SYSTEMATIC REVIEW AND METASYNTHESIS OF PAPERS ON SIGNIFICANCE TESTS

A TEXT FOR PHYSICAL EDUCATION PROFESSIONALS

Abstract

Significance tests continue to be the main means of analyzing the occurrence of differences between groups or associations between variables in the population, using sample data. Several texts have been published in Portuguese about these tests, however, no synthesis of these texts has been carried out to date. Such a synthesis favors a broader understanding, than the isolated reading of each text, regarding the interpretation of significance tests. Given this, we carried out a systematic review and metasynthesis of 26 articles in Portuguese whose main objective was to provide useful information for the interpretation of significance tests. The searches, carried out on April 15, 2021, and updated on August 28, 2023, took place on Google Scholar, which is especially suitable for identifying texts in Portuguese. Two authors participated in the selection of full-text studies and in the qualitative synthesis of data. Based on this metasynthesis, (1) we present fundamental concepts for interpreting significance tests, (2) we discuss the advantages and disadvantages of using arbitrary p values (for example, p < 0.05), (3) we explain how to interpret estimates and confidence intervals, and (4) we point out a relevant gap in the literature. Researchers often interpret p-values according to arbitrary cut-off points (commonly 0.05), rather than considering the estimated effect or association and its confidence interval. This approach ignores information relevant to decisions about the practical importance of the finding and its imprecision (referring to inference for the population). Therefore, our recommendation is that, when possible, determine before carrying out the study what are important effects or associations and interpret the estimate of the effect or association and its confidence interval.