An investigation of the statistical power of inferential research published in mathematics education journal

The severe neglect of statistical power, the probability of correctly rejecting the null hypothesis, can be documented as far back as the 19th century when statistical inference was accepted as the method for making decisions in the social sciences. The research questions for this dissertation address how statistical power and the parameters that directly affect the statistical power should be considered during the planning stages of every inferential research study designed in mathematics education. A post-hoc power analysis was conducted using the major statistical procedure of 100 inferential studies sampled from the literature published during the past five years, 1988 through 1992, inclusively, in the following three mathematics education journals: Journal of Research in Mathematics Education, Educational Studies in Mathematics, and School Science and Mathematics. The sample size was obtained from each article but because the level of significance and effect size were reported by researchers in the literature review as severely neglected parameters, the values for these parameters were pre-defined for all inferential tests to provide consistency in computing the statistical power. These three parameters were used in conjunction with statistical power tables to determine the overall average statistical power of the inferential studies. The articles were examined further to determine if the values for statistical power, level of significance, effect size, and sample size were reported and if the method for determining these values was reported. Finally, a discussion is presented concerning how the neglect of these parameters affects the results of the research designed to address issues in mathematics education. Recommendations are given to encourage current and future researchers and reviewers of mathematics education journals to consider statistical power analyses an essential component of the planning stage of every inferential research design for determining the adequate level of significance, effect size, and sample size needed to correctly evaluate the null hypothesis.