| When the findings of a randomized clinical trial are null, and yet the assumptions of the statistical model are not met, is it appropriate to conclude there is not an effect of the intervention? The purpose of this study is to examine statistical method choice for the analysis of a randomized clinical trial of a strength training and nutrition intervention in a sample of overweight Latino adolescents (N = 54). Results of analysis of covariance (ANCOVA) models were overwhelmingly null, however there were several concerns about the underlying assumptions. Two non-traditional statistical approaches that do not carry the same assumptions as traditional ANCOVA were used to reanalyze these data. The first approach uses a robust analog of ANCOVA that is based on fewer restrictions, and which can increase power while maintaining Type I error rate, even with small samples. Using these robust techniques, the conclusions regarding the effectiveness of the intervention varied widely from those of traditional ANCOVA, as several significant intervention effects were found with these robust methods. In the second approach, developed for this study, a hybrid of two common latent profile analysis models was created to generate a profile of which participants benefited from the intervention. This model tests whether the sample is homogeneous. With this model, it was shown that gender and pre-test values had more influence than the intervention on outcomes, and the intervention appeared to modify these influences. These results suggest that the use of traditional ANCOVA models in the face of assumption violations may lead to missing important effects of an intervention. Expanding the scope of standard techniques for analyzing randomized clinical trials would likely result in a different literature landscape for many disciplines, though the acceptability of the use of these results poses challenges for publication of papers using them. New guidelines may need to be incorporated into recommendations for which methods to use for analyzing randomized clinical trials. |
