An examination of the efficacy of classical and Bayesian meta-analysis approaches for addressing important meta-analysis objectives

This paper examines the efficacy of classical versus Bayesian meta-analytic models for addressing the five important meta-analytic objectives that were proposed by Higgins, Thompson, and Spiegelhalter (2009). In addition, it presents and examines a sixth important meta-analytic objective within the classical and Bayesian frameworks -- a consideration of how meta-analytic inferences may change depending upon the uncertainty in the estimate of the amount of heterogeneity. In order to meet this sixth objective, this study uses a classification system which follows the guidelines proposed by Rothstein, Sutton, and Borenstein (2005) for describing the impact of publication bias. Here, the impact of the way meta-analytic results may change depending upon the uncertainty in the heterogeneity is classified with the use of qualitative indicators akin to those used by Rothstein et al. (2005). Thus, the discrepancy between the best-fitting meta-analytic model and the meta-analytic models used for heterogeneity sensitivity analyses is described as: (a) "minimal", when the fitted meta-analytic models and the estimates remain similar; (b) "modest", when the fitted meta-analytic models remain the same, but the estimates change to a moderate degree; and (c) "severe", when the fitted meta-analytic models and estimates differ substantially from each other.;This research suggests that Bayesian hierarchical linear modeling offers the most complete and accurate approach for addressing all relevant meta-analytic objectives. The project uses five different meta-analytic datasets as illustrative examples. It also provides examples of the code for the classical models for the metafor package, the Bayesian code for the WinBUGS package, and the S-PLUS code for the Bayesian hblm function. Given the complexity and nuances associated with Bayesian model development, a Bayesian quality assurance meta-analysis checklist was refined for this research project.;The use of meta-analytic trace plots produced with the hblm function, which depict the dependency of meta-analytic results on the values of the standard deviation of the between-study variance, is shown to summarize the essence of a fully Bayesian meta-analysis. In a single picture, these plots summarize four out of five of Higgins et al.'s (2009) important metaanalytic objectives. Furthermore, meta-analytic trace plots also provide the additional, important (though underappreciated) advantage of representing how meta-analytic estimates change depending upon the uncertainty in the estimate of the amount of heterogeneity. This paper suggests that the future design of meta-analytic trace plots should also include inlaid curves that depict the estimates for the predicted effect in a new study so that all six important meta-analytic objectives could be addressed in a single graphic display.