A Study Of The Method To Diagnose Publication Bias Based On Publication Date

Objectives To validate the hypothesis that the publication bias is associated with the sequence of publication date, we used the effect size of substudy in Meta-analysis and the sequence of publication date to construct a new method to detect publication bias, then evaluated the accuracy to detect the publication bias of the new method and the consistency of detecting the publication bias for new and traditional methods and put forward the combined diagnosis project of the new and traditional methods to detect the publication bias. Because the new method to detect publication bias is associated with publication date, so we named this new method "the method based on the sequence of publication date".Methods The new method to diagnose publication bias comprises the effect size-the sequence of publication date scatterplot, the total runs of the sequence of effect size sorted by publication date and the effect size-the sequence of publication date linear model.Using the simulated Meta-analysis data being of publication bias and the simulated Meta-analysis data being of no publication bias, the two-side reference interval of the total runs diagnosing publication bias for the method of the total runs was determined according to the principle that the sensitivity to detect publication bias comes to acceptable level and the specificity to detect publication bias reaches to a high level simultaneously; using the simulated Meta-analysis data being of publication bias and the simulated Meta-analysis data being of no publication bias, the sensitivity and specificity of new method and traditional method to detect publication bias were calculated. Afterward, we compared the accuracy of new method with traditional method; using the simulated and published Meta-analysis data, the consistency of detecting the publication bias for new and traditional methods were examined with the kappa index and its 95% confidence interval (CI); using the simulated meta-analysis data, the accuracy of combined diagnosis project (in parallel) with new methods and traditional methods was discussed. According to the accuracy of combined diagnosis project, then the optimal combined diagnosis project with new method and traditional method was put forward.The macro of new methods and traditional methods was written with Statistical Analysis System (SAS) language and carried out in one published Meta-analysis. The macro could output the results of new methods and traditional methods in report forms.The simulated Meta-analysis data were generated by means of Matlab7.0. This study employed the method of Monte Carlo to estimate the accuracy of the new methods and traditional methods to detect publication bias in simulated Meta-analysis for each combination of parameters which was set when generating the simulated Meta-analysis data. According to the pilot simulation study, simulation repeated 1000 times in each different combination of parameters. Simulation was based on different random seed in different combination of parameters. The significant level of hypothesis test was at 0.2.Results The sensitivity to detect the publication bias of the new method based on the sequence of publication date (the method of the total runs and the method of effect size-the sequence of publication date linear model) was low and the specificity to detect the publication bias of the new method was acceptable; the accuracy of the method of the total runs was less than the accuracy of the method of effect size-the sequence of publication date linear model; the sensitivity of the method of unweighted linear model was more than the sensitivity of the methods of weighted linear model, however, the specificity of the method of unweighted linear model was less than the specificity of the method of weighted linear model; the sensitivity of the new method based on the sequence of publication date was less than the sensitivity of the traditional method (Egger's method and Begg's method), but the specificity of the new methods based on the sequence of publication date was not less than the sensitivity of the traditional method; when the population effect size (SMD or ln(OR)) approximated to zero, then the sensitivity of all methods was low, but when the population effect size increased, the sensitivity of all methods increased early and decreased latter and the specificity of all methods immobilized. That is, the accuracy of all methods might relate to the magnitude of the population effect size; when the number of the substudy of Meta-analysis increased, the sensitivity of all methods, except for the method of the total runs, increased and the specificity decreased. That is, the accuracy of all methods might be associated with the number of substudy of Meta-analysis; the consistency for new method and traditional method based on different theories was low; the consistency for the two new methods was still low (the Kappa index was small), but the consistency for the two new methods was better than the consistency for new method and traditional method; the sensitivity of combined diagnosis with new method and traditional method in parallel was more than the sensitivity of combined diagnosis with two traditional methods (Egger's method and Begg's method) in parallel; when the new method and the traditional method were combined in parallel, the sensitivity of the project of combined diagnosis increased, however, the specificity did not decrease obviously; among all projects of combined diagnosis of new method and traditional method, the project of the method of weighted linear model and the Begg's method was optimal; the sensitivity of the project of combined diagnosis of the two new methods was acceptable, but the specificity was small compared to the project of combined diagnosis of the new method and the traditional method; the macro of the new method and the traditional method was written with SAS language and its output was identical to the output of the software in common use (STATA), therefore, the macro made the new method and the traditional method applied conveniently. When the new method and traditional method to detect publication bias were applied to the published Meta-analysis, we found that the results of the method of the linear model were similar to the results of the traditional method for the Meta-analysis with large number of substudy, however, the sensitivity of the method of the linear model was better than the sensitivity of the traditional method for the Meta-analysis with small number of substudy. Because of the magnitude of the total runs was large when the number of substudy in published Meta-analysis was small, the method of the total runs indicated that there might be the publication bias in this published Meta-analysis.Conclusions The accuracy of the new method based on the sequence of the publication date was similar to the traditional method. Because the theories behind the new method and the traditional method were different, therefore, the new method had the special application value and prospects for planning out the project of combined diagnosis with the traditional method in parallel. In all projects of combined diagnosis of the new method and the traditional method, the sensitivity of the project of combined diagnosis of the method of weighted linear model and the Begg's method was acceptable and the specificity reached to a high level, therefore, this project was regarded as an optimal project.Innovations A new method based on publication date to diagnose publication bias was advanced; a method to generate simulated Meta-analysis data with the sequence of publication date was put forward; the optimal project of combined diagnosis with the new method (the method of the total runs and the method of effect size-the sequence of publication date linear model) and the traditional method (the Egger's method and Begg's method) was put forward.