| ObjectivesThis paper aimed to construct an evaluation index system to assess statistical reporting quality of medical observational research papers and get the weight of indices for evaluating statistical reporting quantitatively. Besides, assess the observational studies’ statistical reporting quality through evaluating the statistical reporting quality of observational studies published in medical journals using the established evaluation index system and making analysis of the scoring of these studies. MethodsAn evaluation index system was designed by literature review, group discussion and three rounds of expert consultation. The experts’ evaluation for the importance and feasibility of each index was analyzed. Weights of indices were calculated by the method of analytic hierarchy process(AHP), based on the experts’ evaluation on relative importance of indices. Randomly selected published observational studies and downloaded 270 papers respectively from the SCI and Chinese core medical journals obeyed the stratified randomly sampling principle. Get the score of these papers according to the scoring standards and the weights of indices. Results1 Respond rate of three rounds of expert consultation was 95%,100%,100%, respectively. After the first consultation, five indices were excluded, they were “inclusion criteria”, “exclusion criterion”, and “control setting” with respect to the second-class index of “research subject”, “bias type” and “bias affection” with respect to the second-level index of “research integrity”. And “Statistics” and “P value” in the third-level affiliated to the second-level index of “statistical inference” were combined to one index, which was “statistic and P value”. Two third-level indices “inner validity” and “outer validity” were added, affiliated to the second-level index of “research integrity”. After the adjustments, the questionnaire included 4 first-level indices, 11 second-level indices and 27 third-level indices. In the second round, the mean scores and median scores of 42 indices’ importance and feasibility were larger than 7, and the coefficients of variations were less than 0.25. Finally, 42 indices were included according to the second level of consensus. Statistical reporting quality evaluation index system was composed of 4 first-level indices, 11 of second-level indices and 27 third-level indices.2 The 4 first-level indices were “title and abstract”, “materials and methods”, “results”, and “discussion”. And the corresponding weights were 15.9%, 33.7%, 39.0%, and 11.4%. The largest overall weight of the third-level indices was “statistical method” with respect to “materials and methods”, then followed “statistics and P value”, “model hypothesis”, “variable assignment”, “data description” and “table and diagram”, affiliated to “results”.3 Median of the indices “statistical method” with respect to “title and abstract”, “bias type” and “bias control” with respect to “materials and methods”, “variable assignment” with respect to “results” in both of Chinese and SCI observational papers was zero. The statistical reporting quality difference of the SCI medical journals was higher than that of the Chinese(Z=13.807, P<0.001). There were no significance difference between papers of cross-sectional studies, case-control studies, and cohort studies both in SCI medical journal and Chinese core medical journal(2c =0.578,P=0.749;2c =1.224,P=0.542). Conclusions1 Statistical reporting quality evaluation index system of observational research papers could be used to quantitatively evaluate statistical reporting quality as the higher activity and reliability of experts, and the rigorous development process, and the relative importance of all indices is accordance with the statistical theory.2 Compared to the statistical reporting quality of observational studies published in SCI medical journals, Chinese papers’ statistical reporting quality is rather low and could be improved pointed according to the evaluation index system. Key Words observational studies;statistical reporting;Delphi method;analytic... |
PDF Full Text Request