Comparison, Grouping And Integration Of Mainstream Indicators Of Citation Evaluation Of Science Journals

Background: Since the rising of “Bradford’s law” in 1934 that descriped the distribution of articles or citations in science journals, the theories concerning journal evaluation have gone through nearly one-century development. The process can be divided as: counting published document number—counting the number of citations—evaluating both number and quality of citations — the theory of journal citation networks……From this, it can been seen that the evaluation on journal citation impact has played a leading role in the measure of science journal influence. Among the indicators of journal citation impact, impact factor(IF) is used the most widely home and aborad. However, the defects of IF are recognized gradually, so its role in the citation evalution of science journals has been disputed. The scholars have never stopped exploring new citation evaluation indicators that overcome the shortcomings of IF, so there emerged h index, SCImago Journal Rank(SJR), Eigenfactor Score(ES), Source Normalized Impact per Paper(SNIP) and some derivatives. Most of the derivative indicators are not utilized by databases or scientific institutions for evaluating journal impact after they are designed and studied theoretically. Only some core indicators were applied in practice. Nevertheless, the indicators that the journal databases home and abroad choose for evaluating science journals are greatly different, which leads to the disparity of the evalution results by different databases. This may bring confusion to journal evalution. Therefore, a great number of literature metrologists are making great efforts to find out the best indicator or indicator system for evaluating science journal impact by stuying the characteristics and relationships of these citation evaluation indicators, but it seems that they have not reached a consensus.Objective: The study selected six mainstream indicators of science journal citation evaluation under certain criteria: IF, IF5, h index, SJR, ES and SNIP, which were considered to boost the development and revolution of journal citation evaluation. By theoretical analysis and empirical study on the six indicators, the study aims to make clear their advantages and disadvantages and the relationships among them, and to integrate them into a comprehensive indicator via weighing them and getting rid of the overlapping information in them. We hope to introduce more foreign novel indicators of journal citation evaluation to China, change the only-IF-dependent situation, and offer a new strategy and approach of evaluation of science journals in China.Methods: The study consisted of theoretical analys and empirical study. In the theoretical analysis, we studied the computational principles, compared the characteristics, and investigated the factors influencing the variations of these indicators. In the empirical study, we performed a statistical analysis on the 2013 values of the six indicators of 1881 medical journals from Wo S and Scopus databases. The statistical methods included K-S test to describe the distribution features of the data, nonparametric Spearman correlation analysis combined with Cozby’s correlation ranking to observe the correlations among the six indicators, cluster analysis to group the indicators, and factor analysis to extract potential factors influcing the six indicator’s values and construct a formula of comprehensive evlaution indicator. Finally, with immunological journals and internal medical journals as the representative of special and comprehensive journals, respectively, their comprehensive indicators were calucated and sequenced, and the results were compared with their correponding values of the six indicators.Results: The comparison of the six indicators revealed their characteristics in aspects of computational principles, database resource, citation interval, percentage of self-citation, and so on. For example, h index is easy to calculate and stable, but it does take into account the weight of cited journals, so it is not suitable for the comparative evaluation of journals of different disciplines; both SJR and ES take the weight of citations into their computations, but SJR is still not suitable for the comparison among different disciplines for SJR reflects the average quality of published articles; ES is a measure of the overall impact of science journals, so it is appropriate to employ ES in the evaluation and comparison of journals among different disciplines; SNIP allows direct comparison of source journals in different subject fields, but it does not consider the weight of citations. The above theoretical findings provided reference to the following large sample empirical study. The statistical analysis of the 1881 medical journals showed the distribution of the variables was right-skewed; there were significant correlations among the six indicators; they were classified into two groups: IF, IF5, SNIP, SJR as one group and ES and h index as the other group; two potential factors(also named principle components) were extracted from the six indicators, F1 with bigger factor loadings in IF, IF5, SNIP, SJR and F2 with bigger factor loadings in ES and h index. Based on the indicator characteristics of the two groups, the group composed by IF, IF5, SNIP, SJR was named “average impact indicators” and F1 “average impact factor”; the other group was named “overall impact indicators” and F2 “overall impact factor”. The formula of comprehensive evaluation indicator was established by weighing the two factors.Conclusion: The evaluation of science journals is multidimentional and complicated. The differences in the interval of calculation, subject fields and scource databases have influence on the evaluation results. Any single indicator can not evaluate a journal in absolutely comprehensive aspects. Therefore, we need to construct a indicator system, which involves choosing and weighing indicators and abolishing the overlapping information in indicators. The large sample study indicated that the six indicators had significant correlations statistically, and the overlapping information deciding the correlation coefficients was resulted from the function of the indicators reflecting the average or overall impact of journals instead of the differences in the scource databases, or citation intervals, or self-citation management, or whether to account for the weighes of citation. The number of published documents played a vital role in journal evaluation. The comprehensive indicator that the study constructed was proved empirically being able to eliminate the advase effect of published document number on the evaluation of science journal impact by weighing the overall and average impacts of journals. Compared with the six indicators, it could better present the comprehensive impact of science journals.