Relationship Between Citation Indicators And Peer Review Indicators

BackgroundThe evaluation of the quality of scientific literature is a complex issue that takes into account various factors such as importance,originality,rigor,elegance,and scientific influence.Since there is no mathematical formula to quantify the "quality" of a paper,bibliometrics uses a series of citation-based indicators to evaluate the influence of the paper.At present,the two most commonly used and most representative evaluation methods are citation analysis and peer review.The emergence of F1000 has laid the foundation for quantifying peer review and made the comparative study of citation indicators and peer review indicators feasible.ObjectiveTo explore the relationship between bibliometric evaluation and peer review,and to study the unity and difference degree of bibliometric evaluation and peer review of different attribute disciplines.MethodBibliometric evaluation indexes include the citation frequency of papers in the WoS database and the evaluation index of journals in the JCR database,such as Total Cites,Impact Factor,5-year Impact Factor,Immediacy Index,Eigenfactor Score and Article Influence score.Peer review indicators are derived from the number of paper recommendations and paper recommendation scores in the Faculty of 1000 Prime(F1000Prime)database.This study selects four representative disciplines,covering theoretical and practical disciplines in a targeted manner,and considers citing disciplines with different half-lives when selecting.The final selected disciplines are Infectious Diseases,Oncology,Biochemistry and Structural Biology.Through data cleaning,valid data is finally obtained: 1096 items in Infectious Disease,1073 items in Oncology,1227 items in Biochemistry and 1008 items in Structural Biology.At the level of journal analysis,it covers a total of 716 academic journals,including Biochemistry covering 239 academic journals,Structural Biology covering 198 academic journals,Oncology covering 368 academic journals,and Infectious Diseases covering 331 academic journals.The relevance analysis of citation indicators and peer review indicators is carried out at the paper level and the journal level respectively.ResultsAmong the four disciplines in this study,the peer review and citation of F1000 Prime recommended articles in different disciplines are different.In addition,the ranking of peer reviews and citations in different disciplines is different.The ranking of recommendation times and recommendation scores is Structural Biology = Biochemistry> Infectious Diseases = Oncology,and the total number of citations of WoS database is Oncology> Infectious Diseases> Structural Biology = Biochemistry.At the journal level,Journal F1000 Factors(Including JFF1,journal total number of recommended papers in F1000,JFF2,journal total number of recommended papers in F1000 and JFF3,journal total recommended score in the F1000)have a significant correlation with total citations and feature factors.Compared with the bibliometric indicators of other JCR databases,the correlation coefficients of feature factors and JFF1,JFF2,and JFF3 are larger.Among the three journal evaluation indicators quantified based on the results of peer review of F1000 papers,JFF3 and JCR databases have a statistically significant positive correlation between bibliometric indicators,while JFF1 and JCR databases have a relatively small correlation.The difference between the recommendation times,recommendation scores and citation frequency of different Tag categories.The papers labeled New Finding,Technical Advance,Good for Teaching,Novel Drug Target have more recommendation times,higher recommendation scores and more More citations.ConclusionPeer experts in different disciplines have significant differences in scores.Peer review indicators have a weak correlation with bibliometric indicators.Peer review indicators in different disciplines have different correlations with bibliometric indicators.Whether it is a peer-reviewed index or a bibliometrics index,there are significant differences between different disciplines.Therefore,any academic evaluation must combine the two,and the two cannot replace each other.