Research On The Comprehensive Evaluation Of Academic Journal Impact Based On Editorship And Citation Indicators

As a critical carrier of knowledge dissemination,scientific exchange and civilization inheritance,scientific journals,especially academic journals,have an important implication for presenting the competitiveness of science and technology and the soft power of science and culture.For a long time,the academic group has focused on the issue of the influence evaluation of scholarly journals.Especially,research on the impact measurement of academic journals appears to be critical in the current background of the cultivation and construction of first-class disciplines.As we all know,it is obvious that the measurement of academic journal impact requires a complete indicator system,and no indicator can independently undertake journal evaluation.However,current research on the impact measurement of academic journals mainly relies on a single traditional citation indicator,which only explores the scientific activities from the dimension of citation.Although the characteristics of journal editors,especially their scholarly performance,have gradually been regarded as an important approach to understand the impact of academic journals,the depth and breadth of the current research is still insufficient.Therefore,based on digging into the multi-dimensional data of editorial team,this study combines the evaluation indicators of the dimensions of editorship and citation to comprehensively analyze the influence of academic journals,and further study their evaluation effect.Firstly,from the dimension of citation,this study comprehensively considers these traditional citation indicators of the 46 sample journals in operations research and management science(OR&MS)field developed based on WoS and Scopus citation databases,and this paper mainly focuses on the sources,definitions,advantages,and disadvantages of 15 indicators.We mainly use the correlation results of these indicators to the relationship of the two databases,and further explore their effectiveness in identifying top peer-reviewed UTD24 journals.Secondly,this paper talks about the evaluation of academic journals based on Editorial Team Scholarly Indices(ETSI),which are constructed by editorial scholarly achievements,editorial level,and editorial size of different levels.This study not only analyzes the evaluation results of the OR&MS journals by different ETSI indicators,but also systematically analyzes the evaluation effects of these indexes based on the list of top journals.Finally,we demonstrate the comprehensive evaluation of academic journals based on ETSI and traditional citation indicators.This chapter combines the selected ETSI and traditional citation indicators of the OR&MS journals so that we can comprehensively evaluate the influence of academic journals from the two dimensions of citation and editorship.This study not only uses the three pairwise integrated methods(entropy-TOPSIS,CRITIC-VIKOR,and power Bonferroni means operator),but also applies fuzzy Borda to investigate the evaluation results of above-mentioned methods.More importantly,this paper comparatively analyzes the evaluation effect of all indicators and methods.The main conclusions of this paper are as follows:(1)The results of correlation analysis between these traditional citation indicators developed based on WoS and Scopus citation databases shows that,except for CHL and CngHL,there are a significantly positive and high relationship among other traditional citation indicators.That fully indicates that the majority of traditional citation indexes of the two databases have significantly high consistency,and these indicators are comparable and alternative in assessing the impact of the OR&MS journals.In addition,it is found that AIS and SJR exhibit well in selecting the top journals,while CngHL has the weak performance.(2)The new ETSI indicators are capable of measuring the influence of academic journals more reasonably and scientifically,because they combine quantitative methods and peer review.The results show that the majority of ETSI indicators are basically reasonable and reliable in evaluating the impact of the OR&MS journals,and at the same time,the these ETSI indicators have differentiation while maintaining its own uniqueness.Taking the list of six UTD24 journals as the evaluation benchmark,this study further finds that ETSI(x)has the most excellent performance,while ETSI(p)performs badly.(3)Through the method of fuzzy Borda,the results of these three pairwise integrated methods are combined to evaluate academic journals.It is obvious to measure the influence of the OR&MS journals more systematically so that we not only comprehensively analyze these indicators based on the dimension of citation and editorship,but also fully consider the information of these indicators and their relationship.The results illustrate that the fuzzy Borda and power Bonferroni means operator perform excellently,and the entropy-TOPSIS and CRITIC-VIKOR also maintain high effectiveness in identifying top journals.This study further summarizes that compared to others,AIS and SJR are more effective in identifying top journals in OR&MS field.More importantly,this article shows certain theoretical significance and practical value.(1)Using the theories of editormetrics and citation analysis,we systematically study the comprehensive evaluation of academic journal impact,and enrich and develop related theories in these research field;(2)This study investigates the comprehensive evaluation of academic journal impact by considering the ETSI and traditional citation indicators,and the relevant conclusions are capable of providing decision-making for relevant departments to promote the construction of first-class journals,and for both universities and scientific research institutions to optimize their research evaluation system.