An Empirical Study of Quantile Indicators in Journal Evaluation Field

Abstract

Abstract: Due to the defects of influence factor, the quantile indicators have drawn wide attention in bibliometrics in recent years. The aim of this study is to compare and discuss the actual effect of quantile indicators in journal evaluation, and define the best quantile indicator for journal evaluation. 80 kinds of biology journals in JCR 2015 were selected as sample. Based on InCites database, different quantile indicators were calculated for each journal. The results were compared with the IF5 and peer review results, in order to identify the best quantile indicator. Results showed that the comprehensive quantile indicator can overcome the design defects of IF5. PR6 was the best quantile indicator in journal evaluation with the best evaluation effect and the highest correlation with peer review.

Key words: Journal evaluation, Quantile indicators, Percentile, Percentage rank 6

Wang Li. An Empirical Study of Quantile Indicators in Journal Evaluation Field[J]. Libraly Journal, 2018, 37(7): 80-87.

References

[1] Garfield E. Citation indexes for science: a newdimension in documentation through association ofideas[J]. Science, 1955, 122(3159): 103-111.
[2] Adler R, Ewing J, Taylor P. Citation statistics[J].Statistical Science, 2009, 24(1): 1-26.
[3] Vinkler P. Introducing the current contribution indexfor characterizing the recent, relevant impact ofjournals[J]. Scientometrics, 2008, 79(2): 409-420.
[4] 俞立平. 历史影响因子: 一个新的学术期刊存量评价指标[J]. 图书情报工作, 2015, 59(2): 89-92.
[5] 徐海丽. 影响因子人为操纵案例分析及构建期刊综合评价体系设想[J]. 中国科技期刊研究, 2014,25(5): 691-695.
[6] Bornmann L, Mutz R. Further Steps Towards anIdeal Method of Measuring Citation Performance:The Avoidance of Citation(Ratio) Averages in Fieldnormalization[J].Journal of Informetrics, 2011, 5(1):228-230.
[7] Bornmann L. Towards an ideal method of measuringresearch performance: some comments to theOpthof and Leydesdorff (2010) paper[J]. Journal ofInformetrics, 2010, 4(3): 441-443.
[8] 郑芳, 周群, 陈仕吉, 等. 基于百分位数指标的个人科研绩效评价研究[J]. 科研管理, 2013, 34(S1):189-194.
[9] 周群, 左文革, 陈仕吉. 基于百分位数的文献计量指标研究综述[J]. 现代图书情报技术, 2013(S1):82-88.
[10] Leydesdorff L, Bornmann L. Percentile ranks andthe integrated impact indicator[J]. Journal of theAssociation for Information Science and Technology,2012, 63(9): 1901-1902.
[11] Leydesdorff L, Bornmann L, Mutz R, et al. Turningthe tables on citation analysis one more time:principles for comparing sets of documents[J].Journal of the American Society for InformationScience and Technology, 2011, 62(7): 1370-1381.
[12] Leydesdorff L, Bornmann L. Integrated impactindicators compared with impact factors: Analternative research design with policy implications[J].Journal of the Association for Information Science andTechnology, 2011, 62(11): 2133-2146.
[13] Bornmann L. A better alternative to the h Index[J].Journal of Informetrics, 2013, 7(1): 100.
[14] Leydesdorff L. Alternatives to the journal impactfactor: I3 and the Top-10%(or Top-25%? ) of themost-highly cited papers[J]. Scientometrics, 2012,92(2): 355-365.
[15] Bornmann L, Leydesdorff L, Mutz R. The use ofpercentiles and percentile rank classes in theanalysis of bibliometric data: opportunities andlimits[J]. Journal of Informetrics, 2013, 7(1): 158-165.
[16] Leydesdorff L. Accounting for the uncertainty inthe evaluation of percentile ranks[J]. Journal ofthe American Society for Information Science andTechnology, 2012, 63(11): 2349-2350.
[17] 刘雪立, 周晶, 盖双双, 等. 基于被引频次分布特征的影响因子缺陷矫正——位置指标应用于期刊评价的实证研究[J]. 图书情报工作, 2016,60(9): 99-115.
[18] 王全红, 王红. Google学术计量中文期刊H5指数和H5中位数再评价研究[J]. 现代情报, 2016,36(10): 74-80.
[19] Garfield E. The history and meaning of the journalimpact factor[J]. Journal of the American MedicalAssociation. 2006, 295(1): 90-93.