Downloaded 197 times
![Understanding the Basics of
Journal Metrics
By Leena Shah
Managing Editor, Ambassador for DOAJ
5th Annual Conference of Asian Council of Science Editors [ACSE]
Dubai, 21-22 March 2018 [https://theacse.com/acseconference.php]](https://image.slidesharecdn.com/understandingthebasicsofjournalmetricsconf-181211082505/75/Understanding-the-Basics-of-Journal-Metrics-1-2048.jpg)
![h – index [Source: WoS, Google Scholar or Scopus ]
• Attempts to measure the productivity and citation impact of the
published body of work of an author.
• Definition of h-index: a researcher with an index of h has
published h papers, each of which has been cited at least htimes.
Google Scholar Metrics [ Source: Google Scholar]
• A list of top publications based on the citations within Google,
ordered by their five-year h-index and h-median metrics.
• h5-index is the h-index for articles published in the last 5 complete
years. It is the largest number h such that h articles published in
2011-2015 have at least hcitations each.
Overview of popular journal metrics based
on Scopus/Google Scholar/WoS](https://image.slidesharecdn.com/understandingthebasicsofjournalmetricsconf-181211082505/75/Understanding-the-Basics-of-Journal-Metrics-8-2048.jpg)
![A call for change – DORA
The transformation of the declaration from a statement of intent to a facilitator of policy change
has begun
• San Francisco Declaration on Research Assessment [DORA]
http://www.ascb.org/dora/
- Declaration was conceived in 2012 Annual Meeting of the American
Society for Cell Biology in San Francisco and was announced in May 2013.
- International declaration (18 recommendations) calling on the world
scientific community to eliminate the role of the Journal Impact Factor
(JIF) in evaluating research for funding, hiring, promotion, or institutional
effectiveness
- Over 11800 individual signatories, 463 organizations have signed including
Nature Research, BioMed Central, SpringerOpen, British Library and UK
research councils. Recently OASPA, the Brazilian Institute of Information in
Science and Technology, and the IRCCS Ospedale San Raffaele have also
signed the declaration.](https://image.slidesharecdn.com/understandingthebasicsofjournalmetricsconf-181211082505/75/Understanding-the-Basics-of-Journal-Metrics-13-2048.jpg)
![References
LSE Impact Blog
• Beyond Impact Factors: an Academy of Management report on measuring scholarly impact. (2018, March 02). Retrieved
March 14, 2018, from http://blogs.lse.ac.uk/impactofsocialsciences/2018/03/02/beyond-impact-factors-an-academy-of-
management-report-on-measuring-scholarly-impact/
Libguides
• http://aut.ac.nz.libguides.com/impact/jif-rankings
• http://libguides.lib.umanitoba.ca/facultyhelp/journalmetrics
Blogs
• Neophytou, J. (2014, May 14). How to navigate the world of citation metrics [Web blog post]. Retrieved March 14, 2018, from
https://hub.wiley.com/community/exchanges/discover/blog/2014/05/14/how-to-navigate-the-world-of-citation-
metrics?referrer=exchanges
• Neophytou, J. (2015, April 9). Fake metrics and how to spot them [Web blog post]. Retrieved March 14, 2018, from
https://hub.wiley.com/community/exchanges/discover/blog/2015/04/09/fake-metrics-and-how-to-spot-
them?referrer=exchanges
• Curry, S. (2018, February 7). Let’s move beyond the rhetoric: it’s time to change how we judge research [Web post].
Retrieved March 14, 2018, from https://www.nature.com/articles/d41586-018-01642-w
Websites
• http://www.metrics-toolkit.org/
• https://sfdora.org/
Articles/Reports
• Scully, C., & Lodge, H. (2005). Impact factors and their significance; overrated or misused? British Dental Journal,198(7), 391-
393. doi:10.1038/sj.bdj.4812185
• Wilsdon, J., et al. (2015). The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment
and Management. DOI: 10.13140/RG.2.1.4929.1363](https://image.slidesharecdn.com/understandingthebasicsofjournalmetricsconf-181211082505/75/Understanding-the-Basics-of-Journal-Metrics-16-2048.jpg)
Uploaded byDOAJ (Directory of Open Access Journals)
Understanding the Basics of Journal Metrics
The document provides an overview of journal metrics, detailing their use in evaluating journal quality and influence within scientific disciplines, including popular metrics like the Journal Impact Factor and alternative metrics such as altmetrics. It highlights criticisms of these metrics, particularly their limited effectiveness in assessing individual articles and the influence of various factors on their reliability. The document also discusses initiatives for reform, including the San Francisco Declaration on Research Assessment (DORA) and the Leiden Manifesto, advocating for improved practices in research evaluation.
In this document
Powered by AI
Overview of journal metrics by Leena Shah at the ACSE conference, focusing on their importance in research.
Discusses what journal metrics are, their usages, examples, and critiques, introducing key developments.
Defines journal metrics as rankings determining the quality of journals based on impact and influence.
Lists applications of journal metrics in grant applications, promotions, submissions, library decisions.
Introduces Journal Impact Factor and 5-year Journal Impact Factor as metrics from Journal Citation Reports.
Details Journal Immediacy Index and Eigenfactor metrics based on citations from the Web of Science.
Explains SJR and SNIP metrics from Scopus, highlighting their methodologies for journal ranking.
Introduces the h-index and h5-index as author-level metrics based on citations from various sources.
Defines altmetrics as a measure of social visibility, tracking public engagement and visibility metrics.
Outlines resources to find altmetrics including Impactstory, Plum Analytics, and Altmetric.com.
Advises on recognizing fake metrics and pseudo-impact factor sites, emphasizing research on legitimacy.
Discusses criticisms of Journal Impact Factor including its limitations and biases against non-English journals.
Introduces DORA's declaration urging the shift away from Journal Impact Factor in research evaluations.
Highlights Leiden Manifesto and The Metric Tide report, proposing better practices for metric evaluations.
Summarizes various metrics and criticisms, emphasizing the call for action through DORA and similar initiatives.
Lists references including articles, blogs, and websites for further exploration of journal metrics.
Thanks to institutions and sponsors that supported the event and DOAJ.
Concludes the presentation and provides contact info for further inquiries.
More Related Content
Similar to Understanding the Basics of Journal Metrics
More from DOAJ (Directory of Open Access Journals)
![Directory of Open Access Journals [DOAJ] – An introduction and more](https://cdn.slidesharecdn.com/ss_thumbnails/delhitffocusgroupapril2018final-181211082724-thumbnail.jpg?width=640&height=640&fit=bounds)
Understanding the Basics of Journal Metrics
- 1. Understanding the Basicsof Journal Metrics By Leena Shah Managing Editor, Ambassador for DOAJ 5th Annual Conference of Asian Council of Science Editors [ACSE] Dubai, 21-22 March 2018 [https://theacse.com/acseconference.php]
- 2. Agenda What are journalmetrics? Where are journal metrics used? Overview of popular citation journal metrics Source : Journal Citation Reports (JCR) - WoS Scopus Google Scholar Alternate metrics – Altmetrics Fake metrics Criticism of IF & key developments – DORA, Leiden manifesto & Metric Tide report
- 3. What are JournalMetrics? • Ranking of journals within a specific discipline according to their impact and influence • Proxy measure used to evaluate quality of a journal • Most citation metrics are journal level metrics. Popular e.g. Journal Impact Factor
- 4. Where are journalmetrics used? - Grant applications - Staff applying for promotion & tenure - Make informed decision when you want to submit your article for publication - Make a comparison of journals within a field - Libraries use rankings to make decisions on journal retention or acquisitions - Journals use it to attract editors, sponsorship, author submissions
- 5. Overview of popularjournal metrics based on JCR Source: Web of Science. Published annually in Journal Citation Reports (JCR) • Journal Impact Factor • 5-year Journal IF
- 6. Overview of popularjournal metrics based on JCR Source: Web of Science. Published annually in Journal Citation Reports (JCR) • Journal Immediacy Index Immediacy Index refers to the number of times a paper published in a year is cited within that same year. • Eigenfactor Number of times articles published in a journal over the past 5 years have been cited, with citations from more influential journals weighted more than citations from less influential journals.
- 7. Source: Scopus • SJR– SCImago Journal Rank weighted citations in Year X to papers published in the previous 3 years. Free website: http://www.scimagojr.com/index.php • SNIP – Source Normalized Impact per Paper ratio of a source's average citation count per paper, and the 'citation potential' of its subject field. It aims to allow direct comparison of sources in different subject fields. Overview of popular journal metrics based on Scopus
- 8. h – index[Source: WoS, Google Scholar or Scopus ] • Attempts to measure the productivity and citation impact of the published body of work of an author. • Definition of h-index: a researcher with an index of h has published h papers, each of which has been cited at least htimes. Google Scholar Metrics [ Source: Google Scholar] • A list of top publications based on the citations within Google, ordered by their five-year h-index and h-median metrics. • h5-index is the h-index for articles published in the last 5 complete years. It is the largest number h such that h articles published in 2011-2015 have at least hcitations each. Overview of popular journal metrics based on Scopus/Google Scholar/WoS
- 9. Alternative metrics (Altmetrics) •Bibliometrics measure research impact by analysing citations • Altmetrics monitor social visibility around scientific articles e.g. tweets, blog mentions, news media, social bookmarking, article views, and downloads. Includes web citations in digitised scholarly documents • Applied to journal articles, books/book chapters, datasets • Real-time usage
- 10. Where can Ifind altmetrics? • Impactstory free author profile service that allows users with an ORCID identifier to find and compile their altmetrics into a single profile http://www.impactstory.org/ • Plum Analytics (Elsevier) Plum Analytics offers altmetrics alongside citation metrics and usage statistics in their PlumX suite of products, aimed primarily at universities and funders. http://plumanalytics.com/ • Altmetric.com (Digital Science, Springer Nature) • Individual journals and publishers e.g PLOS • Scopus
- 11. Fake metrics Some examplesof bogus Impact factor sites: Scientific Journal Impact Factor Info Base Index Factor Cosmos Impact Factor Universal Impact Factor Global Impact Factor Citefactor How to identify fake Impact Factors? - Research the company - Are they asking for upfront payment? - Can you validate the mathematical formula for their metric? - Data source?
- 12. Criticism of JournalIF • Journal level metrics which cannot be used to evaluate the impact of individual articles • Scientific field that the journal belongs influences IF • Self citation – authors cite papers in the same journal to manipulate IF • Artificially high IFs in journals that mainly publish review articles • Not reproducible and the data source is not public • Based on a narrow 2-year time frame which is inappropriate for many disciplines • Biased towards English language journals
- 13. A call forchange – DORA The transformation of the declaration from a statement of intent to a facilitator of policy change has begun • San Francisco Declaration on Research Assessment [DORA] http://www.ascb.org/dora/ - Declaration was conceived in 2012 Annual Meeting of the American Society for Cell Biology in San Francisco and was announced in May 2013. - International declaration (18 recommendations) calling on the world scientific community to eliminate the role of the Journal Impact Factor (JIF) in evaluating research for funding, hiring, promotion, or institutional effectiveness - Over 11800 individual signatories, 463 organizations have signed including Nature Research, BioMed Central, SpringerOpen, British Library and UK research councils. Recently OASPA, the Brazilian Institute of Information in Science and Technology, and the IRCCS Ospedale San Raffaele have also signed the declaration.
- 14. Leiden Manifesto andothers.. • Leiden Manifesto http://www.leidenmanifesto.org/ - Named after the conference at which it was conceived - Released in 2015, it offers best practice (10 principles) in metrics-based research assessment - The article published in Nature in 2015 , “has been viewed over 60,000 times on Nature’s website, and has accumulated 259 citations in Google Scholar, 128 in Scopus and 55 in Web of Science.” • ‘The Metric Tide’ Report https://responsiblemetrics.org/the-metric-tide/ - Published in 2015 by Independent Review of the Role of Metrics in Research Assessment and Management - Review looked in detail at the potential uses and limitations of research metrics and indicators, exploring the use of metrics within institutions and across disciplines.
- 15. Summary • Journal-level metricsbased on citations: JIF, 5-year IF, SNIP, SJR, Eigenfactor source: Scopus or JCR/Web of Science • Article-level metrics : Altmetrics source: social media/ web • Author-level metrics: h-index and Google Scholar Criticism of JIF & its over-reliance as a measure of researcher assessment call for action : - DORA - Metric Tide - Leiden Manifesto
- 16. References LSE Impact Blog •Beyond Impact Factors: an Academy of Management report on measuring scholarly impact. (2018, March 02). Retrieved March 14, 2018, from http://blogs.lse.ac.uk/impactofsocialsciences/2018/03/02/beyond-impact-factors-an-academy-of- management-report-on-measuring-scholarly-impact/ Libguides • http://aut.ac.nz.libguides.com/impact/jif-rankings • http://libguides.lib.umanitoba.ca/facultyhelp/journalmetrics Blogs • Neophytou, J. (2014, May 14). How to navigate the world of citation metrics [Web blog post]. Retrieved March 14, 2018, from https://hub.wiley.com/community/exchanges/discover/blog/2014/05/14/how-to-navigate-the-world-of-citation- metrics?referrer=exchanges • Neophytou, J. (2015, April 9). Fake metrics and how to spot them [Web blog post]. Retrieved March 14, 2018, from https://hub.wiley.com/community/exchanges/discover/blog/2015/04/09/fake-metrics-and-how-to-spot- them?referrer=exchanges • Curry, S. (2018, February 7). Let’s move beyond the rhetoric: it’s time to change how we judge research [Web post]. Retrieved March 14, 2018, from https://www.nature.com/articles/d41586-018-01642-w Websites • http://www.metrics-toolkit.org/ • https://sfdora.org/ Articles/Reports • Scully, C., & Lodge, H. (2005). Impact factors and their significance; overrated or misused? British Dental Journal,198(7), 391- 393. doi:10.1038/sj.bdj.4812185 • Wilsdon, J., et al. (2015). The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management. DOI: 10.13140/RG.2.1.4929.1363
- 17. Thanks to All theLibrary Consortia, Universities and Publishers and our Sponsors for the financial support to DOAJ!
- 18. Thank you foryour attention! [email protected]
Editor's Notes
- #7 Eigenfactor