Download to read offline
NISO Altmetrics Recommended Practice 3:AM Conference presentation
The document discusses the NISO recommended practices for adopting altmetrics, highlighting the importance of transparent and accurate metrics in assessing scholarly impact. It defines altmetrics, outlines potential use cases and key themes, and emphasizes the need for standards and a code of conduct for data providers. Additionally, it provides recommendations for improving data metrics and ensuring the reliability of research data citation practices.
More Related Content
Similar to NISO Altmetrics Recommended Practice 3:AM Conference presentation
More from National Information Standards Organization (NISO)
NISO Altmetrics Recommended Practice 3:AM Conference presentation
- 1. NISO Recommended Practices to Support Adoption of Altmetrics Todd A. Carpenter 3:AMConference Bucharest, Romania September 29, 2016
- 2. We’re just notas alternative as we used to be Robert Smith, The Cure
- 3. Ø US-based, non-profit industry trade association accredited by American National Standards Institute Ø Mission of developing and maintaining technical standards related to information, documentation, discovery and distribution of published materials and media ØVolunteer driven organization: 400+ contributors spread out across the world Ø Responsible (directly and indirectly) for standards like ISSN, DOI, Dublin Core metadata, DAISY digital talking books, OpenURL, MARC records, and ISBN About August 20, 2016 2
- 6.
- 7.
- 8.
- 9. Definitions and UseCases Code of Conduct Data Metrics Output Types for Assessment Persistent Identifiers and Assessment August 20, 2016 8
- 10.
- 11. Definitions and UseCases August 20, 2016 10
- 12. Caveats • Citations, usagedata, and altmetrics are ALL potentially important and potentially imperfect • Please don’t use altmetrics (or any metrics) as an uncritical proxy for scholarly impact – consider quantitative and qualitative information too • data quality and indicator construction are key factors in the evaluation of specific altmetrics (read as: garbage in, garbage out!) August 20, 2016 11
- 13. What is Altmetrics?Definition Altmetrics is a broad term that encapsulates the digital collection, creation, and use of multiple forms of assessment that are derived from activity and engagement among diverse stakeholders and scholarly outputs in the research ecosystem. The inclusion in the definition of altmetrics of many different outputs and forms of engagement helps distinguish it from traditional citation-based metrics, while at the same time, leaving open the possibility of their complementary use, including for purposes of measuring scholarly impact. However, the development of altmetrics in the context of alternative assessment sets its measurements apart from traditional citation-based scholarly metrics. August 20, 2016 12
- 14. Use Cases Developed eightpersonas, three themes: Showcase achievement: Indicates stakeholder interest in highlighting the positive achievements garnered by one or more scholarly outputs. Research evaluation: Indicates stakeholder interest in assessing the impact or reach of research. Discovery: Indicates stakeholder interest in discovering or increasing the discoverability of scholarly outputs and/or researchers. August 20, 2016 13
- 15.
- 16. Persona: member ofhiring committee August 20, 2016 15
- 17.
- 18.
- 19. Glossary • Activity. Viewing,reading, saving, diffusing, mentioning, citing, reusing, modifying, or otherwise interacting with scholarly outputs. • Altmetric data aggregator. Tools and platforms that aggregate and offer online events as well as derived metrics from altmetric data providers, for example, Altmetric.com, Plum Analytics, PLOS ALM, ImpactStory, and Crossref. • Altmetric data provider. Platforms that function as sources of online events used as altmetrics, for example, Twitter, Mendeley, Facebook, F1000Prime, Github, SlideShare, and Figshare. • Attention. Notice, interest, or awareness. In altmetrics, this term is frequently used to describe what is captured by the set of activities and engagements generated around a scholarly output. August 20, 2016 18
- 20. Glossary (more...) • Engagement.The level or depth of interaction between users and scholarly outputs, typically based upon the activities that can be tracked within an online environment. See also Activity. • Impact. The subjective range, depth, and degree of influence generated by or around a person, output, or set of outputs. Interpretations of impact vary depending on its placement in the research ecosystem. • Metrics. A method or set of methods for purposes of measurement. • Online event. A recorded entity of online activities related to scholarly output, used to calculate metrics. August 20, 2016 19
- 21. Glossary (and evenmore...) • Scholarly output. A product created or executed by scholars and investigators in the course of their academic and/or research efforts. Scholarly output may include but is not limited to journal articles, conference proceedings, books and book chapters, reports, theses and dissertations, edited volumes, working papers, scholarly editions, oral presentations, performances, artifacts, exhibitions, online events, software and multimedia, composition, designs, online publications, and other forms of intellectual property. The term scholarly output is sometimes used synonymously with research outputs. • Traditional metrics. The set of metrics based upon the collection, calculation, and manipulation of scholarly citations, often at the journal level. Specific examples include raw and relative (field-normalized) citation counts and the Journal Impact Factor. • Usage. A specific subset of activity based upon user access to one or more scholarly outputs, often in an online environment. Common examples include HTML accesses and PDF downloads. August 20, 2016 20
- 22.
- 23. Code of Conduct • Why a Code of Conduct? • Scope •Altmetric Data Providers vs. Aggregators August 20, 2016 22
- 24. Code of Conduct Key Elements • Transparency • Replicability •Accuracy August 20, 2016 23
- 25. Code of Conduct: Transparency • How data are generated, collected, and curated • How data are aggregated, and derived data generated •When and how often data are updated • How data can be accessed • How data quality is monitored August 20, 2016 24
- 26. Code of Conduct: Replicability • Provided data is generated using the same methods over time • Changes in methods and their effects are documented •Changes in the data following corrections of errors are documented • Data provided to different users at the same time is identical or, if not, differences in access provided to different user groups are documented • Information is provided on whether and how data can be independently verified August 20, 2016 25
- 27. Code of Conduct : Accuracy • The data represents what it purports to reflect • Known errors are identified and corrected •Any limitations of the provided data are communicated August 20, 2016 26
- 28.
- 29.
- 30.
- 31. Recommendations re Data Metrics • Metrics on research data should be made available as widely as possible • Data citations should be implemented following the Force11 Joint Declaration of Data Citation Principles, in particular: –Use machine-actionable persistent identifiers – Provide metadata required for a citation – Provide a landing page – Data citations should go into the reference list or similar metadata. June 25, 2016 30
- 32. Recommendations for Data Metrics • Standards for research-data-use statistics need to be developed. – Based on COUNTER; consider special aspects of research data –Two formulations for data download metrics: examine human and non-human downloads • Research funders should provide mechanisms to support data repositories in implementing standards for interoperability and obtaining metrics. • Data discovery and sharing platforms should support and monitor “streaming” access to data via API queries. June 25, 2016 31
- 33.
- 34. Altmetrics for #NISOALMI August 20, 201633 39 presentation slides have been downloaded 32,740 times (as of July 26, 2016 The Phase 1 report published in 2014 downloaded 9,636 times Pages hosting content related to this project were accessed 60,548 times >2,000 people attended the 22 in-person presentations about the project Final Report has been downloaded 2,906 times in the 7 days since publication More than 50 articles/blogs/papers about the initiative
- 35.
- 36.
- 37. Initial • Metrics from provider • Ad-hoc Repeatable •Common measurement criteria from provider • Documented measurements and processes • Comparable and consistent Defined • Measurements defined/confirmed as a standard for provider • Made public • Business processes followed consistently • Transparent Managed • Standards applied • Controls in place • Checks and balances repeated over time • Open for comment and feedback • Accountable Governed • Independent verification or 3rd party audit • Evolving common industry defined standards • Trust and confidence Maturity Model for Standards Adoption Increasing trust and confidence in altmetrics August 20, 2016 36
- 38.
- 39.
- 40.
- 41.
- 42. Key Original Ideas Not Yet Done • What is the role of alternative assessment metrics in research evaluation and identify and what gaps exist in data collection around evaluation scenarios. • Identify best practices for grouping and aggregating multiple data sources •Identify best practices for grouping and aggregating by journal, author, institution and funder. August 20, 2016 41
- 43.
- 44. Thank you tothe dozens of people on the working groups and the hundreds of people who participated in brainstorming and commenting on this effort! August 20, 2016 43
- 45.
- 46. Questions? Todd Carpenter Executive Director [email protected] @TAC_NISO NationalInformation Standards Organization (NISO) 3600 Clipper Mill Road, Suite 302 Baltimore, MD 21211 USA +1 (301) 654-2512 www.niso.org August 20, 2016 45