Download to read offline
![Limitations of existing solutions 2/2
ā¢ Data sparsity
ā¢ Scopus/Web of Science are known to have lower publication/citation
coverage than Google Scholar and Microsoft Academic [Harzing, 2017]
ā¢ Dimensions use primarily citations from I4OC which is an even sparser
citation dataset
ā¢ Lack of APIs for integration of results into existing systems
(Scopus and Web of Science)
ā¢ Slice-and-dice operations on titles/abstracts only
ā¢ Papers without DOI have metrics rarely available](https://image.slidesharecdn.com/20190613or2019analyticsdashboardv2-190718115301/75/CORE-Analytics-Dashboard-5-2048.jpg)
![CORE Analytics Dashboard
ā¢ CORE is the largest full text Open Access aggregation service
[Notay, 2018]
ā¢ Because it is based on Open Access data, it can offer/enable:
ā¢ Slice-and-dice operations based on full text
ā¢ Full access to underlying data
ā¢ API integration with internal university systems](https://image.slidesharecdn.com/20190613or2019analyticsdashboardv2-190718115301/75/CORE-Analytics-Dashboard-6-2048.jpg)
![Conclusions
ā¢ We presented CORE Analytics Dashboard ā a tool designed to
enable users to analyse and compare the performance of
research outputs between universities along a variety of metrics
ā¢ Key difference from existing solutions ā focus on collecting
performance indicators from openly available sources
ā¢ Goal ā add a layer of transparency to research evaluation
ā¢ Goal ā extend to whole world and add additional features
ā¢ E.g. collaboration graphs, UK REF predictions [Pride, 2018]](https://image.slidesharecdn.com/20190613or2019analyticsdashboardv2-190718115301/75/CORE-Analytics-Dashboard-22-2048.jpg)
More Related Content
Similar to CORE Analytics Dashboard
CORE Analytics Dashboard
- 1. CORE Analytics Dashboard PetrKnoth Jozef Harag Drahomira Herrmannova June 13, 2019 ā OR 2019, Hamburg, Germany CORE Big Scientific Data and Text Analytics Group Knowledge Media Institute, The Open University
- 2. Introduction ā¢ Research metricsbecoming a key component of evaluation processes ā¢ Application of research indicators controversial, but demand for them is increasing ā¢ Universities want to understand and monitor the impact of their research outputs ā¢ This is evidenced by subscriptions to expensive products like Elsevierās SciVal, Clarivateās InCites ā¢ Typical cost: negotiated, it is believed Ā£35-Ā£50k per annum per institution
- 3. Existing solutions ā¢ ElsevierSciVal ā¢ Clarivate InCites ā¢ Digital Science Dimensions
- 4. Limitations of existingsolutions 1/2 ā¢ Proprietary data (Scopus and Web of Science) ā¢ Data or results of analyses cannot be easily downloaded and shared with others ā¢ Algorithms used for citation data acquisition not open/known (Scopus and Web of Science) ā¢ Complicates confirming validity/replicating results
- 5. Limitations of existingsolutions 2/2 ā¢ Data sparsity ā¢ Scopus/Web of Science are known to have lower publication/citation coverage than Google Scholar and Microsoft Academic [Harzing, 2017] ā¢ Dimensions use primarily citations from I4OC which is an even sparser citation dataset ā¢ Lack of APIs for integration of results into existing systems (Scopus and Web of Science) ā¢ Slice-and-dice operations on titles/abstracts only ā¢ Papers without DOI have metrics rarely available
- 6. CORE Analytics Dashboard ā¢CORE is the largest full text Open Access aggregation service [Notay, 2018] ā¢ Because it is based on Open Access data, it can offer/enable: ā¢ Slice-and-dice operations based on full text ā¢ Full access to underlying data ā¢ API integration with internal university systems
- 7. CORE Analytics Dashboard ā¢A tool for analytical evaluation of universitiesā research outputs based primarily on CORE data ā¢ Research impact insights on institutional level ā¢ Comparison with other institutions or groups of institutions ā¢ Based on citation data, social impact data ā¢ Enables users to design and create custom graphs and add these to their user area ā¢ Intended for university research officers, repositories, university management
- 8. Use cases ā¢ Auniversity's Research Office manager wants to see how her institution compares to another institution ā¢ A university's Research Office manager wants to have an overlook on research impact of a hot-topic (e.g. Artificial Intelligence, DNA mapping, global warming, etc.) across UK universities ā¢ A non-Russell Group university wants to demonstrate (for marketing reasons) that they have the same research and social impact (or perhaps better) as any of Russell Group universities in a given field.
- 9. Data sources 1. COREdataset ā¢ Primary pivot, provides publication metadata, affiliation information 2. Microsoft Academic Graph ā¢ Citation data 3. Crossref event data ā¢ Publicationsā Wikipedia and Twitter mentions 4. Mendeley ā¢ Readership data, i.e. information about how many Mendeley users have added a certain publication in their Mendeley library
- 10. Data ā¢ Data fromacross different systems were matched using DOIs ā¢ Prototype version: ā¢ UK institutions ā¢ Outputs with a DOI ā¢ Long-term goal is to expand to the whole word
- 11. Interface overview ā¢ Individualfor each user ā¢ Each user assigned a specific institution ā¢ Customizable user area: users can define metrics and graphs to track & display ā¢ Institutional ranking
- 12. User interface: statisticsbar ā¢ Each user assigned a specific institution ā¢ Quick summary overview of institutionās performance
- 13. User interface: dashboard ā¢Customizable area, user-defined charts
- 14. User interface: institutionalranking ā¢ Ranking according to a number of metrics
- 15. Customizable area ā¢ Enablescreating user-defined graphs ā¢ Chart data can be downloaded: ā¢ Original (raw) data ā¢ Aggregate chart data ā¢ Chart PNG ā¢ Full customizable, enables resizing and reorganizing charts
- 16. User area: customizingthe dashboard ā¢ A step-by-step tool (benchmarking wizard) for creating custom visualizations tailored to userās needs ā¢ Works in three steps: 1. Define publications sample filtered to select search criteria 2. Define what indicators to inspect 3. Select visualization chart to apply on the filtered data
- 17. User interface: benchmarkingwizard 1. Filter data ā user specifies which articles to work with ā¢ E.g. using keywords or phrases of interest that will be searched in publicationsā titles, abstracts, and full texts ā¢ Also enables filtering, currently by year and publication type (research article, thesis, etc.) 2. Selecting metrics to track ā user defines which institutions to compare and based on what metrics to compare them 3. Designing the chart ā select chart type and name the chart ā¢ Several chart types available including line charts, bar charts, scatter plots, etc. ā¢ User can customize colors, names of axes, etc.
- 18. User interface: benchmarkingwizard 1. Filter data ā user specifies which articles to work with
- 19. User interface: benchmarkingwizard 1. Filter data ā user specifies which articles to work with
- 20. User interface: benchmarkingwizard 2. Selecting metrics to track
- 21. User interface: benchmarkingwizard 3. Designing the chart
- 22. Conclusions ā¢ We presentedCORE Analytics Dashboard ā a tool designed to enable users to analyse and compare the performance of research outputs between universities along a variety of metrics ā¢ Key difference from existing solutions ā focus on collecting performance indicators from openly available sources ā¢ Goal ā add a layer of transparency to research evaluation ā¢ Goal ā extend to whole world and add additional features ā¢ E.g. collaboration graphs, UK REF predictions [Pride, 2018]
- 23.
- 24. References ā¢ Harzing, Anne-Wil,and Satu Alakangas. (2017). "Microsoft Academic is one year old: The Phoenix is ready to leave the nest." Scientometrics 112.3: 1887-1894. ā¢ Notay, Balviar. (2018). "CORE becomes the worldās largest aggregator". Jisc scholarly communications. https://scholarlycommunications.jiscinvolve.org/wp/2018/06/01/c ore-becomes-the-worlds-largest-aggregator/. Accessed: 2019- 01-16. ā¢ Pride, D. and Knoth, P. (2018). "Peer review and citation data in predicting university rankings, a large-scale analysis". Lecture Notes in Computer Science. Springer.
- 25.
Editor's Notes
- #10Ā We could also include repository download/usage data (IRUS)