Learning analytics – 
opportunities and issues 
Dr Sharon Slade 
The Open University, UK 
21st SAAIR Conference 2014 
Yesterday, today and tomorrow: 21 years of Institutional Research
Learning analytics is the measurement, 
collection, analysis and reporting of data about 
learners to increase our understanding of them 
and their learning needs, and to use that 
understanding to influence their learning.
ā€˜99% of who you are is invisible’ 
R. Buckminster Fuller 
ā€˜ 
http://www.glennsasscer.com/wordpress/wp-content/ 
uploads/2011/10/iceberg.jpg
What do we mean by data about 
learners? 
Background 
Disability 
Gender 
Learning 
behaviours 
Posting to forums - 
content 
Ethnicity 
Study goals 
Study history 
Learning style 
Funding 
issues 
Assignment/test 
scores 
Websites visited 
Age 
Hitting study 
milestones 
Location 
Working status 
Family income 
Language 
Log in frequency 
Posting to forums - 
frequency 
Frequency of contact 
with tutor
It’s everywhere: every learning tool now has an ā€œanalytics 
dashboardā€ (a Google image search) 
8
The OU analytics innovation to impact cycle 
annual cycles of quality enhancement and 
research capability 
and student 
number planning 
maintain analysis 
models 
manage data store 
responsibility for 
architecture 
Innovation Impact 
Research, 
benchmarking and 
rapid prototyping 
Data storage and access for analysis 
Data collection 
Technology architecture 
Actionable insight 
Learning design 
Information advice 
and guidance 
Informed student 
interventions 
Quality 
enhancement 
processes 
Define new 
research questions 
Innovation 
mainstreamed 
into operations 
Requirements for 
new data 
collection 
Manage core 
systems 
and support 
surveys 
ongoing insight 
build and manage systems and tools, provide process 
reports 
New indicators 
identified 
Mainstream 
indicators 
Mainstreamed 
analysis models 
Data harvesting from core student 
record and learning systems 
Systems Students 
Outcomes 
Evidence based 
learning design and 
continuous 
improvement 
→ increased student 
attainment over 
time 
More targeted and 
appropriate IAG 
→ decrease in early 
drop-out 
Interventions are 
better targeted and 
more effective 
→ increased 
retention over time 
Direct feedback from students 
Mainstream BAU 
http://www.jumpoffthescreen.com/analytics.php
Recommender
Purdue’s Course signals 
• Uses a predictive model based on 
– VLE activity and assessment scores 
• Previous academic history and demographic data 
• Has created an ā€˜early warning’ system which 
– Identifies students ā€˜at risk’ of not completing a course 
– Deploys an intervention to increase chances of success 
• System automates the intervention process 
– Student gets ā€˜traffic light’ alert via VLE, and 
– an email/message suggesting corrective action
Purdue University Signals: real time traffic-lights for 
students based on predictive model 
ā€œResults thus far show that students 
who have engaged with Course Signals 
have higher average grades and seek 
out help resources at a higher rate 
than other students.ā€ 
Pistilli, M. D., Arnold, K. and Bethune, M., Signals: Using Academic Analytics to 
Promote Student Success. EDUCAUSE Review Online, July/Aug., (2012). 
http://www.educause.edu/ero/article/signals-using-academic-analytics-promote- 
student-success
Knewton (Arizona State Univ) 
• A continuously adaptive online learning 
platform 
• Logs data about student behaviour and 
performance (e.g. keystrokes, scores, speed, 
etc) 
• Analyses behavioural and 
performance data, comparing it 
with similar students and 
assessing relevance of 
educational content to students 
• Serves each individual student 
the most appropriate learning 
activity for them at a particular 
moment in time
#Learning analytics as a digital Sorting Hat
https://www.flickr.com/photos/uncloned/5370399502
Celebrity photos scandal a wake-up call 
for cloud users 
http://www.thebureauinvestigates.com/category/projects/surveillance-2/
https://www.youtube.com/watch?v=F7pYHN9iC9I
https://www.flickr.com/photos/zigazou76/5824384001/sizes/z
https://www.flickr.com/photos/jes8jes/9655367348
Developing new policy 
Drawing upon existing practice, existing literature 
No comparable policy 
within HE sector 
Sharon Slade and Paul Prinsloo, "Learning Analytics: Ethical Issues and Dilemmas," in 
American Behavioral Scientist, Vol. 57, 2013, p. 1514. doi: 10.1177/0002764213479366
New OU policy for the ethical use 
of learning analytics 
Principle 1: Learning analytics is a moral 
practice, which should align with core 
organisational principles. 
Principle 2: The OU has a responsibility to all 
stakeholders to use and extract meaning from 
student data for the benefit of students where 
feasible.
Principle 3: Students are not wholly defined by 
their visible data or our interpretation of that 
data. 
Principle 4: The purpose and the boundaries 
regarding the use of learning analytics should be 
well defined and visible.
Principle 5: The OU should aim to be transparent 
regarding data collection, and provide students 
with the opportunity to update their own data 
and consent agreements at regular intervals. 
Principle 6: Students should be engaged as active 
agents in the implementation of learning analytics 
(e.g. informed consent, personalised learning 
paths, interventions).
Principle 7: Modelling and interventions based 
on analysis of data should be sound and free 
from bias. 
Principle 8: Adoption of learning analytics 
within the OU requires broad acceptance of the 
values and benefits (organisational culture) and 
the development of appropriate skills across the 
organisation.
the strive for clarity 
https://www.flickr.com/photos/pentog/4495052859
https://www.flickr.com/photos/savvyduck/7903341834
transparency of purpose 
https://www.flickr.com/photos/williamcromar/5338216221
https://www.flickr.com/photos/rooreynolds/46541511
https://www.flickr.com/photos/nffcnnr/5399478788
getting the balance right 
http://www.educause.edu/ero/article/learning-analytics-and-ethics-framework-beyond-utilitarianism 
https://www.flickr.com/photos/pie4dan/4567311801

SAAIR 2014 keynote Sharon Slade

  • 1.
    Learning analytics – opportunities and issues Dr Sharon Slade The Open University, UK 21st SAAIR Conference 2014 Yesterday, today and tomorrow: 21 years of Institutional Research
  • 5.
    Learning analytics isthe measurement, collection, analysis and reporting of data about learners to increase our understanding of them and their learning needs, and to use that understanding to influence their learning.
  • 6.
    ā€˜99% of whoyou are is invisible’ R. Buckminster Fuller ā€˜ http://www.glennsasscer.com/wordpress/wp-content/ uploads/2011/10/iceberg.jpg
  • 7.
    What do wemean by data about learners? Background Disability Gender Learning behaviours Posting to forums - content Ethnicity Study goals Study history Learning style Funding issues Assignment/test scores Websites visited Age Hitting study milestones Location Working status Family income Language Log in frequency Posting to forums - frequency Frequency of contact with tutor
  • 8.
    It’s everywhere: everylearning tool now has an ā€œanalytics dashboardā€ (a Google image search) 8
  • 9.
    The OU analyticsinnovation to impact cycle annual cycles of quality enhancement and research capability and student number planning maintain analysis models manage data store responsibility for architecture Innovation Impact Research, benchmarking and rapid prototyping Data storage and access for analysis Data collection Technology architecture Actionable insight Learning design Information advice and guidance Informed student interventions Quality enhancement processes Define new research questions Innovation mainstreamed into operations Requirements for new data collection Manage core systems and support surveys ongoing insight build and manage systems and tools, provide process reports New indicators identified Mainstream indicators Mainstreamed analysis models Data harvesting from core student record and learning systems Systems Students Outcomes Evidence based learning design and continuous improvement → increased student attainment over time More targeted and appropriate IAG → decrease in early drop-out Interventions are better targeted and more effective → increased retention over time Direct feedback from students Mainstream BAU http://www.jumpoffthescreen.com/analytics.php
  • 11.
  • 12.
    Purdue’s Course signals • Uses a predictive model based on – VLE activity and assessment scores • Previous academic history and demographic data • Has created an ā€˜early warning’ system which – Identifies students ā€˜at risk’ of not completing a course – Deploys an intervention to increase chances of success • System automates the intervention process – Student gets ā€˜traffic light’ alert via VLE, and – an email/message suggesting corrective action
  • 13.
    Purdue University Signals:real time traffic-lights for students based on predictive model ā€œResults thus far show that students who have engaged with Course Signals have higher average grades and seek out help resources at a higher rate than other students.ā€ Pistilli, M. D., Arnold, K. and Bethune, M., Signals: Using Academic Analytics to Promote Student Success. EDUCAUSE Review Online, July/Aug., (2012). http://www.educause.edu/ero/article/signals-using-academic-analytics-promote- student-success
  • 14.
    Knewton (Arizona StateUniv) • A continuously adaptive online learning platform • Logs data about student behaviour and performance (e.g. keystrokes, scores, speed, etc) • Analyses behavioural and performance data, comparing it with similar students and assessing relevance of educational content to students • Serves each individual student the most appropriate learning activity for them at a particular moment in time
  • 17.
    #Learning analytics asa digital Sorting Hat
  • 18.
  • 19.
    Celebrity photos scandala wake-up call for cloud users http://www.thebureauinvestigates.com/category/projects/surveillance-2/
  • 20.
  • 21.
  • 22.
  • 23.
    Developing new policy Drawing upon existing practice, existing literature No comparable policy within HE sector Sharon Slade and Paul Prinsloo, "Learning Analytics: Ethical Issues and Dilemmas," in American Behavioral Scientist, Vol. 57, 2013, p. 1514. doi: 10.1177/0002764213479366
  • 24.
    New OU policyfor the ethical use of learning analytics Principle 1: Learning analytics is a moral practice, which should align with core organisational principles. Principle 2: The OU has a responsibility to all stakeholders to use and extract meaning from student data for the benefit of students where feasible.
  • 25.
    Principle 3: Studentsare not wholly defined by their visible data or our interpretation of that data. Principle 4: The purpose and the boundaries regarding the use of learning analytics should be well defined and visible.
  • 26.
    Principle 5: TheOU should aim to be transparent regarding data collection, and provide students with the opportunity to update their own data and consent agreements at regular intervals. Principle 6: Students should be engaged as active agents in the implementation of learning analytics (e.g. informed consent, personalised learning paths, interventions).
  • 27.
    Principle 7: Modellingand interventions based on analysis of data should be sound and free from bias. Principle 8: Adoption of learning analytics within the OU requires broad acceptance of the values and benefits (organisational culture) and the development of appropriate skills across the organisation.
  • 28.
    the strive forclarity https://www.flickr.com/photos/pentog/4495052859
  • 29.
  • 30.
    transparency of purpose https://www.flickr.com/photos/williamcromar/5338216221
  • 31.
  • 32.
  • 34.
    getting the balanceright http://www.educause.edu/ero/article/learning-analytics-and-ethics-framework-beyond-utilitarianism https://www.flickr.com/photos/pie4dan/4567311801