Learning dashboard for supporting students: from first-year engineering to MOOC students

Learning dashboard for supporting students:
from first-year engineering
to MOOC students
Tom Broos & Tinne De Laet
Tom.Broos@kuleuven.be
@TomBroos
Tinne.DeLaet@kuleuven.be
@TinneDeLaet

“Learning analytics is about
collecting traces that learners
leave behind and using those
traces to improve learning.”
- Erik Duval
Learning Analytics and Educational Data Mining, Erik Duval’s Weblog, 30 January 2012, https://erikduval.wordpress.com/2012/01/30/learning-analytics-and-educational-data-mining/ 2
Learning Analytics?

Learning Dashboards?
3Dashboard Confusion, Stephen Few, Intelligent Enterprise, March 20, 2004
“A dashboard is a visual display of the
most important information needed to
achieve one or more objectives;
consolidated and arranged on a single
screen so the information can be monitored
at a glance.”
- Stephen Few

Successful Transition from secondary to higher
Education using Learning Analytics
enhance a successful transition from
secondary to higher education by means of
learning analytics
 design and build analytics dashboards,
 dashboards that go beyond identifying at-risk
students, allowing actionable feedback for all
students on a large scale.
10
www.stela-project.eu
@STELA_project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD

STELA
11
actionable feedback
student-centered
program level
inclusive
first-year experience
institution-wide
Learning Analytics
actual implementation

[!] Feedback must be “actionable”.
12
Warning!
Male students have
10% less probability to
be successful.
You are male.
Warning!
Your online activity is
lagging behind.
action?
?
action?
?


[!] Start with the available data.
Lots of data may eventually become
available in the future …
…. already start with what is available
13
(*)
(*) Zarraonandia, T., Aedo, I., Díaz, P., & Montero, A. (2013). An augmented lecture feedback system to support learner and teacher communication.
British Journal of Educational Technology, 44(4), 616-628.

[!] Involve stakeholder expertise.
15
visualization experts
practitioners / end-users
researchers LA
researchers first-year
study success
Charleer S., Vande Moere A., Klerkx J., Verbert K., De Laet T. (2017). Learning Analytics Dashboards to Support Adviser-Student Dialogue.
In IEEE Transactions on Learning Technology (http://ieeexplore.ieee.org/document/7959628/).

Student-facing learning dashboards
REX – LASSI – Learning Tracker

founded 1425
research-intensive university
16 faculties → general
university
> 55 000 students
2016-2017-2018 Europe’s
most innovative university
(Reuters ranking)

learning dashboards @KU Leuven
interaction
self-reflection
STUDENT
ADVISOR
STUDENT LASSI –
learning skills
REX - scoresPOS – future
students

Explore one of the dashboards (10 min)
21
inspiration
https://learningdashboards.eu/
Learning Tracker
Be ready to explain what the dashboard
is doing in 2 minutes! (next round)

Explain what the dashboard is doing in
your group (10 minutes)
22
inspiration
https://learningdashboards.eu/
Learning Tracker

[!] Start with the available data.
23
data already available?
administrative (examples)
student records course grades
systems (examples)
LMS access logs advisor meetings
)
Broos T., Verbert K., Van Soom C., Langie G., De Laet T.# (2018). Small data as a conversation starter for learning analytics: exam results dashboard for first-year students in higher
education. Journal of Research in Innovative Teaching & Learning, , 1-14.
demo: https://learninganalytics.set.kuleuven.be/static-demo-rex/ (en) or https://learninganalytics.set.kuleuven.be/demo/rex-1718jan-ir (nl)

Stakeholder involvement: example 2
24

25

26
Student
is
starter?
Study
efficienc
y?
Mail invitation
Student visits
dashboard
Reminder LOW GLOBAL SE
TIPS
REGULATIONS
MID GLOBAL SE
HIGH GLOBAL SE
ADVICE
INTRODUCTION
MAIL
REMINDER
SCORE

27

Evaluation - REX
29
571
76 245 220 353 258 274 1060 187 283 843
242
163
38 130 120 200 151 162 647 128 230 771
269
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
clicked non clicked

What can we learn from dashboard usage?
30Broos T., Verbert K., Van Soom C., Langie G., De Laet T.# (2018). Low-investment, Realistic-Return Business Cases for Learning Analytics Dashboards: Leveraging Usage Data and
Microinteractions. accepted for ECTEL 2018
p<1e-5 p<1e-9p-test

Dashboard learning skills - LASSI
31
students complete LASSI
questionnaire
students received personalized email
with invitation for dashboard
4367 students in 26 programs
in 9 faculties @KU Leuven
demo:
https://learninganalytics.set.kuleuven.be/lassi-1718/ (KU Leuven login)
2 programs @TU Delft

Response
32
3868 students (89%)
used dashboard

Student feedback?
33
http://blog.associatie.kuleuven.be/tinnedelaet/learning-dashboard-for-actionable-feedback-on-learning-and-studying-skills/
How CLEAR is this info?
stars stars

Students that click through
Broos, T., Peeters, L., Verbert, K., Van Soom, C., Langie, G., & De Laet, T. (2017, July). Dashboard for Actionable Feedback on Learning Skills: Scalability and Usefulness.
In International Conference on Learning and Collaboration Technologies (pp. 229-241). Springer, Cham.
34
 better learning skills

More intense users
Broos, T., Peeters, L., Verbert, K., Van Soom, C., Langie, G., & De Laet, T. (2017, July). Dashboard for Actionable Feedback on Learning Skills: Scalability and Usefulness.
In International Conference on Learning and Collaboration Technologies (pp. 229-241). Springer, Cham.
35
 worse learning skills

[!] Acceptance precedes impact.
36
• Involve stakeholders from the start and
value their input!
COmmunication
COoperation
• Demonstrate usefulness.
• Take care of ethics and privacy.
• Best scenario:
students & study advisors as ambassadors
COCO

Impact?
survey before intervention
 2nd year students 2016-2017
 experiences first-year feedback
 41 vragen, 5-point Likert scale
 pen & paper
dashboards
 LISSA
 LASSI (learning skills)
 3 x REX (grades)
Survey after intervention
 2nd year students 2017-2018

Impact?
During the first year I received sufficient information regarding my academic achievements.
38
Engineering Science (p<0.001)

Impact?
The information I received helped to position myself with respect to my peers.
39
Engineering Science (p<0.001)

Impact?
40
The information I received made me reflect.
The information I received made me adapt my behaviour.

Learning analytics for learners
Do learners change their behaviour when confronted
with their learning performance relative to that of
successful learners?
Tracker deployed
in the Drinking
Water MOOC
Dan Davis, Guanliang Chen, Ioana Jivet, Claudia Hauff, and Geert-Jan Houben. Encouraging Metacognition
& Self-Regulation in MOOCs through Increased Learner Feedback. LAL Workshop 2016.
5,462 learners
were exposed to
the intervention
WIS Research Project

Learning Tracker - results
WaterX
C 1,268 160
MOOC COND. N # PASS PASS RATE
T 1,251 188
12.6%
15.0%
UrbanX
C 771 136
T 746 165
17.6%
22.1%
BusinessX
C 164 46
T 160 54
28.0%
33.8%
OVERALL
C 2,203 342
T 2,157 407
15.5%
18.9%
**

Learning Tracker - results
Control 1,150 45
CONDITION N # PASS PASS RATE
3.9%
Indiv. - Promotion 1,147 62 5.4%
Collect. - Prevent. 1,118 51 4.5%
Note: If activity threshold is set from 5 mins
to 1 hour, CalcX Pass rate changes from 5%
to 16%

[!] Context matters!
• available data
• national and institutional regulations
and culture
• educational vision
• educational system, size of population ..
• …
Don’t just copy existing LA solutions!
44

Key messages
ETHICS & PRIVACY
[!] Use all available expertise (study advisers)
[!] Don’t oversimplify. Show uncertainty.
[!] Beware of predictive algorithms.
[!] Wording matters.
[!] Give students “the key” to their data.
[!] Learning Analytics is an opportunity within GDPR:
“useful” view to the data
IMPACT
[!] Acceptance precedes impact.
[!] Impact is hard to measure/obtain.
[!] large scale through: “general” and “specifc”
interventions
DATA
[!] Start with the available data. (small data approach)
[!] Look beyond the obvious data.
[!] Not all data is usable.
[!] Keep Learning Analytics in mind when designing
learning activities.
[!] Learning dashboards themselves create new learning
traces
SCALABILITY & TRANSFERABILITY
[!] Context matters!
[repeat] Start with available data (small data approach)
[!] modularity of software solutions more open important
than open-source
[!] realistic long-term deployments require embedding in
existing IT choices and knowhow of higher education
institute
[!] Portability of data (documentation, meta data) is more
important than standardization

Thank you for participating!
www.stela-project.eu
@STELA_Project

Learning dashboard for supporting students: from first-year engineering to MOOC students

Learning dashboard for supporting students: from first-year engineering to MOOC students

More Related Content

What's hot

Similar to Learning dashboard for supporting students: from first-year engineering to MOOC students

More from Tinne De Laet

Recently uploaded

Learning dashboard for supporting students: from first-year engineering to MOOC students