Download as PDF, PPTX
![[!] Feedback must be “actionable”.
12
Warning!
Male students have
10% less probability to
be successful.
You are male.
Warning!
Your online activity is
lagging behind.
action?
?
action?
?
](https://image.slidesharecdn.com/20180710lalaconference-181113195112/75/Co-designing-learning-dashboards-for-scalable-feedback-12-2048.jpg)
![[!] Start with the available data.
Lots of data may eventually become
available in the future …
…. already start with what is available
14
(*)
(*) Zarraonandia, T., Aedo, I., Díaz, P., & Montero, A. (2013). An augmented lecture feedback system to support learner and teacher communication.
British Journal of Educational Technology, 44(4), 616-628.](https://image.slidesharecdn.com/20180710lalaconference-181113195112/75/Co-designing-learning-dashboards-for-scalable-feedback-14-2048.jpg)
![[!] Involve stakeholder expertise.
16
visualization experts
practitioners / end-users
researchers LA
researchers first-year
study success
Charleer S., Vande Moere A., Klerkx J., Verbert K., De Laet T. (2017). Learning Analytics Dashboards to Support Adviser-Student Dialogue.
In IEEE Transactions on Learning Technology (http://ieeexplore.ieee.org/document/7959628/).](https://image.slidesharecdn.com/20180710lalaconference-181113195112/75/Co-designing-learning-dashboards-for-scalable-feedback-16-2048.jpg)
![[!] Do not oversimplify. Show
uncertainty.
28
• reality is complex
• measurement is limited
• individual circumstances
• need for nuance
• trigger reflection
http://blog.associatie.kuleuven.be/tinnedelaet/the-nonsense-of-chances-of-success-and-predictive-models/](https://image.slidesharecdn.com/20180710lalaconference-181113195112/75/Co-designing-learning-dashboards-for-scalable-feedback-28-2048.jpg)
![[!] Start with the available data.
30
data already available?
administrative (examples)
student records course grades
systems (examples)
LMS access logs advisor meetings
)
Broos T., Verbert K., Van Soom C., Langie G., De Laet T.# (2018). Small data as a conversation starter for learning analytics: exam results dashboard for first-year students in higher
education. Journal of Research in Innovative Teaching & Learning, , 1-14.
demo: https://learninganalytics.set.kuleuven.be/static-demo-rex/ (en) or https://learninganalytics.set.kuleuven.be/demo/rex-1718jan-ir (nl)](https://image.slidesharecdn.com/20180710lalaconference-181113195112/75/Co-designing-learning-dashboards-for-scalable-feedback-30-2048.jpg)
![[!] Context matters!
• available data
• national and institutional regulations
and culture
• educational vision
• educational system, size of population ..
• …
Don’t just copy existing LA solutions!
40](https://image.slidesharecdn.com/20180710lalaconference-181113195112/75/Co-designing-learning-dashboards-for-scalable-feedback-40-2048.jpg)
More Related Content
Similar to Co-designing learning dashboards for scalable feedback
Co-designing learning dashboards for scalable feedback
- 1. Co-designing learning dashboards forscalable feedback Tom Broos & Tinne De Laet [email protected] @TomBroos [email protected] @TinneDeLaet
- 2. largest university inBelgium founded 1425 16 faculties → general university 55 000 students tuition fee for 1 year < 1 3 average monthly income
- 3. “Learning analytics isabout collecting traces that learners leave behind and using those traces to improve learning.” - Erik Duval Learning Analytics and Educational Data Mining, Erik Duval’s Weblog, 30 January 2012, https://erikduval.wordpress.com/2012/01/30/learning-analytics-and-educational-data-mining/ 3 Learning Analytics?
- 4. Learning Dashboards? 4Dashboard Confusion,Stephen Few, Intelligent Enterprise, March 20, 2004 “A dashboard is a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance.” - Stephen Few
- 10. Successful Transition fromsecondary to higher Education using Learning Analytics enhance a successful transition from secondary to higher education by means of learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback for all students on a large scale. Achieving Benefits from Learning Analytics research strategies and practices for using learning analytics to support students during their first year at university developing the technological aspects of learning analytics, focuses on how learning analytics can be used to support students. 10 www.stela-project.eu @STELA_project 2015-1-UK01-KA203-013767 www.ableproject.eu @ABLE_project_eu 562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
- 11. STELA ♥ ABLE 11 actionablefeedback student-centered program level inclusive first-year experience institution-wide Learning Analytics actual implementation
- 12. [!] Feedback mustbe “actionable”. 12 Warning! Male students have 10% less probability to be successful. You are male. Warning! Your online activity is lagging behind. action? ? action? ?
- 13. learning dashboards @KULeuven interaction self-reflection LISSA STUDENT ADVISOR STUDENT LASSI – learning skills REX - scoresPOS – future students
- 14. [!] Start withthe available data. Lots of data may eventually become available in the future … …. already start with what is available 14 (*) (*) Zarraonandia, T., Aedo, I., Díaz, P., & Montero, A. (2013). An augmented lecture feedback system to support learner and teacher communication. British Journal of Educational Technology, 44(4), 616-628.
- 16. [!] Involve stakeholderexpertise. 16 visualization experts practitioners / end-users researchers LA researchers first-year study success Charleer S., Vande Moere A., Klerkx J., Verbert K., De Laet T. (2017). Learning Analytics Dashboards to Support Adviser-Student Dialogue. In IEEE Transactions on Learning Technology (http://ieeexplore.ieee.org/document/7959628/).
- 18. Grade data two dashboards:LISSA & REX
- 19. ERP- CM DWH CSV ETLLA- DWH
- 20.
- 21.
- 22. LISSA dashboard interaction student– study advisor
- 23. Study advisor –student conversations 23 Should I consider another program? Can I still finish the bachelor in 3 years? How should I compose my program for next year? What is the personal situation? How can I help? What is the best next step?
- 24.
- 25. LISSA: status 25 26 programs>4500 students 114 student advisors training of study advisors • Charleer S., Vande Moere A., Klerkx J., Verbert K., De Laet T. (2017). Learning Analytics Dashboards to Support Adviser-Student Dialogue. In IEEE Transactions on Learning Technology • http://blog.associatie.kuleuven.be/tinnedelaet/lissa-learning-dashboard-supporting-student-advisers-in-traditional-higher-education/ • Millecamp M., Gutiérrez F., Charleer S., Verbert K., De Laet T.# (2018). A qualitative evaluation of a learning dashboard to support advisor-student dialogues. Proceedings of the 8th International Learning Analytics & Knowledge Conference. LAK. Sydney, 5-9 March 2018 (pp. 1-5) ACM. dashboards for three examination periods observations, interviews, questionnaires
- 26. How to determine thresholdsfor different groups? Stakeholder involvement: example 1 26 upper and lower group: clear message middle group as small as possible Do not overfit! (nuance)
- 27. Evaluation – interviews “Whenstudents see the numbers, they are surprised, but now they believe me. Before, I used my gut feeling, now I feel more certain of what I say as well”. “It’s like a main thread guiding the conversation.” “I can talk about what to do with the results, instead of each time looking for the data and puzzling it together.” “Students don’t know where to look during the conversation, and avoid eye contact. The dashboard provides them a point of focus”. “A student changed her study method in June and could now see it paid off.” LISSA supports a personal dialogue. the level of usage depends on the experience and style of the study advisors fact-based evidence at the side narrative thread key moments and student path help to reconstruct personal track “I can focus on the student’s personal path, rather than on the facts.” “Now, I can blame the dashboard and focus on collaboratively looking for the next step to take.” 27
- 28. [!] Do notoversimplify. Show uncertainty. 28 • reality is complex • measurement is limited • individual circumstances • need for nuance • trigger reflection http://blog.associatie.kuleuven.be/tinnedelaet/the-nonsense-of-chances-of-success-and-predictive-models/
- 29.
- 30. [!] Start withthe available data. 30 data already available? administrative (examples) student records course grades systems (examples) LMS access logs advisor meetings ) Broos T., Verbert K., Van Soom C., Langie G., De Laet T.# (2018). Small data as a conversation starter for learning analytics: exam results dashboard for first-year students in higher education. Journal of Research in Innovative Teaching & Learning, , 1-14. demo: https://learninganalytics.set.kuleuven.be/static-demo-rex/ (en) or https://learninganalytics.set.kuleuven.be/demo/rex-1718jan-ir (nl)
- 31.
- 32.
- 33. Stakeholder involvement: example2 33 Student is starter? Study efficienc y? Mail invitation Student visits dashboard Reminder LOW GLOBAL SE TIPS REGULATIONS MID GLOBAL SE HIGH GLOBAL SE ADVICE INTRODUCTION MAIL REMINDER SCORE
- 34.
- 35.
- 36. What can welearn from dashboard usage? 36Broos T., Verbert K., Van Soom C., Langie G., De Laet T.# (2018). Low-investment, Realistic-Return Business Cases for Learning Analytics Dashboards: Leveraging Usage Data and Microinteractions. accepted for ECTEL 2018 p<1e-5 p<1e-9p-test
- 37. Design your owngrading dashboard
- 39. Design your own considerations •data • stakeholders • textual and visual elements • incorporating in university practices 39 inspiration https://learningdashboards.eu/
- 40. [!] Context matters! •available data • national and institutional regulations and culture • educational vision • educational system, size of population .. • … Don’t just copy existing LA solutions! 40
- 41. Project team @ 41 SvenCharleer AugmentHCI, Computer Science department PhD researcher ABLE Katrien Verbert AugmentHCI, Computer Science department Copromotor of STELA & ABLE Carolien Van Soom Leuven Engineering and Science Education Center Head of Tutorial Services of Science Copromotor of STELA & ABLE Greet Langie Leuven Engineering and Science Education Center Vicedean (education) faculty of Engineering Technology Copromotor of STELA & ABLE Tinne De Laet Leuven Engineering and Science Education Center Head of Tutorial Services of Engineering Science Coordinator of STELA KU Leuven coordinator of ABLE Francisco Gutiérrez AugmentHCI, Computer Science department PhD researcher ABLE Tom Broos Leuven Engineering and Science Education Center AugmentHCI, Computer Science department PhD researcher STELA Martijn Millecamp AugmentHCI, Computer Science department PhD researcher ABLE Special thanks to study advisors for their cooperation, advice, feedback, and support! Jasper, Bart, Riet, Hilde, An, Katrien, … ♥