Download as KEY, PPTX
Uploaded byIsrael Gutiérrez
The feedback loop revisited
The document discusses improving feedback loops in educational settings through the use of new technologies. It proposes a system called classON that uses web notifications, student interaction tracking, and learning analytics to better orchestrate classroom sessions. This would allow teachers to provide more efficient formative feedback, analyze student engagement and learning, and refine instructional designs. The goal is to actively engage students, support assessment, and increase awareness between teachers and learners.
More Related Content
![[OOFHEC2018] Irene O'Dowd: Examining student engagement with an online toolki...](https://cdn.slidesharecdn.com/ss_thumbnails/ireneodowdhiberniacollege2018final-181128123313-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to The feedback loop revisited
The feedback loop revisited
- 1. The feedback looprevisited Israel Gutiérrez Universidad Carlos III de Madrid
- 2. Context photo: uc3mcom at flickr
- 3. Context Lab session practicesin computer rooms photo: uc3mcom at flickr
- 4. Context Lab session practices in computer rooms Active learning activities: guided and open ended photo: uc3mcom at flickr
- 5. Scenario photo: candicejeanl at flickr
- 6. Scenario Multimedia Applications (AMM) photo: candicejeanl at flickr
- 7. Scenario MultimediaApplications (AMM) Lab sessions are 50% of the lectures photo: candicejeanl at flickr
- 8. Scenario Multimedia Applications (AMM) Lab sessions are 50% of the lectures Work in pairs and in a proposed practice photo: candicejeanl at flickr
- 9. Problems photo: berkeleylab at flickr
- 10. Problems Feedback to students:effective but inefficient photo: berkeleylab at flickr
- 11. Problems Feedback to students:effective but inefficient Issues in the orchestration of the session photo: berkeleylab at flickr
- 12. Problems photo: hackny at flickr
- 13. Problems Feedback to theclass: efficient but ineffective photo: hackny at flickr
- 14. Problems Feedback to theclass: efficient but ineffective Students engagement photo: hackny at flickr
- 15.
- 16. Possible solutions Use abackchannel chat to improve communication
- 17. Possible solutions Usea backchannel chat to improve communication Conole G., and Alevizou P (2010) “A literature review of the use of Web 2.0 tools in Higher Education”. A report commissioned by the Higher Education Academy.
- 18. Possible solutions Usea backchannel chat to improve communication Conole G., and Alevizou P (2010) “A literature review of the use of Web 2.0 tools in Higher Education”. A report commissioned by the Higher Education Academy. Yardi, S. “Whispers in the Classroom." Digital Youth, Innovation, and the Unexpected. Edited by Tara McPherson. The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning. Cambridge, MA: The MIT Press, 2008. 143–164.
- 19.
- 20. Possible solutions Improve orchestrationat enactment by means of lanterns for distributed awareness
- 21. Possible solutions Improve orchestration at enactment by means of lanterns for distributed awareness Alavi, H. S., Dillenbourg, P., & Kaplan, F. (2009). Distributed awareness for class orchestration. Learning in the Synergy of Multiple Disciplines -- Proceedings of the 4th European Conference on Technology Enhanced Learning (pp. 211-225). Springer.
- 22. !" Possible solutions Paulo Blikstein" ! #$%&'(")*"+,("-./("012$%15.'6"7,$8,"1==.74"'(4(1'8,('4"5."%."<18?"19/"@.'5,"$9"5$; 2 1 ( 1 ;("5'18?$9%"
- 23. !" Possible solutions Paulo Blikstein" Learning analytics for assessment ! #$%&'(")*"+,("-./("012$%15.'6"7,$8,"1==.74"'(4(1'8,('4"5."%."<18?"19/"@.'5,"$9"5$; 2 1 ( 1 ;("5'18?$9%"
- 24. !" Possible solutions Paulo Blikstein" Learning analytics for assessment ! #$%&'(")* P. (2011). Using learning analytics to assess students ’ behavior in open ‐ ended programming Blikstein, "+,("-./("012$%15.'6"7,$8,"1==.74"'(4(1'8,('4"5."%."<18?"19/"@.'5,"$9"5$; 2 1 ( 1 ;("5'18?$9%" tasks. Proceedings of the 1st International Conference on Learning Analytics and Knowledge 2011
- 25. Our solution Impact for Impact for Measures students teachers
- 26. Our solution One-way websocketnotifications Impact for Impact for Measures students teachers
- 27. Our solution One-way websocketnotifications Improve attention and effectiveness of general feedback Impact for Impact for Measures students teachers
- 28. Our solution One-way websocketnotifications Improve attention and effectiveness of general feedback Record student interactions and use learning analytics Impact for Impact for Measures students teachers
- 29. Our solution One-way websocketnotifications Improve attention and effectiveness of general feedback Record student interactions and use learning analytics Improve orchestration in enactment and provision of efficient feedback Impact for Impact for Measures students teachers
- 30. Our solution One-way websocketnotifications Improve attention and effectiveness of general feedback Record student interactions and use learning analytics Improve orchestration in enactment and provision of efficient feedback Improve orchestration design for the following session and acquiring evidences for summative assessment Impact for Impact for Measures students teachers
- 31. Our solution One-way websocketnotifications Improve attention and effectiveness of general feedback Record student interactions and use learning analytics Improve orchestration in enactment and provision of efficient feedback Improve orchestration design for the following session and acquiring evidences for summative assessment Engagement in lab sessions (support of other artifacts) Impact for Impact for Measures students teachers
- 32. Our solution One-way websocketnotifications Improve attention and effectiveness of general feedback Record student interactions and use learning analytics Improve orchestration in enactment and provision of efficient feedback Improve orchestration design for the following session and acquiring evidences for summative assessment Engagement in lab sessions (support of other artifacts) Awareness, reflection and reaction Impact for Impact for Measures students teachers
- 33. Our solution One-way websocketnotifications Improve attention and effectiveness of general feedback classON Record student interactions and use learning analytics Improve orchestration in enactment and provision of efficient feedback Improve orchestration design for the following session and acquiring evidences for summative assessment Engagement in lab sessions (support of other artifacts) Awareness, reflection and reaction Impact for Impact for Measures students teachers
- 34.
- 35. classON aCtive Learning ASSessment Orchestration& Notifications
- 36. classON aCtive Learning ASSessment Orchestration & Notifications Enactment phase: on class
- 37. classON aCtive Learning ASSessment Orchestration & Notifications Enactment phase: on class Activate the class: improve orchestration
- 38. classON aCtive Learning ASSessment Orchestration & Notifications Enactment phase: on class Activate the class: improve orchestration Activate the class: students’ engagement
- 39.
- 40. classON Formative assessment andquick feedback
- 41. classON Formative HTML5 & assessment web and quick technologies feedback
- 42. classON Formative HTML5 & Attention & assessment web notification and quick technologies system feedback
- 43. classON Formative HTML5 & Attention & assessment web notification and quick technologies system feedback Learning analytics
- 44. classON Formative HTML5 & Attention & assessment web notification and quick technologies system feedback Orchestrating Learning learning at analytics enactment
- 45. classON Formative HTML5 & Attention & assessment web notification and quick technologies system feedback Orchestrating Learning Data learning at analytics visualization enactment
- 46. Formative assessment and feedback
- 47. Formative assessment and feedback Quick & efficient
- 48. Formative assessment and feedback 21st Century skills Quick & efficient
- 49. HTML5 & webtechnologies
- 50. HTML5 & webtechnologies • Web notifications for forefront messages
- 51. HTML5 & webtechnologies • Web notifications for forefront messages • Web sockets for fast/real-time communication (datavis in real time)
- 52. HTML5 & webtechnologies • Web notifications for forefront messages • Web sockets for fast/real-time communication (datavis in real time) • Key-value databases for a fast and efficient backend: MongoDB
- 53. HTML5 & webtechnologies • Web notifications for forefront messages • Web sockets for fast/real-time communication (datavis in real time) • Key-value databases for a fast and efficient backend: MongoDB • JavaScript libraries for datavis: d3, protovis...
- 54. Attention & notificationsystem
- 55. Attention & notificationsystem • Web notifications in order to make info available without changing attention (supported by websockets)
- 56. Attention & notificationsystem • Web notifications in order to make info available without changing attention (supported by websockets) • One-click subscription to the notifications channel from the wording of the session
- 57. Attention & notificationsystem • Web notifications in order to make info available without changing attention (supported by websockets) • One-click subscription to the notifications channel from the wording of the session • Simple interface for the teacher to send notifications to all the connected students
- 58.
- 59.
- 60. Experience • AMM 2011 •2 sessions
- 61. Experience • AMM 2011 •2 sessions • Increasing complexity
- 62. Experience • AMM 2011 •2 sessions • Increasing complexity • Increasing interest
- 63. Experience • AMM 2011 •2 sessions • Increasing complexity • Increasing interest
- 64.
- 65. Future works • Attentionimprovement delaying notification delivery
- 66. Future works • Attentionimprovement delaying notification delivery • Intelligent Tutoring System based on common problems (notifications log)
- 67. Future works • Attentionimprovement delaying notification delivery • Intelligent Tutoring System based on common problems (notifications log) • Collection of more student events and create real time visualizations
- 68. Keywords of theECTEL’11 short paper
- 69.
- 70. Learning analytics Learning analyticsis the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs. CfP of Conference on Learning Analytics and Knowledge 2011
- 71.
- 72. What to instrumentalize? • Lab session wording containing event collection in JavaScript
- 73. What to instrumentalize? • Lab session wording containing event collection in JavaScript • Login info from LMS in order to associate events to a student id
- 74. Event collection interface !
- 75. Event collection interface ! Buttons to start, finalize and indicate a problem in an activity
- 76. Event collection interface ! Buttons to start, finalize and Todo list indicate a problem in an activity of tasks
- 77.
- 78. Recorded Events • Accessto the wording
- 79. Recorded Events • Accessto the wording • Start working on a section
- 80. Recorded Events • Accessto the wording • Start working on a section • Finish working in a section
- 81. Recorded Events • Accessto the wording • Start working on a section • Finish working in a section • I have a problem / I solved the problem
- 82. Recorded Events • Accessto the wording • Start working on a section • Finish working in a section • I have a problem / I solved the problem • How the problem was solved: individual effort, peer interaction or teacher feedback
- 83. Recorded Events • Access to the wording • Start working on a section • Finish working in a section • I have a problem / I solved the problem • How the problem was solved: individual effort, peer interaction or teacher feedback How to translate that into learning outcomes achievement?
- 84. Contextual data • Physicaldistribution of the labs
- 85. Contextual data • Physicaldistribution of the labs !
- 86. Orchestrating learning at enactment
- 87. Orchestrating learning at enactment The metaphor of “orchestrating learning” has been frequently used to address the challenges of coordinating the complexity of conducting learning activities. Stellar D1.1
- 88.
- 89.
- 90.
- 91. Teacher information • Progress •Stuck in a problem
- 92. Teacher information • Progress •Stuck in a problem • Slow progress
- 93. Teacher information • Progress •Stuck in a problem • Slow progress • Consumed tutoring time
- 94. Teacher information • Progress •Stuck in a problem • Slow progress • Consumed tutoring time • Teacher location
- 95. Teacher information • Progress •Stuck in a problem • Slow progress • Consumed tutoring time • Teacher location • Next step
- 96. Teacher information • Progress 3 • Stuck in a problem • Slow progress • Consumed tutoring time • Teacher location • Next step
- 97. Teacher information • Progress 3 • Stuck in a problem 3 • Slow progress • Consumed tutoring time • Teacher location • Next step
- 98. Teacher information • Progress 3 • Stuck in a problem 3 • Slow progress • Consumed tutoring time • Teacher location • Next step
- 99. Teacher information • Progress 3 • Stuck in a problem 3 • Slow progress • Consumed tutoring time 2’ • Teacher location • Next step
- 100. Teacher information • Progress 3 • Stuck in a problem 3 • Slow progress • Consumed tutoring time 2’ • Teacher location • Next step
- 101. Teacher information • Progress 3 • Stuck in a problem 3 • Slow progress • Consumed tutoring time 2’ • Teacher location • Next step
- 102.
- 103.
- 104. Workflow • Use classONapp to monitor student problems and state in real time
- 105. Workflow • Use classONapp to monitor student problems and state in real time • In a normal situation teacher decides to
- 106. Workflow • Use classONapp to monitor student problems and state in real time • In a normal situation teacher decides to • Directly help one of the teams
- 107. Workflow • Use classONapp to monitor student problems and state in real time • In a normal situation teacher decides to • Directly help one of the teams • Make one team help another one
- 108. Workflow • Use classONapp to monitor student problems and state in real time • In a normal situation teacher decides to • Directly help one of the teams • Make one team help another one • Send a notification (since it’s a common problem)
- 109.
- 110. Data visualization Information visualizationis the interdisciplinary study of the visual representation of large- scale collections of information. Michael Friendly (2008). "Milestones in the history of thematic cartography, statistical graphics, and data visualization"
- 111. Intelligence Amplification (W. Ross Ashby,1956) Visualization Amplifies Cognition (@infosthetics, last Monday)
- 112. Intelligence Amplification (W. Ross Ashby,1956) rs ! h e a c r te l fo Visualization o o t Amplifies Cognition A (@infosthetics, last Monday)
- 113.
- 114. Critiques to datavis •Lack of usability and usefulness
- 115. Critiques to datavis •Lack of usability and usefulness • Not created for the average people
- 116. Critiques to datavis •Lack of usability and usefulness • Not created for the average people • Noise
- 117.
- 118. Datavis for students •Aesthetically attractive
- 119. Datavis for students •Aesthetically attractive • Fluent (straightforward and simple interaction)
- 120. Datavis for students •Aesthetically attractive • Fluent (straightforward and simple interaction) • Make them reflect (in and out of class)
- 121. Datavis for students •Aesthetically attractive • Fluent (straightforward and simple interaction) • Make them reflect (in and out of class) • Awareness of their knowledge and relate it to the average
- 122. Datavis for students •Aesthetically attractive • Fluent (straightforward and simple interaction) • Make them reflect (in and out of class) • Awareness of their knowledge and relate it to the average • React with an designed (re)action
- 123. Summing up: classON One-waywebsocket notifications Improve attention and effectiveness of general feedback Record student interactions and use learning analytics Improve orchestration in enactment and provision of efficient feedback Improve orchestration design for the following session and acquiring evidences for summative assessment Engagement in lab sessions (also other artifacts) Awareness, reflection and reaction Impact for Impact for Measures students teachers
- 124.
- 125. Next steps • Definedatavis for teachers’ and students’ interactions
- 126. Next steps • Definedatavis for teachers’ and students’ interactions • Development of a classON prototype
- 127. Next steps • Definedatavis for teachers’ and students’ interactions • Development of a classON prototype • Save events for every interaction with the system (even events manually annotated)
- 128. Next steps • Definedatavis for teachers’ and students’ interactions • Development of a classON prototype • Save events for every interaction with the system (even events manually annotated) • Use dataset to test the prototype
- 129.
- 130.
- 131. Evaluation • AMM 2012 •Control and experiment groups OR competition between groups
- 132. Evaluation • AMM 2012 •Control and experiment groups OR competition between groups • How can we measure engagement
- 133. Evaluation • AMM 2012 •Control and experiment groups OR competition between groups • How can we measure engagement • Qualitative feedback
- 135. Thank you verymuch! Any questions?
Editor's Notes
- #2 \n
- #3 \n
- #4 \n
- #5 \n
- #6 \n
- #7 \n
- #8 \n
- #9 \n
- #10 \n
- #11 \n
- #12 Literature review and then found an article about experiences of using a chat tool in the classroom\n
- #13 Literature review and then found an article about experiences of using a chat tool in the classroom\n
- #14 Literature review and then found an article about experiences of using a chat tool in the classroom\n
- #15 \n
- #16 \n
- #17 Assessing student learning in unscripted, open-ended environments.\n\n
- #18 Assessing student learning in unscripted, open-ended environments.\n\n
- #19 Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
- #20 Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
- #21 Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
- #22 Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
- #23 Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
- #24 Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
- #25 Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
- #26 Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
- #27 \n
- #28 \n
- #29 \n
- #30 \n
- #31 \n
- #32 \n
- #33 \n
- #34 \n
- #35 \n
- #36 \n
- #37 \n
- #38 \n
- #39 \n
- #40 \n
- #41 \n
- #42 \n
- #43 \n
- #44 \n
- #45 \n
- #46 \n
- #47 \n
- #48 \n
- #49 \n
- #50 \n
- #51 \n
- #52 \n
- #53 \n
- #54 \n
- #55 \n
- #56 \n
- #57 \n
- #58 \n
- #59 \n
- #60 \n
- #61 \n
- #62 \n
- #63 \n
- #64 \n
- #65 \n
- #66 \n
- #67 \n
- #68 \n
- #69 \n
- #70 \n
- #71 \n
- #72 \n
- #73 \n
- #74 \n
- #75 \n
- #76 \n
- #77 \n
- #78 \n
- #79 \n
- #80 \n
- #81 \n
- #82 \n
- #83 \n
- #84 \n
- #85 \n
- #86 \n
- #87 \n
- #88 \n
- #89 \n
- #90 \n
- #91 \n
- #92 \n
- #93 E.g., publish datavis every week at a certain time\n
- #94 E.g., publish datavis every week at a certain time\n
- #95 E.g., publish datavis every week at a certain time\n
- #96 E.g., publish datavis every week at a certain time\n
- #97 E.g., publish datavis every week at a certain time\n
- #98 \n
- #99 \n
- #100 \n
- #101 \n
- #102 \n
- #103 \n
- #104 \n
- #105 \n
- #106 \n
- #107 \n
- #108 \n