Downloaded 15 times
![Drop-down text about strategies
Clickable link.
[Click here to learn about the “What? Why? How?” strategy]](https://image.slidesharecdn.com/howonlineeducationalresourcesprovidenovelaffordancesforconductingpracticalinterventionsanddoingpsych-140125174344-phpapp02/75/How-online-educational-resources-provide-novel-affordances-for-conducting-practical-interventions-and-doing-psychology-experiments-24-2048.jpg)
Uploaded byJoseph Jay Williams
How online educational resources provide novel affordances for conducting practical interventions and doing psychology experiments
Online educational resources can be used to conduct experiments that link real-world learning environments with laboratory research. The document discusses an experiment on Khan Academy that tested the effects of adding motivational messages on student math practice. Growth mindset messages led to reduced dropout rates and increased problems practiced and accuracy compared to practice-as-usual or positive messages. Simultaneous applied and basic research is possible by starting with practical goals within constraints like randomization. Future directions discussed include teaching learning strategies, a learning coach, and MOOC experiments. A MOOClets research group meets weekly to develop digital modules combining research and practice.
More Related Content
Similar to How online educational resources provide novel affordances for conducting practical interventions and doing psychology experiments
How online educational resources provide novel affordances for conducting practical interventions and doing psychology experiments
- 1. How online educationalresources provide novel affordances for conducting practical interventions and doing psychology experiments www.josephjaywilliams.com/experiments-online-education Joseph Jay Williams, Stanford University Lytics Lab (lytics.stanford.edu), Graduate School of Education Office of the Vice Provost of Online Learning (vpol.stanford.edu) [email protected] www.josephjaywilliams.com Originally from Trinidad – apologies for thick accent! 1
- 2. Overview • • • • • • MOOCs VS DigitalOnline Resources Experiments: Link Real World & Laboratory Motivational Messages Experiment Simultaneous Basic & Applied Research Ed-Surge Meetup to link Research & Practice (Wed Feb 19 @ 7) Current Directions Teaching Learning Strategies Learning Coach MOOC Experiment • MOOClets Research & Practice Group 2
- 3.
- 4. Moving from Physicalto Digital World
- 5. Experiments: Link RealWorld & Laboratory • Randomized assignment • Experimental Control • Rich data • Real-world environment • Authentic activities • Practical Challenges
- 6. Experiment • Practical goalon Khan Academy • Extend motivation research 6
- 7. Motivate students topractice math • Online (Math) Exercise Motivational Message: Keep up the good work! 1. Number of Problems Practiced 2. Improve Accuracy 7
- 8. Add motivational messages Practice-as-usual GrowthMindset Message Remember, the more you practice the smarter you become! 8
- 9. Which messages toadd? • • • • • • "This might be a tough problem, but we know you can do it!” Remember, the more you practice the smarter you become! Some of these problems are hard. Just do your best.” Encourage “Growth Mindset” beliefs about intelligence (Dweck, 2008) Fixed Mindset Growth Mindset 9
- 10. Embedded in vivoExperiment Jascha Sohl-Dickstein • Practice-as-usual • Growth Mindset Message • • "Remember, the more you practice the smarter you become.”, "Mistakes help you learn. Think hard to learn from them.” 10
- 11. Results: More motivated? •Growth Mindset Message > Practice-asUsual • reduction in dropout rate • increase in Number of problems practiced • increase in Number Correct • Accuracy: Problems correct/Problems attempted • Accuracy improvement 11
- 12. Interpreting results • Obvious? •Skepticism: Anything’s statistically signficant in a MOOC 12
- 13. Different motivational messages Practice-as-usualMessage Growth Mindset Message Positive Remember, the more you practice the smarter you become! Some of these problems are hard. Do your best! 13
- 14. Does any positivemessage work? • Practice-as-usual • Growth Mindset Message • • "Remember, the more you practice the smarter you become.”, "Mistakes help you learn. Think hard to learn from them.” • Positive Message • • "Some of these problems are hard. Just do your best." "This might be a tough problem, but we know you can do it.” 14
- 15. Effects of PositiveMessages? • Positive Messages not better than Practice-as-Usual! • Growth Mindset better than Positive • reduction in dropout rate • increase in Number of problems practiced • increase in Number Correct • Accuracy improvement 15
- 16. Next: Situationally appropriatemessages • Message blast, can be ignored • Providing motivational message Justin-Time: after incorrect answer • Laboratory experiments • www.assistments.org Neil Heffernan 16
- 17. Simultaneous Applied &Basic Research • Start with “Product” • Constraints Randomization (Technical, use Qualtrics) Behavioral/Learning Measures Williams & Williams (2013). Using Interventions to improve Online Learning. NIPS Data Driven Education workshop. tiny.cc/experimentsonlinelearning • Consider Synthesis Williams, J.J. (2013) Improving Learning in MOOCs by Applying Cognitive Science. AIED MOOCshop & NIPS Data-Driven Workshop. tiny.cc/cognitivescienceinmoocs Reading List: www.josephjaywilliams.com/education Improving outcome of psychosocial treatments by enhancing memory and learning.(in press) Perspectives in Psychological Science. • High Investment but High Return 17
- 18. Multidisciplinary – RelevantCommunities • • • • • Association for Psychological Science 2013. Online Learning and Psychological Science: Opportunities to integrate research and practice. Williams, Saxberg, Means, Mitros. Cognitive Science Society 2013. Online Education: A Unique Opportunity for Cognitive Scientists to Integrate Research and Practice. Williams, Renkl, Koedinger, Stamper. AERA 2014– American Education Research Association. How online resources can facilitate interdisciplinary collaboration. Williams, Thille, Trumbore, Stigler, Siemens. ICLS (under review) – International Conference for the Learning Sciences. Online & Blended Education: Opportunities and Obstacles for the Learning Sciences. Williams, Linn, Slotta, Rose, Dillenbourg. CHI 2014 – Computer Human Interaction. Learning Innovations at Scale. Williams, Kizilcec, Russell, Klemmer. • • • E-learning guild’s DevLearn Educational Data Mining Artificial Intelligence in Education • • • Computer Supported Collaborative Learning Professional & Organizational Development Network American Society for Training & Development
- 19. Laboratory Paradigms &Online Products • A matching problem • www.edsurge.com, www.edudemic.com • [email protected] • Williams, J.J., Teachman, B.A., Richland, L., Brady, S.T, & Aleahmad, T. (2014). Leveraging the Internet to do Laboratory Research in the Real World. Symposium to be conducted at the annual convention of the Association for Psychological Science. San Francisco, CA. 19
- 20. Example: Explanation &Learning • Williams & Lombrozo (2010, Cognitive Science; 2013, Cognitive Psychology; w Rehder, JEP: General) Glorp Right Drent Wrong 20
- 21. EdSurge Meetup WedFeb 19 7 pm CERAS • “Improving Ed-Tech using Research on Learning & Advancing Research on Learning using Ed-Tech” • Ed-Tech Startup Survey • Researcher Survey • Featured Company & Researcher • Panelist & Question suggestions? 21
- 22. Completely New Horizonsfor Experimentation • “Bridging Studies” • www.josephjaywilliams.com/experime nts-online-education • Williams, J. J. (2013). Enhancing Educational Research & Practice using Experiments on Online Educational Resources. Talk at Pittsburgh Science of Learning Center LearnLab Summer School, Pittsburgh, PA. tiny.cc/onlineexperiments 22
- 23. Next: Teaching LearningStrategies • Limited teaching time: Specific content or general strategies? • Online Advantage: Easier to run RCTs & collect longitudinal data Reinforce new educational habits & behaviors • Teach metacognitive self-questioning strategies (Palinscar & Brown, 1984, Cognition and instruction; McNamara, 2004, Discourse Processes; Williams & Lombrozo, 2010, Cognitive Science) 23
- 24. Drop-down text aboutstrategies Clickable link. [Click here to learn about the “What? Why? How?” strategy]
- 25.
- 26. MOOC experiment • MotivationalVideo Control Energetic Thank you & Encouragement Growth Mindset (by Instructor) Growth Mindset (outside Instructor) “Belonging” • Questions for reflection after video None VS Just Think VS Write response 26
- 27. MOOClets Group –Develop Digital Modules • • • • • • • • • E.g. MOOClet: Math exercise, 5 minute interactive video, 30 minute lesson, Study Strategy coach, Quiz/Assessment Instructors can use for blended classroom education (High School, Community College, University) Continuously in use and being improved Easy to build, prototype, test & iterate Researchers directly implement & collect data Small scale & online access facilitates multidisciplinary collaboration & research-practitioner development (see AERA 2014 invited panel) Illustrate/Operationalize instructional design principles MOOClet Reading & Discussion Group meets Tue 11-12 PST @ Barnum (Stanford) or tiny.cc/lyticshangout Multidisciplinary group of researchers & practitioners (tiny.cc/mooclet) 27
- 28. Review • • • • • • • MOOCs VS DigitalOnline Resources Experiments: Link Real World & Laboratory Motivational Messages Experiment Simultaneous Basic & Applied Research Ed-Surge Meetup to link Research & Practice (Wed Feb 19 @ 7) Current Directions Teaching Learning Strategies Learning Coach MOOC Experiment MOOClets Research & Practice Group Meets Tuesdays 11-12 PST (tiny.cc/lyticshangout, tiny.cc/mooclet) 28
- 29. Acknowledgements • • • • • • Jascha Sohl-Dickstein Jace Kohlmeier& Khan Academy PERTS Sam Maldonado Lytics Lab (lytics.stanford.edu) VPOL (Vice Provost of Online Learning, online.stanford.edu) 29
Editor's Notes
- #2 Originally from Trinidad. I’m a Research Fellow in theLytics Lab & Office of Online Learning. Cognitive Science background – experimental psychology, statistical modeling and machine learning.Core interest is being a knowledge broker – reviewing and synthesizing the thousands of published studies in cognitive science, education research, the learning sciences to see which ones are relevant to real-world outcomes, and building products or conducting experiments that improve practically ad financially valuable outcomes. Illustrate this approach with an experiment that shows how to improve students’ motivation while learning from mathematics problems on Khan Academy.
- #12 o.320.27