Carolyn Rosé - WESST Keynote - Technology Support for Effective Team-Based Learning in Online Education
1 like114 views
The document discusses the importance of technology support for effective team-based learning in online education, emphasizing the role of transactivity—collaborative reasoning and interactions among students. It provides empirical support for various design principles and demonstrates how personalized agents can enhance collaboration, learning outcomes, and knowledge integration in team settings. The findings highlight that scaffolding transactive discussions can significantly benefit student engagement and performance across different educational contexts.
![● Students discuss individual work displaying their jigsaw
condition in a discussion forum
● Automated Transactivity detection is applied to posts
● For each pair of students, count number of Transactive
contributions exchanged [Pairwise Transactivity count]
● Assign all students in the course to teams of four using a fast
constraint satisfaction algorithm with the following constraints:
– Each team must have 1 student from each jigsaw condition
– Maximize Average Pairwise Transactivity count of teams
Transactivity-Based Team Formation
Minimal Cost Network Flow
Algorithm (Ahuja and Orlin, 1989)](https://image.slidesharecdn.com/singaporekeynote2017-170816191607/85/Carolyn-Rose-WESST-Keynote-Technology-Support-for-Effective-Team-Based-Learning-in-Online-Education-43-320.jpg)
More Related Content
![Active learning methods in higher education [compatibility mode]](https://cdn.slidesharecdn.com/ss_thumbnails/activelearningmethodsinhighereducationcompatibilitymode-141110040033-conversion-gate02-thumbnail.jpg?width=560&fit=bounds)
Similar to Carolyn Rosé - WESST Keynote - Technology Support for Effective Team-Based Learning in Online Education (20)
Carolyn Rosé - WESST Keynote - Technology Support for Effective Team-Based Learning in Online Education
- 1. Carolyn Penstein Rosé [email protected] http://www.cs.cmu.edu/~cprose http://dance.cs.cmu.edu Language Technologies Institute and Human-Computer Interaction Institute Carnegie Mellon University Technology Support for Effective Team-Based Learning in Online Education
- 2. What is the ultimate in personalized instruction?
- 4. Relationships are critical! Students gain benefits through their interactions with each other We must pay attention to social factors
- 8. •Definition of Transactivity • building on an idea expressed earlier in a conversation • using a reasoning statement Homozygous for both. One parent is orange and the other is not. Orange is dominant. I agree because all the kids are orange also.
- 9. Outline • Overview of technology support for team-based learning
- 10. Outline • Overview of technology support for team-based learning • Transactivity as the DNA of effective collaborative learning
- 11. Outline • Overview of technology support for team-based learning • Transactivity as the DNA of effective collaborative learning • Scaffolding Transactivity with Accountable Talk
- 12. Outline • Overview of technology support for team-based learning • Transactivity as the DNA of effective collaborative learning • Scaffolding Transactivity with Accountable Talk • Transactivity in support of team-based learning
- 13. Outline • Overview of technology support for team-based learning • Transactivity as the DNA of effective collaborative learning • Scaffolding Transactivity with Accountable Talk • Transactivity in support of team-based learning • Resources for Partnership
- 14. Outline • Overview of technology support for team-based learning • Transactivity as the DNA of effective collaborative learning • Scaffolding Transactivity with Accountable Talk • Transactivity in support of team-based learning • Resources for Partnership
- 15. Why Team Based Learning? • Teams can accomplish large and complex goals that individuals would not be capable of • Students benefit both in motivation and learning gains when they learn in teams • Students need to gain effective teamwork skills in order to be successful in the workforce
- 16. Conversational Agent Based Support in Computer Supported Collaborative Learning Students learn 1.25 letter grades more when working with a partner + automated support than working alone (Kumar et al., 2007)
- 17. 17 ▪ A decade and a half of successful classroom studies ▪ Middle school, High school, College level ▪ Urban school districts ▪ Top tier and second tier universities ▪ Math, Science, Engineering, Social Sciences ▪ Massive Open Online Courses (MOOCs) ▪ Demonstrates that success generalizes to massive scale Effective in Multiple Learning Contexts
- 18. Empirical Support for Design Principles • Personalized agents increase supportiveness and help exchange between students (Kumar et al., 2007) • Agents are more effective when students have control over timing of the interaction (Chaudhuri et al., 2008; Chaudhuri et al., 2009) • Agents that employ Balesian social strategies are more effective than those that do not (Kumar et al., 2010; Ai et al., 2010) • Students are sensitive to agent rhetorical strategies such as displayed bias (Ai et al., 2010), displayed openness to alternative perspectives (Kumar et al., 2011), and targeted elicitation (Howley et al., 2012) • Accountable talk agents (Dyke et al., 2013; Adamson et al., 2014)
- 19. Empirical Support for Design Principles • Personalized agents increase supportiveness and help exchange between students (Kumar et al., 2007) • Agents are more effective when students have control over timing of the interaction (Chaudhuri et al., 2008; Chaudhuri et al., 2009) • Agents that employ Balesian social strategies are more effective than those that do not (Kumar et al., 2010; Ai et al., 2010) • Students are sensitive to agent rhetorical strategies such as displayed bias (Ai et al., 2010), displayed openness to alternative perspectives (Kumar et al., 2011), and targeted elicitation (Howley et al., 2012) • Accountable talk agents (Dyke et al., 2013; Adamson et al., 2014)
- 20. Demonstrated impact of social interaction Successfully Deploying Collaborative Chat in Multiple MOOCs ▪ Chat consistently reduces attrition
- 21. Analysis of Student Retention from Medicinal Chemistry MOOC • Reflection activities offered as optional supplements at the end of each unit • If students click to enter the activity, they may be required to do it individually or collaboratively – Out of 14,000 clicks to enter the reflection activities, 25% included an additional student
- 22. What if we’re not concerned about attrition per se?
- 23. Lower attrition = Experience of benefit
- 24. Outline • Overview of technology support for team-based learning • Transactivity as the DNA of effective collaborative learning • Scaffolding Transactivity with Accountable Talk • Transactivity in support of team-based learning • Resources for Partnership
- 26. Transactivity • Students open their minds to other perspectives • Moderating effect on learning (Joshi & Rosé, 2007; Russell, 2005; Kruger & Tomasello, 1986; Teasley, 1995) • Moderating effect on knowledge sharing in working groups (Gweon et al., 2011)
- 28. Outline • Overview of technology support for team-based learning • Transactivity as the DNA of effective collaborative learning • Scaffolding Transactivity with Accountable Talk • Transactivity in support of team-based learning • Resources for Partnership
- 29. Transactivity through Accountable Talk in District Wide Teacher Professional Development Transactivity Accountable Talk
- 30. 30 Accountable TalkUtterance 1 Teacher OK, does anyone want to respond to that? Who wants to respond and can prove that they listened to Marcel’s explanation and can, kind of respond with their own ideas or can add another idea to it? Frank, go ahead. 2 Frank I um I agree with what you said because this for example like if you put-- if you had big um, can- like if you got a big cup of water and you put a- an eraser in there, like a- a ah, like the eraser over there, if you put something like that in a big cup of water, the water level would rise a lot, and, if you put in a copper cube, and it’s not even gonna- it’s not going to rise that much even though that copper cube will weigh more than a eraser. 3 Teacher Ahh ok, anyone agree with Frank’s idea? I like that he kind of, talked about another object and he chose an object that we know has a different volume than the copper and the aluminum cube. So, he gave us an example of an object that has a much greater volume. And can someone explain or repeat for us what Frank thought would happen if I put an object this big, in water? James, what he think would happen if I put an object, this big, into a cup of water? 4 James He said that if you-- if you like a big- bur--like a big bottle of like water 5 Teacher Uh huh 6 James and you put the eraser in it, then it would probably like, rise a lot. Then-- 7 Teacher Woah, woah, rephrase that – it has a what? 8 James a different vol-- a different volume… 9 Teacher And which has a greater volume? 1 0 James The eraser. 1 1 Teacher The eraser. Which means it takes up more space. So if I were to put this into, a big container of water for example, if I were to put it into my little pitcher here, if I had this filled up and I dropped this guy in, well, which direction would my water have to go? (Resnick, Michaels & O’Connors, 2010) Add on Agree/Disagree Explain Other Say More Press for Reasoning Revoice
- 31. • When teachers of math, science, and reading use structured teacher-lead discussion methods… • steep changes in student achievement (Bill, Leer, Reams & Resnick, 1992; Chapin & O’Connor, 2004) • Retention for up to 3 years (Adey & Shayer, 1993, 2001; Shayer, 1999; Topping & Trickey, 2007a, 2007b) • Transfer across domains for up to 3 years (Bill, Leer, Reams & Resnick, 1992; Adey & Shayer, 1993, 2001; Shayer, 1999; Chapin & O’Connor, 2004) • Students perform better on non-verbal reasoning tests e.g. Ravens (Mercer, Wegerif & Dawes, 1999) • Reasoning itself improves (Kuhn & Zillmer, in press; Lin et al 2012) Empirical Support for Accountable Talk Asterhan, C., Clarke, S., Resnick, L (2015). Socializing Intelligence through Academic Dialogue, AERA Press.
- 32. Transactivity through Accountable Talk in District Wide Teacher Professional Development District Context: (2008-2010) • 63% of district students performing below proficient in READING • 56% below proficient in MATH, a large % of which are African American students • Multiple schools and multiple teachers School Context: • Failed to meet Adequate Yearly Progress on standardized tests for 5+ years Subject Area: Focus on 9th Grade Biology • 9th grade Biology Years 1-4 • Added Math in years 3-4
- 33. Agents as Support for Group Learning Employing Accountable Talk Agents 33 Agent support by revoicing
- 34. • Study 1: Year 1, Diffusion Lab – Students learn more on explanation questions in supported conditions (effect size 1 s.d.) – Students in supported conditions more active in whole group discussion (effect size .75 s.d.) • Study 2: Year 2, Diffusion Lab – Students learn more on immediate post test in Revoicing Agent condition (effect size .51 s.d.) • Study 3: Year 2, Punnett Square Lab – Students learned marginally more (p < .1) on delayed post-test in Revoicing Agent condition Positive Effect on Student Learning Automatic Analysis Of Conversation Conversational Interventions Positive Learning Outcomes
- 36. Outline • Overview of technology support for team-based learning • Transactivity as the DNA of effective collaborative learning • Scaffolding Transactivity with Accountable Talk • Transactivity in support of team-based learning • Resources for Partnership
- 37. Challenges in Online Learning • Selecting students to work together • Monitoring teamwork processes • Supporting teamwork processes • Problems increase as scale increases
- 38. ● Analysis of data from two team based MOOCs for teacher professional development ● Data from 177 teams ● 87% of teams failed ● Problems: ○ Finding and keeping team members ○ Starting the discussion ○ Keeping the team active Early Team Based MOOCs
- 39. 39 Transactivity
- 40. Power, Relationships, and Transactivity Cognitive Factors Social Factors
- 42. Proposed Team Formation and Support Paradigm Individual Work Community Deliberation Team Formation with transactivity evidence Transactivity-b ased Collaboration Suport 42
- 43. ● Students discuss individual work displaying their jigsaw condition in a discussion forum ● Automated Transactivity detection is applied to posts ● For each pair of students, count number of Transactive contributions exchanged [Pairwise Transactivity count] ● Assign all students in the course to teams of four using a fast constraint satisfaction algorithm with the following constraints: – Each team must have 1 student from each jigsaw condition – Maximize Average Pairwise Transactivity count of teams Transactivity-Based Team Formation Minimal Cost Network Flow Algorithm (Ahuja and Orlin, 1989)
- 45. Deep Learning Result Maintain high accuracy while achieving domain general performance
- 47. Jigsaw Design Students can only exchange expertise if they have expertise to exchange 47 Coal Wind Hydro Nuclear
- 48. Success Measures We can only know if students shared expertise effectively if the outcome measures are related to expertise sharing • Evidence of knowledge integration processes in collaborative chat • Evidence of knowledge integration in write ups – Comparisons, trade-offs, evaluations of other perspectives 48
- 51. • Student 1: I think solar energy would be ideal for City A, especially if it is in the sunbelt. • Student 2: I don't recall seeing solar energy as an alternative, but good idea. • Student 3: For this particular city, solar energy seems like a perfectly acceptable option. • Student 4: Solar energy was not listed or proposed as an alternative option, and would only be so if City A had sufficient consistent sunlight. We don't have any information proving it would be viable now, though. Transactive Deliberation Example 51 A transactive reply displays reasoning and builds on an earlier contribution in the discussion.
- 53. Results: Knowledge Integration Processes • Teams in the Transactivity Maximization Condition contribute more transactive chat turns during team collaboration (F(2, 51) = 6.74, p = 0.03).
- 54. Results: Knowledge Integration in Product • Teams in the Transactivity Maximization Condition demonstrate better team performance (F(1,52) = 9.26, p = 0.02). 54
- 55. Further pilot results Students benefit from public peer review Higher quality feedback More diverse feedback Students learn more about doing knowledge integration better when scaffolding is explicit
- 56. Deployment Study Team-based MOOC 56 Rise of the Superheroes and the Heroes of the Future
- 57. 57 Post individual work, Community deliberation Team formation, Small team collaboration Small team collaboration Week 3Week 2Week 1
- 58. Success!!! Summary of Results • Correlational results are consistent with our previous experiments • Marginal effect of detected #transactive exchanges prior to collaboration on project completion status (r = 0.23, p = 0.09) • Significant effect of #transactive exchanges prior to collaboration on number of active students within groups (r = 0.32, p = 0.02) • Marginal effect of #transactive exchanges prior to collaboration on whether all the superheroes interacted in the story (r = 0.26, p = 0.06)
- 59. Outline • Overview of technology support for team-based learning • Transactivity as the DNA of effective collaborative learning • Scaffolding Transactivity with Accountable Talk • Transactivity in support of team-based learning • Resources for Partnership
- 60. DANCE is a community of practice with many open source resources Thousands of visitors Hundreds of return visitors each month
- 61. Resources http://dance.cs.cmu.edu DANCE Discussion Forum is compatible with Open edX Includes hooks for interventions like Social Recommendation and Discussion Scaffolding
- 62. Resources LightSIDE Text mining tool bench Over 10,000 users have downloaded LightSIDE Automated collaborative process analysis Automated writing assessment/feedback generation Social Recommendation deployed so far in one MOOC to support help exchange
- 63. Resources DiscourseDB: Data infrastructure to offer discourse data and analytic tools through LearnSphere Bazaar: Tutorial dialogue architecture Dialogue agents for individual or collaborative learning
- 64. Conclusion • Don’t forget social factors in learning • Transactivity is a key construct for understanding what makes collaboration work • Technology can leverage Transactivity for supporting effect team-based learning • Join us: Open Source Resources – Let me know if you would like to collaborate