Downloaded 10 times
Uploaded byJohn Whitmer, Ed.D.
Learning Analytics: Realizing their Promise in the California State University
This document discusses learner analytics and how they are being used at California State University (CSU) campuses. It begins with an overview of the promise of learner analytics, including how they can provide insights into student behavior and performance. Examples of learner analytics tools are presented, including Signals and SNAPP. The document then shares three case studies from CSU campuses: one discusses how analytics were used to help a teacher and student at CSU Northridge, another reviews the GISMO analytics tool at CSU Northridge, and a final case study describes how Vista analytics were used in a course at CSU Dominguez Hills. The presentation concludes with a call to action around increasing analytics reporting capabilities.
More Related Content
Similar to Learning Analytics: Realizing their Promise in the California State University
Learning Analytics: Realizing their Promise in the California State University
- 1. Learner Analytics Realizing their Promise in the CSU John Whitmer, CSU Office of the Chancellor & CSU Chico Kate Berggren, CSU Northridge Hillary Kaplowitz, CSU Northridge Tom Norman, CSU DH Download slides at: http://bit.ly/HqaHBF
- 2. Outline 1. Promise ofLearner Analytics 2. Tools & Systems in Practice 3. CSU Case Studies: • Analytics at Work in the Classroom (Hillary) • GISMO & SQL Query Tools (Kate) • Vista in RELS 180 (John) 4. Q & A
- 3. 1. PROMISE OFLEARNER ANALYTICS
- 4. Steve Lohr, NYTimes, August 5, 2009
- 5. Draft DOE Report releasedApril 12 http://1.usa.gov/GDFpnI
- 6. Economist. (2010, 11/4/2010).Augmented business: Smart systems will disrupt lots of industries, and perhaps the entire economy. The Economist.
- 7. Source: jisc_infonet @Flickr.com 7 Source: jisc_infonet @ Flickr.com
- 8. What’s different withBig Data? 4 V’s: 1. Volume 2. Variety 3. Velocity 4. Variability (IBM & Brian Hopkins, Forrester) 8
- 9. Academic Analytics “Academic Analyticsmarries large data sets with statistical techniques and predictive modeling to improve decision making” (Campbell and Oblinger 2007, p. 3)
- 10. Academic Analytics 1. Termadopted in 2005 ELI research report (Goldstein & Katz, 2005) – Response to widespread adoption ERP systems, desire to use data collected for improved decision making – 380 respondents; 65% planned to increase capacity in near future 2. Call to move from transactional/operational reporting to what-if analysis, predictive modeling, and alerts 3. LMS identified as potential domain for future growth 10
- 11.
- 12. Learner Analytics: “ ...measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.” (Siemens, 2011)
- 13. or said plainly: What are students doing? Does it matter?
- 14. Learner Analytics 1. Analyze combinations of data including: – Frequency of ed tech usage (e.g. clickstream analysis) – Student learning “outputs” (e.g. quiz scores, text answers) – Student background characteristics (e.g. race/ethnicity) – Academic achievement (e.g. grades, retention, graduation) 2. Current rsch: mostly data mining, not hypothesis-driven 3. More complex than Academic Analytics, considering: – Immaturity of ed tech reporting functionality – Translation of usage into meaningful activity – No significant difference: not what technology used, it’s how it’s used, who uses it, and for what purpose
- 15. A few promisesof analytics for faculty and students … 1. Provide behavioral data to investigate student performance 2. Inform faculty about students succeeding or at risk of failing a course 3. Warn students that they are likely to fail a course – before it’s too late 4. Help faculty evaluate the effectiveness of practices and course designs 5. Customize content and learning activities (e.g. adaptive learning materials)
- 16. What’s the promiseof analytics for academic technologists? 1. Decision-making based on actual practices (not just perceptions) and student outcomes 2. Support movement of A.T. into strategic role re: teaching and learning by: – demonstrating the link between technology and learning – distinguishing our role from a technology infrastructure provider
- 17. Our 2 biggestbarriers Image Source: http://bit.ly/Hq9Cdg
- 18. Image Source: UtopianInc http://bit.ly/Hq9sCq
- 19. Image Source: Privacyin the Cloud: http://bit.ly/HrF6zk
- 20. 2. TOOLS &SYSTEMS IN PRACTICE
- 21. SIGNALS Purdue Signals Project http://www.itap.purdue.edu/studio/signals/
- 22. SNAPP SNAPP (Social NetworksAdapting Pedagogical Practice) http://www.snappvis.org/
- 23. KHAN Khan Academy http://www.khanacademy.org/
- 24. OLI CM Open LearningInitiative http://oli.web.cmu.edu/openlearning/initiative/process
- 25. PARCHMENT Parchment http://www.parchment.com/c/my-chances/
- 26. 3. CSU CASESTUDIES
- 27. ANALYTICS AT WORKIN THE CLASSROOM (HILLARY) 27
- 28. How can datahelp teachers and students? Two stories about how data helped students and teachers work better together 28
- 29.
- 30.
- 31. “Hey Professor, I justlooked at my assignments and realized that my Chapter 11 summary did not get submitted, which I'm having trouble believing that I didn't submit it... especially because I see that I did it, and I always submit my assignments as soon as I finish them.” 31
- 32. Now the hardpart…. Do I believe him? If I only I could check… 32
- 33.
- 34.
- 35. And it wasall his idea… The student suggested that I check Moodle and if that didn’t work told me how to check the Revision History in GoogleDocs with step-by- step directions! 35
- 36.
- 37. Hybrid Course WeeklyStructure 4. Post 3. Online questions 1. Watch 2. Read 4. Class 5. Aplia chat and and take lectures textbook meets quiz tutoring practice quiz 37
- 38. “The quiz isunfair” 38
- 39. But the storywas not that simple… » Reports on Moodle painted a different picture » Student was watching the lectures at 10:00 p.m. » Then immediately taking quiz 39
- 40. Enabled constructive feedback… 1.Advised the student how the structure of the course was designed to enhance learning 2. Student revised their study habits 3. Improved grades and thanked the instructor! 40
- 41. What we cando with data now 1. Use Reports in Moodle to verify student claims 2. Review participant list to see last access time 3. Empower students to review their own reports 4. Analyze usage and advise students how to study better 5. Review quiz results to find common misconceptions 41
- 42. And if wehad better tools that are easier to use… 1. Let our students see more details about how their habits affect their grades and encourage them to use them 2. Give instructors access to more information and better tools to organize data so they can see patterns of access and time on task and how they relate to outcomes 3. Have tools that red flag students with teacher set criteria 4. Help streamline workflow for instructors by organizing student information – View all ungraded assignments 42
- 43. Could we helpimprove student learning outcomes if we knew the effect of… Coffee Friends Time Attendance Amount Mobile Textbook LMS LMS Access Activities 43
- 44. GISMO & SQLQUERY TOOL (KATE) 44
- 45. GISMO – CourseBlock 45
- 46. GISMO – AccessOverview 46
- 47. GISMO – Accessby Student 47
- 48. GISMO – QuizOverview 48
- 49.
- 50. List of ContributedQueries 50
- 51.
- 52. Query Results: Most ActiveCourses 52
- 53. Query: Most PopularActivities 53
- 54. Query: Systemwideuse of Activities and Resources 54
- 55. Query: Forum Use Countby Type 55
- 56. Learner Analytics Thomas J. Norman California State University Dominguez Hills
- 57.
- 58. A New Wayof Reading
- 59.
- 63.
- 64. Grading To DoList
- 65.
- 66.
- 67.
- 68. Analysis by AACSBCategories
- 69.
- 70. Performance by Learning Objective/Difficulty
- 71. Ideas? Questions? • [email protected] •[email protected] • 310-243-2146
- 72. CSU CHICO VISTAANALYTICS 72
- 73. LMS Learner Analytics@ Chico State Campus-wide – How are faculty & students using the LMS? – What meaningful activities are being conducted? – How does that usage vary by student background, by college, by department? Course level – What is the relationship between LMS actions, student background characteristics and student academic achievement? (6 million dollar question) – Intro to Religious Studies: redesigned in Academy eLearning, increased enrollment from 80 to 327 students first semester Ultimate goal: provide faculty and administrators with what-if modeling tools to identify promising practices and early alerts 73
- 74. 74 Chart from ScottKodai, Chico State
- 75.
- 76. INTRODUCTION TO RELIGIOUS STUDIES(RELS 180)
- 80.
- 81. Call to Action 1.Metrics reporting is the foundation for Analytics 2. Don’t need to wait for student performance data; good metrics can inspire access to performance data 3. You’re *not* behind the curve, this is a rapidly emerging area that we can (should) lead ... 4. If there’s any ed tech software folks in the audience, please help us with better reporting!
- 82. Want more? Resourceson Analytics Googledoc: http://bit.ly/HrG6Dm
- 83. Q&A and ContactInfo Resources Googledoc: http://bit.ly/HrG6Dm Contact Info: • John Whitmer ([email protected]) • Hillary C Kaplowitz ([email protected]) • Berggren, Kate E ([email protected]) Download presentation at: http://bit.ly/HqaHBF 83
Editor's Notes
- #3 Kathy
- #30 Here is the oldest excuse in the book – “The dog at my homework”
- #31 But now we have new excuses – the electronic dog ate my electronic homework… the computer messed up. I uploaded it. Or they upload the wrong file. Or an empty one. Or the wrong format… or… or….
- #32 So here is an email I got from one of my students
- #33 I want to believe him. He’s an A student but that’s not fair…
- #34 Moodle report by activity and student showed me he accessed it before the deadline but no upload so no way to know if he did it or not.
- #35 But it was a googledoc assignment so I could go into the revision history and verify that he indeed did the work before the deadline!
- #36 He used data to his advantage!
- #37 Next story – students complain the work is too hard! Or… in this case
- #38 Economics class converted to hybrid. Students met only once a week and were given this schedule to follow – which was a carefully designed sequence to help the students learn difficult material that takes time and practice.First watch lecturesThen read bookThen do online activitiesPost questions, take practice quizThen come to class -****with questions and problems to discuss****Then take the quiz online which was graded