Downloaded 17 times
Uploaded bybisg
Analytics Goes to College: Better Schooling Through Information Technology with Vince Kellen, Senior Vice Provost of Information Technology at the University of Kentucky
Higher education faces challenges in addressing economic and readiness problems for students from disadvantaged backgrounds. While free educational content helps, it does not solve complex readiness issues. For-profit models also struggle with serving students in remote, underserved areas. Analytics and technologies show promise in helping institutions address these "last mile" challenges through personalized, adaptive learning approaches. However, significant organizational changes and integration of disparate data sources will be required for institutions to fully leverage these tools. Open discussion is needed around ensuring insights into learning and student success remain available as public goods, not proprietary to any private vendor.
More Related Content
Similar to Analytics Goes to College: Better Schooling Through Information Technology with Vince Kellen, Senior Vice Provost of Information Technology at the University of Kentucky
Analytics Goes to College: Better Schooling Through Information Technology with Vince Kellen, Senior Vice Provost of Information Technology at the University of Kentucky
- 1. Analytics & HigherEducation Vince Kellen, Ph.D. Senior Vice Provost Analytics and Technologies University of Kentucky [email protected] January, 2014 This is a living document subject to substantial revision.
- 2. Higher education hasa ‘last mile’ problem Education in any form is struggling to address families and communities with economic and other readiness problems Free or low-cost educational content does not easily solve readiness problems which have a multitude of factors For profit models rightfully struggle with ‘last-mile’ problems. Public policy matters! 2
- 3. What would AbrahamLincoln think of things? Abraham Lincoln • Autodidactic • Books, books, books • Became a skilled military strategist • Penchant for poetry, Shakespeare, politics and history My nephew • Not an autodidact • Good worker, smart kid, but… • It takes a village • After a few low-security colleges and much money borrowed • He has found an intellectual home 3
- 4. Analytics in highereducation is hot right now A number of vendors, old and new are making sales now • Knewton, Starfish, Civitas, Education Advisor Board, Banner (Signals) and others • Value proposition: we collect lots of data, we have data scientists who can analyze, we collect lots of best practice that we can share with you, we host the data and systems so you don’t have to Adaptive learning got a boost this past year • Gates Foundation spurred implementations with funding for colleges and universities to implement pilot programs • Reports by Education Growth Advisors, Foundation report identified and evaluated eight adaptive learning vendors • APLU formed a Personalized Learning Consortia (also partially funded by the Gates Foundation) to spur collaboration between universities for personalized learning content and platform development • Industry press has picked up coverage (Chronicle, InsideHigherEd) and vendors are making acquisitions 4
- 5. Analytics have beenaround awhile Thousands of studies and research on many aspects of student success and the psychology of learning over several decades Most institutions understand the common causes • High school GPA, test scores, family familiarity with higher education, personal and family expectations, wealth, high identification with an academic discipline, high motivation, conscientiousness, involvement in academics and co-curricular activities, strength of and placement within a social network, problems in course progressions and degree choice Most institutions understand and are starting to act upon the signals • First semester performance, mid-term grades, the first few weeks of progress in academic classes, identification of gaps in needed skills and remediation Many institutions are taking steps to reform the teaching and support • Hybrid designs, active learning designs, guide-on-the-side versus sage-on-the-stage • Early intervention, live and learn communities, peer mentoring, professional advising 5
- 6. What is differentnow? The technology for analytics has undergone a recent renaissance • Different forms of high-speed, big-data processing are coming forward – Structured and unstructured data can be rapidly analyzed – Queries can be run against both structured and unstructured stores simultaneously – The hardware is now more parallel enabling ‘scale-out’ designs to handle big data – Apple Siri, IBM Watson and others have grabbed mainstream attention • Data visualization is well established – Many tools offer interesting ways of visualizing data, enabling better communication of insights • MOOCs have brought attention to eLearning opportunities – MOOCs are expected to incorporate analytics to help improve learning outcomes Organizations and vendors are facing some critical transitions • How do old-line database and analytic vendors change their tools to compete with new approaches (e.g., the Hadoop bandwagon, SAP HANA, etc.) • How do institutions adopt and take advantage of the new tools? What skill sets are lacking? What organizational pieces need to be put into place? Can institutions integrate the data needed? 6
- 7. What we havedone and what we would like to do First steps over the past year • Mobile micro-surveys: Learning from the learner • Student enrollment, retention, demographics, performance, K-Score, facilities utilization, instructor workload and more • High speed, in-memory analytics architectural differences • Open data and organizational considerations Coming down the road? • Micro-segmentation tool to enhance user and IT productivity, develop personalized mobile student interaction/intervention • Models for learner technographics, psychographics, in addition to behaviors, performance, background • Advanced way-finding for streaming content like lecture capture • Content metadata extraction and learner knowledge discovery • Real-time measures of concept engagement and mastery • Real-time learner recommendations and support engine • Use graphing algorithms to perform more sophisticated degree audit what ifs 7
- 8. 8 Model Description Enrollment Enrollmentin a class, midterm and final grades, credit hours attempted and earned, instructor teaching the class Student retention and graduation Student demographics and cohort identification (e.g., John Doe is in the 2009 entering first-year student cohort) Student demographics Demographics, such as age, high school GPA, entrance test scores (SAT, ACT) and subcomponent scores. Also, in a secure location, additional personally identifiable demographic details such as name, address, email, etc. Student performance Present the enrollment data in such a way as to easily show the student’s performance for each term, including credit hours earned, term GPA, cumulative GPA for that term, etc. Student academic career Keep a list of the majors and minors for each student and degrees awarded. Also, include details on students who transfer in and out, including transfer institution, credit hours transferred in, etc. Productivity The room utilization model contains every building, every potential classroom and lets users analyze the room capacity and enrollments for the class or event in the room at five minute intervals. The faculty stats per term model pulls together the number of students and sections taught per term and will contain other important data such as research expenditures per term and grant proposals submitted and won. Micro-surveys Capture questions and answers from the My UK Mobile micro-survey feature Student involvement Interaction history with various applications including the learning management system, clickers, course capture and playback, academic alerts. Provide the basis for calculating the student’s K-Score.
- 9. My UK Mobile usagestats 9
- 10.
- 11.
- 12. Predictive analytics atits finest Our Ph.D. data scientists locked themselves in a room and worked very hard on an approach to more reliably predict first-year student retention from fall to spring term. What they came up with amazed us… 12
- 13. A question sentvia MyUK Mobile to freshman who are not doing so well just prior to midterms: “Do you plan on coming back in the spring?” 13
- 14.
- 15. Academic Health Notifications: Viewin student mobile app
- 18.
- 19. K-Feed: Intelligent, personalizedalerts, news, reminders 19
- 20. Taxonomy? Automatic metadata?Automatic atomic metadata? Let learners navigate an audio/visual stream Let the system learn what are top terms. Let the system map terms to concepts. Let instructional designers lightly ‘bump’ the taxonomy, post production Record student engagement with specific terms / concepts Deliver personalized messages to students 20 See http://p.uky.edu
- 21.
- 22. Key questions • Canthe audio and slides be reliably converted into ‘useful’ text? • Can a concept map be derived automatically from the text generated or easily edited by an instructor? • How easy will it be for designers-instructors to create an assessment and guide its placement in the right location in the video? • Can we personalize the recommendations to reflect prior knowledge, student ability and individual differences in information processing? • Can the interface support real-time integration with high-speed analytic back-ends (e.g., HANA)? • Can advising, learning and general support processes be integrated? • Can this be cost-effective for existing courses? This is just one conceptualization. What other interface designs might exist? How effective will they be?
- 23. Personalize learning andsupport in one architecture • Real-time personalized interactions • Target on-demand peer tutoring based on student’s profile • Deliver micro-surveys and assessments to capture additional information needed to improve personalization • Give students academic health indicators that tell students where they can improve in study, engagement, support, etc. • Let students opt their parents in to this information so the family can support the student • Tailor and target reminder services, avoid over messaging, enable timing of message delivery based on user temporal proclivities • Allow for open personalized learning • How content gets matched to students is psychologically complex • Several theories of how humans learn give many insights • Students differ in the following abilities and attributes: visual-object, visual- spatial, reasoning, cognitive reflection, need for sensation, need for cognition, various verbal abilities, confidence, persistence, prospective memory, etc. • We need an open architecture to promote rapid experimentation, testing and sharing of what works and what doesn’t University of Kentucky
- 24. Is learning analyticsa sustainable opportunity? Let’s look at the pieces to the puzzle of the value proposition • We have the expertise, you don’t – Researchers have been examining many aspects of students success and learning, but do all vendors have “experts” knowledgeable of the breadth of this literature? – Research into how the brain learns is yielding much recent insight, but a single ‘theory of mind’ for learning has not emerged, nor will one soon. There is complexity, counter-intuitiveness and much more to understand regarding how the brain functions – While data science is a scarce skill sets, universities, especially research universities, tend to have these skills • We have large data across multiple institutions that you don’t – Does one need large data to gain valid insights? Exactly where does large data provide learning analytics benefits? For inductive, atheoretical approaches, perhaps big data can help (e.g., millions of rows of clickstream). – Do students brains vary that much that very large sample sizes are needed? – Have we exhausted existing and emerging theoretical approaches? • We have access to multi-institutional best practices that you don’t – Good point 24
- 25. Where are wegoing? Question 1 Should knowledge of how students learn be considered a private or public good? • Learning analytics are nonrival, that is, if I gain knowledge of how students learn that does not simultaneously deprive you of the same opportunity • Learning analytics are excludable, that is, if I have a piece of software that collects data on how students learn, I can prevent you from getting it How excludable are learning analytics? • Most if not all learning analytic companies base their analytics off of published research. While a single vendor’s knowledge of learning analytics might be excludable, the ‘prior art’ is often commonly available • How easy would it be for a competitor or a customer to reverse- engineer an approach or design an alternative? 25
- 26. Questions 2 and3 Is making knowledge about how students learn excludable the right thing to do? • Image a company that has figured out how to improve learning for 90% of human beings by increasing learning outcomes over any time frame by 100% while maintaining or reducing costs of instruction? Suppose this approach is available at the following price: $75,000 per student • Who gets to benefit? Who does not? How would you feel about the same scenario, but now regarding a difficult, life- saving, heart surgical procedure? • While in the U.S. medical procedure patents have been permissible, many countries ban them. Most, if not all medical procedures are based on ‘prior art.’ Many in the medical community are opposed to these types of patents as they can interfere with educating doctors, and impede public health objectives What does this mean for vendors? • Learning analytic procedures (algorithms) would have to be free from reliance on ‘prior art’ which might be difficult for most vendors • While copyright law can protect the software written regarding learning analytics, copyright law cannot compel anyone from revealing what aids learning • Vendor goals and university goals are not always aligned 26
- 27. Is this somethingto worry about? Perhaps not • Cognitive psychology, neuroscience and learning theory are rapidly evolving. Recent brain imaging and sensor advances are expanding knowledge quickly • How humans learn is amazingly complex, and even harder to apply in ‘single event cases.’ How many of you have family in college you just can’t seem to help? • Universities can easily create ‘open source’ versions of learning analytic tools, and sharing specific knowledge about their students with others • As learning analytic knowledge diffuses, universities will then shift the competitive effort on to those activities that their faculty and staff perform (skill of the doctor) versus gaining access to the analytically-power learning tool (medical procedure) • We live in a connected world. Countries might demand learning insights be public goods hurting companies relying on keeping knowledge excludable. Research globally can undermine vendors locally Perhaps so • Universities sometimes move in haste and en masse, and don’t have the expertise to build their own tools, thus will be reliant on vendors • Vendors, by design, are motivated to make data and insights into data excludable • Regulation in a single nation can encourage further privatization of education, thus ‘locking up’ insights into things universities must purchase (and not generally use) 27
- 28. Scarce / Notscarce Scarce 1. Management ability to know how to build an organization to take advantage of analytics 2. Enterprise architecture and data science skills 3. Ability to integrate from disparate sources quickly 4. Order of magnitude improvement in cost-effectiveness Not scarce 1. Ideas for analytics 2. Raw data (dark data) 3. Tools 4. Willing students 28
- 29.
- 30.
Editor's Notes
- #2 1