Higher Education Teachers' Experiences of Learning Analytics in Relation to Student Retention

Higher Education Teachers' Experiences of Learning Analytics in Relation to Student Retention

• 1.
  Higher Education Teachers’Experiences with Learning Analytics in Relation to Student Retention Deborah West & David Heath Acknowledgement of Research Partners and Paper Authors: Professor Carol Miles, Professor Alf Lizzio, Mr Bill Searle, Mr Danny Toohey, Dr Jurg Bronnimann. Acknowledgement of Funding: Australian Government: Office for Learning and Teaching WELCOME
• 2.
  WHAT ARE LEARNINGANALYTICS? ‘measurement, collection, analysis and reporting of data about learners and their contexts, for the purposes of understanding and optimising learning and the environments in which it occurs’ Long, P. & Siemens, G. (2011). Penetrating the Fog: Analytics in learning and education. EDUCAUSE Review, 46(4) July/August. Retrieved 20 May, 2013 from http://www.educause.edu/ero/article/penetrating-fog-analytics-learning-and-education
• 3.
  WHAT ARE LEARNINGANALYTICS? “learning analytics is a new, expanding field that grows at the confluence of learning technologies, educational research, and data science…over time researchers and practitioners with different backgrounds and methodologies have tried to solve two simple but challenging questions: 1. How do we measure the important characteristics of the learning process? 2. How do we use those measurements to improve it? Ochoa, X., Suthers, D., Verbert, K., & Duval, E. (2014). Analysis and reflections on the third Learning Analytics and Knowledge Conference (LAK 2013). Journal of Learning Analytics, 1(2), 5‐22. Next slide reference Bates, S (2015). The anatomy of a 21st century educator: An incomplete guided tour. ETUG Conference, British Columbia, Canada, Published June 23, 2015. http://www.youtube.com/watch?v=ugqYeKVl-w8n (used with permission)
• 4.
  This work islicensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
• 5.
  DATA SCIENCE Poulson, 2014
• 6.
  STUDENT RETENTION Learning Supporting Nelson, K.,Clarke, J., Stoodley, I., & Creagh, T. (2014). Establishing a framework for transforming student engagement, success and retention in higher education institutions. Final Report 2014, Canberra, Australia: Australian Government Office for Learning & Teaching. Resourcing Integrating Belonging
• 7.
  PROJECT AT AGLANCE
• 8.
  1. How muchvariety exists in the online environments where teaching takes place? 2. What involvement do teaching staff currently have in using data to respond to retention issues? 3. In which learning analytics related activities have teaching staff been involved? 4. In which retention applications of learning analytics are participants most interested in? 5. How are institutions supporting learning analytics use amongst teaching staff? RESEARCH QUESTIONS
• 9.
  ACADEMIC LEVEL SURVEY SAMPLE OVERVIEW Variable (n varies dueto missing data) Category Absolute Frequency Relative Frequency Location (n = 274) Australia 269 98% New Zealand 5 2% Primary Work Role (n = 276) Teaching Students 185 67% Learning Support 25 9% Other 24 9% Research 19 7% Management/Administration 12 4% Academic Development 7 3% Student Support 4 1% LMS at Institution (n = 276) Blackboard 175 63% Moodle 89 32% Other 12 4% Employment Basis (n = 275) Full Time 223 81% Part Time 35 13% Casual 15 5% Other 2 1% Academic Level (n = 276) Lecturer 115 42% Senior Lecturer 79 29% Associate Professor 28 10% Associate Lecturer/Tutor 24 9% Professor 18 7% Other 12 4% Length of employment in current institution (n = 251) Less than 1.5 years 18 7% 1.5 – 5 years 57 22% 5 – 10 years 77 31% 10- 20 years 72 29% More than 20 years 27 11% Length of employment in Higher Education Sector (n = 269) Less than 1.5 years 4 1% 1.5 – 5 years 35 13% 5 – 10 years 61 23% 10- 20 years 105 39% More than 20 years 64 24% Enrolment modes of students taught (n = 276) Internally enrolled students only 144 52% A mix of internal and external students 105 38% Externally enrolled students only 14 5% Other 12 4%
• 10.
  LEARNING ANALYTICS DISCUSSIONFREQUENCY 0 30 60 90 120 150 180 210 Teaching Staff (n = 253) Program or Course Co- ordinator (n = 242) School or Faculty Management (n = 243) Learning support staff (n = 241) Students (n = 250) Colleagues in Communities of Practice (n = 243) Central L&T Group Staff (n = 240) Student Support Staff (n = 234) Institutional Management (n = 237) Daily Weekly Fortnightly Monthly < Monthly Never From the Academic Level Survey Notes: n varies between 234 and 253 per group due to missing data GROUPS
• 11.
  LEARNING ANALYTICS ACTIVITYPARTICIPATION 0 30 60 90 120 Delivering training on the use of learning analytics Being part of the group that is leading learning analytics at my institution Conducting formal research and/or publishing work on the topic of learning analytics Attending conferences/presentations/training specifically to learn about learning analytics Advocating for the use of learning analytics to colleagues (informal or formal) Reading about learning analytics for my own professional development Using learning analytics to help with analysis and decision making None of the listed choices* n From the Academic Level Survey Notes: * denotes mutually exclusive response n = 276
• 12.
  TRAINING ATTENDANCE ANDINTERESTS 0 50 100 150 Introduction to learning analytics concepts and applications (n = 168) Accessing data (n = 169) Overview of institutional plan for learning analytics use (n = 167) Interpreting and analysing data (n = 167) Responding to data (n = 166) Has attended Has not attended but will when available Has not attended and would choose not to TRAINING TOPICS
• 13.
  RATING OF INSTITUTIONALPROVISIONS 0 40 80 120 160 Provision of information about how learning analytics use will affect me (n = 232) Provision of information about how learning analytics is being used (n = 230) Opportunities to provide feedback about learning analytics implementation (n = 231) Provision of professional development about learning analytics (n = 232) Ease of visualisation and interpretation of data (n = 231) Relevance and comprehensiveness of data that I can access (n = 232) Ease of learning analytics data access (n = 231) Not Sure Poor or V.Poor Fair Good or V.Good LEARNING ANALYTICS PROVISIONS Notes: n varies between 230 and 232 per category due to missing data n
• 14.
  “Tell me whatdata is available, give me access to it, give me the time to use it and give me guidance in using it”. - interview participant WHAT ACADEMIC STAFF NEED
• 15.
  INTEREST IN LEARNINGANALYTICS APPLICATIONS 0 50 100 150 200 Assistance with decision making about student admissions to the institution Institutional management evaluating and improving teaching practice across the institution Identification of student success with a view to providing an affirmation/reward type of response Informing design and layout of online learning sites and environments Informing potential initiatives to promote student retention (e.g. mentoring) Program teams evaluating and improving their program curriculum Development of broad knowledge base about how effective learning can occur Students monitoring their own progress and identifying actions they could take Teaching staff evaluating and improving their own teaching practice Identification of at- risk students with a view to staff responding to address the risk No Interest A little interest A lot of interest SELECTED LEARNING ANALYTICS APPLICATIONS Notes: n varies slightly across categories due to missing data Excludes those people who indicated ‘not sure’ to better illustrate trends visually.
• 16.
  Being able tobetter understand who is in their class (demographics, prior academic history etc.) Being able to have consolidated information about their individual students at the touch of a button (e.g. seeing how their students are doing in other units, what their demographic data is, whether they are using resources etc. all in one place) INTERESTS OF ACADEMICS Learning analytics being used by people centrally to better justify or evidence directives relating to their teaching (e.g. when academics are told to respond in 24 hours to students is there evidence for this being useful?) Improving BOTH student (e.g. resource access patterns, socialisation) and teacher (e.g. teaching style, unit design) behaviour with respect to learning
• 17.
  Update teaching staffabout expectations Be realistic about how complex the area is Do it with teaching staff Build training and support capacity Incorporate into grad certificates (and other post-grad) around higher education teaching and learning Recognise the leadership challenges around LA and build capacity. MOVING FORWARD
• 18.
  Beer, C., Tickner,R., & Jones, D. (2014). Three paths for learning analytics and beyond: Moving from rhetoric to reality. In B. Hegarty, J. McDonald & S.-K. Loke (Eds.), Critical Perspectives on Education - From Rhetoric to Reality. Proceedings ASCILITE 2014 (pp. 242-250). Dunedin, New Zealand. http://ascilite2014.otago.ac.nz/files/fullpapers/185-Beer.pdf Gaševic, D., Dawson, S., & Siemens, G. (2015). Let's Not Forget: Learning Analytics Are about Learning. TechTrends: Linking Research and Practice to Improve Learning, 59(1), 64-71. http://i.unisa.edu.au/Global/LTU/Learning%20analytics/Lets%20not%20forget%20LA%20are%20about%20learning.p df Kennedy, G., Corrin, L., Dawson, S., Williams, D., Mulder, R., Khamis, S., & Copeland, S. (2014). Completing the loop: returning learning analytics to teachers. Paper presented at the Proceedings Rhetoric and Reality: Critical perspectives on educational technology, 31st ascilite Conference, Dunedin, New Zealand. http://ascilite.org/conferences/dunedin2014/files/concisepapers/76-Kennedy.pdf Dawson, S., Rogers, T., Kennedy, G., & Colvin, C. (in press). Student retention and learning analytics: a snapshot of current Australian practices and a framework for advancement. Strawberry Hills, NSW. www.he-analytics.com Kitto, K., Cross, S, Waters, Z. & Lupton, M. (2015). Learning Analytics beyond the LMS: the Connected Learning Analytics Toolkit. Paper presented at the fifth Learning Analytics and Knowledge (LAK’15) conference. http://eprints.qut.edu.au/81343/1/lak15_submission_116.pdf West, D., Huijser, H., Heath, D., Lizzio, A., Miles, C., Toohey, D., . . . Bronniman, J. (in press). Learning Analytics: Assisting Universities with Student Retention. Strawberry Hills, NSW: Office for Learning and Teaching. www.letstalklearninganalytics.edu.au MAPPING PROGRESS WITH ANALYTICS
• 19.
  Ferguson, R., (2012).Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6), 304–317. Beer, C., Jones, D. & Clark, D. (2012). Analytics and complexity: Learning and leading for the future. Proceedings Future Challenges: Sustainable Futures, ascilite, Wellington, NZ. Retrieved April 13, 2015 from http://eprints.usq.edu.au/23092/2/Beer_Jones_Clark_ascilite_2012_PV.pdf Suthers, D., & Verbert, K. (2013). Learning analytics as a “middle space.” Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), Leuven, Belgium. doi:10.1145/2460296.2460298 ADDITIONAL SOURCES