LEARNING AND STUDY STRATEGIES
A LEARNING ANALYTICS APPROACH FOR FEEDBACK
Tinne De Laet
KU Leuven, Belgium
OUTLINE
1. Who am I? Why I am here?
2. What is learning analytics?
3. Specific context & methodology
4. Results
5. Conclusion & reflections
WHO AM I? WHY I AM HERE?
WHO AM I? WHY I AM HERE?
woman
engineer
Head Tutorial Services
Engineering Science
KU Leuven, Belgium
.
Tinne De Laet
associate
professor
WHO AM I? WHY I AM HERE?
Head of Tutorial Services of Engineering
Science KU Leuven
• Daily experiences of challenges in transition from
secondary to higher education
• looking for opportunities for cross-fertilization
between “first-year experience” and “engineering”
•Forward-looking cooperation project: 562167-
EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
•Successful Transition from secondary to higher
Education using Learning Analytics
•KU Leuven (Belgium), TU Delft (Netherlands),
TU Graz (Austria), Nottingham Trent University
(UK), European Society of Engineering
Education (SEFI)
• http://stela-project.eu/
Coordinator of STELA Erasmus+
forward-looking cooperation project
WHAT IS LEARNING ANALYTICS?
WHAT IS LEARNING ANALYTICS?
no universally agreed definition
7
“the measurement, collection, analysis and reporting of data about learners and their
contexts, for purposes of understanding and optimizing learning and the environments in
which it occurs” [1]
[1] Learning and Academic Analytics, Siemens, G., 5 August 2011, http://www.learninganalytics.net/?p=131
[2] What is Analytics? Definition and Essential Characteristics, Vol. 1, No. 5. CETIS Analytics Series, Cooper, A.,
http://publications.cetis.ac.uk/2012/521
“the process of developing actionable insights through problem definition and the
application of statistical models and analysis against existing and/or simulated future
data” [2]
WAT IS LEARNING ANALYTICS?
8
[3] Learning Analytics and Educational Data Mining, Erik Duval’s Weblog, 30 January 2012,
https://erikduval.wordpress.com/2012/01/30/learning-analytics-and-educational-data-mining/
“learning analytics is about collecting
traces that learners leave behind and
using those traces to improve learning”
[Erik Duval, 3]
† 12 March 2016
no universally agreed definition
IS IT ABOUT INSTITUTIONAL DATA?
•high-level figures:
provide an overview for internal and external reports;
used for organisational planning purposes.
•academic analytics:
figures on retention and success, used by the institution to assess performance.
•educational data mining:
searching for patterns in the data.
•learning analytics:
use of data, which may include ‘big data’,
to provide actionable intelligence for learners and teachers.
[4] Learning analytics FAQs, Rebecca Ferguson, Slideshare,
http://www.slideshare.net/R3beccaF/learning-analytics-fa-qs
SPECIFIC CONTEXT & METHODOLOGY
SPECIFIC CONTEXT
• open admission in the Flemish (Belgium) higher education system
→ a substantial part of first-year students enters without the right qualifications
→ first-year drop-out rate of around 30% in the Faculties of Science & Technology at KU Leuven.
• university invests in advising students before and throughout the first-year
• readySTEMgo Erasmus+ project
• paper-and-pencil questionnaires with first-year students
• five academic skills as measured in the LASSI (The Learning and Study Strategies Inventory) are important for STEM study
success
• concentration, performance anxiety, motivation, the use of test strategies, and time management (Pinxten et al. 2016).
• Results were actively disseminated to the KU Leuven faculties’ student support services and the
central study advice center, who in return adapted their coaching and training based on the results.
• BUT students did not receive feedback
THE DASHBOARD
HTTPS://LEARNINGANALYTICS.SET.KULEUVEN.BE/LASSI.STELA
THE DASHBOARD
THE DASHBOARD
THE INTERVENTION
• 1406 first-year KU Leuven (STEM) students with full LASSI profile
• 12 study programs, 4 faculties
• Students received “personalized email” with invitation to the dashboard
DO STUDENTS ENTER THE DASHBOARD?
1135 studenten “clicked through” (80,7%)
STUDENTS WITH BETTER LEARNING SKILLS GO TO THE
DASHBOARD MORE
17Kruskal-Wallis test, with multiple comparison according to Dunn with a Bonferroni correction
STUDENTS WITH WORSE LEARNING SKILLS VISIT THE
CORRESPONDING TAB AND READ THE TIPS MORE
18Kruskal-Wallis test, with multiple comparison according to Dunn with a Bonferroni correction
STATISTICAL MODEL
19
Logistic regression
DO STUDENTS LIKE IT?
The information is clear
The information is useful
I would like to receive
more similar information
CONCLUSION AND REFLECTIONS
SOME SPECIFICS
• students like “additional” feedback
• challenge to attract “right” students to the platform
• once students are on the platform, the “targeted” students interact more
Future
• repeat intervention and extend (KU Leuven and TU Delft)
• study relation with study results
• qualitative analysis using focus groups and structured interviews with students
LEARNING ANALYTICS & DATA
FOCUS ON
DATA THAT
IS
AVAILABLE
• A lot of data COULD be available
• What IS available?
LEARNING ANALYTICS AND ACTIONABLE
INSIGHTS
“Female students are more successful in higher
education than male students”
70%
successful
60%
successful
so
?
FEEDBACK
SHOULD
ALWAYS BE
ACTIONABLE
LEARNING ANALYTICS & ETHICS &
ADOPTION & …
• Integrate all available expertise DURING development
• educational scientists
• computer scientists & IT experts
• visualization experts
• PRACTITIONERS!!! (study advisors, tutors, etc.)
• students & teachers
•Be wary of out-of-the-box commercial solutions!
• no one-size-fits-all solution
• jeopardizes acceptance of students and staff
• What is underlying the recommendation?
→ actionable!
→ transparency!
•Ethics AND privacy are big issues!
• Ethics: involve practitioners and experts
• Privacy regulations can be hurdle
might be opportunity for learning analytics
→ overview and insight in data that IS gathered!
25
USE ALL
AVAILABLE
EXPERTISE

Learning and study strategies: a learning analytics approach for feedback

  • 1.
    LEARNING AND STUDYSTRATEGIES A LEARNING ANALYTICS APPROACH FOR FEEDBACK Tinne De Laet KU Leuven, Belgium
  • 2.
    OUTLINE 1. Who amI? Why I am here? 2. What is learning analytics? 3. Specific context & methodology 4. Results 5. Conclusion & reflections
  • 3.
    WHO AM I?WHY I AM HERE?
  • 4.
    WHO AM I?WHY I AM HERE? woman engineer Head Tutorial Services Engineering Science KU Leuven, Belgium . Tinne De Laet associate professor
  • 5.
    WHO AM I?WHY I AM HERE? Head of Tutorial Services of Engineering Science KU Leuven • Daily experiences of challenges in transition from secondary to higher education • looking for opportunities for cross-fertilization between “first-year experience” and “engineering” •Forward-looking cooperation project: 562167- EPP-1-2015-1-BE-EPPKA3-PI-FORWARD •Successful Transition from secondary to higher Education using Learning Analytics •KU Leuven (Belgium), TU Delft (Netherlands), TU Graz (Austria), Nottingham Trent University (UK), European Society of Engineering Education (SEFI) • http://stela-project.eu/ Coordinator of STELA Erasmus+ forward-looking cooperation project
  • 6.
    WHAT IS LEARNINGANALYTICS?
  • 7.
    WHAT IS LEARNINGANALYTICS? no universally agreed definition 7 “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” [1] [1] Learning and Academic Analytics, Siemens, G., 5 August 2011, http://www.learninganalytics.net/?p=131 [2] What is Analytics? Definition and Essential Characteristics, Vol. 1, No. 5. CETIS Analytics Series, Cooper, A., http://publications.cetis.ac.uk/2012/521 “the process of developing actionable insights through problem definition and the application of statistical models and analysis against existing and/or simulated future data” [2]
  • 8.
    WAT IS LEARNINGANALYTICS? 8 [3] Learning Analytics and Educational Data Mining, Erik Duval’s Weblog, 30 January 2012, https://erikduval.wordpress.com/2012/01/30/learning-analytics-and-educational-data-mining/ “learning analytics is about collecting traces that learners leave behind and using those traces to improve learning” [Erik Duval, 3] † 12 March 2016 no universally agreed definition
  • 9.
    IS IT ABOUTINSTITUTIONAL DATA? •high-level figures: provide an overview for internal and external reports; used for organisational planning purposes. •academic analytics: figures on retention and success, used by the institution to assess performance. •educational data mining: searching for patterns in the data. •learning analytics: use of data, which may include ‘big data’, to provide actionable intelligence for learners and teachers. [4] Learning analytics FAQs, Rebecca Ferguson, Slideshare, http://www.slideshare.net/R3beccaF/learning-analytics-fa-qs
  • 10.
    SPECIFIC CONTEXT &METHODOLOGY
  • 11.
    SPECIFIC CONTEXT • openadmission in the Flemish (Belgium) higher education system → a substantial part of first-year students enters without the right qualifications → first-year drop-out rate of around 30% in the Faculties of Science & Technology at KU Leuven. • university invests in advising students before and throughout the first-year • readySTEMgo Erasmus+ project • paper-and-pencil questionnaires with first-year students • five academic skills as measured in the LASSI (The Learning and Study Strategies Inventory) are important for STEM study success • concentration, performance anxiety, motivation, the use of test strategies, and time management (Pinxten et al. 2016). • Results were actively disseminated to the KU Leuven faculties’ student support services and the central study advice center, who in return adapted their coaching and training based on the results. • BUT students did not receive feedback
  • 12.
  • 13.
  • 14.
  • 15.
    THE INTERVENTION • 1406first-year KU Leuven (STEM) students with full LASSI profile • 12 study programs, 4 faculties • Students received “personalized email” with invitation to the dashboard
  • 16.
    DO STUDENTS ENTERTHE DASHBOARD? 1135 studenten “clicked through” (80,7%)
  • 17.
    STUDENTS WITH BETTERLEARNING SKILLS GO TO THE DASHBOARD MORE 17Kruskal-Wallis test, with multiple comparison according to Dunn with a Bonferroni correction
  • 18.
    STUDENTS WITH WORSELEARNING SKILLS VISIT THE CORRESPONDING TAB AND READ THE TIPS MORE 18Kruskal-Wallis test, with multiple comparison according to Dunn with a Bonferroni correction
  • 19.
  • 20.
    DO STUDENTS LIKEIT? The information is clear The information is useful I would like to receive more similar information
  • 21.
  • 22.
    SOME SPECIFICS • studentslike “additional” feedback • challenge to attract “right” students to the platform • once students are on the platform, the “targeted” students interact more Future • repeat intervention and extend (KU Leuven and TU Delft) • study relation with study results • qualitative analysis using focus groups and structured interviews with students
  • 23.
    LEARNING ANALYTICS &DATA FOCUS ON DATA THAT IS AVAILABLE • A lot of data COULD be available • What IS available?
  • 24.
    LEARNING ANALYTICS ANDACTIONABLE INSIGHTS “Female students are more successful in higher education than male students” 70% successful 60% successful so ? FEEDBACK SHOULD ALWAYS BE ACTIONABLE
  • 25.
    LEARNING ANALYTICS &ETHICS & ADOPTION & … • Integrate all available expertise DURING development • educational scientists • computer scientists & IT experts • visualization experts • PRACTITIONERS!!! (study advisors, tutors, etc.) • students & teachers •Be wary of out-of-the-box commercial solutions! • no one-size-fits-all solution • jeopardizes acceptance of students and staff • What is underlying the recommendation? → actionable! → transparency! •Ethics AND privacy are big issues! • Ethics: involve practitioners and experts • Privacy regulations can be hurdle might be opportunity for learning analytics → overview and insight in data that IS gathered! 25 USE ALL AVAILABLE EXPERTISE