Download to read offline
Uploaded byRobert Power
EDUC5103G - Week 12 Class Slides (F21)
The 10th class session agenda covered various topics related to survey data analysis in instructional design, including: types of surveys used by educators; student satisfaction surveys; pre-test/post-test analyses; competency/aptitude profiles; best practices and pitfalls of surveys; ethical standards; and common survey analysis tools. The class included breakout activities for students to discuss surveys and their use, as well as a review of qualitative and quantitative data analysis techniques.
More Related Content
Similar to EDUC5103G - Week 12 Class Slides (F21)
More from Robert Power
EDUC5103G - Week 12 Class Slides (F21)
- 1. onlinetechnology EDUC5103G in education 10th ClassSession Agenda: • Check-In • Types of Surveys Most Used by Educators • Break-Out 1: Student Satisfaction Surveys • Data Analysis in Instructional Design • Break • Pre-Test / Post-Test Analyses • Competency / Aptitude Profiles • Break-Out 2: Best Practices & Pitfalls • Ethical Standards • Common Survey Analysis Tools • Check-Out • To Do List and Course Wrap-Up
- 2.
- 3. We acknowledge thelands and people of the Mississaugas of Scugog Island First Nation which is covered under the Williams Treaties. Ontario Tech University is situated on the Traditional Territory of the Mississaugas, a branch of the greater Anishinaabeg Nation which includes Algonquin, Ojibway, Odawa and Pottawatomi.
- 4. Cape Breton University’sDepartment of Education recognizes that Cape Breton Island is in Mi’kma’ki, the ancestral and unceded territory of the Mi’kmaq People. This territory is covered by the “Treaties of Peace and Friendship” which Mi’kmaq and Wolastoqiyik (Maliseet) people first signed with the British Crown in 1725. The treaties did not deal with surrender of lands and resources but in fact recognized Mi’kmaq and Wolastoqiyik (Maliseet) title and established the rules for what was to be an ongoing relationship between nations.
- 5.
- 6. surveyresults in instructional design Threetypes of survey data most likely to be used by educators and instructional designers: 1. Student Satisfaction Surveys 2. Pre-Test / Post-Test Results 3. Competency / Aptitude Profiles
- 7. flippedclass student satisfaction surveys activity StudentSurveys: Using Student Voice to Improve Teaching and Learning https://youtu.be/cP-8OGwQRPI 6 Critical Steps in Conducting Regular Assessments of Student Satisfaction http://blogem.ruffalonl.com/2013/09/12/critical-steps-conducting- regular-assessment-college-student-satisfaction/ Noel-Levitz: Using Satisfaction Data for Retention https://youtu.be/p9Nm-OE_-t0
- 8.
- 9.
- 10. Group Instructions: Watch thevideo / read the blog post (can be opened directly from MindMup) Choose a group recorder and reporter (can be the same person) As a group, use MindMup to: • Create a summary of the key points • How are learner satisfaction surveys being used at your institution? o Institutional Level? o Teacher / Classroom Level? o Course Level? • How would you use the video (blog post), and the key points, to inform your own practice as an educator / instructional designer? • Describe how you would use the video (blog post), and the key points, to frame and plan a professional development session for teachers / trainers at your institution • Report your summaries to the class flippedclass student satisfaction surveys activity
- 11. mindmuplinks Group 1: https://drive.google.com/file/d/1E2Yf7V5mXFgmBuCI7 TZQ3vYKk1iSFmS4/view?usp=sharing Group 2: https://drive.google.com/file/d/1dDMcO9mvYPmL6b8v ydggaJ7gKTwdbj4I/view?usp=sharing Group3: https://drive.google.com/file/d/1pDnEEieS1G8Av1doDt dHTu5KVCJqNVHq/view?usp=sharing
- 12. surveyresults in instructional design February1st March 1st
- 13.
- 14. surveyresults in instructional design URGENT: Thingsthat NEED to be Addressed NOW NICE: Things that CAN be Addressed NOW FUTURE: Things that CANNOT be Addressed NOW NOT POSSIBLE: Things that CANNOT, SHOULD NOT, or DO NOT NEED to be changed More clarity on instructions / due dates, etc Add video intros and tips for specific assignments and activities Offer learners opportunity to change groups and / or topics for presentations Add a video on using the Zotero Tool Adding gamification or games based learning to the topics list Add a course calendar to the website Utilize features of Blackboard? Adding wearables, AR and VR to topics list Adding data analytics to topics list Utilize features of Blackboard?
- 15.
- 17.
- 18. breaktime Please return by7:45 pm ET
- 19. surveyresults cause & effect Correlation≠ Causation
- 20.
- 21. surveyresults pre-test / post-testanalysis Maturation Effect
- 22. surveyresults pre-test / post-testanalysis Comparison to a Control Group Necessary to demonstrate a causal relationship Experimental Design and Survey Data
- 23. surveyresults pre-test / post-testanalysis What are the chances that these changes were random? Less than 5% = Statistically Significant
- 24. surveyresults pre-test / post-testanalysis (mTSES2 – mTSES1) – (TSES2 – TSES1) = Net Change(Intervention Effect) Pre-Test: Self-Efficacy with mLearning Post-Test: Self-Efficacy with mLearning Pre-Test: Self-Efficacy with Teaching Post-Test: Self-Efficacy with Teaching Setting a control within the same group is possible – with careful instrument design (Power, 2015)
- 25. surveyresults pre-test / post-testanalysis 0.19 0.31 0.01 0.57 0.68 0.11 0.57 0.68 0.10 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Student Engagement Instructional Strategies Classroom Management Overall Trends in Self-Efficacy Scores (Power, 2015) TSES mTSES Net Change (mTSES2 – mTSES1) – (TSES2 – TSES1) = Net Change(Intervention Effect)
- 26. surveyresults competency profiles Visual Auditory Reading& Writing Kinesthetic Learning Styles Preferences IQ
- 27. 0.19 0.31 0.01 0.57 0.68 0.11 0.57 0.68 0.10 0 0.2 0.4 0.6 0.8 Student EngagementInstructional Strategies Classroom Management Overall Trends in Self-Efficacy Scores (Power, 2015) TSES mTSES Net Change 0.51 0.64 0.49 0.64 0.62 0.45 0.12 -0.01 -0.04 -0.10 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 Student Engagement Instructional Strategies Classroom Management Overall Trends in Self-Efficacy Scores (ID4ML) TSES (ID4ML) mTSES (ID4ML) Net Change (ID4ML) More info: https://youtu.be/JN4K8XevefM
- 28. flippedclass best practices &pitfalls activity 7 tips for good survey questions https://youtu.be/Iq_fhTuY1hw Confirmation Bias https://youtu.be/B_YkdMwEO5U How Ice Cream Kills! Correlation vs. Causation https://youtu.be/VMUQSMFGBDo
- 29. Group Instructions: Watch theVideo (can be opened directly from MindMup) Choose a group recorder and reporter (can be the same person) As a group, use MindMup to: • Create a summary of the key points • How would you use the video, and the key points, to inform your own practice as an educator / instructional designer? • Describe how you would use the video, and the key points, to frame and plan a professional development session for teachers / trainers at your institution • Report your summaries to the class flippedclass best practices & pitfalls activity
- 30. mindmuplinks Group 1: https://drive.google.com/file/d/1- HUq1PTn0cBMLDCGKqzta13TADq-RdrJ/view?usp=sharing Group 2: https://drive.google.com/file/d/1XhQUXY9lV64ocHwJKFdJlt4KRj PPCNYO/view?usp=sharing Group3: https://drive.google.com/file/d/12UZc-RdrRZQMdWhhUua- ZGx7gSYS2A5F/view?usp=sharing
- 31.
- 32. Canadian Tri-Council PolicyStatement (TPS-2) http://www.pre.ethics.gc.ca/eng/policy- politique/initiatives/tcps2-eptc2/Default/ UOIT Research Services https://research.uoit.ca/faculty/ethics- safety-research-conduct/index.php
- 33. surveyresults common tools Quantitative Tools:Statistics Package for the Social Sciences (SPSS) http://www-01.ibm.com/software/analytics/spss/
- 34.
- 35. surveyresults common tools Quantitative &Qualitative Tools: Excel (Spreadsheet Applications)
- 36. moreresources Videos: Confirmation Bias -https://youtu.be/B_YkdMwEO5U How Ice Cream Kills! Correlation vs. Causation - https://youtu.be/VMUQSMFGBDo Research Ethics - https://youtu.be/9pCfq5QH_v4 Six Critical Steps in Conducting Student Satisfaction Assessments - https://youtu.be/d14AZE61_EE 7 tips for good survey questions - https://youtu.be/Iq_fhTuY1hw Types of Survey Sampling Techniques - https://youtu.be/rASK8PpqakM Using the mTSES to Evaluate and Optimize mLearning Professional Development - https://youtu.be/JN4K8XevefM Ways to Guide Strategic Planning with Student Satisfaction Survey Results - https://youtu.be/ns7ekUDVe1Q Ways to Impact Student Retention with Satisfaction Survey Results - https://youtu.be/HBtuYScGEcQ Ways to Use Student Satisfaction Results for Accreditation - https://youtu.be/9aECPrIDaDw
- 37. moreresources Other Resources: Freelon, D.(2011). ReCal2: Reliability for 2 coders. Retrieved from http://dfreelon.org/utils/recalfront/recal2/ Government of Canada (2015). Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TPS-2 2014). Available from http://www.pre.ethics.gc.ca/eng/policy-politique/initiatives/tcps2-eptc2/Default/ IMB (n.d.). SPSS Software. Available from http://www-01.ibm.com/software/analytics/spss/ Power, R., Cristol, D., Gimbert, B., Bartoletti, R, & Kilgore, W. (in press). Using the mTSES to evaluate and optimize mLearning professional development. The International Review of Research in Open and Distributed Learning. Power, R. (2015). A framework for promoting teacher self-efficacy with mobile reusable learning objects (Doctoral dissertation, Athabasca University). Available from http://hdl.handle.net/10791/63 QSR International (n.d.). Nvivo: The #1 software for qualitative data analysis. Available from http://www.qsrinternational.com/product University of Ontario Institute of Technology (2016). Ethics, Safety and Research Conduct. Available from https://research.uoit.ca/faculty/ethics-safety-research-conduct/index.php
- 38.
- 39. wrappingup To Do List: •Submit Assignment 4 Self-Evaluation by 11:59 pm ET SUNDAY, NOV 28TH! • Submit All Remaining Assignment 3: Learning Reflections by 11:59 pm ET SUNDAY, DEC 5TH! • Have a great break!