Download to read offline
Uploaded bydrdjwalker
eAssessment Policy and Practice
This document discusses the policy and practice of e-assessment at the University of Dundee, outlining its integration into the broader educational technology strategy and the necessity for strategic planning to ensure reliability, security, and pedagogical effectiveness. It emphasizes the importance of stakeholder involvement, continuous development in response to technological advancements, and the inclusion of accessibility considerations in e-assessment practices. Future directions for e-assessment include addressing emerging technologies, enhancing academic-professional development, and fostering shared ownership of policies across institutions.
More Related Content
Similar to eAssessment Policy and Practice
![Foundationsf2f presv2[1]](https://cdn.slidesharecdn.com/ss_thumbnails/foundationsf2fpresv21-120120214714-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
eAssessment Policy and Practice
- 1. e-Assessment Policy andPractice – Making the Connection David Walker (@drdjwalker) University of Dundee
- 2. Overview • • • • • • • Locate e-assessment incontext Conditions for embedding e-assessment Role of e-assessment in wider Technology Enhanced Learning (TEL) strategy Channels/stakeholders informing strategy/policy Embedding inclusion in policy and practice Institutional efforts and annual cycle of e-assessment Consider factors that will shape policy in the future
- 3. Not about bureaucracy… Youcan view the full collection on Flickr: http://www.flickr.com/photos/8181983@N05/ or by visiting www.vimrod.com
- 4. It’s about addressing… “Dowe have a policy on this?” Colleague, University of Galway (2010)
- 5. What is e-Assessment? Youcan view the full collection on Flickr: http://www.flickr.com/photos/8181983@N05/ or by visiting www.vimrod.com
- 6. e-Assessment in Context •Rapid developments in field of e-learning in last 10 years • Expansion of higher education system (mass higher education) – social, philosophical and practical implications • e-assessment becoming particularly prevalent across the sector majority of activity within science/engineering-based disciplines • Growing body of research underpinning practice • Emphasis on alignment of learning and assessment methods with learning outcomes (curriculum alignment)
- 7. Credibility Crisis? e-assessment “…mustsatisfy critics of its pedagogical fitness for purpose”. McKenna, C. & Bull, J. (2000). Quality assurance of computer-assisted assessment: practical and strategic issues. Quality Assurance in Education. 8(1), 24-31. • Can e-assessment be used to assess appropriate levels of learning and skills? • Are appropriate quality assurance procedures in place to ensure the validity/reliability/security of e-assessments?
- 8. Danger of Disjunction “Whileassessment is one of the last things that university teachers consider when planning and developing courses, it is often the first thing that most students think about.” Open University (Undated)
- 9. Embedding e-Assessment • Realising promiseof e-assessment requires strategic approach • Need to address concerns about security and reliability to enhance credibility • Ensure policy/strategy is practical and sustainable and aligned to institutional strategic aims • Foster sense of ownership of policy across institution
- 10. e-Assessment Policy • Useof e- assessment systems and emerging tools requires clear guidance • Dundee first UK institution to develop a Senate approved policy in this area • Policy and Procedures (2003) - broadly aligned with BS7988:2002 • Policy specifies procedures to be followed if e-assessment used in modules • Regularly reviewed/updated
- 11. Informing Policy e-Learning Forum t en En vir on Middle-Out m on vir En m en t Top-Down InstitutionalAims / Sectoral Agenda / Legal Duty Learning Lunches Bottom-Up Environment
- 12. e-Assessment Stakeholders The ‘HiddenTeam’ ‘Front of House’ IT Database Administrator Examinations Office Lecturer Distance Student Is the database correctly configured? Is it receiving and retrieving correct data? Is it properly secured behind a firewall? Are these results 100% reliable and secure? Do our procedures represent the best possible practice? Are procedures in place to address exam irregularities? Is my technology compatible? Is my connection reliable? Is this a replacement for F2F? IT Desktop Administrator eLearning/Assessment Manager Does the desktop software operate correctly? Are the “exam accounts” secure and available? Are the PCs well maintained and reliable? Are the staffing levels correct? Is our software choice appropriate? Are the server specifications correct to cope with the expected load? Are staff education procedures in place? Which software should I use? Who can electronically author my questions? Can we use textbook questions? Is this assessment driving learning? Are my questions too simple? Is this technology reliable? Do the students find it tedious? Where can we obtain challenging questions? Are these assessments aligned with our learning outcomes? IT Server Administrator Timetabling Officer Is the server fully patched, reliable and secure? Is the software functioning correctly and reliably Do we have monitoring and is it working? Are the logging and backup procedures working correctly? Can the proposed exams be fitted into the best rooms? Are assessments receiving appropriate prioritisation? Disability Services Manager Do procedures address needs of students requiring reasonable adjustments? Is appropriate assistive technology available School Secretary Can we book an IT suite for 240 students on Wednesday the 25th? Two students have additional requirements. Learning Technologist Are staff adequately trained and aware of how the systems operate? Are assessments correctly scheduled? Will the exam run smoothly? Are contingency measures in place? Student This is nice and private I am getting real feedback here Do I know the answers? Can I learn from this? I am at home in this online environment
- 13. Evolution of Policy •Original policy redeveloped and re-approved in 2007 and expanded to include guidance on use of emerging assessment tools e.g. ePortfolios, electronic submission etc • Continued development in assessment and Web 2.0 technologies and approaches - prompted extensive revision to policy in 2009 • Revision process involved: – Cross-team ‘framework’ meeting to identify key issues – Input from specialists on matters ranging from technical to procedural. Cross-team approach to policy development – input from members of the ‘hidden team’ important – Review of best practice in field including policies produced by fellow UK HE Institutions
- 14. Policy Issues • Greateremphasis on roles and responsibilities and clearer stipulation on time scales for registration and preparation of electronic examinations • Extended guidance on accessibility and disability support • As use of e-assessment increases technical issues more frequent – greater clarity required on how technical issues will be managed • Need for clarity on invigilation (aligned with University regulations on conduct of examinations)
- 15. Inclusive Practice • Inclusion =supporting disabled staff/students and staff/students who are situationally disabled • e-Assessment (TEL) has huge potential to enhance inclusion – or introduce new barriers • Dundee has an active inclusion support network • Web Accessibility Service co-ordinator involved across the range of TEL strategy activities – important contributor to policy
- 16.
- 17.
- 18. Institutional Efforts How canan institution help to realise the promise of e-assessment? • Enlist small groups across schools to pilot new initiatives over a fixed period • Ensure appropriate channels exist to disseminate good practice e.g. Teaching Awards/Learning Lunches • Offer appropriate academic professional development opportunities – include both internal and external contributors • Support a culture of professional development and a willingness engage with new ideas/approaches
- 19. Annual Cycle ofe-Assessment Sept. Online resit exams Testing & tuning Modules begin Annual framework meeting Summer upgrade window Online exams June December Mid semester window Online exams • Growth may occur each year eAssessment education module • Changes may be made to the systems • A framework allows the process to be sustained
- 20. Summative e-Assessment “Students can,with difficulty, escape from the effects of poor teaching, they cannot… escape the effects of poor assessment.” (Boud, 1995) • High-stakes: concerns about validity, reliability, credibility, fidelity, fairness & transparency • High salience higher levels of test anxiety • Clear policy and procedures crucial
- 21. Formative e-Assessment “The clearadvantage of online testing as part of formative assessment is that it is in a private space and, in most cases, it can be accessed at a time when the student feels appropriate.” (Challis, 2005) • e-assessment widely used for formative assessment • e-Testing: automatic feedback to learners - allow students to practice skills • • Reflective journal/scientific log – comments left by academics Most prominent role for e-assessment?
- 22. Student Views "We needformative assessments! Online formatives should be available after every block.” “So it’s a bit more relaxed sitting on a computer because you can, you can sit back and just click through whereas you’ve got all this going on in your mind but...you don’t have the added thing of writing down.” “I think that on the whole this type of assessment it can work, and it can consolidate learning, and it can enable you to apply knowledge and understanding to situations and problem solving but I think on the other hand it can be a means to an end..."
- 23. Future of e-Assessment •Secure, stable, performing systems - many institutions moving to hosted solutions • Innovation within and beyond existing (dominant) assessment management systems • Policy issues around use of mobile technologies • Risk to large-scale testing posed by closure of IT suites • Stronger links between educational developers/learning technologists/academics – more effective assessments • Increased sharing/sales of item banks • Clearer divide between (compliance) ‘testing in’ corporate sector and ‘assessment’ in education sector
- 24. Closing Remarks • Successfule-assessment/technology enhanced learning dependent on numerous components – strategic, technical and procedural • Range of stakeholders – academic staff, students, e-learning staff, IT staff, administrators, senior management • Challenge is to accommodate growth, consider risk and accept rapid change • Issues of sustainability – IT suite availability, staffing, licensing etc • Policy/strategy should be informed through formal and informal routes – need to achieve shared ownership
- 25. Closing Messages “e-Assessment hascome a long way and is here to stay.” Source: http://www.da.mod.uk/academy-events/e-assessment-in-practice
- 26. Contact Information Dr DavidWalker Library & Learning Centre University of Dundee Email: [email protected] Twitter: @drdjwalker