Assessment 2.0
Luís Tinoca
Institute of Education
University of Lisbon
Agenda
       Web 2.0:
New learning landscapes




    Learning 2.0:
 New learning cultures




    Assessment 2.0:
  the chalenges of e-
      assessment
New learning landscapes
Web 2.0
Bologna Process
    (European Commission, 2008)

challenged higher education to
promote learning environments that are
centered in the development of
competences
The e-learning explosion

The emergence of new digitally
supported learning environments
  Garrisson & Anderson (2003),
  McConnell (2006),
  Pereira et al. (2009)


  •   Collaborative

  •   Student centered
Language and communication


• Four main types of metacompetences
 (Pereira et al., 2009):
    » Problem solving

    » Group work

    » Metacognitive

    » Fluency in ICT use
New learning culture




                       8
Learning
       Complicated  Complex

        Technological mediation

           Learning theories 2.0

Student centered  Participative

           Open  Transparent
Learning 2.0
                     (Redecker, 2009)
• Conectivism (Siemens, 2005)

• Comunities of Learning
  (Wenger et al., 2002)



• Comunities of Inquiry
  (Garrison & Anderson, 2003)



• Produsage (Bruns & Humphreys,
  2007)
C-Learning: learning with others

from e-learning to c-learning
                       Mota (2009)




Comunity
  Comunication
    Colaboration
         Conections



                                         Alec Couros (2006)


Learning is framed by social processes
Assessment 2.0
“Tell me how you assess
I ll tell you how you
teach”
                Abrantes (1990)
A new assessment culture
Emerging from the growing criticism of
traditional testing methods relating to
the unrealistic nature of the tests, the
loss of faith in them as valid measures
of learning, and an over-reliance on
tests as the ultimate
goal of the instruction
process.
Assessment OF learning
assessment focused on measurement
and scaling
Assessment FOR learning
assessment meant
for the students,
through feedback,
to understand their
own learning
processes and
the goals that they
intend to achieve
From psychometrics to edumetrics
Edumetrics criteria are recognized as more valid
and fair for competence based assessment, given
their emphasis in flexibility and authenticity, as
well as their integration into the learning process
valuing the formative function of assessment
The challenge of e-assessment
             “Confusion of tongues”

“e-assessment occurs when there is an automated
marking/response to student input on-screen in a
test, informing on the process of answering a
question and providing feedback to learners and
their teachers through well-crafted advice and
reports”. (Beevers, 2010)

“e-assessment is sometimes used to refer solely
to on-screen assessment but, in its broadest
sense, can refer to all technology-enabled
assessment activities”. (JISC, 2010)
e-assessment
          our definition
e-assessment refers to all technology-enabled
assessment activities where the design and
student activities (complete, present, submit)
must be mediated by technologies.


It is regarded as optional the format in which the
instructor presents the assignment, as well as
the way feedback is provided
@ssess – project financed through FCT (PTDC/CPE-CED/104373/2008)
Steps of an
       e-assessment strategy


                     Student
• design                              • classification
• presentation   • complete           • feedback
                 • present
                 • submit
   Instructor                             Instructor


                         must be mediated by technologies
e-assessment benefits                                    (JISC, 2010)


• Greater variety and authenticity in assessment designs

• Improved learner engagement

• Choice in the timing and location of assessments

• Capture of wider skills and attributes

• Efficient submission, marking, moderation and
  data storage

• Consistent, accurate results

• Increased opportunities for learners to act on feedback
Conceptual framework for
     e-assessment

                authenticity




 consistency   e-assessment      transparency




                practicability
E-assessment




authenticity       consistency       transparency        practicability



                      instruction-
     similarity       assessment       democratization            cost
                       alignment


                        multiple
    complexity                            engagement           efficiency
                       indicators



                        relevant
     adequacy                              visibility        sustainability
                         criteria


                     competences-
    significance      assessment            impact
                       alignment
Authenticity
• Similarity – competeces needed in
  real/professioanl life
• Complexity – cognitive chalenge
• Adequacy – adequate performing
  conditions
• Significance – value for students,
  instructors and employers
Consistency
This dimension emerges as an answer to the
traditional demands for validity and reliability,
associated with psychometric indicators.

• Instruction-assessment alignment

• Relevant criteria

• Competences-assessment alignment

• Multiple indicators – assessment methods,
  contexts and assessors
Transparency
• Democratization – availability and
  participation
• Engagement – participation in the
  definition of the learning
  goals
• Visibility – present/share
  processes and/or products
• Impact – effect on the learning process
  and on course design
Practicability
• Cost – time, digital resources,
  training, …

• Efficiency

• Sustainability – implement and sustain
  the proposed assessment design,
  taking into account the learner profiles
  and the contextual constraints, both for
  the organizations and for the assessors
Discussion
These dimensions are articulated, representing several
degrees of reciprocal interdependence.



The criteria, more than just illustrating
the different features of each dimension,
allow for the operational description of
each criterion stage of implementation, and so contribute
to the evaluation of the achieved assessment strategy
quality level.
Discussion
What can be the contributions of
e-assessment for the assessment culture?

How can e-assessment be used
for internal improvement and
external accreditation?

From a research standpoint should all
dimensions be regarded as equally
important?
Thank You!
           Gracias!
           Obrigado!




         I care so I share
http://www.slideshare.net/luistinoca
          ltinoca@ie.ul.pt
             @luistinoca

Assessment 2.0

  • 1.
    Assessment 2.0 Luís Tinoca Instituteof Education University of Lisbon
  • 2.
    Agenda Web 2.0: New learning landscapes Learning 2.0: New learning cultures Assessment 2.0: the chalenges of e- assessment
  • 3.
  • 4.
  • 5.
    Bologna Process (European Commission, 2008) challenged higher education to promote learning environments that are centered in the development of competences
  • 6.
    The e-learning explosion Theemergence of new digitally supported learning environments Garrisson & Anderson (2003), McConnell (2006), Pereira et al. (2009) • Collaborative • Student centered
  • 7.
    Language and communication •Four main types of metacompetences (Pereira et al., 2009): » Problem solving » Group work » Metacognitive » Fluency in ICT use
  • 8.
  • 9.
    Learning Complicated  Complex Technological mediation Learning theories 2.0 Student centered  Participative Open  Transparent
  • 10.
    Learning 2.0 (Redecker, 2009) • Conectivism (Siemens, 2005) • Comunities of Learning (Wenger et al., 2002) • Comunities of Inquiry (Garrison & Anderson, 2003) • Produsage (Bruns & Humphreys, 2007)
  • 11.
    C-Learning: learning withothers from e-learning to c-learning Mota (2009) Comunity Comunication Colaboration Conections Alec Couros (2006) Learning is framed by social processes
  • 12.
  • 13.
    “Tell me howyou assess I ll tell you how you teach” Abrantes (1990)
  • 14.
    A new assessmentculture Emerging from the growing criticism of traditional testing methods relating to the unrealistic nature of the tests, the loss of faith in them as valid measures of learning, and an over-reliance on tests as the ultimate goal of the instruction process.
  • 15.
    Assessment OF learning assessmentfocused on measurement and scaling
  • 16.
    Assessment FOR learning assessmentmeant for the students, through feedback, to understand their own learning processes and the goals that they intend to achieve
  • 17.
    From psychometrics toedumetrics Edumetrics criteria are recognized as more valid and fair for competence based assessment, given their emphasis in flexibility and authenticity, as well as their integration into the learning process valuing the formative function of assessment
  • 18.
    The challenge ofe-assessment “Confusion of tongues” “e-assessment occurs when there is an automated marking/response to student input on-screen in a test, informing on the process of answering a question and providing feedback to learners and their teachers through well-crafted advice and reports”. (Beevers, 2010) “e-assessment is sometimes used to refer solely to on-screen assessment but, in its broadest sense, can refer to all technology-enabled assessment activities”. (JISC, 2010)
  • 19.
    e-assessment our definition e-assessment refers to all technology-enabled assessment activities where the design and student activities (complete, present, submit) must be mediated by technologies. It is regarded as optional the format in which the instructor presents the assignment, as well as the way feedback is provided @ssess – project financed through FCT (PTDC/CPE-CED/104373/2008)
  • 20.
    Steps of an e-assessment strategy Student • design • classification • presentation • complete • feedback • present • submit Instructor Instructor must be mediated by technologies
  • 21.
    e-assessment benefits (JISC, 2010) • Greater variety and authenticity in assessment designs • Improved learner engagement • Choice in the timing and location of assessments • Capture of wider skills and attributes • Efficient submission, marking, moderation and data storage • Consistent, accurate results • Increased opportunities for learners to act on feedback
  • 22.
    Conceptual framework for e-assessment authenticity consistency e-assessment transparency practicability
  • 23.
    E-assessment authenticity consistency transparency practicability instruction- similarity assessment democratization cost alignment multiple complexity engagement efficiency indicators relevant adequacy visibility sustainability criteria competences- significance assessment impact alignment
  • 24.
    Authenticity • Similarity –competeces needed in real/professioanl life • Complexity – cognitive chalenge • Adequacy – adequate performing conditions • Significance – value for students, instructors and employers
  • 25.
    Consistency This dimension emergesas an answer to the traditional demands for validity and reliability, associated with psychometric indicators. • Instruction-assessment alignment • Relevant criteria • Competences-assessment alignment • Multiple indicators – assessment methods, contexts and assessors
  • 26.
    Transparency • Democratization –availability and participation • Engagement – participation in the definition of the learning goals • Visibility – present/share processes and/or products • Impact – effect on the learning process and on course design
  • 27.
    Practicability • Cost –time, digital resources, training, … • Efficiency • Sustainability – implement and sustain the proposed assessment design, taking into account the learner profiles and the contextual constraints, both for the organizations and for the assessors
  • 28.
    Discussion These dimensions arearticulated, representing several degrees of reciprocal interdependence. The criteria, more than just illustrating the different features of each dimension, allow for the operational description of each criterion stage of implementation, and so contribute to the evaluation of the achieved assessment strategy quality level.
  • 29.
    Discussion What can bethe contributions of e-assessment for the assessment culture? How can e-assessment be used for internal improvement and external accreditation? From a research standpoint should all dimensions be regarded as equally important?
  • 30.
    Thank You! Gracias! Obrigado! I care so I share http://www.slideshare.net/luistinoca [email protected] @luistinoca