Tsl641 principles for call evaluation v 2
- 1. PRINCIPLES OF CALL EVALUATION TSL 641 Chapelle, C. (2001). Computer Applications in SLA. UK: Cambridge 1
- 2. PRINCIPLES FOR CALL EVALUATION2 Three needs must be addressed when evaluating CALL materials: Materials include Courseware – CD ROMs etc Websites – ESL website Mobile Apps - Tasks & – assignments including use of WWW, CMC Activities – Chat, Blog, Podcast, etc
- 3. PRINCIPLES FOR CALL EVALUATION3 Evaluation criteria should incorporate findings and theory-based speculation about ideal conditions for SLA Findings from research on SLA Theories of SLA
- 4. PRINCIPLES FOR CALL EVALUATION Criteria should be accompanied by guidance as to how they should be used i.e. a theory of evaluation needs to be articulated 4
- 5. PRINCIPLES FOR CALL EVALUATION5 Both criteria and theory need to apply not only to software, but also to the task that the teacher plans and that the learner carries out
- 6. PRINCIPLES FOR CALL EVALUATION6 According to Chapelle (2001) 5 principles of CALL evaluation criteria Evaluation of CALL is a situation-specific argument. CALL should be evaluated through two perspectives: judgmental analysis of software and planned tasks, and empirical analysis of learners’ performance. Criteria for CALL task quality should come from theory and research on instructed SLA. Criteria should be applied in view of the purpose of the task. Language learning potential should be the central criterion (most critical) in evaluation of CALL.
- 7. CALL EVALUATION: PRINCIPLE 17 Evaluation of CALL is a situation-specific argument. The outcome of task evaluation for any L2 tasks including those for CALL cannot be a decision about effectiveness. Instead, an evaluation has to result in an argument indicating in what ways a particular CALL task is appropriate for particular learners at a given time, the use at a particular setting Implication: CALL developers need to be familiar with criteria for evaluation which should be applied relative to a particular context.
- 8. CALL EVALUATION: PRINCIPLE 28 CALL should be evaluated through two perspectives: judgmental analysis of software and planned tasks, and empirical analysis of learners’ performance. Judgmental analyses - examine characteristics of the software and task in terms of criteria drawn from research on SLA. Empirical analyses - address the same criteria but through data gathered to reveal the details of CALL use and learning outcomes. Implication: CALL developers need to be familiar with criteria for evaluation which should be applied relative to a particular context.
- 9. LEVELS OF ANALYSIS FOR CALL EVALUATION 9 Level of Analysis Object of Evaluation Example Question Method of Evaluation 1 CALL software Does the software provide learners the opportunity for interactional modifications to negotiate meaning? Judgmental 2 Teacher-planned CALL activities Does the CALL activity designed by the teacher provide learners the opportunity to modify interaction for negotiation of meaning? Judgmental 3 Learners’ performance during CALL activities Do learners actually interact and negotiate meaning while they are working in a chat room? Empirical
- 10. CALL EVALUATION: PRINCIPLE 310 Criteria for CALL task quality should come from theory and research on instructed SLA. Language learning potential Learner fit Meaning focus Authenticity Positive impact Practicality Implication: CALL evaluators need to keep up with and make links to research on instructed SLA.
- 11. CRITERIA FOR CALL TASK APPROPRIATENESS 11 Language Learning Potential •The extent to which the activity can be considered to be a language learning activity rather than simply an opportunity for language use. •The degree of opportunity present for beneficial focus on form. Learner Fit •The individual differences in linguistic ability level and non-linguistic characteristics. •The amount of opportunity for engagement with language under appropriate conditions given learner characteristics. Meaning Focus •The learner’s primary attention is directed toward the meaning of the language that is required to accomplish the task. •The extent to which learners’ attention is directed toward the meaning of the language. Authenticity •The degree of correspondence between an L2 learning task and tasks that the learner is likely to encounter outside the classroom •The degree of correspondence between the CALL activity and target language activities of interest to learners out of the classroom. Positive Impact •Its effects beyond its language learning potential. E.g. – help build metacognitive strategies, willingness to seek out opportunities to use TL. •The positive effects of the CALL activity on those who participate in it. Practicality •How easy it is for the learners and teachers to implement a CALL task within the particular constraints of a class or language program. •The adequacy of resources to support the use of the CALL activity.
- 12. CRITERIA FOR CALL TASK APPROPRIATENESS 12 Language Learning Potential • They learn e.g. how to use modals Learner Fit • Task appropriate at their level, taking into also consideration age, learning style, willingness to communicate Meaning Focus • Meaning of language to accomplish a task. e.g. using language to make decision, exchanging information to accomplish a goal Authenticity • What they would actually encounter outside of the classroom, in the real world Positive Impact • E.g. Seek opportunities to use what they have learned in class outside. Practicality • Easy to implement CALL despite lacking facilities
- 13. CALL EVALUATION: PRINCIPLE 413 Criteria should be applied in view of the purpose of the task. Implication: CALL tasks should have a clearly articulated purpose.
- 14. CALL EVALUATION: PRINCIPLE 514 Language learning potential should be the central criterion (most critical) in evaluation of CALL. Implication: Language learning potential should be one aspect of the purpose of CALL tasks.
- 15. JUDGEMENTAL ANALYSIS OF CALL APPROPRIATENESS15 Qualities Questions Language learning potential Do task conditions present sufficient opportunity for beneficial focus on form? Learner fit Is the difficulty level of the targeted linguistics forms appropriate for the learners to increase their language ability? Is the task appropriate for learners with the characteristics of the intended learners? Meaning focus Is learner’s attention directed primarily toward the meaning of the language? Authenticity Is there a strong correspondence between the CALL task & second language tasks of interest to learners outside the classroom? Will learners be able to see the connection between the CALL task and tasks outside the classroom? Impact Will learners learn more about the target language and about strategies for language learning through the use of the tasks. Will instructors observe sound second language pedagogical practices by using the task? Will both learners and teachers have a positive learning experience with technology through the use of the task? Practicality Are hardware, software, and personnel resources sufficient to allow the CALL task to succeed?
- 16. Empirical Analysis of CALL Appropriateness16 Qualities Questions Language learning potential What evidence suggests that the learner has acquired the target forms that were the focused on during the CALL tasks/ What evidence indicates that learners focused on form during the CALL task? Learner fit What evidence suggests that the targeted linguistic forms are at an appropriate level of difficulty for the learners? What evidence suggests that the task is appropriate to learners’ individual characteristics (age, learning styles, computer experience) Meaning focus What evidence suggests that the learner’s construction of linguistic meaning aids language learning? What evidence indicates that the learners use the language during the task for constructing and interpreting meaning? Authenticity What evidence suggests that learners’ performance in the CALL tasks corresponds to what one would expect to see outside the CALL tasks? What evidence suggests that the learners see the connection between CALL task and tasks outside the classroom? Impact What evidence suggests that the learners learn more about the target language and about strategies for language learning through the use of the tasks? Practicality What evidence suggests that hardware, software and personnel resources prove to be sufficient to allow the CALL task to succeed?
- 17. THE END17