Tblt Presentation

John M. Norris University of Hawai´i Task-Based Language Teaching 2005

This Powerpoint presentation is copyrighted by John M. Norris © September 23, 2005 Citation information: Norris, J. M. (2005, September). The essential roles of assessment, measurement, and evaluation in task-based language teaching . Plenary address presented at the First International Conference on Task-Based Language Teaching, University of Leuven, Belgium (September 23, 2005).

0:00 9:00 3:00 6:00 12:00 21:00 15:00 18:00 What time is it in Leuven ? What time is it in Hawaii ? “ Here and Now” condition “ There and… ” condition Now

0:00 9:00 3:00 6:00 12:00 21:00 15:00 18:00 What is the time difference between Leuven & Hawaii? Leuven Hawaii = 12hrs Task condition for presentation = “ Here _and There_ and Now”

measurement, The essential roles of assessment, and evaluation in task-based language teaching

TBLT 2005: What’s at stake? ??? Language Education ??? Does it work? for whom, when, why? What’s it worth? to whom, when, why? How does it work? for whom, when, why? Can it be improved? by whom, when, why?

TBLT 2005: What’s at stake? … another L2 teaching method bandwagon… The critics say… … cognitive-interactionist SLA hegemony… … won’t work with _____ learners… … not possible in _____ program settings… … too much task, not enough language… … not happening in practice… T B L T

Setting the record straight: TBLT at work TARGET LANGUAGES English Dutch French Czech Chinese Arabic Ukranian Japanese Russian Spanish Korean German

Setting the record straight: TBLT at work PROGRAM TYPES Second Language Foreign Language Vocational Education General Education Local Innovation Small Classes National Ed. Policy Special Purposes Adult Child University Requirement Large Classes Beginning Learners Advanced Learners Face-to- Face Online

Setting the record straight: TBLT foundations “ All genuine education terminates in discipline, but it proceeds by engaging the mind in activities worthwhile for their own sake.” Dewey (1933), pp. 86-87

Setting the record straight: TBLT foundations “ This discussion rejects the doctrine that students should first learn passively, and then, having learned, should apply knowledge. […] In fact, the applications are part of the knowledge. For the very meaning of things known is wrapped up in their relationships beyond themselves. Thus, unapplied knowledge is knowledge shorn of its meaning.” Whitehead, 1947, pp. 218-219

Setting the record straight: TBLT foundations “ For starters, this means having students do tasks, or at least meaningful simulations, that experts do in the various disciplines. Second, it means teaching them to think in ways that experts do when they perform these tasks.” Sternberg (2003), p. 5

Setting the record straight: Other task-based domains Medical school Nursing Curriculum by design Successful intelligence Environmental studies Social work Task-Based Practice Music education

Setting the record straight: So what is TBLT? Task-Based Language Teaching Rationales and Principles Education Philosophy Psychology SLA Language teaching Sociopolitical values L2 Education Programs Learner needs Assessment Curriculum Instruction Teacher development Evaluation Materials

The potential of task-based practice: One example William Reid Reid. W. J. (1992). Task strategies: An empirical approach to social work practice . New York: Columbia University Press. Social work problems : truancy, aging, delinquency, substance abuse, family health

Task-Centered Practice Empirical Justification Empirical Implementation Theory Learning, behavior, cognition Research Psychosocial change, Problem solving Evaluation Case work practice, Intervention outcomes Needs Identify, agree, prioritize Action Plan, rehearse, attempt, feedback Assessment Task accomplishment, Problem change Development Practitioner-friendly methods Improvement Field-tested findings The potential of task-based practice: One example

The potential of task-based practice: One example “ Assessment, process, and outcome data are systematically collected in each case by means of structured recording guidelines. Continuing research is used to improve the model.” (Reid, 1992, p. 132). Development Improvement Field-testing

“ [Reid] was the single most significant figure in social work research” Shaw (2004) p. 113 The potential of task-based practice: One example

Task-Based Language Teaching: A(nother) method for language teaching. The potential of task-based practice: TBLT

Task-Based Language Teaching: A(nother) method for language education practice. The potential of task-based practice: TBLT

Task-Based Language Teaching: An empirical approach to language education practice. The potential of task-based practice: TBLT Assessment Measurement Evaluation

The Role of Measurement in TBLT Interpreting research constructs… … to understand task-based learning

Measurement in TBLT: Responding to research purposes What is measurement? … trustworthy definition, elicitation, observation, analysis, and interpretation of research constructs…

Measurement in TBLT: Responding to research purposes Measurement should… Illuminate task-based learning… L2 dvlpmt. in task, over tasks Noticing, integrating, automatizing Proficiency, motivation, aptitude Timing, sequencing, embedding TBI FonF, modeling, feedback Reliable reporting of TBL variables Synthesis of how TBL works Define clear learning outcomes Identify specific learning processes Clarify learner individual differences Indicate effects of context features Articulate instructional processes Enable communication of findings Foster accumulation of knowledge

Measurement in TBLT: Responding to research purposes “ Perhaps the best recommendation would be that, with research in this area still developing techniques and understandings, the role of A/L work will be to offer suggestions for relevant variables and the standardization of operationalizations of variables…” Tavakoli & Skehan, 2005, p. 273

In short… TBLT Theory/research Constructs Learner variables Setting variables Learning process Instruction process Learning outcomes Practice Empirical Measurement Measurement in TBLT: Responding to research purposes How does TBLT work, for whom, when, why?

Measurement in TBLT: Problems of practice Interpreting research constructs No construct definition Little standardization Missing measures No attention to reliability Impoverished analyses Insufficient reporting Understanding task-based learning Inaccurate interpretation of task-related variables 2. Massive over-generalization of measurement ‘findings’ 3. Minimal meaning for scope, scale, setting of task-based practice

Task influence on L2 performance (adapted from: Skehan, 2001) Measurement in TBLT: Problems of practice 0 + 0 Transformations (planning) 0 + 0 Outcomes + 0 + Structure - + + Dialogic v. Monologic + 0 + Familiarity Fluency Complexity Accuracy Task characteristic

“… giving learners the opportunity to plan before a task is done consistently produces greater complexity of language…” (Skehan, 2003, pp. 395-96) Measurement in TBLT: Problems of practice “ To sum up, both modality and group condition affected complexity at the level of syntax…” (Ellis & Yuan, 2005, p. 184)

Measurement in TBLT: Problems of practice Task condition Planning L2 performance outcome Complexification Modality? writing/speaking Complexity? Subordination Complexity? Other measures Scope of measure? duration/timing Planning? Focus, structure, pressure Learner proficiency? “ L-H”, “N-I-A”, “450-600” Task features? Type, difficulty, form, mono-dialogic, interlocutors ?

Measurement in TBLT: Problems of practice Task condition + strategic planning L2 performance outcome Complexification Complexity? Clauses per C-Unit Learner proficiency? TOEFL: 427-670 Elder & Iwashita (2005) X X X

Measurement in TBLT: Problems of practice Task condition + strategic planning L2 performance outcome Complexification Complexity? Words/T-Unit Clauses per T-Unit Total Subordination Learner proficiency? TOEFL: 420-480 TOEFL: 510-580 TOEFL: 560-610 Kawauichi (2005)

Measurement in TBLT: An empirical approach to practice In what ways are what kinds of planning associated with what kinds of L2 performance for what kinds of L2 users under what conditions…and why?

Measurement in TBLT: An empirical approach to practice “ ... descriptive longitudinal studies which establish norms of performance for particular processes and phenomena in L2 acquisition…” Norris & Ortega, 2003, p. 738 “… attempting to ‘measure’ acquisition without a sound descriptive basis for meaningful differences in particular acquisition-related behaviors would be akin to timing a runner’s performance over a mile without knowing how many times around the track a mile happens to be…”

Measurement in TBLT: An empirical approach to practice Georgetown University German Department: “ Syntactic complexity development in task-based writing” Longitudinal (23) + Cross-sectional (329) Extended and extemporaneous writing tasks Multiple measures, including Multiple measures of SC End-of-curricular levels Across 4 years

Syntactic complexity in curriculum-embedded writing assessment: LONGITUDINAL Global compexification   Phrasal elaboration Subordination 

Syntactic complexity in curriculum-embedded writing assessment: CROSS-SECTIONAL Global compexification  Subordination   Phrasal elaboration

Measurement in TBLT: An empirical approach to practice Implications for measuring syntactic complexity in task-based research Syntactic complexity is multi-faceted; so, measures are multivariate Complexification is a long-term developmental process, best understood in situ At different learner proficiency/curricular levels, complexification occurs in distinct ways Synthesis of research findings on syntactic complexity See: Ortega (2000, 2003)

In short… TBLT in situ Theory/research Constructs Learner variables Setting variables Learning process Instruction process Learning outcomes Practice Empirical Measurement Measurement in TBLT: Responding to research purposes How does TBLT work, for whom, when, why?

The Role of Assessment in TBLT Informing decisions and actions… … to enable task-based learning

Assessment in TBLT: Responding to educative purposes What is assessment? … the use of tests and other procedures for gathering trustworthy information about learners and making decisions and taking actions on the basis of that information, in support of teaching and learning…

Assessment in TBLT: Responding to educative purposes Ensure learner-relevant instruction Support classroom teaching Motivate and focus learners Target clear learning outcomes Identify learning gaps Transform curriculum Counter imposed testing practice Foster range of learning outcomes Certify valued accomplishments Educative assessment should… Enable task-based learning… Needs analysis, intake procedures Feedback on TB pedagogy effects Interesting, meaningful tests Genre, task performance expectation Feedback on task performance Process-product articulation Internal TBA v. external mandate Language + content + task abilities Profile individual abilities in L2 use

Assessment in TBLT: Responding to educative purposes “ In other words, by focusing on task-based assessment—here writing—faculty faced their own assumptions about long-term development in instructed L2 learning and appropriate learning outcomes at particular stages in that sequence.” Byrnes (2002), p. 428

Assessment in TBLT: Responding to educative purposes “… the quality of task completion gave teachers a good understanding of what students did and did not know—often, a better understanding than most tests could provide.” Leaver & Kaplan (2004), p. 52

Assessment in TBLT: Responding to educative purposes “ Focusing on what one can do instead of the competencies one has yields a more direct link with language performance that can be observed in the real world.” Gysen & van Avermaet (2005), p. 51

Assessment in TBLT: Responding to educative purposes In short… TBLT Implementation Curriculum Instruction Materials Learners Teachers Outcomes ETC. Practice Empirical Assessment Does TBLT work, and how can we enable it?

Assessment in TBLT: Problems of practice Tensions in “how to” assess… L2 competence v. task ability Extrapolation v. prediction Construct v. consequence Control v. understand Prescribe v. respond Enabling task-based learning Irrelevant for many educative uses in classrooms/programs Ignorant of interconnections between language and task/ genre/content Irresponsible to alignment of assessment with C&I Incompatible with values and goals of task-based education Dichotomy v. multiplicity

Assessment in TBLT: Problems of practice “… the inferences we want to make are about underlying ‘language ability’ or ‘capacity for language use’ or ‘ability for use’…” Bachman (2002), p. 454 “… assessments can have many different functions. What is appropriate for one assessment purpose may be inappropriate for another…” Council of Europe (2001), p. 180

Advanced intercultural evaluation: An online French course Starkey & Osler (2001) Common European Framework: Language+ Pluriculturalism, Anti-racism Online advanced French learning objectives : Language development Social/cultural awareness Intercultural evaluation Online advanced French teacher assessments (writing feedback): Language development Social/cultural awareness Intercultural evaluation Assessment in TBLT: Problems of practice

Assessment in TBLT: Problems of practice “… is not for the test designer to decide or prescribe. Rather, it should be the test users and stakeholders who inform exactly what needs to be done in the name of language assessment, and it is up to us to enable them to do so.” Norris (2002), p. 344

“ 1. The intended use(s) for task-based assessment within the language programme must be specified…” Long & Norris (2000), p. 600 Assessment in TBLT: An empirical approach to practice

Specifying intended assessment use From: Norris (2000)

Assessment in TBLT: Examples in practice, Specification process Specifying intended uses for assessment in the Georgetown University German Department Clarifying the purposes for assessment, from classroom- to program-level, by… Investigating existing assessment practices (via interviews, surveys, observations) Establishing consensus on the roles to be played by assessment (via meetings, workshops, negotiation among intended users/stakeholders)

Assessment in TBLT: Examples in practice, Specification products Intended assessment use specification Assessment policies Use specs for all tests Priorities for Development Assessment Constraints Quizzes Exams Performance Internal External Time Money Ethics Training Unknowns (Urgent!) Placement Learning outcomes External Accountability Developing Implementing Articulating Communicating Evaluating

Assessment in TBLT: Examples in practice, Use specification Intended use Who : Teachers, curriculum developers, researchers What : Learner written task/genre performance abilities at the end of each curricular level Why : Understanding student achievement of targeted task abilities for improvement of C&I Impact : Feasible (revised?) curricular expectations supported by effective (improved?) pedagogy Constraints Explicitness of curricular expectations Availability/agreement of ‘prototypical’ performance tasks by curricular level Competing uses for assessment (feedback) End-of-semester timing Learner investment, understanding of task expectations Task versus content Task-based writing assessment

Task Content L conventions L focus Assignment 1 Assignment 2 Process writing 4-6 tasks per semester Assignment 3 Assignment 4 Prototypical Performance Writing Task: Level I Level II Level III Level IV Curricular level/genre learning expectations Feedback loop Explicit criteria Assessment in TBLT: Examples in practice, Instruments

Assessment in TBLT: Examples in practice, Impact Curriculum : much greater specificity in learning objectives and articulation, overall and for each level/semester Courses : revised/improved pedagogic focus, consistency Teachers : greater awareness of learner abilities and learning trajectories, enhanced clarity of grading/feedback purpose Learners : enhanced understanding of learning targets & expectations, supportive role of assessment Assessment : clearer articulation with C&I, development of similar assessment systems (e.g., speaking tasks) Program : explicit statement of valued learning outcomes in the form of genres, tasks, content, and language

Assessment in TBLT: Responding to educative purposes In short… TBLT in situ Implementation Curriculum Instruction Materials Learners Teachers Outcomes ETC. Practice Empirical Assessment Does TBLT work, and how can we enable it?

The Role of Evaluation in TBLT Articulating empirical practice… … to improve task-based education … with practical value

Evaluation in TBLT: Responding to program purposes What is evaluation? … gathering trustworthy information about program elements in order to understand, demonstrate, and improve program value…

In short… TBLT in situ Society Values Needs Resources Stakeholders Funders Educators ETC. Practice Empirical Evaluation What is TBLT worth, and how can we improve it? Evaluation in TBLT: Responding to program purposes

Evaluation in TBLT: Responding to program purposes “ Given the checkered history of prescriptions for language teaching, the likelihood that all 10 TBLT MPs will turn out to have been well founded is minimal. Thus, designers … who draw upon these or other sources should build careful evaluation components into their delivery systems.” Doughty & Long (2003), p. 68 “ In contrast to the extensive research into tasks…there have been no more than a handful of evaluations of task-based teaching” Ellis (2003), p. 323

Evaluation in TBLT: Problems of practice Nunan (1989) Crookes & Gass (1993) Willis (1996) Skehan (1998) Lee (2000) Bygate, Skehan, & Swain (2001) Ellis (2003) Leaver & Willis (2004) Nunan (2005) Overview books on TBLT… … with chapters on program evaluation… Ellis (2003)

Evaluation in TBLT: Problems of practice Bangalore Project Evaluation: Task-base program Large-scale effort External evaluator (see Beretta, 1990a, 1990b; Beretta & Davies, 1985) Crisis of conscience Evaluation practice? Evaluation purpose? Evaluation impact? Theory v. practice???

Evaluation in TBLT: An empirical approach to practice Patton (1996) Utilization-focused Evaluation Intended uses Intended users Questions + Methods Understand Improve Educate Demonstrate worth Hold accountable Test theory Empower Teachers Administrators Curriculum writers Learners Parents/public Researchers Funders Values clarification Implementation Process-product Experimentation Synthesis

Evaluation in TBLT: Examples in practice Norris (2004) Evaluation of a US college German FL assessment program Van den Branden (to appear) Evaluation of task-based teacher development in Flanders

Assessment, measurement, and Evaluation in TBLT: Essential roles TBLT in situ Measurement Theory, Research Assessment Imple-mentation Evaluation Society, Value EMPIRICAL PRACTICE

http://www2.hawaii.edu/~jnorris [email_address] See link at: www.tblt.org For more information: Contact me

See you in Hawaii for TBLT 2007!

http://www2.hawaii.edu/~jnorris [email_address] See link at: www.tblt.org

REFERENCES Please see the following web site for a complete pdf file of references and resources from this presentation: http://www2.hawaii.edu/~jnorris/TBLT%20references.pdf

Tblt Presentation

More Related Content

What's hot

Viewers also liked

Similar to Tblt Presentation

More from Dr. Cupid Lucid

Recently uploaded

In this document

Tblt Presentation