1

Task-based Language Assessment(TBLA)

A presentation byUlrike Schuh-Fricke & Wolfgang Stadler

2

Group Presentation (7.2)

• Consists of– A PowerPoint in which we, among others,

• Explain the thinking behind our test design• Say what tasks the students will have to do• Present and describe our assessment instrument• Explain why it is task-based and how it relates to the textbook• Address concerns raised in the literature on TBLA• Reflect on how task-based tests relate to the issues of

generalisability, task difficulty, and task variability

– A task-based language test for a group of students who have done unit 1 of the course book Widgets

3

Structure

• Background (Slide 4)• Task-based language teaching (Slides 5–7) • Course book Widgets, Unit 1 (Slides 8, 9)• Tasks to do on Threshold Level (=B1, CEF) (Slides 10, 11)• Concept behind performance tasks (Slide 12)• Tasks achieved after Unit 1, Widgets (Slide 13)• Task-based language testing (Slides 14–16) • Performance (Assessment) Criteria (Slides 17–21)• The Test (Slides 22–25)• Critical reflection (Slide 26)• Literature (Slide 27)

4

Background• Communicative Language Ability (CLA): Focus on use of language

in various situations (see David’s comment in 7.1)• Action-oriented approach: users and learners of a language are

primarily ‘social agents’, i.e. members of society who have tasks to accomplish in a given set of circumstances, in a specific environment and within a particular field of action. (CEFR, ch. 2.1)

• Focus on the agents’ use of strategies linked to their competences and how they perceive or imagine the situation to be mastered with their language skills

• In accordance with the action-oriented approach, it is assumed that the language learner is in the process of becoming a plurilingual language user developing interculturality (ibid.)

5

Task-based language teaching

• Tasks are a feature of everyday life in the personal, public, educational or occupational domains.

• A task may be quite simple or extremely complex and it may involve a greater or lesser number of steps or embedded sub-tasks.

• Tasks can be extremely varied in nature, and may involve language activities to a greater or lesser extent, for example: creative, skills based, problem solving, routine transactions, interpreting a role in a play, taking part in a discussion, giving a presentation, planning a course of action, reading and replying to (an e-mail) message, etc.

• Task accomplishment involves the strategic activation of specific competences in order to carry out a set of purposeful actions in a particular domain with a clearly defined goal and a specific outcome. (CEFR, ch. 7.1)

6

What is a language task? (Definition)

• “A task [is] any activity in which a person engages, given an appropriate setting, in order to achieve a specifiable class of objectives.“ (J.B. Carroll, 1993)

• According to CEFR, a task is defined as any purposeful action considered by an individual as necessary in order to achieve a given result in the context of a problem to be solved, an obligation to fulfil or an objective to be achieved (CEFR, ch. 2.1)

• Tasks– are closely associated with specific situations– are goal-oriented– involve active participation of the language users

7

Developing a task (Checklist)

• According to Bachman/Palmer’s framework (1996), five aspects need to be considered when developing a task (checklist):– Setting (physical characteristics, participants, time)– Test rubric (instructions, structure, time allotment,

scoring method)– Input (format, language)– Expected response (format, language)– Relationship between input and response (reactivity,

scope of relationship, directness of relationship)

8

Introducing Widgets

• Widgets – designed for an ideal class of 16-24 students – creates a “real-life” English-speaking environment in the classroom and focuses mainly on listening and speaking skills.

• Students imagine that they are new employees at an international company, Widgets Inc., where they must work together in small groups to perform various practical, yet fun, tasks.

http://www.widgets-inc.com/index.php

9

Coursebook Widgets, Unit 1

In Unit 1 students

– learn about the course concept and objectives– learn about Widgets Inc. (the company)– explore customs about business cards, name tags and

handshaking– meet and greet each other in a casual business setting– learn about Victoria Vanderhoff, the company’s president– familiarize themselves with appropriate conversation

topics

10

Tasks to do on Threshold Level(=B1, CEFR)

• Personal identification– Learners can say who they are, spell their name,

state their address, give their telephone number, say when and where they were born, state their age, sex, state whether they are married or not, state their nationality, say where they are from, what they do for a living, describe their family, state their religion, if any, state their likes and dislikes, say what other people are like; elicit/understand similar information from others.

11

Tasks to do on Threshold Level(=B1, CEFR), cont.

• Communication at work– Seek work permits etc. as required;– Enquire (e.g. from employment agencies) about the nature, availability and

conditions of employment (e.g. job description, pay, laws of work, free time and holidays, length of notice);

– read employment advertisements;– Write letters of application and attend interviews giving written or spoken

information about own personal data, qualifications and experience and answer questions about them;

– Understand and follow joining procedures;– Understand and ask questions concerning the tasks to be performed on starting

work;– Understand safety and security regulations and instructions;– Report an accident and make an insurance claim;– Make use of welfare facilities;– Communicate appropriately with superiors, colleagues and subordinates;

12

Concept behind performance tasks

• Students perform authentic, real-life tasks, using the productive skills of speaking and/or writing but also combining skills.

• Teachers elicit authentic communication and may predict students’ future performances in target language use situations.

• If performance assessment is closely linked to the curriculum, a strong positive washback effect may be achieved.

13

What students can do afterUnit 1, Widgets

– Read and understand a letter of employment– View an orientation video of the company and take notes– Distinguish between business cards and name tags and see their purpose– Make their own business cards and name tags– Introduce themselves and shake hands (culture specific elements)– Find missing information (in a CV)– Distinguish between appropriate and inappropriate small talk questions and how

difficult that can be depending on the speakers and the situation (sociolinguistic appropriateness)

– Listen in on a private conversation and enter a dialogue– Lead a more private conversation (pragmatic competence: how to begin, interrupt, keep

up, end a conversation)– Describe, explain and sell a product– Listen to a presentation and find out how a department works– Describe one’s character

14

Task-based language testing

• Characteristics of TBLAPerformance by the candidateAuthenticity of taskSet of criteria to judge the

performance

15

Task-based assessment formsStrong assessment Weak assessment

Task-based Performance-basedFocus on task achievement Focus on elicited languageTask fulfilment ability Language ability

16

Difficulties

• Imitating diverse real-life situations• Selection of tasks• Complex interactions• Reduced generalisability• Decision on successful achievement of task• Integrated tasks• Task difficulty• Task characteristics’ impact on measures of fluency,

complexity and accuracy• Rater training

17

Performance (Assessment) Criteria

• A performance test is a ”test in which the ability of candidates to perform particular tasks, usually associated with job or study requirements, is assessed“ (Davies et al., 1999)

• “< … > the target language is used by the learner for a communicative purpose (goal) in order to achieve an outcome” (Willis, 1996)

18

Our selected set of criteria

• We opt for two steps of assessing tasks• Although– meaning is primary– learners should not be too restricted in their use of

language forms– tasks should bear a relationship to real-world activities– the priority is on achieving the goal of the task– tasks are assessed based on their outcome

(see http://www.widgets-inc.com/teacher/tblt.php)

• we want to assess language as well.

19

Two steps of assessmentStep 1 Was the task appropriately

completed? Would the outcome be reasonably recognizable by an “average” native speaker of English as an example of its “type”? Yes (pass) No (fail) ✓ ✗

Step 2 If “yes”, how good was it? The descriptors below are taken from Widgets teacher‘s resourse centre: http://www.widgets-inc.com/home.php

20

Descriptors for a Pass10 No grammar mistakes worth mentioning. Vocabulary use was very appropriate.

Pronunciation was exceptionally clear. Speech was remarkably smooth and fluent. Gestures, facial expressions and manner were always appropriate and natural.

9 Some small grammar, vocabulary or pronunciation mistakes. However, speech was still very smooth and easy to understand. Gestures, facial expressions and manner were appropriate and natural.

8 Some noticeable grammar, vocabulary or pronunciation difficulties. However, overall speech was easy to follow and understand. Gestures, facial expressions and manner were generally appropriate.

7 Occasional serious difficulties with grammar, vocabulary or pronunciation. Speech was not always smooth and clear, but quite understandable. Did not revert to first language. Gestures, facial expressions or manner may have been somewhat distracting.

6 Serious difficulties with basic grammar, vocabulary or pronunciation. Speech was not always clear. Required some support from the listener. Reverted to first language on occasion. Gestures, facial expressions or manner were often distracting; nevertheless, generally understandable.

21

Descriptors for a Fail5 Serious difficulties with basic grammar, vocabulary or pronunciation. Required

considerable support and patience from the listener. Often reverted to first language; nevertheless, short sections of the speech could sometimes be understandable.

4 Serious difficulties with basic grammar, vocabulary or pronunciation. Required considerable support and patience from the listener. Often reverted to first language. Understandable only to a very sympathetic listener familiar with the student's first language, such as a teacher.

3 Did not display an ability to use basic grammar structures. Spoke in two- or three-word utterances using basic, but appropriate vocabulary. Used other means to support speech, including relying very heavily on first language. Difficult to understand even for a very sympathetic listener; nevertheless, displayed some noteworthy quality, such as an understanding of storytelling conventions.

2 Did not display an ability to use basic grammar structures. Spoke in two- or three-word utterances using only basic vocabulary. Used other means to support speech, including relying very heavily on first language. Extremely difficult to understand, even for a very sympathetic listener.

1 Could not be understood beyond basic set expressions such as “How are you?” Made only single-word utterances, if any at all.

22

The Test

• See attachment (PDF)– It includes the task-based test, the listening tasks

(recorded by Ulrike) and the Widgets rating scales.– See the links in the description

23

Structure of the testTask#

Skill Strategy Expected outcome Time

1 Listening+ taking notes

Listening to locate specific details Can clearly understand co-workers and supervisors; Can take short notes in English (SAQ)

5

2 Listening + taking notes

Listening to understand main ideas and supporting details

Can clearly understand co-workers and supervisors; Can take short notes in English

10

3 Reading Read and collate short information [reading strategy: careful reading] Problem solving (non linguistic ability)

Find best seating decision 5

4 Speaking Monologue first: explain seating order Dialogue with superior

Can speak smoothly without long pauses; Can express opinion and justify arrangements; Can convince superior; Can use appropriate register and style

5

5 Speaking Dialogue with peer Can introduce themselves and use simple greetings; Can ask and answer personal question about one’s own and someone else’s life, about likes and dislikes; Can participate in (short) free topic conversations; Can speak smoothly without long pauses; Can express opinion in small group; Can use appropriate register and style

7

24

Test and tasks

• The test is– a “task-based achievement test”– based on similar tasks in the course book

• Tasks– are authentic and– goal oriented– they allow for meaningful interaction between• test taker and task• test taker and interlocutor (simulation)• test takers (role play)

25

Discussion of assessment criteriaand test design

• The main assumption of the rating process and the scale is:– “… if an appropriate outcome was achieved, then

we already know that the student’s pronunciation, vocabulary and grammar are at an “acceptable” level for the task, communicatively speaking. They would not have managed to complete the task otherwise.”

• Problems: – What is meant by “appropriate outcome”?– Can “task appropriacy” be measured and rated?– If so, how? (see Ben’s thoughts on this in 7.1)

26

Critical reflection• Are learners aware of what they are learning: general English or English for specific purposes?• Do they have to be aware of this?• Are learners able to transfer their competence and use of skills and strategies to new tasks

and real-life situations?• How will students who are used to task-based language learning perform in non task-based

(proficiency) tests?• Real-life tasks seem rather specific, whereas the language used can be fairly general or

practical.• Can task difficulty drastically be altered by changing task specifics (e.g. turn a face-to-face

dialogue into a telephone conversation; change role of interlocutor) and how does this affect the outcome?

• Who chooses tasks and who decides whether the tasks have been successfully solved? Teachers or professionals?

• Rater training seems vital.• Content validity: How to guarantee that tasks are representative of the TLU domain?• Face validity and predictive validity seem to be crucial in TBLA (see also Ben in 7.1) .• How can test developers tackle construct validity and its effect on rating scales?• What about task-based interculturality in a multi-migrant classroom? (cf. Kate in 7.1)

27

Literature• Bachman, L.F. & Palmer, A.S. (1996). Language testing in practice. Oxford: Oxford

University Press (Chapter 3: Describing tasks: Language use in language tests)• Brown, J. D., & Hudson, T. (1998). The alternatives in language assessment. TESOL

Quarterly 32 (4), pp. 653-675. • Colprin, M. & Gysen, S (2006). Developing and introducing task-based language tests. In

K. Van den Branden (Ed.), Task-based language education – From theory to practice, Cambridge, pp. 152 -153.

• Common European Framework of Reference (CEFR).• Eisenmann, M. (2009). Förderung der Sprachproduktion. In Praxis Fremd-

sprachenunterricht 05/09, pp. 4-8.• Nunan, D. (2004). Task-based language teaching, Cambridge, pp.145-146.• Wigglesworth, G. (2008). Task and performance based assessment. In S. Shohamy & N.

H. Hornberger (Eds.), Encyclopedia of language and education (2nd Ed.), Volume 7: Language testing and assessment, pp.111-122.

• Widgets. A task-based course in practical English (Coursebook)• Willis, J. (1996): A framework for task-based learning. Harlow: Longman.

28

© 2009 W. STADLER, U. SCHUH-FRICKE

Thank you for your attention