Testing Testing Vocabulary Vocabulary

Jonathan MagdalenaJonathan Magdalena

What What are we trying to are we trying to MEASUREMEASURE??

“ “A A wordword is is a a microcosmmicrocosm

of of human human consciousnessconsciousness””

(Vygotsky)(Vygotsky)

What What is is vocabularyvocabulary??

LanguageLanguage emerges first as emerges first as wordswords, both historically, and in terms of the way each , both historically, and in terms of the way each of us learned our first and any subsequent languages. The coining of new words of us learned our first and any subsequent languages. The coining of new words never stops. Nor does the acquisition of words. Even in our first language, we are never stops. Nor does the acquisition of words. Even in our first language, we are continually learning new words, and learning new meanings for old words. continually learning new words, and learning new meanings for old words.

Can you think of an example?Can you think of an example?   

How many How many words words can you findcan you find??I like looking for bits and pieces like old second-hand record players and doing them up to look like new.I like looking for bits and pieces like old second-hand record players and doing them up to look like new.

andand is repeated once; is repeated once; like like three timesthree times The first The first likelike is a verb; the other two are prepositions is a verb; the other two are prepositions lookinglooking and and looklook: do they count as two different words or different forms of the same word? : do they count as two different words or different forms of the same word? second-handsecond-hand: : two words or just one? two words or just one? What about What about record playerrecord player?? And And bits and piecesbits and pieces? ? Is this a self-contained unit or three separate words? Is this a self-contained unit or three separate words? Are Are looking looking and and looking for looking for the samethe same?? Doing up Doing up means means renovaterenovate, but it is separated by the word , but it is separated by the word themthem; is it one or two words then?; is it one or two words then?

What What isis aa word word thenthen??A word is a highly complex phenomenon.A word is a highly complex phenomenon.

Words have different functions. Words have different functions. The same word can have a variety of forms.The same word can have a variety of forms. Words can be added to, or combined, to form new words.Words can be added to, or combined, to form new words. Words can group together to form units that behave as if they were single words.Words can group together to form units that behave as if they were single words. Many words commonly co-occur with other words.Many words commonly co-occur with other words. Words may look and/or sound the same but have quite different meanings.Words may look and/or sound the same but have quite different meanings. One word may have a variety of overlapping meanings.One word may have a variety of overlapping meanings. Different words may share similar meanings, or may have opposite meanings.Different words may share similar meanings, or may have opposite meanings. Words can have the same or similar meanings but be used in different situations or for different effectsWords can have the same or similar meanings but be used in different situations or for different effects

What What doesdoes know aknow a word word meanmean??Knowing a word means:Knowing a word means:

having the ability to recognize it in its spoken and written formshaving the ability to recognize it in its spoken and written forms knowing its different meanings.knowing its different meanings. knowing its part of speech [eg. a noun, a verb]knowing its part of speech [eg. a noun, a verb] being able to pronounce it properlybeing able to pronounce it properly being able to use it correctly within a sentence in an appropriate grammatical formbeing able to use it correctly within a sentence in an appropriate grammatical form for technical words, recognizing it in contextfor technical words, recognizing it in context being able to recognize different types of English e.g boot/trunk, lift/elevator [British/American].being able to recognize different types of English e.g boot/trunk, lift/elevator [British/American].

How How areare wordswords learned? learned?“Without grammar very little can be conveyed, without vocabulary “Without grammar very little can be conveyed, without vocabulary

nothing can be conveyed.”nothing can be conveyed.”

What makes you What makes you swerve swerve your car?your car? The limo surges forward and starts to The limo surges forward and starts to swerveswerve wildly over the wildly over the

road.road. The bus driver The bus driver swerved swerved to avoid hitting the cycliststo avoid hitting the cyclists She is one of those rare politicians whom one can trust not to She is one of those rare politicians whom one can trust not to

swerve swerve from from policy and principle.policy and principle. Hitting the brakes would make the bikes Hitting the brakes would make the bikes swerveswerve more. more. The driver made no attempt to The driver made no attempt to swerve swerve out of their pathout of their path Nothing could Nothing could swerveswerve him him Last and surprisingly, for such a small town you can still get Last and surprisingly, for such a small town you can still get

your your swerve swerve on with the nightlife.on with the nightlife. He managed to pass the ball with a perfect body He managed to pass the ball with a perfect body swerve.swerve.

  

How How areare wordswords organized? organized? The mind seems to store words neither randomly or in the form of a The mind seems to store words neither randomly or in the form of a list, but in a highly organized and interconnected fashion — in what is list, but in a highly organized and interconnected fashion — in what is

often called the often called the mental lexicon.mental lexicon.

WhyWhy should we test should we test vocabularyvocabulary??

FeedbackFeedback

Backwash Backwash effecteffect

MotivationMotivation

RecyclingRecycling

HowHow should we test should we test vocabularyvocabulary??

Separated Separated from skills?from skills?

Discrete Discrete point point or integrative items?or integrative items?

Focus on Focus on lexislexis or contextualized meaning? or contextualized meaning?

What are the most common What are the most common test formats test formats for for testing vocabulary?testing vocabulary?

RecognitionRecognition Multiple choice items (Liu, 121)Multiple choice items (Liu, 121) Error-recognition items (Liu, 129)Error-recognition items (Liu, 129) Pairing and matching itemsPairing and matching items Gap-filling itemsGap-filling items Cloze/Modified Cloze (Hughes, 144)Cloze/Modified Cloze (Hughes, 144) Guessing meaning from contextGuessing meaning from context

WhenWhen should we go for should we go for vocabularyvocabulary recognitionrecognition??

When…When… More material needs to be covered. More material needs to be covered. You want to test different levels of You want to test different levels of

learning. learning. You have little time for scoring. You have little time for scoring. You are not interested in evaluating You are not interested in evaluating

how well a test taker can formulate a how well a test taker can formulate a correct answer. correct answer.

You have a large number of test takers. You have a large number of test takers.

What are the most common What are the most common test formats test formats for for testing vocabulary?testing vocabulary?

ProductionProduction Completion items (Hughes, 143) Completion items (Hughes, 143) Transformation items (Liu, 131)Transformation items (Liu, 131) Paraphrase (Hughes, 143)Paraphrase (Hughes, 143) Table completionTable completion Combination and addition itemsCombination and addition items Items involving the changing of wordsItems involving the changing of words

WhenWhen should we go for should we go for vocabularyvocabulary productionproduction??

When…When… You want to evaluate a person’s ability You want to evaluate a person’s ability

to formulate a correct answer. to formulate a correct answer. You have more time to score the itemsYou have more time to score the items You want to test a persons ability to You want to test a persons ability to

apply concepts and information to a apply concepts and information to a new situation. new situation.

You have a clear idea of the aspects You have a clear idea of the aspects and concepts that should be tested. and concepts that should be tested.

Test Test FormatsFormats

Multiple ChoiceMultiple Choice

The flight attendant asked the passengers to ______ The flight attendant asked the passengers to ______ attention to the safety demonstration.attention to the safety demonstration.

a. give b. devote c. pay d. lend a. give b. devote c. pay d. lend They can be tricky or too picky   They can be tricky or too picky   Difficult to test attitudes towards learning Difficult to test attitudes towards learning Knowledge is limited to options provided Knowledge is limited to options provided Difficult to construct at higher levels Difficult to construct at higher levels Encourages guessing (25% chance) Encourages guessing (25% chance) More than one option may be possibleMore than one option may be possible All options must be grammatically possibleAll options must be grammatically possible

Matching Word-DefinitionMatching Word-DefinitionTangle means… Tangle means…

a.a. A type of danceA type of dance

b.b. A tropical forestA tropical forest

c.c. A confused massA confused mass

d.d. A kind of fruit A kind of fruit They are not communicative   They are not communicative   Difficult to test attitudes towards learning Difficult to test attitudes towards learning Words have diverse connotationsWords have diverse connotations Encourages guessing (25% chance) Encourages guessing (25% chance) One word class at a timeOne word class at a time

Single SelectionSingle Selection

Someone was (playing/singing) the tune and for a Someone was (playing/singing) the tune and for a moment you were happy to go (along/away) with moment you were happy to go (along/away) with what seemed a reasonable idea.what seemed a reasonable idea.

They are authentic and communicativeThey are authentic and communicative Words are part of a broader context   Words are part of a broader context   Knowledge is limited to options provided Knowledge is limited to options provided Encourages guessing (50%)Encourages guessing (50%) Difficult to design but easy to markDifficult to design but easy to mark

Correction/EditingCorrection/Editing

Maria bought a skarf for the winterMaria bought a skarf for the winter Tests recognition of mistakesTests recognition of mistakes Is focused on word spelling onlyIs focused on word spelling only Mistakes are ungrammaticalMistakes are ungrammatical Negative backwash Negative backwash Context is important but not essentialContext is important but not essential

Guessing Meaning from ContextGuessing Meaning from Context

Unlike the guitar, the Unlike the guitar, the fiddlefiddle has four strings on a has four strings on a fingerboard without frets.fingerboard without frets.

Tests contextualized knowledgeTests contextualized knowledge Context is crucialContext is crucial Communicative and authenticCommunicative and authentic Positive backwash Positive backwash Difficult for some students (bias)Difficult for some students (bias) Tests various types of word relationshipsTests various types of word relationships

Gap-FillingGap-Filling

The flight attendant asked the passengers to The flight attendant asked the passengers to ______ attention to the safety demonstration.______ attention to the safety demonstration.

They must have a broader context   They must have a broader context   More than one option may be possible More than one option may be possible

(unless tester provides limited options or (unless tester provides limited options or first letter)first letter)

Tests can focus on content wordsTests can focus on content words Production is tested unless options are Production is tested unless options are

provided provided Focus should be on the aspect assessedFocus should be on the aspect assessed

ClozeCloze

John ______ works ______Monday ______FridayJohn ______ works ______Monday ______Friday Tests student’s understanding of the whole Tests student’s understanding of the whole

language   language   Many word types can be testedMany word types can be tested Understanding of context is crucialUnderstanding of context is crucial Difficult to test attitudes towards learning Difficult to test attitudes towards learning Some words may be impossible to guess Some words may be impossible to guess Can test student’s IQ rather than language Can test student’s IQ rather than language

proficiencyproficiency Difficult for some students (bias)Difficult for some students (bias)

Table CompletionTable Completion

Complete the table with positive, negative and Complete the table with positive, negative and neutral adjectivesneutral adjectives

Tests language structureTests language structure Tests word-building   Tests word-building   Many word types can be testedMany word types can be tested Not communicative at allNot communicative at all Tests use but not language usageTests use but not language usage Difficult to test attitudes towards learning Difficult to test attitudes towards learning

Word TransformationWord Transformation

Turn verbs into nouns:Turn verbs into nouns:

1.1. The opera was conducted by the (compose)____The opera was conducted by the (compose)____ Not very communicative (without context)Not very communicative (without context) Clues will limit the answersClues will limit the answers Difficult to test attitudes towards learningDifficult to test attitudes towards learning Some context may be added Some context may be added Instructions are essentialInstructions are essential Few aspects might be assessed (word Few aspects might be assessed (word

classes, synonyms, antonyms…)classes, synonyms, antonyms…)

Five characteristics to measure Five characteristics to measure communicativecommunicative vocabularyvocabulary

1.1. The test must provide more context than The test must provide more context than only a single sentence.only a single sentence.

2.2. The items must assess lexis within a limited The items must assess lexis within a limited number of semantic fields.number of semantic fields.

3.3. The test taker should understand what the The test taker should understand what the communicative purpose of the task is.communicative purpose of the task is.

4.4. He or she should also know who the He or she should also know who the intended audience is.intended audience is.

5.5. He or she must have to focus on meaning He or she must have to focus on meaning and not form to answer correctly.and not form to answer correctly.

6.6. Recognize is not sufficient. The test taker Recognize is not sufficient. The test taker must be able must be able ““to produce grammatical to produce grammatical responses.responses.””

(R. Dickins)

Evidence shows Evidence shows that semantically related that semantically related items are 'stored together'items are 'stored together'. Items are . Items are arranged in a series of associative networks, a arranged in a series of associative networks, a large 'master file', and a variety of 'peripheral large 'master file', and a variety of 'peripheral access files' which contain information about access files' which contain information about spelling, phonology, syntax and meaning. spelling, phonology, syntax and meaning.

Knowing a word, then, is the sum total of all these Knowing a word, then, is the sum total of all these connections — semantic, syntactic, phonological, connections — semantic, syntactic, phonological, orthographic, morphological, cognitive, cultural orthographic, morphological, cognitive, cultural and autobiographical. It is unlikely, therefore, that and autobiographical. It is unlikely, therefore, that any two speakers will 'know' a word in exactly the any two speakers will 'know' a word in exactly the same way.same way.

In ConclusionIn Conclusion