1

Size and Depth of Vocabulary: What Does the Research Show?

Norbert Schmitt

2

History

• Two year review of vocabulary studies pertinent to size vs. depth

• Results reported at AAAL 2012• Manuscript submitted to Language

Learning• 3 reviewers gave plenty of feedback• Latest version: focus on conclusions

3

Size and Depth

• Vocabulary has often been characterized in terms of size vs. depth of knowledge

• The distinction is widespread, but one depth is not easy to pin down

• One reviewer states that depth is “the wooliest, least definable, and least operationalisable construct in the entirety of cognitive science past or present”

4

Size and Depth

• It is time to start thinking about this distinction more rigorously

• The various conceptualizations and measurements of depth make it difficult to start from a theoretical framework

• So start from an empirical perspective to inform the debate:

– Review all studies that have a measurement of size and at least one measurement of depth

5

Vocabulary Size

• Size = the number of lexical items ‘known’

6

Vocabulary Size

• Size = the number of lexical items ‘known’ (to some criterion of mastery, i.e. depth)

7

Vocabulary Size

• Size = the number of lexical items ‘known’ (to some criterion of mastery, i.e. depth)

• Every size test is also a depth test in the sense that a certain criterion of mastery must be met

8

Vocabulary Depth

• Depth / Quality = How well do you know those items?

• What can you do with those items?

• Very broad: can be conceptualized and operationalized in a variety of ways

9

Size and Depth

• The relationship between size and depth depends on:

• How both are conceptualized

• How both are measured

10

Conceptualizing Depth

• Receptive vs. Productive Knowledge

• Usually connected with the 4 skills

11

Reading, Writing, Speaking, Listening

• Depth could be seen as how well word can be employed in the four skills

• Vocabulary size correlates with all four skills

• But little research which shows how well individual lexical items are employed in the skills

12

What is Involved in Knowing a WordNation (2001)

Form spoken receptive/productivewritten word parts

Meaning form and meaningconcept and referentsassociations

Use grammatical functionscollocationsconstraints on use (register, frequency …)

13

Size and Depth

• Knowledge of all of the word knowledge aspects taken together can be conceptualized as a relatively comprehensive depth of knowledge

• But each aspect can be known to various degrees of mastery

14

Degree of Knowledge

Schmitt, 2010a: 38

15

Vocabulary Depth

• So, improving knowledge of any individual word knowledge aspect can be considered as adding to depth

• Not an all-or-nothing concept

• Anything that improves mastery can be considered additional depth

16

Vocabulary Depth

Depth =

•Degree of mastery of the form-meaning link•Polysemous word meanings•Derivations (word family members)•Collocations

•Other word knowledge aspects but were not found in research in conjunction with a size measure

17

Vocabulary Depth

• Depth in this conceptualization concerns individual lexical items

• Only a small number of items can ever be measured, so unclear how generalizable the depth measures can be

18

Conceptualizing Size and Depth

Meara and Wolter, 2004: 89

19

Lexical Organization

• Concerns lexicon as a whole rather than individual lexical items

• Depth could be seen as any of the word knowledge connections between items

• But how to measure it?• Word associations

– Difficult to interpret– Idiosyncratic to individuals– Good measure of organization?– WAF main measure, but ?

20

Lexical Fluency

• Daller, Milton, and Treffers-Daller (2007) see fluency as a separate dimension

• Can see fluency as depth (i.e. depth does not have to be knowledge, but can be seen as employability (skills, automaticity)

21

Recognition and Recall of the Form-Meaning Link

• Laufer and Paribakht (1998) • VLT (form recognition)• PVLT (form recall)• Recall-recognition r=.89 (EFL: Israeli high school)

.72 (ESL: Canadian university)

PVLT ÷ VLT ratio EFL% ESL%

Combined 77 62

2,000 94 84 3,000 76 58 5,000 62 6310,000 46 44

22

Depth of Form-Meaning Link• As vocabulary size increases (and frequency

level decreases), the recall/recognition gap increases

• Learners are more likely to have both form recognition and form recall mastery at the higher frequencies (i.e. smaller gap)

• Less likely to have form recall mastery at the lower frequency levels (i.e. form recognition mastery only)

• Form recall lags both form recognition and meaning recall

23

Knowledge of Written vs. Spoken Word Forms

• van Zeeland (2013) used a meaning recall interview to measure the written and spoken vocabulary knowledge of advanced L2 learners

• Results showed a stronger correlation between written and spoken word knowledge than found by Milton and Hopkins (2006) (r = .85 vs. .68).

• The relationship between learners’ knowledge of written and spoken vocabulary furthermore remained constant as overall scores increased

• These results suggest that knowledge of vocabulary in the two modes may be more closely related than suggested by checklist test results (Milton and colleagues)

24

Depth of Spoken-Written Mastery

• Some evidence that very small vocabularies might be mainly known phonologically

• Somewhat larger vocabulary sizes shift to being known predominately orthographically

? Advanced learners tend to have relatively balanced spoken/written vocabularies, while lower-level students are prone to the type of imbalanced vocabularies found by Milton and colleagues?

25

Knowledge of Derivatives• Correlations between size and derivation/suffix knowledge • Schmitt and Meara (1997) r

– recall of suffix derivation .27 and .35– Recognition of suffix derivation .37 and .41

• Kieffer and Lesaux (2008)– recall of derivation .53 and .46

• Kieffer and Lesaux (2012) – recall of derivation .50 -.57

• Noro (2002) – recall function of affix .42, .54, .69

• Mochizuki and Aizawa (2000)– recall function of affix .54 - .65

26

Knowledge of Derivatives

• Milton (2009) reviews Mochizuki and Aizawa’s results and concludes:

• A vocabulary size of 3,000-5,000 families is necessary for affixes to be mastered

• But even at 5,000 families, some affixes may not be known well

27

Depth of Derivative Knowledge

• Size is only modestly related to knowledge of affixes and derivatives

(system learned before items?)

• Milton suggests that a threshold might exist (3,000-5,000 families?)

28

Knowledge of Collocation

• Gyllstad (2007) high-proficiency Swedish ESL • VLT• 2 collocation tests

– COLLEX 5: a 3-option form recognition test– COLLMATCH 3: a yes/no collocation judgement

• Size - collocation (r=.90)

• 10,000 families >90% on both collocation tests • 5,000 families ≈ 85%• 3,000 families ≈ 70%

• With larger vocabulary sizes, it is possible to recognize collocations well

29

Knowledge of Collocation

• Laufer and Waldman (2011) review the literature and conclude:

– receptive knowledge is related to general vocabulary knowledge

– productive knowledge of collocation lags behind knowledge of individual words

– the problem with collocations is not recognition, but in using them properly, i.e. productive mastery

30

Lexical Fluency • Laufer and Nation (2001) Israeli university

• VORST (computerized, timed, modified VLT)

• There was an increase in lexical access speed at a vocabulary size of around 5,000 word families

• The larger the vocabulary size, the faster the access speed

2,000 – speed (r = -.38) 3,000 – speed (r = -.40) 5,000 – speed (r = -.50)10,000 – speed (r = -.67)

31

Depth of Automaticity

• The larger the vocabulary size, the faster the access speed

• The larger the vocabulary size (and the lower the frequency level) the stronger the relationship between size and fluency

• Hint of a threshold: there is an increase in lexical access speed at a vocabulary size of around 5,000 word families

32

Lexical Organization• Henriksen (2008) Danish high school

Grades 7 10 13

• VLT – association recognition .72 n.s. n.s. • VLT – association recall .85 .69 .55

• The relationship between size and association is stronger at lower grades than more advanced ones

• Since the students had increasing vocabulary sizes at all three grades, we can also interpret the results to show a stronger size-association relationship for smaller vocabulary sizes than larger ones

33

Lexical Organization• Greidanus, et al. (2004) • Advanced Dutch learners of French • Form recognition and form recall • New WAF (paradigmatic, syntagmatic, and analytic)

• Form recognition – association (r= .70)

• Form recall – association (r= .81)

34

Lexical Organization

• Size – association correlations

.18 .36 .39 .43 .49 .51 .55 .61 .62 .69 .69

.70 .70 .72 .78 .81 .81 .82 .85 .85 .86 .89

• Overall, fairly strong correlations

35

Lexical Organization

• Size – association correlations

.18 .36 .39 .43 .49 .51 .55 .61 .62 .69 .69

.70 .70 .72 .78 .81 .81 .82 .85 .85 .86 .89

Association recallAssociation recognition (WAF)

36

Lexical Organization

• Size – association correlations

.18 .36 .39 .43 .49 .51 .55 .61 .62 .69 .69

.70 .70 .72 .78 .81 .81 .82 .85 .85 .86 .89

Lower vocabulary sizeLarger vocabulary size

37

Lexical Organization

• 3 studies showed a trend for larger vocabulary sizes having stronger size – association correlations

• 2 studies showed a trend for larger vocabulary sizes having weaker size – association correlations

38

Lexical Organization

• 3 studies showed a trend for larger vocabulary sizes having stronger size – association correlations

• 2 studies showed a trend for larger vocabulary sizes having weaker size – association correlations

• Does larger size relate to better lexical organization?

• Evidence seems mixed at this point (Problems with measuring organization?)

39

Tentative Conclusions

• Does greater vocabulary size relate to greater depth of vocabulary knowledge?

• Yes, generally• But how strongly depends on what ‘depth’ is

40

Tentative Conclusions

• How one views the size-depth relationship should depend on one’s purpose of use

• If one wishes to discuss the nature of vocabulary in general, particularly with practitioners, then the distinction is useful

41

Tentative Conclusions

• If one’s purpose is to characterize vocabulary knowledge in more precise terms:– theorizing– designing and interpreting research– assessment

• Depth is probably too vague a term to be useful

• Need to state lexical aspect addressed and focus on that

42

Tentative Conclusions

• Virtually all aspects of vocabulary knowledge seem interrelated

• This makes it difficult to discuss any particular conceptualization of depth in isolation

• This makes it difficult to conceptualize overall depth as anything but the combined interrelationships between word knowledge aspects

43

Tentative Conclusions

• The most widely-used vocabulary tests are size tests, and they typically describe their results as the number of words ‘known’

• But they do not define what this actually entails

• Test developers need to explicitly state what correct answers on their tests entail, and what degree of depth they represent

44

Tentative Conclusions

• There can be no clear conceptual distinction between size and depth

• Size by definition is the number of lexical items ‘known’ to some criterion level of mastery

• But the criterion will always be some measure of depth, and so the two will always be confounded

45

Questions / Comments

Comments to help me understand size/depth better? ?

Size? Depth?

www.norbertschmitt.co.uk