Kaftandjieva & Takala

Kaftandjieva & Takala

What is the structure of What is the structure of the space (if it exists) of the space (if it exists) of ‘Can Do’ statements?‘Can Do’ statements?

Kaftandjieva & Takala

The Universe: The Universe: Reading ‘Can Dos’Reading ‘Can Dos’

ALTE ALTE –– 126 126ALTE ALTE –– 126 126ELP – 123ELP – 123ELP – 123ELP – 123

YKI YKI –– 60 60YKI YKI –– 60 60

DIALANG DIALANG –– 36 36DIALANG DIALANG –– 36 36

EOI EOI –– 40 40EOI EOI –– 40 40

CEF – 56CEF – 56CEF – 56CEF – 56

New Set – 54New Set – 54New Set – 54New Set – 54

Kaftandjieva & Takala

The Universe: Levels of Language Proficiency

0102030405060708090

100110120130

A1 A2 B1 B2 C1 C2

New Set

EOI

YKI

DIALANG

ALTE

ELP

CEF

Kaftandjieva & Takala

The GuideThe Guide

Facet theoryprovides an approach for proposing

a "definitional system“, or a model, to describe a universe of observations

a taxonomic device a test of structural hypotheses

Kaftandjieva & Takala

Facet – "a classification of the item domains of a given content universe according to some rule" (Shye,1978)the population facet – foreign language

learnersthe common range facet – CEF levelsthe facets of the content domain

Mapping sentence – “a verbal statement of the domain and of the range of a mapping including connectives between facets as in ordinary language” (Shye, 1978)

The GuideThe Guide

Kaftandjieva & Takala

The HitchhikersThe Hitchhikers

Exact agreement:61% – 88%Mean = 78%

Kappa0.40 – 0.87Mean = 0.60

Contingency coefficient0.58 – 0.77Mean = 0.70

Can

Can

Can

Can

Can

Can

Can

Can

Can CanCan

Can

Can

Can

Can

Can

Can

Can

Kaftandjieva & Takala

Facets: Initial SetFacets: Initial Set

PurposePurpose

Text LengthText

Length

InformationInformation ComprehensionComprehensionFieldField

ModifierModifier

Text Complexity

Text Complexity

VerbVerb

HelpHelp Text Source

Kaftandjieva & Takala

PurposePurpose

Text LengthText

Length

InformationInformation ComprehensionComprehensionFieldField

ModifierModifier

Text Complexity

Text Complexity

VerbVerb

HelpHelp Text Source

Facets: Transitional SetFacets: Transitional Set

Kaftandjieva & Takala

The SpaceThe Space

Homogeneity analysisDimension 1: 64%Dimension 2: 33%

Correlation with CEFComprehension: 0.51Field: 0.57Text length: 0.71 Text complexity: 0.80

Discrimination Measures

0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0 0,2 0,4 0,6 0,8 1

Comprehension

Field

Complexity

Length

Kaftandjieva & Takala

Space StructureSpace Structure

-1,4 -1,2 -1,0 -0,8 -0,6 -0,4 -0,2 0,0 0,2 0,4 0,6 0,8 1,0 1,2 1,4 1,6 1,8 2,0-1,0

-0,8

-0,6

-0,4

-0,2

0,0

0,2

0,4

0,6

0,8

1,0

1,2

1,4

1,6

A1

A2

B1

B2

C1C2

In Part

In General

InDetail

Comprehension

Dimension 1

Dim

ensi

on 2

-1,4 -1,2 -1,0 -0,8 -0,6 -0,4 -0,2 0,0 0,2 0,4 0,6 0,8 1,0 1,2 1,4 1,6 1,8 2,0-1,0

-0,8

-0,6

-0,4

-0,2

0,0

0,2

0,4

0,6

0,8

1,0

1,2

1,4

1,6

A1

A2

B1

B2

C1C2

Short

Average

Long/any

Text Length

Dimension 1

Dim

en

sio

n 2

-1,4 -1,2 -1,0 -0,8 -0,6 -0,4 -0,2 0,0 0,2 0,4 0,6 0,8 1,0 1,2 1,4 1,6 1,8 2,0-1,0

-0,8

-0,6

-0,4

-0,2

0,0

0,2

0,4

0,6

0,8

1,0

1,2

1,4

1,6

A1

A2

B1

B2

C1C2 Simple

Average

Compl/any

Text Complexity

Dimension 1

Dim

en

sio

n 2

-1,4 -1,2 -1,0 -0,8 -0,6 -0,4 -0,2 0,0 0,2 0,4 0,6 0,8 1,0 1,2 1,4 1,6 1,8 2,0-1,0

-0,8

-0,6

-0,4

-0,2

0,0

0,2

0,4

0,6

0,8

1,0

1,2

1,4

1,6

A1

A2

B1

B2

C1C2

Everyday

Own field

Any field

Field

Dimension 1

Dim

ensi

on 2

Kaftandjieva & Takala

Space CoordinatesSpace CoordinatesA1A2B1B2C1C2

Kaftandjieva & Takala

Lost in SpaceLost in SpaceA1A2B1B2C1C2

Can understand a restaurant menu including a wide range of culinary terms. (ALTE 035 – C2)

CAN understand the general meaning of a simplified textbook or article, reading very slowly. (ALTE 035 – A1)

Kaftandjieva & Takala

Found in SpaceFound in Space

Ordinal regressionObjects

Reading ‘Can Dos’Dependent variable

CEF levelsFactors

(x, y) coordinates

PredictionR2 = 0.82r = 0.85 (Spearman)Kappa = 0.43Exact agreement – 53%

The best – A2The worst – A1

Agreement (±1) – 94%Mean difference – 0.56

Kaftandjieva & Takala

Lost and FoundLost and FoundALTE Can Do Project:

http://www.alte.org/can_do/alte_cando.pdf

Can read quickly enough to cope with the demands of an academic course. (p. 15: C1)

Can read quickly enough to cope with the demands of an academic course. (p. 90: C2)

Can understand the general meaning of a simplified textbook or article, reading very slowly. (p. 90: A1)

Can understand the general meaning of a simplified textbook or article, reading very slowly. (p. 15: A2)

Kaftandjieva & Takala

Difference between actual and predicted CEF levels

0%

25%

50%

75%

100%

CEFELP

ALTE

DIALA

NGYKI

EOINew

4

3

2

1

0

0: 42%1: 91%

Lost and FoundLost and Found

0: 67%1: 98%

0: 46%1: 83%

Can understand a restaurant menu including a wide range of culinary terms. (ALTE 035 – C2)

Kaftandjieva & Takala

Text LengthText

LengthComprehensionComprehension

FieldFieldText

ComplexityText

Complexity

Facets: Transitional SetFacets: Transitional Set

Kaftandjieva & Takala

Mapping Sentence: Mapping Sentence: Transitional Stage Transitional Stage

A person {p} who can understand information in1. in part2. in general3. in detail

complexity on topics concerning 1. everyday situations2. his/her own field3. any field

is at least on

Range:A1A2B1B2C1C2

CEF level of language proficency.

length, in a language of

1. short2. average3. long / any

a text of 1. simple2. average3. high / any

Kaftandjieva & Takala

A person {p} who can understand information1. in part2. in general3. in detail

on a topic concerning 1. everyday situations2. my own field of interest3. any field

is at least

in terms of length and/or complexity 1. restricted2. not restricted

in a text

Range:A1A2B1B2C1C2

CEF level.on

Mapping Sentence: Mapping Sentence: Final Stage Final Stage

Kaftandjieva & Takala

I {p} can understand information in a text 1. in part2. in general3. in detail

in terms of length and/or complexity 1. restricted2. not restricted

on a topic concerning 1. everyday situations2. my own field of interest3. any field

Range:

0. NO1. YES

Mapping Sentence: Mapping Sentence: Final Stage Final Stage

Kaftandjieva & Takala

Mapping Sentence: Mapping Sentence: Final Stage Final Stage

Samplen1 = 346 (Bg)n2 = 524 (Fi)

Reliability1 = 0.942 = 0.95

Rasch modelr12 = 0.85

Eta: bi dependentComprehension: 0.50Text: 0.62Field: 0.52CEF: 0.87 (R = 0.86)

-1,5 -1,0 -0,5 0,0 0,5 1,0 1,5

-1,0

-0,5

0,0

0,5

1,0

1,5

2,0Pattern Disposion

323

322

321313

312

311223

222

221

213

212

123

122

121

113

112

211

111

Dim

en

sio

n 2

Dimension 1

Kaftandjieva & Takala