EALTA in Voss (June 2005)

A theory of the reading process tested in a large scale study

The DESI-projectHeiner Willenberg(University of Hamburg)

What is DESI*?

• A follow-up study of PISA in Germany• Initiative and funding: The permanent conferences

of the ministers of education• Round about 11,000 pupils (9th class) have been

tested in September 2003 and summer 2004. • German and English: Mothertongue (L1) / First

foreign language• *Deutsch– Englisch–Schülerleistungen –

International

DESI-German

DESI-Reading

• Two points of testing / 41 Tasks• Four literary texts (prosa, one poem)• Four documents (popular science)

– Added: four tasks which require comparisons of two textstems

Three types of tasks- Multiple Choice- Open tasks- Halfopen: Underlining, marking

Three main discussion topics

• 1. A national reading test for the mother-tongue (or L1) can / should be based on a different theoretical background compared to a world-wide test.

• 2. The reading levels prove parts of the theoretical rationale and show problems.

• 3. Mother-tongue testing goes beyond foreign-language testing and requires a clear extension of the underlying framework.

1a) Reading theory Comparison between DESI and PISA

PISA:

Difficulty of a text:

Hidden information

PISA:

Reading acitivies:

integrating, infering, interpreting linguistic nuances

PISA:

Knowledge as a competence

DESI:

Four or five categories

for describing difficulities in the arrangement of the text

DESI:

Reading process is central:

Reading activities define the levels

DESI:

Knowledge should be activated

General knowledge

Knowledge about text structures

1 b) Reading theory Scientific positions

• In the core: The reading process becomes more difficult when the quantity of text which must be digested is growing. Quantity in time and text leads to more complex demands for readers.

• Dominating trend of psycholinguistics: Speech-reception and sprech-production, cf. Hogrefe encylopaedias from 1999 and 2003

• Work of Walter Kintsch and coworkers (esp. 1998/99)• Comparable with Kirsch et al.: Adult Literacy in OECD

Countries (1998).• These models are valid as well for individuals as for large-

scale testing.

1c) Theory of the reading process

• In the DESI-test• 1. Decoding of words / Identifiying of text

information• 2. Making inferences between sentences• 3. Maybe focussing on a problem (awareness of a

problem) (one point in the text)

1 c) Theory of the reading process

• 4. Parts of the text-structure: Activating of existing knowledge / learned knowledge. Syllabus of 9th graders : Metaphors, irony, funny traits, perspectives, evaluations

• 5. Several paragraphs Combining / Global inferences of divergent paragraphs. Mostly motives of figures, hidden traits

• 6. The whole text or essential parts: Forming a mental model (subjective / objective):

• Figures, time, space, motives, causalities• Possible by comparing texts or analysing condense

quotations

Example of a text: Grandmother

Grandmother´s eyes grumbled [machten Mucken / familiar]. And she didn´t see any longer the little dust spots on her sunday-blouse and the eyes of the sewing needles. We children threaded her files for sewing and mending. „Should I get some glasses for me?“

„For chatting you don´t need any glasses“, said grandfather.One day she spiced the soup of swedes with that gunpowder, grandfather kept in the kitchen

cupboard - instead of pepper. Grandfather spit and the pieces of the swedes-soup flied around in the kitchen. „This woman is poisoning me!“ The soup-terrine went out off the window.

A pair of glasses should be bought for grandmother urgently. Grandfather at first gave her old ones of himself. This glasses didn´t fit for grandmother´s eyes. „I feel dizzy“, she told.

Of course, of course“, said grandfather, „with the glasses you do see the whole swindel/ dizzyness [Schwindel] bigger“, said grandpa.

Grandmother was offended and went to the district town by bus, where she visited an uncle. There she remained for one week.

Grandmother

Grandmother came back with some spectacles. Big round glasses in a nickel-plated frame, left and right of grandmother´s pointed nose. Grandmother looked like a little barn owl and then she made a round-tour through the village. She visited her chatting mates [Klatsch-Kumpankas, familiar] and looked at them through her spectacles: „My God, Bertka, how old did you get in the time I wasn´t here!“

• Grandmother came back from her round and was completey dissatisfied: „How old they´ve become when I wasn´t here. And I myself young and quite chirpy and cheerfull. It´s almost a shame.“ She looked at herself in the mirror and became quiet, she looked from the side and became even quieter.

• „Amen“ said grandfather. Then she took off the glasses and used it only when sewing the socks and when cooking a meal.

• Erwin Strittmatter (1985)

Questions and solutions

• Footnote 1: Mucken: means grumbling in a figurative sense /familiar• 2: Klatschkumpankas = Kumpan + „ka“ means: gossiping-mates

• 1st Level: Where do you find a first mention of problems with grandmas eyes?

• 2nd Level: Inferences: Why does grandmother think, that Bertka has grown old during her absence?

• 3rd Level: Focussing cannot be shown here.• 4th Level: Knowledge about text-structures:„In this story there

are some funny parts or remarks. Do underline them!• 5th Level: Connecting two parts of the text: „The narrator tells

that grandmother uses her spectacles at the end only when she is preparing the meals. Why?

Questions and solutions

• 6th Level: Building a mental model: „Grandfather tells: „With the glasses you do see the whole swindel/ dizzyness [Schwindel] bigger“. What does he mean with this remark?“

• 6th Level: Building a mental model • [Second type: Questions which require a comparison of two stories,

mainly concerning actions and motives]

• In this task you should compare both stories about grandmother and grandfather. Which assertion is correct?

• Grandmother shows emotions like jealousy or vanity in both stories ?» ¨ Right» ¨ False

2 a) Results: Levels

1,00 2,00 3,00 4,00

1 Lexik identifizieren 2 lokale Lektüre 3 Verknüpfen 4 Mentales Modell

-3

-2

-1

0

1

2

3

Item

sch

wie

rig

keit

50%

ige

Wah

rsch

ein

lich

keit

3

3735

2b) Levels

• 1. Identification of words and information:.• 2. Local Reading: • Inferences • Focussing – consciousness of problems or difficult points in text• 3. Connections • Activation if general knowledge• Activation of knowledge about text structures• Connecting divergent paragraphs: mostly for understanding motives

or causality • 4. Mental models: • Bring together figures (characters), time, space, central motive,

causality

2d) Quantity of text per level :1 italic / 2 blue / 4 bold / 5 green / 6 red

Grandmother´s eyes grumbled [machten Mucken / familiar]. And she didn´t see any longer the little dust spots on her sunday-blouse and the eyes of the sewing needles. We children threaded her files for sewing and mending. „Should I get some glasses for me?“

„For chatting you don´t need any glasses“, said grandfather.One day she spiced the soup of swedes with that gunpowder, grandfather

kept in the kitchen cupboard - instead of pepper. Grandfather spit and the pieces of the swedes-soup flied around in the kitchen. „This woman is poisoning me!“ The soup-terrine went out off the window.

A pair of glasses should be bought for grandmother urgently. Grandfather at first gave her old ones of himself. This glasses didn´t fit for grandmother´s eyes. „I feel dizzy“, she told.

Of course, of course“, said grandfather, „with the glasses you do see the whole swindel/ dizzyness [Schwindel] bigger“, said grandfather.

Grandmother was offended and went to the district town by bus, where she visited an uncle. There she remained for one week.

2d) Quantity of text per level :1 italic / 2 blue / 4 bold / 5 green / 6 red

Grandmother came back with some spectacles. Big round glasses in a nickel-plated frame, left and right of grandmother´s pointed nose. Grandmother looked like a little barn owl and then she made a round-tour through the village. She visited her chatting mates [Klatsch-Kumpankas, familiar] and looked at them through her spectacles: „My God, Bertka, how old did you get in the time I wasn´t here!“

• Grandmother came back from her round and was completey dissatisfied: „How old they´ve become when I wasn´t here. And I myself young and quite chirpy and cheerfull. It´s almost a shame.“ She looked at herself in the mirror and became quiet, she looked from the side and became even quieter.

• „Amen“ grandfather said. Then she took off the glasses and used it only when sewing the socks and when cooking a meal.

• Erwin Strittmatter (1985)

2 e) Comparison between DESI and PISA 2003: Quantity of text

• PISA 2003 /2004 • 1. Short part of a text• 2. Part of a text

• 3. Several sentences or parts of a text

• 4. Several paragraphs• 5. Textstems oder

segments of text •

• DESI –PD 2004• 1. Word• 2. Inferences between

sentences• 3. One difficult point in a

text• 4. Several aspects of text-

structure• 5. Combining paragraphs• 6. The whole text or

several texts.

Reading theory II: Difficulties of the text

• 1. Syntax: Do the sentences fit into a window of 3 seconds if the text is read aloud

• 2. Syntax: Junctions in the sentences: temporal, causal and text-steering

• 3. Vocabulary: Levels: Basics (1-2000), concrete, abstract, specialist terminology.

• 4. Text-structure: Coherence in text: Genuine key words.• 5 Text-structure: Literary traits: Metaphors, perspective,

changes of perspectives, levels of time, indirect speech etc.• 6. Style: Elementes of vividness: Persons, actions, emotions

(in documents).

Difficulties of the texts• SYNTAX: Window of three seconds for the subject-

predicate-group• „ „Amen“ grandfather said. /• Then she took off the glasses /• and used it only when sewing the socks and when cooking a meal /3/

• SYNTAX: Junctions: None• VOCABULARY: Mostly basics and concrete words, some

rare concretes: Of course, of course“, said grandfather, „with the glasses you do see the whole swindel/ dizzyness [Schwindel] bigger“, said grandfather. Grandmother was offended and went to the district town by bus, where she visited an uncle. There she remained for one week.

Difficulties of the texts

TEXTSTRUCTURE: Key words: Here generally: Grandfather and grandmotherTEXTSTRUCTURE: Literary aspects: Funny traitsOne day she spiced the soup of swedes with that gunpowder, grandfather

kept in the kitchen cupboard - instead of pepper. Grandfather spit and the pieces of the swedes-soup flied around in the kitchen. „This woman is poisoning me!“ The soup-terrine went out off the window.

Textstructure: Funny traits: Exaggeration, break of expectation

• Texts have different difficulties in the five categories

• But these difficulties nearly don´t correlate with the results / achievements

3 DESI and the CEFR?

• Main areas of testing levels in DESI• First foreign language:• Localisation of information (1) • Activation and understanding of vocabulary (1)• Local inferences (2) / • (Understandig of the topic corrsponding to the syllabus (4))• (Rare global inferences (5))

• Mothertongue: main points of proficiency• Knowledge about text structures (4)• Global inferences / Connections (5): Motives• Mental models (6) complete interpretations, comparisons

4a) Consequences for didactics

• Collection of texts and tasks can be installed

• A databank for testing in mothertongue - education

• Lessons can be analyzed by transcriptions and videos

4b) Didactics and the parallel track(Nebenspur)

a) All possibilities for understanding a text must remain in the centre: Empathy, poetical means, emotions, horizon of one owns understanding (Hermeneutik)

b) Empirical evaluation should be in the parallel track, as an admixture

• Observations

• Different types of tasks and their inherent activation-power

• A theory of partial competencies

• Theory and practice of reading strategies, used by the pupils on their own

4c) Conclusio didactica

• May we smoke when praying? No

• May we pray when smoking? Yes, of course

• Let us integrate diagnosis sometimes into the lessons!

1 e) Theoretical comparison to Kintsch