Post on 22-Dec-2015
transcript
Bridging the Gap from Implementation to Attainment:
Utilising Results from International Comparative
Studies.
Surette van Staden PIRLS 2011 Co National Research Coordinator
University of Pretoria
Edu Lead Conference 14 April 2015
Aim of the Presentation
• Evidence of curricular attainment that has taken place at Grade 4 level across 11 languages in South Africa.
• Interrogation of benchmark results to provide evidence for possible improvement of curricular implementation at classroom level.
• Reflections of how teacher practice can benefit from evidence provided by international comparative studies.
What is PIRLS?
• An international comparative study in reading literacy under the auspices of the IEA
• Measures trends across years
• Monitors policy implementation over time
• Provides a global context for achievement
prePIRLS 2011
Grade 4
Afrikaans
English
isiNdebele
isiXhosa
isiZulu
Sepedi
Sesotho
Setswana
siSwati
Tshivenda
Xitsonga
PIRLS 2011
Grade 5
Afrikaans
English
PIRLS and prePIRLS 2011 Samples
Study Grade Attained sample
No of learners
prePIRLS 4 341 15 744
PIRLS 5 92 3 515
Total 433 19 259
• Implementation in 2011• Reading Literacy
Assessment• Background
Questionnaires
prePIRLS 2011 Overall Results
Colombia International Centre point Botswana South Africa0
100
200
300
400
500
600
700
576
500 463 461
Ave
rag
e sc
ale
sco
re
International Benchmarks
Represented by a four-point scale:
Low International Benchmark (400 points)
Intermediate International Benchmark (475 points)
High International Benchmark (550 points)
Advanced International Benchmark (625 points)
PrePIRLS 2011 Overall Benchmark Performance
0 20 40 60 80 100 120 140 160 180
29% 71% 42% 18% 6%
Not Attained Low Intermediate High Advanced
prePIRLS 2011 Overall Benchmark Performance
% Achieving Benchmarks
Per
cen
tag
e o
f le
ar..
. 29 71
Not Attained Low
30
23
12
6
Low IntermediateHigh Advanced
Of 71%
prePIRLS 2011 Benchmark Performance by Test Language
South Africa
Afrikaans
English
isiNdebele
isiXhosa
isiZulu
Sepedi
Sesotho
Setswana
siSwati
Tshivenda
Xitsonga
29
12
10
31
38
29
57
36
34
24
53
47
6
15
19
0.2
0.4
0.8
0
0.1
0.1
0.25
0
0
Did not reach Low International benchmark Intemediate International BenchmarkHigh International Benchmark Advanced International benchmark
LOW INTERNATIONAL BENCHMARK
400 POINTS
Literary Texts:- Locate and retrieve explicitly stated detail
Informational Texts:- Locate and reproduce 2-3 pieces of
information
Not Attained
Low
Intermediate
High
Advanced
0 10 20 30 40 50 60 70 80
29%
71%
This means learners are probably unskilled in:
• monitoring of their own memory and comprehension
• reading as a meaning-making process • detection of contradictions in text• successfully resolving inconsistencies in
understanding • exerting control of reading processes• realising that they do not understand what is
read
ADVANCED INTERNATIONAL BENCHMARK
625 POINTS
Literary Texts:- Integrate ideas and evidence across
text- Interpret story events to provide
reasons, motivations, feelings, character traits with text-based support
Informational Texts:- Distinguish and interpret complex
information from different parts of text with text-based support
- Integrate information across text to provide explanations, interpret significance and sequence activities
Not Attained
Low
Intermediate
High
Advanced
0 10 20 30 40 50 60 70 80
6%
This means learners are skilled in:
• demonstrating awareness of what they are reading
• knowing reasons why they are reading• having tentative plans or strategies in
place for handling potential problems • monitoring their comprehension of textual
information
The Value of Benchmark Information
• Unequivocally points to the problems in the foundational levels of the education system in getting learners to even master literal comprehension regardless of the language of instruction.
• The international benchmark medians shows that it is possible for Grade 4 learners to attain higher order comprehension skills at this developmental stage.
• Benchmark profiles reveal to the South African system and teachers specifically that learners are not engaged with mastery of reading comprehension or the higher order reading comprehension development needed for further academic achievement in the schooling system.
Implementation Challenges
• Teacher comparisons made only within their own school and neighbouring teachers.
• Uncertainty about activities that are undertaken, pace and the level which these are pitched are often only mediated by curricular expectations and the pace of work within the school as dictated by fellow teachers and school management.
The Value of prePIRLS 2011 Results to Teachers
• Making released reading materials and questions that are used for purposes of the study available to teachers.
• Familiarity with the processes of comprehension the assessment is aimed at: Focus on and retrieve explicitly stated information Making straightforward inference Interpret and integrate ideas and information Examine and evaluate language and text elements
• Making international benchmark descriptions available. • These may serve as a guide to teachers’ own development of
reading material in the classroom against an existing, international framework.
The Value of prePIRLS 2011 Results to Teachers (continued)
• The provision of released passages to teachers to serve as exemplars of levels of expectation at Grade 4 as benchmarked against international standards.
• Exemplary learner responses linked to these items in themselves may provide valuable insights into comprehension processes and developmental requirements to teachers.
• Could serve as a trigger to a more informed understanding of what learners at this developmental stage of learning are actually capable of achieving.
The Value of prePIRLS 2011 Results to Teachers (continued)
• The impact of the PIRLS cycles on macro level policy is irrefutable (e.g. Foundations for Learning, Drop All and Read).
• Yet teacher level feedback from international testing and assessment programmes is still in its infancy and not fully realised in South Africa.
• Feedback to teachers is of cardinal importance, since change at classroom level can only be implemented in the face of empirical evidence of what constitutes effective classroom practice.
• Where international assessment results remain only as grim reminders of South African learners’ poor performance, assessment as integral to learning has failed.
• Assessment results could provide evidence to inform further support or intervention for teachers and learners in need.
• Assessment should provide indicators of those systemic factors that can be changed, adapted or used as leverage against which to ensure improved curricular implementation.
• Failing to do so, assessment as agent of such improved curricular implementation, may become powerless in providing feedback of significant factors that are at work in the home, in classrooms and in schools nationwide.
Conclusions