P rogramme for I nternational S tudent A ssessment. Strong performers and successful reformers. The yardstick for success is no longer improvement by national standards alone but the best performing education systems. Andreas Schleicher - PowerPoint PPT Presentation
1 1 PISA OECD Programme for International Student Assessment What students know and can do Andreas Schleicher Warsaw, 10 February 2011 Strong performers and successful reformers Andreas Schleicher Special advisor to the Secretary-General on Education Policy Head of the Indicators and Analysis Division, EDU Programme for International Student Assessment The yardstick for success is no longer improvement by national standards alone but the best performing education systems
Transcript
11P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011 Strong performers and
successful reformers
Andreas SchleicherSpecial advisor to the Secretary-General on Education Policy
Head of the Indicators and Analysis Division, EDU
Programme for International Student Assessment
The yardstick for success is no longer improvement by national standards alone but the best performing
education systems
22P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
1998PISA countries in 20002001200320062009
77%81%83%85%86%Coverage of world economy 87%
PISA 2009 in brief Over half a million students…
representing 28 million 15-year-olds in 74* countries/economies
… took an internationally agreed 2-hour test… Goes beyond testing whether students can
reproduce what they were taught……to assess students’ capacity to extrapolate from what
they know and creatively apply their knowledge in novel situations
…and responded to questions on… their personal background, their schools
and their engagement with learning and school Parents, principals and system leaders provided data
on… school policies, practices, resources and institutional
factors that help explain performance differences .* Data for Costa Rica, Georgia, India, Malaysia, Malta, Mauritius, Venezuela and Vietnam will be published in December
2011
44P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
1998PISA countries in 20002001200320062009
77%81%83%85%86%Coverage of world economy 87%
PISA 2009 in brief Key principles
‘Crowd sourcing’ and collaboration– PISA draws together leading expertise and institutions from
participating countries to develop instruments and methodologies…
… guided by governments on the basis of shared policy interests
Cross-national relevance and transferability of policy experiences
– Emphasis on validity across cultures, languages and systems– Frameworks built on well-structured conceptual
understandingof assessment areas and contextual factors
Triangulation across different stakeholder perspectives– Systematic integration of insights from students, parents,
school principals and system-leaders Advanced methods with different grain sizes
– A range of methods to adequately measure intended constructs with different grain sizes to serve different decision-making needs
– Productive feedback, at appropriate levels of detail, to fuel improvement at multiple levels .
55P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
What 15-year-olds can do
66P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Average performanceof 15-year-olds in reading – extrapolate and apply
High reading performance
Low reading performance … 17 countries perform below this line
1525354555440.000
460.000
480.000
500.000
520.000
540.000
560.000Shanghai-China
KoreaFinlandHong Kong-China
Singapore CanadaNew ZealandJapanAustralia
NetherlandsBelgium Norway, EstoniaSwitzerlandPoland, IcelandUnited States LiechtensteinSwedenGermany,IrelandFrance, Chinese TaipeiDenmarkUnited KingdomHungary,
Average performanceof 15-year-olds in science – extrapolate and apply
Low average performanceLarge socio-economic disparities
High average performanceLarge socio-economic disparities
Low average performanceHigh social equity
High average performanceHigh social equity
Strong socio-economic impact on
student performance
Socially equitable distribution of
learning opportunities
High reading performance
Low reading performance
88P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Durchschnittliche Schülerleistungen im Bereich Mathematik
Low average performanceLarge socio-economic disparities
High average performanceLarge socio-economic disparities
Low average performanceHigh social equity
High average performanceHigh social equity
Strong socio-economic impact on
student performance
Socially equitable distribution of
learning opportunities
High reading performance
Low reading performance
2009
1313P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
-2 -1 0 1 2300
593
Score
School performance and socio-economic backgroundPoland
Stud
ent p
erfo
rman
ce
AdvantagePISA Index of socio-economic background
Disadvantage
Private school Public school in rural area Public school in urban area
700
1414P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
0
10
20
30
40
50
60
70
80
Percentage of resilient students among disadvantaged students%
More than 30% resilient students
among disadvantaged students
Between 15%-30% of resilient students among
disadvantaged students
Less than 15% resilient students among disadvantaged
students
Resilient student: Comes from the bottom quarter of the socially most disadvantaged
students but performs among the top quarter of students internationally (after
accounting for social background)
1818P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
r7
Dec
embe
r 201
0Policies and practices
Learning climate
Discipline
Teacher behaviour
Parental pressure
Teacher-student relationships
Dealing with heterogeneity
Grade repetition
Prevalence of tracking
Expulsions
Ability grouping (all subjects)
Standards /accountability
Nat. examination
Standardised tests
PolicySystem
RSchool
REquity
E
1919P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Does it all matter?
2020P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Age 19
Age 21
Age 21
048
121620
Level 2Level 3
Level 4Level 5
Increased likelihood of postsec. particip. at age 19/21 associated with PISA reading proficiency at age 15
(Canada)after accounting for school engagement, gender, mother
tongue, place of residence, parental, education and family income (reference group PISA Level 1)Odds ratio
higher education entry
School marks at age 15
PISA performance at age
15
2222P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
What does it all mean?
2525P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Commitment to universal achievement
Goals, gateways,
instructional systems
Capacity at point of delivery
Incentives and
accountability
Resources where they yield most
A learning system
Coherence
Lessons from PISA on successful
education systems
A commitment to education and the belief that competencies can be learned and therefore all children can achieve
Universal educational standards and personalisation as the approach to heterogeneity in the student body…
…as opposed to a belief that students have different destinations to be met with different expectations, and selection/stratification as the approach to heterogeneity
Clear articulation who is responsible for ensuring student success and to whom
2828P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Commitment to universal achievement
Goals, gateways,
instructional systems
Capacity at point of delivery
Incentives and
accountability
Resources where they yield most
A learning system
Coherence
Lessons from PISA on successful
education systems
Clear ambitious goals that are shared across the system and aligned with high stakes gateways and instructional systems
Well established delivery chain through which curricular goals translate into instructional systems, instructional practices and student learning (intended, implemented and achieved)
High level of metacognitive content of instruction
2929P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Commitment to universal achievement
Goals, gateways,
instructional systems
Capacity at point of delivery
Incentives and
accountability
Resources where they yield most
A learning system
Coherence
Lessons from PISA on successful
education systems
Capacity at the point of delivery Attracting, developing and retaining high
quality teachers and school leaders and a work organisation in which they can use their potential
Instructional leadership and human resource management in schools
Keeping teaching an attractive profession System-wide career development
3030P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Lessons from PISA on successful
education systems
Commitment to universal achievement
Goals, gateways,
instructional systems
Capacity at point of delivery
Incentives and
accountability
Resources where they yield most
A learning system
Coherence Incentives, accountability, knowledge
management Aligned incentive structures
For students How gateways affect the strength, direction, clarity and nature
of the incentives operating on students at each stage of their education
Degree to which students have incentives to take tough courses and study hard
Opportunity costs for staying in school and performing wellFor teachers Make innovations in pedagogy and/or organisation Improve their own performance
and the performance of their colleagues Pursue professional development opportunities
that lead to stronger pedagogical practices A balance between vertical and lateral accountability Effective instruments to manage and share knowledge
and spread innovation – communication within the system and with stakeholders around it
A capable centre with authority and legitimacy to act
3131P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
How much autonomy individual schools have over resource allocation
Selecting teachers for hire, OECD average
Poland
Firing teachers, OECD averagePoland
Establishing teachers’ starting salaries, OECD average
Poland
Determining teachers’ salaries increases, OECD average
Poland
Formulating the school budget, OECD average
Poland
Deciding on budget allocations within the school, OECD average
Poland
0% 20% 40% 60% 80% 100%
Only "regional and/or national education author-ity"
Both "principals and/or teachers" and "regional and/or national education author-ity"
Only "principals and/or teachers"
3232P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Establishing student assessment policies, OECD average
Poland
Choosing which textbooks are used, OECD average
Poland
Determining course content, OECD average
Poland
Deciding which courses are offered, OECD average
Poland
0% 20% 40% 60% 80% 100%
Only "regional and/or national education au-thority"
Both "principals and/or teach-ers" and "re-gional and/or national educa-tion authority"
Only "principals and/or teach-ers"
How much autonomy individual schools have over curricula and assessment
3333P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Systems with more accountability Systems with less
accountability
480
490
500
Schools with less autonomy
Schools with more autonomy
495
School autonomy in re-source allocation
System’s accountability arrangements
PISA score in reading
School autonomy, accountability and student performance
Impact of school autonomy on performance in systems with and without accountability arrangements
3434P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Local responsibility and system-level prescription
System-level prescription‘Tayloristic’ work organisation
Schools leading reformTeachers as ‘knowledge workers’
Schools todayThe industrial
model, detailed prescription of
what schools do
Schools tomorrow?
Building capacity
Finland todayEvery school an effective school
Trend in OECD countries
3636P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Commitment to universal achievement
Goals, gateways,
instructional systems
Capacity at point of delivery
Incentives and
accountability
Resources where they yield most
A learning system
Coherence
Lessons from PISA on successful
education systems Investing resources where they can make
most of a difference Alignment of resources with key challenges
(e.g. attracting the most talented teachers to the most challenging classrooms)
Effective spending choices that prioritise high quality teachers over smaller classes
3737P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Commitment to universal achievement
Goals, gateways,
instructional systems
Capacity at point of delivery
Incentives and
accountability
Resources where they yield most
A learning system
Coherence
Lessons from PISA on successful
education systems
A learning system An outward orientation of the system to
keep the system learning, international benchmarks as the ‘eyes’ and ‘ears’ of the system
Recognising challenges and potential future threats to current success, learning from them, designing responses and implementing these
3838P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Commitment to universal achievement
Goals, gateways,
instructional systems
Capacity at point of delivery
Incentives and
accountability
Resources where they yield most
A learning system
Coherence
Lessons from PISA on successful
education systems
Coherence of policies and practices Alignment of policies
across all aspects of the system Coherence of policies
over sustained periods of time Consistency of implementation Fidelity of implementation
(without excessive control)
3939P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Some students learn at high levels
All students need to learn at high levels
Student inclusion
Routine cognitive skills, rote learning
Learning to learn, complex ways of
thinking, ways of workingCurriculum, instruction and assessment
Few years more than secondary
High-level professional knowledge workers
Teacher quality
‘Tayloristic’, hierarchical
Flat, collegial
Work organisation
Primarily to authorities
Primarily to peers and stakeholders
Accountability
Education reform trajectoriesThe old bureaucratic system The modern enabling system
4040P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Beyond schooling
4141P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Lith
uani
a
Ger
man
y
Denm
ark
Croa
tia
Hong
Kon
g-C
hina
Kore
a
Mac
ao-C
hina
Port
ugal
Hung
ary
New
Zea
land
Chile
Italy
Pana
ma
Qat
ar
-100
102030405060
Scor
e po
int d
iffer
ence
Parental support at the beginning of primary school
Score point difference between students whose parents often do (weekly or daily) and those who do not:
"talk about what they had done"
4242P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Performance difference between students who had attended pre-primary school for more than one year and those who did
not
Scor
e po
int d
iffer
ence
Israe
l
Belgium
Macao
-China
France
Switzerl
and
United K
ingdom
Dubai (U
AE)
Kyrgyz
stan
Argen
tina
German
y
New Zea
land
Slovak R
epublic
Brazil
Luxembourg
Thailan
d
Canad
a
Chinese T
aipei
Poland
Kazak
hstan
RomaniaJa
panPeru
Jordan
Norway
Azerbaij
an
ColombiaChile
Lithuan
iaSerb
ia
Netherl
ands
Slovenia
FinlandLatv
ia0
20
40
60
80
100
120Beyond schooling
Observed performance advantage
Performance advantage after accounting for socio-economic
factors
4747P
ISA
OE
CD
Pro
gram
me
for
Inte
rnat
iona
l Stu
dent
Ass
essm
ent
Wha
t stu
dent
s kn
ow a
nd c
an d
oA
ndre
as S
chle
iche
rW
arsa
w, 1
0 Fe
brua
ry 2
011
Thank you !
Find out more about PISA at… OECD www.pisa.oecd.org
– All national and international publications– The complete micro-level database
