Post on 30-Apr-2015
description
transcript
Scottish Learning Festival
International StudiesTheir potential for the practitioner
Presenters
Pete Whitehouse – Head of schools analytical services unit
Audrey MacDougall – Principal Researcher: schools
An analysts view
International studies undertaken by the Scottish Government
• PISA – Programme for International Student Assessment
• PIRLS – Progress in International Reading Literacy Study
• TIMSS - Trends in International Mathematics and Science Study
The Assessment Quadrant
Source: circular no. 02 June 2005 Qualifications, Assessment and Curriculum Division
Why do we do these studies?
• Complements SSA and other national activities
• Contributes to right-hand side of assessment quadrant - external/summative
• Feedback can also contribute to external/formative
• Complements Scotland’s AifL agenda
International study activity Timetable 2006-08
2006
2007
2008
• PISA 2006 main study testing • PIRLS 2006 main study testing
• TIMSS 2007 main study testing• PIRLS 2006 reporting• PISA 2006 reporting
• PISA 2009 Field trial
PIRLS 2006
• 40 countries and 5 Canadian provinces took part
• 150 primary schools with one P5 class per school
• Pupils aged 9 and 10 (P5) • Testing took place in Spring 2006• Results due Nov 2007
PISA 2006
• 57 countries took part including all OECD countries
• Main test domain was science. Reading and maths minor domains
• 15 year old pupils, 110 schools with 35 pupils per school
• Testing took place in spring 2006• Report due in Dec 2007
TIMSS 2007
• More than 60 educational systems participate• Data in mathematics and science at fourth and
eighth grades (P5 and S2)• 150 schools with one class per school• Testing took place in Spring 2007• Results due December 2008
What can each study tell us?
• Attainment• Attitudes, aspirations, ambitions• Impact of socio-economic and background
factors• Information about the learning environment• The curriculum
AttainmentA number of countries perform significantly better than Scotland
PISA 2003
Science Maths Reading
Finland, Japan, Korea Finland, Korea, Netherlands Canada, Finland, Korea
TIMSS 2003 (Science)Singapore, Chinese Taipei, Rep of Korea, Hong Kong, Estonia, Japan, England, Hungary, Netherlands, Australia, Sweden, Slovenia, Lithuania
PIRLS 2001 (Reading)Sweden, Netherlands, England, Bulgaria, Latvia, Lithuania, Hungary, United States, Italy, Germany, Czech Republic
Attitudes, aspirations, ambitions
PISA• Interest and enjoyment• Motivation• Sense of belonging• Self-belief and confidence• Anxieties• Learning strategies• Behaviour and effort
TIMSS• Aspirations• Behaviour• Confidence• Liking school• Learning strategies
Self-confidence in learning science (S2) - TIMSS 2003
High Medium Low
Country % Mean score
% Mean score
% Mean score
International average
48 490 38 445 13 430
**Scotland ** 59 539 28 481 14 459
Norway 60 512 30 475 10 456
New Zealand 41 548 41 509 19 489
Japan 20 595 46 551 34 529
Anxiety in PISA 2003
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
All students Bottomquarter
Top quarter Females Males
Ind
ex m
easu
re
Canada
Finland
Japan
Korea
Netherlands
Scotland
OECD Average
Socio-economic and background factors
PISA• Socio-economic index of occupational status• Index of economic, social and cultural statusTIMSS• Home study and cultural resources• Parental background and home environmentPIRLS 2001• Parental involvement• Literacy resources at home
Impact of socio-economic factors PISA 2003
0.002.004.006.008.00
10.0012.0014.0016.0018.0020.00
Str
eng
th o
f im
pac
t o
f so
cio
-ec
on
om
ic f
acto
rs
470.00
480.00
490.00
500.00
510.00
520.00
530.00
540.00
550.00
Mea
n m
ath
s sc
ore
Economic factors
Maths score
Impact of socio-economic factors PISA 2003
500
505
510
515
520
525
530
535
540
545
550
11 11 12 14 17 18 19
Strength of socio economic relationship
Mea
n m
ath
s sc
ore
Scotland
N’lands
Canada
Finland
Japan
Korea
OECD average
Having a computer at home TIMSS 2003
0 20 40 60 80 100
Scotland
Japan
New Zealand
Norway
Co
un
try
Percentage
Have computer
No computer
497 445
524 489
558 529
516 475
Having a desk at home TIMSS 2003
0 20 40 60 80 100
Scotland
Japan
New Zealand
Norway
Co
un
trie
s
Percentage
Have deskNo desk
496
525 489
554 526
517 489
Early Home Literacy Activities PIRLS 2001
Countries High EHLA
Medium EHLA
Low EHLA
% students
Average scores
% students
Average scores
% students
Average scores
Scotland 82 546 16 529 2 -
England 83 578 14 546 3 513
Sweden 41 572 45 561 14 548
Norway 47 518 41 493 13 474
N’lands 55 566 37 561 8 555
Parents discuss child’s classroom reading work with him/her
PIRLS 2001Countries Almost
every day
Once or twice a week
Once or twice a month
% students
Average scores
% students
Average scores
% students
Average scores
Scotland 44 529 41 552 12 558
England 32 562 43 573 20 580
Sweden 23 551 40 563 29 573
Norway 23 494 42 503 25 509
N’lands 23 555 41 560 28 572
The learning environment
PISA 2003School climate
School policies and practices
Resources invested in education
TIMSS 2003Demographics
Home/school involvement
School resources
School climate
Classroom characteristics
PIRLS 2001DemographicsHome/school involvementSchool resourcesClassroom characteristicsAbsenteeism, bullying and safety
Student views of teacher support PISA 2003
Country Scotland %
Finland%
Korea% Canada%
Netherlands%
The teacher shows an interest in every student's learning
65 54 58 63 49
The teacher gives extra help when students need it
78 77 56 80 66
The teacher helps students with their learning
84 86 79 86 49
The teacher continues teaching until the students understand
74 61 40 71 60
The teacher gives students an opportunity to express opinions
53 62 49 62 54
Impact of teacher support PISA 2003
Emphasis on homework TIMSS 2003 science
0% 20% 40% 60% 80% 100%
Int average
Scotland
Korea
Japan
Netherlands
Australia
Sweden
Lithuania
Co
un
try
% of students
High
Medium
Low
Student experiences of incidences in school
PIRLS 2001
0
5
10
15
20
25
30
35
40
45
Country
Per
cen
tag
e o
f st
ud
ents
Stolen
Bullied
Hit/hurt
The curriculum
TIMSS 2003• Instructional time on maths and science• Emphasis on approaches and processes in the science
curriculum• Assessment
PIRLS 2001• Instructional time on reading• Emphasis on approaches and processes in the reading
curriculum• Assessment
Emphasis in the science curriculum TIMSS 2003
Nature of science
Integrating science
Impact on society
Impact of human on environment
Experience of different ethnic groups
Scotland N Y Y Y N
Korea N N Y Y NN
Japan YY Y Y Y NN
Netherlands N Y YY YY N
Australia YY N Y Y N
Sweden YY N N YY YY
Lithuania YY Y Y YY N
Methods of assessment PIRLS 2001
0
10
20
30
40
50
60
70
80
90
100
Multi-choicequestions
Shortquestions
Longquestions
Listen toreading aloud
Oralquestioning
Oral summary
Type of question
% o
f s
tud
en
ts
Int average
Scotland
England
Sweden
New Zealand
Norway
School level reporting
• Has not been previously undertaken as sample sizes too small for reliable attainment data
• Some elements of school level reporting may be introduced for further studies for some elements of the survey
How your school might conform to the overall findings for Scotland?
Detailed results
PISA
http://www.pisa.oecd.org/pages/0,3417,en_32252351_32236130_1_1_1_1_1,00.htmlPIRLS
http://pirls.bc.edu/pirls2006/index.htmlTIMSS
http://www.timss.org/