Psychological Theory and Educational Reform:

a Finnish perspective using PISA and L2L results, with some Russia/Fin comparisons

using PISA dataJarkko Hautamäki

Using data compiled by Patrik Scheinin, Sirkku Kupiainen and myself

University of Helsinki, FinlandDepartments of Education and Teacher Education

& Centre for Educational Assessment10.3.2011 Moscow, Russia

The next picture is Patrik’s and the place is Hanko, Finland

The Finnish PISA results have been described as a Miracle…Are there any reasons to believe in educational miracles?

School Subjects

Literacies

1pc

Learning-to-LearnCompetencies

L-to-LBeliefs:self-regulated,task-activated,context-supported

Beliefs& Concep-tions

National Curriculaat School

Cross-curricularOutcomes

InternationalAchievementStandards

Life-Long Learning, Transfer, Reflective Theoretical Learning, andConstitution of Knowledge at Worklater on during adult years

ž

ß1 ß2 ß3

GPAInterpretativeModels, i.e.,

Reflective abstraction, inductivelearning,or ...

...

L-to-L

L2L in relation to GPA, National Assessment and PISA

OECD PISA

• Are students prepared to meet the challenges of the future? Do they have the knowledge and skills that are essential for full participation in society? Do they have the capacity to continue learning throughout life? What is our situation compared with other countries?

• The Programme for International Student Assessment (PISA) provides some answers.

• PISA is an internationally standardised assessment that was jointly developed by participating OECD countries.

• Administered to 15-year-olds in schools.• Repeated every three years: 2001, 2003, 2006, and

2009.• In all PISA cycles the domains of reading, mathematical,

and science literacy are assessed, one major/two minor• Tests are typically administered to between 4,500 and

10,000 students in each country.

Learning to learn - Finnish studies since 1996- European Union Pilot Project 2008 as a part of defining and measuring key competencies (8 participating countries) google search : Eu learning to learn pilot

-L2L is used to more detailed modelling national education, to complement PISA and other studies

Learning-to-learn (Def) = competency and willingness to adapt to novel tasks

• activating one’s mastery-of-thinking and perspective-of-hope

• by means of maintaining cognitive and affective self-regulation in-and-of learning activity

• L2L is then understood as a resulting ’vector’ of cognitive and personal beliefs ’vectors’

Basic Issue is

- So what! –

given our psychological understanding or other source of data, PISA,L2L…

Developing national education using causal information

• How are national concerns constituted or identified?• How one can use PISA or any international

comparative study to solve national concerns?• David Olson ‘Psychological theory and educational

reforms’ presents a communication dilemma between psycho-educational studies and policy-related national educational reforms:

• the causal analysis of the factors relevant to the functioning of school as an institution < a break > the intentional analysis of the processes relevant to teaching and learning

• Causal modelling produces variances and correlations, which are not easily translated into intentions and goals

Developing national education using causal information

• Social science knowledge as well as educational and psychological knowledge – is characterized by 3 rules concerning factors of causality, comparison, and multivariate complexity (see Edward Tufte Beautiful evidence, 2006)

• Some factors make a difference, some don’t• The differences aren’t very great• It’s more complicated than that

PISA 2003, 2006 and 2009

Some evidence and some interpretations

PISA 2009 subject means (read, math, science) by country

500 is the OECD mean with 100 SD Countries with best results do well in all tests and vice versa

• The results cannot be explained by factors associated with just one subject.

• Broad explanations are more interesting from the point of educational policy.

• However, what psychological theories can be used /proved?

The PISA results correlate on the country levelMath_Mean Reading_Mean Science_Mean

Math_Mean 1Reading_Mean 0,943 1Science_Mean 0,967 0,934 1Note: N = 40.

** Pearson Correlation is significant at the 0.01 level (1-tailed).

• Money does have a role in the results BUT• Top results are attained with medium costs.

Europe North America Asia South America

Continent

Does money explain PISA results?

Cloud is jittered

Linear Regression

2500 5000 7500 10000

Expenditure per student, 01: Pri

- 2,0

- 1,0

0,0

1,0

F1

PIS

A t

ota

l sco

re

Australia

Austria

Belgium

Czech Rep

Denmark

Finland

France

Germany

Greece

Hungary

Iceland Ireland

Italy

Japan

Korea

Luxembour

Mexico

Netherlan

Norway Poland

Portugal

Slovak Re

Spain

Sweden

Switzerla

United St

R = 0,42R2 = 0,18

Gender as an independent factor, or a fact to be explained?

Parents’ education as an independent factor, or what arethe effective mechanisms?

PISA 09 Reading

• In countries with top results even children from the least educated families perform over or on the OECD average (2009).

Contextual factors where politics decide

And difficult issues of the schooling systems with its regions, schools, and classes

And it all is about mastering the student variation, differences between students,organising schooling by the spesific national institutional solutions

• The average yearly number of hours spent in school correlates negatively with PISA results on the country level. Focus on quality rather than quantity…

• Total reading, math and science learning time per week (PISA 2009)

• Also in 2009 the best results are mostly achieved with relatively low student variation.

Variance components: Between School Variation (2009)

Scandinavian (comprehensive school) model, Asian model, Central European model.

This area is empty!

• 2009 Finnish pupils believe that school is quite interesting and useful.• Among the more positive of those countries that get highest results.

So…What is so special about

Finnish schools?

The Finnish Education System

• Basic education still mostly divided to two separate entities of grades 1–6 and grades 7-9

• Girls outperform boys in most subject on most levels

• Girls outnumber boys in general upper secondary education and in tertiary education except for technical areas

• Vocational/professional education strongly divided into “male” and “female” fields

PISA assessmentpoint/position

• No streaming or ability grouping– Yet, the choice of first foreign language at grade

3 (and a possible second one at grade 5) can affect class formation in some schools

– The same goes for a specific emphasis on music education from grade 1 on and some other “special emphasis” classes (math, science, art) in grades 7-9

• Remedial teaching and special education– Closely integrated into normal teaching; growing

emphasis on inclusion• Free school meal as a fixed part of the school

day• Emphasis on student welfare: health and

dental care, student welfare team, school psychologists, career counsellors (grades 7-9)

• All schools create their own curricula based on the national core curriculum and lesson hour distribution

• No inspection of schools but mandatory self-evaluation of schools by the municipalities and the schools themselves

• No national examinations or testing during (or in the end of) basic education (grades 1-9)

• Sample-based assessment in key subjects at grade 9 by the National Board of Education with results published only at the system level (school-level results only given to the schools themselves for internal use)

• Pedagogy geared for the teaching of heterogeneous groups with stress on the weaker students

EDUCATIONAL EQUITY ACCOUNT

as a way to look for [any] educational and schooling related results from the point-of-view of the educational theory and policy, when repeated and abundant data are available.The amount of data make it increasingly difficult to summarize and draw conclusions [of national gaps].

Educational Equity Account

The most essential educational equity factors or factors that have been shown to impede educational equity or the equal realisation of individual educability are gender, parents’ socio-economic or educational status, immigration status, home- and schooling language and, of course, schools.

There are also other factors that could be taken into account in estimating national educational equity account in specific areas like in Nordic countries, in Europe, in world.

Educational Equity Account in Finland(PISA 2006, Hautamäki & al, 2008)

Factor Level Balance Interpretation

Regional No difference Small difference

only in one region of 6

Regional balance is achieved

Urban/rural Urban M > rural M Urban >0, rural < 0 Real, but so far small differences, monitoring in needed

Parents’ education

Higher means for students with better educated parents

No difference Debates and further analyses still needed; a complex issue!

Finnish/Swedish Finnish > Swedish No difference Need to be analysed even if the diffs were same in PISA 00 and 03

Immigrants Natives > immigrants Native = 0, immigrants > 0

Need to be monitored reading habits?

Gender Girls > boys Boys < 0, girls > 0 Level diff is modest; balance diff is large

What is so special about Finnish schools?

1. Teaching every subject well... 2. … to everybody…3. ... independently of home background...4. ... in every school,5. ... while making every available hour count,6. …and keeping the atmosphere work oriented

(= serious, but pleasant).

This is not possible…without…

This is not possible without:1. Respect for teaching as a profession…2. A nationally coordinated core curriculum

-> makes requirements explicit-> provides basis for teacher education-> and learning material.

3. Masters level for all…4. High quality of teacher education…5. Many applicants -> Possibility of selection…6. Breadth and quality of educational research…7. Combined with research based teacher education and

research litteracy in schools -> Potential for self directed change.

Russia vs Finland PISA 2009

Some comparative data

mean read is a mean of 15 plausible values for readingmean math is a mean of 5 plausible values for mathmean science is a mean of 5 plausible values for science

I have calculated two values for certain analysis level 3 is the 1st principal component of reading, math and science scoresbalance is the 2nd principal component which tell the tilting of the profile, positive values meaning reading score > math score, and negative values mean that reading score of an individual is lower than his/her math score

Descriptive Statistics FINLANDN Min Max Mean SD

mean read 5810 184 744 531 83

mean math 5810 210 783 537 76

mean science 5810 218 799 549 84

Level 3 5810 -2,52 3,04 ,73 ,80

Balance 5810 -2,82 4,38 ,09 ,94

Descriptive Statistics RUSSIAN Min Max Mean SD

mean read 5308 124 750 462 86

mean math 5308 191 797 469 80

mean science 5308 133 796 480 85

Level 3 5308 -3,09 3,12 ,03 ,83

Balance 5308 -3,22 3,89 ,12 ,94

Regional differences in Finland and Russia, PISA 2009 data

Anova, SPSSFinland, F (11/5809)= 14.23 2,6 %Russia, F (44/5307) = 15,7 11,6 %

If rounded, in Russia the regional differences are 4 times larger (3 vs 12) than in Finland

Stratum Level

Adygeya -,288Bashkortostan -,138Dagestan -,455Karelia ,174Komi ,169Marij El -,357Sakha (Yakutia) -,522Tatarstan ,170Udmurtian ,063Chuvashi ,103Altai territory -,038Krasnodar -,145Krasnoyarsk ,104Primorie territory ,111Stavropol -,301Astrakhan region -,044Vladimir region ,226Volgograd region -,003Vologda region -,011Voronezh region -,185Kaluga region -,252Kemerovo region -,015

Kursk region -,062Moscow region ,005Nizhnij Novgorod -,159Novosibirsk ,301Omsk region ,255Orenburg region -,185Perm territory -,145Rostov region -,149Ryazan region -,054Samara region ,249Saratov region ,031Sakhalin region ,059Sverdlovsk region -,258Tambov region ,024Tomsk region ,162Tula region ,208Tyumen region ,341Ulyanovsk region -,266Chelyabinsk region ,279Chita region -,467City Moscow ,742Saint-Petersburg ,472Khanty-Mansijsk -,090

Home Language /Russian Data

A possible end?

• 9th• Vantaa• 2010• Region• School• Classes• Pupils

45

L2L Competence (3rd, 6th and 9th graders),

2010, N for each c. 2000

46

L2L Beliefs (3rd, 6th and 9th)2010, N 2000 for each

47

L2L competence and GPA, 9th graders

48

Students have been divided into 25 % groups according to L2L cognitive test results and then GPA has been estimated for students and results are presented by schools. The results show that there are between school differences in giving schools marks, refering to a possible threat to educational equity

Developing national education using causal information

• Hautamäki, J., Harjunen, E., Hautamäki, A., Karjalainen, T., Kupiainen, S., Laaksonen, S., Lavonen, J., Pehkonen, E., Rantanen, P. & Scheinin, P. with Halinen, I. and Jakku-Sihvonen, R. (2008). PISA 06 Finland. Analyses, reflections and explanations. Ministry of Education Publications 2008:44. Helsinki: Ministry of Education. (www.minedu.fi/english)

• Available also PISA06e.pdf• www.pisa2006.helsinki.fi• www.helsinki.fi/cea

-20

-10

0

10

20

30

40

50

60

70

80

1 2 3 4 5 6 7

Czech_Republic

Finland

Poland

Slovak_Republic

Hungary

-30

-20

-10

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7

Denmark

Finland

Iceland

Norway

Sweden

-20

-10

0

10

20

30

40

50

60

70

80

1 2 3 4 5 6 7

Czech_Republic

Finland

Poland

Slovak_Republic

Hungary

Formal (Piagetian) ThinkingGPA in relation to Formal Operational Thinking, Given no gender difference in formal operational thinking,9th graders, Learning to Learn Studies in Finland

Learning-to-LearnA developmental approach

Centre for Educational AssessmentUniversity of Helsinki, Finland

• longitudinal data• aged 12, 15, 17• basic data: all pupils (N c. 1900) of a

bigger city • presented figures are based on students

participating in all 3 measurements (attrition rate high), (n c. 600)

• y-axis: % of the max points

Deductive reasoningVaria=vocational education, 6th form•2001, 2004, 2006 (long. data, ‘backward’ grouping

Formal operational reasoningVaria=vocational education, 6th form•2001, 2004, 2006 (long. data, ‘backward’ grouping

Thinking self-concept•Gender (girls, boys)•2001, 2004, 2006 (long. data)

L2 or Learning to Learn Scales

• Theoretical model is a combination of several theories of educational psychology

• the competence scales are criterion-scaled in relation to the theory of reflective abstraction

• the belief scales comprise a comprehensive set (more than 30 scales) to cover motivations, self-efficacy, subject related self-conceptions, self-esteem, socio-moral duty, ...