PISA: Behind the headlines and past the rankings

Sue ThomsonDirector, Educational Monitoring and Research, ACER

National Project Manager PISA National Research Coordinator TIMSS, PIRLS

Australia has developed a high quality, world-class schooling system, which performs strongly against other countries of the Organisation for Economic Cooperation and Development (OECD). In international benchmarking of educational outcomes for 15-year-olds in the 2006 OECD Programme for International Student Assessment, Australia ranked among the top 10 countries across all three education domains assessed. Over the next decade Australia should aspire to improve outcomes for all young Australians to become second to none amongst the world’s best school systems.

Ministerial Council on Education, Employment, Training and Youth Affairs (MCEETYA) (2008). Melbourne Declaration on Educational Goals for Young Australians. Carlton: Curriculum Corporation.

National goal

Melbourne Declaration released just prior to release of results from PISA 2006 (December 2008) PISA 2003 data showed that Australia ranked:

equal 5th in maths; equal 2nd in reading equal 4th in science

Data for this presentation focussed on mathematical literacy as this was the major domain for PISA 2003 and PISA 2012, allowing comparisons over time to be made

Context / timing

2003

Hong KongKorea

NetherlandsFinland

AustraliaMacao China

JapanLiechtenstein

SwitzerlandCanada

Belgium

IrelandAustriaFrance

GermanyPoland

Sig. higher than Australia

Not sig. higher to Australia

Sig. lower than Australia

THE RANKINGS …

2003

Hong KongKorea

NetherlandsFinland

2012

AustraliaMacao China

JapanLiechtenstein

SwitzerlandCanada

Belgium

Sig. higher than Australia

Not sig. different to Australia

AustraliaIrelandAustriaFrance

IrelandAustriaFrance

GermanyPoland

Hong KongKorea

NetherlandsFinland

Macao ChinaJapan

LiechtensteinSwitzerland

CanadaBelgium

GermanyPoland

What is the real story?

PISA 2003 PISA 2006 PISA 2009 PISA 2012400

450

500

550

600

Mea

n m

athe

mati

cal l

itera

cy s

core

Was this the case for all students?

2003

2006

2009

2012

250 300 350 400 450 500 550 600 650 700 750 800

Aus

tral

ia

In terms of proficiency levels?

Low - Level 1:Students can answer questions involving familiar contexts where all relevant information is present and the questions are clearly defined. They are able to identify information and carry out routine procedures according to direct instructions in explicit situations. They can perform actions that are almost always obvious and follow immediately from the given stimuli

High -Level 6:Students can conceptualise, generalise and use information based on their investigations and modelling of complex problem situations, and can use their knowledge in relatively non-standard contexts. Students at this level are capable of advanced mathematical thinking and reasoning… Students at this level can reflect on their actions, and can formulate and precisely communicate their actions and reflections regarding their findings, interpretations and arguments, and can explain why they were applied to the original situation

High and low achievers 2003 2006 2009 2012

0

10

20

30

40

50

High achievers Low achievers

Perc

enta

ge o

f stu

dent

s

So, that’s who’s to blame!Girls!

Is this the case, and is it the whole story?

Gender differences

Australia one of SIX countries in the Third International Mathematics and Science Study (TIMSS) 1994/95 with no gender differences in Year 8 maths

One of the few countries with no gender differences in ADVANCED maths at year 12

No gender difference in PISA 2003

Both males and females’ scores significantly higher than OECD average in 2003

but between 2003 and 2012 ... PISA 2003 PISA 2006 PISA 2009 PISA 2012

400

450

500

550

600

Males Females

Mat

hem

atica

l lite

racy

sco

re* * *

*indicates significant difference

where is the decline in scores? 10th 25th 50th 75th 90th

0

5

10

15

20

25Females

Percentile

Diff

eren

ce in

sco

re p

oint

s 20

12 -

2003

Differencein scores between males and females

10th 25th 50th 75th 90th-20-15-10

-505

101520

Percentile

Scor

e po

int d

iffer

ence

mal

es -

fem

ales

2003

2012

Males score higher

Females score higher

Percentage of high and low achievers

2003 2006 2009 20120

10

20

30

40

50

High achievers Females High achievers MalesLow achievers Females Low achievers Males

Prop

ortio

n of

stu

dent

s

So we have learned:

the overall decline in Australia’s score is a reflection of a decline by both males AND females, but more of a decline by females

score for females now not significantly different to the OECD average

Much of the decline for females has been at the lower end of achievement

Gender gap at highest percentile of achievement DECREASED between 2003 and 2012

For both males and females, there are a larger proportion of students failing to achieve the minimum benchmark of Proficiency level 2, and fewer achieving the higher Proficiency levels.

Past the rankings: attitudes and beliefs

Intrinsic motivation

Instrumental motivation

Self-concept

Self-efficacy

Maths anxiety

Intrinsic motivation

Intrinsic motivation: the amount of interest or enjoyment in maths. “I am interested in the things I learn in maths”

46% of females agreed with this statement compared to 61% of males in Australia, and 53% of students on average across the OECD.

On average females scored more negatively than the OECD average, males more positively.

Instrumental motivation

As well as being motivated by enjoyment, students are also influenced to study subjects they perceive will be useful for them in their future.

Measured by items such as “Mathematics is an important subject for me because I need it for what I want to study later on”

80% of Australian males agreed compared to 67% of females.

Both higher than the OECD average but for males this belief was substantially stronger than for females

Self-concept

Self-concept: how confident is the student in general in maths? “I learn maths quickly”

62 per cent of males and 46 per cent of females agree with this, compared to the OECD average of 52 per cent.

The average self-concept in mathematics of Australian females was significantly more negative than both the male students and the OECD on average.

Self-efficacy

Self-efficacy: How confident am I on this particular task? “Calculating the petrol consumption rate of a car”, for example.

This item showed the most difference in confidence levels of males and females in Australia, with 41 per cent of females saying they were confident or very confident of being able to calculate this, compared to 66 per cent of males and 54 per cent of students on average across the OECD.

Maths anxiety

Maths anxiety is the worry or tension felt when confronted with mathematical tasks. It can have a negative impact on students’ ability to demonstrate their potential in a subject.

In PISA 2003 and PISA 2012 anxiety was measured by asking students their level of agreement with five statements:

I often worry that it will be difficult for me in mathematics classes

I get very tense when I have to do mathematics homework

I get very nervous doing mathematics problems I feel helpless when doing mathematics problems I worry that I will get poor grades in mathematics.

Which girls?All girls…

Below level 1

Level 1 Level 2 Level 3 Level 4 Level 5 Level 6-1.20

-.80

-.40

.00

.40

.80

Females Males

Mat

hs a

nxie

ty

Change over time

2003 2012-0.3

-0.2

-0.1

0

0.1

0.2

0.3

Mat

hs a

nxie

ty

*

Females

Males

Implications and consequences

In economic terms, Hanucheck and Woessman have calculated that if all students in Australia were to achieve the minimum OECD proficiency, Proficiency Level 1, there would be a possible 16 per cent increase in Gross Domestic Product (GDP). Imagine if all students were to achieve Level 2, not just in terms of GDP, but also in terms of having a better life.

Too many Australian students are not achieving their potential in mathematics.

Gender differences matter. Lack of engagement by female students in mathematics translates into many of them dropping mathematics as soon as they are able, cutting out many further courses of study and careers.

Looking to the adult population

Unlike reading literacy, in which males catch up to females in adulthood, the gap in mathematical literacy remains into adulthood. This is borne out by findings from the most recent Programme for the International Assessment of Adult Competencies (PIAAC):

Females

Males

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

25

19

35

31

31

34

9

17

Level 1 Level 2 Level 3 Level 4/5

Mat

hem

atica

l lite

racy

What does this all mean for policy?Curriculum?What about for classroom teaching?