Learning for the Future: The

performance of 15-year-olds in

Ireland on reading literacy, science

and mathematics on PISA 2018

3rd December 2019

Overview

PISA 2018 results – key findings and trend comparisons

Reading literacy

Science

Mathematics

Selected key findings from PISA questionnaires

Conclusions

PISA 2018, national team:

Caroline McKeown, National Project Manager

Sylvia Denner, National Data Manager

National team: Gerry Shiel, Sarah McAteer, Lynsey O’Keeffe, and Mary Delaney.

PISA 2018 – Introduction (1)

79 countries, and almost 600,000 students participated

In Ireland, 5,577 students in 157 schools

In Ireland, students were born in 2002, and were aged 15- and 16-years-old

Students were drawn from across five year levels, First-Fifth Year

Third Year was the modal grade, with over three-fifths of students.

In Ireland, PISA was administered entirely on laptops hired and transported to

schools

PISA 2018 – Introduction (2)

Students sat a 2-hour test and completed a 55-minute questionnaire session

In addition:

School principals (or their nominee) completed a school questionnaire (CBA)

Parents of selected students completed a questionnaire (PBA)

Teachers of Junior Cycle English were asked to complete a national questionnaire

(PBA)

Ireland did not participate in the cognitive assessment of Global Competence,

but students in Ireland did respond to questionnaire items on the topic.

PISA 2018 – some data notes

79 countries participated, but:

Data for Vietnam has not been fully validated for international comparability –

applies to all three domains

Reporting on data for Spain in reading literacy is deferred until sub-optimal

response patterns (in the test of reading fluency) are investigated

Comparability of data for Austria in 2009 cannot be ensured, therefore trend

comparisons for Austria are excluded 2009-2018. Only applies to reading literacy.

Reading literacy:

2018 results for 36 OECD countries, 2009-2018 results for 35 OECD countries

Mathematics and science:

2018 results and trend comparisons include all 37 OECD countries

PISA 2018 – interpretation of results

To aid interpretation of results, a longer-term trend is provided in the figures. However, in the report, the focus is on the last time the subject was a major domain:

Reading literacy, 2009-2018

Science, 2015-2018

Mathematics, 2012-2018.

Interpreting scores: not like percent correct

OECD mean was originally set at 500, with SD of 100.

5-point difference is not really meaningful, 10-point difference is small, 15 is moderate, 20-25+ large.

Studies like PISA always involve uncertainty in measurement due to assessment of samples rather than populations, along with use of rotated test design. For example, the measurement error for Ireland for reading literacy is 2.2.

PISA 2018 results – an overview

Ireland’s performance in reading literacy, science and mathematics was

relatively stable between 2015 and 2018, with small but not statistically

significant changes in each domain.

On average across OECD countries between 2015 and 2018, mean scores on

reading literacy and mathematics increased by small non-significant amounts,

while there was a small non-significant drop in science.

Students in Ireland are significantly above the OECD average on all 3 domains

of reading literacy, science and mathematics.

PISA 2018 – reading literacy

On reading literacy, students in Ireland are ranked 4th out of 36 OECD

countries, and 3rd out of 27 EU countries

Ireland’s true rank is between 1st and 5th across OECD countries.

Student performance on PISA reading literacy in Ireland is characterised by:

An above-average percentage of high performers (12.1%)

A small and below-average percentage of low performers (11.8%)

Ireland’s mean score of 518.1 is significantly higher than the OECD average of

487.1

Ireland’s mean score is not significantly different from Hong Kong (China),

Estonia, Canada, Finland, Korea, Poland

Three regions significantly outperform Ireland, B-S-J-Z (China), Singapore and

Macao (China).

8

PISA reading literacy mean scores,

Ireland and the OECD, 2000-2018

526.7

515.5517.3

495.6

523.2 520.8518.1

494.4

493.8

486.1491.2

493.5490.2

487.2

470

480

490

500

510

520

530

2000 2003 2006 2009 2012 2015 2018

Ireland OECD Average

Large drop in PISA 2009; subsequently attributed to statistical model used

to account for missing responses; otherwise stable trend; slight downward

trend in line with OECD average.

PISA reading literacy proficiency levels,

Ireland, 2009-2018

17.2

9.6 10.2

11.8

7.0

11.410.7

12.1

4

6

8

10

12

14

16

18

2009 2012 2015 2018

Below Level 2 At or above level 5

Small, non-significant upwards trend in percentage below Level 2, and Levels 5-6

PISA reading literacy mean scores,

Ireland by gender, 2009-2018

476.3

509.2515.0

506.4

515.5

537.7526.9

529.6

400

420

440

460

480

500

520

540

560

2009 2012 2015 2018

Male Female

Unusually low gender gap in 2015; lower than average gap in 2018

compared to OECD.

PISA 2018 reading literacy proficiency

levels by gender, Ireland and the OECD

15.1

8.510.3

13.8

27.7

17.5

7.1

10.4

0

5

10

15

20

25

30

Male Female Male Female

Below Level 2 Levels 5-6

Ireland OECD

Strong performance by Irish females; room for improvement among Irish

males; both genders do well compared with OECD averages.

PISA 2018 - science

On science, students in Ireland ranked 17th among 37 OECD countries and 11th

out of 28 EU countries.

Ireland’s true rank is between 13th and 21st across the OECD.

Student performance on PISA science in Ireland is characterised by:

An average percentage of high performers (5.8%)

Lower than average percentage of low-performing students (17.0%).

Ireland’s mean score of 496.1 is significantly higher than the OECD average of

488.7

Ireland’s mean score is not significantly different from 11 countries (Germany,

the United States, Sweden, Belgium, Czech Republic, Switzerland, France,

Denmark, Portugal, Norway and Austria)

PISA science mean scores, Ireland and

the OECD, 2006-2018

508.3

508.0

522.0

502.6

496.1

494.8497.8 498.3

490.6488.7

480

485

490

495

500

505

510

515

520

525

2006 2009 2012 2015 2018

Ireland OECD Average

Drop in science scores in 2015 coincided with introduction of interactive

questions (simulated experiments)

PISA science proficiency levels, Ireland,

2015-2018

15.3% 17.0%

7.1%5.8%

0

2

4

6

8

10

12

14

16

18

2015 2018

Below Level 2 Levels 5-6

Trends in performance at proficiency levels mirror overall mean

performance 2015-2018 (small, non-significant changes)

PISA science mean scores, Ireland by

gender, 2015-2018

507.7

495.4

497.2496.9

450

460

470

480

490

500

510

520

2015 2018

Male Female

Narrowing of gender gap due to decline in the mean score of male

students in Ireland

PISA 2018 science, proficiency levels by

gender, Ireland and the OECD

18.1

16.0

6.8

4.9

23.2

20.8

7.36.2

0

5

10

15

20

25

Males Females Males Females

Below level 2 Levels 5-6

Ireland OECD average

PISA 2018 - mathematics

On mathematics, students in Ireland ranked 16th among 37 OECD countries,

and 11th out of 28 EU countries.

Ireland’s true rank is between 12th and 21st across the OECD.

Student performance on PISA mathematics in Ireland is characterised by:

A below average percentage of high performers (8.2%) than on average across

OECD countries.

A below average percentage of low-performing students (15.7%).

Ireland’s mean score of 499.6 is significantly higher than the OECD average of

489.3

Ireland’s mean score in mathematics does not differ significantly from ten

countries/economies (Sweden, United Kingdom, Norway, Germany, Czech

Republic, Austria, Latvia, France, Iceland, and New Zealand).

PISA mathematics mean scores, Ireland

and the OECD, 2012-2018

502.8

501.5

487.1

501.5503.7

499.6

499.2

490.0

491.6

490.4

487.2

489.3

475

480

485

490

495

500

505

2003 2006 2009 2012 2015 2018

Ireland OECD

As with reading, performance in mathematics is relatively stable across

cycles except for a drop in 2009.

PISA mathematics, Ireland by proficiency

levels, 2012-2018

16.9%

15.0%15.7%

10.7%9.8%

8.2%

0

5

10

15

20

2012 2015 2018

Below Level 2 At or above level 5

PISA mathematics mean scores, Ireland

by gender, 2012-2018

509.0 511.6 502.6

493.7 495.4 496.7

440.0

450.0

460.0

470.0

480.0

490.0

500.0

510.0

520.0

2012 2015 2018

Males Females

As with science, narrowing of gender gap due to decline in mean

score of male students in 2018.

PISA 2018 mathematics, proficiency

levels by gender, Ireland and the OECD

15.7 15.7

9.9

6.6

23.9 24.0

12.3

9.5

0

5

10

15

20

25

Male Female Male Female

Below Level 2 Levels 5-6

Ireland OECD

Use of digital technology by students in

PISA 2018

On the index of subject-related digital technology use in class, students in

Ireland had a mean score (-0.37), which was significantly and substantially

below the average across OECD countries (0.00).

Students’ reports of time spent using digital devices for subjects outside class

(this could be at home or in school). The mean score for Ireland was -0.30,

was significantly below the OECD average.

Students in Ireland also reported that across selected subjects (English,

Science, and Mathematics), it was mainly the teacher who used digital

devices in the classroom.

In 2018, under half of participating students had taken a test on computer

prior to PISA, which was up slightly from 2015 (46.5% in 2018 and 42.8% in

2015).

Enhancing teaching and learning with

digital technology in PISA 2018

Principals’ views on the capacity of their schools to enhance teaching and learning through digital technology are generally less favourable in Ireland than on average across the OECD, with some relative strengths.

The percentage of students’ whose principals ‘agreed’ the school’s Internet bandwidth was sufficient (75.9% v 67.5%), adequate software was available (71.9% v 71.3%) and digital devices were sufficiently powerful (73.2% v 68.5%) were about the same or higher than the OECD average.

However, principals in Ireland highlighted relative weaknesses relating to levels of sufficient technical support or assistance (20.7% v 54.1%), the availability of effective professional resources for teachers to learn how to use digital technology (47.4% v 64.7%), and the skill levels of teachers (49.3% v 64.6%) as challenges to successful integration of digital technology in teaching and learning.

Note, perentages: Ireland v OECD

PISA 2018 - Time spent using digital devices

on a normal school day by reading

achievement539.5

461.5

512.0

484.6

528.1

463.9

450.0

460.0

470.0

480.0

490.0

500.0

510.0

520.0

530.0

540.0

550.0

None Less than 1 hour 1 to 3 hours 3 to 5 hours More than 5 hours

Playing computer games (on a digital device)

Chatting or interacting with friends on social networking sites

Watching TV (including online)

Aspects of student well-being in PISA

2018 – Life satisfaction

Well-being in PISA 2018 will be reported on by the ERC in more detail in 2020.

‘Overall, how satisfied are you with your life as a whole these days?’

Slider from ‘0 - not all satisfied’ to ‘10 – completely satisfied’.

Three-fifths (61%) of students in Ireland reported that they were satisfied

(score of 7-10) with their lives, which is significantly lower than the overall

average across OECD countries (67%).

On average, significantly fewer female students in Ireland reported that they

were satisfied with their life (55.5%) compared to male students (67.3%).

Aspects of student well-being in PISA

2018 – Exam stress (national question)

22.6% of students reported they always feel under pressure from teachers to

do well in exams

Students who reported never feeling under pressure from their teachers to do well

scored significantly lower in reading literacy compared to students who reported

sometimes feeling pressure (517.3), often feeling pressure (535.2) or always

feeling pressure (530.1).

10.7% of students reported they always feel physically unwell thinking about

or doing exams and tests

Students who reported that they often or always felt physically unwell thinking

about exams performed less well on average on reading literacy (498.9 and 509.1

respectively), compared to students who never feel physically sick thinking about

exams (546.9).

Conclusions – reading literacy

The relative stability in overall mean scores, notwithstanding the revised PISA

assessment framework (to better reflect how reading has evolved in recent

years) and new test content, highlights the consistently strong performance

of students in Ireland on PISA reading literacy.

This steady performance includes the incorporation digital navigation and

other interactive reading skills and competencies, both in traditional and

newer texts.

The new English specification was introduced in 2014, and thus all

participating students in PISA 2018 studied under it.

Students in Ireland in 2018 reported strong awareness of reading strategies. In

particular, students in Ireland were significantly above the OECD average

(0.00) at 0.21 on an OECD-developed metacognition index, where students

are asked to evaluate the appropriateness of various strategies for assessing

the credibility of an email.

Conclusions - science

A small drop in student performance in science in PISA 2018 was not-statistically significant.

However, between 2012-2018, the decline in performance in science is large and significant (-25.9 score points). This coincided with the introduction of computer-based testing and the introduction of new interactive items.

There is a below average percentage of high-achieving students in science in PISA 2018, even though Ireland’s mean score is significantly higher than the OECD average.

As in mathematics, the gender difference for science is small and non-significant in 2018. This differs from 2015 when male students performed significantly better than females in mathematics and science.

The new science specification was introduced in 2016, with the first round of students sitting that subject as part of the Junior Cycle Profile of Achievement (JCPA) in 2019. Therefore, only a small proportion of students (2%) participating in PISA 2018 studied under the new specification for science.

Conclusions - mathematics

Ireland’s mean score for mathematics remains significantly above the OECD

average in 2018.

However, as in science, there is small but incremental decline in the

proportion of students performing at the highest levels in mathematics (at or

above Proficiency Levels 5 and 6) 2012-2018.

The new specification for mathematics was introduced to First Year students

in 2018; therefore, almost all PISA 2018-eligible students studied under the

previous specification, often referred to as Project Maths.

Mathematics becomes the major assessment domain in PISA 2021, when the

framework will be revised, and items that capitalise on the affordances of

computer-based assessment will be introduced.