Using research to get the best value from the Pupil

PremiumSteve Higgins,

School of Education, Durham Universitys.e.higgins@durham.ac.uk

@stig_01

National Middle Schools’ Forum Conference 2013

21st October 2013Stratford Manor Hotel, Stratford on Avon

Sutton Trust/EEF Teaching and Learning Toolkit

Why we wrote it Best ‘buys’ on average Key messages for spending the Pupil Premium Currently used by about a third of schools

http://educationendowmentfoundation.org.uk/toolkit

The pupil premium Aims:

to close the attainment ‘gap’ between the highest and lowest achieving

to increase social mobility to enable more pupils from disadvantaged backgrounds

to get to the top Universities to provide additional resource to schools to do this

Was £600 in 2012-13 for fsm1 pupils; increased to £900 in 2013-14 and £1300 in 2014-15.

1 any child registered for fsm in the last six years and all looked after children, smaller premium for children of Service families

Resources and learning

Above a minimum threshold – no simple link Conclusion: spending more won’t guarantee

improvement - no simple solution

More money ≠ more learning

Smaller classes? Complex evidence- no clear link with class size and

achievement Experimental trials suggest

Classes need to be less than about 17 for 0.2 effect size…And teachers need to change the way they teach…But support from teaching assistants not as effective

The maths:£900 x 25 pupils x 3 classes with 50% on fsm = £33,750= 1 extra teacher per 3 classesClass size reduction from 25 to 19 – expensive for little gain

One-to-one tuition Highly effective

I hour/ day over at least 6 weeksSupport for class teacher to re-integrateEffect size 0.44

The maths…6 weeks x 5 days x 1 hour = 30 hours4 days teacher time (more effective with an experienced teacher)Approx £700 (ECC models less costly)

Expensive but very effectiveConsider using pairs or triplets?

What should the Pupil Premium ‘buy’? Secondary £5,200 per pupil Primary £4,284 Wide variation

Secondaries £4,000 to £9,000 Primaries £3,000 and £8,000 Middle Schools £3,300 - £8,000 (median £4,100) Excludes Academies and Free Schools

Will £1,300 buy an extra three or four months learning for each pupil eligible for the Pupil Premium?

(In England, data from 2009-10: https://www.gov.uk/government/publications/funding-for-primary-and-service-childrens-education-schools )

The Bananarama Principle

It ain’t what you do it’s the way that you do it…

So how do you spend to “get results”? Or, what does the evidence say is a good

investment or a poor investment for learning? It ain’t what you spend it’s the way that you

spend it…

What we tried to do Summarise the evidence from meta-analysis about the

impact of different strategies on learning (tested attainment) As found in research studies These are averages

Apply quality criteria to evaluations: rigorous designs only

Estimate the size of the effect Standardised Mean Difference = ‘Months of gain’

Estimate the costs of adopting Information not always available

Toolkit

Best ‘buys’...

New entry

Toolkit

Good ‘buys’...

New entry

Toolkit

Low range ...

Worst ‘buys’... (on average)

Toolkit

New entry

Revised entry

SummariesWhat is it?How effective is it?How secure is the evidence?What are the costs?What do I need to know?

The methodology Systematic searches for meta-analyses of

interventions and quantitative estimates of impact Effect sizes aggregated (fixed effect model) Converted to months progress Moderator variables summarised

Costs estimated in terms of outlay for schools Variation within toolkit themes greater than

variation between them: it ain’t what you do… Technical appendix available on EEF website

Toolkit

Fee

dbac

k

Met

a-co

gniti

on a

nd s

elf-

regu

latio

nP

eer

tuto

ring

Ear

ly y

ears

inte

rven

tion

Hom

ewor

k (S

econ

dary

)O

ne t

o on

e tu

ition

Col

labo

rativ

e le

arni

ngM

aste

ry le

arni

ngP

honi

csS

mal

l gro

up t

uitio

nB

ehav

iour

inte

rven

tions

Dig

ital t

echn

olog

y

Soc

ial a

nd e

mot

iona

l asp

ects

of

lear

ning

Par

enta

l inv

olve

men

tO

utdo

or a

dven

ture

lear

ning

Red

ucin

g cl

ass

size

Sum

mer

sch

ools

Spo

rts

part

icip

atio

nA

rts

part

icip

atio

nLe

arni

ng s

tyle

sE

xten

ded

scho

ol t

ime

Aft

er s

choo

l pro

gram

mes

Indi

vidu

alis

ed in

stru

ctio

nH

omew

ork

(Prim

ary)

Men

torin

gA

spira

tion

inte

rven

tions

Blo

ck s

ched

ulin

gP

erfo

rman

ce p

ayP

hysi

cal e

nviro

nmen

tS

choo

l uni

form

Tea

chin

g as

sist

ants

Abi

lity

grou

ping

Rep

eatin

g a

year

-0.4

-0.2

0

0.2

0.4

0.6

0.8

Average Effects

Approaches

Eff

ect

size

Homework (Secondary)

Overview of value for money

Cost per pupil

Effe

ct S

ize

(pot

en

tial m

on

ths

gain

)

£00

1.0

£1000

Meta-cognition

Peer tutoring

EY intervention

1-1 tutoring

Digital technology

Parental involvement

Summer schools

After schoolIndividualised

learning

Performance pay

Teaching assistants

Smaller classes

Ability grouping

Promising

Could be worth it

Needs careful thought

Feedback

Phonics

Key messages

Some things that are popular or widely thought to be effective are hard to make work well in terms of tested attainment

Ability grouping (setting); After-school clubs; Deployment of teaching assistants; Performance pay

Some things look more ‘promising’Effective feedback; Meta-cognition and self regulation strategies; Early years intervention; Peer tutoring; Small group/intensive tuition; Parental involvement and engagement

Issues and limitations

Based on meta-analysis – averages of averages Conversion to ‘months progress’ is a rough

estimate Intervention research is compared with ‘normal’

practice which is varied Not ‘what works’ but what has worked – ‘good

bets’ to support professional enquiry

The challenges (1): implementation These strategies have been shown to be cost-effective

in research studies But when we have tried to implement evidence-based

strategies we have not seen system-wide improvement (e.g. AfL, SureStart)

We don’t know how to get schools/teachers who are not currently doing them to do so in ways that are True to the key principles Feasible in real classrooms – with all their constraints Scalable and replicable Sustainable

The challenges (2) : context

This is what has worked (on average) Where is there leverage for improvement in a

school or class? Will it build capacity? (i.e. is it efficient?)

For learners? For teachers?

Is it inclusive?

For disadvantaged /struggling learners…

One ‘intervention’ won’t be enoughIdentify areas of greatest needClear focus on improving learning, not (just) behaviourMid and high attaining learners can be disadvantaged too!

Effects will need to be cumulativeWhat will build learning capacity and capability?Need to track and evaluate – our best guesses are not always good enough

Evaluate impact

EEF’s DIY Evaluation Guide: http://educationendowmentfoundation.org.uk/library/diy-evaluation-guide

Establishing a baseline

Teacher assessment vs standardised tests? Individual progress vs comparison group? Test score increases or age-standarised progress?

Evidence suggests Teacher Assessment tends to over-estimate progress BUT reliance on standardised tests narrows the curriculum and encourages test preparation

FeedbackMeta-cognitionSelf regulationPeer tutoring

Small group tuitionPhonicsTA support(Parent involvement)(Early years intervention)

One-to-oneSEALBehaviour

But remember….“It ain’t what you do, it’s the way that you do it… that’s what gets results”

Effective identificationAdopting solutions, not bandwagonsHigh quality intervention/supportEvaluate to ensure learning progressQuality counts

Some LinksThe full report can be found on the EEF’s website: http://educationendowmentfoundation.org.uk/toolkit/ The toolkit is recommended by the Department for Education: http://www.education.gov.uk/schools/pupilsupport/premium/b00200492/ppstrategies Official information about the Pupil Premium and LA allocations is available at: http://www.education.gov.uk/schools/adminandfinance/financialmanagement/schoolsrevenuefunding/a00200697/pupil-premium-2012-13 Ofsted’s report is available at: http://www.ofsted.gov.uk/resources/pupil-premium Find out how much each school gets: http://educationendowmentfoundation.org.uk/toolkit/pupil-premium-calculator DIY Evaluation Guide: http://educationendowmentfoundation.org.uk/library/diy-evaluation-guide

Questions…

Using research to get the best value from the Pupil

Premium: workshop session Steve Higgins,

School of Education, Durham Universitys.e.higgins@durham.ac.uk

@stig_01

National Middle Schools’ Forum Conference 2013

21st October 2013Stratford Manor Hotel, Stratford on Avon

‘Best value’

Identify target group Improve effectiveness of provision/support Evaluate impact Track target pupils Adjust plan in response to tracking Use funding to achieve wider outcomes?

Low cost

High Impact

Low Impact

High cost

Preparing for Ofsted Communicate your strategy

Teachers Governors Website

Track target pupils Record data on spending Evaluate accurately Improve plan Justify/argue!

Tracker linksOfsted resources: http://www.ofsted.gov.uk/resources/pupil-premium-analysis-and-challenge-tools-for-schools Leading Learner blog:http://leadinglearner.me/2013/10/10/pupil-premium-analyser-and-tracker/ NAHT: http://www.naht.org.uk/welcome/advice/advice-home/governance-and-infrastructure-advice/pupil-premium-reporting-2012-2013/ TES: http://www.tes.co.uk/teaching-resource/Track-Impact-and-Spend-of-Pupil-Premium-6121277/ Deepings School (CfBT Academy): http://www.deepingschool.org.uk/162/pupil-premium http://www.deepingschool.org.uk/uploads/asset_file/How%20are%20the%20Pupil%20Premium%20pupils%20doing.pdf

For every complex problem there is a solution that is simple, neat…

and WRONG!

H.L. Mencken 1880-1956