Schools, fundingand performance:lessons from NSW
NP
Overview
2
Today’s discussion will look at:
1. Recent changes in school funding
2. Evidence of impact of funding
3. Evidence from evaluations of NSW low SES National Partnerships
4. Conditions for ensuring success
Recent changes inschool funding
Former funding arrangements
Resources allocated to schools through three mechanisms: 1. A school staffing allocation using an enrollment-based
multiplier adjusted for type of school, stage of learning and school size
2. Base and equity-adjusted recurrent operationalfunding allocations
3. Special purpose payments
• All children have access to a level of resources which enables them to achieve a high standard of learning
• Shift from providing inputs according to a staffing formula tolooking at the resources needed to have good outcomes
• Resources to promote school improvement• Important to address differences in learning needs• Focus on fairness and effectiveness (quality and equity) • Transparent • Provide flexibility• Promote accountability
Features of new funding models
• Core funding provided to all schools via a per-student price
• Price adjusted for stages of schooling
• Base allocation to assist smaller schools(tapered by school size)
• Core price adjustments for equity(to address additional student needs)
• Additional lines of funding ‘bundled’ to provide maximum flexibility, built into per-student price and stage weights (components in core).
Needs-based funding model
• What student and school needs should we fund?
• How much should we fund?
• What evidence is there money mattersto improving student outcomes?
Emerging issues for the new model
1. Developmentally ready at pointof entry to school
2. Meets or exceeds achievement benchmark inacademic skills at Year 7
3. Attains a Year 12 or equivalent certificate by Age 19
4. Engaged fully in education, training or work at Age 24.
Key milestones of opportunity
Student and school challenges
Student and school challenges
Early years Developmentally on track in all AEDC domains on entry to school
78.0% 241,185 entrants
22.0% 68,026 entrants
Middle yearsMeets or exceeds achievement benchmark in academic skills at
Year 7
71.6% 198,780 students
28.4% 78,853 students
Senior school yearsAttains a Year 12 certificate or
equivalent by Age 1974.0%
231,106 learners
26.0% 81,199 learners
Early adulthoodEngaged fully in employment,
education or training at Age 2473.5%
258,746 young adults
26.5% 93,289 young adults
58.2% 15.8% 15.8% 10.2%
54.8% 15.6% 16.8% 12.8%
60.0% 11.6% 18.7% 9.7%
Australia
MalesFemales
New South WalesVictoria
QueenslandSouth Australia
Western AustraliaTasmania
Northern TerritoryAustralian Capital Territory
Major CitiesInner RegionalOuter Regional
RemoteVery remote
Low23456789
High
EnglishLBOTE
Northern EuropeanSouthern European
Eastern EuropeanSouthwest and Central Asian
Southern AsianSoutheast Asian
Eastern AsianAustralian Indigenous
Other
Non-IndigenousIndigenousAboriginal
Torres Strait IslanderBoth
20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0
74
69.578.5
72.995100015885276.717481987106676.413330400352
70.99787900212170.9013251560618
59.542640990371447.1561113352158
84.7850678733032
78.22785334673263.9126855210164
61.62032448223756.3585171281089
43.4037692747002
60.605732242509661.3558955506298
64.653037719550865.8995371746423
69.504554543434871.8286111019275
75.656994744042178.6948380030385
81.795340961848389.1374648783332
71.970296568741683.3634642788779
87.769784172661883.4690399545541
87.715765247410874.4438415626696
93.145057766367185.5889876964307
89.643362370635125.1923076923077
71.7178314826911
75.207006750133543.5617860851506
41.942078364565662.6582278481013
58.4045584045584
Gender
State/Territory
Location
SES deciles (Low to High)
Language back-ground
Indigenous status
Year 12 completion
Year 3 Year 9
Within school
Between school
Within school
Between school 82.0 18.0 60.1 39.9 Student Characteristics Socioeconomic
Status 7.7 55.2 6.4 27.2
Female 0.1 0.0 0.5 0.0 ATSI 0.4 2.5 0.6 2.1 LBOTE 0.1 0.1 0.4 4.6
School characteristics
Non-city location 1.6 3.0 School SES 16.2 29.7 ATSI/LBOTE 0.5 7.5 Size of school 0.4 0.0 Selective entry 10.8 Total Variance
explained 8.2 76.4
8.0 84.9
School Effects: Numeracy Year 3, Year 9
Australia New South Wales
Early years Developmentally on track in all AEDC domains on entry to school 22.0% 19.9%
Middle years Meets or exceeds achievement benchmark in academic skills at Year 7 28.4% 28.3%
Senior school years Attains a Year 12 certificate or equivalent by Age 19 26.0% 27.0%
Early adulthood Engaged fully in employment, education or training at Age 24 26.5% 26.4%
Young people missing out
• Task: neutralise the effects of povertyand disadvantage
• 5 billion dollars?
• Based on the numbers missing out = $6.1 billion
• Based on costs of effective interventions = ?
How much funding?
Evidence on impact
• Some researchers (such as Eric Hanushek, Coleman)argue that there is no systematic connection between funding level and student outcomes
• Research on the topic can be confounding- Murnane and Levy.
Does additional money make any difference?
Murnane and Levy conducted a study of 15 low performing schools in Texas serving low income, minority children. Schools were givenan additional $300,000 annually for five years to address low achievement.
Four years later, 13 of these schools still showed poor student attendance and achievement. Yet two of the schools had raised test scores to the state average and had some of the best attendance records in Texas.
The 13 schools invested in smaller class sizes and extra teachers,but did not transform practice or programs
The two schools also lowered class sizes, but also moved special education students into mainstream classes, changed curriculum, brought in health services and used a large portion of their funds to get parents involved.
Murnane and Levy
Does additional money make any difference?
• Some researchers (such as Eric Hanushek, Coleman)argue that there is no systematic connection between funding level and student outcomes
• Research on the topic can be confounding- Murnane and Levy
• System level versus school level.
Review of 24 large-scale studies reporting effect sizes suggeststhat at a system level:
• aggregate measures of per pupil spending are positively associatedwith improved or higher student outcomes
• a positive relationship between funding and outcomes ismore likely to hold:
• For disadvantaged cohorts (primarily low SES students).
• Where schools have greater (and appropriately supported)autonomy over in-school expenditure.
• For mathematics and science subjects (across all cohorts, relative to English/literacy/reading)
• Funding has impact if it is sustained and there is certainty.
Review of evaluation studies
Paper Jurisdiction Key findings Effect size Key variables, conditions or context
Heinesen and Graversen (2005), The effect of school resources on educational attainment: evidence from Denmark.
Denmark • Expenditure per pupil has a small but significant positive effect on educational attainment
• Larger effect for students from disadvantaged backgrounds
Increase in expenditure per pupil by 1 percent increases the probability of obtaining an additional year of education by between 0.097 and 0.112 percentage points
Greenwald, Hedges and Laine (1996), The effect of school resources on student achievement
US, meta-analysis
• Moderate increases in spending may be associated with significant increases in achievement
$1000 increase in per pupil expenditure associated with 0.14 standard deviation increase in student outcomes
Holmlund, McNally and Viarengo (2010), Does money matter for schools?
UK • Expenditure has small positive effect on outcomes at end of primary school
• Higher effects for low SES students
£1000 increase in funding per pupil • English: 0.035 standard
deviations• Maths: 0.029 standard
deviations• Science: 0.034 standard
deviations
Local Education Authority (LEA) schools• Central government
provides funds to LEAs who each use their own funding formulas to allocate money to schools
Cobb-Clark and Jha (2013), Educational achievement and the allocation of school resources .
Victoria, NAPLAN data
• Modest positive relationship between funding and outcomes
• Spending patterns are broadly efficient (ie. Additional expenditure would have similar impact regardless of how it was spent)
$1000 increase in funding per pupil• Grade 5 writing: 0.08
standard deviations• Grade 9 numeracy: 0.05
standard deviations
• Principals have control over school operations
Examples of evaluation studies
Evaluations of school effects and effective practices whichcost money, including class size reduction, welfare support,or case management, suggest that some strategies arepositively associated with student outcomes.
Across studies:
• Some reported effects are larger than others
• there is also variation by student population andschool contextual variables.
Review of evaluation studies: Effective practices
Raising achievement strategies
Dropout prevention strategies
Evidence from NSW
1. Incentives to attract high-performing teachers and principals
2. Adoption of best practice performance management and staffing arrangements that articulate a clear role for principals
3. School operational arrangements that encourage innovationand flexibility
4. Provide innovative and tailored learning opportunities
5. Strengthen school accountability
6. External partnerships with parents, other schools, businesses and communities and the provision of access.
Low SES NP evaluation: Reform areas
Key initiativesReform Area 1: Attract high performance teachers • Establish leadership and strategic positions• Provide incentives to attract high performing
early career teachers• Provide mentoring support to teachers• Provide attractive terms/conditions outside
standard entitlements
Reform Areas 2-3: Staffing & performancemanagement Implement staff performance review procedures• Support early career teachers professionally• Manage the staffing mix and succession planning • Implement professional learning plans for staff• Employ additional paraprofessionals in learning
support roles • Employ additional paraprofessionals in other
support roles
Reform Area 4: Student Learning opportunity• Use assessment and other data to identify
student needs
• Implement targeted approaches to improveoutcomes of students with identified needs
• Implement differentiated teaching methodsto better meet the needs of all students
Reform Area 5: Strengthening school accountability • Use the school plan to drive change to improve
student, teacher and school performance • Use evidence from a range of sources to inform
decision making • Make planning and/or reporting processes
within the school more publicly available
Reform Area 6: External partnerships • Parents and carers generally • Aboriginal and Torres Strait Islander parents and
communities• Employers• Wider community
• A key question for the evaluation was whether or not participation in the Low SES NP has helped improve the education and life opportunities of students from low SES backgrounds through improvements in student outcomes.
• A range of analyses were undertaken using available data across a range of measures including NAPLAN results
• Principal and teacher surveys
• Case studies.
School views impact
9 in 10 of principals who participated in the 2014 survey reported that as a result of the NP their school had been able to:
• better develop their staff capacity,
• develop operational arrangements that encouraged innovationand flexibility in staffing,
• support improved student learning outcomes, and
• build teachers’ and students’ access to more innovativeand tailored learning opportunities
• Improve school culture.
But the acid test is in looking at whether or not it led to improvedstudent outcomes
Results suggest that Low SES NP has had a significant positive effect onstudent NAPLAN achievement.
• Reading scores: 1.98**
• Spelling scores: 1.79**
• Grammar: 3.64**
• Numeracy scores: 2.94**
Trend analysis
Low SES NP was estimated to have a significant positive impact on average student scores with each additional year in place. So, over a period of four years student scores may be expected to increase by four points on average, holding all other student characteristics constant.
Duration of LSNP Reading scores 1.41** Spelling scores 0.88** Grammar 0.37* Numeracy scores 0.54**
Trend analysis
Variance analysis using NAPLAN: Numeracy, Year 3
2009 2013
Within school
Between school Estimate Within
school Between
school Estimate
Constant 82.7 17.3 396.0** 80.7 19.3 391.8**
Student Characteristics
Socioeconomic Status 6.2 52.0 22.9** 7.7 55.2 22.3**
Female 0.1 -0.1 -2.5** 0.1 0.0 -1.9**
ATSI 0.4 2.9 -5.8** 0.4 2.5 -4.3**
LBOTE 0.0 0.0 0.9 0.1 0.1 2.2**
Student Characteristics
Non-city location 0.3 0.3 1.6 -0.9
School SES 14.4 11.3** 16.2 13.9**
LBOTE (%) 0.6 -2.5* 0.4 -2.5**
ATSI (%) 1.7 -2.5** 0.0 -1.3*
Disabilities (%) 1.0 -0.1 0.1 0.6
Females (%) 0.1 3.0 0.0 -0.8
Size of school 0.0 -0.6 0.4 2.0**
Low SES NP 0.0 0.8 0.6 2.1**
Total Variance explained 6.7 72.8 8.2 77.0
2009 2013
Within school
Between school
Estimate
Within school
Between school Estimate
Constant 79.4 20.6 493.4** 79.2 20.8 483.3**
Student Characteristics
Socioeconomic Status 4.1 40.9 22.8** 7.1 49.6 24.1**
Female 0.7 0.1 -5.8** 0.6 0.1 -5.7**
ATSI 0.5 2.6 -5.3** 0.4 2.5 -4.4**
LBOTE 0.4 2.3 5.1** 0.3 2.1 5.5**
School characteristics
Non-city location 0.6 1.4 1.8 0.0
School SES 16.9 15.6** 15.4 16.7**
LBOTE (%) 0.3 0.0 0.4 0.4
ATSI (%) 2.5 -0.9 0.0 -0.9
Disabilities (%) 0.8 1.0 0.3 0.7
Females (%) 0.4 0.5 0.1 -2.1
Size of school 0.0 2.5** 0.7 3.9**
Low SES NP 0.1 1.9* 0.4 2.6**
Total Variance explained 5.7 67.6 8.4 73.2
Variance analysis using NAPLAN: Numeracy, Year 5
Analysis of students in Low SES NP participating schools who remain at or above benchmark standards in NAPLAN across testing points (Year 3 to Year 5, and Year 7 to Year 9). The results are compared to students in schools not participating in Low SES NP.
Measures Description
Dependant variable
Student Outcome
NAPLAN Results are on track
For Year 5 and Year 9 cohorts in Reading and Numeracy respectively
Explanatory variables
Student factors
Prior performance Whether NAPLAN performance on track at Year 3 for Year 5 cohort and at Year 5 for Year 9 cohorts
Family SES Derived based on parental occupation and education levels
ATSI status
LBOTE status Students with Language Background Other Than English (LBOTE)
Gender
School factors
School SES Median of student SES indexes within the school.
ATSI concentration % of students being ATSI LBOTE concentration % of LBOTE students
School Sector Government, Independent or Catholic sector
Size of school Total enrolment School Location ARIA groups
Low SES NP schools Schools participating in Low SES National Partnership programs
NAPLAN analysis based on benchmarks
NAPLAN analysis based on benchmarks
At YR3 At YR5 At YR3 At YR5 At YR7 At YR9 At YR7 At YR9Reading Numeracy Reading Numeracy
YR3-5 match cohort YR7-9 match cohort
79% 79% 76%67% 69% 65% 66%
59%
21% 21% 24% 33% 31% 35% 34% 41%At or Above Bench Mark Below Bench Mark
Year 5 Outcomes Year 9 Outcomes
Explanatory variables Reading Numeracy Reading Numeracy
Student factor
Prior performance 12.50 (+*) 12.50 (+*) 21.30 (+ *) 34.50 (+*)
Family SES 1.02 (+*) 1.02 (+*) 1.02 (+ *) 1.02 (+*)
ATSI status 0.72 (-*) 0.58 (-*) 0.69 (- *) 0.53 (-*)
Gender (Male) 0.80 ( -*) 1.30 (+*) 0.82 (- *) 1.33 (+*)
School factor
School SES 1.02 (+*) 1.02 (+*) 1.04(+*) 1.04 (+*)
School sector
School size
School Location
All Low SES NP schools (VS Non-NP schools) 1.16 (+*) 1.14 (+**) 1.41 (+*)
2011 Low SES NP schools (VS 2011 Non-NP schools) 1.14 (+**) 1.45 (+*) 1.53 (+*) 1.80 (+*)
NAPLAN analysis based on benchmarks
Results of analyses using a repeated measures design for a range of governmentschool performance measures other than NAPLAN revealed that on four measures
• HSC attainment, • HSC achievement, • ATAR scores, and • Attendance
schools participating in Low SES NP had incremental rises in performance relativeto non-NP schools.
Within Low SES NP schools, those commencing in 2009 display relative improvement over time in HSC attainment and apparent retention (both Year 7 to 12 and Year 10to 12). This may reflect an impact of participation of this group of schools (which had been in the program for the longest period) on retaining students in school.
Analysis based on other outcomes
Ensuring success
Aggregate effects conceal marked differences across schoolsin terms of impact.
Based on results we identified three types of schools:
• Schools displaying strong improvement in NAPLAN results,
• Schools displaying positive, but modest improvement in NAPLAN results, and
• Schools displaying no improvement or little change.
Not all schools exhibit improvement
Low Medium High
Reading
Primary 76 202 69 Secondary 26 35 33 Combined 2 39 6
Numeracy Primary 61 164 121 Secondary 14 52 28 Combined 0 34 13
Categories of schools based on impact
Chi-square Probability
School SES-EOI index 1.25 0.74 Percent of LBOTE students 11.01 0.01* School Location 2.21 0.53 Attendance rate 5.83 0.12***
Staff profile Staff Mobility (turnover rate) 9.12 0.03*
Age profile Index 0.79 0.85 Male to female staff ratio 1.39 0.71 Length of current position 4.32 0.23
Program/funding type NP participation year 12.22 0.01*
Improving Teacher Quality programs 5.11 0.02* Literacy and Numeracy program (NP 2009) 1.08 0.3
Priority School Program (2010) 0.22 0.64
Not all schools exhibit improvement
Attracting high performing
teachers
Staffing & performance management
Improving student Learning
opportunity
Strengthening school
accountability External
partnerships
NAPLAN impact category Numeracy
High 1.6 7.6 4.8 3.5 3.3
Medium 1.9 7.7 4.3 3.2 2.9
Low 2.2 7.8 4.1 2.9 3.1
Reading
High 1.9 7.9 4.8 3.5 3.3
Medium 1.7 7.2 4.4 3.2 3.1
Low 2.2 8.5 4.2 3.2 2.7
Number of strategies
Comparing high and low impact schools
Estim Std Error Wald Prob
Attract high performance teachers
Provided mentoring support to teachers -5.56 1.80 9.51 0.00 * Provided attractive terms/conditions outside standard entitlements -3.79 1.79 4.47 0.03 *
Provided opportunities for professional learning and development 3.00 1.62 3.44 0.06 **
Staffing & performance management Introduced greater cooperation with other schools to share resources 2.34 1.35 3.01 0.08 **
Made more use of new technologies in teaching -2.91 1.60 3.33 0.07 ** Provided professional development on a range of behaviour management theories and approaches -2.57 1.59 2.64 0.11 ***
Student Learning opportunity Provided professional learning for teachers on meeting individual learning needs 1.31 0.77 2.88 0.09 **
Provided students with access to learning support services -2.84 1.37 4.28 0.04 *
Provided ILPs for students needing assistance 6.12 1.77 12.03 0.00 *
Strengthening school accountability Used evidence from a range of sources to inform decision making and/or strategic direction setting 2.96 1.72 2.94 0.09 **
Strategies of greatest impact
• They had made marked improvements in attendance rates• They had participated in the Improving Teacher Quality program• They had lower teacher turnover or mobility rates • They had placed greater emphasis than other schools on innovations in
teaching practice and addressing student needs • They had provided stronger welfare and learning support for disadvantaged
students through providing teachers with training around individual learning needs, strengthening the use of ILPs for students, and improving learner support services
• Stronger use of evidence to support decision-making and strategic planning • Stronger emphasis on developing staff through providing opportunities for
professional learning and development
High impact schools
• Some had a stronger emphasis on strategies to attract and retain teachers, particularly through leadership positions, which may itself reflect greater need in this area and schools struggling to recruit and retain quality teachers and leaders
• Less focus on use of evidence for strategic planning and decision-making • Tendency to spread their effort too broadly, rather than in a more focused and
strategic way• Less focus on innovation in teaching, addressing student needs, and
strengthening school accountability and more on staffing and performance management
• Pursued strategies around teacher and leadership recruitment, retention and development suggesting that staff stability was a major issue for them.
Low impact schools
Differences in focus and emphasis reflect the challenges that different schools face and to which they are responding, rather than the effects of the strategies themselves. For example, schools that recorded little change or impact of their reform efforts on student NAPLAN results and placed greater emphasis on attracting high performing teachers and improving the staffing and performance management role of the principal may have issues in staffing and management practices that need (and the schools recognise that need) to be addressed and that seriously undermine their performance. High impact schools, on the other hand, may have more stable staffing profiles (suggested in the results in Table 5.3) and it is their staffing and management stability that has enabled these schools to target areas of improvement in teaching practice and in addressing student needs that has delivered real traction from the NP initiatives they have implemented.
Differences in impact linked to context
• Organisational health of schools critical
• Funding needs to be sustained
• Sufficient resources to overcome the effects of educational disadvantage
• Schools have greater, and appropriately supported, autonomy, combined with accountability and capable school leadership
Conditions for funding impact