Does the Index of Community Socio-Educational Advantage have

Systematic Bias?

Barbara PrestonBarbara Preston Research

21 Boobialla StreetO’Connor ACT 2602

barbara.preston@netspeed.com.auPh. 02 6247 8919

AEU, AGPPA & ASPA NATIONAL SYMPOSIUM

Advice for Ministers and ACARA on NAPLAN, the use of student data, My School and league tables

Friday 23 July 2010

Aerial Function Centre, University of Technology, Broadway, Sydney

Concern only with whether ICSEA has systematic bias against public schools

Not dealing with other apparent biases: – in favour of selective schools of all sorts and against the schools ‘selected’ students would otherwise attend- in favour of high fee private schools and against low fee schools- against schools that actively seek to serve the disadvantaged and ‘unacademic’.

Not dealing with the many other issues around My School and NAPLAN.

The My School website and the presentation of schools’ NAPLAN data

ICSEA and the presentation of statistically similar schools

The high stakes involved

A central element of the My School website is the comparison between ‘statistically similar’ schools regarding their results on the National Assessment Program – Literacy and Numeracy (NAPLAN).

On each school’s web page is a chart setting out the school’s average NAPLAN results. Immediately below are the average scores for ‘statistically similar schools’ (SIM) and for all Australian schools (ALL). There is a bar above each of the SIM and ALL scores that indicates by its colour whether the particular school is ‘substantially above’ (green), ‘above’ (pale green), ‘close to’ (white), ‘below’ (pink), and ‘substantially below’ (red) ‘statistically similar’ schools and all schools respectively. In addition, a page is linked to each school’s main page that lists up to 60 ‘statistically similar’ schools and provides similar data and colour coding.

Thus, a scan of the dominant colours can quickly indicate to the viewer whether or not the school with which they are concerned is substantially above or substantially below ‘statistically similar’ schools, or somewhere between.

My School website

Schools are classified as ‘statistically similar’ if they have a similar ICSEA score.

ICSEA has several components, the most important of which is an index based on 14 variables, covering various income, occupation, education level and other matters for all individuals/households in ABS Census collection districts (a CD has around 250 households), according to 2006 Census data. An index score is calculated for each CD, and the value for a school is derived from the weighted average of the scores of the CDs of students’ home addresses.

The final ICSEA value for a school is adjusted for remoteness and the proportion of Indigenous students, and a small proportion of schools have additional adjustments if considered warranted.

Index of Socio-Educational Advantage (ICSEA)

... if some walk with their feet that’s exactly what the system is designed to do; that is to make sure that school communities are being responsive to the legitimate high expectations of parents and kids ... (Kevin Rudd 2009)

Transparency is critical. To improve schools that are failing their students we need information. And we want parents to drive change ... (Julia Gillard 2009)

High stakes

The nature of Australia’s schooling system:

• Large private sector• Private sector much higher SES

than public sector

Year Government Nongovernment

1890 83% 17%

1900 80% 20%

1940 79% 21%

1954 78% 22%

1964 76% 24%

1971 78% 22%

1976 79% 21%

1981 78% 22%

1986 74% 26%

1991 72% 28%

1996 71% 29%

2001 69% 31%

2006 67% 33%

2009 66% 34%

Share of total enrolments

From 1890 until 1980 the public sector’s share was within 4% of 80%.

It’s now 14% below.

Post WWII peak

Allowing selectivity sets in train a vicious circle of increasing social segregation and the residualisation of the comprehensive and inclusive. (Selectivity includes excluding those not wanted – those who are disruptive, costly to educate, or simply low achieving.)

The realities of our schooling structures are complex and the politics are difficult.

It is relationships that matter:• between the private and the private• between the selective/specialist & the comprehensive• between the high fee and low fee.

Dynamics of selectivity

Low SES students achieve less in low SES schools than they do in high SES schools – social segregation in schooling exacerbates differences in educational outcomes.

Being a ‘positional good’ is inherent to schooling (in a way that it is not to health).

Those already in or aspiring to high SES schools have a vested interest in increasing social segregation (though this may be countered to some extent by their commitment to broader social justice and quality education for all). They tend to be the articulate and politically powerful.

Dynamics of selectivity

The private sector is inherently ‘selective’ (some parts more selective than others)

• Schools can be located where desired (with some constraints and pressures from the particular sector’s clientele community).

• Enrolment numbers for the school as a whole and at grade levels can be planned and assured.

• Individual students can be selected on an individual basis according to criteria of the school’s choosing (with some constraints).

As the OECD noted in School: A Matter of Choice (1994), it is often schools that choose their pupils rather than the reverse.

Type of school attended by primary school students living in localities from most disadvantaged to most advantaged- ABS Census Collection Districts, deciles of SEIFA Index of Education & Occupation, 2006

Type of school attended by secondary school students living in localities from most disadvantaged to most advantaged- ABS Census Collection Districts, deciles of SEIFA Index of Education & Occupation, 2006

Percentage of students in government, Catholic and other nongovernment primary schools with low, medium or high

family incomes, Australia, 2006

P rim a ry s c h o o ls

0 %

1 0 %

2 0 %

3 0 %

4 0 %

5 0 %

6 0 %

G o ve rn m e n t C a th o lic O th e r N o n g o ve rn m e n t A ll s c h o o ls

Low (< $1000/wk)) Medium ($1000-$1699/wk) High (>$1700/wk)

Source: Preston 2007

Percentage of students in government, Catholic and other nongovernment secondary schools with low, medium or

high family incomes, Australia, 2006

Low (< $1000/wk) Medium ($1000-$1699/wk) High (>$1700/wk)

Secondary schools

0%

10%

20%

30%

40%

50%

60%

Government Catholic Other Nongovernment All schools

Source: Preston 2007

Indigenous secondary students

• 84% of all Indigenous secondary students attend government schools.

• While 89% of LOW income Indigenous secondary students attend government schools, only 69% of HIGH income Indigenous secondary students attend government schools.

• In contrast, while only 10% of all Indigenous secondary students attend Catholic schools, 20% of HIGH income Indigenous secondary students attend Catholic schools.

• Similarly, while only 6% of all Indigenous secondary students attend other nongovernment schools, 10% of HIGH income Indigenous secondary students attend other nongovernment schools.

(Pattern is similar at the primary level. This data does not include students not living at home.)

Source: Preston 2007

Area based measures of SES & the ‘ecological fallacy’ – where is the debate?

Systematic bias in ICSEA?

Any systematic bias can only be investigated indirectly:

• Census data is only available for students by type of school attended (and level), not actual schools. • ACARA has not made available the area-based index component of ICSEA.• Students can be classified by CD in which they live, by family income, whether they have an internet connection at home, etc.• CDs can be classified according to various indexes of disadvantage / advantage such as ABS SEIFA indexes.

Findings on family income:

1. In even the most disadvantaged CD (of around 250 households) there are likely to be some HIGH income families, and the children in those families are more likely to attend Catholic or other nongovernment schools than their neighbours in LOW income families.

2. In even the most advantaged CD there are likely to be some LOW income families, and the children in those families are more likely to attend government schools than their neighbours in HIGH income families.

Percentage of all government, Catholic and other nongovernment secondary students living in each decile of disadvantage* who have HIGH family incomes

Disadvantaged Advantaged

Percentage of all government, Catholic and other nongovernment secondary students living in each decile of disadvantage* who have LOW family incomes

* Index of Education and Occupation, all Australian Census Collection Districts. Source: ABS Census 2006

Findings on home internet connection:

In CDs from the most disadvantaged to the most advantaged, students attending government schools (primary or secondary) are less likely to have a home internet connection than their neighbours attending Catholic or other nongovernment schools.

Percentage of all government, Catholic and other nongovernment primary students living in each decile of disadvantage* who have HOME INTERNET CONNECTION

Percentage of all government, Catholic and other nongovernment secondary students living in each decile of disadvantage* who have HOME INTERNET CONNECTION

Issues with judging schools according to:

1. Improvements over time at a given year level:– a school that is performing very badly in the base

year can show great improvement, but still not be performing optimally (of course the improvement should be recognised)

– a school that put much effort into improvement before the base year and was performing very highly may then have little room for improvement.

Issues with judging schools according to:

2. Improvements of particular students from one test to the next several years later– a school that draws from excellent feeder schools

and/or has excellent transition and orientation programs will have students performing very well in the first tests, with little room for improvement before the subsequent tests

– a school that has poor feeder schools and/or poor orientation will have students that have much room for improvement between tests.

Campbell’s law

The more any quantitative indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.

(D. T. Campbell, Assessing the Impact of Planned Social Change, 1976, p. 49)

Thank you.

barbara.preston@netspeed.com.au

barbara.preston59@gmail.com