Is Free School Meal Status a Valid Proxy for Socio-Economic Status?

Graham Hobbs and Anna Vignoles

Centre for the Economics of Education(funded by Department for Education and Skills)

•Family SES potentially key determinant of pupil attainment •FSM proxies family SES in much UK educational research:

– FSM “gap”– school composition effects– social segregation

•FSM as a control variable– Ethnicity “gap”– class size effects/school resource effects,– school effects

CEE FSM Project

CEE Project

• SES measures are generally missing from administrative data sets

• Project assesses the biases that may arise from using FSM as a proxy in particular contexts

• Little in the way of formal evaluation of FSM, with notable exceptions– DfES Statistics of Education: Trends in Attainment

Gaps: 2005– Croxford 2000

CEE Project

• Examine FSM eligibility criteria• Explore national data on benefits to determine what FSM

is actually measuring• Explore these relationships in ALSPAC-NPD merged

data• Present a framework for assessing the validity of proxy

variable as a “control variable” in OLS regression models, recognising that the validity of FSM as a proxy for family SES is model-specifi

• Assess the parameter on FSM when FSM is the “variable of interest”.

1. Merged ALSPAC-NPD data

2. FSM eligibility rules and claimant data

3. FSM status and family SES in ALSPAC-NPD data

4. Proxy variable framework

5. Illustrative example

Outline

•ALSPAC eligibility: Mother resident in Avon. Expected delivery Apr 1991 to Dec 1992.

•ALSPAC enrolment: 14,049 live births

•ALSPAC data: Family income, presence of partner, mother & partner employment, mother & partner education, mother & partner social class

•NPD data: FSM, age, gender, ethnicity, EFL, SEN, & KS2 & KS1 attainment

Merged ALSPAC-NPD data

•Only 11,131 of 14,049 successfully matched to NPD– Slightly non-random sample

•Only 5,924 of 14,049 responded to all SES variables– Item & wave non-response

– Strongly non-random sample

•Family SES observed in pregnancy or aged 4-4½. FSM measured aged 9-11.

Missing data and timing of data

•FSM measure: Eligible & claiming FSM

•FSM eligibility 2005/6: – Eligible & claiming IS or IB-JSA

– Child Tax Credit (but not Working Tax Credit) & income less than £13,480

– Guarantee element of State Pension Credit

– Support under part VI of Immigration & Asylum Act 1999

•FSM eligibility 2001/2: Eligible & claiming IS or IB-JSA

FSM measure and eligibility rules

•No-one working more than 24 hours per week, a low income & limited capital assets (benefit unit)

•92% of dependents of IS & IB-JSA claimants were dependants of IS claimants

•83% of IS claimants with dependents were single parents

•Average IS payment £106 per week for single parents

•34% of ‘lone parent’ claimants received IS for ≥5 years

IS/IB-JSA eligibility and claimant data

Conclusions from eligibility rules

• Children in families claiming either IS or IB-JSA were eligible for FSM

• Majority of these children were in families with a single parent, aged 25-59, and one or two children

• Given IS and IB-JSA eligibility rules, all of the children should be in families without a parent in full-time employment, with low incomes and limited capital assets.

Potential reasons for non claiming

• Stigma effects

• Stigma effects varying by school

• Concerns about diet – differing by ethnic/ cultural background

Story and Chamberlin 2001/ Croxford 2000

FSM status and family SES in ALSPAC

Percentage of: Family SES FSM pupils Non-FSM pupils Sample (% FSM) Workless families 45 4 7,708 (6.6) At least one full-time worker 43 91 7,708 (6.6) One-parent 28 4 7,821 (6.8) Income £199 pw or less 78 20 7,473 (8.1) Mother O-level or less 89 65 10,505 (9.7) Partner O-level or less 76 50 7,396 (6.3) Mother class IIIm, IV or V 43 19 8,505 (7.3) Partner class IIIm, IV or V 70 42 7,072 (6.2)

Ordinal associations between FSM status and family SES Ordinal measure of association: Family SES gamma tau-b Sample (% FSM)

Family employment 0.74 0.26 7,708 (6.6) Partner’s employment 0.81 0.28 7,821 (6.8) Mother’s employment 0.62 0.18 8,039 (7.3) One-parent family status 0.80 0.26 7,821 (6.8) Family income 0.80 0.30 7,473 (8.1) Mother’s education 0.53 0.20 10,505 (9.7) Partner’s education 0.47 0.12 7,396 (6.3) Mother’s social class 0.43 0.14 8,505 (7.3) Partner’s social class 0.42 0.11 7,072 (6.2)

Potential reasons for discrepancies

• Non claiming• Misreporting of income etc. in ALSPAC• Timing of data

– For time-varying family SES variables, the true association between FSM status and family SES is likely to be stronger than reported here

– Observe the family SES variables in pregnancy or between ages 4–4½

– Observe FSM status between ages 9–11

• Sample selection

Structural equation:y = β0 + β1x1 + … + βKxK + γq + v

Linear projection:q = θ0 + ρ1x1 + … + ρKxK + θ1z + r

Proxy variable regression:y = (β0 + γθ0) + (β1 + γρ1)x1 + …+ (βK + γρK)xK + γθ1z + (γr + v)

Proxy variable framework

q variables i.e. ‘true’ SES variables

• Family income

• Parental education

• Social class

• Parental employment

• One parent status

Structural equation:

Proxy variable framework

vqγxββqqxxyEM

mmm

K

kkkMK

11011 ),...,,,...,|(

K

kmmkkmmm rzθxρθq

110

M linear projections:

m=1,…,M

Proxy variable regression:

)()()()(11

11 11

00

M

mmm

M

mmm

K

kk

M

mkmmk

M

mmm rγvzθγxργβθγβy

Illustrating FSM as a proxy variable

• The validity of FSM as a proxy for family SES is model-specific

• Illustrate with example from a (specific) value-added model of Key Stage 2 attainment

• Assessing the validity of FSM when FSM is a control variable

Illustrative example

Model/KS2 maths Age Gender Ethnicity EFL SEN FSM

Proxy variable -0.021** 0.220** 0.075 -0.277* 0.003 -0.073* (0.003) (0.019) (0.073) (0.159) (0.115) (0.041) Structural -0.018** 0.207** 0.058 -0.232 -0.091 (0.003) (0.018) (0.069) (0.187) (0.118) Structural plus FSM -0.018** 0.207** 0.058 -0.232 -0.091 0.001 (0.003) (0.018) (0.069) (0.188) (0.118) (0.042)

Bias -0.003 0.013 0.017 -0.045 0.094 Bias (%) -17.8 6.06 29.0 -19.4 103

Sample = 4,824 (4.4% FSM)

Findings

• In the context of this specific value-added model of KS2 attainment, FSM is an “imperfect proxy” for family SES variables

• Estimates of parameters on some variables are biased as a result of using FSM

• Large biases on SEN and, in some subjects, ethnicity (but small number of minority ethnic group children in our estimation sample prevents us from making strong inferences for other studies)

FSM as variable of interest

• What is the parameter on FSM in the proxy variable regression is actually measuring?

Decomposition of FSM differences in KS2 attainment

KS2 English KS2 maths KS2 science

FSM parameter -0.125** -0.073* -0.172**

Size of FSM parameter accounted for by:

All family SES variables -0.072 58% -0.074 101% -0.073 43%

Mother’s employment 0.014 -11% 0.012 -16% 0.013 -8% Family income -0.032 26% -0.041 56% -0.010 6% Mother’s education -0.029 23% -0.032 44% -0.046 27% Mother’s social class -0.011 9% 0.001 -2% -0.006 4% Partner/partner characteristics -0.014 11% -0.014 18% -0.024 14%

Sample = 4,824 (4.4% FSM)

Findings

• Around half of conditional and unconditional FSM differences in KS2 attainment are accounted for by the eight family SES variables (with the exception of conditional FSM differences in KS2 maths)

• Conditional FSM differences in KS2 attainment are principally accounted for by mother’s education and family income