Using Linked Administrative Data to Enhance Policy Relevant Analysis:Experience from the ADMIN Research Centre
James Brown (University of Southampton)
Lorraine Dearden (Institute of Education)
ISI World Congress, Dublin, 23rd August 2011
2
Outline
• Overview of the ADMIN Research Centre
• Framework for the research agenda
• Introduction to the ‘Pupil-Level Annual School Census’ (PLASC) and the resulting ‘National Pupil Database’ (NPD) Linkage to create the database Linkage with the LSYPE
• Examples using the NPD and other data…
3
ADMIN Research Centre
• ADMIN is a Phase II Node in the Economic and Social Research Council’s (ESRCs) National Centre for Research Methods (NCRM).
• NCRM has a central Hub and then funds three-year research and training nodes. The ADMIN node is approaching the end of its three years of
funding.
• Each Node has a research programme in the area of research methods (quantitative, qualitative, mixed methods). Complementary training programme to enhance the use of
research methods within the Social Science community.
4
ADMIN Research Centre
• Research agenda focused around methods to use administrative data in policy-making.
• Particular focus on the advantages of linked data either across administrative systems and/or between surveys and administrative data.
• Extensively used the school-based administrative data and related linkages for England. Also utilised the Northern Ireland Longitudinal Study (linkage
between health-card registration data and 2001 Census) as another example of linking administrative data.
5
‘Framework’ for the research
• Linkages between administrative systems enhance the utility of the data for policy research. One system may collect the outcomes while another collects the
background information.
• Linking survey data into an administrative system can expand the breadth of information (covariates) available for modelling outcomes. Administrative systems have excellent coverage of units but often
collect little covariate information on the units (beyond what is strictly needed by the system).
6
‘Framework’ for the research
• Linking administrative data into a sample survey can provide enhancement in a variety of ways. Auxiliary information for small area estimation. Contextual/historical information to enhance modelling of
individuals. Provide outcomes that are difficult to measure (income). Allow studies of measurement error in the survey (and potentially
the reverse).
• High quality linkage is clearly an IMPORTANT issue but in our research we have worked with data where unique identifiers aid a near perfect linkage. Focused on using the linkage.
7
NPD/PLASC data
• The Pupil Level Annual (termly) School Census (PLASC) is basic data collected on all pupils within the state schools of England. All pupils are given a unique identifier when the enter the school system. Pupils tracked through to 16 (and then 18) with potential to track them
on into higher education. Information on age, gender, ethnicity, language, free school meals,
special educational needs, local area of home address.
• The National Pupil Database (NPD) links together PLASC data over time and enhances it with exam performance data. KS1, KS2 (at 11), KS3, KS4 (at 16 – end of compulsory schooling)
8
Linked survey data
• The Longitudinal Survey of Young People of England (LSYPE) took a sample of schools and then pupils within schools using the NPD as the sampling frame. Started in 2004 (at age 14) and then followed the cohort of children
through to the end of schooling (and beyond...)
• Survey collects background information on the pupils, their families, non-cognitive outcomes, attitudes to schooling, parental support for schooling, choices post-compulsory education…
9
Example OneLinked administrative data
• The full NPD has been used to measure school performance in England. Also provide information for parents making school choice (but this
role has been extensively criticised). League Tables based on a school’s ‘Contextual Value-Added’
(CVA) measure.
CVA• Scaled level two residual from a two-level (pupils within schools)
random intercepts model. X’s control for prior attainment at pupil and school level, as well as
other pupil background information.
10
Example OneLinked administrative data
• This classic approach to CVA has some issues with respect to the random effects assumptions. Constant additive effect for the CVA of a school. Constant residual variance across the range of prior attainment.
» ‘Capping’ of the outcome to the best eight exams at age 16. Normality assumptions of the residuals.
• We explore utilising an m-quantile approach to measure an individual pupil’s efficiency (given their inputs). Aggregate to explore the impact of higher structures on the
performance of pupils within them.
11
Example OneLinked administrative data
Comparing the fitted mean line (red) with fitted lines (grey) covering quantiles
at 1%, 5%, 25%, 50%, 75%, 95% and 99%.
12
Performance across LAs(marginal measure)
Excluded0.44 - <0.500.50 - <0.540.54 - <0.600.60 - <0.67
13
Performance across LAs(conditional measure)
Excluded0.44 - <0.500.50 - <0.540.54 - <0.60
14
Example TwoSurvey enhancing admin
• Administrative data is widely used for policy making and to measure effectiveness of public service provision. E.g. school and hospital league tables, evaluating policies
• BUT administrative data typically does not have extensive background information on students, patients, … which also impact on outcome of interest. Is the CVA measure really the residual school impact (or is it a function
of unmeasured pupil level effects clustered within schools)?
• Largely untested whether methodological approaches taken actually measures question of interest. Survey linkage can ‘help’ by adding richer covariates…
15
Example TwoSurvey enhancing admin
• Dearden, Miranda and Rabe-Hesketh look at the biases in School CVA estimates when there are missing covariates in the administrative data used to make these estimates. Look at the implication of not having measures of mother’s
education in the administrative data. Linkage to the LSYPE can add that richer set of covariates (but
only for the sample).
• Approach developed by Miranda and Rabe-Hesketh fits a model at the pupil level that corrects for the problem of the missing covariate and allows for ‘informative selection’. Use the model to predict CVA at the school level…
16
Example TwoSurvey enhancing admin
• They find from the survey data that the level of mother’s education varies systematically by school. This creates systematic biases in measures of school CVA with
schools who have relatively low educated mothers doing much worse than those with highly educated mothers.
17
Example TwoSurvey enhancing admin
18
Example ThreeAdmin enhancing survey
• Possibility of new ways of dealing with perennial problems associated with longitudinal data e.g. attrition bias, recall bias… We know exactly how well children who drop out of Next Steps
(LSYPE) perform at school as we can follow them in the administrative data.
Project looking at this by McDonald, Miranda and Rabe-Hesketh.
• Non-response (selection effects) in surveys can bias estimates in multi-level/heirarchical models. Important if trying to measure, for instance, within and between
school effects with full covariates and survey data (Clarke, Crawford, Steele and Vignoles).
19
Example FourNon-pupil example
• The Census Longitudinal Studies (LSs) covering the UK link together samples of census records across censuses. Between censuses the data is enhanced with administrative data
(particularly health related).
• The Northern Ireland LS commenced following the 2001 Census and is just under 30% of the population. Sample based on the health-card registration data.
» This provides on-going information on internal (and international) migration (as well as health-related outcomes).
» Migration data not available in other LSs. Linkage to the Census provides characteristics for LS members.
20
Example FourNon-pupil example
• The administrative data provides flows between and within broad areas of Northern Ireland. Health-card registration data is the basis for internal migration
within the UK.
• Linkage in the Northern Ireland LS provides covariates (at time of census) such as self-reported health status.
• GOAL: Fit contingency table models to the LS data that are consistent with the marginal flow data. Builds on migration estimation work by Willikens, Raymer, and
others and includes developing SE estimates…
21
Brief Discussion
• Linkage of administrative data across data sources and with surveys is growing rapidly within the UK. We are perhaps behind some other European countries and there
are cultural and legislative barriers that are being addressed. Has huge potential to enhance both the utility of the administrative
data as well as improve inference and analysis of survey data.
• ADMIN has been working to create accessible methods to help researchers exploit this new resource. Supported this by running a parallel capacity building programme
to enhance the skills of researchers.
22
Some References
EXAMPLE ONE:Tzavidis, N. and Brown J. J. (2010). Using M-quantile models as an alternative to random effects to model the contextual value-added of schools in London. DoQSS Working Papers 10-11, Department of Quantitative Social Science - Institute of Education, University of London.EXAMPLE TWO:Miranda, A. and Rabe-Hesketh, S. (2010). Missing ordinal covariates with informative selection. DoQSS Working Papers10-16, Department of Quantitative Social Science - Institute of Education, University of London.EXAMPLE THREE:Clarke P., Crawford, C., Steele, F. & Vignoles, A. (2010) The choice between fixed and random effects models: some considerations for educational research. DoQSS Working Papers 10-10, Department of Quantitative Social Science - Institute of Education, University of London.
Institute of EducationUniversity of London20 Bedford WayLondon WC1H 0AL
Tel +44 (0)20 7911 5412Fax +44 (0)20 7612 6686Email [email protected] www.ioe.ac.uk/qss
