SARs User Meeting 2009

23 April 2009

Developing a case for the 2011 SARs

Keith Dugmore

Demographic Decisions

Introduction

• An independent view• Commissioned by University of Manchester• A business case aimed at the Census Offices,

plus ESRC• There’s already extensive literature• Interviews & discussions with > 50 users,

potential users, & Census Office staff (Thank you)• Report completed (it gives more detail) • + 2 issues of special interest……. # # #

The potential value of the 2011 Census

• Censuses as unique sources of information• 2011 cost: £568m• Outputs: published tables, bespoke tables,

microdata, + the Longitudinal Study• Sample surveys as alternatives?• Administrative sources as alternatives?• Census: high cost, but massive amounts of

reliable and detailed information for small populations and small areas

1991 Census SARs files

File Sample Type Geographical coverage and level

Availability

1991 Individual SAR

2% sample of individuals (>1.1 million records)

GB and NI available separately. Divided into a total of 288 SAR areas, with populations >120,000

Online registration and access via CCSR. Data can be downloaded in SPSS, tab or Stata. Nesstar tool available for online data exploration.

1991 Household SAR

1% sample of households (216,000 records, with > 500,000 people)

GB and NI available separately. Regional geography.

Online registration and access via CCSR. Data can be downloaded in SPSS, tab or Stata. Nesstar tool available for online data exploration.

2001 Census SARs files

Geographical detail

SAMS

Individual

Subject

detail

Household

National Regional Local Authority

Special License

CAMS – Individual and Household

End User License

Secure laboratory

Use of SARs 1991 and 2001

Use of SARs, 1991 and 20011991

(Ind & HH files)2001

(Ind & SAMS)2001

(Other files)Registered usersUK Academics (individuals) 308 492UK Non-academic organisations 63 14Overseas organisations 18 17

Special License HH fileProjects 14

CAMS (Ind & HH files)Projects 29

Publications: 1991 c.400; 2001 c.60 to date

2001 SARs – barriers to use c.f. 1991“1991 was so simple & straightforward, compared to 2001”

“You need the dataset on your desktop to learn about it and

experiment”

“I haven't used the 2001 SARs; the bureaucracy was just too slow to get much use out of it. I used

the 1991 SARs as virtually an immediately accessible resource”

“CAMS is a real pain – it really does stop you developing your research”

“Research productivity increases 100-fold if you have the data on your desktop”

“CAMS is a big turnoff – travelling from Newcastle to London for 2 days – unless

it’s a large project”

“It’s sad in comparison to 1991 – you do what you can with the data you can get, so I’ve used the Labour Force Survey rather

than 2001 SARs”

“CAMS is unbelievably bureaucratic and difficult – this discourages people from using

it. It took me 6 months to get to the data”

“Because staff at ONS don’t use the Census data, perhaps they don’t appreciate these things”

2001 SARs: the benefits

• Academic projects & funding• Joseph Rowntree projects• Many other organisations – 6 case studies:

– Joseph Rowntree & DWP – labour market– GfK NOP – market research– University of Leeds – microsimulation– Scottish Executive – health comparisons– Carers UK – characteristics of carers– Office of the First Minister & DFM, Northern Ireland….

Census microdata files in other countries

• CCSR Conference, September 2008• Compare & contrast:

– Canada– Australia– France – USA– Etc.

• IPUMS – an international perspective:

“USA: No need for a license, fantastic detail, used by schoolchildren aged 12”

Looking ahead – users’ needs for 2011 Census information

1. Published tables

2. Bespoke tables

3. Published SARs

4. Bespoke SARs (linking to other sources)

SARs’ attractions• Large numbers (>600,000)• Convenience – if on own PC• No / low cost (c.f. tables)• Speed / Quick queries• Exploration (c.f. ordering table)• Testing before ordering tables• Rich & flexible data• Modelling• Additional variables• Compatibility with 2011 tables• Time series 1991 / 2001 / 2011

Also, strategic needs

• Joseph Rowntree Foundation• Equality & Human Rights Commission• National Data Strategy• ONS’s statistical priorities

Plus:• Needs of National Statistical Institutes

The broader UK policy context – statistics for the public good

• Statistics and Registration Services Act 2007– “Statistics that serve the public good”

• The Code of Practice– Principle 1. Meeting user needs– Principle 5. Confidentiality, including:– “Ensure that arrangements for confidentiality are sufficient to

protect the privacy of individual information, but not so restrictive as to limit unduly the practical utility of official statistics.”

– Protocol 3. The use of administrative sources for statistical purposes

Minimising the risk of disclosure – technical approaches

• Limited datasets? (“Safe data”)• Limited people? (“Safe researchers / analysts”)• Limited locations? (“Safe settings”)• Licensing limitations? (“Safe terms & conditions”)

SARs 2011 – seeking the best balances between risk and utility

“Data utility”?? # # #

• “The utility of microdata that has undergone Statistical Disclosure Limitation methods is based on whether statistical inference can be carried out and the same analysis and conclusions drawn on the perturbed data to the original data.”

• This approach was developed when ONS was seeking to establish the extent to which data could be damaged without rendering it useless

Utility

"That property in any object, whereby it tends to produce benefit, advantage, pleasure, good, or happiness…….It is the greatest good to the greatest number of people which is the measure of right and wrong.”

Jeremy Bentham. Introduction to the Principles of Morals and Legislation (1789)

Utility and Accessibility

Accessibility

Easy Difficult

Utility – numbers of

users and uses

Few

Many

Utility – Maximise or Minimise?

Utility – numbers of

users and uses

Few

Many

Maximise Limit MinimiseDetail Limited Moderate ConsiderablePerson Anyone Approved person Approved person & projectPlace Own PC Own office National LaboratoryLicense End User Special CAMs License

Accessibility

Different SARs for different uses – a recommended way ahead

• A range of uses• A. SARs Popular products

– A1. Individual Subject– A2. Individual Local Areas

• B. SARs Professional products– B1. Individual Subject, with Local Areas– B2. Household– (+ possible bespokes)

• C. Microdata Laboratory Facility– VML + SDS

Different SARs for different uses: Recommended 2011 Census SARs files

Geographical detail

A2 Individual: Local

A1 Individual: Subject

Subject

detail

B2 Household

National Regional Local Authority

Special License

Microdata Laboratory Facility (not a SARs data product)

End User License

Data Access Agreement

B1 Individual: Subject + Local

1981 1991 2001

Occasional & New

Mainstream Analysts

Census Specialists

Census users – the growing pyramid

Numbers of users, and

their expertise

A vision and roadmap for 2011 SARs

Making the vision work in practice – a roadmap

• A range of data products• Product release dates• User advice, training and support for products• Encouraging specialist SARs analysis services• Marketing

“Many commissioners of government research are unaware of SARs”

“It’s hard to overestimate how much you need to sell, sell, sell to make people aware”

“The message that using SARs is better than carrying out expensive new surveys is not getting through to LAs”

“It’s necessary to get SARs out there, into the mainstream”

“Local Authorities have less time – we would like to turn experts to analyse SARs”

The business case

The business case –Potential Benefits

• 2001:– Academic projects, & use by many other organisations

• 2011:– Users’ needs for information, & strategic / policy needs

• Cash values?– ESRC research grants (c.£1.5m 1991/2001)– ESRC SARs support funding (c.£1.45m 2000/6)– Joseph Rowntree Foundation (c.£0.5m 2001)– Northern Ireland OFM & DFM (triggered $150m)– + Depts. such as DWP – say 1% better targeting?

Total costs of 2001 Census, England & Wales

2001 Census costs (1993-2006) England & Wales

Cost (£millions) (%)Policy, Content & Publicity 24 12Support Services 13 6Geography 7 3Data Collection 84 42Data Capture & Processing 61 29Downstream Processing & IT 11 5Output Policy & Production 7 3Total 207 100

Source: 2001 General Report

# # #

Production costs of 1991 & 2001 SARs

Production costs of 1991 & 2001 SARs (£ millions)

1991 20011991 Individual & Household SARs 0.198

Individual SAR 0.079SAM (Small Area Microdata) 0.068Special License Household SAR 0.022CAMS - Individual, Household 0.011

Total 0.198 0.180

Source: ONS Census Division

Outline costs of 2011 SARs(borne by Census Offices & ESRC)

Outline costs of 2011 SARs (£ millions)

SARs Products Lower UpperPolicy & Design (2009-12) 0.100 0.1504 SARs files - Production (2013) 0.120 0.160User support (2013-2018) 0.800 1.200Total 1.020 1.510

Plus similar support for a Microdata Laboratory Facility?

In summary

• Amount invested in 2011 Census: £568m• Proven benefits of 1991 & 2001 SARs files• Cost of SARs files & support: £1.0-1.5m• Similar support for a laboratory facility• Approximately 0.5% of the total Census budget• “Statistics for the public good”• Maximise the returns on the Census investment,

given that the marginal costs of disseminating outputs are so small

• Some specific recommendations…

Recommendations

1. The Census offices should create a family of SARs products

2. UKSA should develop more sophisticated measures of practical Utility that can be weighed against those for Risk

3. The Census Offices should reflect on the weighing of Risk and Utility for 2001 SARs, and whether this might be shifted a little for 2011

4. The Census Offices should develop a roadmap / plan for SARs to achieve their full potential

Keith DugmoreDemographic Decisions Ltd.

Tel: 020 7834 0966

Email: dugmore@demographic.co.uk

Web: www.demographic.co.uk