Eurostat Eurostat
ESS Vision 2020 ADMIN: Assessing Quality
Fabian BACH
ADMIN project manager
Eurostat Unit F.2 – Population and migration
UNECE-Eurostat Expert Meeting on Censuses
Geneva, 27 September 2018
ADMIN data in multisource statistics
Lin
kag
e
Sta
tistic
al
reg
iste
r
Outputs
Po
pu
latio
n
Reg
iste
r
Tran
sfo
rm
atio
n
`
use for survey frames: directly or to supplement / update
(partly) replace data collection: e.g. business tax data survey
direct tabulation
Graph idea: ONS
Registers
Admin. data
Surveys
Full enumeration
ADMIN data in multisource statistics
Lin
kag
e
Sta
tistic
al
reg
iste
r
Outputs
Po
pu
latio
n
Reg
iste
r
Tran
sfo
rm
atio
n
`
Graph idea: ONS
Registers
Admin. data
Surveys
Full enumeration
(mostly) national level! (ESS) quality??
Inputs
Key ADMIN challenge
• as a single variable
• editing, imputation, validation
• indirect use
(calibration, estimation, …)
• in evaluation, including data
confrontation
(e.g. comparison of survey
estimates with related ADMIN
estimates)
Generic Statistical
Business Process Model =
Multisource data: use examples
ESS Vision 2020 ADMIN project (2015-2019)
WP2. Statistical methods
WP5. Frames for social statistics
WP6. Pilot studies &
applications
WP7. Methodological
support
WP4. Eurostat as user of EC data
WP1. Access to & development
of sources
Approved by ESSC: support the ESS to make better use of available ADMIN data without compromising on quality
Work
Packages (
WP)
WP3. Quality
ADMIN: key results
Multisource combination +
estimation methods
Access to ADMIN sources
Use of registers for frames
2019 Q2/2018
Quality of frames for social statistics today!
Multisource quality
Guidelines
WP6 grant summary reports social/business/agricultural stats. + quality lessons learned + knowledge transfer
Best practices
Incl. 2017, 2018 waves
ADMIN: key results
Multisource combination +
estimation methods
Access to ADMIN sources
Use of registers for frames
Quality of frames for social statistics today!
Multisource quality
Guidelines
WP6 grant summary reports social/business/agricultural stats. + quality lessons learned + knowledge transfer
Best practices
Incl. 2017, 2018 waves
Stress on social/pop./ census statistics through all ADMIN!
Reports to ESS Directors
(methodology, social statistics)
ADMIN: Involving ESS Members
Quality of multisource statistics (DK, IE, IT, LT, HU, NL, AT, NO)
• Stock-taking • New indicators • Methodological framework quality guidelines (x2)
ADMIN Steering Group
Governance technical
ADMIN network
Task Force Frames
ESSnet KOMUSO
Workshops
ADMIN WP3: Quality (KOMUSO)
Komusō (image: Wikipedia)
ESSnet
Task 3.1 – Checklists for
evaluating the quality of input data
Task 3.2 – Definition of a
framework for the quality evaluation of statistical outputs using administrative data
Task 3.3 – Dissemination and
practical implementation of main results obtained in tasks 3.1 and 3.2 write guidelines!
https://ec.europa.eu/eurostat/cros/content/essnet-quality-multisource-statistics-komuso_en
2015 until 04/2017:
Outputs published:
KOMUSO Stage 1: Preparatory Work
Take stock on output quality assessment +
reporting Review output quality measures
Review current practices on input quality assessment
• checklists for evaluating the quality of input data
• framework: 6 basis data source configurations
• quality measures and indicators (for frames)
Review existing practices on frames
quality
1st ADMIN quality workshop
(Budapest, 04/2016)
6 basis data source configurations:
1) Non-overlapping microdata without coverage problems
2) Overlapping microdata without coverage problems + (2S) including sample surveys
3) Overlapping microdata with under-coverage
4) Combining microdata + macrodata sources
5) Combining only macrodata sources
6) Longitudinal data
Selected to reflect the most common/typical scenarios
KOMUSO Stage 1: Preparatory Work
KOMUSO Stage 2: Guidelines
04/2017 until 09/2018:
Outputs published (continuing stage 1):
Finalise draft frames guidelines
Continue work on measures/indicators
by data config.
1st progress draft of multisource guidelines
• Quality measures and calculation methods (QMCMs)
• Quality quidelines for frames of social stats.: QGFSS v1.0
• Quality guidelines for multisource statistics: QGMSS v0.7
accuracy (18) coherence (3) timeliness (1)
Structure + drafting progress of the QGMSS v0.7:
KOMUSO Stage 2: Guidelines
Structure + drafting progress of the QGMSS v0.7:
KOMUSO Stage 2: Guidelines
…
…
QMCMs (annex)
Same structure for each section in Part 2:
Excursion: structure of QMCMs (annex to QGMSS)
1) Name of the quality measure/indicator
2) General information
3) Quality measure & computation method in general
4) Situation & computation method in detail
5) References
+ hands-on example where needed
KOMUSO Stage 2: Guidelines
E.g. QMCM_A_10 (Accuracy example 10):
KOMUSO Stage 2: Guidelines
… …
1)
2)
3)
E.g. QMCM_A_10 (Accuracy example 10):
KOMUSO Stage 2: Guidelines
… …
(config. 2)
1)
2)
3)
E.g. QMCM_A_10 (Accuracy example 10):
4)
5)
KOMUSO Stage 2: Guidelines
E.g. QMCM_A_10 (Accuracy example 10):
4)
5)
KOMUSO Stage 2: Guidelines
Detailed calculation algorithm/recipe
E.g. QMCM_A_10 + hands-on example:
KOMUSO Stage 2: Guidelines
…
E.g. QMCM_A_10 + hands-on example:
KOMUSO Stage 2: Guidelines
…
11 steps: Raw input data Combinded DB Final data pool
Multisource quality propagation!
KOMUSO Stage 3: Completion
10/2018 until 09/2019:
Outputs envisaged (continuing stages 1+2):
Finalise measures/indicators QMCMs by data config.
Finalise draft multisource
guidelines (QGMSS)
accuracy contd. (+6) coherence contd. reliability (1) + hands-on examples
• QMCMs contd. incl. hands-on examples
• QGMSS v1.0 tentatively June 2019
2nd ADMIN quality workshop
(winter '18 tbc.)
ADMIN support to Member States
• Workshops: quality + social statistics + other
• (WP6: Grants for ADMIN pilots in Member States)
• WP7: Methodological support
ADMIN helpdesk for NSIs http://ec.europa.eu/eurostat/cros/content/ess-admin-helpdesk_en
Coaching for NSIs (on-site expert support) request through ADMIN helpdesk
ESTP training linked to ADMIN in 2017-2019
winter 2018 tbc.
spring 2019 tbd.
Thank you!
CROS portal: http://www.cros-portal.eu/content/essvip-admin
Annex:
6 basis data source configurations
1) Non-overlapping microdata without coverage problems:
KOMUSO Stage 1: Preparatory Work
most simple
still typical errors:
classification
measurement
progressiveness
incl. single source
reference config.
KOMUSO Stage 1: Preparatory Work
2) Overlapping microdata without coverage problems:
configuration (1) +
linkage errors
inconsistencies
most common configuration
KOMUSO Stage 1: Preparatory Work
2S) Overlapping microdata without coverage problems:
incl. samples
configuration (2) +
sampling errors
weights handling
calibration
KOMUSO Stage 1: Preparatory Work
3) Overlapping microdata with under-coverage:
configuration (2) +
under-coverage
may include samples ( 3S)
KOMUSO Stage 1: Preparatory Work
4) Combining microdata + macrodata sources:
configuration (2) +
reconciliation of microdata (calibr.)
KOMUSO Stage 1: Preparatory Work
5) Combining only macrodata sources:
macro-counterpart of configuration (2)
reconcile overlapping data
E.g. supply/use (SU) tables of National Accounts
KOMUSO Stage 1: Preparatory Work
6) Longitudinal data:
reconcile time series
different frequencies and qualities
now-casting (time series model)
linkage errors