STS Compilation with Multiple Data Sources

Anu PeltolaEconomic Statistics Section, UNECE

UNECE Workshop on Short-Term Statistics (STS) UNECE Workshop on Short-Term Statistics (STS) and Seasonal Adjustmentand Seasonal Adjustment14 – 17 March 2011, Astana, Kazakhstan

March 2011 UNECE Statistical Division 2

Overview

Data collection• Sampling • Administrative data• Combining multiple data sources

Compilation of results• Data editing• Non-response and weighting• Treatment of non-comparable changes

Publication Improvement

March 2011 UNECE Statistical Division 3

Theoretical Concept – A Key to Good Quality

Define the purpose of an indicator Links to the real world

• What should it describe? • Who are the users/uses (internal/external)?• Possible data sources

Links to other statistics• Differences in concepts, scope, methods• Goal variables – national accounts/SBS• Regular benchmarking • Follow-up of differences

Act Plan

Check

Do

Continuous improvement Q

uality

TimeBy Deming

March 2011 UNECE Statistical Division 4

Production Process

Bring the collected data to the level of the intended statistical output!

PublicationCollection

of data

Correction of systematic errors

in data

Index calculation

Check for the most important observations

March 2011 UNECE Statistical Division 5

Data Collection

Statistical Units

Corner stones of business statistics• Legal unit -> enterprise (services) -> enterprise groups• Establishment (for industry/construction)

Business registers are fundamentally important• Bridge between administrative and statistical units• Definition of the economic activity class (ISIC/NACE)• Improve its comprehensiveness – use as a frame• Examine opportunities to use administrative data• Interactive: update with information from STS

UN: International recommendations for the Index of Industrial Production &EC: STS Metholodological manual

March 2011 UNECE Statistical Division 6

Source: Statistics Finland, Strategy for economic statistics

System of Statistics

March 2011 UNECE Statistical Division 7

Give clear instructions• Explain the concepts to the respondents

Revisions to earlier months• Aim to pre-fill the questionnaire with data given earlier • Leave space for reporting revisions

Always test changes to questionnaires Inform the respondents of the use of data Develop useful feedback for respondents

• your company compared to others in the same activity

Data Collection

Questionnaire Design

March 2011 UNECE Statistical Division 8

Many surveys are for units above a size threshold• Burdensome and problems with the coverage of small units

Based on business register and periodically reviewed In drawing a sample, special attention to be paid to:

• Level of details to be published• Resources available• Accuracy and timeliness required• Response burden

Simple/stratified sampling by activity and size

Data Collection

Sampling in Practice

March 2011 UNECE Statistical Division 9

> Business Register to be kept up-to-date with new units

Total population of unitsin the Business Register

Large units Medium units Small units

Stratification by economic activity

Covered on a complete

enumeration basis

Covered by sampling

Covered mainly by

administrative sources

or administrative sources

March 2011 UNECE Statistical Division 10

Administrative registers or datasets can be used as:• Single source in their own right• Frame for sampling via the Business Register• Complementary source• Validation• Data source for small enterprises

For STS limited administrative sources available:• VAT (value added tax)• Social security data (employment and labor cost)• Building permits, etc.

Data Collection

Administrative Data Sources

March 2011 UNECE Statistical Division 11

+ Reduction of response burden

+ Reduction of costs, data collection and manual work

+ Total populations - detailed classifications/regional indicators

+ Better quality and coverage (of smallest units)

- Data content, units, concepts and definitions may differ- Dependence on few large data suppliers- Timeliness - may require use of estimation- Access and confidentiality- Non-observed economy unlikely to be included- Requires good IT capacity by the supplier and the NSO

Data Collection

Pros and Cons of Admin Data?

March 2011 UNECE Statistical Division 12

National ID-system for enterprises New production methods:

• to correct for negative values and different concepts• slow accumulation > estimation of missing data

The most important units to direct collection• Active co-operation with large enterprises

Development of questionnaires:• Simplification – part of information from registers• Efficiency – electronic data collection

Data Collection

Administrative Data and Quality

March 2011 UNECE Statistical Division 13

Compulsory to use existing data (if suitable) in statistics production

Guaranteed access to administrative sources State government and social security institutions

obliged to deliver their data to the NSO• Free of charge or compensation of direct costs• Co-operation in making changes in data collection

To ensure data confidentiality• Individual data collected for statistics should not be

handed over to any use other than statistics or research!

Data Collection

Legislative Issues

March 2011 UNECE Statistical Division 14

Compilation

Central Role of VAT Data

Source: Statistics Finland

March 2011 UNECE Statistical Division 15

VATe.g. 250 000 units

• Turnover• Estimates for output

and missing data

Business Registere.g. 290 000 units

• Unit IDs• Activity code

• Location• Mergers

• LKAU (regional)

1. release

2. release

revision

optimalsampling

small & mediumenterprises

feedbackto BR

Samplee.g. 2000 units

• Turnover• Mergers

sample,basic info

updates to BRactivity of units

combining

Compilation

Linking Admin and Survey Data

March 2011 UNECE Statistical Division 16

Compilation

Data Control and Editing

Studying data to identify errors• Detect errors that have a significant influence• Check whether values are within given ranges • Check whether values for related variables are coherent• Compare to past responses (previous months and a year ago)

Give top priority to outliers and errors that have the largest impact on the results

Outlier values require careful treatment• May be correct but caused by unusual circumstances

Source: Methodology of Short-Term Business Statistics, EC

March 2011 UNECE Statistical Division 17

Compilation

Treating Non-Response

Controlling response burden• Better planning of data collection process• Offering various channels for respondents

Reducing the effect of non-response• Alternative source, e.g. administrative data• Imputation based on historical data• Mean value imputation, donor/nearist neighbour,

regression of variables

March 2011 UNECE Statistical Division 18

Compilation

Comparing Unit Level Data

0

10000

20000

30000

40000

50000

60000

70000

80000

1 2 3 4 5 6 7 8 9 10 11 12Months

Previous year Current year

Change 115%

March 2011 UNECE Statistical Division 19

-2.40

-1.13-0.58

-2.33

-2.53

-4.1229.70

-1.20

60

80

100

120

140

160

180

1 2 3 4 5 6 7 8Months

Index without a unit Index with a unit

Compilation

Impact on the Results

index

March 2011 UNECE Statistical Division 20

Compilation

Non-Comparable Changes (NCCs)

Structural changes in the population: • New units are set up and others stop existing• Units may be taken over, merged or split up • Units may expand, contract or change their activities

Reasons for large changes1) Errors

2) Actual changes that are comparable

3) Actual changes that are non-comparable UN Guide on the Impact of Globalization on

National Accounts > helps with STS as well

March 2011 UNECE Statistical Division 21

Compilation

Example of NCCs

Unit ATurnover = 100 million

Unit BTurnover = 75 million

Exchange of goods50 million

Turnover drops by one third due to a merger!No change in the level of activity!

Unit ABTurnover =

(100-50) + 75 = 125 million

Previous year Current year

March 2011 UNECE Statistical Division 22

Compilation

Alternative Treatments of NCCs

1. All changes are recorded as they are (actual)− Contaminated with apparent, non-comparable changes− Difficult to obtain a picture of economic reality+ Simplicity

2. Panel method• Only same units in both periods are included− Start-ups and closures would be cancelled out− Seriously biased results in highly dynamic populations+ Simplicity

March 2011 UNECE Statistical Division 23

Compilation

Alternative Treatments of NCCs

3. Overlapping method• Actual comparable changes are not adjusted• Other changes are made comparable by

a. Collecting comparable information (largest units)

b. Replacing non-comparable figure by an estimate

c. Taken the unit out of calculation (no effect to results)

− Requires more work+ Results reflect actual changes in economic

activity

Firm X

Firm X

March 2011 UNECE Statistical Division 24

Compilation

Confrontation with Other Sources

Regular confrontation may reveal discrepancies

Aim at coherence:value = price x output

First at the aggregated level and where necessary at lower levels (largest units)

Knowledge of differences between statistics helps communication with users

Quality reviews of indicators to be undertaken

March 2011 UNECE Statistical Division 25

New Requirements for STS?

Globalization• Internationally comparable data needed• Treatment of more complex business activities

Increasing amount of services• Output and price measures, industrial services

Detection of turning points• Longer time series and seasonal adjustment

Coherence• Compare to National Accounts and

between price/volume/value indicators