Centraal Bureau voor de Statistiek Vincent Ohm Vincent Ohm National Accounts National Accounts Statistics Netherlands (CBS) Statistics Netherlands (CBS) contribution for WPNA meeting contribution for WPNA meeting OECD, Paris - 28 October 2011 OECD, Paris - 28 October 2011 Automation in National Accounts: Automation in National Accounts: applied to SNA 2008 revision of applied to SNA 2008 revision of time series for Supply and Use Tables time series for Supply and Use Tables
contribution for WPNA meetingcontribution for WPNA meeting
OECD, Paris - 28 October 2011OECD, Paris - 28 October 2011
Automation in National Accounts:Automation in National Accounts:applied to SNA 2008 revision of applied to SNA 2008 revision of
time series for Supply and Use Tablestime series for Supply and Use Tables
Centraal Bureau voor de Statistiek
OutlineOutline
• ObjectiveObjective• Business case for automationBusiness case for automation• Semi-automatic integration in National AccountsSemi-automatic integration in National Accounts
- recent developments at Statistics Netherlands (CBS)- recent developments at Statistics Netherlands (CBS)
- basic outline of methodology (without technicalities)- basic outline of methodology (without technicalities)• Application : ‘Time series machine’Application : ‘Time series machine’• Practical aspectsPractical aspects
Centraal Bureau voor de Statistiek
ObjectiveObjective
Implementation of SNA 2008 changes in the NA ‘core systems’ (SUT & Implementation of SNA 2008 changes in the NA ‘core systems’ (SUT & Sector Accounts) and satellite accounts for the Sector Accounts) and satellite accounts for the completecomplete time series time series
CoverageCoverage::
– Supply and Use Tables (and IOT) : 1969-2009Supply and Use Tables (and IOT) : 1969-2009
Business case for automationBusiness case for automation
Need for efficiency -Need for efficiency - limited capacity for revision of time series limited capacity for revision of time series
• Staff reduction at CBS 2010-2012 : NA department: approx. -20% in Staff reduction at CBS 2010-2012 : NA department: approx. -20% in number of staff. Additional reductions up to 23% in budget expected for number of staff. Additional reductions up to 23% in budget expected for CBS for 2013-2016 CBS for 2013-2016
• Compiling time series for SNA93 proved to be very time consuming and Compiling time series for SNA93 proved to be very time consuming and costly (costly (8 working years8 working years!)!)
New methodology and software - New methodology and software - available since 2010available since 2010
• developed within the framework of the ‘developed within the framework of the ‘redesign chain of economic redesign chain of economic statisticsstatistics’ programme at CBS’ programme at CBS
Centraal Bureau voor de Statistiek
Recent developments at CBSRecent developments at CBS
• ‘ ‘Redesign chain of economic statisticsRedesign chain of economic statistics’ programme at CBS: initiated in ’ programme at CBS: initiated in 2007 and now nearly completed (beginning 2012)2007 and now nearly completed (beginning 2012)
• Aims to increase Aims to increase tranparency, quality and efficiencytranparency, quality and efficiency in the complete in the complete ‘chain’ of economic statistics : from ‘source’ (data collection) to statistical ‘chain’ of economic statistics : from ‘source’ (data collection) to statistical ‘end products’ (among which is National Accounts)‘end products’ (among which is National Accounts)
• Within the National Accounts department :Within the National Accounts department : focus on redesign of the focus on redesign of the production of NA ‘core systems’ to implement the new CBS process production of NA ‘core systems’ to implement the new CBS process architecturearchitecture
Centraal Bureau voor de Statistiek
CBS process architecture (CBS process architecture (selectionselection) - applied to NA production) - applied to NA production
manuallymanually automatedautomated
source datasource datafor NA systemfor NA system
without large without large discrepanciesdiscrepancies
integrated and integrated and fully consistentfully consistent
systemsystem
Expert Expert knowledge knowledge
requiredrequired
Laborious Laborious and and
tedioustedious
Incident Incident basedbased
Rule Rule basedbased
‘‘rest point’rest point’
Centraal Bureau voor de Statistiek
Automated integration in National Accounts (1)Automated integration in National Accounts (1)
• Since 2005 benchmarking of Quarterly Sector Accounts (QSA) to annual Since 2005 benchmarking of Quarterly Sector Accounts (QSA) to annual totals was achieved using a Matlab implementation of a totals was achieved using a Matlab implementation of a multivariate multivariate Denton methodDenton method which included lineair constraints and weights for the which included lineair constraints and weights for the individual variables in the modelindividual variables in the model
• This implementation however proved to be This implementation however proved to be inadequate for Quarterly inadequate for Quarterly AccountsAccounts (QNA)(QNA) with respect to the maximum problem size and the with respect to the maximum problem size and the relations between the variables: only lineair constraints availablerelations between the variables: only lineair constraints available
• Since 2008 an Since 2008 an extendedextended version of the multivariate Denton method was version of the multivariate Denton method was developed by colleagues of the R&D department and subsequently the developed by colleagues of the R&D department and subsequently the new methodology was implemented by the IT department into new methodology was implemented by the IT department into software software modules: ‘Quarterly machine’modules: ‘Quarterly machine’
Centraal Bureau voor de Statistiek
Automated integration in National Accounts (2)Automated integration in National Accounts (2)
• Since 2009 additional methodology was developed for balancing NA Since 2009 additional methodology was developed for balancing NA systems for a systems for a singlesingle quarter or year based on Stone’s method. The quarter or year based on Stone’s method. The methodology and implementation strongly resemble that of the Quarterly methodology and implementation strongly resemble that of the Quarterly machine: machine: ‘Balancing machine’ ‘Balancing machine’ which will not be discussed herewhich will not be discussed here
• Since 2010 the new ‘machines’ (methodology & software modules) have Since 2010 the new ‘machines’ (methodology & software modules) have been used in been used in QNA & SUT production processQNA & SUT production process, and will also be , and will also be implemented for QSA & Sector Accounts beginning 2012 (currently in implemented for QSA & Sector Accounts beginning 2012 (currently in testing phase)testing phase)
• In addtion In addtion new applicationsnew applications are being developed, for example for the are being developed, for example for the Supply and Use Table time series discussed hereSupply and Use Table time series discussed here
Centraal Bureau voor de Statistiek
Outline of Quarterly machine methodology (1)Outline of Quarterly machine methodology (1)
• Input data: Input data: N ‘periods’ N ‘periods’ consisting ofconsisting of M ‘subperiods M ‘subperiods’, typically we would ’, typically we would have N=3 years and M=4 quarters (or N=9 and M=1 for Time series)have N=3 years and M=4 quarters (or N=9 and M=1 for Time series)
• Quarterly data Quarterly data for each variablefor each variable (e.g. each ‘cell’ in the Supply and Use (e.g. each ‘cell’ in the Supply and Use Table) is viewed in the time direction as a Table) is viewed in the time direction as a series series for which for which Denton’s Denton’s method method (1971) is applied (i.e. quadratic minimization of first differences in (1971) is applied (i.e. quadratic minimization of first differences in the series: either proportionally or additively)the series: either proportionally or additively)
• The benchmarking (or balancing) problem is formulated as a The benchmarking (or balancing) problem is formulated as a quadratic quadratic (convex) optimization problem(convex) optimization problem which is solved numerically using the which is solved numerically using the XPress MP package (XPress MP package (nono technical/mathematical technical/mathematical detailsdetails provided here!) provided here!)
• Each individual seriesEach individual series can be assigned a can be assigned a weightweight for the ‘ for the ‘Denton termsDenton terms’ in ’ in the optimization problem: these weight reflects the reliability of the source the optimization problem: these weight reflects the reliability of the source data of the series data of the series with respect to other series with respect to other series in the systemin the system
Centraal Bureau voor de Statistiek
Outline of Quarterly machine methodology (2)Outline of Quarterly machine methodology (2)
We can define We can define constraintsconstraints on series and on series and between different seriesbetween different series::
• ‘ ‘hard’ linear constraints: hard’ linear constraints: A + B (+…) = c (c = constant)A + B (+…) = c (c = constant) exact exact• ‘ ‘soft’ linear constraints: soft’ linear constraints: A + B (+…) A + B (+…) c c approximately (weight)approximately (weight)• ‘ ‘hard’ inequalities:hard’ inequalities: A + B (+…) > c A + B (+…) > c exact (also for exact (also for < c< c))• ‘ ‘soft’ ratios:soft’ ratios: A / B A / B r (r = constant) r (r = constant) approximately approximately (weight)(weight)• set upper and lower bounds for individual variables set upper and lower bounds for individual variables (subperiods)(subperiods)• set ‘exogeneous’ (fix) individual variables set ‘exogeneous’ (fix) individual variables (subperiods)(subperiods)• impose periodical data on subperiodical data impose periodical data on subperiodical data (exact or approximately)(exact or approximately)
i.e.i.e. q1 + q2 + q3 + q4 q1 + q2 + q3 + q4 == y y or:or: q1 + q2 + q3 + q4 q1 + q2 + q3 + q4 y y (weight)(weight)
Note:Note: In the above A and B can be linear combinations of other variables. In the above A and B can be linear combinations of other variables.
• The bounding years: 2009, 2001, 1995, 1987 ,1979, 1969 are called The bounding years: 2009, 2001, 1995, 1987 ,1979, 1969 are called interface yearsinterface years (in Dutch: steekjaren) which mark the original years where (in Dutch: steekjaren) which mark the original years where previous SNA revisions have been first implemented (in particular 1987, previous SNA revisions have been first implemented (in particular 1987, 1995, 2001 and now 2009). Best data available for these years!1995, 2001 and now 2009). Best data available for these years!
Centraal Bureau voor de Statistiek
Application: Time series for SUT (2)Application: Time series for SUT (2)
• SNA 2008SNA 2008 conceptual changesconceptual changes as well as incidental corrections of errors as well as incidental corrections of errors will be implemented will be implemented manually manually by our NA experts for by our NA experts for all interface yearsall interface years
• Note: Note: There is an There is an extra stepextra step where we first where we first balance the interface yearsbalance the interface years using the balancing machine (not discussed here!) and some manual using the balancing machine (not discussed here!) and some manual corrections where necessary: the result are fully balanced interface years corrections where necessary: the result are fully balanced interface years once these once these are approved are approved by our expertsby our experts
• The The balanced interface yearsbalanced interface years then serve as then serve as fixed boundariesfixed boundaries between between which we will which we will interpolateinterpolate the intermediate years using the the intermediate years using the ‘Time Series ‘Time Series Machine’Machine’ (which is a special application of the Quarterly machine) (which is a special application of the Quarterly machine)
Centraal Bureau voor de Statistiek
Application: Time series for SUT (3)Application: Time series for SUT (3)
• Interpolation by the Time Series Machine will be achieved for Interpolation by the Time Series Machine will be achieved for currentcurrent and and previous year pricesprevious year prices simultaneouslysimultaneously such that price and volume such that price and volume indices indices of the original seriesof the original series will be preserved as much as possible will be preserved as much as possible whereas whereas new levelsnew levels are given by the interface years are given by the interface years
• All All necessary rulesnecessary rules (relations between series as discussed before) as (relations between series as discussed before) as well as well as weightsweights will be set to achieve an optimal result, which however will be set to achieve an optimal result, which however will will have to be approvedhave to be approved of by our NA experts before it is accepted of by our NA experts before it is accepted
• Revision of the time series is therefore an Revision of the time series is therefore an iterative processiterative process, which , which however requires only however requires only limited capacity per iterationlimited capacity per iteration once the model setup once the model setup has been programmed: however has been programmed: however incidentallyincidentally small small adjustments in the adjustments in the rulesrules or or manual intervention (corrections) by expertsmanual intervention (corrections) by experts may be necessary! may be necessary!
Centraal Bureau voor de Statistiek
Application: Time series for SUT (4)Application: Time series for SUT (4)
• The result of The result of one iterationone iteration of the Time series machine is a of the Time series machine is a complete complete (sub)series(sub)series of Supply and Use Tables which are internally of Supply and Use Tables which are internally fully consistentfully consistent and which are matched to the and which are matched to the new levels set by the interface yearsnew levels set by the interface years for for which which SNA 2008 conceptsSNA 2008 concepts have been manually implemented have been manually implemented
• Once the Once the resultsresults of the Time series machine have been of the Time series machine have been carefully carefully examined and approvedexamined and approved by our experst for one (sub)series, the results will by our experst for one (sub)series, the results will be loaded into the database and we will proceed with the next subseries, be loaded into the database and we will proceed with the next subseries, working back in timeworking back in time (so first 2001-2009, then 1995-2001, etc.) (so first 2001-2009, then 1995-2001, etc.)
• After the previous SNA revision (2001) the time series had to be After the previous SNA revision (2001) the time series had to be manually adjusted and balanced year by year manually adjusted and balanced year by year which was very laborious which was very laborious and tedious and which has cost and tedious and which has cost 8 working years8 working years in capacity in capacity (klopt dit???)(klopt dit???)
Centraal Bureau voor de Statistiek
Application: Time series for SUT (5)Application: Time series for SUT (5)
• Although Although revision for the year 2009revision for the year 2009 and then for the and then for the interface years interface years (2001, 1995, etc.)(2001, 1995, etc.) will start by the end of this year, actual work on the time will start by the end of this year, actual work on the time series (using the Time series machine) will start around June 2012series (using the Time series machine) will start around June 2012
• Will this new semi-automated approach work?Will this new semi-automated approach work?
• Although we have not tested the Time series machine Although we have not tested the Time series machine for SNA revisionfor SNA revision of time series before, we are of time series before, we are confidentconfident the approach will be successful: the approach will be successful:
• We have been using the We have been using the Quartely machineQuartely machine for benchmarking quarterly for benchmarking quarterly data to annual totals for SUT for the last two years (2010 & 2011) and we data to annual totals for SUT for the last two years (2010 & 2011) and we have gained sufficent experience using the new software and have gained sufficent experience using the new software and methodology. And compiling the time series is a very similar problem.methodology. And compiling the time series is a very similar problem.
Centraal Bureau voor de Statistiek
Application: Time series for SUT (6)Application: Time series for SUT (6)
• Also we are currently using a (somewhat simplified) version of the Time Also we are currently using a (somewhat simplified) version of the Time series machine for series machine for implementing the new CPA classificationimplementing the new CPA classification into our SUT into our SUT time series, where the approach is essentially the same: time series, where the approach is essentially the same: interpolate interpolate between manually correctedbetween manually corrected and balanced and balanced (fixed)(fixed) interface yearsinterface years
• This time series approach works quite well and the problem size is This time series approach works quite well and the problem size is feasible. However for the SNA 2008 revisions we feasible. However for the SNA 2008 revisions we expect larger changes in expect larger changes in the levelsthe levels of the interface years, which may imply that of the interface years, which may imply that more manual more manual interventionsinterventions and and more iterationsmore iterations for each subseries will be needed to for each subseries will be needed to converge to an optimal end resultconverge to an optimal end result
• Lastly we have Lastly we have capable & helpful colleaguescapable & helpful colleagues both on the both on the technical/methodological aspects as well as on the National Accounts technical/methodological aspects as well as on the National Accounts subject matter and data so we sleep soundlysubject matter and data so we sleep soundly
Centraal Bureau voor de Statistiek
Some practical aspectsSome practical aspects
Centraal Bureau voor de Statistiek
Questions?Questions?
Centraal Bureau voor de Statistiek
Thank you for listeningThank you for listening
For additional information about this subject or theFor additional information about this subject or the
automation methodology (‘machines’) we have developed at CBS, automation methodology (‘machines’) we have developed at CBS,
you are welcome to contact me at: you are welcome to contact me at: [email protected]@cbs.nl
