Snapshot Tool for Implementation of ACSS Code of Practice
Self-Assessment Measures
Presentation by ASEAN Secretariat
Seminar on "Strengthening national statistical systems to meet the challenges
of the UN SDGs"
Bangkok, Thailand, 13-14 December 2016
Outline of Presentation
o Background: ASEAN Community Statistical System (ACSS) Code of Practice (CoP)
o Snapshot Tool o Pilot Exercise o Result of the Pilot Exercise o Conclusions
2
Background
The ACSS Code of Practice (CoP) was adopted at the 2nd Session of the ASEAN Committee Statistical System (ACSS) Committee in Sep 2012
The ACSS CoP is consistent with the Fundamental
Principles of Official Statistics adopted by the UN Statistical Commission in 1994, and reflects the values stipulated in the ACSS Strategic Plan
The 2nd ACSS Committee agreed to implement the 8
principles in the CoP and to conduct self-assessments based on the indicators in the CoP
3
Background These principles are subsumed in 3 major dimensions:
A. Institutional Environment
1. Mandate for data collection 2. Professionalism and integrity 3. Confidentiality 4. Accountability 5. Statistical Cooperation and Coordination
B. Statistical Process 6. Cost Effectiveness 7. Reduced Respondent Burden
C. Statistical Output 8. Commitment to Quality (Relevance, Reliability, Timeliness, Comparability,
Accessibility)
Emphasised that Quality Monitoring Tool (Snapshot) should be in line with the ACSS CoP and assessment measures agreed by the ACSS Committee (for Section A, B and C)
4
The Snapshot Tool
The ‘Snapshot tool’ quality framework was developed by the European Statistical System to assess key aspects of the national statistical systems (institutional environment); as well as in different sectors, specific sector statistics and key indicators for these sectors (statistical output) by asking a series of clear questions with pre-defined answers using Microsoft Excel®.
5
The Snapshot Tool
How assessment is done The pre-defined answers automatically give an intuitive
colour code for good (green), average (orange) and poor (red) achievement for the issue concerned.
Not complying with quality requirements
Partially complying with quality requirements
Fully complying with quality requirements
The Snapshot Tool
How assessment is done This is complemented by an arrow indicating a positive,
unchanged or negative on-going or expected development.
This approach provides an intuitive and easy-to-interpret assessment of the national statistical system or of sector statistics, which is visible at first glance.
The Snapshot Tool
The tool consists of two modules: Module 1: assesses the functioning of the national
statistical system (NSS) as a whole, with a special focus on institutional, legal and organizational aspects, (to be
implemented to CoP Section A). Module 2: assesses the performance of the statistical
system in a specific sector (such as trade in goods, foreign
direct investment, trade in services,…). In this module, there is a special complement to assess the quality of specific sector indicators, (to be implemented to CoP Section B
and C).
8
The Exercise
Purpose of the exercise To test the questionnaire
o this exercise is to test the appropriateness of the self-assessment questionnaire on the implementation of the code of practice for use as an input to the quality report.
To prepare an ASEAN quality report o the ASEAN quality report is to collect in one single place all kind of
information relating to the quality of national statistics but also to present the quality of aggregated estimates at the ASEAN level.
The Exercise
Schedule of Pilot Implementation of the CoP Section A, B and C self assessment using the Snapshot Tool: January 2015: Workshop on Advancing Quality Monitoring April-May 2015: finalization of the self-assessment tool using the snapshot format June-July 2015: training session for AMS and ASEANstats on the use of self-assessment tool July-October 2015: country self-assessments November 2015: Reporting result on assessment to the Committee End of 2015: Regional exchange and discussion on the results of the self-assessments and on the revision of the tool
Jan-Oct 2016 : Pilot exercise for CoP Section B and C (discussed during the Working Group meeting)
10
Result: AMS’ Self-Assessment of Section A Using Snapshot Tool
All 10 National Statistical Office of the Member States participated in the pilot exercise (Thank you, AMSs!!!)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
KP1 KP2 KP2 KP4 KP5
In general, very good application of the 5 Key Principles of the ACSS CoP at the regional level
Result: AMS’ Self-Assessment of Section A Using Snapshot Tool
DETAILED RESULTS PER KEY PRINCIPLE (EXAMPLE KP 1)
KEY PRINCIPLE 1: MANDATE FOR DATA COLLECTION
88%
12%
1.1 THE MANDATE OF THE NATIONAL STATISTICAL AUTHORITIES OF THE AMSS TO COLLECT INFORMATION FOR THE
PRODUCTION AND DISSEMINATION OF OFFICIAL STATISTICS IS STIPULATED IN NATIONAL LAWS AND/OR NATIONAL LEGAL
FRAMEWORKS.
100%
1.1.1 IS THE MANDATE TO COLLECT INFORMATION FOR THE PRODUCTION AND
DISSEMINATION OF OFFICIAL STATISTICS STIPULATED IN AMS’ NATIONAL LAWS AND/
OR NATIONAL LEGAL FRAMEWORKS?
100%
1.1.2 IS THE OBLIGATION TO FURNISH INFORMATION REQUISITIONED BY THE
NATIONAL STATISTICAL OFFICE STIPULATED IN THE AMS’ NATIONAL LAWS AND/ OR
NATIONAL LEGAL FRAMEWORKS?
100%
1.2 OFFICIAL STATISTICS PRODUCED BY NATIONAL STATISTICAL AUTHORITIES OF THE AMSS ARE PROVIDED TO THE
ASEANSTATS FOR THE COMPILATION OF ASEAN STATISTICS, UNDER THE COORDINATION OF THE NATIONAL
STATISTICAL OFFICES.
80%
20% 1.2.1 HAS THE NATIONAL STATISTICAL OFFICE APPOINTED A COUNTRY FOCAL POINT TO
COORDINATE THE PROVISION OF OFFICIAL STATISTICS TO ASEANSTATS?
100%
1.2.2 ARE ACSS STATISTICAL WORK PROGRAMMES AND ACTIVITIES INCORPORATED IN THE
AMS’ NATIONAL STATISTICAL PROGRAMMES AND ACTIVITIES, WHERE RELEVANT?
70%
30% 1.2.3 ARE THERE AGREED DATA TEMPLATE/ PROTOCOL AND DATA DISCLOSURE GUIDELINES/
POLICIES TO HELP THE NATIONAL STATISTICAL OFFICE COORDINATE THE COMPILATION AND
PROVISION OF OFFICIAL STATISTICS TO ASEANSTATS?
70%
30%
Result: SITS: Assessment Measures Sections B (Statistical Process) and C (Statistical Output)
Detailed questions
Key Principle 6: Cost Effectiveness and Adequacy of Resources 60 40 0 0
6.1 Procedures exist to optimise resources utilised in the production and dissemination of ASEAN statistics.
70 30 0 0
6.1.1 Are costing and budgeting procedures in place in AMS and ASEANstats to optimise resources utilised?
70 30
6.3 Productivity potential of information and communications technology is being optimised for data collection, production and dissemination.
50 50 0
6.3.1 Are sufficient resources allocated by AMS and ASEANstats to exploit the full potential of information and communications technology for data collection, production and dissemination?
50 50
6.4 Staff, financial, and technological resources are available both in magnitude and in quality to meet the current needs for ASEAN statistics.
60 40 0 0
6.4.1 Are adequate staff, financial, and technological resources available in AMS and ASEANstats to meet the current needs for ASEAN statistics?
60 40
Result: SITS: Assessment Measures Sections B (Statistical Process) and C (Statistical Output)
Key Principle 8: Commitment to Quality 67 22 3 8
8.3 Timeliness - A release calendar is established for ASEAN statistics at both national and ASEAN levels, and the periodicity of statistics released takes into account user requirements as much as possible.
80 15 0 5
8.3.1 Is an advanced release calendar established by AMS for national statistics and ASEANstats for ASEAN statistics?
80 10 10
8.3.2 Do the periodicity and timeliness of statistics released by AMS for national statistics and ASEANstats for ASEAN statistics follow dissemination standards?
80 20
8.4 Comparability - ASEAN statistics are comparable, and detailed concordance as set by international.
55 25 0 20
8.4.1 Are ASEAN statistics consistent or reconcilable over a reasonable period of time?
40 20 40
8.4.2 Is detailed concordance available as set by international standards between AMS’ national classification systems and the corresponding ASEAN classification systems?
70 30
Result: SITS: Assessment Measures Sections B (Statistical Process) and C (Statistical Output)
8.5 Accessibility - ASEAN statistics are produced and made readily available through dissemination mechanisms utilising modern information and communication technology, and where appropriate, traditional hard copy.
72 18 4 6
8.5.1 Are statistics disseminated by AMS and ASEANstats in forms that meet users’ needs, utilising appropriate information and communication technology?
70 30
8.5.2 Are unpublished statistics that can be disseminated made available by AMS and ASEANstats to users upon request?
50 40 10
8.5.3 Are contact points for AMS and ASEAN statistical releases clearly identified? 90 10
8.5.4 Are updated catalogues of publications, documents and other services to users maintained by AMS and ASEANstats?
70 10 10 10
8.5.5 Are prices of the statistical products and services, where appropriate, clearly disclosed by AMS?
80 10 10
Detailed questions ASEAN
Key Principle 8: Commitment to Quality
8.1 Relevance - ACSS is guided by the statistical requirements reflected in the ASEAN Community Blueprints and user-producer consultations.
8.1.1 Is ASEANstats able to identify major data users and conduct periodical user consultation/ feedback on the statistical requirements reflected in the ASEAN Community Blueprints?
↗
8.1.2 Does ASEANstats regularly update ACSS Committee on ASEAN regional policy issues and initiatives?
↗
8.1.3 Are the requirements prioritised by the ACSS Committee and appropriately incorporated in the ACSS annual work plans and multi-year action plans?
↗
8.1.4 Is user feedback sought on ASEAN statistics disseminated by ASEANstats? →
Result: SITS: Assessment Measures Sections B (Statistical Process) and C (Statistical Output)
Result: SITS: Assessment Measures Sections B (Statistical Process) and C (Statistical Output)
Detailed questions ASEAN
Key Principle 8: Commitment to Quality
8.2 Reliability - Monitoring mechanisms are in place to regularly review and improve the quality of the statistical processes and products.
8.2.1 Are periodic reviews of statistical activities conducted by AMS and ASEANstats to improve quality of statistical processes and products?
↗
8.2.6 Is a regular and transparent schedule established by AMS and ASEANstats for data revisions?
↗
8.2.7 Are preliminary and / or revised data clearly identified by AMS and ASEANstats? →
8.2.8 Are revision studies undertaken by AMS and ASEANstats on a regular basis? →
Result: SITS: Assessment Measures Sections B (Statistical Process) and C (Statistical Output)
Detailed questions ASEAN
Key Principle 8: Commitment to Quality
8.5 Accessibility - ASEAN statistics are produced and made readily available through dissemination mechanisms utilising modern information and communication technology, and where appropriate, traditional hard copy.
8.5.1 Are statistics disseminated by AMS and ASEANstats in forms that meet users’ needs, utilising appropriate information and communication technology?
↗
8.5.2 Are unpublished statistics that can be disseminated made available by AMS and ASEANstats to users upon request?
→
8.5.3 Are contact points for AMS and ASEAN statistical releases clearly identified? →
8.5.4 Are updated catalogues of publications, documents and other services to users maintained by AMS and ASEANstats?
→
Conclusions
Findings from the exercise Snapshot tool is a useful tool, simple, user friendly Easy to understand results (in the future) Initial framework for peer review Observations to improve the self-assessment questionnaire Need to reformulate the questions in order to get useful and
appropriate comments Build as far as possible quantitative indicators Add quality indicators: i.e completeness (information provided by
AMS compared with relevant ASEAN guidelines: calculation of indicators
of completeness); reliability (validation checks at the national and
ASEANstat level), timeliness …
Participative process: in national as well as regional (some of the
questions need to be answered together by AMS and by ASEANstats).