The South Africa I know, the home I understand
The role of the National Statistics Office in the Quality Assessment of Crime Administrative Data in South Africa
Joseph Lukhwareni Manager: Crime and Safety Statistics
Statistics South Africa 15 October 2015
The South Africa I know, the home I understand
Rationale behind Quality Statistics
• The broader National Statistics System (NSS) is characterised by three gaps
I. Statistical information gap • Insufficient supply (Quantity problem), especially small area data
II. Data quality gap
• Data of unknown quality (Quality problem) III. Statistical skills gap
• Insufficient statistical skills (Capacity problem) • To address the Quality Gap, the Statistician-General has gazetted through
parliament, the South African Statistical Quality Assessment Framework (SASQAF)
• Statistician-General is required by the Statistics Act (No.6 of 1999) to coordinate statistical production in the country beyond the confines of Statistics South Africa
The South Africa I know, the home I understand
Definition Official statistics’ definition is statutory Practical criteria of official statistics
– Must be used in the public domain – Are from organs of state and other agencies that are partners in the National
Statistics System [NSS] by signing of MoU – Are sustainable – Have met quality criteria as defined by the Statistician-General [SASQAF]
National statistics’ definition is implicitly statutory
Official sta*s*cs are sta's'cs designated as official sta's'cs by the Sta's'cian-‐General within the provisions of the Sta's'cs Act
Na*onal sta*s*cs are sta's'cs not designated as official Sta's'cs by the Sta's'cian-‐General
The South Africa I know, the home I understand
• The purpose of official statistics is to assist organs of state, businesses, other organisations or the public in:
• planning; • decision-making or other actions; • monitoring or assessment of policies, decision-making or other actions.
• Official statistics must protect the confidentiality of the identity of, and the
information provided by, respondents and amongst other be: • relevant, accurate, reliable and timeous; • objective and comprehensive [have integrity and not be biased]; • compiled, reported and documented in a scientific and transparent manner; • disseminated impartially; • accessible;
More detailed information can be found in SASQAF
Purpose of Official Statistics
The South Africa I know, the home I understand
Dimensions of Quality
• Data quality is inherently not homogenous but
instead built on several dimensions of quality.
The South Africa I know, the home I understand
Relevance
Accuracy
Coherence
Timeliness Interpretability Accessibility
Meeting real needs of clients
Correctly describes phenomena it is designed to measure
Info available at desired reference point Ease of obtaining
info from agency Availability of supplementary info and metadata
Harmonisation of different info within broad analytical and temporal framework
Integrity
Free from political interference: Adherence to objectivity, professionalism, transparency, ethical standards
Methodological soundness
Sound methodologies: • International standards and guidelines - good practice • Agreed practices • Dataset-specific
SASQAF
The South Africa I know, the home I understand
Statistical Value Chain
Need Design Build Collect Process
Analyse Disseminate Archive Evaluate
The South Africa I know, the home I understand
Need • Request is evaluated, the project is planned ,a budget is developed, etc. (e.g. proposal to change a definition)
Design • Develop methodology, design operational requirements, design and testing data collection instrument, sampling design, etc. (e.g. detailed project plan including tabulation plan, concepts and definitions, and classifications)
Build • Build and test technology solution, implement technology solution , training and testing, etc. (design specification, test results, printed documents)
Collect • Data capturing , close off collection (data capturing and verification, concepts and definitions)
Process • Data editing, etc. (processing methods, editing rates, classify and code.)
Analyse • Examine source data, produce clean datasets, etc. (data analysis processes followed)
Disseminate • Update output systems, produce products ,manage release products, manage customer queries , etc. (produce report, manage the release of the product)
• Each phase of statistical production may potentially contribute to errors
The South Africa I know, the home I understand
Key Assessment Steps
DQAT Assessment 8.Indepen-‐dent Assessment
7.Self Assessment 6.Acquisi'on of
metadata documents
3. SASQAF training of
product owner
4.Iden'fica'on of applicable SASQAF indicators
5.Signed document on indicators for assessment
1. Starts with request for evalua'on 2. Evalua'on of
request & Drawing up of
MOU
9.Data Quality Statement, Quality
Improvement Plan
10. SG and DG (assessed publica'on)
meet to finalise process outcome
( DATA/PRODUCT OWNER) 1. Iden'fy the problem 2. Conduct diagnos'c assessment 3. Develop strategy and plans
(problem solu'on) 4. Implement solu'on per
developed strategy 5. SASQAF self assessment
(PRODUCT OWNER/ or with STATS SA)
1. Develop/adopt/adapt standard(s), policy document, or any binding protocol as per the requirements of selected quality indicator. Examples of standards are data capturing error rate threshold, data dissemina'on policy, etc.
Fundamental
in the
assessment
process
DQAT Assessment DQAT Assessment
8.Independent Assessment
7.Self Assessment 6.Acquisi'on
of metadata documents
3. SASQAF training of
product owner
4.Iden'fica'on of applicable SASQAF indicators
5.Signed document on indicators for assessment
1. Starts with request for evalua'on 2. Evalua'on
of request & Drawing up of
MOU
9.Data Quality Statement, Quality
Improvement Plan
10. SG and DG (assessed publica'on)
meet to finalise process outcome
( DATA/PRODUCT OWNER) 1. Iden'fy the problem 2. Conduct diagnos'c assessment 3. Develop strategy and plans
(problem solu'on) 4. Implement solu'on per
developed strategy 5. SASQAF self assessment
(PRODUCT OWNER/ or with STATS SA)
1. Develop/adopt/adapt standard(s), policy document, or any binding protocol as per the requirements of selected quality indicator. Examples of standards are data capturing error rate threshold, data dissemina'on policy, etc.
Fundamental
in the
assessment
process
The South Africa I know, the home I understand
Con'nuous improvement &
future re-‐engineering
Validate current methods, iden'fy gaps and areas of improvement
Recommenda'ons on gaps and areas of improvement
Grow culture of review and con'nuous improvement
Structure: 1. Statistical
value chain 2. Based on
SASQAF quality dimensions
3. Quality indicators at sub-activity level
4. Based on documentary evidence
Process: 1. Process
owners involved
2. Team operationally independent
Evaluation of Methodology and Systems
The South Africa I know, the home I understand
Adherence to SASQAF will:
• encourage compliance to the agreed standards, procedures and guidelines Ø resulting in improvement of quality, closing the quality gap
• ensure that more statistics are certified as official Ø closing the supply gap
• assist the users in assessing the quality of data and products Ø promoting transparency
• ensure that all published products include statements about data quality (quality declaration) Ø informing users about the quality of data and products
• ensure that more statistical products produced in the NSS are declared as “fit for use”
Conclusion
The South Africa I know, the home I understand
THANK YOU
