Maria João Zilhão Planing and Quality Control Unit

« Q 2008 / Rome July

2008

Statistical Auditing and Self-Assessment Benefits for Quality Improvement

«

Concepts ….

Audit

Self-Assessment

An audit is a systematic, independent and documented process for obtaining audit evidence and evaluating it objectively to determine the extent to which the audit criteria are fulfilled

Self-assessment is a comprehensive, systematic and regular review of an organization’s activities and results referenced against a model/framework

The Leadership Group on Quality

The Importance of Auditing and Self-assessment in

statistics

Principle 4 – Quality Commitment

Recommendation nº 16

Recommendation nº 15

European Statistics Code of Practice

A generic checklist should be developed for a simple self-assessment programme for survey managers

DESAP

The methods for auditing on different levels and for diferent purposes such as internal, external ,.., should be reiviewd and recommendations provided

Handbook on experiences on auditing

4.2

4.5

Measurement of Processes and Products

Labelling

Self-Assessment

User Requirements

Standards

Certification (ISO 20252)

Audits

Process Variables

Quality Indicators

Quality Reports

User Surveys

External Environment of DatQAM

Precoditions of DatQAM

The DatQAM map(Handbook on Data Quality assessment Methods and Tools

Eurostat / 2007)

References for auditing

ISO Norms:

• 9000:2000…Implementation of Quality Management Systems

• 19011 - .. Auditing…

• 10013 - …Documentation systems

References for self-assessment

• EFQM Excellence Model

(European Foundation for Quality Management)

• CAF – Common Assessment Framework (EIPA - European Institution for Public Administration)

• DESAP – Checklist for survey Managers(Eurostat)

Top Management support

Documentation System

The results of auditing and self-assessments must be consequent

A Clear communication and involvement of the staff

Preconditions

Proper preparation!

Quality Management Unit

Internal Audits

Departments

External Audits

Documenting Processes

Customer

Satisfaction Surveys

DepartmentsDepartments

Suggestions / Complaints

Performance indicators

Quality Management System

at Statistics PortugalQuality

Documentation System

ISO 19011

ISO 10013

P

D

C

A

COP – Principle 4

Process design Definition of different phases of the processDefinition of responsibilitiesDefinition of minimum documentationHandbooks are produced by pluridisciplinary teams

Documentation System

ISO 10013

Documenting processes

Procedures handbooks

Documentation on survey processes

Phases

I. Conception IV. EvaluationIII. DisseminationII. Operation

II.2. Data collection

II.3. Data treatment and analysis

II.1. Planing and preparation of data collection

I.1. Viability Study

I.2. Methodological Study

I.3. Technical approval

Processes Processes

III.1. Dissemination IV.1. Evaluation

Process Process

Statistical Production Handbook Procedures

Note: Each Process is composed by sub-processes and tasks

Audits

Internal Quality Audits at Statistics Portugal (I)

P

D

C

A

To verify the degree of procedures implementation (handbooks procedures)

To Identify improvement opportunities

To check the internal articulation/communication of units (survey related)

To Improve survey documentation

Objectives

I. Conception IV. EvaluationIII. DisseminationII. Operation

II.2. Data collection

II.3. Data treatment and analysis

II.1. Planing and preparation of data collection

I.1. Viability Study

I.2. Methodological Study

I.3. Technical approval

III.1. Dissemination IV.1. Evaluation

Statistical Production Handbook Procedures

«Internal Quality Audits

at Statistics Portugal (II)

Internal Procedure on auditing

(Clear rules!)

Top Management Support

Exchange of Information / Strong Communication

about auditing, through Top Management, and the Quality Unit

DiscussionTraining of 14 Auditors

(Statisticians) (ISO 19011)

Internal Quality Audits at Statistics Portugal – Preparation

(III)

Internal Quality Audits at Statistics Portugal –(IV)

Principal rules

Top Management defines the Quality/Statistical Audits’ annual plan

No one should audit its own activity

One Audit team needs 2 or 3 auditors

Auditors produce Audit Plans and reports according to Procedures

Improvement actions must be carried out by the audited departments

Improvement actions must be published and followed by the audit teams

External Quality Audits

at Statistics Portugal Auditors are Peers (from other statistical Offices)

Basic rules are the same as for internal auditing Two visits (+/- one week each) Contacts between auditors and audited teams in the mid

period Auditors work recommendations with audited teams Results are presented to the board Action plan is made by audited teams and presented to the

board

Benefits for Quality Improvement

The identification of improvement opportunities in the audited process

The implementation of corrective / preventive actions

The redefinition of procedures

Auditing involves people in the quality program (auditors and audited teams)

Good Recommendations - A look from the outside!

Identification of good practices that can be spread in the organization

Self-assessment and auditing can be combined together

A time to re-think and improve!

Some NSI’s have

experienced Quality

Auditing!

BUT!!!!

-Time consuming activity

-May face resistances

- May be dificult to start

- Better results in the long run if systematically implemented

A Powerfull Tool!

« «

THANK YOU!

Maria João Zilhão

Statistics Portugal

Planing and Quality Control Unit

mjoao.zilhao@ine.pt