1Data Management and Information Delivery

The Data Management and Information Delivery

(DMID) Project

10 Apr 2008 Ashwell Jenneker & Matile

Malimabe

2Data Management and Information Delivery

Topics

Stats SA Background The DMID Project The End to end Statistical Data

Management Facility (ESDMF) Phase 1 Demo Phase 2 Tools ESDMF “Building brick by brick”

3Data Management and Information Delivery

Statistics SA Background Is a national government department accountable to

Minister of Finance, regulated by the Statistics Act (No. 6 of 1999)

Legislative mandate provides for: Promotion of coordination among statistical producers in SA Provision of statistical advice to government departments Liaison with the statistical agencies of other countries and

other international agencies

Vision: The preferred supplier of quality statistics Mission: To provide a relevant and accurate body of

statistics to inform users on the dynamics in the economy and society through the application of internationally acclaimed practices

For more information: visit www.statssa.gov.za

4Data Management and Information Delivery

The DMID Project A business improvement initiative:

Improve the way originating units develop, structure, capture and store data and metadata according to common standards and procedures.

Facilitate better data sharing between the various units Extend the analysis and comparability of data Improve access and use of data, and version control

Central storage of data and metadata is key in improving quality of, and access to, authoritative and reliable statistical information

Adoption of corporate standards for capturing, storage of, access to and management of data and metadata throughout the statistical value chain, from needs analysis to dissemination of data.

5Data Management and Information Delivery

ESDMF: Functional Requirements

Support the Statistical value chain from end to end: Manage data capturing and loading

Centrally store data and metadata

Manage exploratory analysis and analysis for publication

Manage dissemination and archiving

Workflow must be metadata driven and metadata generating

User interface, with role-based access

6Data Management and Information Delivery

Dissemination AreaCentral Data StoreCollections Area

ExternalSources

Metadata Store

Workflow

Data capturingtool

Flat files

Unit Record dataFiles

ValidationEditing

ImputationWeighting

Version control

IDW

Publishableaggregations

CURFs

ODW

Exploratory Analysis

Publicationfacility Publication

Achive

Webpublishing

CD publishing

Hard copypublishing

Collect Process Analyse Disseminate Design Need Build

The ESDMF

7Data Management and Information Delivery

Dataset /Survey metadata• Collect metadata according to the metadata policy using:• standard metadata template

PBT: Development and Implementation

8Data Management and Information Delivery

PBT: Development and Implementation

9Data Management and Information Delivery

Summary of the ESDMF (Phase 0 to 12) 17 Facilities 53 Applications/Tools R 35 Million 3 Year Timeframe

PBT: Development and Implementation

10Data Management and Information Delivery

Phase 1: Tool Demo

We will now show a short Demo of the tool DMID has Developed to Capture Statistical Metadata95% Captured (3 Outstanding)

65% Approved

11Data Management and Information Delivery

Phase 2 Solution DevelopmentPhase 2 Solution Development Concepts and Definitions Management ToolProvides a central register of all concepts and definitions used in Stats SA, recording the terms and their definitions grouped by pre-defined topics and keywords associated with the topics, making use of the registration framework implemented by the Metadata Registration Management Tool.

Variables Management ToolProvides a central register of all variables on which Stats SA collects information making use of the registration framework implemented by the Metadata Registration Management Tool

Classifications Management Tool Provide a central registry of all classifications making use of the registration framework implemented by the Metadata Registration Management Tool.

Metadata Registration Management ToolImplements the chosen registration framework and guides users of the tool through that process of registration so that an environment is created where statistical processes occur in an organised fashion.

12Data Management and Information Delivery

Phase 2 Solution DevelopmentPhase 2 Solution Development

Quality Framework ToolCapture all the indicators, for each dimension, of the SASQAF document and will be able to disseminate the SASQAF report to relevant users.

ESDMF ReportsCreate, execute and disseminate all ESDMF reports. This phase will be limited to the configuring and/or integrating Pentaho with the metadata repositories and registries of products described in this document and that of phase 1.

Survey Metadata Capture Tool (SMCT)Enhanced to integrate with the some or all products described in this phase.

Access Control Management ToolProvides a central point for creating and managing access to all tools of the ESDMF

13Data Management and Information Delivery

Phase 2 In relation to previous and future phases

Phase 1 Phase 2

Survey Metadata Capture

Some parts

of SASQAF

Metadata Capture

Tool

Definitional Metadata

ThesaurusRegister

Process/Methodological

Metadata

Metadata Auditing

Phase 3Process/

Methodological Metadata

Workflow Definition

Workflow Task

Management

Metadata Version Control

Phase 4Workflow

Operational Metadata

Questionnaire Repository

Manual Data Capture

Phase 6Operational Metadata

Classifications Register

Variables Register

Concepts & DefinitionsRegister

SASQAFTool

Admin. of Registration

BusinessRegister

Meth & Stds

CaRS Specs

Out of Scope

In Scope

Within Stats SA

14Data Management and Information Delivery

Thank you