Metadata-driven Business Processin the

Australian Bureau of Statistics

Aurito Rivera, Simon Wall, Michael Glasson – 8 May 2013

Content

1. Why have it?

2. What is meant by metadata-driven business process?

3. What is the ABS doing?

• Two case studies

4. Implications for metadata management

5. Key messages

Why have metadata driven business process

• ABS perspective - the business benefits include• Reduced time and cost of statistics production• Improved quality of statistical products• Increased agility in meeting new demands for statistical products

and services.• Increased agility in harnessing new sources of statistical data.

• Industry perspective - directly supports the HLG’s approach to industrialising and standardising statistics production.

What is metadata-driven business process

In the ABS, it is the systematic and consistent use of metadata to determine the inputs, outputs and behaviour of a statistical business process.

What is metadata-driven business process

Key characteristics1. Metadata is used systematically

• Metadata is used in a planned and managed way across the organisation.

2. Metadata is used consistently• Authoritative ‘single source of truth’ metadata is used throughout the

end-to-end lifecycle of an activity and/or across activities.

3. Metadata is used actively • Metadata is used to automate the definition and execution of statistical

processes• Metadata is structured so as to be machine-consumable.

What is metadata driven business processNecessary components for ABS implementation:

1. GSIM/DDI/SDMX-based metadata• that serves as or identifies inputs to and outputs from processes • standardised and discoverable via a metadata registry• analogous to ‘blood that flows through vital organs’.

2. Libraries of business processes, methods and IT components • that can be assembled in a ‘plug and play’ way• that are atomic or are complex processes that are assembled from atomic

process components

3. Effective governance framework• to ensure the appropriate specification and use of contents• that dictate what and how metadata is used in what processes.

What the ABS is doing in this area• Examples of earlier work:

• 2002 – Business Statistics Innovation Program (BSIP)• 2003 – development of ABS strategy for e-2-e management of

metadata• 2009 – formal adoption of SDMX and DDI Standards• 2010 – Information Management Transformation Program (IMTP).

• Examples of more recent work (under the ABS 2017 Program):• Metadata Registry and Repository (MRR), Statistical Workflow

Management systems (SWM)• Online Forms Prototype Project (eForms)• Enterprise Data Warehouse (EDW).

Case study 1 – MRR/SWM

• Two Core applications that will form the ‘central nervous system’ of future ABS metadata management infrastructure.

• MRR/SWM are intended to guide and quicken the work of Statistics Producers as well improve metadata management.

• MRR/SWM in themselves are not sufficient to realise the business benefits.• Also need Business Process re-engineering, instilling Information

Management governance, etc.

Case study 1 – MRR/SWM• Components

• MRR is a store of statistical metadata that is authoritative, centralised, and standards based• Repository – logically centralised store• Registry – metadata catalogue and data directory.

• SWM helps to assemble business processes from a library of re-useable building blocks

• SWM helps to automate and manage the execution of these business processes.

• SWM in concert with MRR will: • Guide and quicken the work of Statistic Producers• Greatly improve the way we manage metadata.

Case study 1 – MRR/SWM

Registry

Repository

Register key elements of metadata package

E.g. Register metadata

Complete metadata package resides in

Repository

Case study 1 – MRR/SWM

Repository

Search for metadata here

E.g. Metadata search

Returns a list of matches based on search criteria

Registry

?1. Match12. Match23. Match3

Case study 1 – MRR/SWM

Registry

RepositoryE.g. Retrieve metadata based on search results

]

Case study 1 – MRR/SWM

• Proof of Concept undertaken in 2011/2012

• Lessons learnt from POC• Concepts and technologies employed are sound• Automation of business processes feasible.

• First production version due December 2013

Case study 2 – eForms

• Prototype web-based collection tool• Forms the basis for ABS capability for building and deploying

web-based collection tool.

• Test the feasibility of using • DDI 3.1 Instrument specification,• Strategic ABS technologies,

• especially SWM/MRR

• GSBPM-aligned processes.

Case study 2 – eForms

Components• Web Instrument

Production Application

• MRR• SWM• ABS Collections

Electronic System (ACES)

• DDI to ACES Transformer Service.

Brief case study 2 – eForms

Lessons learnt from POC• Automatic generation of web-base collection tool was

possible• Need an ABS (or a wider-NSI) DDI profile• Need for a Metadata Creation and Management Toolbox• Centrally registered, accessible and governed metadata.

Conclusions from the two case studies

• Metadata-driven business process is feasible• There are technical challenges but the more significant ones

are non-technical.

• There are implications for metadata management• Move from siloed systems to integrated metadata

infrastructure• Conceptualise and implement metadata consistently• Govern metadata definition, use, etc. effectively.

Likely implications for metadata management – based on ABS experience

• ‘Templates’ depicting what GSIM object are required as input/output to GSBPM sub-processes are needed• Informal Task Force on Metadata Flows across the GSBPM – a

good start.

• Versioning strategy for metadata is required

• ‘Eco system’ of governance arrangements – e.g. IT governance, Information Management governance, etc. should all cohere

Likely implications for metadata management – based on ABS experience (cont.)

• Statistical Metadata Systems should be made to be shareable

• Broadening and deepening role of metadata practitioners

• Metadata practitioners must be up-to-date with frameworks and Standards.

Key Messages

• Metadata driven business process is feasible

• Metadata driven business process credibly presents tangible business benefits

• Metadata driven business process is consistent with HLG Vision

• Metadata driven business process will require potentially significant changes to the way metadata is managed in an NSI.