Data and Information Framework: Principles

Sue Barrell

Bureau of Meteorology, Australia

CBS-Ext.(14), Asuncion, September 2014

Data Insight

Ref. Robert Logan (2012)

Val

ue

• Applied wisdom insight and value

• Science, numerical models, reports, national accounts, assessments, policies

• Forecasts, warnings, outlooks; Climate record; Metadata about observations, data, products

• Environmental observations: land, oceans, water, atmosphere, space

Observing the environment

• Measure, monitor, observe, sense the environment in which we live

• Snapshots, transects, trends, processes

Ubiquitous sensing, immediate sharing

Increasingly massive data flows & datasets

Meeting insatiable science & service needs

Global Observing Systems

Global

Regional

National

Process, integrate, model data

• Use science to model, understand, interpret, predict, inform

• Using technology, tools, storage, compute power to enable

• Data generating data

• More power, more speed, more capacity

More detail, more knowledge, more data

More challenge

Continuous Improvement in Numerical Weather Prediction

Increasing population

Vulnerability of built environment

Science & technology advances

Lower acceptable risk

Structured emergency management

Personal interface, locally relevant info

Demand for instantaneous

information

Corporate imperatives (i.e. Royal Commissions)

More climate extremes

Apply, inform, decide

• Services designed & delivered to meet user demands

Increasing demand

Demand for services via Web

Share, access and combine to address diverse requirements

• Global/national data sharing and exchange- Maximise value of data, diversify sources of data, quid pro quo

• Access, combine and/or compare observations from one source or system with those from another

• To achieve this, we need:- Metadata & Standards

- Policies & Frameworks

- Collaboration and more

• Through interoperable systems, we turn observations and information into effective data, products, services & insight that meet real user needs

Data: Lifeblood and legacy

• The Bureau's lifeblood and enduring legacy

- Flows through all Bureau products and services

- Essential to delivering value and valued services to our users

- Basis of the climate record and reference datasets

• International obligations

- WMO Res 40 and 25

- Traceability, discoverability and standards

• Deliver value to Australian community

- Discovery, access, interoperability, usability

- Strategic (government) investment and asset

- Open Public Sector Information, Government ICT Strategy

Bureau data challenges

• Operational 24/7 data: weather, flood forecasts & warnings

• Long-term archive: climate, water

• Volume and diversity: increasing demands

• Integration: across space, time, domains, users

• Managing expectations: quality and uncertainty

• Metadata: observations, provenance, discovery

• Stakeholders: international, national, Bureau

• Obligations: exchange, continuity, traceability

Problem/Need

Verify Evaluate Assess

Apply Inform Decide

Observe Monitor Collect

TransmitAccessShare

Process Integrate

Model

DATALIFECYCL

E

Environmental Data Lifecycle Model

Problem/Need

Verify Evaluate Assess

Apply Inform Decide

Observe Monitor Collect

TransmitAccessShare

Process Integrate

Model

DATALIFECYCL

E

Environmental Data Lifecycle Model:it's not just about IT

security exchange

archive

standards metadata

disaster recovery

formats

provenance

channels

stewardship

back-up

licensing

attribution

storage

costing

governance

disposal

discovery

Why?

Who?

monitor

Integrate

record

policy

Data FrameworkProblem/

Need

Verify Evaluate Assess

Apply Inform Decide

Observe Monitor Collect

TransmitAccessShare

Process Integrate

Model

DATALIFECYCL

E

Environmental Data Lifecycle Model

The journey so far

• Reviewed Bureau's approach to data policy and data management

- Bureau data use, policies, practices, needs

- All types of scientific data

• Developed concept of Bureau Data and Information Framework

- Principles-driven approach that spans all aspects of data lifecycle

- Assuring value from/through data, both for internal & external users

• Established governance and responsibility

• Collaboration across agencies and internationally

• Realigned Bureau's organization structure, with data and information as key focus Information Systems and Services Division

Data and Information Framework

Data and Information Framework

• Provides foundation for an enterprise approach to all aspects of data acquisition, management, use, access, archival and disposal

• Respects data assets as a critical national resource that must be managed effectively for current and future generations

Data and Information Principleshttp://www.bom.gov.au/inside/BoMDataFramework_Final.pdf

• Strategic alignment – with Bureau's wider strategic directions, key government & international directions

• Authoritative & trusted – ensure the Bureau is a trusted, authoritative & reliable source of environmental information

• Deliver value – maximise its value by applying D&I to the needs of government, community, industry and stakeholders

• Discoverable & useable – make D&I easily discoverable and useable over time

• Robust management and governance – strong standards-based governance and practices to achieve efficient, effective and secure lifetime management of D&I

• Innovation and leadership –nationally and internationally, from compliance to innovation

Implementing the Bureau's Data Framework

• High-level governance:

- Data and Information Governance Consultative Committee

- Data and Information Reference Group

• Environment Information Management Branch

- Implement & manage the framework, policies, procedures, tools…

• Early priorities:

'Audit' of extant data/information policies and life cycle requirements

Focus on scientific data, but framework includes administrative data

Metadata – categories, definitions, requirements, identification, profiles

Archives Act compliance; develop records authorities

Arrangements for externally sourced data Integrated data life cycle/archival plans

Tiered data/networks/quality Data management guidelines & policies

Licensing Operational data catalogue

Key Messages

• An increased focus on data and information management

- fundamental to harnessing the value of data and information held by the Bureau, now and for the long-term

- improve discovery, access, interoperability, and usability

- serve internal and external stakeholders

• The Data and Information Framework provides a principles-based approach

- for effective data and information management and governance

- to support collaboration, enhanced sharing and use of data and information

• Data delivers value only through being well-managed through its life, accessible, usable and used – the framework will help us achieve this

Problem/Need

VerifyEvaluate Assess

Apply Inform Decide

Observe Monitor Collect

TransmitAccess Share

Process Integrate

Model

DATA

Thank you

Data Framework