Biological and Chemical Oceanography Data Management Office slide 1 of 21

Interoperability ~An Introduction

Cyndy ChandlerBiological and Chemical Oceanography Data Management Office (BCO-DMO)

Woods Hole Oceanographic Institution

26 July 2008

OOS Interoperability Planning WorkshopNational Space Science and Technology Center

University of Alabama at Huntsville, AL USA

Biological and Chemical Oceanography Data Management Office slide 2 of 21

Discussion Points

What do we mean by “interoperability”?

Why does it matter?

How do we achieve interoperability?What are some of the expected challenges and

effective strategies?

Biological and Chemical Oceanography Data Management Office slide 3 of 21

Biological and Chemical Oceanography Data Management Office slide 4 of 21

What do we mean by “interoperability”?the ability to exchange data and information between two or more systems (separated by a recognized boundary) with minimal loss of information clients of interoperable systems may be machines

syntactic interoperability file formats data structures exchange protocols

semantic interoperability term definitions (controlled vocabularies)

Biological and Chemical Oceanography Data Management Office slide 5 of 21

Interoperability . . . a wee bit more from MMI

a client should not be required to possess in depth understanding of the data in order to access them

interoperable systems should be designed to support machine access (not just people clients)

interoperable metadata systems should be designed to support automated, accurate, lossless, machine-to-machine exchange of information

Biological and Chemical Oceanography Data Management Office slide 6 of 21

Why does interoperability matter?improved open access to public data enabling data mining, cataloging systems, portals, mash-ups

ecosystem researchers are asking global questions

model developers need access to data from a variety of domains (biology, chemistry, etc and social and economic data too) we must rise to the challenge of designing interoperable data

systems that are up to these tasks

Biological and Chemical Oceanography Data Management Office slide 7 of 21

How do we achieve interoperability? What are some of the expected challenges?

varied funding sources and governing agencies

projects with different priorities and leadership

distributed data systems

large volume, heterogeneous data

expectation of near real-time availability

Biological and Chemical Oceanography Data Management Office slide 8 of 21

adopt standards

use controlled vocabularies

publish metadata databases

use Semantic Web technologies

build community

What are some of the effective strategies?*

Biological and Chemical Oceanography Data Management Office slide 9 of 21

This is going to be difficult . . .best solutions will be found through community recognition of local implementations

Baker & Chandler, in press. publication expected in DSR II before end of 2008.

Title: Enabling Long-Term Oceanographic Research: Changing Data Practices, Information Management Strategies and Informatics

online:  http://dx.doi.org/10.1016/j.dsr2.2008.05.009 final draft complete: 9-JUL-2008

Biological and Chemical Oceanography Data Management Office slide 10 of 21

The data …

CMarZGEOTRACESIron SynthesisNACPOCBUS GLOBECUS JGOFSUS SOLAS

controlled vocabulariescommunity buildingmetadatastandards

*

Biological and Chemical Oceanography Data Management Office slide 11 of 21

Ocean Observatory Data …expecting large volume, heterogeneous data to be made available in near real timein the past, large volume data sets tended to be more homogeneous (e.g. fewer columns); while data sets with many columns tended to be smaller in size

Homogeneous HeterogeneousSensor data Manual observationsPhysical Biology, Chemistry

Data from ocean observatory sensor arrays are/will be a continuous stream of large volume, heterogeneous data.

Biological and Chemical Oceanography Data Management Office slide 12 of 21

Ocean Observatory Data …

attempting to reduce this challenge by identifying 20 IOOS core variables recognizing the importance of metadata identifying common use cases adopting common standards (OGC and SWE) fostering community-wide involvement and communications sponsoring mysterious, after-dark gatherings in Huntsville, AL

providing little more information beyond street address and suggested footwear . . .

*

Biological and Chemical Oceanography Data Management Office slide 13 of 21

Semantic WebSemantic Web technologies offer one solution set

next four slides are courtesy of Peter Fox (HAO/ESSL/NCAR) and are from an April 2008 presentation

Biological and Chemical Oceanography Data Management Office slide 14 of 21

Semantic Web Basics

The triple: {subject-predicate-object}

Interferometer is-a optical instrument

Optical instrument has focal length

An ontology is a representation of this knowledge

W3C is the primary (but not sole) governing organization for languages, specifications, best practices, etc. RDF - Resource Description Framework OWL 1.0 - Ontology Web Language (OWL 1.1 on the way)

Encode the knowledge in triples, in a triple-store, software is built to traverse the semantic network, it can be queried or reasoned upon

Put semantics between/ in your interfaces, i.e. between layers and components in your architecture, i.e. between ‘users’ and ‘information’ to mediate the exchange

(P. Fox, 2008)

Biological and Chemical Oceanography Data Management Office slide 15 of 21

(P.Fox, 2008)

… … … …

VO Portal

Web Serv.

VO API

DB2 DB3DBn

DB1

Semantic mediation layer - VSTO - low level

Semantic mediation layer - mid-upper-level

Education, clearinghouses, other services, disciplines, etc.

Metadata, schema, data

Query, access and use of data

Semantic query, hypothesis and inference

Semantic interoperability

Added value

Added value

Added value

Added value

Mediation Layer• Ontology - capturing concepts of Parameters,

Instruments, Date/Time, Data Product (and associated classes, properties) and Service Classes

• Maps queries to underlying data• Generates access requests for metadata, data• Allows queries, reasoning, analysis, new

hypothesis generation, testing, explanation, etc.

*

Biological and Chemical Oceanography Data Management Office slide 16 of 21

Semantic Web Methodology and Technology Development Process

Establish and improve a well-defined methodology vision for Semantic Technology based application developmentLeverage controlled vocabularies, etc.

Use Case

Small Team, mixed skills

Analysis

Adopt Technology Approach

Leverage Technology

Infrastructure

Rapid Prototype

Open World: Evolve, Iterate,

Redesign, Redeploy

Use Tools

Science/Expert Review & Iteration

Develop model/ontology

(P. Fox, 2008)

Biological and Chemical Oceanography Data Management Office slide 17 of 21(P. Fox, 2008)

The Information Era: Interoperability

managing and accessing large data setshigher space/time resolution capabilities rapid response requirementsdata assimilation into modelscrossing disciplinary boundaries.

Modern information and communications technologies are creating an “interoperable” information era in which ready access to data and information can be truly universal. Open access to data and services enables us to meet the new challenges of understanding the Earth and its space environment as a complex system:

Biological and Chemical Oceanography Data Management Office slide 18 of 21

Semantic Web technologies being used in some of Peter Fox’s informatics projects:

Semantically-Enabled Science Data Integration (SESDI): http://sesdi.hao.ucar.edu Semantic Knowledge Integration Framework (SKIF): http://skif.hao.ucar.edu

Semantic Web for Earth and Environmental Terminology (SWEET): http://sweet.jpl.nasa.gov

SWEET is a JPL project to provide a common semantic framework for a variety of Earth science initiatives.

Biological and Chemical Oceanography Data Management Office slide 19 of 21

more challenges . . .reality check: insufficient metadata …

researcher aboard R/V Oceanus, August 2008, talking about metadata for shipboard sampling

Data originators recognize that recording metadata is a good idea, and they will even record metadata ~ if we make it easy for them to do so.

*

"I feel like I'm so busy with all the other little things in a cruise that I don't have time to worry about making up log sheets. But if someone puts an already made log sheet in front of me, I'll use it."

"When I see projects with good metadata, it just makes my heart go pitter patter."

Biological and Chemical Oceanography Data Management Office slide 20 of 21

more challenges . . .reality check: heterogeneous data sets …

The BCO-DMO database includes many data sets in which depth and temperature are reported and more than 25 different definitions of each.

Biological and Chemical Oceanography Data Management Office slide 21 of 21

*

Community buildingShare knowledge gained doing implementation at the local level – participate in workshops and publish

Fall 2008 AGU Meeting in San Francisco, CA8-12 December; Earth and Space Science Informatics

IN12: Strategies for Improved Marine and Synergistic Data Access and Interoperability

IN19: From Data to Synthesis: Next-Generation Science Applications

Abstract Submission Deadline: 10 September 2359 UTThank you . . . cchandler@whoi.edu