Select, Overlay, Explore; Multidimensional data

Maintain Distributed Data; Heterogeneous coding , access

Connect providers to users; Homogenize data access

VOYAGER Data Federation Tools and Services Built and Used by a Virtual Community

Layered Map

Time Chart

Providers Users

Vector

GIS Data

XDim DataSQL Table

OLAP

Web

Images

Voyager Federation Web Services

Publish, Find, Bind

Data & Tool Catalog

Uniform Access/Retrieval Scatter Chart

Idaho Fires, 2000: Overlaying Fire Locations, PM2.5, SewWiFS

PM2.5 data from the IMPROVE and AIRS database at Colorado State (VIEWS)

CIRA ColoState-VIEWS

European Space Agency

Generic Map Server

NASA SeaWiFS Project

CAPITA SeaWiFS AOT

FEDERATED DATA SOURCES:

Fire locations from European Space Agency (Project IONIA)

Reflectance image from NASA SeaWiFS Project

Idaho Fires Example (August 2000)

ASTR Fire Locations

Processed SeaWiFS imageshowing aerosol optical depth

Derived aerosol optical depth provides a focused look at smoke dispersion

The SeaWiFS reflectance image is hidden to expose the aerosol optical depth image ‘below’ it

The Dvoy Project

DVOY is Federated Information System for heterogeneous, multidimensional datasets

Voyager is a generic graphic browser for the federated DVOY data.

The initial Dvoy infrastructure is being developed at CAPITA, with NSF support

Further services for data access, processing and viewing are expected from the community

The project evolution is to ride 'web services wave‘ of the Internet

CAPITA projects which use DVOY: – NSF ITR Workgroup Collaboration Tool: Aug 2001 - Aug 2004– EPA Web-based Visibility: Aug 2001 - Apr 2003– NOAA ASOS Visibility: Sep 2001 - Sep 2002– MARAMA Chemical Trajectory Tool: Aug 2002 - July 2003– EPA OAQPS Global Transport Analysis: Nov 2002 – Oct 2003– NSF DigiGov Fire and Smoke Network: May 2003 – Apr 2006 Pending– NASA ESE Satellite Appl. to PM Management: June 2003 – May 2008 Pending

In-kind support by organizations participating in DVOY-based federated data sharing

Collaborators/Partners: CIRA (Schichtel), NRL(Westphal), NASA (Goddard)…many data sources.

CAPITA cast: R. Husar, S. Falke, K. Hoijarvi, J. Colson, R. Zager)

Federated Information System• Data producers maintain their own workspace and resources (data, reports, comments).

• However, part of the resources are shared through a Federated Information System.

• Web-based integration of the shared resources can be across several dimensions:

Planned data sharing federations: • Open GIS Consortium (GIS data layers)• NASA SEEDS network (Satellite data)• NSF Digital Government • EPA’s National Env. Info Exch. Network.

VIEWSRPO

NASANAAPS

RPO Federated Data System

Data, Tools, Methods

SharedPrivate

RPO

Other Federations

Applications

PM Policy

Regulation

Mitigation

RPO Data Sharing

Management of visibility and PM requires coordination among local, state, regional and and federal agencies.

• data collection• data analysis• forecasting/modeling• communication to managers and the public.

A Federated PM Data Sharing Network could augment the dedicated RPO efforts, e.g VIEWS.

• identify and fill data gaps• incorporate new data such as satellite imagery• use community efforts to prepare summary/assessments• present summary/assessments to managers and the public

Render

Service Chaining in Spatio-Temporal Data Browser

Spatial Slice

Find/Bind Data

Data Cube

Time Slice

Time Portrayal

Spatial Portrayal Spatial Overlay

Time Overlay

OGC-Compliant GIS Services

Time-Series Services

Portray Overlay

Homogenizer Catalog

Wrapper

Mediator Client Browser

Cursor/Controller

Maintain Data

Vector

GIS Data

XDim DataSQL Table

OLAP

Satellite

Images

Data Sources

Multi-Dimensional Data Model

Data can be distributed

over 1,2, …n dimensions

1 Dimensional e.g. Time dimension

ij

k

ji

Data Granule

i

1 Dimensional

e.g. Location & Time

1 Dimensional e.g. Location, Time &

Parameter

View 1

Data Space

View 2

Views are orthogonal slices through multidimensional data cubes

Spatial and temporal slices through the data are most common

Web Services Components and Actions

• Service providers publish services to a service broker.

• Service users find the needed service and get access key from a service broker• With the access key, users bind to the service provider• The result is a dynamic binding mechanism between the service users and providers

Service Broker

Service Provider

PublishFind

BindServiceUser

Components: Provider – User – Broker

Actions: Publish – Find - Bind

Web PublishHTTP, FTP

Data Access though a Web Service Adapter

Service Cataloger

Service Consumer

Ordinary web content can be delivered as a Web Service through a Proxy Server.

• The Wrapper Service converts HTTP/FTP service to XML Web service

• The Wrapper Service is published as a web service to the Broker

• The User finds the service from the broker and accesses the Wrapper to use the service

Service Wrapper

HTTP/FTP

ServiceService

User Chain

FindUDDI, WSDL

PublishUDDI, WSDL

AccessSOAP, XML

Interoperability through a Layered Protocol Stack

• Web Services are implemented on a layered stack of technologies and standards

• The lower layers enable binding and exchange of messages; higher levels enable interoperability

• Applications are formed dynamically from distributed components through publish-find-bind mechanisms

TCP/IP, HTTP, FTP

ASCII, XML, etc.

HTML, XML OGC -GML

OGC Coverage, CoordTransfom, WMS

HTTP, SOAP

WSDL

UDDI OGC Catalog

WSFL, XLANG

StandardsInteroperability

Comm. Protocols

Data Encoding

Data Schema

Data Binding

Web Service

Service Integr.

Service Discovery

Service Descript.

Connectivity

Architecture of Dvoy Federated Information System

• The main software components of Dvoy are wrappers, which encapsulate sources and remove technical heterogeneity, and mediators, which resolve the logical heterogeneity (e.g. Busse et. al., 1999).

• Wrapper classes are available for geo-spatial (incl. satellite) images, SQL servers, text files,etc. The mediator classes are implemented as web services for uniform data access to n-dimensional data.

Features of the DVOY XML Web Service Architecture

Interoperability: Platform and language independence; based on Web Services (XML,SVG)

Legacy Support: Encapsulating existing data and exposing them as Web Services. (Access to standard HTTP/FTP servers)

Just-in-time integration: Discovery, access to and ad-hoc chaining of services.(Future: Agile application building)

DVOY draws on three basic web services:

DataCatalogWrapping service is for data registration, finding and wrapping information

DataAccess service provides uniform access to heterogeneous, distributed, multidim. data

DataPortrayal service prepares input data in form suitable for rendering

Data Catalog

NRL Forecast Model for Dust, Smoke and SulfateMETAR Surface Haze

• Real-time model and surface observations are compared spatially and temporally

Dust

Sulfate

Smoke

METAR HazeTime Selector

Quebec Fires, July 6, 2002

SeaWiFS, METAR and TOMS Index superimposed

SeaWiFS satellite and

METAR surface haze shown in the Voyager distributed data

browser

Satellite data are fetched from NASA GSFC; surface data

from NWS/CAPITA servers

Trans-Atlantic Transport of Quebec Smoke

July 11: Smoke approaching Europe

July 10: Quebec smoke over Mid-Atlantic

SeaWiFS Reflectance

TOMS Absorbing Aerosol

SeaWiFS Reflectance

TOMS Absorbing Aerosol

Spain

Spain

E. US

Browsing of Distributed Data from HTTP/FTP Servers

Land Reflectance from SeaWiFS Project, NASA GSFC

AVHRR Oceanic Aerosol

CAPITA, WashU, STL

Fire Pixels, Jan 1997, ESA Ionia Project

NSF-NOAA-EPA/EMAP (NASA)? Project:

Real-Time Aerosol Watch System

Real-Time Virtual PM Monitoring Dashboard.

A web-page for one-stop access to pre-set views of current PM monitoring data including surface PM, satellite, weather and model data.

Virtual Workgroup Website.

An interactive website which facilitates the active participation of diverse members in the interpretation, discussion, summary and assessment of the on-line PM monitoring data.

Air Quality Managers Console.

Helps PM managers make decisions during major aerosol events; delivers a subset of the PM data relevant to the AQ managers, including summary reports prepared by the Virtual workgroups.

wind direction

back trajectories

temperature NAAPS model

PM/Bext time seriesBext contours

PM2.5 contours

web cam

Real-time PM Monitoring DashboardExample Views – Selected from Dozens of spatial, temporal, height cross-sections

satellite image