Post on 18-Jan-2018
description
transcript
1CLASS –Simple NOAA Archive Access Portal
SNAAP Eric Kihn and Rob Prentice
NGDC CLASS Developers Meeting
July 14th, 2008
Simple NOAA Archive Access Portal
2CLASS –Simple NOAA Archive Access Portal
SNAAP Goals
Integrate diverse systems via a standards based, user focused WS interface
Demonstrate the concept of “fundamental separation” of archive and storage from access
Create a tool that shows the benefit of API’s for users Discover new technology and evaluate cutting edge tools
for handling integration and presentation Document integration of multiple data types and sources
through an single API (time-series, grid, swath, etc..)
3CLASS –Simple NOAA Archive Access Portal
The Data Environment Now
CLASS ADS
Web Interface Subscriptions
Human Machine
DataProviders
Visualization Inventory
Order
NM
MR
/Metadata
NGDC/SPIDR
Web Interface
Web Interface
Metadata
SPIDR WS
SNAAP
Web Interface
Metadata
Data Access
4CLASS –Simple NOAA Archive Access Portal
Three Simple Items Search for appropriate data sets
Discover data by time, type, location, abstract, etc. Support for multiple metadata schema Standards based This is in CLASS 4.3.3 – awaiting policy
Order Data Needs a standard way to describe ordering Ordering extensions + Standard metadata Needs some framework for service integration (i.e. transforms)
Integration Layer We are using OGSA-DAI for the demo but this is not a final choice Should not affect the consumer at any rate.
5CLASS –Simple NOAA Archive Access Portal
So a new provider wouldhave to …
Create a standard metadata record for the data set. (They should be doing this anyway).
Create an Ordering extension document. If they are offering web distribution this is trivial.
Write an adapter to the middle layer, unless they have a supported service (like JDBC, FTP, etc..) in which case adapt an existing one.
6CLASS –Simple NOAA Archive Access Portal
Advantages
This approach mirrors that of the highly successful OGC specification for map services (what, make, retrieve)
It minimizes entry burden for new providers Its simplicity makes it scalable It doesn’t preclude or interfere with on-going activity, just
opens up new integration possibilities.
7CLASS –Simple NOAA Archive Access Portal
Current snapshot of the CLASS API architecture
CLASS
CLASS products
Visualization service
SPIDRESG - INE - IDEAS – ESSE
NCEP , ERA-40 , ...
DB cluster
Inventory service
Order service
WebServices
OPeNDAP
OGSA-DAI for CLASS (Globus Toolkit 4)
Plugin PluginPluginPlugin
NOMADS
NARR
NetCDFNetCDF
NetCDF
THREDDS(OPeNDAP)
WCS(OGC)
SABR
OLS DMPS
Previewimage
Plugin
Inventory
GranuleGranuleSatellite
Granule
CLASS web interfaceCLASS
metadata
REST API for CLASS
DB cluster
OGSA-DAI client toolkitGRID client toolkit
8CLASS –Simple NOAA Archive Access Portal
Data Order Flow
OE Web Form Servlet
XSLT get
Client OGSA-DAI
XML Result
Get DataProcess
Mine
SQLXML
GranuleTime
Series
12
Adapter
Storage
ErrorMessage
Data
9CLASS –Simple NOAA Archive Access Portal
CLASS Ordering Extensions (OE) schema upper level
10CLASS –Simple NOAA Archive Access Portal
CLASS OE schema: parameters and time
11CLASS –Simple NOAA Archive Access Portal
CLASS OE schema: map and ROI
12CLASS –Simple NOAA Archive Access Portal
CLASS OE schema: station map
13CLASS –Simple NOAA Archive Access Portal
Why OGSA-DAI?
Standard tool in the Grid community Includes a “process” for extraction Built in support for asynchronous transactions Includes support for domain login Looked at alternatives like OpenDap, WCS, Custom
applications some documentation forth coming Problem: Its very complex and not well accepted in NOAA
14CLASS –Simple NOAA Archive Access Portal
Visualization screens – 1time series and map server
15CLASS –Simple NOAA Archive Access Portal
Visualization screens – 2NASA World Wind plugins
16CLASS –Simple NOAA Archive Access Portal
MS Excel and NASA World Windinteroperability for trajectory data
Excel with track points
Excel with data along tracksDrag & Drop controls
NASA World Wind
17CLASS –Simple NOAA Archive Access Portal
Asynchronous Ordering
CLASS API Demonstration
18CLASS –Simple NOAA Archive Access Portal
Orbit Inventory List
19CLASS –Simple NOAA Archive Access Portal
User Basket (Order & Check-Status)
20CLASS –Simple NOAA Archive Access Portal
Order System Entities
•Granule List now using ECHO XML in the return•New entity “Data Basket” – handle to collection of order•Demo currently orders all data centrally will need to be adapted
21CLASS –Simple NOAA Archive Access Portal
Order Sequence Diagram
•Data basket ID is the API handle•OGSA-DAI activities handle the extraction and packaging•Status check is currently by basket only (XML return)•Finished granules return a URL handle to the entity
22CLASS –Simple NOAA Archive Access Portal
Future Plans
Plan to deliver two documents A discussion of the OGSA-DAI choice and its
implications A detailed discussion of the use of baskets
Would like to identify a CLASS build and date for OE inclusion This would require some review process (including
outside) and a versioning of OE Plan to include AVHRR in in the asynchronous demo Plan to include non-granule based asynchronus (i.e.
models, transforms) data extractions Enhance the connection with GEO-IDE
23CLASS –Simple NOAA Archive Access Portal
Future plans
Extend the OE documentation to the CLASS development team and provide support
Extensive work on the connector layer Expand proto-type to cover more NOAA and non-NOAA
data types and services and document lessons learned Release the work to the public
http://sourceforge.net/projects/vxoware/ Seek outside review