OGC "Geo-interface for Atmosphere, OGC "Geo-interface for Atmosphere, Land, Earth, and Ocean netCDF" Land, Earth, and Ocean netCDF"

(GALEON) Interoperability Experiment (GALEON) Interoperability Experiment

77thth September 2005 September 2005

Norman Barker (RSI)Norman Barker (RSI)

Community ContextCommunity Context

• Many existing client/server data systems in the atmosphere, Many existing client/server data systems in the atmosphere, ocean sciences communities (AIS or ESIS)ocean sciences communities (AIS or ESIS)

• Standards evolving from original base in GIS communityStandards evolving from original base in GIS community• Standards based gateway needed between existing “AIS” data Standards based gateway needed between existing “AIS” data

systems and traditional GIS worldsystems and traditional GIS world• Overall objectivesOverall objectives

– standardstandard interfaces interfaces– gatewaygateway implementation implementation

OGC Web Coverage ServiceOGC Web Coverage Service

• OpenGIS specification with implementations by Ionic, OpenGIS specification with implementations by Ionic, GeoServer and MapserverGeoServer and Mapserver

• Allows temporal / spatial subsetting, and reprojection of Allows temporal / spatial subsetting, and reprojection of underlying data using http requestsunderlying data using http requests

• Clients implemented by ESRI, Cadcorp, Ionic, qvSIG, uDig, and Clients implemented by ESRI, Cadcorp, Ionic, qvSIG, uDig, and ongoing Unidata GALEON project to develop services and ongoing Unidata GALEON project to develop services and clients for netCDF dataclients for netCDF data

• WCS is a superset of the functionality offered by a WMS WCS is a superset of the functionality offered by a WMS allowing integration with thin web clientsallowing integration with thin web clients

• Server side is fully scalable.Server side is fully scalable.

OpenGIS WCS netCDF InterfaceOpenGIS WCS netCDF Interface

WFS

Application

DTED

HDF-EOS

WCS

Static GIS Features

GML

NITF

GeoTIFFNetCDF Interface

ADDEOpenDAP netCDF

file

Service Gateway ObjectivesService Gateway Objectives

• Make communities aware of one anotherMake communities aware of one another• Build standard interface to the many netCDF/OPeNDAP data Build standard interface to the many netCDF/OPeNDAP data

servers already in operationservers already in operation• Allow traditional atmospheric / oceanography data providers to Allow traditional atmospheric / oceanography data providers to

migrate toward one standards-based servicemigrate toward one standards-based service• Allow traditional GIS clients to access netCDF/OPeNDAP Allow traditional GIS clients to access netCDF/OPeNDAP

datasets via standards-based interfacedatasets via standards-based interface

Interface ObjectivesInterface Objectives

• Evaluate effectiveness of ncML-GML in WCS data encoding… Evaluate effectiveness of ncML-GML in WCS data encoding… suggest extensionssuggest extensions

• Evaluate netCDF/OPeNDAP as WCS data transport (encoding) Evaluate netCDF/OPeNDAP as WCS data transport (encoding) mechanism … suggest extension if warrantedmechanism … suggest extension if warranted

• Investigate protocol adequacy for serving and interacting with Investigate protocol adequacy for serving and interacting with (5D) datasets involving multiple parameters (e.g., temperature, (5D) datasets involving multiple parameters (e.g., temperature, pressure, wind speed and direction) in three spatial dimensions pressure, wind speed and direction) in three spatial dimensions with two temporal (actual time, forecast time) dimensionswith two temporal (actual time, forecast time) dimensions

5D Datasets5D Datasets

ParticipantsParticipants

• Unidata/UCAR*Unidata/UCAR*• University of Florence*University of Florence*• NASA Geospatial Interoperability Office*NASA Geospatial Interoperability Office*• George Mason UniversityGeorge Mason University• CadCorpCadCorp• ESRIESRI• JPLJPL• IMAA-CNRIMAA-CNR• Interactive InstrumentsInteractive Instruments• University of Applied SciencesUniversity of Applied Sciences• International University BremenInternational University Bremen• NERC NCAS/British Atmospheric Data CenterNERC NCAS/British Atmospheric Data Center• University of Alabama HuntsvilleUniversity of Alabama Huntsville• Research Systems, Inc. (IDL)Research Systems, Inc. (IDL)• Texas A&M UniversityTexas A&M University

*Initiating institutions*Initiating institutions

Interested ObserversInterested Observers

• EDINA: Edinburgh U. Data LibraryEDINA: Edinburgh U. Data Library• Harvard UniversityHarvard University

Non-member interest in Gateway Non-member interest in Gateway ImplementationImplementation

• University of Rhode Island (OPeNDAP group)University of Rhode Island (OPeNDAP group)• Pacific Marine Environment Laboratory (PMEL)Pacific Marine Environment Laboratory (PMEL)• Center for Land, Ocean, Atmosphere (COLA)Center for Land, Ocean, Atmosphere (COLA)• Marine Metadata Initiative lead by MBARI (Monterey Bay Marine Metadata Initiative lead by MBARI (Monterey Bay

Aquarium Research Institute)Aquarium Research Institute)• GODAE (Global Ocean Data Assimilation Experiment) led by GODAE (Global Ocean Data Assimilation Experiment) led by

FNMOC (Fleet Numerical Meteorological and Oceanographic FNMOC (Fleet Numerical Meteorological and Oceanographic Center)Center)

• Many current THREDDS/OPeNDAP server sitesMany current THREDDS/OPeNDAP server sites

TimelineTimeline

StartupStartup• GALEON WCS-THREDDS IE ad hoc Meeting (NYC TC):   January 19, 2005GALEON WCS-THREDDS IE ad hoc Meeting (NYC TC):   January 19, 2005• Activity Plan draft web publication:   March 18, 2005, revised May 11.Activity Plan draft web publication:   March 18, 2005, revised May 11.

ExecutionExecution

• Review Board Review: June 15, 2005Review Board Review: June 15, 2005• 30-day public review30-day public review• Planned kickoff date (execution start date):   mid-July, 2005Planned kickoff date (execution start date):   mid-July, 2005• Status Updates: intermittently online Status Updates: intermittently online • Update at next OGC TC meeting: November, 2005Update at next OGC TC meeting: November, 2005• Final testing and bug fixing:    5 months after kickoffFinal testing and bug fixing:    5 months after kickoff• Planned end date:Planned end date:    6 months after kickoff6 months after kickoff

Unidata GatewayUnidata Gateway

1.1. WCS gateway site serving NetCDF 5D gridded files. Datasets will be in THREDDS catalog, WCS gateway site serving NetCDF 5D gridded files. Datasets will be in THREDDS catalog, but with no OGC CSW catalog implementation. (GMU has plans for CSW catalog but with no OGC CSW catalog implementation. (GMU has plans for CSW catalog implementation): end of June, 205implementation): end of June, 205

2.2. WCS server ready for other client testing: end of July, 2005 WCS server ready for other client testing: end of July, 2005 3.3. Experiment with netcdf in addition to geotiff: starging August, 2005 Experiment with netcdf in addition to geotiff: starging August, 2005

RSI GALEON ImplementationRSI GALEON Implementation

• Data GenerationData Generation • Data IngestionData Ingestion• Data ProvisionData Provision• Data AccessData Access• Data VisualizationData Visualization• On-going Development and SupportOn-going Development and Support

SoftwareSoftware

• 33rdrd Party Software Party Software– Web Server (Apache)Web Server (Apache)– Web Coverage Server (MapServer)Web Coverage Server (MapServer)

• Bespoke (RSI)Bespoke (RSI)– System Admin/Ingestion ToolSystem Admin/Ingestion Tool– Visualization ToolVisualization Tool– System IntegrationSystem Integration

Ingestion ToolIngestion Tool

• Ingests NetCDF data into GDAL banded netCDF format for Ingests NetCDF data into GDAL banded netCDF format for WCS delivery via MapServerWCS delivery via MapServer

• Supports configurable multithreaded processingSupports configurable multithreaded processing• Accelerated ingestion using native system methods Accelerated ingestion using native system methods • All transactions are logged and can be queried All transactions are logged and can be queried viavia the the ingestion ingestion

GUIGUI• Central database of ingested files allowing other systems to Central database of ingested files allowing other systems to

integrate with the file storeintegrate with the file store

MapServer WCSMapServer WCS

• Open Source, based on GDALOpen Source, based on GDAL• Used extensively as a Web Map Service – been available as a Used extensively as a Web Map Service – been available as a

WCS for the last 18 monthsWCS for the last 18 months• Fully tested with Landsat / DEM type geotiff data, MIMAS, Fully tested with Landsat / DEM type geotiff data, MIMAS,

EDINA, NERC and our experienceEDINA, NERC and our experience• This is the first use of this WCS for serving NetCDF data, work This is the first use of this WCS for serving NetCDF data, work

is beginning to be echoed by GALEON participants is beginning to be echoed by GALEON participants • WCS makes the data store extensible to other clients who can WCS makes the data store extensible to other clients who can

parse XML responses and retrieve data via HTTP. parse XML responses and retrieve data via HTTP.

Vizualisation Tool OverviewVizualisation Tool Overview

• Modular IDL ImplementationModular IDL Implementation• Multi-View LayoutMulti-View Layout• Tab Sheets (1 per view)Tab Sheets (1 per view)• Animation ControlAnimation Control• ImportImport

– WCS (via Data Access Dialog)WCS (via Data Access Dialog)– Raw GeoTIFF FilesRaw GeoTIFF Files

• ExportExport– Image (of View)Image (of View)– MPEG (of View)MPEG (of View)

Data Access GUIData Access GUI

Vizualisation Tool GUIVizualisation Tool GUI

Vizualisation Tool DemoVizualisation Tool Demo

ResultsResults

• Interoperability between clients and serverInteroperability between clients and server• 3D + temporal support for netCDF coverages3D + temporal support for netCDF coverages• IDL WCS client capabilityIDL WCS client capability• Free IDL client for netCDF data via WCS using the IDL Virtual Free IDL client for netCDF data via WCS using the IDL Virtual

MachineMachine

PlansPlans

• WCS and IDL client has been released to the customerWCS and IDL client has been released to the customer• 40 years of climate netCDF data will be available via WCS using 40 years of climate netCDF data will be available via WCS using

either the free IDL client, or A.N.Other WCS client in Decembereither the free IDL client, or A.N.Other WCS client in December• Plans for IDL OpenGIS client functionality to be included in the Plans for IDL OpenGIS client functionality to be included in the

IDL distributionIDL distribution• Migration to true N-D coverage supportMigration to true N-D coverage support

Open Source Community ProgressOpen Source Community Progress

• GDAL added netCDF driver (Denis Nadeau + Frank GDAL added netCDF driver (Denis Nadeau + Frank Warmerdam)Warmerdam)

• Geoserver added WCS support, and working towards N-D Geoserver added WCS support, and working towards N-D server + client functionality (Simone Giannecchini)server + client functionality (Simone Giannecchini)

• NCML-GML (Stefano Nativi)NCML-GML (Stefano Nativi)• CSML (Andrew Woolf)CSML (Andrew Woolf)

Any Questions?Any Questions?