Data DiscoveryTools and Services

Part AA Panel Presentation:

Pat Liggett, Ron Weaver, Sean Hardman, Ben Holt(with help from Dave Gallaher, NSIDC)

October 15, 16 2008Burbank Airport Marriott

User Case Scenario: Scientific and comprehensive data view/access/news for polar regions.Starting point was NSIDC ‘Arctic Sea Ice News and Analysis’ page.

Sea ice extentPresent Daily-Anomaly

Data - AMSR-E, SSM/I, QuikSCAT

Arctic Data Collection

°Region

°Time

°Ice extent

°Sea surface temperature

°Ocean winds

°Ocean color

°Sea ice freeboard

°Sea ice motion

°Clouds

°Albedo

°Atmospheric fields

Etc.

Note: Could extend toland, ice sheets etc

Sea Surface TemperaturePresent

Data - AVHRR, Modis, GHRSST

Ocean WindsPresent Ex.Monthly Climatology

Data - QuikSCAT, AMSR-E, SSM/I

Ocean ColorPresent Ex. Annual (monthly)

Anomaly

Data - Modis, SeaWiFS

Additional products -Sea ice freeboard,Sea ice age, etc

Additional products- Sea ice motion,Clouds, Atmospheric fields,etc

Ex. Annual (monthly)Anomaly

B. Holt-JPL

User Scenario: Arctic IceNASA Views the Polar Region

10/14/08 2008 UWG 3

Joint Implementation Scenario

Example of a Dual Purpose Interface

July 16-17, 2008 PO.DAAC Review, NASA HQ 6

Backup Material

•• Create a simple to use interface(s) for Create a simple to use interface(s) for Cryospheric Cryospheric and Physicaland PhysicalOceanographic dataOceanographic data

•• Get more data distributed to more usersGet more data distributed to more users

•• Build the 80% solution for experts to novicesBuild the 80% solution for experts to novices

•• Get the user just the data they need, in the format they Get the user just the data they need, in the format theywantwant……quickly. Accelerate the use of data.quickly. Accelerate the use of data.

•• Allow users to display and interact with the data Allow users to display and interact with the data

•• Create a standard & flexible framework for new applicationsCreate a standard & flexible framework for new applications

•• BuildBuild a system that allows the user to analyze the data, at thea system that allows the user to analyze the data, at theserver, so they donserver, so they don’’t need to move as much datat need to move as much data

• Allow the system to be expanded by users or other machines

The Objective

1.1. User describes what they wantUser describes what they want

2.2. User selects What data (product)User selects What data (product)

3.3. User selects Where (location)User selects Where (location)

4.4. User selects When (time interval)User selects When (time interval)

5.5. User selects filters (if any) User selects filters (if any) Then:Then:

6.6. If theIf the number of data files is small the usernumber of data files is small the useris given thumbnailsis given thumbnails

7.7. User can view the thumbnailUser can view the thumbnail

8.8. The user can analyze the dataThe user can analyze the data

9.9. User can select the output formatUser can select the output format

Quick search, retrieval, display & deliveryQuick search, retrieval, display & delivery

In the orderIn the orderthe userthe userwants:wants:

Implementation Scenario of Arctic Ice Scenario atPODAAC and NSIDC

• Extend Sea Ice Extent Map to be accessible through both PODAAC and NSIDC.• Single map, accessible from PODAAC and NSIDC.• Map interface is modified and maintained by NSIDC based on shared requirements from PODAAC and NSIDC

• Provide local and remote access to data products identified by region selected via an interactivetable.• PODAAC middle ware developed to allow access to products at both sites. Technology is operational from both sites

and is modified and maintained by PODAAC based on shared requirements from PODAAC and NSIDC.

• Provide Level 2 and Level 3 subsetting capability for all available products for the area of interestand based upon requirements for interoperability and data display and distribution.• PODAAC develops and extends L2, L3 subsetters for PODAAC products.• NSIDC develops and extends L2, L3 subsetters for NSIDC products.

• Provide all products in a compatible form and format for display and further tool input.• PODAAC and NSIDC provide all products in an agreed form and format for display and further tool input and

distribution.

• Provide visualization capability for all selected products.• PODAAC and/or NSIDC develops visualization capability for product overlay to Ice Extent map based on agreed upon formats

and requirements.

• Provide DataCasting availability at both sites for notification of products• JPL developed Datacasting will be ported and configured for NSIDC by PODAAC.

• Provide a single User Interface accessible from both sites that integrates the Sea Ice Extent Map,InteractiveTable of Products, visualization capability, format conversion, datacasting andproduct delivery.• Interface is developed and maintained by NSIDC based upon requirements from both PODAAC and NSIDC.

• Provide machine to machine accessible products from both NSIDC and PODAAC• NSIDC and PODAAC select a single point of access for user selected products.• FTP directory specific to the user selection located at one or the other site.

July 16-17, 2008 PO.DAAC Review, NASA HQ 9

Target Capabilities

•• Query & Display MapQuery & Display Map - Query and display data in thesame widow so the the context of the data relativeto the region of interest can be seen along with thethumbnail view.

•• Value FilteringValue Filtering - Filtering of search results byparameter values, e.g. minimum, maximum,distribution, etc.

•• PeriodicPeriodic - The user may want the same time-seriesperiod for a study, for example, September of thepast 10 years.

•• Multi-Data SelectionMulti-Data Selection - Allow multiple productselections

•• Region SearchRegion Search - Using geographic terms for spatialspecification, e.g. Greenland, Antarctica, Alaskaetc.

Target Capabilities - Continued

•• Display data in Polar coordinatesDisplay data in Polar coordinates - such as PolarStereo

•• Linked List & MapLinked List & Map - Dynamically link the top listedgranule to the top displayed thumbnail

•• Analyze, Plot & AnimateAnalyze, Plot & Animate - Give the user the chance to work withthe data.

•• On demand subsettingOn demand subsetting - select just the data in the region ofinterest.

•• DualDual data displaydata display - Allow comparisons by time, product andlocation.

•• Multi-format outputMulti-format output - Give it to the user in the format they need.•• SubscriptionSubscription - Allow users to get new data as available.•• User modelingUser modeling - Allow users to build their own processes,

models and plug-ins to extend the system. (machine tomachine)

Capabilities - Comparison

NYNNYNNYPolar View

NNNNNNY/NNUserModeling

YYYYNNNNQuery &

MapDisplay

YYYYYY/NYYMulti-DataSelection

Y/N

N

N

N

N

N

N

Y/N

Y

NASA

WIST

Subscription

Multi-formatOutput

DualDispl

ay

On DemandSubsetting

Analyze,Plot &

Animate

Linked List& Map

Regions

Periodic

Value Filter

Agency

System

NNNNNNN

Y/NNNYNNN

NNYNNNN

NNNNYYN

Y/NYY/NYNYY

Y/NYYNNNN

YYYYNNN

NYNY/NYNY

NYNNNY/NY

Alaska OceanObserving

University ofNew Hamp.

University ofArizonaNASANSIDCNASA

GoddardNOAA

Data CatalogExplorerArctic RIMSRangeViewNEODADDIGiovanniCDC

Operational Scenario

• A user at either PODAAC or NSIDC selects a tool for searching for and selecting data. Thetool, by all appearances, is identical at either site. Only the banners are specific to the datacenter. The tool provides the ability to select a region of interest anywhere on the globe.The user is able to enter a time or time range for the area of interest. After entering thisinformation, a list of available products in the area of interest and time at both PODAACand NSIDC (at a minimum) is provided. This list of products allows the user to selectproducts from the list for display and/or download. As products are selected, a display ofthe region of interest is updated with a visualization of the product overlaying the region.The updated display is also annotated with the information from the data related to time,location, name of product and information pertinant to the data (speed, temperature, etc).Products can be overlayed upon each other, or can be displayed individually on the area ofinterest. The user can download a copy of the image currently displayed which willinclude the annotation information. In addition, the user can request data products fordownload in one of several available formats. The selectable formats shall include but notbe limited to: HDF, NetCDF . The user can request to be notified of new products from thislist as they become available in this region. The user can also request notification for anyother products, not in the list but within the region when they become available at any ofthe data locations, PODAAC and NSIDC (at a minimum). The user can request that theseproducts be available as input into other tools or services provided by the data locations,PODAAC and NSIDC (at a minimum) for further manipulation, display or analysis.

July 16-17, 2008 PO.DAAC Review, NASA HQ 13

July 16-17, 2008 PO.DAAC Review, NASA HQ 14

Functional Requirements

• Functional Requirements for a single tool providing both PODAAC orNSIDC data

• JPL’s Requirements List• Support user request for information and/or products within a specified area and time from a range

of products resident locally and remotely.• Provide ability to select area of interest by location and time.• Provide ability to select data available in area of interest by location and time from multiple

measurements both local and remote.• Provide data center identification of data location .• Provide visualization of selected products as an overlay within specified area of interest.• Provide overlay visualization capability of multiple selected products within specified area of

interest.• Provide annotation of the visualized overlays which includes not less than name of product, time of

acquisition, lat/lon, measurement value range and type (speed, height, temperature).• Provide acquisition of annotated image products displayed within area of interest.• Provide acquisition of selected data products displayed within area of interest.• Provide acquired data products in selectable data format of at least HDF and NetCDF• Provide notification of new products as they become available within a specified period of time.• Provide notification of new products as they become available within the specified area of interest.• Provide machine to machine interface for acquisition of products via FTP.• Provide data linkage for selected products to other available tools and services at local and remote

data center.• Provide access to tool described above from either PODAAC or NSIDC.• Provide data center unique product emphasis such as product table list order, or area of interest

emphasis (masking, etc) .

July 16-17, 2008 PO.DAAC Review, NASA HQ 15

Functional Requirements

• Functional Requirements for a single tool providing both PODAAC or NSIDC data• NSIDC’s Requirements List

• A Web-accessible system to provide NSIDC's users the ability to preview and analyze key data setsin our collection. The interface will be straightforward, in the spirit of the CDC reanalysis web site,but will provide "something better" which is TBD. The system will initially focus on the Greenlandarea, but will not be limited to only work in that region

• Performance and Design Objectives1. system response times will depend on action performed:

1. 5-second response time for display results, or in-progress indicator2. ??-second response time for data delivery (this should probably be a rate, as in seconds per file ordered, or

seconds per GB of data ordered)2. the user will be able to access the system with a common web browser, with no plug-ins or special

software for the user to download; Ajax functionality is specifically in-scope; the current list ofsupported browsers is

1. Windows: IE 7.x; Firefox 3.2. Macintosh/Linux: Firefox 3.x

3. system will be built with open source (C) and/or Government-funded technologies (C) (W says"preferably open source")

4. the user experience will be consistent; users will not be required to to know where the data arecoming from, and will not be required to learn this in order to use the system.

5. to maximize reuse and minimize maintenance costs, there will be a standard user interface built foreach kind of information (spatial, temporal, etc). Each of these interfaces may have different waysfor the user to specify input, (e.g. specifying a date using a calendar widget vs. a text box, but therewill not be 2 implementations of a calendar widget)

6. the system will incorporate reasonable security measures, for example, using ssl for at least userauthentication