Post on 23-Feb-2016
description
transcript
Unidata’s CDM: netCDF, OPeNDAP, HDF
Ethan Davis10 January 2014
ESIP Winter Meeting – OPeNDAP BoF
OPeNDAP support in netCDF libraries
• netCDF libraries– netCDF-C (and Fortran, C++, perl, R, python, ...)– netCDF-Java (implements the CDM)
• OPeNDAP support– Existing DAP2 support
• Client: netCDF-C and netCDF-Java/CDM• Server: THREDDS Data Server (TDS)
– DAP4 development• Client: netCDF-Java/CDM• Server: THREDDS Data Server (TDS)
– Future DAP4 support (real soon now!)• Client: netCDF-C
Application
OPeNDAP
NetCDF-3
HDF4
Dispatch
Data Access API
NetCDF-4
…
C Library Architecture
HDF5
NetcdfDataset
Application
Scientific Feature Types
NetCDF-Java/
CDM architecture
THREDDS
Catalog.xml
OPeNDAP
NetCDF-3
HDF4
I/O service provider
GRIB
GINI
NIDS
NetcdfFile
NetCDF-4
…Nexrad
DMSP
CoordSystem Builder
Datatype Adapter
NcML
CDM Architecture
cdmremote
Local Files
Remote Datasets
NetCDF Classic data model(aka netCDF-3)
• Multidimensional arrays of primitive values– byte, char, short, int, float, double
• Key/value attributes• Shared dimensions• Fortran77
OPeNDAP and netCDF-3
• DAP2 carried this data model pretty well– Some translation required
• String vs char*• Shared dimensions vs Grids/Maps
• DAP2 extras– Sequences, relational queries
• HDF-5 extras– Groups, structures, variable length arrays,...
NetCDF Enhanced Data Model(aka netCDF-4)
OPeNDAP and netCDF-4• netCDF-4 extends classic model:
– Groups; Structures; Sequences/varLength arrays• Mapping into DAP2:
– Shared dims vs Grids– Primitive types– No Groups in DAP2– No relational queries in CDM
• DAP4 will carry the netCDF enhanced data model– Still netCDF-4 doesn't do relational queries
• Something for CDM scientific data types (efficient?)• Server-side (TDS) for efficient "relational" queries
Thank You
Commitment to Backward CompatibilityBecause preserving access to archived datafor future generations is sacrosanct:
• NetCDF-4 provides both read and write access to all earlier forms of netCDF data.
• Existing C and Fortran netCDF programs will continue to work after recompiling and relinking.
• Future versions of netCDF will continue to support both data access compatibility and API compatibility.
NetCDF Disk Formats
Thank You