Unidata’s CDM: netCDF, OPeNDAP, HDF

Ethan Davis10 January 2014

ESIP Winter Meeting – OPeNDAP BoF

OPeNDAP support in netCDF libraries

• netCDF libraries– netCDF-C (and Fortran, C++, perl, R, python, ...)– netCDF-Java (implements the CDM)

• OPeNDAP support– Existing DAP2 support

• Client: netCDF-C and netCDF-Java/CDM• Server: THREDDS Data Server (TDS)

– DAP4 development• Client: netCDF-Java/CDM• Server: THREDDS Data Server (TDS)

– Future DAP4 support (real soon now!)• Client: netCDF-C

Application

OPeNDAP

NetCDF-3

HDF4

Dispatch

Data Access API

NetCDF-4

…

C Library Architecture

HDF5

NetcdfDataset

Application

Scientific Feature Types

NetCDF-Java/

CDM architecture

THREDDS

Catalog.xml

OPeNDAP

NetCDF-3

HDF4

I/O service provider

GRIB

GINI

NIDS

NetcdfFile

NetCDF-4

…Nexrad

DMSP

CoordSystem Builder

Datatype Adapter

NcML

CDM Architecture

cdmremote

Local Files

Remote Datasets

NetCDF Classic data model(aka netCDF-3)

• Multidimensional arrays of primitive values– byte, char, short, int, float, double

• Key/value attributes• Shared dimensions• Fortran77

OPeNDAP and netCDF-3

• DAP2 carried this data model pretty well– Some translation required

• String vs char*• Shared dimensions vs Grids/Maps

• DAP2 extras– Sequences, relational queries

• HDF-5 extras– Groups, structures, variable length arrays,...

NetCDF Enhanced Data Model(aka netCDF-4)

OPeNDAP and netCDF-4• netCDF-4 extends classic model:

– Groups; Structures; Sequences/varLength arrays• Mapping into DAP2:

– Shared dims vs Grids– Primitive types– No Groups in DAP2– No relational queries in CDM

• DAP4 will carry the netCDF enhanced data model– Still netCDF-4 doesn't do relational queries

• Something for CDM scientific data types (efficient?)• Server-side (TDS) for efficient "relational" queries

Thank You

Commitment to Backward CompatibilityBecause preserving access to archived datafor future generations is sacrosanct:

• NetCDF-4 provides both read and write access to all earlier forms of netCDF data.

• Existing C and Fortran netCDF programs will continue to work after recompiling and relinking.

• Future versions of netCDF will continue to support both data access compatibility and API compatibility.

NetCDF Disk Formats

Thank You