Netcdf course 2008-10-28

• Intro

• CF convention

• Netcdf excercises

File format

• library for fortran, c, c++, java, etc...

• text, xml and binary format

• From netcdf4 HDF storage

NetCDFNetCDF

Data modelNetCDFNetCDF

Data model for netcdf and others.

Also usable for hdf, opendap, grib, etc. See the java library for details

Data model for netcdf and others.

Also usable for hdf, opendap, grib, etc. See the java library for details

>> url = 'http://tashtego.marine.rutgers.edu:8080/thredds/dodsC/roms/nena/SRTM/w100w20s10n90.nc';>> lon = nc_varget ( url, 'lon' );>> size(lon)

ans =

9600 1

>> lon(1:5)

ans =

-99.9958 -99.9875 -99.9792 -99.9708 -99.9625

Matlab

Using the open source MEXNC you can read and write netcdf from matlab.

Using the open source MEXNC you can read and write netcdf from matlab.

ArcGis

ArcGis also reads and writes netcdf files.

ArcGis also reads and writes netcdf files.

‘VBA interface to netcdfReDim myVar(0 To dimLen - 1)'Open Filestatus = nc_open(fileName, NC_NOWRITE, ncid)status = nc_inq_varid(ncid, varName, varid)status = nc_get_var_int(ncid, varid, myVar(0))

Excel

Excel is limited to 32768 rows. Be careful!Excel only understands 1D & 2D.

Excel is limited to 32768 rows. Be careful!Excel only understands 1D & 2D.

Emacs

xml representation of a netcdf file

xml representation of a netcdf file

Other Tools

NCO#diffncdiff -v time file1.nc file2.nc#compression & packingncpdq -4 -L 9 in.nc out.nc # Deflated packing (~80% lossy compression)#selecting variables by regexncks -v '^Q..' in.nc # Q01--Q99, QAA--QZZ, etc.

IDVVery usefulVery useful

Web hyperslabs, cool!Web hyperslabs, cool!

Not so stable.Not so stable.

Why netcdf?

• Open standard

• Transparent path to improvement

• Prepares us for future innovations

CF CF StandardStandard

Use case

• Customer finds a output file and wants to understand it?

CF CF StandardStandard

• What? (Quantity)

• Where? (Location, cell centered, sigma, coordinate system?, projection?)

• When? (Time, calendar, interval, timezone?)

• How much? (Value, unit, , cell method)

An output file should contain:

Open standard

• Climate & Forecasting Metadata convention

– Version 1.3, 9 October, 2008

– Main contributors: NOAA, MetOffice, UCAR, NCAR, USGS, PCMDI

• Formalized yet open

• Well designed

CF CF StandardStandard

Open standard

• What is standardized?

– Variable names

– Units

– Locations (lat, lon)

– Levels (z, sigma, bar)

– Cells (boundaries, measures)

– Time

CF CF StandardStandard

Variable names

• runoff_amount: "Amount" means mass per unit area. Runoff is the liquid water which drains from land. If not specified, ....

• runoff_flux: Runoff is the liquid water which drains from land. If not specified, "runoff" refers to the sum of surface runoff and subsurface drainage. In accordanceaccordance with common ...

CF CF StandardStandard

Variable names

• sea_water_potential_temperature: Potential temperature is the temperature a parcel of air or sea water would have if moved adiabatically to sea level pressure.

• normalized_difference_vegetation_index: "Normalized_difference_vegetation_index", usually abbreviated to NDVI, is an index calculated from reflectances ....

CF CF StandardStandard

Variable names

• http://cf-pcmdi.llnl.gov/documents/cf-standard-names (v10 21 oktober)

CF CF StandardStandard

Netcdf Excercise

Checkout dhs-cursus

http://svn.wldelft.nl/repos/dhs-cursus/trunk

Add dependency

Create dependency

• Right click project -> project dependencies

f77_netcdf

f90_netcdf

netcdf

program

Add include directory

Data modelNetCDFNetCDF

Data model for netcdf and others.

Also usable for hdf, opendap, grib, etc. See the java library for details

Data model for netcdf and others.

Also usable for hdf, opendap, grib, etc. See the java library for details

Dataset functions

• NF90_CREATE • NF90_OPEN • NF90_CLOSE• NF90_REDEF • NF90_ENDDEF • NF90_INQUIRE Family • NF90_INQ_LIBVERS • NF90_STRERROR • NF90_SET_FILL

NF90_CREATE

function nf90_create(path, cmode, ncid)

character (len = *), intent(in ) :: path

integer, intent(in ) :: cmode

integer, optional, intent(in ) :: initialsize

integer, optional, intent(inout) :: chunksize

integer, intent( out) :: ncid

integer :: nf90_create

NF90_CREATE

Dimensions

• NF90_DEF_DIM

• NF90_INQ_DIMID

• NF90_INQUIRE_DIMENSION

• NF90_RENAME_DIM

NF90_DEF_DIM

function nf90_def_dim(ncid, name, len, dimid)

integer, intent( in) :: ncid

character (len = *), intent( in) :: name

integer, intent( in) :: len

integer, intent(out) :: dimid

integer :: nf90_def_dim

NF90_DEF_DIM

Attributes

• NF90_PUT_ATT

• NF90_INQUIRE_ATTRIBUTE

• NF90_GET_ATT

• NF90_COPY_ATT

• NF90_RENAME_ATT

• NF90_DEL_ATT

NF90_PUT_ATT

function nf90_put_att(ncid, varid, name, values) integer, intent( in) :: ncid, varid character(len = *), intent( in) :: name any valid type, scalar or array of rank 1, & intent( in) :: values integer :: nf90_put_att

NF90_PUT_ATT

Variables• NF90_DEF_VAR: Create a Variable • NF90_DEF_VAR_CHUNKING: Set Var Chunking Parameters • NF90_INQ_VAR_CHUNKING: Get Var Chunking Parameters • NF90_DEF_VAR_DEFLATE: Set Var Compression Parameters • NF90_INQ_VAR_DEFLATE: Get Var Compression Parameters • NF90_DEF_VAR_FLETCHER32: Set Var Fletcher32 Filter • NF90_INQ_VAR_FLETCHER32: Get Var Fletcher32 Filter • NF90_DEF_VAR_ENDIAN: Set Var Endianness • NF90_INQ_VAR_ENDIAN: Get Var Endianness • NF90_INQUIRE_VARIABLE: Get Var Metadata • NF90_PUT_VAR: Write data • NF90_GET_VAR: Read data • NF90_RENAME_VAR

NF90_DEF_VAR

function nf90_def_var(ncid, name, xtype, dimids, varid)

integer, intent( in) :: ncid

character (len = *), intent( in) :: name

integer, intent( in) :: xtype

integer, dimension(:), intent( in) :: dimids

integer, intent(out) :: varid

integer :: nf90_def_var

NF90_DEF_VAR

NF90_PUT_VAR

function nf90_put_var(ncid, varid, values, start, count, stride, map) integer, intent( in) :: ncid, varid any valid type, scalar or array of any rank, & intent( in) :: values integer, dimension(:), optional, intent( in) :: start, count, stride, map integer :: nf90_put_var

NF90_PUT_VAR

NF90_OPEN

function nf90_open(path, mode, ncid, chunksize) character (len = *), intent(in ) :: path integer, intent(in ) :: mode integer, intent( out) :: ncid integer, optional, intent(inout) :: chunksize integer :: nf90_open

NF90_INQUIRE

function nf90_inquire(ncid, nDimensions, nVariables, nAttributes, & unlimitedDimId, formatNum) integer, intent( in) :: ncid integer, optional, intent(out) :: nDimensions, nVariables, & nAttributes, unlimitedDimId, & formatNum integer :: nf90_inquire

NF90_INQUIRE

NF90_INQUIRE_VARIABLE

function nf90_inquire_variable(ncid, varid, name, xtype, ndims, dimids, nAtts) integer, intent( in) :: ncid, varid character (len = *), optional, intent(out) :: name integer, optional, intent(out) :: xtype, ndims integer, dimension(*), optional, intent(out) :: dimids integer, optional, intent(out) :: nAtts integer :: nf90_inquire_variable

NF90_INQUIRE_VARIABLE

Read the data using

• nf90_inquire_attributes

• nf90_get_var