30
Geoapplications development http://rgeo.wikience.org Higher School of Economics, Moscow, www.cs.hse.ru
Geoapplications development
http://rgeo.wikience.org
Higher School of Economics, Moscow, www.cs.hse.ru
Agenda 2
•
•
•
Examples of geospatial raster data 3
Subset types 4
•
•
•
Java libs to read/write rasters 5
•
•
•
Geospatial raster file formats (143 supported by GDAL) 6
Arc/Info ASCII Grid ACE2 ADRG/ARC Digitilized Raster Graphics (.gen/.thf) Arc/Info Binary Grid (.adf) AIRSAR Polarimetric Azavea Raster Grid Magellan BLX Topo (.blx, .xlb) Bathymetry Attributed Grid (.bag) Microsoft Windows Device Independent Bitmap (.bmp) BPG (Better Portable Graphics) BSB Nautical Chart Format (.kap) VTP Binary Terrain Format (.bt) CALS Type I CEOS (Spot for instance) DRDC COASP SAR Processor Raster TerraSAR-X Complex SAR Data Product Convair PolGASP data USGS LULC Composite Theme Grid DirectDraw Surface Spot DIMAP (metadata.dim) ELAS DIPEx DODS / OPeNDAP First Generation USGS DOQ (.doq) New Labelled USGS DOQ (.doq) Military Elevation Data (.dt0, .dt1, .dt2) Arc/Info Export E00 GRID ECRG Table Of Contents (TOC.xml) ERDAS Compressed Wavelets (.ecw) ESRI .hdr Labelled Erdas Imagine Raw
NASA ELAS ENVI .hdr Labelled Raster Epsilon - Wavelet compressed images ERMapper (.ers) Envisat Image Product (.n1) EOSAT FAST Format FIT FITS (.fits) Fuji BAS Scanner Image Generic Binary (.hdr Labelled) GeoPackage Oracle Spatial GeoRaster GSat File Format Graphics Interchange Format (.gif) WMO GRIB1/GRIB2 (.grb) GMT Compatible netCDF GRASS Raster Format GRASS ASCII Grid Golden Software ASCII Grid Golden Software Binary Grid Golden Software Surfer 7 Binary Grid GSC Geogrid Generic Tagged Arrays (.gta) TIFF / BigTIFF / GeoTIFF (.tif) NOAA .gtx vertical datum shift GXF - Grid eXchange File Hierarchical Data Format Release 4 (HDF4) Hierarchical Data Format Release 5 (HDF5) HF2/HFZ heightfield raster Erdas Imagine (.img)
Image Display and Analysis (WinDisp) ILWIS Raster Map (.mpr,.mpl) Intergraph Raster IRIS ISCE raster USGS Astrogeology ISIS cube (Version 2) USGS Astrogeology ISIS cube (Version 3) JAXA PALSAR Product Reader (Level 1.1/1.5) Japanese DEM (.mem) JPEG JFIF (.jpg) JPEG-LS JPEG2000 (.jp2, .j2k) JPEG2000 (.jp2, .j2k) JPEG2000 (.jp2, .j2k) JPEG2000 (.jp2, .j2k) JPEG2000 (.jp2, .j2k) JPIP (based on Kakadu) KEA KMLSUPEROVERLAY KRO NOAA Polar Orbiter Level 1b Data Set (AVHRR) Erdas 7.x .LAN and .GIS FARSITE v.4 LCP Format Daylon Leveller Heightfield NADCON .los/.las Datum Grid Shift MBTiles OziExplorer .MAP In Memory Raster Vexcel MFF Vexcel MFF2
MG4 Encoded Lidar Multi-resolution Seamless Image Database Meteosat Second Generation EUMETSAT Archive native (.nat) NLAPS Data Format NOAA NGS Geoid Height Grids NITF (.ntf, .nsf, .gn?, .hr?, .ja?, .jg?, .jn?, .lf?, .on?, .tl?, .tp?, etc.) NetCDF NTv2 Datum Grid Shift Northwood/VerticalMapper Classified Grid Format .grc/.tab Northwood/VerticalMapper Numeric Grid Format .grd/.tab OGDI Bridge OZI OZF2/OZFX3 PCI .aux Labelled PCI Geomatics Database File PCRaster Geospatial PDF NASA Planetary Data System Planet Labs Mosaics API Portable Network Graphics (.png) PostGIS Raster (previously WKTRaster) Netpbm (.ppm,.pgm) R Object Data Store Rasdaman Rasterlite - Rasters in SQLite DB Swedish Grid RIK (.rik) Raster Matrix Format (*.rsw, .mtw) ROI_PAC Raster Raster Product Format/RPF (CADRG, CIB) RadarSat2 XML (product.xml) Idrisi Raster Sentinel SAFE (manifest.safe) SAGA GIS Binary format SAR CEOS ArcSDE Raster USGS SDTS DEM (*CATD.DDF) SGI Image Format Snow Data Assimilation System Standard Raster Product (ASRP/USRP) SRTM HGT Format Terragen Heightfield (.ter) EarthWatch/DigitalGlobe .TIL TerraSAR-X Product USGS ASCII DEM / CDED (.dem) VICAR GDAL Virtual (.vrt) OGC Web Coverage Service WEBP OGC Web Map Service OGC Web Map Tile Service X11 Pixmap (.xpm) ASCII Gridded XYZ ZMap Plus Grid
CDM architecture 8
Geospatial raster types 9
•
•
•
http://modis-atmos.gsfc.nasa.gov/MOD04_L2/grids.html http://meso-a.gsfc.nasa.gov/val/projects/gpm/swath/index.html
Model VS Storage (1) 10
1 2 3 4 1
2
3
4
1 2
3 4
Model VS Storage (2) 11
Model: several rasters in single file (2) 13
Product Period Datasets Format File name example
AMIP/DOE Reanalysis 2
Year, 6 hours
1 NetCDF pres.sfc.1979.nc pres.sfc.1980.nc
MODIS L3 Atmosphere
Day, Day
>600 HDF4
MOD08_D3.A2000061.051.2010273210218.hdf MOD08_D3.A2000062.051.2010273161753.hdf
CFSR Month, 1 hour
1 Grib2 ocnsst.l.gdas.198401.grb2 ocnsst.l.gdas.198402.grb2
MERRA Day, 1..24 hrs
1.. HDF4
MERRA200.prod.assim.tavg1_2d_lnd_Nx. 20000718.hdf
Aura satellite, OMI radiometer
Day, Day
14 HDF5
OMI-Aura_L3-OMSO2e_2004m1001_v003- 011m0526t144250.he5
raro@ubuntu-pelligrini:/mnt/hgfs/RS_DATA/Landsat8/LC81790212015146-SC20150806075046$
gdalinfo ./LC81790212015146LGN00_sr_band1.tif
Driver: GTiff/GeoTIFF
Files: ./LC81790212015146LGN00_sr_band1.tif
Size is 8191, 8271
Coordinate System is:
PROJCS["WGS 84 / UTM zone 37N",
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433],
AUTHORITY["EPSG","4326"]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",0],
PARAMETER["central_meridian",39],
PARAMETER["scale_factor",0.9996],
PARAMETER["false_easting",500000],
PARAMETER["false_northing",0],
UNIT["metre",1,AUTHORITY["EPSG","9001"]],
AUTHORITY["EPSG","32637"]]
Origin = (224385.000000000000000,6322515.000000000000000)
Pixel Size = (30.000000000000000,-30.000000000000000)
Metadata:
AREA_OR_POINT=Area
Band_1=band 1 surface reflectance
Image Structure Metadata:
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 224385.000, 6322515.000) ( 34d27'55.58"E, 56d57'49.93"N)
Lower Left ( 224385.000, 6074385.000) ( 34d43' 3.37"E, 54d44'27.51"N)
Upper Right ( 470115.000, 6322515.000) ( 38d30'26.82"E, 57d 2'42.39"N)
Lower Right ( 470115.000, 6074385.000) ( 38d32' 5.80"E, 54d48'56.61"N)
Center ( 347250.000, 6198450.000) ( 36d33'23.03"E, 55d54'25.94"N)
Band 1 Block=8191x1 Type=Int16, ColorInterp=Gray
Description = band 1 surface reflectance
GDAL output in WKT for Landsat 8 Moscow scene
don’t worry due to font size – we are looking closer on it
just in several slides
14
http://www.geoapi.org/3.0/javadoc/org/opengis/referencing/doc-files/WKT.html
previous lesson
Origin = (224385.000000000000000,6322515.000000000000000)
Pixel Size = (30.000000000000000,-30.000000000000000)
Metadata:
AREA_OR_POINT=Area
Band_1=band 1 surface reflectance
Image Structure Metadata:
INTERLEAVE=BAND Corner Coordinates:
Upper Left ( 224385.000, 6322515.000) ( 34d27'55.58"E, 56d57'49.93"N)
Lower Left ( 224385.000, 6074385.000) ( 34d43' 3.37"E, 54d44'27.51"N)
Upper Right ( 470115.000, 6322515.000) ( 38d30'26.82"E, 57d 2'42.39"N)
Lower Right ( 470115.000, 6074385.000) ( 38d32' 5.80"E, 54d48'56.61"N)
Center ( 347250.000, 6198450.000) ( 36d33'23.03"E, 55d54'25.94"N)
Band 1 Block=8191x1 Type=Int16, ColorInterp=Gray
Description = band 1 surface reflectance
GDAL metadata for GeoTIFF 15
In this case GeoTIFF contains 1 band (raster), cell data type is 16-bit int. GeoTIFF is very popular for satellite data (along with HDF)
NetCDF (1) 16
Multidimensional array http://www.narccap.ucar.edu/users/user-meeting-08/handout/netcdf-diagram.png
NetCDF (2) 17
Multidimensional array https://trac.osgeo.org/gdal/raw-attachment/wiki/ADAGUC/ADAGUC_NetCDF_dimension_scales.jpg
netcdf file:/d:/RS_DATA/…/wnd10m.gdas.201210.grib2 {
dimensions:
time = 745;
lat = 880;
lon = 1760;
height_above_ground = 1;
variables:
float U-component_of_wind(time=745,height_above_ground=1,
lat=880, lon=1760);
:units = "m s-1";
:long_name = "U-component_of_wind @ height_above_ground";
:missing_value = NaNf; // float
:GRIB_param_discipline = "Meteorological_products";
:GRIB_param_category = "Momentum";
:GRIB_generating_process_type = "Forecast";
:GRIB_product_definition_template = 0; // int
:GRIB_product_definition_template_desc =
"Analysis………/layer at a point in time";
:GRIB_level_type_name = "height_above_ground";
NetCDF metadata for CSFR climate reanalysis 18
Several lines were cut out due to space constraints
NetCDF metadata 19
https://badc.nerc.ac.uk/help/formats/netcdf/netcdf_fig1.gif
ISO 19123 Model 20
See paper in main readings…
CF-CDM Model (netCDF) 21
See paper in main readings…
HDF: now NetCDF-Java reads HDF 22
Main feature: hierarchical variable namespace Includes groups and variables HDF-EOS is the primary format for storing EOS satellite data EOS stands for Earth Observing System
Subsetting data: point 23
1. Define coordinates (lat, lon), e.g. in EPSG:4326
Subsetting data: polygon 24
Quite tricky: usually rasterize polygon to grid resolution
Additional readings 26
Practical lesson 03 27
Subset GeoTIFF, HDF, NetCDF with Java libraries Tools: To feel data: HDF Viewer, Weather and Climate Toolkit (WCT) Java libraries: GDAL, GeoTools, NetCDF-Java Readings: • http://www.unidata.ucar.edu/software/thredds/current/netcdf-
java/tutorial/GridDatatype.html • http://www.smartjava.org/content/access-information-geotiff-using-java • http://gis.stackexchange.com/questions/73210/how-to-crop-an-image-based-
on-a-shapefile-using-geotools • http://docs.geotools.org/stable/tutorials/raster/image.html
Practical lesson 03 28
Feel: play with subsetting NetCDF multidimensional array
Practical lesson 03 29
Feel: browse HDF matrix
