Implementing WMS interface on Imagery Database (BDI)
Frédéric GUILLAUDDSI/DEV
EGOWS Meeting KNMI, June 2009
2
Overview
Context– Towards SOA for production and workstations (Marie-Françoise talk)– DWG Meteorology and WMS IE
Soprano imagery database (BDI) WMS implementation Issues Still to do Conclusion
3
Context
IT SOA pilot project Goals
– Improve our skill with SOA architecture and tools (ESB)– Improve our skill with OGC standards– Early identify issues– Make technical decisions
DWG meteorology IE on WMS– Calendars compatibles (Over 6 month by the end of 2009)– WMS BDI may be a service involved in IE experiments– To do this, WMS should be deployed in DMZ– This lead to security issues
4
Soprano Imagery database (BDI) : Before
RDBMS Oracle 3 days real-time data online (“blob” data type). Data is not archived. About 250 datasets Legacy catalogue content not always consistent … Native formats
– BUFR, TIFF-MeteoFrance (TIFF with - open - specific MF tags…) Native extent depends on dataset origin (MSG, METOP, other
geostationary platforms, radar data ..) Native CRS
– “Space View” (One-Point Perspective Projection ?) and “Polar Stereographic” Data access services
– Low level (application binary executable)– No API available– No remote access possible
5
Soprano Imagery database : product overview
Basic radar reflectivity measurements (for each radar) Mosaics
– Precipitation flux (through reflectivity conversion)– Precipitation accumulation (5’, 15’, …)– Forecasted intensities (2PIR algorithm)
6
Soprano Imagery database : product overview
Satellite data• From MSG, METOP, GOES platforms …• Composite CMS products, …
7
Soprano Imagery database : products overview
Composite products– Radar precipitation intensity calibrated with ground measurements
(ANTILOPE)– Satellite-Based fog diagnostics (CARIBOU)
8
WMS BDI features : Now
About 250 Layers GetMap returns only one LAYER at a time LAYER name is an internal identifier (tokens delimited by “_”, in
order to make easier mapping WMS – RDBMS data access Pre-defined STYLE : Each LAYER has a default colour map, which
is the official colour map on operational workstations at Meteo-France.
User defined styles are not yet supported (SLD=) If TIME not specified, GetMap returns the last image Outputs formats : image/gif; image/png Output CRS : only EPSG:4326 and Polar Stereographic at the
moment No support for metadata layers (e.g. pixel timestamp, pixel
quality, ...) but may be WCS scope ?
9
WMS Implementation : Reuse of legacy components
Data access component : “RDImage”– Application Binary Interface (executable)– Performs data extraction from BDI on criteria :
• Data type (radar, satellite …)• Production type (observation, forecast …)• Data origin (sensor platform, system, algorithm …)• Native domain covered by the dataset • Parameter (rain intensity, accumulation, radiance/wave length …)
Image processing component : “TransImage”– Application Binary Interface (executable)– Performs :
• Sampling a sub-domain• Format translation• Coordinate reference system operations (re-projection)
10
WMS Implementation
Meteo-France WMS framework– Implements WMS interface in PHP– Check on GetMap – GetCapabilities consistency– WMS 1.1.1 or 1.3.0
WMS BDI driver– Links GetMap parameters to RdImage & TransImage ones– Depending on GetMap parameters, invoke TransImage if necessary– Generate GetCapabilities response (XML)
11
WMS implementation overview
BDI (Oracle)
OCI
RDImage (.exe C/C++)
TransImage (.exe C/C++) Capabilities (C )
Meteo-France WMS Framework (PHP)
GetMap GetCapabilities
BDI Driver (PHP)
WMS
12
Issue : GetCapabilities layering
GetCapabilities response – Mainly product metadata– Describe product types (characteristics common to product instances)
GetCapabilities tree : Observations & Measurements approach– 3 Levels of layers :
• Process– Product type, Platform, Algorithm e.g Satellite MSG
• Feature Of Interest– Domain e.g Europe-Atlantic …
• Observed property– Parameter e.g IR 108
– Leaf layer is a named product type (queryable)
13
Issue : GetCapabilities layering
Layer granularity – Balance LAYER / Multi-dimensional LAYER (LAYER plus DIM)– E.g :
LAYER=SA__OBS__MSGC__V_EURATL__WV-62Or
LAYER= SA__OBS__MSGC__V_EURATL DIM_Wavelength=62 Or
LAYER=SA_OBS DIM_Platform=MSGC DIM_NativeDomain=V_EURATL DIM_Wavelength=62
Parameter names, units, … for DIMENSION usage– We need controlled vocabularies
E.g : DIM_Wavelength=62, 73, 108 , DIM_AccumulationTime=5, 15, 60 …DIM_BaseTime , DIM_Quality, DIM_PixelTimeStamp
The granularity of the layers will impact : – The efficiency of catalogue searching – The size of GetCapabilities response and performances
14
Issue : Keywords & Titles
Issue : Finding the layer name to perform GetMap request …– Layer names are quite complicated (internal identifiers), but unique :
• Ex : “RD_CPO_P__OBS__RD_ETRANGER__V_PIEMONT__REFLECTIVITE”– About 250 layers in GetCapabilities– At the moment “Title” == “Name” (that is bad !)
• difficult to be interpreted by a human… Title should be explicit – No abstracts, no keywords
To help catalogue searching, we are looking for keywords, “Controlled vocabularies”, “Ontology” …
– E.g “METEOSAT” “MSG”, “METEOSAT-8” “METEOSAT-9”, … Keywords should be hierarchic
– E.g “TEMPERATURE -- CLOUD -- TOP” Catalogue searching engine tested : GeoNetwork Possible keywords references :
– Global Change Master Directory (GCMD)– Common WMO code tables (Table C13, …)– AMS Glossary of Meteorology (GMET)– EUMETSAT , HMA (for satellite data) ?– Others ? To be defined ?
16
ISO 19128 – ISO 19115 relationship
ISO 19128 (WMS) Metadata elementproperty ISO 19115
Title CI_Citation.title
Abstract MD_DataIdentification.abstract
Keyword element in Keyword list MD_TopicCategoryCodeMD_Keywords
EX_GeographicBoundingBox EX_GeographicBoundingBox
BoundingBox EX_BoundingPolygon
DataURL MD_metadata.dataSetURI
Attribution CI_ResponsibleParty
Identifier and AuthorityURL CI_Citation.identifier
17
Issue : ISO 19128 – ISO 19115 relationship
How to keep consistency between WMS GetCapabilities (ISO 19128) and ISO19115 Metadata common fields ?
Where is the metadata reference or entry point ?– SOPRANO ISO 19115 metadata database
• To be created …• Partially harvested from VGISC metadata (probably not sufficient for BDI
because focussed on GTS products)– BDImage (or other databases) descriptive schema content
• Incomplete (keywords, abstracts are missing …)• Not always consistent (because of the database history…)• Difficult to change (because of backward compatibility)
18
Issue : Time
Time information– It would be useful to include product instance availability information
(ISO 19108) in GetCapabilities response :• Potential availability (time range plus step)• Real time availability (real time list : may be complex and CPU time –
consuming– There is a TIME parameter in GetMap, but no dedicated tags for TIME
information in GetCapabilities response WMS 1.1.1 and 1.3– GetMap : no support for multi-dimensional time (forecasted imagery)
e.g. : Forecasted precipitation intensity = f ( Base time, Validity Time )– TIME semantics may be ambiguous
e.g : Sampling Time, Validity Time, Result Time, Base Time … Possible approach :
– Using “Dimension” / “ExtendedCapabilities” ?
19
Issue : CRS identifiers
There are three methods to identify and describe CRS Namespaces & codes
– We need CRS identifiers not yet define in EPSG database and / or AUTO namespace, others …
• EPSG (not parametric) – Lot of codes to be defined ( all CRS in use at Meteo-France ? )
• AUTO2 (parametric) :– Universal Transverse Mercator AUTO2:42001– Transverse Mercator AUTO2:42002– Ortographic AUTO2:40003– Equirectangular AUTO2:40004– Mollweide AUTO2:40005
• Is it possible to define other codes ?– Ex Polar Stereographic for each prim meridian, “Space View”, .. ?
Reference (URL) to ISO 19111 documents Proj4 “input parameters – like” ?
20
Issue : Performances
It takes about 1 seconds to process full disk MSGC image “on the fly”– Reprojection appears to be very expensive– ”On the fly” processing probably not possible if many concurrent
requests. We probably need pre-defined CRS
– This would allow pre-calculation of the projection matrix for each CRS transformation (in TransImage)
– WMTS implementation ? But we’ll have to pre-compute tiles for each layer, CRS, Zoom Level, Format, Time …
– WMTS implementation is obviously a challenge for real-time Met data.
21
Left to do …
Lot of work on Metadata– Keywords, abstracts, ….– Link GetCapabilities response – ISO 19115 metadata
New output CRS New output formats (does GeoTiff make sense in a WMS ?) Support for multi dimensional times Support for metadata layers (pixel time stamp, pixel quality, … ?) Support for client style (SLD=) Improve performances
Limited set of CRS, projection matrix in TransImage component , WMTS Proj4 – GDAL migration of TransImage ( GEDAL is already in use on
SYNERGIE) TransImage as service Security topics to be investigated
22
Conclusion : BDI WMS works but …
To get really interoperable WMS, we need to agree on :
Metadata implementation rules (at least keywords) Colour Maps / Units of Measure
– E.g : Radar Data : Reflectivity (DBZ) – Intensity ( Kg m-2 / h) CRS policies (EPSG, AUTO2, …) How to manage multiple time dimensions Layer granularity for each product type ( balance LAYER – layer
DIMENSION ) Parameter names, units for DIMENSION usage