Meteorological and Aviation Weather Open Data implementation utilising OGC standards

Meteorological and Aviation Weather Open Data implementation utilising OGC standards

Finnish Meteorological Institute

Finnish Meteorological InstituteRoope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari

15.04.2023 2

Finnish Meteorological Institute opened its data in 2013.

Basically everything that FMI has property rights was opened.

Data is provided in freely in machine readable format.

Meteorological and Aviation Weather Open Data implementation OGC standardsRoope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari

FMI Open Data

https://en.ilmatieteenlaitos.fi/open-data



Data set Description Time Interval

Estimated publish date

Weather Observations

Temperature, Wind, Humidity, Ground Temperature…

10 min Open,older data to be added

Sun Radiation UV, Short and Long Term Radiation…

1 min Open

Marine Observations

Waves, Sea Temperature, Sea Level…

1 h Open

Weather Radars Precipitation Rate, Precipitation Amount…

5 min Open, older data to be added

Lightning Thunder Strikes in Finland

5 min Open

Example of Data Sets

15.04.2023 3Meteorological and Aviation Weather Open Data implementation OGC standardsRoope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari


Data set Description Time Interval Estimated publish date

Real Time Observations

Real Time Observations from specific location(s)

AWS 2010 – Soundings 1959 –Flashes 1998 – Sea Level 1971 – Waves 2005 –

Openolder data will be added

Climatological Observations

Dayly and monthly temperature mean and extreme values from weather stations

1959 - Open

Climatological Observations

Monthly temperature and precipitation rate mean values interpolated to grid

1961 - Open

Climatological Reference

Climatological Reference. Temperature, humidity, pressure, precipitation amount and snow depth.

Reference seasons:1971-2000 1981-2010

Open




Weather forecast model HIRLAM RCR

Point forecasts and grid data

Latest model run(4 times a day) 0…54 h

Open

Sea forecast models Sea level point forecasts, Wave (WAM) and current (HBM) as grid data

Latest model run(4 times a day) 0...54 h

Open

Environmental Monitoring Facilities

Weather observation stations, radars…

2015

Aviation Observations

METAR 30 min open

Ground & mast observations

Special observations from ground and masts

2016 /Open




Air Quality Observations

Air Quality Observations 1h 2015-2016

Silam Model Dispersion Model for Air Quality, Forest Fire and Pollen

Latest model run (once a day)0…96h

2015

HELMI Ice Model Ice forecast model Latest model run(4 times a day) 0...54 h

open

Soundings Temperature, Humidity, Pressure, Wind from ground to 25 km height

2 times a day 2015

Road Weather Observations (LIVI)

Road Weather Observations

10 min open


FMI Open Data Portal follows INSPIRE requirements.

FMI Open Data Portal

Meta data

Data Models

Services

The very same data portal works as Open Data and INSPIRE portal.


Catalog Service (CSW)

o Based on GeoNetwork

View Service (WMS)o Based on GeoServero Only the most common layers

published


Download Service (WFS 2.0)

o Web Feature Service (WFS) 2.0 Simple Profile

o Based on stored querieso Predefined data sets with

possibility for additional parameters (i.e. time and area)

o In-house production


Registrationo Registration is required to use View and Download

Services o Working email address is the only mandatory

informationo After registration the user gets an API key which have to

be added into all requestso GET parameter fmi-apikey=…&

o Header fmi-apikey; …

o Part of url http://wms.fmi.fi/fmi-apikey/…/wms?

o One can create several API keys with one email


Usage Limits

With one API key it’s allowed too do at most 20 000 requests per day to Download Serviceo do at most 10 000 requests per day to View Serviceo do at most 600 requests per 5 minutes to both services

o If all observations from one time step is calculated to as one, little over 17 000 new data sets are published daily

o So, with one API key it’s allowed load everything onceo View service can be used for testing but can not be used as a

back end for popular clients


Open Data Service Cluster

S1 S2 S3

Client Data Service Cluster

S1 S2 S3

Load Balancer

Configuration

Data (NFS)

Configuration (NFS)Database

Please contribute!

https://github.com/fm

idev/

MetoLib

o Open source JavaScript library produced by Finnish Meteorological Institute

o Helps users to load and use the data

o Supports multi point coverage data formato Python version is on the list

Easy requests Cache

Parse the data to as JSON


Data Modelso Observations and point

forecasts as GMLo The same data is published in:

o MultiPointCoverageo MeasurementTimeSerieso SimpleFeature

o Gridded data is provided in appropriate binary format (Grib, NetCDF, GeoTiff…)o WFS members contains the

metadata ‘envelope’ with a link to a actual data


15.04.2023 15

Data Modelsgmlcov:MultiPointCoverage

gml:rangeSetgml:doubleOrNilReasonTupleList

The data is listed for every point defined in domain set.

gml:domainSetgmlcov:simpleMultiPoint

The coverage is defined as a list of

points in 4 dimensional grid (lat,

lon, height, time).

gmlcov:rangeType

The parameters listed in range set

are defined in separate element.


15.04.2023 16

Cons- Not intuitive- No natural

structure of XML XSLT and Xpath don’t work

Pros+ Compact+ Efficient+ Small file size+ Works for many

data types

Data Modelsgmlcov:MultiPointCoverage


15.04.2023 17

Data Modelswml2:MeasurementTimeseries

wml2:MeasurementTimeseries

One member contains time series for one parameter

and one location


15.04.2023 18

Cons- Lots of repetition- Large file size- Heavy for DOM-

based parsers- Don’t work i.e. for

thunder strikes

Pros+ Intuitive+ Easy to use+ XSLT & XPath

works

Data Modelswml2:MeasurementTimeseries


15.04.2023 19

Data ModelsSimpleFeature

SimpleFeature

One member contains one time, one parameter and

one location


15.04.2023 20

Cons- Lots of repetition- Very large file size- Heavy for DOM-

based parsers

Pros+ Intuitive+ Easy to use+ XSLT & XPath

works+ Ready client

support

Data ModelsSimpleFeature


• Aviation weather reposts are delivered as IWXXM• New data model coming into use in aviation

• Consists of the same elements than other messages• om:phenomenonTime, om:procedure, om:featureOfInterest,

om:result

• Content of the METAR is in om:result part as

• extracted into XML elements

• original, “old fashion”, METAR string

Data Modelsaviation observations IWXXM / METARS


http://en.wikipedia.org/wiki/IWXXM

Data Type Data Format

Observations wml2:MeasurementTimeseries

gmlcov:MultiPointCoverage

SimpleFeature

Point Forecasts wml2:MeasurementTimeseries

gmlcov:MultiPointCoverage

SimpleFeature

Lighting Observations gmlcov:MultiPointCoverage

SimpleFeature

Grid Forecasts XML Envelope + Grib2/NetCDF

Radar Images GeoTiff / PNG images

METAR IWXXM


15.04.2023 23

Data Models File size Comparison

Simple

Featu

re

Mea

sure

men

tTim

eser

ies

Mult

iPoin

tCov

erag

e0

20

40

60

80

10081.7

52.9

1.81.3 1.2 0.2

Document Size [MB]

Compressed DocumentSize [MB]

o13

8 ob

serv

atio

n st

atio

ns

o11

par

amet

ers

o72

tim

e st

eps

(12

hour

s)


15.04.2023 24

Data Models Popularity Comparison

0

20

40

60

80

10080

19.8

0.2

Downloads[%]

oBas

ed o

n on

e m

onth

dat

a (0

4/20

14)

oAll d

ata

sets

com

bine

d


And a little over430 000 data downloads

per day (5 req/s)

At the moment about 7200

registered users

Some Experiences


Practically no client supports complex features

Although standards are followed, there’s

a gap between provided data model

and clients’ capabilities

Some Experiences


GeoServer is modified to support stored queries in

WFS 2.0 (released in version 2.7)

Also simple features had to be supported

Some Experiences


Industry is happy to use standardized

services

Amateur and freelancer coders

would prefer simple JSON API

Some Experiences


www.fmi.fi

http://www.slideshare.net/tervo/https://en.ilmatieteenlaitos.fi/open-data

http://www.slideshare.net/tervo/

http://www.slideshare.net/tervo/


Date post:	13-Aug-2015
Category:	Data & Analytics
Upload:	roope-tervo
View:	34 times
Download:	1 times

Meteorological and Aviation Weather Open Data implementation utilising OGC standards

Data & Analytics