© Crown copyright Met Office GISC Exeter Status WMO RA VI Workshop on the Implementation of WIS...

© Crown copyright Met Office

GISC Exeter StatusWMO RA VI Workshop on the Implementation of WISBulgaria, Sofia, 2011-11-01/03

Chris Little, Met Office, Co-chair OGC Met. & Oceanography Domain WG


Contents

• A Little Background

• Progress

• Current Plans

• Demo

• Questions and Answers

Why me?

• Member of vGISC ITT Drafting Teams– (current and previous)

• WMO Regional Association VI Co-Rapporteur for Information Systems & Services – (with Leonid Bezruk, Russia)

• WMO Focal Point for RTH Exeter

• I have been around for a long time– (and partly to blame for GRIB, BUFR, & CREX)


Background

WIS Background - 1CBS-Ext(1998) Karlsruhe: origins of WIS

1999-2000 Inter-Programme Task Team on Future WIS, Melbourne & Monterey

CBS-XII (2000) Geneva: established scope, principles:– Metadata– ‘Push’ and ‘Pull’– Real time and Non-Real time– Ad hoc and Routine Subscription– GTS, IMTN (Clouds I & II) and Internet

• DWD, Météo-France & Met Office propose vGISC• EC-LIII (2001) endorses plans

WMO Expected Result 5

DEVELOPMENT AND IMPLEMENTATION OF THE NEW WMO

INFORMATION SYSTEM

WMO Strategic Thrust 1: Science and Technology Development and Implementation


RA-VI vGISC History2000 RA-VI (Europe+) co-operation proposed

2001 Workshop at ECMWF– Considered politics, funding, technical options

2002-06-26/27 RA-VI Steering/Policy Group established– Formally started programme:

– Western European Virtual GISC

– Technology Pilots, build on GTS and Internet E.g.Unidart

2002-4 Technical subgroups– WMO WIS global groups also meeting

2006 First WMO Metadata Core profile V1.0 (ISO 19115)– SIMDAT live demo at WMO CBS

– Other pilot projects: NCAR, JCOMM

2006 First WIS Project post at Geneva


RA-VI vGISC Procurement Project

2007 Exeter, Offenbach, Toulouse fund Project Office • WMO WIS plans and specifications agreed

2007-12-12 Project started with formal collaboration

• ECMWF, EUMETSAT & Oslo join as DCPCs

• RA-VI vGISC Procurement Steering Group established

• ITT Drafting Team established

2008 ITT developed, intended s/w delivery

2009/2010

2008-05-14 EU Competitive Dialogue, DWD lead


vGISC 2008 Procurement2008-06-10 Draft ITT presented to WMO

2008-06-23 ITT Version 1.0 issued

2008-07-11 Short-listed 4 companies

2008-09-15/17 First Dialogue » Vendors initial proposals of €3-4 million » Gross mismatch between likely price & budgets» 3 major partners unable to subsidise each other

2008-10-24 Friday: Procurement stopped

2008-10-27 Monday: Vendors informed

2008-11-06 RA-VI Steering Group discuss options:» Issue re-scoped/phased ITT» DIY re-factor & enhance SIMDAT software, based at ECMWF» Centres build GISCs separately

2008-12 ITT Version 1.0 available to WMO community

2008-12-19 Choose option at RA-VI SG


vGISC 2008 RIP

• Jacques Anquetil

• Jacques Roumilhac

• Jean-Pierre Aubagnac

• Heinrich Knottenberg

• Siegfried Fechner

• Markus Heene

• Duncan Jeffery

• Chris Little

• Marta Gutierrez

• Øystein Godøy

• Lothar Wolf

• Beatriz Garcia

• Eliot Christian

• Project Support:

• August-Clemens Thole

• Peter Piotrowski

• Karla Schöttler-Jansen

• Karl Solchenbach


vGISC Re-born 2009DWD go alone, MF & UKMO decide to collaborate

Collaboration agreement for new ITT prepared by UKMO

2009-01: Started working

– Backup workshop

– Open source software (Geonetwork, Open SSO, …) gap analysis

– Prototyping for risk reduction

– ITT requirements document (from 280 -> 90 pages)

– ITT definition: firm components, optional components

2009-06: Advert issued, Météo-France procurement

2009-09: Bidder selection

2009-11 to 2010-02: Three rounds of Negotiations

2010-03-19: Winner chosen


vGISC 2009 ITT Approach

Use off the shelf components where possible– Catalogue– Metadata editor– Entity management

Possibly use SIMDAT as ‘glue’

Tender for re-factoring & missing functionality– Monitoring and Administration– Subscription and pull request– Emphasis on interfaces

Tender for baseline and options


vGISC 2009 Project Approach

Open– Other partners (GISC and DCPC) are welcome!– Share the work– Candidate Open Source components proposed

Requirements– Light document: We know what we want, no options– Need to have the main requirements– Reasonable performances but a scalable system– ITT based on V1.5 of vGISC requirements document

Target– Solution that can be used by a large WMO community


OpenWIS Consortium• 2009 Météo-France, MFI, UK Met Office form ITT consortium

• Software named ‘OpenWIS’

• 2010-03 Contract awarded to Akka & sub-contractors:

• CampToCamp – web experts

• VCS – Project Management, technical expertise

• Recruited GeoNetwork expert

• 2010-06-08 Detailed plan agreed with 3 s/w deliveries:

• First delivery for CBS, 2010-11

• Second delivery, ’90%’, for Congress, 2011-04

• Third delivery: monitoring functions, bug fixes, etc 2011-07

• 1 year warranty, optional extra 3 years warranty & development


OpenWIS Software FunctionalityOpenWIS software allows functions:

– GISC– DCPC – NC

Consortium members currently:– Météo-France– UK Met Office– MF International – KMA, Korea Meteorological Administration– BoM, Australia Bureau of Meteorology

At WMO Congress XVI, May 2011:– Moscow announced intention of joining

Others countries interested. Workshop in Toulouse, tbd 2011?


Progress

2010 Proposed GISCs

Australia

Brazil China

France

Germany

India

Islamic Republic Of Iran

Japan

Korea

Russian Federation

Saudi Arabia

UK

USA

2010 Proposed DCPCs

Fiji Finland France 8Germany 9Hong Kong India Italy 2Japan 7Kenya 3Korea 3Netherlands 2

New Zealand 2Niger 2Norway Russian Federation 8Saudi Arabia 2Senegal 3Sweden 3Thailand UK 6USA 10Uzbekistan

Algeria Argentina 5Australia 4Bulgaria Canada Croatia Czech Republic ECMWF Egypt 6EUMETSAT


WIS live catalogue demo! WMO Congress XVI, Geneva 2011

Scenario: How WIS can help with a (imaginary) food security analysis


5 GISCs agreed by WMO4 Technology solutions:• CMA China Meteorological Administration (own software)• DWD Deutscher Wetter Dienst (IBL Moving Weather)• JMA Japan Meteorological Agency (own software, RSS)• Météo-France (OpenWIS)• UK Met Office (OpenWIS)

Interoperability of catalogues, and harvesting of information demonstrated between all nodes, plus Brasil, Iran, NCAR

Harvesting protocol agreed by WMO: OAI-PMH: • Geonetwork (Lucene & Postgres)• Java OAI API (Oracle)

37 DCPCs endorsed by WMO

CMA China Meteorological Administration: 1

HKO Hong Kong, China: 1

DWD Deutscher Wetter Dienst: 10

JMA Japan Meteorological Agency: 8

Météo-France: 7

Norway: 1

UK Met Office: 8

ECWMF: 1

EUMETSAT: 1

http://www.wmo.int/pages/prog/www/WIS/centres/index_en.php

UK Met Office DCPCs• RTH• RSMC-Geographical (NWP)• RSMC-EERA/Dispersion (Nuclear, etc)• Marine Observations Centre• Long Range Forecasting• Ocean/Wave Forecasting• WAFC (ICAO rather than WMO)• VAAC (ICAO rather than WMO)• Opera: European Radar, jointly with M-F


Current Plans


OpenWIS FutureFinal delivery September 2011, Acceptance Start 1 + 3 year warrantyOperational ServiceRelease as Open Source software, with governanceMFI offer commercial installation & supportRollout:

– External DCPCs (ECMWF, EUMETSAT, IPY, …), NCs– Internal DCPCs (9 in Met Office )

• “build your catalogue metadata entries for your data” • Opera Data Centre to be a vDCPC, joint with Météo-France• Other NCs or DCPCs in UK?

Back-up with Météo-France– Out of Country back-up for Public Weather Service

Lots of WMO Technical work:– Tighten WMO Core Metadata interoperability– “Business as Usual” operating procedures


OpenWIS demo (possibly)






Summary: OpenWIS, Open SourceGISC/DCPC/NC functions, configurable, resilient

Geonetwork (Catalogue)• Licence compatibility with Open SSO licence• Use with or without modifications• More secure, more scalable

Open SSO / Open AM (Security)• Licence

Postgres Database • others possible, e.g. Oracle

Interfaces definition• Middleware with standards based interfaces• Possibility to add different software

Consortium established, expanding– Support it yourself, in a community– Commercial support available from MFI


Questions & Answers


Spare slides, to be used only if required

WIS Summary

• Build up for last 10 years

• Change is really happening now

• Less ‘go it alone’ from WMO

• More use of external standards – ISO 19xxx, Inspire, … – OGC Web services: WMS, O&M, SWE, …– NetCDF, XML, … – Etc

• WMO strong point: persistence, bureaucracy!


UserRequests

I3

User Role AssignmentProcedure

Applicable Data Policy

I2

I1

Observations, Productsand Associated Metadata

Reference Information(WIGOS, IOC, etc)

AuthorizedRequests forInformation

Information SearchRequests

Catalogue BrowseRequests

Subscription

Information

« Ad Hoc »

metadata

Monitoring DataFrom all functions

Control Actions toAll functions

O2

Reports of Performanceagainst Requirements

Delivered Informationvia « Push » Services(time-critical &non-time-critical)

Delivered Informationvia « Pull » Services(time-critical &non-time-critical)

O1

Information Search Results

Information Services

DisseminationMetadata

DAR MetadataCatalogue

POSSIBLE CHOICES

SIMDAT base

Sun OpenSSO baseGeonetwork base

MSSSMART / GEMS

A3

Maintain and ExposeCatalogue of Servicesand Information

ManageRequests

CatalogueAccess

« Ad Hoc »

Routine« Push »

Cache

A6Manage SystemPerformance

Monitoring

Administration

Assign User Role &Authorise Access toInformation A2 & A4

Authentication

Authorisation

User &Data Policies

Portal

Make / FollowRequest

Browse /Search DAR

Admin /Monitoring

Portal

A1

Collect Observations,Create Products and Archive Information

Synchro &Harvesting

Replication

A5Deliver Informationto Users

Legacy Databases

Collection

Retrieval

2009 Draft Architecture


VGISC System

DISSMSS/FSS

WIS Harness

Local System

OpenWIS 2009 Draft Architecture Interfaces

Where do the wires actually go?


Other Stuff• WOW: Met Office ‘Crowd sourcing’ of weather obs

http://www.metoffice.gov.uk/climate/uk/wow.html – Met Office stores and shares, and Quality Controls– No OGC standards, only WMO data formats

• EGOWS European Meteorological Graphical Operational Workstations meeting June 2011 in Toulouse http://www.knmi.nl/samenw/egows/ and http://www.meteo.fr/cic/meetings/2011/EGOWS/

– Informal (non-OGC) Interoperability Experiment with WMS– A variety of Servers and Clients– No feedback from Marie-Françoise Voidrot and others yet– Main issue is “non-responses” from servers rather than errors


Regional Meteorological Telecommunication Network for Region VI (Europe)Figure 1 - point-to-point circuits implementation (transmission speed in kilobit/s)

Dublin

Tirana

Beirut

Washington

Nairobi

Cairo

New Delhi

Jeddah

Casablanca

TripoliTunis

Beijing

RTH

NMC

Centre in other region

MTN circuit

Regional circuit

Interregional circuit

Ljubljana

Athens

Brussels

Kishenev

Kiev

Tbilisi

Bucharest

HelsinkiOslo

Zagreb

Warsaw

Minsk

Malta

Amman

Bet Dagan

Reykjavik

SondreStormfjord

DeBilt

Lisbon

Budapest

Bratislava

Vilnius

Riga

Prague

Zurich

Damascus

Ankara

Larnaca

Khabarovsk

Novosibirsk

Tashkent

Tehran

64

64

64

64

9.6

2.4

9.6

Nairobi

0.05

64

19.2

0.05

7.2-28.8

0.050.05

0.164

9.6

9.6

0.2

NI

NI

NI

9.6

9.6

9.6

N/O

9.6NI

NI

Dakar

0.1

Beijing

FR 256 (CIR:64)

64

128

Hanoi

Almaty

Yerevan

XII..2004

Algiers

Madrid

2.4

32

8

32

16

16

256/128

16

24

48

64

64

32

64

8/16

8

8

16

16

16

8/3232/64

64

32

32/96

48/16

32/8

64/8

RMDCN Committed Information Rate

64

8

16/8

24/8

* The RMDCN circuit Helsinki - Tallinn is not yet in the RTMN plan, but replaces the former GTS connection of Tallinn

Belgrade

Exeter

Toulouse

Rome

Copenhagen

Norrköping

Vienna

Tallinn

19.2

ECMWF

16/8

16/64

Offenbach

16/8

64/8

16/8

128/64

19.2

NI

NI

N/O

64/8

Baku

8/16

64/8

Skopje

Moscow

Sofia

NI

N/ON/O

16/8

48

Melbourne

16

16/8

16/8

8

64

http://www.wmo-dra.info/gmap/WMO_NMHS_regions/metservices.html

Antarctic Telecoms



Exeter

Sofia

Melbourne

Buenos Aires

TokyoBeijing

Nairobi

Washington

Jeddah

Prague

Toulouse

Dakar Algiers

Offenbach

Brasilia

Cairo

New Delhi

Improved Main Telecommunication Network (IMTN)

IMTN cloud

Moscow

RA I

RA II

RA III

RA IV

RA V

RA VI


Global Telecommunication SystemGlobal Telecommunications System (GTS)

– Managed by WMO - World Meteorological Organization

Use to exchange– Observations

– Forecasts

– Warnings

Private Network– Node to Node network

– NOT an Internet

– ‘Store and Forward’

– Resilient and inflexible

Global Telecoms System

• It works!

• Developed in early 1960s

• National, regional & international exchange

• Terrestrial & satellite links• Mainly point-to-point circuits, also Multi-point circuits

• X.25, IP & asynchronous links, Mixture of speeds

• Relatively expensive, legacy technologies

• Secure Extranet: limited use of Internet

• No overall system manager

• Store & Forward• No explicit addressing, based on distributed routeing directories

• Very robust, loosely coupled

• Very slow to change, ad hoc development

• Not used much for non-operational WMO data

WIS Background Summary

WMO Telecoms, GTS, is reliable, inflexible, ‘push’ only

Still many point-to-point links

Managed Network in Europe with two roles:– Areal Meteorological Data Comms Network AMDCN– Global Core/Main Trunk Network MTN

Some of Eastern Europe/Caucasus not yet joined

Some global centres still joining MTN– Brasil – Canada– South Africa

Other regions’ AMDCNs not yet happened.

WMO Background Problem

5 GAW World Data Centres

GCOS Data Centres

Global Run-off Data Centre

stopstop

Climate research institutes

Regional Climate Centres

stopstop

stopstopstopstop

stopstop

stopstop

World Meteorologic

al Centres

GTS

Regional/Specialized Meteorological Centres

National Meteorological

Centres

Meteorological and R&D Satellite Operator Centres

International Organizations (IAEA, CTBTO, UNEP, FAO.. )

Commercial Service

Providers

Universities

WMO World Data Centres

WIS Background - 2CBS-Ext (2002) Cairns: Recommended FWISCongress Cg-XIV (2003)

– Multiplicity of systems of different Programmes causes incompatibilities, inefficiencies, duplication of efforts and higher overall costs.

– Congress approves the concept of the Future WMO Information System (FWIS), to provide a single coordinated global infrastructure for the collection and sharing of information in support of all WMO and related international programmes.

Executive Council EC-LVI (2004)– Identified FWIS to be one of the key contributions of

WMO to GEOSS– Established Inter-commission Coordinating Group on WIS

Executive Council EC-LVII (2005)– Renamed FWIS to WIS– Accelerated the development and implementation of WIS– Evolution not revolution, but faster than usual WMO

WIS Interoperability StandardsWMO Data FormatsWMO Telecoms protocols + WMO Extranet + ‘Push’WMO processes

WMO Data Formats Internet + WMO Extranet Global Catalogues

– ISO19115+ISO19139+WMO profile V1.2 +ISO19119– ISO23950+SRU1.3 (+CSW?)– OAI-PMH

Global Cache (Bit Torrent proposed)‘Push & Pull’ Dissemination Ad Hoc & Subscription

(PubSub?)


New WMO Information System


1. Global Information System Centre 1. Global Information System Centre (GISC)(GISC)Global Catalogue + 24 hour operational Global Data Cache

Receives information from NCs and DCPCs

Exchanges information & synchronises with other GISCs

Disseminates, within its area of responsibility, the entire WWW set of data and products for routine global exchange (GTS function) & maintains at least 24 hours worth of data (replicates the ‘Cache’)

Generates, maintains and makes accessible metadata catalogues of its data holdings (synchronizes the catalogue)

Ensures around-the-clock, reliable and secure operations

Enhances/replaces World Met Centres WMCs

WIS Components


2. National Centre (NC)2. National Centre (NC)

Exchanges information collected/generated in a country with GISC/DCPC

Serves as portal for national users & administrates their access to WIS

Generates standardised metadata as well as data

Several NCs in a country are possible (not just the NMC)

Replaces National Meteorological Centres NMCs

WIS Components


3. Data Collection & Production Centre (DCPC)3. Data Collection & Production Centre (DCPC)

Neither GISC or NC

Serves as information production facility, regional or global

Generates, maintains and makes accessible standardized metadata catalogues of its data holdings through GISCs

Supports information “Push” and ”Pull” mechanisms

Facilitates access/exchange from/to NCs and GISCs

Replaces/enhances Regional Telecom Hubs RTHs

Replaces Regional Specialised Meteorological Centres RSMCs

WIS Components


WIS Technologies

Global Catalogue• Metadata: ISO 19115 + 19139

– WMO Profile 1.2– Services could be ISO19119

• Search with ISO23950 + SRU (+ CSW?)• Harvesting: OAI-PMH• Prototype: Geonetwork• Catalogue access for all• Default access policy, others supported

• No more than 24 hours out of date


WIS TechnologiesSubscriptions• Subscriptions & ad-hoc retrievals for authenticated

and authorised users including self registered• Using OpenAM/SAML2 (was OpenSSO)• Need to agree subscription & backup standard

(OGC PubSub?)

Cache• 24 hours worth of all ‘Essential’, critical, data• Need to agree Content Distribution standard

(BitTorrent?)• Stringent global performance standards (2 min

latency end-to-end maximum for Warnings)

Date post:	16-Jan-2016
Category:	Documents
Upload:	stephany-carson
View:	222 times
Download:	4 times

© Crown copyright Met Office GISC Exeter Status WMO RA VI Workshop on the Implementation of WIS...

Documents