THE NEW OGC PUBLISH/SUBSCRIBE 1.0 STANDARD · 2016-09-05 · THE NEW OGC PUBLISH/SUBSCRIBE 1.0...

transcript

http://essi-lab.eu

THE NEW OGC PUBLISH/SUBSCRIBE 1.0 STANDARD

Lorenzo BigagliNational Research Council of Italy – Institute of AtmosphericPollution Research (CNR-IIA), Italylorenzo.bigagli@cnr.it

Aaron BraeckelNational Center for Atmospheric Research (NCAR), USA

Geospatial Sensor WebsConference 2016

Münster (DE), 29-31 August 2016

http://essi-lab.eu

http://www.opengeospatial.org/standards/pubsub

lorenzo.bigagli@cnr.it

http://essi-lab.eu

Outline

¨ Background and introduction¨ PubSub 1.0

¤ Rationale¤ Core profile¤ SOAP Binding

¨ Related work

http://essi-lab.eu

Background

http://essi-lab.eu

OGC PubSub SWG

¨ The Open Geospatial Consortium Publish/Subscribe Standards Working Group investigates a generic mechanism to support publish/subscribe requirements across OGC service interfaces and data types (coverage, feature, etc.)¤ Chartered in November 2010¤ ~80 members/observers

¨ Requirement analysis ended in March 2011¨ 3 documents in scope

¤ PubSub Core (PSC) – approved and published¤ PubSub SOAP Binding (PSSB) – approved and published¤ PubSub RESTful Binding (PSRB) – drafting underway

n To accelerate the process, it was decided to release PSC and PSSB first, and postpone the course of PSRB

http://essi-lab.eu 5

Request/Reply model

¨ May be inefficient when waiting for specific events, e.g. data arrival, server changes, data updates¤ cf. polling

OGC Service (WFS, WCS,

SOS, etc.)

OGC ServiceClient

GetCapabilities( )

Service Capabilities, Dataset Identifiers, etc.

DescribeDataset( identifier )

Dataset Descriptions

GetDataset( identifier, filter )

Filtered Dataset

http://essi-lab.eu

Publish/Subscribe model

¨ Asynchronous delivery of messages¨ Ability to specify an ongoing (persistent)

expression of interest¤ Stateful behaviour

Notification ServiceOGC ServiceClient

Subscribe( identifier, filter )

notify( datasetDescriptor or data )

http://essi-lab.eu

Publish/Subscribe & OGC Web Services

¨ OWS (e.g. WMS, WFS) have primarily addressed the Request/Reply model, considered sufficient for many use cases¤ Clients request data of interest when needed, and may

request periodic updates

¨ Publish/Subscribe is complementary¤ Initial state obtained via Request/Reply, then a subscription

may be created for updates¤ Can result in less (or more predictable) network traffic¤ Can be very well suited to urgent but unpredictable data,

e.g. events, hazard warnings, sensor failures

http://essi-lab.eu

PubSub (SWG) use-cases

¨ Service Filtered Data Push¨ Service Filtered

Notification¨ Notification of Threshold

Crossings¨ Handling Multiple Event

Types and Data Models¨ GeoSynchronization (GSS)¨ Brokered

publish/subscribe¨ Data Distribution

¨ FAA SAA Dissemination Pilot

¨ Emergency/Safety Critical Application

¨ Communication Protocol -Open GeoSMS

¨ Distributed Data Synchronization

¨ Message Aggregation¨ Message Filtering /

Batching Messages

http://essi-lab.eu

PubSub (SWG) rationale

¨ Provide notification capabilities as a module to existing services¤ No impact on existing service semantics¤ Reuse service-specific filtering semantics

¨ Support pushing actual data from OGC Web Services, i.e. WCS, WFS, SOS

¨ Support pushing notifications about new data¤ E.g. service and dataset updates, in order to optimize

harvesting by catalogs

http://essi-lab.eu

PubSub Core (PSC)

¨ Abstract description of the mandatory PubSubfunctionality

¨ Independent of the underlying binding technology¤ Extensions defines

several optional capabilitiesn E.g. batching, heartbeat

http://essi-lab.eu

PSC Conformance Classes

http://essi-lab.eu

PSC Interfaces

http://essi-lab.eu

Basic PubSub interaction

http://essi-lab.eu

Main design elements

¨ Pluggable filter language¤ E.g. OGC Filter Encoding

¨ Publication metadata¨ Pluggable delivery method

¤ E.g. ATOM, WS-Notification, AMQP

http://essi-lab.eu

PubSub SOAP Binding (PSSB)

¨ How the PubSubfunctionality is realized in SOAP¤ Based on OASIS WS-

Notification¤ Additional

conformance class: Reliable Publishern Based on WS-

ReliableMessaging

http://essi-lab.eu

Related work

¨ OGC Testbed 12 Initiative¤ PubSub RESTful Binding (PSRB)

n Drafting underwayn SSE delivery methodn ATOM delivery methodn XML and JSON formats

n Partial prototypes availablen PubSub-CSW (CNR), PubSub-WCS (NCAR), etc.

¤ AMQP delivery method, JMS binding, etc.¤ “OWS Request/Reply Publisher” Conformance Class

n Accepts OWS requests as filtersn Delivers OWS replies as updates/data pushes

http://essi-lab.eu

Example: PubSub-CSW MEP

1. CSW client subscribes with a specific getRecordsRequest2. Time-0 recordset obtained via standard Request/Reply3. Subsequent updates based on CSW Transaction semantics

CSWCSWClient

1. Subscribe( filter = GetRecordsRequest#1 )

3. Notify( TransactionResponse )

4. Notify( TransactionResponse )

2.1. GetRecordsResponse

2. GetRecordsRequest#1

http://essi-lab.eu

Conclusions

¨ The long-awaited OGC PubSub 1.0 standard bridges a gap between the existing SDI standard baseline and the many technologies supporting server-initiated, asynchronous communications

¨ We hope that a growing number of applications will benefit from an interoperable and uniform way of implementing event-based architectures

http://essi-lab.eu

THE NEW OGC PUBLISH/SUBSCRIBE 1.0 STANDARD

Lorenzo BigagliNational Research Council of Italy – Institute of AtmosphericPollution Research (CNR-IIA), Italylorenzo.bigagli@cnr.it

Aaron BraeckelNational Center for Atmospheric Research (NCAR), USA

Geospatial Sensor WebsConference 2016

Münster (DE), 29-31 August 2016

http://essi-lab.eu

EXTRA SLIDES

http://essi-lab.eu

Testbed-12 Asynchronous Messaging for Aviation

¨ The architecture documented in this ER is one of the first implementations of the OGC PubSub 1.0 Specification. Hence, it is of great importance to assess and validate the specification itself, in particular as regards the following aspects:

¨ Extensibility of the PubSub 1.0 specification

¨ Harmonization of PubSub 1.0 specification, namely the SOAP binding, with AMQP 1.0 as a delivery method

¨ Integration and backwards compatibility with Harris Data Exchange (DEX) clients supporting the retrieval of data (e.g. AIXM, FIXM) via JMS

¨ As noted, the ER focuses on the Aviation domain, however a definition of a profile for aviation could serve as an example on how to integrate PubSub into other fieldsthat require near-real time, push-based data dissemination

¨ Use the OGC Filter Encoding 2.0 language to express subscription filters

¨ Pluggability of filter languages, e.g. to reference the correct geometry in

¨ aviation-specific data formats, via XPath/XQuery, etc.

¨ This ER also contributes to some of the activities in the current scope for the PubSub SWG, such as:

¨ Feasibility of adding PubSub capabilities to WFS 2.0 and WMS 1.3.0

¨ Definition of a generic mechanism to reuse service-specific data requests to PubSub-enable the existing OWSs

¨ Definition of lexical constants (e.g. "all" to identify all the publications offered by a Publisher).

¨ AMQP is a broadly useful and popular delivery method, and should be considered for standardization in the PubSub SWG. The ER authors are encouraged to bringan AMQP delivery method document to the SWG for consideration and standardization.

¨ Finally, the ER provides useful indications on future lines of work for the PubSub specification, such as:

¨ OGC PubSub 1.0 profile for moving objects

¨ Specific mechanisms for standardizing delivery methods

¨ Analysis on the role of Message Brokers in enterprise production environments (cf. the PubSub Brokering Publisher)

http://essi-lab.eu

FilterCapabilities

http://essi-lab.eu

FilterCapabilities

http://essi-lab.eu

PublicationsName Definition Data type and

values Multiplicity and

boundingBox The area of interest of the published data contents

BoundingBox [see

OGC 06-121r3]

Zero or more (Optional)

contentType The content type (i.e., mime type) of the published data contents.

Examples: “application/weather+xml”, “text/plain”

MimeType

OGC 06-121r3]

One (Mandatory)

description A human-readable description

DescriptionType [see

OGC 06-121r3]

Zero or one (Optional)

identifier A unique identifier URI One (Mandatory)

metadata Additional metadata on this publication

Metadata [see

OGC 06-121r3]

Zero or one (Optional)

supportedDeliveryMethod

The supported delivery methods for this publication

URI One to many (Mandatory)

supportedFilterLanguage

The filter language identifiers that are offered for filtering

URI Zero to many (Optional)

formalContentDefinition Language

The identifier of the language (e.g., "http://www.w3.org/XML/Schema/1.0”) used to describe the formal publication content definition

URI Zero to many (Optional)

http://essi-lab.eu

Publications

http://essi-lab.eu

DeliveryCapabilities

http://essi-lab.eu

DeliveryCapabilities

http://essi-lab.eu

SSE Delivery

¨ Pull¤ Not needed

¨ Push¤ Over HTTP

n ContentType: "text/event-stream;charset=UTF-8”n Cache-Control: "no-cache”n Connection: "keep-alive”

¨ Format¤ CSW TransactionResponse

http://essi-lab.eu

OAI-PMH Delivery

¨ Pull¤ Polling using “from” parameter

¨ Push¤ Not available

¨ Format¤ CSW TransactionResponse

http://essi-lab.eu

ATOM delivery

¨ Pull¤ Polling using “If-Modified-Since”, “start-index” parameters

¨ Push¤ PubSubHubbub?

¨ Format¤ Feed

n Entry (timestamp)n TransactionResponse

n PUT Record1000n DELETE Record2n UPDATE Record1

http://essi-lab.eu

Proof of Concept – PubSub-CSW

¨ PubSub Core Conformance¤ Basic Publisher

n FilterCapabilities/FilterLanguagen “OWS Request/Reply” Filter Language

n GetRecords, with OpenSearch templatesn Publications/Publication

n All recordsn DeliveryCapabilities/DeliveryMethod

n SSE / WebSockets

¨ Binding¤ RESTful (PSRB)

n The current draft allows a partial implementation¨ Technology

¤ Based on GI-cat

http://essi-lab.eu

Proof of Concept – interface

¨ /¤ GET (GetCapabilities)

¨ /subscription¤ POST (Subscribe)

n Body: SubscribeRequest in XMLn Success: SubscribeResponse with SubscriptionIdentifier,

DeliveryLocation (e.g. SSE endpoint, OAI-PMH endpoint)n Failure: HTTP Error

¤ GET (GetSubscription*)¨ /subscription/{subscriptionId}

¤ DELETE (Unsubscribe)¤ GET (GetSubscription)

http://essi-lab.eu

PubSub Conformance Classes

THE NEW OGC PUBLISH/SUBSCRIBE 1.0 STANDARD · 2016-09-05 · THE NEW OGC PUBLISH/SUBSCRIBE 1.0...

Documents