http://essi-lab.eu
THE NEW OGC PUBLISH/SUBSCRIBE 1.0 STANDARD
Lorenzo BigagliNational Research Council of Italy – Institute of AtmosphericPollution Research (CNR-IIA), [email protected]
Aaron BraeckelNational Center for Atmospheric Research (NCAR), USA
Geospatial Sensor WebsConference 2016
Münster (DE), 29-31 August 2016
http://essi-lab.eu
Outline
¨ Background and introduction¨ PubSub 1.0
¤ Rationale¤ Core profile¤ SOAP Binding
¨ Related work
http://essi-lab.eu
OGC PubSub SWG
¨ The Open Geospatial Consortium Publish/Subscribe Standards Working Group investigates a generic mechanism to support publish/subscribe requirements across OGC service interfaces and data types (coverage, feature, etc.)¤ Chartered in November 2010¤ ~80 members/observers
¨ Requirement analysis ended in March 2011¨ 3 documents in scope
¤ PubSub Core (PSC) – approved and published¤ PubSub SOAP Binding (PSSB) – approved and published¤ PubSub RESTful Binding (PSRB) – drafting underway
n To accelerate the process, it was decided to release PSC and PSSB first, and postpone the course of PSRB
http://essi-lab.eu 5
Request/Reply model
¨ May be inefficient when waiting for specific events, e.g. data arrival, server changes, data updates¤ cf. polling
OGC Service (WFS, WCS,
SOS, etc.)
OGC ServiceClient
GetCapabilities( )
Service Capabilities, Dataset Identifiers, etc.
DescribeDataset( identifier )
Dataset Descriptions
GetDataset( identifier, filter )
Filtered Dataset
http://essi-lab.eu
Publish/Subscribe model
¨ Asynchronous delivery of messages¨ Ability to specify an ongoing (persistent)
expression of interest¤ Stateful behaviour
Notification ServiceOGC ServiceClient
Subscribe( identifier, filter )
notify( datasetDescriptor or data )
notify( datasetDescriptor or data )
notify( datasetDescriptor or data )
http://essi-lab.eu
Publish/Subscribe & OGC Web Services
¨ OWS (e.g. WMS, WFS) have primarily addressed the Request/Reply model, considered sufficient for many use cases¤ Clients request data of interest when needed, and may
request periodic updates
¨ Publish/Subscribe is complementary¤ Initial state obtained via Request/Reply, then a subscription
may be created for updates¤ Can result in less (or more predictable) network traffic¤ Can be very well suited to urgent but unpredictable data,
e.g. events, hazard warnings, sensor failures
http://essi-lab.eu
PubSub (SWG) use-cases
¨ Service Filtered Data Push¨ Service Filtered
Notification¨ Notification of Threshold
Crossings¨ Handling Multiple Event
Types and Data Models¨ GeoSynchronization (GSS)¨ Brokered
publish/subscribe¨ Data Distribution
¨ FAA SAA Dissemination Pilot
¨ Emergency/Safety Critical Application
¨ Communication Protocol -Open GeoSMS
¨ Distributed Data Synchronization
¨ Message Aggregation¨ Message Filtering /
Batching Messages
http://essi-lab.eu
PubSub (SWG) rationale
¨ Provide notification capabilities as a module to existing services¤ No impact on existing service semantics¤ Reuse service-specific filtering semantics
¨ Support pushing actual data from OGC Web Services, i.e. WCS, WFS, SOS
¨ Support pushing notifications about new data¤ E.g. service and dataset updates, in order to optimize
harvesting by catalogs
http://essi-lab.eu
PubSub Core (PSC)
¨ Abstract description of the mandatory PubSubfunctionality
¨ Independent of the underlying binding technology¤ Extensions defines
several optional capabilitiesn E.g. batching, heartbeat
http://essi-lab.eu
Main design elements
¨ Pluggable filter language¤ E.g. OGC Filter Encoding
¨ Publication metadata¨ Pluggable delivery method
¤ E.g. ATOM, WS-Notification, AMQP
http://essi-lab.eu
PubSub SOAP Binding (PSSB)
¨ How the PubSubfunctionality is realized in SOAP¤ Based on OASIS WS-
Notification¤ Additional
conformance class: Reliable Publishern Based on WS-
ReliableMessaging
http://essi-lab.eu
Related work
¨ OGC Testbed 12 Initiative¤ PubSub RESTful Binding (PSRB)
n Drafting underwayn SSE delivery methodn ATOM delivery methodn XML and JSON formats
n Partial prototypes availablen PubSub-CSW (CNR), PubSub-WCS (NCAR), etc.
¤ AMQP delivery method, JMS binding, etc.¤ “OWS Request/Reply Publisher” Conformance Class
n Accepts OWS requests as filtersn Delivers OWS replies as updates/data pushes
http://essi-lab.eu
Example: PubSub-CSW MEP
1. CSW client subscribes with a specific getRecordsRequest2. Time-0 recordset obtained via standard Request/Reply3. Subsequent updates based on CSW Transaction semantics
CSWCSWClient
1. Subscribe( filter = GetRecordsRequest#1 )
3. Notify( TransactionResponse )
4. Notify( TransactionResponse )
2.1. GetRecordsResponse
2. GetRecordsRequest#1
http://essi-lab.eu
Conclusions
¨ The long-awaited OGC PubSub 1.0 standard bridges a gap between the existing SDI standard baseline and the many technologies supporting server-initiated, asynchronous communications
¨ We hope that a growing number of applications will benefit from an interoperable and uniform way of implementing event-based architectures
http://essi-lab.eu
THE NEW OGC PUBLISH/SUBSCRIBE 1.0 STANDARD
Lorenzo BigagliNational Research Council of Italy – Institute of AtmosphericPollution Research (CNR-IIA), [email protected]
Aaron BraeckelNational Center for Atmospheric Research (NCAR), USA
Geospatial Sensor WebsConference 2016
Münster (DE), 29-31 August 2016
http://essi-lab.eu
Testbed-12 Asynchronous Messaging for Aviation
¨ The architecture documented in this ER is one of the first implementations of the OGC PubSub 1.0 Specification. Hence, it is of great importance to assess and validate the specification itself, in particular as regards the following aspects:
¨ Extensibility of the PubSub 1.0 specification
¨ Harmonization of PubSub 1.0 specification, namely the SOAP binding, with AMQP 1.0 as a delivery method
¨ Integration and backwards compatibility with Harris Data Exchange (DEX) clients supporting the retrieval of data (e.g. AIXM, FIXM) via JMS
¨ As noted, the ER focuses on the Aviation domain, however a definition of a profile for aviation could serve as an example on how to integrate PubSub into other fieldsthat require near-real time, push-based data dissemination
¨ Use the OGC Filter Encoding 2.0 language to express subscription filters
¨ Pluggability of filter languages, e.g. to reference the correct geometry in
¨ aviation-specific data formats, via XPath/XQuery, etc.
¨ This ER also contributes to some of the activities in the current scope for the PubSub SWG, such as:
¨ Feasibility of adding PubSub capabilities to WFS 2.0 and WMS 1.3.0
¨ Definition of a generic mechanism to reuse service-specific data requests to PubSub-enable the existing OWSs
¨ Definition of lexical constants (e.g. "all" to identify all the publications offered by a Publisher).
¨ AMQP is a broadly useful and popular delivery method, and should be considered for standardization in the PubSub SWG. The ER authors are encouraged to bringan AMQP delivery method document to the SWG for consideration and standardization.
¨ Finally, the ER provides useful indications on future lines of work for the PubSub specification, such as:
¨ OGC PubSub 1.0 profile for moving objects
¨ Specific mechanisms for standardizing delivery methods
¨ Analysis on the role of Message Brokers in enterprise production environments (cf. the PubSub Brokering Publisher)
http://essi-lab.eu
FilterCapabilities
http://essi-lab.eu
FilterCapabilities
http://essi-lab.eu
PublicationsName Definition Data type and
values Multiplicity and
use
boundingBox The area of interest of the published data contents
BoundingBox [see
OGC 06-121r3]
Zero or more (Optional)
contentType The content type (i.e., mime type) of the published data contents.
Examples: “application/weather+xml”, “text/plain”
MimeType
[see
OGC 06-121r3]
One (Mandatory)
description A human-readable description
DescriptionType [see
OGC 06-121r3]
Zero or one (Optional)
identifier A unique identifier URI One (Mandatory)
metadata Additional metadata on this publication
Metadata [see
OGC 06-121r3]
Zero or one (Optional)
supportedDeliveryMethod
The supported delivery methods for this publication
URI One to many (Mandatory)
supportedFilterLanguage
The filter language identifiers that are offered for filtering
URI Zero to many (Optional)
formalContentDefinition Language
The identifier of the language (e.g., "http://www.w3.org/XML/Schema/1.0”) used to describe the formal publication content definition
URI Zero to many (Optional)
http://essi-lab.eu
Publications
http://essi-lab.eu
DeliveryCapabilities
http://essi-lab.eu
DeliveryCapabilities
http://essi-lab.eu
SSE Delivery
¨ Pull¤ Not needed
¨ Push¤ Over HTTP
n ContentType: "text/event-stream;charset=UTF-8”n Cache-Control: "no-cache”n Connection: "keep-alive”
¨ Format¤ CSW TransactionResponse
http://essi-lab.eu
OAI-PMH Delivery
¨ Pull¤ Polling using “from” parameter
¨ Push¤ Not available
¨ Format¤ CSW TransactionResponse
http://essi-lab.eu
ATOM delivery
¨ Pull¤ Polling using “If-Modified-Since”, “start-index” parameters
¨ Push¤ PubSubHubbub?
¨ Format¤ Feed
n Entry (timestamp)n TransactionResponse
n PUT Record1000n DELETE Record2n UPDATE Record1
http://essi-lab.eu
Proof of Concept – PubSub-CSW
¨ PubSub Core Conformance¤ Basic Publisher
n FilterCapabilities/FilterLanguagen “OWS Request/Reply” Filter Language
n GetRecords, with OpenSearch templatesn Publications/Publication
n All recordsn DeliveryCapabilities/DeliveryMethod
n SSE / WebSockets
¨ Binding¤ RESTful (PSRB)
n The current draft allows a partial implementation¨ Technology
¤ Based on GI-cat
http://essi-lab.eu
Proof of Concept – interface
¨ /¤ GET (GetCapabilities)
¨ /subscription¤ POST (Subscribe)
n Body: SubscribeRequest in XMLn Success: SubscribeResponse with SubscriptionIdentifier,
DeliveryLocation (e.g. SSE endpoint, OAI-PMH endpoint)n Failure: HTTP Error
¤ GET (GetSubscription*)¨ /subscription/{subscriptionId}
¤ DELETE (Unsubscribe)¤ GET (GetSubscription)
