OMG DDS: The Data Distribution Service for Real-Time Systems

Angelo Corsaro, Ph.D.Product Strategy Manager

OMG RTESS and DDS SIG [email protected]

Delivering Performance, Openness, and Freedom

OpenSplice DDS

OMG DDS: The Data Distribution Service for Real-Time Systems

mailto:[email protected]


Age

nda

Motivating Forces

The Standard

Architectural Principles

Programming Model

Future Directions

Complex Information Management

Scalability, Persistence, Security

Distributed Systems

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.

© 2009, PrismTech. All Rights Reserved

The Need for Data DistributionGeographical ScaleTime Scale

Near Real-Time Fault-Tolerant Information

Processing

Throughput, Availability

Real-Time Information Processing

DeterminismParallelism

Systemic Signal

Processing

Data Processing

Parallel Systems

Data Distribution

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Data Exchange / Distribution‣ Network Centric Architectures are emerging as a key trend for next generation military and civil

system of systems

‣ Efficient, scalable and QoS-enabled data dissemination is an enabling technology for Network Centric Systems

Adapted from “The Future of AWACS”,

by LtCol Joe Chapa

Joint Forces

Global Info Grid

Joint Forces

Global Info Grid

The Right Information => To the Right People => At the Right Time

Age

nda

Motivating Forces

The Standard


Programming Model

Future Directions

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Addressing Data Distribution Challenges

The OMG DDS Standard‣ Introduced in 2004 to address the Data

Distribution challenges faced by a wide class of Defense and Aerospace Applications

‣ Key requirement for the standard were its ability to deliver very high performance while seamlessly scaling from embedded to ultra-large-scale deployments

‣ Today recommended by key administration worldwide and widely adopted across several different application domains, such as, Automated Trading, Simulations, SCADA, Telemetry, etc.

DDS is standard designed to address the data-distribution challenges across a wide class of Defense and Aerospace Applications

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


The OMG Data Distribution Service (DDS)

DDS v1.2 API Standard‣ Language Independent, OS and HW architecture

independent

‣ DCPS. Standard API for Data-Centric, Topic-Based, Real-Time Publish/Subscribe

‣ DLRL. Standard API for creating Object Views out of collection of Topics

DDSI/RTPS v2.1 Wire Protocol Standard‣ Standard wire protocol allowing interoperability

between different implementations of the DDS standard

‣ Interoperability demonstrated among key DDS vendors in March 2009

Object/Relational Mapping

Ownership DurabilityContent

Subscription

Minimum Profile

Data Centric Publish/Subscribe (DCPS)

Data Local Reconstruction Layer (DLRL)

DDS Interoperability Wire Protocol

Application

UDP/IP

Real-Time Publish/Subscribe Protocol

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


OMG DDS RecommendationsMandated/Recommended by key Administrations

‣US Navy: Open Architecture

‣DISR/DISA: Net-centric Systems

‣EuroControl: Air Traffic Control Center Operational Interoperability

‣QinetiQ: Recommending DDS for VSI

A Great Time for DDS!

! The infrastructure evolution cycle is churning…– New -> Emerging -> Standard -> Commodity

– Middleware is emerging as OS declines

! …DDS is maturing…– OMG focus

– Wire spec

– Tools

– Enterprise integration

– Multiple products fielded

– Deployed applications!

! …and adoption is on the rise– Navy

– DISR

– FCS/SoSCOE

– Many significant applications

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


OMG DDS AdoptionsAdopted by Most Challenging International Programs

‣US FCS/SOSCOE

‣European Next Generation Flight Data Processor

‣US ZUMWALT [DDG(1000)]

‣VTID adopting DDS for the architectural demonstrator

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


OMG DDS ApplicabilityAdapted from NSWC-DD Open Architecture Documentation

Non Real-Time Soft Real-Time Hard Real-Time Extreme Real-Time

Com

mun

icat

ion

Tech

nolo

gies

Sta

ndar

ds

The DDS is the only technology that spans across the board.

Web SVCs

Java RTSJ

JMS

CORBA RT-CORBA

OMG Data Distribution Service (DDS)

MPI

It guarantees exceptional real-time behavior, while providing unparalleled level of throughput !

Message Passing Interface

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Some DDS Use CasesDefense

‣ Combat Management Systems

‣ Flycatcher system

‣ Unmanned Vehicles (Air + Land)

‣ Tactical Links

‣ Radar Processing

‣ Submarine Systems

‣ Future Combat Systems

‣ Simulation

SCADA/Utilities

‣ Industrial Automation

‣ Power Grids

‣ Rocket Launch Systems

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Some DDS Use Cases

Transportation

‣ Air Traffic Management/Control

‣ Metropolitan Traffic Management

‣ Underground Metropolitan Transportation

Financial Services

‣ Automated Trading Firms

‣ Compliance Systems

‣ Market Data Platforms

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Leading DDS Implementations

‣ Full Implementation of the OMG DDS Standard (DCPS+DLRL + DDSI)

‣ DCPS + DDSI available as Open Source at http://www.opensplice.org

‣ Developed and Commercially supported by PrismTech

‣ Implementation of the OMG DDS Standard (DCPS + DDSI)

‣ Developed and Commercially supported by RTI

‣ Partial Implementation of the OMG DDS Standard (Subset of DCPS)

‣ Available as Open Source at http://www.opendds.com

‣ Developed and Commercially supported by OCI

http://www.opendds.com

http://www.opendds.com

D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m

OpenSplice DDS

DDS vs AMQP

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Scope of Standardization



Subscription

Minimum Profile




Application

UDP/IP


DD

SI

v2

.1D

DS

v1

.2

Interoperability Wire Protocol

(TCP, SCTP, UDP)/IP

Advanced Message Queuing Protocol

AM

QP

v0

.10

API (?)

Application

Wire Protocol Standard

Pub/Sub Standard API

Object Relational Mapping Standard API

AMQP OMG DDS

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Architectural Style

Publisher

SubscriberPublisher

Publisher

Subscriber

Subscriber

Broker

OMG DDS

Publisher

SubscriberPublisher

Publisher

Subscriber

Subscriber

Broker

Broker

Broker

Broker

Broker

AMQP

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Key Differences in Summary

‣ Brokered Communication Architecture

‣ Peer-to-Peer Communication Architecture

AMQP

‣ Wire Protocol Standard‣ Interoperability across AMQP

implementations

‣ Wire Protocol + API Standard‣ Interoperability across DDS implementations

‣ Application Portability across DDS Implementation

‣ Header Based Routing ‣ Content-Based Filtering/Quering

‣ Not geared for High-Performance ‣ Designed bottom-up for maximum performance and determinism

DDS

Age

nda

Motivating Forces

The Standard


Programming Model

Future Directions


OpenSplice DDS

Loosely Coupled Technology

Publisher

Subscriber

Publisher

Publisher

Subscriber

Subscriber

Subscriber

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Counting the “W’s” of Coupling

Client

Request

Reply

Server The “4Ws” of Client/Server‣ Who+Where: Space Coupling

‣ What: Structural Coupling

‣ When: Time CouplingClient/Server CORBA, COM+, Java RMI, .Net Remoting, Web SVCs

DDS

A Single “W” for DDS‣ What: Structural Coupling

DDS features dynamic discovery and full time decoupling!


OpenSplice DDS

High Performance/Availability Pub/Sub

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


High Performance Pub/Sub

‣ Fully distributed, Peer-to-Peer Communication

‣No Single Point of Failure

‣No Single Point of Bottleneck

‣Multicast-enabled

‣High performance and highly scalable

‣High availability

‣hot-swap

‣hot-hot architecture

The right data, at the right place, at the right time

-- All the Time.

Publisher

SubscriberPublisher

Publisher

Subscriber

Subscriber

Brokers


OpenSplice DDS

Data-CentricPub/Sub



Subscription

Minimum Profile




Application

UDP/IP


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Data-Centric Pub/Sub

‣ Distributed Relational Data Model

‣ Local Queries

‣ Continuous Queries / Content Based Subscriptions

‣Windows

‣ Object/Relational Mapping

‣ Support for a subset of SQL-92

Perfect Blend of Data-Centric and Real-Time Publish/Subscribe Technologies

‣ Data-Centric Features are built-in and don’t rely on an external DBMS

‣ Providing thus performance, scalability, and availability

A

B

CD

E

F

m

J

K

Publisher

SubscriberPublisher

Publisher

Subscriber

Subscriber

DBMS

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


“Visualizing” Data-Centric Pub/Sub

A

B

CD

E

F

m

J

K

Publisher

D

E

J

K

A F

CD

A

B D

E

J

K

Subscriber

Subscriber

Subscriber

CD

A

B

Publisher

J

K

A F

Publisher

D

E

A

B

CD

E

F

m

J

K

Publisher

SubscriberPublisher

Publisher

Subscriber

Subscriber

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Organizing Information

‣ All DDS communication is happens within a Domain

‣ Domain can divided into Partitions

‣ Topics are published and subscribed across on or more Partitions

DomainPartition

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topics and Data-Centric Pub/Sub‣ Topics. Unit of information exchanged between

Publisher and Subscribers.

‣ Data Types. Type associated to a Topic must be a structured type expressed in IDL

‣ Topic Instances. Key values in a datatype uniquely identify a Topic Instance (like rows in table)

‣ Content Awareness. SQL Expressions can be used to do content-aware subscriptions, queries, joins, and correlate topic instances

struct TempSensor { long tID; float temp; float humidity; };#pragma keylist TempSensor tID

tID temp humidity

1

2

3

21 62

27 78

25.5 72.3

TempSensor

Instances

tID

temp

humidity

1

21

62

tID

temp

humidity

1

23

65

tID

temp

humidity

1

22

63

Timet1 t2 t3

Samples

Topic Type

Topic

SELECT * FROM TempSensor t WHERE t.temp > 25

tID temp humidity

2

3

27 78

25.5 72.3

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Distributed Relational Information Modeling

‣ Topic Keys can be used to identify instances as well as relationships

‣ Relationships can be navigated by relying on a subset of SQL 92

‣ One-to-many relationships can be captured using foreign keys

‣Many-to-many relationships need to be modeled using a topics

‣ Keys can be represented by an arbitrary number of Topic fields

tID: inttemp: floathumidity: floatrID: int

TempSensorrID: intwidth: floatheight: floatlength: floatfID: int

Room

fID: intlevel: intopen: int

Floor

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Data Centric Publish/Subscribe

Fully Distributed Global Data Space

struct TempSensor { int tID; float temp; float humidity; };#pragma keylist TempSensor tID

tID temp humidity

1 18 60

tID temp humidity

1

2

3

18 60

22 75

21 71

tID temp humidity

3 21 71

TempSensor

TempSensor

tID temp humidity

2 22 75

TempSensor

Publishers Subscribers

TempSensor

tID temp humidity

2

3

22 75

21 71

TempSensor

SELECT * FROM TempSensor t WHERE s.temp > 20

tID temp humidity

1 18 60

TempSensor

s.tID == 1

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Data Centric Publish/Subscribe

Fully Distributed Global Data Space

tID temp humidity

1

2

3

21 62

22 75

21 71

tID temp humidity

3 21 71

TempSensor

TempSensor

tID temp humidity

2 22 75

TempSensor

Publishers Subscribers

TempSensor

TempSensor

SELECT * FROM TempSensor t WHERE s.temp > 20

tID temp humidity

1 21 62

TempSensor

s.tID == 1

tID temp humidity

1 21 62

tID temp humidity

1

2

3

21 62

22 75

21 71


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Processing Content & Structure

P

S1

S2

S3

{iID = 1}

{temp > 21 | humidity > 75}

{ temp <16 }

DDS provides some of the most useful features found in Complex Event Processing platforms!

Complex Event Processing...

tID temp humidity

3 21.5 72.3

tID temp humidity

2 20 78tID temp humidity

1 21 62

‣ Local Queries

‣ Continuous Queries (Content-based Subscriptions)

‣ Topics Joins & Projections

‣ Events windows (via History QoS)

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Processing Content & Structure

P

S1

S2

S3

{iID = 1}

{temp > 21 | humidity > 75}

{ temp <16 }

Complex Event Processing...

tID temp humidity

1 21 62

tID temp humidity

2

3

20 78

21.5 72.3

DDS provides some of the most useful features found in Complex Event Processing platforms!

‣ Local Queries

‣ Continuous Queries (Content-based Subscriptions)

‣ Topics Joins & Projections

‣ Events windows (via History QoS)


OpenSplice DDS

Object-OrientedPub/Sub



Subscription

Minimum Profile




Application

UDP/IP


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



‣ Automatically bridges the Object/Relational Impedance Mismatch

‣ Arbitrary object reconstructions

‣ Automatic Relationships Management

‣ Inheritance

‣ Local Operations

‣ Local/Distributed State

getTemp()getHumidity()

temp: floathumidity: float

TempSensor

set_climate();

width: floatheight: floatlength: floatclimate: int

Room

set_status()set_climate()get_level()

level: intopen: intclimate: int

Floor

setTemp()setHumidity()setFanLevel()start()stop()pause():


Conditioner * 1

*

1

*1


TTempSernsor

rID: intwidth: floatheight: floatlength: floatfID: int

TRoom


TFloor

cID: inttemp: floathumidity: floatfan_level: intstatus: intrID: int

TConditioner

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Object/Relational MappingUnleashing the power of Objects...

Relational

Object Oriented

call_sign: stringroute: listorigin: stringdest: stringaircraft: stringradar_id: string

FlightTopic

x: floaty: floatz: floatradar_id: string

RadarTrackTopicimage: stringrefresh_rate: intvisible: booluid: GUID

FlightViewTopiccall_sign: stringuid: GUIDhandle: Handle

FlightViewAssociation

OO ➡ Relational

‣ Middleware can automatically manage the generation and association between the Object-Oriented Model and the Relational Model

Relational ➡ OO

‣ The Relational Model can be mapped to an Object Oriented model‣ The mapping is under control of the architect

operationOne()operationTwo()


RadarTrack

operationX()operationY()

call_sign: stringroute: listorigin: stringdest: stringaircraft: string

Flight

*1

display()

image: stringrefresh_rate: intvisible: booluid: GUID

FlightView

**

handle: HandleCanvas

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Multiple Object/Relational Mappings

DCPScall_sign: stringroute: listorigin: stringdest: stringaircraft: stringradar_id: string

FlightTopic


RadarTrackTopicimage: stringrefresh_rate: intvisible: booluid: GUID

FlightViewTopiccall_sign: stringuid: GUIDhandle: Handle

FlightViewAssociation

DL

RL

display()

image: stringrefresh_rate: intvisible: booluid: GUIDx: floaty: floatz: floatcall_sign: string

FastFlightView

DLRL

Multiple Mappings‣ Different Object/Relatioal Mappings (ORM) can be used for different subsystems‣ The state is shared across all the local reconstruction (ORMs)



RadarTrack



Flight

*1

display()


FlightView

**

handle: HandleCanvas

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


An Example


FlightViewTopic


FlightTopic



RadarTrack


RadarTrackTopic

display()


FastFlightView

Primary RadarHigh-End 3D Visualization

Flight Data Processor

RadarTrack



Flight

RadarTrackList


RadarTrackTopic


RadarTrackTopic


RadarTrackTopicx: floaty: floatz: floatradar_id: string


RadarTrackTopic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


An Example


FlightViewTopic


FlightTopic



RadarTrack

display()


FastFlightView



RadarTrack



Flight

RadarTrackList


RadarTrackTopic


RadarTrackTopic


RadarTrackTopic


RadarTrackTopic


RadarTrackTopic




RadarTrackTopic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


An Example


FlightViewTopic


FlightTopic



RadarTrack


RadarTrackTopic


RadarTrackTopic

display()


FastFlightView




RadarTrackTopic



RadarTrack

Secondary Radar


RadarTrackTopic

RadarTrack



Flight

RadarTrackList

RadarTrack


RadarTrackTopic

Automatically Manage Relationships

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Custom Filter Example


FlightViewTopic


FlightTopic



RadarTrack


RadarTrackTopic

display()


FastFlightView





RadarTrack

Secondary Radar


RadarTrackTopic

RadarTrack



Flight

RadarTrackList

RadarTrack

SQL FilterCustom Filter

SQL FilterCustom Filter

Filter based on Region-of-interest

Automatically Manage Relationships


RadarTrackTopic


RadarTrackTopic


RadarTrackTopic


RadarTrackTopic




RadarTrackTopic


OpenSplice DDS

...Putting it all Together

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


DCPS Application

‣ The application works directly at the DCPS level

‣When using OO Programming Languages the Object/Relational Impedance Mismatch has to be manually dealt


Subscription

Minimum Profile


DD

S v

1.2


TTempSensor


TRoom


TFloor


TConditioner

Application

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


DLRL Application

‣ The DLRL Layer is used to provide a Language Integrated access to DDS data

‣ The Designer has great freedom in deciding how Objects have to map to Topics

‣ Different Object Reconstruction can be created for different applications Ownership Durability

Content Subscription

Minimum Profile


DD

S v

1.2



TempSensor

set_climate();set_temp()set_humidity()


Room



Floor


temp: floathumidity: floatfan_level: intstatus: int

Conditioner *1

*

1

*

1


TTempSensor


TRoom


TFloor


TConditioner


Data Local Reconstruction Layer (DLRL) DD

S v

1.2

Application

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Generic DDS Application

‣ In the most general case, different portion of the application might rely on DLRL or DCPS depending on their specific needs

‣ DCPS access might be required for accessing and tuning some specific QoS




Subscription

Minimum Profile


DD

S v

1.2



TempSensor

set_climate();set_temp()set_humidity()


Room



Floor


temp: floathumidity: floatfan_level: intstatus: int

Conditioner *1

*

1

*

1


TTempSensor


TRoom


TFloor


TConditioner

Application

Application


OpenSplice DDS

QoS-EnabledPub/Sub

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Communicating with Topics, Partitions and Domains

1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic

Instances1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic

Instances1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic

InstancesDataReaderDataReaderDataReaderDataReaderDataReaderDataReaderDataWriterDataWriterDataWriterDataWriter


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic

Instances1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic

Instances1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic


Partition


PublisherSubscriber

Arrows show structural relationships, not data-flows

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic

Instances1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic

Instances1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic


PublisherSubscriberPartition

Domain

Domain Participant


Arrows show structural relationships, not data-flows

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


QoS Model

‣ QoS-Policies are used to control relevant properties of OpenSplice DDS entities, such as:‣ Temporal Properties

‣ Priority

‣ Durability

‣ Availability

‣ ...

‣ Some QoS-Policies are matched based on a Request vs. Offered Model thus QoS-enforcement

‣ Publications and Subscriptions match only if the declared vs. requested QoS are compatible ‣ e.g., it is not possible to match a publisher which delivers data unreliably with a subscriber which requires reliability

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

......

QoS QoS

Type Matching

DomainParticipant DomainParticipant

QoS QoS

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Sample QoS Policies

‣ Rich set of QoS allow to configure several different aspects of data availability, delivery and timeliness

‣ QoS can be used to control and optimize network as well as computing resource

QoS Policy Applicability RxO Modifiable

DURABILITY

DURABILITY SERVICE

LIFESPAN

HISTORY

PRESENTATION

RELIABILITY

PARTITION

DESTINATION ORDER

OWNERSHIP

OWNERSHIP STRENGTH

DEADLINE

LATENCY BUDGET

TRANSPORT PRIORITY

TIME BASED FILTER

RESOURCE LIMITS

USER_DATA

TOPIC_DATA

GROUP_DATA

T, DR, DW Y N Data AvailabilityT, DW N N

Data Availability

T, DW - Y

Data Availability

T, DR, DW N N

Data Availability

P, S Y N Data DeliveryT, DR, DW Y N

Data Delivery

P, S N Y

Data Delivery

T, DR, DW Y N

Data Delivery

T, DR, DW Y N

Data Delivery

DW - Y

Data Delivery

T, DR, DW Y Y Data TimelinessT, DR, DW Y Y

Data Timeliness

T, DW - Y

Data Timeliness

DR - Y Resources

T, DR, DW N N

Resources

DP, DR, DW N Y ConfigurationT N Y

Configuration

P, S N Y

Configuration

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

......

QoS QoS

Type Matching

DomainParticipant DomainParticipant

QoS QoS

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Mapping QoS

Data Latency

LatencyBudget TransportPriority

Throughput

DeadlineTimeBasedFilter

Control over Latency/Throughput tradeoff Control over data latency Control over data priority

Data Availability

OwnershipOwnershipStrength

Durability

History

Lifespan

Control over data queueing Control over data persistency Control over data sources hot-swap

Data Delivery

Reliability

DestinationOrder

Presentation

Control over data distribution reliability Control over data ordering Control over presentation

OpenSplice DDS provides programmatic QoS-driven support for configuring the most important properties of data distribution!

Which properties does QoS controls?


Data Timeliness

Data Latency

LatencyBudget TransportPriority

Throughput

DeadlineTimeBasedFilter


DEADLINE

LATENCY

BUDGET

TRANSPORT

PRIORITY

T, DR, DW Y Y

Data

Timeliness

T, DR, DW Y Y

T, DW - Y

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


DeadlineThe DEADLINE QoS policy allows to define the maximum inter-arrival time between data samples


DEADLINE T, DR, DW Y Y

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......

You can’t be later than...

‣ DataWriter indicates that the application commits to write a new value at least once every deadline period

‣ DataReaders are notified by the DDS when the DEADLINE QoS contract is violated

Publisher Subscriber

DeadlineDeadlineDeadlineDeadlineDeadline

Deadline Violation

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Latency BudgetThe LATENCY_BUDGET QoS policy specifies the maximum acceptable delay from the time the data is written until the data is inserted in the receiver's application-cache


LATENCY

BUDGET

T, DR, DW Y Y

‣ The default value of the duration is zero indicating that the delay should be minimized

‣ This policy is a hint to the DDS, not something that must be monitored or enforced.

I need to get there in at most...

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......

T1

T2

T3

Latency Budget = Latency = TBuff +T1+T2+T3TBuff

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Transport Priority

The TRANSPORT_PRIORITY QoS policy is a hint to the infrastructure as to how to set the priority of the underlying transport used to send the data.


TRANSPORT

PRIORITY

T, DW - Y

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......

VIP Data, stay clear!


Data Availability

Data Availability

OwnershipOwnershipStrength

Durability

History

Lifespan


DURABILITY

DURABILITY

SERVICE

LIFESPAN

HISTORY

T, DR, DW Y N

Data

Availability

T, DW N N

T, DW - Y

T, DR, DW N N

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Durability

‣ Volatile. No need to keep data instances for late joining data readers

‣ Transient Local. Data instance availability for late joining data reader is tied to the data writer availability

‣ Transient. Data instance availability outlives the data writer

‣ Persistent. Data instance availability outlives system restarts

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......

The DURABILITY QoS controls the data availability w.r.t. late joiners, specifically the DDS provides the following variants:

The DURABILITY_SERVICE QoS provide control over configuration of the service that implements the transient and persistent durability features


DURABILITY

DURABILITY

SERVICE

T, DR, DW Y N

T, DW N N

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Lifespan

‣ The LIFESPAN QoS policy allows to control what happens to stale data

‣ It specifies the validity interval for data written by the DataWriter

‣ The default validity interval is infinite


LIFESPAN T, DW - Y

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......

For how long will this data be available?

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


HistoryThe HISTORY QoS policy controls whether the DDS should deliver only the most recent value, attempt to deliver all intermediate values, or do something in between. The policy can be configured to provide the following semantics:


HISTORY T, DR, DW N N

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......

‣ Keep Last. The DDS will only attempt to keep the most recent “depth” samples of each instance of data identified by its key

‣ Keep All. The DDS will attempt to keep all the samples of each instance of data identified by its key.

‣ On the DataWriter Samples are kept until delivered to all known subscribers

‣ On the DataReader side samples are kept until the application “takes” them

How many data samples should I keep?


Data DeliveryData Delivery

Reliability

DestinationOrder

Presentation


PRESENTATION

RELIABILITY

PARTITION

DESTINATION

ORDER

OWNERSHIP

OWNERSHIP

STRENGTH

P, S Y N

Data Delivery

T, DR, DW Y N

P, S N Y

T, DR, DW Y N

T, DR, DW Y N

DW - Y

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Reliability

The RELIABILITY QoS indicate the level of guarantee offered by the DDS in delivering data to subscribers. Possible variants are:


RELIABILITY T, DR, DW Y N

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......

‣ Reliable. In steady-state the middleware guarantees that all samples in the DataWriter history will eventually be delivered to all the DataReader

‣ Best Effort. Indicates that it is acceptable to not retry propagation of any samples

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Ownership The OWNERSHIP QoS specifies whether it is allowed for multiple DataWriters to write the same instance of the data and if so, how these modifications should be arbitrated. Possible choices are:

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......


OWNERSHIP T, DR, DW Y N

‣ Shared. Multiple writers are allowed to update the same instance and all the updates are made available to the reader

‣ Exclusive. Indicates that each instance can only be owned by one DataWriter, but the owner of an instance can change dynamically -- due to liveliness changes

‣ The selection of the owner is controlled by the setting of the OWNERSHIP_STRENGTH QoS policy

Who owns the data?

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Ownership StrengthThe OWNERSHIP_STRENGTH Specifies the value of the “strength” used to arbitrate among DataWriters that attempt to modify the same data instance

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......QoS Policy Applicability RxO Modifiable

OWNERSHIP

STRENGTH

DW - Y

‣ Data instance are identified by the couple (Topic, Key)

‣ The policy applies only if the OWNERSHIP is EXCLUSIVE

How strong are you?

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Ownership @ Work‣ A Topic can have Shared or Exclusive Ownership

‣ Exclusively owned Topics can be modified by a single writer

‣ Writer strength is used to coordinate replicated writers

R1

R2

R3

symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73

StockQuote

symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97

StockQuote

W1

W2’’

STRENGTH=3

STRENGTH=1


StockQuote


StockQuotesymbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37

StockQuote

W1’

STRENGTH=2


StockQuote



StockQuote


StockQuote


StockQuote

symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37


StockQuote


StockQuote

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Managing Hot-Swap‣ A Topic can have Shared or Exclusive Ownership

‣ Exclusively owned Topics can be modified by a single writer

‣ Writer strength is used to coordinate replicated writers


StockQuote


StockQuote


StockQuote


StockQuote


StockQuote


StockQuote


StockQuote


StockQuote


StockQuote

R1

R2

R3W2’’

STRENGTH=1

Age

nda

Motivating Forces

The Standard


Programming Model

Future Directions


OpenSplice DDS

Shared Global Data Space

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Shared Global Data SpaceCoordination Model

‣ DDS applications are asynchronous and communicate by reading/writing from/to a Global Data Space

‣ DDS applications communicate by simply addressing items in the Global Data Space and without any direct knowledge of the parties involved in the production/consumption of data

Consistency Model

‣ The Shared Global Data Space implemented by DDS, can be configured to supports at most the “Eventual Consistency Model”

‣ Under an Eventual Consistency Model we are guaranteed that eventually all application in the system will have a consistent view of the “world”s

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Eventual Consistency & R/W Caches

Under an Eventual Consistency Model, DDS guarantees that all matched Reader Caches will eventually be identical of the respective Writer Cache

Topic

DataReader

1 1

2 1

3 1

DataReader Cache Topic

1 2

2 3

3 1

DataWriter Cache

DataWriter

1 2

2 2 2 3

Topic

DataReader

1 1

2 1

3 1

DataReader Cache

Topic

DataReader

1 1

2 1

3 1

DataReader Cache

DDS

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


QoS Impacting the Consistency Model

The DDS Consistency Model is a property that can be associated to Topics or further refined by Reader/Writers. The property is controlled by the following QoS Policies:

‣ DURABILITY ‣ VOLATILE | TRANSIENT_LOCAL | TRANSIENT | PERSISTENT

‣ LIFESPAN

‣ RELIABILITY‣ RELIABLE | BEST_EFFORT

‣ DESTINATION ORDER‣ SOURCE_TIMESTAMP | DESTINATION_TIMESTAMP


DURABILITY

LIFESPAN

RELIABILITY

DESTINATION ORDER

T, DR, DW Y N

T, DW - Y

T, DR, DW Y N

T, DR, DW Y N

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


QoS Impacting the Consistency Model

DURABILITY RELIABILITY DESTINATION_ORDER LIFESPAN

Eventual Consistency (No Crash / Recovery)

Eventual Consistency (Reader Crash / Recovery)

Eventual Consistency(Crash/Recovery)

Eventual Consistency(Crash/Recovery)

Weak Consistency

Weak ConsistencyWeak Consistency

VOLATILE RELIABLE SOURCE_TIMESTAMP INF.

TRANSIENT_LOCAL RELIABLE SOURCE_TIMESTAMP INF.

TRANSIENT RELIABLE SOURCE_TIMESTAMP INF.

PERSISTENT RELIABLE SOURCE_TIMESTAMP INF.

ANY ANY DESTINATION_TIMESTAMP ANY

ANY BEST_EFFORT ANY ANY

ANY ANY ANY N

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Eventual Consistency @ Work

B A

B

CD

E

F

m

J

K

A

P1

P2

S1

S2

B

A

B

A

P = {A, B}

S = {A, D}

S= {A, B, J}

P = {D, C, J}

B

S4

S = {A}

A

Eventual Consistency (Reader Crash / Recovery)Eventual Consistency(Crash/Recovery)




{A}

{B}

Weak Consistency ANY ANY ANY N {J}

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Eventual Consistency @ Work

A

B

CD

E

F

m

J

KP2

S1

S2

S3

A

BA

B

S= {A, B, D, J}

J

B

D

J

S = {A, D}

S= {A, B, J}

P = {D, C, J}

S4

S = {A}

A

Eventual Consistency (Reader Crash / Recovery)Eventual Consistency(Crash/Recovery)




{A}

{B}

Weak Consistency ANY ANY ANY N {J}

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Design Guidelines

‣ For all (non-periodic) Topics for which an eventually consistent model is required use the following QoS settings:

‣ For information produced periodically, with a period P, where P is small enough to be acceptable as a consistency convergence delay, the following QoS settings will provide an approximation of the eventual consistency:

Eventual Consistency (Crash / Recovery)



Eventual Consistency (Crash / Recovery)

VOLATILE BEST_EFFORT SOURCE_TIMESTAMP INF.



OpenSplice DDS

Getting the DataRead Semanticsstruct Counter {

int cID; int count;};#pragma keylist Counter cID

Topic Used in next section.

DataReader Cache

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Reading Samples

‣ Read iterates over the available sample instances

‣ Samples are not removed from the local cache as result of a read

‣ Read samples can be read again, by accessing the cache with the proper options (more later)

DataReader

1 1

2 1

3 1

2 2

Samples Read Samples not Read

1 2 1 3 1 4

2 3

3 2 3 3 3 4 3 5

DataReader Cache

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Reading Samples





DataReader

1 1

2 1

3 1

2 2

1 2 1 3 1 4

2 3

3 2 3 3 3 4 3 5

DataReader Cache

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Reading Samples





DataReader

1 1

2 1

3 1

2 2

1 2 1 3 1 4

2 3

3 2 3 3 3 4 3 5

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Sample, Instance and View States

‣ DataReaders are provided with data samples ane state information allowing to detect relevant transitions in the life-cycle of data as well as data writers

‣ Sample State (READ | NOT_READ): Determines wether a sample has already been read by this DataWriter or not.

‣ Instance State (ALIVE, NOT_ALIVE, DISPOSED). Determines wether (1) writer exist for the specific instance, or (2) no matched writers are currently available, or (3) the instance has been disposed

‣ View State (NEW, NOT_NEW). Determines wether this is the first sample of a new (or re-born) instance

Dat

aRea

der

History Depth = 2

DataReader Cache

Samples

1 1

2 2

3 1

1 2

2 3

SampleInfo

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Anatomy of a Read

DDS::ReturnCode_t read( TSeq& samples, DDS::SampleInfoSeq& info, long max_samples, DDS::SampleStateMask samples_state, DDS::ViewStateMask views_state, DDS::InstanceStateMask instances_state)

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Reading Data

TSeq samples;SampleInfoSeq info;

reader->read(samples, info, DDS::LENGTH_UNLIMITED, DDS::NOT_READ_SAMPLE_STATE, DDS::ANY_VIEW_STATE, DDS::ANY_INSTANCE_STATE);

// Do something

reader->return_loan(samples, info);

TSeq samples(N);SampleInfoSeq info(N);

reader->read(samples, info, N, DDS::NOT_READ_SAMPLE_STATE, DDS::ANY_VIEW_STATE, DDS::ANY_INSTANCE_STATE);

// Do something

Example 1 Example 2

Example 3

reader->read(samples, info, N, DDS::NOT_READ_SAMPLE_STATE, DDS::NEW_VIEW_STATE, DDS::ALIVE_INSTANCE_STATE);


OpenSplice DDS

Getting the DataTake Semanticsstruct Counter {

int cID; int count;};#pragma keylist Counter cID


DataReader Cache

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Taking Samples

‣ Take iterates over the available sample instances

‣ Taken Samples are removed from the local cache as result of a take

‣DataReader

1 1

2 1

3 1

2 2

Samples not Taken

1 2 1 3 1 4

2 3

3 2 3 3 3 4 3 5

DataReader Cache

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Taking Samples



Samples not Taken

DataReader 2 2

1 2 1 3 1 4

2 3

3 2 3 3 3 4 3 5

DataReader Cache

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Taking Samples



Samples not Taken

DataReader

1 3 1 4

2 3

3 3 3 4 3 5

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Anatomy of a take

DDS::ReturnCode_t take( TSeq& samples, DDS::SampleInfoSeq& info, long max_samples, DDS::SampleStateMask samples_state, DDS::ViewStateMask views_state, DDS::InstanceStateMask instances_state)


OpenSplice DDS

Writing Datastruct Counter { int cID; int count;};#pragma keylist Counter cID


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Writing Samples

‣ Samples are written in the local cache

‣ Writer control the creation of instances

‣ The DDS ensures that the local caches for the matched DataReader will be eventually consistent with that of the Data Writer DataWriter Cache

Topic

1 1

Last Sample Written

DataWriter

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Writing Samples



‣ The DDS ensures that the local caches for the matched DataReader will be eventually consistent with that of the Data Writer DataWriter Cache

Topic

1 1

Last Sample Written

DataWriter1 2

DataWriter Cache

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Writing Samples



‣ The DDS ensures that the local caches for the matched DataReader will be eventually consistent with that of the Data Writer

1 2

Last Sample Written

DataWriter1 3

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Anatomy of a write

DDS::ReturnCode_t write(const T& sample, InstanceHandle_t handle);

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Writing Data

writer->write(sample, DDS::HANDLE_NIL);

InstanceHandle_t handle = writer->register_instance(sample);// ...writer->write(sample, handle);


OpenSplice DDS

Managing Data History

struct Counter { int cID; int count;};#pragma keylist Counter cID


Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


How many samples?

‣ The History QoS Controls the number of samples-per-instance that will be stored by the middleware on behalf of a Reader

‣ Keep Last K. The History QoS can be set so to always have the latest K samples

‣ Keep All. The History QoS can be set so keep all samples produced by the writer and not yet taken, until resource limits are not reached

DataReader

DataReader

1 1

2 1

3 1

History Depth = 1 (DDS Default)

History Depth = 5

Topic

1 1

2 1

3 1

1 2 1 3 1 4 1 5

2 2 2 3 2 4 2 5

3 2 3 3 3 4 3 5

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


History in Action

Note: The Reliability QoS controls wether data is sent reliably, or best-effort, from the DataWriter to matched DataReaders

Topic

DataReader

1 1

2 1

3 1


DataReader Cache

Topic

1 2

2 3

3 1


DataWriter Cache

DataWriter1 2

2 2 2 3

Network

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


History in Action

Topic

DataReader

1 2

2 2

3 1


DataReader Cache

Topic

1 2

2 3

3 1


DataWriter Cache

DataWriter2 3

Network


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


History in Action

Topic

DataReader

1 2

2 3

3 1


DataReader Cache

Topic

1 2

2 3

3 1


DataWriter Cache

DataWriter

Network


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


History in Action

Topic

DataReader

1 1

2 1

3 1

History Depth = 2

DataReader Cache

Topic

1 2

2 3

3 1


DataWriter Cache

DataWriter1 2

2 2 2 3

Network


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


History in Action

Topic

DataReader

1 1

2 1

3 1

History Depth = 2

DataReader Cache

Topic

1 2

2 3

3 1


DataWriter Cache

DataWriter

1 2

2 2

2 3

Network


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


History in Action

Topic

DataReader

1 1

2 2

3 1

History Depth = 2

DataReader Cache

Topic

1 2

2 3

3 1


DataWriter Cache

DataWriter

1 2

2 3

Network



OpenSplice DDS

Application / DDS Coordination

struct Counter { int cID; int count;};#pragma keylist Counter cID


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Application / DDS Coordination

DDS provides three main mechanism for exchanging information with the application

‣ Polling. The application polls from time to time for new data or status changes. The interval might depend on the kind of applications as well as data

‣WaitSets. The application registers a WaitSet with DDS and waits (i.e. is suspended) until one of the specified events has happened.

‣ Listeners. The application registers a listener with a specific DDS entity to be notified when relevant events occur, such as state changes or


OpenSplice DDS

Putting it All TogetherWriting a DDS Application

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Steps for Writing a DDS ApplicationWriting a DDS application can be decomposed in the following few simple steps:

‣ Step#1: Define Topics

‣ Step #2: Identify QoS representing key non-functional invariants for your system‣ Transport Priority

‣ Deadline

‣ Durability

‣ Step #3: Define Topics / Partition Mapping

‣ Step #4: Identify Topic Readers/Writers

‣ Step #5: Define QoS requirements for Readers/Writers‣ History

‣ Latency Budget

‣ Auto-Dispose

‣ Transport Priority

‣ Deadline

‣ Step #6: Code-it in your favorite programming language

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Making the Hello DDS World

Step #1: Topic definition

‣ We are going to define a simple key-less topic that will carry the name to greet.

module swatch {

struct hello { string<256> name; };#pragma keylist hello

};

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



Step #2: Topic QoS

‣ Reliability QoS: RELIABLE

‣ Durability QoS: TRANSIENT

module swatch {

struct hello { string<256> name; };#pragma keylist hello

};

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



Step #3: Topics/Partitions Mapping

‣ swatch::hello will be mapped into the default-partition (thus no action to take)

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



Step #4: Readers/Writers

‣ A Generic DataReader that will read the topic swatch:: hello ‣We’ll be able to run as many of this as we want

‣ A Generic DataWriter that will read the topic swatch:: hello‣We’ll be able to run as many of this as we want

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



‣ Step #5: Define QoS requirements for Readers/Writers

‣ Writer:‣ Inherit TopicQoS, and

‣ No-Auto Dispose

‣ History QoS: Keep Last N

‣ Reader‣ Inherit TopicQoS, and

‣ History QoS: Keep Last N

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Hello World in a Conceptual Picture

Network

Dat

aRea

der

History Depth = 2

DataReader Cache

Samples

1 1

2 2

3 1

1 2

2 3

SampleInfo

1

2

3

1

2

swatch::hello

History Depth = 2

DataWriter Cache

Samples

1 1

2 2

3 1

1 2

2 3

SampleInfo

1

2

3

1

2

swatch::hello

Dat

aWri

ter

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Visualizing the Strucutre...

PublisherSubscriber Default Partition

Domain

Domain Participant

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Step #6: Coding

How many lines of code is going to take this example?

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


SIMple Dds == SIMD!

‣ Today we’ll write our DDS application using SIMD

‣ SIMD is a C++ library that takes advantage of C++ Template Meta-Programming to:‣ Vastly Improve Productivity

‣ Simplify Usage

‣ Automate Resource Management (All DDS Entities are Garbage Collected via Ref-Counting)

‣ Zero Overhead

SIMD

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Hello-pub.cpp (Default QoS)

int main(int argc, char* argv[]) { if (!parse_args(argc, argv)) return 1;

// -- init the SIMD runtime simd::Runtime::init(); // -- create the DDS Topic simd::Topic<swatch::hello> helloTopic("helloTopic"); // -- create the DDS DataWriter simd::DataWriter<swatch::hello> writer(helloTopic, dwqos);

swatch::hello sample; std::stringstream ss; for (int i = 0; i < N; ++i) { ss << i; std::string tmp = ss.str() + "." + message; ss.str(""); sample.name = DDS::string_dup(tmp.c_str()); std::cout << "<<= " << sample.name << std::endl; writer.write(sample); usleep(period*1000); } std::cout << "[done]" << std::endl; return 0; }

Only 3-lines of DDS-Specific Code Business Logic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Hello-pub.cppint main(int argc, char* argv[]) { if (!parse_args(argc, argv)) return 1; // -- init the SIMD runtime simd::Runtime::init(); simd::TopicQos tqos; tqos.set_reliable(); tqos.set_transient();

// -- create the DDS Topic simd::Topic<swatch::hello> helloTopic("helloTopic", tqos); simd::DataWriterQos dwqos(tqos); dwqos.set_keep_last(history_depth); dwqos.set_auto_dispose(false); // -- create the DDS DataWriter simd::DataWriter<swatch::hello> writer(helloTopic, dwqos);

swatch::hello sample; std::stringstream ss; for (int i = 0; i < N; ++i) { ss << i; std::string tmp = ss.str() + "." + message; ss.str(""); sample.name = DDS::string_dup(tmp.c_str()); std::cout << "<<= " << sample.name << std::endl; writer.write(sample); usleep(period*1000); } std::cout << "[done]" << std::endl; return 0; }

Business Logic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Hello-sub.cpp (Default QoS)

int main(int argc, char* argv[]) { if (!parse_args(argc, argv)) return 1; // -- init the SIMD runtime simd::Runtime::init(); // -- create the DDS Topic simd::Topic<swatch::hello> helloTopic("helloTopic");

// -- create the DDS DataReader simd::DataReader<swatch::hello> reader(helloTopic);

swatch::helloSeq samples; DDS::SampleInfoSeq infos; while (true) { reader.read(samples, infos); for (int i = 0; i < samples.length(); ++i) { std::cout << "=>> " << samples[i].name << std::endl; } if (samples.length() > 0) std::cout << "--" << std::endl; reader.return_loan(samples, infos); usleep(period*1000); } return 0;}Only 3-lines of DDS-Specific Code

Business Logic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Hello-sub.cppint main(int argc, char* argv[]) { if (!parse_args(argc, argv)) return 1; // -- init the SIMD runtime simd::Runtime::init(); simd::TopicQos tqos; tqos.set_reliable(); tqos.set_transient();

// -- create the DDS Topic simd::Topic<swatch::hello> helloTopic("helloTopic", tqos); simd::DataReaderQos drqos(tqos); drqos.set_keep_last(history_depth);

// -- create the DDS DataReader simd::DataReader<swatch::hello> reader(helloTopic, drqos);

swatch::helloSeq samples; DDS::SampleInfoSeq infos; while (true) { reader.read(samples, infos); for (int i = 0; i < samples.length(); ++i) { std::cout << "=>> " << samples[i].name << std::endl; } if (samples.length() > 0) std::cout << "--" << std::endl; reader.return_loan(samples, infos); usleep(period*1000); } return 0;}

Business Logic

Age

nda

Motivating Forces

The Standard


Programming Model

Future Directions

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


DDS TodayDevelopment

‣ UML Profile for DDS

‣ IDL Derived API

Integration

‣ Complete knowledge of the information model

‣ Consistent information model across systems

Deployment

‣ Optimized for LAN, multi-LAN deployment

‣WAN deployments supported via:‣Multicast-enabled VPN

‣DDSI Unicast



Subscription

Minimum Profile




Application

UDP/IP


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


DSS Standard Directions

The DDS Standard will be extended in the following directions:

‣Simplicity. New, API for ISO STD C++, and Java 6

‣Extensibility and Integration. Extensible and Dynamic Topic Types

‣Scale. Support for large-scale WAN deployments

‣Data Access. Support for accessing the DDS Global Data Space via

SOA and Web Technologies (e.g. REST, WS-*, JMS, etc.)


OpenSplice DDS

Simplicity

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


ISO STD C++ PSM for DDSContext

‣ The DDS API is currently derived from the existing IDL2C++ standard

‣ The existing C++ API, although being very powerful, does not always hide accidental complexity

‣ The existing C++ API does not leverage the latest ISO STD C++ features, such as standard containers, iterators, templates, etc.

‣ The existing C++ API leaves the burden of DDS entities manual memory management to the programmer

Upcoming Standard

‣ Fully aligned with the ISO STD C++ Standard and also looking forward to the upcoming C++0x

‣ Simpler, neater, safer API

‣ Automatic Memory Management (still debated)

Status

‣ Final joint revised submission due for the Dec 2009 OMG Meeting


OpenSplice DDS

Extensibility

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Extensible and Dynamic Topic TypesContext

‣ The OMG DDS standard has been designed to effectively support statically defined data models.

‣ This assumption ensures that topics data types are known at compile time, that every member of the DDS global data space agrees precisely on the same topic-type association

‣ Extensions are captured by either DLRL classes inheritance, or by IS-A relations at the DCPS layer

‣ Under this model:‣ it is hard to cope with data models evolving over time unless all the elements of the system

are upgraded consistently.

‣ It is hard to deal with dynamically defined topics, although in some case this is achievable by means of proprietary APIs.

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Extensible and Dynamic Topic TypesUpcoming Standard: Four dimension considered

‣ Topic’s Type Definition

‣Available syntax for defining the types associated with a topic

‣Currently IDL only is supported

‣Support for sparse types as well as for some pre-defined topic types

‣ Encapsulation Definition

‣Controls how a topic is encapsulated on the wire

‣Currently CDR is specified

‣ Extensibility Mechanism

‣Controls how Topics Types can be extended without breaking interoperability w.r.t. portion of the system relying on older version of the same topic

‣ Dynamic API

‣An API which allows to publish and subscribe topics for a Topic that is not known a priory

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Type Extensibility & Evolution‣ Type evolution captures the changes that a type experience in a system throughout its

life-time

‣ Type extension capture, via sub-typing, refinement and specialization.

‣NOTE: Often Type Extension is (ab)used in order to capture type evolution, but this is an “trick” more than a good practice

struct RadarTrack { long trackID; long x; long y;};#pragma keylist RadarTrack trackID

Type Evolution

Type Extension

struct RadarTrack { long trackID; long x; long y; float estErr;};#pragma keylist RadarTrack trackID

struct RadarTrack3D : RadarTrack { long z;};#pragma keylist RadarTrack3D RadarTrack3D::RadarTrack::trackID

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Extensible and Dynamic Topic Types

Status

‣ Joint Revised Submission available on OMG WWW

‣ Joint Revision Submission due on Sep 2009


OpenSplice DDS

Scale

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


DDS for Ultra-Large-Scale SystemsContext

‣ The DDS Standard covers very well the “System” Use Case

‣ Natural extension point exist in the DDS Standard properly support Ultra-Large-Scale Systems, over WAN

‣ Further standardization is required to ensure that the nice DDS properties enjoyed on a “System” are carried on in a “Ultra-Large-Scale System” w/o limiting interoperability or trading it for scalability

Upcoming Standard

‣ Ultra-Large-Scale extensions for DDS

Status

‣ 2 Concept paper submitted by leading vendors to the OMG RTWS ’09

‣ RFP process to be kick-started at the Sept ’09 OMG meeting

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


DDS for Ultra-Large-Scale Systems

‣ DDS-Level Routers will be introduced that establish a Peer-to-Peer overlay that relays DDS data over the WAN

‣ A Network of Super-Nodes is used for ensuring:‣ Scalable (Global) Discovery

‣ Subscription Management (when crossing the boundaries of a System)

‣ Support for establishing communication between Pub and Sub (perhaps via STUNT to circumvent NAT/FIREWALL issues)

N1,0

N1,1

N2,0

N2,1

N2,2 N

2,3

N2,4

N2,5

H1,0,0

H1,0,1

H1,0,k

H1,1,h

H1,1,0

H1,1,1

H2,0,i

H 2,1,j

H 2,1,k H2,1,h

H 2,1,n

H 2,1,m

R

R

R

R

R

R

R

R

SN

SN

SN

SN

SN


OpenSplice DDS

Data Access

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


RESTful DDSContext

‣ REST is becoming the dominant way of accessing resources via the internet and a new style of implementing Web Services, supported by the latest Web2.0 applications, e.g. Twitter, Blogspot, etc.

‣ REST provides basic CRUD functionalities that nicely map to DDS

‣ REST’s resource-based approach nicely maps to DDS Data-based approach, since a Topic nicely maps into a resource

‣ A RESTful access to the DDS global data space is natural and enables to naturally address Topics, e.g.:‣ http://buket.mywww.com/system/domain/partition/topic/

Upcoming Standard

‣ RESTful DDS

Status

‣ Initial RFP Drafting

http://buket.mywww.com/ship/domain/partition/topic

http://buket.mywww.com/ship/domain/partition/topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Concluding RemarksState of the Art

‣ The OMG DDS Standard uniquely addresses some of the key requirements for next generation Mission Critical Systems

‣ The OMG DDS is one of the few middleware standard defining both API as well as Wire-Protocol, thus ensuring application portability and interworking

‣ The DDS Standard has been recommended/mandated by key administration throughout the world as well as adopted by the most challenging Defense programs

Looking Ahead

‣ The DDS Standard is swiftly moving forward to expand its applicability, simplify its use, and facilitate integration



Subscription

Minimum Profile




Application

UDP/IP


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.

Online Resources

http://www.opensplice.com/

emailto:[email protected]

http://www.youtube.com/OpenSpliceTube

http://opensplice.blogspot.com http://bit.ly/1Sreg

http://www.dds-forum.org

http://portals.omg.org/dds


http://twitter.com/acorsaro/







http://opensplice.blogspot.com

http://opensplice.blogspot.com

http://bit.ly/1Sreg

http://bit.ly/1Sreg

http://www.prismtech.com/opensplice-dds

http://www.prismtech.com/opensplice-dds





Date post:	13-Nov-2014
Category:	Technology
Upload:	angelo-corsaro
View:	5,980 times
Download:	4 times