DDS Tutorial -- Part I

OpenSplice | DDS Tutorial-- Part I --

Angelo CORSARO, Ph.D.Chief Technology Officer OMG DDS Sig Co-Chair

[email protected]

Grasping the Idea

DDS is a standard technology for ubiquitous, interoperable, secure, platform independent, and

real-time data sharing across network connected devices

Copyrig

ht 2013, PrismTech – A

ll Rights Reserved.

Data Distribution Service (DDS)

• DDS provides a Global Data Space abstraction that allow applications to autonomously, anonymously, securely and efficiently share data

• DDS’ Global Data Space is fully distributed, highly efficient and scalable

DDS Global Data Space

...

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

Copyrig


ll Rights Reserved.

Data Distribution Service (DDS)

• DataWriters and DataReaders are automatically and dynamically matched by the DDS Discovery

• A rich set of QoS allows to control existential, temporal, and spatial properties of data


...

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

Copyrig


ll Rights Reserved.

Fully Distributed Data Space

Conceptual Model Actual Implementation

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

TopicDQoS

TopicDQoS

TopicAQoS


...

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

Copyrig


ll Rights Reserved.

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

TopicDQoS

TopicDQoS

TopicAQoS

Fully Distributed Data SpaceThe communication between the DataWriter and the DataReader can use UDP/IP (Unicast and Multicast)or TCP/IP

Copyrig


ll Rights Reserved.

Key Highlights

• Elegant and High Level Data Sharing Abstraction

• Polyglot and platform independent

• Java, Scala, C, C++, C#, JavaScript, CoffeeScript etc.

• Android, Windows, Linux, VxWorks, etc.

• Peer-to-Peer by nature, Brokered when useful

Copyrig


ll Rights Reserved.

Key Highlights

• Content and Temporal Filtering (both sender and receiver filtering supported)

• Queries

• 20+ QoS to control existential, temporal, and spatial properties of data

Copyrig


ll Rights Reserved.

Key Highlights

• High Performance and Scalable

• ~35-50 usec latency*

• 7M msgs/sec node-to-node throughput*

(*) Performance measured on Linux Host with i7 processor on a 1Gbps Ethernet network

Who is Using DDS?

Copyrig


ll Rights Reserved.

Use Cases

Integrated Modular Vetronics Training & Simulation Systems Naval Combat Systems

Air Traffic Control & Management Unmanned Air Vehicles Aerospace Applications

Copyrig


ll Rights Reserved.

Use Cases

Agricultural Vehicle Systems

Train Control Systems Complex Medical Devices

Smart CitiesLarge Scale SCADA Systems

High Frequency Auto-Trading

Decomposing DDS

Information Organization

Copyrig


ll Rights Reserved.

Domain

• DDS data lives within a domain

• A domain is identified with a non negative integer, such as 1, 3, 31

• The number 0 identifies the default domain

• A domain represent an impassable communication plane

DDS Domain

Copyrig


ll Rights Reserved.

Partitions

• Partitions are the mechanism provided by DDS to organize information within a domain

• Access to partitions is controlled through QoS Policies

• Partitions are defined as strings:

• “system:telemetry”

• “system:log”

• “data:row-‐2:col-‐3”

• Partitions addressed by name or regular expressions:

• ”system:telemetry”

• “data:row-‐2:col-‐*”

Partitions

Information Definition

Copyrig


ll Rights Reserved.

Topic• A Topic defines a domain-wide information’s class

• A Topic is defined by means of a (name, type, qos) tuple, where

• name: identifies the topic within the domain

• type: is the programming language type associated with the topic. Types are extensible and evolvable

• qos: is a collection of policies that express the non-functional properties of this topic, e.g. reliability, persistence, etc.

TopicType

Name

QoS

...

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

...

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

Copyrig


ll Rights Reserved.

Topic and Instances

• As explained in the previous slide a topic defines a class/type of information

• Topics can be defined as Singleton or can have multiple Instances

• Topic Instances are identified by means of the topic key

• A Topic Key is identified by a tuple of attributes -- like in databases

• Remarks:

• A Singleton topic has a single domain-wide instance

• A “regular” Topic can have as many instances as the number of different key values, e.g., if the key is an 8-bit character then the topic can have 256 different instances

Example

Copyrig


ll Rights Reserved.

Active Floor

• Assume we are building an active floor

• This active floor is made by a matrix of pressure sensors used to detects position, and indirectly movement

• This information is then used by the application that uses the active floor for positioning or entertainment

Cell: (i,j)

Copyrig


ll Rights Reserved.

Active Floor

• The generic active cell can be modeled with a topic that has an instance for each value of (i,j). The topic type can be defined as:

• Each cell is now distinguishable and associated with a topic instance

Cell: (i,j)

struct TCell { short row; short column; float pressure; // in kPa};#pragma keylist TCell row column

Copyrig


ll Rights Reserved.

Active Floor

• How can we know when something is on the cell?

• The detection can be based on the difference between the atmospheric pressure, say P0, and the pressure sensed by the cell

• We can model this as a Singleton Topic ReferencePressure defined by the type:

Cell: (i,j)

struct TReferencePressure { float pressure; // in kPa float precision; };#pragma keylist TReferencePressure

Copyrig


ll Rights Reserved.

Active Floor• Each sensor has associated a topic

instance identified by the (row,column) coordinate -- the instance key

• Each instance produces a stream of pressure values that in DDS terms are called samples

0 1 2 3

0

1

2

3

4

Pressure time

Pressure time

Pressure time

struct TCell { short row; short column; float pressure; // in kPa};#pragma keylist Cell row column

Going 3D

Copyrig


ll Rights Reserved.

Active Floors on a Building

• Let’s assume now that we have a building that uses active floors to detect presence and movement

• How can we organize the out data model?

01

23

0

1

2

3

4

Pressure

time

Pressure

time

Pressure

time

Copyrig


ll Rights Reserved.


• The first thing to do is to introduce the third dimension to our cell:

struct TCell { short row; short column; short floor; float pressure; // in kPa};#pragma keylist TCell row column floor

01

23

0

1

2

3

4

Pressure

time

Pressure

time

Pressure

time

Copyrig


ll Rights Reserved.


• As we move from a single floor to a building we need to add some more structure to our data

• We can now use:

• A Domain for each Building

• A Partition for each Floor

• A Partition for reference value, i.e. Partition

• A Partition for the configuration information, e.g. how many floors, how many rows/cols per floor

01

23

0

1

2

3

4

Pressure

time

Pressure

time

Pressure

time

Copyrig


ll Rights Reserved.


• Thus the resulting structure is:

• Floor Partitions:

• “building:f-‐1”

• “building:f-‐2”

• ...

• Reference Values Partition:• “building:refvals”

• Configuration Partition:• “building:config”

01

23

0

1

2

3

4

Pressure

time

Pressure

time

Pressure

time

Producing Information

Copyrig


ll Rights Reserved.

DataWriter

• A DataWriter (DW) is a strongly typed entity used to produce samples for one or more instances of a Topic, with a given QoS

• Conceptually, the DataWriter QoS should be the same as the Topic QoS or more stringent

• However, DDS does enforce a specific relationship between the Topic and DataWriter QoS

DWTypeTopic

QoS

Copyrig


ll Rights Reserved.

DataWriter

• The DataWriter controls the life-cycle of Topic Instances and allows to:

• Define a new topic instance

• Write samples for a topic instance

• Dispose the topic instance

DWTypeTopic

QoS

Consuming Information

Copyrig


ll Rights Reserved.

DataReader

• A DataReader (DR) is a strongly typed entity used to access and/or consume samples for a Topic, with a given QoS

• Conceptually, the DataReader QoS should be the same as the Topic QoS or less stringent

• However, DDS does enforce a specific relationship between the Topic and DataReader QoS

DRTypeTopic

QoS

Copyrig


ll Rights Reserved.

DataReader

• Depending on its QoS a DataReader may provide access to:

• last sample

• last n samples

• all samples produced since the DataReader was created

0 1 2 3

0

1

2

3

4

Pressure time

Pressure time

Pressure time

Copyrig


ll Rights Reserved.

DataReader


• last sample

• last n samples


0 1 2 3

0

1

2

3

4

Pressure time

Pressure time

Pressure time

n=3

Copyrig


ll Rights Reserved.

DataReader


• last sample

• last n samples


0 1 2 3

0

1

2

3

4

Pressure time

Pressure time

Pressure time

n=3

Copyrig


ll Rights Reserved.

DataReader


• last sample

• last n samples


0 1 2 3

0

1

2

3

4

Pressure time

Pressure time

Pressure time

n=3

Copyrig


ll Rights Reserved.

DataReader


• last sample

• last n samples


0 1 2 3

0

1

2

3

4

Pressure time

Pressure time

Pressure time

n=3

Copyrig


ll Rights Reserved.

DataReader


• last sample

• last n samples


0 1 2 3

0

1

2

3

4

Pressure time

Pressure time

Pressure time

n=3

Copyrig


ll Rights Reserved.

DataReader


• last sample

• last n samples


0 1 2 3

0

1

2

3

4

Pressure time

Pressure time

Pressure time

n=3

Copyrig


ll Rights Reserved.

DataReader


• last sample

• last n samples


0 1 2 3

0

1

2

3

4

Pressure time

Pressure time

Pressure time

n=3

Copyrig


ll Rights Reserved.

DataReader• Samples are stored in the DataReader

Cache

• Samples can be read or taken from the cache

• Samples taken are evicted from the cache

• Samples read remain in the cache and are simply market as read

• The cache content can be selected based on content or state. More on this later...

0 1 2 3

0

1

2

3

4

Pressure time

Pressure time

Pressure time

Putting all Together

Copyrig


ll Rights Reserved.

DDS Entities

• DomainParticipant: Provides access to a data cloud -- called a domain in DDS

• Topic: Domain-wide definition of a kind of Information

• Publisher/Subscriber: Provide scope to data sharing through the concept of partitions

• DataReader/DataWriter: Allow to read/write data for a given topic in the partitions their Subscriber/Publisher are associated with.

Domain (e.g. Domain 123)

Domain Participant

Topic

Publisher

DataWrter

Subscriber

DataReader

Partition (e.g. “Telemetry”, “Shapes”, )

Topic Instances/Samples

TaTb

Tc

Tx

Ty

T1

T1 T3

Copyrig


ll Rights Reserved.

DomainParticipant

• The DomainParticipant is the programming entity that gives access to a DDS domain

• A DomainParticipant is created as follows:

// ISO C++ DDS APIint domain_id = 18;auto dp = DomainParticipant(domain_id);

// Java 5 DDS APIint domain_id = 18;DomainParticipantFactory dpf = DomainParticipantFactory.getInstance(env)

DomainParticipant dp = dpf.createParticipant(domainId);

Copyrig


ll Rights Reserved.

Topic

• Given a DomainParticipant we can define (or discover) Topics within a domain. This can be done as follows:

• As this declaration does not explicitly provide QoS for the Topic, the default QoS will be used

// ISO C++ DDS APIauto topic = Topic<TCell>(dp, “Cell”);

// Java 5 DDS APITopic<TCell> topic = dp.createTopic(“Cell”, TCell.class);

Copyrig


ll Rights Reserved.

Publisher/Subscriber

• Publisher/Subscriber, through the Partitions they are associated with, define the scope of a write/read operation

• Partitions association is done through the Partition QoS Policy

• This association can be defined as a list of string as well as a list of regular expressions

Copyrig


ll Rights Reserved.


• Definition of a Publisher/Subscriber in the default partition:

• Definition of a Publisher/Subscriber with Partition settings

// ISO C++ DDS APIauto pub = Publisher(dp);

// ISO C++ DDS APIauto pub_qos = dp.default_publisher_qos() << Partition(”af:telemetry”);auto pub = Publisher(dp, pub_qos);

// ISO C++ DDS APIauto sub = Subscriber(dp);

// ISO C++ DDS APIauto sub_qos = dp.default_subscriber_qos() << Partition(”af:telemetry”);auto sub = Subscriber(dp, sub_qos);

Copyrig


ll Rights Reserved.


• Definition of a Publisher/Subscriber in the default partition:

• Definition of a Publisher/Subscriber with Partition settings

// Java 5 DDS APIPublisher pub = dp.createPublisher();

// Java 5 DDS APIPublisherQos pubQoS = dp.getDefaultPublisherQos() .with(pf.Partition(”af:telemetry”));

Publisher pub = dp.createPublisher(pubQoS);

// Java 5 DDS APISubscriber sub = dp.createSubscriber();

// Java 5 DDS APISubscriberQos subQoS = dp.getDefaultSubscriberQos() .with(pf.Partition(”af:telemetry”));

Subscriber sub = dp.createSubscriber(subQoS);

Copyrig


ll Rights Reserved.

DataWriter

• A DataWriter with default QoS can be declared as follows:

// ISO C++ DDS APIauto dw = DataWriter<TCell>(pub, topic);// Write the cell c(1,1) using `writer`TCell c11 = {1, 1, 15};dw.write(c11);// Write the cell c(1,2) using the `operator <<`TCell c12 = {1, 2, 5};dw << c12;

// Java 5 DDS APIDataWriter<TCell> dw = pub.createDataWriter<TCell>(topic);TCell c11 = new TCell(1, 2, 15);dw.write(c11);

Copyrig


ll Rights Reserved.

DataReader

• A DataReader with default QoS can be declared as follows:

// ISO C++ DDS APIauto dr = DataReader<TCell>(sub, topic);// Read Samplesauto samples = dr.read();// Do something with itstd::for_each(samples.begin(), samples.end(), do_something);

// Java 5 DDS APIDataReader<TCell> dr = sub.createDataReader<TCell>(topic);

Does DDS “JavaScripts”?

Copyrig


ll Rights Reserved.

Topic

topic = new dds.Topic(0, 'Cell', 'com.activefloor.Cell')

var myTopic = new dds.Topic(domainID, topicName, topicType);

myTopic = new dds.Topic(domainID, topicName, topicType)

JavaScript

CoffeeScript

var topic = new dds.Topic(0, 'Cell', 'com.activefloor.Cell');

Example:

Example:

Copyrig


ll Rights Reserved.

DataWriter

dw = new dds.DataWriter(topic)

cell = {}cell.x = 3cell.y = 5cell.pressure = 3

dw.write(cell)

dw = new dds.DataWriter(topic, qos)

CoffeeScript

Example:

Copyrig


ll Rights Reserved.

DataReader

dr = new dds.DataReader(topic)

dr = new dds.DataReader(topic, qos)

CoffeeScript

Example:

Copyrig


ll Rights Reserved.

Binding a DataReader

• A DataReader can be bound to a user provided function that will handle incoming data or to a cache

• Notice, that as you are in control of how data-readers are bound to cache you can be very creative

Copyrig


ll Rights Reserved.

Binding to User Function

dr.addListener((s) -> console.log(JSON.stringify(s)))

dr.addListener(f)

CoffeeScript

Example:

Copyrig


ll Rights Reserved.

Binding to a Cache

// BindingbindCell = dds.bind((s) -> s.x + “-” + s.y)ccache = new DataCache(historyDepth)bindCell(cache, cdr)

// Working with the Cache: Compute number of active cells

activeCells = ccache.map((c) -> if (c > p0) then 1 else 0).fold(0)((a, c) -> a + c)

cache = new DataCache(historyDepth)bind(keyMapper)(dr, cache)

CoffeeScript

Example:

Demo

Copyrig


ll Rights Reserved.

Shapes Application

• The iShapes application is used by DDS vendors to demonstrate some of the basic mechanism as well as product interoperability

• Three Topics

• Circle, Square, Triangle

• One Type

struct ShapeType { string color; long x; long y; long shapesize;};#pragma keylist ShapeType color

Spotted shapes represent subscriptions

Pierced shapes represent publications

Copyrig


ll Rights Reserved.

Android Shapes

Data Modeling

Copyrig


ll Rights Reserved.

Data Centricity

• DDS is Data Centric, it provides an elegant, efficient and scalable data sharing abstraction

• When designing the architecture of a DDS-based system the Data-Model is the first thing that should be devised

• One of the most common question from DDS novices is how to come up with a good data model. DDS Global Data Space

...

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

Copyrig


ll Rights Reserved.

Data Modeling in DDS

• A good starting point for identifying DDS topics is to describe your system in a natural language and then use the following table:

English Language DDS

Common Noun Topic

Proper Noun Topic Instance

Transitive Verb Relationship

Intransitive Verb Attribute Type

Adjective Topic Attribute

Adverb Relationship Attribute

Copyrig


ll Rights Reserved.

Relationships

• As in DBMS, relationships can be expressed, depending on cardinality and relation attributes, by:

• Keys and foreign keys

• Relationship Topics

Copyrig


ll Rights Reserved.

Normalization

• As in DBMS, Topics should be normalized to improve modularity, efficiency and limit if not avoid anomalies. Typically you should have your Topics in the 3NF

• Notice that the nature of DDS may induce further data normalization to deal with conflicting QoS requirements, such as:

• Mix of frequently and infrequently changing data

• Mix of attributes that have different persistence requirements

Part I Summary & Remarks

Copyrig


ll Rights Reserved.

Summary & Remarks

• DDS provides an elegant Data Space abstraction that allows application to share data ubiquitously and efficiently -- the Data Space implementation is fully distributed

• DDS provides primitives for defining classes of information as well as organizing it

• Data Modeling and normalization draws heavily from DBMS theory and practice with some additional considerations w.r.t. the nature of DDS

¥@prismtech

¥@acorsaro

¥youtube.com/opensplicetube ¥slideshare.net/angelo.corsaro

¥ [email protected]

¥[email protected]

¥opensplice.com ¥forums.opensplice.org

¥opensplice.org ¥[email protected]

:: Connect with Us ::

http://www.youtube.com/opensplicetube

http://www.youtube.com/opensplicetube

http://www.slideshare.net/angelo.corsaro

mailto:[email protected]




http://www.opensplice.com




http://forums.opensplice.org/

http://forums.opensplice.org/

http://www.opensplice.org








Date post:	15-Jan-2015
Category:	Technology
Upload:	angelo-corsaro
View:	849 times
Download:	1 times