Date post: | 10-May-2015 |
Category: |
Technology |
Upload: | angelo-corsaro |
View: | 4,106 times |
Download: | 1 times |
Angelo Corsaro, Ph.D. Chief Technology Officer!OMG DDS Sig Co-Chair PrismTech [email protected]!
The Present and Future "of DDS
The Present
The DDS Standard
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Data Distribution for Real-Time Systems
¨ Introduced in 2004 to address the Data Distribution challenges faced by a wide class of Defense and Aerospace Applications
¨ Key requirement for the standard were to deliver very high, predictable, performance while scaling from embedded to ultra-large-scale deployments
¨ Recommended by key administration worldwide
¨ Widely adopted across several different domains, e.g., Automated Trading, Simulations, SCADA, Telemetry, etc.
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
DDS Standards Family
DDS
App
2004
Standard API
§ Standard API for Data-
Centric Real-Time Pub/Sub § Automatic fail-over § Persistence § Dynamic Discovery § Content Filtering / Queries
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
DDS Standards Family
DDS
App Standard API
DDSI/RTPS network
Interoperable Wire Protocol
2004
2006
DDS
App
DDSI/RTPS
2004
2006
§ High Performance § Interoperable Wire Protocol
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
DDS Standards Family
DDS
App Standard API
DDSI/RTPS network
Interoperable Wire Protocol
UML4DDS
2004
2006
2008
DDS
App
DDSI/RTPS
2004
2006
UML4DDS 2008
§ UML-Based Modeling of DDS applications
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
DDS Standards Family
DDS
App Standard API
DDSI/RTPS network
Interoperable Wire Protocol
UML4DDS
2004
2006
2008
X-T
ype
s 20
10
DDS
App
DDSI/RTPS
UML4DDS
2004
2006
2008
X-T
ype
s 20
10
§ Extensible/Evolvable Type System
§ Dynamic Topic Types, Data Readers & Writers
§ Encoding Negotiation
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
DDS Standards Family
DDS
App Standard API
DDSI/RTPS network
Interoperable Wire Protocol
UML4DDS
ISO-C++ PSM 2010 2004
2006
2008
X-T
ype
s 20
10
DDS
App
DDSI/RTPS
UML4DDS
2004
2006
2008
X-T
ype
s 20
10
§ ISO C++ DDS PSM § Simple, Safe, Efficient,
Elegant and Ergonomic API
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
DDS Standards Family
DDS
App Standard API
DDSI/RTPS network
Interoperable Wire Protocol
UML4DDS
ISO-C++ PSM 2010
Java5 PSM 2010 2004
2006
2008
X-T
ype
s 20
10
DDS
App
DDSI/RTPS
UML4DDS
2004
2006
2008
X-T
ype
s 20
10
§ Java 5 DDS PSM § Simple, Safe, Efficient,
Elegant and Ergonomic API
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
OMG DDS + DDSI/RTPS ¨ Language Independent, OS
and HW architecture independent
¨ Standard API Topic-Based, Real-Time Publish/Subscribe
¨ Built-in support for fail-over, filtering, query, data durability
¨ Standard wire protocol allowing interoperability between different implementations of the DDS standard
Object/Relational Mapping
Ownership DurabilityContent
Subscription
Minimum Profile
Data Centric Publish/Subscribe (DCPS)
Data Local Reconstruction Layer (DLRL)
DDS Interoperability Wire Protocol
Application
UDP/IP
Real-Time Publish/Subscribe Protocol
How does it work?
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
What is Data Distribution?
¨ Data distribution is about making application defined data available where needed and when needed
¨ Data is a first class concept, it can be created, updated, read and eventually disposed
¨ The last value (or last N-values) of a Data is available
Global Data Space
[1/2]
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
What is Data Distribution?
¨ The details of how application data is encapsulated and propagated are dealt by the data distribution technology
[2/2]
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
DDS Topics ¨ The definition of Data in DDS is
tied to the definition of a Topic
¨ A Topic is the association between: ¨ Unique Name
¨ Type
¨ QoS
¨ A Topic can have associated a key in order to express data instances identities
TopicType QoS
Name
com.myco.VehiclePositionTopic!
com::myco::VehiclePosition! {Reliability, ! Deadline, ! Priority, ! Durability} !
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Topic Types ¨ Topic types can be defined
in IDL, XML, Java
¨ Topics types are extensible and evolvable
¨ Rich set of annotations to deal with fields that might be optional, shared, etc.
¨ Each unique key value identifies a Topic Instance
¨ Topic Instance, can be Created, Read, Updated, and Disposed (CRUD)
struct VehiclePosition{ ! string plate; //@Key ! long x; ! longy; !} !
VehiclePositionVehiclePositionVehiclePosition
plate
“A123”
“B456”
“C789”
x y
101 202
303 202
101 606
TrakCo.VehiclePositionTopic
Topic Type
Topic Instances
Topic Key
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Topic Instances and Samples ¨ An update for a Topic Instance is called Sample
¨ DDS can maintain a configurable history of samples per Topic Instance
DDS in Action Demo#1
DDS X-Types
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
The Type System Debate ¨ The debate around type Programming Languages Type
Systems has animated computer scientist over several decades and is far from being resolved
¨ New Programming Languages are continuing to appear on both sides of the fence
F#Statically Typed (ex.) Dynamically Typed (ex.)
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Are you a Type-Phobic Type?
¨ Proponents of Dynamically Typed Programming Languages advocate against strong typing in favor of reduced verbosity and added flexibility
Yet... Are they so sure that a type system always gets between you and what you are trying to achieve?
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Strongly Typed Type Systems
¨ Well designed type-systems don’t add unnecessary verbosity to your application since use sophisticated inference to derive types
¨ Allow the detection of many errors at compile time, thus improving productivity, safety, code quality and reducing the potential for run-time errors
¨ Enable the generation of more efficient code
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Types and Distributed Systems ¨ The “Type System debate” has not spared distributed systems
¨ Today we can find different technologies taking different approaches: from completely untyped to strongly typed
¨ The differences are in this case more complex to organize: ¨ Some technologies support only one type, being a string or an array
of bytes (e.g. AMQP) ¨ Other technologies support some predefined types (e.g JMS) ¨ Other technologies allow the definition of user-specified types (e.g
DDS)
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Types and Distributed Systems T1
T2
T3
T4
T1
T3
T3
T2
T1
T4
T1
T2
?
T1 T2 T3 T4
User Defined Types Example: T1 = RadarTrack T3 = FlightPlanT2 = ClassificationInfo T4 = ...
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Types and Distributed Systems
T2
T3
T4
T1
T3
T2
T1
T4
T1
T2
T1T3
AMQP
octect sequence
T1 T2 T3 T4
User Defined Types
Pub/Sub Types
Example: T1 = RadarTrack T3 = FlightPlanT2 = ClassificationInfo T4 = ...
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Types and Distributed Systems
T2
T3
T4
T1
T3
T2
T1
T4
T1
T2
T1T3
JMS
ObjectMessage TextMessage
ByteMessage
StreamMessage
MapMessage
Pub/Sub Types
Example: T1 = RadarTrack T3 = FlightPlanT2 = ClassificationInfo T4 = ...
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Types and Distributed Systems
T2
T3
T4
T1
T3
T2
T1
T4
T1
T2
T1T3
DDS
T2 T3
T4
T1
T3
T1
Pub/Sub Types == User Types!
Example: T1 = RadarTrack T3 = FlightPlanT2 = ClassificationInfo T4 = ...
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
“Pre X-Types” DDS Type System
¨ Nominal Type-System w/o sub-typing
¨ Designed with the Closed-World Assumption in mind, e.g., the information model is global and immutable
¨ Changes to the information model are either non-transparent or non-interoperable with legacy, e.g. require compile/deploy cyce
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Example ¨ In pre X-Types DDS systems the
RadarTrack and RadarTrack3D cannot be associated with the same Topic by different DDS entities
¨ Nor it is possible for Readers expecting RadarTrack types to match Writers producing RadarTrack3D
struct RadarTrack { ! string id; //@Key ! long x; ! longy; !} !
struct RadarTrack3D { ! string id; //@Key ! long x; ! longy; ! long z; !} !
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
DDS X-Types Type System ¨ Structural Type System
¨ Support extensible and evolvable types
¨ A structural sub-type relationship is used to match DataReader and DataWriters
¨ Annotations + QoS Policy provide control on the sub-typing rule
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Example ¨ Under the X-Types type-system:
¨ RadarTrack3D <: RadarTrack
¨ As a result Subscribers to RadarTrack match Publishers of RadartTrack3D
¨ How about the dual case? ¨ QoS allow to control whether type
widening are allowed and in provide control on widened attributes have to be initialized
struct RadarTrack { ! string id; //@Key ! long x; ! long y; !} !
struct RadarTrack3D { ! string id; //@Key ! long x; ! long z; ! long y; !} !
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
More on X-Types
The DDS X-Types standard also provides
¨ An API for dynamically registering topic types and creating DataReader/Writer for these types
¨ A rich set of annotations for declaring shared and optional attributes, etc.
X-Types Efficiency
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Encoding Efficiency Size for Serialized CoFlight Flight Data Plan
DDS encoding is 10x more space efficient than XML!
¨ DDS serialization is several times more efficient than JSON, XML and YAML
¨ Differences can be quite extreme when dealing with complex types
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Encoding Impact on RTT DDS RTT with different Serialization Format
¨ Inefficient encoding can easily dominate RTT
¨ DDS native encoding is the most efficient among extensible encodings!
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
DDS is very Efficient!
Size (bytes)L
ate
ncy
(u
sec)
¨ DDS implementations are capable of delivering very low and predictable latencies
¨ No other Pub/Sub technology can approach similar level of performance!
ISO C++ PSM
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
PIM/PSM Approach
¨ The DDS standard is composed by a PIM specified in UML and A PSM specified in IDL
¨ Language specific APIs are derived via IDL-based mappings (e.g. IDL2C++, etc.)
PIM (UML)
PSM (IDL)
C JavaC++
IDL2CIDL2C++ IDL2Java
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
What is the Problem?
¨ IDL is very good for describing DDS Topic Types, yet...
¨ IDL’s biggest strength, namely (programming) language independence, becomes its biggest weakness when trying to define APIs that are well integrated with a programming language
¨ For some programming languages (e.g. C++) the IDL2C++ mapping is “seasoned”
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
What is the Problem?
¨ Resulting DDS APIs are more complicated than they should
¨ API don’t feel natural or ergonomic to programmers
¨ API (esp. for C++) don’t integrate well with standard libraries nor implements common idioms/patterns.
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Native C++/Java PSM ¨ The new C++/Java PSM are
derived starting directly from the PIM
¨ The new C++/Java PSM take advantage of all the features available in C++/Java to provide a natural, productive, efficient, safe and portable API
PIM (UML)
ISO C++ PSM Java 5 PSM
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
ISO C++ PSM in a Nutshell ¨ Provides transparent, safe, precise and real-time automatic memory
management
¨ Takes advantage of C++ Template Meta-programming to automate tasks such as type registration and provide a strongly typed API (no downcast ever!)
¨ Provides a DDS API that exploits Iterators and containers as well as other C++ Standard Types
¨ API is extensible, efficient and portable
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Type Mapping
¨ DDS primitive types are mapped to C99/C++ standard types
¨ DDS container types are mapped to standard C++ containers, namely std::vector and std::map
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Topic Type Mapping
struct VehiclePosition {! string plate;! long x;! long y;};#pragma keylist VehiclePosition plate
IDL Topic Type Definition
class VehiclePosition {
! VehiclePosition();! VehiclePosition(const std::string& plate,! ! ! int32_t x, int32_t y);
! virtual ~VehiclePosition();
! const std::string& plate() const;! void plate(const std::string& s);
! const int32_t x() const;! void x(int32_t i);
! const int32_t y() const;! void y(int32_t i);
private:! // Encapsulated state representation};
C++ Topic Type Mapping
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Writing Samples // ================== DataWriter ===================try { DomainId id = 0; DomainParticipant dp = TheParticipantFactory().create_participant(id);
pub::qos::PublisherQos pqos; pqos << policy::Partition("Tracks");
pub::Publisher pub = dp.create_publisher(pqos);
topic::qos::TopicQos tqos; tqos << policy::Reliability::Reliable() << policy::Durability::Transient() << policy::History::KeepLast(10) << policy::TransportPriority(14);
dds::topic::Topic<RadarTrack> topic = dp.create_topic<RadarTrack>("TrackTopic", "RadarTrack", tqos);
pub::qos::DataWriterQos dwqos(tqos);
pub::DataWriter<RadarTrack> dw = pub.create_datawriter(topic, dwqos);
RadarTrack track("alpha", 100, 200);
dw.write(track); // or dw << track; } catch (const dds::core::Exception& e) { }
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Reading Samples // ================== DataReader===================
try { DomainId id = 0; DomainParticipant dp = TheParticipantFactory().create_participant(id);
sub::qos::SubscriberQos sqos; sqos << policy::Partition("Tracks");
sub::Subscriber sub = dp.create_subscriber(sqos);
topic::qos::TopicQos tqos; tqos << policy::Reliability::Reliable() << policy::Durability::Transient() << policy::History::KeepLast(10) << policy::TransportPriority(14);
dds::topic::Topic<RadarTrack> topic = dp.create_topic<RadarTrack>("TrackTopic", "RadarTrack", tqos);
sub::qos::DataReaderQos dwqos(tqos);
sub::DataReader<RadarTrack> dr = sub.create_datareader(topic, drqos); std::vector<RadarTrack> samples(MY_MAX_LEN); std::vector<SampleInfo> info(MY_MAX_LEN); dr.read(samples.begin(), info.begin(), MY_MAX_LEN);
} catch (const dds::core::Exception& e) { }
OpenSplice DDS
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Modern History 2008 • Gartner selects DDS as a Cool Technology and Prismtech/OpenSplice
DDS as the cool vendor among DDS provider
2009 • To propel and accelerate the adoption of DDS beyond Aerospace and
Defense PrismTech Open Sources OpenSplice DDS
2010 • With over 3000 new users in the last year OpenSplice DDS establishes
itself as the most widely used and deployed DDS implementation
OpenSplice DDS#1
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Product Adoption
¨ ~20000 Unique downloads since October 2009
¨ ~3000 Unique OpenSplice DDS Users
0
5000
10000
15000
20000
Dow
nloa
ds
OpenSplice Downloads
0
750
1500
2250
3000
OpenSplice DDS Registered Users
Oct 2010Oct 2009
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Product Organization
¨ Four different editions with growing sets of functionalities
¨ The Community Edition is Open Source. Compact, Professional and Enterprise Edition are available through Commercial Subscriptions / Licensing
Enterprise Edition
Professional Edition
Compact Edition
Community Edition
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Community Edition
¨ OMG DDS v1.2
¨ Real-Time Networking
¨ DDSI/RTPS v2.1
Licensing ¨ LGPLv3
Ownership DurabilityContent
Subscription
Minimum Profile
Real-Time Pub/Sub (DCPS)
Networking Technology
Application
UDP/IP
Interoperable Wire Protocol (DDSI)
DD
SI v
2.1
DD
S v
1.2
Real-Time Networking
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Compact Edition ¨ All Community Ed. features
¨ Power Tools ¨ Eclipse Visual Modeling Tool
¨ Tuner Tool
Licensing ¨ Commercial Subscription / Licenses
¨ PrismTech Source Code License (non-copy-left)
Ownership DurabilityContent
Subscription
Minimum Profile
Real-Time Pub/Sub (DCPS)
Networking Technology
Application
UDP/IP
Interoperable Wire Protocol (DDSI)
DD
SI v
2.1
DD
S v
1.2
Real-Time Networking
PowerTools
MDE
Tuner
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Professional Edition
¨ All Compact Ed. Features
¨ DLRL v1.2
¨ Connectors ¨ SOAP
Licensing ¨ Commercial Subscription / Licenses
¨ PrismTech Source Code License (non-copy-left)
Object/Relational Mapping
Ownership DurabilityContent
Subscription
Minimum Profile
Real-Time Pub/Sub (DCPS)
Object-Oriented Pub/Sub (DLRL)
Application
UDP/IP
DD
SI v
2.1
DD
S v
1.2
SOAP
Connectors
PowerTools
MDE
Tuner
Networking Technology
Interoperable Wire Protocol (DDSI)
Real-Time Networking
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Enterprise Edition ¨ All Professional Ed. Features
¨ Security
¨ Connectors
¨ DBMS
Licensing ¨ Commercial Subscription / Licenses
¨ PrismTech Source Code License (non-copy-left)
Object/Relational Mapping
Ownership Persistence Content Subscription
Minimum Profile
Real-Time Pub/Sub(DCPS)
Object-Oriented Pub/Sub (DLRL)
Networking Technology
Application
UDP/IP
Interoperable Wire Protocol (DDSI)
DD
SI
v2.1
DD
S v
1.2
Real-Time NetworkingSOAP Secure Networking
DBMS
Connectors
PowerTools
MDE
Tuner
Connectors
OpenSplice DDS [Users & Use Cases]
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Some Selected Users
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Some Use Cases Aerospace & Defense
¨ Naval Combat Management Systems
¨ Submarines
¨ Vetronics (Military Vehicles)
¨ Tactical Links
¨ Simulation
¨ Cybercrime
¨ Flycatcher Systems
¨ Data Fusion
¨ Battle Transformation Center
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Some Use Cases Transportation
¨ Drones
¨ Air Traffic Control & Management
¨ Metropolitan Transportation
Financial Services
¨ Automated Trading Firms
¨ Risk Management Firms
The Future
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
DDS Standard Evolution
DDS
App Standard API
DDSI/RTPS network
Interoperable Wire Protocol
UML4DDS
ISO-C++ PSM 2010
Java5 PSM 2010 2004
2006
2008
X-T
ype
s 20
10
DDS
App
DDSI/RTPS
UML4DDS
2004
2006
2008
X-T
ype
s 20
10
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
DDS Standard Evolution
DDS
App Standard API
DDSI/RTPS network
Interoperable Wire Protocol
UML4DDS
ISO-C++ PSM 2010
Java5 PSM 2010 2004
2006
2008
X-T
ype
s 20
10
DDS
App
DDSI/RTPS
UML4DDS
2004
2006
2008
X-T
ype
s 20
10
We
b-D
DS
2011
We
b-D
DS
2011
§ Standardized way of accessing DDS from Web Technologies
§ REST, W3C WS-*, RSS, etc.
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
DDS Standard Evolution
DDS
App Standard API
ULS-DDSI network
Interoperable Wire Protocol
UML4DDS
ISO-C++ PSM 2010
Java5 PSM 2010 2004
2011
2008
X-T
ype
s 20
10
DDS
App
ULS-DDSI
UML4DDS
2004
2011
2008
X-T
ype
s 20
10
We
b-D
DS
2011
We
b-D
DS
2011
§ Ultra-Large Scale Extensions to the DDSI/RTPS wire-protocol
§ New Discovery § Support for TCP, etc.
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
ULS-DDSI ULS-DDSI
DDS Standard Evolution
DDS
App Standard API
network
Interoperable Wire Protocol
UML4DDS
ISO-C++ PSM 2010
Java5 PSM 2010 2004
2006
2008
X-T
ype
s 20
10
DDS
App
UML4DDS
2004
2006
2008
X-T
ype
s 20
10
We
b-D
DS
2011
We
b-D
DS
2011
§ Interoperable Security Architecture
§ Pluggable Policy / Labeling and Tagging
Sec
urity
20
12
Sec
urity
20
12
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
ULS-DDSI ULS-DDSI
DDS Standard Evolution
DDS
App
Standard API
network
Interoperable Wire Protocol
UML4DDS
ISO-C++ PSM 2010
Java5 PSM 2010 2004
2006
2008
X-T
ype
s 20
10
DDS
App
UML4DDS
2004
2006
2008
X-T
ype
s 20
10
We
b-D
DS
2011
We
b-D
DS
2011
§ Remote Method Invocations over DDS
§ Synchronous, Asynchronous and one ways RMI
Sec
urity
20
12
Sec
urity
20
12
DDS-RMI 2012
DDS-RMI 2012
Copyrig
ht 2011, PrismTech – A
ll Rights Reserved.
Concluding Remarks ¨ The DDS specification provides you the most
advanced middleware infrastructure for building mission-critical systems
¨ The Specification is evolving to support System of Systems and Cloud Messaging
¨ OpenSplice DDS is emerging as the leading implementation in the DDS market, quickly expanding well beyond its traditional core market
http://www.opensplice.com/
http://www.opensplice.org/
emailto:[email protected]
http://www.youtube.com/OpenSpliceTube http://opensplice.blogspot.com
http://bit.ly/1Sreg
http://www.slideshare.net/angelo.corsaro
http://twitter.com/acorsaro/
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS