Harnessing DDS in Next Generation

Smart Energy Systems Webcast

November 5, 2014

Webcast Presenters

Stuart Laval – Manager, Technology Development, Duke Energy

Stuart Laval is a member of Duke Energy’s Emerging Technology office, where his primary responsibility is on Smart Grid telecom-related activities. He also brings over 10 years experience in product development at manufacturers of utility equipment, cellular radio modules, and power semiconductor devices. Stuart has contributed to the successful launch of over 20 product innovations in mid-voltage smart grid sensors, 2G/3G wireless communication, consumer lighting, and audio amplifiers. Stuart holds Bachelors and Masters degrees in Electrical Engineering and Computer Science from MIT and a MBA from Rollins College.

Angelo Corsaro – CTO, PrismTech

Angelo Corsaro, Ph.D. is Chief Technology Officer (CTO) at PrismTech where he directs the technology strategy, planning, evolution, and evangelism. Angelo leads the strategic standardization at the Object Management Group (OMG), where he co-chairs the Data Distribution Service (DDS) Special Interest Group and serves on the Architecture Board. Angelo is a widely known and cited expert in the field of real-time and distributed systems, middleware, and software patterns, has authored several international standards and enjoys over 10+ years of experience in technology management and design of high performance mission- and business-critical distributed systems. Angelo received a Ph.D. and a M.S. in Computer Science from the Washington University in St. Louis, and a Laurea Magna cum Laude in Computer Engineering from the University of Catania, Italy.

Angelo  Corsaro,  PhD  Chief  Technology  Officer  

angelo.corsaro@prismtech.com

DDS Overview

Cop

yrig

ht P

rism

Tech

, 201

4

DDS is an Object Management Group (OMG) Standard for efficient, secure and interoperable, platform- and programming-language independent data sharing

DDS standardises: - Programming API - Interoperable wire-protocol - Extensible Type System - Data Modeling - Remote Procedure Call

The DDS Standard

Cop

yrig

ht P

rism

Tech

, 201

4

DDS provides a Global Data Space abstraction that allows applications to autonomously, anonymously, securely and efficiently share data

DDS’ Global Data Space is fully distributed (decentralised), highly efficient and scalable

Data Distribution Service (DDS)

DDS Global Data Space

...

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

Cop

yrig

ht P

rism

Tech

, 201

4DataWriters and DataReaders are automatically and dynamically matched by the DDS Discovery

A rich set of QoS allows to control existential, temporal, and spatial properties of data

Data Distribution Service (DDS)

DDS Global Data Space

...

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

Cop

yrig

ht P

rism

Tech

, 201

4

Elegant and High Level Data Sharing Abstraction

Polyglot and platform independent

• Java, Scala, C, C++, C#, JavaScript, CoffeeScript etc.

• Android, Windows, Linux, VxWorks, etc.

Peer-to-Peer by nature, Brokered when useful

Key Highlights

Cop

yrig

ht P

rism

Tech

, 201

4

Content and Temporal Filtering (both sender and receiver filtering supported)

Queries

20+ QoS to control existential, temporal, and spatial properties of data

Key Highlights

Cop

yrig

ht P

rism

Tech

, 201

4

DDS data lives within a domain

A domain is identified with a non negative integer, such as 1, 3, 31

The number 0 identifies the default domain

A domain represent an impassable communication plane

Domain

DDS Domain

Cop

yrig

ht P

rism

Tech

, 201

4

Partitions are the mechanism provided by DDS to organise information within a domain

Access to partitions is controlled through QoS Policies

Partitions are defined as strings: - “system:telemetry” - “system:log” - “data:row-2:col-3”

Partitions addressed by name or regular expressions: - ”system:telemetry” - “data:row-2:col-*”

Partitions

Partitions

Cop

yrig

ht P

rism

Tech

, 201

4

A Topic defines a domain-wide information’s class

A Topic is defined by means of a (name, type, qos) tuple, where

• name: identifies the topic within the domain

• type: is the programming language type associated with the topic. Types are extensible and evolvable

• qos: is a collection of policies that express the non-functional properties of this topic, e.g. reliability, persistence, etc.

Topic

TopicTypeName

QoS

...

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

Cop

yrig

ht P

rism

Tech

, 201

4

As explained in the previous slide a topic defines a class/type of information

Topics can be defined as Singleton or can have multiple Instances

Topic Instances are identified by means of the topic key

A Topic Key is identified by a tuple of attributes -- like in databases

Remarks: - A Singleton topic has a single domain-wide instance - A “regular” Topic can have as many instances as the number of different key

values, e.g., if the key is an 8-bit character then the topic can have 256 different instances

Topic and Instances

Duke Energy Emerging Technology Office

Harnessing DDS in Distributed Intelligence Platform

Stuart Laval

11/3/2014 page 1 Copyright © 2014 Duke Energy All rights reserved.

Duke Energy Test Areas: Integrated Grid Ecosystems

Sub

stat

ion

• Solar PV • Energy Storage • Dist. Mgmt System • PMU (6) • Weather stations (7)

Sh

erri

ll’s

Ford

, Ran

kin

, M

cAlp

ine

Su

bst

atio

ns

Cu

sto

me

r P

rem

ise

~60

ho

mes

ser

ved

by

M

cAlp

ine

circ

uit

s • Solar PV • Home Energy Manager • PEV • Charging Stations • Smart Appliances • Demand Response • In-home load monitoring

Dis

trib

uti

on

C

ircu

it

6 M

cAlp

ine

circ

uit

s

• Line Sensors (200+) • Solar PV • CES, HES Energy Storage • Comm. Nodes (3,000) • Intelligent Switches • DERMS/DMS • AMI metering (14,000)

11/3/2014 page 2 Copyright © 2014 Duke Energy All rights reserved.

IP Network

11/3/2014 page 3

Smart Meter

Capacitor Bank

Line Sensor

X Street Light

Smart Assets

Distributed Energy Resources

Transformer

Intelligent Switch

DEM

AN

D

ELEC

TRIC

GR

ID

Smart Generation

Continuous Emission Monitoring

Weather Sensor SUP

PLY

Other Nodes

Open Standards Node

Head End A

Head End B

Head End N

Data C

en

ter Message B

us

Network Router

UTILITY DATA CENTER

DIP: “Internet of Things” Platform for the Utility

• Processor(s) + Memory • Linux-based OS • Open API Messaging • 3rd Party Apps • Security / Network Mgr

4G LTE, Wi-Fi, GPS

Ethernet, Serial

PLC, RF ISM, Bluetooth

IP Router Capabilities

Optional Connectivity

Distributed Computing

I/O, Metrology, Fiber

Optional Required

Legend

Copyright © 2014 Duke Energy All rights reserved.

Flexible Hardware & Software Platform

4

Retrofit Inside Cabinet

Pole Mounted Enclosure

Padmount Enclosure

Substation Rackmount Server(s)

Integrated in End Device (as Software)

Copyright © 2014 Duke Energy All rights reserved.

Enabling Distributed Energy Resources with Intelligence at the Edge

Current State – Centralized Decision-Making Future State – Distributed Decision-Making

Meter Sensor

Cellular Network

Utility Office

Battery Storage

Rapid Swing in Production

Meter Line Sensor

Node

Cellular Network

Utility Office

Battery Storage

Rapid Swing in Production

Update Model

Response Decision +

Update Model

Response Decision

>1 Min < 0.25 sec

Transformer Transformer

Line Sensor Head End

Line Sensor Head End

5

Solar PV Solar PV

“Pass-Thru” “Field Message Bus”

Copyright © 2014 Duke Energy All rights reserved.

Field Test: Community Energy Storage

Shifting & Smoothing

In-rush Smoothing

Node w/ Field Msg Bus

Copyright © 2014 Duke Energy All rights reserved.

Field Message Bus: The Distributed “Internet of Things” Enabler

7

• Interoperability between OT, IT, & Telecom

• Modular & Scalable Hardware and Software

• End-to-End Situational Awareness

CIM DDS

Distributed Intelligence

Platform Copyright © 2014 Duke Energy All rights reserved.

AMI Smart Meters

Protection & Control

Distributed Energy Resources

Router

Middleware

Corporate Private

Network

MDM

SCADA

Head end

Upper Tier Central Office

(Utility Datacenter)

Application Processor

Core Processor

Legend

Middle Tier Nodes

(e.g. substation)

Lower Tier Nodes

(e.g. grid)

End Points Devices

Router

Middleware

Router

Middleware

Field Area Network

(FAN)

Wide Area Network (WAN)

Local Area Network

(LAN)

Local Area Network

(LAN) Physical Transport

Virtual Telemetry

Tier 5 DIP Node

Distributed Architecture: Telecom Networking Vision Multi-level Hierarchy: Seamless, Modular, Scalable

11/3/2014 page 8

Firewall

Virtual Firewall

DMS

Copyright © 2014 Duke Energy All rights reserved.

OPEN API MESSAGE BUS

Use-Case App(s)

OT System or Device

Analytics

Messaging

Translation

IT

Pu

blis

h

Sub

scri

be

Pu

blis

h

DNP Modbus

Smart Meter

Cap Bank

Intelligent Switch

FCI line Sensor

Sub

scri

be

OT

Compression

Security

Pu

blis

h

Sub

scri

be

Other

Pu

blis

h

Sub

scri

be

Transformer Telco Router

Battery/PV Inverters

DMS Pi Sandbox

Head-End

Pu

blis

h

Sub

scri

be

Convergence of OT and IT

DDS, MQTT, AMQP, CoAP

Copyright © 2014 Duke Energy All rights reserved.

Why Employ a Field Message Bus Architecture with DDS?

• Pub-Sub Advantages vs. Polling

• Standard Interfaces & Dictionary

• Flexibility & Resiliency

• Unlocks Modularity

• Scalable Infrastructure

• Organizational Efficiencies

page 10 Copyright © 2014 Duke Energy All rights reserved.

Why is the DIP (w/DDS) Important for Duke Energy?

page 11

• Provides accurate control and alleviates intermittency of distributed energy resources

• Provides the ability to scale independently, as needed, without needing a system wide rollout

• Takes cost out of the business by reducing integration time and effort

• Allows Duke to be at the forefront of developing new regulations and policies

Copyright © 2014 Duke Energy All rights reserved.

Angelo  Corsaro,  PhD  Chief  Technology  Officer  

angelo.corsaro@prismtech.com

VORTEX in Smart Energy and Utilities

Cop

yrig

ht P

rism

Tech

, 201

4

Vortex enables seamless, ubiquitous, efficient and timely data sharing across mobile, embedded, desktop, cloud and web applications

Vortex is based on the OMG DDS standard

The Vortex PlatformVortex Device

Tools

Integration

Vortex Cloud

MaaS

Smart Metering with VORTEX

Cop

yrig

ht P

rism

Tech

, 201

4

Tens of millions of Smart Meters for various kinds of utilities, e.g., electricity, water, etc.

More and more utilities companies want react in real-time consumption and usage patterns

Edge analytics are key for scalability, yet aggregated information is also needed

Data should be “scoped” but not sealed, in order terms, when needed an application should be able to get down to a smart counter data stream, regardless of its deployment

Large Scale Smart Metering

Cop

yrig

ht P

rism

Tech

, 201

4

Nations are usually organised in Regioni

Adding Hierarchy

Cop

yrig

ht P

rism

Tech

, 201

4

Each Regione, say Tuscany is further organised in Province

Notice that in this picture, Firenze does not denote the Town but the Provincia headed by the town of Florence

Adding Hierarchy

Cop

yrig

ht P

rism

Tech

, 201

4Each Provincia is further organised in Comuni

Did you notice Vinci? That’s where “Leonardo Da Vinci” comes from. “Da Vinci” is the Italian for “From Vinci”

Adding Hierarchy

Cop

yrig

ht P

rism

Tech

, 201

4

Each Comune is further organised in Quartieri (well… things are a bit more complex in reality)

We will assume that no further hierarchy will be added beyond that of a Quartiere and that the level just below is that of individual users and thus smart counters

Adding Hierarchy

Cop

yrig

ht P

rism

Tech

, 201

4

The Full Picturesmart-counter

smart-counter

smart-counter

smart-counter

smart-counter

smart-counter

Cop

yrig

ht P

rism

Tech

, 201

4

Data Rates

- The volume of data involved at a national scale — several tens of millions of updates per second — push toward an architecture in which data is aggregated throughput the hierarchy

- Nonetheless, we want to make it possible — when necessary — to tap into any un-aggregated data stream

Analytics

- Where are the analytics executed? Does that matter? We’d like the flexibility to deploy analytics wherever make sense and let the data flow to them

Challenges

Cop

yrig

ht P

rism

Tech

, 201

4

Locality

- Communication should exploit locality so to improve efficiency and reduce latency.

Unicast and Multicast

- Depending on the deployment we may be able to take advantage of some form of IP multicast, will the platform we able to exploit it?

Challenges

Cop

yrig

ht P

rism

Tech

, 201

4

Conceptually, it would be nice to have a solution in which the infrastructure, could allow us to concentrate simply on which we want to produce/consume

Yet, the infrastructure was smart enough to ensure that data sharing was efficient and scalable

Conceptual Solution

Cop

yrig

ht P

rism

Tech

, 201

4

System ArchitectureItalia

Toscana

Firenze

Empoli Firenxe

Centro

smart-meter smart-meter

Campo Marte

smart-meter ... smart-meter

... Rifredi

... Vinci

... Siena

Lazio ... Sicily

Catania

Aci Castello Acireale ... Catania

Centro ... Ognina

smart-meter ... smart-meter

... Siracusa

Avola Noto ... Siracusa

Cop

yrig

ht P

rism

Tech

, 201

4

At the bottom level we have the live data coming from smart meters

Higher up in the hierarchy we are interested in aggregated analytics

For instance, the major of the city may be interested in average consumption at a Quartiere-level while the President of the region may be interested in analytics aggregated by the Provincia

Yet at any point in time, anybody should be able to get down to any kind of data

Taking a SliceItalia

Toscana

Firenze

Firenxe

Centro

smart-meter ... smart-meterReal-Time Data

Quartiere-level Analytics

City-level Analytics

Provincia-level Analytics

Region-level Analytics

Nation-level Analytics

Cop

yrig

ht P

rism

Tech

, 201

4

Designing the Information Model

         enum  UtilityKind  {            ELECTRICITY,            GAS,            WATER              };                            struct  Meter  {            string  sn;            UtilityKind  utility;            float  reading;            float  error;              };                                      #pragma  keylist  Meter  sn  

           struct  Index  {            string  key;            float  value;              };  

           typedef  sequence<Index>  IndexSequence;  

           struct  UtilityAnalytics  {            string  scope;            UtilityKind  utility;            IndexSequence  indexes;              };              #pragma  keylist  UtilityAnalytics  scope  

Smart-Meter Analytics

Cop

yrig

ht P

rism

Tech

, 201

4

DDS Partitions will be used to scope data and provide a flexible way of aggregating it

Meter data will be published in a partition composed by nation:region:province:city:quarter:device-­‐sn  - italia:toscana:firenze:vinci:centro:a1b27fdz35  - italia:sicilia:catania:acireale:cappuccini:4cafebabe1

Analytics are produced using data at scope n are injected at scope n-1.

As an example, the average consumption for the quartiere cappuccini is produced using meter data from italia:sicilia:catania:acireale:cappuccini:*  and published into italia:sicilia:catania:acireale:cappuccini and so on

Notice that the use of partitions makes very easy to decide which over which sets of data the analytics have to compute

Information Organisation

Cop

yrig

ht P

rism

Tech

, 201

4

Analytics

Analytics

smart meters

Appliance

low-power radio protocol

Quartiere

City

Provincia

Region

VORTEX Device

VORTEX Cloud

VORTEX Cloud

VORTEX Device

VORTEX Device

VORTEX Cloud

VORTEX Device

VORTEX Cloud

VORTEX Cloud

VORTEX Cloud

VORTEX Cloud

Analytics

Analytics

The different deployment of VORTEX Cloud are federated to behave as a single instance. This logical instance optimally matches and routes information over a very large scale exploiting locality

smart-u https://github.com/kydos/smart-u

Cop

yrig

ht P

rism

Tech

, 201

4

smart-u is a VORTEX demo that illustrates how to implement a smart metering solution

For illustrative purposes, some analytics are computed using ESPER other are hand-coded

As you’ll see with this demo, while scoping information for scalability you can easily access it at any level

In addition analytics can be deployed where it makes the most sense

Through the VORTEX platform data can be injected and consumed across any platform, Web, Mobile, Embedded, Enterprise and Cloud!

smart-u

Cop

yrig

ht P

rism

Tech

, 201

4

In this presentation we have performed how VORTEX can address the challenges of building large-scale real-time smart metering architectures

The combination of Vortex Device and Vortex Cloud made it very easy to create and deploy our Internet-Scale, multi-device applications

Summary

Further information ▶ For further information please contact us directly

▶ angelo corsaro@prismtech.com

▶ www.prismtech.com (on-demand live demo available)

▶ Or via social media

www.twitter.com/prismtech

www.linkedin.com/company/prismtech

info.prismtech.com/facebook

www.slideshare.net/prismtech1