D2.1.2 Requirements and architectures for DSO...

SUNSEED, Grant agreement No. 619437 Page 1 of 105

D2.1.2 Requirements and architectures for

DSO-telecom converged communication networks in dense

DEG smart energy grid networks

Deliverable report


NOTICE The research leading to the results presented in the document has received funding from the European Community's 7th Framework Programme under the Grant Agreement number 619437. The content of this document reflects only the authors’ views. The European Commission is not liable for any use that may be made of the information contained herein. The contents of this document are the copyright of the SUNSEED consortium.


Document Information

1

PU Public

RP Restricted to other programme participants (including the Commission Services)

RE Restricted to a group specified by the consortium (including the Commission Services)

CO Confidential, only for members of the consortium (including the Commission Services)

Call identifier FP7-ICT-2013-11

Project acronym SUNSEED

Project full title Sustainable and robust networking for smart electricity distribution

Grant agreement number 619437

Deliverable number D2.1.2

WP / Task WP2 / T2.1

Type (distribution level)1 PU

Due date of deliverable 31.1.2016 (Month 24)

Date of delivery 31.3.2016

Status, Version Final, V2.0

Number of pages 105 pages

Responsible person, Affiliation Blaž Peternel, TS

Authors David Gustinčič, Božo Mišovič, Radovan Sernec, Milovan Stropnik, Saša

Žbontar, Tomaž Lovrenčič TS

Jimmy Nielsen, AAU

Jurij Jurše, Branko Ožbolt, Gregor Skrt, EP

Žiga Hribar, Bojan Miličič, Janez Zrnec, ES

Kemal Alič, Blaž Kažič, Miha Smolnikar, Maja Škrjanc, Aleš Švigelj, JSI

Ljupco Jorguseski, Max Schreuder, Manolis Chrysallos, Casper van den

Broek, TNO

Ziming Zhu, Zhong Fan, TREL

Herve Ganem, GTOSA

Reviewers Ljupco Jorguseski, TNO


Revision history Version Date Author(s) Notes Status

0.1 17.11.2015 Tomaž Lovrenčič, TS Main chapters Draft

1.0 9.2.2016 Tomaž Lovrenčič, TS Draft

1,9 31.1.2016 Blaž Peternel, TS Editing due all inputs from partners Draft

2.0 15.3.2016 Ales Svigelj, Urgan, Kuhar, JSI

Chapter 6.1 and 6.2.2 changes Draft

2.0 14.2.2016 Marko Pesko, Uros Droftina, Milan Stropnik, TS

Chapter 4.x changes, 6.2.3, GIS upgrade Draft

2.0 16.3.2016 Casper van den Broek, TNO

Final input Chapter 6.3 Draft

2.0 16.3.2016 Jimmy Nielsen, AAU Final input Chapter 3.5 Draft

2.0 10.3.2016 Herve Ganem, GTOSA Final input Chapter 3.5 Draft

2.0 23.3.2016 Blaž Peternel, TS Final changes, corresponding to Webex meeting

Final Draft fro Rewiev


Table of Contents Document Information ........................................................................................................................... 3

List of Figures ........................................................................................................................................... 8

List of Tables .......................................................................................................................................... 10

Abbreviations and acronyms ................................................................................................................. 11

SUNSEED project ................................................................................................................................... 13

Executive Summary ............................................................................................................................... 14

1 Introduction ................................................................................................................................... 16

1.1 Approach ............................................................................................................................... 16

1.2 Relation to the rest of the project ......................................................................................... 16

1.3 Outline of this report ............................................................................................................. 16

2 Smart grid in the context of Sunseed ............................................................................................ 18

2.1 Smart grid marketplace and stakeholders ............................................................................ 18

2.2 Aspects and challenges of smart grid .................................................................................... 20

2.3 Smart grid drivers, space exploration and key building blocks ............................................. 21

2.3.1 Identified cooperation gaps of DSO and telecom operator for smart grids ................. 22

2.4 State of standardisation in smart grids ................................................................................. 24

2.4.1 Standardization bodies .................................................................................................. 25

2.4.2 Smart grid standardization areas for SUNSEED ............................................................. 25

2.5 Overview of smart grid communication networks for WAN and NAN ................................. 26

2.5.1 Neighbourhood Area Network (NAN) ........................................................................... 27

2.5.2 Wide Area Network (WAN) ........................................................................................... 27

2.5.3 Enabling technologies for WAN and NAN ..................................................................... 28

2.5.4 Interoperability issues ................................................................................................... 30

2.5.5 Role of PLC in smart grids today .................................................................................... 30

2.6 DSO and telecom operator communications infrastructures ............................................... 32

2.6.1 DSO communication networks overview on example of EP ......................................... 33

2.6.2 Telecom operator communication networks overview ................................................ 36

3 Selected smart grid use cases for SUNSEED .................................................................................. 39

3.1 Massive prosumer participation for demand response (MPPDR) ........................................ 40

3.2 Advanced distribution network management system platform (ADNMSP) ......................... 43

3.3 Outage management with fault identification & localization via communications networks behaviour (IFIL) .................................................................................................................................. 47

3.4 Summary of use cases’ findings ............................................................................................ 50

3.5 LTE simulations of ADNMPS use case.................................................................................... 50

3.5.1 Number of preambles ................................................................................................... 51

3.5.2 RAO interval ................................................................................................................... 51


3.5.3 Report payload size ....................................................................................................... 52

4 Technical requirements and proposed solutions for future smart grids ...................................... 54

4.1 QoS overview and typical metrics ......................................................................................... 54

4.1.1 Types of services............................................................................................................ 55

4.1.2 QoS operation in a multilayer communication network ............................................... 56

4.1.3 QoS mechanisms ........................................................................................................... 57

4.1.4 Definition of traffic types .............................................................................................. 58

4.2 WAMS nodes communication solutions proposal ................................................................ 59

4.2.1 Requirements of routers in considered communication solutions ............................... 61

4.2.2 Requirements for local aggregation layer nodes .......................................................... 62

4.3 Security requirements ........................................................................................................... 62

4.3.1 Requirements linked to data management .................................................................. 62

4.3.2 Requirements linked to communications ..................................................................... 64

4.3.3 Requirements related to confidentiality ....................................................................... 65

4.3.4 Requirements related to integrity ................................................................................. 66

4.3.5 Requirements related to availability ............................................................................. 67

4.3.6 Requirements related to disaster recovery ................................................................... 68

4.3.7 Requirements related to identification authentication, authorization ......................... 69

4.3.8 Requirements related to risk assessment ..................................................................... 70

4.4 IP/MPLS as communications network core ........................................................................... 71

4.4.1 Services within IP/MPLS network .................................................................................. 71

4.5 Time synchronisation for smart grid observability................................................................ 73

4.6 Potential technologies for future smart grid communication networks .............................. 74

4.6.1 Reengineering existing cellular networks...................................................................... 74

4.6.2 Software defined networking ........................................................................................ 76

4.6.3 Cloud or centralised radio access networking .............................................................. 77

4.6.4 5G .................................................................................................................................. 77

5 Economic and business requirements of cost effective smart grid communication solutions .... 79

5.1 Synergies of DSO and telecom operators for cost effective smart grids .............................. 79

5.2 SLA on smart grid communications network marketplace ................................................... 81

6 Proposal for Sunseed smart grid architecture .............................................................................. 85

6.1 System architecture .............................................................................................................. 85

6.2 Functional building blocks ..................................................................................................... 87

6.2.1 WAMS node in high density distribution smart grids ................................................... 87

6.2.2 Phasor measurement unit ............................................................................................. 94

6.2.3 Visualization with dynamic geographical information system ...................................... 95

6.2.4 FPAI nodes ..................................................................................................................... 98


7 Conclusion ................................................................................................................................... 100

8 References ................................................................................................................................... 102


List of Figures

Figure 1: Domains within smart grid marketplace (black – electricity flow, blue – information flow). 19 Figure 2: Smart grid architecture layers, from physical to business [CEN 2012]. ................................. 19 Figure 3: Smart grid communication patterns today are vertical, lacking horizontal communication among same node types. ...................................................................................................................... 24 Figure 4: IEC smart grid standardisation map with SUNSEED linkage [IEC 2010]. ................................ 26 Figure 5: Smart grid WAN and NAN communication networks. ........................................................... 26 Figure 6: Smart meters with PLC and communication via PLC concentrator to DSO. .......................... 31 Figure 7: PLC signal with IDIS protocol and frequency bins at 65 kHz and 71 kHz................................ 32 Figure 8: PLC signal according to PRIME, measured PLC concentrator. ............................................... 32 Figure 9: DSO – TSO communication network topological partitioning. .............................................. 34 Figure 10: DSO network topology: logical partitioning. ........................................................................ 36 Figure 11: Telecom operator communication network topological partitioning. ................................ 37 Figure 12: Telecom operator communication core network with different domains. ......................... 38 Figure 13: UC-1: Power grid with consumers, no smart metering and low DER penetration. ............. 42 Figure 14: UC-1: Smart grid with prosumers, AMI and high DER penetration. ..................................... 42 Figure 15: UC-2: Classic operation approach in distribution grid. ........................................................ 45 Figure 16: UC-2: Advanced operation and control management approach in distribution grid. ......... 46 Figure 17: UC-3: High level how it works illustration: including telecom operator's modem management system (top) and with smart meter and/or WAMS node dying gasps triggering (bottom). ............................................................................................................................................... 49 Figure 18: Service outage breaking point for different number of preambles. .................................... 51 Figure 19: Service outage breaking point for different RAO intervals. ................................................. 52 Figure 20: Service outage breaking point for different report payload sizes. ...................................... 53 Figure 18: DiffServ extensions [Cisc 2014a]. ......................................................................................... 57 Figure 19: MPLS EXP bits [Cisc 2014a]. .................................................................................................. 57 Figure 20: SUNSEED generalised DSO and/or customer end nodes access options, communication networks and supporting infrastructure. .............................................................................................. 60 Figure 24: Schematic of TS core elements and their outer connections for SUNSEED ......................... 73 Figure 21: OpenFlow API for software control of SDN network elements [ONF 2012]. ....................... 76 Figure 22: Synergies of active cooperation opportunities DSO – telecom operator at various levels and of different types. ........................................................................................................................... 80 Figure 23: Generic SLA flow between DSO-telecom operator for communication networks in smart grid. ........................................................................................................................................................ 82 Figure 24: Per smart grid traffic SLA flow between DSO-telecom operator for converged communication networks in smart grid. ............................................................................................... 83 Figure 25: Systems architecture view and key building blocks of SUNSEED. ........................................ 86 Figure 27: WAMS node vs. smart meter vs. PMU. ................................................................................ 88 Figure 28: WAMS node building blocks. ................................................................................................ 91 Figure 29: WAMS node logical communication paths logical separation. ............................................ 93 Figure 30: PMU key building blocks. ..................................................................................................... 94 Figure 31: PMU data concentrator placement within power grid operator network. ......................... 95 Figure 32: Dynamic GIS concept and actors. ......................................................................................... 95 Figure 33: Dynamic GIS building blocks. ................................................................................................ 97 Figure 34: 34XML Schema representation of time shifert control space update message. ................. 99



List of Tables

Table 1: Comparison of candidate communication technologies for WAN, NAN. ............................... 30 Table 2: Use case UC-1: Massive prosumer participation for demand response (MPPDR). ................. 41 Table 3: Use case UC-2: Advanced distribution network management system platform (ADNMSP). . 45 Table 4: Use case UC-3: Innovative Fault Identification and Localization using external (non-grid) data sources (IFIL). ......................................................................................................................................... 48 Table 5: Considered LTE simulation parameters. .................................................................................. 50 Table 5: Example of types of service classes and bandwidth assignments in MPLS. ............................ 58 Table 6: Security, data management requirements. ............................................................................. 64 Table 7: Security, communications requirements. ............................................................................... 65 Table 8: Security, confidentiality requirements. ................................................................................... 66 Table 9: Security, integrity requirements. ............................................................................................. 67 Table 10: Security, availability requirements. ....................................................................................... 68 Table 11: Security, disaster recovery requirements. ............................................................................ 69 Table 12: Security, identification, authentication, authorization requirements. ................................. 70 Table 13: Security, risk assessment requirements. ............................................................................... 71 Table 14: Some example cases of synergies in DSO – telecom operator cooperation. ........................ 81 Table 15: A sample of selected SLA parameters per smart grid communication network types. ........ 84 Table 16: WAMS node major attributes vs. smart meter and PMU solutions. ..................................... 89 Table 17: Smart grid communication types and technologies in distribution grid. .............................. 91 Table 18: Providers of communication infrastructure by type. ............................................................ 92 Table 19: Logically separate communication interfaces per type of WAMS node. .............................. 94


Abbreviations and acronyms

4G 4th generation mobile technology, LTE

2.5G 2.5th generation mobile technology, GPRS

ADSS All Dielectric Self Supporting combined optical and electrical cable

API Application Programming Interface

ARPU Average Revenue Per User

BSOTA Beyond State Of The Art

CBA Cost Benefit Analysis

CHP Combined Heat and Power

CIM Common Information Model

CPE Customer Premises’ Equipment, modem

CRAN Cloud Radio Access Network

CO2 Carbon Dioxide

DAP Data Aggregation Point

DEG Distributed Energy Generation

DEMS Distributed Energy Management System

DER Distributed Energy Resources

DG Distributed Generation

DMS Demand Management System

DR Demand Response

DSL Digital Subscriber Loop/Line

DSLAM Digital Subscriber Loop/Line Access Multiplexer

DSM Demand Side Management

DSO Distribution System Operator

DWDM Dense Wavelength Division Multiplexing

EMS Energy Management System

ENTSO-E The European Network of Transmission System Operators for Electricity

EV Electric Vehicle

FRR-A Frequency Restoration Reserve - Automatic

FRR-M Frequency Restoration Reserve - Manual

GPON Giga bit Passive Optical Network

GPRS General Packet Radio Service, 2.5G

GPS Global Positioning System

HV High Voltage, in power transmission lines, usually 110 kV or more

ICT Information and Communication Technologies

IPS Intrusion Prevention/Protection System

ISP Internet Service Provider

IT Information Technologies

KPI Key Performance Indicator

LAN Local Area Network

LAR Local Aggregation Router

LTE Long Term Evolution, 4G

LV Low Voltage, 240 V

Mb/s Mega bit per second, also abbreviated as Mbps


MIMO Multiple Input Multiple Output

MPLS Multi Protocol Label Switching

MSAN Multi Service Access Node

MV Medium Voltage, usually 10 kV or 20 kV

MVNO Mobile Virtual Network Operator

NIST National Institute of Standards and Technology

OFDM Orthogonal Frequency Division Multiplexing

OPCC Optical Phase Conductor combined optical and electrical cable

OPGW Optical Ground Wire combined optical and electrical cable

PGW Packet data Gateway in 4G

PLC Power Line Carrier or Communication

PV Photovoltaic

RAN Radio Access Network

RAR Remote Aggregation Router

RES Renewable Energy Sources

ROI Return On Investment

RTT Round Trip Time

SAIDI System Average Interruption Duration Index

SAIFI System Average Interruption Frequency Index

SCADA Supervisory Control and Data Acquisition

SDN Software Defined Network

SDP Service Delivery Point

SGW Signalling Gateway in 4G

SLA Service Level Agreement

SNR Signal to Noise Ratio

SOTA State Of The Art

TDM Time Division Multiplexing

TSO Transmission System Operator

Utility Distribution System Operator or energy retailer

V2G Vehicle to Grid

VPP Virtual Power Plant

VRF Virtual Routing and Forwarding

WAAS Wide Area Augmentation System

WAMS Wide Area Measurement System

WiFi Wireless Fidelity

WLAN Wireless LAN

xDSL X (= A – asymmetric, S – symmetric, V – very high bit rate) DSL

XML Extensible Markup Language


SUNSEED project

SUNSEED proposes an evolutionary approach to utilisation of already present communication networks from

both energy and telecom operators. These can be suitably connected to form a converged communication

infrastructure for future smart energy grids offering open services. Life cycle of such communication network

solutions consists of six steps: overlap, interconnect, interoperate, manage, plan and open. Joint

communication networking operations steps start with analysis of regional overlap of energy and

telecommunications operator infrastructures. Geographical overlap of energy and communications

infrastructures identifies vital DSO energy and support grid locations (e.g. distributed energy generators,

transformer substations, cabling, ducts) that are covered by both energy and telecom communication

networks. Coverage can be realised with known wireline (e.g. copper, fiber) or wireless and mobile (e.g. WiFi,

4G) technologies. Interconnection assures end-2-end secure communication on the physical layer between

energy and telecom, whereas interoperation provides network visibility and reach of smart grid nodes from

both operator (utility) sides. Monitoring, control and management gathers measurement data from wide area

of sensors and smart meters and assures stable distributed energy grid operation by using novel intelligent real

time analytical knowledge discovery methods. For full utilisation of future network planning, we will integrate

various public databases (e.g. municipality GIS, weather). Applications build on open standards (W3C) with

exposed application programming interfaces (API) to 3rd parties enable creation of new businesses related to

energy and communication sectors (e.g. virtual power plant operators, energy services providers for optimizing

home energy use) or enable public wireless access points (e.g. WiFi nodes at distributed energy generator

locations). SUNSEED life cycle steps promise much lower investments and total cost of ownership for future

smart energy grids with dense distributed energy generation and prosumer involvement.

Project Partners

1. TELEKOM SLOVENIJE D.D.; TS; Slovenia

2. AALBORG UNIVERSITET; AAU; Denmark

3. ELEKTRO PRIMORSKA, PODJETJE ZA DISTRIBUCIJO ELEKTRICNE ENERGIJE D.D.; EP; Slovenia

4. ELEKTROSERVISI, ENERGETIKA, MERILNI LABORATORIJ IN NEPREMICNINE D.D.; ES; Slovenia

5. INSTITUT JOZEF STEFAN; JSI; Slovenia

6. GEMALTO SA; GTOSA; France

7. GEMALTO M2M GMBH; GTOM2M; Germany

8. NEDERLANDSE ORGANISATIE VOOR TOEGEPAST NATUURWETENSCHAPPELIJK ONDERZOEK - TNO; TNO;

The Netherlands

9. TOSHIBA RESEARCH EUROPE LIMITED; TREL; United Kingdom

Project webpage

http://www.sunseed-fp7.eu/

http://www.sunseed-fp7.eu/


Executive Summary

Realising the EU target of > 30 % electricity from renewable source in the foreseeable future will require fresh (re)thinking about distribution grids and in particular about communication networks as a foundation of smart distribution grid operations. The objective of SUNSEED is to investigate and test on a large scale field trial the concept of smart grids in distribution based on converged communication networks built out of existing infrastructures of electrical utility (DSO) and telecom operator. Further economic analysis of various DSO – telecom operator cooperation options based on synergies of infrastructures will show what cost benefits can be expected if telecom operator can play active and significant role as a provider of communication networks for dense smart grids in distribution electricity networks. A systematic analysis of possible developments in distribution grid is based on three types of use cases: UC-1 with a dense network of measurement nodes, smart meters and WAMS, on all DER nodes and majority of prosumer locations that will greatly enhance distribution grid observability; UC-2 presents use of real time communication and time synchronous measurements (from 1 s to 15 min reporting intervals) as the solution towards advanced distribution management system with capabilities up to date seen only in transmission grid (e.g. voltage profile from state estimation); UC-3 studies the combined use of converged communication and power networks to pinpoint the type and location of failures. Particular implementations and solutions based on these use cases will be physically realised also on field trial. We build upon the known standards in smart grid realm, particularly IEC 61850 and Smart Grid Architecture Model (SGAM) defined by IEC. The state of the art in communication networks for smart grid in WAN and NAN is covering also technical characteristics of solutions being used to date. PLC has a particular role in AMI and will undoubtable continue to be main technology of choice for communication from smart meters to first aggregation point (usually PLC concentrator). Analysis of contemporary DSO and telecom operator networks reveals interesting similarities (e.g. use of MPLS, timing synchronisation mechanism) and these will be put to use in architecting converged networking solutions. DSO uses physically different networks to transport different traffic types, to assure performance or make access secure (e.g. control/command, teleprotection, data center access, AMI collection). Technical requirements for communication networks in distribution smart grid present a starting point of not to be missed functionalities. Achieving QoS as demanded by characteristics of elements and systems (e.g. teleprotection, control/command, measurement) in distribution grid is essential in future smart grids, particularly those based on converged networks. Information traffic must be marked already at the originating node (e.g. WAMS), so that it is properly classified and given suitable priority further down the communication network, until it reaches desired destination, either network management center of another substation. Access via 4G mobile or PLC, Ethernet, xDSL wireline networks are types of communication solutions in neighbourhood area networks that were given particular attention together with end node routing specification. MPLS operation to achieve service and information traffic separation and interconnection on L2 or L3 is explained further. Techniques of timing synchronisation will be used in WAMS measurement synchronisation to achieve fully synchronous picture of distribution grid behaviour (observability). We have assessed security measures from wider perspective covering data management, communication, confidentiality, integrity, availability, disaster recovery, identification, authentication, authorisation and risk management. We also present some valuable technologies that may be seen in future smart


grid communication solutions (e.g. software defined networking, centralised radio access networks, reengineering of cellular networks) and some will be evaluated during project itself. Larger view of systems architecture covers description of key building blocks and their integration into seamless framework that provides intelligence to WAMS nodes and smart meter measurement data streams. Authentication and access authorisation to selected parts of central software resources is the first step to establish secure information exchange from communication or measurement nodes within smart grid. Other software building blocks, from various project partners, third party open source solutions are integrated with Web services and provide data analytics and fault management capabilities and visualisation of distribution grid operation with dynamic geographical information system. Some provide the interface to involve prosumers in the demand response loop. A small framework for economic and business investigation of various possible cooperation types between DSO and telecom operator is presented. It covers 9 infrastructure types, each from design to end of its life cycle and coupled with detailed cost benefit analysis of various options will provide answers of economically most suitable synergies between two stakeholders. We specifically propose solutions for SUNSEED project throughout this deliverable (marked with “SUNSEED proposed use”). Based on the contents of this deliverable we are able to proceed with field trial specification, its detailed communication network design and architecting the building blocks (WAMS node) and key software modules in the top of the framework pyramid.


1 Introduction

The aim of this deliverable is to outline the requirements of communication network for smart grids, in distribution electricity grid. It targets main building blocks, architectures and mechanisms that should constitute such communication network. The future smart grid presents a huge investment for the society, but it will bring also numerous benefits. We are moving into the realm of fully integrated, distributed energy resources and generators, deployed also by home customers. Integration of these resources within distribution grid, and make it work as a unison, much like the ordinary central electricity generation facility requires the application of novel mathematical techniques that rely on data from a network of measurement nodes dispersed within distribution grid itself. Connecting these measurement units, end users smart meters and control nodes of the distribution grid requires us to rethink the communication networks from ground up. The paramount goal is achieving the connection and resulted benefits with least cost. Reuse and interconnection with telecom operator(s) communication network may provide the possible solution. We call such implementation a converged DSO – telecom operator communication network, since it leverages the present state of the art of already operated networks from both stakeholders, but significantly stretches the requirements towards telecom operator, technical as much as business. This are must requirements since telecom operator in this case is not dealing with just another business customer, but rather with an equal utility type of partner that operates mission critical infrastructure.

1.1 Approach

We set out discussing typical use cases of future distribution grids, knowing the wider EU goals. Three are examined that set the boundaries of our problem space that we are going to explore practically on the field trial with tools defined within the system architecture. The central element to provide distribution grid observability is WAMS node. Beside its measurement capabilities it must seamlessly connect with converged communication network (wireline, wireless) in a secure manner and following specific requirements (e.g. coloring of data traffic, time synchronisation) so that upper communication network layers can assure QoS of different smart grid traffic types.

1.2 Relation to the rest of the project

Contents of this deliverable is going to be applied as guidelines for detailed implementations in WP3 (Communications networking), WP4 (Control management and analytics) and WP5 (Field trial). The requirements outlined in this deliverable should also provide general recipe for implementing converged DSO-telecom operator smart grid environment.

1.3 Outline of this report

Chapter 2 is presenting the smart grid problem space with stakeholders, their interconnection and presents key building blocks and types of communication networks and technologies already in deployment (WAN, NAN). Overview of standardisation, focused on distribution grid is necessary to assure fully integration of new solution within existing environments. DSO and telecom operator networks, with implementation details outline some state of the art practices. Chapter 3 describes three use cases that take into account massive user participation within distribution grid, advanced


distribution network management and fault localisation based on communication network behaviour. Chapter4 is the core of the communication requirements part covering QoS, proposal for WAMS node communication, security and privacy issues, MPLS implantation and time synchronisation across the grid. Interesting technological solutions are also mentioned (from reengineering to 5G). Chapter 5 presents an approach for economic evaluation and business SLA perspective when considering converged DSO – telecom operator network operation. Chapter 6 presents the top level view of the architecture and its main building blocks from WAMS nodes, power load interaction environment for demand response, data analytics and visualisation. Chapter 7 summarizes the findings.


2 Smart grid in the context of Sunseed

We are going to see the most disruption happening in distribution of electricity, which will pull along the whole value chain within the smart grid marketplace. Integration of DER from renewables with EV proliferation promises to improve efficiency via DR (i.e. demand side management). This chapter address the players on the smart grid marketplace and current approaches to communication networks within distribution grid environments. We further compare DSO and telecom operator communication networks, to identify key solution types as these networks are going to be used for converged cooperation, i.e. seamless interconnection to act as a communication foundation of future dense distribution smart grids.

2.1 Smart grid marketplace and stakeholders

The conceptual reference model of smart grid marketplace is based on NIST [NIST 2012]. Each domain is characterised by stakeholders, systems and devices (e.g. smart meter, WAMS node) and applications (e.g. DR). The domains are interconnected with energy flows and information flows (Figure 1). The key advancements of smart grid over central generation distribution grid are the bidirectional energy flow from the user side as well as bidirectional information flow for better coordination between suppliers and customers and control of power grid. Information flows can be further decomposed into measurement, control and other types of information (e.g. for smart grid support or money transfers). We also define use cases that describe how these act together to achieve specific goal (e.g. new type of fault management). We have added telecom operator explicitly to hint at the possibility of its active involvement in converged communication solutions for smart grids. End users, or customers are further divided into industrial and home, where we use the term prosumer (producer + consumer) to indicate possible ownership of DER at home location [Chan 2012]. Home consumers and DER owners can participate in electricity market through VPP provider. Note that DSO is still the owner and managing party of the electricity distribution grid. Communication technologies are key building blocks of smart grids and the condition for all subsequent applications within the smart grid. They provide connectivity between intelligent devices and systems that are integrated into the network within the concept of smart grids. Domain of smart grid technologies is primarily a communication layer in Figure 2. The reference architecture for European smart grids concept with selection of technologies for the implementation of communication infrastructure is given in set of IEC standards [CEN 2012].


Figure 1: Domains within smart grid marketplace (black – electricity flow, blue – information flow).

Figure 2: Smart grid architecture layers, from physical to business [CEN 2012].


2.2 Aspects and challenges of smart grid

Communications networks for smart grids, particularly their efficient (re)use from DSO and telecom operators for the same purpose and operation in power grids is the main aspect of SUNSEED project. We investigate how to architect, deploy, own, operate and maintain such communication networks with optimal total cost of ownership and effective operation for DSO. Future dense smart grids with prosumers scattered throughout distribution grid will move power grid stability challenges down from transmission to distribution. Real time measurements in distribution grid will be of paramount importance. We argue that substantial deployment of WAMS nodes at DER locations and others, beside already installed smart meters and PMUs at selected points in the distribution grid is the necessary step to make behaviour and utilization of distribution grid visible. Such observability enables application of modern control and prediction algorithms. Communication networks of future power grid thus become much more important in terms of handling interconnected M2M devices (e.g. WAMS node, smart meter) and assuring QoS for information exchange of automatic control algorithms between distribution system management centers and key nodes (e.g. substation, protection, DER). Communications infrastructure is a long term investment, second only to distribution electricity grid. Communication infrastructure can grow and be deployed as required by subsequent advancements and requirements of electrical grid nodes in an almost “spaghetti” manner (as it usually happens when new major shifts or disruptions occur). We can also pay particular attention to future needs and architect a long term, scalable, secure solution that can grow in performance, capacity (e.g. number of end nodes) and reliability (e.g. fault tolerant physical links or redundancy protocols). Technical solutions are only the second rate problem, though. Long term financing is the real issue. Setting this aside we must ask another question: Is the communications technology the core competence of DSO, or TSO, or of any electrical operator on the smart grid marketplace at all? However, looking at modern DSO today, we can see a lot of communication networks and IT deployed that enables efficient operations of electrical grid and billing of electricity to end customers. What will happen with new smart grid, so much different that DSO may have problems with? There is a dramatic shift going on in many economies, from oil and nuclear central sources of electricity to distributed, renewable sources [App 2013]. This shift is stimulated either by requirement of being independent on national or regional level from disruptions of supply of key resources from other parts of the world (e.g. oil or gas supply) or by bad experiences of natural disasters that triggered havoc nationwide (e.g. Fukushima, Japan and shutting down of nuclear reactors), the third mover is also the need for a clean environment [EC 2013], [Strick 2011]. Based on these facts we can postulate that DER are here to stay and their proportion and importance will grow, maybe even dramatically in the next decade [IEA 2011], [IEA 2013]. This will result in the requirement to optimise electricity grid in distribution, and to achieve this deployment of new observation methods will be mandatory. It is envisioned that the methods and techniques presently used in transmission network (e.g. use of PMU and sophisticated grid management software) will move to distribution network. But, many more measurement elements may be needed and new methods of operation will be created (e.g. agent based control), to cope with and maintain stable, denser distribution grid with large numbers of DER. Furthermore, new services, non-existent to date will proliferate (e.g. VPP, micro grids, DR, home energy user portals). Dense deployment of DER is placing challenges to balancing of electricity demand and supply, not only on the level of complete distribution grid, but rather on a smaller scale, too: per substation, transformer station, street, even household itself as the smallest unit of consumption and production. All these inevitably result in strained demands on communication networks: more nodes, more traffic data per node and more


severe security requirements. Security in distribution grid will be the issue, since each node with communication capability (e.g. smart meter, WAMS node) is a potential threat injection point to disrupt the operation of grid or to wiretap (intercept) and extract communication data for fraud purposes. These may result in additional requirements on communication networks in terms of availability and agility (e.g. dynamic switching of different physical interfaces or paths). It is thus very valid to propose that DSO deal with electrical distribution grids and ask somebody else to solve the aforementioned challenges, in a secure, scalable and cost effective way [EC 2011], [EC 2014], [Eur 2013], [Grie 2013]. Observing that telecom operators have considerably expanded their access networks in the last two decades (since the Internet explosion) and are operating over copper wires, fiber or wireless media, it is only naturally to systematically explore the possibilities of technical solutions and operating modes of DSO – telecom operator tandem in order to architect the secure, scalable and cost effective, communication and IT parts of the smart grid.

2.3 Smart grid drivers, space exploration and key building blocks

One of key objectives of smart distribution grid is to significantly improve the communication capabilities and cost effectiveness. Converged DSO and telecom operator communication network holds the key to this and has to focus on current and emerging technologies that enable QoS, higher data rates, high reliability, low latency, to accomplish distribution grid observability. Striving to achieve optimal technical performance and lowest total cost of ownership of solutions, we need to optimize both communication networks themselves as well as placement of WAMS nodes within distribution grid. WAMS node should be installed at the following power network nodes:

Main supply substation (SBS): all MV busbars (20 kV).

Transformer station: all LV busbars in transformer station alongside MV feeder sub graph.

Some characteristic electric nodes in LV network (depends on analysis, approximately 15 % of all LV nodes).

Prosumers and DER.

WAMS node typical measuring parameters are:

a) Electrical parameters (e.g. voltage, current, active, reactive power and appearance power with power factor) for state estimation.

b) Power quality parameters (e.g. THD, harmonics and other according to IEC 50160). c) Reliability parameters (e.g. number of outages, outages duration). d) Other parameters for SUNSEED developed KPI. e) Weather and other environmental parameters.

The implementation of all developed concepts and technologies on the field trial have to meet all valid standards covering designing power system facilities (IEC 61850), transmission communications protocols in electrical engineering and power system automation applications (IEC 60870-5 104, DNP3), smart meters (e.g. IEC 62052-11, IEC 62052-21, IEC 62052-22, IEC 62052-23, IEC 62052-24, IEC 62058-11) and synchrophasor types of measurements (IEEE C37.118.1). All developed technology with SUNSEED system architecture will be integrated in separate control unit with all basic prescribed SCADA protocols in terms of connecting to the DSO physical and logical communication network and data storage functionalities and integrated within the SUNSEED data flow system architecture for the field trial purposes.


2.3.1 Identified cooperation gaps of DSO and telecom operator for smart grids

Communication technologies are fundamental part of smart grid and main objective is to reliably connect millions of end nodes (e.g. homes, monitoring nodes, and smart meters), 100 K’s substations and other types of control/command nodes within a single system. Is there a room for greater telecom operator role? The most obvious approaches to deal with communication networks for within smart grid distribution are:

1. Exclusively done by DSO. This presents the easiest way, but is expected to be the most costly solution.

2. Exclusively done by telecom operator. This is unlikely and unreal scenario, considering that DSO already own communication infrastructure.

3. Mix of solutions and services from both DSO and telecom operator. SUNSEED is going to investigate this approach systematically, top to down (Please refer to Chapter 5).

The definitive deciding factor should be total cost of ownership or “EUR efficiency value” [EC 2011], [EC 2014], [Eur 2013].

2.3.1.1 State of distribution grid and relation to communication network issues

It is interesting to note that DSO is practically blind in distribution grid below MV level, i.e. there are currently no real time observation of consumer, neither are there all transformer stations connected. SCADA is just starting to take off within distribution grid of majority of DSOs. Push with new developments (e.g. DR, VPP, micro grids) may accelerate the change. The main dichotomy of DSO communications requirements for smart grids vs. telecom traditional business models lies in simple fact: low income (i.e. low ARPU), low cost AMI requires complete SLA redefinition from telecom operator side (as compared with high income, high bandwidth mobile smart devices). Let us not forget that usual ROI in electricity grid 20 to 40 years compared to a year for mobile smart device (limited by user change with a new model) and 3 to 7 years for communication equipment in telecom operator. Albeit lower AMI revenue per end node, still results in almost guaranteed and steady stream of (almost) guaranteed revenue and that is a stark contrast to highly deregulated communications marketplace and highly mobile user base with 30 % churn rates. DSO entrance into communications networking of smart grid must be through requirements specification that can even be derived in cooperation with telecom operator(s). At present smart meter communications (AMI) are one of the most undemanding tasks, e.g. send data packets from locally installed smart meters ones per day via mobile network. But such nimble requirements will have to change to almost mission critical when DR becomes wide spread, or deep deployment of PMU (or similar) devices become widely used. There are two drivers: 1 or 15 min real time communication of M2M type and almost instant (< 1 min) user observability need of DR or demand management status of DER, EV battery and appliances in homes. Recent use of mobile communication in distribution grid span from AMR (smart meter via PLC and mobile) to substation (PLC and/or xDSL with mobile backup secondary link) connections. Mobile communications were not suitable for mission critical applications (e.g. control/command, protection), predominantly due to non-guaranteed delay bounds and somewhat lower communication link availability than typically required for smart grid protection. Situation will change dramatically in 4G where multiple end devices within base station reach can be assigned different bandwidth and QoS on the L1 and L2 levels (modulation, MAC) [Agil 2011]. However, interconnection (or roaming) with two (or more if applicable) mobile operators is required to achieve true redundancy and meet the communication availability constraints. Each end node must be equipped with communication interface incorporating dual SIM cards to ensure reliable and consistent data transfers. When packet loss threshold and/or SNR threshold is/are exceeded for whatever reason,


such interface dynamically/automatically switch from primary to secondary (i.e. backup) network connection. On the mobile core side it is possible to implement dual HSS policy, too, i.e. having specialized HSS for smart grid SIM card operated equipment, totally separate from smart phone HSS, thus also solving the contention issues. A low latency communication requirement posed by protection switching needs is doable also in telecom operator networks, since it was done before: financial industry required several 100 microseconds latency between nodes connecting banks and stock market computers to enable high frequency trading. MPLS as a L2.5 technology can solve latency issues, but some clever routing rules will be required to achieve also low jitter on same types of geographically dispersed distribution grid nodes. Self-healing MV/LV networks are driven ultimately by regulator specifying needs for mission critical systems. Implementing (tele)protection requires duplicated, fully separate systems, ideally implemented by totally different designs and implementations to enhance reliability, i.e. independence of unknown equipment bugs, protocols. Communication requirements must meet this and be duplicated (i.e. in a high availability mode – HA). High availability communication solutions can be implemented as:

1. Duplication of resources (e.g. dual physical interfaces, control cards, power supplies) 2. Separation of path (e.g. physically separate communication paths, ideally implemented with dual

telecom operators).

Some DSO’s require power autonomy of 24-72 h at critical locations (e.g. main substation). This requirement is even beyond what telecom operators maintain in their own networks today, i.e. usually 4-8 h. To make matters even worse, telecom operators have lowered battery backup times on their equipment since transition from TDM to full IP communication. Voice communications are still required for workforce instructions within smart grid environment, on the field and available in locations down to substation level. A viable solution is VoIP and/or mobile, since TDM is in decommissioning phase in all telecoms. Different types of traffic types and latencies can be met with strict QoS implementations end-2-end and to logically isolate smart grid traffic from other telecom operator’s traffic in order to guarantee bandwidth and delivery to critical data packets (e.g. (tele)protection or even grid observability). Furthermore, all communication and information security processes and solutions must be designed in by telecom operator who has expertise (e.g. ISO27001), since it manages wide spread communications networks itself. Green field physical implementation of fiber within distribution grid is most cost efficiently done with specialized electrical cabling that incorporates multiple fibers at its core. This would automatically create a dense dark fiber network in distribution grid that can be very efficiently used by DWDM and even leased to other service providers. Legal regulation of complete chain from electricity generation to markets is inflexible and to follow the significant changes brought up by dense DER and prosumer demand response participation this must be reflected in legislation, too (e.g. dynamic multi tariff systems, changing energy retailer semi-automatically within day period).


2.3.1.2 Foreseen communication patterns in smart grids

Traditionally we have seen vertical communication, e.g. in AMR from smart meter at end customer via PLC concentrator and 2.5G mobile network to DSO backoffice center (billing), or in SCADA from substations to DSO management center. Wider spread of AMI, DER, and automation at substation nodes (e.g. deployment of PMU or WAMS nodes) forces us to rethink fundamentally communications network organisation for future smart grids, since there is an emergent need for connectivity on the horizontal level of the same types of smart grid nodes (Figure 3). Any these may selectively be also vertically connected to reach DEMS (local). These changed, or additional patters of communication must be considered when designing the reachability of node types (e.g. among substation on the same level) with MPLS, wireless solutions and providing backup paths, as well as assigning QoS (e.g. different traffic types for inter layer vs. intra layer communication). To cope with reliability requirements (e.g. failover redundant paths) and incorporate multiple heterogeneous communications solutions (e.g. fiber, 4G, WiFi) we may consider reorganizing to achieve direct node-node communication on each level, i.e. among WAN nodes and among NAN nodes. This is very much like state of the art telecom networks where nodes on the same layer are partially physically interconnected for redundancy reasons (e.g. RFC3619, RPR) and/or logically, too (e.g. MPLS VRF).

Figure 3: Smart grid communication patterns today are vertical, lacking horizontal communication among same node types.

2.4 State of standardisation in smart grids

Standardization is vital to achieve compatibility, interoperability, safety, quality of future smart grids. This is best illustrated by referring back to Figure 1 and Figure 2, where we want to achieve seamless interoperation among different stakeholders as well as between different levels of the smart grid system. Standardisation is the answer for wide acceptance of new technologies and solutions and will play a key role in massive deployment of prosumer installed DER and their automatic participation in demand response programs tightly integrated with distribution management systems to achieve required impacts (e.g. automatic demand response information flow between DEMS and user’s home appliance).


2.4.1 Standardization bodies

IEC is the most important worldwide organization influencing publication of all electrical, electronic and smart grid standards [IECst 2014]. CENELEC is responsible for standardization in the electrotechnical engineering field on the European level [CENst 2014]. IEEE is the world's largest professional association focuses on communications OSI level and smart grid standardisation [IEEEst 2014]. DLMS works on common language, object model and messaging methods for automatic meter reading and demand side management [DLMSst 2014]. IDIS (http://idis-association.com) (Interoperable Device Interface Specifications Industry Association) was established to maintain and promote publicly available technical interoperability specifications based on open standards and supports their implementation in interoperable products. IDIS is an association for smart metering companies which are committed to providing interoperable products based on open standards. Its current members include Elster, Iskraemeco, Itron and Landis+Gyr [IDISst 2014]. OIML goal is solely worldwide legal metrology [OIMLst 2014], whereas WELMEC is its European equivalent [WELst 2014].

2.4.2 Smart grid standardization areas for SUNSEED

Standardization essentially addresses the core architectural, considerations of the smart grid concept, including public safety, grid compatibility, and communication protocols and data security. In SUNSEED we have to address interoperability between DSO and telecom operator communication networks, gather measurements with different reporting periods from WAMS nodes and those from smart meters use the same common information models to facilitate seamless integration and interoperation with key existing DEMS software elements (e.g. database, analytics engine, GIS). Infrastructure standards address equipment specification and the operational interfaces with existing network on customer and grid sides. Relevant smart meter design standards are interesting for our WAMS node design in order to achieve the same level of measurement performance: IEC 62052, IEC 62053, IEC 62054, IEC 62058, IEC 62059, IEC 50470 and IEC 62056 series. Relation of SUNSEED to major IEC smart grid standards is noted in Figure 4. The depicted standards will influence design of WAMS node, databases, user applications for DR, field trial, information type and transmission implementation between field trial nodes, DEMS and security of the whole composition. Choosing PLC physical interfaces on WAMS must follow related standards in PLC domain (EN 50056) and make certain that designs follow also IEC 61000-4-19, IEC 61000-40-3, CENELEC SC 205A and CLC/TR 50579 to achieve electromagnetic compatibility on power grid installed equipment with other. Smart home automation of end-user of electricity in the home is a key function for a future demand response, providing the opportunity for networks to manage power loads of home appliances, based on agreement that has been developed among electricity retailer, end user and electric utility. This is treated in IEC 14543-3, EN 13321 series, EN 13757 and ISO 16484 series. Demand response interface empowers users with the functionality to manage their demand in a transaction environment and is covered by EN 50090, EN 50428, EN 13321, EN 50491 and ISO 16484 series. Distributed generation of electricity creates a need for development of high level communication protocols to manage interaction of these systems with the national grid (IEC 60904, IEC 62257, and IEC 62446).

http://idis-association.com/


Figure 4: IEC smart grid standardisation map with SUNSEED linkage [IEC 2010].

2.5 Overview of smart grid communication networks for WAN and NAN

A smart grid communication network sits in the middle of the smart grid architecture. It provides interconnection between the physical electric power generation and supply network and the intelligent information processing systems. The hierarchical structure of the three main components of the smart grid communication networks, namely home area network (HAN), neighbourhood area network (NAN) and wide area network (WAN) are illustrated in Figure 5. In this section we provide a short overview of the state of the art (SOTA) in NAN/WAN communications. Since home area networks (HAN) are not going to be utilised in SUNSEED, they are not treated below.

Figure 5: Smart grid WAN and NAN communication networks.

AMI Headend

DAP

DAP

Home Area Network (HAN)

Neighbourhood Area

Network (NAN)Wide Area Network

(WAN)

EVSubstation LAN

Industrial consumer/prosumer LAN

Transmission network

Bulk

generation

Distribution network

Communication links

WAMS

Operation control/

data centre


2.5.1 Neighbourhood Area Network (NAN)

A smart grid NAN is deployed within the distribution domain of the grid, i.e., it forms the communication facility of a power distribution system. NANs offer the capability of monitoring, controlling and responding to the dynamic operational conditions of the grid, as well as the energy demand of consumers. NANs directly connect all the end users in regional areas, forming the most important segment in power grid that can determine the efficiency of the whole grid [Meng 2014]. NAN will play significant role in dense DER deployment within distribution grid (Figure 3) to interconnect DER in a region (horizontal, on the same level communication) and to enable their monitoring and control with DEMS (vertical, between layers communication). The advanced instrumentation technology enabled by real time sensing and data communication will be the most important interface of the power grids for monitoring the demand and supply. For this purpose, the advanced metering infrastructures (AMI) have been proposed to gather and convey (near) real time raw measurement data. Communication network is required for bidirectional information exchange in smart grid NANs. The NAN technologies include various current wired and wireless technologies, such as broadband and narrowband power line communications (PLC), wireless sensor networks (WSNs), wireless local area networks (WLANs), and wireless mesh networks (WMNs). The NANs of AMI communications infrastructure are particularly suitable for wireless deployment, largely due to the ease and low cost of adopting wireless instead of wired solutions. The backhaul network connecting the AMI head-end and the data aggregation points (DAPs) can either be wireless or wired [Zhu 2012]. The link between the DAPs and consumers requires NANs with coverage in the range of thousands of meters. Each DAP can connect to hundreds of smart meters (SMs). As a result, a key requirement of candidate wireless solutions is coverage of wide area, which can also be achieved through mesh network architecture or relay stations. Additionally, the wireless network must be able to provide a certain level of reliability as well as low enough latency not only to satisfy demand side management (DSM) requirements but also to serve all other AMI applications. According to communication requirements from OpenSG [OpSG 2013], this translates to a minimum reliability figure of 99.5 % and a latency requirement of less than 1 second, which is a relatively relaxed figure as compared to the commercial broadband requirements.

2.5.2 Wide Area Network (WAN)

Considering the monitoring of the whole power grids, the current Supervisory Control And Data Acquisition (SCADA) and Energy Management Systems (EMS) both have limitations in the ability of fast monitoring/responding to the dynamic system conditions and time synchronized measurements. The recent Wide Area Measurement System (WAMS) development enables advanced wide area monitoring and control abilities in the smart grid. A WAMS generally consists of Phasor Measurement Units (PMUs), a fast communication network, a central data gathering and analysis system [Kim 2013]. A PMU can measure GPS time synchronized basic data with phasor information, in a relatively high data sampling rate of tens of samples per second. The reader is referred to IEEE C37.118 [IEEE37 2014] and the IEC 61850 [IEC850 2014] standards for more details on the parameters of the PMU and the data message format. WAN is the data transport network that carries the operational data of the grid to central control centers. It requires reliable, high capacity technologies to enable data transmission for the WAMS, handling big data collected from different NANs, industrial consumers, central generations, etc. While the NAN is a more distributed network for local deployments, the WAN will play a centralised role in order to support communication within the whole geographic area of the grid.


WAN technologies span from cellular networks, WiMAX, to wired communications. The aggregation point between NAN and WAN, such as a substation, data concentrator or a RF access point is responsible for collecting the total metering information and send it to the backhaul communication network. Fibre and microwave communication are preferred for high-bandwidth requirements and reliable communication. Broadband power line communication technologies can be used to transfer data from aggregation points to the data centers.

2.5.3 Enabling technologies for WAN and NAN

WiMAX — Implementations of IEEE’s 802.16 standard for metropolitan networks, commonly referred to as WiMAX (worldwide interoperability for microwave access), is a leading candidate for providing connectivity between DAPs and SMs. WiMAX is based on orthogonal frequency division multiple access (OFDMA). The NANs are particularly suitable for wireless deployment, largely due to the ease and low cost of adopting wireless solutions. Although WiMAX is not being widely adopted as a wireless broadband platform, it does not diminish its chance of being a candidate as some utilities are expected to set up dedicated DAPs. As a result, WiMAX is more attractive in the sense that its structure is much less sophisticated as compared to rival cellular standards such as LTE. Additionally, multihop relay capabilities have been added to the standard to enlarge the coverage area using low cost relay stations. UMTS/LTE— Current cellular technologies such as UMTS and LTE also provide attractive solutions for providing NAN coverage. Relaying functionality has also been incorporated in LTE-Advanced, which will allow extended coverage using relay technology. However, the utilities have to be willing to overlay DAP-SM communications over existing communication infrastructure. Although the advantage of overlaying is a lower setup cost since the existing infrastructure can be used, the utility operator will have to work with the telecommunication operators to set up the network which can be contentious due to security and privacy concerns. WiFi — WiFi is the most commonly deployed standard for wireless local area network. As such, WiFi devices and chips are relatively cheap, making it an attractive solution. Various recent improvements of the IEEE 802.11 standard family have been providing significant advanced features [IEEE11 2014]. For example, IEEE 802.11s is to support frame delivery and route selection at MAC layer through radio-aware metrics. The PHY layer of IEEE 802.11s is compatible the existing PHY layer of IEEE802.11, so that high capacity data transmission will be ensured. In this standard, wireless devices can be connected with one another to form a self-configuring multihop wireless mesh network. In terms of the improvements at the PHY layer, comparing to the IEEE 802.11n, IEEE 802.11ac has proposed to use wider channels (80 or 160 MHz vs. 40 MHz) in the 5 GHz band, more spatial streams (up to 8 vs. 4), higher order modulation (up to 256-QAM vs. 64-QAM), and the addition of Multi-user MIMO (MU-MIMO). As of 2013, implementations have achieved a data rate of 1300 Mbit/s total in the 5 GHz band, under the support of 80 MHz channels, three spatial streams, and 256-QAM [6]. Currently, the IEEE 802.11ax working group is aim to develop a high efficiency WLAN, providing 4x the throughput of 802.11ac and in particular to support dense deployment scenarios. In addition, IEEE 802.11ad defines a new physical layer for the future 5G wireless networks to operate in the 60 GHz millimeter wave spectrum. WiFi will have the potential to become a suitable candidate for reliable and high-speed wireless NANs. IEEE 802.22 — An alternative candidate to mainstream broadband wireless is the IEEE 802.22 wireless regional area network, which uses white spaces in the television spectrum. The IEEE 802.22 standard proposes to use cognitive radio technologies to exploit unused spectrum in the frequency


spectrum allocated to television broadcast. As the spectrum used is not dedicated, the latency in data transmission could be higher as compared to other solutions mentioned earlier. SAT – In past a costly solution, satelite communication has now become a feasible technology for covering an ever-growing number of backhaul deployments, particualry in rural environments where alternatives are impractical or even not possible, e.g. without cellular coverage or fiber/cooper infrastructure, since SAT can offer truly global coverage. Technology itself is based on communication to low Earth geosynchronous satellites (LEO) and it requires a line of sight in order to operate. Although the propagation delay is largest compared to other solutions (around 600 - 1000 ms), it has moderate capabilities if truly real-time performance is not required. Also, SAT equipment can be used as redundancy link to existing communications and serves as backup, it since does not depended of neighbouring communication grid. PLC — Power-line communications systems operate by adding a modulated carrier signal to the wiring system. The advantage of PLC over wireless technology lies in communications in highly urbanized areas where attenuation losses are high. In addition, using the existing power line infrastructure as a medium for data communications, PLC communications would incur no external costs, as comparing with cellular technologies. PLC can be divided into two large classes: narrowband and broadband. Narrowband-PLC technologies offer the advantage of being able to communicate through transformers. Lower frequencies are able to cross the transformer more effectively than the higher frequencies utilized by broadband-PLC. Narrowband-PLC can be utilized for longer distances on the grid without the need for repeaters. The installation of repeaters can increase the cost of deployment substantially, which may be against the original intention of using PLC. Although narrowband-PLC has very limited data rate, it is sufficient for the frequent transmission of metering information, time-of-use pricing and control signals from the operation side of the grid. The IEEE 1901 standards provide detailed description of PLC in medium/low voltage lines [IEEE19 2014], which has the potential application in the NANs. In Slovenia, the implementation of PLC systems in the AMI has been successful. This will be further discussed in the following section. Summary of technical characteristics relevant for NAN and WAN in smart grids is in Table 1.

Technology Data rate Coverage

Reliability / Latency

Cost of deployment

WiMAX 219 Mb/s DL 140 Mb/s UL (20 MHz TDD 4x4 MIMO)

Tens of km

High / Low

Medium

UMTS/HSPA (3G)

21,6 Mb/s DL 5,76 Mb/s UL

1-2 km (widely deployed)

High / Low

High (must lease services from cellular carriers)

LTE (4G) 300 Mb/s DL 75 Mb/s UL (UK: 20 MHz FDD @ 1.8/2.6 GHz bands) 800 MHz bands for rural access

1-2 km (under fast deployment) > 10 km for 800 MHz

High / Low

High (must lease services from cellular carriers)


bands

WiFi 54 Mb/s (802.11n 20 MHz) 300 Mb/s (4x4 MIMO)

20-200 m Medium / Medium

Low

IEEE 802.22 19 Mb/s 30 km Medium / Medium

Medium to high

SAT < 1 Gb/s DL < 10 Mb UL

global High / High

High

PLC Narrowband: < 500 kHz MV (< 72 kV)/LV wired area

Medium / Low

Low

Table 1: Comparison of candidate communication technologies for WAN, NAN.

2.5.4 Interoperability issues

A key feature of smart grids is the interconnection of a potentially large number of disparate energy distribution networks, power generating sources and energy consumers. The components of each of these entities will need a way of communicating that will be independent of the physical medium used and also independent of manufacturers and the type of devices. Multiple communication technologies and standards could coexist in different parts of the system. For example, a fibre WAN may contain a number of WiMAX NANs and cellular NANs. A cellular NAN further consists of a number of WiFi based HANs. To this end, interoperability is essential for the communications architectures supporting smart grids. It can be envisaged that in such a complex system, heterogeneous communication technologies are required. Instead of focusing on or defining one particular technology, it is more important to achieve agreement on usage and interpretation of interfaces and messages that can seamlessly bridge different standards or technologies. In other words, one of the main aims of communication standardization for smart grids is ensuring interoperability between different system components. In this context, generic application programming interfaces (APIs) and middleware are useful enabling technologies. The success of commercial deployment of smart metering and smart grid solutions will significantly depend on the availability of open and standard mechanisms that enable different stakeholders and vendors to interoperate and interface in a standard manner. Open interfaces serve many purposes and provide additional benefits in multi-stakeholder scenarios such as smart energy management in home and industrial environments. Further, open APIs provide the means for third parties not directly associated with the original equipment manufacturers to develop a software component which could add functionality or enhancements to the system. On the other hand, smart energy management solutions require access to more information, ideally from different service providers and devices implemented by different vendors. Such information should be available and presented in a usable format to interested parties. Timing and specific configuration of measurements and controls are also critical for dynamic scenarios. Since support for different technologies and some level of cooperation over administrative boundaries are required, proprietary or widely simplified interfaces will not be sufficient in these scenarios. This situation can be improved by standard generic API definitions covering methods and attributes related to capability, measurement and configurations. The design of such APIs should be technology agnostic, lightweight and future-proof [Fan 2013].

2.5.5 Role of PLC in smart grids today

Power line carrier (communication) is today the primary communication solution in NAN, from smart meters to DEMS via PLC concentrators connected wither via fiber or GPRS. In SUNSEED we intend to


use PLC as a smart meter communication technology and will investigate its use as a secondary interface (for failover purposes) on WAMS node, to provide significantly more data bandwidth compared to traditional smart meter PLC (1 reporting/day of 15 min measurements). Communication via the electrical wiring has been used for a long time for management of certain functions in HV grid. With the introduction of smart metering the system is gaining ground in low voltage distribution networks for measurement, control and remote data collection on electricity consumption in the residential areas [EC 2012]. Major benefit of contemporary PLC systems is that the network is already setup, thus greatly lowering the marginal cost of communication solution in NAN for DSO. All major building blocks of existing smart grid support PLC system. PLC is also used in newer concepts of smart house management system.

Figure 6: Smart meters with PLC and communication via PLC concentrator to DSO.

Smart meters have by default built-in PLC communications module to exchange information with PLC concentrator that is typically installed in the LV transformer station. PLC concentrator’s role is to search for potential new smart meters, to record them in the registry, and requires them to provide the needed information and execute commands. Collected data from all smart meters are sent once a day or on demand request via GSM/GPRS mobile network or Ethernet/IP connection to DEMS. This arrangement is primarily used for billing purposes of electricity flow. PLC communication is a relatively new system, and numerous development activities are still in place. Therefore, standardization and regulation have not yet been completed, and various different systems, tied to their own protocols and main carrier frequencies of meters manufacturers, are installed in the network. Smart meter PLC uses 1200 or 2400 Baud physical transmission speed with IDIS protocol [IDIS 2014]. Recent PLC physical transmission adopts also PRIME standard that has much better interference imunity due to OFDM implementation [Prim 2014].


Figure 7: PLC signal with IDIS protocol and frequency bins at 65 kHz and 71 kHz.

The purpose of smart meters is not to fulfil the demands for switching on and off power plants, regulates reactive power and voltage amplitude, or use reservoirs for the needs of primary control of power in real-time. PLC communication with smart meters is primarily designed to transfer accounting data on the consumed energy as well as certain additional parameters. Having separate frequencies for the transmission of own signal does not make sense, because the future technologies (PRIME) will use the entire frequency space for themselves. Further restrictions may be interferences that can occur in a network of arbitrary private consumers, over which the distribution has no control.

Figure 8: PLC signal according to PRIME, measured PLC concentrator.

2.6 DSO and telecom operator communications infrastructures

Telecom operator has been the traditional communications service provider. On the other hand TSO and DSO established their own communication networks to monitor and control transmission and distribution power grids. Telecom operator started cooperation with TSO and DSO either to offer backup communication paths or connect smart meters (via PLC concentrators) to DEMS. In this


chapter we look at contemporary communication infrastructures used by TSO, DSO and telecom operator. Some details describing implementation itself and use of specific network element types shows the capabilities and enables reader to further explore detailed features. This will lead us subsequently to define communication and interconnection requirements in detail and set up the converged DSO-telecom operator communication solutions in WP3.

2.6.1 DSO communication networks overview on example of EP

Although the approaches and standard recommendations for designing and implementation of SCADA and AMI communication networks within TSO and DSO are established and examples well known, we present an implementation detailed view of a communication network within medium distribution system operator (< 200 K end nodes) [Birm 2005], [Gung 2006], [Hopk 2009], [LeM 2008], [McDo 2003], [McGr 2005], [UCA 1999], [Uci 2014]. Such overview provides more data for a credible economic analysis and CBA of various options that lead to converged DSO – telecom operator communication solutions (Please refer to Chapter 5). DSO communication networks were designed and built after establishment of TSO communication networks. Smaller DSO can cost optimize communication networking such that regions of HV/MV are directly connected to TSO operated communication network, and DSO only manages communication networking on the lower levels. In the example of EP the IP/MPLS network was designed by Slovenian national TSO (ELES) (Figure 9). For maximum TSO-DSO interoperability the communication equipment is standardized. The basic concept of building the network follows the design principles of telecom operators and equipment vendors and/or integrators. Design is focused on heavy use of optical fiber from substation level upwards to meet DSO and all electric power sector general needs. Most of further description is based also on EP communication networking solutions. All DSO users are included in the Ethernet aggregation network formed by the powerful Ethernet switches that are interconnected in 10 GE and 1 GE rings. With 1 GE links they are also linked to the IP/MPLS routers to integrate with the IP/MPLS core. Ethernet aggregation network ring structure provides the conditions for high availability services in the network. For that reason it is at the level of L2 used protective mechanism of fast Ethernet switching on redundant links in accordance with RFC3619 recommendation to ensure convergence time < 50 ms, in the case of failure of the transmission paths for services in L2 Ethernet aggregation network. The same mechanism is used in both 10 GE rings, as well as regional 1 GE rings and it has proved to be extremely effective and appropriate substitution to the well-known and effective mechanism ring network security SDH. Ethernet aggregation network architecture is designed in simple and arbitrarily spread manner to meet the need of increasing network capacity, the addition of nodes and the reconfiguration of the network as a result of the constant demands of users for new or changed services.


Figure 9: DSO – TSO communication network topological partitioning.

IP/MPLS network concept is the primary solution for integration with neighbourhood networks, to TSO, other DSO #2 – DSO #5 and service providers, both from energy and telecommunication sectors. Figure 9 shows the topology of an integrated IP/MPLS network - IP/MPLS core network and Ethernet aggregation network. Routers IP/MPLS are interconnected with GE and STM-1 line interface directly over optical fiber and over SDH or WDM transmission systems. STM-1 coupling links are used to protect the primary 1 GE coupling connections between routers. The core IP/MPLS consists of three rings - western, eastern and southern. Ethernet aggregation network form a powerful Metro Ethernet switches that are interconnected in a ring structure in order to ensure conditions of high availability services in the network. The EP area is topologically covered with Ethernet network aggregation western 10 GE ring. In the present IP/MPLS communication network the following equipment is installed:

IP/MPLS core: IP/MPLS routers (e.g. Juniper M series).

Ethernet aggregation network:

o Core switches: Metro Ethernet compliant (e.g. Extreme Networks Black Diamond series)

o 1 GE and 10 GE aggregation rings: Metro Ethernet compliant (e.g. Extreme Networks Summit series)

For the DSO purpose there is no need to use SDH network, since no such requirements exist (e.g. extreme low latency, failover ring topology). However, TSO requires SDH network for distance protection in power lines closed-loop operation. We have to note, that future DER installed in


distribution grid will require SDH like capabilities (e.g. predictable delay, fast switchover to backup path) to meet the power distribution challenges that were up until now only seen in transmission part of power grid (e.g. islanding, looping). Each DSO requires additional communications capabilities to assure both its own business operation and of its clients (typically billing from electricity metering), as well as realise control/command from DEMS to power grid. Typically the number of nodes in control/command increases from DEMS (one or two, geographically distributed) to 10 s or 100 s for transformer stations (HV/MV, MV/LV) and distribution stations. Business operations communication network part interconnects several DSO business locations and smaller local service and maintenance centers. As an example of EP, core network includes two geographically separated, but functionally equivalent locations with primary and secondary data centres connected with LAN links (e.g. 10 GE bundled in Etherchannel) (Figure 10). Interconnection with remote locations can be accomplished with different technologies and, most importantly with different communication network providers. This depends on bandwidth and availability requirements within DSO region. Core network is MPLS and virtualized using VPN, or additional security may be provided by IPsec. Remote locations are equipped with L2/L3 routers. Secure communication to remote locations is performed using IPsec encrypted GRE tunnels, in case of geographic redundancy, multiple are used (e.g. with dynamic multipoint VPN). OSPF protocol is used for IP routing elsewhere. Communication between the services is controlled over central firewall with rules and filters. To achieve levels of QoS and service separation, a MPLS L3 VPN with per service VRF is established and BGP is used for label distribution. Core communication equipment is not only placed in redundant locations, but is itself redundantly built (e.g. dual control and communication modules) to achieve high availability levels.


Figure 10: DSO network topology: logical partitioning.

2.6.2 Telecom operator communication networks overview

The main communication network of the contemporary telecommunication operators is based on MPLS technology with cost effective 10 GE or 100 GE Ethernet transport interfaces. It is usually topologically partitioned into several levels, covering different regions. Telecom operator communication network may have a three layer hierarchy:

Core level is composed of core P type routers (e.g. Cisco CRS-3, Juniper T1600). Core routers

are interconnected in a fully hardware redundant configuration and located at distinct

geographical locations > 100 km apart.

Regional aggregation level is built with PE type routers (e.g. Cisco ASR series, Juniper MX

series). RAR are geographically split into several geographical regions. Care is taken to dual

home each RAR to core routers to achieve link protection (failover redundancy).

Aggregation level is comprised of L2/L3 routers (e.g. Cisco 7600 series, Juniper MX series)

that function as multiservice platform routers. Each is dual homed to upper RAR within the

same region, too.

As an example in Figure 11 we can have topology of aggregation IP/MPLS network divided into two levels (with extra core level shown in Figure 12). Top level is Regional aggregation layer, and lower level is Local aggregation layer. Each is comprised of L3 routers, either regional aggregation (RAR) or local aggregation (LAR) types.

DRC

MPLS

HQ

Mobile WANxDSLOptical

(Ethernet)

Remote Location 1

S3S2 S5S4S1

Remote Location 2

S3S2 S5S4S1

Remote Location 3

S3S2 S5S4S1

SiSi

LAN

CORESiSi

LAN

CORE

Service 1

Service 2

Service 3

Service 4

Service 5

Primary MPLS

aggregation

Secondary MPLS

aggregation

FWFW


Figure 11: Telecom operator communication network topological partitioning.

POP nodes are located in regional aggregation domain with dedicated RAR routers. Traffic aggregation from lower level LAR routers is via multiple 10 GE links. Each LAR has redundant dual upstream connections to a pair or RAR within the same region. The LAR provides access for business and residential customers. POP nodes are interconnected by 10 GE links over dark fiber or DWDM systems. The services provided by the telecom operators are centrally injected into the network through the Service delivery domain POP (SDP). The SDP is usually comprised of redundant pair of high capacity routers (throughput, sessions) located in geographically redundant locations (Figure 12). If there is a lot of unicast traffic (e.g. due to VoD, replay TV), it can be injected locally towards the end users at RAR or LAR domain. Home subscribers are authenticated through geographically redundant BRAS domains, consisting of suitable broadband remote access servers used for PPPoE/IPoE sessions establishment and termination. Each region may have dedicated BRAS server. User data are accesses via Radius from LDAP database. User profiles are dynamically assigned. Peering domain handles or IP peering and IP transit traffic with national ISP, ASP and international ISP and other peering partners. It usually consists of geographically redundant configuration of high throughput routers used as MPLS aware nodes with MPLS services and also used for connecting the business customers for direct IP peering or transit.


Figure 12: Telecom operator communication core network with different domains.


3 Selected smart grid use cases for SUNSEED

The selected three use cases are investigated that cover the problem and design domains that we see will arise in future smart grids. Furthermore, SUNSEED field trial set up and operation will be designed to reflect these use cases in practical operation, albeit on smaller scale, to investigate key development challenges, e.g. density of smart grid nodes in mobile environment, M2M data traffic with QoS mechanisms, cost effective convergence of DSO and telecom operator communication networks [EC 2014], [Eur 2013]. Transition of electricity users to prosumers and active participants in smart grid is already taking place and much more in the near future [EC 2012], [EC 2013]. Massive user participation for demand response will change the control strategies in distribution network and will stress demands on two way communication between prosumers, DER and DEMS.

Use case 1: A dense network of measurement nodes, smart meters and WAMS, not only on selected substation locations, but rather on all DER nodes and majority of prosumer locations will greatly enhance distribution grid observability.

Use case 2: Real time communication and time synchronous measurements (from 1 s to 15 min reporting intervals) is the stepping stone towards advanced distribution management system with capabilities up to date seen only in transmission grid (e.g. voltage profile from state estimation), as presented by the advanced distribution network management system platform use case.

Use case 3: Observability of key electrical parameters distribution grid must be accompanied with the same level of observability of measurement nodes and potential failures in electrical and/or communication grid. Use case outage management with fault identification and localization via communications networks behaviour studies the combined use of converged communication and power networks to pinpoint the type and location of failures.

Use cases are examined in a template form, similar to IEC smart grid use case forms and as used in other industrial sectors (e.g. software development) [IECSGS 2014]. Their scope is chosen rather broad, so that we have room to apply for each use case a subset of scenarios that focus on narrower problem types (e.g. the massive outage in distribution grid scenario within the outage management with fault identification and localization use case).


3.1 Massive prosumer participation for demand response (MPPDR)

Use case name Massive prosumer participation for demand response (MPPDR)

Use case no. UC-1

Use case group Market facilitation

Use case type High-level

Objective in context

Investigate when a large fraction, (e.g. 30 % to 80 %) of today’s electricity consumers, become prosumers, i.e. with their own energy generation capability, connected to power grid the possibility to balance the electricity supply and demand.

Functions / narrative

Involve all prosumers connected to the smart grid (e.g. via VPP concepts or individually) in the balancing of electricity supply and demand in real-time. Today consumers are either passive energy consuming entitles on distribution grid network or are prosumers with installed DER, but they not participate actively in DR programs with DSO or retailer. In the future smart grid prosumers are active participants in DR programs. Retail company sales electricity and possibly other innovative services to prosumers enabled with advance communication network and real time responsiveness of smart grid measurements and control, as compared to non-real time meter reading, even in non-smart meter environments (Figure 13). Under present legislation DSO is responsible for energy measurements for billing purposes, to retailer. Legislative changes can enable VPP formation and their offering of ancillary services influencing quality of electrical grid (e.g. voltage regulation, lowering losses and congestion). Effective DR depends critically on demand management and price/load/renewable energy forecasting, which requires optimization, analytics and automatic process control methods. Future smart grid will have many prosumers and the use case will address what functionality is needed for connecting these prosumers to the grid as well as how to facilitate their communication needs for successful balancing of electricity supply and demand. The relevant technological solutions that will enable the operation of the use case are: WAMS nodes at selected prosumer locations for monitoring & control of DR loads, thorough GIS assisted description of households electricity production, demand, and energy storage status and capability, supply-demand balancing algorithms and forecasting (e.g. based on meteorological data), demand forecasting (e.g. based on time-scheduled events), active communication with loads and storage (e.g. on/off, or protocol).

Actors Energy retail company, DSO, telecom operator, end users residential and business, energy exchange market, regulatory bodies.

Preconditions Smart meter installation at each prosumer and consumer, that participates in DR programs (Figure 14).

AMI infrastructure for billing purposes.

Real time communication from each prosumer for effective DSM (e.g. < 15 min).

Knowledge of production, demand, and electricity storage status and capability for prosumers connected to the smart grid.

Demand forecasting algorithms (e.g. based on time-scheduled events, meteorological data, historical electricity usage/production data)


Remote control of distributed electricity resources, electricity consumers (i.e. appliances), electricity storage.

Scope Smart grid architecture and design for dense prosumer cases.

Quantification criteria

Quantification of mismatch between electricity supply vs. demand, lowering of peaks of supply vs. demand

Security of Supply, System Average Interruption Duration Index (SAIDI), Energy Not Supplied (ENS), Average Interruption Time (AIT), reliability, availability, min max of RES/DER integration potential within certain energy grid

Quantification of security bridge potential

Trigger Market electricity price drives prosumers to change their load profile.

Mismatch (either real-time or forecasted) in electricity supply and demand.

Success end condition

Smaller (or non-existent) mismatch between electricity supply and demand.

Operating parameters of energy grid on the same level as with centralised generation.

Lower electricity supply and demand peaks

Lower electricity loss due to transport in MV/LV grid

Failed end condition

Energy network not operational, unstable due to electricity supply vs. demand mismatches, faults, loops, operating parameters.

Communication network inability to service increased M2M number of end nodes (i.e. WAMS nodes), sessions, bandwidth and fault tolerance.

Occurrence Due to large scale RES and DER deployments balancing will be become norm on < 15 min basis (as is today on the advanced FRR-A markets) to realise potential and coherent and stable operation of distributed resources.

Priority to smart grid

Very critical in years to come, since smart grid is inevitably becoming “crowded space” due to larger RES and DER sources within energy network, EV charging station deployments, EU proposals for three binding targets by 2030: 40 % cut in greenhouse gases (compared to 1990); > 30 % of energy to come from renewable sources; 40 % improvement in energy efficiency. Dense smart grid will have implications on the structure of power grid, particularly changes in distribution grid observability, monitoring and control leading also to redefinition of communications network infrastructure to support it.

Priority to SUNSEED

High. Affects pilot construction and site selection, reengineering communication networks cases. Central to the communication network design of unified DSO-telecom operator solutions.

Open issues and risks

How to cope with security issues? Number of nodes in smart grid rises dramatically, where each node can be potential target for DDoS or on site tempering and vulnerability injection. This is especially true with user in-house installed WAMS devices that facilitate DR programs.

Table 2: Use case UC-1: Massive prosumer participation for demand response (MPPDR).


Figure 13: UC-1: Power grid with consumers, no smart metering and low DER penetration.

Figure 14: UC-1: Smart grid with prosumers, AMI and high DER penetration.


3.2 Advanced distribution network management system platform

(ADNMSP)

Use case name Advanced distribution network management system platform (ADNMSP)

Use case no. UC-2

Use case group Monitoring and control of distribution grid operation

Use case type High-level

Objective in context

To allow integration of higher number of prosumers into the power grid network without additional reinforcements (e.g. installation of new infrastructure such new lines and transformers) the system operator requires tools for closer monitoring of the grid, especially in the mid (20 kV) and low voltage (0,4 kV) parts of the grid. Currently the grid operator controls the grid operation only through voltage and current measurements in main supply substation (110/20 kV) on the busbars and feeders (Figure 15). The main reason for lack of measurements deeper in the distribution network through the feeders is not integration of suitable measurement equipment, but rather in insufficient DSO communication network that do not fulfil the next generation network operational requirements for smart grids.

The new approach with DSO and telecom operator converged communication network enables a huge possibility to build cost effective and technologic multifunction power distribution monitoring and control system to improve better performance of network operation as the basis towards smart grid network operation concepts. Thus the use case for advance distribution network is proposed. A detailed description of the use case is given bellow.

Main objective summarization:

Improve operation conditions in network with high penetration from DER and active consumers with demand response.

Delay or avoid reinforcements due to better utilization of the current grid.


Increased concentration of distributed energy resources (DER) causes changes in the network power flow which converts from traditional one way to both directional ones. On the other hand more and more consumers are proactive considering demand response to optimize their monthly energy cost. Both processes are not correlated in general. Consequently the prescribed limits of voltage violation and in the lesser extend the congestions in the network could occur. There are two approaches to solve the described operational troubles. First is traditional and leads to necessary network reinforcement and the second one is employment of new advanced DEMS functions (e.g. voltage profile, state estimations, congestion managements, load forecasts, KPI monitoring) which manage and integrate the network geographical and topological data from GIS like databases and operational data (SCADA) from measurement nodes more deeper to the prosumers side in the network. With development of novel power protection concepts there is also a possibility to operate in close loop network (like in transmission network) which means that the impedance of the network is reduced by half and thus this higher integration of new prosumers into the network is possible.


Actors DSO, telecom operator

Preconditions Reliable communication infrastructure allowing high capacity and real time communication (Figure 16).

Integration of network information system (geographical, topological and network elements attributes database) with SCADA (operational data) and CRM (prosumers and other customers data).

Scope Monitoring, identification and control of critical operational condition in middle and low voltage network.


The major KPI for proposed use case are measured and estimated voltages and currents (also others basic power parameters aggregated basic ones) alongside the MV and LV network.

The quality of other KPI significantly improves.

Traditional KPI: Security of Supply, Power Quality Index (standard EN 50160), power losses, network investment costs.

Novel proposed KPI:

Relation between total feed power amount from connected DER and consumer load in the supply feeder.

Measurements of resiliency which point to ability of network to operate in case of outages of supply transformers and main power lines.

Trigger Alarm of risk operational state (e.g. approaching voltage or current limits) in DSO control center. Otherwise the ADNMSP runs continuously in real time to make simultaneous calculation and furthermore verification of the operational state parameters (e.g. voltage, current, active and reactive power).


Implementation and integration of system state estimator to obtain current and short term forecasted crucial power parameters (eg. node voltages, branch currents):

On the state estimation basis continuous network observation. Risk operation identification through state estimation prediction. Network management by DSO.

Measuring of typical electrical parameters (e.g. voltage, current, power, power factor), reliability parameters (number of outages, duration of outages), power quality parameters (voltage THD, harmonics, sags, rises…) with universal WAMS node devices.


Risk operational state not identified and/or located and/or managed and/or network observing failed by DSO.

Occurrence Continuously in real time by DSO in network operation control center.


Very high. It is powerful and consecutively indispensable tool for new smart grid operational concept.

Priority to SUNSEED

Very high. With this tool the SUNSEED could efficiently confirm the main goal of DSO and telecom operator converged communication infrastructure.

Open issues Consideration of power distribution SCADA integration protocols.


and risks

Minimal number and characteristic location of WAMS nodes in low voltage grid.

Table 3: Use case UC-2: Advanced distribution network management system platform (ADNMSP).

Figure 15: UC-2: Classic operation approach in distribution grid.


Figure 16: UC-2: Advanced operation and control management approach in distribution grid.


3.3 Outage management with fault identification & localization via

communications networks behaviour (IFIL)

Use case name

Innovative fault identification and localization using external (non-grid) data sources (IFIL)

Use case no. UC-3

Use Case Group

Monitoring of the distribution grid

Use Case Type High-level

Objective in Context

Enhance the reliability of power system service through improved identification and localization of power outages (faults) through monitoring of telecommunications networks, components and customer premises equipment (CPE) as well as smart meters.


The telecom operator (continuously) monitors data modems (coax/fibre cable, DSL, or wireless) in connected households as well as base-station/DSLAM/coax-cable cabinets in its wireless and fixed access network infrastructure. When a power outage occurs, the telecom operator's modem and base-station/DSLAM/coax cabinet management system discovers simultaneous modem unavailability (‘down time’) for group of households in a particular areas (street, neighbourhood, postal area) or 'down time' of a base-station/DSLAM/coax cabinets from a known street/neighbourhood location. The telecom operator's modem and/or wireless/fixed access network management system generates a ‘down time’ alarm, which together with associated relevant GIS information (list of house addresses and/or base-station/DSLAM/coax cabinet locations), is sent from the telecom operator to the DSO grid monitoring system.

Upon reception of the 'down time' alarm from the telecom operator management system, enriched with GIS information, the DSO grid management system can trigger the Fault Identification, localization and Service Restoration process (FLISR).

Alternatively, the DSO can utilise the smart meters and WAMS nodes in order to collect relevant alarms for identification and localisation of the power outage events. For example, the smart meters of WAMS nodes can issue a 'dying gasp' message at the event of power outage that include also node lactation information as defined in the DSO grid management system. These messages are then aggregated (including GIS data) and processed by the DSO smart meter monitoring system.

Consequently, the DSO grid monitoring system triggers the Fault Identification, localization and Service Restoration process (FLISR).

Actors DSO, telecom operator

Preconditions Telecom operator modem management system (e.g. household IP modem connectivity), access network management system (e.g. base station/DSLAM/coax-cable cabinet connectivity) capable of issuing 'down time' alarms associated with node's location information in case of power outages at the respective telecom network nodes.

Smart meter and WAMS nodes capable of issuing 'dying gasps' messages including node locations in case of power outages at the respective smart grid


nodes.

DSO grid monitoring system capable of receiving 'down time' alarm messages from telecom monitoring system and/or 'dying gasps' messages from smart meters and WAMS nodes and processing the embedded location information.

Grid monitoring system (SCADA): Distribution Operation Model and Analysis with fault analysis, fault location relays (schemes), some Distributed Intelligence schemes and Trouble call system.

GIS/ postal data related to telecom operator network topology as well as DSO's grid topology, smart meter and WAMS locations.

Scope Fault identification and localization in MV/LV electricity distribution grid.


Security of Supply, System Average Interruption Duration Index (SAIDI), Energy Not Supplied (ENS), Average Interruption Time (AIT), reliability, availability, potential for DER observation

Trigger Telecom operator alarm (modem or access network infrastructure management system) or DSO's smart meter or WAMS nodes monitoring alarm into DSO Grid Monitoring System.


Fault identified, located and confirmed by DSO


Fault not identified and/or located and/or confirmed by DSO

Occurrence

Outage of the (part of) electricity distribution grid is typically a rare event. Furthermore, the frequency of power outages also depends on in which part of the electricity grid the outage occurs (HV, MV, or LV segment). As the SUNSEED scope is MV and LV electricity distribution the power outage occurrence frequency is typical once in few (e.g. 2 to 3) years.


High, grid complexity is increasing. FLISR and possibly automated (self-organized) FLISR is required to maintain Security of Supply

Priority to SUNSEED

Medium, could affect the grid's potential for RES/DER integration. If increased RES/DER integration leads to increased risk or occurrence of 'faults' then improved fault identification and localization could lead to improved service restoration, thus maintaining high Security of Supply with high RES/DER integration.

Open issues and risks

Table 4: Use case UC-3: Innovative Fault Identification and Localization using external (non-grid) data sources (IFIL).


Figure 17: UC-3: High level how it works illustration: including telecom operator's modem management system (top) and with smart meter and/or WAMS node dying gasps triggering (bottom).

Operational area (houses/street, postal code)

1, 2, 3 ….N1

Modem

DSO grid monitoring centre

1, 2, ….M

1, 2, 3 ….N2

DSLAM

Telco Modem Management

System

1, ...N3 1, 2, 3 …N4

1, 2, 3 ….N5

Outage area (houses/street, postal code)

1, ...N6

Down-time Trigger

Operational area (houses/street, postal code)

DSO grid monitoring centre

Outage area (houses/street, postal code)

1, 2, 3 ….N1 1, 2, 3 ….N2

Smart Meter

‘’Dying Gasp’’ Trigger

0,4 kV

WAMS node instalation


3.4 Summary of use cases’ findings

Some important findings of what we can expect within distribution grid based on presented use cases are concisely summarised below.

a) Dense network of smart meters, installed per user. They will demand low to moderate bandwidth requirements and data transfers at specific times only via concentrators (billing purposes to customers).

b) Installation of WAMS measurement nodes at prosumers and important nodes within distribution grid. They will demand medium to high bandwidth requirements, real time data transfers, and direct connection via communication network to DEMS.

c) High density of metering nodes (either smart meters or WAMS) is the necessary requirement towards more visible, i.e. observable distribution grid.

d) Observability of distribution grid will enable full control of the same, with large pools of DER. e) Extending observability to communication networks can bring the benefits of enhanced fault

management within distribution grid. f) We can cope with increased information flows and realise active observability and fault

management policies in real time only by relying on advanced analytics methods that will add intelligence, state estimation, forecasts required for stable operation and control of distributed smart grid.

g) Visualisation and presentation of results and state of distribution grid will play major role to fully assist operators of the grid itself.

h) Much higher risks of security vulnerabilities due to increased numbers of communication nodes within distribution smart grid, where each node is potential target for eavesdropping or as hacker injection point.

3.5 LTE simulations of ADNMPS use case

In SUNSEED D3.1, early LTE simulations were presented that were based on generic LTE simulation parameters and traffic models based on literature surveys. Since then a better understanding has been gained both of the LTE network configuration that will be used in the SUNSEED trial and of the traffic model of the WAMS-SPM nodes that are responsible for collecting phasor measurements in the distribution grid, as specified in D3.3.1. In the following we present simulation results corresponding to these parameter values. Furthermore, we show how different variations of key parameters such as the number of available preambles, the RAO (random access opportunity) interval and the report payload size impacts the performance of the LTE network. The simulation study uses the parameter values given in Table 5. In the study, the considered performance metric is the service outage Poutage, which is the probability of a measurement report transmission being successfully delivered through the mobile network.

Table 5: Considered LTE simulation parameters.

Parameter Value (bold is default value)

Bandwidth 5 MHz (25 RBs)

Number of available preambles 12, 54

RAO interval (𝛿RAO) 5, 1

Number of allowed retransmissions 9

Modulation scheme QPSK

Payload size 100, 200, 500, 1000, 1500, 2000, 3000 bytes

Reporting interval 1 sec


3.5.1 Number of preambles

The plot presented in Figure 18 shows how increasing the number of available preambles in a single cell can potentially increase the number of supported WAMS-SPM devices by 25% (from 8001000, assuming each has 1 attempt/sec). It should be noted that this result is based on the assumption of having no neighbor cell interference. In case of co-located LTE cells, where the whole set of 54 preambles is shared, the gain may be less. In this case, a preamble transmission in one cell could lead to collisions in neighboring cells if a device is within hearing range of multiple base stations.

Figure 18: Service outage breaking point for different number of preambles.

3.5.2 RAO interval

Another possible parameter variation, is to increase the amount of resources allocated to the PRACH (Physical Random Access Channel), which is represented by the shorter RAO interval 𝛿RAO=1. This means that the PRACH is assigned in every single subframe. Alternatively, the network could be configured to use less resources on PRACH, for example by only assigning the PRACH in fewer subframes, for example every 5th subframe (𝛿RAO=5). Changing this value allows to support approximately 10% more WAMS-SPM devices until the breaking point is reached where the service outage increases rapidly. This improvement does however not come for free, since the allocation of more resources to the PRACH steals resources from the PUSCH (Physical Uplink Shared Channel), which is used for uplink signaling and data. Since the payload size of 500 bytes does not lead to a shortage of resources in the PUSCH (as will be clear in the following subsection) the allocation of more resources to the PRACH may be worth considering. Especially if only 12 preambles2 are

2 This is a typically used configuration by Telekom Slovenije in their LTE networks. Using only 12

preambles per cell is an advantage in the sense that neighbor-cell preamble collisions can be avoided if non-overlapping sets of preambles are assigned to neighboring cells.

0 200 400 600 800 1000 1200

Arrival rate (attempts/sec)

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

Po

uta

ge

12 preambles54 preambles


available in a cell, meaning that the level of collisions quickly becomes significant as the number of devices increases.

Figure 19: Service outage breaking point for different RAO intervals.

3.5.3 Report payload size

While the size of the measurement packets from the WAMS-SPM nodes is expected to be approximately 500 bytes, we investigate also the performance when using both small and larger packet sizes. The plot in Figure 20 reveals that for measurement packet with sizes up to and including 500 bytes, the service outage breaking point is the same around 800 attempts/sec. For larger packet sizes the number of supported WAMS-SPM devices drops significantly, indicating that the PUSCH (where uplink data is transmitted) is the bottleneck.


Figure 20: Service outage breaking point for different report payload sizes.

100 200 300 400 500 600 700 800 900Arrival rate (attempts/sec)

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

Po

uta

ge

100 bytes200 bytes500 bytes1000 bytes1500 bytes2000 bytes3000 bytes


4 Technical requirements and proposed solutions for future smart

grids

This chapter examines requirements and some solutions that can be used in converged DSO – telecom operator communication networks. We emphasise QoS very much and devote the first subchapter to the topics, since it is essential method to achieve fair treatment and bandwidth allocation for heterogeneous smart grid traffic types (e.g. from fast measurements to voice). WAMS measurement building block must be able to colour traffic that originated or transited from it as well as be able to connect to different types of physical interfaces for enhanced availability. MPLS and time synchronous operation are treated as they are going to be implemented to achieve seamless convergence of DSO and telecom networks and provide much needed observability of distribution grid. Approaches to security from a broader perspective must be taken into account at the design stage already as discussed below.

4.1 QoS overview and typical metrics

Quality of Service (QoS) can be defined as the application of features and functionalities needed to satisfy the networking requirements of sensitive applications which are required to meet SLA targets. With QoS mechanisms we strive to ensure predefined quality for certain data traffic types, services or users, either as guaranteed bandwidth, upper latency bounds, increase network communication path and nodes utilization or assure node access from management centre under all network traffic conditions. Available are combination of different traffic marking attributes (e.g. DiffServ) or bandwidth allocation to achieve these goals. QoS mechanisms can determine, during periods of congestion, which packets are more appropriate to be selectively dropped to relieve the congestion of particular physical link. It is mandatory to enable QoS, as close to end node as possible, i.e. at information source, when there are heterogeneous types of traffic, each with its own SLA requirements and different business or home user customers, to assure appropriate level of service during all times. QoS mechanisms should operate across all levels of communication network. Separation and identification of different types of traffic should only take place as close as possible to the user. For different types of traffic, so called traffic colouring can be done at home or business user CPE device or router itself (e.g. assigning CoS – Class of Service value) and it is carried up to LAR. In SUNSEED we are exploiting QoS to assign different priority types to smart grid traffic, for different applications that share communication network (e.g. VoIP and metering or data centre backup) and different types of smart grid end nodes (e.g. WAMS, PMU, protection, control/command node). QoS in a smart grid is a function of several network physical parameters, such are:

1. Latency or delay: The finite amount of time it takes a packet to reach the receiving endpoint after being transmitted from the sending end point. Telecom operators usually measure round trip time/delay (RTT), but certain services (e.g. VoIP) require measurement of one-way delay, that necessitates time synchronization (e.g. GPS) between end points.

SUNSEED proposed use: To time synchronise measurements from WAMS nodes we implement time synchronisation with high-precision GPS module embedded in each WAMS node. This approach allows for centrally synchronised WAMS measurements. Mobile networks (2.5G, 3G nor 4G) are not suitable for conveying information (e.g. command messages for immediate


action, Type 1, Type 4, and Type 7 [IEC850 2014], protection information external to substation [IEEE1646 2004]) for most demanding low latency tasks.

2. Jitter: The variation or the difference in the end-to-end delay between packets. Setting proper queue depth in upper layers of communication networks and minimizing number of communication nodes on end-2-end path minimizes jitter.

SUNSEED proposed use: WAMS nodes should be connected directly to access and/or aggregation layer of communication network without intermediate multiplexers (as in PLC) to minimize jitter. Mobile networks experience high jitter in communication path node – base station (e.g. > 20 ms in 4G) and are not suitable for conveying information (e.g. command messages for immediate action, Type 1, Type 4, Type 7 [IEC850 2014], protection information external to substation [IEEE1646 2004]) for most demanding low latency tasks.

3. Loss: A relative measure of the number of packets that were not received compared to the total number of packets transmitted. Loss is typically a function of availability. Most packet loss is usually experienced in access layer of a communication network.

SUNSEED proposed use: Packet loss minimization is addressed by higher level protocols (e.g. TCP, message bus), but must be carefully applied since this increase jitter. WAMS end nodes may have two physical communication interfaces (e.g. fiber + 4G, dual mobile, etc.) of different types to switch with respect to pre-defined communication and packet loss metric thresholds.

4. Availability: The fraction of time that network connectivity is available between an ingress point and a specified egress point, and defines network SLA. It directly influences service availability that defined as the fraction of time that service is available between a specified ingress point and a specified egress point within bounds of a defined SLA. One of the goals of DSO and telecom operator communication network convergence is to increase network availability.

SUNSEED proposed use: Use of suitable L2 and L3 protocols to handle actions in case of network outages with different switching times, is mandatory (e.g. Ethernet protection switching [RFC3619]).

4.1.1 Types of services

In IP communication networks we typically use two types of service models. Their application in smart grid is briefly discussed in the following.

1. Best effort: Is a single-service model in which an application sends data whenever it must, in any quantity, and without requesting permission or first informing the network. Best-effort service network delivers data without any assurance of reliability, delay bounds, or throughput. It is usually implemented as first-in, first-out (FIFO) queuing, which is suitable for a wide range of network applications such as file transfer and e-mail.

SUNSEED proposed use: Can be used for non-critical information transport from WAMS nodes, e.g. acquiring status, SNMP messages, and Internet connection in substations with service personnel access.

2. Differentiated service is a multiple-service model that can satisfy differing QoS requirements.

It is a per hop behaviour mechanism, i.e. defined and followed in each router node traversal in communication network. For differentiated service, the network tries to deliver a


particular service based on QoS specified by each packet. Specification can be using IP precedence bit settings in packets or source and destination addresses. The network uses the QoS specification to classify, shape, and police traffic and to perform intelligent queuing.

SUNSEED proposed use: Several marked (colored) traffic types, i.e. different services, for all smart grid types of traffic are defined with this mechanism. Traffic coloring must be done at source, i.e. WAMS node.

4.1.2 QoS operation in a multilayer communication network

It is necessary to separate the QoS functions on every layer from CPE (modem) equipment installed at user home, access to core layer. CPE would first do the traffic marking (Please refer to Chapter 4.1.4) and LAR routers would classify IP packet traffic being received from heterogeneous end nodes and classify packets according to colour/marking, i.e. different traffic types. The core layer can give different QoS treatment based received settings. The aggregation layer LAR routers are responsible for examining IP packets arriving from customer side for various application types (Internet, mobile, VoIP, video, FTP) and destinations of traffic. The packets can then be classified (e.g. IP precedence, according to the SLA agreed with the customer). For example, all traffic from a given customer in the telecom network can be given a certain classification, which would mean it would be given higher priority over all other traffic types with lower priorities. LAR also provides ingress bandwidth management and appropriate queuing on egress towards core layer network, ensuring that no one customer or type of traffic can flood (or monopolise) the network bandwidth. Regional aggregation RAR and core layer routers expedite forwarding while enforcing QoS levels assigned at lower levels. The core router does this by associating the Type of Service (TOS) or Experimental (EXP) fields (based on the service type) in the label headers with various egress queues on transmission, which provide the appropriate class of service. A telecom operator provides services to multiple ISP, ASP, business customers and home users. Each requires different service level agreements and is thus assigned with varying QoS policies. Telecom operator adopts a single or multiple QoS policies specifying the traffic classes, their identification mechanism, and expected QoS treatment per class. At the ingress to telecom network the ISP or business customer traffic classes are reclassified (recoloured, remarked) in to the telecom operator classes and IP packets receive the telecom standard QoS treatment throughout its network until exit (egress) from its network. The boundary where ISP or business customer hands off its traffic to telecom operator (or vice versa) is called a trust boundary. It demarcates telecom QoS domain from those of ISP or business customers. Trust boundary may be MSAN, LAR or LAS. Reclassification is achieved by changing the packet’s CoS, IP Precedence, DSCP or MPLS EXP. SUNSEED proposed use: The outlined principles will be directly used in information transport for smart grid on both DSO and telecom operator networks. Traffic colouring and classification must be done with the same mechanisms and settings (i.e. DiffServ classes) in both networks. We require that known smart grid traffic types (e.g. GOOSE [GE 2000], [IEC850 2014], [Wen 2012], [Xu 2013]) are classified according to selected metric (e.g. latency, availability).


4.1.3 QoS mechanisms

The most widely used Ethernet and IP communication networks use a set of several features described below to deliver required class of service, i.e. deal with different traffic types in a different manner or priority.

4.1.3.1 IP DSCP

IP DSCP utilizes 6 bits in the IP header. The IP DSCP bits specify the class of service for each IP packet and can be used to distinguish up to sixty four (64) classes of service in an IP network (Figure 21). This value is set at the edge and enforced in the core layer.

Figure 21: DiffServ extensions [Cisc 2014a].

4.1.3.2 MPLS Experimental Field

The MPLS header consists of several fields. One of them is the 3 bit experimental (EXP) field (Figure 22). The EXP field used to be called CoS field, but was renamed when MPLS became a formal standard, however the EXP field is still used to carry CoS bits. Local aggregation layer routers will mark packets with CoS or EXP values (MPLS). The regional aggregation and core layers will use the EXP markings in order to apply the necessary priorities.

Figure 22: MPLS EXP bits [Cisc 2014a].

4.1.3.3 Policing

Policing functions as rate-limiter of traffic permitted onto a link. It is most useful at the lower network levels to enforce predefined traffic rate from end customer. Policing drops packets in excess of a given rate or can alternatively change the EXP setting of the excess packets so that they are given lower level of service when transiting the network.

4.1.3.4 Weighted Random Early Detection (WRED)

Weighted Random Early Detection (WRED) is a method to improve TCP behaviour in presence of congestion. WRED is designed to avoid congestion in networks before it becomes a problem. WRED interacts with other QoS mechanisms to identify class of service in packet flows. It selectively drops packets from low-priority flows first, ensuring that high-priority traffic gets through. WRED is configured with threshold parameters: a) minimum is the queue depth at which WRED begins to drop packets, b) maximum is the value at which all packets will be dropped. Packets are dropped according to implemented probabilistic function. By changing thresholds we can bound the


maximum latency of traffic through a given queue, change the average latency of the traffic through the system or prevent queue buffer memory loss. SUNSEED proposed use: Described mechanisms can be directly implemented throughout DSO – telecom operator converged communication networks, either at end WAMS nodes or in access and aggregation layers. They require operation in MPLS or IP domains thus imposing the end node and network awareness of these protocols (and others based on these). WRED is applied in information exchange from end nodes when IP protocol uses TCP. Note that it must be bypassed in control/command GOOSE message transmission that is conveyed directly over Ethernet.

4.1.4 Definition of traffic types

Telecom operator usually defines several traffic types, either labelled (e.g. marked or coloured) or not, meaning that deployed QoS needs to address both MPLS and Pure-IP traffic types. It is instructive to take a look at a typical example, since the same principle can be applied later in DSO-telecom operator converged communication network solution carrying different smart grid traffic types. Usually there are at least the following network services, i.e. traffic types in telecom operator’s network:

Internet access: Internet traffic is MPLS switched,

MPLS traffic: MPLS L3VPN and L2 services,

Video on demand and IPTV: Includes uni and multicast video services as pure-IP traffic,

VoIP: MPLS traffic,

Mobile access: MPLS traffic.

With the introduction of QoS mechanisms telecom operator can offer different service level agreements (SLA) and form so called SLA business packages (e.g. standard, premium) in accordance with desired traffic parameters (service availability, latency, jitter, packet loss) and class policies. It is possible to apply traffic engineering (MPLS-TE) for further and deeper QoS of individual services end-2-end (Table 6).

Class Application type DSCP MPLS EXP CoS Bandwidth allocation Control Routing management CS6 6, 7 6, 7 10 %

Real time SyncE, Backhauling, VoIP, VoIP signalling

EF 5 5 20 %

Video IPTV, VoD

CS4 4 4 20 %

Critical Preferred data

Business VPN High speed Internet

AF31, AF21

3, 2 3, 2 10 %

Default Internet CS1 1 1 40 %

Table 6: Example of types of service classes and bandwidth assignments in MPLS.

SUNSEED proposed use: For smart grid communication network operation we define several traffic types (in descending order of priority) e.g. control/command (e.g. GOOSE), (tele) protection (stringent delay requirements), fast period measurements (e.g. PMU, WAMS with low jitter), slow period measurements (e.g. 1 min from smart meters, no delivery deadlines), remote smart grid nodes management (Please refer to Chapter 6.2.1.1).


4.2 WAMS nodes communication solutions proposal

This chapter proposes wired and wireless communication solutions, i.e. different types of access network configurations, considered to fulfil smart grid communication requirements of the SUNSEED project. Figure 20 depicts how each solution enables transport of measurements and control information between the smart meters, WAMS nodes, DEMS and the SUNSEED back-end within the Telekom Slovenije’s network. WAMS has to be a universal unit with standardized communication interfaces to assure reliable information security as standardized interfaces considerably simplify maintenance costs and reduce total cost of ownership. Consequently, they contribute to the end goal of the cost effective distribution grid observability. Our aim is that total cost of WAMS devices and their integration do not significantly exceed the costs of smart meters AMI integration. The WAMS nodes’ functionalities and capabilities differ according to installation purpose and deployment location (e.g. MV, LV, prosumer nodes). WAMS devices deployed within the LV network and on the prosumer side have almost the same characteristics mostly related to taking measurements. However, WAMS devices deployed in transformer stations aggregate also several decentralized network management functionalities. For example, they can operate as smart meters data concentrators to perform ordered data colouring, accomplish end-to-end QoS within the distribution network, reduce individual smart meters transmissions and discharge the processes in the distribution control centre.

PUBLIC INTERNET

APN sunseed.field.trial.area

Mobile networkGPRS / LTE

SUNSEED APNs

SUNSEED Data flow pipeline

SwitchFirewall

DSO and/or customer premises

Database

PC

SwitchFirewall

Database

SUNSEEDDSO (utility)

management center

smart metersensors

Smart gridsubnetwork

1

2

4

5

GPRS / UMTS / LTE

Options

VPN SUNSEED

Internet (IPsec)

GPRS / UMTS / LTE

BS

PPPoEInternet (IPsec)

Remote access services for customers

66.1 Combination of Option 1 and 4 (backup)

6.2 Combination of Option 2 and 5 (backup)

smart metersensors


smart metersensors


smart metersensors

Smart gridsubnetwork Smart Phone

Tablet

DHCP

Telecom operator

Radius

SUNSEED VPN

Core

Core Peer

eBGP

Internet

MSANLAS

Router

RAR

RAR

LAR

IP/MPLS

Streaming protocol

WAMS node

Streaming protocol

WAMS node

Streaming protocol

WAMS node

Streaming protocol

3OPEN VPNsmart meter

sensors

Smart gridsubnetwork Streaming

protocol

WAMS node

SATELLITE NETWORK

SATELLITE modem

xDSL / FTTH /router

MSANLAS

Router

LAR

BRAS

xDSL / FTTH /router

mobile modem

mobile modem

WAMS node


Figure 23: SUNSEED generalised DSO and/or customer end nodes access options, communication networks and supporting infrastructure.

The communications solutions differ with regard to hardware and software used to complete the measure-information transport-process-react and influence loop:

End node, i.e. DSO location (substation, DER) or customer premises: Smart meter or WAMS node with a plurality of measurement sensors and/or controllable actuators.

Home network or DSO location network behind the WAMS node at deployment location: Typical Ethernet LAN, interconnecting other WAMS nodes to increase measurement/control capabilities or other nodes, e.g. VoIP phone for service personnel connection to DEMS, video camera for observation of substation environment.

Router at user premises or DSO location situated after WAMS node or built in it: Enables additional functionality, e.g. security, NAT functionality, routing behind MS, multiple PPPoE tunnels.

Communication protocols between end node and access network: With regard to implementation, WAMS node can implement classical IP protocol stack or use Ethernet level protocols only:

a) L2-L3-IP-UDP/TCP-HTTP/streaming protocol, b) L2-L3-IP-TCP-message bus protocol, c) L2-Ethernet-GOOSE (DSO specific, used in control/command and protection to

guarantee lowest possible delay).

Mobile base station and mobile core network: 2.5G, 3G or 4G. Base stations are connected via MPLS for mobile networks or directly into LAR or to MSAN, depending on the development of the mobile network and backhaul. The preferred arrangement for the SUNSEED field trial is UMTS/LTE.

Wireline access via LAR switch or MSAN: Industrial and/or business type access is into LAR switch or directly into LAR, whereas home users connect to MSAN (Figure 11). Options offer different availability (dual vs. single backhaul links) and throughput. Both options will be tested in the SUNSEED field trial.

IP/MPLS communication network: Its implementation resembles those described in Figure 9, Figure 10, and Figure 11. Authentication, authorisation and service mapping for each WAMS node is done here, as well as interconnection with other operators and/or DSO. In the SUNSEED field trial this is based on converged solution of DSO and telecom operator networks, or takes specific parts of each to achieve particular goal (e.g. lowest latency, best security).

Data centres with systems architecture software: Perform high-level functionality and introduces intelligence into smart grid solutions (via data analytics, forecasting and state estimation modelling). Data centres can be geographically redundant and/or shared between stakeholders (e.g. telecom operator, DSO, cloud services provider).

DSO management centre: It functions as DEMS for the distribution power grid.

User interaction: DR programs require user involvement, regardless of manual or automatic DR implementation and remote access services realise this with personal smart (mobile) devices either as application (e.g. Android, iOS, Windows 8.1) or web service (e.g. HTML5 based).

In case of using mobile networks (Options 1 and 2 in Figure 23) end nodes are connected to mobile network’s private APNs established for the SUNSEED field trial. Private APNs offer better QoS and improved security due to programmed limited services enabled per APN via Radius and PGW, SGW in 4G mobile network core. They are also typical solutions for collecting smart meters’ data (via PLC concentrator or directly). Therefore, this is the preferred mobile access arrangement for the


SUNSEED field trial. Each mobile modem/router acquires an IP address from a central DHCP or obtains a static IP assigned by the RADIUS server. APNs based solution completely separates data transfer from the Internet, but depends on the telecom operator implementation. Reliability of such solution is mostly dependent on the number of mobile cells in reach of each mobile modem/router. When failure occurs on a serving mobile cell, mobile router reconnects to the alternative cell in reach, while transmission characteristics (e.g. latency) heavily depend on the cellular network technology type. Satellite connection access Option 3 will be used in those field trial areas were mobile signal or fixed access points are not available. Satellite deployment locations define some physical restrictions, like full line of sight from SAT dishes to the serving satellite link and lacks fast response timing. Furthermore, the use of satellites is primary limited to one-way communication, like collection of measurement data and not for providing real-time feedback features. In most cases telecom operator does not have own satellite infrastructure and SAT communication link needs to be provided by 3rd party operator. This consequently brings less power (in terms of observability, control and especially security) to full vertical comm. path, which should be taken into account by the SG operator. However, in some cases they are the only suitable solution which can be used in the field. Wireline equivalent access Options to mobile solutions in Figure 23 are Options 4 and 5. These are under full control of telecom operator, which controls the VPN arrangement. Furthermore, VPN access Option 4 offers enhanced availability due to connection to local aggregation with LAR access switch having dual failover uplinks. Finally, access Option 6 is a combination of Options 1 and 4 or 2 and 5, where modem/router is placed after WAMS node when this is required (e.g. NAT capability) or not available in WAMS node itself. In this case end nodes require two different types of physical communication interfaces. The SUNSEED field trial will implement connectivity over combined wireline and mobile router with primary link through wireline DSO or telecom operator network (xDSL/Ethernet) and secondary (backup) link over public mobile network or private APN. It is possible to realize information transport without packet loss even on primary to secondary link switchover, but must be followed by a switchover and/or transport protocols (e.g. TCP-based message bus). Such arrangement greatly enhances availability and is useful for mission critical applications that are not delay bound due to secondary link mobile access method (e.g. some control/command).

4.2.1 Requirements of routers in considered communication solutions

Routing capabilities at end nodes offer functions for realizing multiple communication options (Figure 23). Routers functionality can be integrated in the WAMS nodes or in external devices facing the access network. Which solution is used depends mostly on the end node deployment location within the distribution grid. Routers can be easily integrated and connected into the DSO smart grid communication network via permanently established VPN tunnels and/or can be accessed via private IP addressing (e.g. with telecom operator or third party managed DNS [Dyn 2014]). More enhanced devices can be used for mission critical situations [Cisc 2014b]. Very high communication availability and reliability is achieved by using devices with multiple physical interfaces and/or dual SIM cards to reach access networks of different telecom providers. Automatic switchover between communication interfaces on signal loss or packet loss threshold violation supports seamless, information loss free operation. SIM card is the primary authentication mechanism in a mobile networks, but can be used also to authenticate and authorise access over any physical interface for stringent security in the smart grid. High quality manufacturing, suitable for use in industrial environments to achieve long time robust operation. Remote operations and backup


management of routers can be performed on any physical connection or special console port for syslog and SNMP messages exchange. Implementation of IPSec, OpenVPN or equivalent tunnels provides security during information exchange, while authentication of the routers can be realized by using deposited certificates or pre-shared keys, with multiple encryption algorithms support (e.g. 3DES, AES, key length at least 256 bit). A configurable (user programmed rules) firewall with intrusion protection/prevention system (IPS) must be built in to isolate smart grid communication network form external hacker attacks.

4.2.2 Requirements for local aggregation layer nodes

End nodes and routers within the smart grid typically connect to the back-end through the aggregation layer of converged DSO and telecom operators’ communication networks. To assure consistent treatment of QoS, security and high availability in such scenario the use of multiple access technologies is preferred in order to fully utilise the existing infrastructure. There must be a service platform with multiservice access support to facilitate different types of smart grid end nodes (e.g. control/command, measurement, management, etc.). Support for multiple service providers (per port, per all protocols) on a single node must be performed so that the DSO can communicate from end node’s primary and secondary links to independent telecom operator. Backhauling must be performed via multiple Ethernet 1 GE, 10 GE and GPON ITU-T G.984 links. High availability is achieved with redundant controllers, failover protocols and dual power supplies, with possible carrier-class and/or DC operation.

4.3 Security requirements

The security requirements identified here originate partly from a work of the security and privacy group of the Smart Energy Collective [SEC 2013], NIST [NIST 2010] and from the analysis of the SUNSEED use cases. The broad view of requirements listed below has been classified in 8 categories:

1. Data Management 2. Communications 3. Confidentiality 4. Integrity 5. Availability 6. Disaster recovery 7. Identification, authentication, authorization 8. Risk assessment

Although we present here all categories, in SUNSEED our work will focus on security categories in communications, identification, authentication and authorization that will have most impact when implementing use cases on field trial, particularly UC-1 and UC-2.

4.3.1 Requirements linked to data management

Data management includes, among others, the collection, storing, processing and mining of data. The requirements relate to the questions of knowing which data is being collected for which purpose? How long data is being retained and why? When does the need arise to trace data back to its origin? Who owns which data? The Data Management Principles roughly fall into two main categories. The first one consists of “Minimum Disclose Principles”, the second one of “Ethic of knowledge Principles”.


The Minimum Disclose Principles are the data management principles according to the user point of view. They represent a conservative approach where the data subject does not a priori trust the peer; therefore he/she accepts to disclose data only if such data:

Are required by the service provider.

Provide value for the user.

Requirement Explanation

A Data Policy governs all data in the system. The policy allows clarification of the usage of the data. It permits clear explanation to data subject. It is a relevant way to enable “User Trust” by generalization of “Data Transparency” of the usage of the information.

All data in the system are subject to a Data Protection Impact Assessment.

A Data Protection Impact Assessment (DPIA) is (Art. 33 proposed General Data Protection Regulation) ‘an assessment of the impact of the envisaged processing operations on the protection of personal data. (…).A Data Protection Impact Assessments will be mandatory in the upcoming European General Data Protection Regulation. In addition they form a robust part of data policies mentioned above

Data management is designed in a technology and implementation agnostic way using open standards wherever possible.

Robust and transparent data management is of crucial importance for the viability of smart energy systems as it is a key enabler for trust. Creating an open, interoperable design in which components can be provided by multiple vendors helps to establish a thriving smart energy system.

Disclosure of data is agreed upon in a transparent way by an explicit contract between the actors.

This principle means that “The disclosure of data is clearly stated in the contract between actors”. Transparency with respect to data usage is the main enabler for trust. Users are more likely to disclose personal data when they know and understand the purpose of the disclosure. Clearly stating the conditions under which data are disclosed also allows actors to take relevant actions to fulfil certification and regulation obligations related to data storage.

Data are processed as much as possible on the data subject side.

This implies that the transformation of “raw data” into “information” takes place at the data subject side. Only the information is exchanged, the raw data reside on the client side. Generating information at client side enhances privacy by applying the principle “relevant, adequate and not excessive” and security by generating information only at and for the targeted peer. It also relaxes the security constraints on communication and enables efficient use of network resources.

The collected data is fit for purpose. Collection of data should be, relevant and not excessive. This principle is part of the guiding


principle for privacy in the EU. The principle helps to setup the user contract, fulfil the data storage regulation and enhance trust between actors.

The data controller is responsible for the protection of collected data

The entity collecting the data is responsible for collected data. It needs to fulfil the security and privacy requirements that follow from applicable law and contracts with entities it interfaces with. It protects data from being stolen and/or modified by unauthorized systems.

The data controller allows the data subject control over its Personal Data

Personal data – as stated in the user/service provider contract – is owned by the data subject, that, as result and therefore should be controlled by the data subject.

Anonymous data is not de-anonymized Anonymous data sent by Data subjects may – implicitly or explicitly according the communication protocol – encompass data that could actually be used to identify the data sender via a de-anonymization operation. Such operation should NOT be performed

Data retention times are specified and motivated.

For motivated reasons, system may have to store data in a persistent way. The storage duration is limited.

Information and Knowledge computed from data owned by a single user is considered to be Personal Data

A system may be able to compute information and knowledge from a single data subject. Such data is viewed and processed as Personal data.

Knowledge created from aggregated heterogeneous data is owned by its creator

Knowledge obtained from data aggregation or data mining of heterogeneous users (data from different sources but related using one or more criteria), is owned by the actor performing the computation.

Table 7: Security, data management requirements.

4.3.2 Requirements linked to communications

Smart grids will generate large amounts of data that need to be transported over an infrastructure to the point(s) where it is used. Those requirements define the security level for data communication.


End to end security: The communication channel between source and destination should be protected

End to end security may be understood two ways : The first one requires that in a multihop communication scheme , there should be no hop left without data protection . This understanding of end to end security is a minimum and is compulsory. It does allow however for data to be rekeyed at transition nodes, exposing data in clear at those nodes. This type of end to end security is usually achieved at the transport level. A more stringent understanding of end to end security involves an additional data protection at


the application level enabling to secure data from source to destination with a single set of credentials. This type of end to end security is suitable for data transiting through non trusted nodes, as data will remain ciphered while transiting through those nodes. This type of end to end security should be possible, while not systematically used.

Data sent over an unsecure channel is secured itself

Exchanging data over an unsecure channel exposes the data to be intercepted by a third party, resulting in integrity and/or confidentiality breaches. Since the data controller is obliged to comply with the Data Protection Regulation, he has to take other measures to avoid these breaches. Securing the data means that the data intercepted cannot be read nor changed by the interceptor

Security aspects of the individual data streams are subject to the security aspects of the system as a whole

A data stream between two entities does not stand on its own but plays a role in the entire smart energy system. Because a chain is as strong as its weakest link, achieving the security requirements for the system as a whole means that all individual links should at least fulfil the requirements of the system.

Table 8: Security, communications requirements.

4.3.3 Requirements related to confidentiality

In a smart grid, a vast amount of data is stored and exchanged. Confidentiality refers to limiting information access and disclosure to authorized resources and preventing access by or disclosure to unauthorized resources. The confidentiality requirement defines levels of confidentiality for the different parts of the system.


Information is shared on a need to know basis Access to information other than public information shall be permitted only on a need-to-know basis. In other words, an entity can only access information that is needed to perform the activities it is supposed to do. All other (non-public) information is inaccessible

Information is classified into degrees of confidentiality needed

Not all information has the same sensitivity; therefore information must be classified into degrees of confidentiality needed. Suggested classifications are: Public: available for everyone. Can be freely exchanged Internal: available within the boundary of an organization. Exchanging with third parties is bounded by certain rules. Confidential: available for a limited group within


an organization. Exchanging with third parties is restricted. Secret: available for certain individuals within an organization only. Cannot be exchanged with other organizations

Protect the data, not only the medium Instead of only protecting the access to the medium where the information is stored, the data products themselves are protected as well. This implies that the data is encrypted, so it is only readable by entities that have the key to decrypt it. This provides an additional layer of protection to the common current-day practice of only preventing unauthorized access to information by limiting access to the medium (file share, website, etc.) where the information is stored.

Separate information of different confidentiality classifications.

Separating data according to its confidentiality classification allows for the optimal application of protection measures.

Confidentiality is ensured entity-to-entity A chain is a weak as its weakest link. As a result, to ensure confidentiality during the lifecycle of a piece of information, all entities involved in this lifecycle must have at least taken measures according to the assigned confidentiality level.

Protection is proportional to potential damage The level of protection shall be proportional to the potential damage that may result from information leakage. This implies differentiation is made in measures to be taken for guaranteeing confidentiality for the different information in a smart grid.

Table 9: Security, confidentiality requirements.

4.3.4 Requirements related to integrity

Integrity means that data cannot be modified undetectably. Data integrity concerns the trustworthiness of data (and the information represented in it) over its entire lifecycle. It is a must-have for trust in a smart grid: the energy producers, suppliers and consumers require correct invoices; the consumers need to have trust in the smart energy system that ultimately asks of them to adapt their energy consumption behaviour.


Integrity is upheld for actionable information The level of guarantee for integrity shall at least be proportional to the consequence of the decision

The protection level is proportional to the potential damage

The level of protection shall be proportional to the potential damage that may result from incorrect actions/decisions being taken upon


invalid information. This implies differentiation is made in measures to be taken for guaranteeing integrity for the different types of components.

Confidentiality is ensured entity-to-entity A chain is a weak as its weakest link. As a result, to ensure confidentiality during the lifecycle of a piece of information, all entities involved in this lifecycle must have at least taken measures according to the assigned confidentiality level.

Protection is proportional to potential damage The level of protection shall be proportional to the potential damage that may result from information leakage. This implies differentiation is made in measures to be taken for guaranteeing confidentiality for the different information in a smart grid.

Table 10: Security, integrity requirements.

4.3.5 Requirements related to availability

Availability refers to the availability of information resources including systems, processes and data elements. Availability is often expressed in a percentage, representing the portion of time of a given period – often per year; for SLA-purposes per month is used as well – a system is available. Another way of expressing availability is the “number of nines” – “three nines” corresponds to 99.9%.


vulnerability of assets should be assessed The vulnerability of an asset depends on several aspects, like value, location, etc. To determine the vulnerability of an asset, it is needed to investigate these aspects for each asset

Protection is proportional to potential damage A smart grid consists of many different components. Depending on the role and position in the smart grid, the unavailability of the component has more or less impact. The required measures to ensure availability are determined based on the impact unavailability. To avoid too much granularity in classifications, the M/490 SGIS working group has defined five security levels3 Low: Assets whose disruption could lead to a power loss under 1 MW; Medium: Assets whose disruption could lead to a power loss from 1 MW to 100 MW; Regional/Town Incident; High: Assets whose disruption could lead to a power loss from above 100 MW to 1 GW; Country/Regional Incident; Critical: Assets whose disruption could lead to a power loss from above 1 GW to 10 GW;

3 CEN-CENELEC-ETSI Smart Grid Coordination Group, Smart Grid Information Security, 2012


European/Country Incident; Highly Critical: Assets whose disruption could lead to power loss above 10 GW; Pan European Incident. Although these classifications have a strong focus on physical assets they can be applied to processes and systems controlling these assets too.

Introduce redundancy for systems that need to be highly available

A chain is as weak as its weakest link. As a result, to ensure confidentiality during the lifecycle of a piece of information, all entities involved in this lifecycle must have at least taken measures according to the assigned confidentiality level.

Monitor high-availability systems Systems that need to be highly available must be monitored. Each failure of these systems must be detected and acted upon. Due to the amount of systems on one hand and the speed of the response needed, the monitoring (and the acting upon the status changes) must be automated. Note that the redundancy of systems (see previous principle) should be monitored too.

Unavailability is mitigated by failsafe operation In case the ‘smartness’ in a part of the grid is not available, the grid must switch to a failsafe operation. In this mode, additional services that are needed to optimize the usage of the grid are not available. This implies, among other things, a stepwise degradation of the connection capacity

Table 11: Security, availability requirements.

4.3.6 Requirements related to disaster recovery

Disaster recovery is a subset of business continuity. Disaster recovery is the area of security planning that deals with protecting an organization’s business functions from the effects of significant negative events related to its technology infrastructure.


Investments in Disaster Recovery are based on a risk assessment

A risk assessment is carried out to assess the (financial, reputational) damage associated with negative events. The results of these assessments are used to determine the type and scope of disaster recovery control measures.

The system architecture supports an implementation that matches industry-standard RTO and RPO times

Disaster recovery capabilities are typically specified in terms of RTO (Recovery Time Objective, the duration of time and a service level within which a business process must be restored after a disaster) and RPO (Recovery Point Objective, the maximum tolerable period in which data might be lost from an IT service due to a major incident). A system architecture puts


constraints on the system implementation and therefore on attainable RTO and RPO

The system is designed as highly-cohesive, loosely coupled

Cohesion refers to the number and diversity of tasks that a building block of the system is designed for. Coupling refers to links between separate building blocks of the system

Backup only what needs to be restored A careful assessment should be made which data should be restored in case of a disaster. There is no point in backing up data if there is no intention to restore it after a disaster

Table 12: Security, disaster recovery requirements.

4.3.7 Requirements related to identification authentication, authorization

Identification is the process of knowing who is requesting access to some resource. Authentication is the process of verifying that “you are who you say you are”, authorization is the process of verifying that “you are permitted to do what you are trying to do.


Actors in smart energy systems have unique identifiers within their scope.

A well-defined scope (application, time-based, etc.) determines the name space (or named or unnamed context) and the naming scheme of the identifiers used. Together they guarantee these identifiers are unique. The unique identifiers may be permanently or temporarily assigned to an actor, depending on the scope and application. Non-linkable (by certain actors) IDs aid in privacy preservation. A temporary ID is typically, directly or indirectly, linked to another permanent ID, but not linkable by participating actors at the time when a transaction occurs.

Authorization can be based on either an (authenticated) identity of an actor or (certified) properties of an actor.

Keeping track of the provided authorizations requires a lot of administration. Instead of authorize each actor individually it is desirable to authorize a group of actors based on a (sub)set of their common characteristics (role) to reduce the load on the authorization system. This results in a single authorization to the group instead of an authorization for each single actor.

Identities have a life cycle A life cycle provides a way to distribute, manage and control the identities assigned to actors and transactions, and a way to derive meaning from the identifiers communicated by an actor to other actors participating in Smart Energy Transactions

The use of identity providers is supported All actors in the smart energy system need an identity to participate. Instead of realizing an


own identity provisioning service, an external identity provider can be used. Identity providers aid in issuing identities and typically also authenticate identities to actors participating in a transaction

Authentication mechanisms are fit for use Authentication mechanisms are fit for use implies that they are compatible with resources that are reasonably available for the actors to participate in the transactions

Authentication mechanisms are risk-based Authentication mechanisms shall be based on identified (by risk assessment) risk levels. Risk-based implies that the cost associated with the mechanism substantially outweighs the risks of not using the mechanism.

Authorizations have a life cycle Authorizations change, depending on actions of actors, authorizations may be granted, revoked or suspended

Authorizations are classified into authorization types

Different types of actors have different needs of accessing data – being able to read certain data is sufficient for some actors, while other actors must be able to alter the data as well. Although it is possible to create an authorization for each different action, it is more feasible to combine authorizations to minimize the administration of the authorizations. Making a distinction between ‘read’, ‘change’ and ‘manage’ is in most cases sufficient to prevent unwanted access to data.

Detected unauthorized transactions (or attempts to) are managed according to a predefined policy.

When a smart energy system detects unauthorized actions, or attempts to unauthorized actions, it needs to generate an event in the system log. Depending on the severity of the action, an alarm that is visible in a maintenance room or dispatch center is generated; in case of a severity of the highest category, automated protection measures must be taken to prevent the breakdown of the system

Table 13: Security, identification, authentication, authorization requirements.

4.3.8 Requirements related to risk assessment

Risk assessment is the determination of quantitative or qualitative value of risk related to a concrete situation and a recognized threat.


Risk assessments yield actionable results The outcome of a risk assessment should be in a form that allows for the implementation of specific measures that reduce an identified risk.

Risks are categorized or quantified All risks are categorized (using a fixed set of


categories like low, medium, high) or quantified (using a numeric scale) using a risk assessment methodology which is most appropriate for the management and control of the respective types of risk

Risk assessments are integrated into the smart energy system life-cycle

A smart energy system’s life cycle encompasses, at a minimum, design, implementation, operation and decommissioning phases. During all these phases risks exist which should be identified, quantified and mitigated where needed4.

Risk assessment is based on an auditable method Risk assessments are performed using a method that is auditable, i.e. it yields results that are reproducible and independently verifiable

Table 14: Security, risk assessment requirements.

4.4 IP/MPLS as communications network core

MPLS is a mature L2.5 communication network technology and is used in DSO and telecom operator implementations. Below are described some points of considerations with application to smart grid communication networks. Core communication network design implements BGP protocol, or is BGP-free for smaller DSO and telecom operators (< 2 million users). The exception is for the routers in peering domain that have full BGP routing tables (Figure 12).Within IP/MPLS network IS-IS or OSPF routing protocol is used. Geographical redundancy of router locations and multiple independent transport links must be supported by lower level protocols to handle convergence of routing paths after failover event. For example, fast IGP convergence protection can be implemented to ensure a high control and reliability of telecom operator IP/MPLS network with convergence time < 1 s. Alternatively, MPLS fast reroute mechanism such as RFC4060 can be used to achieve 50 ms path restoration, which is essential for DSO control/command networking. Protection is implemented for all critical services in network for example: L2VPN, VPLS, L3VPN and multicast traffic. IPv6 compliance within DSO and telecom operator network can be in the transition period realised as a dual stack solution implemented in whole network down to including local aggregation level layer.

4.4.1 Services within IP/MPLS network

Multiple service models are inherent in smart grid communication. Example services range from data collection of smart metering measurements, to control/command message distribution to protection and substation elements, relaying video surveillance from sensitive grid locations to supporting voice communication among all grid locations. Each type of service must be linked with its own type of

4 Whether a risk needs to be mitigated is determined by the quantified risk vis a vis the risk appetite.


class of service, i.e. QoS. We briefly consider below how this can be generally implemented on an IP/MPLS network. Services are communication network solutions for business (DSO control/command network, WAMS node network) and home customers (smart meter). Physical connection to business customers is typically via small router installed at business customer’s site that connects to LAR or local access switch (LAS) or to MSAN (Multi Service Access Node). LAS allows connecting one or more business customers, in single or redundant configuration. Redundancy can also be achieved over different physical interfaces (e.g. fiber, with mobile 4G or satellite failover link). Business customers may use many interfaces, depending on the number of acquired services (each service requires one psychical/logical interface). Home and some smaller business customers are connected through copper or fiber to IP/MPLS network via MSAN gateways. These provide ADSL2+, VDSL and FTTH connections and are used like pure L2 devices (Figure 11). LAR layer is providing L2VPN, VPLS, L3VPN and Multicast VPN services. VPN services for business customers usually interconnect through the core to required domains (either SDP or peering). Business customers with region wide dispersed locations may be connected from several LAR to distinct RAR nodes. L2 VPN (PWE/VPLS) or L3 VPN services can be used to fulfil customer demands (e.g. P2P, P2MP (hub-and-spoke) and full-mesh connections). Local aggregation layer with LAR is providing L2VPN (PWE), VPLS and multicast VPN (mVPN) services for home users. IP/MPLS network offers to them a highly resilient transport communication network from home installed customer premises equipment – modem (CPE) to SDP of telecom operator or other ISP, ASP. High speed internet and VPN services are terminated on BRAS router nodes that provide services and offer scalable control mechanism for all customer types. Hierarchical quality of service (QoS) with different classes for each type of traffic is used for ensuring a high service level and traffic separation (per VLAN) between business customers and their services. Larger telecom operators are obliged to provide same level of services to other, ISP or ASP, due to highly deregulated telecom market. So the same level of services described above can be provided as a bitstream access in a wholesale model. Openness of telecom operator class of services includes also DSO in any model as described above. SUNSEED proposed use: MPLS will be the central logical interconnect from WAMS nodes to telecom operator testing and data servers and also to other entities (e.g. DSO). The extension of current EP and TS approaches will be implemented in the field trial. Most of the communication between WAMS nodes and the TS datacentre (where raw storage will be established) will be separated from public internet using multiple APNs (connection via mobile networks) and VPNs (xDSL and satellite connections). Core network data will be terminated with firewalls (for securing connectivity toward MPLS network) and load balancers (for distributing workloads across multiple computing resources). Schematic of TS core network with its outer connections through gateways and firewalls is show in Figure 24.


DATACENTERFW

BALANCER

Multiservice Edge Router

JUNIPER MX840

Aggregation Services Router

JUNIPER M320 LAB

TEST LABSERVERS

WAMS(via APN)

MOBILE NETWORK

PF SENSEFW BALANCER

WAMS(via VPN)

xDSL connection

WAMS(via VPN)

SAT CONNECTION

TS MPLS

Aggregation Services Router

ASR 9000

TS DATACENTER

Figure 24: Schematic of TS core elements and their outer connections for SUNSEED

4.5 Time synchronisation for smart grid observability

Accurate time synchronisation is essential for application of discrete state estimation, observability

and prediction methods, that depend on synchronous data samples from measurement nodes within

distribution grid [Ande 2010], [Hida 2011], [Khan 2011], [Popo 2012], [Popo 2014], [Shik 2011].

There are four mainstream technical solutions to high-accuracy network-time synchronization for key

communication network elements:

1. Synchronous Ethernet: The first is the so-called synchronous-Ethernet, which acts as a

conventional Ethernet with an additional synchronization signal. It is a purely technical

solution on the physical layer (L1) . The technology allows frequency synchronization and is

completely transparent to data transfer (does not affect the transmission bandwidth).

2. IEEE 1588v2: This is a protocol for the transmission cycle via the packet network. It is a pure

L2 service solution and does not require any special hardware. IEEE 1588v2 does not require

updates to existing transmission equipment for the operation. It only requires adding the

IEEE 1588v2 network time servers and clients. It supports frequency and phase

synchronization, which ensures high-quality performance. A synchronization signal is

transmitted in packets over the network that, to some extent, occupies the transmission


capacity of the network. Given the transient and volatile situation in the current networks,

the usage of this protocol is a sensible solution due to its flexibility and scalability.

3. NTP: Is a protocol, NTPv4 being the latest version [RFC5905] that enables purely software

solution to time synchronisation over IP networks. NTP can usually maintain time to within

tens of milliseconds over the public internet, and can achieve better than one millisecond

accuracy in LANs under ideal conditions. In non-ideal conditions (assymetric routes, network

congenstions) errors of 100 ms or more can be caused. It is predominantly used in terms of a

client-server model, but can also be used in peer-to-peer relationships.

4. GNSS: The accuracy of Global Navigation Satellite System (GNSS) time signals is ± 10 ns and

provides atomic clock reference for real global synchronization. It mainly depends on velocity

of receiver (it is static in case of Sunseed), number and position of tracked (i.e. visible) GNSS

satellites that are syncrronized between each other, the type of used GNSS module (e.g.

military, industrial or consumer-grade quality, which mostly differ in acquisition sensitivity)

and WAAS corrections applied. Although only two navigation systems are actually global

(GPS and GLONASS) in the world, other equivalent space-based synchronization systems can

provide similar performance for regional areas, e.g. GALILEO, BeiDou-2.

SUNSEED proposed use: Each of the WAMS nodes will have implemented a GPS module and antenna that will be used for GGSN time synchronization over the GPS system. WAMS nodes will use accurate time for time tagging measurements, which will be sent through the network (LTE, SAT or xDSL) into the raw storage database (see D4.1.1). On the other side, all the servers which consume the WAMS measurements (message broker servers, raw database server) will be time synchronized using NTPv4. Highest possible accuracy will be achieved using an NTP server in the servers’ LAN network. Other synchronisation methods will be implemented only if required for control/command in distribution grid.

4.6 Potential technologies for future smart grid communication

networks

There are several beyond state of the art communication technologies that may be used for future smart grids. Some of them are valuable to extend the life of present mobile access technologies (2.5G, potentially 4G), i.e. reengineering or are valuable due to their low marginal cost entry of DSO into dense communication networks for smart grids, e.g. cloud RAN, SDN. The chapter makes their brief presentation. In SUNSEED we will investigate some of them and their potential via simulation (reengineering) or laboratory, partial field trial implementation (SDN).

4.6.1 Reengineering existing cellular networks

While the future 5G networks will most likely be designed to support M2M traffic natively, these networks are years away, and it is therefore necessary to consider how current cellular networks can be reengineered to support smart grid data traffic now. Current networks such as 2G, 3G and 4G (LTE) have primarily been designed to deliver as high as possible data throughput to mobile handsets and personal devices to fulfil the requirements of


multimedia applications and services online. Here, a device is typically connected for a limited time in which a substantial data throughput is expected. However, for M2M applications, communication is typically at a much lower data rate but for much longer periods of time. Furthermore, since such communication in M2M applications is often used to facilitate feedback in a real-time control loop, the requirements to reliability and latency can be much stricter than for multimedia content where for example buffering can mitigate occasional glitches. The specific requirements of smart grid data traffic that can pose challenges to existing cellular networks, relate to:

1. Massive number of devices, 2. Low latency communication, 3. Ultra-reliable communication.

In the following, a few examples from the literature on reengineering current cellular networks are given. Existing GPRS networks that are no longer able to provide the data rates required by users, are considered a promising solution to deliver reliable low data rate connectivity in M2M applications. However, a massive number of devices can be problematic for GPRS especially in the way that uplink resources are allocated to users. In [Mad 2013] it is shown how the limited size of the Uplink Status Flag (USF) in GPRS makes it impossible to multiplex more than 7 users per PDCH, meaning that in total only 49 users can be simultaneously allocated in a cell. The authors further propose a reengineering approach, in which a multiplexing scheme is used expand the uplink address space and thereby let users access only every Mth multiframe. This allows to support a large number of devices (>105) with a sustained (low) bit rate. Another challenge lies with the random access methods that are allocating uplink resources to users. In current cellular systems the random access part is based on ALOHA-type protocols that have the disadvantages that:

1. They are essentially inefficient with a throughput efficiency of up to 0.37. 2. They are incapable of dealing with sudden bursts of massive arrivals in an efficient way.

A recently proposed way to deal with this is to use Coded Random Access, for which collisions are not wasted but instead buffered and used for later decoding using Successive Interference Cancellation. This is possible because users content with replicas of the original packet that was part of a collision. When at some point this packet becomes successfully received, the receiver can subtract it from the buffered collisions and then with higher probability decode the other colliding packet. This alternative contention mechanism can be easily implemented in current cellular systems [Paol 2014], where it only requires an upgrade of the Base Station software. As some types of M2M traffic are periodic in nature, e.g., status reports from smart meters or voltage fluctuation measurements from PMUs, it makes sense to not do default random access for such expected messages. In [Corr 2014] it is considered how the LTE uplink resources can be structured in two periodically occurring resource pools, where the first is reserved for periodic status reports and the second is used for additional unforeseen transmissions. The paper showed how the second resource pool should be dimensioned to ensure a certain reliability level (successful packet decoding probability). Compared to a legacy LTE system, the proposed resource pooling use much less channel capacity and is thus able to serve many more devices. The proposed approach requires changes to both device and base station software, however, the idea of supporting periodic transmissions is in line with the development of LTE and may well be included in some form in future releases.


SUNSEED proposed use: Focus of the considered reengineering approaches will ensure that the smart grid communication:

1. Supports high densities of end nodes with low throughput, primarily for advanced metering by use of smart meters and other distributed measurement nodes.

2. Obeys latency requirements of different magnitudes in different parts/layers of the smart grid, where for some also high data rates are required.

3. Requires as few as possible changes to existing systems, preferably only software updates on base station.

4.6.2 Software defined networking

Software defined networking (SDN) approach effectively decouples data plane from control plane via an open API (e.g. OpenFlow protocol [OFP 2014]) that allows remote control of information packet forwarding between high throughput switches in such network [Nasc 2011]. Single controller, centrally or regionally situated can control multiple forwarding instances, each having different programmed packet match content capabilities (e.g. MAC and VLAN and IP and port) (Figure 25). Furthermore, packets can be altered on a per hop basis. Several SDN routing protocols (e.g. RouteFlow) allow implementation on commodity server hardware (running Linux and routing engines XORP, BIRD) that may be situated in DSO, telecom operator or even at independent data center service hosting provider. Such solutions offer broad design space with possibility for virtual routers and IP networks as a service. SDN has a proven track record in many application areas, e.g.: MPLS, virtual overlay M2M cellular network, distributed BRAS, FlowER open switch implementation in Erlang. All major equipment vendors (e.g. Cisco, Juniper, Brocade, IBM) have embraced the SDN paradigm and already offer key building blocks, whereas large cloud providers operate intercontinental, inter data center communication networks with SDN concept (e.g. Google). For smart grid environments SDN has the potential to realise so called smart grid communication network as a service model where data plane is implemented by/within telecom operator communication network and control plane be in control of DSO. DSO thus gets complete, secure control of different information flows for smart grid operation with low marginal costs.

Figure 25: OpenFlow API for software control of SDN network elements [ONF 2012].


SUNSEED proposed use: Part of field trial communication network (or laboratory network) connecting nodes within DSO, telecom operator and WAMS nodes in MPLS enabled SDN implementation, with commercial off the shelf data plane and open source control plane.

4.6.3 Cloud or centralised radio access networking

The Cloud/Centralised RAN (CRAN) is a mobile communication concept based on and combining technologies of small cells, distributed antenna systems (DAS), active antenna arrays and remote radio heads (RRH) or units (RRU), radio over fiber (RoF), and centralised signal processing and cloud based compute load scheduling [TI 2011]. It promises to create significantly denser cellular networks with lower equivalent power consumption, investment cost and fast deployment [CMRI 2010]. Key principles are also studied in 3GPP standardisation body and planed for future releases (beyond Rel 12). We have witnessed much dense compute processing capabilities for the same power budget due to innovation in computer architecture (e.g. VLIW and DSP multicore, GPU, heterogeneous cores) and this progress pushed to implementation the idea of centralized processing of fiber transported radio signals from separate base stations to centralised data center. CRAN concept decomposes main building blocks (baseband unit, radio remote head, backhauling) of classical base station and connects multiple RRH/RRU with antennas over RoF or DWDM fiber links (microwave implementations also exist) to a single centralized location (private, secure cloud) responsible for all signal processing, MAC and L3 of baseband unit, covering wide geographical area (up to 20 km radius). RRH/RRU is only responsible for antenna connection via low nose amplifier and transmit power amplifier, analog to digital (and vice versa) conversion (although experimental solutions exist with direct analog signal transmission over fiber) and powering. Such arrangement results in lower complexity (that has moved completely into software running on data center servers), reduced purchasing and maintenance costs and power consumption. Wireless network cloud has the ability to efficiently share compute resource to accommodate processing of all connected RRH/RRU, since not all are fully occupied during every time instant. CRAN concept is especially attractive for DSO and smart power grid operators in general, due to its lower cost and simplified equipment at the antenna side (i.e. RRH/RRU) that can be powered with RES and requires only fiber or microwave backhaul, which can be leased or operated as a virtual operator on telecom operator infrastructure. In such arrangement DSO can install RRU with antenna array on transformer stations and substations, such that it radio covers all power grid subscribers that are attached to this local power grid. In SUNSEED we will look into placement of such radio nodes on suitable DSO points in power grid and use this as input for techno-economic analysis. SUNSEED proposed use: Radio planning of CRAN like solution, i.e. RRH/RRU at transformer and substation location, with full dynamic GIS support and visualisation to establish the end node coverage/reach fraction within distribution grid. Will be presented as a business case of DSO applying CRAN infrastructure for smart grid (Please refer to Chapter 5.1).

4.6.4 5G

For the next generation of cellular communication networks, referred to as 5G, it is commonly agreed upon that simply making a faster version of the current 4G technologies such as LTE, will not suffice to meet the demands that we will face in the coming years. As outlined in [Bocc 2014] a number of radical changes, our so-called disruptive technologies are believed to be necessary for the future 5G networks to be able to satisfy the extremely high aggregate data rates and very low latencies requirements that the future interconnected society requires. Specifically, the mentioned disruptive technologies are:


1. Device-centric architectures: Reconsider traditional concepts as uplink/downlink and control and data channels.

2. Millimeter Wave (mmWave): Exploiting mmWave spectrum to remedy current microwave frequency spectrum scarcity.

3. Massive-MIMO: Utilize a much higher number of antennas. 4. Smarter devices: Less reliance on infrastructure than with 2G-3G-4G, thus putting more

intelligence at the device side. 5. Native support for Machine-to-Machine (M2M) communication: Support for massive number

of devices, sustainment of minimal data rate in virtually all circumstances, and very low latency data transfer.

In relation to enabling communication in future smart-grid systems, such as in the SUNSEED project, the most interesting of these technologies is 5., in that it addresses the foreseen challenges related to supporting a large number of devices such as smart meters or WAMS nodes, where it is crucial to be able to communicate status, control and alarm messages reliably and with very low latency. In the EU co-funded project METIS [Meti 2013], Mobile and wireless communications Enablers for the Twenty-twenty (2020) Information Society, a similar vision for the 5G networks exists. Within METIS, a set of horizontal topics has been defined that characterize the capabilities of the future networks. The topics are:

1. D2D - Direct Device-to-Device Communication 2. MMC - Massive Machine Communication 3. MN - Moving Networks 4. UDN - Ultra-Dense Networks 5. URC - Ultra-Reliable Communication.

For smart grid, the most related topics are 2, 4, and 5, which characterize the communication challenges. SUNSEED proposed use: We are not going to deal with 5G directly, but it is essential that future smart grid considers 5G as a natural follow up of 4G for NAN and WAN usage. Since smart grids are considered by several 5G-PPP Horizon projects as appropriate use case scenarios for implementing the low latency, security and multitenancy functionality, we have proposed Sunseed pilot and knowledge as a testbed for a use case scenario in CHARISMA P5-PPP project and it was accepted.


5 Economic and business requirements of cost effective smart grid

communication solutions

In SUNSEED we propose a comprehensive techno-economic treatment of smart grid communication to achieve wider cost benefits that originate from DSO and telecom operator converged approach to realization of communication networks. One of the SUNSEED goals is to create a set of guidelines on how to approach and execute this challenge as an evolutionary growth path from existing communication network domains of both stakeholders. A study of economic benefits of overlapped, to a certain extend combined and interconnected communication networks from DSO, telecom operator and even third parties (e.g. ISP, municipality, local community) will discuss several cases in a systematic way, i.e. converge, DSO and telecom communication networks for dense DEG environment smart energy grids. Multitude of possibilities in cost benefit analysis (CBA) will be analysed with heuristic toolsets (e.g. decision tree methods) to derive optimal solutions for DSO. Optimality criterion is either lowest marginal cost (barrier to entry) or lowest total cost of ownership of the resulting communication infrastructure solutions. We take into account the complete life cycle of providing technological solutions, i.e. from design to implementation, usage and exploitation, management and supporting activities organised on top of these infrastructures (e.g. billing in data center connected to communication network gathering measurement data from smart meters or analytic decisions from WAMS nodes). So the emphasis is not only on the (best, most recent, highest performance) technology, but on cost benefits, and we treat cost as the decisive factor to achieve best smart grid communication solution for DSO and other stakeholders.

5.1 Synergies of DSO and telecom operators for cost effective smart

grids

Before defining any possible synergies, we must ask what it takes to plan, set up, operate and maintain any infrastructure and communications networks in particular. The complete ICT domain is important in smart grids, so it is instructive to look at this from wider perspective. Figure 26 shows coarse grain infrastructure types involved. We start with physical types (e.g. cabling, data center location), advance through logical and end at software infrastructures required to operate communication and ICT for the smart grid. Each of infrastructures is decomposed into several tasks that constitute its life cycle (plan, build, manage, share). Planning is a wider term that constitutes also financing and procurement process. Building is associated with erecting physical infrastructure, whereas installation is at level of equipment, or program when we talk about virtual partitioning of equipment among more operators (DSO, telecom, ISP). Managing is a long term process of ensuring constant levels of performance and constitutes maintenance, software or hardware upgrades throughout the infrastructure or equipment life expectancy. We specifically list also the possibility for sharing of each infrastructure type and this is the point where true cooperation and convergence will be achieved (Table 15). By systematically traversing Figure 26, we can arrive at several possible points of cooperation between DSO and telecom operator, or even other third parties. Wider evaluation is possible on the task level. The CBA of most interesting cases will be done with decision tree support tools (e.g. Dexi [Dexi 2014]). Note that the bottom up analysis (from back office support software) is also possible


(than resembling OSI layers view of physical infrastructure at bottom and software at top), where this is more like the investment ladder, and is usually done in telecom operators’ environment for analysis of cost models (e.g. inter operator).

Figure 26: Synergies of active cooperation opportunities DSO – telecom operator at various levels and of different types.

Cooperated infrastructures and

tasks

Model name or purpose Comment

6D Simple AMI only Used for smart meter reading. Smart meters are connected to mobile network via APN of telecom operator as managed service and DSO is interconnected with VPN in single point,


also as a managed service. Although we operate two kinds of infrastructures (mobile and wireline), only single term (6) is used.

4D 5D 6D

DSO as MVNO and data center hosting

For DSO smart meter data traffic over mobile network, DSO is operating as a MVNO on top of telecom operator infrastructure and is leasing also data center hosting services, whereby the physical servers and software is DSO investment installed at telecom operator data center. The same VPN data transport connection applies as before.

1A, 1C, 1D 2A, 2C, 2D 5D

Sharing common fibre for dual infrastructures

DSO and telecom operator plan together fiber layouts, for mutual benefits to cover wider area. DSO builds, by third party system integrator, CRAN like nodes on selected transformer station for WAMS node data collection and offers data bandwidth for selected mobile telecom services. Fiber and CRAN infrastructure is subsequently managed by telecom operator, but observability is fully within DSO network management center.

Table 15: Some example cases of synergies in DSO – telecom operator cooperation.

SUNSEED proposed use: DSO – telecom operator synergies will be systematically evaluated through 9 step procedure outlined, for cases that we find on the field trial geographical region and other interesting cases to fulfil dense smart grid distribution grid needs. Thorough CBA follows and decision tree post analysis will provide guidelines of most cost beneficial cooperation strategies and subsequently converged network implementations (within technical constraints).

5.2 SLA on smart grid communications network marketplace

Service level agreement is a contract between a telecom operator and a customer which specifies a required level of service, and commits the telecom operator. A SLA typically contains a specified level of service including availability (uptime or downtime), identification/classification of traffic into traffic classes, committed loss, latency, delay variation per traffic class, enforcement or penalty provisions for services not provided, and a specified level of customer support. From a conceptual standpoint, a telecom operator may offer several tiers of services with various levels of SLAs suitable for different types of applications run by its customers (e.g. best effort service for non-mission critical, not delay sensitive applications or low latency service). It is not necessary that all communication network metrics and parameters are specified and/or enforced by each SLA. SLA ensures elements and commitments of managed services for provision of consistent communication networks operation and support between telecom operator and business customer. Specific SLA comes into place between DSO (utility) and telecom operator in order to offer complete ICT solutions, support and services necessary to operate communication and ICT infrastructure of future smart grid. It provides clear reference to service ownership, accountability, roles and/or


responsibilities, measurable description of service provision to DSO and matches perceptions of expected service provision with actual service support and delivery. The highly reliable, secure smart grid communication network operation is assured through traffic classification and appending QoS parameters early, at the source (e.g. WAMS node) and treating smart grid traffic as prescribed throughout the communication network, i.e. end-2end (Figure 27). A set of KPI (metric) are defined, along with measurement tools (e.g. measurement methods, protocols, management software). A common understanding of required service levels is agreed and put forward in business agreement between parties. It is required that tracking of SLA parameters is available also to business customer. Note that not only communication parameters are part of SLA, but also of other ICT equipment, thus offering coverage of all ICT elements required for smart grid. Specific mention of ICT equipment covers for the smart grid communications equipment itself and IT equipment (e.g. servers, storage) used for backoffice support (e.g. billing).

Figure 27: Generic SLA flow between DSO-telecom operator for communication networks in smart grid.

It is prudent to refine the generic model of SLA flow and adjust it to DSO specifics due to highly heterogeneous nature of smart grid communications, different types of communications networks coexisting and all functioning to deliver reliable smart grid operation (Figure 28). We may follow the path for several SLA, specific for each type of DSO communication network and/or equipment, producing more business agreements, too. Such decomposition allows DSO the possibility to work with several communication network operators (telecoms) in order to achieve best technological operating performance (e.g. lowest latency, highest availability), lowest total operating cost, ability to adapt to market conditions rapidly (e.g. changing to other telecom for certain communication network or service type). For larger DSO it is also possible that multiple telecoms take care of SLA for different geographical regions. Role of IT expands greatly in smart grid with WAMS nodes deployed and gathering real time information about smart grid status, much more emphasis is given to data analytics and offering results in a form of Web services to other smart grid stakeholders (e.g. VPP, micro grid operator). This approach will be considered in SUNSEED techno-economic analysis and business models development and methods used to derive them (e.g. CBA, decision tree).


Figure 28: Per smart grid traffic SLA flow between DSO-telecom operator for converged communication networks in smart grid.

A sample of selected SLA metrics for different types of smart grid networks is shown in Table 16, with inserted numbers only as a reference point. A demarcation router is usually situated between customer premises, i.e. DSO and telecom operator and this is valid for all communication network types. Note, however that for grid specific elements, like SCADA, PMU, telecom operator may not be the right partner to manage them, too. WAMS node management by telecom operator is given as an example. Control networks (e.g. protection, SCADA) may have very stringent requirements on availability or latency or both, whereas data acquisition from WAMS places less demanding constraints that are straightforward for any modern telecom network type (e.g. wireline fiber, mobile 4G). There are some specifics in smart grid SLA compared to classical business ones (video, VoIP, data center interconnect). Whereas in classical business we see predominantly requirements for high bandwidth and low latency in smart grid this may not be the case:

1. Multiple, simultaneously active, heterogeneous communications network types. 2. Specific SLA requirements per each network type. 3. New types of SLA parameters, e.g. low bandwidth with extremely low latency and 5x9

availability (protection), medium bandwidth with low latency and time synchronised end points (PMU), higher bandwidth with medium latency and time synchronised end points (PMU).

Network (function) type

Equipment Availability Latency

AMI Not managed smart meter 96 % Not specified

WAMS Managed demarcation router, WAMS node, at location IP

99,5 % 500 ms


network

SCADA Managed demarcation router, at location IP network

99,99 % 1 s

PMU Managed demarcation router, at location IP network

99,9 % 50 ms

Protection Managed demarcation router, at location IP network

99,999 % 8 ms

VoIP Managed demarcation router, at location IP network

95 % 150 ms

Video Managed demarcation router, at location IP network

95 % 100 ms

Table 16: A sample of selected SLA parameters per smart grid communication network types.

SUNSEED proposed use: New approach to define SLA for network and services for distribution grid communication networks will be derived from converged DSO – telecom operator long term cooperation. Primary goal is to fully take into account specific DSO requirements, of all their communication network types (from smart meter AMI to control/command and video or VoIP) and suitably place them within the DSCP prioritization scheme.


6 Proposal for Sunseed smart grid architecture

The objectives of SUNSEED smart grid architecture can be summarised as:

a) Provide software framework that will bring about intelligence to WAMS nodes and smart meter measurement data streams.

b) Control, configure and monitor WAMS nodes. c) Integrate several software building blocks, from various project partners, third party open

source solutions, via Web services (e.g. RESTful API) into a coherent framework. d) Provide real time visualisation of distribution grid operation and evaluation of KPI on power

and communication grids as set out in Chapter 3. e) Involve prosumers in the demand response loop. f) Allow access to the system only to known devices and assure full communication security.

Further provide access only to selected services to which end nodes are subscribed.

6.1 System architecture

SUNSEED architecture, depicted in Figure 29, consists of different entities, which communicate between each other through well-defined interfaces (APIs) exploiting either DSO or telecom communication’s infrastructure. There are different types of data sources originating from devices (at the bottom of the figure) observing the grid (providing real time measurements), and some of them serving also as the entities, which can control the grid (i.e. WAMS-PMC). DSO database is describing the grid (static data), relevant information describing households, offices and companies (static data). In addition, there are used also other data sources that also influence the energy consumption and production e.g. weather information, calendar (e.g. weekends, holidays). All the relevant data (measured and from other data sources) are collected in Raw data storage, which serves as the main repository.


IP

XMPP broker MQTT broker

SUNSEED data

Distribution System State Estimation

e.g. weather

Analytics / Forecasting

Raw data storage (Mongo, SQL, ...)

datalocal resources

other data

datalocal resources

local visualisation

SE, LF, ... calculation

data cleaning

WAMS CP-PMC

Power measures

FPAI Agent & driver

WAMS CP-PMC

Interface unit

FPAI Agent & driver

Energy Device

WAMS CP-PMC

Power measures

FPAI Agent & driver

WAMS CP-PMC

Interface unit

FPAI Agent & driver

Energy Device

AMI AMIWAMS CP-SPM

Phasor measures

MQTT Agent

WAMS CP-SPM

Phasor measures

MQTT Agent

Landis+Gyr(Aggregator)

Reporting & visualisation (GIS applications

visualization)

Other smart grid services(e.g. Demand response (MPPDR),

Monitoring (ADNMSP), KPI monitoring, …)

e.g. topology

DSO data

Security(Authentication,

Authorisations and credential management)

Figure 29: Systems architecture view and key building blocks of SUNSEED.

Real time measurements are provided through AMI and WAMS nodes. We assume that security is provided on end nodes as described in Chapters Error! Reference source not found., Error! Reference source not found., 4.3. The latter also run the FPAI (Please refer also to Chapter 6.2.4 and deliverable D4.4.1), which can act as:

a) control space (device status from the device) b) allocation (control information to the device).

FPAI abstracts the nature of an appliance/energy prosuming (producing and/or consuming) device into a control space that tells in what period the device will consume or produce energy according to a certain profile, and an allocation that controls (within the boundaries set by the device) when the actual production or consumption will take place. AMI nodes perform periodic measurements that are collected at pre-scheduled times using the smart meter aggregator and thus arrive to the Raw data storage with the delay. The AMI data collection periods depend on the regulatory demands.


Smart meter (AMI) and WAMS nodes measurement data are both used in analytical processing in later stages. AMI nodes are all property of the same DSO operator; hence the communication to the other modules is through one DSO (in our case via Landis+Gyr aggregator). WAMS nodes with or without FPAI, with data collection or actuator functionality may belong to different entities (e.g. users, DSO, telecom operator, etc.). M2M platform (i.e. XMPP or MQTT) placed between the different data source and other entities manage all different data sources from different entities in a unified and secure manner. The main functionality of M2M platform is to ensure that a device or external data source and an analytical platform may talk together even if they have not been written to work together. In addition, M2M platform will also encompass the trust platform that will deal with devices and application authentication and authorization. The Reporting & visualisation module incorporates GIS that is the main visualisation reporting tool as described by dynamic GIS tool set. All results are presented using different GUI, which correspond to different Applications (e.g. use case types). Collected data from services will be transformed in to the common format, so that they can be cleaned, fused and different stream aggregates (e.g. moving averages, normalisation, interpolation) can be performed on data stream in the same way. The “application/service” modules, can be in general divided into the following parts, broadly discussed in deliverable D4.1.1, 4.2.1 and D4.3.1:

A monitoring part that monitors the current state of the entire DSO grid, including the state estimation calculation.

A prediction part that predicts the future state of the DSO grid based on the known and predicted energy consumption and production.

A visualization part that presents the status, results, etc in GIS application. A control part that checks if current and predicted state of the electricity grid are within safe operational limits according to selected use cases and possibly takes measures to assure safe operation of the grid.

6.2 Functional building blocks

We describe main building blocks envisaged on SUNSEED: WAMS node, visualisation tool sets, FPAI. Since WAMS is closely related to smart meters and PMU designs, we contrast both and make a case for WAMS capability list below.

6.2.1 WAMS node in high density distribution smart grids

WAMS node is the essential element in distribution smart grid providing its observability on a wider space and finer time scales. It solves the present lack of visibility of distribution grid and opens the opportunity for cost effective solution (more expensive than a smart meter, but much cheaper than a PMU), especially when deployed much more massively compared to PMU. It is intended to be installed at DER locations, majority of substations, larger electricity consumers. Its function is to work in parallel to smart meters, but provide more functionality, faster and more measurement types (Figure 30, Table 17). WAMS node is intended to take the equivalent role in the distribution grid as PMU has it in transmission grid.


Figure 30: WAMS node vs. smart meter vs. PMU.

Attribute Smart meter WAMS node PMU

Purpose Billing of electricity Observation of distribution grid network behaviour and state

Observation of transmission grid network behaviour and state

Deployment place Home user Industrial site

Distribution grid, all DER, MV–LV substation

Transmission grid, HV network, HV–MV main substation, MV–LV substation

Deployment quantity Each end node in distribution grid

All DEG, MV–LV substation, selected nodes in distribution grid

Selected nodes in high voltage grid.

Measurement quantities

P (home) P, Q (industrial)

P, Q, cos, U, I, f, THD (grid metering)

Phasors: U, I

P, Q, cos, U, I, f, THD, harmonics

Phasors: U, I

Measurement accuracy

1 % (home) 0,5 % (industrial) 0,2 % (grid metering)

0,2 % - 1 % TVE < 1 % (total vector error)

Dynamic range < 12 bit (home) < 14 bit (industrial)

12 bit – 16 bit > 14 bit

Measurement < 8 kHz < 8 kHz < 20 kHz


sampling frequency (16 ms result integration, 200 ms result write into log file)

Measurement period 15 min (home) 1 min (industrial)

1 s 10 – 60 Hz

Reporting period 24 h (home) 1 h, 15 min, 1 min (industrial)

1 s, 15 s, 1 min 10 – 60 Hz (China 100 Hz)

Communications interface

PLC (home) PLC, GPRS, RS485, Ethernet (industrial)

2.5G, 3G, 4G, Ethernet RS232 Ethernet

Communications protocol

DLMS (home) DLMS, TCP/IP (industrial)

UDP/IP, TCP/IP, streaming

UDP/IP, TCP/IP, streaming

Communication network structure

Two layer: smart meter – PLC concentrator – Ethernet / GPRS

Single layer: Wireless: WAMS node –4G Wireline: a) WAMS node – IP stack and IP core b) WAMS node – Ethernet and L2 core

Two layer: PMU – PMU data concentrator (PDC) – IP

Applicable standards IEC 62052-11, IEC 62052-21, IEC 62053-21, IEC 62053-22, IEC 62053-23, IEC 62053-24, IEC 62058-11, IEC 62058-31, EN 50470-X ANSI C12.1, ANSI C12.20 M/490 (EU mandate)

As guidelines: IEC 61850 IEEE C37.118

IEC 61850 IEEE C37.118

Other features GPS or NTPv4 or IEEE 1588v2 synchronised

measurements, 1 s to 10 ms time stamp resolution

GPS synchronised

measurements, 1 s to 1 ms time stamp resolution

Table 17: WAMS node major attributes vs. smart meter and PMU solutions.

WAMS design sweet spot can be stated in a nutshell as: up to 100 times shorter reporting period as smart meter (10 times higher compared to PMU, though), with time synchronised additional measurement quantities and real time data transmission. WAMS node usage focus is in distribution grid. Whereas PMU are used in transmission grid, and only recently there were concerns raised as how to employ them also in distribution grid [Musc 2014]. WAMS can take the role of monitoring element in distribution grid, taking into considerations also:

Distribution grid observability and control (e.g. frequency of DC/AC converter of DER), even protection can be greatly enhanced with data coming from WAMS installed at key nodes in distribution grids.


Installed DER will push more distribution system dynamics and distortions (harmonics) compared to transmission systems thus requiring faster and denser tracking of vital smart grid parameters.

Accuracy of measurements are even more important due to smaller area of distribution grid, as we still require detection of parameter changes at different nodes of distribution grid (e.g. voltage drop, phase change), but more WAMS nodes may provide enough data for post processing with statistical methods.

WAMS design goal is to provide enough accurate measurement information, and be widely deployed in distribution grid with real time communication capability. We are pursuing design with more measurement capability compared to smart meter (Table 17) in terms of number of measurement quantities (e.g. harmonics) and quantity of measurement channels (e.g. 2 x 3 phases). Since all distribution grid nodes are polyphase, we require that N x 3 phase measurement is handled by single WAMS node. Time synchronisation of all measurements across distribution grid is required to achieve the same level of observability, gather enough state information and apply the same mathematical modelling as is used in transmission grid from PMU measurements. Furthermore, it is possible to reuse the same communication protocol and data model as PMU and interoperate such WAMS with existing PMU management packages. This will give us distribution grid state snapshots with reasonable fine grain reporting period (e.g. 1 s) that can be used in modern DEMS and coordination of pool DER or provide additional information for micro grids or VPP. Reporting period is a function of cost/performance of the measurement data acquisition building block (e.g. discrete antialiasing filters, ADC, time synchronisation vs. single chip smart meter solution) and communication transmission requirements with foreseeable technology (e.g. LTE). In WAMS design we are pursuing the single chip (or multiple identical for more measurement channels) data acquisition solution to achieve abovementioned reasonable reporting period. This future deployment of dense network of moderate bandwidth WAMS nodes requires rethinking of smart grid communication network, design approaches, topologies, technologies, operation, management to achieve economic benefits for smart grid stakeholders. WAMS node is composed of three main building blocks Figure 31

1. Core with data processing capabilities, storage. It realises measurement pre-processing and local analysis, control algorithms, encapsulates real time data into transport protocols (e.g. TCP/IP, IEEE C37.118) and authenticates with centrally located IoT/M2M platform.

2. Measurement and control. Realises key added value of the SUNSEED WAMS part: accurate measurements of multiple quantities and enables safe control of power loads (e.g. DR).

3. Communication interfaces. This block may be implemented within WAMS node itself or as an external router that connects to WAMS via Ethernet interface. This depends on the capabilities/price/performance issues arising where in distribution grid WAMS node is used.

Dual physical communication links (Table 18, Table 19) may be required to meet DSO demands for mission critical applications. Such approach solves the communication network availability issue, by providing core measurement node with dual physical communication links, preferably of different technology (e.g. PLC vs. xDSL vs. fiber and physical medium (power grid vs. communication network vs. wireless). Furthermore, it gives DSO the freedom and ability to choose the provider of communication network for the primary or secondary link. It was already discussed in Chapter Error! Reference source not found. that it is technically viable to have dual mobile connectivity. There are many more possible combinations that must be addressed on a regional specific basis, i.e. which telecom operators, municipalities, or public communication networks are available in the region that DSO operates in. however, we must be careful to select only those providers that operate their own physical infrastructure to really address the high availability and thus eliminate single points of communication failures. Note that the proposed arrangement must also be supported on the lower


protocol levels implementing protection switching (e.g. RFC3619, RPR) and on transmission protocol level to assure message persistency (e.g. message bus protocols, like 0MQ, RMQ).

Figure 31: WAMS node building blocks.

Communication design option Primary PHY

Secondary PHY

Application type, usage

Single PHY, wireline PLC xDSL

- AMI, PLC concentrator

Single PHY, wireline Fiber - PLC concentrator, SCADA low latency non critical

Single PHY, wireless GPRS, 4G - AMI, SCADA mid latency non critical

Single PHY, wireless SAT - High latency, non critical

Dual PHY, wireline-wireline

PLC xDSL fiber

PLC concentrator SCADA mid latency non critical

Dual PHY, wireline-wireline

fiber fiber Protection, SCADA low latency very critical

Dual PHY, wireline-wireless

PLC xDSL

GPRS, 4G WiFi

PLC concentrator SCADA mid latency non critical


fiber GPRS, 4G WiFi

SCADA low latency critical

Dual PHY, wireless-wireless

GPRS, 4G GPRS, 4G WiFi

SCADA mid-low latency critical


PLC xDSL fiber

SAT SCADA mid-low latency, non critical high latency backup


GPRS, 4G SAT PLC concentrator SCADA mid latency non critical


WiFi SAT PLC concentrator high latency non critical

Table 18: Smart grid communication types and technologies in distribution grid.

Communication Provider Provider Notes


infrastructure Primary PHY Secondary PHY

PLC DSO - DSO is physical owner of electrical distribution grid.

fiber DSO telecom 1 municipality

telecom 2 municipality

DSO places fiber on MV distribution grid. Municipality or local community fiber networks.

GPRS telecom 1 telecom 2 Dual access to two independent mobile networks. MVNO are not allowed.

4G DSO telecom 1

telecom 2 DSO builds its own mobile network or is as MVNO.

WiFi - DSO telecom 1 municipality

DSO builds its own WiFi network. Municipality or local community WiFi networks.

SAT 3rd party 3rd party Satellite backhaul is owned by 3rd party provider offering B2B or residential WAN access.

Table 19: Providers of communication infrastructure by type.

6.2.1.1 WAMS node traffic characterisation and separation for QoS

There are several requirements for WAMS node communication capabilities, regardless of physical interface type (e.g. copper xDSL, fiber, 4G, WiFi) in ascending order of priority:

1. Mark different types of traffic. 2. Place different types of traffic in separate queues. 3. Assign VLAN per queue associated with different traffic types. 4. Fairly schedule queues to single output (egress) port.

The best solution encompasses all three types of traffic handling and is so called CQS capable router (C = classification, Q = queuing, S = scheduling). First step for efficient solution to both is network traffic characterisation and marking [RFC2697], [RFC2698], [RFC4115]. Based on known traffic types found in smart grid networks, to date, we propose 8 VLAN types to logically separate (colour) traffic already at the lowest level, i.e. at the source itself (Figure 32) Traffic is assigned into multiple traffic queues. Queues are scheduled in pre assigned orders of priority (except FIFO) with some advanced scheduling technique, such as Weighted Fair Queuing (WFQ), Class-Based Weighted Fair Queuing (CBWFQ), Priority Queuing (PQ), Low Latency Queuing (LLQ) also called PQCBWFQ as CBWFQ with a Priority Queue (PQ). Such arrangement, with some or all of the capabilities listed, best solves smart grid traffic classification requirement already at the originating node. Ultimately communication solution must be interoperable with networks of different communication providers.


Figure 32: WAMS node logical communication paths logical separation.

The marking process only marks data traffic and eventually does prioritization of egress flow by queue scheduling algorithms. Prioritisation of marked and/or logically separated traffic from end node is done in the aggregation layer of the network. What communication interfaces can do is assign different buffer queues to each of the VLAN_type data traffic types and have the ability to adjust different queue depth in order to influence latency and/or jitter and to take into account the upper layer protocol usage (e.g. WRED for TCP) at the end node already. VLAN_Management is always present for remote management and configuration of WAMS node [BBF 2013]. Summary of the proposed requirements is in Table 20.

Usage Required VLAN_type Main objective Queue depth optimization

Notes

AMI VLAN_AMI VLAN_Management

Low performance

No specific requirements

Home DR management

WAMS VLAN_WAMS VLAN_Video VLAN_Internet VLAN_Management

Higher bandwidth, low latency variation

Minimize jitter WAMS node in distribution grid, located at substations, DER locations.

SCADA VLAN_SCADA VLAN_VoIP VLAN_Video VLAN_Internet VLAN_Management

Medium bandwidth

Low latency All present SCADA locations.

PMU VLAN_PMU Constant Lowest jitter All substation types.


VLAN_Management latency, higher bandwidth

Protection VLAN_Protection VLAN_Management

Lowest delay communication, reliability

Lowest latency Protection, control/command locations.

Table 20: Logically separate communication interfaces per type of WAMS node.

6.2.2 Phasor measurement unit

Our WAMS node design proposal takes notable features from PMU (i.e. time synchronised measurements, fast reporting periods), so we are including a generic PMU description in this chapter. The PMU is a power system device capable of measuring the synchronized voltage and current phasor in a power system. Synchronicity among Phasor Measurement Units is achieved by same-time sampling of voltage waveforms using a common synchronizing signal from GPS. The GPS signal is used to provide a time stamp for each measurement using coordinated universal time as the reference. After analogue to digital conversion, single-tone discrete Fourier transform (DFT) is applied to estimate phase angle difference (with respect to the reference PMU node), and magnitude, of the measured analog signal given samples taken at appropriate intervals. The ability to calculate synchronized phasors makes the PMU one of the most important measuring devices in the future of power system monitoring and control. The technology behind PMUs traced back to the field of computer relaying. In this equally revolutionary field of power system protection, microprocessor technology made possible the direct calculation of the sequence components of phase quantities, which are necessary for some fault detection algorithms. A PMU at a substation measures voltage with time tagging of when the measurement was taken. It also computes frequency, and rate of change of frequency (ROCOF). Measurements can be reported at a rate of 1-50 times per second. PMU technology is well suited to track grid dynamics in real time; this is a significant improvement over the SCADA/EMS whose refresh rate is seconds to minutes. Each utility has its own PDC to aggregate and align data from various PMUs based on the time tag. Measurements from each utility’s PDC are sent to the Central Facility.

Figure 33: PMU key building blocks.

The anti-aliasing filter is used to filter out from the input waveform frequencies above the Nyquist rate. The phase locked oscillator (PLL) converts the GPS 10 MHz reference clock into a sampling clock


required by the sigma/delta ADC. The microprocessor executes the phasor calculations. Finally, the phasor is time-stamped and uploaded to a collection device known as a data concentrator. PDC (Phasor data concentrator) plays strong role as collects, concentrate, correlate and synchronize phasor data samples. Samples with the same creation time are encapsulated in one packet, and then transmitted as a single stream to the phasor data server. In addition to that, PDC performs a number of quality checks, such as inserting the appropriate flag in the correlation of data, checking for disturbance flags and recording the data for offline analysis.

Figure 34: PMU data concentrator placement within power grid operator network.

Major benefits of using PMUs are presented in the following applications:

Real time monitoring and control

Power system state estimator

Post disturbance analyses

Power system restoration

Adaptive protection

Planned power system separation

Real time congestion management

Benchmarking, validation and fine-tuning of system models

6.2.3 Visualization with dynamic geographical information system

Figure 35: Dynamic GIS concept and actors.

Geographic Information System (GIS) can be used for a static representation of spatial topological datasets, e.g. electricity grid network. In Sunseed, on the other hand, we need highly efficient and dynamic GIS, capable of providing real-time data about energy and communication data flows within local smart-grid networks (Figure32).


mGISmap application works in many modern web browsers (IE, Firefox, Chrome, Safari) on different OS (Windows, Linux, OS X, Android, iOS), so all necessary data can be accessed from trained personnel within network management center, or technicians, installing WAMS nodes on the field. Data from databases (e.g. MySql, PostgreSQL/PostGIS) is rendered in real time (e.g. update interval 15 min) and presented in graphical form with all other spatial data layers. Sunseed will use several data sources, stored in databases, for efficient real time presentation of electrical and communications networks status and topology:

1. Electrical grid network topology (provided by EP) 2. Communications network topology of utility (provided by EP) 3. Communications network topology of telecom operator (provided by TS) 4. Mobile communication network radio coverage (provided by TS) 5. Cartographic map data of Slovenia (aerial imagery, topographic maps, linear features,

provided by GURS, GiS and Monolit) 6. Measurements from smart meters (set up on Sunseed field trial and historical data provided

by EP) 7. Measurements from WAMS nodes network (set up on Sunseed field trial) 8. Other relevant data (e.g. weather)

For the GIS platform which delivers geospatial information within the Sunseed project we use free and open source software (FOSS) web-based framework mGISmap. It incorporates various proved OSGeo [Osg 2014] projects for web mapping, such as MapServer [Maps 2014] and OpenLayers [Open 2014]. The authoring capabilities are provided by team of GIS specialists who use Quantum GIS [Qgis 2014] as the authoring tool of choice. Several geospatial libraries, e.g., GDAL/OGR, provide the means for displaying location-aware information in real time. The geographical data are encoded using the PostgreSQL-PostGIS combo [Gdal 2014], which allows users to execute interactive spatial queries arbitrary combining any displayed elements and/or data. For efficient handling of many simultaneous requests MapCache [Map 2014] is used for map-tile caching in order to minimize the delivery time of the read-only geospatial information, whereas the interactive queries are handed over to the MapServer rendering engine. In order to deliver a mature and professional look, we chose the AngularJS [Ang 2014] with Bootstrap [Boot 2014] for the in-browser GUI design elements, many of which were custom developed and adapted for the framework needs (Figure33).


Figure 36: Dynamic GIS building blocks.

The driving forces behind open GIS are some leading FOSS projects of OSGeo foundation that supports collaborative development of open source geospatial software. The main FOSS projects incorporated in Sunseed GIS application are:

MapServer: platform for publishing spatial data and interactive mapping applications to the web,

OpenLayers: pure JavaScript library for displaying map data in most modern web browsers, with no server-side dependencies,

PostgreSQL: powerful, open source object-relational database system,

PostGIS: spatial database extender for PostgreSQL object-relational database to adds support for geographic objects allowing location queries to be run in SQL.

GDAL/OGR: translator library for raster and vector geospatial data formats; it presents a single raster abstract data model and vector abstract data model to the calling application for all supported formats.

Additionally, Sunseed GIS application also uses:

AngularJS with Bootstrap: JavaScript application framework for building rich interactive web applications


6.2.4 FPAI nodes

In the SUNSEED project, the FPAI node in essence does nothing more than make a physical connection to a smart grid device, collect the information from the device, abstract the information to a common format specified in the EFI (Energy Flexibility Interface) and forward the abstracted information to clients that are interested in that information using XMPP. In the same way it also allows control of some of the smart grid devices by letting clients that want to control a device, send abstracted control information specified in the EFI to the FPAI node using XMPP. The FPAI will translate the abstracted control information to commands the device will understand and physically connect to the device to transfer the control signals. This is explained in more detail in D4.4.1. Within the SUNSEED project the FPAI nodes are used in two ways:

As measurement devices, for this a further distinction can be made into o PMU measurement devices o Power measurement devices

As control devices for demand response

FPAI as PMU measurement device When FPAI is used as PMU measurement device it will run on the hardware of a WAMS Node (see section 6.2.1) that is capable of performing PMU measurements (WAMS SPM). The FPAI driver interfaces with the WAMS node PMU measurements, abstracts the measurements and puts them in an XML format. This XML message is packaged into an XMPP packet and using the communication hardware of the WAMS Node this packet is sent via the XMPP server to clients that are interested in this these measurements. FPAI as Power measurement device When FPAI is used as a power measurement device it will also run on the hardware of WAMS node but this time a WAMS node that is only capable of doing power measurements (a WAMS PMC). The FPAI driver interfaces with the WAMS node power measurements, abstracts the measurements and puts them in an XML format. This XML message is packaged into an XMPP packet and using the communication hardware of the WAMS Node this packet is sent via the XMPP server to clients that are interested in this these measurements. Alternatively the FPAI could also run on an low cost device such as a Raspberry Pi and interface with a smart meter using a serial connection. This setup will most probably not be used in the SUNSEED project due to lack of arrangements with participating smart meter users in the SUNSEED field trial areas. FPAI as control device for demand response When FPAI is used as control device for demand response, it will again run on the hardware of a WAMS node, in this case the WAMS node that is capable of interfacing with external equipment/appliances. This is the same WAMS node that is capable of doing the power measurements, the WAMS PMC. The FPAI driver will connect to the appliance (via the interfaces provided on the WAMS PMC) and is able to collect information about the energy flexibility of the connected appliance (that its desired energy consumption and/or production over time and its ability to change its behaviour). This information is abstracted to a so called control space in XML format as defined by the EFI. This XML message is packaged into an XMPP packet and using the communication hardware of the WAMS Node this packet is sent via the XMPP server to clients that are interested in this information. A client receiving this information and desiring to control an appliance can send an allocation xml message (as defined by the EFI) packaged in an XMPP packet to


the appliance they want to control. FPAI will convert this abstracted allocation towards specific control signals for the appliance and send the control signals to the appliance using the interfaces of the WAMS PMC module. FPAI can be installed at different type of users. With respect to the demand response it can be installed in households as well as power plants while with respect to measurements it can be installed anywhere within the smart grid. The measurement frequency will depend on the type of application of FPAI. For demand response ‘measurments’ (or rather information that needs to be exchanged) will be available when an appliance/energy prosuming device knows when they will consume or produce energy or when an application wanting to control the device sends control information. For measurement devices FPAI will follow the frequency with which the device produces its measurements. The data format of the information transmitted by FPAI nodes has been converted from the original java classes to an xml interface Figure 37 shows, as an example, the xml schema for the message with wich a time shiftable appliance can update its information about its energy flexibility..

Figure 37: 38XML Schema representation of time shifert control space update message.

See D4.4.1 for a more comprehensive explanation of the FPAI data formats and the options of communication between the FPAI node and smart grid applications.


7 Conclusion

The key objective from DSO perspective within SUNSEED is to significantly improve the communication capabilities and cost effectiveness with converged DSO and telecom operator communication networks. The communication network convergence has to focus on current and emerging technologies that enable high data rates, to accomplish distribution grid observability in real time, QoS on multiple types of data streams, low latency with high reliability. Such converged communication network forms the foundation for the much needed next generation of power distribution smart grid network. Selected use cases implementations on the field trial will demonstrate and evaluate capabilities, efficiency and cost effectiveness of communication network operation, its converged integration concept, utilised technologies and business models. Advanced distribution network management system platform (ADNMSP) is the crucial use case for SUNSEED from DSO operational perspective. Based on ideas outlined in ADNMSP use case on the field trial, the distribution power network will be upgraded with advanced communication and measurement equipment (WAMS nodes) to improve observability of the distribution power network. This is the first step towards further implementation of advanced distribution management system control functionalities which characterized by smart grids. The ADNMSP use case scopes whole MV feeder from the main supply substation with all secondary MV to LV delivery substation (i.e. transformer station) and their belonging LV feeders supplying and connecting consumers, producers and prosumers. The core of ADNMSP is state estimation for real time and short term forecasted operational states in the power network. The operational data for state estimations come from traditional data acquisition devices in main supply substation (e.g. voltage and current transformer on the MV busbar) and from novel WAMS nodes. The state estimations are more precise by additionally including smart meters (AMI) measurement on the consumer side. LTE network simulations of TS' network has shown that there is more than enough capacity to support the expected required number of WAMS nodes. WAMS node has to be universal unit with standard interfaces to communication and information network assuring high security. This will simplify maintenance and reduce total cost of ownership considerably thus contributing to end goal of cost effective distribution grid observability. Our aim is that total cost of WAMS node devices and their integration do not significantly exceed the AMI smart meters system integration. In the field trial 4 basic connectivity scenarios will be used: WAMS (SPMs/PMCs) which are using mobile or fixed modem/router (Scenario A), WAMS SPMs which are using satellite modem/router (Scenario B), smart meters which are connected to PLC data concentrator (DC), which is connected to mobile or fixed modem/router (Scenario C) and smart meters which are connected to mobile or fixed modem/router via RS485 to Ethernet converter (Scenario D). The implementation of all developed concepts and technologies on the field trial have to meet all valid standards covering designing power system facilities (IEC 61850), transmission communications protocols in electrical engineering and power system automation applications (IEC 60870-5 104, DNP3), smart meters (e.g. IEC 62052-11, IEC 62052-21, IEC 62052-22, IEC 62052-23, IEC 62052-24, IEC 62058-11) and synchrophasor types of measurements (IEEE C37.118.1).


All developed technology with SUNSEED system architecture will be integrated in separate control unit with all basic prescribed SCADA protocols in terms of connecting to the DSO physical and logical communication network and data storage functionalities and integrated within the SUNSEED data flow system architecture for the field trial purposes. To achieve integration of multitude information flows within converged DSO – telecom operator communication network it is mandatory to strictly follow the good practices of traffic classification from source, WAMS node, throughout the network and to its destination in DEMS. Such operation can provide QoS as required by DSO for reliable and secure operation of distribution grid. MPLS will be the central logical interconnect from WAMS nodes to telecom operator testing and data servers and also to other entities (e.g. DSO). The extension of current EP and TS approaches will be implemented in the field trial. Most of the communication between WAMS nodes and the TS datacentre (where raw storage will be established) will be separated from public internet using multiple APNs (connection via mobile networks) and VPNs (xDSL and satellite connections). Core network data will be terminated with firewalls (for securing connectivity toward MPLS network) and load balancers (for distributing workloads across multiple computing resources).


8 References

[Agil 2011] Introducing LTE-Advanced, Agilent, 2011. [Ande 2010] R. Anderson, et.al., Adaptive Stochastic Control for the Smart Grid, SIP2010, IEEE. [App 2013] J. Appen, et. al., The Challenge of High PV Penetration in the German Electric Grid, IEEE Power and energy mag., Mar. 2013. [BBF 2013] TR-069 CPE WAN Management Protocol, Broadband Forum, Nov. 2013. [Birm 2005] K. Birman, et.al., Overcoming Communications Challenges in Software for Monitoring and Controlling Power Systems, Proc. Of IEEE, Vol. 93, No. 5, May 2005. [Bocc 2014] Boccardi, F., Heath Jr, R. W., Lozano, A., Marzetta, T. L., & Popovski, P. (2014). Vodafone. Communications Magazine, IEEE, 52(2), 74-80. [Chan 2012] S.C. Chan, et. al., Load/price forecasting and managing demand response for smart grids, IEEE Signal processing mag., Sept. 2012. [CEN 2012] Smart Grid Reference Architecture, CEN-CENELEC-ETSI Smart Grid Coordination Group, 2012. [CENst 2014] http://www.cenelec.eu [Cisc 2014a] QoS: Classification Configuration Guide, Cisco IOS XE Release 3S, Cisco, 2014. [Cisc 2014b] http://www.cisco.com/en/US/prod/collateral/routers/ps10906/ps380/ps11615/data_sheet_c78-678459.html; accessed 2.7.2014. [CMRI 2010] China mobile research institute, C-RAN: The road towards green RAN", C-RAN International Workshop, April 2010. [Corr 2014] Corrales Madueno, G.; Stefanovic, C.; Popovski, P. (2014, May)."Reliable Reporting for Massive M2M Communications with Periodic Resource Pooling," Wireless Communications Letters, IEEE , vol.PP, no.99, pp.1,1 [Dexi 2014] DEXi: A Program for Multi-Attribute Decision Making, http://www-ai.ijs.si/MarkoBohanec/dexi.html. [DLMSst 2014] http://www.dlms.com [Dyn 2014] http://dyn.com/dns/; accessed 2.7.2014. [EC 2011] Regulation on guidelines for trans European telecommunications networks and repealing Decision, No 1336/97/EC, EC, 19.10.2011. [EC 2012] Recommendation on preparation for the rollout of smart metering systems, 2012/148/EU, 2012. [EC 2013] Incorporating demand side flexibility, in particular demand response, in electricity markets, EC, 5.11.2013. [EC 2014] Directive 2014/61/EU on measures to reduce the cost of deploying high-speed electronic communications networks, 15.5.2014. [Eur 2013] New EU broadband proposals aim for synergies between telecoms and power grids, Eurelectric, 18.4.2013. [Fan 2013] Z. Fan, P. Kulkarni, S. Gormus, C. Efthymiou, G. Kalogridis, M. Sooriyabandara, Z. Zhu, S. Lambotharan, and W.H. Chin, “Smart Grid Communications: Overview of Research Challenges, Solutions, and Standardization Activities,” IEEE Communications Surveys & Tutorials, vol.15, no.1, pp. 21-38, 2013. [Gdal 2014] http://www.gdal.org/; accessed 4.7.2014. [GE 2000] Practical considerations in application of UCA GOOSE, GE, 2000. [Gras 2014] http://grass.osgeo.org/; accessed 4.7.2014.

http://www.cenelec.eu/

http://www.cisco.com/en/US/prod/collateral/routers/ps10906/ps380/ps11615/data_sheet_c78-678459.html

http://www.cisco.com/en/US/prod/collateral/routers/ps10906/ps380/ps11615/data_sheet_c78-678459.html

http://dyn.com/dns/

http://www.gdal.org/

http://grass.osgeo.org/


[Grie 2013] M. Griera, Synergies electricity telecoms, Innogrid2020+: The Transmission and Distribution R&D Seminar, 21.2.2013. [Gung 2006] V. Gungor, F. Lambert, A Survey on Communication Networks for Electric System Automation, 2006; www.ece.gatech.edu/research/labs/bwn/papers/2006/j4.pdf. [Hida 2011] N. Hidayatullah, et.al., Impact of Distributed Generation on Smart Grid Transient Stability, Smart Grid and Renewable Energy, May 2011. [Higg 2013] T. Higgins, “AC1900: Innovation or 3D Wi-Fi?” http://www.smallnetbuilder.com/wireless/wireless-features/32238-ac1900-innovation-or-3d-wi-fi [Hopk 2009] K. Hopkinson, et.al., Quality-of-Service Considerations in Utility Communication Networks, IEEE Trans. Power Delivery, Vol. 24, No. 3, July 2009. [Huaw 2014] http://www.4gltemall.com/huawei-b593-4g-lte-cpe-industrial-wireless-router.html; accessed 2.7.2014. [IDIS 2014] IDIS (Interoperable Device Interface Specification), http://www.ngusummitmena.com/media/whitepapers/LandisGyr_WhitePaper_IDIS.pdf. [IDISst 2014] http://idis-association.com [IEA 2011] Technology Roadmap Smart Grids, IEA, 2011. [IEA 2013] IEA: Renewables To Beat Out Natural Gas By 2016, http://www.renewgridmag.com/e107_plugins/content/content.php?content.10055#.UdQFvG0sa4o; accessed 28.6.2013. [IEC 2010] IEC Smart Grid Standardization Roadmap, IEC, 2010. [IEC850 2014] IEC 61850, http://www.iec.ch/smartgrid/standards/ [IECst 2014] http://www.iec.ch/ [IECSGS 2014] Core IEC Smart Grid Standards, http://www.iec.ch/smartgrid/standards/ [IEEE11 2014] IEEE 802.11 Working Group, http://www.ieee802.org/11/ [IEEE1646 2004] IEEE standard Communication delivery times performance requirements for electric power substation automation, IEEE, 2004. [IEEE19 2014] IEEE 1901 standards, available http://grouper.ieee.org/groups/plcsc/PLCSC/Home.html [IEEE37 2014] C37.118.1-2011 IEEE Standard for Synchrophasor Measurements for Power Systems, http://standards.ieee.org/findstds/standard/C37.118.1-2011.html [IEEEst 2014] http://www.ieee.org [Kim 2013] J. Kim, B. Lee, S. Han, J. Shin, T. Kim, S. Kim, and Y. Moon, "Study of the Effectiveness of a Korean Smart Transmission Grid Based on Synchro-Phasor Data of K-WAMS," Smart Grid, IEEE Transactions on , vol.4, no.1, pp.411,418, March 2013 [Khan 2011] H. Khan, A Novel Island Detection Methodology for the Realization of Smart Grid, Smart Grid and Renewable Energy, Nov. 2011. [LeM 2008] M. LeMay, An Integrated Architecture for Demand Response Communications and Control, Proceedings of the 41st Hawaii International Conference on System Sciences, 2008. [Mad 2013] Madueno, G. C., Stefanovic, C., & Popovski, P. (2013, June). How Many Smart Meters can be Deployed in a GSM cell?. In Communications Workshops (ICC), 2013 IEEE International Conference on (pp. 1263-1268). [Maps 2014] http://mapserver.org/; accessed 4.7.2014. [McDo 2003] J. McDonald, Substation automation IED integration and availability of information, IEEE Power & energy magazine, March 2003. [McGr 2005] M. Granaghan, F. Goodman, Technical and system requirements for advanced distribution automation, CIRED 18th International Conference on Electricity Distribution, 2005. [Meng 2014] W. Meng, R. Ma and H. Chen, “Smart Grid Neighborhood Area Networks: A Survey”, IEEE Network, vol. 28, no. 1, pp.26-32, January/February 2014.

http://www.ece.gatech.edu/research/labs/bwn/papers/2006/j4.pdf

http://www.smallnetbuilder.com/wireless/wireless-features/32238-ac1900-innovation-or-3d-wi-fi

http://www.4gltemall.com/huawei-b593-4g-lte-cpe-industrial-wireless-router.html

http://www.ngusummitmena.com/media/whitepapers/LandisGyr_WhitePaper_IDIS.pdf

http://idis-association.com/

http://www.renewgridmag.com/e107_plugins/content/content.php?content.10055#.UdQFvG0sa4o

http://www.iec.ch/smartgrid/standards/

http://standards.ieee.org/findstds/standard/C37.118.1-2011.html

http://www.ieee.org/

http://mapserver.org/


[Meti 2013] METIS Deliverable D6.2, Initial report on horizontal topics, first results and 5G system concept, https://www.metis2020.com/wp-content/uploads/deliverables/METIS_D6.2_v1.pdf; accessed 2.7.2014. [Meti 2014] METIS webpage: < www.metis2020.com>; accessed 2.7.2014. [Musc 2014] C. Muscas, Opportunities and Challenges for PMU Deployment in Distribution Systems, http://smartgrid.ieee.org/july-2014/1126-opportunities-and-challenges-for-pmu-deployment-in-distribution-systems; accessed 6.7.2014. [Nasc 2011] M. Nascimento, et.al., Virtual Routers as a Service: The RouteFlow approach leveraging Software Defined Networks, CFI ’11, June 2011. [NIST 2010] Introduction to NISTIR 7628 Guidelines for Smart Grid Cyber Security, NIST, 2010. [NIST 2012] NIST Framework and Roadmap for Smart Grid Interoperability Standards, NIST, 2012. [OFP 2014] OpenFlow, https://www.opennetworking.org/sdn-resources/onf-specifications/openflow; accessed 2.7.2014. [OIMLst 2014] https://www.oiml.org [Open 2014] http://openlayers.org/; accessed 4.7.2014. [OpSG 2013] OpenSG, “SG Network Systems Requirement Specification Release Version 5 (final),” Feb. 2013, available http://osgug.ucaiug.org. [ONF 2012] SDN, OpenFlow and the ONF; Open Networking Foundation, 2012. [Osg 2014] http://www.osgeo.org; accessed 4.7.2014. [Popo 2012] T. Popović, M. Kezunović, Measures of value, IEEE Power and energy magazine, Oct. 2012. [Paol 2014] E. Paolini, C. Stefanovic, G. Liva, P. Popovski, "Coded Random Access: How Coding Theory Helps to Build Random Access Protocols", submitted to IEEE Communications Mag., 2014. [Popo 2014] T. Popović, et.al., Implementation requirements for automated fault data analytics in power systems, Int. Trans. Electrical Energy Systems, 2014. [Prat 2013] Pratas, N. K., Thomsen, H., Stefanovic, C., & Popovski, P. (2012, December). Code-expanded random access for machine-type communications. In Globecom Workshops, 2012 IEEE (pp. 1681-1686). [Prim 2014] http://www.prime-alliance.org/ [Qgis 2014] http://www.qgis.org/en/site/; accessed 4.7.2014. [RFC2697] A Single Rate Three Color Marker, IETF, 1999. [RFC2698] A Two Rate Three Color Marker, IETF, 1999. [RFC3619] Ethernet Automatic Protection Switching (EAPS), IETF, 2003. [RFC4115] A Differentiated Service Two-Rate, Three-Color Marker with Efficient Handling of in-Profile Traffic, IETF, 2005 [RFC5905] Network Time Protocol Version 4: Protocol and Algorithms Specification, IETF, 2010. [RPR 2011] IEEE 802.17 Resilient packet ring (RPR) access method and physical layer specifications, IEEE, 2011. [SEC 2013] An introduction to the Universal Smart Energy Framework, http://ec.europa.eu/energy/gas_electricity/smartgrids/; accessed 9.7.2014. [Senc 2014] http://www.sencha.com/products/extjs/; accessed 4.7.2014. [Shik 2011] N. Shikhah, et.al., Medium-Term Electric Load Forecasting Using Multivariable Linear and Non-Linear Regression, Smart Grid and Renewable Energy, May 2011. [Strick 2011] E. Strickland, 24 Hours at Fukushima, IEEE Spectrum, Nov. 2011. [TI 2011] Creating cloud base stations with TI’s KeyStone multicore architecture, Texas Instruments, 2011. [Tile 2014] http://tilecache.org/; accessed 4.7.2014. [TDT 2014] http://www.tdt.de/en/products/wifi-dsl-isdn-vpn-router/c1550lw; accessed 2.7.2014. [UCA 1999] Utility Communications Architecture (UCATM) Version 2.0, IEEE SA TR 1550, 1999.

https://www.metis2020.com/wp-content/uploads/deliverables/METIS_D6.2_v1.pdf

http://www.metis2020.com/

http://smartgrid.ieee.org/july-2014/1126-opportunities-and-challenges-for-pmu-deployment-in-distribution-systems

http://smartgrid.ieee.org/july-2014/1126-opportunities-and-challenges-for-pmu-deployment-in-distribution-systems

https://www.opennetworking.org/sdn-resources/onf-specifications/openflow

https://www.opennetworking.org/sdn-resources/onf-specifications/openflow

http://openlayers.org/

http://osgug.ucaiug.org/

http://www.osgeo.org/

http://www.qgis.org/en/site/

http://ec.europa.eu/energy/gas_electricity/smartgrids/

http://www.sencha.com/products/extjs/

http://tilecache.org/

http://www.tdt.de/en/products/wifi-dsl-isdn-vpn-router/c1550lw


[Uci 2014] Integrated Energy and Communications System Architecture (IECSA), http://www.uci-usa.com/Projects/pr_List/Systems/Architecture/IECSA.html; accessed 14.7.2014. [WELst 2014] http://www.welmec.org [Wen 2012] J. Wen, et.al., Wide-Area Ethernet Network Configuration for System Protection Messaging, Georgia Tech Protective Relay Conference, April 2012. [Xu 2013] J. Xu, et.al., Towards Implementation of IEC61850 GOOSE Messaging in IEC61499 Environment, IEEE Conference on Industrial Informatics (INDIN’13), July 2013. [Zhu 2012] Z. Zhu, S. Lambotharan, W. H. Chin, Zhong Fan, "Overview of demand management in smart grid and enabling wireless communication technologies," Wireless Communications, IEEE , vol.19, no.3, pp.48,56, June 2012.

http://www.uci-usa.com/Projects/pr_List/Systems/Architecture/IECSA.html

http://www.uci-usa.com/Projects/pr_List/Systems/Architecture/IECSA.html

Date post:	19-Aug-2018
Category:	Documents
Upload:	hoangthien
View:	214 times
Download:	0 times

D2.1.2 Requirements and architectures for DSO...

Documents