Substation and IoT-Smart Building Testbed

Jeffrey Reed, Bob McGwier, Ryan GerdesElectrical and Computer Engineering & Hume Center, Virginia Tech

S2ERC Showcase November, 2017

false-data injection

false-data injection

controller

actuation

false-actuation injection

plant (dynamics) simulator

(system states & control input)sensor readings/

actual state

sensor readings/ actuation

database

simulated sensors/actuators

hardware-in-the-loop simulation

cyber attack

related work:1. lack hardware-in-the-loop simulation

(limited to select plants)

2. lack industry standard components3. primarily on cyber threats4. single system in a system

LAN/WAN

7

4) Multilin D400

Advanced Substation Gateway

GE's Multilin™ D400 is a secure, substation-hardened gateway that collects metering, status, event, and fault report data from serial or LAN based Intelligent substation devices. The D400 summarizes data from the substation devices and makes it available locally/remotely through a standard secure web browser (HTTPS).

It supports serial and/or LAN connections to SCADA masters. TCP/IP network connections are supported over the built-in Ethernet and the modem interface.

Key Benefits

• Increase the operational and non-operational analog data management through Analog Report Generation capability

• Increase the availability and reliability of data through the presence of a third Ethernet interface, hot-standby functionality and Parallel Redundancy Protocol (PRP), IEC 62439-3 Ed 2 scheme

• Advanced security features including centralized user authentication (LDAP & TACACS+), access control, and auditing enabling compliance with latest NERC/CIP requirements

• Full suite of SCADA protocols allow for direct communications with Master Stations

• Secure pass through / terminal services allow personnel to access all substation devices, remotely

• Offline editor and secure LogicLinx connection provides simplified device configuration utilizing powerful SGConfig software toolset

• Direct support for industry standard communication protocols (including IEC 61850) ensures connectivity with new and legacy substation devices

• Graphical configuration with pre-configured device maps, offline editor, and secure LogicLinx connection simplifies device configuration utilizing SGConfig software toolset

• Application flexibility with complete IEC 61131 SoftLogic Capabilities

• Enable predictive maintenance with local or remote access to device status, annunciator, and data log

• Advanced HMI including File Explorer interface and streamlined Alarm page, simplifying device and system management

• Supports automatic record retrieval from IEDs via TFTP, FTP, SFTP, 61850 MMS, and SEL ASCII

Applications

• Advanced Gateway - Substation Data collection, concentration and visualization

• Advanced Automation - Automate substation procedures using IEC 61131 compliant tools

• Fault Recording & Data Logging - Extract valuable data such as digital fault records and event files

• Secure Remote Access - Securely access substation device locally and remotely

MultiLin D400 GW

12

• Alarm Relays for Monitoring Systems • Aside from extensive software based alarms, the hardware based alarm of the S300 relays

further enhance Alarming choices. • Customer Tested Time Server Design • The 5th generation Microsemi SyncServer S300 has decades of design experience behind

it. Customer input is evident in every detail. From the front and rear panel design configurations to the state-of-the art web interface, the S300 is unsurpassed in reliability, accuracy, security and ease of use.

• Upgrade to IEEE 1588 / PTP • All S300 SyncServers are IEEE 1588 / PTP grandmaster ready with built in hardware

based, nanosecond caliber time stamping. An optional key code enables the PTP operations on the LAN2 port. Order your S300 with the PTP option or upgrade later when you are ready to deploy PTP on your network.

• Bottom Line • The SyncServer S300 is your answer to bringing perfect timing and therefore the highest

performance to your network - securely, reliably and easily.

9) Multilin D20MX Substation Controller

Distribution and transmission industries are under pressure to ensure that their grids are reliable and to prolong the usability of their assets. Data from these assets can be collected, aggregated and processed, to provide visibility of system conditions. The Multilin™ D20MX Substation Controller is a specialized computing platform designed to execute communications and energy management applications for the monitoring and control of electrical substations. The D20MX is capable of amalgamating data from multiple slave devices and D20 I/O modules connected via communication channels into a single database using various protocols. The D20MX can execute local logic, compile data, process it through one of multiple applications and report the results upstream to master stations through different server protocols.

Key Benefits:

• Reduces legacy D20 RTU upgrade expenditures by over 50% through backwards compatibility with existing D20 installed accessories, such as chassis, modems and D20 I/O peripheral modules

• Minimizes operation and maintenance costs of existing D20-based SCADA infrastructure by leveraging existing designs, processes and infrastructure

• Introduces a new and modern network security feature suite that enables effective compliance with NERC-CIP requirements through the application of native cyber security features built into the D20MX Substation Controller

MultiLin D20MX Substation Controller

11

8) SyncServer S300 Network Time Server • High Performance, Enhanced Security GPS Network Time Server (NTP or PTP) • Faster, more accurate, more secure, increased redundancy, and maximum reliability -

these features all contribute to higher levels of performance and assure optimal integrity within the Next Generation IT network.

• • Overview • The SyncServer ® S300TM is a high performance, enhanced security enterprise class GPS

Network Time Server. It sets standards for security, accuracy, reliability, and redundancy in network time servers.

• Unmatched High Performance with Unparalleled Flexibility and Security • The S300 has four dedicated and isolated Ethernet ports, one of which is Gigabit Ethernet

- more than enough to meet the need of servicing thousands of NTP requests per second while maintaining microsecond caliber timestamp accuracy. Multiple ports provide the flexibility to adapt to different network topologies as networks grow and change. It supports a wide range of network protocols including IPv4 and IPv6, for easy management and seamless integration into your existing and future network. An optional upgrade to IEEE 1588 PTP grandmaster operations with hardware based, nanosecond accurate time stamping is available.

• Enhanced Security - Secure and Easy Network Integration and Management • The S300 provides very reliable and secure network synchronization technology by

combining multi-port network interfaces with multiple time reference technology and enhanced security protocols. TACACS+, RADIUS, SSL, Autokey, MD5, passwords, access control lists and more are standard for maximum security. All of the expected network management and monitoring protocols are standard in the S300.

• Easy To Set Up and Maintain • SyncServers are the easiest to set up and maintain network time servers in the world. The

front panel of the SyncServer S300 is designed to quickly bring the server online with a few front panel keystrokes or DHCP. To fully configure the unit, use the very intuitive web interface or the step-by-step wizards for the most common operations.

• Redundancy and Time Assurance • The internal modem is standard to connect directly to legal time provided by national

time authorities. An optional AM radio is available to synchronize to national time broadcasts, which can be an alternative to GPS when GPS is not viable option.

• Assured Perfect Timing • The Stratum 1 level S300 derives extremely accurate time directly from the atomic clocks

aboard the GPS satellite system. Reliability is further enhanced via Stratum 2 operation by retrieving time from other user-designated time servers. All SyncServers can be upgraded to an internal Rubidium atomic oscillator to keep the time server accurate if the GPS signal is lost.

S300 Network Time Server

13

10) Multilin F60 Feeder Protection System

The F60, a member of the UR Family of protection relays, provides high performance feeder protection,

control, monitoring and metering in an integrated, economical, and compact package. The F60 includes GE

Multilin’s unique high-impedance fault detection for fast and reliable detection of downed conductors.

Key Benefits • The most flexible protection and control device for distribution feeder applications

• Advanced IEC61850 Ed. 2 certified implementation, complete settings via SCL files and IEC 61850-9-2 process bus solution enable resource and platform managing optimization and reduce cost of ownership

• Routable GOOSE (R-GOOSE) enables GOOSE messages going beyond the substation, which enables wide area protection and control applications

• Unique and secure downed conductor detection

• Flexible load encroachment allows secure operation during heavy load conditions

• Advanced automation capabilities

• Ambient temperature monitoring with alarms

• Voltage and frequency elements to provide load shedding and transfer schemes

• Application flexibility with multiple I/O options and programmable logic (FlexLogic)

• Robust network security enabling critical intrastructure protection

• Advanced fault and disturbance recording

• High-speed inter-relay communications reducing relay-to-relay wiring and costs

• Phasor Measurement Unit (synchrophasor) according to IEEE® C37.118 (2011) and IEC® 61850-90-5 support

• Supports English, French, Russian, Chinese, Turkish and German languages on the front panel, software and manuals

• Supports latest edition of waveform capture (COMTRADE 2013) simplifying fault records management

MultiLin F60 Feeder Protection System

6

3) Multilin ML3000 Managed Switch 19'' Rack-mounted Managed Switch with IEEE 1588v2 Timing

Managed Ethernet Switches The MultiLink™ ML3000 Series of managed Ethernet switches has been designed for the specific requirements of devices used in utility and industrial environments. The MultiLink ML3000 Series includes the ML3000, ML3001, ML3100 and ML3101, and supports many unique features that allow for full redundancy under network fault conditions.

Key Benefits

• Supports up to 32-ports Power over Ethernet, reducing wiring complexity and cost (model dependent)

• Supports 1588v2 for high precision timing applications

• Models with optional field replaceable power supplies available

• High density substation Ethernet switch

• Up to 36-ports copper (model dependent)

• Up to 18-ports fiber (model dependent) Key Features Industrially Hardened

• UL listed/CE agency approved

• IEC 61850 and IEEE 1613 approval for operation in electric substation environments

• Redundant and mixed power supply options for increased reliability

• Harsh chemical environment options ensures product function and viability Secure

• Secure management via SSL

• Port security prevents unauthorized devices from gaining access to the network

• Multi-level passwords with levels of privilege and command for different users or groups

• Complete event logging for forensic and regulatory auditing and reporting Managed Networks

• Supports 1588v2 timing and the C37.238 Power Profile

• Supports SNMPv3 with full backwards compatibility for v1 and v2

• Traffic segregation and prioritization control via IEEE 802.1p and IEEE 802.1Q

• Hardware and software alarm contacts for detection of critical network or switch events

• LLDP to support topology discovery in Network Management Systems (NMS) Ease-of-use

• Support for industrial protocols (e.g. Modbus)

• IP out-of-the-box for easy installation and setup

MultiLin ML300 Switch

14

Applications • Primary protection and control for feeders on solidly, impedance, or resonant (Petersen coil) ground

systems

• Busblocking/interlocking schemes

• Distribution load shedding schemes based on voltage and frequency elements

• High-speed fault detection for arc flash mitigation

• Throwover schemes (bus transfer scheme applications)

• Backup protection for transmission lines, feeders, and transformers

Distributed generation (DG) interconnect protection

11) SEL-734 Advanced Metering System California ISO tested and passed the SEL-734 for revenue metering use in ISO-metered entity applications. This certification process verified metering accuracy, communications, and integration capability with the California ISO communications system. High-accuracy metering, Ethernet communications, and Itron® MV-90® compatibility enabled the SEL-734 to exceed ISO requirements, making it ideal for grid intertie metering applications in California. The affordability of the SEL-734 will allow California utilities to increase the coverage of revenue and power quality metering and, ultimately, to improve power quality. The SEL-734 includes advanced revenue metering and communications capabilities that provide additional value for the utility.

SEL-734 Adv. Metering System

15

12) SEL-734T Advanced Digital Transducer

Replace monitors and transducers; monitor real-time power, energy, and power quality; and communicate these data in real-time with the new surface-mount SEL-734T Advanced Digital Transducer. Priced at just $1,144, the surface-mount transducer provides a low-cost monitoring and control solution for distribution and integrated power systems.

SEL-734T Digital Transducer

16

13) 2*SEL-735 (Power quality and revenue meter)

This product’s metering accuracy and the ability to integrate with the California ISO communications system is verified. The SEL-735 exceeds ISO requirements through its high-accuracy metering, Ethernet communications, advanced load trending, and Itron MV-90 compatibility, making it ideal for grid intertie metering applications in California. With these features, the SEL-735 integrates with practically any billing system.

Building on the SEL-734 Advanced Metering System, the SEL-735 adds improved ease-of-use and metering accuracy at a lower price. The affordability of the SEL-735 allows California utilities to increase the coverage of revenue and power quality metering, improving efficiency and power quality. The SEL-735 includes

advanced revenue metering and communications capabilities that provide additional value for utilities, exceeding accuracy class 0.2 requirements.

SEL advanced metering systems meet or exceed the strictest ANSI, UL, and IEC requirements of utilities and industrial companies worldwide. SEL metering solutions can measure and control any of today’s new methods of generation. They also lead in versatility, communications, price, and support. In addition to indoor mounting options, prewired outdoor enclosures from SEL facilitate high-end metering at virtually any location.

SEL-735 Power Quality Meter

Power Input (from Utility)

24

• Programming with SCL: Like the other controllers, the S7-1200 can now also be programmed with the high-level language SCL (Structured Control Language).

The S7-1200 range incorporates 4 different CPU's offering a variety of performance and IO options. Each CPU is available with a choice of supply voltage AC or DC, and with AC, DC, or relay outputs.

17) Multilin EPM 9900 Power Quality Meter

High Performance Power Quality and Transient Recorder Meter

The Multilin™ EPM 9900P is one of the most advanced monitoring products on the market today, providing a comprehensive perspective of energy usage and power quality metrics for critical energy circuits. Features such as 0.06% accuracy, available high speed 50MHz Transient Recorder, advanced communication interface/protocol capabilities, and up to 4GB of logging make the EPM 9900P perfect for industrial and utility substation automation applications where both power quality monitoring, high accuracy and easy integration/commissioning are required.

Key Benefits

Ideal for revenue and power quality monitoring in applications such as of utility substations, advanced industrial manufacturing, datacenters and hospitals with high resolution transient recording (up to 50Mhz) and high accuracy 0.06% Watt/Hr energy metering with demand and time of use capture.

Constant Calibration metrology self calibrates every 10 seconds ensuring highly stable readings

Large 4GB memory makes it possible to log years of captured data

Easy integration with flexible communications options and protocols supporting simultaneous Modbus, DNP and IEC 61850 communications

MultiLin EPM 9900 Power Qual. Meter

Electric Arc Furnace Datacenter

10

The main chassis contains the central CPU(s) and communication ports and provides the data concentration, protocol conversion, and customizable local automation functionality. Available D20 main chassis include: • D20 main chassis - Single or 2 CPU options available • D200 main chassis - Up to 7 CPUs

A complete family of substation hardened I/O modules makes the D20 controller scalable to both large and small substations. A distributed architecture with both ring and star topologies allows for easy expandability and remote placement of I/O modules. The following I/O modules are available in a variety of models with I/O ranges up to 300VDC. • D20S - 64 channel status input module • D20A - 32 channel DC Analog input module • D20K - 32 channel control output module • D20KI - 8 external interposer relay pairs module • D20C - 16 status input, 8 control output, optional 16 DC analog inputs or 8 analog inputs and 8 analog outputs • D20AC - 15 channel direct AC input, 1 DC analog input module

The DNP I/O modules are data collection and control devices that combine the DNP 3.0 communications protocol with our existing industry proven and robust line of I/O modules. The DNP I/O modules implement the industry standard DNP 3.0 (Level 2) slave protocol over a single RS485 interface to provide expandable I/O functionality for the D400, D25 and iBox product lines, as well as third-party products. The following DNP I/O modules are available: • DNPIO-DI (D20S) • DNPIO-DCAI (D20A) • DNPIO-CO (D20K) • DNPIO-C (D20C)

D20 I/O

9

6) QUINT POWER

The QUINT-PS/1AC/24DC/10 is a 1-phase primary-switched Power Supply with SFB technology. Compact power supply units of the new QUINT POWER generation maximize the availability of your system. Even the standard power circuit-breakers can be tripped reliably and quickly with the SFB technology (Selective Fuse breaking Technology) and six times the nominal current for 12ms. Defective current paths are disconnected selectively, the defect is limited and the important system parts remain in operation. A comprehensive diagnostics is carried out by continuously monitoring the output voltage and current. This preventive function monitoring visualizes the critical operating modes and reports them to the control unit before an error occurs.

• Quick Tripping of Standard Power Circuit breakers with Dynamic SFB Technology Power Reserve

• Reliable Starting of Difficult Loads with Static POWER BOOST Power Reserve • Preventive Function Monitoring • Can be used World wide • High degree of operational safety due to high MTBF >500000h, Long mains buffering

times

Device contains dangerous live elements and high levels of stored energy, never carry out work when the power is turned on.

7)WESDAC D20K, WESDAC D20A, WESDAC D20C, 2 WESDAC D20S

The D20 substation controller offers an industry leading design embedded with high value

substation automation applications that provide cost savings, increased reliability, and improved operational efficiencies in your substations. The D20 device acts as the gateway to SCADA master stations for IEDs in the station, or for downstream substations or feeders. A large protocol library facilitates communication to most existing substation devices for improved visibility and remote control. The distributed, expandable I/O architecture and mission-critical automation control applications reinforce why the D20 controller is being used in over 40,000 installations around the world.

Quint Power Supply

Building Model

18

15) Twido Twdlmda40duk

range of product

Twido

product or component type Modular base controller

discrete I/O number 40

discrete input number 24

discrete input logic Sink or source

discrete input voltage 24 V

discrete input voltage type DC

discrete output number 16 for transistor (sink)

[Us] rated supply voltage 24 V DC

Twido PLC

23

16) SIMENS S7-1200

S7-1200 PLC

The SIMATIC S7-1200 controller is modular and compact, versatile, a secure investment, and perfectly fits a wide variety of applications. A scalable and flexible design, a communication interface that fulfills the highest standards of industrial communicat ion and a full range of powerful integrated technology functions make this controller an integral part of a complete and comprehensive automation solution.

Highlights • Scalable and flexible design:

The SIMATIC S7-1200 controller family has been designed with maximum flexibility to fit your individual machine requirements. This allows you to custom design your controller system to meet your needs; it also makes future system expansions quick and easy.

• Integrated Industrial Ethernet/PROFINET interface: The Industrial Ethernet/PROFINET interface integrated into SIMATIC S7-1200 offers seamless communication with distributed I/O with SIMATIC HMI Basic Panels for visualization and additional controllers for CPU-to-CPU communication. Also with devices from third parties for extended integration possibilities as well as the SIMATIC STEP 7 Basic engineering system for configuring and programming.

• Integrated technology functions: The name SIMATIC has been synonymous with reliability in the field of automation for many years. Based on long years of experience, we have integrated our proven and innovative technology functions into our new controller – ranging from counting and measuring, speed, position and duty cycle control to simple process control functionality. This wide variety of functionality enables you to solve a wide array of applications.

Siemens S7-1200 PLC

8

5) CPU 65150 PENTIUM CONTROLLER

Capacity 166MHz

Catalog Description Programmable Logic Controller Processor (CPU) (Modicon Quantum), 512Kb, 166MHz, (1) RS 232/485 Modbus/ASCII - (1) Modbus Plus - (1) Ethernet TCP/IP - (1) USB port

Catalog Number 140CPU65150 Country of Origin FR EU RoHS Indicator Y GTIN 00785901758488 Gross Weight 3.200 Invoice Description English QUANTUM PROCESSOR- UNITY W/ 1024/7168K

Mfr/Vendor Square D by Schneider Electric Program Memory 512Kb Select Code AUTOMATION ShortDescriptionStripped Schneider-Electric-Square-D-140CPU65150-Quantum

Special Features (1) RS 232/485 Modbus/ASCII - (1) Modbus Plus - (1) Ethernet TCP/IP - (1) USB port

Sub Brand Modicon Quantum Type Processor (CPU) UPC 785901758488 description SQD 140CPU65150 QUANTUM PROCESSOR-

65150 PLC

HVAC System proposed testbed

5

2) Multilin D.20TMRIO (Distributed I/O Controller)

GE's Multilin D.20 RIO Distributed I/O Controller is a stand-alone, small form factor device designed to provide distributed I/O capabilities for easy connection to the Multilin D400™ gateway through any point in the substation LAN. The D.20 RIO provides an interface to GE's Multilin D20 Series of I/O modules.

Key Benefits

• Reduced copper wiring between I/O modules and substation controllers by adding I/O near the monitored device and communicating to the D400 over the substation LAN • Easy installation of the small form factor D.20 RIO into existing control panels • Cost effective deployment of new and retrofit substation automation projects through compatibility of D400 Substation Gateways and Multilin D20 Input / Output peripheral modules Application • Optimize life cycle management of aging D20 RTUs. The D20 module interface enables users to maintain and leverage existing engineering designs, processes and automation infrastructure • Simplify implementation of distributed substation automation architectures by installing I/O where it's needed and reducing copper wiring between I/O and substation controller

MultiLin Distribute I/O

year one: a testbed that is adaptable to many building infrastructures that focuses on one with a substation

1. Creation of a hardware-in-the-loop architecture to facilitate interaction of hardware components with simulated plant models, to include building

temperature dynamics and power consumption 2. integration of existing hardware into “building in a box”

3. preliminary vulnerability of assessment of components/processes integrated into the testbed.