Experience you can trust.

SCADA/EMS Cyber Security–An Industry Status

EMS Users Group

September 18, 2006

Joe Weiss, PE, CISM KEMA, Inc.(408) 253-7934

September 18, 2006 EMS Users Group 2

Why Are There So Few Experts

IT

IT Security Control Systems

Control System Cyber Security

September 18, 2006 EMS Users Group 3

“Progress”• Awareness

– End-users– Vendors– Government

• Definitions– DOS, cyber, etc

• Standards• Interdependencies• Solutions

– Hardware/software– Policies

• Leadership– Coordination

September 18, 2006 EMS Users Group 4

Standards

• Little coordination– eg, Multiple standards on IED security

• Inconsistencies– eg, Meet NIST 800-53 also meets NERC CIP, not

vice versa

September 18, 2006 EMS Users Group 5

Myths

• Firewalls make you secure• VPNs make you secure• Encryption makes you secure• IDSs can identify possible control system

attacks• Messaging can be one-way• Field devices can’t be hacked• You can keep hackers out• You are secure if hackers can’t get in• More and better widgets can solve security

problems• …

September 18, 2006 EMS Users Group 6

Cultural Change is Needed• Productivity considerations are pushing the use of

vulnerable systems and connections– Eliminating “Islands of Automation” can have

unexpected consequences• Operations and IT view the other as the risk

– Operations views O&M as their driver; security is an impediment

– Recent PI UG survey • Engineers like “toys” ; IT likes COTS

– Both can be vulnerable

September 18, 2006 EMS Users Group 7

Cyber Security is an On-going Process• System vulnerabilities and threats are constantly

changing– Any modification, integration, upgrade, or test can

affect cyber vulnerability– Vulnerability assessments are a snap-shot in time

• There is NO silver bullet– No single technology is sufficient to protect control

systems– Relevant control system security policies and

procedures are closest – Without appropriate policies, any technology can be

defeated

September 18, 2006 EMS Users Group 8

New Technologies can be Cyber Vulnerable• New technology and information flow is improving

productivity – Telecom including BPL, VOIP, Bluetooth, 802.11– RFID, Smart Dust– Reliability Centered Maintenance (RCM)/Machinery

monitoring– Smart grid, Substation automation, Automated meter

reading– Boiler control, Condenser/cooling tower system

optimization– Advanced field devices – System integration, Data warehousing– Nanotechnology

September 18, 2006 EMS Users Group 9

New Technologies can be Cyber Vulnerable• They will be used, but… they come with a price tag -

cyber vulnerabilities– Need to address how to best utilize these

technologies

September 18, 2006 EMS Users Group 10

Other Cyber Issues

• Dial-ups still being used with new equipment– Many dial-up connections are not even owned by

the end-user– War-dialing may not be possible if telephone line

installed by vendor• Use of wireless modems, web services,Telnet, and

other vulnerable applications in new equipment

September 18, 2006 EMS Users Group 11

Typical Cyber Vulnerabilities

• Disgruntled employee• Viruses/Trojans• Prohibited software• Vendor updates• Software malfunction• Hacker reconnaissance• Contractors• Inappropriate

policies/testing

• New/modified files• New sockets/new

processes• Removable media/games• Files modified• Process termination• NIDS alert• Rogue devices• Performance degradation

September 18, 2006 EMS Users Group 12

Generational Issues with Control Systems• Legacy equipment

– Security agnostic– Vulnerabilities backfit and security often turned off– Will be around for at least another 5 years

• New equipment– Vulnerabilities designed in– Will become pervasive in about 5 years for the

next 15-20 years• Future equipment

– Security and performance part of initial design criteria

– Probably about 20 years away before pervasive

September 18, 2006 EMS Users Group 13

Disclosure

• Minimal disclosures to “White Hat” community– Very few public cases– Reticence to disclose– Myths– FUD

• Technical disclosures to “Black Hat” community– Step-by-step instructions on how to hack Modbus,

DNP3, UCA, GOOSE– http://toorcon.org/2005/slides/mgrimes/mgrimes-

scadaexposed.pdf

September 18, 2006 EMS Users Group 14

SCADA Impacted• Event: Insecure GIS mapping system with no firewall

into SCADA led to vulnerability allowing targeted attack from Internet resulting in loss of SCADA

• Industry: Electric Transmission & Distribution• Location: North America• Information Source: SCADA Engineer’s presentation at

4th KEMA Cyber Security Workshop – August 2004• Impact: • No SCADA servers or mapping system for two weeks • Installation of firewalls, proxy servers, IDS and LAN

monitors• Neighboring utility networks went from trusted to

untrusted• 4 Man-months to recover• Lessons learned: • Isolate SCADA system from corporate LAN• Install firewall between the DSL router and the corporate

LAN • Install group of firewalls between the frame relay and

neighbors to isolate all ports that are not business-related

September 18, 2006 EMS Users Group 15

SCADA Impacts Plant• 350 MW Gas-fired power plant• Dispatch computer issued incorrect dispatching

requests• Unit dispatched for rapid load changes over a 3-hour

period• DCS maintains all control variables with ramp rates

approaching 40 MW/minute• GE 1000 hour cyclic life curve exceeded 3 times in 3

hours– Rate of temperature change averaged 1000oF/hr

with peak rates of 1600oF– GE curves extended only to 600oF/hr

• New ramp rate 28 mw/min

September 18, 2006 EMS Users Group 16

SCADA Vulnerability Demonstration • Objectives:

– Demonstrate cyber can remotely impact SCADA systems – Use encryption to camouflage the compromised data– Impact control and operator displays in a modern SCADA

system– Cause the operators to question HMI data

• How:– Simulate a small utility's transmission environment

consisting of six feeders. • Bad data was injected using an OPC client from the

hotel in Portland to the OPC server at PNNL(200 miles away)

• Feeder one was represented by the connection from Portland, feeders 2 through 5 were simulated using power flow software, and feeder 6 was connected to a relay

September 18, 2006 EMS Users Group 17

SCADA Vulnerability Demonstration

• Results– Bad data caused an alarm to trigger for feeder one– Feeders 2 through 5 were fed fictitious data over a

period of ten minutes with no alarms – HMI screens caused operator confusion– Feeder 6 caused relay mal-operation

September 18, 2006 EMS Users Group 18

UNIT SUBSTATIONS NOW WEB-ENABLED TO SIMPLIFY ACCESS TO POWER TRANSFORMER DATA

Aug. 29, 2005 – Equipped with an Ethernet interface and Web server, Vendor A Unit Substations now provide simple, affordable access to power system information – including transformer coil temperatures – using a standard Web browser. The pre-engineered equipment ships in standard lead-times and connects to a customer's existing Ethernet Local Area Network much like adding a PC or printer.

Unit substations include a Temperature Controller, which provides remote access to transformer data, in addition to its primary role in controlling cooling fans. With a simple click of a mouse, it is easy to monitor transformer coil temperatures per phase, and verify cooling fan status at a glance. Among the many potential benefits, these new capabilities make it possible to correlate circuit loading with transformer temperatures to extend equipment life.

The typical unit substation incorporates Medium Voltage Metal-Enclosed Switchgear on the primary side and Low Voltage Switchgear or Low Voltage Switchboard on the secondary.

Vendor A was the first manufacturer in the world to embed an Ethernet interface and Web server into its power distribution equipment, allowing customers easier access to power system information. The family of power distribution equipment includes medium and low voltage switchgear, unit substations, motor control centers, switchboards and panelboards.

September 18, 2006 EMS Users Group 19

Other New Technologies

September 18, 2006 EMS Users Group 20

September 18, 2006 EMS Users Group 21

Summary

• Leaping from mid-80’s to mainstream networking technologies has advantages and disadvantages– We need to understand them enough to make

prudent decisions or we will become less secure• We need to be able to specify security in products

and employ relevant best practices– This requires an understanding of security and

system performance