Rome, May 31, 2011

Jonathan Pollet conference

Part 1: Control System Vulnerabilities

control system vulnerabilities > analysis of 5 years of field data

Jonathan Pollet, CISSP, CAP, PCIP

Red Tiger Security

[on behalf of the DHS CSSP program - contract #240704]

1

Jonathan Pollet – CISSP, PCIP, CAP

2

�  12 Years of Electrical Engineering, SCADA, Industrial Controls, and IT Experience �  PLC Programming and SCADA System Design and Commissioning �  Wireless RF and Telecommunications Design and Startup �  Front-end Web Development for SCADA data �  Backend Database design for SCADA data �  Acting CIO for Major Oil Company for 2 years – Enterprise IT Management

�  Last 8 Years Focused on SCADA and IT Security �  Published White Papers on SCADA Security early in 2001 �  Focused research and standards development for SCADA Security since 2002 �  Conducted over 120 security assessments on Critical Infrastructure systems �  Conducted over 75 International conferences and workshops on CIP �  Developed safe security assessment methodology for live SCADA Systems �  Co-developed the SCADA Security Advanced 5-day training course

outline �  background on the project

�  review of ISA99 architecture model

�  source for data used in the analysis

�  interesting results �  avg. # of days between vulnerability disclosure and discovery �  where in the architecture are most vulns being discovered �  does the type of vulnerabilities change throughout the architecture �  workstation HMI vulnerabilities ranked by OS �  network vs. host/application vulns throughout the architecture �  interesting security findings on control system networks

�  Q & A

3

project background �  Over 38,000 control system vulnerabilities collected over 5 years

from mid-2002 to 2008

�  Over 100 security assessments performed on critical infrastructure facilities such as electric power generation plants, transmission energy control centers, chemical plants, water plants, and oil/gas production, refining, and pipeline systems

�  Vulnerability analysis and classification conducted under research project facilitated by INL and funded through the DHS Control Systems Security Program contract #240704

�  ISA99 architecture model used to classify where the vulnerabilities were discovered in the systems

4

5

data source – what was collected? �  From mid-2002 to 2008, vulnerability data was stripped of any client

information and the raw vulnerabilities were captured in a database �  Vulnerability ID (auto-numbered from entry number 1)

�  Vulnerability Title (title for the vulnerability)

�  Security Zone or Location (location based on the ISA99 model where the vulnerability was located)

�  Disclosure Date (date when vulnerability was disclosed) �  Discovery Date (date when vulnerability was discovered by the team and

entered into the database)

�  Days Between Disclosure and Discovery (time between disclosure and detection)

�  Vulnerability Detailed Description

�  Vulnerability Suggested Remediation Steps

6

interesting results �  avg. # of days between vulnerability disclosure and discovery

�  all field data was exported from the database to an excel spreadsheet containing over 38,000 rows, and much of the analysis had to be performed manually

�  since we captured when the vulnerability was disclosed in the public, and also captured when the vulnerability was discovered and entered into the database, we were able to perform a simple diff against these two fields

�  vulnerabilities that were never disclosed in the public were thrown out of this particular exercise since negative or zero entries would throw off the calculations

�  the maximum number of days between when a vulnerability was disclosed in the public and when it was found during an assessment was over 3 years!

�  the average was 331 days, or close to 1 year. this means that on average most SCADA and process control environments contained latent vulnerabilities, probably with compiled exploits, and were not discovered until almost a year later, and would not have been discovered had not the asset owner funded the assessment.

7

where are the vulnerabilities being discovered?

8

16,9%

24,7%

46,3%

11,8%

0,3% 0,0%

Vulnerabilities by Location in Architecture

Level 5 - Internet DMZ zone

Level 4 - Enterprise LAN zone

Level 3 - Operations DMZ

Level 2 - Supervisory HMI LAN

Level 1 - Controller LAN

Level 0 - Instrumentations bus network

does the type of vulnerabilities change throughout the architecture? �  classified each vulnerability by the system that was impacted and where

the vulnerability was found in the architecture

�  The data set emerged a common set of system types at each network zone or segment: �  Email Server Applications

�  Web Server Platforms (Apache and IIS)

�  Business Applications

�  Shopping Cart Applications

�  Applications written on PHP platform

�  Applications written on ASP or .NET platform

�  Database Servers (MS SQL, mySQL, and Oracle)

�  FTP Servers

�  Portal Servers (Blogs, Forums, etc…)

�  Workstation (client) vulnerabilities

9

systems impacted at the Internet DMZ zone

10

12,7%

23,3%

7,8%

2,2%

25,8%

10,0%

5,6%

1,2% 11,4%

0,0%

Internet DMZ Vulnerabilities Email Server Applications

Web Server Platforms (Apache and IIS)

Business Applications

Shopping Chart Applications

Applications written on PHP platform

Applications written on ASP or .NET platform

Database Servers (MS SQL, mySQL, and Oracle)

FTP Servers

Portal Servers (Blogs and Forums)

Workstation (client) vulnerabilities

systems impacted at the Enterprise LAN zone

11

9,7%

19,3%

23,4%

5,0%

4,6%

5,9%

12,6%

1,2% 5,9%

12,5%

Enterprise LAN Vulnerabilities Email Server Applications

Web Server Platforms (Apache and IIS)

Business Applications

Shopping Chart Applications

Applications written on PHP platform

Applications written on ASP or .NET platform

Database Servers (MS SQL, mySQL, and Oracle)

FTP Servers

Portal Servers (Blogs and Forums)

Workstation (client) vulnerabilities

systems impacted at the Operations DMZ zone

12

5,5%

41,4%

15,3% 1,1%

1,5%

2,3%

19,8%

3,9%

3,3% 6,0%

Operations DMZ Vulnerabilities Email Server Applications

Web Server Platforms (Apache and IIS)

Business Applications

Shopping Chart Applications

Applications written on PHP platform

Applications written on ASP or .NET platform

Database Servers (MS SQL, mySQL, and Oracle)

FTP Servers

Portal Servers (Blogs and Forums)

Workstation (client) vulnerabilities

workstation HMI vulnerabilities ranked by OS

13

62,2%

0,9%

2,2%

6,7%

8,3%

2,4%

4,4% 11,5%

1,4%

Supervisory HMI LAN Vulnerabilities Microsoft-based Operating System or Applications

Red Hat Linux Operating System or Applications

Tru64 Operating System or Applications

HPUX Operating System or Applications

IBM AIX Operating System or Applications

FreeBSD Operating System or Applications

SCO UNIX Operating System or Applications

Sun Solaris Operating System or Applications

SuSE Linux Operating System or Applications

only logged 105 controller LAN vulnerabilities, but QnX showed up as the most typical source

14

19,0%

65,7%

15,2%

Controller LAN Vulnerabilities

Vulnerabilities in Controller LAN due to Phone/Telecom Equip

Vulnerabilities in Controller LAN due to QNX

Misc. Vulnerabilities

network vs. host/application vulns throughout the architecture

15

3,4% 3,3% 9,3%

35,4%

95,1% 96,7% 90,7%

64,0%

Level 5 - Internet DMZ zone Level 4 - Enterprise LAN zone

Level 3 - Operations DMZ Level 2 - Supervisory HMI LAN

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Network versus Host/Application Vulnerabilities by Location in Architecture

Host/Application

Network

interesting security findings on control system networks

�  VOIP (Voice over IP) Systems

�  Network Video Recording Devices

�  Network Surveillance Equipment and Software

�  Adult Video Directory Scripts

�  Online Dating Service Databases

�  Advanced Forensics Format (AFF) archives

�  Gaming Software Servers

�  aGSM - a freeware game server info monitoring utility

�  Alien Arena 2006 Gold Edition

�  Counter Strike

�  Brood Wars

�  Battlefield 1942 Server and Clients

�  Quake 2 and Quake 3 Game Servers found in Supervisor HMI LAN

�  Soldier of Fortune II

16

�  Software license cracking executables (CD-key generators)

�  Torrent client software on Supervisor HMI LAN

�  Paging Software Server (i.e. Air Messenger Server connected to both the SCADA and Internet for SMTP relay out)

�  America Online Clients

�  MP3 Music and Video Playing Software including iTunes

�  Streaming Music and Radio software with vulnerabilities

�  BitTorrent Clients (for peer-to-peer file sharing)

�  MSN and other IM chat clients

�  Anonymous FTP Servers running waiting for connections

but wait…there’s more

�  Apache Web Servers and Linux hosts un-patched for over 2 years

�  APC Battery Backup UPS systems with vulnerable Web Interface

�  Several web blog site engines running in control system DMZ

�  Office grade Linksys, Belkin, and D-Link WiFi devices on Supervisory HMI LAN

�  IM clients found installed and contained vulnerabilities on Supervisory HMI LAN

�  Windows 95 found installed on hosts in Supervisory HMI LAN (no longer supported by MS)

17

�  Windows NT found installed on hosts in Supervisory HMI LAN (no longer supported by MS)

�  Windows Vista found used as OS for operator consoles in Supervisory HMI LAN

�  IRC Chat Servers found installed on hosts in the Operational DMZ LAN

�  Nintendo Entertainment System (NES) Game Simulator

�  Netscape Browser vulnerabilities detected in Supervisor HMI LAN

�  Multi-function Printer/Fax/Scanner device vulnerabilities

summary / take away points �  331 = the average time in days between when a vulnerability was disclosed in the public

versus when it was discovered in an industrial control systems assessment

�  the intermediate Operations DMZ network that sites between the Enterprise network and the industrial control systems had the most vulnerabilities attributed to its zone

�  web server and back-end database vulnerability findings comprised the largest number of vulnerabilities found in these Operations DMZ network – we need more web app testing!

�  network devices are better managed in the Internet DMZ and Enterprise LAN networks where the IT or IS department has clear ownership of managing the network devices

�  number of client workstation vulnerabilities also increased deeper into the real-time operations networks, thus proving we still have a patch problem in our industry

�  vulnerabilities with Windows operating systems or Windows applications also accounted for the overwhelming majority of vulnerabilities for systems in the Supervisory HMI LAN

�  Vulnerabilities and Exploits will continue to be found at a rapid pace for SCADA HMI Applications built for Windows, Web-Enabled SCADA Browser Applications, and Embedded PLC devices

18

q & a

�  contact info jonathan pollet principal consultant jpollet@redtigersecurity.com office: +1.877.387.7733 mobile: +1.281.748.6401

19