Cisco Cyber Threat Defense Chad Mitchell – CCIE# 44090 Consulting Systems Engineer
Detecting and Protecting Against Insider Threat
Cisco Public 2 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Consider these guys…
All were smart. All had security. All were seriously compromised.
Cisco Public 3 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Increased Attack Surface
APTS Cyberware
Spyware and Rootkits Worms
Antivirus (Host-Based)
IDS/IPS (Network
Perimeter)
Reputation (Global) and Sandboxing
Intelligence and Analytics
(Cloud)
Enterprise Response
2010 2000 2005 Tomorrow
The Threat Landscape is evolving
Cisco Public 4 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
DEFEND Assess Environment & Threat
Visibility & Investigation Contain
Fix
Advanced Content Analysis Behavior Anomaly Detection
Policy & Access Control Blocking Quarantine Re-Routing Traffic
Re-Think Security Process and Technology
Advanced Targeted Attacks Inside the Network
Cisco Public 5 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
The Advanced Malware Attack Lifecycle PLAN EXPLOIT / ATTACK INFECT / SPREAD STEAL / DISRUPT
Attacker determines possible entry points,
formulates a plan of attack
Attacker exploits vulnerabilities and delivers
its weapon
Malware moves laterally through the internal network in search of
additional resources and data
Attacker takes action on its objectives and
exfiltrates data or disrupts systems
HACKER
Cisco Public 6 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
NetFlow Generator
Source IP Address Destination IP Address Source Port Destination Port
Layer 3 Protocol TOS byte (DSCP) Input Interface
NetFlow Key Fields
Flow Information Packets Bytes/packet
Address, ports... 11000 1528
...
NetFlow Cache
StealthWatch FlowCollector
1
2
3
Source Des5na5on
Introduction to NetFlow
Cisco Public 7 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Internet
Internal Network
NetFlow Data
NetFlow Collector
Key NetFlow Fields
• Packet count • Byte count
• Source IP address • Destination IP address
• Start sysUpTime • End sysUpTime
• Packet count • Byte count
• Input ifIndex • Output ifIndex
• Type of Service • TCP flags • Protocol
• Next hop address • Source AS number • Dest. AS number • Source prefix mask • Dest. prefix mask
Usage
Time
Port Utilization
QoS
From/To
Application
Routing and Peering
The Network as a Scalable Source of Truth
Cisco Public 8 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cisco Infrastructure Unsampled NetFlow is key to internal network visibility
Sampled = Partial • Subset of traffic, usually less than 5%, • Gives a snapshot view into network activity • Similar to reading every 20th page of a book
Unsampled = All • All traffic is collected • Provides a comprehensive view into all activity
on the network • Equivalent to reading every word on every
page of a book
Sampling is sufficient for network performance monitoring, not security
Cisco Public 9 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Versions of NetFlow Version Major Advantage Limits/Weaknesses
V5 Defines 18 exported fields Simple and compact format Most commonly used format
IPv4 only Fixed fields, fixed length fields only Single flow cache
V9 Template-based IPv6 flows transported in IPv4 packets MPLS and BGP nexthop supported Defines 104 fields, including L2 fields Reports flow direction
IPv6 flows transported in IPv4 packets Fixed length fields only Uses more memory Slower performance Single flow cache
Flexible NetFlow (FNF) Template-based flow format (built on V9 protocol) Supports flow monitors (discrete caches) Supports selectable key fields and IPv6 Supports NBAR data fields
Less common Requires more sophisticated platform to produce Requires more sophisticated system to consume
IP Flow Information Export (IPFIX) AKA NetFlow V10
Standardized – RFC 5101, 5102, 6313 Supports variable length fields, NBAR2 Can export flows via IPv4 and IPv6 packets
Even less common Only supported on a few Cisco platforms
NSEL (ASA only) Built on NetFlow v9 protocol State-based flow logging (context) Pre and Post NAT reporting
Missing many standard fields Limited support by collectors
Cisco Public 10 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
NetFlow Version 5 (Common Record) Fixed format
Cisco Public 11 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
How do I want to cache information
Which interface do I want to monitor?
What data do I want to meter? Router(config)# flow record my-record Router(config-flow-record)# match ipv4 destination address Router(config-flow-record)# match ipv4 source address Router(config-flow-record)# collect counter bytes
Where do I want my data sent? Router(config)# flow exporter my-exporter
Router(config-flow-exporter)# destination 1.1.1.1
Router(config)# flow monitor my-monitor
Router(config-flow-monitor)# exporter my-exporter
Router(config-flow-monitor)# record my-record
Router(config)# interface s3/0
Router(config-if)# ip flow monitor my-monitor input
1. Configure the Exporter
2. Configure the Flow Record
3. Configure the Flow Monitor
4. Apply to an Interface
Configuring Flexible NetFlow
Best Practice: include all v5 fields
Cisco Public 12 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
NetFlow Challenges: Flow Stitching
12
10.2.2.2 port 1024
10.1.1.1 port 80
eth0
/1
eth0
/2
Start Time Interface Src IP Src Port Dest IP Dest Port Proto Pkts Sent Bytes Sent
10:20:12.221 eth0/1 10.2.2.2 1024 10.1.1.1 80 TCP 5 1025
10:20:12.871 eth0/2 10.1.1.1 80 10.2.2.2 1024 TCP 17 28712
Start Time Client IP Client Port
Server IP Server Port Proto Client Bytes
Client Pkts Server Bytes
Server Pkts Interfaces
10:20:12.221 10.2.2.2 1024 10.1.1.1 80 TCP 1025 5 28712 17 eth0/1 eth0/2
Uni-directional flow records
Bi-directional: • Conversation flow record • Allows easy visualization and analysis
Cisco Public 13 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
NetFlow Challenges: De-duplication
13
Router A
Router B
Router C
10.2.2.2 port 1024
10.1.1.1 port 80
Router A: 10.2.2.2:1024 -> 10.1.1.1:80 Router B: 10.2.2.2:1024 -> 10.1.1.1:80 Router C: 10.1.1.1:80 -> 10.2.2.2:1024
Duplicates
• Without de-duplication: • Traffic volume can be misreported • False positives would occur
• Allows for the efficient storage of flow data • Necessary for accurate host-level reporting • Does not discard data • Includes NAT
Cisco Public 14 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
CTD Solution Components
Cisco ISE
Flow Sensor
Flow Collector
StealthWatch Management
Console
Catalyst 3500-X
10G-Servicemodule Catalyst 3850* Catalyst 4500
Supervisor 7E/L Catalyst 6500 Supervisor 2T
ASA-5500-X(NSEL) Nexus 1k*
Nexus 7k M Series* Nexus 7k F2 Series*
Nexus 6k* Nexus 2k on 7k*
NGA-3240 ASR 1000(NBAR)
ISR-G2(incl. NBAR) WLC (incl. NBAR)**
Catalyst 2960X LanBase* Net
flow
Ena
bled
Dev
ice
LanC
ope
Ste
alth
Wat
ch
Cis
co Id
entit
y S
ervi
ce E
ngin
e
Connection Information
Monitoring
Identity Information
Device User
Collection/Analysis
Presentation
Cisco Public 15 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cisco Network
StealthWatch FlowCollector
StealthWatch Management
Console
NetFlow
Users/Devices
Cisco ISE
NBAR NSEL
StealthWatch Solution Design
StealthWatch FlowSensor
StealthWatch FlowSensor
VE
NetFlow
StealthWatch FlowReplicator
Other tools/collectors
Cisco Public 16 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Devices Internal Network
Use NetFlow Data to Extend Visibility to the Access Layer
Unify Into a Single Pane of Glass for Detection, Investigation and
Reporting
Enrich Flow Data With Identity, Events and Application to Create Context
WHO
WHAT WHERE
WHEN
HOW Hardware-enabled NetFlow Switch
Cisco ISE
Cisco ISR G2 + NBAR
Cisco ASA + NSEL
Context
Visibility, Context, and Control
Cisco Public 17 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Conversational Flow Record - Visibility Who Who What
When
How
Where More context
• Highly scalable (enterprise class) collection • High compression => long term storage
• Months of data retention
Cisco Public 18 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Situational Awareness
Shopping cart? “it's knowing what is going on around you …”
www.sans.edu/research/management-laboratory/article
Cisco Public 19 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Adding Context and Situation Awareness
NAT Events
Known Command & Control Servers
User Identity
Application Application & URL
Cisco Public 20 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
LIMIT ACCESS Switch port control
INCREASE SCRUTINY Inform IPS, FW to selectively use stringent policy
ROUTE DIFFERENT PATH Route traffic through advanced security stack
ROUTE COPY OF TRAFFIC Selectively archive all packets of suspicious users / devices
Close the loop from detection to mitigation
• Save money by leveraging network, itself, to enforce policy • Reduce risk / latency by focusing resource intensive analysis • Protection policy will follow the user and/or device • Takes advantage of ISE, TrustSec, and pxGrid
DEFEND
Network-Integrated Mitigation - Control
Cisco Public 21 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Behavioral Analysis & Anomaly Detection
Behavioral Analysis • Leverages knowledge of known bad
behaviour
Anomaly Detection • Identify a change from
“normal”
Cisco Public 22 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
DEFEND
Today – Advanced Visibility & Investigation
• Partner with Lancope to deliver NetFlow visibility and security intelligence • Enhance with Identity, device, application awareness
Cisco ISE
Cisco ISR G2 + NBAR
Firewall
IPS
Web Sec
N-AV
Email Sec
Threat Detection
Routers
Switches
Firewall
NetFlow
Visibility
Scalable Network Defense
Cisco Public 23 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
DEFEND
Firewall
IPS
Web Sec
N-AV
Email Sec
Threat Detection
Routers
Switches
Firewall
NetFlow
Visibility
Advanced
Analysis
Next – Advanced Discovery
• Next-gen analysis utilizing Artificial Intelligence, Game Theory and Predictive Algorithms
• Leverage NetFlow, Local Web Data, DNS, Identity and SIO Context
• Reduce human analysis • Mitigation – TrustSec with ISE, Cisco ONE
SIO
Local Web Data, DNS,
Identity Global Reputation
URL, IP, File, Domain
Scalable Network Defense
Cisco Public 24 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Collect & Analyze Flows
1 2 • # Concurrent flows • Packets per second • Bits per second • New flows created • Number of SYNs sent • Time of day
• Number of SYNs received
• Rate of connection resets
• Duration of the flow • Over 80+ other
attributes
Establish Baseline of Behaviors
Alarm on Anomalies & Changes in Behavior
threshold
threshold
threshold threshold
Critical Servers Exchange Server Web Servers Marketing
Anomaly detected in host behavior
3
Flow-based Anomaly Detection
Cisco Public 25 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Detailed Flow Information – StealthWatch 6.6
Cisco Public 26 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Attribute Flows and Behaviors to a User and Device
26
Policy Start Active Time
Alarm Source Source Host Groups
Source User Name
Device Type
Target
Desktops & Trusted Wireless
Jan 3, 2013 Suspect Data Loss
10.10.101.89 Desktops, San Jose
jchambers Windows7-Workstation
Multiple Hosts
Cisco Public 27 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
A Note on StealthWatch and NSEL
• Flow Action field can provide additional context
• State-based NSEL reporting is taken into consideration in StealthWatch’s behavioral analysis • Concern Index points accumulated for Flow Denied events
• NAT stitching
Cisco Public 28 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Behavior-Based Attack Detection
High Concern Index indicates a significant number of suspicious events that deviate from
established baselines
Host Groups Host CI CI% Alarms Alerts
Desktops 10.10.101.118 865,645,669 8,656% High Concern Index
Ping, Ping_Scan, TCP_Scan
Cisco Public 29 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Identifying Reconnaissance Activity
NetFlow Capable
Internal Network
Devices
Management StealthWatch FlowCollector
StealthWatch Management
Console
1. Infected host performs random pings and sweeps in the internal network
2. Infrastructure generates records of the activity using NetFlow
3. Collection and analysis of NetFlow data
4. Contextual information added to NetFlow analysis
5. Concern index increased Suspicious network scanning activity alarms generated
Cisco ISE
29
Cisco Public 30 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Detecting Internally Spreading Malware
NetFlow Capable
Devices
Management StealthWatch FlowCollector
StealthWatch Management
Console 3. Collection and analysis of NetFlow data
4. Contextual information added to NetFlow analysis
5. Concern index increased Worm propagation Alarm generated
Cisco ISE
Initial Infection
Secondary Infection
1. Infection propagates throughout the internal network as attacker executes their objective
2. Infrastructure generates records of the activity using NetFlow
Internal Network
30
Cisco Public 31 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Detecting Internally Spreading Malware
Devices
Management StealthWatch FlowCollector
StealthWatch Management
Console 3. Collection and analysis of NetFlow data
4. Contextual information added to NetFlow analysis
5. Concern index increased Worm propagation Alarm generated
Cisco ISE
Tertiary Infection
Initial Infection
Secondary Infection
2. Infrastructure generates records of the activity using NetFlow
Internal Network
NetFlow Capable
1. Infection propagates throughout the internal network as attacker executes their objective
31
Cisco Public 32 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Infection Tracking
Tertiary Infection
Secondary Infection
Initial Infection
32
Cisco Public 33 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Attack Detection without Signatures
High Concern Index indicates a significant number of suspicious events that deviate from
established baselines
Host Groups Host CI CI% Alarms Alerts
Desktops 10.10.101.118 338,137,280 8656% High Concern index Ping, Ping_Scan, TCP_Scan
Monitor and baseline activity for a host and within host groups.
Cisco Public 34 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Detecting Botnet Command and Control SLIC Feed
Alarm indicating communication with known
BotNet Controllers
IP Address Source user name
Policy that triggered alarm
Policy Start Active Time
Alarms Source Source Host Groups
Source User Name
Target Target Host Group
Inside Hosts
Jan 27, 2014
Host Lock Violation
10.35.88.171 Remote VPN Bob ZeusCCServer.com Zeus BotNet Controllers
34
Cisco Public 35 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Detecting Suspect Data Loss
Policy Start Active Time
Alarm Source Source Host
Group
Source Username
Target Details
Inside Hosts 8-Feb-2014 Suspect Data Loss
10.34.74.123 Wired Data
Bob Multiple Hosts
Observed 4.08G bytes. Policy Maximum allows up to 81.92M
bytes.
35
Cisco Public 36 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Detecting attacks like Heartbleed
• Attack payload size and response is consistent
• Application & • Ports and Protocol are well
known • Connections from attacker
are typically very long (hours-days)
Cisco Public 37 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
What Heartbleed looks like in StealthWatch
Secure HTTP Server App = SSL Client Ratio ~ 5%
Duration is typically long – multiple hours
Cisco Public 38 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
What can I do? What has been done?
Cisco Public 39 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ Preparation § Scanned 1.2M vulnerable servers – 300 in need of repair § Developed signatures for Cisco IPS and Cisco NGIPS (SourceFire) § Deployed Signatures to IPS/IDS
§ Monitoring and Response § Discovered 25 Attacks: 21 Benign, 4 Malicious § Researched attack via StealthWatch to discern normal connections vs
anomalous and malicious.
Cisco CSIRT response to Heartbleed
Cisco Public 40 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Combine Network and Security Management
Cisco Public 41 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Combine Network and Security Management
Top Conversations:
Cisco Public 42 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
StealthWatch: Visibility into the Network
Cisco Public 43 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
“1-touch” network mitigation action – from 3rd party partner console
pxGrid ANC API
ISE as unified policy point
User/Device Quarantine
Dynamic ACLs, Increase Inspection
Adaptive Network Control for ISE. It provides the ability to: • Quarantine user devices from 3rd party products, such as StealthWatch & Threat Identification systems
• Enlist other Cisco infrastructure in the network response – such as dynamic ACLs or SGTs on switches and ASA or increase IPS inspection levels
pxGrid: Adaptive Network Control Transforms the Cisco Infrastructure into a Unified Event Response Network
Cisco Public 44 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
StealthWatch 6.6 – Quarantine w/ Cisco ISE ANC
• Utilizes pxGrid to initiate action
• Actions can be initiated Manually from the StealthWatch UI or Automatically with StealthWatch rules
Cisco Public 45 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Manipulating User Traffic using ANC & SGT
Classified Data Server
ISE
RADIUS (Access Request)
SGT = Classified_Operator
Security Group Filtering
Classified Operator
Allow access to Data Center
IP 10.45.1.70 = Classified_Operator
Cisco Public 46 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Manipulating User Traffic using ANC & SGT
Classified Data Server
ISE
RADIUS (Access Request)
SGT = Quarantine
Security Group Filtering
Quarantine
QoS = Bronze
Filter Traffic Block access to DC Allow remediation Full Packet Capture
Malware Traffic
Re-Route
Punt to IPS for further inspection
Cisco Public 47 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Walking the Talk - CSIRT NetFlow Collection at Cisco
250,000 Hosts 180 Flow Exporters
180,000 FPS
16 Billion Flows Daily
90+ Days Flow Retention
Cisco Public 48 © 2013-2014 Cisco and/or its affiliates. All rights reserved. 48
RTP San Jose Amsterdam
Bangalore
Sydney
Tokyo
15.6 billion flows / day 90 day retention
Walking the Talk - CSIRT NetFlow Collection at Cisco
Cisco Public 49 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Summary
Provides Rich Context Unites NetFlow data with identity and application ID to provide security context
Leverages Cisco Network for Security Telemetry
NetFlow-enabled Cisco switches and routers become security telemetry sources Cisco is the undisputed market leader in Hardware-enabled NetFlow devices
Cisco ISE
Cisco Network
Provides Threat Visibility and Context
Single pane of glass that unifies threat detection, visibility, forensics analysis, and reporting
Cisco ISR G2 + NBAR
+ +
+ NetFlow
FlowSensor
FlowCollector StealthWatch Management
Console
Cisco ASA Who What Where When How