Date post: | 11-Nov-2014 |
Category: |
Internet |
Upload: | malaysia-network-operators-group |
View: | 155 times |
Download: | 0 times |
Traffic Analysis for Peering, and Security Utilizing xFlow Technologies
August 2014 Julie Liu
Agenda
What is xFlow Technology?
What values can xFlow propose to xSPs? Traffic Visibility for Peering Analysis Infrastructure Security
What is xFlow Technology? (A quick foreword, in case you are not familiar with it…)
Definition of a Flow A unidirectional set of packets that arrive at a router on the same
interface, have the same source/destination IP addresses, Layer 4 protocol, TCP/UDP source/destination ports, and the same ToS byte in the IP headers
A technology to gather information on forwarded packets In router/switch caches And exported to collectors
Client
Server Request
Response
TWO flows for ONE TCP connection
Client Server Content
ONE flow for ONE UDP Stream
Flow Cache Table • Active Timeout • Inactive timeout
How xFlow Can Benefit xSPs?
Traffic Matrix
Visibility
Security Protection
Capacity Planning
xFlow Collection & Analysis
Traffic Engineer-
ing
Peering Analysis
Anomaly Detection
In-cloud Mitigation
TRAFFIC MATRIX VISIBILITY
Traffic Matrix
Visibility
Security Protection
Capacity Planning
xFlow Collection & Analysis
Traffic Engineer-
ing
Peering Analysis
Anomaly Detection
In-cloud Mitigation
Why Traffic Matrix Visibility?
Traffic Matrix Visibility The amount of data transmitted between every pair of network
"instances" (router-level, Pop-level, network-level) Provide end-to-end, network-wide traffic visibility, in contrast to
the individual link load stats
Traffic Matrix Visibility for what purposes?
Capacity planning (build capacity where needed) Traffic engineering (steer traffic where capacity is available) Peering Analysis (support peering decisions, TE at the border) Better understand traffic patterns (what is normal or abnormal)
Challenges of xFlow Only Flow duplicates
Collect from where? Usually multiple Flow measurement sources in a data path However, if collecting from multiple
xFlow sources in a data path, will duplicate the Flow data results for counting traffic toward a network instance
Network topology data is needed! Count once on the
network boundary of the instance to be monitored
Challenges of xFlow Only From point into path measurements
xFlow only Can tell you where traffic is going now
Some simple information about origin-AS or peer-AS
Not only peer or origin, the transit ASes also matter! Embed a BGP (passive) peer on the Flow Collector to correlate
Flow data with all the BGP attributes (path, communities, etc.)
Use of full AS Path information to determine where traffic is going and coming from and how existing transit/peer is used
BGP carries the topology (i.e. path) information helps extend local measure view to completely across the Internet
Peering Analysis What is Peering? (Just a quick reminder…)
What is Peering? The Internet is a collection of many individual networks (ASes),
who interconnect with each other under the common framework of ensuring global reachability between any two points
There are 3 primary positions for this interconnection:
Transit Provider – Typically someone you pay money to, who has the responsibility of routing your packets to/from the entire Internet
Transit Customer – Typically someone who pays you money, with the expectation that you will route their packets to/from the entire Internet
Peer – Two networks who get together and agree to exchange traffic between each others’ networks, typically for free
Peering Analysis Peering and its benefits…
One major benefit of Peering Reduced operating costs
Peering traffic is “free”. If you no longer pay a transit provider to deliver some portion of your traffic, it reduces your transit bills
Provider A
Provider B
Provider C
Customer
Customer Customer Customer Customer
Customer
Multi-homed Customer
Peering Transit
Peering Analysis Why traffic visibility for Peering?
To decide if you should peer with a new network To convince other networks to peer with you To manage traffic engineering to other networks To defend your network against depeering actions To make intelligent transit purchasing decisions
Peering Analysis Peering traffic requirements
Traffic Volume A peer may be required to exchange a certain minimum amount
of traffic to be considered
Traffic Ratios Inbound vs. outbound traffic ratio Traffic is “hot potato” routed (i.e. get it off your network ASAP)
Push traffic coming from Network A gets hauled primarily by Network B, and vice versa
If the ratio is 1:1, both peers share backhaul costs equally
Others: PoP requirements, interconnect locations, routing stability, operations requirements, business concerns…
Peering Analysis Peering evaluation questions
" A Business Case for Peering," William B. Norton
Does the AS send me about as much traffic as I send to it?
How much of the traffic originates from the potential peer?
Does the volume of traffic justify a direct peering effort?
How much traffic is transited through the potential peer?
AS101
AS100
AS21
AS C
AS23
AS4
AS1
AS2
AS3
Home
Internet
Peer AS Origin AS
Transit AS
Peering Analysis Route-flow fusion analysis answers this…
" A Business Case for Peering," William B. Norton
Source-sink/transit traffic distribution
TopN ASNs sourcing-sinking/transiting traffic with me 1
In/Out traffic ratio 2
3
Peering Analysis Peering cost analysis
In theory, peering is “free” right? The fact is that the overhead associated with peering can be
higher than transit costs (if the peered traffic is not huge enough)
How much does it save/ cost?
Which transit provider(s)?
How much transit traffic can be offloaded?
US$ Internet Transit Price
Transit A $1.6 per Mbps
Transit B $1.8 per Mbps
Transit C $1.2 per Mbps
AS
101
AS
100
AS
21 Peer
Candidate
AS
23
AS
4
Transit A
Transit B
Transit C
Home
Internet
SECURITY PROTECTION
Traffic Matrix
Visibility
Security Protection
Capacity Planning
xFlow Collection & Analysis
Traffic Engineer-
ing
Peering Analysis
Anomaly Detection
In-cloud Mitigation
Infrastructure Security Threats DDoS attacks
DDoS attack traffic consumes SP network
capacity
DDoS attack traffic saturates in-line security
devices
DDoS attacks launched from compromised
systems (bots)
DDoS attack traffic targets applications and
services
Internet Service Provider
Network Enterprise or
IDC
Bots Victim
Why traditional in-line security solution fails preventing infrastructure security threats? Volumetric attacks must be removed from the cloud
Tradition security products are easy targets of it (stateful in-line solution) Deployment costs
Single point of failure and latency
Anomaly traffic Normal traffic
Infrastructure Security A Flow-based solution
Flow-based solution building blocks
Flow-based Learning
Flow-based Detection
Cloud-based Mitigation
•Network-wide: Collects xFlow data from various router locations and correlates the data into a comprehensive network model •Dynamic Behaviour Analysis: During peace time, the system creates a network-wide view of the traffic patterns and learns thresholds for representing 'what is 'normal'
•Detection Engines: compare the collected real-time Flow data and thresholds •Once significant threshold violations identified, the system sends alarms and enable cloud-based mitigation actions
•Cloud-based mitigation action options: - Remote Triggered Black Hole (RTBH) - BGP FlowSpec -OOP Traffic Cleaning
Flow-based Learning & Detection The idea
Flow-based Network Behavior Anomaly Detection (NBAD) DOES:
Analyze Flows data (IP header info, byte/pkt count) from routers Detect anomalies by observing network traffic behaviors – knowing
what is normal, and hence identify abnormal when it happens DOESN'T:
Analyze L7, packet contents from raw packets Detect anomalies by matching content signatures – knowing what is
bad, and then catch the bad from the good
First-line protection for the network infrastructure Trading DPI precision off for carrier-grade scalability and performance
Flow-based Learning & Detection Network behavior analysis examples
What's Normal? What can be Abnormal? Example
A server accepts requests from clients
Over 5,000 SYN requests per second and lasts over 3 minutes
TCP SYN Flooding
A client connects to few destination hosts / ports
Over 100 connection requests per second to destination hosts / ports
Port Scan / IP Scan
Various packet sizes Fixed packet size (e.g. UDP/1434, packet size = 404)
SQL Slammer
The source address ≠ the destination address
The source address is the same as the destination address
LAND Attack
The traffic rate for this network scope is usually around 150M bps
Over 180M bps traffic rate appears in this network scope
Zero-day attack (generic traffic floods)
Flow-based Learning & Detection The mechanisms
Flow-based NBAD
Mechanism Type Detection Engine Examples
Fingerprint-based
Protocol anomaly TCP Flag Null, IP Fragment, IP Protocol Null, Land Attack, Ping of death, TCP XMAS attack…
Flood attack ICMP Flooding, UDP Flooding, TCP SYN Flooding, TCP RST Flooding, TCP ACK Flooding…
Specific behaviour attack
IP Scan, Port Scan, DNS Flooding, e-Mail Spam, Trojan Heloag, MS Blaster, Sasser, Code Red, SQL Slammer…
Baseline Heuristic Baseline deviation Zero-day attacks (generic traffic floods)
1-Jul 11-Jul 21-Jul 31-Jul
Traf
fic L
evel
Learning peacetime Flow data samples "Baseline": what is the "normal" traffic rates?
Infrastructure Security Cloud-based mitigation with RTBH
All traffic to the victim is discarded
Remotely triggered black hole filtering at SP edge
BGP prefix with next-hop set to a pre-defined
black hole route
Internet Service Provider Network
Enterprise or IDC
Bots Victim
RTBH RTBH
Anomaly traffic Normal traffic BGP announcement
Infrastructure Security Cloud-based mitigation w/ BGP FlowSpec
Suspicious traffic recognized is filtered at the SP network edge
Only filtered traffic is delivered to the enterprise/IDC network
BGP FlowSpec distributes traffic filter
lists to routers
Internet Service Provider Network
Enterprise or IDC
Bots Victim
RFC 5575;Selectively drop traffic flows based on L3/L4 information
FlowSpec
Anomaly traffic Normal traffic BGP FlowSpec
FlowSpec
Infrastructure Security Cloud-based mitigation w/ OOP cleaning
Suspicious traffic is diverted at the SP network edge
Divert victim prefix traffic via BGP
Internet Service Provider Network
Enterprise or IDC
Bots Victim
The "Cleaning Centre" is typically a shared resource in the network infrastructure to reduce the deployment costs
Malicious traffic Benign traffic BGP announcement
Cleaned traffic tunnelled back
DPI-capable mitigation appliance (application-layer attack,
asymmetric detection)
Cleaning Centre
No impacts to other traffic to other networks
Infrastructure Security xFlow-based OOP solution value proposition
Flow Technology Network Resource Impact Issues
NetFlow data volume? 1K FPS ≒ 338K bps NetFlow traffic
However, to estimate Flows/ second based on the given network traffic
bps is a much more complex task! Typically 1~4% link rate
Leverage data reduction techniques: Partial coverage (i.e. a few POPs, selective boundaries) Tune the active & inactive timeouts Flow Sampling
In addition to the data volume, 'full NetFlow’ may inflict a burden on memory and router CPU intensive. Therefore sampled xFlow is preferred…
Flow/sec Pkt/sec Byte/sec bps
1,000 33 49,500 338.37K
Flow Technology Flow Sampling
To alleviate the performance penalty incurred by turning on xFlow on routers Allow users to sample one out of every “N” IP packets being forwarded
(a user can configure the “N” interval) Substantially decreases the CPU utilization needed to account for
Flow packets CPU utilization varies, depending on the sampling rate and the routers
Example: Cisco 12000 Series Router to handle 65K flows
In “full-flow” mode required 24% more CPU; the same router using 1:100 sampling required only 3% additional CPU
Cisco 7500 Router
References
Yann Berthier, "NetFlow to guard the infrastructure," NANOG 39, 2007
Thomas Telkamp, “Best Practices for Determining the Traffic Matrix in IP Networks V 3.0,” NANOG 39, 2007
Richard A Steenbergen, "A Guide to Peering on the Internet," NANOG 51, 2011
William B. Norton, " A Business Case for Peering in 2010," http://drpeering.net/white-papers/A-Business-Case-For-Peering.php
RFC 5575, Dissemination of Flow Specification Rules Leonardo Serodio, "Traffic Diversion Techniques for DDoS
Mitigation using BGP Flowspec," NANOG 58, 2013 Cisco Systems Inc., "NetFlow Performance Analysis," 2007