Scalable DDoS mitigation
Peter Filo
Senior Systems Engineer
ALEF Distribution SK
Agenda
• Traditional DDoS Mitigation
– Remote Triggered Blackhole Filtering
• Scalable DDoS Mitigation
– BGP FlowSpec
• Cloud DDoS Protection
– F5 Silverline
DDoS Overview
• Distributed denial-of‐service (DDoS) attacks target network infrastructures or computer services by sending overwhelming number of service requests to the server from many sources.
• Server resources are used up in serving the fake requests resulting in denial or degradation of legitimate service requests to be served
• Addressing DDoS attacks– Detection
• Detect incoming fake requests
– Mitigation• Diversion – Send traffic to a specialized device that removes the fake packets
from the traffic stream while retaining the legitimate packets• Return – Send back the clean traffic to the server
DDoS Detection
• NetFlow / IPFIX / sFlow– How many flows/sec can your routers meter, and how fast is your
collector/analyzer?– What are you going to look at?
• SNMP– Are you looking at all the right values?– Are you polling your devices every second, every minute, every hour?
• SYSLOG– Need to set up proper rules to filter out the events you want to see
• RADIUS/TACACS+ logging– Watch those authentication failures and changes to the nodes
• Packet capturing– Do you use TAPs/Splitters?
Goals of DDoS Mitigation
• Stop the attack
• Drop only the DDoS traffic
• Application aware filtering, redirection, mirroring
• Dynamic and adaptive technology
• Simple to configure
• Easy to disseminate
Remote Triggered Black-Hole Filtering (RTBH)
• Once the attack has been detected, traffic related to the DDoS should be discarded on the edge of the service provider network
• BGP router (trigger) signals over BGP to the edge routers that traffic causing DDoS should be discarded (forwarded to null interface)
• Destination-based RTBH– Traffic going to the IP addresses of the customer is discarded on the
edge
• Source-based RTBH– Traffic coming from the IP addresses of the attacker is discarded on
the edge– Uses strict uRPF with BGP signalling
Destination-based RTBH
Customer
SP
AS 65535
Attacker
PE2 PE1
Signalling router
! PE1 router!ip route 192.0.2.1 255.255.255.255 Null0
!
interface Null0
no ip unreachables
! Signalling router!router bgp 65535 .redistribute static route-map static-to-bgp.!route-map static-to-bgp permit 10
match tag 66set ip next-hop 192.0.2.1set local-preference 200set community no-export set origin igp
!route-map static-to-bgp permit 20
! Signalling router / adding a static route when under attack!ip route 172.19.61.1 255.255.255.255 Null0 Tag 66
172.19.61.0/24
Gi0/0
192.168.10.0/24
Source-based RTBH
Customer
SP
AS 65535
Attacker
PE2 PE1
Signalling router
! PE1 router!ip route 192.0.2.1 255.255.255.255 Null0
!
interface Null0
no ip unreachables
!
interface GigabitEthernet0/0/0
ip verify unicast source reachable-via rx
! Signalling router!router bgp 65535 .redistribute static route-map static-to-bgp.!route-map static-to-bgp permit 10
match tag 66set ip next-hop 192.0.2.1set local-preference 200set community no-export set origin igp
!route-map static-to-bgp permit 20
! Signalling router / adding a static route when under attack!ip route 192.168.10.0 255.255.255.0 Null0 Tag 66
172.19.61.0/24
Gi0/0/0
192.168.10.0/24
RTBH as a Service
• Ask your uplink providers for blackhole BGP community• Provide blackhole BGP community to your customers
SP
AS 65535Internet
CE PE1
! PE2 routerrouter bgp 65535.neighbor cust route-map from-customer in!ip community-list standard BH permit 65535:666!route-map from-customer permit 10match community BHset ip next-hop 192.0.2.1set local-preference 200set community no-export
!route-map rm-community-in permit 20!
! CE routerrouter bgp 65500.network 172.19.61.0 mask 255.255.255.0redistribute static route-map static-to-bgp!route-map static-to-bgp permit 5match tag 666set community additive 65535:666!ip route 172.19.61.1 255.255.255.255 FastEthernet0/0 tag 666!!
web server 172.19.61.1/24 DDoS Traffic
PE2
BGP: 172.19.61.0/24
BGP: 172.19.61.1/32
Com: 65535:666 172.19.61.1/32 Discard
F0/0
Remote Triggered Black-Hole Filtering (RTBH)
• No more DDoS traffic on my web server
• But no more traffic at all on my webserver
• IP based solution only
• Is this the solution you were looking for?
Policy Based Routing ?
• Identification of DDoS traffic based around conditions regarding MATCH statements– Source/Destination address– Protocol– Packet Size– Port Number– Etc...
• Actions upon DDoS traffic– Discard– Rate Limiting– Redirection– Etc...No more DDoS traffic on my web server
• Does not this sound as a great solution?
Policy Based Routing ?
• Good solution for – Done with hardware acceleration for carrier grade routers
– Can provide very good precision of match statements and actions to impose
• But...– Customer need to call its Service Provider
– Service Provider has to accept and run this filter on each of their peering routers
– Customers need to call the Service Provider and remove the rule after
• Not scalable...
Solution: BGP FlowSpec
• Makes static PBR a dynamic solution
• Allows to propagate PBR rules
• Existing control plane communication channel is used
• Uses your existing MP-BGP infrastructure
RFC5575 Dissemination of Flow Specification Rules
• Published in August 2009
• New Flow Specification NLRI type encoded using MP_REACH_NLRI/MP_UNREACH_NLRI
• Inter-domain support • Point-to-multipoint with Route-Reflectors • Networking engineers and architects understand perfectly BGP
• Capability to send via a BGP address Family– Match criteria (NLRI)– Action criteria (Extended communities)
• Three elements– Controller– Client– Route-reflector (optional)
BGP FlowSpec Components
• Controller– Injects rules remotely in the clients– Needs to implement at the minimum the Control Path– Examples of BGP FS Controllers:
• Router (ASR9K, CRS, NCS6000, XR12000)• Server (ExaBGP, Arbor PeakFlow SP Collector Platform)• Virtual router (XRv)
• Client– Receives rules from Controller(s) and programs the match/action in hw– Needs to implement both Control Plane and Data Plane– Examples of BGP FS Clients:
• Router (ASR9K, ASR1K)
• Route-Reflector (optional)– Receives rules from Controller(s) and distributes them to Clients– Examples of BGP FS Route-Reflectors:
• ASR9K; CRS; NCS6000 or XRv
RFC5575 Dissemination of Flow Specification Rules
• New NLRI defined (AFI=1, SAFI=133) to describe the traffic of interest
1. Destination IP Address (1 component)2. Source IP Address (1 component)3. IP Protocol (+1 component)4. Port (+1 component)5. Destination port (+1 component)6. Source port (+1 component)7. ICMP Type8. ICMP code9. TCP Flags10. Packet length11. DSCP12.Fragment
Notice from the RFC: “Flow specification components must follow strict type ordering. A given component type may or may not be present in the specification, but if present, it MUST precede any component of higher numeric type value.”
+---------------------------------------------------------+
| Address Family Identifier (2 octets) |
+---------------------------------------------------------+
| Subsequent Address Family Identifier (1 octet) |
+---------------------------------------------------------+
| Length of Next Hop Network Address (1 octet) |
+---------------------------------------------------------+
| Network Address of Next Hop (variable) |
+---------------------------------------------------------+
| Reserved (1 octet) |
+---------------------------------------------------------+
| Network Layer Reachability Information (variable) |
+---------------------------------------------------------+
The MP_REACH_NLRI – RFC 4760
RFC5575 Dissemination of Flow Specification Rules
• Traffic Action is defined in extended communities (RFC4360)
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type high | Type low(*) | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Value |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Type Description Encoding
0x8006 Traffic-rate 2bytes ASN; 4 bytes as float
0x8007 Traffic-action Bitmask
0x8008 Redirect 6 bytes RT (Route Target)
0x8009 Traffic-marking DSCP value
Cisco IOS XR Routers BGP FS Implementation
Platform Hardware Control Plane Support Data Plane Support
ASR9K – Typhoon LC 5.2.0 5.2.0
ASR9K – Thor LC 5.2.0 5.2.2
ASR9001 5.2.0 5.2.2
ASR9K – Tomahawk Target 5.3.x Target 5.3.x
CRS – Taiko LC 5.2.0 5.2.0
CRS – Topaz LC 5.2.0 Target 5.3.1
XRv 5.2.0 N/A
C12K 5.2.0 Not planned
NCS6000 Target 5.2.3/5.2.4 Target 5.2.3/5.2.4
• In Cisco IOS 15.5(S) release, BGP flow specification is supported only on a route reflector.• IOS XE software supports BGP flow specification client function and does not support BGP
flow specification controller function. • Mixing of address family matches and actions is not supported in flow spec rules. For
example, IPv4 matches cannot be combined with IPv6 actions and vice versa.
Cisco IOS XR Routers BGP FS Implementation
NLRI type Match fields Value input method
XR PI ASR9K CRS NCS6000
Type 1 IPv4 Destination address
Prefix length
Type 2 IPv4 Source address Prefix length
Type 3 IPv4 protocol Multi value range
Type 4 IPv4 source or destination port
Multi value range
Type 5 IPv4 destination port
Multi value range
Type 6 IPv4 source port Multi value range
Type 7 IPv4 ICMP type Multi value range
Type 8 IPv4 ICMP code Multi value range
Type 9 IPv4 TCP flags Bit mask
Only lower Byte reserved and NS bit
not supported
Only lower Byte reserved and NS bit
not supported
Only lower Byte reserved and NS bit
not supported
Type 10 IPv4 packet length Multi value range
Type 11 IPv4 DSCP Multi value range
Type 12 IPv4 fragmentation bits
Bit mask
Only indication of fragment
Cisco IOS XR Routers BGP FS Implementation
NLRI type Match fields Value input method XR PI ASR9K CRS NCS6000
Type 1 IPv6 Destination address
Prefix length
Type 2 IPv6 Source address Prefix length
Type 3 IPv6 Next header Multi value range
Type 4 IPv6 source or destination port
Multi value range
Type 5 IPv6 destination port
Multi value range
Type 6 IPv6 source port Multi value range
Type 7 IPv6 ICMP type Multi value range
Type 8 IPv6 ICMP code Multi value range
Type 9 IPv6 TCP flags Bit mask
Only lower Byte reserved and NS bit
not supported
Only lower Byte reserved and NS bit
not supported
Only lower Byte reserved and NS bit
not supported
Type 10 IPv6 packet length Multi value range
Type 11 IPv6 Traffic class Multi value range
Type 12 Reserved N/A N/A N/A N/A N/A
Type 13 IPv6 Flow Based Multi value range x x x x
Configuring BGP FlowSpec on IOS XR Routers
• Signalisation: Use of a new Address-family flowspec
!
router bgp1
bgprouter-id 6.6.6.6
address-family ipv4 flowspec
!
neighbor-group ibgp-flowspec
remote-as 1
update-source loopbook0
address-family ipv4 flowspec
!
!
neighbor 25.2.1.3
use neighbor-group ibgp-flowspec
!
neighbor 25.2.1.4
use neighbor-group ibgp-flowspec
!
!
flowspec
address-family ipv4
service-policy type pbr FS
!
Controller
!
router bgp1
bgprouter-id 3.3.3.3
address-family ipv4 flowspec
!
neighbor-group ibgp-flowspec
remote-as 1
update-source loopback0
address-family ipv4 flowspec
!
neighbor 25.2.1.11
use neighbor-group ibgp-flowspec
!
!
flowspec
local-install interface-all
!
Client
Advertise policy FS
Install all rules
on all interfaces
Configuring BGP FlowSpec on IOS XR Routers
• Verifying the Session Establishment (on Client)
RP/0/RP0/CPU0:Client#sh bgp ipv4 flowspec summary
BGP router identifier 3.3.3.3, local AS number 1
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 7072
BGP main routing table version 7072
BGP NSR Initial initsyncversion 0 (Reached)
BGP NSR/ISSU Sync-Group versions 7072/0
BGP scan interval 60 secs
BGP is operating in STANDALONE mode.
Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 7072 7072 7072 7072 7072 7072
Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
25.2.1.11 0 1 106269 105679 7072 0 0 1w1d 1001
RP/0/RP0/CPU0:Client#
Configuring BGP FlowSpec on IOS XR Routers
• Configuring Rules on the Controller
• In many aspects, the rules configuration on the controller is similar to the MQC (Modular QoS Configuration)
• Rules are defined in Cisco Common Classification Policy Language (C3PL) format:
– Traffic Matching is defined in class-map
– Action is defined in a policy-map and refers a class-map
– This policy-map is advertised by the “service-policy type pbr”
Configuring BGP FlowSpec on IOS XR Routers
• Configuring Rules on the Controller
!
class-map type traffic match-all match-UDP53
match destination-port 53
match protocol udp
end-class-map
!
class-map type traffic match-all match-src-ipv4-addr
match destination-address ipv4 25.1.104.0 255.255.255.0
end-class-map
!
!
policy-map type pbr FS
class type traffic match-src-ipv4-addr
police rate 100000 bps
!
!
class type traffic match-UDP53
redirect next 192.42.52.125
!
!
class type traffic class-default
!
end-policy-map
!
flowspec
address-family ipv4
service-policy type pbr FS
!
Configuring BGP FlowSpec on IOS XR Routers
• Configuring Rules on the Controller
class-map type traffic match-all MATCH-UDP123
match destination-port 123
match protocol udp
end-class-map
!
class-map type traffic match-all MATCH-SRCv4
match destination-address ipv4 2.1.1.0/24
end-class-map
!
policy-map type pbr FS1
class type traffic MATCH-SRCv4
police rate 100000 bps
!
end-policy-map
!
policy-map type pbr FS2
class type traffic MATCH-UDP123
redirect nexthop192.168.2.5
!
end-policy-map
!
flowspec
address-family ipv4
service-policy type pbr FS1
service-policy type pbr FS2
class-map type traffic match-all MATCH-UDP123
match destination-port 123
match protocol udp
end-class-map
!
class-map type traffic match-all MATCH-SRCv4
match destination-address ipv4 2.1.1.0/24
end-class-map
!
policy-map type pbr FS
class type traffic MATCH-SRCv4
police rate 100000 bps
!
class type traffic MATCH-UDP123
redirect nexthop192.168.2.5
!
end-policy-map
!
flowspec
address-family ipv4
service-policy type pbr FS
!
Configuring BGP FlowSpec on IOS XR Routers
• Configuring Type 1 – match “Destination IP”
RP/0/0/CPU0:Ctrl(config)#class-map type traffic match-all MATCHING-RULE
RP/0/0/CPU0:Ctrl(config-cmap)#match destination-address ipv4 81.253.193.0/24
RP/0/0/CPU0:Ctrl(config-cmap)#
RP/0/RP0/CPU0:Client#sh flowspec ipv4 detail
AFI: IPv4
Flow :Dest:81.253.193.0/24
Actions :Traffic-rate: 100000 bps (bgp.1)
Statistics (packets/bytes)
Matched : 0/0
Transmitted : 0/0
Dropped : 0/0
RP/0/RP0/CPU0:Client#sh flowspec ipv4 nlri
AFI: IPv4
NLRI (Hex dump) : 0x011851fdc1
Actions :Traffic-rate: 100000 bps (bgp.1)
RP/0/RP0/CPU0:Client#
Type Prefix length Prefix
1 byte 1 byte Variable
1 /24 81.253.193
0x01 0x18 0x51 fd c1
0x011851fdc1
Cisco IOS XR Routers BGP FS Implementation
NLRI type Match fields Value input method
XR PI ASR9K CRS NCS6000
Type 1 IPv4 Destination address
Prefix length
Type 2 IPv4 Source address Prefix length
Type 3 IPv4 protocol Multi value range
Type 4 IPv4 source or destination port
Multi value range
Type 5 IPv4 destination port
Multi value range
Type 6 IPv4 source port Multi value range
Type 7 IPv4 ICMP type Multi value range
Type 8 IPv4 ICMP code Multi value range
Type 9 IPv4 TCP flags Bit mask
Only lower Byte reserved and NS bit
not supported
Only lower Byte reserved and NS bit
not supported
Only lower Byte reserved and NS bit
not supported
Type 10 IPv4 packet length Multi value range
Type 11 IPv4 DSCP Multi value range
Type 12 IPv4 fragmentation bits
Bit mask
Only indication of fragment
Configuring BGP FlowSpec on IOS XR Routers
• Mixing several matching statements
class-map type traffic match-all MATCHING-RULE1
match source-port 10 20 30-40 50-52 60-70
match protocol udp
match dscp ef
match packet length 10-100 102-200 202-400 402-1500
match destination-port 80
match destination-address ipv4 11.200.4.0 255.255.255.0
end-class-map
RP/0/RSP0/CPU0:Client#sh flowspec afi-all detail
AFI: IPv4
Flow
:Dest:11.200.4.0/24,Proto:=17,DPort:=80,SPort:=10|=20|>=30&<=40|>=50&<=52|>=60&<=70,Length:>=10&<=100|>=
102&<=200|>=202&<=400|>=402&<=1500,DSCP:=46
Actions :Traffic-rate: 314152 bps (bgp.1)
Statistics (packets/bytes)
Matched : 0/0
Dropped : 0/0
RP/0/RSP0/CPU0:Client#sh flowspec afi-all nlri
AFI: IPv4
NLRI (Hex dump) :
0x01180bc80403811105815006010a0114031e452803324534033cc5460a030a4564036645c803ca550190130192d505dc0b812e
Actions :Traffic-rate: 314152 bps (bgp.1)
RP/0/RSP0/CPU0:Client#
Configuring BGP FlowSpec on IOS XR Routers
• We can mix several Actions:– Rate-limit + Redirect VRF/IP– Rate-limit + DSCP Marking– Redirect VRF/IP + DSCP Marking– Rate-limit + Redirect VRF/IP + DSCP Marking
• It’s not possible to mix:– Redirect VRF + Redirect NH IP– Redirect NH IP@A + Redirect NH IP@B
RP/0/RP0/CPU0:Client#sh flowspec ipv4 detail
AFI: IPv4
Flow :Dest:25.1.102.1/32,Proto:=17,Length:>=500&<=1550
Actions :Traffic-rate: 100000 bps DSCP: ef Nexthop: 25.3.9.3 (bgp.1)
Statistics (packets/bytes)
Matched : 75899782/106259694800
Dropped : 75686514/105961119600
RP/0/RP0/CPU0:Client#
Rate limit
DSCP marking
Redirect
Benefits of DDoS Mitigation with BGP FS
• Single point of control to program rules in many clients
• Allows a very precise description/matching of the attack traffic
• Can be used for both mitigation and diversion of the attack traffic, without impact the course of the rest of the traffic targeted to the victim
• Filtering stateless attacks on the edge route permits mitigation of millions of PPS of dirty traffic while liberating precious CPU cycle on the scrubbing device for more advanced mitigation needs
• The Cisco ASR9000 supports Arbor Peakflow SP TMS software on the VSM service card
• XRv can be used as a controller – Free to test with CCO account
DDoS Mitigation on ASR9K
Virtualised Service Module
• Cisco/Arbor Partnership• Peakflow SP TMS embedded on
VSM• Supported with
– RSP440 onwards (not RSP2)– All 9000 chassis except 9001
• Multi-purpose service card– CGN– IPSec– Mobile GW– DPI– ASAv– DDoS Mitigation
• Service chaining• KVM virtualised environment
F5 Silverline DDOS protection - Global Coverage
24/7 Support
F5 Security Operations Center (SOC) is available 24/7 with security experts ready to respond to DDoS attacks within minutes
- Seattle, WA US
SOC
Global Coverage
Fully redundant and globally distributed data centers world wide in each geographic region
- San Jose, CA US
- Ashburn, VA US
- Frankfurt, DE
- Singapore, SG
Industry-Leading Bandwidth
Scrubbing capacity of over 2.0 Tbps
Guaranteed bandwidth with Tier 1 carriers
F5 Silverline DDOS protection – Service Options
Always availableAlways onPrimary protection
available on-demand
The Always Available service runs on stand-by and can be initiated when
under a DDoS attack. F5 Silverline will being mitigation as soon as your traffic
is sent to us.
Primary protection as the first line of defense
The Always On service stops bad traffic from ever reaching your
network by continuously processing all traffic through the cloud-scrubbing service and returning only legitimate
traffic through your website.
F5 Silverline DDOS protection
Two Ways to Direct Traffic to Silverline Scrubbing Centers
Multiple Ways to Return Clean Traffic
L2VPN / VIRTUAL ETHERNET SERVICE
IP REFLECTION ™
GRE TUNNELS
PROXY
BGP (BORDER GATEWAY PROTOCOL)ROUTED MODE
DNS PROXY MODE
EQUINIX CLOUD EXCHANGE
Routed Configuration
Data Center
TCP Connection: SYNSRC: 86.75.30.9:27182DST: 1.2.3.4:80
86.75.30.9
F5 SilverlineDDoS
Protection
TCP Connection:SRC: 69.86.73.76:4243DST: 1.2.3.4:80
69.86.73.76
ISP Router
F5 Router
Internet
GRE Tunnel
Customer Admin
BGP Configuration Change:withdraw advertisement for 1.2.3.0/24
BGP Route Advertisement:F5 route for 1.2.3.0/24 becomes preferred
F5 Router Customer Router
1.2.3.4
1.2.3.5
1.2.3.6
1.2.3.7
TCP Connection: SYN-ACKSRC: 1.2.3.4:80DST: 86.75.30.9:27182
Clean traffic is returned via GRE Tunnel to
customer’s data center
Proxy Configuration
Data CenterDNS Query: www.abc.com
DNS Query:www.abc.com DNS Query: www.abc.com
DNS Response: www.abc.com 5.6.7.8
DNS Response:www.abc.com
5.6.7.8
Local DNS Public DNSServers
5.6.7.8 Proxy
1.2.3.4
DNS Response:www.abc.com
5.6.7.8
TCP Connection:SRC: 86.75.30.9:27182DST: 5.6.7.8:80
86.75.30.9
TCP Connection:SRC: 9.9.9.18:31415DST: 1.2.3.4:80
NAT Pool9.9.9.0/24
Customer Router
F5 SilverlineDDoS
Protection
Customer Admin
TCP Connection:SRC: 69.86.73.76:4242DST: 1.2.3.4:80
ISP Router ACLpermit: 9.9.9.0/24 1.2.3.4/32deny: any 1.2.3.4/32
DNS Configuration Change#www.abc.com 1.2.3.4www.abc.com 5.6.7.8
Authoritative DNS
TCP Connection:SRC: 69.86.73.76:4243DST: 5.6.7.8:80
69.86.73.76
ISP Router
DDoS Architecture Scrubbing Center
Cloud
CloudScrubbing
Service
Volumetric attacks and floods, operations
center experts, L3-7 known signature attacks
InspectionToolsets
Scrubbing Center
Inspection Plane
Traffic ActionerRoute Management
Flow Collection
Portal
Switching Routing/ACLNetwork
Mitigation
Routing(Customer VRF)
GRE Tunnel
Proxy
IP Reflection
X-Connect Customer
Data Plane
Netflow Netflow
Copied trafficfor inspection
BGP signaling
Signaling
Visibility
Management
Proxy Mitigation
Switching mirrors traffic to Inspection
Toolsets and Routing layer
Inspection Tools provide input on attacks for Traffic Actioner & SOC
Traffic Actioner injects blackholeroutes and steers
traffic
Network Mitigation removes advanced
L4 attacks
Proxy Mitigation removes L7
Application attacks
Flow collection aggregates attack
data from all sources
Egress Routing returns good traffic back to customer
Portal provides real-time reporting and
configuration
Ingress Router applies ACLs and blackholes traffic
Summary
• Traditional DDoS Mitigation
– Remote Triggered Blackhole Filtering
• Scalable DDoS Mitigation
– BGP FlowSpec
• Cloud DDoS Protection
– F5 Silverline
Thank you