BGP Flowspec(RFC5575) Case study and Discussion
Shishio Tsuchiya
• BGP Flowspec Overview
• BGP Flowspec case study
• JANOG35 Q&A
Agenda
Affect of DDOS attack
Customer
aggregation
node/line
Bandwidth of Backbone
Customer
line/node/servic
e
Target
Service203.0.113.1
The affect would be all of network wide…
RTBH(Remote Triggered Black Hole Filtering)
Target
Service203.0.113.1
203.0.113.1 via
192.0.2.1
192.0.2.1 null0
203.113.1 192.0.2.1
192.0.2.1 null0
203.113.1 192.0.2.1
192.0.2.1 null0
203.113.1 192.0.2.1
• RTBH(RFC5635) is well known technic in ISP
• static route to null(Black hole) preliminarily
• If incidence happen then BGP advertises route
• DDOS traffic will be stopped
Netflow+BGP Attribute
Why BGP Flow Specification will be needed
Non DDOS user also would be stopped.
It is difficult to discover/ attempt rule against DDOS attack which rapidly change and increasing
BGP Flowspec(RFC5575)+draft-ietf-idr-flow-spec-v6
Dst IP
Src IP
protocol
port
Dst port
Src Port
ICMP Type
ICMP Code
TCP Flags
Packet Length
DSCP
Fragment
traffic-rate
traffic-action
redirect
traffic-marking
Flow Type Action Rule+---------------------------------------------------------+
| AFI(2 octets) 1 and 2 |
+---------------------------------------------------------+
| SAFI (1 octet) 133 and 134 |
+---------------------------------------------------------+
| Length of Next Hop Network Address (1 octet) |
+---------------------------------------------------------+
| Network Address of Next Hop (variable) |
+---------------------------------------------------------+
| Reserved (1 octet) |
+---------------------------------------------------------+
| Network Layer Reachability Information (variable) |
+---------------------------------------------------------+
SAFI
133 Dissemination of flow specification rules
134 L3VPN dissemination of flow specification rules
BGP Flowspec defined in RFC5575. draft-ietf-idr-flow-spec-v6 for IPv6 BGP Flowspec
Flow type to identify traffic , Action Rule to execute policy against the traffic
“Flow Type” and “Action Rule” will be advertised by BGP update
BGP Flowspec(RFC5575)
Target
Service203.0.113.1
A,B,C to
203.0.113.1 drop
D and E to 203.0.113.1 100kbps
F markdown to dscp 0
100kbps
Netflow
collector
Flowspec uses netflow to collect traffic information
Flow rule and action will be distributed by BGP
• BGP Flowspec Overview
• BGP Flowspec case study
• JANOG35 Q&A
Agenda
• DDOS Problem
• Affect Large/Often to end user
• Not only end user but also Infrastructure Risk
• OPEX increase
• DDoS Analysis
• Large DDOS attack by botnet armies/Script Kiddies
• TCP Syn Flood greater than 1Mpps
• UDP fragment
• Most of Attack source APNIC(Chinese) IP source , difficult to track due to national NAT
• Deployed Flowspec for Peer & Transit router from RR
• Mitigation from egress point to cleaning vrf
• What was missing ?
• Multi vendor support (deployed Juniper and Arbor)
• Inter-Carrier
• Matching DSCP
Flowspec Use case 1 world wideTime Warner Telecom (TWTC) NANOG38 2006 Deployment Experience With BGP Flow Specification
https://www.nanog.org/meetings/nanog38/presentations/labovitz-bgp-flowsp
• Compare RTBH/PBR and Flowspec
• RTBH(Remote Triggered Black Hole)
Website can protect from DDOS attack, but no more traffic on website
• PBR(Policy Based Routing)
Can control traffic precisely by hardware
But need contact to service provide operator to run/remove policy when ddos detect
• Flowspec
Makes static PBR to dyanmic/Propagate PBR rules/do no need additional communication channel
• Deployed Flowspec on transit router
Would like to use on eBGP as architecture but can not trust customer/don’t like to use flow for ebgp session for stability reason
• What’s Next
• IPv6 and VPNv6 support
• Traffic Monitoring
• More vendors(only Juniper and Alcatel support at that time)
Flowspec Use case 2 world wideNeo Telecoms FRNOG18 2011Flowspechttp://media.frnog.org/FRnOG_18/FRnOG_18-6.pdf
• Background
• Attacker use zombies, if number of army of zombies then DDOS traffic will be massive (ex. DNS amp)
• Need Better tools
- Granularity : per flow
- Action : drop/rate-limit/redirect,
- Speedy/ Efficiency / Automation / Manageability
• Deployed FireCircle
• Wizard based UI to define policy from customer
• Apply XML configuration to BGP flowspec router via NETCONF
• eBGP flowspec propagate policy to GRNET router
• Expanding the service to GEANT community
https://fod.grnet.gr/
Flowspec Use case 3 world wideGRNET(Greek Research and Technology Network) TNC2012FireCircle: GRNET’s approach to advanced network security services’ management via bgp flow-spec and NETCONFhttps://tnc2012.terena.org/core/presentation/41
NETCONF
FireCircle
GRNET
GEANT
Participant
NREN
• DDOS Volume(average)
• JAPAN Q2:491.63Mbps Q3:365.8Mbps
• Asia Q2:530.5Mbps Q3:588.74Mbps
• World Wide Q2:759.83Mbps Q3:858.98Mbps
• NTP Amp trend(average volume)
• JAPAN Q2:3.22Gbps Q3:281.76Mbps
• Asia Q2:2.57Gbps Q3:2.70Gbps
• Attack Duration
• 92% DDOS stops within 1hour
• JAPAN: >1hour 92% average 3h21m
• Asia: >1hour 94.1% average 31m
• Professional DDOS service is exist ex)5min free 4$/hour
Atlas DDOS Trend report
Services UDP Source Port
Q3Maximum
DDOS Volume
Q3Average
DDOS Volume
SNMP 161 3.75Gbps 769.1Mbps
Chargen 19 21.26Gbps 1.12Gbps
DNS 53 43.45Gbps 1.31Gbps
SSDP 1900 51Gbps 5.11Gbps
• What’s Next• NTP Amp attack can create big volume.
• So Attacker using other protocol.
• SSDP(1900) is increasing
http://www.janog.gr.jp/meeting/janog35/files/2014/2077/3840/janog35-bgpfs-agatsuma-1.pdf
• ISP who is interesting in BGP Flowspec
• Amp attack are increasing under 5%-> over 70%
• and valuable
• Src 53 Dst 0/Src 123/Src 1900/Dst 80
Flowspec Use case 1
Protect Method For Point If Flowspec deployed
RTBH rapid action protect short duration DDOS more specific flow
can use policer for DDOS amp
ACL permanent action flexible/need time to deploy to be rapidly/manage acl rule
Mitigation premier service expensive would be effective
• ISP who already deployed by Juniper
• and would like to deploy to be more wide by Cisco
• Flowspec is very useful feature against today’s DDOS, but one consideration point is scalability spec of forwarding router
• Rule was too long, so forwarding router could not apply filter as the result not only DDOS but also normal traffic down
Flowspec Use case2
DDOS detect/BGP update send Rule was too long for forwarding router, cold not apply filter
• BGP Flowspec Overview
• BGP Flowspec case study
• JANOG35 Q&A
Agenda
• JANOG had a session of BGP Flowspec in JANOG35Shishio Tsuchiya Cisco Systems G.K.
Shojiro Hirasawa BIGLOBE Inc.
Satoshi Agatsuma TOYO Corporationhttp://www.janog.gr.jp/en/index.php?JANOG35_Meeting%2FJANOG35_Program_Contents%2FBGPFS
http://www.janog.gr.jp/meeting/janog35/program/bgpfs/
• Share question/discussion on JANOG35 meeting
Discussion summary
• Let’s confirm in detail for RFC and IETF WG draft.
Q1. Does Flowspec really useful?
Typ
e
IPv4
(RFC5575)
IPv6
(flow-spec-v6)
1 Destination Prefix Destination IPv6 Prefix
2 Source Prefix Source IPv6 Prefix
3 IP Protocol Next Header
4 Port Port
5 Destination port Destination port
6 Source port Source Port
7 ICMP type ICMP type
8 ICMP code ICMP type
9 TCP flags TCP flags
10 Packet length Packet length
11 DSCP DSCP
12 Fragment Fragment
13 N/A Flow Label
Flow Type has operator code which can
specify lt(less than) gt(grater than)
eq(equal) .
• Most of action rule is defined both IPv4 and IPv6.
• But redirect IP seems confusing , should watch idr wg activity
Q1. Does Flowspec really useful? cont’dtype extended community Actual Action RFC/draft
0x8006 traffic-rate Policing rate
0:drop
RFC5575
0x8007 traffic-action specific acction
Terminal bit:(0 is terminal)
Sample bit:(1 is logging/sampling)
RFC5575
0x8008
0x8208
0x800b
redirect AS-2byte
redirect AS-4byte
redirect IPv6 specific AS
redirect to specific vrf flowspec-redirect-rt-bis
flowspec-redirect-rt-bis
flow-spec-v6
0x8108 redirect IPv4 address
redirect IPv6 address
redirect to next hop address
redirect to next hop address
flowspec-redirect-rt-bis
flowspec-redirect-ip
flowspec-redirect-ip
0x8009 traffic-marking marking DSCP values flowspec-redirect-rt-bis
flow-spec-v6
• Cisco
IOS-XR:5.2.0-
IOS-XE3.14 –(RR)
Forwarding router in 3.15
• Juniper
JUNOS 7.3-
• Alcatel-Lucent
SR-OS 9.0R1-
Implementation status
• Arbor NetworksPeakFlow 6.0-
• Genie Networks5.5.1-
• ExaBGP
Q2. How about interoperability in multi vendor?
Cisco IOS Cisco IOS-XR JNPR
JUNO
S
ALU
SR-OS
Arbor Genie
Cisco
IOS
Cisco
IOS-
XR
JNPR
JUNOS
ALU
SR-OS
Arbor
Genie
• There is some intorop report but may need more interop test to deploy ISP network
Q3.Flow is really enough to monitor ISP traffic?
DDOS Traffic
Normal Traffic
Inline type model offramp model
need many equipment to monitor all
of subscribers
can use shared resource
have to monitor huge traffic only suspect traffic will transit to
mitigation
when mitigation fail, the failed
equipment should just transit traffic
when mitigation fail, then advertise
BGP to change rule
offramp solution
would be reasonable
• Today’s most of mobile carrier deployed CGN as solution of IPv4 exhaustion problem.
• Malware/DDOS tool of android already exist.
• Flow based filtering will be more importance to reduce side affect of DDOS
Q4.How is DDOS on mobile network?
Global Address Global
Address
RFC6598 ISP Shared Address
or
RFC1918 Private Address
• It’s depends on router architecture.
APNIC38 Geoff Huston (APNIC) - What's so special about 512?
APRICOT2012 Greg Hankins, Brocade Pushing the Limits, A Perspective on Router Architecture Challenges
• Usually QoS/PBR is used on TCAM, so performance impact would be minimize .
Q5.Performance issue?
https://supportforums.cisco.com/document/105496/asr9000xr-understanding-route-scale
• Flowspec should work in eBGP peer. But eBGP validation rule for received route should be relaxed.
• On transit AS/Router server on IXP, it would be desirable service. Because if one AS sends DDOS then affects to another AS.
• Validation rule should be relax so maybe we should consider co-exist solution with RPKI to be more powerful security solution.
• Should check “Revised Validation Procedure for BGP Flow Specifications” draft-ietf-idr-bgp-flowspec-oid
Q6.eBGP Use case?
ROA
Transit AS Route Server on IXPco-Exist with RPKI
• There is Openflow DDOS protection solution.
• Hybrid OF use TCAM also.
• Difference point are network architecture(full distributed vs controller) and API(OF vs BGP)
Q7.How is OpenFlow DDOS solution?
• Current DDOS are high volume/short duration/amp attack variable and increasing
• BGP Flowspec is useful solution against today’s DDOS attack
• BGP Flowspec is almost ready to deploy in ISP network.
• Need detail implementation information of each of vendors(scalability/nexthop address/IPv6) and interoperability test result.
• eBGP should work and customer may desire on-demand Firewall/PBR services like a FireCircle.
Summary