133
1
1
22 v1.0
IXP & Routing Tutorial– BGP Basics
W E B I N A R C O U R S E
3 v1.03
Acknowledgements
• Philip Smith
• Cisco Systems
4 v1.04
Overview
• What is BGP?
• BGP Features
• Path Vector Routing Protocol
• BGP General Operation
• BGP Terminology
• Inserting Prefixes into BGP
5 v1.05
What is BGP?
• Border Gateway Protocol
• A Routing Protocol used to exchange routing information between different
networkso Exterior gateway protocol
• Described in RFC4271o RFC4276 gives an implementation report on BGP
o RFC4277 describes operational experiences using BGP
Interior Gateway Protocol Exterior Gateway Protocol
OSPF, IS-IS … BGP
6 v1.06
BGP Features
• Path Vector Protocol
• Incremental Updates
• Many options for policy enforcement
• Widely used for Internet backbone
• Autonomous systems
• Classless Inter Domain Routing (CIDR)
7 v1.07
What is an Autonomous System?
• Group of Internet Protocol-based networks with the same routing policyo Usually under single ownership, trust or administrative control
• The AS is used both in the exchange of exterior routing information (between
neighboring ASes) and as an identifier of the AS itself
• The Autonomous System is the cornerstone of BGPo It is used to uniquely identify networks with a common routing policy
8 v1.08
Autonomous System Number (ASN)
• Globally unique identifiers for IP networks
• ASN uniquely identifies each network on the Internet, allocated to each
Autonomous System (AS) for use in BGP routing
• 2-byte only AS number range : 0 – 65535
• 4-byte only AS number range : 65,536 - 4,294,967,295
9 v1.09
Autonomous System Number (ASN)Number Bits Description Reference 0 16 Reserved RFC7607 1 - 23455 16 Public ASN's
23456 16 Reserved for AS_TRANS RFC6793
23457 - 64495 16 Public ASN's
64496 - 64511 16 Reserved for use in documentation and sample code RFC5398
64512 - 65534 16 Reserved for private use RFC6996 65535 16 Reserved RFC7300
65536 - 65551 32 Reserved for use in documentation and sample code RFC5398
65552 - 131071 32 Reserved 131072 - 4199999999 32 Public 32-bit ASN's
4200000000 - 4294967294 32 Reserved for private use RFC6996
4294967295 32 Reserved RFC7300
https://www.iana.org/assignments/as-numbers/as-numbers.xhtml
10 v1.010
What is Path Vector Routing Protocol
• A path vector routing protocol is used to span different autonomous systems
• It defines a route as a collection of a number of AS that it passes through from
source AS to destination AS, i.e.
{65001 65002 65003 65007}
• This list of AS numbers is called AS path and used to avoid routing loop
• AS path is also used to select path to destination
11 v1.011
Path Vector Routing Protocol
AS64500 AS64501
AS64502AS64503
172.16.0.0/16
172.16.0.0/16 64503172.16.0.0/16
64501 64503
172.16.0.0/16
12 v1.012
Definitions
• Transit
§ carrying traffic across a network, usually for a fee
• Peering
§ exchanging routing information and traffic
13 v1.013
Peering and Transit example
A and B can peer, but
need transit
arrangements with D to
get packets to/from C
provider A
provider C
provider B
Backbone Provider D
IXP-EastIXP-West
14 v1.014
BGP General Operation
• Learns multiple paths via internal and external BGP speakers
• Picks the best path and installs it in the routing table (RIB)
• Best path is sent to external BGP neighbours
• Policies are applied by influencing the best path selection
15 v1.015
BGP Attributes
• Well-known attributes – must be supported by every BGP implementationo Mandatory attributes – must be included with every route entry. If one attribute is missing, it will result in an
error message
o Ex: ORIGIN, AS_PATH, NEXT_HOP
o Discretionary attributes – every BGP router must recognize, but they don’t have to be present with every
route entry
o Ex. ATOMIC_AGGREGATE, LOCAL_PREF
• Optional attributes – not necessarily supported by all BGP implementations. It can be either
transitive or non-transitive.o COMMUNITY, AGGREGATOR, MULTI_EXIT_DISC
16 v1.016
Internal & External BGP
• eBGP used to:o Exchange networks/routes between ASes
- Aggregates and sub-aggregates
o Implement routing policies- To manipulate inbound and outbound traffic
• iBGP is used to:o Carry customer networks/prefixes
o Internet routes (some or all) across the AS backbone
17 v1.017
BGP Message Types
• There are 5 types of BGP messages for communication:o Openo Keepaliveo Updateo Notificationo Route-Refresh
18 v1.018
BGP Message Types
• Open:o After a TCP connection has been established between two BGP routers, an Open message is
sent- Once the open message is confirmed (keepalive), the BGP session is established – become BGP
peers/neighbors!
o Contains:- Sender’s ASN
- BGP version
- BGP router ID
- Hold-time (3 x keepalive interval)
19 v1.019
BGP Message Types
• Keepalive:o Exchanged initially to acknowledge Open messages
o Exchanged periodically (60 secs) to maintain BGP session- Dataless packet
• Update:o BGP peers exchange network information through Update messages
- One update for each path!
o Contains:- Withdrawn routes – no more reachable- Path attributes – attributes for this path to reach the destinations specified by the NLRI- NLRI – list of networks reachable through this path
20 v1.020
BGP Message Types
• Notification:o Sent when an error condition is detected
o The BGP session is torn down immediately!
o Contains:- Error code
- Error sub-code
- Data related to error
21 v1.021
BGP Neighbor States
• There are six BGP neighbour states:
o Idle
o Connect
o Active
o OpenSent
o OpenConfirm
o Established
22 v1.022
BGP Neighbor States
• A BGP router goes through six different states• Idle
- The router is looking for a route to its neighbor
• Connect- BGP router moves from Idle to Connect state if it has found a route to its neighbor, and has started
the TCP handshake
- If the TCP session successful, sends an Open message (and transitions to OpenSent)
- Else, move to Active state
23 v1.023
BGP Neighbor States
• Active- A router transitions to Active state if the initial TCP connection was not successful (in Connect state)
- Restarts the TCP connection
- If successful, sends an Open message
- Else, falls back to Idle state
• OpenSent- An Open message has been sent to the neighbour
- Waiting for Open message from neighbour
- If it receives an Open message and there are no mismatches (version, source addr same as TCP
addr, ASN, router-ID, TTL, md5), sends KeepAlive, moves to OpenConfirm
- Else (if mismatches/errors), sent Notification and falls back to Idle
24 v1.024
BGP Neighbor States
• OpenConfirm- waiting for the initial KeepAlive
- If received, transitions to Established
- If holdtimer expires or Notification received, moves to Idle
• Established- The BGP neighbor relationship (session) is established!
- Routing information can now be exchanged
- If holdtimer expires/error, moves back to Idle
25 v1.025
BGP State Machine
Idle
Active
Connect
OpenSent
OpenConfirm
Established
26 v1.026
BGP/IGP model used in ISP networks
• BGP is used internally (iBGP) and externally (eBGP)
• iBGP – used to carry some/all Internet prefixes across ISP backbone and ISP’s
customer prefixes
IGP
iBGP
IGP
iBGP
IGP
iBGP
IGP
iBGP
eBGP eBGP eBGP
AS65001 AS65002 AS65003 AS65004
• eBGP – used to
exchange prefixes
with other ASes and
implement routing
policy
27 v1.027
eBGP Neighbor Relationship
• eBGP neighbors/peerso BGP session established between routers in different ASes
o Generally directly connected!- Session established using directly connected interface IP
- Peering address must match the TCP session!
o Else, we need a static route to reach the neighbor and change the eBGP TTL value
(default 1)
AS 65001 AS 65000
router bgp 65001neighbor 172.16.12.2 remote-as 65000!address-family ipv4neighbor 172.16.12.2 activate!
172.16.12.0/30
.1
28 v1.028
iBGP Neighbor Relationship
• iBGP neighbors/peerso BGP session established between routers within the same AS
o Does not need to be directly connected
- IGP ensure reachability (TCP connection)
o Generally using loopback addresses
AS 65001 router bgp 65001neighbor 10.10.10.2 remote-as 65001
!
29 v1.029
iBGP Operation
• iBGP routers must:
o Originate directly connected routes
• Carry routes learned from outside the AS to all routers within the ASo Fully-meshed instead of redistributing!
o Advertise routes learned from eBGP peers to all iBGP peers!
• To prevent routing loops (in a fully-meshed network)o iBGP routers are not allowed to advertise iBGP learned routes to other iBGP peers!
30 v1.030
iBGP full-mesh
router bgp 65400neighbor 10.10.10.2 remote-as 65400neighbor 10.10.10.3 remote-as 65400neighbor 10.10.10.4 remote-as 65400!
AS65400
R1
R2
R3
R4
R1:
router bgp 65400neighbor 10.10.10.1 remote-as 65400neighbor 10.10.10.3 remote-as 65400neighbor 10.10.10.4 remote-as 65400!
R2:
Example configuration on R1 and R2
31 v1.031
Sourcing iBGP from Loopback
• By default, routers use the exit-interface address as the source address for locally
originated packets (updates)o If the BGP TCP session was established using any other interface (loopbacks) addresses, the
source address for BGP updates must match!
• In Cisco IOS, the update-source loopback command achieves this
router bgp 65400neighbor 10.10.10.1 remote-as 65400neighbor 10.10.10.1 update-source loopback 0!
32 v1.032
Insert Prefixes into BGP
Examples in IOS Functionnetwork 192.168.1.0 mask 255.255.255.0
Add the specific route 192.168.1.0/24 into BGP routing table.
redistribute OSPF redistribute all the routes in OSPF routing table into BGP routing table.
33 v1.033
Inserting prefixes into BGP – network command
• Configuration Example
• A matching route must exist in the routing table before the network is announced
• Forces origin to be “IGP”
router bgp 65400network 10.10.32.0 mask 255.255.254.0
ip route 10.10.32.0 255.255.254.0
34 v1.034
Configuring Aggregation – Network Command
• Configuration Example
• A matching route must exist in the routing table before the network is announced
• Easiest and best way of generating an aggregate
10.10.1.0 255.255.255.0
10.10.2.0 255.255.255.0
10.10.3.0 255.255.255.0
…
router bgp 64500network 10.10.0.0 mask 255.255.0.0
ip route 10.10.0.0 255.255.0.0 null0
3535 v1.0
IXP & Routing Tutorial– BGP Attributes
W E B I N A R C O U R S E
36 v1.036
BGP Path Attributes
• Attributes describe the path to a network(s)/NLRIo Used to enforce routing policies for path control!
Well-known Mandatory
Well-known Discretionary
Optional Transitive
Optional Non-transitive
AS_PATHNEXT_HOP
ORIGIN
LOCAL_PREFATOMIC_AGGREGATE
COMMUNITYAGGREGATOR
MED
Always included in BGP updates Can be included (for path control)!
37 v1.037
AS_PATH
• Indicates the list of ASes a route has passed through to reach the local ASo the list of ASes to reach a destinationo can influence path selection!
AS6450110.10.0.0/16
AS6450210.20.0.0/16
AS64503 AS64509
10.10.0.0/16 64503 64502 6450110.20.0.0/16 64503 64502
10.10.0.0/16 64502 64501 10.20.0.0/16 64502
38 v1.038
AS_PATH
• Used to ensure a loop-free exchange of routing info between ASes o If own AS is seen in an update from an eBGP peer, loop is detected (Update is
dropped)!
AS6450110.10.0.0/16
AS6450210.20.0.0/16
10.10.0.0/16 64503 64502 6450110.20.0.0/16 64503 6450210.30.0.0/16 64503AS64503
10.30.0.0/16
X
39 v1.039
NEXT_HOP
• Indicates the next hop address to reach the destinationo Source of the update packet!
• For eBGPo eBGP neighbor address (to reach the next AS)
• For iBGPo Generally the loopback address
40 v1.040
NEXT_HOP
• eBGP learned routes are advertised to iBGP peers without changing the next hopo Routers within the AS need to be able to reach the next hop (IGP or static)o Else, external routes not installed in the routing table!
AS 6450210.20.0.0/16
AS 6450310.30.0.0/16
10.20.0.0/30
.1 .2AS 64501eBGP
iBGP
R1 R2
R3R3:10.30.0.0/16 130.10.0.110.20.0.0/16 130.10.0.1
41 v1.041
NEXT-HOP-SELF
• Override the eBGP next hop default behavior with next-hop-self commando Advertises itself as the next hop for external routes
▸ Reachable through IGP
AS 6450210.20.0.0/16
AS 6450310.30.0.0/16
10.20.0.0/30.1 .2AS 64501eBGP
iBGP
R1 R2
R3
10.10.0.12
10.10.0.13
R2:router bgp 64501neighbor 10.10.0.13 remote-as 64501neighbor 10.10.0.13 next-hop-self
!
R3:10.20.0.0/16 10.10.0.1210.30.0.0/16 10.10.0.12
42 v1.042
ORIGIN
• Indicates the origin of the route
Origin Methods Examples
IGP(i) Interior to the originating AS. Generated by BGP “network” statement
network 172.16.16.0 mask 255.255.254.0
EGP(e) By EGP (not used now)
Incomplete(?) Route’s origin is unknown. Usually redistributed from another routing protocol
redistribute ospf
43 v1.043
ORIGIN
44 v1.044
LOCAL_PREF
• Local preference tells routers within the AS (local) the preferred path to exit the ASo Path with highest local_pref wins
▸ Outbound traffic!
• Local to the ASo Advertised only to iBGP peers!
AS 64502130.10.0.0/16
AS 64503
AS 64501
R1
R2AS 64505
LP-200
LP-500
R3
45 v1.045
MED
• Multi-exit discriminator is inter-AS non-transitive o Indicates to neighbor AS about the preferred entry points into the local AS (incoming
traffic)
• The path with lowest MED wins!
AS 200
160.10.0.0/16R1
R2
MED-10
MED-200
MED-10
MED-200R4
R3AS 300
R5
46 v1.046
COMMUNITY
• Used to group prefixes (incoming/outgoing) and apply policies to the communitieso A prefix can belong to more than one community
• Is (was?) a 32-bit integer o Represented as two 16-bit integers [ASN:number], for example: 64501:200
▸ Works well for 2-byte ASN
• With 4-byte ASNso Common to see [private-ASN:number]o RFC 8092 (BGP Large Communities): 96-bit integer
▸ [32-bit ASN:32-bit:32-bit], for example: 64496:4294967295:2
47 v1.047
BGP Best Path Selection
Highest Local Preference
Locally originated routes
Shortest AS Path
Lowest Origin Code (i
48 v1.048
BGP Operation
• BGP learns routes from iBGP and eBGP peers
o Selects best path based on the attributes
o Installs best path in the routing table
o Advertises the best paths to its other BGP peers▸ eBGP learned routes to iBGP peers▸ iBGP learned routes to eBGP peers
49 v1.049
BGP Operation
Routing Table
Local Router
PeerPeer
Inbound updates
Outbound updates
(best paths)BGP Table
Best Paths
Filters (Policy)
50 v1.050
BGP Tables
• Neighbor Tableo List of all BGP neighbors
• BGP Tableo List of routes learned from all BGP neighborso (And locally originated routes!)
• Routing (Forwarding) Tableo All best paths
▸ selected based on attributes and whose next-hops are reachable!
5151 v1.0
IXP & Routing Tutorial– BGP ScalingW E B I N A R C O U R S E
52 v1.052
Agenda
• BGP Peer Group
• BGP Route Reflection
53 v1.053
BGP Peer Group
• Problem: number of BGP updates in an iBGP mesho BGP updates generated for each neighbor individually
▸ CPU wasted on repeat calculations
o iBGP neighbors receive the same update▸ Contain same info
• Solution: Peer Groupso Group neighbors with the same outbound update policyo Updates are generated once per group
54 v1.054
BGP Peer Group
• Still need to establish TCP sessions individually
• Useful when many neighbors have the same outbound policieso Runs through the outbound filters only once for the group (applied to all members)
• Members can have a different inbound policy!
• Simplifies configurationo Define the peer groupo Add neighbors to the peer group
▸ Still need to configure peering individually
o Apply filters (outbound) to the group
55 v1.055
Peer Group – Best Practices
• Always configure peer-groups for iBGPo Even if there are only a few iBGP peerso Easier to scale network in the future
• Consider using peer-groups for eBGPo Especially useful for multiple BGP customers using same AS (RFC 2270)o Also at IXPs where ISP policy is generally the same for each IX peer
56 v1.056
BGP Loop Prevention
• eBGPo AS-PATH attributeo If the local ASN is seen in a route received from a eBGP peer, a routing loop has
occurred▸ Drop the route!
• iBGPo BGP router is not allowed to advertise iBGP learned routes to other iBGP peers within
the ASo How do all iBGP routers learn about each other’s networks?
▸ iBGP full-mesh!
57 v1.057
Scaling iBGP mesh
• Number of iBGP sessions n(n-1)/2
▸ 10 routers = 45 sessions▸ 100 routers = 4950 sessions
• Number of BGP updateso Every update needs to be sent to all iBGP
peer
10 routers = 45 iBGP sessions
58 v1.058
Solution
• Route Reflection (RFC4456)o RR client peers only with the RRo RR and its clients form a CLUSTERo Non-clientso An AS can have multiple clusters
RRs do not affect the actual traffic path; only affects the path BGP messages take!
59 v1.059
RR Operation
• When a RR receives an Update:o If from a client peer, the route is
reflected (advertised) to other clients, and non-client peers.
o If from a non-client, only reflected to client peers
o If from a eBGP peer, reflected to both client and non-client peers.
AS 65001
AS 17821
Clients
Route Reflector
D
A C
B
Routing loops can happen with RRs!
60 v1.060
Avoiding loops in RR
• Originator_ID attributeo The BGP router id of the originator, created by the RR
▸ If you see your router id in the Originator_ID attribute, loop has occurred.
• Cluster_List attributeo In a cluster with a single RR, the Cluster ID is the router ID of the RRo If more than one RR in a cluster, a 4-byte Cluster ID configuredo Cluster_List reflects the sequence of clusters a route has passed througho When a RR reflects routes (from clients) to non-clients, it appends the local Cluster ID to the
Cluster_List▸ When a RR receives an update, if the local Cluster ID is seen in the list, a loop has occurred!
(drops the update)
61 v1.061
RR Design
• Divide the AS into multiple clusterso At least one RR and few clients in a clustero Could have more than one RR in a cluster for redundancy
▸ NOT recommended!
• Peering between clients in a cluster not necessary (but could be)o The RR reflects routes between them
• RRs in different clusters must be fully meshed with each other and with any iBGP router that’s not a part of any cluster
62 v1.062
RR example
• RR configuration
• RR Client config
router bgp 17821
neighbor 2406:6400:2 remote-as 17821neighbor 2406:6400:2 route-reflector-clientneighbor 2406:6400:3 remote-as 17821neighbor 2406:6400:3 route-reflector-clientneighbor 2406:6400:4 remote-as 17821neighbor 2406:6400:4 route-reflector-client
router bgp 17821
neighbor 2406:6400:1 remote-as 17821
Only peers with the RR
63 v1.063
RR Selection
• Best practice is to follow the physical topology
o Ensures traffic forwarding paths wont be affected
o Prevents routing loops
E – client to both RRs
LOOP!
2406:6400:A8::1
BA
RR - DRR - C
2400:6400:C8::/64
2406:6400:B8::1
64 v1.064
RR Redundancy
• Most ISP networks would overlay two clusters o Each client peers with RRs in different clusters (same POP) for redundancy (NEVER two RRs
in the same cluster!)o All RRs fully-meshed!o Can have full-mesh between clients in the same cluster
RR2RR1
X
A
X
router bgp 17821bgp cluster-id 1.1.1.1
CLUSTER_LIST
AS 17821
Cluster 1Cluster 2
A
RR
RR
RR
RR
AS 65000
6565 v1.0
IXP & Routing Tutorial– Policy ControlW E B I N A R C O U R S E
66 v1.066
Influence Path Selection – Policy Control
Routing Table
Local Router
PeerPeer
Inbound updates
Outbound updates
(best paths)BGP Table
Route-map
Filter-list
Prefix-list
Best Paths
67 v1.067
Policy Tools
• Prefix-listo To filter routes/prefixes
▸ More granularity than as-path filters
• Filter-listo To filter based on AS-patho To apply AS-path ACLs
• Route-mapo modify attributes based on condition matches
68 v1.068
Path control - Attributes
• Inbound Traffic:o AS-Path, MED, Community
• Outbound Traffic:o Local Preference
69 v1.069
Prefix List
• Ex 1:
– Allows any prefix with prefix length between 8 and 24– Implicit DENY at the end!
ip prefix-list name/num [seq#] permit | denyprefix/length [ge value][le value]
ip prefix-list TEST permit 0.0.0.0/0 ge 8 le 24
• Ex 2:
– Permit the prefix 2406:6400::/32 up to /48– Implicit DENY at the end!
ipv6 prefix-list TEST-v6 permit 2406:6400::/32 le 48
70 v1.070
Prefix List
• Ex 3:
– Deny default route
ip prefix-list TEST deny 0.0.0.0/0
• Ex 4:
– Deny IPv6 default routes
ipv6 prefix-list TEST-v6 deny ::/0
71 v1.071
Prefix List
router bgp 17821network 100.100.0.0 mask 255.255.224.0neighbor 20.20.20.1 remote-as 20neighbor 20.20.20.1 prefix-list MY-PREFIX outneighbor 20.20.20.1 prefix-list PEER-PREFIX in!ip prefix-list MY-PREFIX permit 100.100.0.0/19 ip prefix-list MY-PREFIX deny 0.0.0.0/0 le 32!ip prefix-list PEER-PREFIX permit 200.200.0.0/16ip prefix-list PEER-PREFIX deny 0.0.0.0/0 le 32
72 v1.072
AS-path ACL
• AS-path access list use regular expressions
. Matches any one character* Matches any sequence of pattern before *+ match at least one preceding expression^ beginning with$ ending with_ matches start, end, space, comma, braces
ip as-path access-list num [permit|deny] regex
73 v1.073
AS-path ACL
• Example regular expressions:^$ locally originated routes_100$ originated by AS 100_100_200_ passing through 100 and 200^(_100)+$ originated by 100, multiple occurrence
ip as-path access-list 10 permit ^100$
• Example 1:
– Allow any prefix originated and received from AS100– Implicit DENY at the end
• Use filter-list to apply AS-PATH access-lists
74 v1.074
AS-path ACL
router bgp 17821network 100.100.0.0 mask 255.255.224.0neighbor 30.30.30.1 remote-as 30neighbor 30.30.30.1 filter-list 30 outneighbor 30.30.30.1 filter-list 40 in!ip as-path access-list 30 permit ^$ip as-path access-list 40 permit ^30$
75 v1.075
Route Map
• Default is permito Implicit DENY at the end!
route-map name [permit | deny] [sequence]
If {(A or B or C)and D} matchThen {set X and Y}
ElseIf E matchesThen set Z
Else (for everything else)Do/set nothing
route-map TEST permit 20match Eset Z
route-map TEST permit 30
route-map TEST permit 10match A B Cmatch Dset Xset Y
76 v1.076
Match (conditions) & Set (actions)
Command Descriptionmatch community BGP community tagmatch as-path AS-path access listmatch ip address Access list or prefix-list
Command Descriptionset as-path Modify AS-pathset community Apply BGP community tagset metric Modify MEDset local-preference Modify local preference
77 v1.077
Route Map
router bgp 17821neighbor 30.30.30.1 remote-as 30neighbor 30.30.30.1 route-map AS-OUT outneighbor 30.30.30.1 route-map LP-IN in!route-map AS-OUT permit 10set as-path prepend 17821 17821 17821!route-map LP-IN permit 10match as-path 1set local-preference 150!route-map LP-IN permit 20!ip as-path access-list 1 permit _30$
78 v1.078
Route Map
router bgp 17821network 100.100.0.0 mask 255.255.224.0 route-map SET-AGGneighbor 20.20.20.1 remote-as 20neighbor 20.20.20.1 send-communityneighbor 20.20.20.1 route-map TR-IN in
!route-map SET-AGG permit 10set community 100:1000
!route-map TR-IN permit 10match community 5set local-preference 150
!route-map TR-IN permit 20!ip community-list 5 permit 20:3000ip community-list 5 permit 20:4000
• Setting and Matching communities:
79 v1.079
Applying Policy Filters
• Incoming/Outgoing updates are filtered through policieso BGP table does not contain routes rejected by policies
• Whenever there is a BGP policy change, we need too Trigger an update to force in/outbound routes through the new filters (else only the
ones already in BGP table)o either through a Hard Reset or a Soft Reset (Route Refresh)
• If the filter is applied to:o Outbound routes: need to resend its BGP table through the filtero Inbound routes: need its neighbors to resend their BGP tables
80 v1.080
Hard Reset
• Hard reset of a BGP sessiono Tears down the TCP connectiono Re-establish the TCP sessiono Resend the BGP table to neighbors affected by the reseto Relearn all routes from neighbors
o Disrupts network connectivity▸ Same as a router reboot!
clear ip bgp *clear ip bgp clear bgp ipv6 unicast *clear bgp ipv6 unicast
81 v1.081
Route Refresh
• Route refresh (soft reset) does not tear down BGP sessiono Not disruptive!
• No manual configuration requiredo Negotiated automaticallyo Peering routers need to support route refresh
sh ip bgp neighbor sh ip bgp all neighborssh bgp ipv6 unicast neihgbors
Neighbor capabilities: Route refresh: advertised and received
82 v1.082
Route Refresh
o Tells neighbor to resend its BGP table
o Resends full BGP table to its neighbor
clear ip bgp [soft] inclear bgp ipv6 unicast [soft] in
clear ip bgp [soft] outclear bgp ipv6 unicast [soft] out
83 v1.083
Soft Reconfiguration
• All current routers should support route refresho If the router (local or peer) does not have route refresh capability, use soft-
reconfiguration
• With soft-reconfiguration, the router stores a copy of the received routes in addition to the BGP table (allowed by policy filters)o Thus, requires additional memory!
8484 v1.0
IXP & Routing Tutorial– Introduction of Route Servers
W E B I N A R C O U R S E
85 v1.085
What is a Route Server?
• A route server is for internet exchange (IX) operators that provides an alternative to full eBGP mesh peering among the service providers who have a presence at the IXP.
• Also:o Announces routes to participating IXP members according to their routing policy
definitions
86 v1.086
Features of a Route Server
• Helps scale routing for large IXPs
• Simplifies Routing Processes on ISP Routers
• Optional participationo Provided as service, is NOT mandatory
• Optionally uses Policy registered in IRR
87 v1.087
Diagram of N-squared Peering Mesh
• For large IXPs (dozens for participants) maintaining a larger peering mesh becomes cumbersome and often too hard
88 v1.088
Peering Mesh with Route Servers
• ISP routers peer with the Route Servers– Only need to have two eBGP sessions rather than N
RS RS
89 v1.089
RS based Exchange Point Routing Flow
TRAFFIC FLOWROUTING INFORMATION FLOW
RS
90 v1.090
Advantages of Using a Route Server
• Helps scale Routing for very large IXPs
• Separation of Routing and Forwarding
• Simplify Routing Configuration Management on ISPs routers
90
91 v1.091
Disadvantages of using a Route Server
• ISPs can lose direct policy controlo If RS is only peer, ISPs have no control over who their prefixes are distributed to
▸ Some IXPs provide community based filtering option
• Completely dependent on 3rd partyo Configuration, troubleshooting, etc…
92 v1.092
Typical usage of a Route Server
• Route Servers may be provided as an OPTIONAL serviceo Most common at large IXPs (>50 participants)o Examples: LINX, HKIX, AMS-IX, etc
• ISPs peer:o Directly with significant peerso With Route Server for the rest
93 v1.093
Things to think about...
• Would using a route server benefit you?o Avoids having to maintain a large number of eBGP peerso But can you afford to lose policy control? (An ISP not in control of their routing policy
is what?)
9494 v1.0
IXP & Routing Tutorial– Multihoming Techniques
W E B I N A R C O U R S E
95 v1.095
ISP Hierarchy
95
Tier-1
Tier-1 Tier-1
Tier-1
Regional ISPs Regional
ISPsRegional
ISPs
Regional ISPs
Access ISPs Access
ISPs
Access ISPs
Access ISPs Access
ISPs
Access ISPs
IXP IXP
Source: Philip Smith “Introduction to Internet”
96 v1.096
Exchanging Routes
• Pay someone to advertise your networkso TRANSITo Make sure they have good onward peering/transit!
• Interconnect with as other ASes to exchange locally originated routes and traffico PEERINGo Private Peering
▸ Between two ASes o Public Peering
▸ at an IXP (domestic/global)
96
97 v1.097
Achieving Redundancy
97
• More than one path to the same ISP– Dual-homed
YOU
YOU ISP
ISPYOU
ISP
Single-homed
Dual-homed
98 v1.098
Achieving Redundancy – Multihoming
98
• More than one upstream ISP– Multi-homed
ISP2
ISP1
YOU
YOU
ISP2
ISP1
99 v1.099
Multihoming
99
• One upstream and local peering
You
ISP-A
Internet
Transit
Local PeerPeering
100 v1.0100
Multihoming
100
• More than one upstream ISP and local peering
You
ISP-BISP-A
Internet
Transit
Local PeerPeering
101 v1.0101
Multihoming
101
• More than one upstream ISP with local and public peering
You
ISP-BISP-A
Internet
Transit
Local PeerPeering
IXP
Peering
102 v1.0102
Recap: Path control Attributes
• Inbound Traffic:o AS-PATH, MED, Community (sub-aggregates/more specifics)
• Outbound Traffic:o Local Preference
103 v1.0103
Two Upstream – One backup
• We want:o Both incoming and outgoing traffic via AS20 (R1)o AS30 (R2) path to be used only if the link to AS20 fails
• AS-PATH to control incoming traffico Prepend outbound on R2
• LOCAL-PREF for outgoing traffico Higher LP for inbound routes on R1 AS 17821
AS 30AS 20
Internet
Primary BackupR1 R2
104 v1.0104
Two Upstream – One backup
• Always announce the aggregate on both links!
• R1 (main link) config:
router bgp 17821network 61.45.248.0 mask 255.255.248.0neighbor 20.20.20.1 remote-as 20neighbor 20.20.20.1 prefix-list AGGR outneighbor 20.20.20.1 prefix-list DEF in
!ip prefix-list AGGR permit 61.45.248.0/21ip prefix-list DEF permit 0.0.0.0/0!ip route 61.45.248.0 255.255.248.0 null0
Prefix-list applied to outbound routes
Prefix-list applied to inbound routes
Advertise aggregate
Define the prefix-lists
Aggregate should exist in the routing table
(pull-up route)
105 v1.0105
Two Upstream – One backup
• R2 (backup) config:
router bgp 17821network 61.45.248.0 mask 255.255.248.0neighbor 30.30.30.1 remote-as 30neighbor 30.30.30.1 prefix-list AGGR outneighbor 30.30.30.1 route-map BACKUP-OUT outneighbor 30.30.30.1 prefix-list DEF inneighbor 30.30.30.1 route-map BACKUP-IN in!ip prefix-list AGGR permit 61.45.248.0/21ip prefix-list DEF permit 0.0.0.0/0!ip route 61.45.248.0 255.255.248.0 null0!route-map BACKUP-OUT permit 10set as-path prepend 17821 17821 17821!route-map BACKUP-IN permit 10set local-preference 80
Route-map applied to outbound routes
Advertise aggregate in BGP
Define the prefix-lists
BACKUP-OUT prepends the AS-PATH for all outbound
BGP updates
Route-map applied to inbound routes
BACKUP-IN sets lower local pref for all inbound BGP
updates
106 v1.0106
Two Upstream – Load Sharing (Inbound Traffic)
• Load share incoming and outgoing traffic on the two links
• Announce one-half of the sub-aggregate on first, and the other half on the second linko Always announce the aggregate on both!
• Requires good address planningo Customers need to be assigned from both
address blocksAS 17821
AS 30AS 20
Internet
Load Share
R1 R2
107 v1.0107
Two Upstream – Load Sharing (Inbound Traffic)
• R1 config:
router bgp 17821network 61.45.248.0 mask 255.255.248.0network 61.45.248.0 mask 255.255.252.0neighbor 20.20.20.1 remote-as 20neighbor 20.20.20.1 prefix-list SUB-A outneighbor 20.20.20.1 prefix-list DEF in!ip prefix-list SUB-A permit 61.45.248.0/21ip prefix-list SUB-A permit 61.45.248.0/22ip prefix-list DEF permit 0.0.0.0/0!ip route 61.45.248.0 255.255.248.0 null0ip route 61.45.248.0 255.255.252.0 null0
Advertise sub-aggregate along with
the aggregate
Advertise both aggregate and first sub-prefix in BGP
Sub-aggregate should exist in the routing
table (pull-up route)
108 v1.0108
Two Upstream – Load Sharing (Inbound Traffic)
• R2 config:
router bgp 17821network 61.45.248.0 mask 255.255.248.0network 61.45.252.0 mask 255.255.252.0neighbor 30.30.30.1 remote-as 30neighbor 30.30.30.1 prefix-list SUB-B outneighbor 30.30.30.1 prefix-list DEF in!ip prefix-list SUB-B permit 61.45.248.0/21ip prefix-list SUB-B permit 61.45.252.0/22ip prefix-list DEF permit 0.0.0.0/0!ip route 61.45.248.0 255.255.248.0 null0ip route 61.45.252.0 255.255.252.0 null0
Advertise sub-aggregate along with
the aggregate
Advertise both aggregate and second
sub-prefix in BGP
Sub-aggregate should exist in the routing
table (pull-up route)
109 v1.0109
Load Sharing – Outbound (Full)
• What about outbound traffic load balancing?
• Case I: Full Internet routes (more memory/CPU)o Accept full route from both (AS20 and AS30)
o For routes from AS30 (R2)▸ Higher local pref prefixes originated by AS30 and its immediate neighbors (one AS hop away) –
traffic goes via AS30
▸ Lower local pref all other routes (lower than 100) – traffic to these goes via AS20
o For routes learned from AS20 (R1)▸ default local pref
110 v1.0110
Load Sharing – Outbound (Full)
AS 17821
AS 30
AS 20
Internet
Rest of the Internet
R1 R2
AS X
111 v1.0111
Load Sharing – Outbound (Full)
• R1 configuration: nothing doing
router bgp 17821neighbor 20.20.20.1 remote-as 20neighbor 20.20.20.1 prefix-list ALL in
!ip prefix-list ALL permit 0.0.0.0/0 le 32!
Accept full internet feed(default local-pref)
112 v1.0112
Load Sharing – Outbound (Full)
• R2 config:
router bgp 17821neighbor 30.30.30.1 remote-as 30neighbor 30.30.30.1 prefix-list ALL inneighbor 30.30.30.1 route-map TWO-HOPS in!!ip prefix-list ALL permit 0.0.0.0/0 le 32!ip as-path access-list 30 permit ^(30_)+$ip as-path access-list 30 permit ^(30_)+_[0-9]+$!route-map TWO-HOPS permit 10match as-path 30set local-preference 150route-map TWO-HOPS permit 20set local-preference 50
Accept full internet feed
Accept routes local to and received from AS30
(AS-path prepend included)
Received from AS30 but AS-PATH length of two
(its neighbor ASes) High-pref AS30 and its neighbor AS originated
routesLow-pref everything else
113 v1.0113
Load Sharing – Outbound (Partial)
• Partial Routes – less HW resources!
• Case II: Partial Internet routeso Accept default from AS20o Default and full from AS30 (well-connected than AS20)o filter to only accept prefixes originated by AS30 and its neighbor ASes (AS-Path ACLs)
▸ Higher pref those routes
▸ Low pref the default route▸ so that traffic to these goes via AS20
o Traffic to rest of Internet via AS 20
114 v1.0114
Load Sharing – Outbound (Partial)
AS 17821
AS 30
AS 20
Internet
Rest of the Internet
R1 R2
AS X
115 v1.0115
Load Sharing – Outbound (Partial)
• R1 configuration:
router bgp 17821neighbor 20.20.20.1 remote-as 20neighbor 20.20.20.1 prefix-list DEF in!ip prefix-list DEF permit 0.0.0.0/0!
116 v1.0116
Load Sharing – Outbound (Partial)
• R2 config:
router bgp 17821neighbor 30.30.30.1 remote-as 30neighbor 30.30.30.1 filter-list 30 inneighbor 30.30.30.1 prefix-list ALL inneighbor 30.30.30.1 route-map DEF-LOW in!ip prefix-list DEF permit 0.0.0.0/0prefix-!!ip prefix-list ALL permit 0.0.0.0/0 le 32!ip as-path access-list 30 permit ^(30_)+$ip as-path access-list 30 permit ^(30_)+_[0-9]+$!route-map DEF-LOW permit 10match ip address prefix-list DEFset local-preference 90route-map DEF-LOW permit 20
Accept full internet feed
Filter inbound routes with AS-PATH ACL using filter-list
Purely for redundancy (if path via AS 20 fails)
Accept routes local to and received from AS30
(AS-path prepend included)
Received from AS30 but AS-PATH length of two
(its neighbor ASes)
Low-pref default route
117 v1.0117
Using Communities
• Community attribute provides greater flexibility for traffic shaping than prefix-listo Simplifies BGP configurationo Greater policy control
• Not sent by default to BGP peerso explicitly send (neighbor x.x.x.x send-community)
• Can carry policy informationo Example:
▸ ASN:80 (set local-pref 80)▸ ASN:1 (set as-path prepend ASN)▸ ASN:888 (set ip next-hop 192.0.2.1 – Cymru bogons)
118 v1.0118
Route Server Policy
0:(IXP-AS) Do not announce prefixes to all peers0:(PEER-AS) Do not announce prefixes to certain peer(IX-AS):(PEER-AS) Advertise to a certain peer(IX-AS):(IX-AS) Advertise prefixes to all peers
119 v1.0119
Setting Communities
router bgp 17821neighbor 20.20.20.1 remote-as 20neighbor 20.20.20.1 send-community!address-family ipv4 unicastnetwork 61.45.248.0 mask 255.255.248.0 route-map SET-COMM-AGGnetwork 61.45.248.0 mask 255.255.254.0 route-map SET-COMM-4Gnetwork 61.45.250.0 mask 255.255.254.0 route-map SET-COMM-BBnetwork 61.45.252.0 mask 255.255.254.0 route-map SET-COMM-ENTnetwork 61.45.254.0 mask 255.255.254.0 route-map SET-COMM-CORP
!ip route 61.45.248.0 255.255.248.0 null0ip route 61.45.248.0 255.255.254.0 null0 254ip route 61.45.250.0 255.255.254.0 null0 254ip route 61.45.252.0 255.255.254.0 null0 254ip route 61.45.254.0 255.255.254.0 null0 254!
120 v1.0120
Setting Communities
!route-map SET-COMM-AGG permit 10set community 17821:1000
!route-map SET-COMM-4G permit 10set community 17821:1101
!route-map SET-COMM-BB permit 10set community 17821:1102
!route-map SET-COMM-ENT permit 10set community 17821:1103
!route-map SET-COMM-CORP permit 10set community 17821:1104
!
121 v1.0121
Grouping Communities
• We can group communities together using community-list:
!ip community-list 20 permit 17821:1000ip community-list 21 permit 17821:1101ip community-list 22 permit 17821:1102ip community-list 23 permit 17821:1103ip community-list 24 permit 17821:1104!
122 v1.0122
Two Upstream and IXP – using Communities
AS 17821
AS 30AS 20
Internet
Transit
IXP
Peering
AS111 AS222
R1 R2
R3
123 v1.0123
Two Upstream and IXP – IX Router
• R3 (IXP) configuration:o both incoming and outgoing traffic, IXP should be the preferred path!
router bgp 17821neighbor IX-PEERS peer-groupneighbor 12.12.12.111 remote-as 111neighbor 12.12.12.111 peer-group IX-PEERSneighbor 12.12.12.222 remote-as 222neighbor 12.12.12.222 peer-group IX-PEERS
!address-family ipv4neighbor IX-PEERS send-communityneighbor IX-PEERS remove-private-asneighbor IX-PEERS route-map IX-IN inneighbor IX-PEERS route-map IX-OUT out
Add neighbors to the peer group
Define peer-groups for all IX peers
Define common policies applied to all neighbors on the peer-group- Send communities- Remove private
ASNs
Apply inbound and outbound routing policies
124 v1.0124
Two Upstream and IXP – IX Router
• R3 (IXP) configuration (contd..):
!ip community-list 20 permit 17821:1000ip community-list 21 permit 17821:1101ip community-list 22 permit 17821:1102ip community-list 23 permit 17821:1103ip community-list 24 permit 17821:1104!route-map IX-IN permit 10set local-preference 250set community 17821: add
!route-map IX-OUT permit 10match community 20 21 22 23 24set metric 10
!
Define the communities
High-pref routes received from IX peers
(outbound traffic via IX)
Send all our prefixes (aggregates and sub-aggregates)
Define a community for all routes learned via IXP
Set lower MED for all routes sent to IX peers
(inbound traffic via IX)
125 v1.0125
Two Upstream and IXP – Transit Router
• For Transit/Upstream:o Tier-1 ISPs (or ISPs who are run properly) use communities to group their regional
prefixeso Filter based on those to shape outbound traffic to Internet!
▸ Ex: receive US routes from one ISP, and Europe routes from the other
o Example:▸ NTT US – 2914:3000▸ NTT Europe – 2914:3200▸ NTT Asia – 2914:3400▸ NTT South America – 2914:3600
126 v1.0126
Two Upstream and IXP – Transit Router
• For Inbound traffic:o We can use our sub-prefixes to balance incoming traffic
o Ex: Advertise half of our routes to one, and the other half to the other ▸ keep playing until we reach symmetry!
o But remember to announce the aggregates to both (REDUNDANCY!)
127 v1.0127
Two Upstream and IXP – TR1
• R1 configuration:o Let us assume NTT (AS2914) as transit here
router bgp 17821neighbor 29.29.29.1 remote-as 2914neighbor 29.29.29.1 description eBGP with NTT
!address-family ipv4neighbor 29.29.29.1 send-communityneighbor 29.29.29.1 route-map NTT-IN inneighbor 29.29.29.1 route-map NTT-OUT out
!! We want Asia, US and SA routesip community-list 1 permit 2914:3000 !USip community-list 1 permit 2914:3400 !ASip community-list 1 permit 2914:3600 !SAip community-list 2 permit 2914:3200 !EU
- Send communities- Apply inbound and
outbound routing policies
Define communities for NTT global routes- In this example, we
will source US and Asia routes from NTT
128 v1.0128
Two Upstream and IXP – TR1
• R1 configuration (contd..):!route-map NTT-IN permit 10match community 1set local-preference 210
route-map NTT-IN permit 20match community 2set local-preference 50
route-map NTT-IN permit 40!route-map NTT-OUT permit 10match community 20match community 21match community 22
!
Route-map to influence outbound traffic- Set higher local-pref for US, Asia,
and SA routes (outbound traffic)- Still lower than IX!
Lower local-pref for EU routes (will prefer the second ISP, but available if that link fails)
Route-map to influence inbound traffic- Send our aggregate (in case ISP2
fails)- And half of our sub-prefixes
129 v1.0129
Two Upstream and IXP – TR2
• R2 configuration:o Let us assume Zayo (AS6461) as transit here
router bgp 17821neighbor 64.64.64.1 remote-as 6461neighbor 64.64.64.1 description eBGP with Zayo
!address-family ipv4neighbor 64.64.64.1 send-communityneighbor 64.64.64.1 route-map ZAYO-IN inneighbor 64.64.64.1 route-map ZAYO-OUT out
!! Zayo Europe routesip community-list 3 permit 6461:5996ip community-list 3 permit 6461:5998ip community-list 3 permit 6461:5999! Zayo Global routesip community-list 4 permit 6461:5997
- Send communities- Apply inbound and
outbound routing policies
Define communities for Zayo global routes- In this example, we will
source EU routes from Zayo
130 v1.0130
Two Upstream and IXP – TR2
• R2 configuration (contd..):!route-map ZAYO-IN permit 10match community 3set local-preference 210
route-map ZAYO-IN permit 20match community 4set local-preference 50
route-map ZAYO-IN permit 40!route-map ZAYO-OUT permit 10match community 20match community 23match community 24
!
Route-map to influence outbound traffic- Set higher local-pref for EU
routes (outbound traffic)- Still lower than IX!
Lower local-pref for global routes (NTT is preferred, but will work if that link fails)
Route-map to influence inbound traffic- Send our aggregate (in case
ISP1 fails), and- other second-half of our sub-
prefixes
131 v1.0131
Acknowledgements
• Philip Smith
• Cisco Systems
132132 v1.0
Thank You!END OF SESSIONThank You!
END OF SESSION
133
