Packet Forwarding in Transit Autonomous Systems
Packet Forwarding in Transit Autonomous Systems
Packet Forwarding in an Autonomous System
• All core routers must receive external routes - they must run BGPThis is the only scalable design, route redistribution into
IGP is not scalable
AS 42
AS 12 AS 14
R-14
R-12
Rtr-A Rtr-B
Rtr-DRtr-C
Router on a transit path needs to know all external destinations for
proper packet forwarding
Packet Forwarding for External Destinations
• Routes learned via BGP don’t have outgoing interface associated with them in the routing table
• Recursive lookup is performed to forward IP packets toward external destinations
wg3pe2#show ip route 128.51.0.0Routing entry for 128.51.0.0/16 Known via "bgp 5", distance 200, metric 0 Tag 99, type internal Last update from 192.168.5.1 01:21:25 ago Routing Descriptor Blocks: * 192.168.5.1, from 192.168.5.1, 01:21:25 ago Route metric is 0, traffic share count is 1 AS Hops 5wg3pe2#show ip route 192.168.5.1Routing entry for 192.168.5.1/32 Known via "ospf 1", distance 110, metric 1563, type intra area Redistributing via ospf 1, rip Advertised by rip metric 3 Last update from 192.168.5.17 on Serial1/0.121, 02:09:15 ago Routing Descriptor Blocks: * 192.168.5.17, from 192.168.5.1, 02:09:15 ago, via Serial1/0.121 Route metric is 1563, traffic share count is 1
wg3pe2#show ip route 128.51.0.0Routing entry for 128.51.0.0/16 Known via "bgp 5", distance 200, metric 0 Tag 99, type internal Last update from 192.168.5.1 01:21:25 ago Routing Descriptor Blocks: * 192.168.5.1, from 192.168.5.1, 01:21:25 ago Route metric is 0, traffic share count is 1 AS Hops 5wg3pe2#show ip route 192.168.5.1Routing entry for 192.168.5.1/32 Known via "ospf 1", distance 110, metric 1563, type intra area Redistributing via ospf 1, rip Advertised by rip metric 3 Last update from 192.168.5.17 on Serial1/0.121, 02:09:15 ago Routing Descriptor Blocks: * 192.168.5.17, from 192.168.5.1, 02:09:15 ago, via Serial1/0.121 Route metric is 1563, traffic share count is 1
BGP next-hop
No outgoing interface
Route toward BGP next-hop
Outgoing interface
Recursive Lookup in IOS
BGP tableAddress Prefix AS-Path Communities Other attr.Next hop10.0.0.0 /8 42 13 37:121.2.3.4
... ... ... ... ......
IP routingtable
Next-hop Outgoing interfaceAddressProtocolBGP
1.5.4.1 Ethernet 01.2.3.0OSPF--- Ethernet 01.5.4.0conn.
Prefix
/24/24
1.2.3.4 ---10.0.0.0 /8
Address Prefix
... ...
Switchingcache
IP address
...ARP cache
MAC address
...
L2 header
...10.0.0.0 /8 MAC header
Entries in routing table are built from BGP tableOutgoing interface is never associated with a BGP route
Recursive lookup is performed to forward the packet toward external destination
1.5.4.1 0c.00.11.22.33.44 ARP cache lookup is performedto build layer-2 header
Lookup result is stored in the switching cache
Recursive Lookup in IOS
• Traditional IOS switching mechanisms perform recursive lookup when forwarding the first packet– Fast switching, Optimum switching
• Cisco Express Forwarding (CEF) pre-computes the forwarding table– All recursive lookups are performed while the
forwarding table is built
Routing Protocols in a Transit Autonomous System
• With IBGP running on all core routers, is IGP still needed in the core?
• IGP is needed to resolve BGP next hops and perform fast convergence after a failure in the core network
AS 42
AS 12 AS 14
R-14
R-12
Rtr-A Rtr-B
Rtr-DRtr-C
All core routers are running IBGP
Interactions Between BGP and IGP
Ideally, there would be no interaction between BGP and IGP–BGP carries external and customer routes–IGP carries only core subnets–IGP is not affected by external route flaps–BGP is not affected by failures internal to the
network as long as the BGP next-hop remains reachable
–The only link between BGP and IGP should be the recursive lookup
Interactions Between BGP and IGP
Sometimes, BGP and IGP will propagate the same route–Usually due to bad network design–In this case, routes are believed in
EBGP/IGP/IBGP order based on administrative distances of the routes
Routing protocol Default administrativedistance
EBGP 20IGP 90 - 170IBGP 200
Change the Administrative Distance of BGP Routes
• Sets administrative distance for EBGP, IBGP and local routes
• Applies only to routes received after the command has been entered (similar to filters)
Defaults:• EBGP routes have distance 20, and IBGP routes have
distance 200
Defaults are usually OK, don’t change them
distance bgp external internal
router(config-router)#
Monitoring and Troubleshooting
IBGP
Monitoring and Troubleshooting
IBGP
show ip bgp neighbor
router(config)#
• Displays whether a neighbor is an IBGP neighbor
IBGP-related IOS Show Commands
show ip bgp
router(config)#
• Uses a special marker (i) for IBGP routes
show ip bgp prefix
router(config)#
• Displays whether the prefix is an IBGP route
Show ip bgp neighbor
Router#show ip bgp neighbor 192.168.3.101BGP neighbor is 192.168.3.101, remote AS 3, internal link BGP version 4, remote router ID 192.168.3.101 BGP state = Established, up for 00:56:08 Last read 00:00:08, hold time is 180, keepalive interval is 60 seconds Neighbor capabilities: Route refresh: advertised and received Address family IPv4 Unicast: advertised and received Received 82 messages, 0 notifications, 0 in queue Sent 97 messages, 0 notifications, 0 in queue Route refresh request: received 0, sent 0 Minimum time between advertisement runs is 5 seconds
Router#show ip bgp neighbor 192.168.3.101BGP neighbor is 192.168.3.101, remote AS 3, internal link BGP version 4, remote router ID 192.168.3.101 BGP state = Established, up for 00:56:08 Last read 00:00:08, hold time is 180, keepalive interval is 60 seconds Neighbor capabilities: Route refresh: advertised and received Address family IPv4 Unicast: advertised and received Received 82 messages, 0 notifications, 0 in queue Sent 97 messages, 0 notifications, 0 in queue Route refresh request: received 0, sent 0 Minimum time between advertisement runs is 5 seconds
Show ip bgp prefix
Router#show ip bgp 197.99.1.0BGP routing table entry for 197.99.1.0/24, version 3Paths: (1 available, best #1) Advertised to non peer-group peers: 192.168.3.103 99 192.168.21.99 (metric 20) from 192.168.3.101 (192.168.3.101) Origin IGP, metric 0, localpref 100, valid, internal, best
Router#show ip bgp 197.99.1.0BGP routing table entry for 197.99.1.0/24, version 3Paths: (1 available, best #1) Advertised to non peer-group peers: 192.168.3.103 99 192.168.21.99 (metric 20) from 192.168.3.101 (192.168.3.101) Origin IGP, metric 0, localpref 100, valid, internal, best
BGP Confederations
IBGP Transit AS - Problems
IBGP requires full-mesh between all BGP-speaking routers
–large number of TCP sessions–unnecessary duplicate routing traffic
Solutions–route reflectors modify IBGP split horizon rules–BGP confederations modify IBGP AS Path
processing
AS 61 AS 62
AS 63 AS 64
BGP Confederation - Split Transit AS in smaller AS
AS 12
AS 14
Splitting the AS into smaller AS would reduce the number of IBGP sessions, but we cannot get extra AS numbers
AS 42
Confederations enable us to hide internal AS numbers and announce only one (external) AS number to the EBGP neighbors
real EBGPsession
Intra-confederationEBGP session
IBGP session
AS Path Changes within BGP Confederation
IBGP session
Intra-confederation EBGP session
EBGP session with external peer
• AS path is not changed• Intra-confederation AS
number is prepended to AS path
• Intra-confederation AS numbers are removed from AS path
• External AS number is prepended to the AS path
AS Path Changes within BGP Confederation
AS 42
AS 12
AS 14
AS 61 AS 62
AS 63 AS 64
X (61) 12
X 12 X (61) 12
X (61) 12
X (61) 12
X (62 61) 12
X (63 61) 12 X 42 12
Details of AS Path Processing
• Intra-confederation AS path is encoded as a separate segment of the AS path– Displayed in parenthesis when using IOS
show commands
• All routers within the BGP confederation have to support BGP confederations– A router not supporting BGP confederations
will reject AS path with unknown segment type
Other Properties of Intra-Confederation EBGP Session
• Behaves like EBGP session during session establishment– EBGP neighbor has to be directly connected or you
have to configure ebgp-multihop on the neighbor
• Behaves like IBGP session when propagating routing updates– Local preference, MED and next-hop attributes are
retained– The whole confederation can run one IGP, giving
optimal routing based on next-hop attribute in BGP routing table
Deploying BGP ConfederationsDeploying BGP Confederations
BGP Confederation Planning
Divide transit AS into smaller areas–Follow physical topology of the network
Define AS number for each area. –Use AS numbers reserved for private use (higher
than 64512)
Verify IOS release level–All routers have to support BGP confederations
Convert each area into autonomous system–Total rewrite of BGP configuration is required
Configuring BGP Confederation
• Start BGP process with member AS number
• Specify external AS number– Must be defined in all routers within
confederation
• List all member AS numbers in the confederation– Must be defined in all routers with an EBGP
session
Configuring BGP Confederation
no router bgp as-numberrouter bgp member-AS-number
router(config)#
• Remove old BGP process and configure BGP process with member AS number
bgp confederation identifier external-as-number
router(config-router)#
• Configure external confederation-wide AS number
bgp confederation peers list-of-intra-confederation-AS
router(config-router)#
• Define all the other autonomous systems in the confederation
Autonomous system 123
Internal AS65003
Internal AS65002
Internal AS 65001
1.0.0.11.0.0.2
1.0.0.3 1.0.0.4
2.7.1.1EBGP inAS 222
BGP Confederation Configuration Example
router bgp 65001 ! internal AS!! Confederation parameterbgp confederation identifier 123bgp confederation peers 65002 65003! ! IBGP neighborneighbor 1.0.0.3 remote-as 65001 !! EBGP with intra-confed ASneighbor 1.0.0.2 remote-as 65002neighbor 1.0.0.1 remote-as 65003!! real EBGPneighbor 2.7.1.1 remote-as 222
Monitoring BGP Confederation
show ip bgp neighbor
router#
• Displays whether a neighbor is within the confederation
show ip bgp prefix [mask]
router#
• Displays internal and external segments of the AS Path
• Displays whether the path is external, internal or intra-confederation external
Monitoring Intra-Confederation EBGP Neighbors
Wilma#show ip bgp neighbor 1.0.0.2BGP neighbor is 1.0.0.2, remote AS 65002, external link Index 2, Offset 0, Mask 0x4 BGP version 4, remote router ID 12.1.2.3 Neighbor under common administration BGP state = Established, table version = 5, up for 00:09:15 Last read 00:00:16, hold time is 180, keepalive interval is 60 seconds Minimum time between advertisement runs is 30 seconds Received 13 messages, 0 notifications, 0 in queue Sent 13 messages, 0 notifications, 0 in queue Prefix advertised 1, suppressed 0, withdrawn 0 Connections established 1; dropped 0 Last reset never 1 accepted prefixes consume 32 bytes 0 history paths consume 0 bytes External BGP neighbor may be up to 255 hops away.
Wilma#show ip bgp neighbor 1.0.0.2BGP neighbor is 1.0.0.2, remote AS 65002, external link Index 2, Offset 0, Mask 0x4 BGP version 4, remote router ID 12.1.2.3 Neighbor under common administration BGP state = Established, table version = 5, up for 00:09:15 Last read 00:00:16, hold time is 180, keepalive interval is 60 seconds Minimum time between advertisement runs is 30 seconds Received 13 messages, 0 notifications, 0 in queue Sent 13 messages, 0 notifications, 0 in queue Prefix advertised 1, suppressed 0, withdrawn 0 Connections established 1; dropped 0 Last reset never 1 accepted prefixes consume 32 bytes 0 history paths consume 0 bytes External BGP neighbor may be up to 255 hops away.
Monitoring Confederation Routes
Fred#show ip bgp 14.0.0.0BGP routing table entry for 14.0.0.0/8, version 5Paths: (2 available, best #2, advertised over IBGP, EBGP) (65001) 387 1.3.0.3 (metric 54357248) from 1.0.0.1 (11.0.0.1) Origin IGP, metric 0, localpref 60, valid, confed-internal (65001) 387 1.3.0.3 (metric 54357248) from 1.0.0.2 (10.1.1.1) Origin IGP, metric 0, localpref 60, valid, confed-external,
best
Intra-confederation part of AS-PathIntra-confederation part of AS-Path
External part of AS-PathExternal part of AS-Path
Route received from intra-confederation EBGP sessionRoute received from intra-confederation EBGP session
Route received from intra-confederation IBGP sessionRoute received from intra-confederation IBGP session
Next-hop points to real EBGP peer in both casesNext-hop points to real EBGP peer in both cases
BGP Route Reflectors
IBGP Transit AS - Problems
IBGP requires full-mesh between all BGP-speaking routers
–Large number of TCP sessions–Unnecessary duplicate routing traffic
Solutions–Route reflectors modify IBGP split horizon
rules–BGP confederations modify IBGP AS Path
processing
Route Reflectors - Modification of Split Horizon Rules
Classic BGP - IBGP routes arenot propagated to other IBGP peers. Full mesh of IBGP peers is therefore required
Route reflector can propagate IBGP routes to other IBGP peers. Full mesh of IBGP peers is nolonger required
Route reflector
EBGP route
Route Reflector Split Horizon Rules
Autonomous system
Reflector
Reflector ReflectorClient
Client
Client
ClientEBGP peer
EBGP peer
Client
EBGP peer
Routes received from external peers are propagated to all internal peersRoutes received from external peers are propagated to all internal peers
Routes received from a client are propagated to all other peers
Routes received from a client are propagated to all other peers
More Route Reflector Split Horizon Rules
Autonomous system
Reflector
Reflector ReflectorClient
Client
Client
ClientEBGP peer
EBGP peer
Client
EBGP peer
Routes received from non-client IBGP neighbors are sent to clients and EBGP peers
Routes received from non-client IBGP neighbors are sent to clients and EBGP peers
All IBGP and EBGP routes are sent to EBGP peersAll IBGP and EBGP routes are sent to EBGP peers
Route Reflectors Reduce the Number of IBGP Sessions
Autonomous system
Reflector
Reflector ReflectorClient
Client
Client
Client
EBGP peer
EBGP peer
Client
EBGP peer
But the reflectorcould be a single
point of failure
But the reflectorcould be a single
point of failure
Design requirement: reflectors must be redundant
Redundant Route Reflectors
Autonomous system
Reflector
Reflector ReflectorClient
Client
Client
Client
EBGP peer
EBGP peer
Client
EBGP peer
Redundant reflectors solve high availability requirementRedundant reflectors solve high availability requirement
But they might also cause routing loopsBut they might also cause routing loops
The concept of “clusters” is introduced to prevent IBGP routing loops with route reflectors
Route Reflector Clusters
• A group of redundant route reflectors and their clients form a cluster
• Each cluster must have a unique cluster-ID• Each time a route is reflected, the cluster-ID is
added to cluster-list BGP attribute• The route that already contains local cluster-ID
in the cluster-list is not reflected
A utonomous system
C luster C luster
Ref lec tor
Ref lec tor Ref lec torClient
Client
Client
Client
EBGP peer
EBGP peer
Client
EBGP peer
Redundant Route Reflectors in a Cluster
Route is rejected since the cluster-ID is already in cluster-list
Route is rejected since the cluster-ID is already in cluster-list
Additional Route Reflector Loop Prevention Mechanisms
• Every time a route is reflected, the router-ID of the originating IBGP router is stored in originator-ID BGP attribute
• A router receiving an IBGP route with originator-ID set to its own router-ID ignores that route
• BGP path selection procedure is modified to take in account cluster-list and originator-ID.
Network Design with BGP Route ReflectorsNetwork Design with
BGP Route Reflectors
Route Reflectors - Design
• Divide transit AS into smaller areas (called clusters)
• Each cluster contains route reflectors and route reflector clients
• Routers that don’t support route reflector functionality act as one-router cluster or as route reflector client
Route Reflectors - Sample Network
Autonomoussystem
Redundantcluster
Non-redundantcluster
Reflector
Client Client Client
EBGP peer
EBGP peer
Client
Reflector
Reflector Non-RRrouter
Client
Client
EBGP peer
Route Reflector IBGP Session Rules
• All clients in a cluster must have IBGP session with and only with all route reflectors in the cluster
• IBGP full-mesh between all route reflectors within the AS is required
• Non-route reflector capable routers can participate in IBGP full-mesh or be route reflector clients
Hierarchical Route Reflectors
Problem:• In very large networks, a single layer of
route reflectors might not be enough
Solution:• A hierarchy of route reflectors can be
established– A route reflector can be a client of another
route reflector– The hierarchy can be as deep as needed
Hierarchical Route Reflector Example
Autonomoussystem
Cluster27
Cluster 11 Cluster12
Reflector/Client
Client Client Client
EBGP peer
Client
EBGP peer
Client Client
Reflector/Client
Reflector/Client
Reflector Reflector
Client Client Client
EBGP peer
This router is a reflector in Cluster 11 and client in Cluster 27
This router is a reflector in Cluster 11 and client in Cluster 27
Deploying BGP Route ReflectorsDeploying BGP
Route Reflectors
Configuring BGP Route Reflectors
• Configure cluster-ID on route reflectors
• Configure BGP neighbors as route reflectors clients on the route reflectors
• No configuration is needed on the route reflector clients
Configuring Route Reflectors - Router Configuration Commands
bgp cluster-id cluster-id
router(config-router)#
• Optionally assigns a cluster-ID to the route reflector (default value is router-ID)
• Required only for clusters with redundant reflectors
• Cluster-ID cannot be changed after the first client is configured
neighbor ip-address route-reflector-client
router(config-router)#
• Configures an IBGP neighbor to be a client of this reflector
Route Reflector Configuration Example
Autonomoussystem 123
Cluster 175
1.0.0.1reflector
1.0.0.2reflector
1.0.0.3client
1.0.0.4client
1.2.0.6IBGP peer
2.7.1.1EBGP inAS 222
router bgp 123! cluster IDbgp cluster-id 175! RR clientsneighbor 1.0.0.3 remote-as 123neighbor 1.0.0.3 route-reflectorneighbor 1.0.0.4 remote-as 123neighbor 1.0.0.4 route-reflector! other IBGP neighborsneighbor 1.0.0.2 remote-as 123neighbor 1.2.0.6 remote-as 123! EBGP neighborsneighbor 2.7.1.1 remote-as 222
router bgp 123! cluster IDbgp cluster-id 175! RR clientsneighbor 1.0.0.3 remote-as 123neighbor 1.0.0.3 route-reflectorneighbor 1.0.0.4 remote-as 123neighbor 1.0.0.4 route-reflector! other IBGP neighborsneighbor 1.0.0.2 remote-as 123neighbor 1.2.0.6 remote-as 123! EBGP neighborsneighbor 2.7.1.1 remote-as 222
Monitoring Route Reflector Operation
show ip bgp neighbor
router#
• Displays whether a neighbor is a route reflector client
show ip bgp network [mask]
router#
• Displays additional path attributes (originator and cluster-list)
Monitoring Route Reflector Clients
Barney#show ip bgp neighbors 1.0.0.1BGP neighbor is 1.0.0.1, remote AS 213, internal link Index 1, Offset 0, Mask 0x2 Route-Reflector Client BGP version 4, remote router ID 11.0.0.1 BGP state = Established, table version = 5, up for 01:33:24 Last read 00:00:24, hold time is 180, keepalive interval is 60 seconds Minimum time between advertisement runs is 5 seconds Received 257 messages, 0 notifications, 0 in queue Sent 264 messages, 0 notifications, 0 in queue Connections established 5; dropped 4 Last reset 01:33:33, due to : User reset request No. of prefix received 1
Barney#show ip bgp neighbors 1.0.0.1BGP neighbor is 1.0.0.1, remote AS 213, internal link Index 1, Offset 0, Mask 0x2 Route-Reflector Client BGP version 4, remote router ID 11.0.0.1 BGP state = Established, table version = 5, up for 01:33:24 Last read 00:00:24, hold time is 180, keepalive interval is 60 seconds Minimum time between advertisement runs is 5 seconds Received 257 messages, 0 notifications, 0 in queue Sent 264 messages, 0 notifications, 0 in queue Connections established 5; dropped 4 Last reset 01:33:33, due to : User reset request No. of prefix received 1
Monitoring Reflected BGP Routes
Routes received from the client as seen on the reflector
Reflected routes as seen on the client
Barney#show ip bgp 11.0.0.0BGP routing table entry for 11.0.0.0/8, version 3Paths: (1 available, best #1, advertised over IBGP) Local, (Received from a RR-client) 1.0.0.1 (metric 40640000) from 1.0.0.1 (11.0.0.1) Origin IGP, metric 0, localpref 100, valid, internal, best
Barney#show ip bgp 11.0.0.0BGP routing table entry for 11.0.0.0/8, version 3Paths: (1 available, best #1, advertised over IBGP) Local, (Received from a RR-client) 1.0.0.1 (metric 40640000) from 1.0.0.1 (11.0.0.1) Origin IGP, metric 0, localpref 100, valid, internal, best
Wilma#sh ip bgp 14.0.0.0BGP routing table entry for 14.0.0.0/8, version 30Paths: (1 available, best #1) Not advertised to any peer Local 1.0.0.3 (metric 41152000) from 1.0.0.2 (14.1.2.3) Origin IGP, metric 0, localpref 100, valid, internal, best Originator: 14.1.2.3, Cluster list: 0.0.2.55
Wilma#sh ip bgp 14.0.0.0BGP routing table entry for 14.0.0.0/8, version 30Paths: (1 available, best #1) Not advertised to any peer Local 1.0.0.3 (metric 41152000) from 1.0.0.2 (14.1.2.3) Origin IGP, metric 0, localpref 100, valid, internal, best Originator: 14.1.2.3, Cluster list: 0.0.2.55