Packet Forwarding in Transit Autonomous Systems

Packet Forwarding in Transit Autonomous Systems

Packet Forwarding in an Autonomous System

• All core routers must receive external routes - they must run BGPThis is the only scalable design, route redistribution into

IGP is not scalable

AS 42

AS 12 AS 14

R-14

R-12

Rtr-A Rtr-B

Rtr-DRtr-C

Router on a transit path needs to know all external destinations for

proper packet forwarding

Packet Forwarding for External Destinations

• Routes learned via BGP don’t have outgoing interface associated with them in the routing table

• Recursive lookup is performed to forward IP packets toward external destinations

wg3pe2#show ip route 128.51.0.0Routing entry for 128.51.0.0/16 Known via "bgp 5", distance 200, metric 0 Tag 99, type internal Last update from 192.168.5.1 01:21:25 ago Routing Descriptor Blocks: * 192.168.5.1, from 192.168.5.1, 01:21:25 ago Route metric is 0, traffic share count is 1 AS Hops 5wg3pe2#show ip route 192.168.5.1Routing entry for 192.168.5.1/32 Known via "ospf 1", distance 110, metric 1563, type intra area Redistributing via ospf 1, rip Advertised by rip metric 3 Last update from 192.168.5.17 on Serial1/0.121, 02:09:15 ago Routing Descriptor Blocks: * 192.168.5.17, from 192.168.5.1, 02:09:15 ago, via Serial1/0.121 Route metric is 1563, traffic share count is 1

wg3pe2#show ip route 128.51.0.0Routing entry for 128.51.0.0/16 Known via "bgp 5", distance 200, metric 0 Tag 99, type internal Last update from 192.168.5.1 01:21:25 ago Routing Descriptor Blocks: * 192.168.5.1, from 192.168.5.1, 01:21:25 ago Route metric is 0, traffic share count is 1 AS Hops 5wg3pe2#show ip route 192.168.5.1Routing entry for 192.168.5.1/32 Known via "ospf 1", distance 110, metric 1563, type intra area Redistributing via ospf 1, rip Advertised by rip metric 3 Last update from 192.168.5.17 on Serial1/0.121, 02:09:15 ago Routing Descriptor Blocks: * 192.168.5.17, from 192.168.5.1, 02:09:15 ago, via Serial1/0.121 Route metric is 1563, traffic share count is 1

BGP next-hop

No outgoing interface

Route toward BGP next-hop

Outgoing interface

Recursive Lookup in IOS

BGP tableAddress Prefix AS-Path Communities Other attr.Next hop10.0.0.0 /8 42 13 37:121.2.3.4

... ... ... ... ......

IP routingtable

Next-hop Outgoing interfaceAddressProtocolBGP

1.5.4.1 Ethernet 01.2.3.0OSPF--- Ethernet 01.5.4.0conn.

Prefix

/24/24

1.2.3.4 ---10.0.0.0 /8

Address Prefix

... ...

Switchingcache

IP address

...ARP cache

MAC address

...

L2 header

...10.0.0.0 /8 MAC header

Entries in routing table are built from BGP tableOutgoing interface is never associated with a BGP route

Recursive lookup is performed to forward the packet toward external destination

1.5.4.1 0c.00.11.22.33.44 ARP cache lookup is performedto build layer-2 header

Lookup result is stored in the switching cache

Recursive Lookup in IOS

• Traditional IOS switching mechanisms perform recursive lookup when forwarding the first packet– Fast switching, Optimum switching

• Cisco Express Forwarding (CEF) pre-computes the forwarding table– All recursive lookups are performed while the

forwarding table is built

Routing Protocols in a Transit Autonomous System

• With IBGP running on all core routers, is IGP still needed in the core?

• IGP is needed to resolve BGP next hops and perform fast convergence after a failure in the core network

AS 42

AS 12 AS 14

R-14

R-12

Rtr-A Rtr-B

Rtr-DRtr-C

All core routers are running IBGP

Interactions Between BGP and IGP

Ideally, there would be no interaction between BGP and IGP–BGP carries external and customer routes–IGP carries only core subnets–IGP is not affected by external route flaps–BGP is not affected by failures internal to the

network as long as the BGP next-hop remains reachable

–The only link between BGP and IGP should be the recursive lookup

Interactions Between BGP and IGP

Sometimes, BGP and IGP will propagate the same route–Usually due to bad network design–In this case, routes are believed in

EBGP/IGP/IBGP order based on administrative distances of the routes

Routing protocol Default administrativedistance

EBGP 20IGP 90 - 170IBGP 200

Change the Administrative Distance of BGP Routes

• Sets administrative distance for EBGP, IBGP and local routes

• Applies only to routes received after the command has been entered (similar to filters)

Defaults:• EBGP routes have distance 20, and IBGP routes have

distance 200

Defaults are usually OK, don’t change them

distance bgp external internal

router(config-router)#

Monitoring and Troubleshooting

IBGP

Monitoring and Troubleshooting

IBGP

show ip bgp neighbor

router(config)#

• Displays whether a neighbor is an IBGP neighbor

IBGP-related IOS Show Commands

show ip bgp

router(config)#

• Uses a special marker (i) for IBGP routes

show ip bgp prefix

router(config)#

• Displays whether the prefix is an IBGP route

Show ip bgp neighbor

Router#show ip bgp neighbor 192.168.3.101BGP neighbor is 192.168.3.101, remote AS 3, internal link BGP version 4, remote router ID 192.168.3.101 BGP state = Established, up for 00:56:08 Last read 00:00:08, hold time is 180, keepalive interval is 60 seconds Neighbor capabilities: Route refresh: advertised and received Address family IPv4 Unicast: advertised and received Received 82 messages, 0 notifications, 0 in queue Sent 97 messages, 0 notifications, 0 in queue Route refresh request: received 0, sent 0 Minimum time between advertisement runs is 5 seconds

Router#show ip bgp neighbor 192.168.3.101BGP neighbor is 192.168.3.101, remote AS 3, internal link BGP version 4, remote router ID 192.168.3.101 BGP state = Established, up for 00:56:08 Last read 00:00:08, hold time is 180, keepalive interval is 60 seconds Neighbor capabilities: Route refresh: advertised and received Address family IPv4 Unicast: advertised and received Received 82 messages, 0 notifications, 0 in queue Sent 97 messages, 0 notifications, 0 in queue Route refresh request: received 0, sent 0 Minimum time between advertisement runs is 5 seconds

Show ip bgp prefix

Router#show ip bgp 197.99.1.0BGP routing table entry for 197.99.1.0/24, version 3Paths: (1 available, best #1) Advertised to non peer-group peers: 192.168.3.103 99 192.168.21.99 (metric 20) from 192.168.3.101 (192.168.3.101) Origin IGP, metric 0, localpref 100, valid, internal, best

Router#show ip bgp 197.99.1.0BGP routing table entry for 197.99.1.0/24, version 3Paths: (1 available, best #1) Advertised to non peer-group peers: 192.168.3.103 99 192.168.21.99 (metric 20) from 192.168.3.101 (192.168.3.101) Origin IGP, metric 0, localpref 100, valid, internal, best

BGP Confederations

IBGP Transit AS - Problems

IBGP requires full-mesh between all BGP-speaking routers

–large number of TCP sessions–unnecessary duplicate routing traffic

Solutions–route reflectors modify IBGP split horizon rules–BGP confederations modify IBGP AS Path

processing

AS 61 AS 62

AS 63 AS 64

BGP Confederation - Split Transit AS in smaller AS

AS 12

AS 14

Splitting the AS into smaller AS would reduce the number of IBGP sessions, but we cannot get extra AS numbers

AS 42

Confederations enable us to hide internal AS numbers and announce only one (external) AS number to the EBGP neighbors

real EBGPsession

Intra-confederationEBGP session

IBGP session

AS Path Changes within BGP Confederation

IBGP session

Intra-confederation EBGP session

EBGP session with external peer

• AS path is not changed• Intra-confederation AS

number is prepended to AS path

• Intra-confederation AS numbers are removed from AS path

• External AS number is prepended to the AS path

AS Path Changes within BGP Confederation

AS 42

AS 12

AS 14

AS 61 AS 62

AS 63 AS 64

X (61) 12

X 12 X (61) 12

X (61) 12

X (61) 12

X (62 61) 12

X (63 61) 12 X 42 12

Details of AS Path Processing

• Intra-confederation AS path is encoded as a separate segment of the AS path– Displayed in parenthesis when using IOS

show commands

• All routers within the BGP confederation have to support BGP confederations– A router not supporting BGP confederations

will reject AS path with unknown segment type

Other Properties of Intra-Confederation EBGP Session

• Behaves like EBGP session during session establishment– EBGP neighbor has to be directly connected or you

have to configure ebgp-multihop on the neighbor

• Behaves like IBGP session when propagating routing updates– Local preference, MED and next-hop attributes are

retained– The whole confederation can run one IGP, giving

optimal routing based on next-hop attribute in BGP routing table

Deploying BGP ConfederationsDeploying BGP Confederations

BGP Confederation Planning

Divide transit AS into smaller areas–Follow physical topology of the network

Define AS number for each area. –Use AS numbers reserved for private use (higher

than 64512)

Verify IOS release level–All routers have to support BGP confederations

Convert each area into autonomous system–Total rewrite of BGP configuration is required

Configuring BGP Confederation

• Start BGP process with member AS number

• Specify external AS number– Must be defined in all routers within

confederation

• List all member AS numbers in the confederation– Must be defined in all routers with an EBGP

session

Configuring BGP Confederation

no router bgp as-numberrouter bgp member-AS-number

router(config)#

• Remove old BGP process and configure BGP process with member AS number

bgp confederation identifier external-as-number

router(config-router)#

• Configure external confederation-wide AS number

bgp confederation peers list-of-intra-confederation-AS

router(config-router)#

• Define all the other autonomous systems in the confederation

Autonomous system 123

Internal AS65003

Internal AS65002

Internal AS 65001

1.0.0.11.0.0.2

1.0.0.3 1.0.0.4

2.7.1.1EBGP inAS 222

BGP Confederation Configuration Example

router bgp 65001 ! internal AS!! Confederation parameterbgp confederation identifier 123bgp confederation peers 65002 65003! ! IBGP neighborneighbor 1.0.0.3 remote-as 65001 !! EBGP with intra-confed ASneighbor 1.0.0.2 remote-as 65002neighbor 1.0.0.1 remote-as 65003!! real EBGPneighbor 2.7.1.1 remote-as 222

Monitoring BGP Confederation

show ip bgp neighbor

router#

• Displays whether a neighbor is within the confederation

show ip bgp prefix [mask]

router#

• Displays internal and external segments of the AS Path

• Displays whether the path is external, internal or intra-confederation external

Monitoring Intra-Confederation EBGP Neighbors

Wilma#show ip bgp neighbor 1.0.0.2BGP neighbor is 1.0.0.2, remote AS 65002, external link Index 2, Offset 0, Mask 0x4 BGP version 4, remote router ID 12.1.2.3 Neighbor under common administration BGP state = Established, table version = 5, up for 00:09:15 Last read 00:00:16, hold time is 180, keepalive interval is 60 seconds Minimum time between advertisement runs is 30 seconds Received 13 messages, 0 notifications, 0 in queue Sent 13 messages, 0 notifications, 0 in queue Prefix advertised 1, suppressed 0, withdrawn 0 Connections established 1; dropped 0 Last reset never 1 accepted prefixes consume 32 bytes 0 history paths consume 0 bytes External BGP neighbor may be up to 255 hops away.

Wilma#show ip bgp neighbor 1.0.0.2BGP neighbor is 1.0.0.2, remote AS 65002, external link Index 2, Offset 0, Mask 0x4 BGP version 4, remote router ID 12.1.2.3 Neighbor under common administration BGP state = Established, table version = 5, up for 00:09:15 Last read 00:00:16, hold time is 180, keepalive interval is 60 seconds Minimum time between advertisement runs is 30 seconds Received 13 messages, 0 notifications, 0 in queue Sent 13 messages, 0 notifications, 0 in queue Prefix advertised 1, suppressed 0, withdrawn 0 Connections established 1; dropped 0 Last reset never 1 accepted prefixes consume 32 bytes 0 history paths consume 0 bytes External BGP neighbor may be up to 255 hops away.

Monitoring Confederation Routes

Fred#show ip bgp 14.0.0.0BGP routing table entry for 14.0.0.0/8, version 5Paths: (2 available, best #2, advertised over IBGP, EBGP) (65001) 387 1.3.0.3 (metric 54357248) from 1.0.0.1 (11.0.0.1) Origin IGP, metric 0, localpref 60, valid, confed-internal (65001) 387 1.3.0.3 (metric 54357248) from 1.0.0.2 (10.1.1.1) Origin IGP, metric 0, localpref 60, valid, confed-external,

best

Intra-confederation part of AS-PathIntra-confederation part of AS-Path

External part of AS-PathExternal part of AS-Path

Route received from intra-confederation EBGP sessionRoute received from intra-confederation EBGP session

Route received from intra-confederation IBGP sessionRoute received from intra-confederation IBGP session

Next-hop points to real EBGP peer in both casesNext-hop points to real EBGP peer in both cases

BGP Route Reflectors

IBGP Transit AS - Problems

IBGP requires full-mesh between all BGP-speaking routers

–Large number of TCP sessions–Unnecessary duplicate routing traffic

Solutions–Route reflectors modify IBGP split horizon

rules–BGP confederations modify IBGP AS Path

processing

Route Reflectors - Modification of Split Horizon Rules

Classic BGP - IBGP routes arenot propagated to other IBGP peers. Full mesh of IBGP peers is therefore required

Route reflector can propagate IBGP routes to other IBGP peers. Full mesh of IBGP peers is nolonger required

Route reflector

EBGP route

Route Reflector Split Horizon Rules

Autonomous system

Reflector

Reflector ReflectorClient

Client

Client

ClientEBGP peer

EBGP peer

Client

EBGP peer

Routes received from external peers are propagated to all internal peersRoutes received from external peers are propagated to all internal peers

Routes received from a client are propagated to all other peers

Routes received from a client are propagated to all other peers

More Route Reflector Split Horizon Rules

Autonomous system

Reflector

Reflector ReflectorClient

Client

Client

ClientEBGP peer

EBGP peer

Client

EBGP peer

Routes received from non-client IBGP neighbors are sent to clients and EBGP peers

Routes received from non-client IBGP neighbors are sent to clients and EBGP peers

All IBGP and EBGP routes are sent to EBGP peersAll IBGP and EBGP routes are sent to EBGP peers

Route Reflectors Reduce the Number of IBGP Sessions

Autonomous system

Reflector

Reflector ReflectorClient

Client

Client

Client

EBGP peer

EBGP peer

Client

EBGP peer

But the reflectorcould be a single

point of failure

But the reflectorcould be a single

point of failure

Design requirement: reflectors must be redundant

Redundant Route Reflectors

Autonomous system

Reflector

Reflector ReflectorClient

Client

Client

Client

EBGP peer

EBGP peer

Client

EBGP peer

Redundant reflectors solve high availability requirementRedundant reflectors solve high availability requirement

But they might also cause routing loopsBut they might also cause routing loops

The concept of “clusters” is introduced to prevent IBGP routing loops with route reflectors

Route Reflector Clusters

• A group of redundant route reflectors and their clients form a cluster

• Each cluster must have a unique cluster-ID• Each time a route is reflected, the cluster-ID is

added to cluster-list BGP attribute• The route that already contains local cluster-ID

in the cluster-list is not reflected

A utonomous system

C luster C luster

Ref lec tor

Ref lec tor Ref lec torClient

Client

Client

Client

EBGP peer

EBGP peer

Client

EBGP peer

Redundant Route Reflectors in a Cluster

Route is rejected since the cluster-ID is already in cluster-list

Route is rejected since the cluster-ID is already in cluster-list

Additional Route Reflector Loop Prevention Mechanisms

• Every time a route is reflected, the router-ID of the originating IBGP router is stored in originator-ID BGP attribute

• A router receiving an IBGP route with originator-ID set to its own router-ID ignores that route

• BGP path selection procedure is modified to take in account cluster-list and originator-ID.

Network Design with BGP Route ReflectorsNetwork Design with

BGP Route Reflectors

Route Reflectors - Design

• Divide transit AS into smaller areas (called clusters)

• Each cluster contains route reflectors and route reflector clients

• Routers that don’t support route reflector functionality act as one-router cluster or as route reflector client

Route Reflectors - Sample Network

Autonomoussystem

Redundantcluster

Non-redundantcluster

Reflector

Client Client Client

EBGP peer

EBGP peer

Client

Reflector

Reflector Non-RRrouter

Client

Client

EBGP peer

Route Reflector IBGP Session Rules

• All clients in a cluster must have IBGP session with and only with all route reflectors in the cluster

• IBGP full-mesh between all route reflectors within the AS is required

• Non-route reflector capable routers can participate in IBGP full-mesh or be route reflector clients

Hierarchical Route Reflectors

Problem:• In very large networks, a single layer of

route reflectors might not be enough

Solution:• A hierarchy of route reflectors can be

established– A route reflector can be a client of another

route reflector– The hierarchy can be as deep as needed

Hierarchical Route Reflector Example

Autonomoussystem

Cluster27

Cluster 11 Cluster12

Reflector/Client

Client Client Client

EBGP peer

Client

EBGP peer

Client Client

Reflector/Client

Reflector/Client

Reflector Reflector

Client Client Client

EBGP peer

This router is a reflector in Cluster 11 and client in Cluster 27

This router is a reflector in Cluster 11 and client in Cluster 27

Deploying BGP Route ReflectorsDeploying BGP

Route Reflectors

Configuring BGP Route Reflectors

• Configure cluster-ID on route reflectors

• Configure BGP neighbors as route reflectors clients on the route reflectors

• No configuration is needed on the route reflector clients

Configuring Route Reflectors - Router Configuration Commands

bgp cluster-id cluster-id

router(config-router)#

• Optionally assigns a cluster-ID to the route reflector (default value is router-ID)

• Required only for clusters with redundant reflectors

• Cluster-ID cannot be changed after the first client is configured

neighbor ip-address route-reflector-client

router(config-router)#

• Configures an IBGP neighbor to be a client of this reflector

Route Reflector Configuration Example

Autonomoussystem 123

Cluster 175

1.0.0.1reflector

1.0.0.2reflector

1.0.0.3client

1.0.0.4client

1.2.0.6IBGP peer

2.7.1.1EBGP inAS 222

router bgp 123! cluster IDbgp cluster-id 175! RR clientsneighbor 1.0.0.3 remote-as 123neighbor 1.0.0.3 route-reflectorneighbor 1.0.0.4 remote-as 123neighbor 1.0.0.4 route-reflector! other IBGP neighborsneighbor 1.0.0.2 remote-as 123neighbor 1.2.0.6 remote-as 123! EBGP neighborsneighbor 2.7.1.1 remote-as 222

router bgp 123! cluster IDbgp cluster-id 175! RR clientsneighbor 1.0.0.3 remote-as 123neighbor 1.0.0.3 route-reflectorneighbor 1.0.0.4 remote-as 123neighbor 1.0.0.4 route-reflector! other IBGP neighborsneighbor 1.0.0.2 remote-as 123neighbor 1.2.0.6 remote-as 123! EBGP neighborsneighbor 2.7.1.1 remote-as 222

Monitoring Route Reflector Operation

show ip bgp neighbor

router#

• Displays whether a neighbor is a route reflector client

show ip bgp network [mask]

router#

• Displays additional path attributes (originator and cluster-list)

Monitoring Route Reflector Clients

Barney#show ip bgp neighbors 1.0.0.1BGP neighbor is 1.0.0.1, remote AS 213, internal link Index 1, Offset 0, Mask 0x2 Route-Reflector Client BGP version 4, remote router ID 11.0.0.1 BGP state = Established, table version = 5, up for 01:33:24 Last read 00:00:24, hold time is 180, keepalive interval is 60 seconds Minimum time between advertisement runs is 5 seconds Received 257 messages, 0 notifications, 0 in queue Sent 264 messages, 0 notifications, 0 in queue Connections established 5; dropped 4 Last reset 01:33:33, due to : User reset request No. of prefix received 1

Barney#show ip bgp neighbors 1.0.0.1BGP neighbor is 1.0.0.1, remote AS 213, internal link Index 1, Offset 0, Mask 0x2 Route-Reflector Client BGP version 4, remote router ID 11.0.0.1 BGP state = Established, table version = 5, up for 01:33:24 Last read 00:00:24, hold time is 180, keepalive interval is 60 seconds Minimum time between advertisement runs is 5 seconds Received 257 messages, 0 notifications, 0 in queue Sent 264 messages, 0 notifications, 0 in queue Connections established 5; dropped 4 Last reset 01:33:33, due to : User reset request No. of prefix received 1

Monitoring Reflected BGP Routes

Routes received from the client as seen on the reflector

Reflected routes as seen on the client

Barney#show ip bgp 11.0.0.0BGP routing table entry for 11.0.0.0/8, version 3Paths: (1 available, best #1, advertised over IBGP) Local, (Received from a RR-client) 1.0.0.1 (metric 40640000) from 1.0.0.1 (11.0.0.1) Origin IGP, metric 0, localpref 100, valid, internal, best

Barney#show ip bgp 11.0.0.0BGP routing table entry for 11.0.0.0/8, version 3Paths: (1 available, best #1, advertised over IBGP) Local, (Received from a RR-client) 1.0.0.1 (metric 40640000) from 1.0.0.1 (11.0.0.1) Origin IGP, metric 0, localpref 100, valid, internal, best

Wilma#sh ip bgp 14.0.0.0BGP routing table entry for 14.0.0.0/8, version 30Paths: (1 available, best #1) Not advertised to any peer Local 1.0.0.3 (metric 41152000) from 1.0.0.2 (14.1.2.3) Origin IGP, metric 0, localpref 100, valid, internal, best Originator: 14.1.2.3, Cluster list: 0.0.2.55

Wilma#sh ip bgp 14.0.0.0BGP routing table entry for 14.0.0.0/8, version 30Paths: (1 available, best #1) Not advertised to any peer Local 1.0.0.3 (metric 41152000) from 1.0.0.2 (14.1.2.3) Origin IGP, metric 0, localpref 100, valid, internal, best Originator: 14.1.2.3, Cluster list: 0.0.2.55