Troubleshooting - MPLS - Cisco Live

#CLUS

#CLUS

Vinit Jain – CCIE# 22854@vinugenieBrad Edgeworth – CCIE# 31574@bradedgeworthTECMPL-3201

Troubleshooting MPLS – On All Cisco Platforms

© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS

• Troubleshooting LDP Issues

• BGP, LDP, RSVP

• Troubleshooting MPLS LSP

• OAM, Multipath Trace

• Troubleshooting MPLS L3 VPNs

• Troubleshooting PE-CE Interaction (RD, RT, VPN Services)

• Interactions with Traffic Engineering

• Segment Routing

• Migration

• On Demand Next-Hop (ODN)

Agenda

TECMPL-3201 3

Questions? Use Cisco Webex Teams to chat with the speaker after the session

Find this session in the Cisco Live Mobile App

Click “Join the Discussion”

Install Webex Teams or go directly to the team space

Enter messages/questions in the team space

How

Webex Teams will be moderated by the speaker until June 16, 2019.

1

2

3

4


Cisco Webex Teams

cs.co/ciscolivebot#TECMPL-3201

4

MPLS Trivia Question


Fun with MPLS Trivia

R4R3R2

R1, R2, R3, R4 and R5 all have OSPF and MPLS enabled.

What changes can be made on R2 and/or R3 to prevent only R1’s Loopback (192.168.1.1) from pinging R5’s Loopback (192.168.5.5)?

We will explain some of the concepts that make this work.

Lo0: 192.168.1.1 Lo0: 192.168.5.5

R1 R5

R1#ping 192.168.5.5 so 192.168.1.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.168.5.5, timeout is 2 seconds:

Packet sent with a source address of 192.168.1.1

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 3/4/5 ms

TECMPL-3201 6

Configuring and Basic LDP Operations


Troubleshooting LDP Issues

IOS / IOS XE

MPLS LDP Configuration

IOS XR

mpls label protocol ldp

!

interface Gig 0/0

mpls ip

mpls label protocol ldp

exit

!

mpls ldp router-id

loopback0 force

mpls ldp

router-id x.x.x.x

interface gi 0/0/0/0

interface gi 0/0/0/1

install feature-set mpls

feature-set mpls

feature mpls

mpls ldp configuration

router-id x.x.x.x

!

interface ethernet 2/1

mpls ip

NX-OS

TECMPL-3201 8


Establishing Adjacency & Swapping Labels

• First the IGP (OSPF / IS-IS) is established and routes are exchanged between all routers

Populating the RIB

R1 R4R3R2

InLabel

OutLabel Network Out Int

N/A N/A 10.1.0.0/16 Gi0/0

N/A N/A 10.1.1.0/24 Gi0/0

N/A N/A 10.12.1.0/24 Gi0/0

N/A N/A 10.23.1.0/24 Gi0/1

N/A N/A 10.34.1.0/24 Gi0/1

N/A N/A 10.4.0.0/16 Gi0/1

N/A N/A 10.4.4.0/24 Gi0/1

InLabel


N/A N/A 10.1.0.0/16 Gi0/0

N/A N/A 10.1.1.0/24 Gi0/0

N/A N/A 10.12.1.0/24 Gi0/0

N/A N/A 10.23.1.0/24 Gi0/0

N/A N/A 10.34.1.0/24 Gi0/0

N/A N/A 10.4.0.0/16 Gi0/1

N/A N/A 10.4.4.0/24 Gi0/1

InLabel


N/A N/A 10.1.0.0/16 Gi0/2

N/A N/A 10.1.1.0/24 Gi0/1

N/A N/A 10.12.1.0/24 Gi0/0

N/A N/A 10.23.1.0/24 Gi0/0

N/A N/A 10.34.1.0/24 Gi0/0

N/A N/A 10.4.0.0/16 Gi0/0

N/A N/A 10.4.4.0/24 Gi0/0

TECMPL-3201 9



• Local Labels are automatically generated for all prefixes in the RIB.(MPLS Label 3 is reserved for Implicit-Null – directly connected routes)

• This includes local network prefixes

Creating the Local Labels

R1 R4R3R2

InLabel


200 N/A 10.1.0.0/16 Gi0/0

201 N/A 10.1.1.0/24 Gi0/0

3 - 10.12.1.0/24 Gi0/0

3 - 10.23.1.0/24 Gi0/1

204 N/A 10.34.1.0/24 Gi0/1

205 N/A 10.4.0.0/16 Gi0/1

206 N/A 10.4.4.0/24 Gi0/1

InLabel


300 N/A 10.1.0.0/16 Gi0/0

301 N/A 10.1.1.0/24 Gi0/0

302 N/A 10.12.1.0/24 Gi0/0

3 - 10.23.1.0/24 Gi0/0

3 - 10.34.1.0/24 Gi0/0

305 N/A 10.4.0.0/16 Gi0/1

306 N/A 10.4.4.0/24 Gi0/1

InLabel


3 - 10.1.0.0/16 Gi0/2

3 - 10.1.1.0/24 Gi0/1

3 - 10.12.1.0/24 Gi0/0

103 N/A 10.23.1.0/24 Gi0/0

104 N/A 10.34.1.0/24 Gi0/0

105 N/A 10.4.0.0/16 Gi0/0

106 N/A 10.4.4.0/24 Gi0/0

TECMPL-3201 10



R1 R4R3R2

InLabel


200 POP 10.1.0.0/16 Gi0/0

201 POP 10.1.1.0/24 Gi0/0

3 - 10.12.1.0/24 Gi0/0

3 - 10.23.1.0/24 Gi0/1

204 POP 10.34.1.0/24 Gi0/1

205 305 10.4.0.0/16 Gi0/1

206 306 10.4.4.0/24 Gi0/1

InLabel


300 200 10.1.0.0/16 Gi0/0

301 201 10.1.1.0/24 Gi0/0

302 POP 10.12.1.0/24 Gi0/0

3 - 10.23.1.0/24 Gi0/0

3 - 10.34.1.0/24 Gi0/0

305 405 10.4.0.0/16 Gi0/1

306 406 10.4.4.0/24 Gi0/1

InLabel


3 - 10.1.0.0/16 Gi0/2

3 - 10.1.1.0/24 Gi0/1

3 - 10.12.1.0/24 Gi0/0

103 POP 10.23.1.0/24 Gi0/0

104 204 10.34.1.0/24 Gi0/0

105 205 10.4.0.0/16 Gi0/0

106 206 10.4.4.0/24 Gi0/0

• Local Labels are exchanged with downstream routers

• Labels are all exchanged at the same time.(This animation was done to show you the correlation of tables)

TECMPL-3201 11




LDP neighborship is formed on TCP port 646

Discovery Mechanism:

Basic Discovery – Multicast UDP hellos for directly connected neighbors

Extended Discovery – Targeted Unicast UDP hellos for non-directly connected neighbors

• Parameters

• Session Keepalive = 60 sec. & Hold time = 180 Sec.

• Discover Hello interval = 5 sec. and Hold Time = 15 sec.

• Can be viewed using the command show mpls ldp parameters

LDP Neighborship

TECMPL-3201 13



LDP Router-ID must have a specific routing entry in the RIB

Authentication parameters must match

Multiple L3 links between LDP devices

Adjacency Requirements

TECMPL-3201 14


Troubleshooting LDP IssuesLDP Neighborship Negotiation

TECMPL-3201 15


Troubleshooting LDP IssuesVerifying LDP Neighborship

PE1#sh mpls ldp neighbor

Peer LDP Ident: 10.13.1.101:0; Local LDP Ident 10.13.1.61:0

TCP connection: 10.13.1.101.11031 - 10.13.1.61.646

State: Oper; Msgs sent/rcvd: 58/60; Downstream

Up time: 00:39:27

LDP discovery sources:

Ethernet0/0, Src IP addr: 10.13.1.5

Ethernet1/0, Src IP addr: 10.13.1.9

Addresses bound to peer LDP Ident:

10.13.1.9 10.13.1.5 10.13.2.5 10.13.1.101

PE1#show tcp brief| i 646

43ABB020 10.13.1.101.11031 10.13.1.61.646 ESTAB

PE1#

TECMPL-3201 16



• Ensure reachability between the LDP router ID’s

• Verify no ACL in path blocking TCP port 646 and other Multicast traffic for LDP Hello’s.

Reachability and ACL verification

PE1#ping 192.168.11.11 source lo0


Sending 5, 100-byte ICMP Echos to 192.168.11.11, timeout is 2 seconds:

Packet sent with a source address of 192.168.1.1

.....

Success rate is 0 percent (0/5)

PE1#telnet 192.168.11.11 646 /source-interface lo0

Trying 192.168.11.11, 646 ...

% Destination unreachable; gateway or host down

Check Routing Configuration

Verify ACLs in the path or on the routers itself

TECMPL-3201 17



• If router-id is not set manually, router checks all operational interfaces on the router(including loopbacks) and chooses the highest IP address as the LDP router-id.

• LDP_ID should be hardcoded via

• “mpls ldp router-ID <interface>”

• The above configuration will not help unless:

• <interface> is UP when LDP gets started

• Existing LDP_ID (usually an interface) is shut

• Following avoids both shortcomings

• “mpls ldp router-ID <interface> force”

LDP Router-id

TECMPL-3201 18


Troubleshooting LDP issuesVerifying LDP Connection

“show mpls ldp discovery [detail]”

• Must show xmit/recv on LDP enabled interface

PE1#show mpls ldp discovery

Local LDP Identifier:

192.168.1.1:0

Discovery Sources:

Interfaces:

GigabitEthernet0/1 (ldp): xmit/recv

LDP Id: 192.168.11.11:0

Local LDP_ID

Discovered Neighbors’ LDP_ID

Xmited and Recvd Hellos on that interface

TECMPL-3201 19


Troubleshooting LDP issuesProblem with xmit / recv



192.168.1.1:0

Discovery Sources:

Interfaces:

GigabitEthernet0/1 (ldp): xmit

R1#debug mpls ldp transport connections

07:00:06.106: ldp: Scan listening TCBs



PE1 P1

P1#show mpls ldp discovery


192.168.11.11:0

Discovery Sources:

Interfaces:

GigabitEthernet0/1 (tdp): xmit

Lo0=192.168.1.1 Lo0=192.168.11.11

Label Protocol is TDP

PE1(config-if)#mpls label protocol ldp

TECMPL-3201 20


Troubleshooting LDP issues

Problem: Default route towards the peering router

LDP No Route Problem



192.168.1.1:0

Discovery Sources:

Interfaces:

Gi0/1 (ldp): xmit/recv

LDP Id: 192.168.11.11:0; no route

PE1 P1

P1#show mpls ldp discovery


192.168.11.11:0

Discovery Sources:

Interfaces:

Gi0/1 (ldp): xmit/recv

LDP Id: 192.168.1.1:0

Lo0=192.168.1.1 Lo0=192.168.11.11

PE1#show ip route 192.168.11.11

% Network not in table

TECMPL-3201 21


Troubleshooting LDP issuesProblem due to Summarization

PE1 P1

PE1#show mpls ldp neighbor 192.168.11.11



192.168.1.1:0


LDP Id: 192.168.11.11:0


Routing entry for 192.168.11.11/32

Known via "ospf 100", distance 110, metric 2, type

intra area

Last update from 10.1.111.11 on Gi0/1, 00:04:34 ago

Routing Descriptor Blocks:

* 10.1.111.11, from 192.168.11.11, 00:04:34 ago,

via GigabitEthernet0/1

Route metric is 2, traffic share count is 1

PE2#sh mpls ldp neighbor 192.168.1.1



192.168.11.11:0


LDP Id: 192.168.1.1:0


Routing entry for 192.168.1.0/24

Known via "bgp 100", distance 200, metric 0

Tag 1, type internal

Last update from 192.168.1.12 20:10:38 ago

Routing Descriptor Blocks:

* 192.168.1.12, from 192.168.12.12, 20:10:38

ago

Route metric is 0, traffic share count is 1

AS Hops 5

TECMPL-3201 22



RP/0/0/CPU0:PE2#show mpls ldp trace peer last 20

0/0/CPU0 t1 [PEER]:506: VRF(0x60000000): Peer(192.168.11.11:0): Peer FSM: Stepped, pp=0x102d9548, event=0, state 0 -> 1

0/0/CPU0 t1 [PEER]:581: VRF(0x60000000): Peer(192.168.11.11:0): DOWN - reason 'TCP connection closed'

0/0/CPU0 t1 [PEER]:3262: VRF(0x60000000): Release Peer(192.168.11.11:0): rsn 'TCP connection closed' ('Success')

0/0/CPU0 t1 [PEER]:3625: Peer(192.168.11.11:0): Unsupported/Unknown TLV (type 0x506, U/F 1/0) rcvd in INIT msg

0/0/CPU0 t1 [PEER]:506: VRF(0x60000000): Peer(192.168.11.11:0): Peer FSM: Stepped, pp=0x102d9520, event=0, state 0 -> 1

0/0/CPU0 t1 [PEER]:575: VRF(0x60000000): Peer(192.168.11.11:0): DOWN - reason 'Received Notification message from peer' (more_info 'KeepAlive Timer Expired')

0/0/CPU0 t1 [PEER]:3262: VRF(0x60000000): Release Peer(192.168.11.11:0): rsn 'Received Notification message from peer' ('KeepAlive Timer Expired')

0/0/CPU0 t1 [PEER]:3625: Peer(192.168.11.11:0): Unsupported/Unknown TLV (type 0x506, U/F 1/0) rcvd in INIT msg

MPLS LDP Trace on IOS XR

Also good to check “show mpls ldp trace discovery”

TECMPL-3201 23



• When a link comes up, LDP and IGP compete to converge; Labeled traffic drops if IGP wins.

• When LDP session on a link drops, IGP may continue forwarding labeled traffic to that link and cause traffic dropped.

LDP & IGP Sync

TECMPL-3201 24



• Link up:

• If LDP peer is reachable (alternate route exists), defer IGP adjacency on the link.

• If LDP peer is not reachable (no alternate route), IGP advertise max-metric to reach neighbor through the link.

• LDP session down:

• IGP advertises max-metric to reach neighbor through the link.

LDP & IGP Sync – Solution

TECMPL-3201 25


Troubleshooting LDP IssuesLDP & IGP Sync

• LDP IGP Sync feature is enabled under IGP (OSPF/ISIS)• - “sync-igp-shortcuts” for TE tunnel interfaces, “sync” for all other types.

router (config-isis-if-af) # mpls ldp sync [ level <1-2> ]

router (config-ospf) # mpls ldp sync + (config-ospf-ar), (config-ospf-ar-if)

router (config-ospf) # mpls ldp sync-igp-shortcuts + (config-ospf-ar)

TECMPL-3201 26


Troubleshooting LDP IssuesLDP & IGP Sync

router (config-ldp) # igp sync delay on-session-up <sec>

router (config-ldp) # igp sync delay on-proc-restart <sec>

LDP IGP Sync delays are configured under LDP

TECMPL-3201 27



• Problem:

I. When a link flaps (for a short time),

II. LDP hello adjacency over the link flaps

III. LDP session is torn down then re-setup

IV. LDP re-exchanges label bindings when LDP session is setup (i.e. LDP re-convergence).

• Solution:

• When LDP session supported by link hello is setup, create a targeted hello to protect the session.

• When link is down, the targeted hello remains through other path and keeps the LDP session up.

• When link restores, re-discover neighbors, re-program forwarding.

LDP Session Protection

TECMPL-3201 28


Troubleshooting LDP IssuesLDP Session Protection

router (config-ldp) # log session-protection

router (config-ldp) # session protection [ for <peer-acl> ] [ duration { <sec> | infinite } ]

TECMPL-3201 29



• 3 routing processes between R1 and R2

• Lo0 defined as the LDP router-id on both routers

• LDP adjacency is formed just across one link, down on other two

Case Study - 1

IP RAN10.12.2.0/24

CORE10.12.3.0/24

R1 R2

192.168.1.1 192.168.2.2

10.12.1.0/24

TECMPL-3201 30


Troubleshooting MPLS LSP

RTR#show mpls ldp bindings detail

tib entry: 10.1.1.0/30, rev 10

local binding: tag: imp-null

Advertised to:

10.1.2.2:0 10.1.2.6:0 10.1.2.4:0

remote binding: tsr: 10.1.2.2:0, tag: imp-null

remote binding: tsr: 10.1.2.6:0, tag: 12304


Looking at the LIB

TECMPL-3201 31



• The LFIB stores local and remote labels for prefixes that are used to forward packets

• Prefixes that are used = prefixes in routing table (RIB)

• Labels are derived from LIB

Label Forwarding Information Base (LFIB)

RIBLIB LFIBprefix + next-hop

prefix, next-hop and in-

label, out-label

get in- and out-label for

(prefix, next-hop)

LDP TDP

(prefix, LDP Ident,

label)(prefix,next-hop,

in-label, out-label)

(prefix, next-hop)

TECMPL-3201 32



• Verify the TCP connection – You will find the clue

• Router-ID is configured with Lo0 (forced)

• If one of the interfaces is configured with mpls ldp discovery

transport-address interface, then this behavior can be noticed.

Case Study - 1

TECMPL-3201 33

Troubleshooting LSP Issues



• Broken LDP adjacency

• MPLS not enabled

• Mismatch labels

• Software/hardware corruption

Reasons for LSP to Break

PE1

192.168.1.1/32

PE2

192.168.2.2/32

CE1

Lo0=172.16.1.1/32

CE2

Lo0=172.16.2.2/32

P1

192.168.11.11/32

MP-IBGP – VPNv4

10.1.111.0/24 10.1.211.0/24 172.16.22.0/24172.16.11.0/24

LDP + IGP

TECMPL-3201 35



• LIB stores local and remote bindings

• Local Binding:

• Prefix in own routing table + local label

• One binding

• Remote Binding:

• Prefix + remote label received from LDP neighbor

• Holds LDP router-id

• One binding per LDP neighbor

• LIB stores all labels from all LDP (BGP) neighbors, even the ones that are not used for packet forwarding (now)

Label Information Base (LIB)

TECMPL-3201 36



RTR#show mpls ldp bindings detail

tib entry: 10.1.1.0/30, rev 10

local binding: tag: imp-null

Advertised to:

10.1.2.2:0 10.1.2.6:0 10.1.2.4:0

remote binding: tsr: 10.1.2.2:0, tag: imp-null



Looking at the LIB

TECMPL-3201 37



• The LFIB stores local and remote labels for prefixes that are used to forward packets

• Prefixes that are used = prefixes in routing table (RIB)

• Labels are derived from LIB

Label Forwarding Information Base (LFIB)

RIBLIB LFIBprefix + next-hop

prefix, next-hop and in-

label, out-label

get in- and out-label for

(prefix, next-hop)

LDP TDP

(prefix, LDP Ident,

label)(prefix,next-hop,

in-label, out-label)

(prefix, next-hop)

TECMPL-3201 38


Troubleshooting MPLS LSPBuilding the LFIB

P1#show ip route 3.3.3.4Routing entry for 3.3.3.4/32* 10.1.2.1, from 10.1.2.1, 13:28:32 ago, via Ethernet0/0

P1#show mpls ldp neighbor 10.1.2.1Peer LDP Ident: 3.3.3.3:0; Local LDP Ident 2.2.2.2:0

P1#show mpls ldp binding 3.3.3.4 255.255.255.255

lib entry: 3.3.3.4/32, rev 18

remote binding: lsr: 3.3.3.3:0, label: imp-null

P1#show mpls forwarding 3.3.3.4Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 20 Pop Label 3.3.3.4/32 0 Et0/0 10.1.2.1

TECMPL-3201 39



• Defined in RFC 4379

• LSP Ping and Traceroute provide ability to monitor MPLS Label Switched Paths and quickly isolate MPLS forwarding problems.

• Two messages

• MPLS Echo Request: MPLS labeled IPv4 or IPv6 UDP packet

• MPLS Echo Reply IPv4 or IPv6 UDP packet

• Ping mode: Connectivity check of an LSP

• Test if a particular “FEC” ends at the correct egress LSR

• Traceroute mode: Hop by Hop fault localization

• Packet follows data path

MPLS OAM

TECMPL-3201 40



• ping mpls ?

ipv4 Target specified as an IPv4 address

pseudowire Target VC specified as an IPv4 address and VC ID

traffic-eng Target specified as TE tunnel interface

• traceroute mpls ?

ipv4 Target specified as an IPv4 address

multipath LSP Multipath Traceroute

pseudowire Target VC specified as an IPv4 address and VC ID

traffic-eng Target specified as TE tunnel interface

FEC Types Supported

TECMPL-3201 41



• Simple and efficient mechanism to detect data plane failures in MPLS LSPs

• Verify data plane against the control plane

• Sending “echo request” and receiving “echo reply”

• Verify that packets belonging to a FEC exit the LSP on the correct egress LSR

• Modelled after the well known IP ping and traceroute

• Ping verifies connectivity, traceroute verifies path

• LSP Ping/trace leave the LSR with the correct label stack for the LSP to be tested

LSP Ping (ping mpls . . . )

TECMPL-3201 42


Troubleshooting MPLS LSPPacket Format

Version Number Must Be Zero

Message Type Reply Mode Return Code Return Subcode

Sender’s Handle

Sequence Number

Timestamp Sent (seconds)

Timestamp Sent (microseconds)

Timestamp Received (seconds)

Timestamp Received (microseconds)

TLV …

TECMPL-3201 43



• Version number: 1

• Message Type

• MPLS Echo Request

• MPLS Echo Reply

• Reply Mode

1 Do not reply

2 Reply via an IPv4/IPv6 UDP packet

3 Reply via an IPv4/IPv6 UDP packet with Router Alert

4 Reply via application level control channel

• Timestamp

• Time-of-day in seconds and microseconds

Packet Format

TECMPL-3201 44



• Reply Mode – Do Not Reply

• This mode is useful for a keepalive application running at the remote end

• Such an application would trigger state changes if it does not receive a LSP ping packet within a predefined time

• An MPLS echo request with “do not reply” may also be used by the receiving router to log gaps in the sequence numbers and/or maintain delay/jitter statistics

Reply Modes

TECMPL-3201 45



• Reply Mode – Reply via an IPv4 UDP Packet

• The Reply via UDP packet implies that an IP V4 UDP packet should be sent in reply to an MPLS echo request

• This will be the most common reply mode for simple LSP pings sent to periodically poll the integrity of an LSP

• This is the default reply mode

Reply Modes

TECMPL-3201 46



• Reply Mode – Reply via an IPv4 UDP Packet with Router Alert

• In this mode when the destination router replies it appends a label of “1” to the packet

• This forces all the intermediate routers, on the way back, to process switch the reply

• This mode is CPU intensive and should generally be used if the reply fails for “reply with IPv4 UDP packet”

• This mode is useful when we have inconsistency between IP and MPLS

Reply Modes

TECMPL-3201 47


Troubleshooting MPLS LSPReturn Codes

Value Meaning

0 The Error Code Is Contained in the Error Code TLV

1 Malformed Echo Request Received

2 One Or More of the TLVs Was Not Understood

3 Replying Router Is an Egress for the FEC

4 Replying Router Has No Mapping for the FEC

5 Replying Router Is Not One of the “Downstream Routers”

6Replying Router Is one of the “Downstream Routers”, and Its Mapping for this FEC on the Received Interface Is the Given Label

TECMPL-3201 48


Troubleshooting MPLS LSPMPLS Echo Request

R1#ping mpls ipv4 192.168.2.2/32 verbose

destination 127.0.0.2 repeat 1 exp 7 pad 0xFFFF

Sending 1, 100-byte MPLS Echos to 10.200.254.4/32,

timeout is 2 seconds, send interval is 0 msec:

Codes: '!' - success, 'Q' - request not transmitted,

'.' - timeout, 'U' - unreachable,

'R' - downstream router but not target


! Reply address 10.1.211.2, return code 3

TECMPL-3201 49



• We use the same label stack as used by the LSP and this makes the echo to be switched inband of LSP

• The IP header destination address field of the echo request is a 127/8address

• An Echo reply, which may or may not be labelled, has the egress interface IP address as the source; destination IP address/port are copied from the echo-request’s source address/port

• Presence of the 127/8 address in the IP header destination address field causes the packet to be consumed by any routers trying to forward the packet using the ip header

• In this case P1 would not forward the echo-req to PE1 but rather consumes the packet and sends a reply to PE2 accordingly

MPLS Ping (Operational Theory)

TECMPL-3201 50


Troubleshooting MPLS LSPMPLS Ping Packet Capture

TECMPL-3201 51


Operation

• For LSP ping we generate an MPLS echo request

• The payload includes the LDP/RSVP/L2 Circuit sub-TLV depending on the LSP we use

• Echo request is appropriately labelled and sent out• Ping mode: MPLS TTL = 255• Traceroute mode: TTL = 1, 2 ,3 etc.

• MPLS Echo Request always has FEC Stack TLV

• The LSP ping sender sets the return code to 0.

• The replying router would set it accordingly based on the table shown previously

MPLS OAM Caveats

TECMPL-3201 52



• Only the TTL field in the label at the top of the stack counts

• The outgoing TTL value is only a function of the incoming TTL value

• Outgoing TTL is one less than incoming TTL

• If outgoing TTL = 0, packet is not forwarded (not even stripped and forwarded as an IP packet)

• When an IP packet is first labelled, the TTL field is copied from the IP header to the MPLS header (after being decremented by 1)

• When the label stack is removed, the outgoing TTL value is copied to the TTL field in the IP header

• Unless MPLS TTL > IP TTL

TTL Field in Labels

TECMPL-3201 53



• Receiving LSR checks that label stack of received packet matches with the received FECs in FEC Stack

• MPLS Echo Reply is sent in response to MPLS Echo Request– Destination IP address is source IP address of Echo Request– IP TTL = 255– Reply Mode: (You do not control if return packet is sent over IP or MPLS)

• IPv4

• IPv4 with Router Alert (IP Option)

– If over MPLS, then Router Alert Label as topmost label is added in the label stack

– Hardware forwarding bypassed; packet is sent to RP process level forwarding

Operation

TECMPL-3201 54


Traceroute in MPLS Network

In Label Prefix Output Interface

Out Label

- 172.16.2.2/32 Y 19 24008

16 172.16.1.1/32 X -


Out Label

22 192.168.1.1/32 X pop

19 192.168.2.2/32 Y pop


Out Label

24008 172.16.2.2/32 Y -

- 172.16.1.1/32 X 22 16

PE1 P1 PE2

CE1 CE2

Y

X

Y

X

192.168.1.1/32 192.168.2.2/32

172.16.1.1/32 172.16.2.2/32

TECMPL-3201 55


Troubleshooting MPLS LSPTraceroute in MPLS Network

PE1 P1 PE2CE1 CE2

192.168.1.1/32 192.168.2.2/32 172.16.2.2/32

172.16.2.2 TTL=2

UDP port 35678

172.16.2.2 TTL=1

UDP port 35678

172.16.2.2 TTL=255, ICMPTTL Exceeded

172.16.1.1 TTL=254ICMP TTL Exceeded

Label 24008

Label 19, TTL=1

Label 24008, TTL=255



172.16.1.1/32

Label 16

Label 22, TTL=254

Label 16, TTL=253

Aggregate Outgoing Label, IP Lookup

done in CEF for VRF

TECMPL-3201 56



• The ICMP messages “TTL exceeded” are forwarded along the LSP until the end of the LSP. So, the router does not lookup the source ip address in the global routing table to return the ICMP message.

• Reason : P routers do not have knowledge of VPN prefixes : all traceroutes initiated from within a VPN would fail

• ICMP messages are forwarded with EXP bits = 6

MPLS Trace

TECMPL-3201 57



• This command prohibits the copying of the TTL from the IP header to the MPLS shim header and vice versa (TTL is set to 255)

• It should be configured on the routers that do the label imposement(LSR edge routers), which is the PE routers.

• Providers like to use it so that the customers see the MPLS network as one hop when tracerouting

MPLS Trace Hiding

no mpls ip propagate-ttl forwarded

TECMPL-3201 58


Troubleshooting MPLS LSPMPLS Trace Hiding

CE1#traceroute 172.16.2.2 source 172.16.1.1


Tracing the route to 172.16.2.2

1 172.16.11.2 [AS 100] 3 msec 3 msec 3 msec

2 10.1.111.11 [MPLS: Labels 19/24008 Exp 0] 122 msec 25 msec 19 msec

3 10.1.211.2 [MPLS: Label 24008 Exp 0] 21 msec 16 msec 23 msec

4 172.16.12.1 [AS 100] 23 msec * 22 msec

remote PE

Plocal PE

remote CE

(mpls ip propagate-ttl forwarded)

CE1#traceroute 172.16.2.2 source 172.16.1.1



VRF info: (vrf in name/id, vrf out name/id)

1 172.16.11.2 [AS 100] 4 msec 3 msec 3 msec


3 172.16.12.1 [AS 100] 24 msec * 28 msec

remote PElocal PE

remote CE

(no mpls ip propagate-ttl forwarded)

TECMPL-3201 59


Troubleshooting MPLS LSPMPLS Trace with no mpls ip propagate-ttl on PE routers

PE1 P1 PE2CE1 CE2

172.16.2.2/32

172.16.2.2 TTL=2

UDP port 35678

172.16.2.2 TTL=1

UDP port 35678

172.16.2.2 TTL=1

UDP port 35678

172.16.1.1 TTL=254, ICMP

Port Unreachable

Label 24008

Label 19, TTL=1

Label 24008, TTL=255

172.16.1.1 TTL=254, ICMP

Port Unreachable

172.16.1.1 TTL=254, ICMP

Port Unreachable

172.16.1.1/32

Label 16

Label 22, TTL=255

Label 16, TTL=254

172.16.2.2 TTL=1

UDP port 35678

172.16.1.1 TTL=255, ICMP

Port Unreachable

udp port35678?

Aggregate Outgoing Label

TECMPL-3201 60



• MPLS LSP ping / trace is useful tool to validate the health of a label switched path

• In case of multiple paths, LSP ping may not serve useful to validate all the available paths

• Multipath MPLS trace allows users to identify all LSP failures

• The multipath LSP trace, sends probe by setting the destination to loopback address (127.x.x.x), which can help detect failure in LSP by avoiding the packet to get IP routed.

Multipath MPLS Trace

TECMPL-3201 61


R6 R1 R4

R2

R3

192.168.6.6/32 192.168.1.1/32

192.168.3.3/32

192.168.2.2/32

192.168.4.4/32

1

2

Troubleshooting MPLS LSPMultipath MPLS Trace

Echo Request

SRC – 10.1.16.6DEST – 127.0.0.0

1

Echo ReplySRC – 10.1.16.1DEST – 10.1.16.6

DS Mapping – 127.0.0.124002 - 10.1.13.3

DS Mapping – 127.0.0.030002 - 10.1.12.2

2

TECMPL-3201 62



Echo Request

SRC – 10.1.16.6DEST – 127.0.0.0

3


DS Mapping – 127.0.0.0pop - 10.1.24.4

4

R6 R1 R4

R2

R3

192.168.6.6/32 192.168.1.1/32

192.168.3.3/32

192.168.2.2/32

192.168.4.4/323

4

TECMPL-3201 63



Echo Request

SRC – 10.1.16.6DEST – 127.0.0.1

5


DS Mapping – 127.0.0.0pop - 10.1.34.4

6

R6 R1 R4

R2

R3

192.168.6.6/32 192.168.1.1/32

192.168.3.3/32

192.168.2.2/32

192.168.4.4/32

5

6

TECMPL-3201 64



PE1#traceroute mpls multipath ipv4 192.168.4.4/32

Codes: '!' - success, 'Q' - request not sent, '.' - timeout,

'L' - labeled output interface, 'B' - unlabeled output interface,

<snip>


LL!

Path 0 found,

output interface Gi0/1 nexthop 10.1.16.1

source 10.1.16.6 destination 127.0.0.1

0 10.1.16.6 10.1.16.1 MRU 1500 [Labels: 18 Exp: 0] multipaths 0

L 1 10.1.16.1 10.1.12.2 MRU 1500 [Labels: 30002 Exp: 0] ret code 8 multipaths 2

L 2 10.1.12.2 10.1.24.4 MRU 1500 [Labels: implicit-null Exp: 0] ret code 8 multipaths 1

! 3 10.1.24.4, ret code 3 multipaths 0

L!

Path 1 found,

output interface Gi0/1 nexthop 10.1.16.1

source 10.1.16.6 destination 127.0.0.0

0 10.1.16.6 10.1.16.1 MRU 1500 [Labels: 18 Exp: 0] multipaths 0

L 1 10.1.16.1 10.1.13.3 MRU 1500 [Labels: 24002 Exp: 0] ret code 8 multipaths 2

L 2 10.1.13.3 10.1.34.4 MRU 1500 [Labels: implicit-null Exp: 0] ret code 8 multipaths 1

! 3 10.1.34.4, ret code 3 multipaths 0

Paths (found/broken/unexplored) (2/0/0)

Echo Request (sent/fail) (5/0)

Echo Reply (received/timeout) (5/0)

Total Time Elapsed 192 ms

Multipath MPLS Trace

TECMPL-3201 65

Demo - Multipath MPLS Trace



With MPLS, the idea is to de-couple the forwarding from the IP header

The forwarding decision is based on the MPLS header, not the IP header

The above is true once the packet is inside the MPLS network

Forwarding is still based on the IP header at the edge where the packet first enters the MPLS network

CEF must be configured on all the routers in a MPLS network.

CEF takes care of the crucial “recursion” and “resolution” operations

MPLS Forwarding Plane

TECMPL-3201 67


Troubleshooting MPLS LSPWhat happens when CEF disabled?

PE1#show mpls forwarding-table

Local Outgoing Prefix Bytes Label Outgoing Next Hop

Label Label or Tunnel Id Switched interface

16 No Label 172.16.1.1/32 0 drop

17 No Label 192.168.12.12/32 0 drop

20 No Label 192.168.2.2/32 0 drop

21 No Label 10.1.212.0/24 0 drop

22 No Label 10.1.211.0/24 0 drop

23 No Label 192.168.11.11/32 0 drop

24 No Label 172.16.11.0/24 0 drop

25 No Label 172.16.14.0/24 0 drop

TECMPL-3201 68



• Outgoing label also conveys what treatment the packet is going to get. It could also be:

I. Pop - Pops the topmost label

II. Untagged - Untag the incoming MPLS packet

III. Aggregate - Untag and then do a FIB lookup

Label values 0-15 are reserved.

MPLS Forwarding Plane – Outgoing Labels

PE1#show mpls forwarding-table 192.168.2.2

Local Outgoing Prefix Bytes Label Outgoing NextHop


20 19 192.168.2.2/32 0 Gi0/1 10.1.111.11

PE1#

TECMPL-3201 69


Troubleshooting MPLS LSPMPLS Forwarding Plane: Outgoing Labels

PE1#sh mpls forwarding-table

Local Outgoing Prefix Bytes tag Outgoing Next Hop

tag tag or VC or Tunnel Id switched interface

16 2002 10.13.1.22/32 0 Et0/0 10.13.1.5

2002 10.13.1.22/32 0 Et1/0 10.13.1.9

18 Pop tag 10.13.1.101/32 0 Et1/0 10.13.1.9

Pop tag 10.13.1.101/32 0 Et0/0 10.13.1.5

19 Pop tag 10.13.2.4/30 0 Et1/0 10.13.1.9

Pop tag 10.13.2.4/30 0 Et0/0 10.13.1.5

20 Untagged 5.5.5.5/32[V] 0 Se2/0 point2point

21 Pop tag 10.13.21.4/30 0 Et1/0 10.13.1.9

Pop tag 10.13.21.4/30 0 Et0/0 10.13.1.5

24 Aggregate 200.1.61.4/30[V] 0

26 Untagged 30.30.30.1/32[V] 0 Se2/0 point2point

PE1#

TECMPL-3201 70



Untagged• Convert the incoming MPLS packet to an IP packet and forward it.

Pop• Pop the top label from the label stack present in an incoming MPLS

packet and forward it as an MPLS packet.• If there was only one label in the stack, then forward it as an IP packet.

SAME as imp-null label.

Aggregate• Convert the incoming MPLS packet to an IP packet and then do a FIB

lookup for it to find out the outgoing interface.

MPLS Forwarding Plane: Outgoing Labels

TECMPL-3201 71



Three cases in the MPLS forwarding:1) Label Imposition - IP to MPLS conversion

2) Label swapping - MPLS to MPLS

3) Label disposition - MPLS to IP conversion

So, depending upon the case, we need to check:1) FIB - For IP packets that get forwarded as MPLS

2) LFIB - For MPLS packets that get forwarded as MPLS

3) LFIB - For MPLS packets that get forwarded as IP

MPLS Forwarding Plane - Lookup

TECMPL-3201 72



MPLS Loadsharing (due to multiple paths to a prefix) is no different from that of IP

Hashing-algorithm is still the typical ‘FIB based’ i.e per-destloadsharing by default **

So the “show commands” are still relevant

• “Show ip cef exact-route <source> <dest>” etc.

But the <dest> must be known in the FIB table, otherwise the command won’t work.

• Won’t work on P routers for the VPN prefixes.

MPLS Forwarding Plane: Loadsharing

TECMPL-3201 73



• “mpls mtu <bytes>” can be applied to an interface to change the MPLS MTU size on the interface

• MPLS MTU size is checked by the router

• while converting an IP packet into a labeled packet or transmitting a labelled packet

• Label imposition(s) increases the packet size by 4 bytes/label, hence the outgoing packet size may exceed ‘interface MTU’ size, hence the need to tune MTU

• ‘mpls mtu <bytes>” command has no effect on “interface or IP MTU” size.

• By default, MPLS MTU = interface MTU

• MPLS MTU setting doesn’t affect MTU handling for IP-to-IP packet switching

MPLS Forwarding Plane: MTU Setting

TECMPL-3201 74



• If the label imposition makes the packet bigger than the MPLS MTU size of an outgoing interface, then:- If the DF bit set, then discard the packet and send ICMP reply back (with code=4)

- If the DF bit is not set, then fragment the IP packet (say, into 2 packets), and then impose the same label(s) on both the packets, and then transmit MPLS packets

MPLS Forwarding Plane: MTU Setting

TECMPL-3201 75



“show mpls forwarding”

• Shows all LFIB entries (vpn, non-vpn, TE etc.)

“show mpls forwarding <prefix>”

LFIB lookup based on a prefix

“show mpls forwaring label <label>”

LFIB lookup based on an incoming label

“show mpls forwarding <prefix> detail”

Shows detailed info such as L2 encap etc

MPLS Forwarding Plane: Show Commands

TECMPL-3201 76


Troubleshooting MPLS LSPMPLS Forwarding Plane: Show Commands

R2#show mpls forwarding 10.13.1.11 detail

Local Outgoing Prefix Bytes tag Outgoing Next Hop

tag tag or VC or Tunnel Id switched interface

45 51 10.13.1.11/32 0 Fa1/1/1 10.13.7.33

MAC/Encaps=14/18, MRU=1500, Tag Stack{51}

0003FD1C828100044E7548298847 00033000

No output feature configured

Per-packet load-sharing

R2#

14/18 means that the L2 header is of 14 bytes, but

L2+label header is 18 bytes (one label is 4 bytes)

TECMPL-3201 77

MPLS Labels Case Study

BGP 3107


Mobile Transport Market Conditions• High Capacity requirements from Edge to Core

• 100Mbps eNB, 1Gbps Access, 10Gbps Aggregation, 100Gbps Core

• Higher scale as LTE drives ubiquitous mobile broadband

• Tens- to hundred-of-thousands of LTE eNBs and associated CSGs

• Support for multiple and mixed topologies

• Fiber and microwave rings in access, fiber rings, hub and spoke in aggregation and core networks

• Need for graceful service integration and integration into existing infrastructure

• Need to support transport for all services from all locations

• Optimized operations with consistent packet transport

TECMPL-3201 87


Core Edge AggregationAccess

IP/MPLSCross-Domain Convergence

MPLS as Network Convergence TechnologyOptimizing Service Delivery

LS Challenges with differing Access technologies

• Complexity of achieving 50 millisecond convergence with TE-FRR

• Splitting large networks into domains while delivering services end-to-end

• Common end-to-end convergence and resiliency mechanisms

• End-to-end provisioning and troubleshooting across multiple domain

Unified MPLS addresses these challenges

with elegant simplicity and scaleTECMPL-3201 88


Seamless MPLS Overview

• An efficient MPLS transport architecture

• Virtualized to support many services on one infrastructure

• Relying on an intelligent hierarchy to scale to new challenges

• Enabling seamless operation for network and service resilience

• Separating transport from service operations with single touch point service

enablement and contiguous OAM

• Integrating alternate access technologies on same infrastructure while still

enabling Fixed and Mobile Services

TECMPL-3201 89


MPLSMPLS

MPLS

Seamless MPLS Operation Transport & Service Decoupling

Typically, a service has to be configured on every network element via operational points. The management system has to know the topology.

• Goal is to minimize the number of operational points

• Only with the integration of all MPLS islands, the minimum number of operational points is possible

Service provisioning only at the Edge

Unified MPLSAccess AGG AGG

LER LSR LER

AGG AGG Access

Operational Points

TECMPL-3201 90


Unified MPLS = Classical MPLS with a few additions

RFC

3107

BGP

filtering

LFA

R-LFA

BGP

PIC

IGP/LDP

Domainisolation

E2E

OAM

Classical MPLS

Scalability Security Simplification Multi-Service

UnifiedMPLS

Architecture

Flex

AccessL2/IGP/BGP/MPLS-

TP/LDP DoD

TECMPL-3201 91


RFC-3107

• RFC 3107 was approved May 2001, main purpose being scaling of MPLS

• RFC 3107 is BGP IPv4 with the ability to distribute labels

• BGP Filtering supported via BGP Communities in a secure manner

RFC 3107 basis:

• BGP can be used to distribute MPLS labels in the same way it can distribute a route

• The label mapping information for a particular route is piggybacked in the same BGP Update message that is used to distribute the route itself.

• If two immediately adjacent Label Switched Routers (LSRs) are also BGP peers, then label distribution can be done without the need for any other label distribution protocol.

TECMPL-3201 92


LFA & R-LFA• What is LFA FRR?

• RFC 5286 basic fast re-route mechanism with local protection in pure IP and MPLS/LDP networks

• Pre-computing available paths at source node that do not create loops

• Gives benefits of TE-FRR, but no configuration or design required

• What is Remote LFA?

Defined in draft “http://tools.ietf.org/html/draft-shand-remote-lfa”

Remote LFA uses automated IGP/LDP behavior to extend basic LFA FRR to arbitrary topologies

A node dynamically computes its remote loop free alternate node(s)– Done during SFP calculations using PQ algorithm (see draft)

Automatically establishes a directed LDP session to it– The directed LDP session is used to exchange labels for the FEC in question

On failure, the node uses label stacking to tunnel traffic to the Remote LFA node, which in turn forwards it to the destination

TECMPL-3201 93


Remote LFA FRR - Protection

• C2’s LIB

• C1’s label for FEC A1 = 20

• C3’s label for FEC C5 = 99

• C5’s label for FEC A1 = 21

• On failure, C2 sends A1-destined traffic onto an LSP destined to C5

• Swap per-prefix label 20 with 21 that is expected by C5 for that prefix, and push label 99

• When C5 receives the traffic, the top label 21 is the one that it expects for that prefix and hence it forwards it onto the destination using the shortest-path avoiding the link C1-C2.

A1

C1

C2

C3

E1

C4

A2

Backbone

Access Region

C5Directed LDP

session

21

20

99

21 99

21 X

21

TECMPL-3201 94


BGP Prefix-Independent Protection (PIC)/BGP FRR

• BGP Fast Reroute (BGP FRR)enables BGP to use alternate paths

• Algorithm uses a pointer to move all prefixes to new next hop, not a hop by hop rewrite

• ~ 100 msec protection

• Prefix-Independent

• Default behavior, entirely automated computation

• Enables 3107 BGP+labels operation to scale via hierarchy while maintaining fast convergence characteristics

• For Transport and Service convergenceTECMPL-3201 95


Unified MPLS Architecture Models• Architecture Models based on:

• Access Type: Ethernet TDM or MPLS access

• Network Size: Small/Medium (1000 nodes or less) or Large

• End to Labeled Switch Path

Deployment Model

Network Size Access Type Core/Aggregation LSP

1 Small/Medium Ethernet/TDM Flat LDP

2 Small/Medium MPLS Hierarchical Labeled BGP

3 Large Ethernet Hierarchical Labeled BGP

4 Large MPLS Hierarchical Labeled BGP for Core, Aggregation and Access

5 Large MPLS Hierarchical Labeled BGP for Core, Aggregation with redistribution in Access

TECMPL-3201 96


1 – Small Network: Ethernet/TDM Access

• Core and Aggregation Networks form one IGP and LDP domain.

• Scale recommendation is less than 1000 IGP/LDP nodes

• Packet Microwave links aggregated in Aggregation Nodes

• Mobile Access is based on TDM

• All services –Mobile and Wireline– enabled by Aggregation Nodes

Flat LDP LSP across Core and Aggregation Networks

Distribution Node

Core and Aggregation

IP/MPLS Domain

Core Node

Aggregation Node

Core Node

Core Node

Core Node

IGP/LDP domain

Aggregation Node

Aggregation Node

Aggregation Node

Aggregation Node

Pre-AggregationNode

IP/Ethernet

Fiber and Microwave3G/LTE

TDM and Packet Microwave, 2G/3G/LTE

Mobile Transport GW

Mobile Transport GW

Business

CSG

TECMPL-3201 97


2 – Small Network: MPLS Access

• The Core and Aggregation form a relatively small IGP/LDP domain (1000 nodes)

• MPLS enabled RAN, each RAN forms a different IGP/LDP domain

• The Core/Aggregation and RAN Access Networks are integrated with labelled BGP LSP

• The Access Network Nodes learn only the MPC labelled BGP prefixes and selectively and optionally the neighbouring RAN networks labelled BGP prefixes.

Hierarchical BGP LSP Across Core + Aggregation and Access Networks

Core and Aggregation

IP/MPLS domain

IGP Area

Aggregation Node

Aggregation Node

Aggregation Node

Aggregation Node

Pre-AggregationNode

RANIP/MPLS Domain

LDP LSP LDP LSP LDP LSP

iBGP Hierarchical LSP

RANIP/MPLS Domain

Pre-AggregationNode

Mobile Transport GW

Core Node

Core Node

Core Node

Core Node

Mobile Transport GW

CSG

CSG

CSG

CSG

CSG

CSG

TECMPL-3201 98


Core Network

IP/MPLS Domain

IP/Ethernet

Fiber and Microwave3G/LTE

3 – Large Network: Ethernet/TDM access

• Core and Aggregation Networks enable Unified MPLS Transport

• Core and Aggregation Networks are organized as independent IGP/LDP domains

• Core and Aggregation Networks may be in same or different Autonomous Systems

• The network domains are interconnected with hierarchical LSPs based on RFC 3107, BGP IPv4+labels

• No MPLS in Access Domain

• Aggregation Node enable Mobile and Wireline Services over Unified MPLS transport.

Hierarchical BGP LSP Across Core Network and Aggregation Networks

Pre-Aggregation Node

Aggregation Network

IP/MPLS

Domain

Aggregation Node

AggregationNode

Aggregation Network

IP/MPLS

Domain

Core Node

LDP LSP LDP LSP LDP LSP

iBGP (eBGP across ASes) Hierarchical LSP

TDM and Packet Microwave, 2G/3G/LTE

Aggregation Node

Aggregation Node

Aggregation Node

Core Node

Core Node

Core Node

Mobile Transport GW

Mobile Transport GW

CSG

CSG

TECMPL-3201 99


4 – Large Network: MPLS Access

• Core, Aggregation, Access Network enable Unified MPLS Transport• Core, Aggregation, Access are organized as independent IGP/LDP domains• Core and Aggregation Networks may be in same or different Autonomous Systems• Network domains are interconnected with hierarchical LSPs based on RFC 3107, BGP IPv4+labels. • Intra domain connectivity is based on LDP LSPs• The Access Network Nodes learn only the required labelled BGP FECs

Hierarchical BGP LSP Across Core, Aggregation and Access Networks

RANIP/MPLSdomain

Core Node

Core Node

Core Node

Core Node

LDP LSP LDP LSP LDP LSP LDP LSP LDP LSP

iBGP (eBGP across ASes) Hierarchical LSP

RANIP/MPLS domain

Core Network

IP/MPLS Domain

Pre-Aggregation Node

Aggregation Network

IP/MPLS

Domain

Aggregation Node

Pre-AggregationNode

Aggregation Network

IP/MPLS

Domain

Core Node

Aggregation Node

Aggregation Node

Aggregation Node

Core Node

Core Node

Core Node

Mobile Transport GW

Mobile Transport GW

CSG

CSG

CSGCSG

CSG

CSG

TECMPL-3201 100


5 - Large Network, MPLS Access

• Core and Aggregation are distinct IGP/LDP domains that enable inter domain hierarchical LSPs

• Core and Aggregation Networks may be in same of different Autonomous Systems

• Redistribution of Core/Aggregation LSPs into Access Networks IGP

Hierarchical BGP LSP with IGP/LDP Redistribution in Access Network

RANMPLS/IP

IGP Area/Process

RANMPLS/IP

IGP Area/Process

MPC iBGP community

into RAN IGP

RAN IGP CSN Loopbacks

into iBGP

Core

Core

Core

Core

LDP LSP LDP LSP LDP LSP LDP LSP

LDP LSP

i/eBGP Hierarchical LSP

Core Node

Core Node

Core Node

Core Node

Core Network

IP/MPLS Domain

Aggregation Network

IP/MPLS

Domain

Aggregation Node

Pre-AggregationNode

Aggregation Network

IP/MPLS

Domain

Core Node

Aggregation Node

Aggregation Node

Aggregation Node

Core Node

Core Node

Core Node

Mobile Transport GW

Mobile Transport GW

Pre-AggregationNode

MPC iBGP community

into RAN IGP

RAN IGP CSN Loopbacks

into iBGP

CSG

CSG

CSGCSG

CSG

CSG

TECMPL-3201 101


Cell Site

AccessLayer

AggregationLayer

PGW SGW

Aggregation

node

Distribution

node

Cell site

Router

Pre-AggregationLayer

Core

node

Core Network Aggregation Network

Unified MPLS ArchitectureSummary

Aggregation NodeCore ABR

Sample Routing Architecture

IGP/LDP IGP/LDP

L2

iBGP/eBGP

CoreLayer

Core ABRPre-Aggregation

NodeAccess

Network

IGP/LDP

Aggregation Node EPC GatewayAccess Node

Access NodeCentralised RR

Flexible L2 & L3 transport virtualisation to support GSM, 3G &

LTE, wholesale & retail options

New levels of Scale for MPLS transport and optimal routing

through RFC 3107 with BGP hierarchical LSPs

Simplified MPLS Transport with E2E OAM, performance

management, provisioning with seamless resiliency

TECMPL-3201 102

Demo –LDP interop with BGP 3107

Troubleshooting MPLS L3 VPNs



• CE – Customer edge router, connects to the CE network and the PE

• Forwards only IP packets – no awareness of the MPLS network is needed

• Routes between the CE internal network and the PE router

• PE – Provider Edge router, connects to P and CE routers

• Maintains separate routing table per VRF

• Uses MP-BGP to exchange VRF routing information (RD + RT)

• Performs LFIB and FIB lookups, VPN label imposition and disposition

• P – Provider core router, connects to P and PE routers

• Does not need to run BGP with the PE’s

• Performs LFIB MPLS forwarding for outer label traffic (PE to PE)

Nodes and their Roles

TECMPL-3201 105


Troubleshooting MPLS L3 VPNsIP Addressing Concerns

CE1

CE2

CE3

CE4

Customer A

Site 1

Customer A

Site 2

Customer B

Site 1

Customer B

Site 2

172.16.1.0/24

172.16.2.0/24 172.16.4.0/24

172.16.3.0/24

PE1

TECMPL-3201 106


Troubleshooting MPLS L3 VPNsIsolation Through the Use of VRFs

CE1

CE2

CE3

CE4

Customer A

Site 1

Customer A

Site 2

Customer B

Site 1

Customer B

Site 2

172.16.1.0/24

172.16.2.0/24 172.16.4.0/24

172.16.3.0/24

VRF VPN01

VRF VPN02

PE1

TECMPL-3201 107


L3VPN By PartsThe Edge:

• VRF = VPN Routing Forwarding instance

Isolated routing table, kind of like a VM

• Any routing protocol between the PE and CE

The Core:

• BGP VPNv4 and/or VPNv6 between PEs

• Labeled Switch Path between PEs

CE

MP-EBGP

PEPEP2

CE

PE-CE Protocol PE-CE Protocol

TECMPL-3201 108


MP-BGP (Multiprotocol BGP) for MPLS VPNs

• No new rules, still requires full mesh or RRs

• RRs need to support additional capabilities

• For MPLS only PEs need to speak BGP or know CE routes

• L3VPN Relies on Extended Communities

• Extended Communities are arbitrary TLVs attached to BGP prefixes

• BGP is used to Exchange the MPLS Label specific to the VPN prefix

• Outer MPLS Label is used to forward traffic between PEs

MP-BGP (Multi Protocol BGP)

TECMPL-3201 109



1. Packet is received on local PE

2. Remote VPN Label is assigned

3. Remote PE Label is assigned

Visualizing Data Flow

CE

MP-EBGP

PEPEP2

CE

100.64.6.6

100 20

TECMPL-3201 110



1. The P router next to destination PE router POPs the outer label

2. The packet is forwarded onto the Destination PE router

3. The VPN Label is examined and POP’d

4. The packet is forwarded out to the VRF

Visualizing Data Flow

CE

MP-EBGP

PEPEP2

CE

100.64.6.620100 100.64.6.620100 100.64.6.620 100.64.6.620

TECMPL-3201 111



• Address-family (AFI) “vpnv4”, “ipv4 unicast vrf” introduced

• vpnv4 AFI for PE to PE (label information)

• ipv4 unicast vrf for PE to CE

• Neighbor must be “activated” for each AFI supported

MP-BGP: Address-Families

router bgp 100neighbor 192.168.3.3 remote-as 100

!address-family vpnv4neighbor 192.168.3.3 activateneighbor 192.168.3.3 send-community

extended!address-family ipv4 unicast vrf redneighbor 192.168.4.4 remote-as 400neighbor 192.168.4.4 activate

Remote PE

Local CE

TECMPL-3201 112



BGP maintains a table for each AFI (vpnv4, ipv4, vrf…)

CE routes are placed into the vpnv4 BGP table

• BGP routes in a vrf AFI are automatically turned into vpnv4 routes

• If BGP is not PE-CE protocol routes must be redistributed into ipv4 vrfAFI

All vpnv4 routes get an assigned label

vpnv4 routes are exchanged between vpnv4 peers (PEs)

MP-BGP: Advertising CE Routes

TECMPL-3201 113



• VRFs have 3 parts:

1. VRF name (case sensitive)

2. Route Distinguisher (RD)

3. Route Target(s) (RT)

• RD and RT are for MPLS; RD must always be defined

• RD must be unique to the VRFs on the local PE

RTs and RDs: Creating the VRF

ip vrf redrd 100:100route-target import 200:200route-target export 201:201

TECMPL-3201 114



Prefix

Locally Assigned Label

RD

Route Target

vrf definition VPN01rd 200:1

route-target export 200:1

TECMPL-3201 115



• Route Distinguisher

• There is only one VPNv4 table

• How are routes distinguished from another?

• Prepending the RD to the route to creates a VPNv4 route

• Only used to make routes unique VPNv4 prefixesIPv4 Route: 192.168.1.0/24

RD: 100:100

VPNv4 Route: 100:100:192.168.10/24

Understanding RDs

Let’s Investigate This Further

TECMPL-3201 116


Troubleshooting MPLS L3 VPNsUnderstanding RDs

172.31.31.31

Route Reflector

TECMPL-3201 117



• Route Target

• RT is a BGP extended community (extra information on the update)

• “route-target export” adds the community to the outbound update

• “route-target import” defines which routes to bring into the VRF

• Multiple imports and exports allowed

Understanding the RT

ip vrf redrd 1:1route-target import 100:100route-target import 200:200

route-target export 201:201route-target export 44:313Let’s Investigate This Further

TECMPL-3201 118


Troubleshooting MPLS L3 VPNsVPN Services

TECMPL-3201 119


Troubleshooting MPLS L3 VPNsUnderstanding the RT

TECMPL-3201 120



• AS_Path is a loop prevention mechanism

• PE routers can use a special feature called AS-Override.

• Any prefixes with the same AS that the is used by the CE is changed to the AS of the PE

Fixing the BGP AS_Path Problem

XR3

router bgp 200

neighbor 172.32.36.6

remote-as 500

address-family ipv4 unicast

route-policy PASSALL in

route-policy PASSALL out

as-override

R1

router bgp 200

address-family ipv4 vrf VPN01

redistribute connected

neighbor 172.16.15.5 remote-as 500

neighbor 172.16.15.5 activate

neighbor 172.16.15.5 as-override

TECMPL-3201 121

Live Troubleshooting Demo



CE6PE1

AS500 AS500

PE3P2

Route Reflectors

CE5

RR

• CE5 cannot ping CE6

• IP Addressing is exactly the same as before.

• PE1 and PE3 now connect to a Route Reflector (192.168.10.10)

What do we do first and why?

TECMPL-3201 123

Inter-AS MPLS VPNs


Inter-AS MPLS VPNs

• Previous section – VPNs within Single-AS boundary

• Inter-AS MPLS VPN – VPNs spanning across multiple AS boundaries

• Types:

• Option 1 – Back to Back VRF

• Option 2 – Inter-Provider VPNs using ASBR-to-ASBR approach

A. Next-Hop-Self Method

B. Redistribute Connected Method

C. Multi-hop EBGP between ASBRs

• Option 3 – MP-EBGP between RR and EBGP between ASBR

Flavors

TECMPL-3201 125


Inter-AS MPLS VPNsOption 1 - Back-to-Back VRF Method

AS100 AS200

PE-ASBR1

Lo0-11.11.11.11/32

PE-ASBR2

Lo0-22.22.22.22/32

RR-P1 RR-P2

PE1 PE2

CE1 CE2

VRF- ABCVRF- XYZ

IPv4 + IGP/BGP

TECMPL-3201 126


Inter-AS MPLS VPNsOption 2a – ASBR-to-ASBR with Next-Hop-Self Method

AS100 AS200

PE-ASBR1

Lo0-11.11.11.11/32

PE-ASBR2

Lo0-22.22.22.22/32

RR-P1 RR-P2

PE1 PE2

CE1 CE2

MP-eBGP

v1172.16.1.1

172.16.1.1 172.16.2.2

neighbor x.x.x.x next-hop-self

• No LDP or IGP required on the link between the two ASBRs.

• Configure no bgp default route-target filter on ASBRsTECMPL-3201 127


Inter-AS MPLS VPNs

• Both ASBRs allocate VPN labels for prefixes received from the other AS.

• When MP-eBGP peering is configured between ASBRs, below configuration is done to complete LSP• mpls bgp forwarding – on Cisco IOS devices

• no bgp default route-target filter configured on ASBR not having VRF configured.

• Default behavior – deny vpnv4 prefixes that are not imported in any local VRF

• On XR – retain route-target all

Option 2a – ASBR-to-ASBR with Next-Hop-Self Method

TECMPL-3201 128


Inter-AS MPLS VPNsOption 2b – ASBR-to-ASBR with Redistribute Connected Method

AS100 AS200

PE-ASBR1

Lo0-11.11.11.11/32

PE-ASBR2

Lo0-22.22.22.22/32

RR-P1 RR-P2

PE1 PE2

CE1 CE2

• No LDP or IGP required on the link between the two ASBRs.

• Configure no bgp default route-target filter on ASBRs

MP-eBGP

v1172.16.1.1

172.16.1.1 172.16.2.2

TECMPL-3201 129


Inter-AS MPLS VPNs

• Redistribute the link between ASBR into IGP in local AS

• Required on both ASBR routers.


• VPN label V1 is advertised from AS100 towards ASBR-PE2 in AS200.

• Since the NH changes on ASBR-PE2, ASBR-PE2 swaps that label with V2 and advertises it towards the core.

Option 2b – ASBR-to-ASBR with Redistribute Connected Method

TECMPL-3201 130


Inter-AS MPLS VPNsOption 2c – ASBR-to-ASBR with Multi-Hop EBGP between ASBRs Method

AS100 AS200

PE-ASBR1

Lo0-11.11.11.11/32

PE-ASBR2

Lo0-22.22.22.22/32

RR-P1 RR-P2

PE1 PE2

CE1 CE2• Loopback to loopback peering between ASBRs

• Configure no bgp default route-target filter on ASBRs

MP-eBGP

v1172.16.1.1

172.16.1.1 172.16.2.2

TECMPL-3201 131


Inter-AS MPLS VPNs

• Loopback to loopback MP-EBGP peering between ASBRs.

• IGP or static route required between the ASBR link


• VPN label V1 is advertised from AS100 towards ASBR-PE2 in AS200.

• Since the NH changes on ASBR-PE2, ASBR-PE2 swaps that label with V2 and advertises it towards the core.

Option 2c – ASBR-to-ASBR with Multi-Hop EBGP between ASBRs Method

TECMPL-3201 132


Inter-AS MPLS VPNsOption 3 – Multi-Hop MP-EBGP between RR and EBGP between ASBRs

AS100 AS200

PE-ASBR1

Lo0-11.11.11.11/32

PE-ASBR2

Lo0-22.22.22.22/32

RR-P1 RR-P2

PE1 PE2

CE1 CE2

• Neighbor send-label required on eBGP peers on ASBR.

MP-eBGP

172.16.1.1 172.16.2.2

eBGP +

Send-label

TECMPL-3201 133


Inter-AS MPLS VPNs

• RR & ASBR loopbacks are advertised via EBGP on ASBR

• The remote ASBR redistributes the received loopbacks into local IGP

• MP-EBGP peering configured between RR’s on each AS

• Configure neighbor next-hop-unchanged

Option 3 – Multi-Hop MP-EBGP between RR and EBGP between ASBRs

TECMPL-3201 134

Troubleshooting IPv6 VPNs


Troubleshooting 6VPEReference Topology

PE1

PE2

CE1 RR-P PE5 CE2

IPv4 – 192.168.1.1/32

IPv6 – 2001:DB8::1/128

IPv4 – 192.168.2.2/32

IPv6 – 2001:DB8::2/128IPv4 – 192.168.5.5/32

IPv6 – 2001:DB8::5/128IPv6 – 2001:DB8::7/128IPv6 – 2001:DB8::6/128

AS 100

AS 200 AS 300IPv4 – 192.168.4.4/32

Service Provider Core

IPv4 – IGP

MPLS

TECMPL-3201 136


Troubleshooting 6VPE

• IPv6 enabled VRF’s are configured in the same way as IPv4 VRF’s

• On Cisco IOS, use command vrf definition to configure both IPv4 and IPv6 capable VRF’s

VRF Configuration

vrf definition VPN01

rd 1:1


route-target import 1:1




. . .

interface Gi0/0

vrf forwarding VPN01

ipv6 address xx:xx:xx::y/64

vrf VPN01


import route-target

1:1

2:2

export route-target

1:1


. . .

interface Gi0/0/0/0

vrf VPN01

ipv6 address xx:xx:xx::y/64

TECMPL-3201 137


6VPE Configuration – Cisco IOSrouter bgp 100

bgp router-id 192.168.1.1

bgp log-neighbor-changes

no bgp default ipv4-unicast


neighbor 192.168.4.4 update-source Loopback0

!

address-family vpnv6


neighbor 192.168.4.4 send-community extended

neighbor 192.168.4.4 next-hop-self

!

address-family ipv6 vrf red

neighbor 2001:DB8:0:16::6 remote-as 200

neighbor 2001:DB8:0:16::6 activate

exit-address-family

TECMPL-3201 138


6VPE Configuration – IOS XRrouter bgp 100


address-family vpnv6 unicast

!


remote-as 100

update-source Loopback0


next-hop-self

!

vrf red

rd 100:1


!

neighbor 2001:db8:0:26::6

remote-as 200


route-policy pass in

route-policy pass out

TECMPL-3201 139



• Since both control plane and data plane works in opposite direction, verify the IPv6 VPN prefix on PE5.

Verifying Control Plane

PE5#show ipv6 route vrf red

! Output omitted for brevity

B 2001:DB8::6/128 [200/0]

via 192.168.1.1%default, indirectly connected

B 2001:DB8::7/128 [20/0]

via FE80::7, GigabitEthernet0/2

TECMPL-3201 140



• Verify the VPNv6 prefix in BGP along with the local label


PE5#show bgp vpnv6 unicast vrf red 2001:db8::7/128

BGP routing table entry for [100:5]2001:DB8::7/128, version 38

Paths: (1 available, best #1, table red)

Advertised to update-groups:

2

Refresh Epoch 1

300

2001:DB8:0:57::7 (FE80::7) (via vrf red) from 2001:DB8:0:57::7

(192.168.7.7)

Origin IGP, metric 0, localpref 100, valid, external, best

Extended Community: RT:100:1

mpls labels in/out 23/nolabel

rx pathid: 0, tx pathid: 0x0

TECMPL-3201 141



• The remote IOS PE - PE1, receives the VPNv6 prefix as the out label of 23.


PE1#show bgp vpnv6 unicast vrf red 2001:db8::7/128

BGP routing table entry for [100:1]2001:DB8::7/128, version 7

Paths: (1 available, best #1, table red)

Advertised to update-groups:

1

Refresh Epoch 1

300, imported path from [100:5]2001:DB8::7/128 (global)

::FFFF:192.168.5.5 (metric 3) (via default) from 192.168.4.4 (192.168.4.4)

Origin IGP, metric 0, localpref 100, valid, internal, best

Extended Community: RT:100:1

Originator: 192.168.5.5, Cluster list: 192.168.4.4

mpls labels in/out nolabel/23

rx pathid: 0, tx pathid: 0x0

TECMPL-3201 142


Troubleshooting 6VPEVerifying Control PlaneRP/0/0/CPU0:PE2#show bgp vpnv6 unicast vrf red 2001:db8::7/128

BGP routing table entry for 2001:db8::7/128, Route Distinguisher: 100:1

Last Modified: Feb 4 22:46:29.408 for 1d05h

Paths: (1 available, best #1)

Not advertised to any peer

Path #1: Received by speaker 0

Not advertised to any peer

300

192.168.5.5 (metric 3) from 192.168.4.4 (192.168.5.5)

Received Label 23

Origin IGP, metric 0, localpref 100, valid, internal, best, group-best,

import-candidate, imported

Received Path ID 0, Local Path ID 1, version 5

Extended community: RT:100:1

Originator: 192.168.5.5, Cluster list: 192.168.4.4

Source VRF: default, Source Route Distinguisher: 100:5

TECMPL-3201 143


Troubleshooting 6VPEVerifying Data Plane

PE1#show ipv6 cef vrf red 2001:db8::7/128 detail

2001:DB8::7/128, epoch 0, flags [rib defined all labels]

recursive via 192.168.5.5 label 23

nexthop 10.1.14.4 GigabitEthernet0/2 label 19

PE1#show mpls forwarding-table 192.168.5.5



21 19 192.168.5.5/32 0 Gi0/2 10.1.14.4

TECMPL-3201 144


Troubleshooting 6VPEVerifying Data Plane on IOS XR

RP/0/0/CPU0:PE2#show cef vrf red ipv6 2001:db8::7/128

2001:db8::7/128, version 7, internal 0x5000001 0x0 (ptr 0xa140c5f4) [1],

0x0 (0x0), 0x208 (0xa14db230)

Updated Feb 4 22:46:29.731

Prefix Len 128, traffic index 0, precedence n/a, priority 3

via ::ffff:192.168.5.5, 3 dependencies, recursive [flags 0x6000]

path-idx 0 NHID 0x0 [0xa176b0bc 0x0]

recursion-via-/128

next hop VRF - 'default', table - 0xe0000000

next hop ::ffff:192.168.5.5 via ::ffff:192.168.5.5:0

next hop 10.1.24.4/32 Gi0/0/0/1 labels imposed {19 23}

RP/0/0/CPU0:PE2#show mpls forwarding-table prefix 192.168.5.5/32



24001 19 192.168.5.5/32 0 Gi0/0/0/1 10.1.24.4

TECMPL-3201 145


Verifying Ingress Hardware Programming – IOS XRPE2#show cef vrf red ipv6 2001:db8::7/128 hardware ingress detail loc0/0/CPU0


0x0 (0x0), 0x208 (0xa14db230)

Updated Feb 4 22:46:29.730

[1 type 1 flags 0x48089 (0xa14f5398) ext 0x0 (0x0)]

LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0]

gateway array update type-time 1 Feb 4 22:46:29.730

LDI Update time Feb 4 22:46:29.730



recursion-via-/128




Ingress platform showdata is not available.

Load distribution: 0 (refcount 1)

Hash OK Interface Address

0 Y Unknown ::ffff:192.168.5.5:0

TECMPL-3201 146


Verifying Egress Hardware Programming – IOS XRPE2#show cef vrf red ipv6 2001:db8::7/128 hard egr det loc 0/0/CPU0


0x0 (0x0), 0x208 (0xa14db230)

[1 type 1 flags 0x48089 (0xa14f5398) ext 0x0 (0x0)]

LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0]

gateway array update type-time 1 Feb 4 22:46:29.730

LDI Update time Feb 4 22:46:29.730



recursion-via-/128




Egress platform showdata is not available.

Load distribution: 0 (refcount 1)

Hash OK Interface Address

0 Y Unknown ::ffff:192.168.5.5:0

TECMPL-3201 147


Troubleshooting 6VPE / MPLS

• Verify the interface counters for mpls forwarding

• If there is forwarding problem, check the counters and ensure they are not increasing.

• Initiate the VPNv6 prefix ping and verify the counters again to see if they increased

Verifying Counters on Interface

RP/0/0/CPU0:PE2#show interface gigabitethernet0/0/0/1 accounting

GigabitEthernet0/0/0/1

Protocol Pkts In Chars In Pkts Out Chars Out

IPV4_UNICAST 261333 20337753 46929 2305821

IPV6_UNICAST 21017 2062274 20995 1964348

MPLS 10 1180 14426 968553

ARP 84 5040 84 3528

IPV6_ND 13296 1193736 10306 742016

TECMPL-3201 148

Troubleshooting MPLS

Traffic-Engineering


Troubleshooting MPLS TEThe “Fish” Problem

6050

100

40

8045

70

25

35M Traffic

10M Traffic

X

TECMPL-3201 150


Troubleshooting MPLS TE

• Increase efficiency of bandwidth resources

• Prevent over-utilised (congested) links whilst other links are under-utilised

• Ensures the most desirable/appropriate path for certain traffic types based on certain policies

• Override the shortest path selected by the IGP

• The ultimate goal is COST SAVING

Motivation

TECMPL-3201 151



• CSPF (executed at ingress) –computes an optimal explicit path based on constraints

• Bandwidth requirements

• Hop limitations

• Administrative groups (link colors)

• Priority (setup and hold)

• Explicit route

• Link attributes

• Reservable bandwidth of the links (static bandwidth minus the currently reserved bandwidth

CSPF – The TE AlgorithmDijkstra(G, w, s):

Initialize-single-source(G,s);

S = empty set;

Q = V[G];

While Q is not empty {

u = Extract-Min(Q);

S = S union {u};

for each vertex v in Adj[u] {

relax(u, v, w);

}

}

In which:

G: the graph, represented in some way (e.g. adjacency list)

w: the distance (weight) for each edge (u,v)

s (small s): the starting vertex (source)

S (big S): a set of vertices whose final shortest path from s have already been determined

Q: set of remaining vertices, Q union S = V

TECMPL-3201 152



1. CSPF process begins at ingress router with parameters of bandwidth, setup priority, hold priority and method used incase of equal cost multipath such as random, least fill or most-fill. It determines the final destination (Egress router).

2. It checks for maximum hop count, include and exclude constraints configured.

3. Check each node for metric and hop count starting with Ingress.

4. For each node check if endpoint is already visited ,if yes then skip the verification. if not check the link for metric, color and bandwidth (for constraints). The information on each node includes administrative groups (Color), metrics, static bandwidth, reservable bandwidth, and available bandwidth priority level. The information contained in the traffic engineering database should be the same across all routers in the same traffic engineering domain.

5. If it fails then remove this link.

6. If it passes then select the link with shortest path to neighbor router, go to next link and repeat the step 4.

draft-manayya-cspf-00

TECMPL-3201 153



• Repeat the steps 3 to 5 for all nodes

• The result of the CSPF algorithm is formed into a strict-hop ERO (Explicit Route Object)

• When the ERO is completed, the ERO is passed to the RSVP (Resource Reservation Protocol) process, where it is used for signaling and establishing the LSP in the network.

• If it is not possible to find the path then indicate about not finding a route then retry after retry interval.

draft-manayya-cspf-00 (contd…)

TECMPL-3201 154



• Once the path is calculated, it must be signaled across the network

• Reserve any bandwidth to avoid “double booking” from other TE reservations

• Priority can be used to pre-empt low priority existing tunnels

• RSVP used to setup TE LSP

• PATH messages (from head to tail) carries LABEL_REQUEST

• RESV messages (from tail to head) carries LABEL

• When RESV reaches headend, tunnel interface = UP

• RSVP messages exist for LSP teardown & error sig

RSVP Overview

Headend Midpoint Tailend

TECMPL-3201 155



• On receipt of PATH message

• Router will check there is bandwidth available to honour the reservation

• If bandwidth available then RSVP accepted

• On receipt of a RESV message

• Router actually reserves the bandwidth for the TE LSP

• If preemption is required lower priority LSP are torn down

• OSPF/ISIS updates are triggered

RSVP Overview – Admission Control

Does RSVP actually allocates the b/w across the path for TE tunnel?

TECMPL-3201 156


Troubleshooting MPLS TERSVP Overview – Admission Control

6050

100

40

8045

70

25

RSVP Path

Message

(10M)

BW=10 30

PATH RSVP

RESV

Message

TECMPL-3201 157



• RSVP should be enabled on relevant interfaces

• mpls traffic-eng should be enabled

• Globally

• Interface level

• IGP Level

• Tunnel Interface Configuration

• Allowing traffic through TE Tunnel

• Decision on Path Selection Process

• Dynamic

• Explicit-path

Configuration / Feature requirements

6050

100

40

8045

70

25

TECMPL-3201 158



• Used to include TE LSP in SPF calculations

• Tunnel is treated as a directly connected link to the tail

• IGP adjacency is NOT run over the tunnel!

• Using autoroute announce, all nodes behind the headend are routed via tunnel

Autoroute Announce

IOS – IOS-XE (Config under Tunnel Interface)tunnel mpls traffic-eng autoroute announce

IOS-XR (Configuration under Tunnel-te Interface)autoroute announce

NX-OS (Configuration under Tunnel-te Interface)autoroute announce

TECMPL-3201 159


R1

R2

R3

R4 R5

R6 R7

R8


• Autoroute does not advertise the LSP into the IGP

• There may be requirement to advertise the existence of TE tunnels to upstream routers

• Allow upstream routers to compute a better path to destination a over downstream TE tunnel

Forwarding Adjacency

All links have metric of 10

TECMPL-3201 160



• Verifying RSVP Interfaces

• Show ip rsvp interface

• Verifying TE Tunnels• Show mpls traffic-eng tunnels tunnel <num>

• Show mpls traffic-eng forwarding (XR)

• Show mpls traffic-eng forwarding-adjacency

• Verifying FRR Database• Show mpls traffic-eng fast-reroute database

Verification Commands

TECMPL-3201 161



RP/0/0/0:R1#sh rsvp counters messages summary

All RSVP Interfaces Recv Xmit Recv Xmit

Path 0 25 Resv 30 0

PathError 0 0 ResvError 0 1

PathTear 0 30 ResvTear 12 0

ResvConfirm 0 0 Ack 24 37

Bundle 0 Hello 0 5099

SRefresh 8974 9012 OutOfOrder 0

Retransmit 20 Rate Limited 0

RSVP Troubleshooting

IOS - Show ip rsvp counters summary

IP proto 0x2e – Can use this for performing packet capture

TECMPL-3201 162


Verify Basic TE Tunnel Forwarding

RP/0/RP0/CPU0:PE2#show mpls traffic-eng tunnels briefTUNNEL NAME DESTINATION STATUS STATEtunnel-te400 192.168.4.4 up up

PE1_t100 192.168.2.2 up upPE4_t100 192.168.2.2 up upPE1_t101 192.168.2.2 up up

RP/0/RP0/CPU0:PE2#show mpls traffic-eng tunnels 400

Name: tunnel-te400 Destination: 192.168.4.4 Ifhandle:0x580

Status:

Admin: up Oper: up Path: valid Signalling: connected

path option 1, type dynamic (Basis for Setup, path weight 1)

G-PID: 0x0800 (derived from egress interface properties)

Bandwidth Requested: 0 kbps CT0

Creation Time: Thu Jun 15 19:22:40 2017 (00:15:46 ago)

Config Parameters:

Bandwidth: 0 kbps (CT0) Priority: 7 7 Affinity: 0x0/0xffff

Metric Type: TE (global)

Path Selection:

<snip>

Fast Reroute: Disabled, Protection Desired: None

Path Protection: Not Enabled BFD Fast Detection: Disabled

Reoptimization after affinity failure: Enabled

Soft Preemption: Disabled

History: Tunnel has been up for: 00:15:46 (since Thu Jun 15 19:22:40 UTC 2017)

Current LSP: Uptime: 00:15:46 (since Thu Jun 15 19:22:40 UTC 2017)

Path info (OSPF 100 area 0):

Node hop count: 1

Hop0: 10.24.1.4

Hop1: 192.168.4.4

TECMPL-3201 163



• Configuration

• Logging

• Logging events lsp-status reoptimize (XR TE Tunnel interface config)

• Logging events lsp-status reroute (XR TE Tunnel interface config)

Re-optimization Configs

TECMPL-3201 164


No Destination

configured under

Tunnel interface

Very verbose

reason given here

on this line for

config errors

RP/0/RP0/CPU0:PE2# show mpls traffic-eng tunnel 400 detailWed May 29 14:07:50.428 UTC Name: tunnel-te 400 Destination: 0.0.0.0Status:Admin: up Oper: down Path: not valid Signalling: Downpath option 10, type dynamic (Basis for Setup, path weight 2)ospf 100 area 0 G-PID: 0x0800 (internally specified)

Config Parameters:Bandwidth: 0 kbps (CT0) Priority: 7 7 Affinity: 0x0/0xffffMetric Type: TE (default)AutoRoute: disabled LockDown: disabled Loadshare: 0 equal loadsharesAuto-bw: disabled(0/0) 0 Bandwidth Requested: 0Direction: unidirectionalEndpoint switching capability: unknown, encoding type: unassignedTransit switching capability: unknown, encoding type: unassigned

Reason for the tunnel being down: No destination is configuredHistory:Prior LSP:

ID: path option 10 [13]Removal Trigger: signalling shutdown

Troubleshooting MPLS TETroubleshooting : TE Tunnel does not come up

TECMPL-3201 165



RP/0/RP0/CPU0:PE2#show mpls traffic-eng tunnel 400 detailName: tunnel-te400 Destination: 192.168.4.4Status:Admin: up Oper: down Path: not valid Signalling: Downpath option 10, type dynamic (Basis for Setup, path weight 2) ospf 100 area 0G-PID: 0x0800 (internally specified)

Config Parameters:Bandwidth: 1 kbps (CT0) Priority: 7 7 Affinity: 0x0/0xffffMetric Type: TE (default)AutoRoute: enabled LockDown: disabled Loadshare: 0 equal loadsharesAuto-bw: disabled(0/0) 0 Bandwidth Requested: 1Direction: unidirectionalEndpoint switching capability: unknown, encoding type: unassignedTransit switching capability: unknown, encoding type: unassigned

History:Prior LSP:

ID: path option 1 [21]Removal Trigger: path verification failedLast Error:

PCALC:: No path to destination, 192.168.4.4(bw)

Insufficient RSVP b/w.

Bandwidth command not

configured under rsvp.

or

is misconfigured

Troubleshooting : TE Tunnel does not come up

TECMPL-3201 166



RP/0/RP0/CPU0:PE2#show mpls traffic-eng tunnel 400 detailName: tunnel-te400 Destination: 192.168.4.4Status:Admin: up Oper: down Path: not valid Signalling: Downpath option 10, type dynamic (Basis for Setup, path weight 2) ospf 100 area 0G-PID: 0x0800 (internally specified)

Config Parameters:Bandwidth: 1 kbps (CT0) Priority: 7 7 Affinity: 0x0/0xffffMetric Type: TE (default)AutoRoute: enabled LockDown: disabled Loadshare: 0 equal loadsharesAuto-bw: disabled(0/0) 0 Bandwidth Requested: 1Direction: unidirectionalEndpoint switching capability: unknown, encoding type: unassignedTransit switching capability: unknown, encoding type: unassigned

History:Prior LSP:

ID: path option 1 [21]Removal Trigger: path verification failedLast Error:

PCALC:: No path to destination, 192.168.4.4(reverselink or exclude-link)

Troubleshooting : TE Tunnel does not come up

Tunnel has no

alternative path

Or

Explicit path is

misconfigured.

TECMPL-3201 167



• RSVP Signaling in progress• Show rsvp sessions dst-port

• No path available• Show mpls traffic-eng igp-area

• Show mpls traffic-eng topology model-type rdm|mam

(Russian Dolls / Maximum allocation)

• Show mpls traffic-eng link-management interface x/y

• Cannot reach dst x.x.x.x from y.y.y.y• Show rsvp interface

• Or check TE topology database

TE Tunnel not up (Summary)

TECMPL-3201 168


Class-Based Tunnel Selection – CBTS

• EXP-based selection between multiple tunnels to same destination

• Local Tunnels (Head-end) configured with allowable EXP values

• Tunnels may be configured as default

• No IGP extensions, VRF aware

• Simplifies use of DS-TE tunnels & similar to PVC Bundling

Destination NH: PE2 EXP: 4PE2

PE3

PE4

PE1

Destination NH: PE2 EXP: Default

Destination NH: PE3 EXP: 5

Destination NH: PE3 EXP: 3,4


Destination NH: PE4 EXP: 5


TECMPL-3201 169



• BW pool applies to one or more classes

• Global BW pool (BC0) equals MRB

• BC0..BCn used for computing unreserved BW for class n

• Current implementation supports BC0 and BC1

• BC0 – Global Pool

• BC1 – Sub Pool

• Supported by Traditional and IETF implementation

Russian Dolls Model (RDM)

BC2

BC1

BC0All

Classes

(Class0+

Class1 +

Class2)Class1

+ Class2

Class2

Maximum

Reservable

Bandwidth

(MRB)

TECMPL-3201 170


All

Classes


• BW pool applies to one class

• Sum of BW pools may exceed MRB

• Sum of total reserved BW may not exceed MRB

• Current implementation supports BC0 and BC1

• Supported by IETF Implementation only

Maximum Allocation Model (MAM)

Maximum

Reservable

Bandwidth

(MRB)

BC2

BC1

BC0

Class1

Class0

Class2

TECMPL-3201 171


CBTS – Configuration Example interface Tunnel65

ip numbered loopback0

tunnel destination 192.168.2.2

tunnel mode mpls traffic-eng

tunnel mpls traffic-eng bandwidth sub-pool 30000

tunnel mpls traffic-eng exp 5

interface Tunnel66

ip numbered loopback0

tunnel destination 192.168.2.2

tunnel mode mpls traffic-eng

tunnel mpls traffic-eng bandwidth 50000

tunnel mpls traffic-eng exp default

Both tunnels to same

destination but different QoS

TECMPL-3201 172


Troubleshooting MPLS TETraces to collect on IOS XR

Module Trace commands

MPLS TE Control Show tech support mpls traffic-eng

RSVP Show tech support rsvp

CEF (forwarding) Show cef mpls trace location <line card location>

Show cef platform trace all all location <line card location>

Show cef trace location <line card location>

Show mpls forwarding tunnel detail

Show mpls forwarding labels hardware ingress/egress detail loc

Show cef mpls adj tunnel-te <> hardware ingress/egress detail loc <>

SONET Show sonet-local trace location <line card location>

Bundles Show tech-support bundles

Interface Manager Show tech-support pfi

TECMPL-3201 173



• Mechanism to mitigate packet loss during a failure

• Pre-provisioned protection tunnels that carry traffic when a protected link or node goes down

• MPLS TE protection also known as FAST REROUTE

• Protects against LINK FAILURE• For example, Fibre cut, Carrier Loss, ADM failure

• Protects against NODE FAILURE• For example, power failure, hardware crash, maintenance

Tunnel Protection

TECMPL-3201 174



• Local Protection

• Link Protection

• Node Protection

• Protects a segment of the tunnel (Node or Link)

• 1:N Scalability

• Faster failure recovery

• Path Protection

• Protects individual tunnels

• 1:1 Scalability

• More resource consumption

Categories of FRR

TECMPL-3201 175


Troubleshooting MPLS TELink Protection

PE1 P-2

P-3

P-4

P-5 PE2

VPN Label

TE Label

pop

TECMPL-3201 176


Troubleshooting MPLS TEFRR kicks in…

PE1 P-2

P-3

P-4

P-5 PE2

VPN Label

TE Label

pop

PLR

TECMPL-3201 177


Troubleshooting MPLS TENode Protection

PE1 P-2

P-3

P-4

P-5 PE2

VPN Label

TE Label

pop

TECMPL-3201 178



PE1 P-2

P-3

P-4

P-5 PE2

VPN Label

TE Label

pop

TECMPL-3201 179



PE1 P-2

P-3

P-4

P-5 PE2

VPN Label

TE Label

pop

TECMPL-3201 180

MPLS Traffic-Engineering Demo

SP SDN –Segment Routing


Segment Routing

• LDP had its own challenges

• Extra process required (LDP) + It creates complicated interaction with IGP (LDP-IGP Sync)

• RSVP-TE – Deployment and scalability issues (Only 10% SP space uses RSVP-TE and that too with FRR use-case)

• Always-on Feature, even when TE is not required in the network

• Need network that could understand application requirements

Path towards Segment Routing

TECMPL-3201 183


Segment Routing

• SR originally meant “Strade Romane” – network of roads which were built by Roman Empire

• The name was later changed to Segment Routing

• SR is nothing but Application Engineered Routing, where application makes request to the network (controller) to provide it a path that would serve the needs of the application

• SR is a source based routing, where the source chooses a path based on the application requirements

• The chosen path is encoded in the packet header as an ordered list of segments

• Segment – ID for any type of instruction

• Forwarding or service

Overview

TECMPL-3201 184


Segment Routing – Forwarding Plane• MPLS: an ordered list of segments is represented as a stack of labels

• IPv6: an ordered list of segments is encoded in a routing extension header

• This presentation: MPLS data plane

• Segment → Label

• Basic building blocks distributed by the IGP or BGP

TECMPL-3201 186


IGP segments• Two basic building blocks distributed by IGP

• Prefix Segments

• Adjacency Segments

TECMPL-3201 187


IGP Prefix Segment

• Shortest-path to the IGP prefix

• Equal Cost MultiPath (ECMP)-aware

• Global Segment

• Label = 16000 + Index

• Advertised as index

• Distributed by ISIS/OSPF

1 2

3 4

5

16005

16005

16005

16005

16005

16005

16005

1.1.1.5/32

All nodes use default SRGB

16,000 – 23,999

TECMPL-3201 188


IGP Prefix Segment• Shortest-path to the IGP prefix

• Equal Cost MultiPath (ECMP)-aware

• Global Segment

• Label = 16000 + Index

• Advertised as index


1 2

3 4

5

16004

16004

16004

16004

16004

16004

16004

1.1.1.4/32


16,000 – 23,999

TECMPL-3201 189


IGP Adjacency Segment• Forward on the IGP adjacency

• Local Segment

• Advertised as label value


1 2

3 4

524042

24045

Adj to 5

Adj to 2

24043

Adj to 3

TECMPL-3201 190


Combining IGP Segments• Steer traffic on any path through the network

• Path is specified by a stack of labels

• No path is signaled

• No per-flow state is created

• Single protocol: IS-IS or OSPF

1 2

3 4

5

16004 24045

16004

24045

Packet to 5

24045

Packet to 5


16,000 – 23,999

TECMPL-3201 191


Segment Routing – 3 Segments Example

• Source routing – ordered list of segments

• Stack of MPLS labels

• IPv6 Routing Extension

• MPLS labels are advertised by the IGP

• Simplicity

A B C D

E F G H

I

1700

1900

3000

1700

1900

1700

1700

segment 1

segment 3

Global label

3000

Global label

1700

Adjacency

label 1900

segment 2

PHP

TECMPL-3201 192

Segment Routing - Control Plane & Data Plane


MPLS Control and Forwarding Operation with Segment Routing

PE1 PE2

IGPPE1 PE2

Services

IPv4 IPv6IPv4

VPN

IPv6

VPNVPWS VPLS

Packet Transport

LDP

MPLS Forwarding

RSVP BGPStatic IS-IS OSPF

No changes to

control or

forwarding plane

IGP or BGP label

distribution for

IPv4 and IPv6.

Forwarding plane

remains the same

MP-BGP

TECMPL-3201 194


SID Encoding

• Prefix SID

• Label form SR Global Block (SRGB)

• SRGB advertised within IGP via TLV

• In the configuration, Prefix-SID can be configured as an absolute value or an index

• In the protocol advertisement, Prefix-SID is always encoded as a globally unique index

Index represents an offset from SRGB base, zero-based numbering, i.e. 0 is 1st index

E.g. index 1 SID is 16,000 + 1 = 16,001

• Adjacency SID

• Locally significant

• Automatically allocated by the IGP for each adjacency

• Always encoded as an absolute (i.e. not indexed) value

SRGB = [ 16,000 – 23,999 ] – Advertised as base = 16,000, range = 8,000

Prefix SID = 16,001 – Advertised as Prefix SID Index = 1

Adjacency SID = 24000 – Advertised as Adjacency SID = 24000

SR enabled node

TECMPL-3201 195


SR IS-IS Control Plane Summary

• IPv4 and IPv6 control plane

• Level 1, level 2 and multi-level routing

• Prefix Segment ID (Prefix-SID) for host prefixes on loopback interfaces

• Adjacency Segment IDs (Adj-SIDs) for adjacencies

• Prefix-to-SID mapping advertisements (mapping server)

• MPLS penultimate hop popping (PHP) and explicit-null signaling

TECMPL-3201 196


router isis 1


metric-style wide

segment-routing mpls

!


metric-style wide


!

interface Loopback0

passive


prefix-sid absolute 16001

!



!

!

IS-IS Configuration – Example

Ipv4 Prefix-SID value for loopback0

Wide metrics

enable SR IPv4 control plane and SR MPLS data plane on all ipv4 interfaces in this IS-IS instance

Wide metrics

enable SR IPv6 control plane and SR MPLS data plane on all ipv6 interfaces in this IS-IS instance

Ipv6 Prefix-SID value for loopback0

SID index 1

1.1.1.11.1.1.2

1.1.1.4 1.1.1.6

DIS

TECMPL-3201 197


SR OSPF Control Plane Summary

• OSPFv2 control plane

• Multi-area

• IPv4 Prefix Segment ID (Prefix-SID) for host prefixes on loopback interfaces

• Adjacency Segment ID (Adj-SIDs) for adjacencies

• Prefix-to-SID mapping advertisements (mapping server)

• MPLS penultimate hop popping (PHP) and explicit-null signaling

TECMPL-3201 198


router ospf 1

router-id 1.1.1.1


area 0

interface Loopback0

passive enable


!

!

!

OSPF Configuration Example

Prefix-SID for loopback0

Enable SR on all areas

SID index 1

1.1.1.11.1.1.2

1.1.1.5 1.1.1.3

DR

1.1.1.4

TECMPL-3201 199


MPLS Data Plane Operation (labeled)

• Packet forwarded along IGP shortest path (ECMP)

• Swap operation performed on input label

• Same top label if same/similar SRGB

• PHP if signaled by egress LSR

Payload

SRGB [16,000 – 23,999 ]

X

Payload

Swap

X

Payload

SRGB [16,000 – 23,999 ]

Y

Payload

Pop

Y

Adjacency

SID = X

X

Prefix SID Adjacency SID

Packet forwarded along IGP adjacency

Pop operation performed on input label

Top labels will likely differ

Penultimate hop always pops last adjacency SID

TECMPL-3201 200


Payload

VPN Label

MPLS Data Plane Operation (Prefix SID)

SRGB [16,000 – 23,999 ] SRGB [16,000 – 23,999 ] SRGB [16,000 – 23,999 ] SRGB [16,000 – 23,999 ]

Loopback X.X.X.X

Prefix SID Index = 41

A B C D

Payload

16041

Payload

Push

Push

Swap Pop

Payload Payload

VPN Label

16041

VPN Label

Pop

TECMPL-3201 201


Payload

VPN Label

MPLS Data Plane Operation (Adjacency SIDs)

Payload

16041

Payload

Push

Push

Push

Pop Pop

Payload Payload

VPN Label

16041

VPN Label

Pop

Adjacency

SID = 30206

30206

A B X D Loopback X.X.X.X

Prefix SID Index = 41

SRGB [16,000 – 23,999 ] SRGB [16,000 – 23,999 ] SRGB [16,000 – 23,999 ] SRGB [16,000 – 23,999 ]

TECMPL-3201 202

LDP-SR Migration


Simplest migration LDP to SR

• Initial state: All nodes run LDP, not SR

1 LDP

3 4

2

5 6

LDP Domain

LDP LDP

LDP LDP

LDP LDP

Assumptions:

• all the nodes can be upgraded to SR

• all the services can be upgraded to SR

TECMPL-3201 204




• Step1: All nodes are upgraded to SR

• In no particular order

• leave default LDP label imposition preference

1 LDP

3 4

2

5 6

SR+LDP Domain

SR+LDP SR+LDP

SR+LDP SR+LDP

SR+LDP SR+LDP

Assumptions:



TECMPL-3201 205







• Step2: All PEs are configured to prefer SR label imposition


SR+LDP Domain

1 SR

3 4

2

5 6

SR+LDP SR+LDP

SR+LDP SR+LDP

SR+LDP SR+LDP

sr-prefer

Assumptions:



TECMPL-3201 206


Simplest migration LDP to SR• Initial state: All nodes run LDP, not SR




• Step2: All PEs are configured to prefer SR label imposition


• Step3: LDP is removed from the nodes in the network


• Final state: All nodes run SR, not LDP

SR Domain

1 SR

3 4

2

5 6

SR SR

SR SR

SR SR

Assumptions:



TECMPL-3201 207


ISIS SRISIS Level-2

AS 64001

XR-3

3.3.3.3

XR-1

1.1.1.1

XE-2

2.2.2.2

XR-4

4.4.4.4

XR-5

5.5.5.5

XR-6

6.6.6.6

XE-7

7.7.7.7

XR-1010.10.10.10

XR-9

9.9.9.9

SRGB: 16000-23999

ISIS SRISIS Level-2

ISIS SRISIS Level-1AS64002

SR Topology

XE-8

8.8.8.8

16001

16002

16003

16004

16005

16006

16007

16010

16008 16009

Prefix SID Prefix SID

2401224010

24011 24008

PeerAdj SID*

Note (*) = PeerAdj SID values are dynamically allocated

Node X

Lo0 – 1.1.1.x/32

Link XY – 99.X.Y.0/24; X < Y

Prefix SID – 16000 + X

TECMPL-3201 208


Enabling Segment Routing – XR and XEIOS-XR

segment-routing

!

router isis SR-AS-1



!

interface Loopback0



!

commit

IOS-XE

XE-2(config)#segment-routing mpls

XE-2(config-srmpls)#connected-prefix-sid-map

XE-2(config-srmpls-conn)#address-family ipv4

XE-2(config-srmpls-conn-af)#2.2.2.2/32 absolute 16002 range 1

XE-2(config-srmpls-conn-af)#exit

XE-2(config-srmpls-conn)#exit

XE-2(config-srmpls)#exit

XE-2(config)#router isis SR-AS-1

XE-2(config-router)#segment-routing mpls

TECMPL-3201 209

Segment Routing Migration Demo

ODN


SR-PCE

• SR-PCE is an IOS XR multi-domain stateful SR Path Computation Element (PCE)

• IOS XR: XTC functionality is available on any physical or virtual IOS XR node, activated with a single configuration command

• Multi-domain: Real-time reactive feed via BGP-LS/ISIS/OSPF from multiple domains; computes inter-area/domain/AS paths

• Stateful: takes control of SRTE Policies, updates them when required

• SR PCE: native SR-optimized computation algorithms

• SR-PCE is fundamentally distributed

• Not a single all-overseeing entity (“god box”), but distributed across the network; RR-alike deployment

TECMPL-3201 212


SR-PCE Building Blocks

213TECMPL-3201

DeployCollect

Topo

DBCompute

REST API

IGP

BGP-LS

BGP

PCEP

Multi-Domain Topology

Native SR algorithms

SR-PCE runs on virtual or physical IOS-XR node

WAE Custom app


ISIS SRISIS Level-2AS 64001

XR-33.3.3.3

XR-11.1.1.1

XE-22.2.2.2

XR-44.4.4.4

XR-55.5.5.5

XR-66.6.6.6

XE-77.7.7.7

XR-1010.10.10.1

0

XR-99.9.9.9

BGP

BGP

ISIS SRISIS Level-2


ODN Workflow

XE-88.8.8.8

BGP RRBGP RR

BGP VPNv4

BGP VPNv4

• Routes tagged with a user-defined COLOR to convey SLA requirements

• VPN routes propagated via BGP

BGP VPNv4

Y/24BGP color comm.

“gold”

BGP color comm.“gold”

TECMPL-3201 214



XR-33.3.3.3

XR-11.1.1.1

XE-22.2.2.2

XR-44.4.4.4

XR-55.5.5.5

XR-66.6.6.6

XE-77.7.7.7

XR-1010.10.10.1

0

XR-99.9.9.9

BGP

BGP

ISIS SRISIS Level-2


XTC-ASR PCE

ODN Workflow

XE-88.8.8.8

• Ingress PE matches on user-specified BGP “color” community • Ingress PE enforces a “template” associated with the color community

Need a path to node (9)?Minimizing TE metric

SRTEOn-demand color “gold”

contact PCErequest path to

BGP NHminimize TE metric

PCReq

TECMPL-3201 215



XR-33.3.3.3

XR-11.1.1.1

XE-22.2.2.2

XR-44.4.4.4

XR-55.5.5.5

XR-66.6.6.6

XE-77.7.7.7

XR-1010.10.10.1

0

XR-99.9.9.9

BGP

BGP

ISIS SRISIS Level-2


XTC-ASR PCE

XTC-BSR PCE

PCEP

XE-88.8.8.8

PCEPPCEP

TECMPL-3201 216

Demo - ODN

© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS TECMPL-3201 218

https://goo.gl/M0EOfb

https://goo.gl/Y6d2D3


Complete your online session evaluation

• Please complete your session survey after each session. Your feedback is very important.

• Complete a minimum of 4 session surveys and the Overall Conference survey (starting on Thursday) to receive your Cisco Live water bottle.

• All surveys can be taken in the Cisco Live Mobile App or by logging in to the Session Catalog on ciscolive.cisco.com/us.

Cisco Live sessions will be available for viewing on demand after the event at ciscolive.cisco.com.


http://ciscolive.cisco.com/us

https://ciscolive.cisco.com/

Thank you

#CLUS

#CLUS

Reference Slides

Troubleshooting MPLS with NX-OS


Troubleshooting MPLS with NX-OSSoftware Architecture

LDP IGP

URIB

Netstack

ULIB

PSS

System

Manager

CLI

Feature

Manager

SNMP

Message QueueShared Memory

IM/OIM/

MPLS-Mgr

MTS

License

Manager

L3VM

TECMPL-3201 225



• Interact with ULIB

• Allocate local label for prefixes

• Interact with URIB

• Learn routes

• Program outgoing labels

• Interact with Netstack

• UDP socket (Hello messages)

• TCP sockets (Other LDP messages)

• Interact with IM/OIM/MPLS_mgr

• Learn interface status

• Learn interface address

• Enable MPLS forwarding on interface

Component Functions

TECMPL-3201 226



• Interact with L3VM

• VRF table id

• Interact with IGP

• LDP-IGP sync

• LDP auto-configuration

• Interact with platform services

• Enabling LDP feature enables multiple services:

• LDP service

• ULIB service

• mpls_mgr service

• mpls_oam service

Component Functions (contd…)

TECMPL-3201 227



• LDB – L2 Features; Perform LDP lookup to derive LIF / BD for ingress packet

• L2FT – Perform SMAC and DMAC lookup; DMAC should be router MAC

• ILM – Lookup ingress LIF MAP table and identify feature enabled, i.e. MPLS

• FIB – Deals with both PI and PD programming

• ADJ – FIB results provides adjacency points to egress LIF

• ELM – Egress LIF has the DI for egress interface

• RIT – Generate the rewrite (SMAC, DMAC and Label rewrite [push, pop, swap])

MPLS Packet Flow

LDBL2FT

(DMAC)ILM

FIB TCAM

ADJ ELM RIT

TECMPL-3201 228


Troubleshooting MPLS with NX-OSLDB – Check if the router BD is set in the LDP entrymodule-1# show hardware internal forwarding interface e1/1

Software Tables:

Interface = Ethernet1/1 LTL Index = 0x422 LIF = 0x4002

State(up) Layer(L3) Mode(0x0) VDC(1) Local Port(yes)

Number of Member Ports(0x0)

LDB Sharing(no) LDB Base(0xc801) LDB Port Features(no)

Hardware Tables:

Instance: 0x1

L2-LIF-MAP entry with index = 0x422

ldb_base = 0xc801 add_vlan = 0

Instance: 0x1

L2-LIF entry with index = 0xc801

pt_cam_en = 0 ipv4_igmp_snoop = 0 ipv4_pim_snoop = 0 ipv6_mld_snoop = 0

ipv6_pim_snoop = 0 bd = 0x2 l2v4 = 0 ingr_lif = 0x4002

<snip>

TECMPL-3201 229


Troubleshooting MPLS with NX-OSCheck if the router BD is set in the LDP entrymodule-1# show hardware internal forwarding interface e1/1

Software Tables:

Interface = Ethernet1/1 LTL Index = 0x422 LIF = 0x4002

State(up) Layer(L3) Mode(0x0) VDC(1) Local Port(yes)

Number of Member Ports(0x0)

LDB Sharing(no) LDB Base(0xc801) LDB Port Features(no)

Hardware Tables:

Instance: 0x1

L2-LIF-MAP entry with index = 0x422

ldb_base = 0xc801 add_vlan = 0

Instance: 0x1

L2-LIF entry with index = 0xc801

pt_cam_en = 0 ipv4_igmp_snoop = 0 ipv4_pim_snoop = 0 ipv6_mld_snoop = 0

ipv6_pim_snoop = 0 bd = 0x2 l2v4 = 0 ingr_lif = 0x4002

<snip>

TECMPL-3201 230



show hardware mac address-table

FE | Valid| PI| BD | MAC | Index| Stat| SW | Modi| Age| Tmr| GM| Sec| TR| NT| RM| RMA| Cap| Fld|Always

| | | | | | ic | | fied|Byte| Sel| | ure| AP| FY| | |TURE| | Learn

---+------+---+------+---------------+-------+-----+-----+-----+----+----+---+----+---+---+---+----+----+----+------

0 1 1 2 0022.557a.32c1 0x00400 1 0x000 0 6 0 1 0 0 0 0 0 0 0 0

0 1 0 1 0100.0cff.fffe 0x00421 1 0x001 0 6 0 0 0 0 0 0 0 0 0 0

Verify L2FT and ILM

NX-OS# show hardware internal forwarding interface Ethernet 1/1 module 10 | inc mpls_en

l2l3_lkup_cfg = 0 mpls_en = 1 sm_en = 0 red_ids_chk_fail_en = 1 v4_rpfv3_en = 0

ipv4_en = 1 eompls_en = 0 mpls_en = 1

L2FT

ILM

N7k-1# show hardware internal forwarding interface e1/2 module 1 | in mpls_en

mpls_vpn_sel : 0x0 l2_tunnel_type : 0x0 mpls_en : 0x1

TECMPL-3201 231



N7k-1# show forwarding route module 1

----------------+----------------------------------------+----------------------+-----------------

Prefix | Next-hop | Interface | Labels

----------------+----------------------------------------+----------------------+-----------------

<snip>

192.168.2.2/32 nxthop 10.12.1.2 Ethernet1/2 NO-OP

192.168.3.3/32 nxthop 10.12.1.2 Ethernet1/2 PUSH 21

192.168.4.4/32 nxthop 10.12.1.2 Ethernet1/2 PUSH 22

Verifying FIB - PI

N7k-1# show forwarding route detail

Prefix 192.168.2.2/32,

No of paths : 1 Update time: Wed Jun 14 08:46:21 2017

nxthop 10.12.1.2 Ethernet1/2 NO-OP DMAC: 001b.54c2.3342

packets: 0 bytes: 0

Prefix 192.168.3.3/32,


nxthop 10.12.1.2 Ethernet1/2 PUSH 21 DMAC: 001b.54c2.3342

packets: 0 bytes: 0

Prefix 192.168.4.4/32,


nxthop 10.12.1.2 Ethernet1/2 PUSH 22 DMAC: 001b.54c2.3342

packets: 0 bytes: 0

TECMPL-3201 232



N7k-1# show forwarding mpls module 1

--------+-----------+-------------------+----------------+-------------+-------

Local |Prefix |FEC |Next-Hop |Interface |Out

Label |Table Id |(Prefix/Tunnel id) | | |Label

--------+-----------+-------------------+----------------+-------------+-------

18 |0x1 |192.168.2.2/32 |10.12.1.2 |Ethernet1/2 |Pop Label

19 |0x1 |192.168.3.3/32 |10.12.1.2 |Ethernet1/2 |21

20 |0x1 |192.168.4.4/32 |10.12.1.2 |Ethernet1/2 |22

Verifying FIB – PI – Forwarding and Adjacency Info

N7k-1# show forwarding adjacencyIPv4 adjacency informationnext-hop rewrite info interface-------------- --------------- -------------10.1.12.2 001b.54c2.3342 Ethernet1/2

N7k-1# show forwarding adjacency mpls

IPv4 adjacency information, adjacency count 1

next-hop rewrite info interface

-------------- --------------- -------------

10.1.12.2 Ethernet1/2 001b.54c2.3342 NO-OP 3

10.1.12.2 Ethernet1/2 001b.54c2.3342 PUSH 21

10.1.12.2 Ethernet1/2 001b.54c2.3342 PUSH 22

TECMPL-3201 233



N7k-1# show system internal forwarding mpls detail

Table id = 0x1

------------------

----+--------+--------+------------+----------+----------+-----------+--------+

Dev | Index |Priority| In-label | AdjIndex | LIF | Out-label | Op

----+--------+--------+------------+----------+----------+-----------+--------+

0 0x5624 0x23c2 16 0x5c 0x1fe0 0 POP ONE

0 0x5625 0x23c3 17 0x5c 0x1fe0 0 POP ONE

0 0x5224 0x23c4 18 0x62 0x2 3 POP ONE

0 0x5225 0x23c5 19 0x60 0x2 21 SWAP ONE

0 0x5c24 0x23c6 20 0x64 0x2 22 SWAP ONE

0 0x5c25 0x23c7 21 0x65 0x3 0 POP ONE

Table id = 0x2a

------------------

----+--------+--------+------------+----------+----------+-----------+--------+

Dev | Index |Priority| In-label | AdjIndex | LIF | Out-label | Op

----+--------+--------+------------+----------+----------+-----------+--------+

No labels in table

Aggregate Table id = 0x2a

------------------

--------+--------+

label | vpn_id

--------+--------+

0 492287 0x2a

Verifying FIB – PD – MPLS Programming

TECMPL-3201 234



pe1# show system internal forwarding mpls label

show system internal forwarding mpls

Table id = 1

------------------

----+--------+------------+----------+----------+-----------+--------+

Dev | Index | In-label | AdjIndex | LIF | Out-label | Op

----+--------+------------+----------+----------+-----------+--------+

0 0x1ffa9 18 0x62 0x2 3 POP ONE

0 0x5225 19 0x60 0x2 21 SWAP ONE

0 0x5c24 20 0x64 0x2 20 SWAP ONE

Verify Label Information in Hardware

FIB TCAM

Index

Egress LIF

(LTL)

FIB DRAM

Adjacency

Index

TECMPL-3201 235



• Use the following command to check the route in FIB PD

• Show system internal forwarding route

• Use the following command to check the adjacency in FIB PD

• Show system internal forwarding adjacency

• Use the following command to check the MPLS adjacency in LFIB PD

• Show system internal forwarding mpls adjacency

• Use the following command to check the hardware adjacency to verify if the packet is getting forwarding out correct interface

• Show system internal forwarding adjacency entry <adj> detail

Route Update PD Verification

TECMPL-3201 236



• Check for L3VM process for the event-traces to verify the events that occurrent for the VRF

Troubleshooting L3VPN VRF Issues

N7k-1# show system internal l3vm event-history vrf

VRF events for L3VM Process - Bufsize 1000 KB2017

2017 Jun 14 09:10:02.139925 l3vm [5710]: [5830]: Updated interface Ethernet1/1 cmd <vrf member TEST>

2017 Jun 14 09:10:02.139757 l3vm [5710]: [5830]: Interface Ethernet1/1 (IOD 37) changing from VRF default to VRF TEST - Count 1

2017 Jun 14 09:10:02.139728 l3vm [5710]: [5830]: Interface Ethernet1/1 (IOD 37) will be down, VRF default UP-IF count 1

2017 Jun 14 09:10:02.139680 l3vm [5710]: [5830]: Moving Ethernet1/1 (ifindex: 0x1a000000 iod: 37) from VRF default to VRF TEST

2017 Jun 14 09:10:02.139522 l3vm [5710]: [5830]: Deleting all L3VM_PSS_IF_KEY config for interface Ethernet1/1

2017 Jun 14 09:10:02.137418 l3vm [5710]: [5830]: [VSH] Process interface Eth1/1 cmd <vrf member TEST>

2017 Jun 14 09:06:24.460917 l3vm [5710]: [5830]: Updated vrf TEST cmd <address-family ipv4 unicast>

2017 Jun 14 09:06:24.460771 l3vm [5710]: [5830]: [VSH] Process vrf TEST cmd <address-family ipv4 unicast>

2017 Jun 14 09:06:24.426293 l3vm [5710]: [5830]: l3vm_pd_process_l3vm_mts_msg_from_ctrl: Received l3vm notification (mtype: 4)



2017 Jun 14 09:06:24.424511 l3vm [5710]: [5829]: VRF TEST:ipv4:base table (Up:--) sending: Table create

2017 Jun 14 09:06:24.424372 l3vm [5710]: [5829]: VRF TEST:ipv6:base table (Up:--) sending: Table create

2017 Jun 14 09:06:24.424256 l3vm [5710]: [5829]: VRF TEST (Up:--) sending: VRF create

2017 Jun 14 09:06:24.424006 l3vm [5710]: [5829]: VRF TEST - Created

2017 Jun 14 09:06:24.424002 l3vm [5710]: [5829]: VRF TEST (Up:--) sdb ack

2017 Jun 14 09:06:24.423008 l3vm [5710]: [5829]: gsdb_op_callback() - gsdb context 0x0003ce86

2017 Jun 14 09:06:24.421933 l3vm [5710]: [5830]: Updated cmd <vrf context TEST>

TECMPL-3201 237

Inter-AS MPLS VPN


Inter-AS MPLS VPNs

• Previous section – VPNs within Single-AS boundary

• Inter-AS MPLS VPN – VPNs spanning across multiple AS boundaries

• Types:

• Option A – Back to Back VRF

• Option B – Inter-Provider VPNs using ASBR-to-ASBR approach

1. Next-Hop-Self Method

2. Redistribute Connected Method

3. Multi-hop EBGP between ASBRs

• Option C – MP-EBGP between RR and EBGP between ASBR

Flavors

TECMPL-3201 239


Inter-AS VPN Topology

AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8


Inter-AS VPN Option A: Back to Back VRFs

• Terminate VRFs on ASBRs

• Advertise Peering Link to VRF/BGP

• Exchange routes across peering link

• Simple

• Doesn’t Scale Well

AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

IPv4 + IGP/BGP

VRF VPN01 VRF VPN02



AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

router bgp 100





!




exit-address-family

!

address-family ipv4 vrf VPN01




exit-address-family



AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

router bgp 200

vrf VPN02

rd 200:1



!


remote-as 100




© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS 244


AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

CE7#trace 100.64.8.8




1 172.16.57.5 3 msec 3 msec 3 msec

2 172.16.12.1 [AS 100] [MPLS: Label 204 Exp 0] 4 msec 4 msec 5 msec

3 172.16.12.2 [AS 100] 5 msec 5 msec 4 msec


5 172.32.68.8 [AS 200] 11 msec * 11 msec


Inter-AS VPN Option B1: ASBR to ASBR w/ Next-Hop-Self

No LDP or IGP required on the link between the two ASBRs.

Configure no bgp default route-target filter on ASBRs

ASBRs advertise to RRs with Next-Hop-Self

AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

Next-Hop-Self Next-Hop-Self

172.16.1.1v1


Inter-AS MPLS VPNs

• Enabled by default by all non-RRs.

Problems with Route Filtering

R1#debug bgp vpnv4 unicast updates

BGP updates debugging is on for address family: VPNv4 Unicast

R1#clear bgp vpnv4 unicast * sof

*Jun 20 19:35:50.710: BGP: nbr_topo global 192.168.3.3 VPNv4 Unicast:base (0x110FC570:1) rcvd

Refresh Start-of-RIB

*Jun 20 19:35:50.710: BGP: nbr_topo global 192.168.3.3 VPNv4 Unicast:base (0x110FC570:1)

refresh_epoch is 3

*Jun 20 19:35:50.711: BGP(4): 192.168.3.3 rcvd UPDATE w/ attr: nexthop 192.168.5.5, origin ?,

localpref 100, metric 0, originator 192.168.5.5, clusterlist 192.168.3.3, merged path 700, AS_PATH

, extended community RT:100:1

*Jun 20 19:35:50.714: BGP(4): 192.168.3.3 rcvd 100:1:100.64.7.0/24, label 5003 - DENIED due to:

extended community not supported;

TECMPL-3201 246



AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

interface GigabitEthernet0/2

ip address 172.16.12.1 255.255.255.0

mpls bgp forwarding

!

router bgp 100



no bgp default route-target filter




!






neighbor 192.168.3.3 next-hop-self

R1



AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

router bgp 200



retain route-target all

!


remote-as 100




!

!


remote-as 200



next-hop-self


Inter-AS MPLS VPNs

• Routes will not install on remote Pes if they have different RTs

• AS 100 was using 100:1

• AS 200 was using 200:1

• Check to see if the routes make it on ASBRs or RRs

Problems with Route Installation at Remote PEs

TECMPL-3201 249


Inter-AS MPLS VPNsProblems with Route Installation: Checking on the RRsR3#show bgp vpnv4 unicast all | b Netw

Network Next Hop Metric LocPrf Weight Path

Route Distinguisher: 100:1

*>i 100.64.7.0/24 192.168.5.5 0 100 0 700 ?

*>i 172.16.57.0/24 192.168.5.5 0 100 0 ?


*>i 100.64.8.8/32 192.168.1.1 0 100 0 200 700 ?

*>i 172.32.68.0/24 192.168.1.1 0 100 0 200 ?

RP/0/0/CPU0:XR4#show bgp vpnv4 unicast | b Netw

Network Next Hop Metric LocPrf Weight Path


*>i100.64.7.0/24 192.168.2.2 100 0 100 700 ?

*>i172.16.57.0/24 192.168.2.2 100 0 100 ?


*>i100.64.8.8/32 192.168.6.6 0 100 0 700 ?

*>i172.32.68.0/24 192.168.6.6 0 100 0 ?

TECMPL-3201 250


Inter-AS MPLS VPNs

Simple Solution, but does it scale?

Problems with Route Installation: Solution 1 – Additional Import Statements

R3 (IOS PEs)

vrf definition VPN01

rd 100:1




XR4 (IOS XR PEs)

vrf VPN02


import route-target

200:1

100:1

!

export route-target

200:1

TECMPL-3201 251


Inter-AS MPLS VPNs

IOS ASBRs (R1)

Problems with Route Installation: Solution 2 – Route Target ReWrite on ASBRs

ip extcommunity-list 1 permit rt 200:1

route-map REWRITE permit 10

match extcommunity 1

set extcomm-list 1 delete

set extcommunity rt 100:1 additive

!

router bgp 100




neighbor 172.16.12.2 route-map REWRITE in

TECMPL-3201 252


Inter-AS MPLS VPNs

IOS XR ASBRs (XR2)

Problems with Route Installation: Solution 2 – Route Target Re-Write on ASBRs

route-policy REWRITE

if extcommunity rt matches-any AS100VPN01

then

set extcommunity rt AS200VPN02

endif

pass

end-policy

!

extcommunity-set rt AS100VPN01

100:1

end-set

!

extcommunity-set rt AS200VPN01

200:1

end-set

router bgp 200


remote-as 100


route-policy REWRITE in


TECMPL-3201 253



AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

router bgp 200




!


remote-as 100




!

!


remote-as 200



next-hop-self



AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

RP/0/0/CPU0:XR2#show mpls forwarding

Local Outgoing Prefix Outgoing Next Hop Bytes

Label Label or ID Interface Switched

------ ----------- ------------------ ------------ --------------- ------------

24000 Pop 192.168.6.6/32 Gi0/0/0/2 10.26.1.6 796

24001 Pop 192.168.4.4/32 Gi0/0/0/0 10.24.1.4 12010

24003 60003 200:1:100.64.8.8/32 \

192.168.6.6 0

24004 60004 200:1:172.32.68.0/24 \

192.168.6.6 208

24005 Aggregate 172.16.12.0/24 default 0

24006 206 100:1:100.64.7.0/24 \

172.16.12.1 0

24007 207 100:1:172.16.57.0/24 \

172.16.12.1 0

XR2



AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

router static


172.16.12.1/32 GigabitEthernet0/0/0/1

XR2



AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP




------ ----------- ------------------ ------------ --------------- ------------

24000 Pop 192.168.6.6/32 Gi0/0/0/2 10.26.1.6 1070

24001 Pop 192.168.4.4/32 Gi0/0/0/0 10.24.1.4 12383

24003 60003 200:1:100.64.8.8/32 \

192.168.6.6 0

24004 60004 200:1:172.32.68.0/24 \

192.168.6.6 20176

24006 206 100:1:100.64.7.0/24 \

Gi0/0/0/1 172.16.12.1 0

24007 207 100:1:172.16.57.0/24 \

Gi0/0/0/1 172.16.12.1 0

24008 Pop 172.16.12.1/32 Gi0/0/0/1 172.16.12.1 0

XR2



AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R8

MP-EBGP

XR2

CE7#trace 100.64.8.8




1 172.16.57.5 3 msec 2 msec 3 msec




5 172.32.68.8 [AS 200] 16 msec * 18 msec

R7


Inter-AS VPN Option B2: Advertise Peering Link

No LDP or IGP required on the link between the two ASBRs.


ASBRs redistribute/advertise peering link into IGPAS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

Redistribute

Connected

172.16.1.1v1

Redistribute

Static Route



AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP


ip address 172.16.12.1 255.255.255.0

mpls bgp forwarding

!

router ospf 1

redistribute connected subnets

network 10.0.0.0 0.255.255.255 area 100

network 192.168.0.0 0.0.255.255 area 100

!

router bgp 100






!






R1



AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

router ospf 1

redistribute static

area 200

interface Loopback0

interface GigabitEthernet0/0/0/0


!

router bgp 200



!


remote-as 100




!


remote-as 200




Inter-AS VPN Option B3: Multi-hop

Static route on ASBRs to reach remote ASBR loopback

ASBRs peer with each other via Loopback interface. Requires EBGP Multi-Hop


ASBRs advertise remote loopback into BGPAS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

Redistribute

Connected

172.16.1.1v1

Redistribute

Connected


Inter-AS VPN Option B3: MultiHop

AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

ip route 192.168.2.2 255.255.255.255 172.16.12.2


ip address 172.16.12.1 255.255.255.0

mpls ip

!

router ospf 1

redistribute static subnets

network 10.0.0.0 0.255.255.255 area 100

network 192.168.0.0 0.0.255.255 area 100

!

router bgp 100




neighbor 192.168.2.2 ebgp-multihop 255


!





R1


Inter-AS VPN Option B3: Multihop

AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

router ospf 1

redistribute static

area 200

..

!

router bgp 200



!


remote-as 100

ebgp-multihop 255





!


remote-as 200




Inter-AS VPN Option C: MBGP between RRs

VPNv4 session is established between RRs

RRs use Next-Hop-Unchanged

ASBRs exchange RRs and PE loopbacks as labeled routesAS 700

100.64.7.7AS 700

100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGPPE & RR Labels

172.16.1.1v1

MP-EBGPVPNv4 Routes/Labels


Inter-AS VPN Option C: RRs Peer Direct

AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

XR2

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

router ospf 1

redistribute bgp 100 subnets

network 10.0.0.0 0.255.255.255 area 100

network 192.168.0.0 0.0.255.255 area 100

!

router bgp 100




!

address-family ipv4

network 192.168.1.1 mask 255.255.255.255

network 192.168.3.3 mask 255.255.255.255

network 192.168.5.5 mask 255.255.255.255


neighbor 172.16.12.2 send-label

R1



AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3 XR4

R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

router ospf 1

redistribute bgp 200

area 200

interface Loopback0



!

router bgp 200



network 192.168.2.2/32

network 192.168.4.4/32

network 192.168.6.6/32

allocate-label all

!


remote-as 100

address-family ipv4 labeled-unicast



XR2



AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

XR4

XR2

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

R1

router bgp 100




neighbor 192.168.4.4 ebgp-multihop 255




!




neighbor 192.168.4.4 next-hop-unchanged




neighbor 192.168.5.5 route-reflector-client

R3



AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

R5 PEXR6

R3

R1

ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

MP-EBGP

XR2

router bgp 200



!


remote-as 100

ebgp-multihop 255





next-hop-unchanged

!

!


remote-as 200



route-reflector-client

XR4


Inter-AS VPN: Examining the VPNv4 Routes

Verify the source/destination routes at entry/remote PE and local/remote ASBR

AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

PE

R3 XR4ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

IPv4 + IGP/BGP

Check RoutesCheck Routes

Check Routes

Check Routes

XR6

R1 XR2

R5

TECMPL-3201 270


Inter-AS VPN: Examining the MPLS FECs

Verify the source/destination is label switched towards destination with numbered labeles or ASBRs have POP

Remember IOS XR needs a /32 entry for the FEC to populate.AS 700

100.64.7.7AS 700

100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

PE

R3 XR4ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

IPv4 + IGP/BGP

Check MPLS

Forwarding

Table

R1

Check MPLS

Forwarding

Table

R5

Check MPLS

Forwarding

Table

XR2

Check MPLS

Forwarding

Table

XR6

TECMPL-3201 271


Troubleshooting Inter-AS VPN: Tip

Sometimes the issue may not appear directly.

Add a loopback interface on ASBR, and place into a VRF.

Provides a method of checking connectivity across the ASBR link.

AS 700100.64.7.7

AS 700100.64.8.8

ASBR192.168.1.1

ASBR

PE

RR

RR

CE CE

PE

R3 XR4ASBR192.168.2.2

AS100VPN02

AS200VPN02

R7 R8

IPv4 + IGP/BGP

Add a VRF

Check Here

R1

R5

Add a VRF

Check Here

XR2

XR6

TECMPL-3201 272

MPLS Carrier Supporting Carrier (CSC)


Carrier Supporting Carrier (CSC)• CSC allows MPLS services across discontiguous areas. Typically

when MPLS services cannot be provided end-to-end because of geography reasons.

CE

Customer Customer

Service Provider 1

Service Provider 2 Service Provider 2

CE

TECMPL-3201 274


Carrier Supporting Carrier (CSC) Roles

R7

AS 700 AS 800

R3

R8

CSC-PE CSC-PE

PE

CSC-CECSC-CE

R5 XR6

XR4

PE

CE CE

CustomerCarrier

Backbone

Carr

ier

CustomerCarrier

XR2R1


CSC is not running MPLS inside its POP Sites

CE7

AS 700 AS 800

R3

CE7

LDP + IGP

MP-IBGP

R5

XR4

XR6

EBGP EBGP

LDP + IGP

or

Labeled BGPLabeled

BGP

IBGP + RR Client

IBG

P

IBG

P

CustomerCarrier

XR2R1



CE7

AS 700 AS 800

R3

CE7

LDP + IGP

MP-IBGP

R5

XR4

XR6

EBGP EBGP

LDP + IGP

Labeled

BGP

IBGP + RR Client

IBG

P

IBG

P

CustomerCarrier

XR2


description to R3

vrf forwarding CORE

ip address 172.16.13.1 255.255.255.0

mpls ip

!

router ospf 10 vrf CORE

redistribute bgp 100 subnets

network 172.16.0.0 0.0.255.255 area 200

!

router bgp 100

address-family ipv4 vrf CORE

redistribute ospf 10

R1



CE7

AS 700 AS 800

R3

CE7

LDP + IGP

MP-IBGP

R5

XR4

XR6

EBGP EBGP

Labeled BGP Labeled

BGP

IBGP + RR Client

IBG

P

IBG

P

CustomerCarrier

XR2


description to R3

vrf forwarding CORE

ip address 172.16.13.1 255.255.255.0

mpls bgp forwarding

!

router bgp 100

address-family ipv4 vrf CORE

network 172.16.13.0 mask 255.255.255.0



neighbor 172.16.13.3 as-override

neighbor 172.16.13.3 send-label

R1



CE7

AS 700 AS 800

R3

CE7

LDP + IGP

MP-IBGP

R5

XR4

XR6

EBGP EBGP

LDP + IGP

or

Labeled BGPLabeled

BGP

IBGP + RR Client

IBG

P

IBG

P

CustomerCarrier

R1

router bgp 100

vrf CORE

rd 100:1



allocate-label all

!


remote-as 200

address-family ipv4 labeled-unicast



as-override

XR2



CE7

AS 700 AS 800

R3

CE7

LDP + IGP

MP-IBGP

R5

XR4

XR6

EBGP EBGP

LDP + IGP

or

Labeled BGPLabeled

BGP

IBGP + RR Client

IBG

P

IBG

P

CustomerCarrier

R1




------ ----------- ------------------ ------------ --------------- ------------

24000 Pop 192.168.1.1/32 Gi0/0/0/1 10.12.1.1 1271

24002 100 172.16.13.0/24[V] 192.168.1.1 0

24003 104 172.16.35.0/24[V] 192.168.1.1 0

24004 Aggregate CORE: Per-VRF Aggr[V] \

CORE 4280

24005 103 192.168.3.3/32[V] 192.168.1.1 0

24006 106 192.168.5.5/32[V] 192.168.1.1 1022

24007 Pop 172.32.46.0/24[V] 172.32.24.4 0

24008 Pop 192.168.4.4/32[V] 172.32.24.4 49920

24009 44005 192.168.6.6/32[V] 172.32.24.4 8312796

XR2

Where is

172.32.24.4/32



CE7

AS 700 AS 800

R3

CE7

LDP + IGP

MP-IBGP

R5

XR4

XR6

EBGP EBGP

LDP + IGP

or

Labeled BGPLabeled

BGP

IBGP + RR Client

IBG

P

IBG

P

CustomerCarrier

R1




------ ----------- ------------------ ------------ --------------- ------------

24000 Pop 192.168.1.1/32 Gi0/0/0/1 10.12.1.1 296

24001 Pop 172.32.24.4/32[V] Gi0/0/0/0 172.32.24.4 1210

24002 100 172.16.13.0/24[V] 192.168.1.1 0

24003 104 172.16.35.0/24[V] 192.168.1.1 0

24004 Aggregate CORE: Per-VRF Aggr[V] \

CORE 4280

24005 103 192.168.3.3/32[V] 192.168.1.1 0

24006 106 192.168.5.5/32[V] 192.168.1.1 252

24007 Pop 172.32.46.0/24[V] Gi0/0/0/0 172.32.24.4 0

24008 Pop 192.168.4.4/32[V] Gi0/0/0/0 172.32.24.4 48880

24009 44005 192.168.6.6/32[V] Gi0/0/0/0 172.32.24.4 8092044

router static

vrf CORE


172.32.24.4/32 GigabitEthernet0/0/0/0

XR2



CE7

AS 700 AS 800

R3

CE7

LDP + IGP

MP-IBGP

R5

XR4

XR6

EBGP EBGP

LDP + IGP

or

Labeled BGPLabeled

BGP

IBGP + RR Client

IBG

P

IBG

P

CustomerCarrier

R1

CE7#trace 100.64.8.8




1 172.16.57.5 3 msec 3 msec 3 msec





6 172.32.46.6 [AS 200] 15 msec 15 msec 16 msec

7 172.32.68.8 [AS 200] 16 msec * 19 msec

XR2


CSC is running MPLS inside its POP Sites

CE7

AS 700 AS 800

R3

CE7

LDP + IGP

MP-IBGP

R5

XR4

XR6

EBGP EBGP

LDP + IGP

or

Labeled BGPLabeled

BGP

IBGP

CustomerCarrier

LD

P

IGP

LD

P

IGP

XR2R1


CSC is running MPLS inside its POP Sites

AS 700 AS 800

R3

CE7

LDP + IGP

MP-IBGP

R5

XR4

XR6

EBGP EBGP

LDP + IGP

or

Labeled BGPLabeled

BGP

IBGP

CustomerCarrier

LD

P

IGP

LD

P

IGP

XR2R1

CE7#trace 100.64.8.8




1 172.16.57.5 3 msec 3 msec 3 msec





6 172.32.46.6 [AS 200] 15 msec 15 msec 16 msec

7 172.32.68.8 [AS 200] 16 msec * 19 msec

CE7


CSC is running MPLS VPN inside its POP Sites

CE7

AS 700 AS 800

R3

CE7

LDP + IGP

MP-IBGP

XR4

EBGP EBGP

LDP + IGP

or

Labeled BGP

Labeled

BGP

MP-IBGP

CustomerCarrier

LD

P

IGP

LD

P

IGP

XR2R1

R5 XR6


CSC is running MPLS VPN inside its POP Sites

AS 700 AS 800

R3

CE7

LDP + IGP

MP-IBGP

XR4

EBGP EBGP

LDP + IGP

or

Labeled BGP

Labeled

BGP

MP-IBGP

CustomerCarrier

LD

P

IGP

LD

P

IGP

XR2R1

R5 XR6

CE7# trace 100.64.8.8 so lo0




1 172.16.57.5 5 msec 3 msec 4 msec






7 172.32.68.8 [AS 800] 26 msec * 19 msec

CE7 Customer Carrier

VPN Label

Backbone Carrier

Forwarding Label

Complete your online session evaluation

• Please complete your session survey after each session. Your feedback is very important.

• Complete a minimum of 4 session surveys and the Overall Conference survey (starting on Thursday) to receive your Cisco Live water bottle.

• All surveys can be taken in the Cisco Live Mobile App or by logging in to the Session Catalog on ciscolive.cisco.com/us.

Cisco Live sessions will be available for viewing on demand after the event at ciscolive.cisco.com.


http://ciscolive.cisco.com/us

https://ciscolive.cisco.com/


Continue your education

Related sessions

Walk-in labsDemos in the Cisco campus

Meet the engineer 1:1 meetings

TECMPL-3201 288

Thank you

#CLUS

#CLUS

Date post:	26-Feb-2023
Category:	Documents
Upload:	khangminh22
View:	1 times
Download:	0 times

Troubleshooting - MPLS - Cisco Live

Documents