Cisco Confidential 1© 2013 Cisco and/or its affiliates. All rights reserved.

BGP Monitoring ProtocolBertrand Duvivier (bduvivie@cisco.com)Sr Product Manager

September, 2014

The BGP Monitor Protocol

Provides access to the Adj-RIB-In (pre BGP inbound filtering) of a peer (BR: Internet Border Router or PE: Provider Edge Router) on an ongoing basis and a periodic dump of certain statistics that the monitoring station can use for further analysis.

BMP overviewWhat is BMP ?

http://tools.ietf.org/html/draft-ietf-grow-bmp-07.txt

BMP overview

Adj-RIB-in (pre-inbound-filter)BGP Monitor Protocol update

BMP collector

BMP clientInbound filteringpolicing

Loc-RIB (post-inbound-filter)iBGP update

BMP message

Adj-RIB-in (pre-inbound-filter)eBGP update

BMP client/collector

BGP peer’s (external)

BGP peer (internal)

BMP overview

1.BMP devices (e.g. routers) send BMP messages to a BMP collector.

2.Open Daylight (ODL) controller SQL plugin with SQL using Yang interfaces with the BMP receiver database. ODL in this fashion provides an abstract view of all BMP collector data.

3.Admins, Network Engineers, automated programs/scripts, etc. interact via ODL northbound interfaces to run various BMP analytics.

4.Admins, Network Engineers, automated programs/scripts, etc. can also go direct to the BMP database as needed

Full picture

BMP overview

There are many reasons to use BMP, but to highlight a few common ones:

1. Looking Glasses - IPv4, IPv6, and VPN4

2. Route Analytics - Track convergence times, history of prefixes as they change over time, monitor and track BGP policy changes, etc...

3. Traffic Engineering Analytics - Adapt dynamically to change and know what is the best shift

4. BGP pre-policy What-Ifs - Pre-policy routing information provides insight into all path attributes from various points in the network allowing nonintrusive what-if topology views for new policy validations

many more

Why BMP ?

BMP implementationBMP client

- ASR1K, CSR1000v : IOS-XE 3.11

- ASR9K, CRS, NCS6K, XRv: IOS-XR 5.2.2

- N3K, N7K, N9K: TBD

- Others: Juniper (older version of the draft)

BMP collector

- OpenBMP: Open Source: using MySQL

https://www.openbmp.org/

- BMP receiver: Open Source: Writen from scratch in Python https://code.google.com/p/bmpreceiver/

BMP client

• BMP receiver can be configured with both ipv4 & ipv6 host addresses.

• The BGP speaker process is referred to as the BMP Client.

• BMP client provides only pre-policy view of the ADJ-RIB-IN of a peer• Post-policy view is not supported

• A BGP peer can be monitored by multiple BMP reciever• Any update message from the peer ( irrespective of the address-family ) is sent to the

BMP receiver

• Multiple BMP receivers can be configured across all BGP instances• Each BGP instance will send update messages for peers under it to the BMP receivers

monitoring the corresponding peers

Design… (1)

BMP Client

• Maximum of 8 BMP receivers can be configured.

• No limitation on number of BGP peers to be monitored by a receiver

• No limitation on the number of clients interacting with a receiver

• Each BGP instance maintains a queue per BMP receiver

• Once the session with the BMP receiver comes up, the client starts sending the current view (unless configured not to…) and any subsequent updates.

• The BGP client is the Active side & the BMP receiver is the passive listener

Design… (2)

BMP Client

• Once the TCP connection with a BMP receiver comes up, BMP client does the following• Sends the Initiation message to BMP receiver

This contains the host name and system software info (version)

• Sends Peer-Up message for every ESTABLISHED peer

• Send Route-Refresh message to all ESTABLISHED peers (unless configured not to…)

• Peer-Down message for every peer which goes down

• Periodic Stats-Report messages

• Route-monitoring messages for any update received from the monitored peer

• Termination message if session with BMP receiver is brought down gracefully ( unconfigured )

Design … (3)

BMP Client

• When BMP client receives updates from a peer, it makes a copy of the message and add it to the queue of all BMP receivers monitoring the peer • A separate thread in BGP speaker process then flushes the updates out on the tcp

session to the BMP receiver

• A copy of the message is made for every monitoring BMP receiver

• If a TCP session with the BMP receiver cannot be established, the Client process will retry the connect every 7 secs

Design… (4)

BMPBMP messages

Type Description Implementation brief

0 Route Monitor Received update messages

1 Statistics Report No Cisco specific statistics

2 Peer Down Notification BGP peer down

3 Peer Up Notification BGP peer up

4 Initiation BMP session starts, will carry the session description and basic informations

5 Termination BMP session ends

BMP BMP Stats Reports

Stat type

0 (standard) Number of prefixes rejected by inbound policy.

1 (standard) Number of (known) duplicate prefix advertisements. Currently sending 0.

2 (standard) Number of (known) duplicate withdraws. Currently sending 0.

3 (standard) Number of updates invalidated due to CLUSTER_LIST loop.

4 (standard) Number of updates invalidated due to AS_PATH loop.

5 (standard) Number of updates invalidated due to ORIGINATOR_ID.

6 (standard) Number of updates invalidated due to AS_CONFED loop. 7 (standard) Number of routes in Adj-RIBs-In.8 (standard) Number of routes in Loc-RIB.

BMP Client

bmp receiver <1-8>

description <string>

dscp <dscp-value>

host <name | ipv4/6 address> port <1-65535>

initial-delay <1-3600> // before attempting to connect to bmp receiver

initial-refresh {delay <1-3600> spread <1-3600> | skip}

precedence <precedence-value>

shutdown

stats-reporting-period <1-3600>

update-source <interface-id>

vrf <vrf-id>

router bgp <asn>

neighbor x.x.x.x

bmp-activate receiver <1-8>

Client config.

OpenBMPOverview

1. OpenBMP use MySQL to store BMP record– use MySQL Workbench (client) to read it

2. Live OpenBMP server: – db1.openbmp.org  (login/pw  = demo/demo)

3. Few command:

Command MySQL Qweries

Show current status of BGP peers select * from v_peers order by RouterName

Show BMP Routers select * from routers

Show BGP peer statistics interval based report info

select * from v_peer_prefix_report_last

OpenBMPMySQL Worbench initial config

OpenBMPMySQL Qwery

BMP summary

1. Allows to collect all peers adj-RIB-in BGP updates

2. BMP cisco implementation is based on draft BMPv7

3. Supported on IOS-XE 3.11 and IOS XR 5.2.2

4. Easy to deploy using OpenBMP & BMPreceiver open source receiver

Question: ask-bmp@cisco.com

Thank you.

BMP Client - ShowRP/0/0/CPU0:R2#sh bgp bmp receiver 5BMP receiver 5

Host 10.0.101.1 Port 16666

Connected for 00:25:07 since Dec 31 16:25:07.269

Precedence: internet

BGP neighbors: 1

VRF: - (0x60000000)

Update Source: 9.9.9.9 (Lo9)

Update Source Vrf ID: 0x60000000, Active: Y

Message Stats:

Total messages sent: 60

INITIATION: 1

TERMINATION: 0

STATS-REPORT: 0

PER-PEER messages: 59

Neighbor 20.0.101.11

Messages pending: 0

Messages sent: 59

PEER-UP: 1

PEER-DOWN: 0

ROUTE-MON: 58

RP/0/0/CPU0:R2#sh bgp bmp receiver 1BMP receiver 1

Host 10.0.101.1 Port 16666

Connected for 00:30:40 since Dec 31 16:30:40.613

Precedence: internet

BGP neighbors: 1

VRF: - (0x60000000)

Update Source: - (-)

Update Source Vrf ID: 0x0, Active: N

Message Stats:

Total messages sent: 31

INITIATION: 1

TERMINATION: 0

STATS-REPORT: 0

PER-PEER messages: 30

Neighbor 20.0.101.11

Messages pending: 0

Messages sent: 30

PEER-UP: 1

PEER-DOWN: 0

ROUTE-MON: 29

BMP Client - Show

RP/0/0/CPU0:R2#sh bgp nei 20.0.101.11 | beg BMP

BGP Monitoring(BMP) activated for receivers:

1, 3, 5

….

BMP Client – Debug – debug bgp bmp

Type 0 - Route monitoring message:

RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: [BMP Messagw Dump : receiver (NULL), nbr 20.0.101.11, type Route Monitoring]:

RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 0300 0000 cb00 0000 0000 0000 0000 0000

RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 0000 0000 0000 0000 0000 0000 1400 650b

RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 0000 0065 1400 650b 53b3 28cd 0000 6e5c

P/0/0/CPU0:RJul 1 14:31:57 : bgp[1039]: [default-iord]: ffff ffff ffff ffff ffff ffff ffff ffff

RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 009b 0200 0000 2040 0101 0040 0304 1400

RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 650b 4002 0a02 0400 6500 c900 ca00 cbf0

RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 0800 0400 6503 e720 4702 0101 2047 0201

RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 0220 4702 0103 2047 0201 0420 4702 0105

RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 2047 0201 0620 4702 0107 2047 0201 0820

RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 4702 0109 2047 0201 0a20 4702 010b 2047

RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 0201 0c20 4702 010d 2047 0201 0e20 4702

RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 010f 2047 0201 1020 4702 0111 2047 0201

RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 1220 4702 0113 2047 0201 14

Type 1 - Stat-report message:

RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: [BMP Message Dump : receiver 10.0.101.1, nbr 20.0.101.11, type Statistics Report]:

RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: 0300 0000 6401 0000 0000 0000 0000 0000

RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: 0000 0000 0000 0000 0000 0000 1400 650b

RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: 0000 0065 1400 650b 53f2 363b 000b 20f1

RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: 0000 0005 0001 0004 0000 0090 0004 0004

RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: 0000 0028 0005 0004 0000 0038 0007 0008

RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: 0000 0000 0000 0018 0008 0008 0000 0000

RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: 0000 0018

BMP Client – Debug – debug bgp bmp

BMP Client – Debug – debug bgp bmp

Type 2 - Peer-down message:

RP/0/0/CPU0:Aug 18 11:01:14 : bgp[1039]: [default-mgmt]: [BMP Message Dump : receiver (NULL), nbr 20.0.101.11, type Peer Down Notification]:

RP/0/0/CPU0:Aug 18 11:01:14 : bgp[1039]: [default-mgmt]: 0300 0000 4602 0000 0000 0000 0000 0000

RP/0/0/CPU0:Aug 18 11:01:14 : bgp[1039]: [default-mgmt]: 0000 0000 0000 0000 0000 0000 1400 650b

RP/0/0/CPU0:Aug 18 11:01:14 : bgp[1039]: [default-mgmt]: 0000 0065 1400 650b 53f2 3f6a 000c fd3a

RP/0/0/CPU0:Aug 18 11:01:14 : bgp[1039]: [default-mgmt]: 01ff ffff ffff ffff ffff ffff ffff ffff

RP/0/0/CPU0:Aug 18 11:01:14 : bgp[1039]: [default-mgmt]: ff00 1503 0602

BMP Client – Debug – debug bgp bmp

Type 3 – Peer-up message:RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: [BMP Message Dump : receiver (NULL), nbr 20.0.101.11, type Peer Up Notification]:

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 0300 0000 ac03 0000 0000 0000 0000 0000

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 0000 0000 0000 0000 0000 0000 1400 650b

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 0000 0065 0000 0000 53f2 3fea 0005 3282

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 0000 0000 0000 0000 0000 0000 1400 0002

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 00b3 7f51 ffff ffff ffff ffff ffff ffff

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: ffff ffff 003b 0104 0064 00b4 0202 0202

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 1e02 0601 0400 0100 0102 0280 0002 0202

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 0002 0641 0400 0000 6402 0440 0200 78ff

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: ffff ffff ffff ffff ffff ffff ffff ff00

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 2d01 0400 6500 b414 0065 0b10 0206 0104

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 0001 0001 0202 8000 0202 0200

BMP Client – Debug – debug bgp bmp

Type 4 - Initiation message:

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: [BMP Message Dump : receiver 10.0.101.1, nbr (None), type Initiation Message]:

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: 0300 0000 6b04 0001 005b 4369 7363 6f20

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: 494f 5320 5852 2053 6f66 7477 6172 652c

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: 2056 6572 7369 6f6e 2035 2e32 2e32 2e31

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: 3849 5b44 6566 6175 6c74 5d0a 436f 7079

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: 7269 6768 7420 2863 2920 3230 3134 2062

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: 7920 4369 7363 6f20 5379 7374 656d 732c

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: 2049 6e63 2e00 0200 0252 32

BMP Client – Debug – debug bgp bmp

Type 5 - Termination message:RP/0/0/CPU0:Aug 18 09:37:33 : bgp[1039]: [default-mgmt]: [10.0.101.1] receiver close called with reason 0x3 [state ESTAB]

RP/0/0/CPU0:Aug 18 09:37:33 : bgp[1039]: [default-mgmt]: [BMP Message Dump : receiver 10.0.101.1, nbr (None), type Termination Message]:

RP/0/0/CPU0:Aug 18 09:37:33 : bgp[1039]: [default-mgmt]: 0300 0000 1f05 0000 000f 636f 6e66 6967

RP/0/0/CPU0:Aug 18 09:37:33 : bgp[1039]: [default-mgmt]: 2073 6875 7464 6f77 6e00 0100 0200 00

RP/0/2/CPU0:Aug 18 09:37:33 : bgp[1039]: [default-mgmt]: [10.0.101.1] receiver close called with reason 0x3 [state NONE]