Cisco Catalyst 6500 Switch Architectured2zmdbbm9feqrf.cloudfront.net/2013/usa/pdf/BRKARC-… ·...

Cisco Catalyst 6500 Switch Architecture BRKARC-3465

Scott Hodgdon

Senior Technical Marketing Engineer

© 2013 Cisco and/or its affiliates. All rights reserved. BRKARC-3465 Cisco Public

Session Goal

To provide you with a thorough understanding of the Catalyst® 6500 switching architecture, packet flow, forwarding engine functions, and key feature operations.

3


Agenda

Chassis and Power Supplies

Supervisor Engine and Switch Fabric Architectures

Module Architectures

Layer 2 Forwarding

IP Unicast Forwarding

NetFlow

Access Control Lists

Packet Walks

4


Catalyst 6500 E-Chassis Family a

5

6509-V-E 6513-E 6509-E 6506-E 6504-E 6503-E

7 Chassis Members – From 3 Slot to 13 Slot


Catalyst 6500 E-Series Chassis Inside the Chassis

6

BU

S

FABRI

C

Supervisor

32/720/2T

Slots

Linecard

Slots

Linecard

Slots


Catalyst 6500 Switch Backplanes Catalyst Bus and Fabric Overview

7

Classic (32Gb) BUS Backplane

DBUS

RBUS

EOBC

Linecard Linecard Linecard

Data Bus (DBUS) allows L/C to forward data to Supervisor for forwarding decision Results Bus (RBUS) returns forwarding result from Supervisor back to L/C Ethernet Out of Band Channel (EOBC) provide out of band management between Supervisor and LC

720Gb / 2Tb Crossbar Backplane

CROSSBAR

Linecard Linecard

Crossbar is a matrix of “N” channels to provide a data path between linecards Sup720 supports 18 channels at 8G/20G per channel (speed autodetected) Sup2T supports 26 channels at 20G/40G per channel (speed autodetected)


Slot 6503/

6503-E

6504-E 6506/

6506-E

6509/

6509-E

6509-

NEBS-A

6509-V-E 6513 6513-E

1 Dual Dual Dual Dual Dual Dual Single Dual



4 - Dual Dual Dual Dual Dual Single Dual

5 - - Dual Dual Dual Dual Single Dual

6 - - Dual Dual Dual Dual Single Dual

7 - - - Dual Dual Dual Single Dual

8 - - - Dual Dual Dual Single Dual

9 - - - Dual Dual Dual Dual Dual

10 - - - - - - Dual Dual

11 - - - - - - Dual Dual

12 - - - - - - Dual Dual

13 - - - - - - Dual Dual

Catalyst 6500 Linecard Slot Support a

8

In order to take advantage

of the dual fabric channels

in slots 1 – 8 of the 6513-E

chassis, the Supervisor 2T

is required.

With any version of the

Supervisor 720, the 6513-E

fabric channel distribution Is

the same as the 6513.


Slot 6503

6503-E 6504-E

6506

6506-E

6509

6509-E

6509-

NEBS-A 6509-V-E 6513 6513-E

1 Sup/LC Sup/LC LC LC LC LC LC LC

2 Sup/LC Sup/LC LC LC LC LC LC LC

3 LC LC LC LC LC LC LC LC

4 - LC LC LC LC LC LC LC

5 - - Sup/LC Sup/LC Sup/LC Sup/LC LC LC

6 - - Sup/LC Sup/LC Sup/LC Sup/LC LC LC

7 - - - LC LC LC Sup/LC Sup

8 - - - LC LC LC Sup/LC Sup

9 - - - LC LC LC LC LC

10 - - - - - - LC LC

11 - - - - - - LC LC

12 - - - - - - LC LC

13 - - - - - - LC LC

For Your Reference

Catalyst 6500 Supervisor Slot Support a

9


Power Supplies for six, nine and thirteen

slot chassis are located at front bottom of

chassis

Power Supplies for three and four slot

chassis are located in the rear

AC Power Supplies DC Power Supplies

1400W

2700W

3000W

4000W

6000W

8700W

950W

2500W

2700W

4000W

6000W

BLUE = 6503E

RED = 6504E

BLACK = 6506E, 6509E,

6509-V-E, 6513

and 6513-E

For Your Reference

Catalyst 6500 Power Supplies a

10


Three “backplanes” exist in 6500

1. Ethernet Out of Band Channel for chassis control

2. 32G Shared “Classic Bus” for legacy linecards

3. Switch Fabric (720G or 2T) for fabric linecards

11


Redundant Mode

• Each supply provides ~50% of power needs

• Neither supply operates at >60% or <40%

capacity

• Either supply can power the system on its

own

• This is BEST PRACTICE

Power Supply 1 Power Supply 2

Catalyst 6500

Combined Mode

• Each supply provides up to 83% of its

capacity

• Total power available is 167% o a single

supply

• A single supply may not power the whole

system

• NOT the recommended mode for production

Power Supply 1 Power Supply 2

Catalyst 6500

Power Supply Redundancy Modes of Operation

12

Use the Cisco Power

Calculator

on cisco.com to

determine which

supplies and which

mode of operation is

needed for your

system.


Agenda




Layer 2 Forwarding


NetFlow


Packet Walks

13


Catalyst 6500 Supervisors Supervisor 720-10G: Some Facts

14

Supervisor 720-10G Quick Facts

Integrated 720Gbps Switch Fabric

Integrated Policy Feature Card 3 (PFC3) supporting

hardware acceleration for select features

Integrated Multilayer Switch Feature Card 3 (MSFC3)

supporting two CPUs for Layer 2 and Layer 3

functionality

IPv6 unicast and multicast forwarding support in

hardware

Virtual Switching System (VSS) support

All uplinks can be active in systems with redundant

Supervisors

(more information in the notes)

Supervisor 720-10G 3C / 3CXL


Crossbar Fabric Channels

Switch

Fabric

RP

Flash

RP

DRAM

SP

Flash

SP

DRAM

RP

SP

1

G

1

G

MSFC3 1

G

Port

ASIC

Fabric /

Replication

ASIC

Port

ASIC

Classic BUS

MET

FIB TCAM

QOS ACL

Security ACL

Counters

Netflow TCAM

Netflow Table

Netflow Stats

Policy Feature Card

(PFC3)

Layer 2 FWD Engine

L2 CAM

FIB Table

Adjacency

Adj Stats

Layer 3

FWD

Engine

Supervisor 720 3A / 3B / 3BXL Block Diagram

15

Dbus

Rbus EOBC

For Your Reference

© 2013 Cisco and/or its affiliates. All rights reserved. BRKARC-3465 Cisco Public 16


Switch

Fabric Fabric

ASIC

Fabric /

Replication

ASIC

RP

Flash

RP

DRAM

SP

Flash

SP

DRAM

RP SP

1

G

MSFC3

Classic BUS

Port

ASIC

Port

ASIC

MET

Layer

2/3

FWD

Engine

Policy Feature Card

(PFC3)

L2 CAM FIB TCAM

QOS ACL

Security ACL

Counters

Netflow TCAM

Netflow Table

Netflow Stats

FIB Table

Adjacency

Adj Stats

10

G

10

G 1

G

1

G

Quad Port PHY

Dbus

Rbus EOBC

20Gbps

Supervisor 720-10G Block Diagram


Catalyst 6500 Supervisors Supervisor 2T: Some Facts

17

Supervisor 2T Quick Facts

Integrated 2-Tbps Switch Fabric

Integrated Policy Feature Card 4 (PFC4) supporting

hardware acceleration for select features

Integrated Multilayer Switch Feature Card 5 (MSFC5)

supporting a single CPU for L2 and L3 functionality

Connectivity Management Processor (CMP) for

improved management capability

One external compact flash slot (power controlled by

IOS)

All uplinks can be active in systems with redundant

Supervisors

(more information in the notes)

Supervisor 2T

PFC4 / PFC4XL



Switch

Fabric Fabric

ASIC

Fabric /

Replication

ASIC

DRAM Flash

CPU

1G MSFC5

Classic BUS

Port

ASIC

Port

ASIC

MET

Layer

2/3

FWD

Engine

Policy Feature Card (PFC4)

L2

C

A

M

FIB TCAM

QOS ACL

Security

ACL

Counters

Netflow

TCAM Netflow

Table Netflow

Stats

FIB Table

Adjacency

Adj Stats

10G 10G 1G 1G

Quad Port PHY

PFC4

NetFlow

L2

Forwarding

Engine

L2 CAM (128K)

LIF DB

LIF Stats

ACE Counter

CL1 TCAM

CL2 TCAM

FIB

ADJ

RPF Table

LIF Table

L3/4

Forwarding

Engine

Supervisor 2T PFC4 / PFC4XL Block Diagram

18

Dbus

Rbus EOBC

20Gbps


Supervisor Chassis Requirements Switch Fabric

19

Supervisor 720-10G

Supervisor 2Ts

Chassis All E-Series

All non-E Series Only E-Series

Fan Trays

E-Fans for E-Series

Fan2 for non-E Series

E-Fans for E-Series

Power Supplies 2500W AC / DC or greater

Supervisor Slots

3-Slot : 1 and 2

4-slot : 1 and 2

6-slot : 5 and 6

9-slot : 5 and 6

13-slot : 7 and 8

E-Fan cannot be used in non-E Series Chassis

Fan2 cannot be used in E-Series

With Supervisor 2T and 6513-E, only Supervisors are allowed in the Supervisor Slots

With Supervisor 720 and 6513-E, the fabric channel distribution is the same as with Supervisor 720 and 6513.


Catalyst 6500 Supervisors Switch Fabric

20

The Supervisor 720 and Supervisor 2T support a Switch Fabric which offers

each connected linecard a set of discrete communication paths into the switch

backplane…

Linecard

Slot #3

Linecard

Slot #4

Supervisor

Slot #5

Linecard

Slot #6

Linecard

Slot #7

Linecard

Slot #2

Linecard

Slot #1

Linecard

Slot #9

Linecard

Slot #8

Data

Flows


Catalyst 6500 Supervisor 720-10G The 720Gbps Switch Fabric

21

-Integrated 720Gbps Switch Fabric

-Provides backplane interconnects between linecards

-Fabric Traces are distributed across each linecard slot

- Each Fabric Trace can run at 8Gb/sec OR 20Gb/sec

Switch Fabric


Catalyst 6500 Supervisor 2T The 2Tbps Switch Fabric

22

Switch Fabric

- Integrated 2Tbps Switch Fabric

- 26 Channels to support the 6513-E

-Provides backplane interconnects between linecards

-Fabric Traces are distributed across each linecard slot

- Each Fabric Trace can run at 20Gb/sec OR 40Gb/sec


Checking Fabric Utilization Checking Fabric Utilization

23

6509E#show platform hardware capacity fabric

Switch Fabric Resources

Bus utilization: current: 25%, peak was 75% at 19:28:31 UTC Mon Feb 2 2012

Fabric utilization: Ingress Egress

Module Chanl Speed rate peak rate peak

1 0 20G 10% 50% @13:49 06Jan12 20% 50% @13:49 06Jan12

1 1 20G 20% 50% @13:49 06Jan12 10% 50% @13:49 06Jan12

2 0 20G 0% 1% @20:30 13Jan12 0% 1% @20:46 06Jan12

2 1 20G 0% 1% @20:47 16Jan12 0% 1% @16:52 06Jan12

3 0 20G 20% 40% @13:49 06Jan12 0% 0% @13:49 06Jan12

6 0 20G 0% 1% @17:44 06Jan12 0% 1% @00:36 08Jan12

8 0 8G 0% 3% @16:33 12Feb12 50% 100% @13:49 06Jan12


Switch Fabric = Switch Backplane

A set of dedicated fabric channels, which interconnect all cards…

24


MSFC5

MSFC3

MSFC Serves as Control Plane for 6500

Supervisors 720 and 32 have Two CPU’s –

SP and RP SP serves as L2 control plane

RP serves as L3 control plane

Supervisor 2T has One CPU Single CPU performs L2 and L3 functions

CMP on MSFC5 provides CPU,

file system, and boot management

Local Bootflash holds IOS images Only SP Bootflash holds Native IOS images

for Supervisor 720

Config held in NVRAM

Catalyst 6500 Multilayer Switch Feature Card Checking Fabric

25


The Connectivity Management Processor (CMP) supports

new capabilities that will aid Network Administrators in

managing the system:

CPU Image Recovery

- TFTP boot of the system

CPU File Transfer

- Image via TFTP

Remote CPU Reset - Hard or Soft reset

CPU Console Logging - Record CPU console log for troubleshooting

USB Support - USB serial console access

Catalyst 6500 Supervisor 2T MSFC5: Connectivity Management Processor (CMP)

26


REFERENCE : MSFC3 vs. MSFC5

27

Feature MSFC3 (Supervisor 720) MSFC5 (Supervisor 2T)

CPU Speed SP CPU – 600Mhz RP CPU – 600Mhz

Dual core with each core @ 1.5Ghz

Number of CPU cores 1 2

DRAM SP CPU – Up to 1GB RP CPU – Up to 1GB

1 x 2GB (default)

2 x 2GB (upgrade option)

Connectivity Management

Processor (CMP) CPU No

Single core @ 266Mhz

32MB Boot Flash

256MB System Memory

NVRAM 2MB 4MB

OBFL Flash No 4MB

Bootflash / Bootdisk SP CPU – 1GB (CF)

RP CPU – 64MB (flash) 1GB (CF)

External CF slot 1 1

For Your Reference


MSFC = Multilayer Switch Feature Card

It’s the Software “Control Plane” for the Switch, where IOS runs…

28


Catalyst 6500 Policy Feature Card Overview of PFC3 and PFC4

29

PFC3

PFC4

PFC Serves as Data Plane for 6500

Two primary ASICs – L2 and L3

TCAM’s used for high speed lookup into Forwarding (FIB), ACL (Security and QoS) and Netflow Tables

PFC3 – 48Mpps Maximum Forwarding

PFC4 – 60Mpps Maximum Forwarding

Common features supported in hardware by PFC3 and PFC4 include:

IPv4 - IPv6 - MPLS - Multicast - Policing - Classification - RACL - VACL - PACL - GRE - Tunneling - URPF - Control Plane Policing - and more

Features introduced by the PFC4 include: Flexible NetFlow - ACL Dry Run - ACL Hitless Commit - Cisco TrustSec – VPLS - Egress NetFlow - IPv6 uRPF - Roles Based Access Control – 512K Multicast Routes – Improved EtherChannel Hash – and more


REFERENCE: PFC3 vs. PFC4

30

Feature PFC3B/BXL PFC3C/CXL PFC4/XL

IPv4 Forwarding Up to 30Mpps Up to 48Mpps Up to 60Mpps

IPv6 Forwarding Up to 15Mpps Up to 24Mpps Up to 30Mpps

FIB TCAM IPv4 256K / 1M 256K / 1M 256K / 1M

FIB TCAM IPv6 128K / 500K 128K / 500K 128K / 500K

Adjacency Table 1M 1M 1M

Netflow Table Up to 256K (XL) Up to 256K (XL) Up to 1M (XL)

(Ingress 512K : Egress 512K)

MAC Table 64K (32K) 96K (80K) 128K

Egress Netflow No No Yes

Flexible Netflow No No Yes

MPLSoGRE No No Yes

IPv6 uRPF No No Yes

For Your Reference


Feature PFC3B/BXL PFC3C/CXL PFC4/XL

ACL Labels 4K 4K 16K

Security ACEs Up to 32K Up to 32K Up to 192K (XL Default)

QoS ACEs Up to 32K Up to 32K Up to 64K (XL Default)

Port ACLs 2K 2K 8K

Aggregate Policers 1023 1023 6K

Shared Microflow Policers 63 63 512

Egress Microflow Policing No No Yes

Distributed Policers No No Yes

Packet or Byte Based

Policing

No No Yes

RPF Interfaces 2 2 16

Native VPLS No No Yes

VSS No Yes Yes

REFERENCE: PFC3 vs. PFC4

31

For Your Reference


PFC = Policy Feature Card

It’s the Hardware “Control Plane” for the Switch, based on

the information learned by MSFC…

32


Agenda




Layer 2 Forwarding


NetFlow


Packet Walks

33


Port

ASIC

Dbus

Rbus

EoBC

Ingress and Egress packet queuing

and scheduling is done in the Port

ASIC

All other functions (Lookups, Policing,

Replication, etc) are performed on the

Supervisor

There is no connection to the Switch

Fabric

Packets destined to fabric-attached

modules utilize the Supervisor’s

switch fabric connection

Lin

ecard

Catalyst 6500 Classic Module Architecture a

34


Port

ASIC

EoBC

Port

ASIC

Port

ASIC

Port

ASIC Replication

ASIC

Dbus

Rbus 8Gb Fabric Channel to Switch Fabric

Dbus

Rbus

CEF256 provides connection to Bus

and Switch Fabric

Ingress and Egress packet queuing

and scheduling is done in the Port

ASIC

Can use either Bus or Fabric for data

transmission

Local replication ASIC for multicast

and SPAN replication

Lin

ecard

Fabric ASIC

Catalyst 6500 CEF256 Module Architecture a

35

For Your Reference


Port

ASIC

EoBC

Port

ASIC

Port

ASIC

Port

ASIC Replication

ASIC

Dbus

Rbus

dCEF256 adds local distributed

forwarding linecard (DFC3)

DFC3 contains same forwarding

ASICs as PFC

DFC3 provides local switching @

48Mpps

No need for DBus or RBus when

DFC3 in used

DFC3

8Gb Fabric Channel to Switch Fabric

L2

L3 Lin

ecard

Fabric ASIC

Catalyst 6500 dCEF256 Module Architecture a

36

For Your Reference


Port

ASIC

EoBC

Port

ASIC

Dbus Rbus

Centralized

Forwarding Card

Port

ASIC

Port

ASIC

20Gbps Fabric

Channel

20Gbps Fabric

Channel

CEF720 has no local forwarding

Uses CFC card to forward Packet header

to Supervisor over BUS for forwarding

lookup

Ingress and Egress packet queuing and

scheduling is done in the Port ASIC

Data sent over fabric channel to destination

linecard

Lin

ecard

Fabric and

Replication

ASIC

Fabric and

Replication

ASIC

Catalyst 6500 CEF720 Module Architecture a

37


Catalyst 6500 dCEF720 Module Architecture a

38

Port

ASIC

Port

ASIC

Port

ASIC

Port

ASIC

20Gbps Fabric

Channel

20Gbps Fabric

Channel

dCEF720 uses DFC3 / DFC4 for local

forwarding

Module has no connection to Dbus or

Rbus

DFC3 / DFC4 contains same hardware and logic

as PFC3 / PFC4 on Supervisor

Ingress and Egress packet queuing and

scheduling is done in the Port ASIC

Lin

ecard

….

.

….

.

EoBC

Distributed

Forwarding Card

L2 FWD

L3 FWD

Fabric and

Replication

ASIC

Fabric and

Replication

ASIC


Ingress and Egress packet queuing and scheduling is

done in the Port ASIC

CTS ASICs provide wire-rate encryption / decryption

Catalyst 6500 dCEF2T Module Architecture WS-X6908-10G / -10G-XL

39

40G 40G

Switch Fabric

PORT

ASIC

FPGA FPGA

Supervisor

CTS

ASIC

PORT

ASIC

PORT

ASIC

PORT

ASIC

PORT

ASIC

PORT

ASIC

PORT

ASIC

PORT

ASIC

CTS

ASIC

CTS

ASIC

CTS

ASIC

CTS

ASIC

CTS

ASIC

CTS

ASIC

CTS

ASIC

FIRE

ASIC

FIRE

ASIC

FIRE

ASIC

DFC4

FABRIC INTERFACE

FIRE

ASIC

dCEF2T uses DFC4 for local forwarding and other operations (ACL,

NetFlow, QoS, MPLS, etc)

Linecard has no connection to Rbus or Dbus


Ingress ASIC Egress ASIC Ingress ASIC Egress ASIC

Port FPGA / CTS Port FPGA / CTS

40 G CFP - Port 1 40 G CFP - Port 2 40 G CFP - Port 3 40 G CFP - Port 4

S

F

P

+

5

S

F

P

+

6

S

F

P

+

7

S

F

P

+

8

S

F

P

+

9

S

F

P

+

1

0

S

F

P

+

1

1

S

F

P

+

1

2

S

F

P

+

1

7

S

F

P

+

1

8

S

F

P

+

1

9

S

F

P

+

2

0

S

F

P

+

1

3

S

F

P

+

1

4

S

F

P

+

1

5

S

F

P

+

1

6

CFP Daughter Card

DFC4

Catalyst 6500 dCEF2T Module Architecture WS-X6904-40G / -40G-XL

40

Replication

Engine

Fabric Interface

ASIC

Fabric ASIC

Replication

Engine

Fabric Interface

ASIC

Replication

Engine

Fabric Interface

ASIC

Replication

Engine

Fabric Interface

ASIC

Interface

ASIC

Interface

ASIC

Switch Fabric Supervisor

40G 40G


Catalyst 6500 Module Architecture Centralized Forwarding Cards (CFC)

41

The Centralized Forwarding Card (CFC) provides

BUS connectivity for the CEF720 linecards…

The CFC is available only for certain

CEF720 modules:

WS-X6704-10GE

WS-X6724-SFP

WS-X6748-SFP

WS-X6748-GE-TX

The CFC provides the connection to the

Dbus and Rbus

The CFC is used to communicate with the

Supervisor when centralized forwarding is

used


Catalyst 6500 Module Architecture Distributed Forwarding Card 3 (DFC3)

42

The DFC3 supports forwarding rates up to

48Mpps

The DFC3 stores a local copy of the

forwarding table, as well as Security and

QoS ACL’s that are centrally defined

The DFC3 IS field upgradeable and is

supported only with Sup720

Three different versions of the DFC3

are supported…

DFC3A

DFC3B/DFC3BXL

DFC3C/DFC3CXL


Catalyst 6500 Module Architecture Distributed Forwarding Card 4 (DFC4)

43

The DFC4 supports forwarding rates up to

60Mpps

The DFC4 also stores a local copy of the

forwarding tables, as well as Security and

QoS ACL’s that are centrally defined

The DFC4 is located underneath a protective

cover that protects the daughtercard from

getting damaged when the linecard is

inserted or removed from a chassis

The DFC4 IS field upgradable

Two different versions of the DFC4 are

supported…

DFC4-A / AXL

DFC4-E / EXL


Catalyst 6500 Module Architecture DFC3/4 Interoperability with PFC3/4

44

DFC3s work only with PFC3s, and DFC4s work only with PFC4s.

When mixing DFCs and PFCs of different capabilities, the lower common denominator is in effect:

Example 1 : A PFC3BXL on the Supervisor with a DFC3B on the module will result in the PFC3BXL running in PFC3B mode.

Result : The larger FIB and NetFlow tables of the XL will not be used as they will need to be programmed to match the smaller tables sizes of the non-XL.

Example 2: A PFC3C on the Supervisor with a DFC3B on the module will result in the PFC3C running in PFC3B mode.

Result : The VSS capability of the PFC3C will be disabled when it runs in PFC3B mode since PFC3B mode does not support VSS.

Mixing of different PFCs in the same chassis is not supported.

When inserting a module with a lower level DFC than the PFC on the Supervisor, the system must be reloaded for the PFC to reprogram itself to the lower mode.


RESOURCE : Catalyst 6500 Modules DFC3/4 Interoperability with PFC3/4

45

PFC3A PFC3B PFC3BXL PFC3C PFC3CXL PFC4 PFC4XL

DFC3A Operate

as PFC3A Operate

as PFC3A Operate as

PFC3A Operate

as PFC3A X X

DFC3B Operate

as DFC3A Operate

as PFC3B Operate as

PFC3B Operate

as PFC3B X X

DFC3BXL Operate

as DFC3A Operate

as DFC3B

Operate as PFC3B

and DFC3B

Operate as

PFC3BXL X X

DFC3C Operate

as DFC3A Operate

as DFC3B

Operate as PFC3B

and DFC3B

Operate as PFC3C

X

X

DFC3CXL Operate

as DFC3A Operate

as DFC3B

Operate as

DFC3BXL

Operate as DFC3C

X X

DFC4 X X X X X Operates as

PFC4

DFC4XL X X X X X Operates as DFC4

For Your Reference


Mixing Linecard Types Flow Through Mode

46

CLASSIC

LINECARD

SUPERVISOR

CLASSIC

LINECARD

Used for traffic between classic (non-fabric)

modules, and for traffic between a Classic and

the Supervisor…

100% centralized performance @ 15Mpps

DBUS

RBUS

EOBC


Mixing Linecard Types Truncated Mode

47

CLASSIC

LINECARD

SUPERVISOR

FABRIC

LINECARD

FABRIC

LINECARD

DBUS

RBUS

EOBC

Used for traffic between fabric-enabled linecards,

when a non-fabric (classic) linecard is installed.

In this mode, centralized forwarding reverts back to

15Mpps.

SWITCH FABRIC


Mixing Linecard Types Compact Mode

48

SUPERVISOR

FABRIC

LINECARD

FABRIC

LINECARD

DBUS

RBUS

EOBC

Used when only ALL fabric-enabled linecards

used in a chassis.

This mode uses a compact form of DBUS

header which optimizes centralized lookup

performance at 30Mpps

SWITCH FABRIC


CFC or DFC = Centralized or Distributed

CFC connects to DBUS and RBUS so that the PFC can perform forwarding lookups

DFC enables local (distributed) forwarding lookups on each linecard

49


Agenda




Layer 2 Forwarding


NetFlow


Packet Walks

50


Catalyst 6500 Internals L2 Forwarding Steps

L2 flooding

No

L2 forwarding Yes

Known MAC?

No

L3 forwarding Yes

Update entry

No

Layer 2 Table

Learn Yes

Layer 2 Table

Router MAC? New MAC?

Frame received

Source MAC

Lookup

Destination MAC

Lookup Layer 2 Table Layer 2 Table

51


MAC Table

16, 24,

or 32

pages 4096

rows PFC

The PFC has an

integrated CAM Table

that supports 4096

rows * X pages =

MAC address space

PFC3B/BXL = 16 pages (64K entries)

PFC3C/CXL = 24 pages (96K entries)

PFC4/XL = 32 pages (128K entries)

MAC Table

Table MAC

A

B

C

D

E

F

Port

1

2

3

4

5

6

Catalyst 6500 Internals Layer 2 Table Structure

52


0000.2222.7777 | 20

0000.1111.cccc | 10

0000.bbbb.ac1c | 30

0000.dddd.a112 | 30

Frame

VLAN MAC

Hash

MAC Table Row

HIT!!! 1. Hash result identifies the starting Page and Row in MAC table

2. Lookup key (VLAN + MAC) compared to contents of indexed line on each page, sequentially

3. Destination lookup: Match returns destination interface(s), Miss results in Flood

4. Source lookup: Match updates age of matching entry, Miss installs new entry in table

PFC

16, 24,

or 32

Pages

MAC Table

4096

Rows

Catalyst 6500 Internals Layer 2 Forwarding Operation

53


Displaying the Layer 2 Table a

54

6513E.SUP2T.SA.2#show mac address-table

Legend: * - primary entry

age - seconds since last seen; n/a - not available; S - secure entry;

R - router's gateway mac address entry; D - Duplicate mac address entry

Displaying entries from active supervisor:

vlan mac address type learn age ports

----+----+---------------+-------+-----+----------+-----------------------------

* 192 00d0.0053.bc00 dynamic Yes 5 Gi7/3

R 205 0024.c4dc.d740 static No - Router

R 20 0024.c4dc.d740 static No - Router

* 192 0014.5e31.4220 dynamic Yes 65 Gi7/3

* 60 00d0.2bfc.23f5 dynamic Yes 30 Gi5/14

* 192 00e0.1e5d.e9ff dynamic Yes 30 Gi7/3


Catalyst 6500 Internals EtherChannel

55

Combines multiple physical interfaces into ONE logical interface

EtherChannel Load Sharing Deterministic

PFC3 algorithm supports 8 results (3 bits)

PFC4 algorithm supports 256 results (8 bits)

Load Sharing is by flow and NOT per packet

EtherChannel can be configured for L2 and L3 interfaces


E/Chan

Bundle

Link1 Link2 Link3 Link4 Link5 Link6 Link7 Link8

2 Links 50% 50% -- -- -- -- -- --

3 Links 37.5% 37.5% 25% -- -- -- -- --

4 Links 25% 25% 25% 25% -- -- -- --

5 Links 25% 25% 25% 12.5% 12.5% -- -- --

6 Links 25% 25% 12.5% 12.5% 12.5% 12.5% -- --

7 Links 25% 12.5% 12.5% 12.5% 12.5% 12.5% 12.5% --

8 Links 12.5% 12.5% 12.5% 12.5% 12.5% 12.5% 12.5% 12.5%

Frame 1 2 3 4 5 6 7 8

EtherChannel Hash 3 bit result

Even Distribution for Flows is for those cases highlighted in RED

EtherChannel “Power-of-2” Ports PFC3 Flow Distribution

56


E/Chan

Bundle

Link1 Link2 Link3 Link4 Link5 Link6 Link7 Link8

2 Links 50% 50% -- -- -- -- -- --

3 Links 33.6% 33.2% 33.2% -- -- -- -- --

4 Links 25% 25% 25% 25% -- -- -- --

5 Links 20.4% 19.9% 19.9% 19.9% 19.9% -- -- --

6 Links 16.8% 16.8% 16.8% 16.8% 16.4% 16.4% -- --

7 Links 14.5% 14.5% 14.5% 14.5% 14% 14% 14% --

8 Links 12.5% 12.5% 12.5% 12.5% 12.5% 12.5% 12.5% 12.5%

1 2 3 256

Even Distribution for Flows is for those cases highlighted in RED

………

..

EtherChannel “Power-of-2” Ports PFC4 Flow Distribution

57

Frame

EtherChannel Hash 8 bit result


Reference: PFC3 EtherChannel Inputs a

dst-ip Destination IP Address dst-mac Destination Mac Address dst-mixed-ip-port Destination IP Address and TCP / UDP Port * dst-port Destination TCP/UDP Port mpls Load Balancing for MPLS packets src-dst-ip Source XOR Destination IP Address src-dst-mac Source XOR Destination Mac Address src-dst-mixed-ip-port Source XOR Destination IP Address abd TCP / UDP Port * src-dst-port Source-Destination TCP/UDP Port src-ip Source IP Address src-mac Source Mac Address src-mixed-ip-port Source IP Address and TCP / UDP Port * src-port Source TCP/UDP Port * Requires 12.2(33)SXH or newer and PFC3C or PFC3CXL mode

EtherChannel Uses a Load Balancing Algorithm to Determine which Link in the Bundle to Use—the Inputs to

the Algorithm Are a Combination of L2, L3 or L4 addresses

For Your Reference


Reference: PFC4 EtherChannel Inputs a

dst-ip Dst IP Addr dst-mac Dst Mac Addr dst-mixed-ip-port Dst IP Addr and TCP/UDP Port dst-port Dst TCP/UDP Port mpls Load Balancing for MPLS packets src-dst-ip Src XOR Dst IP Addr src-dst-mac Src XOR Dst Mac Addr src-dst-mixed-ip-port Src XOR Dst IP Addr and TCP/UDP Port src-dst-port Src XOR Dst TCP/UDP Port src-ip Src IP Addr src-mac Src Mac Addr src-mixed-ip-port Src IP Addr and TCP/UDP Port src-port Src TCP/UDP Port vlan-dst-ip Vlan, Dst IP Addr vlan-dst-mixed-ip-port Vlan, Dst IP Addr and TCP/UDP Port vlan-src-dst-ip Vlan, Src XOR Dst IP Addr vlan-src-dst-mixed-ip-port Vlan, Src XOR Dst IP Addr and TCP/UDP Port vlan-src-ip Vlan, Src IP Addr vlan-src-mixed-ip-port Vlan Src IP Addr and TCP/UDP Port

EtherChannel Uses a Load Balancing Algorithm to Determine which Link in the Bundle to Use—the Inputs to

the Algorithm Are a Combination of L2, L3 or L4 addresses

For Your Reference


Agenda




Layer 2 Forwarding


NetFlow


Packet Walks

60


Catalyst 6500 IP Unicast Forwarding

This session covers IP Unicast forwarding.

There is a dedicated Breakout Session at Cisco Live for IP Multicast

Forwarding with the Catalyst 6500:

BRKARC-3322 Catalyst 6500 IP Multicast Architecture


4K VLAN POOL

CoPP Etc

VLANs L3 Ports

SVI Tunnels

• VLANs used for both L2 bridging

and L3 routing

• L3 interfaces internally consume

VLANs from the 4K VLAN pool

Supervisor 2T

Catalyst 6500 Interface Management a

62

Supervisor 720

16K Bridge

Domains

VLAN 1…4K

128K Logical

Interfaces

• Separate L2 bridging and L3 routing

• Break the 4K VLAN barrier

• Allows VLAN reuse on a per port basis

• Massive scale of L3 interfaces

VLAN 1…4K

VLAN 1…4K CoPP Etc

L3 Ports

SVI Tunnels


L3

Engine

L2 MAC Table 1

Netflow TCAM

Netflow Table

Netflow Statistics

Adjacency Statistics

FIB TCAM &

SSRAM

Security ACL

TCAM

QoS ACL TCAM

Adjacency Table

L2 Engine 2 IP Packet Parse

3 IP Packet Parse

4

4

4

5

6

7

8

8

8

Catalyst 6500 PFC3/DFC3 Lookup Process a

63


L3 Engine

GV IF RP CL1

CL2

NF RI PL L3

PO

Packet Header

L2 Engine

IFE process:

1.IF: Get Port and Ingress LIF QoS info

2.RP: Src FIB Lookup, Source QoS

3.CL1: Ingress ACL TCAM Lookup

4.CL2: Select Ingress Class and Policy

5.NF: Ingress NetFlow lookup

6.L3: Dst FIB Lookup, Dst QoS

7.PL: Apply Ingress Policing and Marking

Architecturally, the PFC/DFC4 is almost the same as the PFC/DFC3

What changes is the Dual-Cycle Input (IFE) and Output (OFE) Processing

Here we perform the Input Forwarding Engine (IFE) pass...

Catalyst 6500 PFC4/DFC Lookup Process Input Forwarding Engine Lookup

64


L3 Engine

OFE process:

1.IF: Get Egress LIF QoS info

2.CL1: Egress ACL TCAM lookup

3.CL2: Select Egress Policy and Class

4.NF: Select NF Egress Policy and Class

5.PL: Apply Egress Policing and Marking

6.RI: Generate RBUS result

GV IF CL1

CL2

NF RI PL

PO

RBUS Result

L3

RP

L2 Engine

Architecturally, the PFC/DFC4 is almost the same as the PFC/DFC3

What changes is the Dual-Cycle Input (IFE) and Output (OFE) Processing

Here we perform the Output Forwarding Engine (OFE) pass...

Catalyst 6500 PFC4/DFC Lookup Process Output Forwarding Engine Lookup

65


Hardware Based CEF Process

1. FIB lookup based on destination prefix (longest-match)

2. FIB “Hit” returns Adjacency pointer

3. Adjacency contains Rewrite (next-hop) information

4. ACL, QoS & NetFlow lookups occur in parallel, and effect final result

Routing Protocols OSPF, EIGRP, ISIS, BGP, etc

Routing Protocols receive routing updates

from the network... Control Plane (RP)

Holds routing tables in

Routing information Base

(RIB) from Static Routes

and all running

Routing Protocols

Software CEF Takes RIB and builds a

Forwarding

Information Base (FIB)

containing IP/mask

prefixes

Hardware CEF Loads FIB into PFC

& distributes to DFC’s

FIB (on PFC/DFC)

FIB & ADJ tables are used by

EARL to perform L3 lookups

and forwarding

Catalyst 6500 IP Unicast Forwarding Layer 3 Forwarding on PFC

66


IF, MACs, MTU

IF, MACs, MTU

IF, MACs, MTU

IF, MACs, MTU

FIB

TCAM

Adjacency

Table

Located on the PFC are the “FIB” and “Adjacency Table”…

The FIB contains:

L3 entries are arranged logically from MOST to LEAST specific (based on /mask)

Overall FIB hardware shared by:

– IPv4 Unicast

– IPv4 Multicast

– IPv6 Unicast

– IPv6 Multicast

– MPLS

The Adjacency Table:

– L2 “Re-Write” information and / or pointers for replication

– Hardware adjacency table also shared among protocols

Catalyst 6500 IP Unicast Forwarding Layer 3 Forwarding on PFC

67

10.1.0.0

172.20.45.1

10.1.1.100

…

10.1.3.0

10.1.2.0

MASK (/24)

MASK (/32)

…

MASK (/16)

172.16.0.0

…

0.0.0.0

MASK (/0)


10.1.0.0

172.16.0.0

…

172.20.45.1

10.1.1.100

…

10.1.3.0

10.1.2.0

…

0.0.0.0

MASK (/24)

MASK (/16)

MASK (/32)

MASK (/0)

Assuming a lookup was performed for a packet with a

destination of 10.1.5.2 /24, then the following would occur…

Packet 1

2 Key Gen

3 Lookup Key

HIT!

IF, MACs, MTU

IF, MACs, MTU

IF, MACs, MTU

IF, MACs, MTU

Load-Sharing Hash

4

5

6 7

FIB TCAM

Adjacency

Table

Catalyst 6500 Internals Layer 3 Forwarding on PFC

68


Reference: IPv4 FIB TCAM Lookup Process

1. Destination IP read from packet

2. Lookup key created based on destination IP

3. As lookup key compared to TCAM entries, associated mask applied

4. Longest match returns index to adjacency block and number of adjacencies in load-sharing block

5. Packet flow data input to load-sharing hash function

6. Hash result returns adjacency offset value, selecting an adjacency entry (containing next-hop information) in the indexed adjacency block

For Your Reference


Supervisor FIB TCAM Resources Defaults and Changes

IPv6 and IPv4 multicast require 2 entries

MPLS and IPv4 only one

XL PFCs = 1M entries

Non-XL PFCs = 256K entries

By default TCAM is allocated as seen in the table

70

NON-XL PFCs

XL PFCs

IPv4, MPLS 192k 512k

IPv6, Multicast 32k 256k

SUP720-3CXL Example 6509E#sh mls cef maximum-routes

FIB TCAM maximum routes :

=======================

Current :-

-------

IPv4 + MPLS - 512k (default)

IPv6 + IP Multicast - 256k (default)

Changing default (requires Reboot!)

6509E(config)#mls cef maximum-routes ?

ip number of ip routes

ip-multicast number of multicast routes

ipv6 number of ipv6 routes

mpls number of MPLS labels


Displaying IPv4 Forwarding Summary s

6509E#show platform hardware capacity forwarding

<snip>

L3 Forwarding Resources

FIB TCAM usage: Total Used %Used

72 bits (IPv4, MPLS, EoM) 196608 28 1%

144 bits (IP mcast, IPv6) 32768 7 1%

detail: Protocol Used %Used

IPv4 28 1%

MPLS 0 0%

EoM 0 0%

IPv6 1 1%

IPv4 mcast 3 1%

IPv6 mcast 3 1%

Adjacency usage: Total Used %Used

1048576 171 1%

<snip>

71


Displaying Hardware IPv4 Prefix Entries s

72

6509E#show platform hardware cef

Codes: decap - Decapsulation, + - Push Label

Index Prefix Adjacency

68 255.255.255.255/32 receive

75 10.10.1.1/32 receive

76 10.10.1.0/32 receive

77 10.10.1.255/32 receive

78 10.10.1.2/32 Gi1/1, 0030.f272.31fe

3200 224.0.0.0/24 receive

3201 10.10.1.0/24 glean

3202 10.100.0.0/24 Gi1/1, 0030.f272.31fe

3203 10.100.1.0/24 Gi1/1, 0030.f272.31fe

3204 10.100.2.0/24 Gi1/1, 0030.f272.31fe

3205 10.100.3.0/24 Gi1/1, 0030.f272.31fe

<…>


Finding the Longest-Match Prefix Entry s

73

6509E#show platform hardware cef 171.1.1.0



6509E#show platform hardware cef lookup 171.1.1.0



3531584 171.0.0.0/8 Vl192 ,00d0.0053.bc00

6509E#show platform hardware cef ipv6 lookup FF00::

Codes: + - Push label


512 FF00::/8 glean


Agenda




Layer 2 Forwarding


NetFlow


Packet Walks

74


Catalyst 6500 NetFlow a

75

Netflow

Collection

Server

Netflow

Data Flow (PFC)

Exported Netflow

Record (MSFC)

Netflow is a process designed to collect information about traffic flows that pass through the switch


Catalyst 6500 NetFlow PFC3 Flow Masks

76

Flow Masks supported by PFC3 / DFC3

© 2013 Cisco and/or its affiliates. All rights reserved. BRKARC-3465 Cisco Public Alias CAM

NetFlow

Table Index

Result

128K/256K

entries

128K/256K

rows

Statistics Mask

Key Key

Key Key Key Key Key

Key

Mask

Key Key Key

Flow Data Flow Data

Flow Data Flow Data Flow Data Flow Data Flow Data

Flow Data

Flow Data Flow Data

Flow Data Flow Data

Key

Catalyst 6500 NetFlow TCAM Lookup on PFC3

77

Netflow TCAM Netflow Table

Compare

Flow Key

Hash Key Hash Key

HIT!

HIT!

128 entries

Compare

Hash Function

Hash Key

2

3

4

5

6

7 Flow Key

Packet 1


Reference: PFC3 NetFlow Processing

1. Layer 3 and Layer 4 information (based on flow mask) extracted from packet header

2. NetFlow lookup key generated based on packet information

3. NetFlow lookup key input to hash function

4. NetFlow hash key compared to contents of NetFlow TCAM and Alias CAM

5. On hit in NetFlow TCAM, result returns NetFlow table index; hit in Alias CAM may return additional index

6. Lookup key compared to contents of indexed location(s) in NetFlow table

7. On match, statistics for flow updated On miss, Alias CAM entry installed On Alias CAM full, no stats maintained for new flow

78

For Your Reference


Catalyst 6500 NetFlow NetFlow Export Process

79

Netflow Collector

Direct Export supported with Supervisor 720 and : WS-X6708-10GE-3C/3CXL WS-X6716-10x-3C/3CXL Direct Export supported with Supervisor 2T and : WS-X6716-10x upgraded with DFC4-E / DFC4-EXL WS-X6816-10x-2T/2TXL WS-X6908-10G-2T/2TXL WS-X6904-40G-2T/2TXL

EOBC

WS-X6908-10G-2T\2TXL

Netflow

Data

Netflow

Data

Netflow

Export

Supervisor

Netflow

Data

WS-X6848-TX-2T w\DFC4


Flexible

Netflow

Increased customization by selecting the fields to match and collect for both IPv4 and IPv6

CPU Friendly

Export

Optimal CPU utilization

with Yielding Netflow

Data Export, direct

export from a

module

Up to 13M

Flows /

System

Bigger tables mean

more entries per

system, up to 13

million entries with a

13 slot chassis, giving

you better visibility in

your network Sampled

Netflow in

Hardware To optimize the Netflow

tables utilization and

minimize load on

analyzers

Egress

Netflow

Allow to use netflow after ingress lookup is done (NetFlow on CoPP)

Allow to account for multicast traffic per destination instead of per group

Sup2T

Netflow

Catalyst 6500 NetFlow Supervisor 2T Enhancements

80


512K

entries

Index

Index

Index

Index

Index

Index

Index

Index

Index

Index

Index

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Index

Index

Index

Index

Index

Index

Index

Index

Index

Index

Index

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Index

Index

Index

Index

Index

Index

Index

Index

Index

Index

Index

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Index

Index

Index

Index

Index

Index

Index

Index

Index

Index

Index

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

Data Key

NetFlow Lookup Table

Data Key

Flow Data

Flow Data

Flow Data

Flow Data

Flow Data

Flow Data

Flow Data

Flow Data

Flow Data

Flow Data

Flow Data

NetFlow Data Table

1

Statistics

Statistics

Statistics

Statistics

Statistics

Statistics

Statistics

Statistics

Statistics

Statistics

Statistics

NetFlow

Statistics

Table

7

Lookup

Key

Update Stats

Indexes row in Lookup Table 3

4

5

Index to

NF Data

Table

Compare

all pages

Flow Key Flow Key Compare

Flow Data

6

Data Key

HIT! HIT!

2

Hash Function

10.1.1.10 10.1.2.11 0x6 80 33992

SRC IP DST IP Proto SRC Port DST Port

81

Catalyst 6500 NetFlow TCAM Lookup on PFC4


Flow

Monitor

Key Field Non-Key Field

Flow Export

Flow Monitor

Flow Record

Ingress

or/and

Egress

Interfaces

Key Field Non-Key Field

… …

Export Profile

Export Profile

…

Ingress

or/and

Egress ….

Key Fields trigger the creation of a new Flow entry

every time their value change

Non-Key Fields are data that is indexed by the Key Fields.

Key Fields are defined using the “match” statement

Non-Key-Fields are defined using the “collect” statement

Multiple Exporters

can be associated

with a single FNF

monitor

Same Flow Monitor

can be associated

with multiple

Interfaces.

82

Catalyst 6500 NetFlow Configuring Flexible NetFlow


flow record SAMPLE-FLOW

match ipv4 source address

match ipv4 destination address

match transport source-port

match transport destination-port

match flow direction

collect counter bytes

collect counter packets

collect timestamp sys-uptime first

collect timestamp sys-uptime last

flow exporter SAMPLE-EXPORT-1

description SAMPLE FnF v9 Exporter

destination 11.1.1.1 vrf MGMT

source Loopback0

transport udp 999

flow exporter SAMPLE-EXPORT-2

description SAMPLE FnF v9 Exporter

destination 12.1.1.1 vrf MGMT

transport udp 999

flow monitor SAMPLE-MONITOR

description SAMPLE FnFf v9 Monitor

record SAMPLE-FLOW

exporter SAMPLE-EXPORT-1

exporter SAMPLE-EXPORT-2

interface GigabitEthernet1/1/1

ip address 172.16.0.1 255.255.255.0

ip flow monitor SAMPLE-MONITOR input

ip flow monitor SAMPLE-MONITOR output

logging event link-status

interface Vlan10

ip address 172.16.1.1 255.255.0

ip flow monitor SAMPLE-MONITOR input

ip flow monitor SAMPLE-MONITOR output

logging event link-status

NON-KEY

KEY

Interfaces support multiple

monitors if their key fields

do not overlap *

For Your Reference REFERENCE : Catalyst 6500 NetFlow

Flexible NetFlow Configuration

83

© 2013 Cisco and/or its affiliates. All rights reserved. BRKARC-3465 Cisco Public © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential BRKARC-3465 84

For Your Reference REFERENCE : PFC4 Key and Non-Key Fields

84


NDE increases

export rate until

threshold reached

Wait 5 seconds and then

step up export rate again

When threshold

reached, NDE quickly

backs off export rate

70% Yielding NDE

threshold

CPU

Utilization

30% CPU before

NDE begins

85

Catalyst 6500 NetFlow CPU Friendly Export


Displaying NetFlow Utilization a

86

6509E#show platform hardware capacity netflow

Netflow resources:

Netflow table size: 515032 entries total

Netflow table usage: Module/Instance Input flows Output flows

3 10% 10%

7 25% 25%


Agenda




Layer 2 Forwarding


NetFlow


Packet Walks

87


PFC

PFC

DFC

DFC

DFC

Hardware Support Policy Feature Card

(PFC)

Distributed Forwarding

Card (DFC)

Router ACLs

Vlan ACLs

Port Based ACLs

Role Based ACLs

2

Hardware- Assist

Features

Netflow

WCCP

Reflexive ACLs

Network Address

Translation

Cisco Trust Sec

3

IP Access-List extended Internet

permit ip any host 10.2.2.4




Create the ACL or traffic

classification policy using CLI or

Network Management System

1

Catalyst 6500 Access Control Lists Hardware Support


Catalyst 6500 Access Control Lists Three Forms of Security ACLs

89

The PFC3/PFC4 supports three forms of Security ACLs: the RACL, VACL and PACL…

Router ACL (RACL) VLAN ACL (VACL) Port ACL (PACL)

Used to permit or deny the

movement of traffic

between Layer 3 Subnets

Applied as an input or

output policy to a Layer 3

interface


movement of traffic

between Layer 3

Subnets/VLANs or within a

VLAN

Applied as a policy to a

VLAN - is inherently

applied to both inbound

and outbound traffic


movement of traffic

between Layer 3

Subnets/VLANs or within a

VLAN

Applied as a policy to a

Layer 2 Switch port

interface - is applied for

inbound traffic only


Catalyst 6500 Access Control Lists ACL Order of Processing

90

VACL VACL

Input RACL Output RACL

Source

Destination

Note that no

Output PACL

exists

Input PACL


1

2

3

4

5

6

7

8

1

2

3

4

5

6

7

8

Permit

Permit

Deny

Deny

Permit

Deny

Deny

Permit

00000000 FFFFFFFF 00 0000 0000

Masks Values

xxxxxxxx 10.1.2.100 xx xxxx xxxx



00000000 00000000 FF 0000 FFFF

xxxxxxxx xxxxxxxx 06 xxxx 0016



xxxxxxxx xxxxxxxx 11 xxxx 00A1


Dest IP

Protocol

Source IP

Source Port Dest Port

1=“Compare”

0=“Mask”

Catalyst 6500 Access Control Lists PFC3 TCAM Population

ip access-list extended example


deny ip any host 10.1.68.101


permit tcp any any eq 22

deny tcp any any eq 23

deny udp any any eq 514


permit udp any any eq 161





3

1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8

Permit

Result

Compare

00000000 FFFFFFFF

00 0000 0000

00000000 00000000 FF

0000 FFFF




xxxxxxxx xxxxxxxx 11 xxxx 00A1


ip access-list extended example





deny tcp any any eq 23

deny udp any any eq 514


permit udp any any eq 161 92

Generate

Lookup

Key

SIP=10.1.1.10

DIP=10.1.2.11

Protocol=TCP (6)

SPORT=33992

DPORT=80

Packet

Entries

matching only

destination IP

Entries matching

only protocol and

destination port

Lookup Key

Masks Values

1

2

4

xxxxxxxx xxxxxxxx 06 xxxx 0050 xxxxxxxx 10.1.2.11 xx xxxx xxxx 10.1.1.10 | 10.1.2.11 | 06 | 84C8 | 0050

HIT!s

Catalyst 6500 Access Control Lists PFC3 TCAM Lookup


Reference: ACL TCAM Lookup Process

1. Layer 3 and Layer 4 information read from packet

2. Lookup key generated based on packet information

3. As lookup key compared to TCAM entries, associated mask applied

4. “First” match returns ACL result (permit, deny, redirect, punt, etc.)

93

For Your Reference


permit ip 10.1.1.0 0.0.0.255 any

permit ip 10.2.1.0 0.0.0.255 any

permit ip 10.3.0.0 0.0.255.255 any

MASK

0.0.0.255

10.1.1.0 permit

10.2.1.0 permit

- - - - - - - - - - - -

MASK

0.0.255.255

10.3.0.0 permit

- - - - - - - - - - - - - -

PFC3 ACL TCAM 10.1.1.0 permit

10.2.1.0 permit

- - - - - - - - - -

10.3.0.0 permit

Mask 0.0.0.255

Mask 0.0.0.255

- - - - -

Mask 0.0.255.255

- - - - - -

- - -

PFC4 ACL TCAM

94

Catalyst 6500 Access Control Lists Mask Utilization : PFC3 vs PFC4

16 ACEs

used

3 ACEs

used


BANK 1

VACL

BANK 0

QoS

BANK 3

RACL

BANK 2

SGT

Classification Module 1

TCAM A TCAM B


ACL

Labels

ACL

LOUs

TCAM Controller

Packet Header Information

1

2 X Lookup

Keys

2

3

4 X

Results

4

4 X Result

Data

5

6

ACE

Counters

(L2 ASIC)

7

Final Result to

Netflow

8

Forwarding Engine

(PFC4 or DFC4)

95

Catalyst 6500 Access Control Lists PFC4 TCAM Lookup


REFERENCE : Catalyst 6500 PFC4 Access Control List Lookup Example

CL1 = Classification Module 1

CL2 = Classification Module 2

IFE = Input Forwarding Engine pipeline which performs Ingress functions – input classification, input QOS, ACLs, RPF checks, Ingress Netflow and L3 FIB-based forwarding.

OFE = Output Forwarding Engine pipeline which performs the Egress functions – adjacency lookup, egress classification, rewrite instruction generation.

1 - Packet header information enters the CL1

2- CL1 derives the TCAM Classification Lookup keys, one for TCAM A and one for TCAM B. The TCAM lookup key is derived from the ACL Labels, LOUs, and packet header information

3- TCAM controller uses the lookup key to perform a dual-bank lookup per TCAM, so two separate feature lookup per TCAM are supported at the same time (four total lookups).

This allows multiple feature lookups to occur at the same time.

4- The four individual results are sent back to the CL1 module for packaging to the CL2 module, the CL1 module receives the results from the TCAM which includes a pointer to the TCAM SSRAM (not pictured in the diagram), the data in the SRAM is retrieved and is packaged into a header to be forwarded to the CL2 module. The header also includes a Precedence value indicting the order of precedence for the four results

5- The four individual results are sent to the CL2 module

6- The CL2 module receives the results and reads the precedence 96

For Your Reference


Make sure the ACL will fit in the TCAM before you apply the ACL - ACLs that do not fit can cause

software forwarding and possible high CPU utilization

Special configuration session - Create and edit ACls

- Verifies if the changes will fit within the hardware resources

The actual changes are not programmed into the hardware during the configuration session

Configuration changes can be verified step by step

97

SUP2T-E# show configuration session test status

====================================

Status of last config validation:

Timestamp: 2010-02-20@17:27:06

======================================

SLOT = [1] Result = Configuration will fit in TCAM

Catalyst 6500 Access Control Lists PFC4 ACL Dry Run Feature


For Your Reference

REFERENCE : Catalyst 6500 PFC4 ACL Dry Run Configuration Example

98


ACL Updates

MAC

IPv4 IPv6

99

Allows updates to an ACL without interrupting traffic

Multiple features updated at once

IPv4, IPv6, MAC…

RACL, VACL, PBR…

Global configuration option (default is on)

Feature does consume double the number of TCAM entries

Catalyst 6500 Access Control Lists PFC4 ACL Hitless Update


BANK 1

VACL-1

VACL-2

BANK 0

QoS-1

QoS-2

BANK 3

RACL-1

RACL-2

BANK 2

SGT-1

SGT-2


TCAM A TCAM B

2 X Lookup Keys 4 X Results

ACL

Labels

1, 2

ACL LOUs

TCAM Controller

Each ACL feature is initially programmed into two different spaces into the TCAM

Primary space (Label -1)

Shadow space (label-2)

While an ACL is being updated the PFC4 will use a temporary label that points to the shadow TCAM space

Once the ACL changes have been completed the then PFC4 will then use the original label again

100

Catalyst 6500 Access Control Lists PFC4 ACL Hitless Update


Agenda




Layer 2 Forwarding


NetFlow


Packet Walks

101


Centralized Forwarding: Classic to Classic a

Layer 2 Engine

Layer 3 Engine

Supervisor Engine 2T

PFC4

Fabric / Bus

Interface and

Replication ASIC

Port ASIC A Port ASIC B

Slot 1 Classic


Dbus

Rbus

Slot 2 Classic

Switch Fabric Michael

Engineering

Amanda

Marketing

P

P

H

= Packet

= Header

P

1

2

P P P P

P

H

3

R = Result

R R

R

R R R R 4

102


Reference: Classic to Classic

1. Unicast IPv4 packet (P) received on Classic Module A; entire packet is flooded on DBUS and all devices, including the PFC on the supervisor engine, receive it

2. PFC makes a forwarding decision for the packet based on the header (H) information

3. PFC floods forwarding decision result (R) on RBUS

4. Egress port ASIC on Classic Module B is selected to transmit the packet—all other devices on the bus discard the packet

For Your Reference

103


Layer 2 Engine

Layer 3 Engine


PFC4

Fabric / Bus

Interface and

Replication ASIC


Slot 1 Classic


Dbus

Rbus

Slot 2 CEF720

Switch Fabric

FIRE ASIC A FIRE ASIC B

CFC

Centralized Forwarding: Classic to CEF720 a

Michael

Engineering

Amanda

Marketing

P

P

1

2

P P

P

H

3

R R

R

R R

4

5

P

H

= Packet

= Header

R = Result

104


Reference: Classic to CEF720

1. Unicast IPv4 packet (P) received on Classic Module in Slot 1; entire packet is flooded on DBUS and all devices, including the PFC on the supervisor engine, receive it; CFC on CEF720 Module in Slot 2 ignores the packet

2. PFC makes a forwarding decision for the packet based on the header (H) information

3. PFC floods forwarding decision result (R) on RBUS; all devices on the bus discard the packet since the Egress port is on linecard CEF720 Module in Slot 2; CFC on CEF720 Module in Slot 2 ignores the result

4. The packet is forwarded to the Switch Fabric ASIC and is transmitted to CEF720 Module in Slot 2.

5. CEF720 Module in Slot 2receives the packet and transmits the packet to the egress port ASIC when then transmits to the egress port

For Your Reference

105


Centralized Forwarding: CEF720 to Classic a

Layer 2 Engine

Layer 3 Engine


PFC4

Fabric / Bus

Interface and

Replication ASIC


Slot 1 Classic


Dbus

Rbus

Slot 2 CEF720

Switch Fabric


CFC

Michael

Engineering

Amanda

Marketing

P

1

2

P P

P

H

3

R R

4 5

H

H R

5

R

R

R

6

7

8

9

P

H

= Packet

= Header

R = Result

106


Reference: CEF720 to Classic

1. A Unicast IPv4 packet (P) is received on CEF720 in Slot 2; the entire packet is forwarded to the FIRE ASIC for buffering.

2. The FIRE ASIC buffers the packet, creates the lookup header (H) and sends it to the CFC Bus Interface ASIC.

3. The CFC forwards the header to the Data Bus (DBUS); the header is received by the PFC and is ignored by any other device attached to the DBUS (Port ASICs don’t accept lookup headers).

4. The PFC makes a forwarding decision for the packet based on the header information.

5. The PFC floods the forwarding decision result (R) on RBUS; the source Bus Interface (CFC) processes the result; the Supervisor keeps a copy of the result since it sees this is a flow from a fabric-attached module to a non-fabric-attached module; other devices on the RBUS ignore the result

6. Based on the result, the packet (still in the FIRE ASIC buffer) is sent from the FIRE ASIC to the Sup720 across the switch fabric

7. The Sup720 floods the packet onto the DBUS and all devices receive it; CFC on CEF720 Module in Slot 2 ignores the frame

8. The Sup720 generates a new result, identical to the original except the L3 result has no rewrite information, and floods it onto the RBUS; CFC on CEF720 Module in Slot 2 ignores the result

9. Egress port ASIC on Classic Module in Slot 1 is selected to transmit the packet—all other devices on the bus discard the packet

For Your Reference

107


Centralized Forwarding: CEF720 to CEF720 a

Layer 2 Engine

Layer 3 Engine


PFC4

Fabric / Bus

Interface &

Replication ASIC


Dbus

Rbus

Slot 2 CEF720

Switch Fabric


CFC

Michael

Engineering

Amanda

Marketing

H

3 6


Slot 1 CEF720


CFC

H

R

P

1

2

4 H 5

5

R R

7

P

H

= Packet

= Header

R = Result

108


Reference: CEF720 to CEF720 1. A Unicast IPv4 packet (P) is received in Port ASIC A on the CEF720 in Slot 1; the entire

packet is forwarded to the FIRE ASIC A for buffering.

2. The FIRE ASIC A buffers the packet, creates the lookup header (H) and sends it to the CFC Bus Interface ASIC.

3. The CFC forwards the header to the Data Bus (DBUS); the header is received by the PFC and is ignored by any other device attached to the DBUS (the CFCs don’t accept lookup headers).

4. The PFC makes a forwarding decision for the packet based on the header information.

5. The PFC floods the forwarding decision result (R) on RBUS; the source Bus Interface (CFC) processes the result; other devices on the RBUS ignore the result

6. Based on the result, the packet (still in the FIRE ASIC A buffer) is sent from the FIRE ASIC A on the CEF720 Module in Slot 1 to FIRE ASIC B on the CEF720 Module in Slot 2 across the switch fabric

7. FIRE ASIC B on the CEF720 in Slot 2 forwards the packet to Port ASIC B which then sends out the selected port and to the receiver

For Your Reference

109


Layer 2 Engine

Layer 3 Engine


PFC4

Fabric / Bus

Interface and

Replication ASIC


Dbus

Rbus

Slot 2 CEF720/DFC4

Switch Fabric


DFC4

Michael

Engineering

Amanda

Marketing

3

5


Slot 1 CEF720/DFC4


DFC4

R

P

1

2 4

6

L

2

L

3

L

2

L

3

H

Distributed Forwarding: dCEF720 to dCEF720 a

P

H

= Packet

= Header

R = Result

110


Reference: CEF720 w/DFC to CEF720 w/DFC

1. A Unicast IPv4 packet (P) is received in Port ASIC A on the CEF720/DFC4 in Slot 1; the entire packet is forwarded to the FIRE ASIC A for buffering

2. FIRE ASIC A sends just the packet header to the DFC4

3. DFC4 makes a forwarding decision for the packet

4. DFC4 returns the forwarding decision result to FIRE ASIC A

5. Based on the result, the packet (still in the FIRE ASIC A buffer) is sent from the FIRE ASIC A on the CEF720 Module in Slot 1 to FIRE ASIC B on the CEF720 Module in Slot 2 across the switch fabric

6. FIRE ASIC B on the CEF720 in Slot 2 forwards the packet to Port ASIC B which then sends out the selected port and to the receiver

For Your Reference

111


The Catalyst 6500 architecture provides a robust infrastructure upon which the system can provide hardware-based forwarding at high speeds

L2 and L3 switching are done via the same hardware forwarding process, so there is no difference in performance between the two

Enabling features such as Netflow, QoS and ACLs can be done without impact to forwarding performance as these features are processed in hardware in parallel to the L2 and L3 lookup processes

The Catalyst 6500 architecture is designed so that unicast and multicast can coexist within the same infrastructure, providing a versatile platform for the networks of today and tomorrow

112

Summary a


You should now have a thorough understanding of the Catalyst 6500 switching architecture, packet flow, and key forwarding engine functions… Any Questions?

113

Conclusion a


Maximize your Cisco Live experience with your

free Cisco Live 365 account. Download session

PDFs, view sessions on-demand and participate in

live activities throughout the year. Click the Enter

Cisco Live 365 button in your Cisco Live portal to

log in.

Complete Your Online Session Evaluation

Give us your feedback and you could win fabulous prizes. Winners announced daily.

Receive 20 Cisco Daily Challenge points for each session evaluation you complete.

Complete your session evaluation online now through either the mobile app or internet kiosk stations.

114

Date post:	10-Mar-2018
Category:	Documents
Upload:	duongdieu
View:	249 times
Download:	6 times

Cisco Catalyst 6500 Switch Architectured2zmdbbm9feqrf.cloudfront.net/2013/usa/pdf/BRKARC-… ·...

Documents