+ All Categories
Home > Documents > Pfa Smo Fcoe

Pfa Smo Fcoe

Date post: 25-Dec-2015
Category:
Upload: innovite
View: 242 times
Download: 0 times
Share this document with a friend
Description:
Pfa Smo Fcoe
Popular Tags:
60
IBM Systems and Technology Group P. FALCOU / S Monney IBM ESCC Mainz, June 2009 © 2009 IBM Corporation FCoE INTRODUCTION P. Falcou ([email protected]) S. Monney ([email protected])
Transcript

IBM Systems and Technology Group

P. FALCOU / S Monney IBM ESCC Mainz, June 2009 © 2009 IBM Corporation

FCoE INTRODUCTION

P. Falcou ([email protected])

S. Monney ([email protected])

© 2009 IBM Corporation2 FCoE: P. FALCOU / S Monney June 2009

� Thanks to Brocade, SNIA, CISCO, Pierre Falcou

� Agenda

– Why FCoE ?

– Ethernet

– FCoE encapsulation

– HBA/Switch

– Cabling

– Protocols // CEE

– Some words on some vendors

WHY ARE YOU HERE ?

© 2009 IBM Corporation3 FCoE: P. FALCOU / S Monney June 2009

FCoE is now a standard

"The FCoE standardization activity started in April 2007 is now successfully completed.

On Wednesday June 3rd, 2009, the FC-BB-5 working group of T11 completed the development of the draft standard and unanimously approved it as the final standard. The following daythe plenary session of T11 approved forwarding the FC-BB-5 standard to INCITS for the publication process as an ANSI standard.

( this contain also FIP)

© 2009 IBM Corporation4 FCoE: P. FALCOU / S Monney June 2009

WHY IS DATA CENTER GOING TO FCoE ?

� WAY = TECHNOLOGY :10 Gb ethernet ,

– 40 gigabit and 100 gigabit are in the plan .

� PUSH : Ethernet infrastructure

� ATTRACTION : Consolidation , Virtualization ,TCO

– Less power , cooling , cable , switch , management, skills

density….

� I WANT TO BE : greener, smarter

© 2009 IBM Corporation5 FCoE: P. FALCOU / S Monney June 2009

FC roadmap

10GFCoE

40GFCoE

100GFCoE

10GFC

20GFC

100GFC

8GFC

16GFC

32GFC

MBps

2009

Market

Market

FCoE

2004

2008??

Market

Base 10

2008

2011

Market

Base 2

Market DateProduct

© 2009 IBM Corporation6 FCoE: P. FALCOU / S Monney June 2009

FC roadmap

Juniper claims first with 100G Ethernet

Network World , 06/08/2009

Juniper says, with the IEEE 802.3ba standard for 40G and 100G Ethernet, even though that standard is not expected to be ratified for another year.

Juniper expects the interface to be deployed in customer test networks with DWDM optical transmission gear before the end of 2009.

© 2009 IBM Corporation7 FCoE: P. FALCOU / S Monney June 2009

Fibre Channel

Traffic

Ethernet

� FC View: FC gets a new transport in the form of lossless Ethernet (CEE)

� Ethernet view: A new upper-layer protocol, or storage application, that runs over a new lossless Ethernet

FCoE : IT IS ONLY ENCAPSULATION

Encapsulation of FC frames in Ethernet frames

© 2009 IBM Corporation8 FCoE: P. FALCOU / S Monney June 2009

Encapsulation protocol for transporting FC over Ethernet

(Lossless Ethernet: CEE) Enhanced Ethernet Transport

FC Traffic IP Traffic

Eth

ern

et

Head

er

FC

S

FC

oE

Head

er

EO

F

FC

Head

er

CR

C

FC Payload (SCSI..)

� FC frame remains intact: FC does not change

� No TCP/IP

TraditionalEthernet IP Traffic

FCoE : IT IS ONLY ENCAPSULATION

© 2009 IBM Corporation9 FCoE: P. FALCOU / S Monney June 2009

FCoE specificity

� Ethernet needs a larger frame: Larger than 1.5 KB

Maximum FC frame size is 2148 bytes => we will then use 2.3 KB (baby jumbo frame)

� Ethernet must become lossless to carry storage data with integrity

FCoE flow control (pause, QoS)

� Not routable (no IP)

� Not 100% reliable (no TCP)

� SOF and EOF are encoded into Ethernet fields

Ethernet frame CRC replaces FC frame CRC (same algorithm)

© 2009 IBM Corporation10 FCoE: P. FALCOU / S Monney June 2009

FCoE specificity

� FC has a minimum allowable frame size of 36 bytes and a maximum of 2148 bytes

© 2009 IBM Corporation11 FCoE: P. FALCOU / S Monney June 2009

FCoE specificity

� Some sort of framing standard or format was required to ensure that the most effective solution was found. It was decided to add padding to all frames to avoid any need for a length field.

� Ethernet frames have a minimum allowable frame size of 64 bytes

© 2009 IBM Corporation12 FCoE: P. FALCOU / S Monney June 2009

FCoE uses two different Ethernet Packets:

– FCoE Initialization Protocol (FIP) Ethertype, 0x8914.

– FCoE Data Plane

� FIP is the control plane protocol:

– Used for login to and logout off the FC fabric

– Discovers FC entities connected to the Ethernet fabric

FIPHeader Descriptors

EthernetHeader

FCS

FIP Frame

PROTOCOLS AND FEATURES

� FC data plane packets are used in the data transfer phase

© 2009 IBM Corporation13 FCoE: P. FALCOU / S Monney June 2009

FCoE specificity

� 1 to 1 correspondence between FC frames and Ethernet frames

� Each FC frame = 1 Ethernet frame

� Multiple short FC frames are not put into the same Ethernet frame

© 2009 IBM Corporation14 FCoE: P. FALCOU / S Monney June 2009

Protocol stack comparison

© 2009 IBM Corporation15 FCoE: P. FALCOU / S Monney June 2009

� Encapsulation protocols do not replace storage device interfaces such as FC or SCSI

� Not FCIP replacement . FCIP remains for traffic beyond Data Center (WAN / routing)FCIP use IP

� FCoE is an encapsulation protocol with less overhead than iSCSI but a little bit

more overhead than FC.

� The framing efficiency of FCoE when using

“baby jumbo” Ethernet frames (~ 2.5 KB) is

96.17%

� – This is within approximately 1% of the

framing efficiency on a native Fibre Channel

link

Overhead

© 2009 IBM Corporation16 FCoE: P. FALCOU / S Monney June 2009

FC-0

FC-1 Fibre Channel

Transport

Layer 1

Layer 2

Layer 3IP FCoE

FC-2

FC-3

FC-4FC Services

Encapsulation

Legacy

EthernetFibre Channel

TodayTraditional Ethernet

Today

CEE

Transport

FCoE

ENCAPSULATION

© 2009 IBM Corporation17 FCoE: P. FALCOU / S Monney June 2009

FCoE : IT IS ONLY ENCAPSULATION

FCoE frame

© 2009 IBM Corporation18 FCoE: P. FALCOU / S Monney June 2009

FCoE minimum

FCoE requires reliable frame delivery

“Lossless, full-duplex Ethernet providing in-order frame delivery”

– No frame drop due to buffer overflow or congestion

– No collisions on 1 Gbit and 10 Gbit Ethernet

– 1 Gb and 10 Gb Ethernet both specify a bit error rate of 10-12 (Same requirement as native Fibre Channel)

© 2009 IBM Corporation19 FCoE: P. FALCOU / S Monney June 2009

– When a frame is sent, the available credit is decremented (consumed)

At the PLOGI a receiving port gives a sending port permission to send a specified

number of frames. That permission is called credit

„Hello – I have 2 BBcredits“

– As long as a port has available credit, it may send additional frames

– If the credit is exhausted, frame transmission is suspended until the credit is replenished

Frame sentCredit = Credit -1

Reply ( ACK) Credit = Credit + 1

– When a reply is received, the available credit is incremented (replenished)

(should be within E_D_TOV = 2sec)

Ethernet ( with TCP/IP) use packet drop flow (not acceptable for storage traffic)

© 2009 IBM Corporation20 FCoE: P. FALCOU / S Monney June 2009

Ethernet has an optional “pause” based flow control

– Described in IEEE 802.3 Annex 31B

– Receiver tells the sender when to pause or resume frame transmission

(done in hardware, not software)

– Receiver must send pause while there is enough buffer space to

accommodate any frames in transit plus time for the pause to be

received and processed

© 2009 IBM Corporation21 FCoE: P. FALCOU / S Monney June 2009

FCoE preventing frame drop

One of the problems with the PAUSE mechanism is that it applies no intelligence to the PAUSE, and arbitrarily pauses all traffic.

With “per-priority” flow control each priority, or traffic class,

can be controlled independently

– “Per-Priority Pause” frame has time values for each priority class

– Can selectively pause or resume each priority class independently

– PFC is being standardized by IEEE 802.1Qbb

© 2009 IBM Corporation22 FCoE: P. FALCOU / S Monney June 2009

FCoE preventing frame drop

Pause vs BB_credit

Both mechanism alow to avoid dropping frames

The pause mechanism requires at least the bandwidth x delay product on a link as buffer space

The pause frame is handled by the MAC layer ( similar to R_Ready handling by the FC-1 level

The BB credit mechanism allow to not lose frames over link

© 2009 IBM Corporation23 FCoE: P. FALCOU / S Monney June 2009

• One or more FC interfaces (B) to the host• One or more NIC interfaces (A) to the host• FCoE function: Not seen by the host:

• Performs the encapsulation and de-encapsulation

10GbE10GbE

10GbE10GbE

NIC

FCoE

FC BA

PCIe

WHAT IS CNA ?

© 2009 IBM Corporation24 FCoE: P. FALCOU / S Monney June 2009

Fibre Channel Drivers

Ethernet Drivers

Operating System

Fibre Channel Drivers

Ethernet Drivers

Operating System

PCIe

Fib

re

Ch

an

nel

Eth

ern

et

10G

bE

10G

bE

Link

Fib

re

Ch

an

ne

l

PCIe

4/8

GH

BA

4/8

GH

BA

Link

PCIe

Eth

ern

et

10

Gb

E

10

Gb

E

Link

LAN

Next: One CNA, single interface

HBA

Now: Two adapters and interfaces

WHAT IS CNA ?

© 2009 IBM Corporation25 FCoE: P. FALCOU / S Monney June 2009

CNA on Windows Server

© 2009 IBM Corporation26 FCoE: P. FALCOU / S Monney June 2009

CNA on VMware Server

© 2009 IBM Corporation27 FCoE: P. FALCOU / S Monney June 2009

CNA @IBM

Brocade 10Gb CNA for IBM System x Brocade 10Gb CNA for IBM System x 42C1820 (FC 1637)Brocade 10Gb SFP+ SR Optical Transceiver 49Y4216 (multimode) FC 0069

QLogic 10Gb CNA for IBM System xQLogic 10Gb CNA for IBM System x 42C1800 QLogic 10Gb SFP+ SR Optical Transceiver 49Y4218

ftp://ftp.software.ibm.com/common/ssi/pm/sp/n/xsd03045usen/XSD03045USEN.PDF

ftp://ftp.software.ibm.com/common/ssi/pm/sp/n/xsd03044usen/XSD03044USEN.PDF

© 2009 IBM Corporation28 FCoE: P. FALCOU / S Monney June 2009

Port Naming convention

In an end device, we have the:

• PN_Port = Physical Fibre Channel node port, and one or more,

• VN_Port(s) = Virtual Fibre Channel node port(s)

– In a switch connected to an end device, we have the:

• PF_Port = Physical Fibre Channel fabric port, and one or more,

• VF_Port(s) = Virtual Fibre Channel fabric port(s)

– In a switch connected to another switch, we have the:

• PE_Port = Physical Fibre Channel expansion port, and one or

more,

• VE_Port(s) = Virtual Fibre Channel fabric port(s)

© 2009 IBM Corporation29 FCoE: P. FALCOU / S Monney June 2009

FC and FCoE addressing

Addressing

FC use standard 24 bit address

Domain ID, Area ID, Node address

© 2009 IBM Corporation30 FCoE: P. FALCOU / S Monney June 2009

Addressing

FCoE use mac address (two type can be used)

Server-Provided MAC Addresses

SPMA is a MAC address that is set by the manufacturer at installation.

Fabric-Provided MAC Addresses

The FPMA is a fabric-unique address that is assigned by the fabric. The low-order 24 bits are equivalent to the N_Port ID (FC-ID) assignedduring fabric login, and the high-order 24 bits are equal to the FCoEMAC address prefix (FC-MAP) associated with the fabric.

When a frame is transmitted across FCoE the frame will keep S_ID and D_ID but will encapsulate the FC frame and add a new source and destination MAC address. (Mapping between Fibre Channel N_port IDs (aka FCIDs) and Ethernet MAC addresses )

© 2009 IBM Corporation31 FCoE: P. FALCOU / S Monney June 2009

© 2009 IBM Corporation32 FCoE: P. FALCOU / S Monney June 2009

� FCoE N_Port , F_Port , E_port has the same function as in FC but is on top of Ethernet

� They are calle VN_Port , VF_Port , VE_port

� FCoE pathing and forwarding (routing) use FSPF

� Non FCoE ethernet traffic STP/MSTP (Spanning Tree Protocol )

� STP/MSTP is at layer below FCoE

� TRILL to move the ethernet frames

> RED BOOK LAST CHART

WHAT ABOUT FCoE SWITCH ?

© 2009 IBM Corporation33 FCoE: P. FALCOU / S Monney June 2009

• FC switch is the FCoE/FC forwarding

entity• Has multiple FC ports that connect to

the SAN• Has multiple 10 GbE ports that connect

to the messaging network

• FCoE entity that handles FCoE/FC

encap/decap function

• Enhanced Ethernet switch handles LAN

and SAN traffic:• Forwards FCoE traffic to FCoE entity

• Forwards Enet traffic to Enet ports

Enhanced Ethernet Switch

FCoE

Enet Port

Enet Port

Enet Port

Enet Port

FC Switch

FC Port

FC Port

FC Port

FC Port

Connection to FC not required but possible

Switch with ethernet ports and FCoE capabilities

FCoE must support advanced features

WHAT ABOUT FCoE SWITCH ?

© 2009 IBM Corporation34 FCoE: P. FALCOU / S Monney June 2009

WHAT ABOUT FCoE SWITCH ? NEXUS

© 2009 IBM Corporation35 FCoE: P. FALCOU / S Monney June 2009

� conf t (From EXEC mode, enter the configure terminal command interface)

� interface eth 1/x ( Enters interface configuration mode for the specified interface)

� fcoe mode on (Enable FCoE mode)

� switchport access vlan 100 (Sets the access mode of the interface to the specified VLAN)

� end

� conf t (From EXEC mode, enter the configure terminal command interface)

� interface vfc x/1 (Creates a virtual Fibre Channel interface (if it does not already exist) and entersinterface configuration mode -The only valid number for the port value is 1.)

� bind interface eth 1/y (Associates the virtual interface group to the specified physical interface. Slot port)

� no shutdown (enable the interface) == do not forget

� end

� conf t (From EXEC mode, enter the configure terminal command interface)

� vsan database (Configures the database for a VSAN.)

� vsan 10 interface vfc x (Assigns the membership of the specified interface to the VSAN)

WHAT ABOUT FCoE SWITCH ? NEXUS

© 2009 IBM Corporation36 FCoE: P. FALCOU / S Monney June 2009

WHAT ABOUT FCoE SWITCH ? B8K

© 2009 IBM Corporation37 FCoE: P. FALCOU / S Monney June 2009

Step 1. Configure a CEE interface as a Layer 2 switch port:

switch#config t (Enter configuration commands)

switch(config)#interface tengigabitethernet 0/0(Specify the CEE interface (interface 0/0 is used in this example).

switch(config-if)#switchport (Configure the interface as a Layer 2 switch port)

switch(config-if)#no shutdown (Enable the CEE interface)

Step 2. Create an FCoE VLAN and add an interface to it:

switch#config t

switch(config)#vlan classifier rule 1 proto fcoe encap ethv2(Configure a protocol-based VLAN classifier rule)

switch(config)#vlan classifier group 1 add rule 1(Create a VLAN classifier group and add a rule)

switch(config)#interface vlan 5 (Specify the VLAN interface)

switch(conf-if-vl-5 )#fcf forward (To make a VLAN FCoE-capable, you must enable the forwarding of FCoE traffic on the VLAN interface

switch(conf-if-vl-5 )#interface tengigabitethernet 0/0 (Specify the interface)

WHAT ABOUT FCoE SWITCH ? B8K

© 2009 IBM Corporation38 FCoE: P. FALCOU / S Monney June 2009

switch(config-if-te-0/0)#switchport (Configure the interface as a Layer 2 switch port)

switch(config-if-te-0/0)#switchport mode access (Sets the Layer 2 interface as access alternative is trunk)

switch(config-if-te-0/0)#vlan classifier activate group 1 vlan 5Specify the VLAN classifier group to activate and associate it with a VLAN interface (group 1 and VLAN 5 are used in this example).

switch(config-if-te-0/0)#qos flowcontrol pfc 3 tx on rx on

(Enable an Ethernet Pause on the interface for both TX and RX traffic)

switch(config-if-te-0/0)#no shutdown (enable port)

WHAT ABOUT FCoE SWITCH ? B8K

© 2009 IBM Corporation39 FCoE: P. FALCOU / S Monney June 2009

Step 3: Create a CEE Map to carry LAN and SAN traffic and apply it to an interface:

switch#config t (enter the config mode)

switch(config)#cee-map default (Enters the CEE map configuration mode)

switch(conf-cee-map)#priority-group-table 1 weight 40 pfc(Configures the bandwidth for Priority Group. Define the CEE map with for PGID 1. enables the Priority-based Flow Control)

switch(conf-cee-map)#priority-group-table 2 weight 60(Configures the bandwidth for Priority Group)

switch(conf-cee-map)#priority-table 2 2 2 1 2 2 2 2(Provisions the CEE Priority-to-Priority Group Table, this table maps each of the eight ingress CoSinto a Priority Group)

switch(conf-cee-map)#interface tengigabitethernet 0/0(Specify the CEE interface (interface 0/0 is used in this example).

switch(conf-if-te-0/0)#cee default (apply CEE map on an interface)

WHAT ABOUT FCoE SWITCH ? B8K

© 2009 IBM Corporation40 FCoE: P. FALCOU / S Monney June 2009

Step 4. Configure LLDP for FCoE:

(Link Layer Discovery Protocol: detect links between neighboring switches and if possible create trunk)

switch:admin>cmsh

(takes you to the CEE Brocade8000 CLI prompt)

switch#enable (From the EXEC mode, enter the

enable command.)

switch#config t ( enter the config mode)

switch(config)#protocol lldp (specify a protocol)

switch(conf-lldp)#advertise dcbx-fcoe-app-tlv

(Advertises application Type, Length, Values (TLVs) to ensure interoperability of traffic over the Data Center Bridging eXchange protocol (DCBX), which runs over LLDP to negotiate an FCoE application TLV.)

switch(conf-lldp)#advertise dcbx-fcoe-logical-link-tlv

(Advertises to any attached device the FCoE status of the logical link.)

WHAT ABOUT FCoE SWITCH ? B8K

© 2009 IBM Corporation41 FCoE: P. FALCOU / S Monney June 2009

100Mb 1Gb 10Gb

UTP Cat 5 UTP Cat 5

SFP Fiber

10Mb

UTP Cat 3

Mid 1980s Mid 1990’s Early 2000’s Late 2000s

Cable

Transceiver

Latency (link)

Power

(each side)DistanceTechnology

Twinax ~0.1µs~0.1W10mSFP+ CU

Copper

MM 62.5µm

MM 50µm~01W

82m

300mSFP+ SRshort reach

MM OM2

MM OM3~01W

10m

100mSFP+ USR

ultra short reach

Cat6Cat6a/7Cat6a/7

2.5µs2.5µs1.5µs

~8W~8W~4W

55m100m30m

10GBASE-T

X2

SFP+ Cu (BER better than 10 )

SFP+ FiberCat 6/7

-18

SFP+ Cu

SFP+ to SFP+

WHAT ABOUT CABLES ?

© 2009 IBM Corporation42 FCoE: P. FALCOU / S Monney June 2009

WHAT ABOUT CABLES ?

The major differences between the SFP and SFP+ modules are

SFP+ can run data rates up to 10

Gbps

The modules do not contain the

SERDES (MUX/DEMUX) and

equalization (pre-emphasis and

EDC-electronic dispersion compensation) – these are

moved to the host.

This allows for a much more

cost effective general module

that can be targeted at several

different 10-Gigabit markets

SFP+ are also different to XFP

(30% smaller, more power efficient, and lower cost!)

© 2009 IBM Corporation43 FCoE: P. FALCOU / S Monney June 2009

6

72

4

36

Total

3636Cables

22Switches

42Uplink Ports

1818Adapters*

FCEthernet18 Servers

0

0

0

0

Ethernet

6

36

2

18

Total

036Cables

02Switches

42Uplink Ports

018Adapters*

FCCEE18 Servers

LAN

SAN A

SAN B

2

2

2

2 LAN

SAN A

SAN B

2

2

2

2

50% fewer adapters, switches,

and cables

SOLUTION ?

© 2009 IBM Corporation44 FCoE: P. FALCOU / S Monney June 2009

SOLUTION

© 2009 IBM Corporation45 FCoE: P. FALCOU / S Monney June 2009

FCoE and CEEEthernetFC

Converged CNA’s(CEE/FCoE Traffic)

Storage

SAN CorporateLAN

Top-of-Rack

SwitchesFCoE

SwitchFCoE

Switch

Unified I/O Use Case

FC HBAs(FC Traffic)

NICs(Ethernet Traffic)

LAN SAN A SAN B

FC Switch

FC Switch

BEFORE AFTER

SOLUTION

© 2009 IBM Corporation46 FCoE: P. FALCOU / S Monney June 2009

SOLUTION

SOLUTION

© 2009 IBM Corporation47 FCoE: P. FALCOU / S Monney June 2009

Ethernet is standardized as IEEE 802.3 / 1980 / Token Ring ; FFDI ; ARCNET

From CSMA/CD (Carrier sense multiple access with collision detection) on

coax to full duplex fiber on switches

From software bridges to full hardware switches

+ Spanning-tree protocol to maintain the active links of the network as a

tree while allowing physical loops for redundancy.

+ Link aggregation to Add bandwidth to overloaded links

Provide some measure of redundancy, (although the links won't

protect against switch failure because they connect the same pair of switches)

+

Ethernet FLOW CONTROL = FRAME DROP

ETHERNET HAS BEEN IMPROVED ..BUT

© 2009 IBM Corporation48 FCoE: P. FALCOU / S Monney June 2009

The Spanning Tree Protocol is an OSI layer-2 protocol that ensures a loop-free topology for any bridged LAN.

Spanning tree allows a network design to include spare (redundant) links to

provide automatic backup paths if an active link fails, without the danger of bridge

loops, or the need for manual enabling/disabling of these backup links. Bridge loops must be avoided because they result in flooding the network.

The Spanning Tree Protocol (STP), is defined in the IEEE Standard 802.1D. As the name suggests, it creates a spanning tree within a mesh network of

connected layer-2 bridges (typically Ethernet switches), and disables those links

that are not part of the tree, leaving a single active path between any two network nodes

PROTOCOLS AND FEATURES : STP

© 2009 IBM Corporation49 FCoE: P. FALCOU / S Monney June 2009

PROTOCOLS AND FEATURES : STP

A Spanning Tree port can be in one of four states. These include:

Listening. In this state, a port is listening to Spanning Tree messages (BPDUs) and attempting to figure out how the network is configured.

Learning. In this state, a port is adding addresses to its MAC table, butnot yet forwarding frames.

Forwarding. When in this state, a port is sending and receiving data as normal. During normal operation, a port will be in either a forwarding or a blocking state.

Blocking. When in this state, a port will neither send nor receive data, but will listen to network messages relating to Spanning Tree. By default, all ports are in blocking mode when a switch is first powered on.

© 2009 IBM Corporation50 FCoE: P. FALCOU / S Monney June 2009

PROTOCOLS AND FEATURES : RSTP

There are only three port states left in RSTP (Rapid spanning tree) that correspond to the three possible operational states

© 2009 IBM Corporation51 FCoE: P. FALCOU / S Monney June 2009

PROTOCOLS AND FEATURES : RSTP

RSTP bridge port roles:

Root - A forwarding port that has been selected for the spanning-tree topology

Designated - A forwarding port for every LAN segment

Alternate - An alternate path to the root bridge. This path is different than using the root port.

Backup - A backup/redundant path to a segment where another bridge port already connects.

Disabled - Not strictly part of STP, a network administrator can manually disable a port

© 2009 IBM Corporation52 FCoE: P. FALCOU / S Monney June 2009

Recover bandwidth, multiple active paths; no spanning tree L2 Multipathing: TRILL in IETF

Feature/Standard Benefit

Priority Flow Control (PFC)IEEE 802.1Qbb

Enable storage and networking traffic types to share a common CEE link without interfering with each other ( -> ballot letter

11.09)

ETS: Enhanced Transmission

Selection (Bandwidth Management) IEEE 802.1Qaz

Enable bandwidth management by assigning bandwidth segments to different traffic flows ( -> ballot letter 11.09)

Congestion Management IEEE 802.1Qau

End-to-end congestion management for L2 network (-> approval 11.09)

Data Center Bridging Exchange Protocol (DCBX)

Management protocol for CEE

CEE

* Transport for FCoE storage traffic

* Lossless Ethernet that creates the potential for I/O consolidation at the server

edge by transporting storage and networking traffic over CEE

*Jumbo frames required

*Pause to handle congestion IEEE 802.3 multicast MAC address of 01-80-C2-00-00-01.

FCoE PROTOCOLS AND FEATURES

© 2009 IBM Corporation53 FCoE: P. FALCOU / S Monney June 2009

Priority Flow Control (PFC):

� Standard track in IEEE 802.1Qbb (04/09)

� Enables the use of PAUSE command per priority, using Ethernet priorities 0-7 assigned to various traffic flows

� “No drop” attribute

� Avoids the shortcomings of link-level PAUSE

� Eliminates traffic interference among priority flows

Bandwidth management (ETS) (07/09)

� Standard track in IEEE 802.1Qaz

� Assigns one or more priorities to a Priority Group

� Allocates bandwidth to assigned priority groups using percentage values

Priority 0: FCoE

Priority 1: FCoE

Priority 2: LAN

Priority 6: User x

Priority 7: User Z

Priority Group 1: Storage 60%

Priority Group 2: LAN 30%

Priority Group 3: IPC 10%

PROTOCOLS AND FEATURES

© 2009 IBM Corporation54 FCoE: P. FALCOU / S Monney June 2009

CEE

PFC (Priority flow control): Link level flow control is the only way to guarantee zero

loss due to congestion

802.3 pauses all traffic including control traffic.

PFC affect only traffic that need it after PFC is negociated using DCBX 802.3 shall not be

used (Don’t generatre and Ignore pause frames)

PROTOCOLS AND FEATURES

© 2009 IBM Corporation55 FCoE: P. FALCOU / S Monney June 2009

DCE Bridging Capability Exchange Protocol

DCBX protocol is an extension of the Link Layer Discovery Protocol (LLDP).

DCBX end points exchange request and acknowledgment messages.

For flexibility, parameters are coded in a type-length-value (TLV) format.

Used between switches to handle DCE parameters : known and coherent

parameters (COS , bandwidth , lossless? ,…

During normal operation of FCoE between the switch and the adapter, the DCBX

protocol provides link-error detection.

DCBX is also used to negotiate capabilities between the switch and the adapter

and to send configuration values to the adapter.

The capability reduces the possibility of configuration error and simplifies administration of the adapters.

STILL under development in IEEE

PROTOCOLS AND FEATURES : DCBX

© 2009 IBM Corporation56 FCoE: P. FALCOU / S Monney June 2009

� Why DCBX?

– Defines the limits of the CEE-capable cloud

– Detects misconfiguration between peers

– Can be used to configure a peer

� DCBX enhances LLDP with additional TLVs

– PFC TLV

– ETS TLV

– FCoE Application TLV

PROTOCOLS AND FEATURES : DCBX

© 2009 IBM Corporation57 FCoE: P. FALCOU / S Monney June 2009

Transparent Interconnection of Lots of Links (TRILL):

The TRILL WG will design a solution for shortest-path frame routing in multi-hop IEEE 802.1-compliant Ethernet networks with arbitrary topologies, using an

existing link-state routing protocol technology.

Eliminate Spanning tree

The design should have the following properties:

- Minimal or no configuration required

- Load-splitting among multiple paths

- Routing loop mitigation (possibly through a TTL field)- Support of multiple points of attachment

- Support for broadcast and multicast- No significant service delay after attachment

- No less secure than existing bridged solutions

May 2009 Submit base protocol specification to IEEE/IETF expert review

Jun 2009 Base protocol specification submitted to the IESG for publication as

a Proposed Standard RFC

PROTOCOLS AND FEATURES : TRILL

© 2009 IBM Corporation58 FCoE: P. FALCOU / S Monney June 2009

PROTOCOLS AND FEATURES : CNA

© 2009 IBM Corporation59 FCoE: P. FALCOU / S Monney June 2009

An Introduction to Fibre Channel over

Ethernet, and Fibre Channel over Convergence Enhanced Ethernet

http://www.redbooks.ibm.com/redpieces/abstracts/r

edp4493.html?Open

REFERENCE

© 2009 IBM Corporation60 FCoE: P. FALCOU / S Monney June 2009

FCoE Terminology

CEE: Converged Enhanced Ethernet or

DCE: Data Center Ethernet

DCBX: Data center bridging exchange

FCoE: Fibre channel over ethernet

LAG: Link aggregation Group

LLDP: Link Layer discovery portocol

VLAN: Virtual Local area network

VN_Port: FCoE equivalent of a N_port

VF_Port: FCoE equivalent of a F_port

Enode: FCoE servers and storage

FCF: FCoE copnverter

FIP: FCoE initialisation portocol

CNA: Converged Network Adapters (CNAs)


Recommended