MYFCOE

IBM Systems and Technology Group

P. FALCOU IBM ESCC Mainz, May 2009 2009 IBM Corporation

FCOE INTRODUCTION

2009 IBM Corporation2 FCOE MAY 05

Thanks to Brocade ,SNIA Agenda

Why FCOE ? Ethernet FCOE encapsulation HBA/Switch Cabling Protocols //CEE Some words on some vendors


Why is Data Center going to FCOE ?

More NICs and More HBAs and More cables More powerfull servers

Servers virtualization / consolidation Green/Smart 10 Gigabit ethernet


Ethernet is standardized as IEEE 802.3 / 1980 / Token Ring ; FFDI ; ARCNET

From CSMA/CD on coax to full duplex fiber on switches

From software bridges to full hardware switches

+ Spanning-tree protocol to maintain the active links of the network as a tree while allowing physical loops for redundancy.

+ Various port protection features, as it is far more likely an attacker willbe on an end system port than on a switch-switch link.

+ VLANs to keep different classes of users separate while using thesame physical infrastructure.

+ Fast routing at higher levels to route between those VLANs. + Link aggregation to add bandwidth to overloaded links and to provide

some measure of redundancy, although the links won't protect againstswitch failure because they connect the same pair of switches

FLOW CONTROL = FRAME DROP

ETHERNET is a long story


Fibre Channel Traffic

Ethernet

FC View: FC gets a new transport in the form of lossless Ethernet (CEE)

Ethernet view: A new upper-layer protocol, or storage application, that runs over a new lossless Ethernet

FCOE : IT IS ONLY ENCAPSULATION


Encapsulation protocol for transporting FC over Ethernet

(Lossless Ethernet: CEE) Enhanced Ethernet Transport

FC TrafficIP Traffic

E

t

h

e

r

n

e

t

H

e

a

d

e

r

F

C

S

F

C

o

E

H

e

a

d

e

r

E

O

F

F

C

H

e

a

d

e

r

C

R

C

FC Payload

FC frame remains intact: FC does not change Ethernet needs a larger frame: Larger than 1.5 KB => 2.3 KB Ethernet must become lossless to carry storage data with integrity

TraditionalEthernet IP Traffic



FC-0

FC-1 Fibre ChannelTransport

Layer 1

Layer 2

Layer 3IP FCoE

FC-2

FC-3

FC-4FC Services

Encapsulation

LegacyEthernet

Fibre Channel TodayTraditional Ethernet

Today

CEETransport

FCoE

ENCAPSULATION


Encapsulation protocols do not replace storage device interfaces such as FC or SCSI

Not FCIP replacement . FCIP remains for traffic beyond Data Center

FCoE is an encapsulation protocol with less overhead than iSCSI but more overhead than FC

OTHER PROTOCOLS


One or more FC interfaces (B) to the host One or more NIC interfaces (A) to the host FCoE function: Not seen by the host:

Performs the encapsulation and de-encapsulation

10GbE10GbE

10GbE10GbENIC

FCoE

FC BA

PCIe

WHAT IS CNA ?


Fibre Channel Drivers

Ethernet Drivers

Operating System

Fibre Channel Drivers

Ethernet Drivers

Operating System

PCIe

Fibre

Chan

nel

Ethern

et10G

bE

10GbE

Link

Fibre

Ch

an

nel

PCIe

4/8 G

HB

A

4/8 G

HB

A

Link

PCIe

Ethe

rnet

10GbE

10GbE

Link

LAN

Next: One CNA, single interface

HBA

Now: Two adapters and interfaces

WHAT IS CNA ?


CNA on Windows Server


CNA on VMware Server


FC switch is the FCoE/FC forwarding entity

Has multiple FC ports that connect to the SAN

Has multiple 10 GbE ports that connect to the messaging network

FCoE entity that handles FCoE/FC encap/decap function

Enhanced Ethernet switch handles LAN and SAN traffic:

Forwards FCoE traffic to FCoE entity Forwards Enet traffic to Enet ports

Enhanced Ethernet Switch

FCoE

Enet Port

Enet Port

Enet Port

Enet Port

FC Switch

FC Port

FC Port

FC Port

FC Port

Connection to FC not required but possibleSwith with ethernet ports and FCOE capabilitiesFCOE must support advanced features

WHAT ABOUT FCoE SWITCH ?


FCOE N port , F port , E port has the same functionas in FC

FCOE pathing and forwarding use FSPF STP ( Spanning Tree Protocol ) .TRILL to move the

ethernet frames> RED BOOK LAST CHART



WHAT ABOUT FCoE SWITCH ? NEXUS


conf t interface eth 1/x fcoe mode on switchport access vlan 100 end conf t interface vfc x bind interface eth 1/y no shutdown == do not forget end conf t vsan database vsan 10 interface vfc x

WHAT ABOUT FCoE SWITCH ? NEXUS


WHAT ABOUT FCoE SWITCH ? B8K


Step 1. Configure a CEE interface as a Layer 2 switch port:switch#config tswitch(config)#interface tengigabitethernet 0/0switch(config-if)#switchportswitch(config-if)#no shutdownStep 2. Create an FCoE VLAN and add an interface to it:switch#config tswitch(config)#vlan classifier rule 1 proto fcoe encap ethv2switch(config)#vlan classifier group 1 add rule 1switch(config)#interface vlan 5switch(conf-if-vl-5 )#fcf forwardswitch(conf-if-vl-5 )#interface tengigabitethernet 0/0switch(config-if-te-0/0)#switchportswitch(config-if-te-0/0)#switchport mode accessswitch(config-if-te-0/0)#vlan classifier activate group 1 vlan 5switch(config-if-te-0/0)#qos flowcontrol pfc 3 tx on rx onswitch(config-if-te-0/0)#no shutdown



Step 3: Create a CEE Map to carry LAN and SAN traffic and apply it to an interface:switch#config tswitch(config)#cee-map defaultswitch(conf-cee-map)#priority-group-table 1 weight 40 pfcswitch(conf-cee-map)#priority-group-table 2 weight 60switch(conf-cee-map)#priority-table 2 2 2 1 2 2 2 2switch(conf-cee-map)#interface tengigabitethernet 0/0switch(conf-if-te-0/0)#cee defaultStep 4. Configure LLDP for FCoE:switch:admin>cmshswitch#enableswitch#config tswitch(config)#protocol lldpswitch(conf-lldp)#advertise dcbx-fcoe-app-tlvswitch(conf-lldp)#advertise dcbx-fcoe-logical-link-tlv



100Mb 1Gb 10Gb

UTP Cat 5 UTP Cat 5SFP Fiber

10Mb

UTP Cat 3

Mid 1980s Mid 1990s Early 2000s Late 2000s

CableTransceiver

Latency (link)Power

(each side)DistanceTechnology

Twinax ~0.1s~0.1W10mSFP+ CUCopper

MM 62.5mMM 50m ~01W

82m300m

SFP+ SRshort reach

MM OM2MM OM3 ~01W

10m100m

SFP+ USRultra short reach

Cat6Cat6a/7Cat6a/7

2.5s2.5s1.5s

~8W~8W~4W

55m100m30m

10GBASE-T

X2SFP+ Cu (BER better than 10 )SFP+ FiberCat 6/7

-18

SFP+ CuSFP+ to SFP+

WHAT ABOUT CABLES ?


6

72

4

36

Total

3636Cables

22Switches

42Uplink Ports

1818Adapters*

FCEthernet18 Servers

0

0

0

0

Ethernet

6

36

2

18

Total

036Cables

02Switches

42Uplink Ports

018Adapters*

FCCEE18 Servers

LAN

SAN A

SAN B

2

2

2

2


FCoE and CEEEthernetFC

Converged CNAs(CEE/FCoE Traffic)

Storage

SAN CorporateLAN

Top-of-RackSwitches

FCoE Switch

FCoE Switch

Unified I/O Use Case

FC HBAs(FC Traffic)

NICs(Ethernet Traffic)

LAN SAN A SAN B

FC Switch

FC Switch

BEFORE AFTER

SOLUTION


SOLUTION

SOLUTION


FCoE uses two different Ethernet Packets: FCoE Initialization Protocol (FIP) Ethertype, 0x8914. FCoE Data Plane

FIP is the control plane protocol: Used for login to and logout off the FC fabric Discovers FC entities connected to the Ethernet fabric

FC data plane packets are used in the data transfer phase

FIPHeader Descriptors

EthernetHeader FCS

FIP Frame

PROTOCOLS AND FEATURES


Recover bandwidth, multiple active paths; no spanning tree L2 Multipathing: TRILL in IETF

Feature/Standard Benefit

Priority Flow Control (PFC)IEEE 802.1Qbb

Enable storage and networking traffic types to share a common CEE link without interfering with each other

Enhanced Transmission Selection (Bandwidth Management) IEEE 802.1Qaz

Enable bandwidth management by assigning bandwidth segments to different traffic flows

Congestion Management IEEE 802.1Qau End-to-end congestion management for L2 network

Data Center Bridging Exchange Protocol (DCBX) Management protocol for CEE

CEE * Transport for FCoE storage traffic* Lossless Ethernet that creates the potential for I/O consolidation at the server edge by transporting storage and networking traffic over CEE*Jumbo frames required *Pause to handle congestion IEEE 802.3 multicast MAC address of 01-80-C2-00-00-01.



Priority Flow Control (PFC): Standard track in IEEE 802.1Qbb (04/09) Enables the use of PAUSE command per

priority, using Ethernet priorities 0-7 assigned to various traffic flows

No drop attribute Avoids the shortcomings of link-level

PAUSE Eliminates traffic interference among

priority flows

Bandwidth management (ETS) (07/09) Standard track in IEEE 802.1Qaz Assigns one or more priorities to a

Priority Group Allocates bandwidth to assigned

priority groups using percentage values

Priority 0: FCoEPriority 1: FCoEPriority 2: LAN

Priority 6: User xPriority 7: User Z

Priority Group 1: Storage 60%

Priority Group 2: LAN 30%

Priority Group 3: IPC 10%



The Spanning Tree Protocol is an OSI layer-2 protocol that ensures a loop-free topology for any bridged LAN.

Spanning tree allows a network design to include spare (redundant) links to provide automatic backup paths if an active link fails, without the danger of bridge loops, or the need for manual enabling/disabling of these backup links. Bridge loops must be

avoided because they result in flooding the network.The Spanning Tree Protocol (STP), is defined in the IEEE Standard 802.1D. As the

name suggests, it creates a spanning tree within a mesh network of connected layer-2 bridges (typically Ethernet switches), and disables those links that are not part of the

tree, leaving a single active path between any two network nodes

PROTOCOLS AND FEATURES : STP


DCE Bridging Capability Exchange ProtocolDCBX protocol is an extension of the Link Layer Discovery Protocol (LLDP). DCBX end points exchange request and acknowledgment messages. For flexibility, parameters are coded in a type-length-value (TLV) format.Used between switches to handle DCE parameters : known and coherent parameters(COS , bandwidth , lossless? , During normal operation of FCoE between the switch and the adapter, the DCBX protocol provides link-error detection.DCBX is also used to negotiate capabilities between the switch and the adapter andto send configuration values to the adapter.The capability reduces the possibility of configuration error and simplifies administration of the adapters.

STILL under development in IEEE

PROTOCOLS AND FEATURES : DCBX


Why DCBX? Defines the limits of the CEE-capable cloud Detects misconfiguration between peers Can be used to configure a peer

DCBX enhances LLDP with additional TLVs

PFC TLV ETS TLV FCoE Application TLV

PROTOCOLS AND FEATURES : DCBX


The TRILL WG will design a solution for shortest-path frame routing inmulti-hop IEEE 802.1-compliant Ethernet networks with arbitrarytopologies, using an existing link-state routing protocol technology.

Eliminate Spanning treeTransparent Interconnection of Lots of Links (TRILL):The design should have the following properties:

- Minimal or no configuration required- Load-splitting among multiple paths- Routing loop mitigation (possibly through a TTL field)- Support of multiple points of attachment- Support for broadcast and multicast- No significant service delay after attachment- No less secure than existing bridged solutionsMay 2009 Submit base protocol specification to IEEE/IETF expert review Jun 2009 Base protocol specification submitted to the IESG for publication as a Proposed Standard RFC

PROTOCOLS AND FEATURES : TRILL


PROTOCOLS AND FEATURES : CNA


BROCADE CISCO QLOGIC NSERIES

SOME WORDS ON

And other vendors .

OEM


An Introduction to Fibre Channel overEthernet, and Fibre Channel overConvergence Enhanced Ethernet

http://www.redbooks.ibm.com/redpieces/abstracts/redp4493.html?Open

REFERENCE

Date post:	21-Nov-2015
Category:	Documents
Upload:	innovite
View:	218 times
Download:	1 times

MYFCOE

Documents