IBM Systems and Technology Group
P. FALCOU / S Monney IBM ESCC Mainz, June 2009 © 2009 IBM Corporation
FCoE INTRODUCTION
P. Falcou ([email protected])
S. Monney ([email protected])
© 2009 IBM Corporation2 FCoE: P. FALCOU / S Monney June 2009
� Thanks to Brocade, SNIA, CISCO, Pierre Falcou
� Agenda
– Why FCoE ?
– Ethernet
– FCoE encapsulation
– HBA/Switch
– Cabling
– Protocols // CEE
– Some words on some vendors
WHY ARE YOU HERE ?
© 2009 IBM Corporation3 FCoE: P. FALCOU / S Monney June 2009
FCoE is now a standard
"The FCoE standardization activity started in April 2007 is now successfully completed.
On Wednesday June 3rd, 2009, the FC-BB-5 working group of T11 completed the development of the draft standard and unanimously approved it as the final standard. The following daythe plenary session of T11 approved forwarding the FC-BB-5 standard to INCITS for the publication process as an ANSI standard.
( this contain also FIP)
© 2009 IBM Corporation4 FCoE: P. FALCOU / S Monney June 2009
WHY IS DATA CENTER GOING TO FCoE ?
� WAY = TECHNOLOGY :10 Gb ethernet ,
– 40 gigabit and 100 gigabit are in the plan .
� PUSH : Ethernet infrastructure
� ATTRACTION : Consolidation , Virtualization ,TCO
– Less power , cooling , cable , switch , management, skills
density….
� I WANT TO BE : greener, smarter
© 2009 IBM Corporation5 FCoE: P. FALCOU / S Monney June 2009
FC roadmap
10GFCoE
40GFCoE
100GFCoE
10GFC
20GFC
100GFC
8GFC
16GFC
32GFC
MBps
2009
Market
Market
FCoE
2004
2008??
Market
Base 10
2008
2011
Market
Base 2
Market DateProduct
© 2009 IBM Corporation6 FCoE: P. FALCOU / S Monney June 2009
FC roadmap
Juniper claims first with 100G Ethernet
Network World , 06/08/2009
Juniper says, with the IEEE 802.3ba standard for 40G and 100G Ethernet, even though that standard is not expected to be ratified for another year.
Juniper expects the interface to be deployed in customer test networks with DWDM optical transmission gear before the end of 2009.
© 2009 IBM Corporation7 FCoE: P. FALCOU / S Monney June 2009
Fibre Channel
Traffic
Ethernet
� FC View: FC gets a new transport in the form of lossless Ethernet (CEE)
� Ethernet view: A new upper-layer protocol, or storage application, that runs over a new lossless Ethernet
FCoE : IT IS ONLY ENCAPSULATION
Encapsulation of FC frames in Ethernet frames
© 2009 IBM Corporation8 FCoE: P. FALCOU / S Monney June 2009
Encapsulation protocol for transporting FC over Ethernet
(Lossless Ethernet: CEE) Enhanced Ethernet Transport
FC Traffic IP Traffic
Eth
ern
et
Head
er
FC
S
FC
oE
Head
er
EO
F
FC
Head
er
CR
C
FC Payload (SCSI..)
� FC frame remains intact: FC does not change
� No TCP/IP
TraditionalEthernet IP Traffic
FCoE : IT IS ONLY ENCAPSULATION
© 2009 IBM Corporation9 FCoE: P. FALCOU / S Monney June 2009
FCoE specificity
� Ethernet needs a larger frame: Larger than 1.5 KB
Maximum FC frame size is 2148 bytes => we will then use 2.3 KB (baby jumbo frame)
� Ethernet must become lossless to carry storage data with integrity
FCoE flow control (pause, QoS)
� Not routable (no IP)
� Not 100% reliable (no TCP)
� SOF and EOF are encoded into Ethernet fields
Ethernet frame CRC replaces FC frame CRC (same algorithm)
© 2009 IBM Corporation10 FCoE: P. FALCOU / S Monney June 2009
FCoE specificity
� FC has a minimum allowable frame size of 36 bytes and a maximum of 2148 bytes
© 2009 IBM Corporation11 FCoE: P. FALCOU / S Monney June 2009
FCoE specificity
� Some sort of framing standard or format was required to ensure that the most effective solution was found. It was decided to add padding to all frames to avoid any need for a length field.
� Ethernet frames have a minimum allowable frame size of 64 bytes
© 2009 IBM Corporation12 FCoE: P. FALCOU / S Monney June 2009
FCoE uses two different Ethernet Packets:
– FCoE Initialization Protocol (FIP) Ethertype, 0x8914.
– FCoE Data Plane
� FIP is the control plane protocol:
– Used for login to and logout off the FC fabric
– Discovers FC entities connected to the Ethernet fabric
FIPHeader Descriptors
EthernetHeader
FCS
FIP Frame
PROTOCOLS AND FEATURES
� FC data plane packets are used in the data transfer phase
© 2009 IBM Corporation13 FCoE: P. FALCOU / S Monney June 2009
FCoE specificity
� 1 to 1 correspondence between FC frames and Ethernet frames
� Each FC frame = 1 Ethernet frame
� Multiple short FC frames are not put into the same Ethernet frame
© 2009 IBM Corporation15 FCoE: P. FALCOU / S Monney June 2009
� Encapsulation protocols do not replace storage device interfaces such as FC or SCSI
� Not FCIP replacement . FCIP remains for traffic beyond Data Center (WAN / routing)FCIP use IP
� FCoE is an encapsulation protocol with less overhead than iSCSI but a little bit
more overhead than FC.
� The framing efficiency of FCoE when using
“baby jumbo” Ethernet frames (~ 2.5 KB) is
96.17%
� – This is within approximately 1% of the
framing efficiency on a native Fibre Channel
link
Overhead
© 2009 IBM Corporation16 FCoE: P. FALCOU / S Monney June 2009
FC-0
FC-1 Fibre Channel
Transport
Layer 1
Layer 2
Layer 3IP FCoE
FC-2
FC-3
FC-4FC Services
Encapsulation
Legacy
EthernetFibre Channel
TodayTraditional Ethernet
Today
CEE
Transport
FCoE
ENCAPSULATION
© 2009 IBM Corporation17 FCoE: P. FALCOU / S Monney June 2009
FCoE : IT IS ONLY ENCAPSULATION
FCoE frame
© 2009 IBM Corporation18 FCoE: P. FALCOU / S Monney June 2009
FCoE minimum
FCoE requires reliable frame delivery
“Lossless, full-duplex Ethernet providing in-order frame delivery”
– No frame drop due to buffer overflow or congestion
– No collisions on 1 Gbit and 10 Gbit Ethernet
– 1 Gb and 10 Gb Ethernet both specify a bit error rate of 10-12 (Same requirement as native Fibre Channel)
© 2009 IBM Corporation19 FCoE: P. FALCOU / S Monney June 2009
– When a frame is sent, the available credit is decremented (consumed)
At the PLOGI a receiving port gives a sending port permission to send a specified
number of frames. That permission is called credit
„Hello – I have 2 BBcredits“
– As long as a port has available credit, it may send additional frames
– If the credit is exhausted, frame transmission is suspended until the credit is replenished
Frame sentCredit = Credit -1
Reply ( ACK) Credit = Credit + 1
– When a reply is received, the available credit is incremented (replenished)
(should be within E_D_TOV = 2sec)
Ethernet ( with TCP/IP) use packet drop flow (not acceptable for storage traffic)
© 2009 IBM Corporation20 FCoE: P. FALCOU / S Monney June 2009
Ethernet has an optional “pause” based flow control
– Described in IEEE 802.3 Annex 31B
– Receiver tells the sender when to pause or resume frame transmission
(done in hardware, not software)
– Receiver must send pause while there is enough buffer space to
accommodate any frames in transit plus time for the pause to be
received and processed
© 2009 IBM Corporation21 FCoE: P. FALCOU / S Monney June 2009
FCoE preventing frame drop
One of the problems with the PAUSE mechanism is that it applies no intelligence to the PAUSE, and arbitrarily pauses all traffic.
With “per-priority” flow control each priority, or traffic class,
can be controlled independently
– “Per-Priority Pause” frame has time values for each priority class
– Can selectively pause or resume each priority class independently
– PFC is being standardized by IEEE 802.1Qbb
© 2009 IBM Corporation22 FCoE: P. FALCOU / S Monney June 2009
FCoE preventing frame drop
Pause vs BB_credit
Both mechanism alow to avoid dropping frames
The pause mechanism requires at least the bandwidth x delay product on a link as buffer space
The pause frame is handled by the MAC layer ( similar to R_Ready handling by the FC-1 level
The BB credit mechanism allow to not lose frames over link
© 2009 IBM Corporation23 FCoE: P. FALCOU / S Monney June 2009
• One or more FC interfaces (B) to the host• One or more NIC interfaces (A) to the host• FCoE function: Not seen by the host:
• Performs the encapsulation and de-encapsulation
10GbE10GbE
10GbE10GbE
NIC
FCoE
FC BA
PCIe
WHAT IS CNA ?
© 2009 IBM Corporation24 FCoE: P. FALCOU / S Monney June 2009
Fibre Channel Drivers
Ethernet Drivers
Operating System
Fibre Channel Drivers
Ethernet Drivers
Operating System
PCIe
Fib
re
Ch
an
nel
Eth
ern
et
10G
bE
10G
bE
Link
Fib
re
Ch
an
ne
l
PCIe
4/8
GH
BA
4/8
GH
BA
Link
PCIe
Eth
ern
et
10
Gb
E
10
Gb
E
Link
LAN
Next: One CNA, single interface
HBA
Now: Two adapters and interfaces
WHAT IS CNA ?
© 2009 IBM Corporation27 FCoE: P. FALCOU / S Monney June 2009
CNA @IBM
Brocade 10Gb CNA for IBM System x Brocade 10Gb CNA for IBM System x 42C1820 (FC 1637)Brocade 10Gb SFP+ SR Optical Transceiver 49Y4216 (multimode) FC 0069
QLogic 10Gb CNA for IBM System xQLogic 10Gb CNA for IBM System x 42C1800 QLogic 10Gb SFP+ SR Optical Transceiver 49Y4218
ftp://ftp.software.ibm.com/common/ssi/pm/sp/n/xsd03045usen/XSD03045USEN.PDF
ftp://ftp.software.ibm.com/common/ssi/pm/sp/n/xsd03044usen/XSD03044USEN.PDF
© 2009 IBM Corporation28 FCoE: P. FALCOU / S Monney June 2009
Port Naming convention
In an end device, we have the:
• PN_Port = Physical Fibre Channel node port, and one or more,
• VN_Port(s) = Virtual Fibre Channel node port(s)
– In a switch connected to an end device, we have the:
• PF_Port = Physical Fibre Channel fabric port, and one or more,
• VF_Port(s) = Virtual Fibre Channel fabric port(s)
– In a switch connected to another switch, we have the:
• PE_Port = Physical Fibre Channel expansion port, and one or
more,
• VE_Port(s) = Virtual Fibre Channel fabric port(s)
© 2009 IBM Corporation29 FCoE: P. FALCOU / S Monney June 2009
FC and FCoE addressing
Addressing
FC use standard 24 bit address
Domain ID, Area ID, Node address
© 2009 IBM Corporation30 FCoE: P. FALCOU / S Monney June 2009
Addressing
FCoE use mac address (two type can be used)
Server-Provided MAC Addresses
SPMA is a MAC address that is set by the manufacturer at installation.
Fabric-Provided MAC Addresses
The FPMA is a fabric-unique address that is assigned by the fabric. The low-order 24 bits are equivalent to the N_Port ID (FC-ID) assignedduring fabric login, and the high-order 24 bits are equal to the FCoEMAC address prefix (FC-MAP) associated with the fabric.
When a frame is transmitted across FCoE the frame will keep S_ID and D_ID but will encapsulate the FC frame and add a new source and destination MAC address. (Mapping between Fibre Channel N_port IDs (aka FCIDs) and Ethernet MAC addresses )
© 2009 IBM Corporation32 FCoE: P. FALCOU / S Monney June 2009
� FCoE N_Port , F_Port , E_port has the same function as in FC but is on top of Ethernet
� They are calle VN_Port , VF_Port , VE_port
� FCoE pathing and forwarding (routing) use FSPF
� Non FCoE ethernet traffic STP/MSTP (Spanning Tree Protocol )
� STP/MSTP is at layer below FCoE
� TRILL to move the ethernet frames
> RED BOOK LAST CHART
WHAT ABOUT FCoE SWITCH ?
© 2009 IBM Corporation33 FCoE: P. FALCOU / S Monney June 2009
• FC switch is the FCoE/FC forwarding
entity• Has multiple FC ports that connect to
the SAN• Has multiple 10 GbE ports that connect
to the messaging network
• FCoE entity that handles FCoE/FC
encap/decap function
• Enhanced Ethernet switch handles LAN
and SAN traffic:• Forwards FCoE traffic to FCoE entity
• Forwards Enet traffic to Enet ports
Enhanced Ethernet Switch
FCoE
Enet Port
Enet Port
Enet Port
Enet Port
FC Switch
FC Port
FC Port
FC Port
FC Port
Connection to FC not required but possible
Switch with ethernet ports and FCoE capabilities
FCoE must support advanced features
WHAT ABOUT FCoE SWITCH ?
© 2009 IBM Corporation35 FCoE: P. FALCOU / S Monney June 2009
� conf t (From EXEC mode, enter the configure terminal command interface)
� interface eth 1/x ( Enters interface configuration mode for the specified interface)
� fcoe mode on (Enable FCoE mode)
� switchport access vlan 100 (Sets the access mode of the interface to the specified VLAN)
� end
� conf t (From EXEC mode, enter the configure terminal command interface)
� interface vfc x/1 (Creates a virtual Fibre Channel interface (if it does not already exist) and entersinterface configuration mode -The only valid number for the port value is 1.)
� bind interface eth 1/y (Associates the virtual interface group to the specified physical interface. Slot port)
� no shutdown (enable the interface) == do not forget
� end
� conf t (From EXEC mode, enter the configure terminal command interface)
� vsan database (Configures the database for a VSAN.)
� vsan 10 interface vfc x (Assigns the membership of the specified interface to the VSAN)
WHAT ABOUT FCoE SWITCH ? NEXUS
© 2009 IBM Corporation37 FCoE: P. FALCOU / S Monney June 2009
Step 1. Configure a CEE interface as a Layer 2 switch port:
switch#config t (Enter configuration commands)
switch(config)#interface tengigabitethernet 0/0(Specify the CEE interface (interface 0/0 is used in this example).
switch(config-if)#switchport (Configure the interface as a Layer 2 switch port)
switch(config-if)#no shutdown (Enable the CEE interface)
Step 2. Create an FCoE VLAN and add an interface to it:
switch#config t
switch(config)#vlan classifier rule 1 proto fcoe encap ethv2(Configure a protocol-based VLAN classifier rule)
switch(config)#vlan classifier group 1 add rule 1(Create a VLAN classifier group and add a rule)
switch(config)#interface vlan 5 (Specify the VLAN interface)
switch(conf-if-vl-5 )#fcf forward (To make a VLAN FCoE-capable, you must enable the forwarding of FCoE traffic on the VLAN interface
switch(conf-if-vl-5 )#interface tengigabitethernet 0/0 (Specify the interface)
WHAT ABOUT FCoE SWITCH ? B8K
© 2009 IBM Corporation38 FCoE: P. FALCOU / S Monney June 2009
switch(config-if-te-0/0)#switchport (Configure the interface as a Layer 2 switch port)
switch(config-if-te-0/0)#switchport mode access (Sets the Layer 2 interface as access alternative is trunk)
switch(config-if-te-0/0)#vlan classifier activate group 1 vlan 5Specify the VLAN classifier group to activate and associate it with a VLAN interface (group 1 and VLAN 5 are used in this example).
switch(config-if-te-0/0)#qos flowcontrol pfc 3 tx on rx on
(Enable an Ethernet Pause on the interface for both TX and RX traffic)
switch(config-if-te-0/0)#no shutdown (enable port)
WHAT ABOUT FCoE SWITCH ? B8K
© 2009 IBM Corporation39 FCoE: P. FALCOU / S Monney June 2009
Step 3: Create a CEE Map to carry LAN and SAN traffic and apply it to an interface:
switch#config t (enter the config mode)
switch(config)#cee-map default (Enters the CEE map configuration mode)
switch(conf-cee-map)#priority-group-table 1 weight 40 pfc(Configures the bandwidth for Priority Group. Define the CEE map with for PGID 1. enables the Priority-based Flow Control)
switch(conf-cee-map)#priority-group-table 2 weight 60(Configures the bandwidth for Priority Group)
switch(conf-cee-map)#priority-table 2 2 2 1 2 2 2 2(Provisions the CEE Priority-to-Priority Group Table, this table maps each of the eight ingress CoSinto a Priority Group)
switch(conf-cee-map)#interface tengigabitethernet 0/0(Specify the CEE interface (interface 0/0 is used in this example).
switch(conf-if-te-0/0)#cee default (apply CEE map on an interface)
WHAT ABOUT FCoE SWITCH ? B8K
© 2009 IBM Corporation40 FCoE: P. FALCOU / S Monney June 2009
Step 4. Configure LLDP for FCoE:
(Link Layer Discovery Protocol: detect links between neighboring switches and if possible create trunk)
switch:admin>cmsh
(takes you to the CEE Brocade8000 CLI prompt)
switch#enable (From the EXEC mode, enter the
enable command.)
switch#config t ( enter the config mode)
switch(config)#protocol lldp (specify a protocol)
switch(conf-lldp)#advertise dcbx-fcoe-app-tlv
(Advertises application Type, Length, Values (TLVs) to ensure interoperability of traffic over the Data Center Bridging eXchange protocol (DCBX), which runs over LLDP to negotiate an FCoE application TLV.)
switch(conf-lldp)#advertise dcbx-fcoe-logical-link-tlv
(Advertises to any attached device the FCoE status of the logical link.)
WHAT ABOUT FCoE SWITCH ? B8K
© 2009 IBM Corporation41 FCoE: P. FALCOU / S Monney June 2009
100Mb 1Gb 10Gb
UTP Cat 5 UTP Cat 5
SFP Fiber
10Mb
UTP Cat 3
Mid 1980s Mid 1990’s Early 2000’s Late 2000s
Cable
Transceiver
Latency (link)
Power
(each side)DistanceTechnology
Twinax ~0.1µs~0.1W10mSFP+ CU
Copper
MM 62.5µm
MM 50µm~01W
82m
300mSFP+ SRshort reach
MM OM2
MM OM3~01W
10m
100mSFP+ USR
ultra short reach
Cat6Cat6a/7Cat6a/7
2.5µs2.5µs1.5µs
~8W~8W~4W
55m100m30m
10GBASE-T
X2
SFP+ Cu (BER better than 10 )
SFP+ FiberCat 6/7
-18
SFP+ Cu
SFP+ to SFP+
WHAT ABOUT CABLES ?
© 2009 IBM Corporation42 FCoE: P. FALCOU / S Monney June 2009
WHAT ABOUT CABLES ?
The major differences between the SFP and SFP+ modules are
SFP+ can run data rates up to 10
Gbps
The modules do not contain the
SERDES (MUX/DEMUX) and
equalization (pre-emphasis and
EDC-electronic dispersion compensation) – these are
moved to the host.
This allows for a much more
cost effective general module
that can be targeted at several
different 10-Gigabit markets
SFP+ are also different to XFP
(30% smaller, more power efficient, and lower cost!)
© 2009 IBM Corporation43 FCoE: P. FALCOU / S Monney June 2009
6
72
4
36
Total
3636Cables
22Switches
42Uplink Ports
1818Adapters*
FCEthernet18 Servers
0
0
0
0
Ethernet
6
36
2
18
Total
036Cables
02Switches
42Uplink Ports
018Adapters*
FCCEE18 Servers
LAN
SAN A
SAN B
2
2
2
2 LAN
SAN A
SAN B
2
2
2
2
50% fewer adapters, switches,
and cables
SOLUTION ?
© 2009 IBM Corporation45 FCoE: P. FALCOU / S Monney June 2009
FCoE and CEEEthernetFC
Converged CNA’s(CEE/FCoE Traffic)
Storage
SAN CorporateLAN
Top-of-Rack
SwitchesFCoE
SwitchFCoE
Switch
Unified I/O Use Case
FC HBAs(FC Traffic)
NICs(Ethernet Traffic)
LAN SAN A SAN B
FC Switch
FC Switch
BEFORE AFTER
SOLUTION
© 2009 IBM Corporation47 FCoE: P. FALCOU / S Monney June 2009
Ethernet is standardized as IEEE 802.3 / 1980 / Token Ring ; FFDI ; ARCNET
From CSMA/CD (Carrier sense multiple access with collision detection) on
coax to full duplex fiber on switches
From software bridges to full hardware switches
+ Spanning-tree protocol to maintain the active links of the network as a
tree while allowing physical loops for redundancy.
+ Link aggregation to Add bandwidth to overloaded links
Provide some measure of redundancy, (although the links won't
protect against switch failure because they connect the same pair of switches)
+
Ethernet FLOW CONTROL = FRAME DROP
ETHERNET HAS BEEN IMPROVED ..BUT
© 2009 IBM Corporation48 FCoE: P. FALCOU / S Monney June 2009
The Spanning Tree Protocol is an OSI layer-2 protocol that ensures a loop-free topology for any bridged LAN.
Spanning tree allows a network design to include spare (redundant) links to
provide automatic backup paths if an active link fails, without the danger of bridge
loops, or the need for manual enabling/disabling of these backup links. Bridge loops must be avoided because they result in flooding the network.
The Spanning Tree Protocol (STP), is defined in the IEEE Standard 802.1D. As the name suggests, it creates a spanning tree within a mesh network of
connected layer-2 bridges (typically Ethernet switches), and disables those links
that are not part of the tree, leaving a single active path between any two network nodes
PROTOCOLS AND FEATURES : STP
© 2009 IBM Corporation49 FCoE: P. FALCOU / S Monney June 2009
PROTOCOLS AND FEATURES : STP
A Spanning Tree port can be in one of four states. These include:
Listening. In this state, a port is listening to Spanning Tree messages (BPDUs) and attempting to figure out how the network is configured.
Learning. In this state, a port is adding addresses to its MAC table, butnot yet forwarding frames.
Forwarding. When in this state, a port is sending and receiving data as normal. During normal operation, a port will be in either a forwarding or a blocking state.
Blocking. When in this state, a port will neither send nor receive data, but will listen to network messages relating to Spanning Tree. By default, all ports are in blocking mode when a switch is first powered on.
© 2009 IBM Corporation50 FCoE: P. FALCOU / S Monney June 2009
PROTOCOLS AND FEATURES : RSTP
There are only three port states left in RSTP (Rapid spanning tree) that correspond to the three possible operational states
© 2009 IBM Corporation51 FCoE: P. FALCOU / S Monney June 2009
PROTOCOLS AND FEATURES : RSTP
RSTP bridge port roles:
Root - A forwarding port that has been selected for the spanning-tree topology
Designated - A forwarding port for every LAN segment
Alternate - An alternate path to the root bridge. This path is different than using the root port.
Backup - A backup/redundant path to a segment where another bridge port already connects.
Disabled - Not strictly part of STP, a network administrator can manually disable a port
© 2009 IBM Corporation52 FCoE: P. FALCOU / S Monney June 2009
Recover bandwidth, multiple active paths; no spanning tree L2 Multipathing: TRILL in IETF
Feature/Standard Benefit
Priority Flow Control (PFC)IEEE 802.1Qbb
Enable storage and networking traffic types to share a common CEE link without interfering with each other ( -> ballot letter
11.09)
ETS: Enhanced Transmission
Selection (Bandwidth Management) IEEE 802.1Qaz
Enable bandwidth management by assigning bandwidth segments to different traffic flows ( -> ballot letter 11.09)
Congestion Management IEEE 802.1Qau
End-to-end congestion management for L2 network (-> approval 11.09)
Data Center Bridging Exchange Protocol (DCBX)
Management protocol for CEE
CEE
* Transport for FCoE storage traffic
* Lossless Ethernet that creates the potential for I/O consolidation at the server
edge by transporting storage and networking traffic over CEE
*Jumbo frames required
*Pause to handle congestion IEEE 802.3 multicast MAC address of 01-80-C2-00-00-01.
FCoE PROTOCOLS AND FEATURES
© 2009 IBM Corporation53 FCoE: P. FALCOU / S Monney June 2009
Priority Flow Control (PFC):
� Standard track in IEEE 802.1Qbb (04/09)
� Enables the use of PAUSE command per priority, using Ethernet priorities 0-7 assigned to various traffic flows
� “No drop” attribute
� Avoids the shortcomings of link-level PAUSE
� Eliminates traffic interference among priority flows
Bandwidth management (ETS) (07/09)
� Standard track in IEEE 802.1Qaz
� Assigns one or more priorities to a Priority Group
� Allocates bandwidth to assigned priority groups using percentage values
Priority 0: FCoE
Priority 1: FCoE
Priority 2: LAN
Priority 6: User x
Priority 7: User Z
Priority Group 1: Storage 60%
Priority Group 2: LAN 30%
Priority Group 3: IPC 10%
PROTOCOLS AND FEATURES
© 2009 IBM Corporation54 FCoE: P. FALCOU / S Monney June 2009
CEE
PFC (Priority flow control): Link level flow control is the only way to guarantee zero
loss due to congestion
802.3 pauses all traffic including control traffic.
PFC affect only traffic that need it after PFC is negociated using DCBX 802.3 shall not be
used (Don’t generatre and Ignore pause frames)
PROTOCOLS AND FEATURES
© 2009 IBM Corporation55 FCoE: P. FALCOU / S Monney June 2009
DCE Bridging Capability Exchange Protocol
DCBX protocol is an extension of the Link Layer Discovery Protocol (LLDP).
DCBX end points exchange request and acknowledgment messages.
For flexibility, parameters are coded in a type-length-value (TLV) format.
Used between switches to handle DCE parameters : known and coherent
parameters (COS , bandwidth , lossless? ,…
During normal operation of FCoE between the switch and the adapter, the DCBX
protocol provides link-error detection.
DCBX is also used to negotiate capabilities between the switch and the adapter
and to send configuration values to the adapter.
The capability reduces the possibility of configuration error and simplifies administration of the adapters.
STILL under development in IEEE
PROTOCOLS AND FEATURES : DCBX
© 2009 IBM Corporation56 FCoE: P. FALCOU / S Monney June 2009
� Why DCBX?
– Defines the limits of the CEE-capable cloud
– Detects misconfiguration between peers
– Can be used to configure a peer
� DCBX enhances LLDP with additional TLVs
– PFC TLV
– ETS TLV
– FCoE Application TLV
PROTOCOLS AND FEATURES : DCBX
© 2009 IBM Corporation57 FCoE: P. FALCOU / S Monney June 2009
Transparent Interconnection of Lots of Links (TRILL):
The TRILL WG will design a solution for shortest-path frame routing in multi-hop IEEE 802.1-compliant Ethernet networks with arbitrary topologies, using an
existing link-state routing protocol technology.
Eliminate Spanning tree
The design should have the following properties:
- Minimal or no configuration required
- Load-splitting among multiple paths
- Routing loop mitigation (possibly through a TTL field)- Support of multiple points of attachment
- Support for broadcast and multicast- No significant service delay after attachment
- No less secure than existing bridged solutions
May 2009 Submit base protocol specification to IEEE/IETF expert review
Jun 2009 Base protocol specification submitted to the IESG for publication as
a Proposed Standard RFC
PROTOCOLS AND FEATURES : TRILL
© 2009 IBM Corporation59 FCoE: P. FALCOU / S Monney June 2009
An Introduction to Fibre Channel over
Ethernet, and Fibre Channel over Convergence Enhanced Ethernet
http://www.redbooks.ibm.com/redpieces/abstracts/r
edp4493.html?Open
REFERENCE
© 2009 IBM Corporation60 FCoE: P. FALCOU / S Monney June 2009
FCoE Terminology
CEE: Converged Enhanced Ethernet or
DCE: Data Center Ethernet
DCBX: Data center bridging exchange
FCoE: Fibre channel over ethernet
LAG: Link aggregation Group
LLDP: Link Layer discovery portocol
VLAN: Virtual Local area network
VN_Port: FCoE equivalent of a N_port
VF_Port: FCoE equivalent of a F_port
Enode: FCoE servers and storage
FCF: FCoE copnverter
FIP: FCoE initialisation portocol
CNA: Converged Network Adapters (CNAs)