Fibre Channel over Ethernet (FCoE) - Advancing Storage · PDF fileFCoE Fabrics must be built...

Fibre Channel over Ethernet (FCoE)

John L Hufferd, Brocade

Fibre Channel over Ethernet (FCoE) © 2008 Storage Networking Industry Association. All Rights Reserved. 2

SNIA Legal Notice

The material contained in this tutorial is copyrighted by the SNIA. Member companies and individuals may use this material in presentations and literature under the following conditions:

Any slide or slides used must be reproduced without modificationThe SNIA must be acknowledged as source of any material used in the body of any document containing material from these presentations.

This presentation is a project of the SNIA Education Committee.


Abstract

A new concept is currently moving through the Fibre Channel (T11) standards committee called Fire Channel over Ethernet (FCoE). The FCoE standard will specify the encapsulation of Fibre Channel frames into Ethernet Frames and the amalgamation of these technologies into a network fabric that can support Fibre Channel protocols and other protocols such as TCP/IP, UDP/IP etc.The tutorial will show the Fundamentals of the FCoE concept and describe how it might be exploited in a Data Center environment.


Agenda

Introduction

Goals & Requirements

Consolidation

Architecture

Discovery & Link Instantiation

Topologies

Scenarios

Summary


IntroductionThis presentation provides an overview of Fibre Channel over Ethernet (FCoE)

One should think about FCoE as placing the FC protocol on a new physical link

Ethernet links instead of physical FC linksBut it is still Fibre Channel

The protocol is being defined in the INCITS Fibre Channel (T11) technical committee

Many details of the protocol still need to be defined but the significant major issues have been resolved

Target T11 Standards completion is the 2H08


The Origins of Data Center Ethernet

The technology has evolved continuously, showing a great ability to adapt to new technologies and increasing business requirements

Increasing Scalability, Feature, Function

Incr

easi

ng P

erfo

rman

ce Integration of Carrier-grade features

Incremental Protocol Enhancements

Logical Partitioning

Evolution from shared media to dedicated media

1973

2008-2009 CEE

Introduction of Ethernet

10Mbps

100Mbps

1Gbps

10Gbps(Converged Enhanced Ethernet)


Goals/Requirements


FCoE Goals/Requirements (part 1)

FCoE Fabrics must be built with FCoE Switches Switches with Ethernet ports that provide FCoE capabilities and services – Called, in the standard, an FC Forwarder FCFSwitches that include the functions of traditional FC switchesStandard Ethernet switches may also exist in the fabric but switches with FCoE capabilities are required

FCoE fabrics must operate seamlessly with real FC Fabrics

FC services must operate identically on FCoE fabrics and Fibre Channel fabrics

FCoE must support all Fibre Channel advanced features (e.g. virtual fabrics, IFR, security, etc.) transparently

FCoE is NOT a replacement for FCIP or iFCPFCIP & iFCP use TCP/IPFCIP/iFCP is for inter-switch links beyond the Data Center


Integrated Ethernet & FCoE switch (FCF) with FC connectionsSome implementations may combine the features and capabilities of an Ethernet Switch with the features and capabilities of a FC switch

• Support Ethernet and IP standards for switching, pathing and routing

• Support FC standards for switching, pathing and routing

• Support current and enhanced Ethernet Standards

• Adapt between FCoE and FC

An FCoE Port (N_Port, F_Port or E_Port)

• Has the same function as in FC

• But is layered on top of Ethernet

• Called VN_Port, VF_Port and VE_Port

Because many logical (virtual)Ports can share one physical port

Note:FCF Pathing and Forwarding utilizes the FSPF (Fabric Shortest Path First) protocolNon-FCoE Ethernet traffic is relayed using conventional 802.1 defined mechanisms such as STP (Spanning Tree Protocol) and MSTP (Multiple STP)

See additional FSPF and STP explanation in Appendix

FCFCFC

CEE Ethernet Ports (with IP & FCoE VF_Port &

VE_Port capabilities)


Connections to a Combo FCoE Switch

Applications

TCPUDPIP

SCSI

Fibre Channel

FCoE

Lossless Ethernet MAC (CEE)

• Fibre Channel is carried over lossless Ethernet as a L3 protocol

Combo Lossless Ethernet (CEE) Switchwith FCoE Switch (FCF) capabilities

(FCoE VN_Port)

Ethernet port with IP & FCoE VF_Port capabilities

IP address 123.45.67.89

iSCSI

SCSI



FCoE requires specific Ethernet extensions to be implementedLossless switches and fabrics (e.g., supporting IEEE 802.3 PAUSE) configurations are requiredJumboframe support is required (not a standard, but widely available)

Deployments of FCoE should utilize the advances in Ethernet currently being discussed in IEEE 802.1, specifically:

Priority-based Flow Control (PFC)Enhanced Transmission Selection (ETS)DCB Capability eXchange Protocol (DCB CXP)

These 802.1 advances are important for Consolidated Flows (Messaging, Clustering and Storage)This set of functions has been called Data Center Ethernet, or CEE –

Converged Enhanced Ethernet (intended for a Data Center Environment)

FCoE should require no changes to FC software


Consolidation


Dramatic reduction in adapter, switch ports and cabling4-6 cables to 2 cables per server

Seamless connection to the installed based of existing SANs and LANsRequires high performance lightweight frame mappers vs. heavy weight gateways

FCoE has no need to terminate and re-initiate a SCSI connection(e.g. iSCSI to FC)

Effective sharing of high bandwidth links

High End 10GE Server & NIC/HBA Consolidation

Today

With CEEOS3 DB Server

OS2 App Server

OS1 Web Server

Messaging

MPIRDMA

FC HBA

OS3 DB Server

OS2 App Server

OS1 Web Server

IB/Ethernet Cluster

Hyper Visor(vmWare, Xen, etc)

E-HBA(CEE)

Data CenterCEE Network

EthernetTCP/IP

FC SAN

Data CenterCEE Network

E-HBA(CEE)

E-HBA(CEE)

•NIC•TCP acceleration.•MPI, RDMAover Ethernet

•FCoE

OS3 DB Server

OS2 App Server

OS1 Web Server


Architecture


FC Encapsulation Into Network Packets (2 FCoE Related Packet types)

IEEE 802.3Layers

FC Levels(Unchanged)

FCoE Mapping

FC-4

FC-3FC-2

FC-1

FC-0

FC-4

FC-3FC-2

MACPHY

Frame Check Sequence(CRC)

Protocol control information: Version, SOF, EOF, etc. FC Imbedded Frames: Same as in Physical FC

EthernetHeader FCSFCoE

HeaderFC Header SCSI Commands/Data

Ethertype “FCoE”

Fibre Channel over Ethernet (Fibre Channel over Ethernet (FCoEFCoE) Packets) Packets

Protocol control information: Version, Op-codes, etc. Discovery and Login/Logout Parameters

EthernetHeader FCSFIP

HeaderDescriptors

Ethertype “FIP”

FCoE Initialization Protocol (FCoE Initialization Protocol (FIPFIP) Packets) Packets

Ethernet Header provides things needed for the physical network, including “Ethertype”


FIP Protocol and FCoE Protocol

Discovery PhaseFCFs Discovery each other and form a FabricENodes and FCFs Discover

Potential VN_Port VF_Port pairingCapabilities of Potential pairing

Login PhaseENodes chose among discovered FCFs’ PortsCreates association between ENode Ports and FCF Ports

VN_Port VF_Port Logical FC LinkTwo allowed alternatives for the ENode MAC Addressing

Fabric Provided MAC Addresses (FPMA)Server Provided MAC Addresses (SPMA)Chosen by FCF (FPMA & SPMA Described latter)

Uses: FLOGI, FLOGI ACC, LOGO, …

Data Transfer PhasePLOGI/PRLIAll other FC protocol frames (ELS, FC4 ULPs. etc.)

FCoE Initialization Protocol (FIP)

FCoE Protocol


FIP Operation FormatWord 31-24 23-16 15-8 7-0

0 Destination MAC Address (6 Bytes)

12 Source MAC Address (6 Bytes)

3 ET=FIP (16 bits) Ver (4b) Reserved (12 bits)

4 FIP Operation Code5 Reserved FIP subcode Descriptor List Length

6 Flags F P

S P

S F

PAD to minimum length or mini-Jumbo length

…

n

n+1 Ethernet FCS

Optional IEEE 802.1q4 ByteTag goes here

Descriptor listvariesIn size

Ethernet framesizeIs 64Bytes to 2220Bytes

FIP Operation Code Reserved FIP SubCode

Descriptor List Length Flags FP SP S F

Descriptor List

Solicited bit FCF bitCapability Bits (SPMA or FPMA)

See Appendix for Descriptor list items


FC’s Encapsulation in Ethernet (FCoE)Word 31-24 23-16 15-8 7-0

0 Destination MAC Address (6 Bytes)

12 Source MAC Address (6 Bytes)

3 ET=FCoE (16 bits) Ver (4b) Reserved (12 bits)

4 Reserved5 Reserved6 Reserved SOF (8 bits)

7 Encapsulated FC FrameFC Frame = Minimum 28 Bytes (7 Words)

Maximum 2180 Bytes (545 Words)(including FC-CRC)

…

n

n+1 EOF (8 bits) Reservedn+2 Ethernet FCS

Optional IEEE 802.1q4 ByteTag goes here

This field varies

In size

Ethernet framesizeIs 64Bytes to 2220Bytes


FCF Model

Link End

Point (LEP)


ENode (HBA) Model

Each ENode (HBA) may have multiple Physical Ethernet PortsEach Physical Port may have multiple Logical VN_Ports

Link End

Point (LEP)


Model of the ENODE with Multiple Logical FC interfaces

FCoEController

Lossless Ethernet MAC Ethernet_Port

FCEntityVN_Port

FCoEEntity

FCoE_LEP

FC-3 /FC-4s

MAC Address of “Burnt-in

MAC

FCEntityVN_Port

FCoEEntity

FCoE_LEP

FC-3 /FC-4s

. . .

• For each logical N_Port (VN_Port) there is one FLOGI and perhaps 100’s of FDISC

• Each VN_Port is seen by the Host as a separate (logical) FC connection

• The number of (logical) FC connections is implementation dependent

• Only one MAC Address is required for the FCoE Controller and the VN_Ports on a single physical MAC (aka Server Provided MAC Address – SPMA)• FCF may chose to specify new MAC addresses for each VN_Port (aka Fabric Provided MAC Address – FPMA)

In this model this is where

FC-2 functions

live

MAC Address of FCoE_LEP(VN_Port)

May or may not be the same as the FCoE controller

In this model this is where the

Encapsulation /De-Encapsulation

functions live

Multiple FC NPIV instances on a single

logical FC Host interface


The Logical FC Link is defined by a MAC Address pair• A VN_Port MAC Address • A VF_Port MAC Address

For a logical FC link the FCoE Frames are always sent to and received from a specific FCF’s MAC Address• Therefore, pathing to and from the FC driver is always defined by the MAC Address of the partner FCF’s VF_Port

Multiple Logical FC connections via a single Ethernet MAC

Examples of single MACs with connections to two different FCFs Switch


Functions of an FCoE Initiator ASIC

NIC Function

FCoE

function

FCFunction

Host PCIe interface

ExternalPort

FCoE

ASIC

• Has a Normal NIC interface (A) to the Host

• Has one or more Normal FC interfaces (B,C) to the Host

• FCoE functions not seen by the Host

• FCoE functions perform the Encapsulation and De- encapsulation

• The FCoE function Instantiates a Logical FC N_Port, called a VN_Port

Lossless Ethernet

MAC

A B C


HBA with Multiple Logical FC Interfaces

• MAY have one “burnt-in MAC address for both IP and FCoE/FIP packets

Or

• MAY have different “Burnt- in” MACs for IP and FCoE/FIP packets

• Used to separate HW based FCoE from other Ethernet Traffic

• Most NICs come with several “Burnt-in” MAC Addresses

• The FCoE controller will perform the FIP functions and will instantiate new VN_Ports as FCoE Link End Point (LEP)• With the same MAC address as the FCoE Controller (SPMA)• Or with a new MAC address specified by the FCF (FPMA)

FCoE

Controller

FC-3 /FC

-4sFCEntity

VN_Port

FCoE

EntityFC

oE_LEP

Lossless Ethernet MACEthernet_Port

NIC

Etype=FCoE

or FIP?

FCoE Chip


MAC

FCoE Function

FC-3 /FC

-4s

FCEntity

VN_Port

FCoE

EntityFC

oE_LEP


MAC

A B C


FIP (FCoE Initiation Protocol) Discovery and Link Instantiation


ENode to FCF Logical Links

FCfabric

H1

H2 ClassicalEthernet

LAN

FCF B

LosslessEthernetSwitch

FCF A

VN_Ports,VN_Port_Names

FCoE_LEP

VN_Port

FCoE_LEP

VN_Port

FCoE_LEP

VN_Port

FCoE_LEP

VN_Port

FCoE_LEPs

VF_Port

FCoE_LEPs

VF_Port

VF_Port,VF_Port_Name

FCoE_LEPs

VF_Port

FCoE_LEPs

VF_Port

FCF-MAC(A)

FCF-MAC(B)

MAC(H1)

MAC(H2)

FCF-MAC(x): A MAC address of a Lossless Ethernet port of FCF xMAC(y): A MAC address of a Lossless Ethernet port of ENode y

Note: The FCFs must discover each other and create a Fabric before the ENode Discover FCFs

An ENode must Locate FCFs with a Discovery protocol and then establish a Logical FC Link with an FCF (VN_Port VF_Port) before an Normal FC frame flow


Initial Login Flow ladder (2 Phases)

End-Node FCF

Discovery Multicastto “ALL FCFs”Unicast FCF MAC Address et.al. in Jumbo Frame

FLOGI

FLOGI ACC with the FCF’s chosen VN_Port MAC address as a descriptor value

FC Command(Using the FCF selected MAC Address as the SA) FC Command responses

(using the command’s SA as a DA)

Discovery Phase

Login Phase

Normal FC Processing

FCoE Initialization Protocol

FCOE ProtocolSee Appendix for more details in Discovery Consideration and Actions


Subsequent Login Flow ladder (1 Phase)

End-Node FCF

FLOGI

Login Phase

Normal FC Processing

FCoE Initialization Protocol

FCOE ProtocolSee Appendix for more details in Discovery Consideration and Actions

FC Command(Using the FCF selected MAC Address as the SA)

FC Command responses(using the command’s SA as a DA)

FLOGI ACC with the FCF’s chosen VN_Port MAC address as a descriptor value


Topologies


FCoE Switch & FC Fabric

FCFCFCFCoE Switch

An FCoE Switch may connect to a normal FC switchVia the FC E-Port Ethernet

FC

Note: FCoE servers and storage will probably use an FCoE HBA (or chip set)


Multiple FCoE Switch (FCF) Topologies

EthernetFC

FCFCFCLossless Ethernet Fabric

FCFCFCLossless Ethernet

Fabric

FCoE Switch

A Lossless Ethernet Fabric can be made up of Combo FCoE SwitchesLossless Ethernet switches configured into a Lossless Ethernet

Fabric can Front the FCoE Switch

FCFCFC

FCoESwitch

FCoESwitch

FCoESwitch

Lossless Ethernet

FCoE Switches deployed at the edges of the Lossless Ethernet FabricFCoE Switches connected via VE_Ports and Lossless Ethernet

FCFCFC

FCoESwitch

FCoESwitch

LosslessEthernet

1

LosslessEthernet

2

LosslessEthernet

4

FCoESwitch

LosslessEthernet

3

A VE_Port in an FCF may connect to other VE_Port in another FCFAnd an FCF FC E_Port may connect to an FC switch E_Port


Status of Current Data Center Networking

There can be 3 different networks

System Area Network (SyAN)Used for Clustering/Low Latency

Storage area Network (SAN)Used to Access to Storage

LAN/WAN External NetworksUsed for General messagingUsed for Client-Server MessagingUsed for NAS

Often divided into at least 3 management domains

Data Center Server (clustering) NetworkData Center Storage NetworkOutfacing (IP) Network

LAN/WANs– Messaging– NAS

Data Center

Remote Offices

System Area Network (SyAN)•Clustering Fabric•InfiniBand•Myrinet•Ethernet•CTC

Data CenterServer & Storage Network

Management GroupOutfacing (IP) Network

Management Group

LAN/WAN •Messaging•NAS

Storage Area Network- Fibre Channel

Note: with multiple Data Centers there may also be interconnects with DWDM, FCIP/iFCP, etc.


Current FabricsRemoteRemoteOfficesOffices

Clustering Network

There is a FC Storage Network, a Clustering Network & an “Outfacing” NetworkIP Network Management Group

FC

FC

FC

Ficon

Ficon StorageController

Mainframe

Data Center Server & Storage Network Management GroupFile Storage Arrays (NAS)

iSCSIStorage

FCNetwork

Ficon

(LAN/WAN)(LAN/WAN)••MessagingMessaging••NASNAS

Outfacing IP Network

Local & Remote Local & Remote Business CampusBusiness Campus

Focus: Low Latency & High

Bandwidth

Focus: Protection, Bandwidth/Congestion

Management

FC Link

EthernetLink

Ficon Link

Clustering Network



View of the Future Data Center Networks

Remote Remote OfficesOffices

Data Center Server & Storage Network

Management Group

Outfacing (IP) NetworkManagement Group

Storage & System Storage & System Area NetworkArea Network••Logically Single FabricLogically Single Fabric

(with FCoE)(with FCoE)

Other IP NetsOther IP Nets(LAN/WAN)(LAN/WAN)••MessagingMessaging••NASNAS

The The Server/Storage NetworksServer/Storage Networks will become a Consolidated will become a Consolidated FabricFabric

Managed by the Data Center Managed by the Data Center System and Storage System and Storage Management GroupManagement Group

–– Includes Storage and Includes Storage and clustering provisioningclustering provisioning

–– SAN and SyAN managed SAN and SyAN managed as a single fabricas a single fabric

Focus: High Bandwidth and Focus: High Bandwidth and Low LatencyLow Latency

The Outfacing (The Outfacing (IP) Management IP) Management GroupGroup remains the sameremains the same

Focus: Protection, Bandwidth Focus: Protection, Bandwidth and Congestion Managementand Congestion Management



Phase in: FC & High Performance Lossless Ethernet Unify into a Data Center Fabric

Remote Remote OfficesOffices

FC

FC

FCoE

FC

(LAN/WAN)(LAN/WAN)••MessagingMessaging••NASNAS

Outfacing IP Network

Ficon

iSCSIStorage

Ficon

Ficon StorageController

Mainframe

File Storage Arrays (NAS)

• FCoE permits intermixing of multiple Connection types/protocols• Clustering messaging, General Messaging, and Storage

• The DataCenter Fabric will “Trunk” to the “Outfacing” Network (including iSCSI sys)• Some Customers may want keep a mixed environment on-going

DataCenterFabric

EthernetSW

EthernetSW

FC & EthernetSW Blades

FC Link

EthernetLink

Ficon Link

Business Campus

with iSCSI connections

Including iSCSI Gateways

Note: with multiple Data Centers there may also be interconnects

with DWDM, FCIP/iFCP, etc.


Scenarios


Scenario 1: FCoE & IP Flows

FCFCFC

LosslessEthernet

ClassicalEthernet

FCoESwitch

Internet

FCoESwitches

LosslessEthernet

FCoE Flows

IP Flows


Scenario 2: FCoE Right & Wrong

FCFCFC

LosslessEthernet

ClassicalEthernet

FCoESwitch

Internet

FCoESwitches

LosslessEthernet

FCoE Flows

Invalid FCoE Flows


Summary


Summary

FCoE is a simple, efficient mechanism for encapsulating Fibre Channel in Ethernet frames

FCoE is being standardized in INCITS Fibre Channel (T11) technical committee

Target completion is 2H08

Maximum benefit of Fibre Channel is achieved:Evolutionary model of FC Switches and FC SANsEmphasis placed on capitalizing on the benefits of Converged Enhanced Ethernet (CEE)

Being discussed in the IEEE 802.1 standards working group

Thank You!


Q&A / Feedback

Please send any questions or comments on this presentation to SNIA: [email protected]

For additional information refer to http://www.t11.org/fcoe

Many thanks to the following individuals for their contributions to this tutorial.

SNIA Education Committee

Claudio DeSanti Howard Goldstein Walter DeyRobert Snively Suresh Vobbilisetty Silvano GaiJoe Pelissier John Hufferd

mailto:[email protected]

http://www.t11.org/fcoe


Appendix

Additional RequirementsFCoE Relation to ISO LayersFlowsAdditional TopologiesFSPF and STPFIP Considerations and ActionsFIP DescriptorsPause vs. BB_Credit



FCoE must be a direct mapping of Fibre Channel over an Ethernet networkFCoE must be layered on top of Ethernet

FSPF used to route FCoE packetsEthernet Spanning Tree (STP), MSTP, etc, is at a layer below

FCoE to allow an evolutionary approach towards consolidation of fabrics

The Fibre Channel N_Port, F_Port and E-Port constructs must be retained

With FCoE, ports may be connected with Logical Ethernet Links– May pass through Ethernet switches– Identified by pairs of end point MAC addresses

Physical Ethernet Links can replace physical FC Links Physical Ethernet Links can carry all Ethernet traffic, including FCoE, but combined traffic needs the CEE capabilities



“Combo FCoE Switches” may be built that support normal Ethernet traffic, FCoE traffic , & FC trafficThe FCoE solutions should appear as a Fibre Channel to a Fibre Channel experienced customerFCoE should keep the Fibre Channel operations independent from Ethernet forwarding

Keeps management /Troubleshooting simpleCommon physical structures, different logical structures

Based on Ethertype (Ethertype = FCoE)

Storage Management should be unchanged


FCoE Relation to ISO Layers


Flows


Logical Fabric Topology

FCFCFC

FCoESwitch

FCoESwitch

FCoESwitch

LosslessEthernet

LosslessEthernet

A

H1

H2

H3H4 S2

EthernetDestination

& Source

EncapsulatedFC Frame

D_ID

S_ID

FCoE-A MACFCoE-H2 MAC

FC_ID for S1FC_ID for H2




FCoE-B MACFCoE-A MAC

FCoE-C MACFCoE-B MAC

FCoE-S1 MACFCoE-C MAC

EthernetFC

LogicalTransaction Path

An FCoE Switch receives FCoE frames addressed to its FC-MAC address and forwards them based on the D_ID of the encapsulated FC frame

An FCoE Switch rewrites the SA and DA of an FCoE frame

LosslessEthernet

LosslessEthernet

S1

Path #1 Path #2 Path #3 Path #4

#1#2

#3

#4B

C


Single Ethernet Fabric with FCoE Switches

FCFCFCFCoE

SwitchFCoE

SwitchLossless Ethernet

AC

H1

H2

H3

S1

S2

FCoE-A MACEthernet

Destination& Source FCoE-H2 MAC


D_ID

S_ID




FCoE-C MACFCoE-A MAC

FCoE-S1 MACFCoE-C MAC

Ethernet Switch

Ethernet Switch

Ethernet Switch

Ethernet Switch

Ethernet Switch

EthernetFC


Path #1 Path #2 Path #3

#1

#2 #3


FC Host to FCoE Storage

FCFCFCFCoE

Switch

FCoESwitch

Lossless EthernetH1

H2

H3

S1

S3

FCoE-S2 MACEthernet

Destination& Source FCoE-A MAC


D_ID

S_ID




FCoE-A MACFCoE-C MAC

Ethernet Switch

Ethernet Switch

Ethernet Switch

Ethernet Switch

H5S2

Ethernet Switch

EthernetFC


Path #3 Path #2

Path #1

#1

#2

#3

CA


Additional Topologies


Additional Topologies (1)

FCoESwitch (A)

FCoESwitch (B)

FCoESwitch (E)

FCoESwitch (F)

LosslessEthernet Switch


FCFCFC

Example of Topologies with Rack Mount servers


Additional Topologies (2)

FCoESwitch (B)

FCoESwitch (E)

FCoESwitch (F)

LosslessEthernet Switch


FCFCFC

FCoESwitch (A)

Equivalent to Blade servers with N_Port_ID Virtualization (NPIV) SupportExample of Topologies with Blade Servers


FSPF and STP


FSPF & STP Concepts with FCoE (basic)


FSPF & STP Concepts with FCoE (interconnected)


Equivalent FC topology


FIP Considerations and Actions


FCoE Discovery Considerations

The Discovery phase of the FCoE Initialization Protocol (FIP) uses two types of messages, Solicitations and AdvertisementsThe FIP Discovery Phase helps define the FCF Ports that are available for the Link instantiation Phase The ENodes discover the FCF ports that can become VF_Ports and FCFs discover other FCF ports that can become VE_Ports

ENodes Solicit (via Multicast) Advertisements from FCFs while specifying their capabilitiesIn response FCF’s Advertise their availability and capabilities back to the ENodesFCF’s Multicast their existent to other FCF

The FIP Discovery phase exchanges solicitation and/or Advertisements between (HBA and/or FCF) “FCoE Controllers”


FCF Discovery ActionsFCF supporting VE_Ports:

Discovers other VE_Port capable FCF-MACs, connected to the same Lossless Ethernet segment, by:

Transmitting a multicast Solicitation to ‘All-FCFs’(with the FCF bit set to one)

Receives back Jumbo Unicast Advertisements from VE capable MACs

To verify the support of Ethernet Jumbo frames in the pathIn response to receipt of a Multicast to “All-FCFs” from another FCF

Has its own VE capable MACs sends their own Jumbo Unicast Advertisements

Instantiates VE_Port to VE_Port connections and Exchanges FC ELP (Extended Link Protocol) and Fabric configuration (using Ethertype=FCoE) with the other FCFs VE_Port capable MACs


ENode ActionsWhen an ENode becomes operational:

The ENode discovers the VF_Port capable FCF-MACs connected to the same Lossless Ethernet segment

Transmits a multicast Solicitation to ‘All-FCFs’ (with the FCF bit set to zero)Receives Jumbo unicast Advertisements from select compatible VF_Port capable FCF-MACsStores the discovered FCF-MACs in an FCF list

When an ENode receives an Advertisement that a new FCF is available, it may send a unicast Solicitation to it and receive a Jumbo unicast Advertisement in reply

To verify the support of Ethernet Jumbo frames in the pathMay then perform FLOGIs (with Ethertype-FIP) to a vendor specific subset of the FCF-MACs in the FCF list


FIP Descriptors


FIP Descriptors (1)


FIP Descriptors (2)


Pause vs. BB_CreditBoth mechanisms are used to avoid dropping frames

With different trade-offs

The Pause mechanism requires at least the (2 x RTT x bandwidth) product on a link as buffer space

But allows Buffer handling in an arbitrary wayWell suited for networks with limited (bandwidth x delay) product (e.g. within the data center)

The Pause frame is handled by the MAC layerSimilar to the R_RDY handling by the FC-1 level

The BB_Credit mechanism prevents loosing frames over any link

But links go under-utilized if link credits (& buffers) are < that needed for (RTT x BW)Requires buffer handling in maximum frame size units

Date post:	20-Mar-2018
Category:	Documents
Upload:	trinhdung
View:	219 times
Download:	2 times

Fibre Channel over Ethernet (FCoE) - Advancing Storage · PDF fileFCoE Fabrics must be built...

Documents