SII14 FCoE Introduction

© 2009 IBM CorporationSTG Technical Conferences 2009

sII14 - Fibre Channel over Ethernet:An Introduction

Jim Blue IBM SAN Central

STG Technical Conferences 2009

© 2009 IBM Corporation2 sII14 – FCoE: An Introduction

Trademarks and DisclaimersThe following are trademarks of the International Business Machines Corporation in the United States and/or other countries. For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml:

The following are trademarks or registered trademarks of other companies:

Java and all Java based trademarks and logos are trademarks of Sun Microsystems, Inc., in the United States and other countries or both

Microsoft, Windows,Windows NT and the Windows logo are registered trademarks of Microsoft Corporation in the United States, other countries, or both.

Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

UNIX is a registered trademark of The Open Group in the United States and other countries or both.

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

Other company, product, or service names may be trademarks or service marks of others.

NOTES:

Any performance data contained in this document was determined in a controlled environment. Actual results may vary significantly and are dependent on many factors including system hardware configuration and software design and configuration. Some measurements quoted in this document may have been made on development-level systems. There is no guarantee these measurements will be the same on generally-available systems. Users of this document should verify the applicable data for their specific environment.

IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.

Information is provided “AS IS” without warranty of any kind.

All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.

This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.

STG Technical Conferences 2009


Trademarks and Disclaimers (continued)NOTES:

All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.

Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

Prices are suggested US list prices and are subject to change without notice. Starting price may not include a hard drive, operating system or other features. Contact your IBM representative or Business Partner for the most current pricing in your geography.

Any proposed use of claims in this presentation outside of the United States must be reviewed by local IBM country counsel prior to such use.

The information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.

IBM makes no representation or warranty regarding third-party products or services including those designated as ServerProven, ClusterProven or BladeCenter Interoperability Program products. Support for these third-party (non-IBM) products is provided by non-IBM Manufacturers.

IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive, Armonk, NY 10504-1785 USA.

STG Technical conferences 2009


Agenda

Overview of Fibre Channel over Ethernet (FCoE) as a product

Protocol Summary

Based on – Version 2 of FC-BB5 (ratified)

– July 2009 versions of select IEEE standards drafts

– FC-BB6 is already in development



What Is Fibre Channel over Ethernet?

A direct mapping of Fibre Channel upper layers onto Ethernet as a transport mechanism

Aims to reduce cost through sharing of Ethernet infrastructure

New standard specified by ANSI T11 Committee within FC-BB-5

Ethernet enhancements are being defined by IEEE in the Ethernet standards

Backed by virtually entire storage networking industry, including IBM, Brocade, Cisco, QLogic, HP, and Intel



General Characteristics

Simple – No complicated flow control or frame loss detection

– Entire standard 44 pages (FC-BB_E, Section 7)

Will make use of enhancements being added to Ethernet standards

Alternative to native Fibre Channel

Generally intended for, but not limited to, 10Gb or faster Ethernet



Basic FCoE Installation



Why use FCoE?

Large hosts generally use two networks right now: Ethernet and Fibre Channel

FCoE eliminates need for additional Host Adapters– Reduces Cost

– Reduces slot and bus space usage• Very helpful for Blades



Protocol / Stack Comparison

iSCSI iFCP or FCIP FCP FCoE



FCoE vs. FCIP/iFCP

Not really equivalent

FCoE – Direct replacement for FC-0, FC-1 and limited portions of FC-2 layers while not using TCP/IP

FCIP/iFCP – Generally used for Distance Extension– FCoE unsuitable for long distances or complex

networks (later version of standards may allow NAT implementation)

No FCIP or iFCP storage devices or HBAs

FCoE capable storage devices are very likely



FCoE vs. iSCSI

Frame loss recovery built into protocol stack

Frame loss can quickly become catastrophic

TCP/IP well understood by most IT staff

Storage administrators know Fibre Channel well

Overhead varies – typically higher

Low overhead – similar toFibre Channel

Substantial complexity on top of TCP

Simple encapsulation of Fibre Channel

Allows many hops, lossyconnections and high latency

Local-area, lossless links, no routing allowed

iSCSIFCoE



FCoE vs. iSCSI long-termNo overwhelming advantages for one vs. other at this time for now

FCoE appears to be off to a better start than iSCSI

FCoE appropriate for existing SAN installs– No complex, gateway for non-iSCSI-native systems

FCoE expands the choices for new solutions– Can take advantage of existing IP infrastructure

Lossless Network needed for FCoE helpful to both

And the winner will be ?????



Software vs. Hardware ImplementationsFCoE is available as a software driver for many operating systems

– Primary use is proof-of-concept and interoperability testing

Virtually all production deployments expected to be hardware based

– New “Converged Network Adapters” (CNA)• In reality, FC HBAs with an Ethernet MAC layer or vice-versa • Will appear as another SCSI / RAID controller



FCoE Disadvantages

No flow control (not even buffer credits)– Relies on Ethernet extensions under development

• 802.1Qaz - Enhanced Transmission Selection (ETS) –Sponsor Ballot voted 07/2009 / full ratification by end of 2009 ???

• 802.1Qau -Congestion Notification –Sponsor Ballot target date scheduled for November 2009 and Forwarding target date of March 2010

– Required to deal with potential FC congestion, even if Ethernet is not congested



FCoE Disadvantages

Currently limits transmission rate– 802.3-2008 – PAUSE mechanism (all or nothing)

– 802.1Qbb – Priority Flow Control (PFC) (requires priority tagging to be in place)

Needs support for “small” jumbo frames to work efficiently (frame size of approximately 2500 bytes)

Assumes high-quality network– Lossless Ethernet network required



Possible Industry Deployment

FCoE intended for 10Gb Ethernet links **– 1Gb functional, not recommended

Blades with Ethernet/FCoE “splitter” on in-Chassis switch **

Edge switches ** / Director blades

Entire Directors / Core Ethernet switches

Storage EMC and NetApp have announced that they are working on products

** Products already announced and available



Summary of FCoE as a Product

FCoE, in summary:– A new standard designed to converge Fibre Channel

SANs and Ethernet networks

– Simple in concept and design

– Details still being developed

– Fundamentally different from iFCP and FC/IP

– In competition with iSCSI

Questions? (Information on protocol details to follow)



Protocol Terms and Concepts

FCoE Mapper (FCM) –Provides translation between Ethernet and Fibre Channel addresses

– Present in NICs and switches

FCoE Forwarder (FCF) – Ethernet switch that understands Fibre Channel and FCoE encapsulation

– Also provides FCoE discovery and login services

– Does not need actual Fibre Channel ports

– Provides same services as FC switch (FC-SW4)

FCoE ENode (ENode) – Ethernet port w/ FCoE abilities (software, hardware or both)



Terms and Concepts

FC-MAP – Fabric ID– Usually 0E:FC:0z (z = a Fabric ID number, default is 0)

– Used for first three bytes of Fabric-Assigned MAC Addresses

Once Discovery and FLOGI (similar to normal FC operations) are completed, FCoE_LEP (link end point) is created at CNA and in the FCF for virtual ports and regular FC operations continue.



Terms and Concepts

FCoE_LEP (Link End Point) – FCoE component which handles FC frame encapsulation or decapsulation as well as the transmission or reception of encapsulated frames across a single virtual link. Each LEP has its own MAC unique MAC address.

FCoE_LEP keeps up with 3 items as well as encapsulation duties:− MAC address of itself− Associated N_Port ID− MAC address of FCoE_LEP at other end of virtual link



Terms and Concepts

Virtual Ports – The port types associated with FCoE_LEP instances that provides the functionality of physical FC port types. Valid types are: − VN_port (on the ENode)

− VF_port (FCF port to VN_port)

− VE_port (FCF to FCF)



FCoE Standard

Listed in ANSI T11, FC-BB-5, Chapter 7

Two protocols in one standard:– FCoE – The encapsulation standard

– FCoE Initialization Protocol (FIP) – Protocol to provide device discovery and login services

– Use different EtherTypes• Facilitates FIP “snooping” by non FCF switches



FCoE and the Protocol Stack



FCoE Frame Format

FC Frame Delimiters start with “K28.5” which must be changed in the FCoE header and trailer

“Mini-Jumbo” Ethernet frames strongly suggested to carry maximum FC payload without needing to fragment

2 new EtherTypes:

8906 – FCoE type

8914 – FCoE Initialization Protocol (FIP)



FCoE Frame Forwarding

Relies on Ethernet Spanning Tree Protocol (STP) or variations of Spanning Tree mechanism

– Rapid STP (RSTP)

– Per VLAN STP (PVST) Cisco proprietary

– Multiple STP (MSTP)

No built-in routing such as FSPF

Load balancing generally poor, but should improve as standards are developed and vendors add unique refinements/enhancements



More-Complex FCoE Topology is on the horizon

SAN A

SAN B

FCF Ethernet

Not a redundant network design!



FCoE Network Without Much FC is also coming



FCoE Network Currently Supported and working



Addressing of Virtual ports

Two options– Server Provided MAC Address (SPMA)

• Typical Ethernet standard globally unique address• Some HBA vendors support, no FCF vendors yet

– Fabric Provided MAC Address (FPMA)• Format XX:XX:XZ:YY:YY:YY

– XX:XX:XZ = FC-MAP (Z = Domain ID, 0E-FC-00 is default)– YY:YY:YY = FCID (or N_Port ID assigned during FLOGI)

No relation to MAC of the lossless Ethernet adapter



Addressing (cont.)

Multiple MACs per adapter expected– One for regular Ethernet traffic (i.e. TCP/IP)

• ENode MAC Address– One for each occurance of FCoE_LEP

• VN_Port MAC Address

NPIV supported



FCoE Discovery

Special Fabric Initialization Protocol (FIP) frames– Separate from regular FCoE frames

– Used for FLOGI/FDISC, LOGO, ELP and FCoE discovery solicitation/advertisement

– Source MAC is the burned-in MAC Address of the lossless Ethernet port

– Initial Destination MAC is multicast MAC for all FCF



FCoE Discovery

Ethernet devices use MAC addresses so well-known names will not work

Ethernet has the concept of Multicast (group of ports)

FCoE uses several multicast addresses – All FCoE MAC 01:10:18:01:00:00

– All ENodes MAC 01:10:18:01:00:01

– All FCF MAC 01:10:18:01:00:02



FCoE Discovery and FC Login Process

1. Discover available VLANs which support FCoE**** Brocade defaults to VLAN 1002 only and ignores FIP VLAN Requests.

2. Send multi-cast Ethernet frame to All-FCF-MACs multicast address with “Discovery Solicitation”payload.– Contains:

• MAC Address used to receive response• The host’s name• Maximum Receive Size (Max_FCoE_Size)• If an FCF itself, includes FC-MAP (Mapped Address

Prefix if using FPMA), similar to FC Domain ID



FCoE Discovery and FC Login Process (cont.)

3. FCF responds with Discovery AdvertisementAdvertisement contains:

– “Priority” – 1-byte value used to determine which FCF-MAC to use, if multiple respond

– MAC Address of the FCF-MAC– If using FPMA, the FC-MAP of the FCF– Name of the Switch– Name of the FabricThis response will be Max_FCoE_Size bytes to verify that the network is capable of handling that size frame



FCoE Discovery and FC Login Process (cont.)4. ENode sends FIP FLOGI

– Just like FC FLOGI, only encapsulated within a FIP frame

– Returned FCID also used to build FPMA MAC Address

– For FCFs, FIP ELP sent instead

For NPIV, FIP FDISC is used– Works just like counterpart

5. The FCF responds with a FIP LS_ACC (or ELP_SW_ACC)− At this point, VF_port at FCF and VN_port on ENode have

been created and are now ready to work with Name Server to register itself and query zoning database.




L/E

MAC

ENode FCF

L/E

MAC

1. FIP Request for FCoE VLAN(s) using “All FCF multicast MAC as SRC

FIP VLAN info (currently only Cisco)




L/E

MAC

ENode FCF

L/E

MAC



2. FIP Discover Solicitation using “All FCF multicast MAC” as Src

3. FIP Discover Advertisement




L/E

MAC

ENode FCF

L/E

MAC





4. FIP FLOGI Request using FCF MAC as Destination




L/E

MAC

ENode FCF

L/E

MAC





4. FIP FLOGI Request using FCF MAC as Destination

5. FIP FLOGI Response which returns ** FCID with new MAC **

FCoE-LEP

VN-Port**

FCoE-LEP

VF-Port

FCID: 04-0B-00

MAC: 0E-FC-00-04-0B-00 All future FCoE traffic will now use this MAC address

Virtual link established oncethe FLOGI is accepted



Ethernet Enhancements for FCoE“Pause” mechanism (described in 802.3x updates) is used to stop and resume Ethernet traffic to prevent input buffer congestion. The Receiving port sends the Pause frame to the transmitter.

FCF

L/E

MAC

L/E

MAC

ENode

FCoE-LEP

VN-Port**

FCoE-LEP

VF-Port

FCF input buffer utilization

100%

0

PauseTime = xxxx

PauseTime = 0

Traffic being sent by ENode

No frames

sent



Ethernet Enhancements for FCoE

Per Priority Pause Flow Control (PFC) in IEEE 802.1Qbb allows the Pause mechanism to be used on an individual traffic class.

FCF

L/E

MAC

L/E

MAC

ENode

FCoE-LEP

VN-Port**

FCoE-LEP

VF-Port

01234567




Congestion Notification being standardized by IEEE 802.1Qau. Congestion Point (CP) detects congestion and signals Reaction Point (RP) to limit associated traffic.



Ethernet Enhancements for FCoEEnhanced Transmission Selection (ETS) being defined in IEEE 802.1Qaz is adding QoS to Priority Groups based on 802.1Q VLAN tag and priority. Priority levels (or class of service) can be mapped into Groups and each priority group can then be allocated a portion of total bandwidth.




IEEE 802.1Qaz also describes Data Center Bridge Capability Exchange (DCBX which primarily enhances Link Level Discovery Protocol (LLDP). Main functionality inlcudes:

DCE peer discovery and link configuration

Detection of configuration mismatches

Negotiation Layer

Enable multipathing and load balancing where possible

Provides link error detection



Summary

FCoE has two parts– FCoE frames – simple Ethernet frame to transport a

Fibre Channel frame.

– FIP protocol – handles device discovery, fabric login functions, and address assignment to allow traffic to start



Special Session – sSM60

"Meet The Storage Experts “Ludwig von Beethoven II

Thursday, 22 October

17:30 to 19:30



Thank you!

Date post:	18-Jan-2016
Category:	Documents
Upload:	innovite
View:	9 times
Download:	0 times

SII14 FCoE Introduction

Documents