Date post: | 13-Jan-2016 |
Category: |
Documents |
Upload: | aron-holland |
View: | 222 times |
Download: | 0 times |
11/05/07 1TDC564-08
TDC 564TDC 564Local Area Networks Local Area Networks
Lecture 8Lecture 8IP-based Storage Area IP-based Storage Area
NetworkNetwork
11/05/07 2TDC564-08
Course OutlineCourse Outline IP over FC (RFC 2625)IP over FC (RFC 2625) IP-SANIP-SAN
iSCCI (RFC 3720)iSCCI (RFC 3720) IP and FC-SAN IP and FC-SAN
InterworkingInterworkingFC Encapsulation (RFC 3643)FC Encapsulation (RFC 3643)FCIP (RFC 3821) – FC over IPFCIP (RFC 3821) – FC over IPiFCP (RFC 4172)iFCP (RFC 4172)
11/05/07 3TDC564-08
RFC 2625 – IP and ARP RFC 2625 – IP and ARP over FCover FC
FC supports multiple higher layer FC supports multiple higher layer protocols, and SCSI is the most widely protocols, and SCSI is the most widely used one.used one.
Needs/Applications of IP over FC:Needs/Applications of IP over FC: Access data in SAN from IP-based serversAccess data in SAN from IP-based servers interworking between NAS and SANinterworking between NAS and SAN
RFC 2625 addresses two issues.RFC 2625 addresses two issues. A scheme to encapsulate IP and ARP A scheme to encapsulate IP and ARP
packets inside the FC frame (as the FC packets inside the FC frame (as the FC payload)payload)
A procedure to resolve the address mappingA procedure to resolve the address mapping
11/05/07 4TDC564-08
IP over FC (RFC 2625)IP over FC (RFC 2625)App-1: accessing SAN from IP-based App-1: accessing SAN from IP-based
serversservers
SAN
Data
IP
L2
PHY
IP
L2
PHY
IP
RFC 2625FC FC
IP
RFC 2625FC
Data
SAN
FC/IP Gateway
FC-basedStorage Device
11/05/07 5TDC564-08
IP over FC (RFC 2625)IP over FC (RFC 2625)(App-2: interworking between SAN and (App-2: interworking between SAN and
NAS)NAS)
SAN
Data
IP
L2
PHY
IP
L2
PHY
IP
RFC 2625FC FC
IP
RFC 2625FC
L2
PHY
IP
Data
IP
L2
PHY
SAN
FC/IP Gateway
FC/IP Gateway
NAS-basedStorage Device
11/05/07 6TDC564-08
FC Address Resolution FC Address Resolution Three steps schemesThree steps schemes
ARP - IP to MACARP - IP to MAC MAC => World-Wide Port Name (WW_PN)MAC => World-Wide Port Name (WW_PN) FC-ARP - WW_PN => Port_IDFC-ARP - WW_PN => Port_ID
Optional Header is used for Network Header by Optional Header is used for Network Header by RFC 2625RFC 2625
D_NAA (4)Dst. Network Address (60)
S_NAA (4)
Src. Network Address (60) 16 bytes
NAA: Network Address Authority
unused (12 bits)IEEE 802.3 MAC Address (48 bits)
Format of Network Header
Format of Network Address (WW_PN)
11/05/07 7TDC564-08
RFC 2625: IP and ARP over RFC 2625: IP and ARP over FC (1)FC (1)
ARP Request (broadcast)- Sender HW (MAC) Address = HW(A)- Sender IP Address = IP(A)- Target HW (MAC) Address = ??- Target IP Address IP(B)
A
B C D
FC switch
FC SAN
11/05/07 8TDC564-08
RFC 2625: IP and ARP over RFC 2625: IP and ARP over FC (2)FC (2)
ARP Reply (unicast):- Sender HW (MAC) Address = HW(B)- Sender IP Address = IP(B)- Target HW (MAC) Address = HW(A)- Target IP Address IP(A)
A
B C D
11/05/07 9TDC564-08
RFC 2625: IP and ARP over RFC 2625: IP and ARP over FC (3)FC (3)
FC-ARP Request:- Requester N_Port_Name (WW_PN)- Requester N_Port_ID- Responder N_Port_Name (WW_PN)- Responder N_Port_ID = ?? A
B C D
unused (12 bits)IEEE 802.3 MAC Address (48 bits)WW_PN
11/05/07 10TDC564-08
RFC 2625: IP and ARP RFC 2625: IP and ARP over FC (4)over FC (4)
FC-ARP Reply:- Requester N_Port_Name- Requester N_Port_ID- Responder N_Port_Name- Responder N_Port_ID
A
B C D
Advantages of IP for SANAdvantages of IP for SAN
11/05/07 11TDC564-08
IP Network CapabilitiesIP Network Capabilities
11/05/07 12TDC564-08
IP-SAN ProtocolsIP-SAN Protocols
11/05/07 13TDC564-08
11/05/07 14TDC564-08
IP - SANIP - SAN
FC -SAN
IP
IP
Storage Devices
11/05/07 15TDC564-08
IP storage networking – carrying storage traffic IP storage networking – carrying storage traffic over IPover IP
Uses TCP, a reliable transport for deliveryUses TCP, a reliable transport for delivery Can be used for local data center and long haul Can be used for local data center and long haul
applicationsapplications Two primary IETF protocols/standards:Two primary IETF protocols/standards:
iSCSIiSCSI – Internet SCSI – allows block storage to be – Internet SCSI – allows block storage to be accessedaccessed over a TCP/IP network as though it were locally over a TCP/IP network as though it were locally attachedattached
FCIPFCIP – Fibre-Channel-over-IP – used to tunnel Fibre – Fibre-Channel-over-IP – used to tunnel Fibre Channel frames over TCP/IP connectionsChannel frames over TCP/IP connections
IP - Storage Area IP - Storage Area Network (SAN)Network (SAN)
IPIP IPIP TCPTCP TCPTCP iSCSIiSCSI iSCSIiSCSI SCSISCSI SCSISCSI DataData DataData
IPIP IPIP TCPTCP TCPTCP FCIPFCIP FCIPFCIP FCFC FCFC SCSISCSI SCSISCSI DataData DataData
11/05/07 16TDC564-08
Internet SCSI (iSCSI) Internet SCSI (iSCSI) iSCSI is a proposed industry standard that allows iSCSI is a proposed industry standard that allows
SCSI block I/O protocols (commands, sequences, and SCSI block I/O protocols (commands, sequences, and attributes) to be sent over a network using the attributes) to be sent over a network using the popular TCP/IP protocol. popular TCP/IP protocol.
A way to access storage across an IP network as A way to access storage across an IP network as though it was locally attached.though it was locally attached.
Transports SCSI protocol commands and data across Transports SCSI protocol commands and data across an IP networkan IP network
Cisco and IBM co-authored original iSCSI protocol Cisco and IBM co-authored original iSCSI protocol draftdraft
iSCSI Protocol is a standard maintained by the IETFiSCSI Protocol is a standard maintained by the IETF IP Storage (IPS) Working GroupIP Storage (IPS) Working Group RFC 3720RFC 3720
iSCSIiSCSI
11/05/07 17TDC564-08
iSCSI over TCP/IPiSCSI over TCP/IP
11/05/07 18TDC564-08
Command SCSI
Command Data Command Data
TCP Header TCP Payload
11/05/07 19TDC564-08
iSCSI ApplicationsiSCSI Applications Connectivity : iSCSI can be used for Connectivity : iSCSI can be used for
DAS or SAN connections. DAS or SAN connections. iSCSI capable devices could be placed on iSCSI capable devices could be placed on
an existing LAN (shared with other an existing LAN (shared with other applications) in a similar way to NAS applications) in a similar way to NAS devices.devices.
iSCSI capable devices could be attached iSCSI capable devices could be attached to a LAN which is dedicated to storage to a LAN which is dedicated to storage I/O (in other words an IP SAN), or even I/O (in other words an IP SAN), or even on a LAN connected to only one on a LAN connected to only one processor (like a DAS). processor (like a DAS).
11/05/07 20TDC564-08
iSCSI Protocol StackiSCSI Protocol Stack
Data
SCSI
iSCSI
TCP
IP
EthernetPHY
Driver
TCP
IP
EthernetPHY
Driver
Data
SCSI
iSCSI
TCP
IP
EthernetPHY
Driver
Data
SCSI
iSCSI
AdapterCard
HostProcessing
Host Bus Adapter (HBA)
Software iSCSIiSCSI TCP Offload
Engine(TOE)
iSCSI Host Bus Adapter
(HBA)
iSCSI Performance iSCSI Performance ConsiderationConsideration
11/05/07 21TDC564-08
iSCSI BenefitiSCSI Benefit
11/05/07 22TDC564-08
Host Bus Adapter (Fibre Host Bus Adapter (Fibre Channel)Channel)
11/05/07 23TDC564-08
QLogic PCI-EXPRESS 4Gb 1PT Fibre Host Bus Adapter – STD FW• PCI-Express • • 1 Port (Fibre Channel)• Data Rate: 4Gb • Standard Firmware
Price Quote: $951Source: http://www.cdw.com
11/05/07 23TDC564-08
Host Bus Adapter (iSCSI)Host Bus Adapter (iSCSI)
11/05/07 24TDC564-08
The QLE4060C iSCSI Host Bus Adapter (HBA) provides connectivity to SANs over Ethernet and TCP/IP network infrastructures for PCI Express bus-based servers. Using a widely deployed and familiar networking technology, iSCSI can lower the total cost of ownership to better manage and support storage networking.
Source: http://www.cdw.orgPrice quote: $815
10/100/1000BaseTPCI
iSCSI StorageiSCSI Storage
11/05/07 25TDC564-08
Promise Technology VTrak M-Class iSCSI RAIDPrice quote: $4,929Source: //www.cdw.com
Network: Gigabit Ethernet (2)Protocol: TCP/IP, iSCSICapacity: 16-drive bay (500G – 1TB per drive)
11/05/07 26TDC564-08
iSCSI iSCSI ApplicationApplicationData
SCSI
iSCSI
TCP
IP
Layer2
PHY
iSCSI
TCP
IP
Layer2
PHYIP
SCSI
FCP
FC 0-2
SCSI
FCP
FC 0-2
Data
Cisco iSCSI Configuration Example
iSCSIGateway
Server
StorageDevice
Multi-TCP SessionsMulti-TCP Sessions
11/05/07 27TDC564-08
HBA: Multi-Path I/OHBA: Multi-Path I/O
11/05/07 28TDC564-08
LUN: Logical [Storage] Units
HBA iSCSI vs. Software iSCSI
No performance advantage of using iSCSI HBA Throughput Round trip time
Significantly higher cost of iSCSI HBA $800 vs. $100 (Server NIC)
What are the advantages of iSCSI HBA? Answer: CPU Utilization (off-load)
Reference: http://download.intel.com/network/connectivity/products/iSCSIcomp0407.pdf
11/05/07 29TDC564-08
Is IP-SAN similar Is IP-SAN similar to NAS?to NAS?
What are the What are the advantages, if any, of advantages, if any, of IP-SAN vs. NAS?IP-SAN vs. NAS?
11/05/07 TDC564-08 30
Performance Analysis: iSCSI vs. NAS
(software based – no HBA)
http://technomagesinc.com/pdf/ip_paper.pdf
iSCSI NFS
11/05/07 31TDC564-08
Sequential Read/Write Tests
Conclusion: 1. Comparable performance in character read/write and block write2. Significant advantage of iSCSI in block read (20-25%)
11/05/07 32TDC564-08
Small Files Read/Write
11/05/07 33TDC564-08
IOGen Test (Emulation of Database)
11/05/07 34TDC564-08
11/05/07 35TDC564-08
FC-SAN vs. iSCSIFC-SAN vs. iSCSI Since the iSCSI appliance attaches to the existing Since the iSCSI appliance attaches to the existing
Ethernet network, NAS and iSCSI are very similar in Ethernet network, NAS and iSCSI are very similar in network architecturenetwork architecture However, the performance would be significantly different.However, the performance would be significantly different.
Both iSCSI and SAN use Block I/O to transport data, Both iSCSI and SAN use Block I/O to transport data, whereas NAS uses File I/O. whereas NAS uses File I/O.
SAN offers better performance (c.f. NAS), but is more SAN offers better performance (c.f. NAS), but is more expensive and requires a higher skill set to implement. expensive and requires a higher skill set to implement. iSCSI and NAS offer better pricing and skills may already iSCSI and NAS offer better pricing and skills may already be in place to implement them. be in place to implement them.
Both SAN and iSCSI offer the performance benefit of Both SAN and iSCSI offer the performance benefit of
Block I/O.Block I/O.
Case Study: FC-SAN vs. iSCSI
http://www.promise.com/marketing/whitepaper/file/2_MClass-iSCSI%20SCSI%20and%20Fibre-10-07-
05.pdf11/05/07 36TDC564-08
4G bps
11/05/07 37TDC564-08
11/05/07 38TDC564-08
FC over IP FC over IP • The encapsulation of Fibre Channel frames into IP
packets and tunneled through an existing TCP/IP network infrastructure, in order to connect remote islands of FC-based SANs
FC SAN IP FC SAN
IPIP IPIP TCPTCP TCPTCP FCIPFCIP FCIPFCIP FCFC FCFC SCSISCSI SCSISCSI
DataData DataData
FC Frame Encapsulation (RFC 3643)
Describes the common Fibre Channel (FC) frame encapsulation format and a procedure for the measurement and calculation of frame transit time through the IP network
Encapsulation HeaderSOFFC Frame BodyEOF
11/05/07 39TDC564-08
FC Encapsulation Header
Q: Why is time stamp needed in the FC encapsulation header?
11/05/07 40TDC564-08
FC Encapsulat
ion
11/05/07 41TDC564-08
FCIP (RFC 3821)Fibre Channel Over TCP/IP (FCIP) describes mechanisms that allow the interconnection of islands of Fibre Channel storage area networks over IP-based networks to form a unified storage area network in a single Fibre Channel fabric
11/05/07 42TDC564-08
FCIP Protocol Stacks
11/05/07 43TDC564-08
11/05/07 44TDC564-08
FC over IP FC over IP
IP SAN
Data
SCSI
FC 0-2 FC 0-2
FCIP
TCP
IP
L2
PHY
IP
L2
PHY
TCP
IP
L2
PHY
FC 0-2
Data
SCSI
FC 0-2
FCP FCP
SAN
an IP tunnel for FC-based SANApplication: interconnect SAN over IP-WAN.
FCIPFCP FCP
FCIP Configuration Example
11/05/07 45TDC564-08
11/05/07 46TDC564-08
Internet Fiber Channel Internet Fiber Channel Protocol (iFCP)Protocol (iFCP)
iFCP – iFCP – RFC 4172 (Sept. 2005)(Sept. 2005) iFCP is a protocol that uses the FCP iFCP is a protocol that uses the FCP
over IP networks to interconnect over IP networks to interconnect FC-based storage devices.FC-based storage devices.
It is considered a It is considered a gatewaygateway protocol, protocol, while FCIP is considered a while FCIP is considered a tunnel tunnel protocol.protocol.
iFCP (RFC 4172) iFCP specifies an architecture and a gateway-to-
gateway protocol for the implementation of fibre channel fabric functionality over an IP network.
This functionality is provided through TCP protocols for fibre channel frame transport and the distributed fabric services specified by the fibre channel standards.
The architecture enables internetworking of fibre channel devices through gateway-accessed regions with the fault isolation properties of autonomous systems and the scalability of the IP network
11/05/07 47TDC564-08
iFCP Network Example
11/05/07 48TDC564-08
11/05/07 49TDC564-08
iFCP (RFC 4172) iFCP (RFC 4172)
IP
Data
SCSI
FC 0-2
FC 0-2
FCP
TCP
IP
L2
PHY
IP
L2
PHY
TCP
IP
L2
PHY
FC 0-2
Data
SCSI
FC 0-2
FCP FCP
Gateway approachApplication: interconnect SAN over IP-WAN.
FCP
iFCPiFCP
iFCP GW iFCP GW
F-port
F-port
11/05/07 49TDC564-08
iFCPiFCP
11/05/07 50TDC564-08
FCIP vs. iFCPFCIP vs. iFCP
11/05/07 51TDC564-08
http://whatis.techtarget.com/magPrintFriendly/0,293813,sid35_gci1096482,00.html
FCIP vs. iFCP
McDATA
11/05/07 52TDC564-08
11/05/07 53TDC564-08
SAN Performance SAN Performance ComparisonComparison
http://www.netapp.com/library/tr/3423.pdf
11/05/07 54TDC564-08
SummarySummary Needs for large storage – continual growthNeeds for large storage – continual growth
10109 9 (G) => 10(G) => 101212 (T) => 10 (T) => 101515 (P) => 10 (P) => 1018 (E) ……. (E) ……. From dedicated solution to network-based From dedicated solution to network-based
solutionsolution DAS => NAS => SAN => IP-SANDAS => NAS => SAN => IP-SAN
Convergence of SAN and IP-LAN/WANConvergence of SAN and IP-LAN/WAN It is an IP world!It is an IP world!
SCSI is the protocol for block data SCSI is the protocol for block data transmissiontransmission SCSI over FCSCSI over FC SCSI over IP (iSCSI)SCSI over IP (iSCSI)
FC and IP interworking protocolsFC and IP interworking protocols IP over FCIP over FC FC over IP (FCIP) and iFCPFC over IP (FCIP) and iFCP