Post on 06-Aug-2020
transcript
IBM Systems
Next generation Ethernet connect to All FLASH: iSERand NVMeF
Subhojit RoySenior Technical Staff Member, IBM
IBM Systems
§ Flash growth and dimensions of change
§ Shared SAN storage requirements
§ What is iSER?
§ What is NVMeF?
§ All FLASH on SAN – roadmap
§ What are we doing at IBM?
Agenda
| 2
IBM Systems
Flash Growth and Dimensions of Change…
StorageSWArchitecture
StorageMedia
Layer2Interconnect(L2)
HDDà SASSSDsàPCIe NVMe àPM(3DXP)
KernelModeà UserMode(SPDK)
FC8G/16Gà Eth10Gà FC32Gà EthRDMA25/40/50/100G
UpperLevelProtocols(ULP) FCPà iSER &NVMeoF (Eth&FC)
StorageHWArchitecture AFServerà AF-HyperConvergedà AF-Arraysà AFDisaggregatedStorage
WorkloadsTraditionalEnterpriseWorkloadsàNewageFlashworkloads
(Tier0:SAPHANA,RTAnalytics,Tier1:OLTP,VDI,SocialMediaApps)
ü Flashisdrivingchange ü Changeismultidimensional
ü Changewouldnotbeallatonce.Onedimensionatatime
IBM Systems
Shared SAN storage requirements
Ø Multipath (for high availability)
Ø Reservation (SCSI Persistent Group Reservation)
Ø vVols (offload Copy Services to Storage Controller)
Ø Data movement/migration without involving host CPU (XCOPY, ODX)
Ø Unmap: Optimize space deallocation on Thin Provisioned storage
Ø Atomic Compare And Write (CAW): Alternative to SCSI3 Reservations
Ø Error Handling: Abort Task, Abort Task Set, LUN Reset, Target Reset etc.
| 4
5
What is iSER?
OperatingSystem/Applications
SCSILayer
FCP iSCSI
TCP
IP
NICDriver
iSER
iWARP Driver RoCE DriverFiberChannelDriver
OFED IB verbs
InfinibandAdapterDriver
Sof
twar
e D
river
sH
ardw
areFCHBA NIC iWARP rNIC RoCE rNIC IBHCA
FC iSCSI iSER
Ω
IBM Systems
iSER: Current state of affairs
6
LowLatency,LowCPUUtilization(Eliminatescopiesto/fromTCP/IPbuffers)
NoChangestoiSCSIadministration(vSphere, Widows, OpenStack work asis)
VendorandTechnologyIndependent(works oniWARP, RoCE &Infiniband HCAs)
WorksonStandardEthernetequipment(10G and25/50/100G switches)
Enterpriseapplicationsjustwork!(vVols, Clustering, Multipath etc.)
SuitableforAll FLASHoverHighSpeedEth(10,25,40,50,100Gbps andbeyond)
Nodisruptiontoadministrationmodel
Fitswell intoSoftwareDefined Storage(SDS)paradigm
CostSavings
Idealforsharedstorage(bothforFLASHandHDD)
iSER is iSCSI with RDMA Data path
iSER is ready for Shared All FLASH SAN storage today!
7
What is NVMeF?
OperatingSystem/Applications
SCSI
iSCSI
iSER
iWARP Driver RoCE Driver
OFED IB verbs
InfinibandAdapterDriver
Sof
twar
e D
river
sH
ardw
are
iWARP rNIC RoCE rNIC IBHCA
iSER
NVMeF
iWARP Driver RoCE DriverFCAdapterDriver
iWARP rNIC RoCE rNIC FCHBA
NVMeF
IBM Systems
NVMeF: Current state of affairs
8
LowLatency,LowCPUUtilization(Primarilyscuts downonHosts/wstack)
NewAdministrativeModel
VendorandTechnologyIndependent(works oniWARP, RoCE &Fiber Channel)
WorksonStandardEthernetequipment(10G and25/50/100G switches)
Applicationsmustchangetoexploitparallelism
SuitableforAll FLASHoverHighSpeedEth(10,25,40,50,100Gbps andbeyond)
ChangestovSphere,Openstack etc.
YesandNo!Needcommonuserspacelayer2
CostSavings
Applicationsneedtotransformyet
NVMeF is still evolving to adapt to Shared Storage Applications!
IBM Systems
HDD
< 2010
SCSI
FC
HDD/SSD
2012-2016…
SCSI
FC
HDD
2010+….
SCSI
iSCSI/Eth
….
Flash/NVMe
2015-….
SCSI
FC
….
Flash/3DXP
2017
NVMeoF
NVMf/RDMA
…. ….
SCSI/FC ruled the
Enterprise Shared Storage World
iSCSI started
penetration in low end
market
PCIe NVMeFlash debuted
as high performance
storage
….
Flash environments
used SCSI with FC as an interconnect
Flash/NVMe
2016-….
SCSI
iSER/RDMA
iSER came as an alternative
to FC for connecting
external Flash
NVMeoFspecification
and technology matured for
Tier0 usecase
….Timeline ForMaturity
Technology
AdoptionStatus
Flash/3DXP
SCSI/NVMeoF
iSER/NVMf
NVMeoF/Eth RDMA
matured for Shared Storage Usecase
UsecaseDetails
1990s – 2010 : HDD/SCSI/FC Rule
2007- 2016 SSD/Flash media evolution, maturity
2015 - … NVM evolution
2016 to 2020 – iSER/SCSI
2019/2022 –iSER/NVMeF
2019….
All Flash SAN roadmap!
IBM Systems
Host HostHost Host Host
Host SAN
HostSVC
HostSVC
HostSVC
HostSVC
Vdisk 1 Vdisk 1 Vdisk 1 Vdisk 1
Device SAN
RAIDCtlr
RAIDCtlr
RAIDCtlr
RAIDCtlr
RAIDcontrollerLUNS
Clustering of Nodes on iSER
Host attach over iSER
Ø Host Attach over iSER –Linux, VMWare
Ø Clustering over iSER
What are we doing at IBM?
IBM Systems 11
iSER vs Fibre channel
I/O iSER (40Gb) Fibre Channel(16Gb)
Read4KiB 50(us) 80(us)
Write4KiB 139(us) 195(us)
Read64KiB 95(us) 196(us)
Write64KiB 209(us) 337(us)
iSER:Fiber Channel benefitsminus theadditional costs
IBM Systems
Thank You!
12