© Copyright 2020 Dell Inc.2 of 32Restricted - Confidential
Agenda
• What are NVMe™ and NVMe over Fabrics™?
• Workloads and use cases by Fabric type
• What do customers expect from NVMe over Fabrics?
• Project Gilgit
© Copyright 2020 Dell Inc.3 of 32Restricted - Confidential
0 2 4 6 8 10 12
HDD
ms
HDD vs SSD
Read Time Transfer Time Misc SSD Link Transfer Platform & Adapter Software
Source: Intel Measurements
Source: Jim Handy, SDC 2015
https://www.snia.org/sites/default/files/SDC15_presentations/gen_sessions/JimHandy_The_Long-Term_Future_PrintVersion.pdf
Time scale in
milliseconds
© Copyright 2020 Dell Inc.4 of 32Restricted - Confidential
0 2 4 6 8 10 12
HDD
ms
HDD vs SSD
Read Time Transfer Time Misc SSD Link Transfer Platform & Adapter Software
Read Access timeTiny
Rainbow
Source: Intel Measurements
Source: Jim Handy, SDC 2015
https://www.snia.org/sites/default/files/SDC15_presentations/gen_sessions/JimHandy_The_Long-Term_Future_PrintVersion.pdf
Time scale in
milliseconds
© Copyright 2020 Dell Inc.5 of 32Restricted - Confidential
0 2 4 6 8 10 12
HDD
SSD
ms
HDD vs SSD
Read Time Transfer Time Misc SSD Link Transfer Platform & Adapter Software
Source: Intel Measurements
Source: Jim Handy, SDC 2015
https://www.snia.org/sites/default/files/SDC15_presentations/gen_sessions/JimHandy_The_Long-Term_Future_PrintVersion.pdf
Time scale in
milliseconds
© Copyright 2020 Dell Inc.6 of 32Restricted - Confidential
0 20 40 60 80 100 120
HDD
SSD
us
HDD vs SSD
Read Time Transfer Time Misc SSD Link Transfer Platform & Adapter Software
Source: Intel Measurements
Source: Jim Handy, SDC 2015
https://www.snia.org/sites/default/files/SDC15_presentations/gen_sessions/JimHandy_The_Long-Term_Future_PrintVersion.pdf
Time scale in
microseconds
© Copyright 2020 Dell Inc.7 of 32Restricted - Confidential
0 20 40 60 80 100 120
HDD
SSD
us
HDD vs SSD
Read Time Transfer Time Misc SSD Link Transfer Platform & Adapter Software
Transfer Time
Misc. SSD
Media Access Time
Platform & Adapter
Software
Source: Intel Measurements
Source: Jim Handy, SDC 2015
https://www.snia.org/sites/default/files/SDC15_presentations/gen_sessions/JimHandy_The_Long-Term_Future_PrintVersion.pdf
Link Transfer
© Copyright 2020 Dell Inc.8 of 32Restricted - Confidential
0 20 40 60 80 100 120
MLC NAND SATA 3 ONFi 2
us
SSD Access times
Read Time Transfer Time Misc SSD Link Transfer Platform & Adapter Software
Source: Intel Measurements
Source: Jim Handy, SDC 2015
https://www.snia.org/sites/default/files/SDC15_presentations/gen_sessions/JimHandy_The_Long-Term_Future_PrintVersion.pdf
© Copyright 2020 Dell Inc.9 of 32Restricted - Confidential
0 20 40 60 80 100 120
MLC NAND SATA 3 ONFi 2
MLC NAND SATA 3 ONFi 3
us
SSD Access times
Read Time Transfer Time Misc SSD Link Transfer Platform & Adapter Software
ONFi 3 reduces transfer time
Source: Intel Measurements
Source: Jim Handy, SDC 2015
https://www.snia.org/sites/default/files/SDC15_presentations/gen_sessions/JimHandy_The_Long-Term_Future_PrintVersion.pdf
© Copyright 2020 Dell Inc.10 of 32Restricted - Confidential
0 20 40 60 80 100 120
MLC NAND SATA 3 ONFi 2
MLC NAND SATA 3 ONFi 3
MLC NAND PCIe x4 G3 ONFi 3
us
SSD Access times
Read Time Transfer Time Misc SSD Link Transfer Platform & Adapter Software
ONFi 3 reduces transfer time
PCIe reduces Link Transfer & Platform & Adapter time
Source: Intel Measurements
Source: Jim Handy, SDC 2015
https://www.snia.org/sites/default/files/SDC15_presentations/gen_sessions/JimHandy_The_Long-Term_Future_PrintVersion.pdf
© Copyright 2020 Dell Inc.11 of 32Restricted - Confidential
0 20 40 60 80 100 120
MLC NAND SATA 3 ONFi 2
MLC NAND SATA 3 ONFi 3
MLC NAND PCIe x4 G3 ONFi 3
Future NVM PCIe x4 G3
us
SSD Access times
Read Time Transfer Time Misc SSD Link Transfer Platform & Adapter Software
ONFi 3 reduces transfer time
PCIe reduces Link Transfer & Platform & Adapter time
NVM Shrinks Read, Transfer & Misc SSD Times
Source: Intel Measurements
Source: Jim Handy, SDC 2015
https://www.snia.org/sites/default/files/SDC15_presentations/gen_sessions/JimHandy_The_Long-Term_Future_PrintVersion.pdf
© Copyright 2020 Dell Inc.12 of 32Restricted - Confidential
0 20 40 60 80 100 120
MLC NAND SATA 3 ONFi 2
MLC NAND SATA 3 ONFi 3
MLC NAND PCIe x4 G3 ONFi 3
Future NVM PCIe x4 G3
us
SSD Access times
Read Time Transfer Time Misc SSD Link Transfer Platform & Adapter Software
ONFi 3 reduces transfer time
PCIe reduces Link Transfer & Platform & Adapter time
NVMe to replace the SCSI SW
Reduce latency down to ~<10us
Source: Intel Measurements
Source: Jim Handy, SDC 2015
https://www.snia.org/sites/default/files/SDC15_presentations/gen_sessions/JimHandy_The_Long-Term_Future_PrintVersion.pdf
© Copyright 2020 Dell Inc.13 of 32Restricted - Confidential
• Non-Volatile Memory Devices (NVM/SSD/Flash drives) can use
a PCIe interface rather than a serial I/O controller.
NVMe™, and NVMe™ over TCP
PCIe
PCIe
NVMe
NVMe over fabrics
PCIe
NVMe
IP
Network
• The Non-Volatile Memory Express (NVMe) Base specification
that controls the PCIe interface of an SSD/Flash (NVM) drive. – 64K queues with up to 64K commands per queue
• The NVMe over Fabrics Transport specifications describe how
NVMe commands may be carried over a fabric.– The NVMe over Fabrics (NVMe-oF) Transports include: FC (NVMe/FC),
RDMA (NVMe/RoCE, NVMe/IB, and NVMe/iWARP) and TCP (NVMe/TCP).
– TCP’s advantage is support for arbitrary networks without specialized network
configuration (e.g., Lossless operation) or the need to specialized adapters.
– New standards TP-8009 and TP-8010 for NVMe/TCP: Dell and other vendors
collaborate to design standardized automated discovery process.
© Copyright 2020 Dell Inc.14 of 32Restricted - Confidential Pre-GA: Material subject to change.
NVMe-oF™ EvolutionFrom SCSI to NVMe
• Application running on a host that is accessing
external array-based storage via either FC or
iSCSI.
Application
SCSI
SCSI
FCPiSCSI NVMe-oF
SCSI
FCPiSCSI
NVMe
SATA-SSD
Ho
st
Sto
rag
e A
rra
y
NVMe-oF
NVMe
NVMe-SSD
SCSI
Local SATA-SSD Local NVMe-SSD
• NVMe Drives were first introduced on the
host in 2015 and were used mainly for
caching and boot drives
• NVMe-SSDs improve storage array
performance locally but using the SCSI
protocol can add significant latency.
• NVMe-oF™ can run over either Ethernet or
Fibre Channel with low latency.
Fibre
Channel
IP
Network
© Copyright 2020 Dell Inc.20 of 32Restricted - Confidential
Agenda
• What are NVMe and NVMe over Fabrics?
• Workloads and use cases by Fabric type
• What do customers expect from NVMe over Fabrics?
• Project Gilgit
© Copyright 2020 Dell Inc.21 of 32Restricted - Confidential
NVMe/TCPNVMe/RoCE or IB
Workloads and use cases by Fabric
Logos are indicative of workload characteristics only.
High Perf Use Cases
General purpose use cases
BCA = Business Critical Apps
HCI = Hyper Converged Infrastructure
EBOF = Ethernet Bunch of Flash
vMotion
EBOF
FC-NVMe
In-house
BCAs
Storage
vMotion
Business Critical Apps
Specialized Use Cases
© Copyright 2020 Dell Inc.22 of 32Restricted - Confidential
Agenda
• What are NVMe and NVMe over Fabrics?
• Workloads and use cases by Fabric type
• What do customers expect from NVMe over Fabrics?
• Project Gilgit
© Copyright 2020 Dell Inc.23 of 32Restricted - Confidential
Operational simplicity
Performance Applications
Scale
SecurityCost
23
What do customers expect from
NVMe over Fabrics?
© Copyright 2020 Dell Inc.24 of 32Restricted - Confidential
Configuration used for comparison of fabric types
• The test results/data used to generate the following graphs are based on testing performed at Dell
Technologies.
• These results are only intended to compare the fabric types that are available for use.
– In other words, all things being equal on both ends, the results demonstrate the benefit of using one fabric versus
another.
• These test results DO NOT represent what end-users will experience when using one fabric versus
another to access storage capacity on every storage platform.
– This will be much more dependent on the storage platform implementation than on the fabric in use.
• The test configuration consisted of:
1. An EZFIO client running on a physical server running Linux that has an appropriate adapter type installed
2. All adapters used were either 32GFC, 25GbE or 100GbE based.
3. A single 32GFC or 100GbE switch was used for “SAN” connectivity
4. An LIO Target running on a physical server running Linux that is using RAMDISK for storage capacity.
• The Test results shown are all 4k Random Writes performed with varying numbers of threads
© Copyright 2020 Dell Inc.25 of 32Restricted - Confidential
Defining Hero Numbers
• Hero numbers are:– performance metrics that are captured at the extreme ends of the support envelop in order to highlight
what is possible.
– usually focus on Latency and IOPS/BW
– typically captured under ideal conditions (e.g., non-oversubscribed)
• Hero numbers do not:– directly correlate to a performance benefit many end-users will experience with their workloads
– capture the infrastructure ecosystem implications of using one protocol versus another. This is not
only about CapEx, OpEx and Security… You need to consider the operating environment as well as
the end-devices/appliances and their support for the protocol.
© Copyright 2020 Dell Inc.26 of 32Restricted - Confidential
Typical operating
range for Web, DB,
and file access
applications
Hero numbers
IOPS
© Copyright 2020 Dell Inc.27 of 32Restricted - Confidential
Latency
Typical operating range
for Web, DB, and file
access applications
(No Congestion)
Hero numbers
© Copyright 2020 Dell Inc.28 of 32Restricted - Confidential
I/Os per Percent CPU Utilization
Typical operating range for
Web, DB,
and file access applications
FC @ 32GFC
TCP & ROCE @
25GbE
© Copyright 2020 Dell Inc.29 of 32Restricted - Confidential
Why TCP vs. RoCE?
Aspect RoCE TCP/IP
Performance Excellent Good, getting better
Interoperability Fair, getting better Excellent
Interop Testing costs High (FC-like) Moderate
Network congestion impacts Visible, unexpected Moderate, expected
Network management impacts New protocol, missing
end-to-end functionality
Part of normal
operations
poor excellent
• NVMe-oF/Ethernet provides standard, interoperable, high-speed, light-weight, low-latency, cost effective block storage access
• 25GE provides essentially the same throughput as 32G FC, but at a fraction of the cost
• NVMe-oF/TCP delivers better performance and reduced overhead compared to typical iSCSI
• TCP/IP for NVMe-oF transport just works by default
• Specialized configuration of TCP is not required
• Realistic performance is similar to RoCE (better as NVMe/TCP offloads emerge)
• TCP/IP is a better fit to Edge, IoT, Client deployments due to price & hardware
• TCP/IP allows a wide variety of network topologies (fully routable and fully flow controlled as needed)
© Copyright 2020 Dell Inc.30 of 32Restricted - Confidential
Operational simplicity
Performance Applications
Scale
SecurityCost
30
© Copyright 2020 Dell Inc.31 of 32Restricted - Confidential
NVMe Fabric type comparison
FC-NVMe NVMe/RoCE NVMe/TCP
High Speed Performance
Software Defined Storage
Centralized Provisioning1
State Change Notifications
Edge/Distributed System at Scale
Cloud Operating Model/Automation
CapEx Cost Advantage
OP
EX
ben
efi
ts
1. Source: https://brasstacksblog.typepad.com/brass-tacks/2017/12/nvme-over-fabrics-discovery-problem.html
© Copyright 2020 Dell Inc.32 of 32Restricted - Confidential
Initial Configuration Steps by Fabric type
Single Host to Single Target
Transport Protocol Host Steps Network Steps Storage Steps Total
FC 2 5 7 14
FC-NVMe 2 5 7 14
NVMe/TCP 4->1 3 8 15->12
NVMe/RoCE 5->2 7 9 21->18
Source: http://brasstacksblog.typepad.com/brass-tacks/2012/02/fc-and-fcoe-versus-iscsi-network-centric-versus-end-node-centric-provisioning.html
Source: http://brasstacksblog.typepad.com/brass-tacks/2015/05/fibre-channel-is-better-than-ethernet.html
© Copyright 2020 Dell Inc.33 of 32Restricted - Confidential
Operational simplicity
Performance Applications
Scale
SecurityCost
33
© Copyright 2020 Dell Inc.34 of 32Restricted - Confidential
Backend
RoCE or IB
Network
JBOF
Frontend
Embedded Scale Out Back End
© Copyright 2020 Dell Inc.35 of 32Restricted - Confidential
FC SAN Alternative
▪SAN features at higher performance▪ Better utilization:
capacity, rack space, and power
▪ Scalability▪Management▪ Fault isolation
▪Driver support in Linux, VMware and Windows(3rd party)
Ethernet InfiniBand
Fibre Channel
© Copyright 2020 Dell Inc.36 of 32Restricted - Confidential
Compute Storage Disaggregation
Switch
Compute
Compute
Compute
Compute
Compute
Compute
Compute
Compute
Compute
Compute
Compute
Switch
Compute
Compute
Compute
Compute
Compute
Compute
Compute
Compute
Compute
▪Also called
Composable
Infrastructure and
Rack Scale
▪NVMe over
Fabrics/RoCE enables
with nearly local disk
performance
▪NVMe/TCP may be
used depending on
performance
requirements
© Copyright 2020 Dell Inc.37 of 32Restricted - Confidential
Hyperconverged and Scale-Out Storage
▪Scale-out ▪ Cluster of commodity servers▪ Software provides storage
functions
▪Hyperconverged collapses compute & storage▪ Integrated compute-storage
nodes & software▪NVMe/RoCE performs like
local/direct-attached SSD▪NVMe/TCP may be used
depending on performance requirements
Scale out Storage
Mellanox x86 Switch
Compute Nodes
Storage Application
VM VMVM VM
NVMe NVMe NVMe
NVMe NVMe NVMe
Storage
App
HCI Nodes
© Copyright 2020 Dell Inc.38 of 32Restricted - Confidential
Fabrics at Scale
Cross Rack / Large Cluster
NVMe/TCP/RoCE
NVMe/TCP
NVMe/TCP/RoCE/FC
StorageController
JBOF
Compute
NVMe/RoCE/IB
NVMe/TCP/RoCE/FC
StorageController
JBOF
Compute
Intra Rack / Smaller Scale
NVMe/RoCE/IB
StorageArray
GPU
Remote Datacenter(s)
GPU
NVMe/RoCE
NVMe/TCP/RoCE/FC
© Copyright 2020 Dell Inc.39 of 32Restricted - Confidential
Operational simplicity
Performance Applications
Scale
SecurityCost
39
© Copyright 2020 Dell Inc.40 of 32Restricted - Confidential
Must secure NVMe Data in flight and rest
Potential DC NVMe-oF Security Threats
Sniffing
Storage Traffic
Data
Corruption
Storage
Masquerading
Session Hijacking
© Copyright 2020 Dell Inc.41 of 32Restricted - Confidential
Securing NVMe-oF
FC-SP-2 (FC-NVMe)
▪ FC-SP-2 is an ANSI/INCITS standard (2012) that defines protocols to –
▪ Authenticate Fibre Channel entities
▪ Setup session encryption keys
▪ Negotiate parameters to ensure per frame integrity and confidentiality
▪ Define and distribute security policies over FCP and FC-NVMe
▪ Concurrent FCP and FC-NVMe
▪ HBA provide full offload
TLS for NVMe/TCP
• TLS is a protocol suite defined by the IETF –
• Privacy and data integrity between two communicating applications
• Higher-level protocols like NVMe that can layer on top of the TLS protocol transparently.
• NVMe-oF over TCP already supports TLS
• The NVM Express Technical Proposal TP 8011 added support for TLS 1.3
• Many NICs can provide full offload
IPsec for
NVMe/TCP/RoCE
▪ IPsec is a secure network protocol
suite defined by the IETF –
▪ Authenticates and encrypts the
data packets over IP networks
▪ Higher-level protocols like NVMe that
can layer on top of the IPsec protocol
transparently.
▪ NVMe-oF/TCP/RoCE already
supports IPSEC
▪ Many NICs can provide full offload
© Copyright 2020 Dell Inc.42 of 32Restricted - Confidential
NVMe/TCP Security
• Authentication– Authentication support for DH-HMAC-CHAP Negotiation (NVMe.org TP 8006) in the NVMe-oF
Transport Layer.
– Including support for a 3rd Party Authentication Verifier (such as RADIUS) workflow to interface with
the NVMe-oF Transport Layer.
• Transport Security– TLS Support (TLS 1.3 via NVMe.org TP8011) in the NVMe-oF Transport Layer.
– Includes TLS control plane.
• Redfish/DMTF Credential Bootstrap– As discussed in Boot; mechanism works for Boot and non-Boot volumes for credential provisioning.
– It’s mandatory for Boot (Security).
– Doorbell (DSP0270 v1.3 ratified, Redfish structure still in definition)
© Copyright 2020 Dell Inc.43 of 32Restricted - Confidential
Agenda
• What is NVMe over Fabrics?
• Workloads and use cases by Fabric type
• What do customers expect from NVMe over Fabrics?
• Project Gilgit
© Copyright 2020 Dell Inc.44 of 32Restricted - Confidential
Configuration Steps with Direct Discovery (Existing)
Direct
Discovery
Controller
(DDC)
IO
Controller
NVM SubsystemIP Network
NVMe over
TCP Storage
Adapter
1
1
2
Host sends connect to
Discovery Controller at
IP Address supplied by
admin
Storage admin
provisions
Namespaces
(Storage) to the Host
NQN3 Host Admin uses nvme
connect-all to Discover
and connect to IO
Controllers on that
subsystem.
2
Host
3
4 Repeat 1-3 on all
Hosts for each
subsystem
© Copyright 2020 Dell Inc.45 of 32Restricted - Confidential
Configuration Steps with Centralized Discovery (New)
Direct
Discovery
Controller
(DDC)
IO
Controller
NVM SubsystemIP Network
NVMe over
TCP Storage
Adapter
1 Zoning performed on
CDC (optional)
3 After zoning, Host
receives AEN, uses get
log page, and connects
to each IO Controller
2
Host
3
4 Repeat 1-2 for each
Hosts on each
subsystem
Centralized
Discovery
Controller
(CDC)
0 Host and subsystems
automatically discover
the CDC, connect to it
and Register
Discovery info
2 Storage admin
provisions
namespaces to the
Host NQN. Storage
may send zoning info
to CDC
1
0
0
© Copyright 2020 Dell Inc.46 of 32Restricted - Confidential
Direct vs Centralized Discovery at scale
Direct Discovery config steps
1. Host: Determine subsystem Discovery controller IP -> connect
2. Storage: Provision storage
3. Host: Discover / connect all
Centralized Discovery config steps1. Host: N/A
2. CDC: Configure Zoning (optional)
3. Storage: Provision storage
What the chart doesn’t show1. Direct becomes impractical @ >64 hosts
2. Direct requires interaction with each host every time a storage subsystem is added or removed.
3. Direct may lead to extended discovery time if many subsystem interfaces are present.
© Copyright 2020 Dell Inc.47 of 32Restricted - Confidential
Network Fabric
Automated End-to-End Discovery
• Enables end devices to discover
other end devices without manual
configuration
• Avoids iSCSI like user experience
Storage Arrays
Automated Fabric Formation
• Enables End-to-End reachability
• Standards based
Switch Switch
Switch Switch
Switch
Ethernet/IP Fabric formation services
• SFS (Automated Underlay Config) Dell
only
• Many other solutions available
NVMe/TCP Discovery services
• Discovery Service (CDC service)
• Discovery Client (CDC client)
• OS Specific - SW based
• CDC – See TP-8010
• CDC Discovery - See TP-8009
FC Fabric formation services
• Vendor Specific
• Non-interoperable
Host OS
ESX, Linux, etc
Discovery
Client
Discovery
Client
FC Discovery services
• Discovery Service
• Vendor Specific (e.g., Name
Server)
• Discovery Client
• HBA specific – HW based
Discovery: FC vs NVMe/TCP
NVMe/TCP
Discovery
Service
Underlay Fabric
Config Service
Fibre Channel
© Copyright 2020 Dell Inc.48 of 32Restricted - Confidential
Introducing the NVMe IP SAN - Powered by Dell TechnologiesModern storage connectivity with a standards-based, end-to-end NVMe/TCP portfolio
Automation of complex storage and networking tasks
and management
Reliability with global support & services
Accelerating the evolution of storage connectivity with a modern, automated and secure NVMe IP SAN ecosystem
Agility with cutting-edge NVMe storage technology
© Copyright 2020 Dell Inc.49 of 32Restricted - Confidential
Introducing the NVMe IP SAN - Powered by Dell TechnologiesModern storage connectivity with a standards-based, end-to-end NVMe/TCP portfolio
De
ll E
MC
Glo
ba
l S
up
po
rt &
Se
rvic
es
VMware ESXPowerEdge
PowerEdge MXPowerStore
PowerMax PowerFlex
PowerSwitch
Automation of complex storage and networking tasks
and management
Reliability with global support & services
Accelerating the evolution of storage connectivity with a modern, automated and secure NVMe IP SAN ecosystem
Agility with cutting-edge NVMe storage technology
© Copyright 2020 Dell Inc.50 of 32Restricted - Confidential
Introducing the NVMe IP SAN - Powered by Dell TechnologiesModern storage connectivity with a standards-based, end-to-end NVMe/TCP portfolio
De
ll E
MC
Glo
ba
l S
up
po
rt &
Se
rvic
es
VMware ESXPowerEdge
PowerEdge MXPowerStore
PowerMax PowerFlex
PowerSwitch
SmartFabric Services (SFS)Automated Ethernet Storage networking
Automation of complex storage and networking tasks
and management
Reliability with global support & services
Accelerating the evolution of storage connectivity with a modern, automated and secure NVMe IP SAN ecosystem
Agility with cutting-edge NVMe storage technology
© Copyright 2020 Dell Inc.51 of 32Restricted - Confidential
Introducing the NVMe IP SAN - Powered by Dell TechnologiesModern storage connectivity with a standards-based, end-to-end NVMe/TCP portfolio
De
ll E
MC
Glo
ba
l S
up
po
rt &
Se
rvic
es
VMware ESXPowerEdge
PowerEdge MXPowerStore
PowerMax PowerFlex
PowerSwitch
SmartFabric Services (SFS)Automated Ethernet Storage networking
SmartFabric Storage Software (SFSS)Policy driven storage and data services
Automation of complex storage and networking tasks
and management
Reliability with global support & services
Accelerating the evolution of storage connectivity with a modern, automated and secure NVMe IP SAN ecosystem
Agility with cutting-edge NVMe storage technology
© Copyright 2020 Dell Inc.52 of 32Restricted - Confidential
Smart Fabric Storage Software (SFSS)Standards based Centralized Discovery Controller (CDC) for NVMe/TCP hosts and targets
• Discovery Service
– NVMe/TCP end points dynamically discover the SFSS instance
– Listen and respond to mDNS queries from end points in the fabric
• End Point Registration Service
– NVMe-oF end point – host or subsystem register their information with SFSS
• End Point Query Service
– NVMe/TCP Hosts query SFSS to discover the NVMe/TCP subsystems they can communicate with
– NVMe/TCP subsystems query SFSS to discover the NVMe/TCP hosts that can connect to them
Equivalent of Fibre Channel Name Server Database
© Copyright 2020 Dell Inc.53 of 32Restricted - Confidential
Smart Fabric Storage Software (SFSS)Standards based Centralized Discovery Controller (CDC) for NVMe/TCP hosts and targets
• Discovery Service
– NVMe/TCP end points dynamically discover the SFSS instance
– Listen and respond to mDNS queries from end points in the fabric
• End Point Registration Service
– NVMe-oF end point – host or subsystem register their information with SFSS
• End Point Query Service
– NVMe/TCP Hosts query SFSS to discover the NVMe/TCP subsystems they can communicate with
– NVMe/TCP subsystems query SFSS to discover the NVMe/TCP hosts that can connect to them
• Zone Service
– Soft Zoning – Get Log Page responses only include sub-systems zoned for the querying host
– Hard Zoning – Enforcement in the network with ACLs – (Future integration with SFS)
Equivalent of Fibre Channel Name Server Database
Equivalent of Fibre Channel Zone Server Database
© Copyright 2020 Dell Inc.54 of 32Restricted - Confidential
Smart Fabric Storage Software (SFSS)Standards based Centralized Discovery Controller (CDC) for NVMe/TCP hosts and targets
• Discovery Service
– NVMe/TCP end points dynamically discover the SFSS instance
– Listen and respond to mDNS queries from end points in the fabric
• End Point Registration Service
– NVMe-oF end point – host or subsystem register their information with SFSS
• End Point Query Service
– NVMe/TCP Hosts query SFSS to discover the NVMe/TCP subsystems they can communicate with
– NVMe/TCP subsystems query SFSS to discover the NVMe/TCP hosts that can connect to them
• Zone Service
– Soft Zoning – Get Log Page responses only include sub-systems zoned for the querying host
– Hard Zoning – Enforcement in the network with ACLs – (Future integration with SFS)
• Asynchronous Notifications
– Asynchronous Event Registration – subscribe to state change notifications from end points
– Asynchronous Event Notifications – send notifications to end points for state changes
Equivalent of Fibre Channel Name Server Database
Equivalent of Fibre Channel Zone Server Database
Equivalent of Fibre Channel Registered State Change Notification
(RSCN)
© Copyright 2020 Dell Inc.55 of 32Restricted - Confidential
Introducing the NVMe IP SAN - Powered by Dell TechnologiesModern storage connectivity with a standards-based, end-to-end NVMe/TCP portfolio
De
ll E
MC
Glo
ba
l S
up
po
rt &
Se
rvic
es
VMware vCenter
VMware ESXPowerEdge
PowerEdge MXPowerStore
PowerMax PowerFlex
PowerSwitch
SmartFabric Services (SFS)Automated Ethernet Storage networking
SmartFabric Storage Software (SFSS)Policy driven storage and data services
Dell EMCCloudIQ
Dell OpenManage
Enterprise
Storage Orchestrators
Automation of complex storage and networking tasks
and management
Reliability with global support & services
Accelerating the evolution of storage connectivity with a modern, automated and secure NVMe IP SAN ecosystem
Agility with cutting-edge NVMe storage technology
© Copyright 2020 Dell Inc.56 of 32Restricted - Confidential
NVMe IP SAN operations
NVMe IP SAN Fabric
SmartFabric Storage Software
SmartFabric Services
Orchestrator(vCenter)
PowerEdge Servers (with vSphere)(NVMe/TCP Hosts)
PowerSwitch Switches
PowerStore Storage(NVMe/TCP Subsystems)
OpenManage NetworkIntegration (OMNI)
1Automated deployment of the fabric based on defined policies
Th
is s
lid
e h
as a
nim
atio
n
© Copyright 2020 Dell Inc.57 of 32Restricted - Confidential
NVMe IP SAN operations
NVMe IP SAN Fabric
SmartFabric Storage Software
SmartFabric Services
Orchestrator(vCenter)
PowerEdge Servers (with vSphere)(NVMe/TCP Hosts)
PowerSwitch Switches
PowerStore Storage(NVMe/TCP Subsystems)
OpenManage NetworkIntegration (OMNI)
1Automated deployment of the fabric based on defined policies
Th
is s
lid
e h
as a
nim
atio
n
2Server and storage systems discover SFSS
© Copyright 2020 Dell Inc.58 of 32Restricted - Confidential
NVMe IP SAN operations
NVMe IP SAN Fabric
SmartFabric Storage Software
SmartFabric Services
Orchestrator(vCenter)
PowerEdge Servers (with vSphere)(NVMe/TCP Hosts)
PowerSwitch Switches
PowerStore Storage(NVMe/TCP Subsystems)
OpenManage NetworkIntegration (OMNI)
1Automated deployment of the fabric based on defined policies
Th
is s
lid
e h
as a
nim
atio
n
2Server and storage systems discover SFSS
3Server and storage systems
register with
SFSS
© Copyright 2020 Dell Inc.59 of 32Restricted - Confidential
NVMe IP SAN operations
NVMe IP SAN Fabric
SmartFabric Storage Software
SmartFabric Services
Orchestrator(vCenter)
PowerEdge Servers (with vSphere)(NVMe/TCP Hosts)
PowerSwitch Switches
PowerStore Storage(NVMe/TCP Subsystems)
OpenManage NetworkIntegration (OMNI)
1Automated deployment of the fabric based on defined policies
Th
is s
lid
e h
as a
nim
atio
n
2Server and storage systems discover SFSS
4Operator or orchestrator reads the SFSS Name Server database and sets up server/storage zones
3Server and storage systems
register with
SFSS
© Copyright 2020 Dell Inc.60 of 32Restricted - Confidential
Orchestrator(vCenter)
NVMe IP SAN operations
NVMe IP SAN Fabric
SmartFabric Storage Software
SmartFabric Services
PowerEdge Servers (with vSphere)(NVMe/TCP Hosts)
PowerSwitch Switches
PowerStore Storage(NVMe/TCP Subsystems)
OpenManage NetworkIntegration (OMNI)
1Automated deployment of the fabric based on defined policies
5SFSS notifies servers and storage systems of a zoning change
Th
is s
lid
e h
as a
nim
atio
n
2Server and storage systems discover SFSS
4Operator or orchestrator reads the SFSS Name Server database and sets up server/storage zones
3Server and storage systems
register with
SFSS
© Copyright 2020 Dell Inc.61 of 32Restricted - Confidential
NVMe IP SAN operations
NVMe IP SAN Fabric
SmartFabric Storage Software
SmartFabric Services
Orchestrator(vCenter)
PowerEdge Servers (with vSphere)(NVMe/TCP Hosts)
PowerSwitch Switches
PowerStore Storage(NVMe/TCP Subsystems)
OpenManage NetworkIntegration (OMNI)
1Automated deployment of the fabric based on defined policies
5SFSS notifies servers and storage systems of a zoning change
Th
is s
lid
e h
as a
nim
atio
n
6Servers connect to storage and start transferring data
2Server and storage systems discover SFSS
4Operator or orchestrator reads the SFSS Name Server database and sets up server/storage zones
3Server and storage systems
register with
SFSS
© Copyright 2020 Dell Inc.63 of 32Restricted - Confidential
R740
R740
R740
R740
R740
R740
R740
R740
R740
R740
R740
R740
R740
R740
R740
R740
Storage
SAN A SAN B
R640
SFSS
SAN A / SAN B
Two primary deployment models
R740
R740
R740
R740
R740
R740
R740
R740
R740
R740
R740
R740
R740
R740
R740
R740
Storage
R640
SFSS
Z-Series
S-Series
Converged network
© Copyright 2020 Dell Inc.64 of 32Restricted - Confidential
OOB Mgmt Network OOB Mgmt Network
Dell Technologies SAN Interconnects for NVMe/TCP with external SFSS
PowerEdge MX Converged Compute
PowerStoreStorage
PowerEdgeCompute
PowerMaxStorage
PowerFlexSoftware
Defined Storage
ETHERNET LAN CONNECTIVITY
NVMe/TCP SAN A
NVMe/TCP SAN B
NVMe/TCP Interconnect with
SMARTFABIC SERVICES
NVMe/TCP Interconnect with
SMARTFABRIC SERVICES
SM ARTFABRIC STORAGE SOFTWAREAutomated Storage Services for NVMe/TCP
SM ARTFABRIC STORAGE SOFTWAREAutomated Storage Services for NVMe/TCPOM NI
Centralized Management(VM installed on ESXi/KVM
POWERSWITCH POWERSWITCH
© Copyright 2020 Dell Inc.65 of 32Restricted - Confidential
OOB Mgmt Network OOB Mgmt Network
Dell Technologies SAN Interconnects for NVMe/TCP with embedded SFSS
PowerEdge MX Converged Compute
PowerStoreStorage
PowerEdgeCompute
PowerMaxStorage
PowerFlexSoftware
Defined Storage
ETHERNET LAN CONNECTIVITY
NVMe/TCP SAN A
NVMe/TCP SAN B
NVMe/TCP Interconnect
with SFS & SFSS
NVMe/TCP Interconnect
with SFS & SFSS
Network & NVMe/TCP Storage Automation Services running locally on the SAN Interconnects
POWERSWITCH POWERSWITCH
OM NICentralized Management(VM installed on ESXi/KVM
© Copyright 2020 Dell Inc.66 of 32Restricted - Confidential
Converged LAN / SAN Fabric Deployment for NVMe/TCP
PowerEdge MX Converged Compute
PowerStoreStorage
PowerEdgeCompute
PowerMaxStorage
PowerFlexSoftware
Defined Storage
Ethernet LAN & SAN Connectivity
Converged Fabric for Application & Storage traffic with Network & NVMe/TCP Storage Automation Services
POWERSWITCH
AUTONOMOUS
FABRIC DEPLOYMENT Plug and Play fabric for Compute
& Storage connectivity with
SmartFabric Services
Fabric Interconnect
AUTOMATED STORAGE
FABRIC SERVICESNVMe-oF TCP End Point Policies
driven by SFSS deployed as an
external Virtual Application
INTEGRATED
FABRIC MANAGEMENT Centralized Management for
LAN and Storage Services with
OMNI / vCenter
CONVERGED