Date post: | 08-Sep-2018 |
Category: |
Documents |
Upload: | duongnguyet |
View: | 217 times |
Download: | 0 times |
IP-Based NVMe Development Platform
Mickael GuyardProduct Marketing Director (IP-Maker)
Flash Memory Summit 2018 Santa Clara, CA 1
Agenda
The need for NVMe IPs NVMe device platform NVMe host platform Use cases and applications
Flash Memory Summit 2018 Santa Clara, CA 2
Part 1 – The need for NVMe IPs
Flash Memory Summit 2018 Santa Clara, CA 3
NVMe the new universal interface
The new universal interface for storage First specification released in 2011 13 board members 90 companies with NVMe-based products (G2M
research report) But not only…
Flash Memory Summit 2018 Santa Clara, CA 4
NVMe applications
Storage: PCIe SSD Cache: PCIe MRAM and NVRAM Processing accelerator
Flash Memory Summit 2018 Santa Clara, CA 5
Heterogenous architecture
Flash Memory Summit 2018 Santa Clara, CA 6
FPGA in the data centers
Flash Memory Summit 2018 Santa Clara, CA 7
The need for NVMe IPs
Flash Memory Summit 2018 Santa Clara, CA 8
Massive usage of FPGA in data centers NVMe as a universal interface New architectures
=>NVMe IPs for FPGA are needed Both device and host
IP-Maker IPs
Flash Memory Summit 2018 Santa Clara, CA 9
Part 2 – NVMe Device Platform
Flash Memory Summit 2018 Santa Clara, CA 10
NVMe protocol
Flash Memory Summit 2018 Santa Clara, CA 11
3- NVMe command submission fetch
2- NVMe command ready
CPU
1-Host driverSetup PCIe
Phy + CtrlPCIe rootcomplex
NVMe IP
DDRDMA4- Data transfer DDR
5- NVMe command executed
NVMe IPs
Flash Memory Summit 2018 Santa Clara, CA 12
Key Features1.3 NVM Express specificationAutomatic NVMe commandUp to 65536 I/O queuesQueue arbitrationAll mandatory commands / log managementLegacy interrupt/MSI/MSI-XAXI/Avalon interface Up to 32 Read DMA channels + 32 write DMA channelsScalable data buswidth (64/128/256 bits)Available for PCIe Gen1/2/3
Full hardware
Hardware + SoftwareFor more flexibility, such as vendor
specific commands
HW/SW architecture
Automatic command processing => Low latency
Multi-channel DMA => IOPS acceleration
Flash Memory Summit 2018 Santa Clara, CA 13
Validated platforms
Flash Memory Summit 2018 Santa Clara, CA 14
VC709 – Virtex7
KCU105 – Kintex UltrascaleNallatech 250S+ - Kintex Ultrascale+
Fidus Sidewinder – Zynq Ultrascale+
Reference design
Flash Memory Summit 2018 Santa Clara, CA 15
PCIe Interface
IPM-NVMe
DMADMADMADMA
AutomaticCommandProcessing
Bridge Bridge
DDR3 Controll
er
FPGA
PCIe hard IP DDR controller soft IPHost
CPU
DDR
NVMe driver
Device
DDR3
Performances
Setup• Hardware : reference design• Standard NVMe driver• Use of standard benchmark tool for storage: FIO
• Latency• IOPS
Flash Memory Summit 2018 Santa Clara, CA 16
Latency
QD=1, IO=4kB
Flash Memory Summit 2018 Santa Clara, CA 17
IOPS
Gen3x4, QD8, 4kB IO, random R/W• 700kIOPS
High IOPS at low queue depth Scalable data path : up to Gen3x16, Gen4 x8
Flash Memory Summit 2018 Santa Clara, CA 18
Part 3 – NVMe Host Platform
Flash Memory Summit 2018 Santa Clara, CA 19
NVMe Host IP overview
Flash Memory Summit 2018 Santa Clara, CA 20
Memory orFIFO
PCIe
Roo
tPo
ort
Automatic initengine(state
machine)User InterfaceData to Transfer NVMe SSD
NVMe Command Manager
Control
Data transferengine
AXI/Avalon
AXI/Avalon
IP-Maker IP
3rd party IP
Theory of operation
Flash Memory Summit 2018 Santa Clara, CA 21
Data transfer request
Init with a state machinePCIe rootport/endpoint settingsNVMe device/host configuration
NVMe commandssetup by the host
Data transfer
Memory orFIFO
PCIe
Roo
tPo
ort
Automatic initengine(state
machine)User InterfaceData to Transfer NVMe SSD
NVMe Command Manager
Control
Data transferengine
0
1
2 3
Different configurations
Flash Memory Summit 2018 Santa Clara, CA 22
Single port, up to 128 queues N* root ports, 1 queue
*depending on FPGA interfaces
Multiroot
Flash Memory Summit 2018 Santa Clara, CA 23
Key FeaturesNVM Express CompliantAutomatic NVMe Command managementAutomatic PCIe/NVMe initMulti rootport supportSingle I/O queueSingle NamespaceVendor specific commandsUp to PCIe Gen 3x8
Advanced
Flash Memory Summit 2018 Santa Clara, CA 24
Key FeaturesNVM Express CompliantAutomatic NVMe Command managementAutomatic PCIe/NVMe init128 I/O queuesVendor specific commandsSingle NamespaceUp to PCIe Gen 3x8
Open Channel support
• Full submission command controlall vendor specific commands are possiblesThe complete control is possible.
• Full completion control
Flash Memory Summit 2018 Santa Clara, CA 25
Validated platforms
Flash Memory Summit 2018 Santa Clara, CA 26
KCU105 – Kintex UltrascaleNallatech 250S+ - Kintex Ultrascale+
Fidus Sidewinder – Zynq Ultrascale+
Reference design
Host IP configuration• 1 root port• 1 queue version
Embedded test bench
Flash Memory Summit 2018 Santa Clara, CA 27
Ultrascale Xilinx FPGA
Samsung NVMe 960EVOGen3x 4 PCIe interface
Performance
Performance• Write : 2.2 GB/s• Read : 3.2GB/s
Flash Memory Summit 2018 Santa Clara, CA 28
Part 4 – Applications
Flash Memory Summit 2018 Santa Clara, CA 29
Use cases and applications
PCIe Flash NVRAM Emerging NVM Smart SSD HBA NVME2NVMe
Flash Memory Summit 2018 Santa Clara, CA 30
NVRAM Reference design
Flash Memory Summit 2018 Santa Clara, CA 31
NVMe device ref design close to a end-product Detected as a NVMe device by the driver
Just need to add « non-volatile » feature
PCIe NVRAM
Using NVDIMM-N like technology Or using directly a NVDIMM-N
Flash Memory Summit 2018 Santa Clara, CA 32
PCIe NVRAM
Specification Up to 32GB 1.5MIOPS on Gen3x8 10us latency
Flash Memory Summit 2018 Santa Clara, CA 33
Flash Controller IP
Flash Memory Summit 2018 Santa Clara, CA 34
Key FeaturesONFI 3/4 CompliantSLC / MLC / TLC2 ChannelsSupported modes
Async, DDR, DDR2, DDR3AXI/Avalon interfaceConfigurable ECC
BCHLDPC
Evolution
• MRAM support
Flash Memory Summit 2018 Santa Clara, CA 35
MR
AM
NVMe to NVMe HBA
For NVMe SSD aggregation:• Better performance and reliability
Flash Memory Summit 2018 Santa Clara, CA 36
HBACPU
NVMe to NVMe HBA
• 2.5 x86 cores full time at 3GHz required to sustain 750kIOPS on each SSD• 10 cores total!
• =>Need of hardware accelerator enginesFlash Memory Summit 2018 Santa Clara, CA 37
HBA:10 x86 cores?CPU
NVMe to NVMe HBA
• Let’s use both NVMe device and host IPs
Flash Memory Summit 2018 Santa Clara, CA 38
NVMe Device IP
NVMe Host IP
CPU
NVMe to NVMe
Flash Memory Summit 2018 Santa Clara, CA 39
PCIe Interface NVMe Device
DDR3 Controller
PCIeInterfaceNVMe Host
PCIeInterfaceNVMe Host
PCIeInterfaceNVMe Host
PCIeInterfaceNVMe Host
Namespacemanagement
software
4 x Gen3x4
Gen3x 16or
Gen4x4
Namespace management
Many configurations• Basic capacity aggregation: one namespace across the 4
SSDs• Asymmetric : one namespace on one SSD and 10
namespaces on the 3 other SSDs.• Multi namespaces with different characteristic (encryption,
compression…) seen only as one storage SSD.• Raid 1 storage totally transparent for the host software.
Flash Memory Summit 2018 Santa Clara, CA 40
Evolution
Path to computational storage Advanced computing accelerators can be added
such as key-value store, search engine and deep learning
Flash Memory Summit 2018 Santa Clara, CA 41
Thanks
Visit IP-Maker booth #710NVMe live demo!
Flash Memory Summit 2018 Santa Clara, CA 42