Kathryn Vandiver, vSAN ReadyLabs Engineering, VMwareVivek Sarathy, Non-Volatile Memory (NVM) Solutions Group, Intel
STO1053BES
#VMworld2017 #STO1053BES
Redefine vSAN Deployments with Next Generation Intel® Xeon® processor, Intel® Optane™ and Intel® 3D NAND SSDs
VMworld 2017 Content: Not fo
r publication or distri
bution
• This presentation may contain product features that are currently under development.
• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.
• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
• Technical feasibility and market demand will affect final delivery.
• Pricing and packaging for any new technologies or features discussed or presented have not been determined.
Disclaimer
#STO1053BES CONFIDENTIAL 2
VMworld 2017 Content: Not fo
r publication or distri
bution
3
Legal Notices and Disclaimers
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn
more at intel.com, or from the OEM or retailer.
No computer system can be absolutely secure.
Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect
actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information
about performance and benchmark results, visit http://www.intel.com/performance.
Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and
configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction.
This document contains information on products, services and/or processes in development. All information provided here is subject to change
without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps.
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site
and confirm whether referenced data are accurate.
Intel, the Intel logo, Xeon, Intel® Optane™, and 3D XPoint are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
© 2017 Intel Corporation.
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
Agenda
1 Business Overview
2 Emerging Technology and Trends
3 vSAN All Flash – Overview and Customer Adoption
4 vSAN on Next Gen Platform – Intel® Optane™ Technology
5 Manageability Improvements for Next Gen Intel based Platform
6 Summary
4#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
Market Segmentation for Hardware
5
1. General Purpose
Rack Servers - General purpose workloads
BCA, Database, VDI, ROBO, DR, Test/Dev
2. Storage Dense
Rack Servers - Storage Heavy workloads
Archiving, video streaming, analytics
3. Compute Intensive
Blade Servers - Compute Heavy workloads
Web apps, HPC, real time analytics, In-memory DB
4. ComposableInfrastructure
Scaling storage & compute independently for disaggregated workloads
Data warehouses, Search engines databases, Log aggregation
vSAN for various customer workloads#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
vSAN Ready Node: Many Platform Choices to Maximize Flexibility
Many Platform Choices15 server vendors
164 platform choices
Flexible SupportIndividual vendor support
Single support from partner
Software FlexibilityBring your own software
Factory install software
Licensing OptionsBring your own license
Purchase license with platform
https://www.vmware.com/resources/compatibility/search.php?deviceCategory=vsan#STO1053BES CONFIDENTIAL 6
VMworld 2017 Content: Not fo
r publication or distri
bution
vSAN Hardware Enhancements in 2016 & 2017
7
Overview
➢ Enable high density vSAN ReadyNode™ with support for 512e
Drives – vSAN 6.5
➢ Certified Intel® Optane™ SSD as caching tier for vSAN
➢ LED Management for controllers & NVMe
➢ Revised caching guidelines for usage of larger capacity cache
drives in Q1, 2017
➢ Hardware Lifecycle Management: Automated controller
driver/firmware updates for major OEMs in Q1, 2017
Benefits
➢ Larger capacity on high density server configs supported only in
512e/4Kn format
➢ Better All Flash TCO* with improved caching guidelines
➢ Simplified and automated HW life cycle management
* https://vsansizer.vmware.com/vsansizing/html/dcScaleDeploymentsOptions.html#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
Technology Trends
CPU densities
continues to
increase
Hi-Density Flash and
NVMe will dominate
Enterprise Storage
High speed
interconnect to keep
up with fast storage
Impact of Key Technology Trends on vSAN
• All Flash Market is growing at a rapid rate – 38% CAGR over last 5 years
• Enterprise customers are adopting slowly moving away from using magnetic drives
• Newer high speed storage & network interconnects are disrupting the HCI market.
• Leverage hardware trends to deliver All Flash vSAN at the best $/IOP & $/Gig
#STO1053BES CONFIDENTIAL 8
VMworld 2017 Content: Not fo
r publication or distri
bution
Intel® Xeon® Scalable Processors optimized for Storage
Platform
Performance
Server
Agility
Efficiency
at Scale
Platform to drive storage transformation
Intel Xeon
Processor
Intel Xeon
Processor
UPI
DDR4 DDR4
X16 PCIe
VMD VMD
ISA-LSPDKVROC
Memory Drive
Integrated Networking
RDMA
#STO1053BES CONFIDENTIAL 9
VMworld 2017 Content: Not fo
r publication or distri
bution
Cost Delay MORELESSHIGHERLOWER
Intel Technology Innovations Fill the Memory and Storage Gap
Intel® Optane™
Technology
Higher
performance
Intel 3D NANDTechnology
Lower cost & higher
density
#STO1053BES CONFIDENTIAL 10
VMworld 2017 Content: Not fo
r publication or distri
bution
Caching
DataPersistencevSAN
vSAN All Flash Today
11
• 2 Tier Architecture:
• Tier 1 Caching: High performance,
high endurance flash for caching
writes
• Tier 2 Data Persistence: Read
intensive, low endurance drives for
capacity
• Customer Adoption: Gaining significant
momentum, aligned with enterprise
adoption of flash, particularly NVMe
NVMeSSD NVDIMM
Capacity TierFlash Devices
Reads primarily from capacity tier
Writes cached first,Reads from capacity tier
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
NVMe Use Cases for vSAN All Flash
High Performance Caching & Capacity Device
NVMe Configurations for vSAN
✓ NVMe as Caching Tier for vSAN (2-4 NVMe as caching tier per server)
✓ NVMe as Capacity Tier for vSAN (7 NVMe capacity tier per cache device)
✓ ALL NVMe (caching & capacity) platforms (with PCIe switches, up to 24 drives per server)
Advantages:
✓ General trend of customers moving toward NVMe – better TCO*, more power efficiency, compact form factors and better error handling
✓ Good for Latency sensitive workloads such as OLTP/SQL, VDI – Full Clones
✓ Good for write intensive workloads requiring high IOPS – analytics, database, media streaming as an example
NVMe/PCIe
* https://estimator.intel.com/sddc12#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
Based on Telemetry Data &Customer Surveys
Customer Adoption of vSAN All Flash
• 84% of customers are expected to adopt Purley within the next year
• Higher adoption of OEM’s who are offering all NVMe ReadyNodes
• Increase in VCG Listings and ReadyNodes with All Flash over Hybrid
Today: 65%
#STO1053BES CONFIDENTIAL 13
VMworld 2017 Content: Not fo
r publication or distri
bution
Use Case
• vSAN All-Flash supporting over 1200+ stores
• Tier-1 Apps including store inventory management, customer portals, credit card
transaction processing, customer loyalty systems
• BCA like Microsoft SQL Server, Exchange, SharePoint
• 1.2 PB, 4000 VMs
• Supermarket chain across Denmark and Europe
• 1200+ stores
COOP
Challenges
• Competitive retail industry meant constant changes, IT need to respond fast
• Traditional storage, networking, maintenance very costly with limited staff
Benefits
• Standardization: Single, integrated VMware stack with full flexibility to grow
• Best TCO Compared to Any Other Vendor: best price/performance ratio
compared to storage vendors and Azure Cloud
• Performance Exceeding Expectations: Latency-sensitive business-critical
applications far exceeded performance goals
#STO1053BES CONFIDENTIAL 14
VMworld 2017 Content: Not fo
r publication or distri
bution
Use Case
• Tier-1 OLTP applications like mission-critical billing and reservation systems on SQL
• 800TB – 1PB with 48 nodes across 4 clusters
• All-Flash for latency sensitive workloads
• Plans to move entire Tier-1 Apps to vSAN
• Apps are low compute/high capacity
“For airline apps, more important than IOPS
is reliably low latency, which we could
achieve with vSAN All-Flash”- Philipp Schwarzhuber, Director of SDDC
Hardware Platform
• Dell R730XD servers
Largest Airline in EuropeTier-1 Applications on vSAN
Challenges
• Standardization: SDDC initiative to standardize on Dell
• Meet demanding performance needs of billing and reservation systems
• Need cost savings and ease of scale with increasing application usage
Benefits
• Convergence of teams: team is less siloed, more productive and builds new skillsets
• OPEX improvement: Day-to-day operations are lean and efficient
• Easy to scale: grow with unpredictable airline business
#STO1053BES CONFIDENTIAL 15
VMworld 2017 Content: Not fo
r publication or distri
bution
Use Case
• SDDC Initiative to go 100% HCI with All-Flash
• 48 Node Cluster managing 500TB capacity
• All-Flash for latency sensitive Financial Apps running DB2
• Leading global financial services firm
• Providing Investment banking, securities,
wealth management, and services
Leading Global Financial Services Firm
Challenges
• Standardization: SDDC initiative to standardize on All Flash
• Financial Apps (business critical) & Desktop Virtualization
• Tier 1 workloads requiring high performance / low latency for DB2 and VDI
Benefits
• VSAN All Flash delivers highest performance on NVMe at the lowest price
• Ease of management
• Scalable All Flash VDI solution for Full Clones
Large Financial Institute
#STO1053BES CONFIDENTIAL 16
VMworld 2017 Content: Not fo
r publication or distri
bution
Intel® Optane™ and Intel® 3D NAND SSDs for vSAN
VMworld 2017 Content: Not fo
r publication or distri
bution
18
5-8x faster at low Queue
Depths1
Vast majority of applications
generate low QD storage
workloads
1. Common Configuration - Intel 2U Server System, OS CentOS 7.2, kernel 3.10.0-327.el7.x86_64, CPU 2 x Intel® Xeon® E5-2699 v4 @ 2.20GHz (22 cores), RAM 396GB DDR @ 2133MHz. Configuration – Intel®
Optane™ SSD DC P4800X 375GB and Intel® SSD DC P3700 1600GB. Performance – measured under 4K 70-30 workload at QD1-16 using fio-2.15.
Intel® Optane™ Breakthrough Performance
Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance.
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
Intel® Optane™ Storage Performance Characterization
Higher is better
Low
er
is b
etter
10x latency reduction
▪ < 10usec latency †
100x QoS improvement
▪ < 200usec 99.999th r/w †
Latency vs. Load: NAND SSD vs. Intel® Optane™ SSD (Intel® SSD DC P3700 vs. Intel® Optane™ SSD DC P4800X)
† vs. NAND based SSD
QD
~12
19#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
Intel® 3D NAND SSDs. Optimized for Capacity and Cost
Arr
ay
Smaller FG cell
Architectural efficiency (CMOS under array)
Scale to bigger capacities
Manufacturing efficiency
More
GBper wafer
Advantages
up to
20%higher areal density1
#STO1053BES CONFIDENTIAL 20
VMworld 2017 Content: Not fo
r publication or distri
bution
21
Intel® 3D NAND SSDs. Optimized for Power-Efficient Capacity
48U Open Rack with 12 kw Budget
Compute and Networking
50%of rack space*
50%of rack space*
U.2 15mm 2 TB at 25 W 720 TB, 9 kW
25%of power budget*
75%of power budget*
U.2 7mm 4 TB Intel SSD DC P4501 Series at 10W, 1.44
PB, 3.6 kW
OR
25%of power budget*
30%of power budget*
45%of power budget
SAVED! *
Sources – Intel and Facebook. Facebook Lightening JBOF: https://code.facebook.com/posts/989638804458007/introducing-lightning-a-flexible-nvme-jbof/. Intel – Specifications for Intel® SSD DC P3520 and Intel® SSD
DC P4501. Each rack populates 24U with 15 4TB drives in each rack unit. Storage on the P3520 48U rack consumes 9kW (24U x 15 SSDs/U x 25W/drive max power of U.2 form factor for power budget) while the P4501
rack consumes 3.6kW (24U x 15 SSDs/U x 10W/drive).
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
22
vSphere vSAN
vSAN Datastore
NVMe NVMe
Cache
Capacity
All Writes are served by the Cache Tier
– Absorbs the bulk of the write rate from the application
– Data is de-staged or drained from Write Cache to
Capacity Tier only when needed, extending the life of
the capacity tier
– Deduplication / compression occurs in the de-staging
process
Reads are served directly from the Capacity Tier
• Read performance from capacity flash is more than
sufficient.
Intel® Optane™ SSD provides benefit for low latency
and high performing transactional workloads
Caching and Persistent Tiers with All Flash
SSD SSD
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
Evaluator Group Study with Intel® Optane™ SSD and Intel® Xeon® Scalable Processor
▪ Real-world application profiles
Tests utilized “Application Workloads”
▪ IOmark-VM = General Database and Server VM
▪ HammerDB TPC-C = High Performance Database
Two Different Application Workload Profiles
#STO1053BES CONFIDENTIAL 23
VMworld 2017 Content: Not fo
r publication or distri
bution
Up to 10x Performance w/ Latest HCI Technology on vSAN
24
80
288
480
800 800
1120
0
200
400
600
800
1000
1200
IOmark-VM’s
SSD + HDD
NVMe + SSD
All-NVMe
Optane + S3520
Optane + P4500
Optane + P4500( No Dedupe)
IOmark-VM Performance: 4-Node HC
Optane + NVMe1st Gen NVMeHybrid HCI
Hybrid HCI
1st Gen NVME
Optane + NVMe
https://www.evaluatorgroup.com/document/intel-next-generation-technology-powering-new-performance-levels/#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
Up to 9x Better Price/Performance with Latest HCI Technology on vSAN
25
$2,048
$675 $545
$293 $302 $237
$0
$500
$1,000
$1,500
$2,000
$2,500
$ / IOmark-VM-HC
SSD + HDD
NVMe + SSD
All-NVMe
Optane + S3520
Optane + P4500
Optane + P4500( No Dedupe)
Price / Performance: 4-Node HC
Optane NVMe1st Gen NVMeHybrid HCI
Hybrid HCI
1st Gen NVMe
Optane NVMe
https://www.evaluatorgroup.com/document/intel-next-generation-technology-powering-new-performance-levels/
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
vSAN Database Results (Running a TPC-C “like” workload)
• Nearly 2X better performance and 2X lower price / performance
– 89% higher TPM Performance
– 44% lower cost per TPM
26
$0.05
4,828,595
9,116,024
-
1,000,000
2,000,000
3,000,000
4,000,000
5,000,000
6,000,000
7,000,000
8,000,000
9,000,000
10,000,000
Total TPM
Total TPM for 4-Node vSAN
P4600+P4500
Optane+P4500
$0.03
$-
$0.01
$0.01
$0.02
$0.02
$0.03
$0.03
$0.04
$0.04
$0.05
$0.05
$ / TPM HC
$ / TPM 4-Node vSAN
P4600+P4500
Optane+P4500
https://www.evaluatorgroup.com/document/intel-next-generation-technology-powering-new-performance-levels/
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
HCIBench Tests:Up to 3x lower latency with Latest HCI Technology
*Other names and brands names may be claimed as the property of others
27
System Configuration: 4 Node vSAN* Cluster. Per Node configuration: A:Supermicro* SuperServer 2028U-TN24R4T+ Dual Intel® Xeon® E5-2687Wv4 (12 Core @ 3.0 Ghz), B. Supermicro* SuperServer 2029U-TN24R4T+ Dual Intel® Xeon® scalebale processor, Supermicro* Server Board, 256 GB DDR4 RAM, Boot Drive, 1x Intel® SSD DC S3710 Series (200 GB, 2.5”), vSAN Cluster: Virtual SAN SSDs - 2 Disk Groups comprised of 2x Intel® SSD DC P4800x Series (375GB, 2.5” SFF), 8x Intel® SSD DC P4500 Series (4 TB, 2.5” SFF), vSAN Cluster: Virtual SAN SSDs - 2 Disk Groups comprised of 2x Intel® SSD DC P3700 Series (400GB, 2.5” SFF), 8x Intel® SSD DC P3520 Series (2 TB, 2.5” SFF),
Server: Supermicro* 2028U-TN24R4T with
2 x Intel® Xeon® E5
:
Cache Tier:
2x Intel® SSD DC P3700 @ 400 GB
Transition
to
New Intel® Technology
based vSAN
2xup to
Increase Storage
Efficiency
Capacity Tier:
8x Intel® SSD DC P3520 @ 2TB
Server: Supermicro 2029U-TN24R4T with
2 x Intel® Xeon® Scalable Processor
Cache Tier:
2x Intel® SSD DC P4800x @ 375 GB
Capacity Tier:
8x Intel® SSD DC P4500 @ 4TB
3xup to
Lower
Latency
A B
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
Manageability Improvements for Next Gen Intel® based platform
VMworld 2017 Content: Not fo
r publication or distri
bution
29
Storage at Scale Requires Efficiency
Simple, Reliable and
Manageable
IT Efficiency starts with reliable and serviceable solutions
Storage inspired Intel® Xeon® Platforms enable efficiency to
manage data at scale
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
30
Efficiency Challenge: Lack of Reliability & Serviceability
Did your vSAN node PSOD due to surprise PCIe* SSD removal?
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
31
Which SSD Needs Replacement? Whoops!
These tiny LEDs
are critical
important.
Efficiency Challenge: Lack of Reliability & Serviceability
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
Intel® Volume Management Device
PCIeRoot Complex
Operating System Operating System
Intel® Xeon® E5v4 Processor
Intel® Xeon® Scalable
ProcessorPCIe Root Complex
Intel® VMD
PCIeSSDs
PCIeSSDs
After Intel® VMDBefore Intel® VMD
32
– Intel® VMD is a new technology to enhance solutions with PCIe* storage
– Benefit for VMware* vSAN
– Intel® VMD enables:
• Isolating fault domains for device surprise hot-plug and error handling
• Provide consistent framework for managing LEDs
• Simplify PCIe storage software stacks
Intel® VMD enables customers to simplify and harden solutions using PCIe storage.
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
vSAN NVMe SSDs Hot Plug and LED Management
33
Without Intel® VMD
Supported in vSAN:
• Hot Insert: Supported if the server platform
supports PCIe Hot Plug
• Prepared Hot Removal: Needs user
management operations to hot remove
NVMe SSDs
Not Supported in vSAN:
• Surprise Hot Plug Removal
• Surprise Hot Insert
• LED Management for NVMe SSDs
With Intel® VMD
Benefit to vSAN:
• Surprise Hot Plug Removal ✓
• Surprise Hot Insert ✓
• LED Management for NVMe SSDs ✓
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
Intel® NVMe SSDs Firmware Auto Update
34
vSphere vSAN
vSAN DatastoreNVMe NVMe ….. NVMe…..
Intel® SSD Data Center Tool
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
#STO1053BES CONFIDENTIAL 35
VMworld 2017 Content: Not fo
r publication or distri
bution
What’s next with Intel® NVMe SSDs and vSAN?
VMworld 2017 Content: Not fo
r publication or distri
bution
37
“Ruler” form factor for Intel® SSDs.
Designed from the ground up to optimize rack efficiency, the new Ruler
Form Factor delivers unparalleled Space-Efficient Capacity,
Operationally- Efficient Design and Scalable Manageability.
“Ruler” form factor for Intel® SSD DC P4500 available by end of 2017.
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
1PB IN 42u W/2 TB HDDs
Enabling up to 1PB of storage in a 1U server rack
Opening up new use cases in
datacenter storage with disruptive
total cost of ownership
Storage
Capacity
1PB in 1U Intel® 3D NAND SSD, 32TB
ruler in 2018
38#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
“Ruler” form factor for Intel® SSDs with vSAN - (Future)
“Ruler” form factor for
Intel® SSDs
NVMe Over Fabric
✓ Scale compute and storage independently (disaggregated)
✓ Expand storage (dynamically) to serve compute blades
Workloads for disaggregated storage
✓ Data warehouses, Search databases, Log aggregation, analytics
NVMeOF
EthernetFabric
PCIe
Switch
Compute
Blades
Next Gen
NVMe/PCIe
#STO1053BES CONFIDENTIAL 39
VMworld 2017 Content: Not fo
r publication or distri
bution
Summary:
40
Jointly committed to delivering vSAN HCI
solutions built on cutting-edge Intel® Flash
Technology
Intel® Optane™ and Intel ® Xeon™
Scalable Processors Delivers All Flash
vSAN at the best $/IOP, $/Gig and low
latency
Breakthrough performance for HCI
Higher responsiveness to meet today’s
modern workload needs
Intel ® Optane ™ SSD andIntel® SSD DC P4500
Intel ® Xeon ™ Scalable Platforms
https://www.evaluatorgroup.com/document/intel-next-generation-technology-powering-new-performance-levels/
#STO1053BES CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
VMworld 2017 Content: Not fo
r publication or distri
bution
VMworld 2017 Content: Not fo
r publication or distri
bution