LSI Data Center Solutions GroupDecember 17, 2013Khai Le

VMware Acceleration with Nytro WarpDrive PCI Flash and Proximal Data AutoCache

2Confidential

New Product Terminology and Solution Strategy

Nytro XD-V AutoCache

vAppAutoCache_1.1.21.ova

Nytro WarpDrive (Legacy)

LSI support for Nytro XD and XD-V EOL!

Third Party Solutions – Proximal AutoCache, EMC VF Cache, NetApp Flash Cache, SanDisk Express Cache and others

Nytro XP6209 and Nytro XP6210

3Confidential

Objectives:

Understand how AutoCache accelerates storage in virtualized environment

Understand how to test AutoCache performance scaling using synthetic benchmark

Understand how to tune for optimal AutoCache performance in virtualized environment

Understand how to monitor AutoCache IO workload and performance statistics in virtualized environment

4Confidential

Agenda

Why performance is important?

Customer satisfaction

Reduce solution and operating costs

Enable Nytro WarpDrive acceleration in virtualized environments

Product

AutoCache Overview Review Performance Improvement

VMware Storage Acceleration

Summary of results

Available Resources

5Confidential

Proximal Data AutoCache & OS SupportCache Device Auto Cache 1.1 Auto Cache 2.0

Intel® SSD 313 Series √ √

Intel® SSD 320 Series √ √

Intel® SSD 330 Series √ √

Intel® SSD 520 Series √ √

Intel® SSD 710 Series √ √

Intel® SSD 910 Series √ √

LSI Nytro WarpDrive WLP Series

√ √

LSI Nytro WarpDrive BLP Series

√ √

LSI Nytro WarpDrive SLP Series

√ √

Micron P400e SSD Series √ √

Micron P320h SSD Series √ √

More OS Support Auto Cache 1.1 Auto Cache 2.0

VMware ESXi 4.1, 5.0, 5.1

√ √

VMware ESXi 5.5 √

6Confidential

vCenter Diagram

7Confidential

Storage Acceleration for Server Virtualization

DASSANNAS

+

NytroXD-VCaching Software+

Nytro WarpDrive

AutoCacheSoftware

8Confidential

System Components impacting Max IOPS Nytro WarpDrive PCIe Flash Controller

AutoCache Flash Caching Software

Baseline (dataStorage - DAS /SAN /NAS)

Application I/O workload

Performance in Virtualized Environments

9Confidential

Single VM measure IOPS

Notes:• Baseline (R10-16HDD): Single 16-drive RAID-10 HDD datastore baseline• WarpDrive_100G_vmdk: Nytro WarpDrive 6203 as VM datastore, no caching• NXD-V 100G_Hot_Region: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software

10Confidential

Single VM measure Latency (milliseconds)

Notes:• Baseline (R10-16HDD): Single 16-drive RAID-10 HDD datastore baseline• WarpDrive_100G_vmdk: Nytro WarpDrive 6203 as VM datastore, no caching• NXD-V 100G_Hot_Region: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software

Use log scale so you can see the lower latency better?

11Confidential

VMware VisualESXTOP capture on AutoCacheMax IOPS & VM IOPS -Workload 8KB 100% RR

Virtual Disks (vmdk): 4 * 20K IOPS ~ 80K IOPS for Nytro WarpDrive 6203

12Confidential

Configuration

Vmware– ESXi version 5.0 Update 2– 64bit build 914586

AutoCache v1.1.21– AutoCache: Pdi-cache ver-500.1.1.21-1381159898– Cache Statistics Monitor: proximal ver-500.1.1.21-1381159898

Nytro WarpDrive 6203– 1.6TB (eMLC)

MegaRAID (TB) controller– VMware Data Storage (VD)

• 16 SAS 6G HDD RAID-10– WT, NRA, DIO, DCD, SS 64KB

VM information:– Windows 2008 R2 SP1– 4 vCPU @ 2.0 GHz– 4 GB of vRAM– 50 GB vmdk

13Confidential

Nytro WarpDrvie 6203 Information

Model: Nytro WarpDrive 6203 1.6TB DFF flash type: eMLC Over Provision: 28% Default Capacity: 1.46 TB after OP setting Firmware: 110.00.00.00 (Phase 10 components) Driver: mpt2sas 17.00.00.00.1vmw (Phase 10 components)

IOmeter ver-2008.06.18-RC2. - Highly patterned compressible data Workload: 8KB block Size, 100% Read 0% Write, 100% Random IO Queue Depth 64, 8KB Aligned I/O 1 worker 50G Hot Region Per VM

Workload

14Confidential

PowerCLI Script

Syntax for mulitple VMs kick off performance test at the same time– Ensures each VM receives balanced percentage of resources that

balances total available performance between VMS

$cred = Get-CredentialGc test.txt | % { Invoke-VMScript –RunAsync –VM $_ -ScriptText { C:\iom2008\DEMO_RUNME.bat } –GuestCredential $cred }

Test.txt vsiTarget001

vsiTarget002 vsiTarget003 vsiTarget004

DEMO_RUNME.bat C:\iom2008\iometer.exe /c iom2008\8kb100R0W.icf /r iom2008\vm4_results.csv

15Confidential

Run time270 intervals of 10 sec each = 45 mins

16Confidential

HDD Volume Performance Baseline4 VMs (vmdk) data Stores on same base volume

4 VMs * 500 IOPS/VM ~ 2,000 IOPS total

17Confidential

Accelerated Multiple (4) VMs Performance Scaling

AutoCache v1.1.21 Accelerate Data Storage (4 VMs -vmdk)Nytro WarpDrive 6203 PCIe Flash

Notes:• Baseline: DAS RAID-10 for datastore• NytroXD-V: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software

Hot data size = 50GB/VM

4 VMs * 2,000 IOPS/VM ~ 80,000 IOPS total

18Confidential

Nytro WarpDrive 6203 PCIe Flash – no AutoCache4 VMs (vmdk) on 1.6TB eMLC

4 VMs * 25,000 IOPS/VM ~ 100,000 IOPS totalShows AutoCache v1.1.21 Caching Overhead ~22.5us

19Confidential

Accelerated Multiple (4) VMs on SAN Storage

AutoCache v1.1.21 + Nytro WarpDrive 6203 PCIe FlashMix Workload: 8KB 100% Random, 70/30 (R/W)

Hot data size = 50GB/VM

4 VMs * 1750 IOPS/VM ~ 7,000 IOPS total

Notes:• SAN: backend connect to Snowmass,16-HDDs create RAID-10 for datastore• NXD-V: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software

20Confidential

Accelerated Multiple (4) VMs on SAN Storage

AutoCache v1.1.21 + Nytro WarpDrive 6203 PCIe FlashMix Workload: 8KB 100% Random, 70/30 (R/W)

Notes:• SAN: backend connect to Snowmass,16-HDDs create RAID-10 for datastore• NXD-V: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software

AutoCache reduces IO latency

21Confidential

AutoCache VMware Cache ManagementCapture for 4 VMs on SAN

Notes:• AutoCache Cache policy: Write Through mode

Mix workload 70/30 (R/W)

22Confidential

Performance Tip

Format Nytro WarpDrive 6203 PCIe Flash Controller Syntax:

./ddcli –c 1 –format ./ddmfgcli –c 1 –format

Precondition flash Cache device iometer.exe /c iom2008\8kb100%SW.icf /r iom2008\precondition_results.csv

Increase Nytro WarpDrive 6203 Driver Queue Depth 1. Syntax adjust SATA Q.Depth: esxcfg –module –s mpt2sas_SATA_queue_depth=128 mpt2sas

2. Syntax adjust max controller Q.Depth: esxcfg-module --set-options "max_queue_depth=620” mpt2sas

23Confidential

ESXi Host, Disk Queue Depth:VMware -> configuration -> Software ->

Advanced Settings -> Disk.SchedNumReqOutStanding=256

Create vmdk select option Independent disks

are not affect by snapshots

Continue Performance Tip

24Confidential

Performance Tuning:Driver Queue Depth and Disk Schedule Outstanding Request Count

Default Queue Depth

1VM Performance Shown

Notes:• IOmeter: 8KB 100% random read• NytroXD-V: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software

25Confidential

Esxtop follow cmd options:

-v disk vm-u device Queue Depth-d disk Adapter-c CPU utilization-m memory-n Network-p Power

VMware Real Time Performance Monitor

26Confidential

Cmd: esxtop v, dHDD Baseline Performance

500 IOPS x 4VM ~ 2000 IOPS

27Confidential

Cmd: esxtop v, dAccelerated AutoCache v1.1.21 Performance

28Confidential

How to save esxtop statistics:

esxtop -b -d 2 -n 100 > esxtopcapture.csv

esxtop -b -a -d 10 -n 100 | gzip -9c > esxtopoutput.csv.gz -

29Confidential

IO Analyzer under virtualization

Vmware Visual ESXtop– Connect to a live

Server vCenter, ESXi host Connect multiple windows– Chart metrics– Select metrics counter– Save results to a file– Replay data to chart or

graph

30Confidential

MySQL Database Workload Example

Commands/sec (total)

Reads/sec

Writes/sec

31Confidential

Additional Statistics Monitoring Options

Cmds/s -the number of cmd issued per second

DAVG/cmd -the average device latency(m.secs) per cmd

DAVG/rd -the average device latency per/read

DAVG/wd -the average device latency per/write

DQLEN -the storage device Q. Depth.

GAVG/cmd -the average Guest OS latency m.sec per cmd

KAVG/cmd -the average ESX Vmkernel latency per cmd

LOAD -ratio of ESX VMKernel active cmd +queue cmd

MBREAD/s –Mega bytes read per second

MBWRTN/s-Mega bytes write per second

QAVG/cmd –the average queue latency (ms)per cmd

QUED -# of cmd in ESX Vmkernel currently queued

READ/s –the number of read cmd issued per second

WRITE/s-the number of write cmd issued per second

More

32Confidential

AutoCache v1.1.21 Multiple NWD Performance Scaling

Second Nytro WarpDrive 6203 provides additionalcaching capacity but no performance improvement.

Notes:• IOmeter: 8KB 100% random read• NytroXD-V: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software

33Confidential

AutoCache v1.1.21 Average Response Time

Zoom in at 20 msec, so latency between 1 NXP and 2 NXPs about the same

1x Nytro WaprDrive 6203 Latency

2x Nytro WarpDrive 6203Latency

Notes:• IOmeter: 8KB 100% random read• NytroXD-V: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software

34Confidential

AutoCache 2.x Multiple NWD Performance Scaling

2x Nytro WarpDrive 6203 performance AutoCache v2.0 demonstrates Nytro WarpDrive 6203 PCIe flash performance scaling

1x Nytro WarpDrive 6203 performance

35Confidential

AutoCache 2.x Average Response Time

(milliseconds)

1x Nytro WarpDrive6203 Latency

2x Nytro WaprDrive 6203 Latency

36Confidential

AutoCache Summary

Nytro WarpDrive 6203 Scaling - 1 and 2 PCIe Flash Devices– AutoCache 1.1.21

• Significant performance improvement• Added flash caching capacity• Added performance for hot region size > 1st Nytro WarpDrive 6203 capacity

– AutoCache 2.0• Significant performance improvement• Added flash caching capacity• Added performance for any hot region size

Performance Ramp Up Time– Dependent on total hot region size (i.e., #VMs * Hot Region Size/VM)

Nytro WarpDrive 6203 Driver Performance Optimizations– Increasing Queue Depth and Disk Schedule Outstanding Request Count

• Significant acceleration improvement

37Confidential

Resources

Nytro WarpDrive and Nytro XP http://

www.lsi.com/products/flash-accelerators/pages/default.aspx#tab/product-family-tab-3

AutoCache Resources http://www.proximaldata.com/product/resources.phpv

Proximal Download Portal http://www.proximaldata.com/support/software_download_portal.php

Sphere Best Practices http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.0.pdf

esxtop Download http://www.yellow-bricks.com/esxtop/ 

VMware io Analyzer Download http://labs.vmware.com/flings/io-analyzer

Q/A?