Building a Management Cluster with VSANDAN BARR, PENN STATE ARL

CENTRAL PA VMUG, APRIL 6, 2016

4/5/2016 1

Why a Management Cluster?Dedicated resources for infrastructure management and monitoring

Isolated from production & development workloads for security and performance

Ability to troubleshoot production issues out-of-band

As vSphere environment grows, so do management and monitoring needs

Getting vCloud Suite licensing – even more management appliances

4/5/2016 2

Source: https://vvirtual.wordpress.com/2011/12/01/manage-your-virtual-environment-wisely-separate-your-vmware-vsphere-management-servers-into-different-cluster/

Why VSAN?Wanted dedicated storage for the management cluster◦ If production storage is having a bad day, tools to fix it

are still available

◦ Reduce I/O load on primary storage

Limited choices for dedicated arrays with decent capacity & HA for small $$

VSAN integrated into vSphere – no additional management tools

Simple

4/5/2016 3

WorkloadsvSphere Management:

◦ vCenter (VCSA + external PSC), VUM, vSphere Replication

Storage/Server Management:◦ Compellent Enterprise Manager, Dell OpenManage, EMC InsightIQ, WhatsUp

Network Management:◦ Switch NAC & policy manager, firewall monitoring, IPAM, vulnerability scanners

Supporting Infrastructure:◦ AD, DNS, MS SQL Server, management jump servers (RDS hosts)

Total: 30 VMs, 74 vCPU, 170 GB RAM, 6.2 TB disk

Future: SRM, vRealize Suite, NSX = 35 vCPU, 90 GB RAM, 600 GB disk

4/5/2016 4

HardwareTargeted 2:1 vCPU:Logical Core ratio, 12 TB usable storage

Factored in current & future workloads plus organic growth

All components on VSAN hardware compatibility list

Modest IO needs, so Hybrid VSAN chosen over all-flash

Three Dell R730 2U servers, each with:◦ 24 cores (2x Intel E5-2680 v3), 256 GB RAM

◦ 4x 10GbE (2x Intel X710)

◦ PERC H730 controller in HBA Mode

◦ 2x 400 GB SATA mix-use MLC SSD (cache)

◦ 8x 1 TB 7200 RPM NL-SAS (capacity)

24 TB raw capacity, 12 TB usable with VSAN FTT=1 (mirroring)

4/5/2016 5

Source: dell.com

Disk Layout

E S T

6 74 52 30 1 14 1512 1310 118 9

SA

S

1T

B 7

.2k

SA

S

40

0G

B S

SD

SA

S

40

0G

B S

SD

SA

S

1T

B 7

.2k

SA

S

1T

B 7

.2k

SA

S

1T

B 7

.2k

SA

S

1T

B 7

.2k

SA

S

1T

B 7

.2k

SA

S

1T

B 7

.2k

SA

S

1T

B 7

.2k

4/5/2016 6

Disk Group 1 Disk Group 2

Two Disk groups per host, each with 1x 400 GB SSD cache disk & 4x 1 TB HDD capacity disks20% SSD to usable capacity ratio (best practice is 10% minimum)Room to expand each disk group to 7 capacity disks (max for a VSAN disk group), or could add 3rd DG

Networking

4/5/2016 7

vmnic0vmnic1 vmnic4

vmnic5

VMsManagement Replication ... iSCSI-2vMotion iSCSI-1 VSAN

Uplink 1 Uplink 2 Uplink 1 Uplink 2

Unused

Standby

Active

vmk0 vmk6 vmk2 vmk3 vmk4 vmk5

VDS-FrontEnd VDS-BackEnd

Port Group

Virtual Distributed Switch

Virtual Standard Switch

VMkernel NIC

Health ChecksVSAN Health Check Plugin integrated with vSphere 6.0 Update 1

◦ Checks health of all components

◦ Includes checking compatibility of hardware, drivers, firmware

◦ Proactive tests: VM creation, multicast performance, and storage performance

4/5/2016 8

PerformanceMultiple test workloads available

Results using “70/30 read/write mix, realistic, optimal flash cache usage” test:

41,896 IOPS, 163.7 MB/sec throughput, 1.1ms avg. latency

4/5/2016 9

Issues/ChallengesBug in 6.0 U1b:

◦ Warning shown, but all health checks are green

◦ KB 2143214 – cosmetic, resolved in 6.0 U2

◦ Can clear it by restarting management agents on hosts

Performance Monitoring:◦ No performance charts in vSphere Web Client as of VSAN 6.1 (vSphere 6.0 U1) – new in 6.2

◦ Need to use VSAN Observer via RVC (Ruby vSphere Console) on vCenter to generate live perf stats

Multicast Performance Test:◦ Only runs on 2 hosts in a 3-host cluster, shows lower than expected bandwidth, or fails

◦ Known issue, still present in 6.2

Bootstrap vCenter onto VSAN:◦ vCenter needed to configure VSAN, so how do we get it there to start?

◦ Temporary iSCSI volume mounted to first host

4/5/2016 10

TipsCheck the HCL!

◦ Always important with vSphere, even more with VSAN

Read the VSAN 6.0 Design and Sizing Guide◦ http://www.vmware.com/files/pdf/products/vsan/VSAN_Design_and_Sizing_Guide.pdf

Use the VSAN Sizing Calculator◦ https://vsantco.vmware.com

Document your shutdown & startup procedures◦ Potential chicken & egg problem with vCenter running on VSAN

◦ http://www.virtuallyghetto.com/2014/07/quick-tip-steps-to-shutdownstartup-vsan-cluster-wvcenter-running-on-vsan-datastore.html

Set up alerts!◦ Lots available, set notifications for anything health-related

4/5/2016 11

Thank You!Questions?

Contact:

Twitter: @vDanBarr

LinkedIn: https://www.linkedin.com/in/vdanbarr

Email: drb45@psu.edu

4/5/2016 12

This document was created using the official VMware icon and diagram library. Copyright © 2012 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at http://www.vmware.com/go/patents.

VMware does not endorse or make any representations about third party information included in this document, nor does the inclusion of any VMware icon or diagram in this document imply such an endorsement.