WHITE PAPER 1
DELL EMC VXRACK SYSTEM SDDCTM TECHNOLOGY OVERVIEW
ABSTRACT
VxRack System SDDC powered by VMware Cloud Foundation is a turnkey, rack-scale,
hyper-converged engineered system. Fully integrated hardware and software provides
an agile and flexible infrastructure foundation that IT organizations can leverage as part
of their IT transformation into an easy to manage cloud operating model. This is
accomplished through system features such as workload domains and automated
infrastructure deployments that are comparable to public cloud offerings, providing
elasticity and scalability while reducing costs.
This paper provides an overview of VxRack SDDC, describes its major components,
and describes its value as a turnkey Software-Defined Data Center system.
October 2017
Document H15859-1
The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.
Use, copy, and distribution of any software described in this publication requires an applicable software license. Copyright © 2017 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be the property of their respective owners. Published in the USA 2/17 VxRack SDDC Whitepaper.
Dell EMC believes the information in this document is accurate as of its publication date. The information is subject to change without notice.
Dell EMC VxRack System SDDC
2
TABLE OF CONTENTS
EXECUTIVE SUMMARY ...........................................................................................................3
INTENDED AUDIENCE .............................................................................................................3
INTRODUCTION ........................................................................................................................3
ENGINEERED SYSTEM OVERVIEW .......................................................................................5
CLOUD FOUNDATION SOFTWARE STACK ..........................................................................6
vSphere ........................................................................................................................................... 77
vSPHERE VIRTUAL NETWORKING ............................................................................................................................. 8
vCENTER SERVER FEATURES FOR AVAILABILITY AND PERFORMANCE ............................................................ 8
NSX ................................................................................................................................................... 9
NSX COMPONENTS ................................................................................................................................................... 10
vSAN ............................................................................................................................................... 10
vSAN Datastore ............................................................................................................................................................ 11
vSAN Storage Policy-Based Management ................................................................................................................... 11
vSAN Features ............................................................................................................................................................. 11
SDDC Manager ............................................................................................................................... 12
Simplified Resource Management with Workload Domains ........................................................................................ 12
Lifecycle Management .................................................................................................................................................. 14
vRealize Suite—VMware’s Cloud Management Platform (CMP) .................................................... 14
Horizon Suite—VMware’s VDI and App Virtualization Platform ...................................................... 15
HARDWARE CONFIGURATION ............................................................................................ 16
Hardware Components ................................................................................................................... 17
Cabinets ........................................................................................................................................................................ 17
Server Nodes ................................................................................................................................................................ 18
Server Node Storage Options ...................................................................................................................................... 18
Network Switches ......................................................................................................................................................... 19
NETWORK TOPOLOGY ........................................................................................................ 21
Network Topology for Single-Rack Configuration ............................................................................ 21
Network Topology for Multi-Rack Configuration .............................................................................. 22
Traditional and Rack Interconnect Network Architecture ................................................................. 22
CONCLUSION ........................................................................................................................ 25
Dell EMC VxRack System SDDC
3
EXECUTIVE SUMMARY
The Dell EMC VxRack™ SDDC consists of hyper-converged, rack-scale engineered systems with integrated networking to achieve the
scalability and management requirements of traditional and cloud-native workloads. Dell EMC and VMware have expanded the
industry’s broadest hyper-converged infrastructure portfolio to provide a turnkey rack-scale, hyper-converged system with VxRack
SDDC. The VxRack SDDC is a fully integrated hardware and software solution with VMware Cloud Foundation. A complete turnkey
solution at rack scale, the VxRack SDDC provides industry-best lifecycle management and assurance for VMware SDDC solutions
adding rack-scale capabilities to complement the Dell EMC Vblock® Systems and VxRail™ Appliances.
The hyper-converged VxRack SDDC is designed for enterprise-scale deployments of virtual infrastructure, IaaS, and virtual desktops
infrastructure (VDI) in a VMware environment. Each VxRack SDDC system is based on a standardized architecture that combines Dell
EMC and Cisco hardware with pre-loaded, pre-integrated software components in a complete and validated system. The VxRack
SDDC is a turnkey VMware SDDC system.
Together with VMware, only Dell EMC can provide a tightly engineered hyper-converged solution with storage, networking, computing,
and software. VxRack SDDC brings standardization, modular scale, lifecycle management, and industry-best support for on-demand IT
services that further accelerate business outcomes and the transformation to cloud computing.
INTENDED AUDIENCE
This document is intended for Dell EMC sales and field personnel, partners, and customers involved in designing, acquiring, and
managing a VxRack SDDC solution. It may also be a useful resource for System Administrators and Dell EMC Solutions Architects.
INTRODUCTION
The IT infrastructure market is undergoing unprecedented transformation. The most significant transformation is reflected by two major
trends: a deployment trend toward converged and hyper-converged infrastructure (HCI) and a design trend toward a software-defined
data center architecture. Both are responses to the IT realities of infrastructure clutter, complexity, and high cost; they represent
attempts to simplify IT and reduce the overall cost of infrastructure ownership.
Today’s legacy infrastructure environments are typically comprised of multiple hardware and software products from multiple vendors,
with each product offering a different management interface and requiring different training. Each product in this type of legacy stack is
likely to be grossly overprovisioned, using its own resources (CPU, memory, and storage) to address the intermittent peak workloads of
resident applications. The value of a single shared resource pool, offered by server virtualization, is still generally limited to the server
layer. All other products are islands of overprovisioned resources that are not shared. Therefore, low utilization of the overall stack
results in the ripple effects of high acquisition, space, and power costs. Too many resources can be wasted in traditional legacy
environments.
In the modern data center, software defined, converged, and hyper-converged infrastructure (HCI) are cornerstones of virtually every
customer’s IT strategy. Converged and hyper-converged infrastructures mean that multiple pre-engineered and pre-integrated
components operate under a single controlled architecture with a single point of management and a single source for end-to-end
support. HCI provides a localized single resource pool that enables a higher overall resource utilization than with a legacy island-based
infrastructure. There is no separate Storage Area Network (SAN) infrastructure layer to manage. Overall acquisition cost is lower and
management is simplified. In the data center, HCI typically has a smaller footprint with less cabling and can be deployed much faster
and at lower total cost than traditional infrastructure.
Industry infrastructure deployment is transforming as customers begin to shift from a “build” to a “buy” approach. This deployment shift
is being driven by the need for IT to focus limited economic and human capital resources on driving business innovation, resulting in
less resources available to focus on infrastructure. While a “build-your-own” deployment strategy can achieve a productive IT
infrastructure, this strategy can be difficult to maintain and vulnerable to higher operating costs. It is susceptible to greater risk related to
component integration, configuration, qualification, compliance, and management. A “buy” deployment strategy for HCI provides the
benefits of previously integrated, configured, qualified, and compliant components. Buying an HCI system provides a single optimized
IT infrastructure base which is quick and easy to deploy. A “buy” deployment strategy for HCI provides a simple and effective alternative
to “build-your-own”, and it is widely adopted.
Dell EMC VxRack System SDDC
4
The software-defined data center (SDDC) is a significant design trend driving an unprecedented transformation of the IT infrastructure
market. SDDC is VMware’s architectural vision for the modern, emerging software-centric data center. Dell EMC and VMware share
this vision and together are developing new technologies to make this vision a reality for customers. While the concept is still evolving,
the SDDC is a software-centric architectural design for the data center based on virtualization and automation. It logically defines all
data center infrastructure services by applying the widely successful principles of server virtualization—abstraction, isolation, and
pooling—to the remaining network and storage infrastructure services. SDDC management is automated through policy-based software
that controls resources both on premises and off premises. With SDDC, traditional enterprise applications can be supported in a more
flexible and cost-effective manner. SDDC represents the epitome of the agile digital business model, where pooled resources adapt
and respond to shifting application requirements. The SDDC architectural vision is being driven by VMware, a thought leader in the
industry. See Figure 1 below for a typical SDDC Architecture.
Figure 1: Software-Designed Data Center (SDDC) Architecture
In the SDDC, virtualized servers represent a familiar software-defined IT model, in which hypervisors running on a cluster of hosts
allocate hardware resources to virtual machines (VMs). In turn, VMs can function with a degree of autonomy from the underlying
physical hardware. Software-defined storage (SDS) and software-defined networking (SDN) are based on a similar premise: physical
resources are aggregated and dynamically allocated based on predefined policies with software abstracting control from the underlying
hardware. The result is the logical pooling of compute, storage, and networking resources. Physical servers function as a pool of CPU
resources hosting VMs, while network bandwidth is aggregated into logical resources, and pooled storage capacity is allocated by
specified service levels for performance and durability.
Once the SDDC has abstracted resources, SDDC services make the data center remarkably adaptable and responsive to business
demands. In addition to virtualized infrastructure, the SDDC includes automation, policy-based management, and hybrid cloud services.
The policy-based model eases compliance capabilities and insulates users from the underlying standardized technology, and policies
balance and coordinate resource delivery. Resources are allocated where needed, absorbing utilization spikes while maintaining
consistent and predictable performance. Conceptually, SDDC encompasses more than the IT infrastructure itself; it also represents an
essential departure from traditional methods of delivering and consuming IT resources. The SDDC vision requires changes in the IT
organization technology, people, and process which then enables a business’s transformation to a cloud model and toward a digital
transformation as a whole. In the SDDC, infrastructure, platforms, and software become cloud computing services and are called
Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS), Software-as-a-Service (SaaS) and Container-as-a-Service (CaaS).
SDDC is the fundamental architecture that underpins the most sophisticated cloud computing environments.
Dell EMC VxRack System SDDC
5
Cloud services (IaaS, PaaS, SaaS, and CaaS) are accessed via a cloud management platform (CMP) that provides end users with a
self-service catalog of IT services with financial transparency. The VMware SDDC architectural vision includes CMP layers running on
top of SDDC infrastructure layers to create a private or hybrid cloud. This requires a SDDC infrastructure with an extremely high level of
efficiency and serviceability, such as the VxRack SDDC system. VxRack SDDC is the simplest and most complete underlying
infrastructure system needed when building a SDDC with private or hybrid cloud computing services. VMware’s cloud management
platform, vRealize Suite, when combined with VxRack SDDC delivers the most complete solution for a private or hybrid cloud
computing environment.
VxRack SDDC, a hyper-converged engineered system designed as one by Dell EMC and powered by VMware Cloud Foundation, is
comprised of innovative technology from three leading industry providers—Dell EMC, Cisco, and VMware. VxRack SDDC provides a
tightly integrated, turnkey hardware and software system with virtualized computing, networking and storage. VxRack SDDC provides
standardization, modular scale, lifecycle management, and industry-best support for on-demand IT services that further accelerate a
business’ transformation to cloud computing. VxRack SDDC is evolutionary in its design and engineering.
ENGINEERED SYSTEM OVERVIEW
The VxRack SDDC hyper-converged infrastructure system, shown in Figure 2, is powered by VMware’s Cloud Foundation, an
integrated virtualization software platform that includes vSphere, NSX, vSAN, and the SDDC Manager. The hyper-converged VxRack
SDDC is a system which is sized and optimized for customer workloads, and manufactured to customer specifications. The system
arrives on site ready to be configured and integrated into the data center network, and to provision virtual servers within hours.
This new hyper-converged system is designed for companies that want a complete cloud platform infrastructure with improved
economics and security. VxRack SDDC has been developed for companies that have moved to standardize on VMware virtualization
and cloud technologies and want a premier foundation for automated data center operations. VxRack SDDC is an agile system that
enables customers to build an IaaS platform based on a complete SDDC architecture. Its modular design allows datacenter scalability
and flexibility as budgets for next-generation cloud workloads and enterprise applications demand. It’s designed for deployments
involving large numbers of VMs and provides the following features:
Automated bring-up delivers best-in-class day 0 experience – from power-on to system ready in a few hours
Low provisioning complexity with simple-to-use functionality, optimized to consolidate thousands of applications with proven
VMware technologies
Flexible and variable configuration options to support a variety of application workloads
Horizontal scaling by adding, re-assigning, and extending nodes on the fly to extend compute, storage, and networking resources
exponentially
Easy day-to-day management and one-click software upgrades
Pre-configured, pre-loaded, pre-tested, and fully optimized hardware and software stack, delivered as a fully assembled and single
vendor supported system
Enhanced networking and security capabilities built-in and automatically configured
Future proof with full release certification of software and hardware upgrades to insure reliability
A complete, turnkey system, the Dell EMC hyper-converged VxRack SDDC provides the easiest and fastest way to stand up a VMware
based private cloud while at the same time introducing the performance, flexibility, and automation required to make IT departments
more agile. VxRack SDDC powered by VMware Cloud Foundation delivers a rack scale hyper-converged system that simplifies the
transformation to a cloud computing model.
Dell EMC VxRack System SDDC
6
Figure 2: VxRack SDDC
VMWARE CLOUD FOUNDATION SOFTWARE STACK
The VxRack SDDC architecture defines data center resources in terms of software. It minimizes compute, network, and storage
hardware constraints and increases infrastructure service agility. This is the evolution from simple server virtualization to complete
virtualization and automation of a modern data center. VMware SDDC software provides virtualization and automation across fully
integrated components.
VMware Cloud Foundation provides the core software platform for this virtualization and automation in the VxRack SDDC. Cloud
Foundation has been designed to extend and optimize the familiar VMware vSphere experience with robust software-defined solutions
for storage and networking as well as simple management. It provides the opportunity to leverage a single standardized SDDC platform
for both on-premises private cloud and off-premises public clouds which ensures compatibility for a true hybrid cloud architecture. It’s a
complete SDDC solution that provides a highly available, resilient, on-demand infrastructure.
Figure 3: VMware Cloud Foundation
Dell EMC VxRack System SDDC
7
Cloud Foundation is a unified SDDC software platform that combines leading virtual-computing technology into a natively and fully
integrated, enterprise-ready infrastructure for private and public clouds. It’s made up of vSphere, vSAN, and NSX, and is managed by
the SDDC Manager as shown in Figure 3 above. The following sections examine the Cloud Foundation software components as
implemented in VxRack SDDC.
vSPHERE
VMware vSphere, the compute virtualization component of VxRack SDDC, delivers server virtualization within a highly available,
resilient, and efficient on-demand infrastructure. vSphere is an established, industry-leading software platform with the following
advantages:
Continuous availability and fault tolerance with data protection and replication
Simplified customer experience for automation and management at scale
Optimal hypervisor supported by a broad ecosystem and scalable management
Comprehensive built-in security for protecting data, infrastructure, and access
Load-balanced workloads and prioritized access to resource for top performance and compliance
Rapid provisioning and deployment of workloads and desktops
VxRack SDDC leverages two key components of vSphere: ESXi and vCenter. ESXi is the hypervisor software which is installed directly
onto physical VxRack SDDC nodes, the nodes are powered by the Dell EMC PowerEdge server platform. Each physical server
functions as a virtual ESXi server (ESXi host) made up of partitioned logical servers referred to as virtual machines (VMs). VMs are
configured on top of the ESXi server. VMware vCenter is available under a separate license and is a centralized management
application that is used to manage the vSphere ESXi servers (hosts) and VMs. vCenter communicates with each ESXi host using an
agent that relays tasks to perform management operations directly on the hosts.
VMWARE vCENTER SERVER
The vCenter Server software provides the primary point of management for server virtualization and storage using vSAN, and is the
enabling technology for advanced capabilities such as vMotion, DRS and HA. vCenter scales to enterprise levels where a single
vCenter can support up to 1000 ESXi servers and 10,000 VMs. vCenter supports a logical hierarchy of data centers, clusters, and
hosts, which allows resources to be segregated by use cases or line of business and can also scale dynamically as needed.
vCenter is a single interface that provides a number of services, including task scheduling, statistics logging, alarms, event
management, and VM provisioning and configuration. vCenter also provides distributed services such as vSphere vMotion, vSphere
DRS, and vSphere HA. VxRack SDDC uses vCenter for its own system management cluster as well as for user provisioned workload
cluster management.
VMWARE vSPHERE ESXI
VMware vSphere ESXi is an enterprise-class hypervisor that deploys and services VMs. Figure 4 illustrates vSphere ESXi basic
architecture. ESXi partitions a physical server into multiple secure and portable VMs that can run side by side on the same physical
server. Each VM represents a complete system—with processors, memory, networking, storage, and BIOS—so any operating system
(guest OS) and applications can be installed and run on the virtual machine without any modification. The hypervisor provides physical
hardware resources, dynamically to support the operation of VMs. The hypervisor enables the VMs to operate independently from the
underlying physical hardware. For example, a virtual machine can be moved from one physical host to another. Also, the VM’s virtual
disks (VMDKs) can be moved from one type of physical storage to another without affecting the functioning of the virtual machine. ESXi
also isolates VMs from one another, so when a guest operating system running in one VM fails, other VMs on the same physical host
are unaffected and continue to run. VMs share access to CPUs, and the hypervisor is responsible for CPU scheduling. In addition, ESXi
assigns each VM a region of usable memory and provides shared access to the physical network cards and disk controllers associated
with the physical host.
Dell EMC VxRack System SDDC
8
Figure 4: vSphere ESXi Architecture
vSPHERE VIRTUAL NETWORKING
vSphere also provides a rich set of virtual networking capabilities, which are managed through vCenter. Virtual-switch technology
allows communication among ESXi servers and among VMs in a cluster, using the same protocols that would be used over physical
switches. The virtual switch supports VLANs and forwards frames at the data-link layer and virtual Ethernet adapters each have their
own IP and MAC address. As a result, VMs have the same properties as physical machines from a networking perspective. The virtual
adapters connect both VMs and the ESXi server console to external networks.
VxRack SDDC clusters use the VMware Virtual Distributed Switch (VDS), a single switch that spans across multiple ESXi hosts in the
same cluster. This switch enables VMs to maintain consistent network configurations as they migrate across multiple hosts. VDS is
configured in vCenter Server at the data-center level and makes the configuration consistent across all ESXi hosts. vCenter Server
stores the state of distributed ports in the vCenter Server database. Networking statistics and policies migrate with VMs when the VMs
are moved from host to host. As discussed in upcoming sections, vSAN relies on VDS for its storage virtualization functionality, and
VDS is also the underlying technology which NSX leverages for its network virtualization functionality.
vCENTER SERVER FEATURES FOR AVAILABILITY AND PERFORMANCE
VxRack SDDC leverages the following vSphere software suite technologies to ensure a high level of availability and load balanced
performance during planned and unplanned system outages.
vMotion for VM migration
Distributed Resource Scheduler (DRS) for VM load balancing
vSphere HA for node failover protection
vMotion. VMware vMotion™ enables live migration of running VMs from one physical server to another with no downtime, continuous
service availability, and complete transaction integrity. vMotion is a key enabling technology for creating a dynamic, automated, and
self-optimizing data center. vMotion continuously and automatically allocates VMs within resource pools. It also improves availability by
conducting maintenance without disrupting business operations. The advanced capability for migrating workloads without disruption is
one of the features that distinguish the VxRack SDDC system from other hyper-converged systems.
Dell EMC VxRack System SDDC
9
Distributed Resource Scheduler. The Distributed Resource Scheduler (DRS) balances computing capacity across a collection of
VxRack SDDS resources that have been aggregated into logical pools. It continuously balances and optimizes compute resource
allocation among the VMs. When a VM experiences an increased workload, DRS evaluates the VM priority against user-defined
resource-allocation rules and policies. If justified, DRS allocates additional resources. It can also be configured to dedicate consistent
resources to the VMs of particular business-unit applications to meet SLAs and business requirements. DRS allocates resources to the
VM either by migrating the VM to another server with more available resources or by making more “resources” for the VM on the same
server by migrating other VMs off that server. In VxRack SDDC, all ESXi hosts are part of a vMotion network. The live migration of VMs
to different node servers is completely transparent to end users through VMotion. DRS adds tremendous value to the VxRack SDDC
clusters by automating VM placement, ensuring consistent and predictable application-workload performance.
vSphere HA. vSphere HA failover technology protects VMs with rapid recovery from outages, providing cost-effective high availability
for VM applications. vSphere HA uses a Fault Domain Manager (FDM) agent to proactively monitor host availability and power state.
When a host fails, vSphere HA restarts the affected VMs on another host. Once vSphere HA is configured, all workloads are protected.
No actions are required to protect new VMs, and applications and VMs need no special software. vSphere HA provides several points
of protection for applications including:
Circumventing any server failure by restarting the VMs on other hosts within the cluster.
Continuous monitoring of VMs and resetting of any detected VM failures.
Protecting against datastore accessibility failures and providing automated recovery for affected VMs.
Protecting VMs against network isolation by restarting them if their host becomes isolated on the management or VMware vSAN
network. This protection is provided even if the network has become partitioned.
NSX
Physical and virtual network design and planning for growth when deploying a cloud model infrastructure is critical to maintaining
operational performance as the environment scales. The Dell EMC VxRack SDDC integrates the latest in networking best practices at
rack scale—both physical and software defined. Physical networking consists of a Cisco topology with top-of-rack (ToR) and rack
interconnect switches. Software-defined networking (SDN) consists of VMware NSX and VMware SDDC Manager for management
which provides network configuration, control, and management of the physical and virtual network elements.
NSX network virtualization delivers the operational model of a VM to VxRack SDDC’s network infrastructure. As an integral component
in the VxRack SDDC architecture, NSX SDN injects improved security into the entire data center infrastructure. With NSX, network
functions including switching, routing, and firewalling are embedded in the hypervisor and distributed across the environment. This
effectively creates a “network hypervisor” that acts as a platform for virtual networks and services as shown in Figure 5.
Figure 5: NSX Software-Defined Networking
Dell EMC VxRack System SDDC
10
NSX virtual networks leverage automated, policy-based provisioning and multi-tenant isolation to simplify network management, even
for complex multi-tier network topologies and multiple workload domains. NSX reproduces the entire network model in software,
enabling any network topology to be created and provisioned in seconds. Users can create multiple virtual networks with diverse
requirements, leveraging a combination of the services offered via NSX, to build inherently more agile and secure environments.
NSX COMPONENTS
The NSX virtualized SDN is built on several key components described below.
NSX Manager. NSX Manager provides a central point to deploy and configure the virtualized network components, including
controller cluster systems and VMware Installation Bundles (VIB) for ESXi, VXLAN, logical switching, logical firewall, and logical
routing. NSX Manager can also deploy and configure Edge gateway systems and its services.
NSX Manager is deployed as a single VM in the management domain and controller clusters are deployed on the workload domain
cluster for which the NSX Manager is responsible for managing. Each workload domain has an NSX Manager VM per vCenter
Server VM with NSX virtual switches installed in the hypervisors. SDDC Manager uses NSX APIs to configure a VXLAN with the IP
parameters specified for the workload domain, and to configure the ToR ports associated with the servers.
NSX Distributed Logical Router. The Distributed Logical Router DLR) is responsible for forwarding and routing all packets
through the virtualized SDN networks. The NSX LDR can provide virtual network segmentation. Creating multiple VMware NSX
LDRs enables multi-tenancy or separate security zones. Each LDR can create virtual switches that function in the same context as
a physical VLAN. The LDR is accessed independently on both the Control Plane and Data Plane. Each VMware host has a copy of
the NSX LDR running in the hypervisor. All the gateway interfaces and IP addresses are distributed throughout the VMware
cluster. This allows VMs to directly access their default gateway at the local hypervisor.
NSX Controller. NSX Controller (NSX-C) provides the Control Plane functionality to distribute logical routing, so VXLAN network
information can reach the underlying hypervisor. Controllers are deployed as virtual appliances and should reside in the same
vCenter that connects to NSX Manager. NSX-C nodes are deployed in sets of three within a cluster and they divide the workload
equally in an active-active cluster scenario. Additionally, NSX-C removes the dependency on multicast routing in the physical
network and suppresses broadcast traffic in VXLAN networks.
NSX Edge Services Gateway. The NSX Edge Services Gateway (ESG) offers a rich set of services that include network address
translation, routing, load balancing, firewall, L2/L3VPN and DHCP/DNS relay. Individual services can be deployed, configured, and
consumed on demand. The ESG is a virtual machine, deployed via NSX Manager and accessed using the vSphere web client.
Distributed Firewall. NSX provides a complete L2–L4 stateful distributed firewall that runs in the ESXi hypervisor kernel. Because
the firewall is a function of the ESXi kernel, it provides massive throughput and performs at near line-rate speeds. The distributed
firewall service (DFW) installs in the kernel by deploying the kernel VIB in conjunction with the VMware Inter-networking Service
Insertion Platform (VSIP). VISP is responsible for monitoring and enforcing security policies on all traffic flowing through the Data
Plane and provides anti-spoofing functionality and traffic redirection for third-party appliance extensions and services. DFW
protects virtual-to-virtual or virtual-to-physical traffic. DFW policies can also restrict traffic between VMs and external networks.
Individual VMs without a firewall-protection requirement can be added to the DFW exclusion list.
NSX delivers significant advantages to the VxRack SDDC system including, improved security, on-demand service delivery, faster and
more agile deployment, and operational efficiency. NSX allows IT organizations to break free of the constraints and limitations of
hardware-based data center networking infrastructure. Customers can choose to leverage these features with the VxRack SDDC at the
onset of deployment or can gradually introduce NSX capabilities as use case requires.
vSAN
vSAN is a Software Defined Storage (SDS) technology that includes deep integration with vSphere and the VMware ecosystem to
make it a compelling, effective storage solution, well suited for VxRack SDDC. vSAN decouples software from the underlying hardware
and it implements a notably efficient architecture, built directly into ESXi hypervisor. This distinguishes vSAN from solutions that
typically install a virtual storage appliance (VSA) that runs as a guest VM on each host. Embedding vSAN into the ESXi-kernel layer
Dell EMC VxRack System SDDC
11
has clear advantages in performance and memory requirements. It has very little impact on CPU utilization (less than 10 percent) and
self-balances based on workload and resource availability.
vSAN aggregates locally attached disks of VxRack SDDC nodes to create a pool of distributed shared storage. This enables on-
demand provisioning and consumption according to policy. VxRack SDDC supports multiple vSphere clusters with up to 64 nodes per
cluster and up to a total of 192 nodes in a single VxRack SDDC system. Storage characteristics are configured using Storage Policy
Based Management (SPBM), which allows VM object-level policies to be set and modified on the fly to control storage provisioning and
day-to-day management of storage service-level agreements (SLAs).
vSAN DATASTORE
vSAN technology creates a virtual SAN from the local datastore on clustered ESXi instances. The solution provides a single vSAN
datastore spanning all the hosts within each vSphere cluster. In the VxRack SDDC, it leverages the local disk storage on the Dell
PowerEdge servers, which function as ESXi cluster nodes. The resulting virtual SAN becomes shared storage for the ESXi hosts
contributing storage.
vSAN is preconfigured for the management domain when VxRack SDDC is initialized and managed through vCenter. After the
initialization of the management domain, the additional vSAN datastore for user workloads are dynamically provisioned by SDDC
manager from the selected nodes to create a vSphere cluster for that workload domain at the time of workload domain creation. The
VxRack SDDC system total storage capacity depends on the storage disk configuration of the VxRack SDDC server nodes. Each node
contains ten storage devices in two separate disk groups configured for either hybrid storage, with flash SSD drives for the cache tier
and HDD drives for the capacity tier, or for all-flash storage, with flash SSDs for both the cache and capacity tiers.
vSAN STORAGE POLICY-BASED MANAGEMENT
vSAN is entirely policy driven and designed to simplify storage provisioning and management. It automatically and dynamically matches
requirements with underlying storage resources based on VM-level storage policies. vSAN policies define VM storage requirements for
performance and availability. They determine how storage objects are provisioned and allocated within the datastore to guarantee the
required level of service. vSAN policies also define storage-specific attributes for VM objects, including availability and replication
settings.
vSAN FEATURES
Today’s data center management teams have high expectations of their storage platforms in terms of scalability, availability,
performance, and efficiency. vSAN delivers the same advanced technology found in the most robust storage systems.
Scalability. The vSAN distributed architecture suits the modular architecture of VxRack SDDC. Customers can non-disruptively
scale out by adding nodes for capacity and performance.
Availability. As a core VxRack technology, vSAN is tightly integrated with vSphere’s availability feature set, including vSphere HA,
DRS, vMotion, Fault Tolerance, and snapshot technology. vSAN enforces availability specifically at the storage level with efficient
data-replication functionality and Fault Domain technology, which protects the environment from rack-level failures.
Performance. vSAN’s tight kernel-level integration with vSphere gives vSAN a notable advantage in performance as it reduces
CPU and memory overhead. Furthermore, as part of the VMware SDDC software stack, vSAN dynamically self-tunes, adjusting to
ongoing changes in workload conditions to load balance storage resources, ensuring each VM adheres to its defined storage
policies. In addition, vSAN’s flash-optimized design minimizes storage latency by accelerating read and write IO with built-in
caching. The caching tier for each disk group functions as a read/write buffer for hybrid configurations and a write buffer for all-flash
configurations.
Efficiency. Storage-capacity requirements continue to grow exponentially and vSAN uses data deduplication and compression to
increase capacity utilization at a lower cost for all-flash configurations. Meanwhile, vSAN deduplication and compression have only
a minimal impact on CPU overhead and memory. Typical virtual-server workloads with highly redundant data such as full-clone
virtual desktops or homogenous-server operating systems benefit most from data deduplication and compression. vSAN’s kernel-
level integration with vSphere not only enhances performance, but it places the storage close to the application and removes the
complexity of storage management and provisioning. vSAN sits directly in the IO data path, delivering the highest levels of
performance without taxing CPU or memory resources. It’s the optimal SDS backbone for the VxRack SDDC.
Dell EMC VxRack System SDDC
12
SDDC MANAGER
SDDC Manager is a new, innovative system management solution for VxRack SDDC designed to deliver a radically simplified user
experience. SDDC Manager automates critical operations across physical and virtual infrastructure such as system initial build-up,
configuration of servers and switches, auto-discovery of new physical capacity, resource provisioning, and lifecycle management of
hardware and software components. SDDC Manager serves as the primary interface for an operator’s day-to-day tasks and provides
an integrated view of both the physical and virtual infrastructure. It complements well-known VMware management tools such as
vCenter Server and vRealize Operations that continue to be available for advanced administration tasks and integration with third-party
software tools.
SIMPLIFIED RESOURCE MANAGEMENT WITH WORKLOAD DOMAINS
The value of an SDDC platform is simple resource management. In VxRack SDDC, physical compute, storage and network
infrastructure becomes part of a single shared pool of virtual resources that is managed as one system using the SDDC Manager. From
this shared pool, customers can carve out separate pools of capacity called workload domains, each with its own set of specified CPU,
memory, network, and storage requirements to support various workloads types. Two types of customer workload domains are
supported currently, Virtual Desktop Infrastructure (VDI) and Virtual Infrastructure (VI). As new physical capacity is added to the
VxRack SDDC, added resources are automatically recognized by SDDC Manager and made available for consumption. The entire
system is managed as one thereby removing any physical constraints of a single physical server or rack.
SDDC Manager abstracts and aggregates physical resources into these customer defined logical workload domains which are actually
physical vSphere clusters provisioned from available nodes in the VxRack SDDC system. SDDC Manager then automates the
configuration of each physical vSphere cluster, creating a dedicated vCenter server, the vSAN datastore, and virtual network based on
the underlying configuration parameters specified by the administrator. Workload domains are a policy-driven approach for defining
performance, availability, and security parameters. SDDC Manager automatically implements a deployment workflow to translate the
workload domain specifications into the underlying pool of resources. Through the automation of tasks and workflows, that SDDC
Manager simplifies the provisioning, monitoring, and ongoing management of both the logical and physical resources of the VxRack
SDDC.
In the VxRack SDDC, system management utilities are designed to function via a management domain. The management domain is a
special purpose workload domain which runs Cloud Foundation infrastructure components including vCenter Servers, PSCs, NSX
Managers, SDDC Manager components, LCM, vR Ops, vR LI. The management domain normally consumes the first four nodes in rack
one. See Figure 6.
Figure 6: SDDC Manager abstracts and aggregates physical resources into workload domains.
Dell EMC VxRack System SDDC
13
The management cluster runs two virtual machine appliances that make up the SDDC Manager management services as shown in
Figure 7: The SDDC Manager Controller VM and the SDDC Manager Utility VM. These VM’s run the “system” level management
services. The SDDC Manager Controller VM runs the SDDC Manager user interface (UI), Hardware Management, Physical and Virtual
Resource Manager services, datastore and coordination services, and lifecycle management services.
The Hardware Management Service (HMS) runs as a service inside the SDDC Manager Controller VM instead of on the physical
management switch within the system. This service is responsible for being the translation layer to the physical hardware infrastructure
enabling SDDC Manager to perform operations such as configuring the VLANS on physical switch ports for example.
The datastore and coordination services track system inventory, coordinate activities, and provide a persistent datatore for the system
metadata and configuration information.
The SDDC Manager Utility VM is used for lifecycle management operations and is responsible for storing patches and upgrade bundles
as well as applying these updates to the workload domains. As new updates become available admins are notified to schedule a time
to download the updates and apply them. With Cloud Foundation, individual workload domains can be updated independently. The
entire patch/upgrade process for Cloud Foundation components is fully automated using SDDC Manager.
Other management components that run in the management domain include a dedicated vCenter server and a pair of Platform
Services Controllers (PSCs). Storage is provided by vSAN and NSX Manager and NSX Controllers are also deployed. A clustered
instance of vRealize Log Insight will also run on the management domain as well as other optional software such as vRealize
Operations.
In VxRack SDDC, a Cloud Foundation Standard Architecture is used. This means that the management domain cluster is physically
separated as its own vSphere cluster to run only system management workloads. By default, the first 4 nodes in Rack 1 of the system
will be reserved for the management domain cluster. User workloads would be then be deployed using physically separate VI/VDI
workload domain clusters.
While the deployment and configuration of the management cluster is fully automated, once running, management operations for the
workload domain clusters are just like any vSphere cluster using the vSphere Web Client.
Figure 7: Management Domain
Dell EMC VxRack System SDDC
14
All workload domains (VI/VDI and management) also include a dedicated NSX Manager and cluster of 3 NSX Controller VM’s. While
the NSX Managers for all workload domains get deployed and run on the management domain, the NSX Controllers for a give workload
domain will run on the workload domain cluster itself.
Specifically, for VDI workload domains, the desktop management software such as Horizon and AppVolumes will also be deployed on
the management domain cluster leaving the workload domain cluster to run the NSX controllers, other NSX components and desktop
VM’s themselves. Figure 7 illustrates the components that run on the management domain cluster.
LIFECYCLE MANAGEMENT
SDDC Manager also streamlines and automates lifecycle management for VxRack SDDC components as shown below in Figure 8.
Given the complexities of validating new firmware and patches in an interconnected, hyper-converged infrastructure, conventional
methods of performing upgrades and patches are unfortunately prone to configuration and implementation errors. SDDC Manager
automates upgrades and patch management for the logical infrastructure. Lifecycle management can be applied to the entire
infrastructure or to individual workload domains. vMotion and DRS allow administrators to perform upgrades while the system and the
VMs remain up and running. VxRack SDDC comes with 24/7 world-class support and includes physical and logical lifecycle
management for the entire system, both the hardware and software, through a single Dell EMC phone number.
Figure 8: SDDC Manager Lifecycle Management for VxRack SDDC
vREALIZE SUITE—VMWARE’S CLOUD MANAGEMENT PLATFORM (CMP)
Most software-defined data centers will consist of a workloads mix of traditional and modern application architectures. They will be
provisioned in an increasingly virtualized mix of physical and virtual environments managed both on-premises in private clouds and in
off-premises in public clouds. The concept of a cloud management platform has evolved as a response to this complex set of
management requirements. Available as a separate purchase, but fully supported to work with a VxRack SDDC environment VMware’s
vRealize cloud management platform delivers the management capabilities to effectively manage the complete lifecycle of services
delivered in a private cloud environment.
VMware’s vRealize cloud management platform includes:
vRealize Operations which provides intelligent health, performance, capacity, and configuration management. vRealize Operations
offers performance and health monitoring and capacity planning for the VxRack SDDC as well as custom dashboards, capacity
modeling, and customized alerting. These insights help administrators maintain compliance and efficiently detect and resolve any
operational issues that may arise.
Dell EMC VxRack System SDDC
15
vRealize Log Insight which provides real time log management and log analysis. vRealize Log Insight lets administrators monitor
physical and virtual infrastructure to avoid failures and performance issues. vRealize Log Insight provides centralized log aggregation
and analysis with search and filter capabilities for the entire VxRack SDDC system. This provides the ability to monitor all workload
domains from a single place. Each VxRack SDDC system contains a management domain cluster with an instance of the vRealize Log
Insight virtual appliance for management domain components. This cluster covers logging for the management domain only. When
deploying VxRack SDDC, vRealize Log Insight is deployed by default and is included as part of Cloud Foundation for use with
management domain only. Optional licensing can be purchased enabling Log Insight to be configured to receive and process log
events for all workload domains in the system
vRealize Automation which automates delivery of personalized infrastructure, applications and custom IT services.
vRealize Business for Cloud which automates costing, usage metering, and service pricing of virtualized infrastructure.
The vRealize Suite can be purchased with VxRack SDDC. As previously mentioned, Log Insight is deployed by default at time of
system installation. The rest of the vRealize Suite components are optional add ons. These are not deployed by SDDC Manager today,
but are planned for a future release of Cloud Foundation.
HORIZON SUITE—VMWARE’S VDI AND APP VIRTUALIZATION PLATFORM
Most IT environments offer end-user computing as a service. VMware Horizon Suite, the platform for workforce mobility, connects end
users to their data and applications on any device without sacrificing IT security and control. IT can transform technology silos of
desktops, data and applications into centralized IT services—and improve operational efficiency, security and agility through policy-
based management of those services. Horizon Suite is the market-leading desktop-virtualization, end user computing solution and
includes Horizon View and App Volumes. Horizon can be easily integrated and supported on the VxRack SDDC.
VMware Horizon View (Horizon) is VMware’s VDI and desktop-management environment. Horizon provisions user desktops using a
flexible and secure delivery model. The desktop environments are accessed by the user from almost any device, including mobile
devices, with the security and resiliency of the datacenter. Because the application software and data components reside in the
datacenter, traditional security, backup, and disaster recovery approaches may be applied. If a user's device is lost or the hardware
fails, the recovery is straight forward. The user simply restores the environment by logging in using another device. With no data saved
on the user's device, if the device is lost or stolen, there is much less chance that critical data could be retrieved and compromised.
Figure 9 below shows how Horizon View encapsulates the OS, applications, profiles, and user data into isolated layers and dynamically
assembles desktops on demand to provide users with a personalized view of their individual environments.
Figure 9: Highly Available and Secure Desktops
Dell EMC VxRack System SDDC
16
Availability and security, along with ease of management and support, are compelling reasons for moving from traditional physical
desktops and laptops to VDI.
VMware App Volumes provides application and user management and monitoring with enterprise-scale capabilities across virtual
desktop and published/remote application environments powered by VMware Horizon. VMware App Volumes supports real-time
application delivery to virtualized desktop environments. With Horizon 7 and App Volumes, IT can build a real-time application delivery
system that ensures all applications are centrally managed. Applications are delivered to virtual desktops through VMDK virtual disks
without modifying the VM or applications themselves and can be scaled out to virtual desktops with superior performance, at lower
costs and without compromising end-user experience. These are deployed as part of a VDI workload domain on the VxRack SDDC.
HARDWARE CONFIGURATION
VxRack SDDC implements an integrated, scalable hardware architecture (Figure 10), ready to deploy as a complete SDDC
infrastructure. Minimum configurations begin with eight nodes and scale to 192 nodes in eight racks.
Figure 10: VxRack SDDC Integrated and Scalable Architecture
The VxRack SDDC system includes industry-leading Cisco Nexus IP 9000 series network switches for ToR and rack interconnect
switches and Dell PowerEdge servers that run the VMware SDDC stack: ESXi hypervisor, NSX software-defined networking (SDN),
and vSAN software-defined storage (SDS).
A VxRack SDDC configuration contains a set of compute and storage components as well as fixed network resources. The minimum
single-rack configuration has eight server nodes, four of which are reserved for the management domain cluster, one Dell Networking
S3048-ON switch for management and a pair of Cisco Nexus 93180YC-EX switches. Storage is part of the hyper-converged
infrastructure and based solely on the associated nodes. Storage for the default four node management cluster is provided by vSAN
using a highly available configuration. The remaining four nodes that make up the eight node minimum in a VxRack SDDC system can
be leveraged to expand the management domain cluster, to more than the four node default or can be used as a separate workload
domain cluster entirely.
Multiple-rack configurations add a pair of Nexus 9332PQ switches in the second rack as a rack interconnect layer backbone. Two
switches maximum are needed for each system and are located in the second rack. This provides inter-rack connectivity at multiple
40GB links. The server or node connectivity to the ToR switching is 10GB via the Nexus 93180YC-EX, and these switches
Dell EMC VxRack System SDDC
17
accommodate customer uplinks (10Gbps or 40Gbps). Customers can dynamically scale VxRack SDDC as demands increase. To
simplify future expansion, best practice is to size the initial configuration with the maximum number of ports and include cross-rack
networking.
HARDWARE CONFIGURATION
The VxRack SDDC includes the following hardware components:
Panduit cabinet (or rack) with an IPI appliance and two PDUs
Support for a maximum configuration of 192 Dell PowerEdge R630 servers (or 8 fully populated racks)
o Minimum of eight server nodes per rack
o Maximum of 24 server nodes per rack
Two Cisco 93180YC-EX Top of the Rack (ToR) switches per rack
Two Cisco 9332PQ rack interconnect switches included in multi-rack configurations only
One Dell Networking S3048-ON switch per rack for management
HARDWARE COMPONENTS
VxRack SDDC takes advantage of the most reliable enterprise-quality server hardware from Dell and the industry’s top network
switches from Cisco. VxRack SDDC uses Panduit intelligent cabinets for rack hardware, and system components are pre-installed and
configured within the rack.
CABINETS
In each VxRack SDDC, the compute and network-layer components are pre-installed and distributed within Panduit cabinets as shown
in Figure 11. Distributing the components balances out the power draw and reduces the size of the required PDUs. This distributed
design improves flexibility during upgrades or expansion. In addition, each Panduit cabinet has an Intelligent Physical Infrastructure
(IPI) appliance which allows users to collect and monitor environmental data, power, and security.
Figure 11: VxRack SDDC uses Panduit cabinets equipped with an IPI appliance.
Dell EMC VxRack System SDDC
18
SERVER NODES
VxRack SDDC nodes are high-density, two-socket Dell PowerEdge R630 servers (picture in Figure 13) which utilize the latest Intel
based CPUs. These single unit (1RU), rack-mount servers come in a choice of configurations with either hybrid or all-flash storage.
Currently, hybrid and all-flash storage cannot be mixed in the same VxRack SDDC.
Front View
Rear View Figure 12: Dell PowerEdge 630 VxRack SDDC Node
SERVER NODE STORAGE OPTIONS
Local disks within the Dell PowerEdge R630 Server provide storage in the VxRack SDDC. Each server has direct attached storage
(DAS) that becomes aggregated by VMware vSAN to create a single datastore shared among all the VMs running on the VxRack
SDDC Workload Domain clusters. Each node contains ten disks in two separate vSAN disk groups configured either as a hybrid
storage with two Solid State Drives (SSDs) for the cache tier and eight Hard Disk Drives (HDD) for capacity tier or as all-flash storage
with two SSDs for the cache tier and eight SSDs for the capacity tier.
VXRACK SDDC ENCLOSURES SPECIFICS (ALL FLASH)
BASE PLATFORM PowerEdge R630
FORM FACTOR 1U1N
CPU 5th generation Intel™ Xeon™ E5-2600 Family
CORES PER CPU 6 - 22
MEMORY GB 768 GB – 1536 GB
TOTAL RAW CAPACITY (TB) 15 TB – 30 TB
CAPACITY STORAGE DEVICE 1.92 TB SSD 1WPD 2.5" or 3.84 TB SSD 1WPD 2.5”
NETWORK Dual 10 GbE SFP+
NIC Intel Ethernet X520 DP 10Gb DA/SFP/+ I350 DP
STORAGE CONTROLLER PERC H730 mini Integrated RAID Controller
CACHE SSD Dual 800 GB 10WPD 2.5"
BOOT DEVICE 64 GB SLC SATADOM
TPM Optional: TPM 1.2 pre-installed or no TPM
POWER SUPPLY Dual 750W PSU 100-240VAC
Dell EMC VxRack System SDDC
19
Figure 14 lists current VxRack SDDC node configurations with Dell PowerEdge R630 as of the writing of this paper. As Dell EMC
continues to qualify and build new Dell EMC PowerEdge systems these choices will grow. Check with your Dell EMC account team for
any new VxRack SDDC node configurations.
VXRACK SDDC ENCLOSURES SPECIFICS (HYBRID)
Base Platform PowerEdge R630
Form Factor 1U1N
CPU Dual E5-2680v4 2.4 Ghz
Cores per CPU 14
Memory GB 384 - 768
Total Raw Capacity (TB) 10 TB
Capacity Storage device 1.2 TB HDD 10K 2.5"
Network Dual 10 GbE SFP+
NIC Intel Ethernet X520 DP 10Gb DA/SFP/+ I350 DP
Storage Controller PERC H730 mini Integrated RAID Controller
Cache SSD Dual 800 GB 10WPD 2.5"
Boot Device 64 GB SLC SATADOM
TPM TPM 1.2 pre-installed or no TPM
Power Supply Dual 750W PSU 100-240VAC
Figure 13 VxRack SDDC Node Configuration Table
NETWORK SWITCHES
Physical networking for a VxRack SDDC consists of a rack interconnect topology with top-of-rack (ToR) switches and a single
management switch. Each physical rack contains two physical network switches that control network traffic, provide redundancy, and
manage out-of-band connectivity. With scale-out across multiple racks, east-west traffic is fully self-contained. Two interconnect rack
switches provide connectivity between racks. VxRack SDDC comes with Cisco Nexus 93180YC-EX switches installed in pairs at the
top of the rack.
Dell EMC VxRack System SDDC
20
Expansion racks require a pair of switches in the second rack (and only the second rack) to provide connectivity for inter-rack east-west
traffic. Both the Nexus 93180YC-EX top-of-rack switches and the Nexus 9332PQ rack interconnect switches have two links to the Dell
S3048-ON management switch to communicate with out-of-band components. Both the Nexus 9372PX-E switches and the Nexus
9332PQ switches have two links to the Dell S3048-ON management switch to communicate with out-of-band components. The table in
Figure 14 lists each VxRack SDDC network switch and their functions.
Switch Switch Type Function Ports
Cisco Nexus 93180YC-EX Top of Rack / ToR Switch Deployed in pairs in each rack
Northbound traffic to customer network is through these ToR switches in the 1st rack only
Customer data uplink
48x 1/10/25GbE + 6x
40Gb/100GbE
One 40Gbps QSFP per
93180YC-EX switch to
connect to each of the two
rack interconnect switches
in multi-rack configuration
Four 10GbE links between
each pair of 93180YC-EX
2x 10GbE links between
server and each 93180YC-
EX
For 93180YC-EX in the 1st
rack only - up to four
10GbE or 40GbE links to
customer data center
switches
Cisco Nexus 9332PQ Rack Interconnect Switch
(multi-rack configurations)
Inter-rack communications
Deployed in a pair in 2nd rack only
Only 2 needed per VxRack SDDC system
32x 40GbE
1x 40Gbps QSFP to
connect to each of the
9372PX-E ToR switches
2x 40Gbps QSFP to
connect to the other
9332PQ
Dell Switch S3048-ON Management Switch Single switch deployed on each rack
48x 1GbE + 4x 10GbE
1x 1GbE link to each
server nodes, 9372PX-E
and 9332PQ (if exists) in
the same rack
1x 10GbE link to each
9372PX-E in the same
rack
Figure 14: VxRack SDDC Switches
The ToR switches in the first rack connect to the data center aggregation switch using up to 4x10 or 40Gbps links per 93180YC-EX.
Dell EMC VxRack System SDDC
21
NETWORK TOPOLOGY
Each rack configuration uses a pair of Cisco Nexus 93180YC-EX ToR switches and a single Dell Switch 3048-ON management switch.
VxRack SDDC nodes connect to the ToR switches through 10G SFP+ (one per switch) to communicate with the in-band components of
the infrastructure. The 93180YC-EX switches have two links to the management switch to communicate with the out-of-band
components. Expansion rack(s) requires a single pair of Cisco Nexus 9332PQ switches for the rack interconnect network. These reside
only in rack #2 and allow connectivity between all racks. Each expansion rack also has two Cisco Nexus 93180YC-EX ToR switches
that connect to the Nexus 9332PQ rack interconnect switches in rack #2.
The server nodes connect to the Ethernet component of the network layer. The southbound interconnects link to the Cisco Nexus
switches in the Ethernet network through 10GbE port channels. All racks are part of the same VxRack SDDC management domain, up
to the supported maximum scale-out configuration of 192 nodes, or eight racks.
The uplinks used to connect to a customer’s datacenter network in rack 1 can be configured using L2 connectivity. This is the preferred
and default method used for VxRack SDDC systems. However, if required, connecting the uplinks using L3 is also available.
NETWORK TOPOLOGY FOR SINGLE-RACK CONFIGURATION
The diagram in Figure 15 illustrates the single-rack configuration network design.
Figure 15: Single Rack Network Design
ToR Pairs Cisco ToR Switch
93180YC-EX
Cisco ToR Switch
93180YC-EX
Dell EMC VxRack System SDDC
22
NETWORK TOPOLOGY FOR MULTI-RACK CONFIGURATION
The diagram in Figure 17 illustrates the multi-rack configuration network design.
Figure 16: Multi-Rack Network Design
TRADITIONAL AND RACK INTERCONNECT NETWORK ARCHITECTURE
In a conventional data center network hierarchy, hosts at the access layer connect to network switches which, in turn, rely on routing
services (data center services as well services outside the data center, including the Internet) from a third tier. This traditional core-
aggregate-access (three-tier) network model is efficient for traffic that travels “north-south,” which is traffic that travels in and out of the
data center. North-south traffic typically has a lot of remote client/server communication. This traditional network architecture is usually
built for redundancy and resiliency and is still very widely used for service-oriented types of traffic that travel north-south. However, the
trends in traffic patterns are changing as the workloads in today’s data centers are changing. Traffic in the new SDDC data center
typically is east-west traffic, or server-to-server traffic. This traditional network model becomes prone to bottlenecks for highly scalable
SDDC infrastructures.
Figure 17 shows the traditional network data center topology with a three-layer architecture: the access layer, where users connect to
the network; the aggregation layer, where access switches intersect; and the core, where aggregation switches interconnect to each
other and to networks outside of the data center. In this traditional network architecture, if a server connected to the left-most access
switch needs to communicates with a server connected to the right-most access switch, this “east-west” communication must travel all
the way to the core switch and back down again. Clearly this is not the most efficient path and will cause more latency while consuming
more bandwidth.
Rack #2
9332PQ Rack #2
9332PQ
Rack Interconnect Switches in Rack 2
Cisco ToR Switch
93180YC-EX
Cisco ToR Switch
93180YC-EX
Cisco ToR Switch
93180YC-EX
Cisco ToR Switch
93180YC-EX
Dell EMC VxRack System SDDC
23
Figure 17: Traditional Network Architecture
In modern SDDC data centers, compute and storage infrastructure alterations change the predominant network traffic patterns from
“north-south” to “east-west”. To address network bottleneck because of the shift from north-south to east-west traffic, one solution is to
create a rack interconnect architecture. This architecture has two main components: rack interconnect switches and top of rack (ToR)
switches as shown in Figure 18.
Figure 18: Top of Rack and Rack Interconnect Network Architecture
Rack interconnect switches are composed of high-throughput Layer 3 switches with high port density. ToR switches are like the access
layer; they provide network connection points for servers, as well as uplink to the rack interconnect switches. The most important part of
Rack Interconnect Switch Pair
ToR Switches
Dell EMC VxRack System SDDC
24
this architecture is that every ToR switch connects to both rack interconnect switches in rack two. This point is important because no
matter which ToR switch a server is connected to, it always has to cross the same amount of devices to get to another server (unless
the other server is located on the same rack). This keeps the latency down to a predictable level and provides much greater overall
bandwidth.
VxRack SDDC uses a unique network design. In Cloud Foundation all the physical racks are on the same L2 network. The ToR
switches are dual connected to the two rack interconnect switches through redundant 40Gbps links configured in a Multi-Chassis Link
Aggregation Group (MC-LAG). Spanning Tree Protocol is not used because looping is avoided using MC-LAG.
As an option, Layer 3 (northbound traffic) is only supported with the Cisco Nexus 93180YC-EX switches in the first rack. SDDC
Manager configures a Switched VLAN Interface (SVI) for each requested VLAN and configures a static route between the two
93180YC-EX ToR switches in the first rack and the upstream router. Full redundancy is accomplished by setting up iBGP between ToR
switches and an eBGP between each of the two 93180YC-EX ToR switches in the first rack and the upstream router.
VxRack SDDC with VMware Cloud Foundation integrates with an existing data center network infrastructure and provides full uplink
compatibility to existing switches such as Cisco, Juniper, and Brocade. L3 or L2 uplink connectivity is supported, however, L2 is
preferred. The VMware SDDC Manager provides support for configuring, controlling, and managing the physical network elements, and
software defined networking is delivered through VMware NSX.
In Figure 19 VxRack SDDC with VMware Cloud Foundation physical switches features are shown. Each rack contains two ToR
switches and all hosts (Server 1…24) in the physical rack are dual connected to these two ToR switches with 10Gb links. Each NIC on
all hosts is connected to one of the two ToR switches in a Multi-Link Aggregation (MLAG) bond configuration. MLAG is a type of link
aggregation group (LAG) with constituent ports that terminate on separate chassis, primarily for the purpose of providing redundancy in
the event one of the chassis fails. They operate as if they are connected to a single, logical switch. The BMC port on each host is
connected to the management switch over a 1G connection used for out-of-band (OOB) management. The ToR switches are
connected to each other over 2 links for control traffic and redundancy. The ToR switches are further connected to the rack interconnect
switches in a dual-MLAG (Multi-Link Aggregation) configuration. The data path between the hosts across multiple racks can tolerate a
failure of one link or one ToR switch or one rack interconnect switch using link aggregation between hosts and ToR switches and also
between ToR switches and rack interconnect switches. For uplink connectivity, ToR switches are connected to the existing LAN
environment using 40 or 10 GbE with MLAG/Channel over 160 GbE.
Figure 20: VxRack SDDC powered by Cloud Foundation Physical Networking
Rack Interconnect Switches
ToR ports 49, 50 for Interconnect switches
Dell EMC VxRack System SDDC
25
CONCLUSION
The modern CIO knows there are conflicting priorities for the IT organization: Quickly adopt new technology, but minimize technical
debt; be strategic and focus on business objectives, but maintain operational excellence; encourage innovation, but stay secure and
maintain compliance. The overarching challenge in the cloud era: Support business need for speed and agility, while maintaining
control and reducing IT costs.
The IT market is changing faster than ever, driven by continuous acceleration of business and modern data center initiatives. Business-
as-usual no longer works, and hyper-converged infrastructure (HCI) and cloud computing are now cornerstones of every business’
digital and IT transformation strategy. IT organizations undertaking modernization and transformation efforts are looking to use
technology to reduce costs as well as gain competitive advantage. The Dell EMC VxRack SDDC is a turnkey HCI engineered
infrastructure system with single vendor support and lifecycle management. It is the fastest and simplest way to deploy a VMware
Cloud Foundation based SDDC infrastructure. VxRack SDDC is unique in the industry and is offered exclusively by Dell EMC as a co-
engineered system with VMware. The VxRack SDDC powered by VMware Cloud Foundation delivers a rack scale hyper-converged
system that simplifies the journey to the private cloud.