+ All Categories
Home > Documents > Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf ·...

Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf ·...

Date post: 28-Jul-2020
Category:
Upload: others
View: 1 times
Download: 0 times
Share this document with a friend
86
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID Carlos Campos UCS Consulting Systems Engineer Latam CCIE#16993 Storage/R&S [email protected] Virtual Machines Fabric Extender, UCS Fabric and VM's - Extending FEX direct to VM's in UCS
Transcript
Page 1: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 1

Carlos Campos

UCS Consulting Systems Engineer – Latam

CCIE#16993 Storage/R&S

[email protected]

Virtual Machines Fabric Extender, UCS Fabric and VM's - Extending FEX direct to VM's in UCS

Page 2: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 2

Virtualization/Fabric Extender Overview

Virtual Machine Fabric Extender (VM-FEX)

Benefits of VM-FEX

Forwarding and Latency considerations with FEX Technologies (NEW)

VM-FEX UCS Generals

VM-FEX VMware on UCS

VM-FEX KVM on UCS

Security and Segmentation of with VM-FEX

Operations Model with VM-FEX (Stats and SPAN)

Summary

Agenda

What we hope to share

Page 3: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 3

Vmware/Hyper-V and virtualization

Virtualized environments are the natural transiton to 10G

Reduce cabling

Reduce total number of ports

Reduce virtual machine oversubscription

Resources underutilized

Several NICs used nowadays (VMKernel, console, data, backup) and not all of them are as used as data ones

Inconsistent responsabilities/configurations

Network configurations now also being part of servers area (vSwitch) which leads to inconsistent configurations

Uplink validation (security/QoS) might be needed before Vmotion

Monitoring is not possible for saturation, DoS attacks, etc within VMWare

No IPS, IDS information exporting

Virtual-machine flexibility

Diversity in virtual-machines is not possible due to Vmnic sharing and no QoS policing enforced

Page 4: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 4

Hyperv

isor

Hyp

erv

iso

r

Hyperv

iso

r

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

Phase I Virtualization Consolidated Environment

Page 5: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 5

Hyperv

isor

Hyp

erv

iso

r

Hyperv

iso

r

App

OS

App

OS

App

OS

vSwitch vSwitch vSwitch

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

Phase I Virtualization More virtual infrastructure needed

Page 6: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 6

Hyperv

isor

Hyp

erv

iso

r

Hyperv

iso

r

vSwitch vSwitch

App

OS

vSwitch

App

OS

Net State Net State Net State

App

OS

Phase 2 Virtualization: Mobility Network policies tied to VSwitch: Not mobile

Page 7: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 7

Hyperv

iso

r

Hyp

erv

iso

r

Hyperv

iso

r

App

OS

App

OS

vSwitch vSwitch

App

OS

vSwitch

Net State Net State Net State

Cisco VN-link: Virtualization Aware Network Network Policies tied to Virtual Machine

Page 8: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 8

Hyperv

isor

Hyp

erv

iso

r

Hyperv

isor

vSwitch vSwitch

App

OS

vSwitch

App

OS

Net State Net State Net State

App

OS

vNetwork Distributed Switch VN-Link Domain

Cisco VN-link: Virtualization Aware Network Network policies trascend server boundaries

Page 9: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 9

Before/after VMWare considerations

Before

After

Page 10: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 10

Before/after VMWare considerations

Before

After

Page 11: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 11

Before/after VMWare considerations

Before

After

Page 12: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 12

Cisco Nexus 1000V Industry First 3rd Party Virtual Distributed Switch

Nexus 1000V provides enhanced VM switching for VMW ESX environments

Features VN-Link capabilities:

Policy-based VM connectivity

Mobility of network and security properties

Non-disruptive operational model

Ensures visibility and continued connectivity during VMotion

Enabling Acceleration of Server Virtualization Benefits

VMW ESX

Server 1

VMware vSwitch Nexus 1000V

VMW ESX

VMware vSwitch Nexus 1000V

Server 2

Nexus 1000V

VM

#4

VM

#3

VM

#2

VM

#1

VM

#8

VM

#7

VM

#5

VM

#5

VM

#2

VM

#3

VM

#4

VM

#5

VM

#6

VM

#7

VM

#8

VM

#1

Page 13: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 13

Cisco Nexus 1000V Components

VMW ESX

Server 3

VM

#9

VM

#12

VM

#11

VM

#10

VEM

VMW ESX

Server 2

VM

#5

VM

#8

VM

#7

VM

#6

VEM

VMW ESX

Server 1

VM

#1

VM

#4

VM

#3

VM

#2

VEM

Virtual Ethernet Module(VEM)

Replaces existing vSwitch

Enables advanced switching capability on the hypervisor

Provides each VM with dedicated “switch ports”

Virtual Supervisor Module(VSM)

CLI interface into the Nexus 1000V

Leverages NX-OS 4.01

Controls multiple VEMs as a single network device

Virtual Center

Nexus 1000V

VSM

Page 14: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 14

Nexus 1000V ‘Virtual Chassis’ Model

One Virtual Supervisor Module managing multiple Virtual Ethernet Modules

•Dual Supervisors to support HA environments

A single Nexus 1000V can span multiple ESX Clusters

SVS-CP# show module

Mod Ports Module-Type Model Status

--- ----- -------------------------------- ------------------ ----------

--

1 1 Supervisor Module Cisco Nexus 1000V active *

2 1 Supervisor Module Cisco Nexus 1000V standby

3 48 Virtual Ethernet Module ok

4 48 Virtual Ethernet Module ok

--More--

Page 15: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 15

Single Chassis Management

Upstream-4948-1#show cdp neighbor

Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge

S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone

Device ID Local Intrfce Holdtme Capability

Platform Port ID

N1KV-Rack10 Gig 1/5 136 S Nexus

1000V Eth2/2

N1KV-Rack10 Gig 1/10 136 S Nexus

1000V Eth3/5

N1KV-Rack10 Gig 1/12 136 S Nexus

1000V Eth21/2

A single switch from control plane and management plane perspective

Protocols such as CDP operates as a single switch

XML API and SNMP management appears as a single ‘virtual chassis’

Page 16: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 16

Virtual Supervisor Options

VSM

VSM

VSM Virtual Appliance

ESX Virtual Appliance

Special dependence on CPVA server

Supports up to 64 VEMs

VMW ESX

Server 3

VM

#9

VM

#12

VM

#11

VM

#10

VEM

VMW ESX

Server 2

VM

#5

VM

#8

VM

#7

VM

#6

VEM

VMW ESX

Server 1

VM

#1

VM

#4

VM

#3

VM

#2

VEM

VSM Physical Appliance

Cisco branded x86 server

Runs multiple instances of the VSM virtual appliance

Each VSM managed independently

Page 17: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 17

Virtual Ethernet Module

VEM is a light weight (~10MB RAM) module that provides switching capability on the ESX host

Single VEM instance per ESX host

Relies on the VSM to provide configuration

Stores basic configs locally (system VLANs, Domain ID, etc…)

Can run in last known good state without VSM connectivity

Some will not work (Vmotion) in this state

Must have VSM connectivity upon reboot to switch VM traffic

Page 18: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 18

Switching Interface Types

Physical Ethernet Ports

- NIC cards on each server

- Appears as ‘Eth’ interface on a specific module in NX-OS

Example – ‘Eth10/7’

- Static assignment as long as the module ID does not change

- Up to 32 per host

Virtual Ethernet Ports

- Virtual Machine facing ports

- Appears as ‘Veth’ within NX-OS.

- Not assigned to a specific module to simplify VMotion

Example – ‘Veth68’

Page 19: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 19

Cisco Nexus 1000V Scalability @ FCS

A single Nexus 1000V

• 66 modules (2x Supervisors and 64x Ethernet Modules)

• 4x64 modules in Nexus 1010

Virtual Ethernet Module:

• 32 physical NICs

• 256 virtual NICs

Limit Per Nexus 1000V

• 512 Port Profiles

• 2048 physical ports

• 8,192 virtual ports (vmknic, vswif, vnic)

Virtual Supervisor - Standby

VEM

VEM

VEM

VEM

VEM

VEM

VEM

VEM

VEM

VEM

Virtual Supervisor - Active

Nexus 1000V

Page 20: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 20

Virtual Supervisor to Virtual Center

One way API between the VSM and Virtual Center

Certificate (Cisco self signed or customer supplied) ensures secure communications

Connection is setup on the Supervisor

N1K-CP# show svs connections

Connection VC:

IP address: 10.95.112.10

Protocol: vmware-vim https

vmware dvs datacenter-name: PHXLab

ConfigStatus: Enabled

OperStatus: Connected

Nexus 1000V

VSM

Virtual Center

Page 21: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 21

Introducing the Cisco Nexus 1010

• Allows network administrators to manage the Nexus 1000V Virtual Supervisor Module (VSM) as a standard Cisco® switch, with all Nexus 1000V features and with Virtual Services Nodes (VSN)

• Physical appliance for virtual network services (VSM, NAM, etc.).

• Supported by CiscoWorks LAN Management Solution (LMS).

• Cisco Nexus® 1010 is a networking appliance that can host four Nexus 1000V Virtual Supervisor Modules.

Page 22: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 22

Supervisor to Ethernet Module

Two distinct virtual interfaces are used to communicate between the VSM and VEM

•Control

• Carries low level messages to ensure proper configuration of the VEM.

• Maintains a 2 sec heartbeat what the VSM to the VEM (timeout 6 seconds)

•Packet

•Carries any network packets between the VEM and the VSM such as CDP/LLDP

Must be on two separate VLANs

Supports both L2 and L3 designs

VMW ESX

VM

#1

VM

#4

VM

#3

VM

#2

VEM

Nexus 1000V

VSM

Page 23: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 23

Introduction to Port Profiles

Port Profiles are a collection ‘interface’ commands

i.e.:

switchport mode access

switchport access vlan 57

no shutdown

Applied at the interface level using to either physical or virtual interfaces

Dynamic configuration

Port Profile changes are propagated immediately to all ports using that profile

Interfaces can be configured manually in conjunction with a profile

Page 24: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 24

VMW ESX

Server

Nexus 1000V - VEM

VM

#1

VM

#4

VM

#3

VM

#2

What Can A Profile Contain?

Policy definition supports:

VLAN, PVLAN settings

ACL, Port Security, ACL

Redirect

Cisco TrustSec (SGT)

NetFlow Collection

Rate Limiting

QoS Marking (COS/DSCP)

Remote Port Mirror (ERSPAN)

Nexus 1000V

VSM

Virtual Center

Page 25: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 25

Port Profiles Propagation

Port profiles are pushed via the Virtual Center API

Upon connection/reconnection with Virtual Center the VSM re-verifies the correct port profile configuration exists within Virtual Center

Port profile ‘state’ and ‘type’ must be set for propagation to occur

N1K-CP(config-port-prof) state enable

N1K-CP(config-port-prof) vmware port-group (optional name)

Page 26: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 26

Network Administrator View

Page 27: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 27

VMware Administrator View

Consistent Workflow: Continue to select Port Groups when configuring a VM in VMware Virtual Infrastructure Client

Page 28: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 28

Port Profile Mobility – Simplified VMotion

VMW ESX

Server 2

Nexus 1000 -—VEM

VMW ESX

Server 1

Nexus 1000V—VEM Nexus 1000V

VM

#5

VM

#8

VM

#7

VM

#6

VM

#1

VM

#4

VM

#3

VM

#2

VM

#1

Nexus 1000V

VSM

VM

#1

Virtual Center

Page 29: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

Fabric Extender Overview

Page 30: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 30

LAN LAN Switch port extended

over Fabric Extender

Lo

gic

al

Sw

itch

Collapse networking tiers, reduce network management points!!!

Switch

Switch

Legacy multi-tier architecture FEX architecture

Switch

FEX

Fabric Extender (FEX) Concept

Switch

FEX

Page 31: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 31

Over 6000 Nexus 5K/2K customers Over 5400 UCS Customers

LAN

Parent Switch

Parent Switch + FEX

Single Access Layer

SAN

FEX

Page 32: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 32

Over 6000 Nexus 5K/2K customers Over 5400 UCS Customers

LAN

Parent Switch + FEX

Single Access Layer

SAN

Page 33: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 33

FEX Building Block #1: VNTAG

1. VNTAG

D: Direction, P: Unicast/Multicast, L: Loop

Virtual addresses switch local

VNTAG Ether type

Destination Virtual Interface

Source Virtual Interface ver

D P

L R

Application

Payload

TCP

IP

Ethernet

VNTAG

FEX architecture

Switch

FEX

LAN

Frame

VNTAG

Frame

Layer 2 Tag To Mimic Forwarding Vector

Page 34: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 34

FEX Building Block #2: Policy

FEX architecture

Switch

FEX

LAN

Frame

VNTAG

Frame

Policy Associated with Virtual Interfaces NOT Physical

1. VNTAG

D: Direction, P: Unicast/Multicast, L: Loop

Virtual addresses switch local

2. Policy

VLAN, QoS, Rate limit

VNTAG Ether type

Destination Virtual Interface

Source Virtual Interface ver

D P

L R

Application

Payload

TCP

IP

Ethernet

VNTAG

Page 35: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

Virtual Machine Fabric Extender

Page 36: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 36

LAN

Parent Switch

Parent Switch + FEX + Virtual Switch

Physical and Virtual Infrastructure

SAN

FEX

App

OS

App

OS

App

OS

vSwitch

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

Physical Network

Virtual Network

Separate virtual and physical infrastructures!!!

Page 37: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 37

Lo

gic

al

Sw

itch

Virtualized Deployment

Switch

FEX

Hypervisor vSwitch

App

OS

App

OS

App

OS

LAN

Lo

gic

al

Sw

itch

VM-FEX architecture

Switch

FEX

Hypervisor

LAN

App

OS

App

OS

App

OS

VM-FEX

Cascaded Fabric Extenders

Lo

gic

al

Sw

itch

Extending FEX Architecture to VMs Cascaded Fabric Extenders

Page 38: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 38

LAN

Parent Switch

Parent Switch + FEX + VM-FEX

Single “Physical-Virtual” Access Layer

SAN

FEX

App

OS

App

OS

App

OS

VM-FEX

App

OS

App

OS

App

OS

Collapse virtual and physical networking tiers!!!

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

Page 39: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 39

LAN

Parent Switch + FEX + VM-FEX

Single “Physical-Virtual” Access Layer

SAN

App

OS

App

OS

App

OS

App

OS

Connect 1000s of VM to Distributed Modular Switch

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

App

OS

Page 40: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 40

FEX Building Block #3: PCIe Virtualization

FEX architecture

Switch

FEX

LAN

Frame

VNTAG

Frame

UCS Virtual Interface Card

1. VNTAG

D: Direction, P: Unicast/Multicast, L: Loop

Virtual addresses switch local

2. Policy

VLAN, QoS, Rate limit

3. PCIe Virtualization

Ability to create interfaces inside the host

Hypervisor

App

OS

App

OS

App

OS

VM-FEX

Page 41: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 41

Flexibility and Performance

UCS Virtual Interface Card 1280

Flexibility

256 PCIe devices

Devices can be vNICs or vHBAs

Each device has a corresponding switch interface

VM-FEX with VMDirectPath Capability

Performance

Dual 40Gb

vNICs/vHBAs NOT limited to 10Gb

PCIe Gen2 x 16

Page 42: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 42

Physical Network

Virtual Network

Hyp

erv

iso

r

Hyp

erv

iso

r

VM VM VM VM VM VM VM VM

VETH

VNIC

VM-FEX Operational Step: Background

Server

Page 43: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 43

VM-FEX Operational Step #1: Pre Boot Config

Hyp

erv

iso

r

Hyp

erv

iso

r

Step1: Pre boot config

Anonymous PCIe Enumerations

Host discovers PCIe devices

Server Server

VM-FEX

Switch

VETH

VNIC

Page 44: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 44

VM-FEX Operational Step #2: Define Policy

Hyp

erv

iso

r

Hyp

erv

iso

r

Step1: Pre boot config

Anonymous PCIe Enumerations

Host discovers PCIe devices

Step2: Policy Definition

Networking Policy definition

VLAN, QoS, Rate limit, etc

Server Server

VM-FEX

Switch

Port Profiles

Definition

WEB Apps

HR

DB

Compliance

Network

Manager

Page 45: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 45

VM-FEX Operational Step #3: Export Policy

Hyp

erv

iso

r

Hyp

erv

iso

r

Step1: Pre boot config

Anonymous PCIe Enumerations

Host discovers PCIe devices

Step2: Policy Definition

Networking Policy definition

VLAN, QoS, Rate limit, etc

Step3: Port Profile Export

Policy exported to Hypervisor manager

Server Server

VM-FEX

Switch

Network

Manager Hypervisor

Manager

Export Port Profiles

Page 46: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 46

VM-FEX Operational Step #4: VM Association

Hyp

erv

iso

r

Hyp

erv

iso

r

Step1: Pre boot config

Anonymous PCIe Enumerations

Host discovers PCIe devices

Step2: Policy Definition

Networking Policy definition

VLAN, QoS, Rate limit, etc

Step3: Port Profile Export

Policy exported to Hypervisor manager

Step4: VM Creation

Policy resolution

Server Server

VM-FEX

Switch

Network

Manager Hypervisor

Manager

Export Port Profiles

VM

VM VM VM

Page 47: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 47

Emulated Mode VMDirectPath

Standard Mode

12%-15% CPU performance improvement

vMotion supported

High Performance Mode

Co-exists with Standard mode

Bypasses Hypervisor layer

30% improvement in I/O performance

vMotion supported with ESX 5.0

Modes of VM-FEX

Page 48: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 48

0

2

4

6

8

10

12

0 10 20 30 40 50 60 70 B

and

wid

th (

Gb

ps)

Time (sec)

Temporary transition

from VMDP to

standard I/O

vMotion to secondary

host

• 8GB VM, sending UDP stream using pckgen (1500MTU)

• UCS B200 blades with UCS VIC card

• vSphere 5 technology preview

VMDirectPath Operations

Page 49: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 49

Physical Network Simplicity

One infrastructure

Consistent features, performance and management

Benefits of VM-FEX #1: Simplicity

Virtual Network

Hyp

erv

iso

r

Hyp

erv

iso

r

VM VM VM VM VM VM VM VM

VETH

VNIC

Page 50: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 50

Physical Network

Virtual Network

VM VM VM VM VM VM VM VM

VLANs

Simplicity

One infrastructure

Consistent features, performance and management

Robustness

Programmability

Trouble shooting

Traffic engineering

Benefits of VM-FEX #2: Robustness

Page 51: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 51

Simplicity

One infrastructure

Consistent features, performance and management

Robustness

Programmability

Trouble shooting & Traffic engineering

Performance

Near bare metal I/O performance

Benefits of VM-FEX #3: Performance

VMDirectPath

Page 52: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

Forwarding and Latency considerations with FEX Technologies

Page 53: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 53

LAN

Parent Switch

Parent Switch + FEX

De-Coupling of the Modular Switch

SAN

FEX

Page 54: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 54

Virtualizing the Switchports with Unified I/O Mapping of Ethernet and FC

Wires over Ethernet

Service Level enforcement

Multiple data types (jumbo, lossless, FC)

Individual link-states

Fewer Cables

oMultiple Ethernet traffic co-exist on same cable

Fewer adapters needed

Overall less power

Interoperates with existing Models

oManagement remains constant for system admins and LAN/SAN admins

Possible to take these links further upstream for aggregation

Individual

Ethernets

DCB Ethernet

Individual

Storage (iSCSI, NFS, FC)

Access Layer Consolidation & Virtualization

Blade Management

Channels (KVM, USB,

CDROM, Adapters)

Page 55: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 55

FEX Data Forwarding Is Switching local to Blades Always Better?

High variance in server to server bandwidth metrics

High variance in server to server latency metrics

Need to build out multiple fabrics

This led to static server designs and we needed to fix the locations and engineer the infrastructure

Very stateful in nature

Latency can be minimized – but challenged in store-and-forward

Page 56: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 56

FEX Data Forwarding Is Intra-Host Switching Always Better?

VM-VM data switching within host will consist of mapping memory between structures

Latency will be optimal – but only for VMs on same host and can vary greatly

Moving memory structures of packets between emulated ports

Store and Forward handling of full packets through virtual switch

With VM-FEX and multiple port ASICs on Fabric Interconnects we get forwarding closer to a physical or virtual server (not a centralized fabric)

With cut-through switching on our port ASICs latency is greatly reduced

Page 57: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

VM-FEX UCS Generals

Page 58: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 58 2009

IOM links to FI decide number of interfaces available for the host (# IOM links * 63) – 2 (v2.x Hardware)

(# IOM links * 15) – 2 (v1.x Hardware)

Set a “Dynamic Policy” for the server in the UCS

Policy include: Number of dynamic VIF’s

Adapter policy (interrupt coalescing timers, buffering, etc.)

vCon policy (which adapter in a multi-adapter blade to use)

Dynamic vNICs are Protected

VM-FEX Dynamic vNIC Policies with VIC

Page 59: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 59

Double-wide blades have multiple Mezzanine slots

UCS Administrator Sets the physical adapter that vNIC and vHBA’s will map to

vCon is an Adapter

vNIC/vHBA can be A, B, A-B, B-A on the given vCon (not between)

Typical option is just to allow automatic system mapping

Can define a policy to control

Service Profiles with Full width Blades

Page 60: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 60 2009

• Makes use of vCon first if separation by adapter is required

• UCS Administrator Sets the Dynamic vNIC Connection Policy

• 1 static vNIC to fabric A and 1 static vNIC to fabric B, for redundancy

• Actual VLAN, QoS, Pinning, etc. configuration done in port profiles

VM-FEX Service Profile with VIC

Page 61: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 61 2009

• UCS Administrator Creates Profiles

• Sets Trunk VLANs, Access VLAN, etc.

• Profiles available for use on multiple vCenter defined DVS

VM-FEX Port Profiles

Page 62: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 62 2009

• UCS Administrator Sets Policy per profile

• Sets CoS, Rate Limiting (VM to fabric direction only), Network Control, Uplink Connectivity, etc.

UCS Port Profiles (QoS, Control, Pinning)

Page 63: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

VM-FEX VMware on UCS

Page 64: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 64

Configuration for VMDirectPath2

Page 65: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 65 2009

ESX VM-FEX for VMware: Easy VM-FEX Tool

Eliminate Virtual Networking in minutes

Page 66: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 66 2009

• Connections for up to 8 vCenter DVS’ within a UCS

• Keying for each of the connections

• Define DVS within the vCenter provider

ESX VM-FEX for VMware: UCSM/vCenter Connection

Page 67: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 67

ESX VM-FEX: vCenter View

Page 68: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 68

ESX VM-FEX: View from VM Settings

Page 69: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

VM-FEX KVM on UCS

Page 70: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 70

VM-FEX with KVM Architecture

Uplink Port

Virtual ports

PF VF2 VFn

eth0 eth2 ethn

Macvtap 2

User

Kernel

MacVTap

Interfaces Netdev

Interface …

Libvirt

Management

Tools

Netlink Socket

Guest

Userspace

virtio-net

Guest OS

KVM

UCS Fabric

Interconnect

vhost-net

Guest

Userspace

Guest OS

virtio-net

veth2

VIC

adapter

veth1

VF1

eth1

Macvtap 1

vhost-net

Port Profile1:

QoS1, VLAN1 Port Profile2:

QoS2, VLAN2

QoS2, VLAN2 QoS1, VLAN1

Physical Port

IO Module

Page 71: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 71

Install Red Hat as Virtualization Host

Unlike VMware no VEM to load (utilizes libvirt)

Works with live migration feature for VMs

Scripted nature of configuration at FCS No current RHEV-M for RHEL KVM 6.x

RHEV-M 3.0 will have RHEL 6.2 hooks for VM-FEX configuration assistance

MacVTap has 3 distinct modes Bridge mode for normal end points connecting together within host (Adapter FEX has value here)

802.1Qbg “VEPA” for traffic hair-pinning on next upstream device

802.1Qbh “private-mode” for traffic always passing to controlling bridge (UCS FI)

VM-FEX uses private-mode

Virtual Machine interface management via editing of VM domain XML file

Trunking to guest TBD

UCS VM-FEX for KVM

RHEL 6.1 Administration

Page 72: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 72

Create profiles for system

Can Create Clusters of KVM hosts

At FCS, Single Default Cluster only as there is no mechanism of a Cluster ID

Publish to client Cluster

Client can now be under the Cluster of KVM devices

UCS VM-FEX Configuration for KVM

Port Profiles in the VM Tab

Page 73: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 73

KVM Domain XML Configuration

Page 74: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

Security and Segmentation of with VM-FEX

Page 75: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 75

Nexus 1000V

VSM

Virtual

Center

Virtual Network

Manager

(VNM)

VMW ESX

Server 2

VM

#3

VM

#4

Virtual

Service

Node #3

Service Data Path (SDP)

Virtual

Service

Node #2

VMW ESX

Server 1

VM

#2

VM

#1

Service Data Path (SDP)

Virtual

Service

Node #1

VMW ESX

Server N

VM

#5

VM

#6

Service Data Path (SDP)

Nexus 1000V VEMs

VM

#8

VM

#7

Virtual Security Gateway

VC: Server policies

VSM: Networking policies

VNM: Security policies

Page 76: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 76

Firewall VSN Policy Model

VM

#1

VM

#8

VM

#7 VM

#6

VM

#4

VM

#3

VM

#2 VM

#5

DB Zone

Tenant_A

Nexus 1000 DVS

Zone 0 APP Zone Web Zone

Firewall

VSN

Internet/Intranet

Internet Zone Branch Zone

10.10.*.* +

10.12.10.*

Zone support

Multi-tenancy support

Context based (supports VM attributes as well as network constructs)

Page 77: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 77

VM

#7

Zone 1 Zone 2

Firewall

VSN

VEM

Flow Lookup Service

Lookup Service Data Path (SDP) VEM

Flow Lookup Service

Lookup Service Data Path (SDP)

VM

#3

VM

#11

Zone 3

VM

#1

VM

#2 VM

#6

Zone 3

VM

#12

VM Mobility

VM traffic Flow continues to be handled by appropriate Firewall VSN

Traffic State is re-built from Firewall VSN

VSN Mobility

Overlay tunnel ensures firewall VSN is reachable in the new host.

Traffic state moves with firewall VSN VM

Firewall VSN and VM Mobility Support

Page 78: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 78

Security Zones and Unified Compute

3 4

1 3 4 5 6 7 8

Chassis IO Module A

1 2

Server Ports

3 4

1 3 4 5 6 7 8

1 2

Server Ports

VN Tag @ 10Gbe

2 2

Chassis IO Module B

Internal Connections 2 3 4 5 6

0 1

2 3 4 5 6 UCS 6x00 Physical Ports

Chassis IOM Ports

UCS 6x00 Physical Ports

Chassis IOM Ports

VIC CPU

Virtual Interface Control Logic

Virtual Interface Control Logic

vCenter

Controlled

interfaces on VMs

with forwarding

rules enforced on

dynamic adapters

and signaled on

Private Interfaces

1 1

8 7

Ethernet Uplink Ports

2 1

Fiber Channel Uplink Ports

8 7

Ethernet Uplink Ports

2 1

Fiber Channel Uplink Ports

6 5 6 5

0

Mgmt Uplink

0

Mgmt Uplink

CIMC KVM etc.

Cisco Adapter

UCS Fabric Interconnect B (port profiles) UCS Fabric Interconnect A (port profiles)

UCS Interaction with VMware and VM-FEX

Page 79: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

Operations Model with VM-FEX (Stats and SPAN)

Page 80: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 80

VM-FEX UCSM Visibility VM Visibility on VM Tab

Page 81: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 81

VM-FEX UCSM Visibility Direct VM SPAN Sessions

Page 82: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 82

www.YouTube.com/ciscodatacenter

Playlist UCS Technical

Videos

http://www.youtube.com/ciscodatacenter#p/c/

F04A2C6AA04DF055

Overview Cisco UCS

Advantage

http://www.youtube.com/watch?v=IW4zHXIjpPU

UCS Advantage Videos on YouTube

Page 83: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 83

Category Title URL

UCS server Service Profiles and Templates http://www.youtube.com/watch?v=JW-YtVN75R0

UCS server Organizations and Roles http://www.youtube.com/watch?v=tb-L0zv3If

UCS server Extended Memory Technology http://www.youtube.com/watch?v=kS3ehPRcVDo

UCS server Server Pre-Provisioning http://www.youtube.com/watch?v=o7BuEE3hNPE

UCS server BIOS Policies http://www.youtube.com/watch?v=Pr6EptC9JXQ

UCS server RAID Policies http://www.youtube.com/watch?v=Vcs56wjUWuI

UCS server Firmware Policies http://www.youtube.com/watch?v=vjj8Xz0NqI4

UCS server Server Pools and Qualification Policies http://www.youtube.com/watch?v=KTw7M3T-VOw

UCS server Maintenance Policies http://www.youtube.com/watch?v=QQTlm98NgTI

UCS server High Availability During Upgrades http://www.youtube.com/watch?v=57HXMGn88HA

UCS server Monitoring with BMC BPPM http://www.youtube.com/watch?v=mdoEZf7tM5E

UCS server Microsoft Hyper-V on UCS http://www.youtube.com/watch?v=G3x_YOYK-Fo

Page 84: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 84

Category Title URL

UCS I/O Adapter Templates http://www.youtube.com/watch?v=KpVEn3DhfOM

UCS I/O Network Interface Virtualization http://www.youtube.com/watch?v=njjbCEblxVc

UCS I/O Adapter Fabric Failover http://www.youtube.com/watch?v=tlu8RSq6T_M

UCS I/O Extend the Network to the Virtual Machine http://www.youtube.com/watch?v=Ylizxq18yxE

UCS I/O Traffic Analysis of All Servers http://www.youtube.com/watch?v=PHTdXy_8Zdg

UCS I/O Ethernet Switching Modes http://www.youtube.com/watch?v=roX8MRN66UM

UCS I/O Fibre Channel and Switch Modes http://www.youtube.com/watch?v=VSetsgOYYCo

UCS I/O FC Port Channels and Trunking http://www.youtube.com/watch?v=PpzKPguRTXc

Page 85: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 85

Category Title URL

UCS

Infrastructure

Lights-Out Management http://www.youtube.com/watch?v=QEO1d_1vTxs

UCS

Infrastructure

Easy VM-FEX Deployment http://www.youtube.com/watch?v=0aAuj80cNvg

UCS

Infrastructure

Server Power Grouping http://www.youtube.com/watch?v=EgoFe33YoD8

UCS

Infrastructure

Blade and Rack-Mount Management http://www.youtube.com/watch?v=aOsx4YMiOho

UCS

Infrastructure

Manager Platform Emulator http://www.youtube.com/watch?v=ZNNrs2e0wvk

UCS

Infrastructure

Cisco Developer Network and Sandbox http://www.youtube.com/watch?v=Syhl6SAiwew

Page 86: Virtual Machines Fabric Extender,d2zmdbbm9feqrf.cloudfront.net/2011/mex/pdf/BRKCOM-2005.pdf · CCIE#16993 Storage/R&S ccampost@cisco.com Virtual Machines Fabric Extender, ... Certificate

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 86

Thank you.


Recommended