Nexus 5000 Deep Dive - Cisco · Nexus 5020. 56-Port L2 Switch • 40 Ports 10GE/FCoE, fixed • 2...

transcript

Nexus 5000 Deep DiveNexus 5000 Deep Dive

Agenda

System Hardware Overview

Internal Architecture

Fabric Data Path

Lossless data path

Forwarding and Policy Enforcement

NX-OS Overview

Select Switch Features

Network design

System Hardware Overview

Nuova Systems Product PortfolioIndustry’s First I/O Consolidation Virtualization Fabric for

Enterprise Data Center Industry’s First I/O Consolidation Virtualization Fabric for

Enterprise Data Center

Nexus 502056-Port L2 Switch• 40 Ports 10GE/FCoE, fixed• 2 Expansion Modules

Cisco Fabric Manager and Cisco Data Center Network Manager

Cisco NX-OS

FC + Ethernet • 4 Ports 10GbE/FCoE • 4 Ports 1/2/4G FC

Expansion Modules

Ethernet • 6 Ports

10GE/FCoE

Eco System PartnersEco System Partners

Rear PanelsNX5020NX5020

Expansion Module(s)

Cables connect in the rear for ease of server wiringCables connect in the rear for ease of server wiring

Power Entry

Base 10GE 10/100/1000

Out of Band Mgmt Console

All 10GE ports are FCoE capable!

Support for 1 GE Support for crypto

Front PanelsNX5020NX5020

N+1 redundant fans

Replaceable components on the front for easy accessReplaceable components on the front for easy access

Dual redundant power supplies

NX5020Power Supply

Fully redundant, load sharing and hot swappableFully redundant, load sharing and hot swappable

Maximum Power – 750WTypical Operating Power – 480WAC Input - 208 VoltsEfficiency - 82-88%Protection - 110%-150% max load

Expansion Modules

Ethernet Expansion Module

CombinationExpansion Module

Six 10G Ethernet

Four 10G Ethernet

Four 1/2/4GFibre Channel

Cooling

NX5020Cooling Module

Max RPM - 12KFailover - N+1Op Temp - 0 to 40 CHumidity - 95% non-condensingElevation - 10K feet

SFP+ Transmission Media

CableTransceiver

Latency (link)Power

(each side)DistanceTechnology

Twinax ~0.25 μs~0.1W10mSFP+ CU Copper

MM OM2 MM OM3 ~0.1 μs1W82m

300mSFP+ SR short reach

MM OM2 MM OM3 ~0.1 μs1W10m

100mSFP+ USR

ultra short reach

Cat6 Cat6a/7 Cat6a/7

2.5μs 2.5μs 1.5μs

~8W ~8W ~4W

55m 100m 30m

10GBASE-T

•Low power consumption•Low cable cost•Low transceivers latency•Low error rate (10 exp-17)

Internal Architecture

Hardware Architecture

Supervisor Details

CPU 1.66 GHz Intel LV Xenon - LF80538KF0281M

IO Chip Set Intel 3100 South Bridge for embedded applications

DRAM 2 GBytes of DDR2 400 (PC2 3200) in two DIMM slots

Program Store 1 GBytes of USB based (NAND) Flash

Boot/BIOS 2 Mbytes of EEPROM with locked recovery image

On-board Fault Log 64 MBytes of Flash for failure analysis Kernel Stack traces, boot record and fault logs

NVRAM 2 Mbytes of SRAM – Syslog and licensing information

Secure Keystore Renesas AE46C1 – Credentials and secure RNG

Management Interfaces RS-232 console port – console0

10/100/1000BASE-T – mgmt0 partitioned from inband VLANs

Unified Crossbar Fabric58 port crossbar and scheduler

3 unicast and 1 multicast crosspoints

Central tightly coupled scheduler

Request, propose, accept, grant, acknowledge semanticsPacket enhanced iSLIP scheduler

Distinct unicast and multicast schedulers

Eight classes of serviceEgress buffer creditsDWRR class of serviceDWRR ingress interface

Total SRAM 24.6 MbitsGates 12.4 MillionTransistors ~200 MillionMetal Layers 7Signal Pins 1286

SerDes 232 @ 3.75Gbps

Unified Port Controller

Media Access Controllers1/10G Ethernet and 1/2/4G Fibre Channel

Packet Buffering and QueuingTotal of 1.875 MBytes used in four slices

Forwarding ControllerEthernet, Fibre ChannelLayered policy engine

Four data path slicesOne 1/10G Ethernet or two 1/2/4G Fibre Channel portsConnects to one Altos port

All switching done in Altos crossbar480 KBytes of buffering

Total SRAM 35 Mbits

Total TCAM 1 Mbit

Logic Gates 18 Million

Transistors ~300 Million

Metal Layers 7

Total Pins 900

SerDes 32 @ 3.75Gbps

Slice 4Slice 2 Slice 3

Switch ASIC Architecture

Slice 1

Forwarding

1/10G MAC

Transceiver

Virtual Queues

Egress Queues

Packet Buffer

Virtual Queues

Egress Queues

Packet Buffer

Slice 2 Slice 3Slice 1

Virtual Queues

Egress Queues

Packet Buffer

Slice 4

4 @ 3.75G – 12Gbps 4 @ 3.75G – 12Gbps 4 @ 3.75G – 12Gbps

XAUI – 10 Gbps4 @ 3.125G

Transceiver

XAUI – 10 Gbps4 @ 3.125G

1/10GE Attached Server

10GE LAN Uplink

58 source busses in total

Parsing &Editing

Forwarding

?Forwarding

Parsing &Editing

1/10G MACFC MAC

FC MAC

Fibre Channel SAN Uplinks

1/2/4G Fibre Channel1 @ 1.0625/2.125/4.25G

Parsing &Editing

Fabric Buffer Fabric Buffer Fabric BufferUnicast and

Multicast Schedulers

Slice 4Slice 2 Slice 3

Packet Walkthrough

Slice 1

Forwarding

1/10G MAC

Transceiver

Virtual Queues

Egress Queues

Packet Buffer

Fabric Buffer

Virtual Queues

Egress Queues

Packet Buffer

Fabric Buffer

Slice 2 Slice 3Slice 1

Virtual Queues

Egress Queues

Packet Buffer

Slice 4

Forwarding

Fabric BufferUnicast and

4 @ 3.75G – 12Gbps 4 @ 3.75G – 12Gbps 4 @ 3.75G – 12Gbps

XAUI – 10 Gbps4 @ 3.125G

Transceiver

XAUI – 10 Gbps4 @ 3.125G

1/10GE Attached Server

10GE LAN Uplink

?Forwarding

Parsing &Editing

1/10G MACFC MAC

FC MAC

Fibre Channel SAN Uplinks

1/2/4G Fibre Channel1 @ 1.0625/2.125/4.25G

Parsing &Editing

1. Decode, align, synchronize bytes

Decrypt, verify, authenticate frames

5. Queue frames and manage crossbar service requests

4. Store frame content when waiting

6. Match requests, available outputs,

and fairness criteria

7. Landing place for frames in flight

9. Evaluate frame fields for, filtering,

and editing

3. Evaluate frame fields for forwarding, filtering, and editing

2. Extract frame fieldsAdd/remove headers

and edit frame contents

10. Encrypt frames and encode bytes

8. Extract frame fieldsAdd/remove headers and

edit frame contents

Switch Fabric Data Path

Unified Crossbar Fabric

Port 4

Data Path Deep Dive

Media Access Controllers

Crossbar operationUnicastMulticast

Latency

Port 1

Forwarding

1/10G MAC

Transceiver

Virtual Queues

Egress Queues

Packet Buffer

Fabric Buffer

Virtual Queues

Egress Queues

Packet Buffer

10GE Attached Servers

Parsing &Editing

1/10G MAC

Parsing &Editing

Transceiver

Media Access Controllers

Each Unified Port Controller slice has…One 1 Gigabit Ethernet MACOne 10 Gigabit Ethernet MACTwo 1/2/4 Gigabit Fibre Channel MACs

Two of the slices in each Gatos have an 802.1AE LinkSec encryption engine

Integrated Flow Control handlingEthernet – 802.3X “PAUSE” and Cisco Priority Flow ControlFibre Channel – BB_credits

Crossbar Overview

Tightly coupled scheduler and crosspoint20% link speedup

12 Gbps

Unicast SchedulerVirtual Output Queuing3x fabric speed up

3 crosspointsMultiple frames transferred per scheduling event

“Superframing”

Multicast SchedulerSystem Class queuingSeparate crosspoint

Unicast Virtual Output Queuing

Day in the Life of a Unicast Frame

1. Frame arrive into Packet buffer

2. Frame pointer posted to Virtual Output Queue

3. VOQ posts request to Scheduler

4. Scheduler arbitrates and grants access

5. Frame sent to Fabric Buffer

6. Fabric Buffer sends to egress queue

7. Egress port sends frame on wire

8. Egress buffer indicates freed buffer resources

Unicast scheduler overview

Each VOQ send a request to egress Scheduler associated with the corresponding portEgress Scheduler choose an ingress and make a proposal

A priority is selected (Fixed priority, or DWRR)An ingress is selected within that priority

Ingress Scheduler choose an egress and send acceptSelection is based on fixed Round Robin

Egress Scheduler send a grant to the VOQ

Unicast scheduler overview (continue)

Multicast

Multicast, fabric replication

Ingress Fabric Egress

BMMcast

AUcast

BMcast

AU-VOQU-VOQ

BBU-VOQU-VOQ

U-VOQU-VOQ

Use cases• Ethernet multicast

M-VOQM-VOQAA

SPAN, ingress replication

M-VOQM-VOQ

Ingress Port configured for SPAN

Fabric Egress

BMUnicast

AUcast

BMcast

U-VOQU-VOQ

Use cases•Ingress SPAN sessions•Egress SPAN sessions

VOQ for SPAN packets

Multicast Class Queuing

Multicast Scheduling Algorithm

Port 4

Latency3.2 µsec port-to-port

First-In-First-OutFull featured forwarding

6.7 µsec kernel to kernelStateless offloads

no DDP1.4 µsec host send2.1 µsec host receiveIncreases with OS, interrupt, and transport overheads

Socket layer app-to-appLinux 2.6Raw – 10.1 µsecUDP – 11.2 µsecTCP – 11.8 µsec

Port 1

Forwarding

1/10G MAC

Transceiver

Virtual Queues

Egress Queues

Packet Buffer

Fabric Buffer

Virtual Queues

Egress Queues

Packet Buffer

Parsing &Editing

1/10G MAC

Parsing &Editing

Transceiver

Lossless Data PathLossless Data Path

QoS Flow (animated)

EntersNuova Switch

DropPolicy

Queue 1

Queue 2

Queue 3

Queue 8

QoS Actions at Ingress Unified Port Controller

UnifiedCrossbar

Fabric

QoS Actions atEgress Unified Port

Controller

• No Drop• Drop

RXRXSystemClassmap

Ingresspolicer

Cos classification

Input portclassification

Egress Queuing (with Priority and Bandwidth)&

BCNTXTX Buffering Marking

Buffering

Class Based Data Path

Different classes of traffic require different treatment, e.g.FC class of traffic requires lossless or no drop treatmentMarket Data Ethernet class traffic may also require no dropRemaining Ethernet Data may only require best effort

Nexus 5000 data path resource and features are all per class based; for example,

Per class VOQs and egress queues, buffers, MTU, drop behavior

Per Class behavior should be consistently configured system wideand network wide

Nexus 5000 supports Modular QoS CLI (MQC) for all QoS configuration

System is a new target introduced at the global cfg levelSystem classes are instantiated within a system policy System policy is a service-policy attached to the ‘system’ target

At FCS, parameters configurable under system class:MTUDrop, no Drop

At ingress, packets are classified into a system classAt FCS, classification can be based on .1p or interface

Once classified, this class assignment travels with the packet through the entire system to select per class treatment at every step

At Egress, 802.1p rewrite is supported. 802.1p value can then be consistently used throughout the network to select the same system class treatment

System Class

switch(config)# systemswitch(config-system)# service-policy dc-policy

switch(config)# policy-map dc-policyclass eth-lo

mtu 9000class eth-hi

pause no-drop mtu 1500

class dc-controlmtu 4000

switch(config)#class-map eth-lomatch cos 2

class-map eth-himatch cos 4

class-map dc-controlmatch cos 5

System Policy Example

System Class Defaults

System classes defaults allow for plug N play IOC datapathBy default, there are 4 classes:

FC-default no drop, FC and FCoE traffic automatically belong to this class

Sup-hi – high priority control plane traffic such as BPDUs, FSPFSup-low – low priority control plane traffic such as CDPEth-default – all other traffic types classified into this class

Drop (best effort class)

User cannot remove Sup-HI, Sup-Low and FC-defaultUser can modify parameters in FC-default class but not Sup classes

User can define up to 5 new system classes including the Ethernet class-default

PFC and BB_Credits

IEEE 802.3x Pause provides no drop flow control similar to BB credits for FC

Priority Flow Control is a finer grained mechanism of flow control over standard pause or link level BB credits

Priority Flow Control uses .1p CoS value mapping to a system class to send appropriate pause to previous hop

Priority Flow ControlPriority based Flow ControlPriority based Flow Control

• Enables lossless behavior for each class of service

• PAUSE sent per priority when buffers limit exceeded

Priority based bandwidth managementPriority based

Bandwidth Management Priority based

Bandwidth Management

• Enables Intelligent sharing of bandwidth between traffic classes control of bandwidth

• 802.1Qaz Enhanced Transmission

Forwarding

Port 4Port 1

Data Path

Unified Crossbar Fabric

Forwarding

1/10G MAC

Transceiver

Virtual Queues

Egress Queues

Packet Buffer

Fabric Buffer

Virtual Queues

Egress Queues

Packet Buffer

Parsing &Editing

1/10G MAC

Parsing &Editing

Transceiver

Forwarding Pipeline

Wire rate “fixed” latency

Parsed frame fields, configuration, and control plane state are evaluated to determine destination(s)

Policy engine filters based on configuration, bindings, and layered ACLs

Layered equal cost multi path expansion

Fibre ChannelEtherChannel/ SAN PortChannel

Destination address

Source address

Ethertype = IP

Ethertype = .1Q VLANCoS d

TOS Total lenVer IHLIdentification Flg Frgm offset

TTL Proto Header cksumSource address

Destination address

IP options

Src port Dst portSeq numberAck number

Hdr len Flags Win sizeCksum Urgent ptr

TCP options and data

checksum check

FCS check

Parsing ethernet IP packets

Destination address

Source address

Ethertype = FCoE Ver

ReservedSOF

EOF Reserved

r_ctl d_id

seq_id df_ctl seq_cntox_id rx_id

Payload

Parameters

cs_ctl s_idtype f_ctl

CRC check FCS check

Fibre Channel frames are FCoE encapsulated prior to

forwarding

Parsing FCoE packets

Acquiring Interface StatePhysical Interface Table

Physical interface related configuration and state

Virtual Interface TableVirtual interface related configuration and state

VLAN State TableVLAN related configuration and state

Ethernet Forwarding16K Entry dLeft hash table (StationTable)

Searched by {VLAN, destination address}

SelectsLocal port/PortChannelMulticast index

Unknown addresses forwarded by VLAN multicast vectors

Unknown unicastUnregistered multicastBroadcast

IP Multicast forwarded by MAC addressIP multicast groups registered by IGMP snooping

Same mechanism forwards Fibre Channel in the local domain and N_port Virtualizer

Fibre Channel Forwarding

4K Entry dual index search tableSearched by {VSAN, domain_id}Misses are Fibre Channel exceptions

SelectsLocal port or PortChannel

Ethernet Address LearningIngress and Egress learning searches

Line rate on for all framesFacilitates distributed table population

Ingress notifies Supervisor to develop

database

Supervisor pushes new addresses to

all Unified Port ControllersAdds entries if missedRe-enforces existing entries

Supervisor queries tables to check for

consistencyMaintains aging state

CPU removes entries that are obsolete

Policy Enforcement Frames evaluated by multi-stage engineSearches occur in parallel

Results evaluated in pipelineDiagnostics and control plane “tap” pipeline at any point

Multipath Expansion

VLAN Membershipcheck

Interface, VLAN, and MAC Binding

MAC and L3 Binding(IP & Fibre Channel)

Fibre Channel Zone membership check

Port ACLs

VLAN ACLs (ingress)

QoS ACLs (ingress)

Role Based ACLs (egress)

Control Plane R

edirect/Snooping

Switch Port Analyzer (SPAN

) and Diagnostic Sam

failpass

pass fail

permit

policer drop

to Supto SPANsession

ACL Search Engine

2048 Ternary match ACEsEach entry available to all functions

Labels allow sharing of Access Control Entries (ACEs)ACLs have a labelPolicy definition points select a label

Interfaces, VLANs, RolesLabels and frame fields form search keys

Flexible region assignmentTune ACL resource allocation to network policies

ACLs scopeVLAN and Control plane are Global scope – same on all Unified Port ControllersPort, QoS, RB, and SPAN are local scope – specific to each Unified Port Controller

TCAM 2Kx432

Priority

Search KeyPort ACLs

(576)QoS ACLs

(64 ingress) Role Based ACLs

(egress)

Vlan ACLs(1024)

SPAN and Diagnostic ACLs (64)

Control Plane ACLs (128)

Priority

Multipath Expansion

Two stage expansion processEach can lead to the nextSame mechanism for all expansionsConfiguration of expansion unique to each expansion

Fibre Channel SwitchingSelects a path to a target Fibre Channel switch

Fibre Channel Shortest Path First (FSPF)

Etherchannel/PortChannelSelects a path to a physically adjacent device

Fibre Channelswitching?

FC Multipath Expansion

EtherChannel/ PortChannel Expansion

List of Virtual Output Queues

Policy Enforcement

Expansion Algorithm

Relevant frame fieldsEthernet SA and DA always availableIP frames allows inclusion of IP v4/v6 SA and DA

TCP/UDP frames can include source and destination portsFibre Channel frames can include D_ID and S_ID

OX_ID can also be included per VSAN

Each field is divided by one of two CRC-8 polynomialsResult of field CRC division is combined via bitwise XORResult selected using modulo division by number of equal cost paths

256 possibilities are reduced to avoid biasWorst case imbalance is 6%

Ethernet DA

Ethernet SA

IP DA or FC D_ID

IP SA or FC S_ID

TCP DP

TCP SP or FC OX_ID

CRC-8 A

Field selectPolynomial select

XOR Modulo

Number of equal paths

256 possibilities Selected

CRC-8 B

Editing FC packets for VOQ

Destination address

Source addressr_ctl d_id

Payload

Parameters

CRCEthertype = FCoE Ver

ReservedSOF

EOF Reserved

r_ctl d_id

Payload

Parameters

NX-OS Overview

Nexus 5000 Software Architecture Overview Consistent user experience across NX 7000, NX 5000 and MDS

Nuova Extensions to Support converged

interfaces

Layer-2 Protocols I/O ConsolidationFeatures & Protocols

(Nuova Addition)

Storage Protocols(SANOS3.1(3)+ NPV)

Interface Management

Chassis Management

Kernel

Chip/Driver Infrastructure

VSANmgr

F_portSvr

Zoning

WWNMgnt

DCBXQoS Mgt

Priority Flow ControlSPAN

802.1X

FC flow

……

Protocol Stack (IPv4, IPv6 , L2, fc2)

VLANmgr

IGMPsnp

Nuova HW architectureSpecific Components

DCOS4.0 code base

Ongoing Syncing and Committing

NX-OS Features and Benefits

Software compatibility Common software throughout the data center Modular software design Quick development of enhancements and problem fixes Troubleshooting and diagnostics

Smart Call Home Cisco GOLD

Programmatic XML interface Simple Network Management Protocol (SNMP)Role-based access control (RBAC)

Nexus 5000 NX-OS Image Layout and Upgrade

Field Upgradeable BIOS and Bootloader stored together in one flash

Field Upgradeable Kickstart and System images stored in a separate flash

BIOS and Bootloader image packaged in System Image

Image RecoveryIf system image is bad, kickstart can retrieve image from networkIf kickstart+system image flash is bad, bootloader can netboot image

NX-OS Install and Version Check Infrastructure

Future ISSU leveraging NX-OS stateful process restart and upgrade infrastructure

Nexus 5020 Licenses

NX-OS Licensing InfrastructureAll support time based license and grace periodsLicenses are enabled through the same workflow as Nexus 7000 and

Base software includes Ethernet featuresIncluded with every 5020, no license enablement required

Two feature licenses are available for 5020 N5020-SSK9 Nexus 5020 Storage Protocol

ServicesN5000FMS1K9 Nexus 5000 Fabric Manager Server

N5020-SSK9 includes protocol support for FC and FCoERequired for any fibre channel expansion module.

Nexus 5000 Feature Overview

I/O Consolidation Ethernet and Fibre Channel

IO Consolidation

virtual-ethernet interface (veth)Paired with host’s Ethernet deviceConfiguration point for all Ethernet features

virtual-fc interface (vfc)Paired with host’s HBA deviceConfiguration point for all Fibre Channel features

virtual-interface-group (vig)Logical representation of a switch port

Consists of one veth and one vfcConfigured online or offlineBound to physical switch port for deployment

Etherchannel post FCS

SCSI IP

SAN A SAN B

vethvfc

Ethernet ForwardingFC Forwarding

Ethernet

Connecting LAN and SAN on a single physical link

fc fc eth eth

IO consolidation: interface configurationCreate virtual-interface-group and bind to physical interface

switch(config)# interface vig 20

Bind virtual-interface-group to physical interfaceswitch(config)# interface vig 20switch(config-if)# bind Ethernet 1/1

Configure virtual-ethernet and virtual-fcswitch(config-if)# interface veth 20/1switch(config-if)# interface vfc 20/1

vfc30/1veth30/1vfc20/1veth20/1

Eth1/1

Eth1/33

Fibre Channel N_Port Virtualizer

Provides physical port level virtualization of multiple FC end nodes to one F_Port off a FC Switch

Nuova Switch operates in N_Port Proxy Mode (not in FC Switch mode)Simplifies multi-vendor interoperation

Eliminates the FC domain on Nuova switch

Simplifies management

Used in conjunction with NPIV

FC N_Port Virtualizer

Nuova Switch

Fibre Channel Attachments

N_Port Virtualizer terminology

Border Interface

N_ports

Server interface

F_Port

NP_Port

N_Port Virtualizer

Nuova Server Switch

F_Port F_Port F_Port

NP_Port NP_Port

N_Port Virtualizer in detail

NPIV Proxy captures all login associated packets from the HBA and Border Interfaces (BI)

Hosts pinned to Border Interfaces

Support NPIV over Server Interfaces (SI)

Relies on NPIV on Border interfaces

Retry failed login requests from one Border Interface on a different interface

Handle Events by generating proxy LOGOs

SISI SI

N_Port Virtualizer

MDS #1 MDS #2

HBA #1 (NPIV)

HBA #2(NPIV)

HBA #3

Nuova Ethernet Host Virtualizer

Eliminates Need for Spanning Tree Protocol on Uplink Bridge Ports

Reduces CPU load on upstream switches

Allows Multiple Active Uplinks from Nuova Switch to Network

Doubles effective bandwidth vs STP

Prevents Loops by Pinning a MAC Address to Only One Port

Completely Transparent to Next Hop Switch

Ethernet Host Virtualizer

Nuova Switch

Active-Active

Pinning

Border interface

Server interface

Outgoing traffic: known unicast

Border

Traffic sourced by a station connected to a SIF goes out of its pinned border interface

Outgoing traffic: Multicast/broadcast

Border

Local replication to all SIFs is done by the End Host Virtualizer switch

One copy of the packet is sent out of the source SIF’s pinned border interface

Incoming traffic: Reverse Path Forwarding

Border

Packets destined to a station behind a SIF are accepted only by the SIF pinned border interface

A station can’t send anything out until her MAC has fully propagated to all Gatos (otherwise RPF can’t be honored)

Incoming traffic: Multicast/broadcast portal

Border

One border interface is elected to receive broadcast, multicast and unknown unicast traffic for all the SIFs

Incoming traffic: Deja-vu check

Border

If the source MAC belongs to a local station

The multicast/broadcast portal drops the packetThe pinned port accepts the packet, but no replication is done

This is regardless of the destination MAC (known/unknown unicast, multicast or broadcast)

Configuration Exchange and Validation

DCB Capability Exchange Protocol

Link level capability and configuration exchangeSimilar to FLOGI and PLOGI in Fibre ChannelAllows either full configuration or configuration checking

Based on LLDP (Link Level Discovery Protocol)Added reliable transportLink partners can choose supported features and willingness to accept configuration from peer

Feature TLVsPriority Groups (Link Scheduling)Priority-based Flow ControlCongestion Management (Backwards Congestion Notification)Application (frame priority usage)Logical Link Down

Host Redundancy

NIC Bonding (active-standby)Server uplink redundancyMultiple NICs connect to different switchesOne IP interface to host

One MAC address

SCSI Multi-Pathing Presents logical devices to the host applicationsLoad balances I/O requests across redundant linksactive/active or active/standby

Based on target capabilities

SCSI multipathing

file system

SAN A SAN B

Active for SAN

Active for LAN and SAN

SPANSupport local SPAN and ERSPAN(After FCS)2 SPAN sessions per boxSPAN source can be a port, VLAN, or EtherChannelExpand the SPAN to support traffic monitoring between FC and Ethernetsource vlan/vsan list to filter spanned traffic for port and port-channel source

Souce SPAN Destination SPAN SPAN type

Ethernet Ethernet Local SPAN, ERSPAN

Fibre Channel Fibre Channel Local SPAN

Fibre Channel Ethernet Local SPAN ERSPAN(FC FCoE)

Virtual Ethernet Ethernet Local SPAN, ERSPAN

Virtual Fibre Channel Fibre Channel Local SPAN(FCoE FC)

Virtual Fibre Channel Ethernet Local SPAN, ERSPAN

Network Design

DISTRIBUTION

Server Cabinet Pair 1 Server Cabinet Pair N

MDS 9500

ACCESSNexus 5020

10GE/FCoE CNA

SAN-BLAN Core

STP BLK

EthernetFibre ChannelFCoE

I/O consolidation with TOR Design

POD 1 POD N

Server Cabinet Pair 1 Server Cabinet Pair N Server Cabinet Pair 1 Server Cabinet Pair N

ACCESSNexus 5020

10GE/FCoE CNA

SAN-BLAN Core

End Host ModeActive/Active

DISTRIBUTIONMDS 9500

I/O consolidation with TOR Design End Host Mode

POD 1 POD N

DISTRIBUTION

Server Cabinet Pair 1 Server Cabinet Pair N Server Cabinet Pair 1 Server Cabinet Pair N

MDS 9500

ACCESSNexus 5020

10GE/FCoE CNA

SAN-BLAN Core

VSS Supportat Aggregation

I/O consolidation with TOR Design VSS Support at Aggregation Layer

4 x 4G FC

ACCESSNexus 5020

Blade Switch = 10GE x 1GE

Server Rack 1 Server Rack N

MDS9134

2 x 10GE

MDS9134

LAN Core

DISTRIBUTIONMDS 9500

Nexus 5020 in Blade Center Environment 10G Aggregation

Nexus 5000 Deep Dive - Cisco · Nexus 5020. 56-Port L2 Switch • 40 Ports 10GE/FCoE, fixed • 2...

Documents