Deployment Considerations with Interconnecting Data...

© 2012 Cisco and/or its affiliates. All rights reserved. BRKDCT-3060 Cisco Public

Deployment Considerations

with Interconnecting Data Centers Patrice Bellagamba

Distinguished SE

Cisco Europe

BRKDCT-3060


Session Objectives

The main goals of this session are:

• Highlighting the main business requirements driving Data Center Interconnect (DCI) deployments

• Understand the functional components of the holistic Cisco DCI solutions

• Get a full knowledge of Cisco LAN extension technologies and associated deployment considerations

• Integrate routing aspect induced by the emerging application mobility offered by DCI

This session does not include:

• Network services integration (Firewall / Load Balancer)

‒ This is the objective of BRK… session

• Storage extension considerations associated to DCI deployments

3


Session-ID Session Name

TECIPM-3191 Advanced LISP Techtorial

BRKDCT-2049 Overlay Transport Virtualization

BRKDCT-2081 Cisco FabricPath Technology and Design

BRKDCT-2131 Mobility and Virtualization in the Data Center with LISP and OTV

Related Cisco Live 2011 Events DCI Sessions

An important companion to this session is: BRKDCT-2312 - Design consideration for Network and Security services stretched

over multiple locations

4


• DCI Business Drivers and Solutions Overview

• LAN Extension Deployment Scenarios

Ethernet Based Solutions

MPLS Based Solutions

IP Based Solutions

• LISP for DCI Deployments

LISP and Path Optimization

LISP as L3 DCI

• Summary and Q&A

Agenda

5


DCI

Drivers Business Solution Constraints IT Technology

Business

Continuity

Disaster Recovery

HA Framework

Stateless

Network Service Sync

Process Sync

GSLB

Geo-clusters

HA Cluster

Operation Cost

Containment

Data Center Maintenance / Migration / Consolidation

Host Mobility Distributed Virtual

Data Center

Business

Resource

Optimization

Disaster Avoidance

Workload Mobility

VLAN Extension

Statefulness

Bandwidth & Latency

VM Mobility

Cloud Services Inter-Cloud Networking

XaaS

Flexibility

Application mobility

VM Mobility

Automation

Data Centers are extending beyond traditional

boundaries

Virtualization applications are driving DCI across PODs

(aggregation blocks) and Data Centers

Data Center Interconnect Business Drivers

6

© 2012 Cisco and/or its affiliates. All rights reserved. BRKDCT-3060 Cisco Public 7

DC1 DC2 DC3

STP domain

STP domain

STP domain

SiSiSiSi SiSi SiSi

ALT GW ALT ALT GW GW

Path Optimization

Dual-Homing

Storage extension

Any type of links

STP Domain isolation + Storm-control

Data Center Interconnect LAN Extension Model


Type T0 Limited to a single access layer

Type T1 Extended inside an aggregation block (POD)

Type T2 Extended between PODs part of the same

DC site

Type T3 Extended between twin DC sites connected

via dedicated dark fiber links

Type T4 Extended between twin DC sites using non

5*9 connection

Type T5 Extended between remote DC sites

T1 T2

T0

T3

T4

Fabric-path

/ vPC

Fabric-path

/ vPC

OTV/VPLS

OTV/VPLS

LAN Extension for DCI VLAN Types

8


Ethernet

MPLS

IP

Over dark fiber or protected D-WDM

VSS & vPC

Dual site interconnection

FabricPath (TRILL)

MPLS Transport

EoMPLS Transparent point to point

A-VPLS Enterprise style MPLS

H-VPLS Large scale & Multi-tenants

IP Transport

OTV Enterprise style Inter-site MAC Routing

Campus style

SP style

IP style

LAN Extension for DCI Technology Selection Criteria

9


• Transport

Fiber

LOS report / Protected DWDM

L2 SP offer (HA=99.7+)

IP

• Scale

Site

VLAN (102 or 103 or 104)

MAC (103 or 104 or 105)

• Multi-tenants

Tagging (VLAN / 2Q / VRF)

Overlapping / Translation

• Multi-point or point to point

• Greenfield vs. Brownfield

Ethernet only for 5*9 HA link

MPLS/IP for WAN quality link

Ethernet for medium scale

IP for low scale

MPLS for high scale

MPLS for multi-tenancy features

LAN Extension for DCI Technology Selection Criteria

10





VSS, vPC and FabricPath


IP Based Solutions



LISP as L3 DCI

• Summary and Q&A

Agenda

11


WAN

L

3

L

3

Server Cabinet Pair 1 Server Cabinet Pair N Server Cabinet Pair 1 Server Cabinet Pair N

L

2

L

2

SiSi SiSi

Primary Root Primary Root

On DCI Etherchannel:

STP Isolation (BPDU Filtering)

Broadcast Storm Control

FHRP Isolation

Link utilization with Multi-Chassis EtherChannel

DCI port-channel

- 2 or 4 links

Requires protected DWDM or Direct fibers

vPC does not support L3 peering:

Use dedicated L3 Links for Inter-DC routing!

Validated design:

200 Layer 2 VLANs + 100 VLAN SVIs

1000 VLAN + 1000 SVI (static routing)

interface port-channel10

desc DCI point to point connection

switchport

switchport mode trunk

vpc 10

switchport trunk allowed vlan 100-600

spanning-tree port type edge trunk

spanning-tree bpdufilter enable

storm-control broadcast level 1

storm-control multicast level x

Dual Site Interconnection Leveraging EtherChannel between Sites

12

DC Site 2 DC Site 3 DC Site 4 DC Site 1

L2 DCi is NOT LAN Switching! Is FabricPath a valid solution for DCi ?

FabricPath is primarily positioned for Clos-based architectures

• Perception on FabricPath DCi

‒ Plug and play

‒ No Spanning Tree events shared between DC sites

‒ Can do IP routing over FP DCi

‒ One single protocol to manage end to end

‒ One single Fabric end to end

‒ Works also with N5K only scenarios

13


• Dependencies with L1 WAN links

- Requires point to point high quality connections

- Golden rule : WAN links must support Remote Port Shutdown and micro flapping protection

• Multidestination traffic impacts

- Must tune multicast tree to avoid local traffic to fly over root tree site

- Cannot avoid multicast to fly over root tree site for DCI multicast

• IP routing over FabricPath

- Ship in the night effect

- OSPF hellos are multicast and will fly over root site

• STP interactions with FabricPath DCI

- The Fabric becomes STP root for all propagated VLAN, means that twin site vPC will be blocking

• FabricPath & HSRP Localization

- HSRP Control-plane can be isolated with mismatching authentication key

- But HSRP data-plane cannot be isolated when DC is also FP, leading to flapping vMAC

• High Availability

- L2 ISIS fine tuning is required: allocate-delay timer, transition-delay, linkup-delay, spf-interval, lsp-gen-interval

- Sub second convergence, except node recovery in 3s

FabricPath DCI - Lessons learned

Root MDT1 S1

R2

10 20 40

40

5

40 40

Site C

Site B

R1

Site A

FabricPath DCI - Key Takeaways

• On DCi, FabricPath is not so Plug and Play actually

‒ No specific DCI functions compared to OTV, VPLS

‒ Several designs gotchas but do not impact all customers

‒ Multidestination Trees capacity planning may be very complex

‒ Multiple Topologies will enhance the overall solution

• By default, OTV/VPLS should be the first solutions to promote

‒ Cisco Validated Designs (CVDs)

‒ Specific DCi features

‒ Offer an efficient independence between DC

• FabricPath is a valid DCi solution when :

‒ Short distances between DCs (tromboning is not a issue)

‒ Multicast is not massively used

Customer

references

Operations

simplicity

Domino effect

prevention

DCi link quality

mgmt

3+ Sites

optimization High Availability L2 functions

L3 Unicast

functions

Multicast

functions Scalability

FabricPath

15






EoMPLS

VPLS

H-VPLS

IP Based Solutions



LISP as L3 DCI

• Summary and Q&A

Agenda

16


interface

interface

PE1

PE2

interface g1/1

description EoMPLS port mode connection

no switchport

no ip address

xconnect 2.2.2.2 vcid 1 encapsulation mpls

T-LDP

LSP Label VC Label Ethernet

Header

Ethernet

Payload 0x8847 DA’ SA’ FCS’

1518 8

EoMPLS Port Mode xconnect

17


Aggregation

Layer DC1

DCI DCI Aggregation

Layer DC2

On DCI Etherchannel:

STP Isolation (BPDU Filtering)

Broadcast Storm Control

FHRP Isolation

MPLS Core

Active PW

Active PW

Encryption Services with 802.1AE

Requires a full meshed vPC 4 PW

EoMPLS Usage for DCI End-to-End Loop Avoidance using Edge to Edge LACP

18


Aggregation

Layer DC1

DCI DCI Aggregation

Layer DC2

IP Core

Active PW

Active PW

crypto ipsec profile MyProfile

set transform-set MyTransSet

interface Tunnel100

ip address 100.11.11.11 255.255.255.0

ip mtu 9216

mpls ip

tunnel source Loopback100

tunnel destination 12.11.11.21

tunnel protection ipsec profile MyProfile

EoMPLS Usage for DCI Over IP Core

19


MPLS Core

Aggregation

Layer DC1

DCI DCI Aggregation

Layer DC2

PE receives the PW down

notification and shutdown its

transmit signal toward

aggregation

X X SiSi

SiSi

Active PW

Active PW

X

ASR1000 / ASR903 feature configuration:

interface GigabitEthernet1/0/0

xconnect 1.1.1.1 1 pw-class eompls

remote link failure notification ! (default)

Failover

(msec)

Fallback

(msec)

Bridged

traffic

281 54

453 300

Dealing with PseudoWire (PW) Failures Remote Ethernet Port Shutdown

20


Instead of xconnecting physical port, xconnect port-channel

LACP is kept local, no more extended over EoMPLS

PW is virtual on both VSS members

SSO protection in 12.2(33)SXJ

Requires VSS or Nexus as DC device

Limited support of L3 routing with vPC

MPLS

Local LACP Local LACP

One PW

Aggregation

Layer DC1

SiSi

SiSi

VSS

SiSi

SiSi

VSS Aggregation

Layer DC2

EoMPLS Deployment on VSS Point to Point EoMPLS with Port-Channel xconnect

21






EoMPLS

VPLS

H-VPLS

IP Based Solutions



LISP as L3 DCI

• Summary and Q&A

Agenda

22


MPLS

Core

One extended bridge-domain built using:

VFI = Virtual Forwarding Instance

( VSI = Virtual Switch Instance)

VFI

VFI

VFI

PW

PW

PW

VLAN VLAN

VLAN

SVI

SVI SVI

Mac address table population

is pure Learning-Bridge

PW = Pseudo-Wire

SVI = Switch Virtual Interface

xconnect

Multi-Point Topologies What is VPLS?

23


• Using clustering mechanism

‒ Two devices in fusion as one

• VSS Sup720

• VSS Sup2T

• ASR9K nV virtual cluster

One control-plane / two data-planes

• Dual node is acting as one only device

• Native redundancy (SSO cross chassis)

• Native load balancing

• Capability to use port-channel as attachment circuit

SUP720+ES

SUP2T

ASR9K nV

VPLS Cluster Solutions

24


SiSi

SiSi

SiSi

SiSi

LDP session protection & Loopback usage allows

PW state to be unaffected

LDP + IGP convergence in sub-second Fast failure detection on Carrier-delay / BFD

Immediate local fast protection Traffic exit directly from egress VSS node

X

VSS Failover

(msec)

Fallback

(msec)

Bridged

traffic

258 218

162 174

mpls ldp session protection

mpls ldp router-id Loopback100 force

VPLS Redundancy Making Usage of Clustering

25


SiSi

SiSi

SiSi

SiSi

If failing slave node: PW state is unaffected

If failing master node:

• PW forwarding is ensured via SSO

• PW state is maintained on the other side using

Graceful restart

Edge Ether-channel convergence in sub-second

Traffic is directly going to working VSS node

Traffic exits directly from egress VSS node

Quad sup SSO for SUP2T in 1QCY13

VSS Failover

(msec)

Fallback

(msec)

Bridged

traffic

224 412

326 316

mpls ldp graceful-restart

X

VPLS Redundancy Making Usage of Clustering

26


Problem

Remote VSS are having two un-equal cost path to others, so one only route is put in RIB

10.100.1.1

/ Solution

Stops forwarding traffic for 2mn when primary route is removed

(there is no control-plane to insert backup route)

Build a symmetric core with two ECMP paths between each VSS

X

sh ip route 10.100.1.1

Known via "ospf 2”

via GigabitEthernet1/3/0/1 Route metric is 2

via GigabitEthernet2/3/0/1 Route metric is 2

VPLS Deployment Considerations “Symmetry is Good”

27


Si

Si Si

Si

SiSi

SiSi

SiSi

SiSi Rem: One PW per VLAN per destination

Any card type facing edge

SUP720 + SIP-400 facing core (5Gbps) or

SUP720 + ES-40 (40Gbps) support with

12.2(33)SXJ

SUP2T

#sh mpls l2 vc

Local intf Local circuit Dest address VC ID Status

------------- ------------- ------------ ----- ------

VFI VFI_610_ VFI 10.100.2.2 610 UP

VFI VFI_610_ VFI 10.100.3.3 610 UP

VFI VFI_611_ VFI 10.100.2.2 611 UP

VFI VFI_611_ VFI 10.100.3.3 611 UP

interface Virtual-Ethernet1

transport vpls mesh

neighbor 10.100.2.2 pw-class Core

neighbor 10.100.3.3 pw-class Core

pseudowire-class Core

encapsulation mpls

switchport


switchport trunk allowed vlan 610-619

VSS - A-VPLS CLI SUP2T in 15.1SY

28


Si

Si Si

Si

SiSi

SiSi

SiSi

SiSi

One GRE tunnel per site

Native on SUP2T

int tunnel 1

tunnel mode gre ip

mpls ip

tunnel source 10.1.1.1

tunnel destination 10.2.2.2

…

interface virtual-ethernet 1

transport vpls mesh

neighbor 10.2.2.2 pw-class cl1

…

switchport


switchport trunk allowed vlan 10, 20

ip route 10.2.2.2 255.255.255.255

Tunnel1

…

VSS - A-VPLS over GRE SUP2T in 15.1SY

29


PW

ASR9K VPLS Set-up

l2vpn

router-id 10.0.1.1

bridge group BG

bridge-domain BD

interface TenGigE0/0/0/4

interface TenGigE0/0/0/5

!

vfi VFI

vpn-id 4003

neighbor 10.0.1.2 pw-id 4003

30






EoMPLS

VPLS

H-VPLS

IP Based Solutions



LISP as L3 DCI

• Summary and Q&A

Agenda

31


DHD

Active POA

Standby POA

Redundancy

Group

MPLS

Terminology:

mLACP : Multi-Chassis Link Aggregation Control

Protocol

MC-LAG : Multi-Chassis Link Aggregation Group

DHD : Dual Homed Device (Customer Edge)

DHN : Dual Homed Network (Customer Edge)

POA : Point of Attachment (Provider Edge)

ICCP

ICCP synchronizes event/states between multiple

chassis in a redundancy group

ICCP runs over reliable LDP / TCP

ICCP relies on BFD/IP route-watch as keepalive

ICCP message to synch state

Ex: LACP, IGMP query …

draft-ietf-martini-pwe3-iccp C7600 SRE with ES facing edge

ASR9K XR4.0

ASR903 Q3CY13

DC Access Multi-Homing Inter Chassis Communication Protocol - ICCP

32


DHD

Active POA

Standby POA

Redundancy

Group

MPLS

Terminology:

mLACP : Multi-Chassis Link Aggregation Control

Protocol

MC-LAG : Multi-Chassis Link Aggregation Group

DHD : Dual Homed Device (Customer Edge)

DHN : Dual Homed Network (Customer Edge)

POA : Point of Attachment (Provider Edge)

ICCP

Multi-Chassis LACP synchronization:

LACP BPDUs (01:80:C2:00:00:00) are exchanged on each

Link

System Attributes: Priority + bundle MAC Address

Port Attributes: Key + Priority + Number + State

redundancy

iccp

group <ig-id>

mlacp node <node id>

mlacp system mac <system mac>

mlacp system priority <sys_prio>

member

neighbor <mpls device>

interface <bundle>

mlacp iccp-group <ig-id>

mlacp port-priority <port prio>

interface <physical interface>

bundle id <bundle id> mode active

DC Access Multi-Homing Inter Chassis Communication Protocol - ICCP

33


http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DCI/vpls/vpls_asr9k.html

SiSi

SiSi

MPLS

core

1

2 3

4

5

6

7

8

Only error 2/3/4 are leading to ICCP convergence

Rem: 2 & 4 are dual errors

500 VLAN Unicast: Link error sub-1s & Node error sub-2s

1200 VLAN unicast: Link error sub-2s & Node error sub-4s

MC-LAG to VPLS Testing

34


1. Selective Trunk Support

Group multiple VLAN in one only core bridge domain

• QinQ model or PBB Model

• VLAN overlapping

2. VLAN translation 121 / 222 / …

Inter-DC VLAN numbering independency

3. Scale to 4000 * 4000 VLAN

Scale above 4000 VLAN

4. Routing for multi-TAG

Multi-tenant default gateway

IRB - IP routing / VRF routing for QinQ tagged frames

Flexible VLAN Handling Ethernet Virtual Circuit - EVC

35


Control-Plane Distribution of Customer MAC-

Addresses using BGP

PE continues to learn C-MAC over AC

When multiple PEs announce the same C-MAC, hash

to pick one PE

MP2MP/P2MP LSPs for Multicast Traffic

Distribution

MP2P (like L3VPN) LSPs for Unicast Distribution

Full-Mesh of PW no longer required !!

BGP

PE PE

PE PE

E-VPN (aka Routed VPLS) Main Principles

36


Nexus 7000 - Data Center Interconnect with VPLS Q3 CY 2013

VFI VLAN X

Primary N7K WAN Edge

MCT = Multi-chassis Trunk Interface

VFI = Virtual Forwarding Instance

Layer 2 switchport Trunk Portchannel

VFI VLAN X

MCT

Secondary N7K WAN Edge

Virtual Port

Channel (vPC)

Vlan X

Vlan X

VLAN tied to Active VFI

with neighbors to remote

DC sites

VLAN tied to Standby

VFI with neighbors to

remote DC sites


Data Center Interconnect with VPLS

Sample Configuration – Nexus 7000

vlan 80-81

!

vlan configuration 80

member vfi vpls-80

!


member vfi vpls-81

!

l2vpn vfi context vpls-80

vpn id 80

redundancy primary

member 10.0.0.3 encapsulation mpls


!


vpn id 81

redundancy secondary



!



switchport trunk allowed vlan 80,81

vlan 80-81

!


member vfi vpls-80

!


member vfi vpls-81

!


vpn id 80

redundancy secondary



!


vpn id 81

redundancy primary



!



switchport trunk allowed vlan 80,81

VFI

VFI

VFI

VFI

PE1 10.0.0.1

PE2 10.0.0.2

PE3 10.0.0.3

PE4 10.0.0.4

VFI

VFI

VFI

VFI

PE1 10.0.0.1

PE2 10.0.0.2

PE3 10.0.0.3

PE4 10.0.0.4

PE 1 PE 2

EVEN VLANs

ODDVLANs

• Primary VFI owner for

ODD vlans

• Secondary owner for

EVEN vlans

• Primary VFI owner for

EVEN vlans

• Secondary owner for

ODD vlans

vPC

vPC

Note: Virtual Port Channel (vPC) configuration not shown


vPC

Operational Primary

Nexus 7000 - Layer 3 + Layer 2 Extension

• Dual VDC or additional VPLS PE

layer required for L3 and L2

extension for the same vlan

‒ Double-sided vPC design (dual vPC

peer links)

‒ No VPLS and IRB support

39

Access

X

Peer Link

vPC

Operational Primary

vPC

Operational Secondary DP RP

DP DP

RP

AGG

A

AGG

B

VFI

200

PE1 PE2

100, flow 1 100, flow 2

VFI

100

Primary Root Vlan 100

Secondary Root Vlan 100

Peer Link

vPC

Operational Secondary

Active VFI

Vlan 100

Standby VFI Vlan 100

CFS AGG A AGG B

VFI

100

VFI

200






IP Based Solutions

OTV Technology Overview

OTV Deployment Considerations



LISP as L3 DCI

• Summary and Q&A

Agenda

40


Overlay Transport Virtualization Technology Pillars

41

OTV is a “MAC in IP” technique to

extend Layer 2 domains

OVER ANY TRANSPORT

Protocol Learning

Built-in Loop Prevention

Preserve Failure Boundary

Site Independence

Automated Multi-homing

Dynamic Encapsulation

No Pseudo-Wire State

Maintenance

Optimal Multicast

Replication

Multipoint Connectivity

Point-to-Cloud Model

First platform to support OTV (since 5.0 NXOS Release)

Nexus 7000

Now also supporting OTV (since 3.5 XE Release)

ASR 1000


Overlay Transport Virtualization OTV Control Plane

Edge Device (ED): connects the site to the (WAN/MAN) core and responsible for

performing all the OTV functions

Internal Interfaces: L2 interfaces (usually 802.1q trunks) of the ED that face the site

Join Interface: L3 interface of the ED that faces the core

Overlay Interface: logical multi-access multicast-capable interface. It encapsulates

Layer 2 frames in IP unicast or multicast headers

OTV

Internal Interfaces

Core L2 L3

Join Interface

Overlay Interface

42


Transport Infrastructure

OTV OTV OTV OTV

MAC TABLE

VLAN MAC IF

100 MAC 1 Eth 2

100 MAC 2 Eth 1

100 MAC 3 IP B

100 MAC 4 IP B

MAC 1 MAC 3

MAC TABLE

VLAN MAC IF

100 MAC 1 IP A

100 MAC 2 IP A

100 MAC 3 Eth 3

100 MAC 4 Eth 4

Layer 2 Lookup

6 IP A IP B MAC 1 MAC 3 MAC 1 MAC 3

Encap 3

Decap 5

MAC 1 MAC 3 West

Site Server 1 Server 3

East

Site

4

7

IP A IP B

1

IP A IP B MAC 1 MAC 3

OTV Data Plane Inter-Site Packet Flow

43


IP A

West

East

3 New MACs are

learned on VLAN 100

Vlan 100 MAC A

Vlan 100 MAC B

Vlan 100 MAC C

South

VLAN MAC IF

100 MAC A IP A

100 MAC B IP A

100 MAC C IP A

4

OTV updates exchanged via

the L3 core 3

3

2

VLAN MAC IF

100 MAC A IP A

100 MAC B IP A

100 MAC C IP A

4

3 New MACs are

learned on VLAN 100

1

Overlay Transport Virtualization OTV Control Plane

44

• Neighbor discovery and adjacency over

Multicast (Nexus 7000 and ASR 1000)

Unicast (Adjacency Server Mode currently available with Nexus 7000 from 5.2 release)

• OTV proactively advertises/withdraws MAC reachability (control-plane learning)

• IS-IS is the OTV Control Protocol - No specific configuration required


OTV Failure Domain Isolation Spanning-Tree Site Independence

45

Site transparency: no changes to the STP topology

Total isolation of the STP domain

Default behavior: no configuration is required

BPDUs sent and received ONLY on Internal Interfaces

L2

L3

OTV OTV

The BPDUs stop here

The BPDUs stop here


OTV Failure Domain Isolation Preventing Unknown Unicast Storms

46

No requirements to forward unknown unicast frames

Assumption: end-host are not silent or uni-directional

Default behavior: no configuration is required

L2

L3

OTV OTV

MAC TABLE

VLAN MAC IF

100 MAC 1 Eth1

100 MAC 2 IP B

- - -

MAC 1 MAC 3

No MAC 3 in the MAC Table

Remote OTV Device MAC

Table

VLAN MAC IF

100 MAC 1 IP A

101 MAC 2 IP B

47

• Automated and deterministic algorithm (not

configurable)

• In a dual-homed site:

Lower IS-IS System-ID (Ordinal 0) = EVEN VLANs

Higher IS-IS System-ID (Ordinal 1) = ODD VLANs

• Future functionality will allow to tune the behavior

OTV OTV

Internal peering for AED election

AED ODD VLANs

AED EVEN VLANs

IP B IP A

Site Adjacency*

Overlay Adjacency

OTV-a# show otv vlan

OTV Extended VLANs and Edge Device State Information (* - AED)

VLAN Auth. Edge Device Vlan State Overlay

---- ------------------ ---------- -------

100 East-b inactive(Non AED) Overlay100

101* East-a active Overlay100

102 East-b inactive(Non AED) Overlay100

OTV-b# show otv vlan

OTV Extended VLANs and Edge Device State Information (* - AED)

VLAN Auth. Edge Device Vlan State Overlay

---- ------------------ ---------- -------

100* East-b active Overlay100

101 East-a inactive(Non AED) Overlay100

102* East-b active Overlay100

OTV-a OTV-b

OTV Multi-homing VLANs Split Across AEDs

*Supported from 5.2 NX-OS release






IP Based Solutions

OTV Technology Overview

OTV Deployment Considerations



LISP as L3 DCI

• Summary and Q&A

Agenda

48


OTV and SVI Routing Introducing the OTV VDC

49

• Guideline: The current OTV implementation on the Nexus 7000 enforces the

separation between SVI routing and OTV encapsulation for any extended VLAN

• This separation can be achieved with having two separate devices to perform

these two functions

• An alternative cleaner and less intrusive solution is the use of Virtual Device

Contexts (VDCs) available with Nexus 7000 platform:

A dedicated OTV VDC to perform the OTV functionalities

The Aggregation-VDC used to provide SVI routing support

Aggregation OTV

VDC

OTV

VDC L2

L3


L2-L3 boundary at aggregation

DC Core performs only L3 role

STP and L2 broadcast Domains

isolated between PODs

Intra-DC and Inter-DCs LAN extension

provided by OTV

Requires the deployment of dedicated

OTV VDCs

Ideal for single aggregation block

topologies

Recommended for Green Field

deployments

Nexus 7000 required in aggregation

vPC vPC

SVIs SVIs SVIs SVIs

Placement of the OTV Edge Device OTV in the DC Aggregation

50


OTV VDC

OTV VDC

Link-1 Link-2 Po1

N7K-A

N7K-B

Link-1

Link-3 Link-4

Link-2 Po1

Physical View

Logical View OTV VDC OTV VDC

N7K-A

N7K-B

Routing VDC Routing VDC

Layer 3 Layer 2

May use a single physical link for Join and

Internal interfaces

Minimizes the number of ports required to

interconnect the VDCs

Single link or physical node (or VDC)

failures lead to AED re-election

50% of the extended VLANs affected

Failure of the routed link to the core is not

OTV related

Recovery is based on IP convergence

Single Homed OTV VDC Simple Model

51


Logical Port-channels used for the Join

and the Internal interfaces Increases the number of physical interfaces

required to interconnect the VDCs

Traffic recovery after single link failure

event based on port-channel re-hashing

No need for AED re-election

Physical node (or VDC) failure still

requires AED re-election

In the current implementation may cause few

seconds of outage (for 50% of the extended

VLANs)

OTV VDC

OTV VDC

Po1

N7K-A

N7K-B

Link 5 Link 7

Po1

Physical View

Logical View OTV VDC OTV VDC

N7K-A

N7K-B

Links 1-2

Layer 3 Layer 2

Links 3-4


Links 1-2 Links 3-4

Link 8

Link 6

Dual Homed OTV VDC Improving the Design Resiliency

52


AED role negotiated between the two OTV VDCs (on a

per VLAN basis)

Internal IS-IS peering on the site VLAN

Recommended to carry the site VLAN on vPC links and vPC

peer-link

For a given VLAN all traffic must be carried to the AED

Device

Part of the flows carried across the vPC peer-link

Optimized traffic flows is achieved in the most resilient model

leveraging Port-Channels as Internal Interfaces

The AED encapsulates the original L2 frame into an IP

packet and send it back to the aggregation layer device

The aggregation layer device routes the IP packet

toward the DC Core/WAN edge

L3 routed traffic bypasses the OTV VDC

Most Resilient Model

Aggregation

Simple Appliance Model

OTV VDC

OTV VDC

AED

Aggregation

OTV VDC

OTV VDC

AED

OTV in the DC Aggregation Site Based Per-VLAN Load Balancing

53


OTV in the DC Aggregation Using F-Series Linecards

55

• F1 and F2 linecards do not support OTV natively

• As of today, the OTV VDC must use only M-series ports for

both Internal and Join Interfaces

Recommendation is to allocate M1 only interfaces to the OTV VDC

• Native OTV support on F-series is targeted for 6.2 release

(Q2CY13)

© 2012 Cisco and/or its affiliates. All rights reserved. BRKDCT-3060 Cisco Public ** Could use static default route or ospf stub

Routing VDC

OTV VDC

hostname routing-vdc

!

interface Ethernet1/1

switchport


switchport trunk allowed vlan 100,600-700

!


ip address 3.3.3.1/24

ip router ospf 1 area 0.0.0.0

ip ospf passive-interface

ip pim sparse-mode

ip igmp version 3

!

ip pim rp-address 33.33.33.33 group-list 224.0.0.0/4

ip pim ssm range 232.0.0.0/8

hostname otv-vdc

feature otv

!

otv site-vlan 100

!


description Internal Interface

switchport



!


description Join Interface


ip igmp version 3

!

interface Overlay100

otv join-interface Ethernet2/2

otv control-group 239.1.1.2

otv data-group 232.1.1.0/24

otv extend-vlan 600-700

!

ip route 0.0.0.0 0.0.0.0 3.3.3.1

N7K-Agg1 N7K-Agg2

e1/1 e1/2

e2/2 e2/1

L3 Link

L2 Link


OTV VDC

OTV VDC

Establish L3 peering

on a dedicated VLAN

PIM enabled interfaces

OTV in the DC Aggregation Configuration (Multicast Transport)

56


Routing VDC

OTV VDC

hostname routing-vdc

!


switchport



!



ip router ospf 1 area 0.0.0.0

ip ospf passive-interface

hostname otv-vdc

feature otv

!

otv site-vlan 100

!


description Internal Interface

switchport



!


description Join Interface


!

interface Overlay100

otv join-interface Ethernet2/2

otv adjacency-server*

otv use-adjacency-server 10.1.1.1 11.1.1.1

otv extend-vlan 600-700

!

ip route 0.0.0.0 0.0.0.0 3.3.3.1

N7K-Agg1 N7K-Agg2

e1/1 e1/2

e2/2 e2/1

L3 Link

L2 Link


OTV

VDC

OTV

VDC

Establish L3 peering

on a dedicated VLAN

* Needed only on the Adjacency Server

Release 5.2

and above OTV in the DC Aggregation Configuration (Unicast Transport)

57


Brownfield

SiSi SiSi

SiSi

Leverage OTV capabilities on Nexus 7000 (Greenfield) and ASR 1000 (Brownfield)

Build on top of the traditional DC L3 switching model (L2-L3 boundary in Agg, Core

is pure L3)

Possible integration with the FabricPath/TRILL model

Greenfield

L3

L2

FabricPath

OTV Virt. Link

ASR 1K

Nexus 7K

SiSi

Greenfield

Nexus 7K

Nexus 7K L2

L3

L2

L3

L2

L3

Nexus 7K OTV OTV OTV OTV

OTV OTV

OTV OTV

Placement of the OTV Edge Device Connecting Brownfield and Greenfield Data Centers

60


6.2 (Q2CY13) OTV – New functionality

• Selective Unicast Flooding

(for unidirectional MACs & silent hosts)

• OTV VLAN translation

• Dedicated Data Broadcast Multicast Group

• Multiple Uplinks / Loopback ED IP

• Scalability to large deployment

• Fast convergence

‒ AED synchronization

‒ Fast remote convergence using Site-ID

‒ Fast local convergence using pre-population

‒ Fast ED failure detection using BFD & route tracking

• F1 and F2e as internal interfaces (proxy mode)

Key features for larger DCI usage






IP Based Solutions



LISP as L3 DCI

• Summary and Q&A

Agenda

62


L2 Links (GE or 10GE)

L3 Links (GE or 10GE)

Layer 3 Core ISP A

ISP B

Access

Agg

Access

Agg

DC A DC B

VLAN A

Public Network

Data-Base Front-End

DB

144.254.100.0/25 & 144.254.100.128/25

EEM or RHI can be used to get very granular

Move the whole application tier

Optimize the whole path:

Client to Server

Server to Server

Server to Client

Server-Server

Path Optimization

Egress Path Optimization: Server-Client Egress Path Optimization: Server-Client

Ingress Path Optimization:

Clients-Server

Path Optimization and DCI Avoid Suboptimal Traffic Path After Workload Motion

63


East-DC

LISP Site

IP Network

ETR

5.1.1.1

5.3.3.3

1.1.1.1

5.2.2.2

10.2.0.0/24 West-DC

10.1.0.0/24

ITR S

D

DNS Entry: D.abc.com A 10.2.0.1

1

10.1.0.1 -> 10.2.0.1

2

EID-prefix: 10.2.0.1/32

Locator-set:

2.1.1.1, priority: 1, weight: 50 (D1)


Mapping Cache

Entry

3

10.1.0.1 -> 10.2.0.1

1.1.1.1 -> 2.1.1.1

4

10.1.0.1 -> 10.2.0.1

5

2.1.1.1 2.1.2.1 3.1.1.1 3.1.2.1

Mapping DB

Inbound Path Optimization LISP Host Mobility

64

LAN Extension

10.2.0.1


East-DC

LISP Site

IP Network

ETR

5.1.1.1

5.3.3.3

1.1.1.1

5.2.2.2

10.2.0.0/24 West-DC

10.1.0.0/24

ITR S

D

DNS Entry: D.abc.com A 10.2.0.1

EID-prefix: 10.2.0.1/32

Locator-set:



2.1.1.1 2.1.2.1 3.1.1.1 3.1.2.1

Mapping DB

Inbound Path Optimization LISP Host Mobility

65

LAN Extension

10.2.0.1

Workload Move

10.2.0.1

10.1.0.1 -> 10.2.0.1

1.1.1.1 -> 3.1.1.1

8

6

9

EID-prefix: 10.2.0.1/32

Locator-set:



Mapping Cache

Entry Update 7






IP Based Solutions


L3 Host Mobility using LISP

LISP as L3 DCI

• Summary and Conclusions

• Q&A

Agenda

67


Use Case Focus for LISP as L3 DCI

• Data-Center migration

‒ Capability to install a device in a brownfield DC to ensure subnet continuity for

migration to a greenfield DC

‒ VM migration / Physical migration with no change on workload IP address

• Hybrid Cloud

‒ Insertion of SP resource in customer local subnet (like SaaS)

‒ Cloud bursting (provisioning of resource in Cloud)

‒ Migration

• Backup services

‒ Partial Disaster Recovery

‒ This require capability of moving back resource to the original site

ETR

MS/MR

PiTR with Mobility on a stick NOT default gateway

Does not receive any traffic before move

Brownfield DC Greenfield DC 10.17.0.0/24 10.17.0.0/24

This is ASM mode with same subnet value both side Home subnet is Greenfield (registers with MS 10.17.0.0/24), Dynamic part is Brownfield (detects and registers any /32)

LISP ETR (using M1-32) on a stick Default gateway for the moved traffic

Does not receive any traffic before the move

PxTR

Enable LISP on a stick

69

Packet Flow from Client & Server in Brownfield North-South Traffic

Traffic to a non moved resource does not reach LISP nodes on a stick

70

Symmetric Packet Flow from Client & Server in Greenfield North-South Traffic

Existence of a Firewall between WAN edge & PxTR requires symmetrical flow

Use PeTR

PeTR allows return flow to go thru LISP Path, nevertheless it requires ETR to work with default routing

PeTR

71


• There are three mechanisms to handle convergence

‒Route watch / Route notification

Mandate RLOC /32 to be received remotely

This /32 must not be part of an aggregated route

As fast convergence as the routing protocol

The one used in this solution

‒EID/RLOC probing

Probes every EID

60s convergence

‒LSB bits

Data-plane bits indicating local RLOC status

Not supported with M1-32

Convergence Considerations All Failures are Leading to Sub 3s Convergence with IGP/BGP tuning

72






IP Based Solutions


L3 Host Mobility using LISP

LISP as L3 DCI

• Summary and Q&A

Agenda

73


Data Center Interconnect - DCI Model Connecting Virtualized Data Centers

74

L2 Domain Elasticity - LAN Extension

VN-link notifications

Path Optimization - Optimal Routing - Route Portability

Storage Elasticity - SAN Extensions

OTV

OTV

OTV

OTV

Sync or Async replication modes are driven by the applications, hence the

distance/latency is a key component to select the choice

Localization of Active Storage is key Distance can be improved using IO accelerator or caching

Virtual LUN is allowing Active/Active

STP Isolation is the key element Multipoint Loop avoidance + Storm-Control

Unknown Unicast & Broadcast control Link sturdiness Scale & Convergence

Considerations Network and Security services deployment Server-Client Flows Server-Server Flows Path Optimization Options Egress

Addressed by FHRP Filtering Ingress:

Addressed by LISP


Data Center Interconnect Where to Go for More Information

75

http://www.cisco.com/go/dci

http://www.cisco.com/en/US/netsol/ns749/networking_solutions_sub_program_home.html








Recommended Reading for BRKDCT-3060

76 76 76


Call to Action

• Visit the Cisco Campus at the World of Solutions to experience Cisco innovations in action

• Get hands-on experience attending one of the Walk-in Labs

• Schedule face to face meeting with one of Cisco’s engineers

at the Meet the Engineer center

• Discuss your project’s challenges at the Technical Solutions Clinics


Date post:	29-Jun-2018
Category:	Documents
Upload:	duongnhu
View:	252 times
Download:	2 times

Deployment Considerations with Interconnecting Data...

Documents