SDN with Cisco ACI or EVPN · SDN with Cisco ACI or EVPN Robert Turnšek Senior IT Arhitect NIL

SDN with Cisco ACI or EVPN

Robert Turnšek

Senior IT Arhitect

NIL

© 2018 NIL, Security Tag: SECURED 2

Why even bother?


Network Infrastructure

Reality Full/partial meshed topology

1GE and some 10GE

Non-deterministic latency

L2 domains spread across and between DC locations

Multiple „contextualized“ and stretched physical firewalls

Wishes Speed deployment of new segments (minimize configuration)

Securely separate applications and environments (TEST, DEV, PROD, …)

Isolate core network from edge

Distribute/scale internal security


Network Deployment TO-DO List

Multiply by the number of switches...

Create VLANs

Configure physical links/interfaces

Configure trunks (allowed VLANs)

Create IP interfaces

Configure FHRP (HSRP, anycast GW,…)

Update routing (OSPF, static, …)

ACLs

Apply ACL

…


Multitenancy

Challenges

Control of resource sharing

Security

Creation & removal

Scaling

Solution

Dedicated infrastructure

Shared infrastructure

Examples

TEST/DEV/PROD

Customers

Applications


Redesign >> Redeploy >> Redesign …


Application Architecture

DMZInternet

Internal

Users (WAN & LAN)

Users (via Internet)

Edge FW

Core FW

Data Center

Internet

VDSvCenter

SQL DBsComposer

ThinApp Infra vCenter

LB

...

infrahost01 infrahosti02 infrahost0Y

Infra Cluster

...

User communication

Infra/mgmt communication

VUM

VDI Cluster

...vdihost01 vdihost02 vdihost0Y

RDS Cluster

...rdshost01 rdshost02 rdshost0Y

RDS Servers ...

VDIs


Application Traffic Profiles - Blueprints

Internet

Users InetRTR EdgeFW Sec.Conn.

Server Farm

EdgeFW LB Connection

Server Farm

VDS Pool DFS

Cluster

GW GWCoreFW GW

External &Mobile Users

Internal Users

Users WAN/LAN LB Connection

Server Farm

VDS PoolGW GW DFS ClusterCoreFW GW

GW CoreFW GW

Infra vCenterInfra cluster x

Infra

Admin

DFS Cluster

Infrastructure Management & Services

Composer

GW CoreFW GW

VDS

vCenter

VDI cluster x

VDS

Admin

ThinApp

DFS Cluster

RDS cluster x

Sec.Conn.Server Farm

VDS Administration & Provisioning

EdgeFWConnection

Server Farm

GW AD/DNSInternal

Cluster

DNS Clients –

vCenters, ESXi, ...

SQL DBsMS SQL Clients – GW

vCenters, Composer,...


Service Insertion

Service insertion options

Virtualized service appliances

Transparent (L2)

Gateway on core switch

FW/LB in transparent mode

Clients L3Gateway

L2Appliance

SRV

VIPs, Real IPs

VLAN Stitching

Clients L3Gateway

SRV

Separate VLANs

Subnet XSubnet Y

Routed (L3)

Gateway on service appliance

FW/LB in routed mode


Cisco ACI, EVPN

Differences Controllers

Environment (virtual vs. physical/mixed)

Built-in automation

Network services insertion/integration

Distributed firewall (filtering)

Similarities VXLAN overlays

Routed network fabric

Distributed gateway

Leaf-spine for equidistant end-points

Keep predictable latency

Physical connectivity (1/10/40...)

Scaling


Lifecycle

Design (day-0) Sizing

Support

Future requirements

Implementation (day-1) Pre-requisites

Implementation specifics

Maintenance (day-2) Upgrades (dependencies !!!)

Scaling

Monitoring & alerts


Cisco ACI


ACI Components

SpineSwitches

Nexus 9500 with 9700 linecards

Nexus 9300 spine

LeafSwitches

Nexus 9300

1/10/25/40/100 GE

Copper or fiberAPIC ControlCluster

APIC controllers

Medium – up to 1000 edge ports

Large – more than 1000 ports

...


ACI Deployment

PHASE 1Prepare environment

Design ACI

Implement physical fabric

Initialize fabric

Integration

PHASE 2Migration to ACI fabric

Implement ACI objects

Map workload to ACI object(s)

Implement IP subnets

Implement fabric access policies

PHASE 3On-going changes

Add/change ACI objects

Scale fabric

Change policies

...1

23

Initialize SeedAPIC

Cluster configuration ...Enter the fabric name [ACI Fabric1 #1]: aciPrimaryEnter the number of controllers in the fabric (1-16) [3]:Enter the controller ID (1-3) [1]:Enter the controller name [apic1]: ssdcAPIC1Enter address pool for TEP addresses [10.0.0.0/16]: 172.16.0.0/16Enter the VLAN ID for infra network (1-4094)[]: 3067Enter pool for BD multicast addresses (GIPO) [225.0.0.0/15]:

Out-of-band management configuration ...Enter the IP address for out-of-band management: 10.176.109.51/24Enter the IP address of the default gateway [None]: 10.176.109.1Enter the interface speed/duplex mode [auto]:...

DEPLOY


ACI Configuration Aspects

......

Access Policies

Fabric Policies

Tenant X

VRF A

Subnet A

Bridge Domain 1

Subnet B

EPG1A

EPG1B


ACI Objects

Tenant 1

root

Private Network 1 Private Network X

Bridge Domain 1-1

10.1.1.1/24

10.1.2.1/24

EPG1-1A

EPG1-1B

EPG1-1C

Bridge Domain 1-2

10.2.0.1/16

EPG1-2A

EPG1-2B

Bridge Domain X-1

172.16.1.1/24

EPGX-1A

EPGX-1B

10.254.1.1/24

Customer/BU/Group/...

Private Routed Domains

L2 Boundary

IP Space(s)

Application Network Profiles

172.31.1.1/24

Tenant X Mgmt Infra Common

... ...


ACI Fabric Design

ACI fabric

Initialization parameters

Management

VTEP IP pool

Infra VLAN

GIPo

Physical topology

Compute, border, service leaves

Workload communication

Tenants, VRFs, BDs, EPGs

Define security (black/whitelist mode)

External connectivity

Workload connectivity

Access policies

VLAN pools, domains, …

Access ports > VLANs using policies

DEPLOY


ACI Topology Design

APIC Node ID Mgmt IP Connected to Leaf

apic01 1 10.1.1.1 1st leaf, 2nd leaf

apic02 2 10.1.1.2 2nd leaf, 3rd leaf

apic03 3 10.1.1.3 3rd leaf, 4th leaf

Tenant ID Description

tnt1 1 Tenant1 leaf nodes

tnt2 2 Tenant2 leaf nodes

common 0 Shared leaves

Spine Node ID Mgmt IP

spine01 101 10.1.1.51

spine02 102 10.1.1.52

Dual-fabric design

• Redundant leaf pairs

• Fabric A – denoted with 1

• Fabric B – denoted with 2

...A B A B

101 102

1 2 3

Type Fabric A Node ID

Hostname Fabric B Node ID

Hostname Max leaf switch qty.

Description

compute U1y leafU1y U2y leafU2y 180 Only servers are connected, limit to 180 (last node sequence ID90)

border U31x leafU31x U32x leafU32x 20 Only border devices – routers

service U41x leafU41x U42x leafU42x 20 Only service appliances – FW, LBs

mixed U91x leafU91x U92x leafU92x 20 Mixed – routers, FWs, LBs, ...


Designing Access Policies

Switch

Interface VLAN, VXLAN, .... pools VMM integration >> dynamic VLAN pool Blocks/ranges

Attach pool Associate AEP(s)

Switch Policies

VMM IntegrationPhysical Domains

DESIGN PROCESS- Objects/Policies- Properties- Relations- Checklists ?

Pools

Define VMM domain(s) Built-in VSW(VMware DVS) or AVS? Attach pool Associate AEP(s)

Leaf switch policies (STP, vPC, ...) Leaf switch policy groups Leaf switch profiles Bind leaf selectors & interface profiles

Interface Policies

Interface policies (L2, LLDP, pc, ...) Leaf interface policy groups Leaf interface selectors Leaf interface profiles Bind interface selectors with policy groups and profiles Bind AEP to interface policy group

Access Entity Profiles

VMM integration > Infrastructure VLAN? Attach domains – physical/VMM/ext Attach specific EPGs

Physical/VMM Domain

Interface Policy Group

Interface Profile

Interface Selector

Switch Policy Group

Switch Profile

Leaf Selectors

Pool

Access Entity Profile

Design Document


Designing ACI Objects

Single vs. multiple? Dedicated vs. everything in Common ? Specific monitoring policy? Who should manage it?

DESIGN PROCESS- Objects- Properties- Relations- Checklists

Single vs. multiple? Should policy control be enforced? If used, which direction? Specific monitoring policy? Do routes need to be tagged?

Single vs. multiple? Should forwarding be customized (e.g. ARP flooding)? Should unicast routing be disabled? Will BD MAC address be manually configured? What IP subnets are defined at the BD level? Should IP learning be limited to subnet only? Do subnets have to be announced to external domains? Is specific monitoring required?

Gateway IP – should it be virtual IP? Network scope – VRF bound, shared or advertised

externally? Any route profiles attached?

Tenants VRFs BDs

SubnetsEPGs

Application Profile? QoS class? Will micro segmentation be used (uSeg EPGs)? Does isolation need to be enforced within EPG? Do we have info to enforce contracts/filters for communication? Are contracts reused and we need to prevent transitive

communication?

Contracts

Scope – AP, VRF, tenant, global? List subjects that should be used Where and how will contract be reused? Will taboo contracts be used?

Subjects

Apply in both directions? Reverse filter ports List filters to be used

Filters

Define entries Ether type, ARP flag, IP protocol, stateful, src/dst to/from ports, ...

DesignDocument

?


Security Policy Example

Traffic Profile

BKPProxies

BKPServer

vCenter

Cluster 1

BKP Targets

Filter Entries

flt-ssh ent-ssh, IP, tcp, dst port 22

flt-t443 ent-t443, IP, tcp, dst port 443

ftl-t902 ent-t902, IP, tcp, dst port 902

flt-bkptrgt ent-t135, IP, tcp, dst port 135 ent-u135, IP, udp, dst port 135ent-t137-9, IP, tcp, dst port range 137- 139ent-u137-9, IP, udp, dst port range 137-139

ent-t445, IP, tcp, dst port 445 ent-u445, IP, udp, dst port445

TCP 22

TCP902

bd

-in

fra-1

vCenter ESXi cluster X

epg-cluster1

10.1.1.1/24

10.1.2.1/24

10.2.1.1/24

10.2.2.1/24

epg-veam-pxyepg-veam-bksr

BKP Server BKP Proxies

epg-vc

Contract Common properties Subject list

ct-bkpsrv Apply both directions Reverse fitler ports

sbj-ctrlch, flt-sshsbj-bkprst, flt-t443 sbj-datatrsf, flt-t902

ct- bkprxy Apply both directions Reverse fitler ports

sbj-bkprst, flt-t443 sbj-datatrsf, flt-t902 sbj-bkptrgt, flt-bkptrgt

Parameter ValueContract common parameters

Scope VRF

TCP 443

TCP 902

ct-bkpsrv ct-bkppxy

epg-bkptrgt

BKP Targets


Virtual Infrastructure Deployment Options

No Integration Static VLAN pool Physical domain Static path assignment Any virtual switch

...

Cluster 1 Cluster 2

vCenter1

...

...

...vCenter2

APIC Cluster

node1 node2 nodeX

dvsInfra

vSw0 vSw0 vSw0

...

pg1 pg2

node1 node2 nodeX

dvsACI

vSw0 vSw0 vSw0

...

tnt1|apVDS|epgVDI1 tnt1|apVDS|epgVDI2


Virtual Infrastructure Integration Design

? vSwitch orDVS No/single/multi-hop transit infrastructure

possible Cluster wide VLAN consistency (port-

groups) Static VLAN pool

- EPG = VLAN (administratively assigned) Static path selection (physical port) MAC pinning = route based on virtual port

OpFlex used Enable infra VLAN to hosts Multi-hop transit infrastructure

possible- Pre-configure infra VLAN- DHCP relay for AVS VTEP vmk

ACI Admin Design physical connectivity Design access policies (vPC, PC, ...) Define APIC to VMM integration Design application policies (BD, EPG, contracts, ...)

- Policy resolution – pre-prov/Immediate/on-demand

VMM Admin Attach hosts to VDS Assign VMs with desired port-groups

IntegrationNo

? Transit switch infrastructure? DVS orAVS? Load-balancing? VLAN or VXLAN for EPG

Yes

Dynamic VLAN pool Max single-hop transit infrastructure

- Pre-configure dynamic VLANs- Enable LLDP/CDP

VDS – local switching MAC pinning EPG = VLAN (assigned by APIC)

Dynamic VLAN pool If transit infrastructure used

- Pre-configure dynamic VLANs VDS – local switching

Multicast pool VDS – local or no-local switching


EVPN


EVPN Components

Fabric - leaf and spine switches


EVPN Deployment

Prepare environment

Design EVPN

Internal IP addressing

VLANs, VXLANs for workload „addressing“ plan

Fabric & workload physical & logical interfaces

Routing - internal, reachability, external

Implement physical topology

Implement internal, reachability, external routing

Migration to EVPN fabric

Configure workload facinginterfaces

Create VLAN(s)

Create VLAN-to-VXLAN mapping

Enable anycast gateway

On-going changes

(Re)configure workload ports

Add segments (VLAN, VXLAN)

Update routing (any)

PHASE 1 PHASE 2

PHASE 3

DEPLOY


EVNP Fabric Design

Leaf & Spine PhysicalTopology

• Fabric links &oversubscription

• Workload physicalconnectivity

• Internal fabric routing & IPaddressing

• Dynamic routing, loopbacks

• Redundancy

• Reachability information

• VLAN & VxLANaddressing & mapping

• External routing

• L2

• L3 with default gateway

• L3 with dynamic routing

Logical Fabric

Connectivity requirements

DESIGN


EVPN Internal Routing

DWDM

40Gb/s

mx40Gb/s

40Gb/s – vPC

10Gb/s –DWDMISIS – area0010.0091.0001 ISIS – area0010.0091.0002

Spine-01Spine-02Spine-01 Spine-02

DC site #1 DC site #2

Access-01 Access-02 Access-03 Access-04 Edge-01 Edge-02 Leaf-01 Leaf-02 Leaf-01 Leaf-02 Edge-01 Edge-02


VXLAN Overlays

40Gb/s

mx40Gb/s

40Gb/s – vPC

10Gb/s –DWDM

VXLAN „overlay“

DWDM

Services networks

Services network

LAN/User access networks



Access-01 Access-02 Access-03 Access-04 Edge-01 Edge-02 Leaf-01 Leaf-02

Compute networks

Leaf-01 Leaf-02

Compute networks

Edge-01 Edge-02

INETFW VPNFW ESXi physical ESXi INETFW VPNFW

hosts hosts hosts


Distributing Reachability Information

DWDM

40Gb/s

mx40Gb/s

40Gb/s – vPC

10Gb/s –DWDMMP-iBGP – AS65001 MP-iBGP – AS65002

MP-eBGP



Access-01 Access-02 Access-03 Access-04 Edge-01 Edge-02 Leaf-01 Leaf-02 Leaf-01 Leaf-02 Edge-01 Edge-02


Security & Virtualization integration

Access-list between L3 segments ? Just static access-list

EVPN is distributed -> multiple points to configure

Virtualization integration No integration -> similar to ACI with no integration

Static VLAN pool Physical domain Static path assignment Any virtual switch


A word about automation


Automation Options

Migration

Scripting for flexibility

Collect current configuration

Large roll-out

PowerShell/Python + REST API

VMware vSphere

Physical network

Operations

Orchestration

Define and use workflows/blueprints for well-defined & repetitive processes

Deploy new tenant/clone tenant

Decommission tenant


Cisco ACI Deployment TO-DO

1. Add leaf switch Implement physical connectivity

2. Confirm it in the ACI management portal

3. Connect workload Configure physical port


EVPN Deployment TO-DO

1 . Add leaf switch

Implement physical connectivity

Add BGP neighbor

2. Add network segment

Add VLAN, VXLAN and mapping

Update BGP routing

L2 only connectivity

L3 anycast gateway

L3 with OSPF routing

3. Connect workload

Configure physical port

Apply configuration usingcustom scripts

CSV + XML template + REST-functions>> config

Leaf Switch Configvlan ###VLANID###

name ###VLANname###vn-segment ###VXLANID###

!interfacenve1member vni ###VXLANID###

supress-arpingress-replication protocol bgp

!evpn

vni ###VXLANID### l2rd autoroute-target import 65000:###VXLANID### route-target export 65000:###VXLANID###


Summary


Summary

Technology by itself does not solveproblems

Applying technology appropriatelydoes

Design is crucial

Evolution“ based approach Conducted in phases

PoCs, testing, …

„It‘s all about delivery“

Date post:	25-Sep-2018
Category:	Documents
Upload:	phungkhuong
View:	246 times
Download:	5 times

SDN with Cisco ACI or EVPN · SDN with Cisco ACI or EVPN Robert Turnšek Senior IT Arhitect NIL

Documents