Date post: | 29-Mar-2018 |
Category: |
Documents |
Upload: | truongkhuong |
View: | 220 times |
Download: | 5 times |
Migrating Your Existing WAN to Cisco’s IWAN
Brad Edgeworth, CCIE#31574, Systems Engineer @BradEdgeworth
BRKCRS-2007
Mani Ganesan, CCIE#27200, Consulting Systems Engineer @Mani_Cisco
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Introduction
• Who we are?
• “Advanced” Class
This is not an ‘Introduction to IWAN’ session
This is not an ‘IWAN Design’ session. Some design aspects will be discussed
This session is about how to migrate your existing WAN to Cisco’s Intelligent WAN
A lot of things will technically work, but IWAN is prescriptive design.The design keeps thing simple…..
This session is focused primarily on transport independence and performance routing. Specifically how to deploy it.
We tried to keep things in a logical order as much as possible, but there are some couldn’t; so STAY AWAKE!
Housekeeping
For yourreference only
Preferred or Recommended
BRKCRS-2007 3
• Sequence of Migration
• Migration Planning and Tools
• End State IWAN Concepts:
• QoS
• DMVPN and Routing
• DMVPN Hub Router Placement Strategies
• Migrating Branch Routers
• Other Migration Scenarios (Dual MPLS Hybrid Model Migration, IPsec Migration)
• Performance Routing (PfR)
BRKCRS-2007: Migrating Your Existing WAN to Cisco’s IWAN
Introduction
Intelligent WAN Solution Components
Internet
Branch
3G/4G-LTE
AVC
MPLS
PrivateCloud
VirtualPrivateCloud
PublicCloudWAAS PfR
Application Optimization
Secure Connectivity
• Certified strong encryption
• Comprehensive threat
defense with ASA and IOS
firewall/IPS
• Cloud Web Security (CWS)
for scalable secure direct
Internet access
Intelligent Path Control
• Application best path based
on delay, loss, jitter, path
preference
• Load balancing for full utilization
of all bandwidth
• Improved network availability
• Performance Routing (PfR)
TransportIndependent
• Consistent operational model
• Simple provider migrations
• Scalable and modular design
• DMVPN IPsec overlay design
• AVC: Application monitoring
with Application Visibility and
Control
• WAAS: Intelligent Edge Caching
with Akamai Connect
• WAAS: Application Acceleration
and bandwidth savings
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Where to start ?
IWAN is not all or nothing – so deploy in phases if that’s easier
DIA and App Optimization ( WAAS and Akamai ) can be deployed anytime during the process.
Start with transport independence before adding path control - DMVPN is needed to run Performance Routing (PfRV3)- Provides us consistent overlay routing across all transports
This session is focused on Transport Independence, PfR and Connectivity. This matters the most during migration
BRKCRS-2007 7
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN Topology• Lan Prefixes:
• 10.0.0.0/8 (Site Location is 2nd Octet)
• HQ is 10.1.0.0/16 & 10.2.0.0/16
• Remote Sites:
• 10.3.0.0/16
• 10.4.0.0/16
• 10.5.0.0/16
• DMVPN Hub Routers
• R11 & R21 MPLS Transport
• R12 & R22 Internet Transport
• Transport:
• 172.16.0.0/16 MPLS
• 100.64.0.0/16 Internet
DC2DC1
BRKCRS-2007 8
Planning the Migration
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Mastering The MigrationPeople + Process + Technology
• Avoid implementation that doesn’t map back to logical design determined necessary to address key requirements.
• Must have strong understanding of current state environment to ensure implementation success
..
BRKCRS-2007 10
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Why Migration Planning is critical ?
• Moving all branch traffic from underlay to Overlay tunnels Can be complicated
• WAN Migration may last for weeks for months
• Need to Maintain Universal connectivity between legacy and IWAN sites that are migrated
• Choose the right sites to act as migration sites ( during migration phase ) – based on circuit speeds and device capacity
• What is being migrated? All Branches or leaving some sites on the legacy WAN?
BRKCRS-2007 11
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Where Do We Start Our IWAN Migration?
Gather Information and document them
• Inventory
• Licenses
• Software Version
• Top applications with AVC
• Existing Routing Design
• QoS Design
• Sites with Backdoor Links
BRKCRS-2007 12
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Carrier 2
VPN
Carrier 1
VPN
Internet
Internet
Internet
Capacity Management - WAN/Backbone
WAN Interface
Utilization >60%
Dropped Packets > 1%
Delay > 1
WAN Interface
Utilization >75%
Dropped Packets > 5%
Delay > 2 WAN Interface
Utilization >60%
Dropped Packets > 1%
Delay > 1 opco STATE_PROVINCECITY NetworkElementName ProductID capacity maxdelay mindelay rxavgutil rxbusy4avgutil txavgutil txbusy4avgutil
FXE CO DENVER BKFArspm01 CISCO2821 1.54MB 2964 36 10 31 0 0
5 16 3 7
COTotal
GA MACON MCNArm01 CISCO2811 1.54MB 2016 19 9 23 1 3
GATotal
MA SOUTHBOSTON BVYArm01 CISCO2851 1.54MB 3089 35 10 24 0 0
8 21 5 9
MATotal
FXF TN MEMPHIS MEM-2811-SPRINT CISCO2811 1.54MB 3906 6 13 30 1 3
MEM-2811-VOIP-ATT CISCO2811 1.54MB 3897 6 22 39 5 8
3.07MB 3897 6 22 39 5 8
TNTotal
BRKCRS-2007 13
Capacity Management - Branch
Branch Optimization Analysis
c881#show flow monitor FLOWMON cache agg app name
Processed 32 flows
Aggregated to 9 flows
APP NAME flows bytes pkts
============= ========== ========== ==========
prot icmp 1 4272 12
port http 4 7981530 8242
port netbios-ns 1 1794 23
cisco unclass 14 636420 1320
port ms-wbt 4 407184 506
port ssh 1 14352 198
cisco dhcp 1 328 1
port dropbox 4 1216 6
port isakmp 2 58 2
SiSi
Core/Dist Switches
Access Switches
AT&T/SPRINT
MPLS
WLC
APs
Internet
SiSi
SiSiSiSi
HDTV
IP Desktop Video
Video Conferencing
PC
Surveillance Camera
Signage
VVVVMedia Gateway
Cache Engine
Branch Optimization Analysis
Mon 21 Oct 2013 01:16 PM – ATL-xxx
Input Output
----- ------
Protocol 5min (bps) 5min (bps)
5min Max (bps) 5min Max (bps)
------------ --------------- ---------------
exchange 0 120000
2811000 1958000
skype 0 0
2678000 1879000
rtp 0 0
1595000 966000
ftp 0 0
2147000 61000
h323 0 0
1152000 569000
edonkey 0 0
810000 750000
Total 1409000 469000
30711000 16394000
BRKCRS-2007 14
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Capacity Management – Branch NBAR View
BU3 (top 10 apps) – 3Mbps sites Max bps (input) * Max bps (output) * ObservationsHTTP 2.9Mbps 2Mbps Bandwidth HogSkype 2.4Mbps 2.2Mbps Unauthorized App/Bandwidth HogExchange 2.7Mbps 1.6Mbps Bandwidth HogFTP 1.9Mbps negligible High Bandwidth UsageeDonkey 1Mbps 1Mbps Unauthorized/High Bandwidth UsageRTP 1.3Mbps 750Kbps High Volume/High Bandwidth UsageNovadigm 1.1Mbps 400Kbps InvestigateSkinny 1.6Mbps negligible High Volume/High Bandwidth UsageFasttrack 700Kbps 270Kbps Unauthorized/High Bandwidth Usage
Citrix 1.2Mbps negligible High Bandwidth Usage/Monitor Latency
BU1 (top 10 apps) – 3-6Mbps sites Max bps (input) * Max bps (output) * ObservationsSYSLOG negligible Max Capacity Bandwidth HogHTTP Max Capacity 1Mbps Bandwidth HogSecure HTTP Max Capacity 600Kbps Bandwidth HogIMAP 950Kbps 700Kbps High Bandwidth UsageSMTP 30Kbps 800Kbps High Bandwidth UsageExchange 1.7Mbps 400Kbps High Bandwidth UsageSkype 600Kbps 1.2Mbps Unauthorized/High Bandwidth UsageeDonkey 250Kbps 600Kbps Unauthorized/High Bandwidth UsageCitrix 450Kbps 200Kbps Monitor LatencyXwindows 500Kbps 500Kbps Check Security Impact
Depending on the type of network traffic, DIA deployment could be accelerated.
BRKCRS-2007 15
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Application Profile (Branch)
Application
Weekly
Average Kbps
Daily Average
Kbps Peak Kbps Average Delay Max Delay Voice/Video Variance Classification
http 748 2271 9276 37ms 9s Transactional
secure-http 436 710 2410 40ms 3s Transactional
ssl 292 4695 38ms 3s Transactional
outlook-web-service 128 2292 55ms 3s Transactional
ldap, cifs, active-directory, sqlnet 60 81 2804 33ms 212-332ms Transactional
sqlserver 4 23 1387 28ms 68ms Transactional
share-point, ms-office-web-apps, ms-office-365, ms-
update, oracle-sqlnet, sap 3 3 477 35ms 36-84ms Transactional
rtp 6 13 93 30ms jitter (97% within) Voice
ms-lync 0 49 59ms 124ms Voice
webex-meeting, h323 1 42 Interactive Video
sip-tls, skinny, rtsp, mgcp, rtcp, rsvp 2 89 VoIP Control
youtube 132 3201 35ms 2s Streaming Video
unknown 340 2104 17608 37ms 3s Bulk
amazon-instant-video, rtmpt, amazon-web-services,
flash-video 211 5530 35ms 52ms Bulk
video-over-http 101 5250 35ms 48ms Bulk
binary-over-http 80 2355 38ms 11s Bulk
facebook, gmail 54 1289 35ms 104-115ms Bulk
itunes 10 5695 96ms 3s Bulk
audio-over-http 6 3614 34ms 40ms Bulk
BRKCRS-2007 16
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN/Offload Application BenefitsClassification* Branch Traffic Volume PfR Primary Path Offload Option
VOICE 151 Kbps MPLS N
VOIP CONTROL 42 Kbps MPLS N
INTERACTIVE_VIDEO 89 Kbps MPLS N
STREAMING_VIDEO 3778 Kbps INET Y
TRANSACTIONAL_DATA 1711 Kbps MPLS Y (Selected Cloud Apps)
BULK_DATA 776 Kbps INET Y
• IWAN will provide distinct paths to improve the application performance for key transactional and voice/video apps, redirecting bulk and streaming video to the
alternate Internet backhaul path
• CWS and direct offload will then allow cloud apps and general Internet traffic to be directly offloaded avoiding backhaul bandwidth expense
BRKCRS-2007 17
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Migration steps
• Finalize the Design
• Deploy IWAN via a POC or Production Pilot
• Learn the technology
• Learn the applications
• Test the migration strategy
• Collect results from any POC/Production Pilot
• Identify sites for migration
• Make changes to infrastructure (if H/W upgrades are needed)
• Hub deployment
• Cut-Over Branches
• Clean-Up
BRKCRS-2007 18
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Tools to simplify Deployment and Migration
• Application Policy Infrastructure Controller (APIC-EM)
• Prime Infrastructure IWAN Workflow
• CLI
BRKCRS-2007 19
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco Intelligent WAN App for APIC-EM
Business Policy Dictates Network Action
IT Admin
Business
Policy:
App SLA
APP DMVPN
SLA
QoS
Security
Path
Selection
Access Application
Network Profile
NETWORK
SDN
Simple Workflow
Templates
Zero Touch
ProvisioningBusiness
Level Policies
Open
Architecture
Network, Applications
Monitoring
BRKCRS-2007 20
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Launch the IWAN workflow
from the new Converged
Menu
How can I easily connect new sites to the data center and
enable the IWAN technologies ?
Cisco Prime IWAN WorkflowsSimplifying Configuration and Deployment
BRKCRS-2007 21
End State IWAN Concepts
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Dynamic Multipoint VPNTunneling Technology that uses:mGRE, NHRP, and IPsec.
• Zero-touch provisioning
• Scalable Deployment
• Dynamic Spoke-to-Spoke Communication
• Spoke-to-Spoke Tunnels requires traffic to hair-pin on the Hub tunnel interface
• Provides Transport Independence
R11
R41
R51
DMVPN Hub
DMVPN Spoke
DMVPN
SpokeDMVPN
Spoke
R31
BRKCRS-2007 23
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
DMVPN Spoke-To-Spoke Tunnel Creation
Traffic has
hairpinned on my
DMVPN tunnel1
2
3
4
Traffic has
hairpinned on my
DMVPN tunnel
BRKCRS-2007 24
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
DMVPN Spoke-To-Spoke Tunnel Creation (continued)
4
5
6
7
BRKCRS-2007 25
End State IWAN Concepts:Quality of Service
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Need for QoS from IWAN Perspective
• Replacing expensive MPLS service with business class internet
• PfR to load balance / provide resiliency / best path
• DMVPN overlay on MPLS and Internet
• Up to 2,000 remote sites per hub router in a single domain
• MPLS transport will have SP QoS, but with Internet transport we assume none
BRKRST-2043 IWAN AVC-QoS Design
BRKCRS-2007 27
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Hub
BR
T1
Branch
T1
Branch
T3
Branch
10 Mbps
Branch
T3
Branch
IWAN QoS Requirements
80 Mbps
1.5 Mbps
1.5 Mbps
45 Mbps
10 Mbps
45 Mbps
Service
Rate
GE
Shape for
Service Rate
Per Site
Bandwidth Sharing
Within Tunnel
Shape for
Remote Site
Last Mile
Bandwidth Sharing
Between Tunnels
BRKCRS-2007 28
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
CE
CE
CE
CE
CE
CE
CE
CE
CE
CE
802.1q
trunk
100 Mbps
50 Mbps
50 Mbps
20 Mbps
20 Mbps
10 Mbps
10 Mbps
Shape only(100 Mbps)
100 Mbps in to DMVPN cloud can easily
overrun the lower speed committed rates at
spoke sites
• Per-Site Shaping to Avoid Overruns
• Hub to spoke only
DMVPN Per Tunnel QoS
BRKCRS-2007 29
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Per-Tunnel QoS• Tunnels created from Hub to Spoke sites will have QoS applied per-tunnel
• Pre-configured QoS policy applied to the tunnel based on NHRP Group name
passed from Spoke to Hub
• Although many spokes can be put into the same NHRP group, the tunnel traffic
for each spoke is measured individually for shaping and policing.
• Per-tunnel QOS policy controls only Hub to Spoke traffic, it is not bidirectional
- Branches run their own QOS policies from spoke side
BRKCRS-2007 30
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
policy-map POLICY-TRANSPORT-1-SHAPE-ONLY
class class-default
shape average 100000000
!
interface GigabitEthernet0/0/3
bandwidth 100000
service-policy output POLICY-TRANSPORT-1-SHAPE-ONLY
interface Tunnel10
nhrp map group RS-GROUP-10MBPS service-policy output RS-GROUP-10MBPS-POLICY
nhrp map group RS-GROUP-20MBPS service-policy output RS-GROUP-20MBPS-POLICY
nhrp map group RS-GROUP-50MBPS service-policy output RS-GROUP-50MBPS-POLICY
policy-map RS-GROUP-50MBPS-POLICY
class class-default
shape average 50000000
service-policy WAN
Separate shaper policies for
each remote-site bandwidth
DMVPN Hub Per Tunnel QoSImplementing Per-Site Traffic Shaping
policy-map RS-GROUP-20MBPS-POLICY
class class-default
shape average 20000000
service-policy WAN
policy-map RS-GROUP-10MBPS-POLICY
class class-default
shape average 10000000
service-policy WAN
Add a class-default shape-only policy on the hub physical interface
interface GigabitEthernet0/0
bandwidth 10000
service-policy output POLICY-TRANSPORT-1
!
interface Tunnel10
bandwidth 10000
nhrp group RS-GROUP-10MBPS
tunnel source GigabitEthernet0/0
tunnel vrf IWAN-TRANSPORT-1
interface GigabitEthernet0/0
bandwidth 20000
service-policy output POLICY-TRANSPORT-1
!
interface Tunnel10
bandwidth 20000
nhrp group RS-GROUP-20MBPS
tunnel source GigabitEthernet0/0
tunnel vrf IWAN-TRANSPORT-1
interface GigabitEthernet0/0
bandwidth 50000
service-policy output POLICY-TRANSPORT-1
!
interface Tunnel10
bandwidth 50000
nhrp group RS-GROUP-50MBPS
tunnel source GigabitEthernet0/0
tunnel vrf IWAN-TRANSPORT-1
Spoke Tunnel Configurations
10 Mbps spoke
20 Mbps spoke
50 Mbps spoke
Shape(100 Mbps)
50 Mbps
50 Mbps
20 Mbps
20 Mbps
10 Mbps
10 Mbps
Per tunnel shapers
Parent
shaper
Signal from the
spoke to the hub
to use the correct
policy for each
remote site
List all available policies as map groups on hub tunnel interface
BRKCRS-2007 31
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IPSec Anti-Replay
Crypto Engine
(Adds Sequence
Number)
22
Packets In
P1
priority data class-default
Police
2123 Enqueue
2426
2223
21
26 242227 21
Packets Out
25Dropped
By Policer27 28
Queue
Tail Drop
23
• Decryption side keeps a sliding history of packets
received (default is 64 packets)
• Provides anti-replay protection against an attacker
duplicating encrypted packets
• Increasing the anti-replay window size has no impact on
throughput or security
• The impact on memory is insignificant because only an
extra 128 bytes per incoming IPsec SA is needed
IWAN Conclusion: Use the maximum replay
window-size of 1024 for each supported platform
crypto ipsec security-association replay window-size 1024
BRKCRS-2007 32
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
PfR Policies rely on QOS marking
• Create the PfR classes with matching policy names and DSCP values to simplify the configuration
• Define the path preference for traffic
• Load balance non-priority traffic
domain IWAN
vrf default
master hub
load-balance
class VOICE sequence 10
match dscp ef policy voice
path-preference MPLS fallback INET
class INTERACTIVE_VIDEO sequence 20
match dscp cs4 policy real-time-video
match dscp af41 policy real-time-video
match dscp af42 policy real-time-video
match dscp af43 policy real-time-video
path-preference MPLS fallback INET
class LOW_LATENCY_DATA sequence 30
match dscp cs2 policy low-latency-data
match dscp cs3 policy low-latency-data
match dscp af21 policy low-latency-data
match dscp af22 policy low-latency-data
match dscp af23 policy low-latency-data
path-preference MPLS fallback INET
IWAN Master Controller
class BULK_DATA sequence 40
match dscp af11 policy bulk-data
match dscp af12 policy bulk-data
match dscp af13 policy bulk-data
path-preference MPLS fallback INET
class SCAVENGER sequence 50
match dscp cs1 policy scavenger
path-preference INET fallback MPLS
class DEFAULT sequence 60
match dscp default policy best-effort
path-preference INET fallback MPLS
BRKCRS-2007 33
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
• QoS is based upon the following logic:
• Ingress traffic is classified and marked accordingly (if not done elsewhere)
• Egress traffic is shaped/queue based on QoS marking
• PFR maps traffic to classes based on the DSCP marking or application names. LAN Traffic should be marked on Ingress or before hitting the BRs
• As a best practice, use the same class names in PFR that were used for the QoS policies. Match DSCP for each PfR class with the DSCP used for the QoS policies.
• Ensures DSCP is consistent between QOS and PFR policies
• Makes it easier to identify the PFR policies
QOS settings for PFR
BRKCRS-2007 34
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
The Diffserv class view is preserved across the enterprise even though we are treating it differently in the router and sending it to different channels within the SP network.
The classes remain intact on the inner header and the outer header is discarded after leaving the tunnel interface
Enterprise to SP QoS Mapping
BRKCRS-2007 35
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Video Flow from
Term-A To Term-B
class-map match-all MULTIMEDIA_CONFERENCING-NBAR
match protocol attribute traffic-class multimedia-conferencing
match protocol attribute business-relevance business-relevant
!
policy-map traffic-marking
class MULTIMEDIA_CONFERENCING-NBAR
set dscp af41
!
int gig0/0/0
service-policy in traffic-marking
Term-A
10.3.0.1
Term-B
GRE
Tunnel
10.1.0.1
Gig0/0/0
10.1.0.2
10.2.0.1
10.2.0.2
10.3.0.2
SP
Network
L2
Dest
L2
SrcType
User IP
Header
User
Data
Src IP: 10.1.0.1
Dst IP: 10.3.0.1
DSCP: 0
Packet View 1
Src IP: 10.1.0.1
Dst IP: 10.3.0.1
DSCP: af41
Src IP: 10.1.0.1
Dst IP: 10.3.0.1
DSCP: af41
Src IP: 172.16.0.1
Dst IP: 172.16.0.2
DSCP: af41
Packet View 3
DSCP copied Inner-to-Outer
Tun10
172.16.0.1
Tun10
172.16.0.2
Gig0/0/1
192.168.0.1
192.168.0.2
Src IP: 10.1.0.1
Dst IP: 10.3.0.1
DSCP: af41
L2
Dest
L2
SrcType
User IP
Header
User
Data
Packet View 2
L2
Dest
L2
SrcType
User IP
Header
User
Data
GRE IP
Header
L2
Dest
L2
SrcType
User IP
Header
User
Data
Packet View 4
Enterprise to SP MappingDefault SP Marking
BRKCRS-2007 36
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
class-map INTERACTIVE-VIDEO
match dscp af41
!
policy-map egress-queuing
class INTERACTIVE-VIDEO
set dscp af31
!
int gig0/0/1
service-policy out egress-queuing
Term-A
Term-B
GRE
Tunnel
10.1.0.1
Gig0/0/0
10.1.0.2
10.2.0.1
10.2.0.2
SP
Network
L2
Dest
L2
SrcType
User IP
Header
User
Data
Src IP: 10.1.0.1
Dst IP: 10.3.0.1
DSCP: 0
Packet View 1
Src IP: 10.1.0.1
Dst IP: 10.3.0.1
DSCP: af41
Src IP: 10.1.0.1
Dst IP: 10.3.0.1
DSCP: af41
Src IP: 172.16.0.1
Dst IP: 172.16.0.2
DSCP: af31
Packet View 3
Tun10
172.16.0.1
Tun10
172.16.0.2
Gig0/0/1
192.168.0.1
192.168.0.2
Src IP: 10.1.0.1
Dst IP: 10.3.0.1
DSCP: af41
L2
Dest
L2
SrcType
User IP
Header
User
Data
Packet View 2
L2
Dest
L2
SrcType
User IP
Header
User
Data
GRE IP
Header
L2
Dest
L2
SrcType
User IP
Header
User
Data
Packet View 4
DSCP copied Inner-to-Outer *BUT*
we over-write Outer after the copy
10.3.0.1
10.3.0.2
Enterprise to SP MappingSet dscp outbound on physical (Branch)
class-map match-all MULTIMEDIA_CONFERENCING-NBAR
match protocol attribute traffic-class multimedia-conferencing
match protocol attribute business-relevance business-relevant
!
policy-map traffic-marking
class MULTIMEDIA_CONFERENCING-NBAR
set dscp af41
!
int gig0/0/0
service-policy in traffic-marking
Video Flow from
Term-A To Term-B
BRKCRS-2007 37
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
class-map INTERACTIVE-VIDEO
match dscp af41
!
policy-map egress-queuing
class INTERACTIVE-VIDEO
set dscp tunnel af31
!
int tun10
service-policy out egress-queuing
Term-A
Term-B
GRE
Tunnel
10.1.0.1
Gig0/0/0
10.1.0.2
10.2.0.1
10.2.0.2
SP
Network
L2
Dest
L2
SrcType
User IP
Header
User
Data
Src IP: 10.1.0.1
Dst IP: 10.3.0.1
DSCP: 0
Packet View 1
Src IP: 10.1.0.1
Dst IP: 10.3.0.1
DSCP: af41
Src IP: 10.1.0.1
Dst IP: 10.3.0.1
DSCP: af41
Src IP: 172.16.0.1
Dst IP: 172.16.0.2
DSCP: af31
Packet View 3
Tun10
172.16.0.1
Tun10
172.16.0.2
Gig0/0/1
192.168.0.1
192.168.0.2
Src IP: 10.1.0.1
Dst IP: 10.3.0.1
DSCP: af41
L2
Dest
L2
SrcType
User IP
Header
User
Data
Packet View 2
L2
Dest
L2
SrcType
User IP
Header
User
Data
GRE IP
Header
L2
Dest
L2
SrcType
User IP
Header
User
Data
Packet View 4
10.3.0.1
10.3.0.2
Enterprise to SP MappingSet dscp tunnel outbound on tunnel (Hub)
class-map match-all MULTIMEDIA_CONFERENCING-NBAR
match protocol attribute traffic-class multimedia-conferencing
match protocol attribute business-relevance business-relevant
!
policy-map traffic-marking
class MULTIMEDIA_CONFERENCING-NBAR
set dscp af41
!
int gig0/0/0
service-policy in traffic-marking
‘Set dscp tunnel’ means don’t copy
but instead remember and mark this
value once tunnel header is imposed
Video Flow from
Term-A To Term-B
BRKCRS-2007 38
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Use “set dscp tunnel” on Hub’s per tunnel, “set dscp” remarks inner
header at hub
• Branch policy applied on physical uses “set dscp” : just remarks Ipsec,
inner untouched
• If “set dscp” used on hub, DSCP Values for Traffic Class from branch
and hub will not be the same, as a result channels will not establish
DSCP remarking - Impact on PFR channels
BRKCRS-2007 39
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN QOS Summary Hub
- Per-Tunnel QoS for Branches, child policy drives per-app bandwidth ( voice, video )
- with per-tunnel, the encapsulating interface ( physical ) supports only a class default shaper
Branch
- Shaper and Child-Policy on Physical WAN Interface
- No shaper required if line-rate interface
Maximize or Disable anti-replay window as queueing is done post encryption
- Window size varies with platform. Make as large as possible
BRKRST-2043 IWAN AVC-QoS Design
BRKCRS-2007 40
End State IWAN Concepts:DMVPN Tunnels and Routing
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Various Acceptable DMVPN Layouts
R11 – DMVPN Hub R41 – DMVPN Spoke
CE Router at Hub and Spoke
FW Protects Hub
Complex
Scenario
Direct Connection
BRKCRS-2007 42
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Centralized Access Model
Internet and Internal traffic routes across the WAN
A simple default route can be used for Internet traffic and Internal traffic
Internet Access Models
Distributed Access Model
Internet traffic routes direct to the ISP
A simple default route can be used for Internet traffic pointing to ISP
Internal traffic routes across the WAN
A simple default route can NOT be used for Internal traffic.
BRKCRS-2007 43
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Route Summarization
• All DMVPN hubs advertise Enterprise prefix summary routes (10.0.0.0/8) for all the LAN and WAN networks
• DMVPN hubs advertise a default route that provides Internet connectivity.
• DC Specific Summaries:
• 10.1.0.0/16
• 10.2.0.0/16
De
fau
lt R
ou
te
10
.0.0
.0/8
Su
mm
ary
R
ou
te
De
fau
lt R
ou
te
Internet Internet
10
.1.0
.0/1
6
DC
1
10
.2.0
.0/1
6
DC
2
BRKCRS-2007 44
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
NHRP Interaction with Route Table
R31-Spoke#show ip route
10.0.0.0/8 is variably subnetted, 3 subnets, 3 masks
D 10.0.0.0/8 [90/26885120] via 192.168.100.11, 00:29:28, Tunnel100 Summary Route from DMVPN Hub
C 10.3.3.0/24 is directly connected, GigabitEthernet0/2
192.168.100.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.100.0/24 is directly connected, Tunnel100
R31-Spoke#show ip route
10.0.0.0/8 is variably subnetted, 4 subnets, 3 masks
D 10.0.0.0/8 [90/26885120] via 192.168.100.11, 00:31:06, Tunnel100
C 10.3.3.0/24 is directly connected, GigabitEthernet0/2
H 10.4.4.0/24 [250/255] via 192.168.100.41, 00:00:22, Tunnel100 NHRP Installed Route
192.168.100.0/24 is variably subnetted, 3 subnets, 2 masks
C 192.168.100.0/24 is directly connected, Tunnel100
H 192.168.100.41/32 is directly connected, 00:00:22, Tunnel100 NHRP Installed Route
Routing Table with Spoke-to-Spoke Traffic
Routing Table with Spoke-to-Hub Traffic
BRKCRS-2007 45
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN Routing Protocol Selection
• Prescriptive design that uses EIGRP or IBGP for scalability.
• EIGRP and BGP do not flood routes
• IBGP supports dynamic peers, supports zero-touch DMVPN hub and templatable spoke configuration
• IBGP allows usage of Local Preference to allow centralized routing policy change
• DMVPN topologies can support up to 2,000 spokes. Routing protocol must be able scalable.
• PfR interacts with EIGRP and BGP
BRKCRS-2007 46
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Same EIGRP AS # for LAN and WAN
• DMVPN Hub advertise Default and Summary Route
• Delay added on to influence PfR uncontrolled traffic
• EIGRP Stub Site Feature on Branches
IWAN EIGRP Routing Design
BRKCRS-2007 47
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
EIGRP Stubrouter eigrp IWAN
address-family ipv4 unicast autonomous-system 1
eigrp stub
BRKCRS-2007 48
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
EIGRP Stub-Siterouter eigrp IWAN
address-family ipv4 unicast autonomous-system 1
af-interface Tunnel100
stub-site wan-interface
exit-af-interface
!
af-interface Tunnel200
stub-site wan-interface
exit-af-interface
eigrp stub-site 1:4
BRKCRS-2007 49
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN Deployment – EIGRP• Single EIGRP process for Branch, WAN and
POP/hub sites
• Extend Hello/Hold timers for WAN
• Adjust tunnel interface “delay” to ensure WAN path preference (MPLS primary, INET secondary)\
• Adjust LAN interface “delay” to ensure proper path selection
• Hubs
• Disable Split-Horizon
• Advertise Site summary, enterprise summary, default route to spokes
• Summary metrics: A summary-metric is used to reduce computational load on the DMVPN hubs.
• Ingress filter summary routes on tunnels.
• Spokes
• EIGRP Stub-Site functionality builds on stub functionality that allows a router to advertise itself as a stub to peers on specified WAN interfaces, but allows for it to exchange routes learned on LAN interface
R31 R41
R10
Site1 Site2
R20
MPLS INET
DCIWAN Core
Delay 1,000
Delay 25000 Delay 25000 Delay 25000 Delay 25,000
Set TunnelDelay to
influence best path
EIGRPStub Site
Delay 2,000
R11 R12 R21 R22
Delay 24,000 Delay 24,000
Delay 20,000Delay 1,000Delay 1,000
Delay 20,000
R51 R52Delay 20,000
Delay 20,100 Delay 20,100
BRKCRS-2007 50
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN BGP Routing Flow
Branches with Directly Connected
Branches with Multiple Routers
BRKCRS-2007 51
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN Deployment – BGP on WAN & OSPF on LAN• A single iBGP routing domain is used for WAN
• Appropriate Hello/Hold timers for WAN(20 hello / 60 hold)
• BGP Neighbor Weight is set to 50k
• Hub:
• DMVPN hub routers function as BGP route-reflectors for the spokes.
• BGP dynamic peer feature configured for Tunnel Networks
• Spokes:
• Peer to the DMVPN hubs for that transport
RR RR
For yourreference only BRKCRS-2007 52
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN Deployment – BGP on WAN & OSPF on LAN• Traffic Engineering for traffic when PfR is uncontrolled
state.
• Set Local-Preference:
• 100,000 for first selection (MPLS DC1)
• 20,000 for second selection (MPLS DC2)
• 3,000 for third selection (Internet DC1)
• 400 for fourth selection (Internet DC2)
LP100,000 RR
LP3,000
LP400
LP20,000RR
R31-Spoke# show bgp ipv4 unicast
! Output omitted for brevity
Network Next Hop Metric LocPrf Weight Path
* i 0.0.0.0 192.168.200.22 1 400 50000 i
* i 192.168.200.12 1 3000 50000 i
* i 192.168.100.21 1 20000 50000 i
*>i 192.168.100.11 1 100000 50000 i
* i 10.0.0.0 192.168.200.22 0 400 50000 i
* i 192.168.200.12 0 3000 50000 i
* i 192.168.100.21 0 20000 50000 i
*>i 192.168.100.11 0 100000 50000 i
* i 10.1.0.0/16 192.168.200.12 0 3000 50000 i
*>i 192.168.100.11 0 100000 50000 i
* i 10.2.0.0/16 192.168.200.22 0 400 50000 i
*>i 192.168.100.21 0 20000 50000 i
For yourreference only BRKCRS-2007 53
DMVPN Migration:Hub Routers and Routing Logic
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
We did a lot of research in Vegas!
Not Everyone’s WAN is the same.
BRKCRS-2007 55
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Network Traffic Flows During Migration
• Site-to-Site Traffic in Legacy WAN
• Site-to-Site Traffic in IWAN
• Traffic between Legacy and IWAN networks must flow through a migration site. This is located with the DMVPN hubs
BRKCRS-2007 56
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Three Methods of Hub Deployment or Migration
DMVPN
Hub*DMVPN
Hub*
Greenfield
• New DMVPN Hub Routers
• New Circuits
• Simple Design
Intermediate (IBlock)
• New DMVPN Hub Routers
• Existing Circuits
• Medium Design
Condensed
• Existing CE Routers
• Existing Circuits
• Increased Complexity
Spoke Migration is not impacted by the Hub model.57
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Transport Drawing Connectivity showed logical structure
Physical connectivity looks like
Sub-Interfaces can separate:
• P2P traffic
(/30 IP on Sub-Interface)
• Transit switching
(VLAN on MLS)
The same concept can apply to
transport connectivity too
BRKCRS-2007 58
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Greenfield Deployment
Greenfield
• New DMVPN Hub Routers
• New Circuits
• Simple Design
• Not restricted to constraints of existing network
• The only routing interaction required with the existing network is connectivity to the LAN (Migration Site)
• Simple Post-Migration CleanupRemoval of CE1 and CE2
• Typically used when deploying new circuits or a parallel network
BRKCRS-2007 59
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Greenfield Migration Routing Pattern
Benefits:
• Isolated environment. Changes on CE1 do not impact IWAN environment.
• Simple routing configuration
• Easy to troubleshoot and trace packet flows
• Bandwidth is sized appropriately for DMVPN traffic only.
• QoS policy on DMVPN hub is separated from Legacy QoS policy
Cons:
• Cost and timeline for new circuits
BRKCRS-2007 60
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Intermediate DeploymentIntermediate (IBlock)
• New DMVPN Hub Routers
• Existing Circuits
• Medium Design
• Some constraints of existing network
• Existing circuits to SP are used. New links (logical/physical) between CEs and DMVPN hubs are required.
• CEs must advertise these new links to the SP so that spokes know how to reach the DMVPN hubs.
• Connectivity to the LAN is straightforward.
• Post-migration cleanup may be required
BRKCRS-2007 61
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Intermediate Migration Routing Pattern
Benefits:
• Simple routing configuration
• Easy to troubleshoot and trace packet flows
• QoS policy on DMVPN hub is separated from Legacy QoS policy
Cons:
• Bandwidth for CE1 to the SP network must be sized accordingly.
• Changes on CE1 could impact IWAN environment.
• Some Clean-Up after Migration
BRKCRS-2007 62
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN Routing Protocol Diagram During MigrationEIGRP
BRKCRS-2007 63
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN Routing Protocol Diagram During MigrationBGP
BRKCRS-2007 64
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Condensed Deployment
Condensed
• Existing CE Routers (verify capability)
• Existing Circuits
• Increased Complexity (QoS / Routing)
Do not Deviate from the IWAN CVD with this model, or be
prepared to face problems or complications during migration
BRKCRS-2007 65
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Condensed Migration Routing PatternBenefits:
• Cost
• No real Clean-Up after Migration
Cons:
• Outage to all WAN networks is required during cutover.
• Advanced Routing (VRF Leaking)
• Hiearchical QoS is Not Supported on transport interface. If needed for legacy network, this prevents per-tunnel-QoS on DMVPN tunnel.
• Does your existing WAN have per-tunnel QoS?This could be enabled later
BRKCRS-2007 66
Condensed - Leaking Routes Between BGP Global & VRF Tables
vrf definition MPLS01
address-family ipv4
import ipv4 unicast map VRF-LEAK-TO-MPLS01
export ipv4 unicast map VRF-LEAK-FROM-MPLS01
! These route-maps are used to Permit/Block Routes between the
! VRF and Global BGP Tables
route-map VRF-LEAK-TO-MPLS01 permit 10
match ip address prefix-list LEAK-TO-MPLS01
route-map VRF-LEAK-FROM-MPLS01 permit 10
match ip address prefix-list LEAK-FROM-MPLS01
ip prefix-list VRF-LEAK-TO-MPLS01 permit 0.0.0.0/0 le 32
ip prefix-list VRF-LEAK-FROM-MPLS01 permit 0.0.0.0/0 le 32
router bgp 10
address-family ipv4 vrf MPLS01
neighbor 172.16.11.2 remote-as 65000
neighbor 172.16.11.2 activate
! The local-as command is not required; but allows you to use a standard ASN
! for IWAN and still peer to MPLS SP using the ASN they want you to use
neighbor 172.16.11.2 local-as 11 no-prepend replace-as dual-as
BRKCRS-2007 67
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Condensed - Leaking Routes Between BGP Global & VRF Tables
R11-DC1-Hub1#show bgp ipv4 unicast
Network Next Hop Metric LocPrf Weight Path
*> 10.0.0.0 0.0.0.0 32768 i
*> 10.1.0.0/16 0.0.0.0 32768 i
s> 10.1.0.11/32 0.0.0.0 0 32768 ?
s> 10.1.12.0/24 0.0.0.0 0 32768 ?
s> 10.1.111.0/24 0.0.0.0 0 32768 ?
s>i 10.3.0.31/32 192.168.100.31 0 100 50000 ?
s>i 10.3.3.0/24 192.168.100.31 0 100 50000 ?
s> 10.4.0.41/32 172.16.11.2 0 65000 41 ?
s> 10.4.4.0/24 172.16.11.2 0 65000 41 ?
s> 10.5.0.51/32 172.16.11.2 0 65000 51 ?
s> 10.5.0.52/32 172.16.11.2 0 65000 51 ?
BRKCRS-2007 68
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Condensed - Routing Table with Route Leaking
R11-DC1-Hub1#show ip route bgp
!SNIP
10.0.0.0/8 is variably subnetted, 24 subnets, 4 masks
B 10.0.0.0/8 [19/0], 04:34:53, Null0
B 10.1.0.0/16 [19/0], 04:34:53, Null0
B 10.3.0.31/32 [19/0] via 192.168.100.31, 00:22:19
B 10.3.3.0/24 [19/0] via 192.168.100.31, 00:22:19
B 10.4.0.41/32 [201/0] via 172.16.11.2 (MPLS01), 00:28:19
B 10.4.4.0/24 [201/0] via 172.16.11.2 (MPLS01), 00:28:19
B 10.5.0.51/32 [201/0] via 172.16.11.2 (MPLS01), 00:28:19
B 10.5.0.52/32 [201/0] via 172.16.11.2 (MPLS01), 00:28:19
B 10.5.5.0/24 [201/0] via 172.16.11.2 (MPLS01), 00:28:19
B 10.5.12.0/24 [201/0] via 172.16.11.2 (MPLS01), 00:28:19
BRKCRS-2007 69
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Other Condensed Techniques May Technically Work…..
Be aware of your traffic patterns:
• IWAN to Legacy
• IWAN to DC
• Legacy to DC
Additional load for transit traffic
Clean-up is still needed later on:
• Encapsulating tunnel IP changes
Going off the tried and true path
may lead to problems later!
BRKCRS-2007 70
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Hub Deployment Summary
DMVPN
Hub*DMVPN
Hub*
Greenfield Intermediate (IBlock) Condensed
• Keep It Simple Stupid (KISS). Remember your operations staff.
• Use Greenfield or IBlock when possible
• Depending on bandwidth CSR1000Vs could be used
• Don’t go crazy if you go Condensed
BRKCRS-2007 71
DMVPN Migration:Branch Routers
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Branch Pre-Migration Tasks
• Make a list of what network applications work and what applications do not work before migrating the branch
• Backup the existing router configurations to the local router & centralized repository.
• Allow local authentication / authorization. to allow access to the router in a timely manner (assuming that TACACS or radius servers cannot be reached).
• Allow remote console sessions on routers from the workstation, and any peer routers.
BRKCRS-2007 73
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Branch Migration ActivitiesDuring the migration the following tasks are done:
- DMVPN tunnel configuration
- Certificate enrollment if IPsec Tunnel Protection uses PKI
- Association of FVRF to the Encapsulating Interface
- Routing protocol changes
- PfR configuration deployed
BRKCRS-2007 74
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Connectivity During Migration
• When the FVRF is associated to the transport interface, the IP address is removed from that interface.
• If there is a backdoor between sites, migrate those sites together
- prevents possibility of route loops and transit routing
R31-Site3(config-if)#vrf forwarding MPLS01
% Interface GigabitEthernet0/1 IPv4 disabled and address(es)
removed due to enabling VRF MPLS01
R31-Site3(config-if)#ip address 172.16.31.1 255.255.255.252
BRKCRS-2007 75
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Assess the Connectivity Model at Branch
• Single router with single transport• Cold Migration Only
• Single router with dual transport• Cold Migration• Warm Migration
• Dual router with dual transport• Cold Migration • Warm Migration
Decide if migrations are remote or on-site
Depending on the site’s connectivity model, the migration could be executed without loss of service to the users at the branch.
76
Migration Scripts • Cisco tools use these or can be used for CLI
• Prevents for Typos/Fat-Fingering
• Allows for off-site migration
Example: EEM script allows for multiple commands to be entered even if console connectivity is lost.
event manager applet MIGRATE-PORTION
event none
action 010 cli command "enable"
action 020 cli command "configure terminal"
action 030 cli command "interface GigabitEthernet0/2"
action 040 cli command "vrf forwarding INET01"
action 050 cli command "ip address dhcp“
! Wait 20 seconds to allow DHCP to get a packet before no shutting tunnel
action 060 wait 20
action 070 99 syslog msg “FVRF Associated to Gi0/2"
BRKCRS-2007 77
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Advanced EEM Script that Configures Routing Too!event manager applet MIGRATE
event none
action 010 cli command "enable"
action 020 cli command "configure terminal"
! This section enables the MPLS FVRF and No Shuts the MPLS Tunnel
action 030 cli command "interface GigabitEthernet0/1"
action 040 cli command "vrf forwarding MPLS01"
action 050 cli command "ip address 172.16.31.1 255.255.255.252"
action 060 cli command "ip route 0.0.0.0 0.0.0.0 Tunnel100 192.168.100.11 250"
action 070 cli command "interface Tunnel 100"
action 080 cli command "no shut"
! This section enables the Internet FVRF and No Shuts the Internet Tunnel
action 090 cli command "interface GigabitEthernet0/2"
action 100 cli command "vrf forwarding INET01"
action 110 cli command "ip address dhcp"
! The wait command allows for the interface to obtain an IP address from DHCP
! Before the Internet DMVPN tunnel is brough online
action 120 wait 15
action 130 cli command "interface Tunnel 200"
action 140 cli command "no shut"
action 150 syslog msg "Interface Configurations Performed "
! The last section is to remove the previous routing protocol configuration.
! And then configure the routing protocols. Only a portion of this activity
! is shown, but this section should be completed based on your design.
action 160 cli command "no router bgp 65000"
action 170 cli command "no router ospf 1"
action 180 cli command "router eigrp IWAN"
! Continue with rest of routing protocol configuration
action 999 syslog msg "Migration Complete"
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Migrating a Branch Router
Configure DMVPN
Configure EEM applet
** Copy run start
** Reload in 15
Execute EEM
Connect back to router
• Either on Tunnel or FVRF
Configure overlay routing
• Remove any existing routing
** reload cancel
Verify connectivity
Tunnel will remain down with
no FVRF interface
The entire process could be
captured by an script
** Recommended for CLI MigrationsBRKCRS-2007 79
Post-Migration Cleanup
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Post- MigrationIf the final IWAN design does not migrate all devices to IWAN, then stop here!
Migration is considered complete once :
• All of the planned sites are communicating only via overlay tunnels
• The service provider network is used only for transport between DMVPN routers.
• The last task is to clean up the environment:
• Greenfield – Remove previous WAN routers
• Intermediate (IBlock) – Removal of link between LAN and CE RoutersPotential removal of CE links
• Condensed – Remove BGP Route Leaking Configuration
BRKCRS-2007 81
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Post-Migration Clean-Up for Intermediate
Link Not
Needed
BRKCRS-2007 82
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
CE1 could be removed depending on the following factors:
• Who owns the device? Your organization or the service provider?
• What additional value does CE1 add to the design or operational perspective?
Removal of the CE Device
BRKCRS-2007 83
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Post Migration Clean upCE Removal
• While removing CE1, if the cable connecting to the MPLS network & CE1 is pulled from CE1 and plugged into R11, DMVPN connectivity is going to break.
• R11’s IP address is on the 172.16.11.0/30 network and the service provider’s PE router is on the 172.16.13.0/30 network. One of the devices will have to change their IP address.
• DMVPN Spoke mappings is configured to the 172.16.11.1 NBMA Address.
BRKCRS-2007 84
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Post Migration Clean upHow to fix IP Addressing Problem
Connectivity is restored by:
• Re-configure the NHRP on every branch site
• Either add a second NBMA address (only 1 active at a time on each spoke)
• Terminate the DMVPN Tunnel on a Loopback
• Little more complexity in VRF Routing & additional IP addresses consumed.
• Coordinate IP address change with SP and migrate 1 DMVPN hub at a time.
• SP would change the IP addressing on the peer link.
BRKCRS-2007 85
Migration of VPLS or Metro Ethernet Topologies
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Router cannot forward L3 and L2 on the same interface
• Requires Insertion of a Switch from VPLS Hand-off
• QoS Shaping can be done outbound on newly inserted switch
DMVPN Hub Setup for VPLS Migration
Same Subnet on
CE1 and DMVPN
FVRF Interface
BRKCRS-2007 87
Migration from Dual MPLS to Hybrid Model
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Migration from Dual MPLS to Hybrid Model
• Traditional Dual MPLS with Mutual Redistribution between IGP and BGP
• Install new MPLS1 DMVPN Hub (Just like shown earlier)
• Install new Internet DMVPN Hub
• Turn up DMVPN interfaces on MPLS and Internet Hubs
• Migrate Branch Sites.
• MPLS1 MPLS1 DMVPN Tunnel
• Install new Internet Circuit
• Internet DMVPN Tunnel turned up
• MPLS2 Shutdown and Circuittermination
BRKCRS-2007 89
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Clean-Up from Dual MPLS to Hybrid Model
Now that all sites have migrated on to IWAN, there is not a need for connectivity to the MPLS SP2.
• Remove CE2 (Connected to MPLS SP2)
• Remove the link between MLS5and CE1
BRKCRS-2007 90
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Clean-Up from Dual MPLS to Hybrid Model (continued)
Now comes the decision to remove CE1 or keep it. If it is removed, then this is what your topology will look like.
BRKCRS-2007 91
Alternative to Using a Migration Site
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Alternative to Using a Migration Site
Sometimes routing traffic through a Migration site may not work due to:
• End-to-End Latency
• Bandwidth at Hubs
Where possible, see if you canadd another Hub and advertisemore specific routes.
If that cannot be done, thereis another option for routing
experts, and requires route leaking at the IWAN branch.
BRKCRS-2007 93
Alternative to Using a Migration SiteReceiving Routes (IWAN Path)Hub receives the route, but advertises a summary that contains it.
Branch receives the hub summary and tags it. That route is not leaked from Global to FVRF.
10.6.1.0/24
Branch tags on receipt and
blocked from insertion to
FVRF
VRF Export Map Blocks Tag
BRKCRS-2007 94
Alternative to Using a Migration SiteReceiving Routes (Transport Path)
Branch tags on receipt and
blocked from
advertisement to Hub
Branch receives the branch route in a FVRF routing protocol and tags it.
Route is leaked from FVRF into Global.
Route is blocked frombeing advertised to the hubs.
BRKCRS-2007 95
Alternative to Using a Migration SiteReceiving RoutesLongest match wins.
IWAN Branch will go direct through SP transport
BRKCRS-2007 96
Alternative to Using a Migration SiteAdvertising Routes (Branch via Hub)
10.3.1.0/24
AS100:100
Branch advertises the route to Hub
Hub advertises to CE router
CE router prepends AS or blocks
SP advertises to R61
BRKCRS-2007 97
Alternative to Using a Migration SiteAdvertising Routes (Branch)Branch advertises route to SP with BGP community.
SP advertises route to Migration CE, and is blocked by community.
Route via IWAN Path is preferred.
SP advertises route to remote branch
Branch route
is filtered on
CE inbound
from transport
BRKCRS-2007 98
Alternative to Using a Migration SiteAdvertising Routes (Branch)Shortest AS-Path Wins
Traffic from R31’s transport (leaked) interface is preferred
BRKCRS-2007 99
Alternative to Using a Migration SiteAdvertising Routes (CE)CE advertises routes to SP with BGP Community 100:200
SP advertises route to Remote Branch which accepts the route.
SP advertises route to IWAN Branch which discards based on community.
IWAN Branch uses Summary Route (via R11)
IWAN Branch discards
route based on 100:200
BGP CommunityBRKCRS-2007 100
Keep in Mind About Not Using a Migration Site• There is a lot of route tagging and leaking between VRFs.
• This can cause confusion for operation staff and Junior Network Engineers
• If this is the path you want to pursue, please engage Cisco or a Cisco Partner for assistance
BRKCRS-2007 101
Migration of Existing Point-to-Point IPsec Topologies
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Add the DMVPN hub router into the network
• The placement of hub depends on where the IPSEC tunnels are currently terminated – Firewall or a router
If IPSEC is terminated on FW, then place the hub router behind it ( pass-through)
• Migrate sites based on traffic patterns- Non-transit sites first
Migrating P2P IPSEC WAN to IWAN
R4 R5
DMVPN
Hub
R2R1
DMVPN
Tunnel
R3
BRKCRS-2007 103
Important PfR Concepts for IWAN
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Performance Routing v3Running in an Enterprise Domain
BranchMPLS
Internet
Central Site
Branch • One Master Controller defined as the Hub MC
• Centralized location for policy definition
Hub Master Controller
Branch Master Controller
MC
BR1
BR2
MC/BR
MC/BR
BRKRST-3362 Implementing Performance Routing
BRKCRS-2007 105
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Enterprise Domain
Branch
MPLS
Internet
Central Site
Network
Discovers the
Applications
WAN Edge measures
application performance
WAN Edge peers,
learns SP SLA,
manages congestion
Send performance
feedback to peers
Peering & Coordination at WAN Edge MC
BR2
BR1
MC/BR
BRKCRS-2007 106
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
• DMVPN is a requirement for the PFR solution
- Can’t support multiple next-hops and multiple data centers with the same prefix when the carrier is your routing partner
• Tunnel Bandwidth must be configured(otherwise default is 100kbps)
- Load Balancing
- Performance classes when first controlled have no bandwidth, but before they can be moved available bandwidth is verified
Deploying Intelligent Path Control- Best practices
BRKCRS-2007 107
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Policy
• Start with a Single Class and Load Balancing disabled- All other classes will follow routing
• Enable an additional class- Monitor Traffic Classes and Load on the Network ( CPU, Interface Utilization etc..)
• Enable additional classes and load balancing
• Three Performance Classes, Voice, Video, and Critical Application, plus Load Balancing is a good start to baseline.
Deploying Intelligent Path ControlPrepare to run PFR
BRKCRS-2007 108
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Built-in Policy TemplatesMatching QoS Best Practices
Pre-defined
Template
Threshold Definition
Voice priority 1 one-way-delay threshold 150 threshold 150 (msec)
priority 2 packet-loss-rate threshold 1 (%)
priority 2 byte-loss-rate threshold 1 (%)
priority 3 jitter 30 (msec)
Real-time-video priority 1 packet-loss-rate threshold 1 (%)
priority 1 byte-loss-rate threshold 1 (%)
priority 2 one-way-delay threshold 150 (msec)
priority 3 jitter 20 (msec)
Low-latency-
data
priority 1 one-way-delay threshold 100 (msec)
priority 2 byte-loss-rate threshold 5 (%)
priority 2 packet-loss-rate threshold 5 (%)
Pre-defined
Template
Threshold Definition
Bulk-data priority 1 one-way-delay threshold 300 (msec)
priority 2 byte-loss-rate threshold 5 (%)
priority 2 packet-loss-rate threshold 5 (%)
Best-effort priority 1 one-way-delay threshold 500 (msec)
priority 2 byte-loss-rate threshold 10 (%)
priority 2 packet-loss-rate threshold 10 (%)
scavenger priority 1 one-way-delay threshold 500 (msec)
priority 2 byte-loss-rate threshold 50 (%)
priority 2 packet-loss-rate threshold 50 (%)
BRKCRS-2007 109
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Ensure Parent Route is present to match site-prefix in PFR
• Routing Protocols are checked in this order:NHRP, BGP, EIGRP, Static, RIB
• If a route is found in the BGP table for 10.0.0.0/8 over your discovered paths and you are looking for 10.1.0.0/16 which is in EIGRP and the RIB, BGP will be utilized. PfRv3 is an Enterprise Protocol and does not expect multiple routing protocols within a single Enterprise.
Deploying Intelligent Path ControlPrepare to run PFR
BRKCRS-2007 110
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Use Standard attributes in site and enterprise prefix-list , they do not support extended prefix-list attributes
Examples :
ip prefix-list site-prefix seq 5 deny 10.1.10.0/24” invalid,
only permit is supported
“ip prefix-list site-prefix seq 10 permit 10.1.0.0/16 le
24” invalid, it will be advertised as 10.1.0.0/16 alone
Deploying Intelligent Path Control- Best Practices
BRKCRS-2007 111
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
• With an increase in number of traffic-classes to the Data Center,
Manually break the site-prefix into smaller blocks to increase load-balancing granularity.
ip prefix-list site-prefix seq 5 permit 10.1.1.0/24
ip prefix-list site-prefix seq 10 permit 10.1.16.0/20
ip prefix-list site-prefix seq 15 permit 10.1.32.0/20
ip prefix-list site-prefix seq 20 permit 10.1.48.0/20
ip prefix-list site-prefix seq 25 permit 10.1.0.0/16
• Longest prefix always wins
Deploying Intelligent Path Control-Best Practices
BRKCRS-2007 112
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Pfr Topology
BRKCRS-2007 113
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Enterprise Prefix
PFR Enterprise & Site Prefix Lists
Without Enterprise-Prefix: all the
traffic between PfR sites will be
learned as PfR Internet traffic class
and delay, jitter, etc. cannot be
monitored.
PfR Internet
Site prefixes for particular sites
with PFRv3 enabled
Branches learn Site Prefixes
Dynamically (or statically
configured)
Hubs act as transit sites –site-
prefix statically defined
Branch
Site
Prefixes
* Only Routing is used between
Non-PfR and PfR enabled site in
Enterprise Prefix
**Legacy
Site
Prefixes
**Placing Legacy Site Prefixes at
Hub Sites, provides PfR for half of
the path
Hub
Site
Prefixes
BRKCRS-2007 114
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Hubs: Site-Prefix lists before anything is migrated
DMVPNMPLS
DMVPNINET
R31
R12 R21 R22
R10 R20
R11
10.3.3.0/24
10.1.0.0/16
10.0.0.0/8
BGP
10.2.0.0/16
10.0.0.0/8
BGP
10.1.0.0/16 10.2.0.0/16
SITE1
PfR Site-Prefix
10.1.0.0/16
SITE2
PfR Site-Prefix
10.2.0.0/16
R41
10.4.4.0/24
DMVPNMPLS
Enterprise Prefix
10.0.0.0/8
Site Prefix is
10.1.0.0/16
BRKCRS-2007 115
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Hub1 Site-Prefix Table Before Anything is MigratedHub MC (R10)
domain IWAN
vrf default
master hub
enterprise-prefix prefix-list ENTERPRISE_PREFIX
site-prefixes prefix-list SITE_PREFIX
!
ip prefix-list ENTERPRISE_PREFIX seq 10 permit 10.0.0.0/8
ip prefix-list SITE_PREFIX seq 10 permit 10.1.0.0/16
BRKCRS-2007 116
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Hub1 Site-Prefix Table Before Anything is MigratedR10-DC1-MC#show domain IWAN master site-prefix
Change will be published between 5-60 seconds
Next Publish 01:46:29 later
Prefix DB Origin: 10.1.0.10
Prefix Flag: S-From SAF; L-Learned; T-Top Level; C-Configured; M-
shared
Site-id Site-prefix Last Updated DC Bitmap Flag
----------------------------------------------------------------------
10.1.0.10 10.1.0.10/32 00:13:41 ago 0x1 L
10.1.0.10 10.1.0.0/16 00:13:41 ago 0x1 C,M
255.255.255.255 *10.0.0.0/8 00:13:41 ago 0x1 T
----------------------------------------------------------------------
BRKCRS-2007 117
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
R31 on Site 3 migrated to IWAN
DMVPNMPLS
DMVPNINET
R31
R12 R21 R22
R10 R20
R11
10.3.3.0/24
10.1.0.0/16
10.0.0.0/8
BGP
10.2.0.0/16
10.0.0.0/8
BGP
10.1.0.0/16 10.2.0.0/16
SITE1
PfR Site-Prefix
10.1.0.0/16
SITE2
PfR Site-Prefix
10.2.0.0/16
R41
10.4.4.0/24
DMVPNMPLS
Enterprise Prefix
10.0.0.0/8
Site Prefix is
10.1.0.0/16
10.2.0.0/16
BRKCRS-2007 118
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Hub1 Site Prefix Table After R31 is MigratedR10-DC1-MC#show domain IWAN master site-prefix
Change will be published between 5-60 seconds
Next Publish 01:46:29 later
Prefix DB Origin: 10.1.0.10
Prefix Flag: S-From SAF; L-Learned; T-Top Level; C-Configured; M-
shared
Site-id Site-prefix Last Updated DC Bitmap Flag
----------------------------------------------------------------------
10.1.0.10 10.1.0.10/32 00:23:41 ago 0x1 L
10.1.0.10 10.1.0.0/16 00:23:41 ago 0x1 C,M
10.3.0.31 10.3.0.31/32 00:01:11 ago 0x0 S
10.3.0.31 10.3.3.0/24 00:01:11 ago 0x0 S
255.255.255.255 *10.0.0.0/8 00:23:41 ago 0x1 T
----------------------------------------------------------------------
BRKCRS-2007 119
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
No PFR control for Site 3 to Site 4 traffic ( IWAN to Non-IWAN site )
DMVPNMPLS
DMVPNINET
R31
R12 R21 R22
R10 R20
R11
10.3.3.0/24
10.1.0.0/16
10.0.0.0/8
BGP
10.2.0.0/16
10.0.0.0/8
BGP
10.1.0.0/16 10.2.0.0/16
SITE1
PfR Site-Prefix
10.1.0.0/16
SITE2
PfR Site-Prefix
10.2.0.0/16
Enterprise Prefix
10.0.0.0/8
Site Prefix is
10.1.0.0/16 R41
10.4.4.0/24
DMVPNMPLS
Ro
utin
g
BRKCRS-2007 120
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Add 10.0.0.0/8 to Hub1 Site-PrefixHub MC (R10)
domain IWAN
vrf default
master hub
enterprise-prefix prefix-list ENTERPRISE_PREFIX
site-prefixes prefix-list SITE_PREFIX
!
ip prefix-list ENTERPRISE_PREFIX seq 10 permit 10.0.0.0/8
ip prefix-list SITE_PREFIX seq 10 permit 10.1.0.0/16
ip prefix-list SITE_PREFIX seq 20 permit 10.0.0.0/8
BRKCRS-2007 121
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
R10-DC1-MC#show domain IWAN master site-prefix
Change will be published between 5-60 seconds
Next Publish 01:46:29 later
Prefix DB Origin: 10.1.0.10
Prefix Flag: S-From SAF; L-Learned; T-Top Level; C-Configured; M-
shared
Site-id Site-prefix Last Updated DC Bitmap Flag
----------------------------------------------------------------------
10.1.0.10 10.1.0.10/32 00:28:42 ago 0x1 L
10.1.0.10 10.1.0.0/16 00:28:42 ago 0x1 C,M
10.3.0.31 10.3.0.31/32 00:06:19 ago 0x0 S
10.3.0.31 10.3.3.0/24 00:06:19 ago 0x0 S
10.1.0.10 *10.0.0.0/8 00:00:30 ago 0x1 T
----------------------------------------------------------------------
After 10.0.0.0/8 is added to Hub1 Site-Prefix
Previously this was 255.255.255.255
BRKCRS-2007 122
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
After 10.0.0.0/8 is added to Hub1 Site-Prefix
DMVPNMPLS
DMVPNINET
R31
R12 R21 R22
R10 R20
R11
10.3.3.0/24
10.1.0.0/16
10.0.0.0/8
BGP
10.2.0.0/16
10.0.0.0/8
BGP
10.1.0.0/16 10.2.0.0/16
SITE1
PfR Site-Prefix
10.0.0.0/8
10.1.0.0/16
SITE2
PfR Site-Prefix
10.0.0.0/8
10.2.0.0/16
R41
10.4.4.0/24
DMVPNMPLSP
FR
Enterprise Prefix
10.0.0.0/8
Site Prefix is
10.0.0.0/8
10.1.0.0/16
BRKCRS-2007 123
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Hub1 Site-Prefix Table After Site4 is MigratedR10-DC1-MC#show domain IWAN master site-prefix
Change will be published between 5-60 seconds
Next Publish 01:46:29 later
Prefix DB Origin: 10.1.0.10
Prefix Flag: S-From SAF; L-Learned; T-Top Level; C-Configured; M-
shared
Site-id Site-prefix Last Updated DC Bitmap Flag
----------------------------------------------------------------------
10.1.0.10 10.1.0.10/32 00:33:41 ago 0x1 L
10.1.0.10 10.1.0.0/16 00:33:41 ago 0x1 C,M
10.3.0.31 10.3.0.31/32 00:11:24 ago 0x0 S
10.3.0.31 10.3.3.0/24 00:11:24 ago 0x0 S
10.4.0.41 10.4.0.41/32 00:01:09 ago 0x0 S
10.4.0.41 10.4.4.0/24 00:01:09 ago 0x0 S
10.1.0.10 *10.0.0.0/8 00:05:19 ago 0x1 T
----------------------------------------------------------------------
BRKCRS-2007 124
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
R41 on site 4 is migrated to IWAN
DMVPNMPLS
DMVPNINET
R31
R12 R21 R22
R10 R20
R11
10.3.3.0/24
10.1.0.0/16
10.0.0.0/8
BGP
10.2.0.0/16
10.0.0.0/8
BGP
10.1.0.0/16 10.2.0.0/16
SITE1
PfR Site-Prefix
10.1.0.0/16
SITE2
PfR Site-Prefix
10.2.0.0/16
R41
10.4.4.0/24
DMVPNMPLS
PFR
Enterprise Prefix
10.0.0.0/8
Site Prefix is
10.0.0.0/8
10.1.0.0/16
BRKCRS-2007 125
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Dual Router Branch
• Must be Layer 2 Adjacent for SAF Establishment
• Can use static GRE tunnel, dedicated, or dot1q sub-interface
Deploying Intelligent Path ControlPrepare to run PFR
BRKCRS-2007 126
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
• 5 VRFs supported by default
• IOS- XE 3.16.2 adds support to configure up to 20 VRF’s ( requires TCAM re-carving )
• Global Table is configured as one “vrf default”
• VRF-Lite, no label support
Deploying Intelligent Path ControlVRF considerations
BRKCRS-2007 127
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Spoke-to-spoke Considerations for PFR
• If the interface does not have routes in the RIB (blind interface), then NHRP will not allow a shortcut to be installed. PfR is verifying Parent Routes via the BGP Table or EIGRP Topology. So NHRP’s check must be disabled, “no nhrp route-watch”
• Only a NHRP host route to the destination sites site-id, PfR Master Controller source interface, will be installed. PfR will then control traffic on this path. Check using “show domain <name> border traffic-class” or “show ip route overrides pfr”
Deploying Intelligent Path Control- Best Practices
BRKCRS-2007 128
Summary
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Documenting the existing network.
• Create a high-level migration plan.
• Deploy a proof-of-concept or production pilot of the network. The first remote site should always be in a lab. This allows for the operational teams to be comfortable with the technology while they start to learn about the actual applications in use in the network. As well, any issues to the IWAN routing architecture should not impact production during this phase.
• Testing the execution plans in a lab environment and modify accordingly.
• Deploying DMVPN hub routers.
• Migrate Branch routers.
• Post-migration cleanup tasks.
• Migrating other WAN transports/technologies
• PfR
Session Summary
Ask your boss for a raise!
You improved business
application responsiveness while
saving the company $$$$
BRKCRS-2007 130
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Recommended Reading
Coming
Soon
BRKCRS-2007 131
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
• TECCRS-2004 – Implementing the Intelligent WAN
• BRKCRS-2000 – Intelligent WAN Architecture
• BRKRST-2043 – IWAN AVC/QoS Design
• BRKCRS-2002 – IWAN Design and Deployment Workshop
• BRKRST-2362 – IWAN Implementing Performance Routing (PfRv3)
• BRKRST-3413 – IWAN Serviceability: Deploying/Monitoring/Operating
• BRKCRS-2007 – Migrating Your Existing WAN to Cisco’s IWAN
• BRKRST-2514 – IWAN Application Optimization and Provisioning
• CCSRST-2000 – IWAN Migration Case Study
• BRKNMS-1040 – IWAN Management with Cisco Prime Infrastructure
Other IWAN Related Sessions
BRKCRS-2007 132
Cisco Live On Demand
Cisco Live U.S. Content will be
out in about 3-4 weeks
BRKCRS-2007 133
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Complete Your Online Session Evaluation
Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online
• Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 Amazon gift card.
• Complete your session surveys through the Cisco Live mobile app or from the Session Catalog on CiscoLive.com/us.
BRKCRS-2007 134
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Lunch & Learn
• Meet the Engineer 1:1 meetings
• Related sessions
BRKCRS-2007 135
Please join us for the Service Provider Innovation Talk featuring:
Yvette Kanouff | Senior Vice President and General Manager, SP Business
Joe Cozzolino | Senior Vice President, Cisco Services
Thursday, July 14th, 2016
11:30 am - 12:30pm, In the Oceanside A room
What to expect from this innovation talk
• Insights on market trends and forecasts
• Preview of key technologies and capabilities
• Innovative demonstrations of the latest and greatest products
• Better understanding of how Cisco can help you succeed
Register to attend the session live now or
watch the broadcast on cisco.com
Thank you