Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
May 13-14, 2015
Virtual Network Overlays Working Group
• Follow up from last ONUG use case and fire side discussions – ONUG users wanted to see formalized feedback – ONUG users wanted to see progression in use case definition / adoption – ONUG user poll – ONUG Fall 2014
• Working group, with both end-users and vendors, created for specific use cases intended to address ONUG audience feedback – Collect and document end-user requirements – Clarify and refine use case requirements based on vendor inputs – Validate and show case use case capabilities via vendor PoCs
• Working group will be working on set of deliverables, which will be presented at the upcoming Spring 2015 ONUG – Showcase results of PoC / validation tests, addressing the top 10 Virtual Network/Overlay use case
requirements
User Poll – ONUG Fall 2014
Virtual Network Overlays
L2/L3 Network Edge Node
Edge Node
Virtual Network Overlay
Overlay Controller(s)
Orchestration
Network Overlay
End-Point
Open Interface
Network Overlay
End-Point
Underlay
Cloud Orchestration
Vendor Approaches
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Cisco Overlay Verifica0on Tes0ng
Mike Cohen Director of Product Management
Cisco
Overlay WG Use Case – Feature Verification of Top Ten Requirements Results
1. Virtual network overlay forwarding based on Open vSwitch and open documented encapsulation techniques. VXLAN used in this test.
2. Virtual network overlay endpoint termination on hypervisors, Linux containers and physical switches
3. Virtual network control plane scale and acceptable convergence for deployments scaling up to 100,000 endpoints -
4. Open documented southbound controller-to-endpoint interface(s) for virtual network overlay endpoint provisioning and traffic policy configuration
5. Open documented controller-to-controller interfaces, enabling multi-domain federation of separate virtual overlay networks -
6. Open documented northbound virtual network controller interface to cloud orchestration systems
7. Layer-2 and layer-3 traffic mapping and distributed virtual network overlay packet forwarding
8. Networking Virtual Edge (NVE) monitoring capabilities, using standard management protocols and APIs
9. End-to-end network overlay monitoring to validate network reachability and performance characteristics
10. Correlation of virtual network overlays with physical network underlay state and performance (Vendor’s choice)
System Under Test
Products Software Versions
Nexus 9504 11.0(3n)
2 Nexus 9396 11.0(3n)
Application Policy Infrastructure Controller 1.0(3n)
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Cisco Applica0on Centric Infrastructure Test Topology
• Cisco Applica@on Centric Infrastructure – Leaf-‐spine network
topology – APIC*
• Physical Ixia • Ubuntu server running docker containers
• OpenStack (including OVS) Ixia
VLAN traffic
VXLAN traffic
Nexus 9504
Nexus 9396
Ubuntu 14.04
OpenStack VM
* Production deployments require 3 APICs
OVS
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Cisco Applica0on Centric Infrastructure Highlights
Open Visibility Investment Protection
Automation Lowering Opex
and TCO
Security
Test (2): ACI can integrate with bare metal, mul@ple hypervisors, and Linux containers
Test (4/6): ACI offers open REST APIs as well as open southbound APIs (Device Packages / OpFlex)
Test (9/10): ACI delivers strong physical + virtual integra@on integra@on through health scores, fault repor@ng, and atomic counters
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Test 10: Physical + Virtual Example
• Problem: A tenant or applica@on reports connec@vity issues or diminished performance in a virtual network
• Solu@on: APIC offers health scores tracking tenant to root causes that may exist in the physical network!
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
More Informa0on
h`p://www.cisco.com/go/aci
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Big Switch Networks Overlay Verifica0on Tes0ng
Kyle Forster Founder
Big Switch Networks
Overlay WG Use Case – Feature Verification of Top Ten Requirements Results
1. Virtual network overlay forwarding based on Open vSwitch and open documented encapsulation techniques. VXLAN used in this test.
2. Virtual network overlay endpoint termination on hypervisors, Linux containers and physical switches
3. Virtual network control plane scale and acceptable convergence for deployments scaling up to 100,000 endpoints -
4. Open documented southbound controller-to-endpoint interface(s) for virtual network overlay endpoint provisioning and traffic policy configuration
5. Open documented controller-to-controller interfaces, enabling multi-domain federation of separate virtual overlay networks
6. Open documented northbound virtual network controller interface to cloud orchestration systems
7. Layer-2 and layer-3 traffic mapping and distributed virtual network overlay packet forwarding
8. Networking Virtual Edge (NVE) monitoring capabilities, using standard management protocols and APIs - 9. End-to-end network overlay monitoring to validate network reachability and performance characteristics
10. Correlation of virtual network overlays with physical network underlay state and performance (Vendor’s choice)
System Under Test
Products Software Versions
Big Cloud Fabric 2.6.0
VMWare vSphere 5.5u2
Vmware NSX 6.1.3
Apache Hadoop with Cloudera Manager (version CDH 5.3.3)
2.5
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Test Set-‐Up
• 38 switch in SDN leaf-‐spine underlay (Big Cloud Fabric)
• 16 ESXi hosts in SDN overlay (NSX-‐v)
• Pathological L2/L3 underlay versus L2/L3 overlay design
• Hadoop Terasort workload (Cloudera)
Big Cloud Fabric SDN Controllers
(active-standby pair)
Switch Light OS on Spine
(6 40G bare metal switches)
NSX-v Controllers (active-standby pair)
Switch Light OS on Leaf (32 10G/40G bare metal switches)
<--- 13 racks of servers / 48k VMs simulated using massive scale 1024 x 10GE traffic generator --->
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Chaos Monkey!
Force-‐failed a random BCF controller every 70 seconds, a random switch every 8 seconds and a random link every 4 seconds
No change to terasort run.mes
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Nuage Networks Overlay Verifica0on Tes0ng
Jerrod Gerth Principal Solu@ons Architect
Nuage Networks
Overlay WG Use Case – Feature Verification of Top Ten Requirements Results
1. Virtual network overlay forwarding based on Open vSwitch and open documented encapsulation techniques. VXLAN used in this test.
2. Virtual network overlay endpoint termination on hypervisors, Linux containers and physical switches
3. Virtual network control plane scale and acceptable convergence for deployments scaling up to 100,000 endpoints
4. Open documented southbound controller-to-endpoint interface(s) for virtual network overlay endpoint provisioning and traffic policy configuration
5. Open documented controller-to-controller interfaces, enabling multi-domain federation of separate virtual overlay networks
6. Open documented northbound virtual network controller interface to cloud orchestration systems
7. Layer-2 and layer-3 traffic mapping and distributed virtual network overlay packet forwarding
8. Networking Virtual Edge (NVE) monitoring capabilities, using standard management protocols and APIs
9. End-to-end network overlay monitoring to validate network reachability and performance characteristics
10. Correlation of virtual network overlays with physical network underlay state and performance (Vendor’s choice)
System Under Test
Products Software Versions
Nuage Virtualized Services Platform (incl. Virtualized Services Directory - VSD, Virtualized Services Controller - VSC, and Virtualized Router Switch - VRS)
3.0
VSG 7850, Nuage Virtualized Services Assurance Platform (incl. DC Network Service Navigator, DC Fault Management)
Alcatel 7750 12.0
Arista DCS-7150S 4.14.3
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Nuage Networks – Test Topology
Hypervisor
Hypervisor
Hypervisor
DC 1 Rack 1 Hypervisor
Hypervisor
Hypervisor
DC 1 Rack 2
Nuage VSG 7850 Arista 7150S
Alcatel-Lucent 7750
Hypervisor
Hypervisor
Hypervisor
DC 1 Rack 1 Hypervisor
Hypervisor
Hypervisor
DC 1 Rack 2
Nuage VSG 7850 Arista 7150S
Nuage VSC Nuage VSC
Alcatel-Lucent 7750
Availability Zone 1 Availability Zone 2
Alcatel-Lucent 7750
Federation
Nuage VSD
Nuage VSAP Route Monitor
Nuage VSAP
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Nuage Networks – Scale Test
• Create 200 networks and 200 VRFs (router contexts) • Use 200 hypervisors • Single VSD process, 4 VSCs
– All running as AWS instances
• Create 100K docker containers – 500 containers per network (showing scale of broadcast domains) – 20 networks per hypervisor
• Nets 0-‐19 in hypervisor 1, 1-‐20 in hypervisor 2, and so on • Demonstrates that solu@on scales in real environments
• Demonstrate convergence in under 10 minutes
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Nuage Networks and the ONUG Use Cases
Software only solution
Open interfaces
Scale and Robustness
Any underlay, any server hardware
ReST, OVSDB, OpenFlow, BGP
Internet scale architecture Multi-zone federation
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Nuage Networks & the DevOps Loop
Business Requirements
Dev & Test
Deployment Operations
Customer Feedback
Overlay & Underlay Correlation
Service Navigation
Fault Management
WHAT’S NEEDED?
Overlay & Underlay Correlation
Service Navigation
Fault Management
Nuage Networks Virtualized Services Assurance PlaVorm (VSAP)
Announcing Today:
Operations Nuage Networks Virtualized Services PlaVorm (VSP)
Application analytics
Security & Policy Management
Flexibility in Choice of Hardware
Nuage Networks Virtualized Services PlaVorm (VSP)
Application-driven Design
Policy-based, Declarative Control
Network Flexibility
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Pluribus Networks Overlay Verifica0on Tes0ng
Sunay Tripathi Founder and CTO Pluribus Networks
Overlay WG Use Case – Feature Verification of Top Ten Requirements Results
1. Virtual network overlay forwarding based on Open vSwitch and open documented encapsulation techniques. VXLAN used in this test.
2. Virtual network overlay endpoint termination on hypervisors, Linux containers and physical switches
3. Virtual network control plane scale and acceptable convergence for deployments scaling up to 100,000 endpoints
4. Open documented southbound controller-to-endpoint interface(s) for virtual network overlay endpoint provisioning and traffic policy configuration
5. Open documented controller-to-controller interfaces, enabling multi-domain federation of separate virtual overlay networks -
6. Open documented northbound virtual network controller interface to cloud orchestration systems
7. Layer-2 and layer-3 traffic mapping and distributed virtual network overlay packet forwarding
8. Networking Virtual Edge (NVE) monitoring capabilities, using standard management protocols and APIs
9. End-to-end network overlay monitoring to validate network reachability and performance characteristics
10. Correlation of virtual network overlays with physical network underlay state and performance (Vendor’s choice)
System Under Test
Products Software Versions
F64-M 3.0
E28-Q 3.0
Ixia IxNetwork 7.40 EA
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Test Configura0on
VXLAN Tunnel
L3 CORE
Automatic tunnel provisioning! On-demand,
dynamic VXLAN encap/decap!
VLAN 100 VLAN 100 VLAN 300 VLAN 300
100K VMs • 95% line rate@10Gbps
• 0% packet drop • Convergence:
• Cold: 25.54sec • Warm: 7.35
F64 (Intel FM6000) E28 (Broadcom Trident 2)
VLAN 400 VLAN 500 VLAN 500
VLAN 400 VLAN 500 VLAN 500
Copyright 2015 Open Networking User Group. All Rights Reserved Confiden@al Not For Distribu@on
Unified Overlay and Underlay w/ Switch VTEP Off-‐load
VXLAN over BGP fabric or VXLAN for L2 POD extension § Netvisor SDN Fabric to orchestrate
and automate VXLAN tunnel offload on Leaf switches
§ Keep server networking simple and eliminate VXLAN encap/decap performance tax
§ ONE fabric with seamless overlay-‐underlay (VXLAN) visibility
!
VM VM VM
VM VM VM
VM VM VM
VM VM VM
VM VM VM
VM VM VM
L3#
L2#$#VLAG# L2#$#VLAG# L2#$#VLAG# L2#$#VLAG#
IP/ECMP'Fabric'
!
VM VM VM
VM VM VM
VM VM VM
VM VM VM
VM VM VM
VM VM VM
L3#
L2#$#VLAG# L2#$#VLAG# L2#$#VLAG# L2#$#VLAG#
IP/ECMP'Fabric'VXLAN Tunnel
POD #1 POD #2 L3 CORE
!
VM VM VM
VM VM VM
VM VM VM
VM VM VM
VM VM VM
VM VM VM
L3#
L2#$#VLAG# L2#$#VLAG# L2#$#VLAG# L2#$#VLAG#
IP/ECMP'Fabric'VXLAN Tunnel
100K VMs • 95% line rate@10Gbps
• 0% packet drop • Convergence:
• Cold: 25.54sec • Warm: 7.35
Wrap-up
• For more information: – ONUG web site: http://opennetworkingusergroup.com/spring-2015-downloads/
• Network Overlay working group • Network Overlays white paper • Network Overlay test plans
– Vendors! • What’s next – Refinement of network overlay use case tests – Open source network overlay test scripts: https://github.com/onug
• Poll results