Enterprise/Govt.SMBResidential Mobile
Access Metro Backhaul/Core Data Center
1
Ethernet Virtual Private Networks (EVPN)for Data Center Interconnect and other Applications
Tutorial
2
Introduction
Ethernet VPN delivers next-generation business, cloud, video and mobile transport services, including those applicable to 5G, in a
way that ensures optimal customer experience by means of world class flexible, efficient and highly resilient services.
This tutorial goes through the market drivers, technology overview and key use cases of EVPN services.
3
Agenda 1. Ethernet VPNs and Market Drivers
2. EVPN Concepts and Service Interfaces
3. EVPN Use-Cases
4. Technology Deepdive
5. Network Virtualization Overlay with EVPN
6. Conclusion
Topics
Enterprise/Govt.SMBResidential Mobile
Access Metro Backhaul/Core Data Center
4
Ethernet VPNs and Market Drivers
5
Ethernet VPN Service Revenues
• With more than $70B predicted by 2020, Ethernet technology and services continues to be a huge industry-wide source of revenue
• Significant growth in cloud infrastructure build-up
6
IP VPN Service Revenues
• > $30B by 2020, IP VPN services continue to be a key revenue source
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
2013 2014 2015 2016 2017 2018 2019 2020
Rev
enu
es (
$m
)
IPsec-based
MPLS-based
7
EVPN’s Competitive Advantage
• EVPN agility– Simplicity – One technology for today’s and future services
– Enables Scalable, Flexible and Seamless connectivity across the datacenter and WAN
Services
ELAN
ELINE
ETREE
L3 VPN
Traditional
VPLS, PBB-VPLS
PW
VPLS ETREE
RFC4364
Cloud and DCI*
Unified architecture for VPN and cloud based services
Cloud and DCI de-facto standard for seamless connectivity
Higher efficiency, scale, maximum flexibility and greater control
Evolution
EVPN
Networking technologies
8
Cloud-based Data Center Interconnect
• Data Center Interconnect (DCI) at L2/L3 extends connectivity between data centers and from the data center to end users
• Infonetics estimates that routers for DCI will make up between 5% and 10% of the overall router market over the next few years [Router and Switch Vendor Leadership – Global Service Provider Survey June 30, 2015, page 6]
IP/MPLS Network
E2E EVPN
Control plane (EVPN)Data plane
(VXLAN/MPLS)
Non SDN-enabled data
center
Data center network IP/MPLS WAN Data center network
HVM
HVM
SDN Controller
Route Reflector
CE
SDN-enabled data center
DC-GW DC-GW
PEPE
PE PE CE
e.g. VLAN
Enterprise/Govt.SMBResidential Mobile
Access Metro Backhaul/Core Data Center
9
EVPN Concepts and Service Interface Overview
10
EVPN Concepts (ESI, E-Tag, EVI, …)
CE
PE
LAG
PE
EVI 1
EVI 1
EVI 1EVI 1
EVI 1
EVI 1 EVPN MAC/IP
BGP updates
PE
PE
LAG
CE
PEPE
Control Plane learningPEs Advertise MAC Addresses and Next Hops from connected CEs using MP-BGP
Data Plane learningDynamic or Static (Provisioned), Management Protocol
Customer Edge (CE)Host, Router or Switch
EVPN Instance (EVI)Identifies a VPN
Ethernet TagBroadcast or Bridge Domain in the EVI
Data Plane encapsulationMPLS or IP
Ethernet Segment Identifier (ESI)Identifies all links that connect a given CE to the PEs (ESIs are unique across the network)
All-Active ModeMultihomed, two or more active PEs
Single-Active ModeMultihomed, one active PE
11
EVPN Service Interface Overview
12
EVPN Technology Benefits
What requirements does EVPN address
• Support All-Active multi-homing
• Minimizes flooding of BUM traffic / improve Learning
• Improves convergence using aliasing
• Support optimized VM mobility
• Allows fine grained, policy driven control on route advertisement control
• Integrate with next generation of transport technologies (e.g. VXLAN/IP)
• Simplified deployment with a single VPN technology for L3 and L2 VPNs
How is EVPN better than today’s technology
• EVPN provides ability to program remote MAC-addresses in the control plane using BGP as the transport protocol
• Data plane learning is limited to PE-CE
• Protocol natively supports multi-homing, resiliency, MAC mobility and L3 aware inter-subnet routing features
• Supports integrated routing and bridging solution with MAC/IP reachability across different VLAN
Enterprise/Govt.SMBResidential Mobile
Access Metro Backhaul/Core Data Center
13
EVPN Use Cases
14
Application of Ethernet VPN Technology
EVPN Use Cases
• Next gen L2VPN technology for E-Line/E-LAN/E-Tree services
• DC Interconnect – for L2 or L3 service stretched between two DCs over WAN
• EVPN control-plane for VXLAN forwarding plane for L2 or L3 aware subnet overlay on IP networks (in DC or WAN)
Which customers are interested in EVPN and why
• Service providers that offer E-LAN / E-Line services– EVPN technology improves their service offering
– Operators can replace VPLS, VPWS with more efficient technology
• Data Center Builders – SPs, Enterprises, Content providers– EVPN allows multi-tenant L2 service stretch between DCs
– EVPN with VXLAN for L2 or L3 aware service stretch between VMs on a IP fabric DC
15
BBF TR-350 – Ethernet Services
• Specifies end-end architecture, equipment requirements and common set of feature options to support carrier Ethernet services using BGP MPLS based EVPN
• Promotes multi-vendor interoperability
• Supports the following Ethernet service capabilities:
– MEF Carrier Ethernet services ( E-LAN currently supported. E-Line and E-Tree work ongoing)
– Ethernet service capabilities required by RFC 7209 (e.g. multi-homing with All-Active, load balancing, policy)
– Multi-service broadband access and aggregation (TR-178)
• MEF defined service attributes are supported using BGP MPLS EVPNs
16
EVPN Ethernet service overview
• Ethernet service traffic is tunneled over a packet network such as MPLS or IP
• Unicast, unknown unicast and multi-destination traffic is supported
• Remote MAC addresses are learnt using a control-plane protocol (MP-BGP) but local MAC addresses continue to be learnt using standard IEEE procedures
• CE treats multiple links connected to PE(s) as a LAG. It uses local hashing to map traffic flows onto links in the LAG
PE
LAG
PE
MP-BGP
PE
PE
RR
CE
MEF UNI
CEMEF UNI
CEMEF UNI
CE
MEF UNI
LAG
17
Ethernet Private LAN service
• EVPN Port-based Service Interface is used
• Ethernet Segment is the entire UNI
• CE-VLAN configuration at customer sites does not need co-ordination with the service provider. VID is preserved across the network
PE
LAG
PE
IP/MPLS
Network
PE
PE
CE
MEF UNI
CEEthernet Segment
/ MEF UNICEMEF UNI
CE
MEF UNI
LAG
CE CEMEF UNIMEF UNI
EP-LAN 1
EP-LAN 2
18
Ethernet Virtual Private LAN service
• EVPN service interfaces that are VLAN based are used
• Services can be multiplexed on a PE-CE link
• All to one bundling is not allowed
PE
LAG
PE
IP/MPLS
Network
PE
PE
CE
MEF UNI
CEEthernet Segment
/ MEF UNI
CE
MEF UNI
LAG
CE MEF UNI
EVP-LAN 1
EVP-LAN 2
19
Ethernet Line service
• An EVI would have exactly two endpoints– Both Private Line and Virtual Private Line services are provided
• Supports all-active multi-homing
• Supports flow based load-balancing in the MPLS network
• New BGP extended community for layer 2 attributes– e.g. Control-word, MTU, multihoming
PE
LAG
PE
IP/MPLS
Network
PE
PE
CE
MEF UNI
CEEthernet Segment
/ MEF UNICEMEF UNI
CE
MEF UNI
LAG
EP-LINE 1
EP-LINE 2
20
Ethernet Tree service
• E-Tree is a bidirectional communication service between root and leaf nodes
• An E-Tree has the restriction that leaf nodes cannot communicate with each other
• Work is in progress at IETF to extend RFC 7432 to support E-Tree (draft-ietf-bess-evpn-etree)
• Both Ethernet Private Tree (EP-Tree) and Ethernet Virtual Private Tree (EVP-Tree) can be supported
• RFC 7432 may be used as-is to implement TR-221 (E-Tree*)
21
EVPN for Data Center Interconnect
Benefits
• Seamless interconnect for DCI – L3 aware and L2 stretch between DCs
• Seamless workload migration - VM mobility across DCs
• Wide Applicability – Interconnects Native L2 or L3 and overlay DC technologies like VXLAN, MPLS in DC
IP/MPLS Network
BGP Control Plane based learning in the DC
BGP Control Plane based learning in the WAN
PE
PE
PE
PE
SDN Controller
EVPN/VXLANData Center network
LegacyData Center network
Data Center Site 1 Data Center Interconnect Data Center Site 2
22
L2 or L3 VPNs over IP WAN
EVPN implemented on enterprise' edge routersVXLAN data plane over operator's IP WAN
Benefits
• Data Center / Enterprises buy Simpler IP connectivity in WAN– No need to buy expensive VPN service, depending on requirements (e.g. SLA)
– EVPN overlay is transparent to service providers
• VXLAN Tunnel Endpoints (VTEP) on enterprises’ WAN (CE) routers
EVPN-VXLAN
SP WANNetwork
PE
EVI1
EVI1
EVI1
EVI1
CorporateNetwork
EVPN MAC/IP
BGP updates
CEPE
PE PE
CE
CE
CECorporateNetwork
23
L2 Peering Service
Benefits
• Allows policy driven peering relationship to clients
– Different types of peering relationships (public peering / private peering) with same infrastructure, driven by policies
• Supports explosive traffic growth and high availability
– Active/Active multi-homing with load balancing
– Each ASBR connecting with two peering switches improves availability
– Capability to do ARP/ND proxy
PE
PE
PE
PE
RR
ASBR
LAG
ASBR
LAG Peering providerIP/MPLS network
ISP 1network
ISP 2network
BGP signaling on WAN exchange MAC/IP routes
Peering handoffpoint
Peering handoffpoint
MP-BGP
24
EVPN Use-Cases in Next-Gen Applications
EVPN for GiLAN
• EVPN can be used over IP fabric of DC and WAN IP/MPLS network
• Many Gi-LAN virtualized VAS can be deployed in the packet core and chained as needed
• Virtualized VAS can be scaled up/down on demand and deployed in a redundant manner
• EVPN with VXLAN can support VM migration
• RR can be virtualized and deployed flexibly in the network
EVPN for IoT
• EVPN supports large number of endpoints as required by IoT
• Simple endpoint provisioning enables creating services that can be deployed cost-effectively
• Diverse access technologies can be translated to a simple Ethernet access to an EVPN Instance
• QoS over the core ensures that differentiated services can be created with the required QoE (Quality of Experience)
IoT GW
IoT GW
IoT GW
PE
PE
Transport network
CE
CEAccess 1(Zigbee)
Access 2(MACSec)
Access 3(3G/4G/LTE/Wi-Fi)
68o
68o
68o
Mobile Backhaul
SDN Controller
EVPN/VXLANData Center network
PGW VM
VM
VM
VM
VM
DC-GW
DC-GW
PE
IP/MPLS network
CE
CE
PE
PE
VPN1
VPN2
RR
Enterprise/Govt.SMBResidential Mobile
Access Metro Backhaul/Core Data Center
25
Technology Deepdive
26
EVPN Route Types and Benefits
Route Usage Business Benefit
Ethernet A-D Route (Type 1) • Advertising Split-Horizon Labels• MAC Mass-Withdraw• Aliasing
Loop Avoidance Fast Convergence
MAC Advertisement Route (Type 2)
• Advertise MAC Address Reachability• Advertise IP/MAC Bindings
Policy control
Inclusive Multicast Route (Type 3)
• PMSI tunnel attribute• BUM flooding
Set up path for BUM traffic
Ethernet Segment Route (Type 4)
• DF Election for multi-homing Loop avoidance
IP Prefix Route(Type 5)
• IP route advertisement without host MAC
L3 Routing Integration
27
MAC Route
• Advertises host MAC (and host IP) reachability with “service label”
– Allows Control Plane based MAC learning for remote PEs
– Minimizes flooding across WAN
– Allows PE to do proxy-ARP for remote hosts locally
• IRB MAC address route has default GW extended community
– Used in VM motion when default GW of VM remains same
• If IRB MACs and IP are same across MH PEs, avoids flooding after node failure
ESI
RD
Ethernet Tag ID
MAC Address
IPv4 or IPv6 Address
Service tags (MPLS labels)
MAC reachability advertisement
Establishes Reachability
PE
PE
PE
PE (DF)
LAGLAG
IP/MPLSnetwork
MP-BGP signaling on WAN
CECE
PEs learn MACs on CE-PE link and advertise its reachability in EVPN MAC routes
EVPN MAC/IP
BGP updates
Single-Active ESI All-Active ESI
28
Inclusive Multicast Route
• Allows PE to send BUM traffic from a CE on a VLAN in an EVI, to all the other PEs that span that VLAN in that EVPN instance
• Uses Existing MVPN defined constructs for signaling and transport– P2MP Tunnel: If advertising PE uses a P-Multicast tree for EVPN, the PMSI Tunnel attribute MUST
contain tree identity
– Ingres Replication: Route includes PMSI Tunnel attribute with Tunnel Type set to Ingress Replication and Tunnel ID as PE address
• Able to carry the traffic of more than one EVPN instance on the same tree using ’Aggregation’
IMET Route advertisement
Sets up path for BUM trafficPer VLAN per EVI
Ethernet TAG ID
RD
Originating PE IP Address
PE
PE
PE
LAGLAG
IP/MPLSnetwork
MP-BGP signaling on WAN
CECE
EVPN IMET
BGP updates
Single-Active ESI All-Active ESI
PE (DF)
RR
29
Unique EVPN Capabilities
Key Control Plane Features• All-Active Multihoming and Designated
Forwarder Election
• All-Active Multihoming and Split Horizon
• Aliasing for load balancing
• MAC Mass-Withdraw for fast convergence
• Default Gateway Inter-Subnet Forwarding
• ARP proxy for peer’s IRB address of a BD in an EVI
• MAC Mobility Extended Community
• ARP/ND Proxy and Unknown Unicast Flooding Suppression for remote hosts
EVPN Data Plane Options• MPLS or IP/GRE as transport for EVPN
• PBB-EVPN – PBB frames are encapsulated over MPLS similar to Ethernet
• VXLAN – VXLAN overlay network transport Ethernet frames over UDP/IP
30
Carrier Class scaling with EVPN
Control Plane
• Propagation of EVPN routes for a given VLAN/subnet is constrained by the provisioning of PEs
• Route Reflector infrastructure
• Use of RTC to limit EVPN route distribution to participating PEs
Data Plane
• The presence of a MAC address in the control plane does not imply that it must be installed in the forwarding plane
• PBB for EVPN can further ease MAC table sizes
31
PBB-EVPN Key Concepts (RFC 7623)
BMAC Control Plane learning• System BMACs are advertised by MP-BGP• ESI BMACs are advertised by MP-BGP
CMAC Data Plane learning• CMAC-local AC or CMAC to remote BMAC
mapping is learnt in the data plane
PBB-EVPN combines 802.1ah and EVPN• PEs have I-components mapped to B-
components (EVIs)• Reduces the number of MACs in EVPN by
aggregating CMACs with BMACs
Used for layer-2 EVPN networks• All EVPN Multihoming functions are supported,
including Single-Active and All-Active• Per-ISID flooding tree are supported• The B-component EVI uses MPLS data plane
PE
LAG
PE
PE
PE
CE
MEF UNI
CE
MEF UNI
LAG
BMAC BGP
update
PBB MAC
Mapping
B-component
EVI 1
EVI 1EVI 1
EVI 1
ISID-1
IP/MPLSnetwork
32
Integrated Routing and Bridging (IRB)
EVPN provides layer-2 and layer-3 services• Both services are provided through the same logical
AC to the customer• One VPN technology for both services, no need for
multiple protocols• VXLAN or MPLS data planes are possible
Required EVPN features• IP-prefix advertisement and inter-subnet forwarding• All-active multihoming for load balancing• Single-active multihoming for better determinism
Asymmetric IRB model: draft-ietf-bess-evpn-inter-subnet-forwarding
Symmetric IRB model: draft-ietf-bess-evpn-prefix-advertisement
PE
PE
PE
CECE
MEF UNI
LAG
EVPN MAC/IP
updates
IP/MPLSnetwork
MEF UNI
EVI 3EVI 2
EVI 1
VRF
EVI 3
EVI 2
EVI 1
EVI 3
EVI 2
EVI 1
LAG
Enterprise/Govt.SMBResidential Mobile
Access Metro Backhaul/Core Data Center
33
Network Virtualization Overlay with EVPN
34
` ` `` ` `
Traditional Datacenter Design
Challenges:• MAC Address Explosion• VLAN ID Limitations• Subnet alignment with L2 domains
Result in:• Asset isolation• Complex compute migrations• Complex provisioning of new endpoints• Vendor lock within the DC• Constrained growth / moves within a POD
L2 Networking- VLAN Separation- xSTP or MC-LAG- Vendor specific L2
enhancements
VM Move / Balance Domain- Rows or Rack
WANNetwork
35
Data Center network direction
• Data Center networks will have to increasingly adapt virtualized workload and on going enterprise transition to private and hybrid clouds
• DC architecture is changing from traditional L2 to Spine – Leaf
• Infonetics Research report on “Data Center Strategies” Sept. 2014 provides preferred DC fabric features. They are:– High speed
– Automatically adjust to VM mobility
– Low latency
– Multipath connectivity
• Virtual Machines per servers are increasing by 50% in 2 years
• Data Center Interconnect mechanism has to support: Bridging, Routing and Overlay
36
` ` ` ` ` `
Combination of EVPN and VXLAN for scalable DC
VXLAN (RFC 7348)• Scalable IP tunneling encapsulation that allows
multi-tenancy (VNI)• Uses UDP source port to provide entropy and
ECMP in the IP fabric• De-facto standard in server NICs and DC gateways
BGP-EVPN (RFC 7432)• Provides massive control plane scalability• Auto-discovery of remote VTEPs• Advertising of MACs, IPs and IP prefixes• Cloud-optimized:• Mobility• Protection• Proxy-ARP/ND• Inter-subnet forwarding
WANNetwork
MAC
Pay
load
MAC
IP
UDP
VXLAN
MAC
Pay
load
MAC
Pay
load
DCNetwork
VMTenant 2
EVI 1
EVI 2
EVI 1
EVI 2
VMTenant 1 VM
Tenant 2
VMTenant 1
Hypervisor
EVI 1
EVI 2
EVI 1
EVI 2
Enterprise/Govt.SMBResidential Mobile
Access Metro Backhaul/Core Data Center
37
Conclusion
38
EVPN standardization highlights
• RFC 7209: Requirements for Ethernet VPN (EVPN)
• RFC 7432: BGP MPLS Based Ethernet VPN: RFC 7432
• RFC 7623: Provider Backbone Bridging Combined with Ethernet VPN (PBB-EVPN)
• draft-ietf-bess-evpn-vpws: VPWS support in EVPN
• draft-ietf-bess-evpn-etree: E-TREE Support in EVPN & PBB-EVPN
• draft-ietf-bess-evpn-inter-subnet-forwarding: Integrated Routing and Bridging in EVPN
• draft-ietf-bess-evpn-prefix-advertisement: IP Prefix Advertisement in EVPN
• draft-ietf-bess-evpn-overlay / draft-ietf-bess-dci-evpn-overlay: EVPN Overlay networks (interconnect to WAN services)
• BBF TR-350: Ethernet Services using BGP MPLS Based EVPN
39
Summary
• EVPN is a versatile technology that overcomes the limitations of VPLS and supports multiple deployment options
• EVPN’s architectural principles are similar to the well deployed BGP/MPLS IP VPNs. It can use multiple data plane technologies such as MPLS, VXLAN and IP
• EVPN provides several key benefits (i.e. integrated services, network efficiency, design flexibility, simplified provisioning) to Service Providers and simplifies their network while allowing to offer advanced services to customers. Further it reduces OPEX
• Data Centers interconnected using EVPN support business continuity, disaster prevention and workload mobility
• Cloud computing and NFV are shifting DC networks to SDN-based DCs where VXLAN and EVPN provide the required capabilities
40
Contributors
• Guiu Fabregas – Nokia, Editor
• J. Rao Cherukuri – BBF Distinguished Fellow
• Sachin Natu – Juniper
• Ravi Shekhar – Juniper
Enterprise/Govt.SMBResidential Mobile
Access Metro Backhaul/Core Data Center
41Thank you
Getting InvolvedAccelerate new opportunities for your company by actively participating in the work of the BBF
More at broadband-forum.org
42
AbbreviationsARP – Address Resolution Protocol
AS – Autonomous System
BBF – Broadband Forum
BGP – Border Gateway Protocol
B-MAC – Provider Backbone MAC
CE – Customer Edge device
C-MAC – Customer/Client MAC
DF – Designated Forwarder
DWDM – Dense Wave Division Multiplexing
EVI – EVPN Instance
ES – Ethernet Segment
ESI – Ethernet Segment Identifier
FRR – Fast ReRoute
IETF – Internet Engineering Task Force
IP/MPLS – Internet Protocol / Multi Protocol Label Switching
43
Abbreviations (2)IRB – Integrated Routing and Bridging
I-SID – Service Instance Identifier
L2VPN – Layer 2 Virtual Private Network
LLDP – Link Layer Discovery Protocol
MP-BGP – Multiprotocol BGP
MPLS – Multiprotocol Label Switching
MTU – Maximum Transmission Unit
ND – Neighbor Discovery
NFV – Network Function Virtualization
NLRI – Network Layer Reachability Information
P2MP – Point to Multipoint
P2P – Point to point
PBB – Provider Backbone Bridging
PE – Provider Edge (Node)
PMSI – Provider Multicast Service Interface
PW - Pseudowire
44
Abbreviations (3)RD – Route Distinguisher
RFC – Request For Comments
RT – Route Target
SDN – Software Defined Networks
SLA – Service Level Agreement
STP – Spanning Tree Protocol
TR – Technical Report
UNI – User to Network Interface
UDP – User Datagram Protocol
VID – VLAN ID
VLAN – Virtual Local Area Network
VPLS – Virtual Private LAN Service
VPN – Virtual Private Network
VRF – Virtual Routing and Forwarding
VTEP – VXLAN Tunnel Endpoint
VXLAN – Virtual eXtensible Local Area Network
4545