1
Attilla de Groot
Host multitenancy
Attilla de Groot | Sr. Systems Engineer, HCIE #3494 | Cumulus Networks
EVPN to the host
2
Agenda
EVPN to the Host§ Multi tenancy use cases§ Deployment issues§ Host integration with EVPN§ Caveats and future work
3
Multi tenancy use casesVirtualization environments
VM environments§ MLAG to hypervisors§ Vxlan between hosts§ Double overlay§ Dedicated network nodes
Mlag
VM1 VM1
vswitch1 vswitch2
VM1 VM1
vswitch1 vswitch2 vswitch1 vswitch2
vrouter
hypervisor1 hypervisor2 networknode
Mlag
BGP BGP
4
Multi tenancy use casesVM deployment issues
Architecture issues§ MLAG to hypervisors§ Vxlan between hosts§ Double overlay§ Dedicated network nodes§ Orchestration problems
Mlag
VM1 VM1
vswitch1 vswitch2
hypervisor1
bare metal host
Mlag
OVSDBML2
Controller
5
Multi tenancy use casesContainer environments
Container environments§ BGP to hosts§ Host route advertisement § Docker management§ Container overlay
Container1
containervisor1 containervisor2
Container2 Container3 Container4
RoutingDaemon
IP fabric
RoutingDaemon
BGP
BGP BG
P
BGP
ACLsACLs
6
Multi tenancy use casesContainer deployment issues
Architecture issues§ Host networking§ Multi-tenancy§ IP prefix overlap§ ACL management
containervisor
RoutingDaemon
BGP
BGP
ACLs
Container210.1.1.1/32
2001:DB8::1/128
Container110.1.1.1/32
2001:DB8::1/128
tenant1 tenant2
7
Host integration with EVPNOpen standards
EVPN on hosts§ Vlan aware bridge§ VRF in Linux kernel§ VxLAN§ Free Range Routing with EVPN§ Ifupdown2§ Iproute2
cumulus@server01:~$ uname -aLinux server01 4.17.0-041700-generic #201806041953 SMP Mon Jun 4 19:55:25 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
cumulus@server01:~$ vrf list
VRF Table---------------- -----mgmt 1001tenant3 1002tenant2 1003tenant1 1004
8
Host integration with EVPNHost connectivity
Host connectivity§ Routing to hosts§ BGP unnumbered§ RFC5549§ Loopback advertisement§ EVPN address family
auto eth1iface eth1
auto eth2iface eth2
router bgp 65501bgp router-id 10.250.250.1bgp bestpath as-path multipath-relaxneighbor FABRIC peer-groupneighbor FABRIC remote-as externalneighbor FABRIC timers 1 3neighbor eth1 interface peer-group FABRICneighbor eth2 interface peer-group FABRIC!address-family ipv4 unicastredistribute connected route-map loopbacksexit-address-family!address-family l2vpn evpnneighbor FABRIC activateadvertise-all-vniadvertise ipv4 unicastadvertise ipv6 unicast
exit-address-family!route-map loopbacks permit 10match interface lo
interface eth1ipv6 nd ra-interval 10no ipv6 nd suppress-rainterface eth2ipv6 nd ra-interval 10no ipv6 nd suppress-ra
9
Host integration with EVPNL2 tenancy
L2 tenancy§ Vlan-aware bridge§ ARP / ND suppression § L2VNIs§ EVPN Type-2
vlan xxxVM1 eth0 vethX
Vlan aware bridge
Hypervisor1
vlan xxx
Vlan aware bridge
Hypervisor2
VM2vethX eth0
vni xxx vni xxxVxlan tunnel
assigned to local vlan
Vxlan tunnel
assigned to local vl
an
BGPdFRR
BGPdFRR
learn
ed m
acs
adde
d to f
db
learned macs
added to fdb
EVPN Type 2,3 (VNI & MAC/IP) exchange.Messages via spine/leaf nodes
10
Host integration with EVPNDistributed routing
Distributed routing§ Gateway location§ SVIs on host§ Anycast gateway§ Local host routing
vlan yyy
Hypervisor1
VM2
vethX
vrf xxx
vlan xxx
vethX
VM1
eth010.1.1.10/24
svi10.1.1.1/24
vrf yyyl3vni l3vni
eth1 eth2
svi10.1.2.1/24
l2vnil2vni
eth010.1.2.10/24
vlan yyy
Hypervisor2
VM2
vethX
vrf xxx
vlan xxx
vethX
VM1
eth010.1.1.11/24
svi10.1.1.1/24
vrf yyyl3vni l3vni
eth1 eth2
svi10.1.2.1/24
l2vnil2vni
eth010.1.2.11/24
11
Host integration with EVPNL3 tenancy
L3 tenancy§ VRFs on host§ Prefix advertisement§ L3VNIs§ EVPN Type-5
vlan xxxVM1 eth0 vethX
Vlan aware bridge
Hypervisor1
vlan yyy
Vlan aware bridge
Hypervisor2
VM2vethX eth0
Vxlan tunnel
assigned to local VRF
Vxlan tunnel
assigned to local VRF BGPd FRR BGPdFRR
learned macs/ips
added to fd
b and
routing table
learned macs/ips
added to fdb and
routing table
EVPN Type 2,3,5 (VNI, MAC/IP & prefix route) exchange.Messages via spine/leaf nodes
vrf xxxvrf xxx l3vni xxx
svi xxx
l3vni xxx
svi yyy
12
Host integration with EVPNContainer integration
Container integration§ Container IP redistribution§ Host route advertisement§ Prefix overlap§ No ACLs for tenant segregation
Containervisor
container2
vrf xxx
container1
eth010.1.1.10
vrf yyyl3vni l3vni
eth010.1.2.10
Dockerplugin
FRR
routing table X
routing table Y
13
Host integration with EVPNBare metal integration
Bare metal integration§ Integration with RFC standard§ L2 stretching§ L3 tenancy§ Distributed routing
VM1 VM2
hypervisor1
bare metal host
Mlag
FRR
L2VNIL2VNI
BGP
BGP
L2VNI L3VNI L2VNI L3VNI
VTEPAnycast
14
Future workProof of technology
Proof of technology§ Commercial support
§ Software / tools availability
Linux kernel
FRR
§ Demo
cumulus@server01:~$ vrf list
VRF Table
---------------- -----
mgmt 1001
tenant3 1002
tenant2 1003
tenant1 1004
cumulus@server01:~$ bridge vlan
port vlan ids
bridge 1 PVID Egress Untagged
1010
1011
1012
1013
1014
cumulus@server01:~$ uname -a
Linux server01 4.17.0-041700-generic
#201806041953 SMP Mon Jun 4 19:55:25 UTC
2018 x86_64 x86_64 x86_64 GNU/Linux
cumulus@server01:~$ sudo vtysh -c "show bgp evpn vni"
VNI Type RD Import RT Export RT Tenant VRF
* 101010 L2 10.250.250.1:5 65501:101010 65501:101010 tenant1
* 101012 L2 10.250.250.1:3 65501:101012 65501:101012 tenant2
* 101014 L2 10.250.250.1:7 65501:101014 65501:101014 tenant3
* 404001 L3 172.30.11.1:8 65501:404001 65501:404001 tenant1
* 404002 L3 172.30.13.1:9 65501:404002 65501:404002 tenant2
* 404003 L3 172.30.15.1:10 65501:404003 65501:404003 tenant3
https://gitlab.com/CumulusNetworks/evpn-to-the-host
15
Future workOrchestration & Caveats
Orchestration§ Openstack neutron§ Kubernetes / Swarm§ Host / Network management
Head-end replication§ BUM anycast§ Merchant silicon limitations§ Multicast replication§ PIM-SM
16
Future workRoute leaking & Micro segmentation
Route-leaking§ Inter tenant traffic§ Service tenant§ FRR implementation
Micro segmentation§ Host traffic filtering§ Filtering with BPF§ Flowspec for ACL distribution
17
EVPN to host demo
18
EVPN to the Host
Questions ?
19
Thank you!Visit us at cumulusnetworks.com or follow us @cumulusnetworks
© 2018 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark
Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.