Post on 20-Jan-2016
transcript
ITDepartment
1
Using VPLS for VM mobility
Carles.Kishimoto @ cern.chDavid.Gutierrez @ cern.ch
HEPIX Fall 2015
ITDepartment
2
Agenda
• CERN data center
• The objective
• The idea
• The solution
• Conclusions
ITDepartment
3
Agenda
• CERN data center
• The objective
• The idea
• The solution
• Conclusions
ITDepartment
4
• Switzerland• 1000 racks
• Hungary• 300 racks• Wigner Research Centre for Physics• Since January 2013• 2x 100GbE links
CERN Data Center (I)
ITDepartment
5
• Network equipment• Brocade routers• HP Procurve / HP H3C / Brocade switches
CERN Data Center (II)
ITDepartment
6
CERN network (I)
x100GbE links
Distribution CoreAccess
x10GbE
ToR x10GbE
ITDepartment
7
CERN network (II)
• Routed network (OSPF)• No vlans, no spanning-tree• ECMP and LACP• IPv4 / IPv6 Dual stack since 2013• x10GbE switch uplinks (LCG)• 100GbE p2p router interconnects (LCG)• MPLS deployed in the backbone
ITDepartment
8
Agenda
• CERN data center
• The objective
• The idea
• The solution
• Conclusions
8
ITDepartment
9
The objective (I)• Migrate virtual machines transparently to the new hardware
VirtualMachines
ITDepartment
The objective (II)
• Migrate running virtual machines transparently to the new hardware
• Conditions :• Keep the same ip address on all virtual machines• Live migration
• Restrictions:• CERN is a routed network• We do not extend VLANs across racks
10
ITDepartment
11
Agenda
• CERN data center
• The objective
• The idea
• The solution
• Conclusions
ITDepartment
The idea
MPLS / IP network
Switch B
Router A Router B
Switch A
12
Row A Row B
VM
ITDepartment
The idea
MPLS / IP network
Switch B
Router A Router B
Switch A
Multimode fiber
13
Row A Row B
VM
ITDepartment
The idea
MPLS / IP network
Switch B
Router A Router B
Switch A
VMVM
VM
14
Row A Row B
ITDepartment
The idea
MPLS / IP network
Switch B
Router A Router B
Switch A
• Connect the switches at router level
15
21 1
2
VM
ITDepartment
The idea
MPLS / IP network
Switch B
Router A Router B
Switch A
• Connect the switches logically using VPLS
16
21 1
2
VM
ITDepartment
What is VPLS ?
• Virtual Private LAN Service• RFC4761 / RFC4762
• Signaling based on BGP or LDP
• From Wikipedia: “Virtual Private LAN Service (VPLS) is a way to provide Ethernet-based multipoint to multipoint communication over IP or MPLS networks”
• You need an MPLS enabled backbone
17
ITDepartment
What is VPLS ?
MPLS / IP network
Switch B
Router A Router B
Switch A
• VPLS emulates an Ethernet switch
18
21 1
2
VM
ITDepartment
19
Agenda
• CERN data center
• The objective
• The idea
• The solution
• Conclusions
ITDepartment
The solution
vlan 51 name S513-C-IP790 untagged ethe 5/1 router-interface ve 51 loop-detection!
interface ethernet 5/1 no flow-control load-interval 30 enable!
interface ve 51 port-name VPLS#S513-C-IP790 ip ospf area 128.142.0.0 ip ospf cost 1 ip ospf passive ip address 10.10.10.1/24!
Layer 1
Layer 2
Layer 3 MPLS
20
Switch A Switch B
ITDepartment
The solution
vlan 51 name S513-C-IP790 untagged ethe 5/1 router-interface ve 51 loop-detection!
interface ethernet 5/1 no flow-control load-interval 30 link-fault-signaling!
interface ve 51 port-name VPLS#S513-C-IP790 ip ospf area 128.142.0.0 ip ospf cost 1 ip ospf passive ip address 10.10.10.1/24!
Layer 1
Layer 2
Layer 3 MPLS
21
vpls 1 1 auto-discovery vlan 51 untagged ethe 5/1 router-interface ve 51
Switch A Switch B
ITDepartment
The solution
vlan 51 name S513-C-IP790 untagged ethe 5/1 router-interface ve 51 loop-detection!
interface ethernet 5/1 no flow-control load-interval 30 link-fault-signaling!
interface ve 51 port-name VPLS#S513-C-IP790 ip ospf area 128.142.0.0 ip ospf cost 1 ip ospf passive ip address 10.10.10.1/24!
Layer 1
Layer 2
Layer 3 MPLS
22
vpls 1 1 auto-discovery vlan 51 untagged ethe 5/1 router-interface ve 51
Switch A Switch B
(config)# vlan 51(config-vlan-51)# no router-interface ve 51(config-vlan-51)# no untagged eth 5/1(config-vlan-51)# router mpls (config-mpls)# vpls 1 1 (config-mpls-vpls-1)# router-interface ve 51
ITDepartment
The loop cable (I)
MPLS / IP network
Switch B
Router A Router B
Switch A
• Loop cable installed in distribution routers
23
1 12 2
VM
ITDepartment
The loop cable (II)
MPLS / IP network
Switch B
Router A Router B
Switch A
• Routing still provided by Router A
24
VM
Internet
ITDepartment
The loop cable (III)
MPLS / IP network
Switch B
Router A Router B
Switch A
• Increase VRRP priority in router B
25
Internet
VM
ITDepartment
The loop cable (IV)
• Implemented with 10GigabitEthernet ports• Advantage
• It allows us to create the circuit transparently• We can deploy multiple links in parallel if needed• It can be used for multiple migrations
26
ITDepartment
Operations
• Workflow
1. Service manager will create a ticket
2. Network team will enable the circuit
3. Migration of virtual machines
4. The circuit will be removed
• Open questions:
• Duration of the circuit ?
• Number of circuits simultaneously ?
27
ITDepartment
28
Agenda
• CERN data center
• The objective
• The idea
• The solution
• Conclusions
28
ITDepartment
Conclusions
• It allows us to connect the broadcast domain of different IP services transparently
• It supports multiple circuits at the same time• The solution is based on standard protocols (RFC) • It works… although not tested in production yet• Our tools are ready to support this configuration• It scales as we could connect two IP services from
different data centers (Geneva vs Budapest)
29
ITDepartment
30
Thank you