Large-scale Virtualization in the Emulab Network Testbed

Mike Hibler, Robert Ricci, Leigh Stoller Jonathon Duerig

Shashi Guruprasad, Tim Stack, Kirk Webb, Jay Lepreau

Proc. Of the 2008 USENIX Annual Technical Conference

Hoon-gyu Choihgchoi@mmlab.snu.ac.kr

2008.10.27

Contents

• Introduction• Motivation• Key techniques• Results• Conclusion

23年 4月 8日 MMLAB 2

Introduction

• Testbed is more realistic than simulation– Experimenters can use real OS, other software, and

obtain actual performance measures

• Emulab: Network Testbed– Provide time- and space-shared public facility

23年 4月 8日 MMLAB 3

Motivation

• Emulab is not scale– Too small– Inefficient

• Original Emulab maps virtual network nodes and links one-to-one onto dedicated PCs and switched Ethernet link

• Solution– Multiplex logical nodes and networks onto the physical

infrastructure– Use Virtualization to perform network experiments using

fewer physical resources– Should be transparent to applications and should

preserves experiment fidelity23年 4月 8日 MMLAB 4

Three Goals

• Application transparency– Real application on virtual machines– Keep most semantics of unshared machines

• Application fidelity– Physical results ≈ Virtual results– Avoid virtual node interference

• System capacity– Low overhead– Don’t prolong experiments

23年 4月 8日 MMLAB 5

Key techniques

• Virtualization technology– Host and network

• Resource mapping• Feedback-directed emulation

23年 4月 8日 MMLAB 6

Virtualization

• Extending FreeBSD jail– Namespace isolation– Virtual disks– Network virtualization

• Ability to bind to multiple interfaces• New virtual network device (veth)• Separate routing tables

23年 4月 8日 MMLAB 7

Virtualization

23年 4月 8日 MMLAB 8

Mapping

• Map the virtual topology to a physical topology• Good mapping

– Pack using resources efficiently• Packs virtual hosts, routers, and links on to a minimum

number of physical nodes without overloading the physical nodes

– Do it quickly

• Use assign– A solver for the Network Testbed Mapping Problem– Pack both nodes and link– Use Resource-based Packing

• Avoid scarce resources• Works well for heterogenious virtual and physical nodes

23年 4月 8日 MMLAB 9

Mapping quickly

• Apply a heuristic pre-pass to the virtual graph before running assign– Exploit the structure of the input topology– Two coarsening algorithms

• Combine all leaf nodes from the same LAN• Graph partionining

23年 4月 8日 MMLAB 10

Mapping quickly

23年 4月 8日 MMLAB 11

200

Mapping quality

23年 4月 8日 MMLAB 12

Feedback

• Do I know how tightly I can pack my virtual nodes?– No, But we have a closed, repeatable world

• Process1. Pick a packing2. Run experiment3. Monitor for artifacts

• CPU near 100%, significant paging activity, disk utilization

4. Re-pack & Repeat• Measure resource use• Feed into resource-based packing

23年 4月 8日 MMLAB 13

Results

• Application fidelity• Adaptation• Application transparency

23年 4月 8日 MMLAB 14

Application Fidelity

• Setup– All results gathered on Emulab’s low-end “pc850”

machines• 850 MHz PCs with 512MB RAM and four 100Mb

Ethernet interfaces– Uses Pathrate bandwidth measurement tool– Running p2p file sharing application, Kindex

• Total 60 clients• Each client

– Uploads a single file’s index, and random searches for other files

– generates 20-40 request per minute

– Network topology consists of six 10Mbps campus LANs• Each LAN is connected to a router via a 3Mbps

23年 4月 8日 MMLAB 15

Application Fidelity

• Application Fidelity

23年 4月 8日 MMLAB 16

Adaptation Results

• Setup– Java-based web server on one host with 69 clients

• Each clients continually download a 64KB file• Three different types

– 9 clients were evenly spread across 3 links on a single router using 2MB LANs (to emulate cable modems)

– 40 clients were directly connected to a single router using 2MB multiplexed link (to emulate DSL modems)

– 20 clients were directly connected to a single router using 56Kb multiplexed link (to emulate phone modems)

23年 4月 8日 MMLAB 17

Adaptation results

• Feedback case study

23年 4月 8日 MMLAB 18

Conclusion

• Virtualization increase Emulab’s capacity– Transparently– Preserves fidelity

• Requires solving several challenging problems• But, can be useful

23年 4月 8日 MMLAB 19