Xen and the Art of

Virtualization

Ian PrattUniversity of Cambridge and Founder of XenSource Inc.

Computer Laboratory

Outline

Virtualization overview Xen Today : 2.0 Overview Architecture Performance Live VM Relocation Xen 3.0 Roadmap (Q2 2005)

Virtualization Overview Single OS image: Ensim, Vservers, CKRM

Group user processes into resource containers Hard to get strong isolation

Full virtualization: VMware, VirtualPC Run multiple unmodified guest OSes Hard to efficiently virtualize x86

Para-virtualization: UML, Xen Run multiple guest OSes ported to special arch Arch Xen/x86 is very close to normal x86

Xen Today : 2.0 Features Secure isolation between VMs Resource control and QoS Only guest kernel needs to be ported

All user-level apps and libraries run unmodified Linux 2.4/2.6, NetBSD, FreeBSD, Plan9

Execution performance is close to native Supports the same hardware as Linux x86 Live Relocation of VMs between Xen nodes

Para-Virtualization in Xen Arch xen/x86 : like x86, but replaces

privileged instructions with Xen hypercalls Avoids binary rewriting and fault trapping For Linux 2.6, only arch-dep files modified

Modify OS to understand virtualised env. Wall-clock time vs. virtual processor time

• Xen provides both types of alarm timer Expose real resource availability

• Enables OS to optimise behaviour MMU virtualisation: direct vs. shadow mode

I/O Architecture Xen IO-Spaces delegate guest OSes

protected access to specified h/w devices Virtual PCI configuration space Virtual interrupts

Devices are virtualised and exported to other VMs via Device Channels Safe asynchronous shared memory transport ‘Backend’ drivers export to ‘frontend’ drivers Net: use normal bridging, routing, iptables Block: export any blk dev e.g. sda4,loop0,vg3

Xen 2.0 Architecture

Event Channel Virtual MMUVirtual CPU Control IF

Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE)

NativeDeviceDriver

GuestOS(XenLinux)

Device Manager & Control s/w

VM0

NativeDeviceDriver

GuestOS(XenLinux)

UnmodifiedUser

Software

VM1

Front-EndDevice Drivers

GuestOS(XenLinux)

UnmodifiedUser

Software

VM2

Front-EndDevice Drivers

GuestOS(XenBSD)

UnmodifiedUser

Software

VM3

Safe HW IF

Xen Virtual Machine Monitor

Back-End Back-End

System Performance

L X V USPEC INT2000 (score)

L X V ULinux build time (s)

L X V UOSDB-OLTP (tup/s)

L X V USPEC WEB99 (score)

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

1.1

Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U)

TCP results

L X V UTx, MTU 1500 (Mbps)

L X V URx, MTU 1500 (Mbps)

L X V UTx, MTU 500 (Mbps)

L X V URx, MTU 500 (Mbps)

0.0

0.1

0.20.3

0.40.50.60.7

0.80.9

1.0

1.1

TCP bandwidth on Linux (L), Xen (X), VMWare Workstation (V), and UML (U)

Scalability

L X2

L X4

L X8

L X16

0

200

400

600

800

1000

Simultaneous SPEC WEB99 Instances on Linux (L) and Xen(X)

Live VM Relocation Why is VM relocation useful?

Managing a pool of VMs running on a cluster Taking nodes down for maintenance Load balancing VMs across the cluster

Why is it a challenge? VMs have lots of state Some VMs will have soft real-time

requirements• E.g. web servers, databases, game servers

Can only commit limited resources to migration

Rate Limited Migration

Quake 3 Server migration

Roadmap for Xen 3.0 SMP guest OSes

Prototype now working, undergoing tuning Required careful design to get good

performance and retain security guarantees

Support for Intel VT-x extensions Run ‘legacy’ unmodified OSes

Other ports : x86/64 and ia64 (ppc) Both Xen x86/64 and ia64 now boot!

Roadmap for Xen 3.0 Better cluster management tools

Manage pool of VMs across a set of nodes

Better tools for QoS control New GUI management tool Improved hardware compatibility

Graphics cards, ACPI, APM

Research Roadmap : 4.0 Cluster load balancing algorithms

Exploit properties of live migration Software fault tolerance

Exploit deterministic replay System debugging

Lightweight checkpointing and replay VM forking

Lightweight service replication, isolation Secure virtualization

Multi-level secure Xen

Conclusions Xen is a complete and robust GPL VMM Outstanding performance and scalability Excellent resource control and protection Live relocation makes seamless migration

possible for many real-time workloads

http://xensource.com http://xen.sf.net