Challenges to build virtual machines Performance isolation
Process Scheduling
Memory Usage
Network Traffic
Disk Access
Support for various OS platforms
Minimizing performance overhead
Conventional
Full Virtualization GuestOS runs without
modification
Guest OS cannot access the hardware directly
Problematic for certain privileged instructions (e.g., traps)
No real-time guarantees
Xen
Paravirtualization Modifications to the
GuestOS necessary
Guest OS runs in parallel with other modified systems
Provides some exposures to the underlying HW
Multiplexes resources at the granularity of an entire OS Follows the ideology of Exokernel
As opposed to process-level multiplexing
Price: higher overhead
Target: 100 virtual OSes per machine
Depending on the hardware supports Software managed TLB
Associate address space IDs with TLB tags Allow coexistence of OSes Avoid TLB flushing across OS boundaries
X86 does not have software managed TLB Xen exists at the top 64MB of every address space Avoid TLB flushing when an guest OS enter/exist Xen Each OS can only map to memory it owns Writes are validated by Xen
X86 supports 4 levels of privileges 0 for OS, and 3 for
applications
Xen downgrades the privilege of OSes
System-call and page-fault handlers registered to Xen
“fast handlers” for most exceptions, Xen isn’t involved
0
1
2
3
Privilege Level of
Guest OS
Separation of policy and mechanism
Domain0 hosts the application-level management software Creation and deletion of
virtual network interfaces and block devices
Control Plane
Software
GuestOS (XenoLinux)
User Software
GuestOS (XenoLinux)
User Software
GuestOS (XenoBSD)
User Software
H/W (SMP x86, Phys Mem, eNet, SCSI/IDE)
GuestOS (XenoXP)
Xeno - Aware Device Drivers
Xeno - Aware Device Drivers
Xeno - Aware Device Drivers
Xeno - Aware Device Drivers
Domain0 Control Interface
Virtual x86 CPU
Virtual Physical Memory
Virtual Network
Virtual Block
Device
X E N
Hypercall: synchronous calls from a domain to Xen Analogous to system calls Allows domains to perform a synchronous software trap
into the hypervisor to perform privileged operation
Events: asynchronous notifications from Xen to domains Replace device interrupts Lightweight notification of important system events,
similar to Unix signal Event handling can be deferred by domain
Safe indirect way to share I/O devices among OSes
Circular queue accessible by Xen and a domain
Borrowed virtual time scheduling Allows temporary violations of fair sharing to favor
recently-woken domains
Goal: reduce wake-up latency
Xen provides several different types of timers Real Time (time that always advances regardless of the
executing domain) Virtual Time (time that only advances within the context of
the domain) Wall Clock Time (time that takes in to account local offsets
for time zone and DST)
No shadow pages (VMWare)
Xen provides constrained but direct MMU updates
All guest OSes have read-only accesses to page tables
Updates are batched into a single hypercall
Updates must be validated by Xen
Guest OSes are responsible for allocation and managing pages within their own domain
Xen exists in a generally unused section at the top of every address space to prevent paging out
Reserved at domain creation times
Memory statically partitioned among domains
Does not guarantee contiguous regions of memory
Supports hardware~physical mapping by providing shared translation array readable by all domains
Virtual firewall-router attached to all domains
Round-robin packet scheduler
To send a packet, enqueue a buffer descriptor into the transmit rang
Use scatter-gather DMA (no packet copying) A domain needs to exchange page frame to avoid
copying Page-aligned buffering
Only Domain0 has direct access to disks
Other domains need to use virtual block devices Use the I/O ring Reorder requests prior to enqueuing them on the ring If permitted, Xen will also reorder requests to improve
performance
Use DMA (zero copy)
SPEC INT2000 score
CPU Intensive (Little I/O and OS interaction)
SPEC WEB99
180Mb/s TCP traffic (Disk read-write on 2GB dataset)
Since version 3.0.2 Xen supports unmodified Microsoft OSes and Linux when the host runs on Intel VT or AMD-V hardware
As of 2009 most Linux distributions include Xen packages to interact with the Xen hypervisor and start additional domains
XenServer was made 100% open source 2 years after being acquired by Citrix Systems. Citrix has also formed Xen Project Advisory Board (Xen AB), which currently has members from IBM, Intel, Hewlett-Packard, Novell, Red Hat, Sun Microsystems and Oracle
Questions?
Discussion
Slide is available at: http://www.slideshare.net/tarequeh/xen-the-art-of-
virtualization