Download - XPDS13: Xen Hypervisor Development Update - George Dunlap, Citrix

Intro OSS Development Xen 4.3 Xen 4.4 Updates

Xen Development update

George Dunlap

Edinburg – 21-23 October, 2013


Overview

I Principles of OSS Development

I Xen development process

I Review of 4.3 development process

I 4.4 development: Plans and predictionsI Project updates

I EFII Network driversI FreeBSDI libvirt integrationI NUMA

Edinburg – 21-23 October, 2013 Xen Development update 2 / 24


Overview








Overview








Overview




I 4.4 development: Plans and predictions

I Project updates




Overview








Overview





I EFI

I Network driversI FreeBSDI libvirt integrationI NUMA



Overview





I EFII Network drivers

I FreeBSDI libvirt integrationI NUMA



Overview





I EFII Network driversI FreeBSD

I libvirt integrationI NUMA



Overview





I EFII Network driversI FreeBSDI libvirt integration

I NUMA



Overview








Principles

I Open-source: A force multiplier

I Limiting factor: Trust

I Goal: Meritocracy



Principles



I Goal: Meritocracy



Principles



I Goal: Meritocracy



Practice: Code development

I All patches posted to list

I Discussion and consensusI Committers are “referees”

I Determine that consensus has happenedI Break ties when consensus is not achieved

I Similar process for bigger decisions





I Discussion and consensus

I Committers are “referees”















I Determine that consensus has happened

I Break ties when consensus is not achieved


















Releases

What we want:

I Visibility into the work that’s going on

I Clear picture of the state everything is in

I A roadmap of important featuresI A good release

I A bug-free releaseI An awesome releaseI An on-time release



Releases

What we want:







Releases

What we want:



I A roadmap of important features

I A good release




Releases

What we want:







Releases

What we want:




I A bug-free release

I An awesome releaseI An on-time release



Releases

What we want:




I A bug-free releaseI An awesome release

I An on-time release



Releases

What we want:







Releases

I Keep a list of outstanding features and bugs

I Things actively being worked onI Things we think would be good to work on

I Send the list every few weeks with updatesI Mirror on wiki:

I wiki.xen.org/wiki/Xen Roadmap/4.4



Releases

I Keep a list of outstanding features and bugsI Things actively being worked on

I Things we think would be good to work on





Releases

I Keep a list of outstanding features and bugsI Things actively being worked onI Things we think would be good to work on





Releases


I Send the list every few weeks with updates

I Mirror on wiki:




Releases






Releases






Release phases

I Open development

I Feature freeze: No new features (3 weeks)

I Code “freezing point”: Begin rejecting features (3 weeks)

I RCs: Bug fixes only (6 weeks)



Release phases

I Open development






Release phases

I Open development






Release phases

I Open development






Interlude



Xen 4.3 Planning Retrospective

I Goal: 9 month release cycle

I Scheduled release: June 2013

I Actual release: July 2013

I No “howlers”







I No “howlers”







I No “howlers”







I No “howlers”




Aug ’12

Jan ’13 Jul ’13

NUMA-aware scheduler (NUMA affinity) 3

3 3

ARM server port 3

3 3

openvswitch integration 3

7 3

16 TiB 3

3 3

backend scripts 3

3 3

qemu stubdoms 3

3 7

event channel limit 3

3 7

NUMA memory migration 3

3 7

virtual NUMA topology 3

3 7

PVH mode 3

3 7

pv audio 3

7 7

blktap3 3

7 7

vm import/export 3

7 7

xl USB hot-plug for HVM

3 7

libvirt libxl driver

3 3

XSM can replace IS PRIV

3 3

vTPM updates

3 3

osstest

3 3

qxl spice

3 7




Aug ’12 Jan ’13

Jul ’13

NUMA-aware scheduler (NUMA affinity) 3 3

3

ARM server port 3 3

3

openvswitch integration 3 7

3

16 TiB 3 3

3

backend scripts 3 3

3

qemu stubdoms 3 3

7

event channel limit 3 3

7

NUMA memory migration 3 3

7

virtual NUMA topology 3 3

7

PVH mode 3 3

7

pv audio 3 7

7

blktap3 3 7

7

vm import/export 3 7

7

xl USB hot-plug for HVM

3 7

libvirt libxl driver

3 3

XSM can replace IS PRIV

3 3

vTPM updates

3 3

osstest

3 3

qxl spice

3 7




Aug ’12 Jan ’13

Jul ’13

NUMA-aware scheduler (NUMA affinity) 3 3

3

ARM server port 3 3

3

openvswitch integration 3 7

3

16 TiB 3 3

3

backend scripts 3 3

3

qemu stubdoms 3 3

7

event channel limit 3 3

7

NUMA memory migration 3 3

7

virtual NUMA topology 3 3

7

PVH mode 3 3

7

pv audio 3 7

7

blktap3 3 7

7

vm import/export 3 7

7

xl USB hot-plug for HVM 3

7

libvirt libxl driver 3

3

XSM can replace IS PRIV 3

3

vTPM updates 3

3

osstest 3

3

qxl spice 3

7




Aug ’12 Jan ’13 Jul ’13NUMA-aware scheduler (NUMA affinity) 3 3 3

ARM server port 3 3 3

openvswitch integration 3 7 3

16 TiB 3 3 3

backend scripts 3 3 3

qemu stubdoms 3 3 7

event channel limit 3 3 7

NUMA memory migration 3 3 7

virtual NUMA topology 3 3 7

PVH mode 3 3 7

pv audio 3 7 7

blktap3 3 7 7

vm import/export 3 7 7

xl USB hot-plug for HVM 3 7

libvirt libxl driver 3 3

XSM can replace IS PRIV 3 3

vTPM updates 3 3

osstest 3 3

qxl spice 3 7




Don’t try to predict.



Xen 4.4 Planning


I Feature freeze: 18 October (past)

I Code freeze: Extended to catch a few straglers



Xen 4.4 Planning






Xen 4.4 Planning






Xen 4.4 Planning

Things that look likely:

I Multi-vector MSI (done)

I Improved spice support for libxl (done)

I PVH for domUs

I Non-udev scripts for driver domains(NetBSD driver domains)

I A ton of fixes from Coverity reports

I NUMA per-vcpu affinity

I USB hotplug for libxl



Xen 4.4 Planning




I PVH for domUs







Xen 4.4 Planning




I PVH for domUs







Xen 4.4 Planning




I PVH for domUs







Xen 4.4 Planning




I PVH for domUs







Xen 4.4 Planning




I PVH for domUs







Xen 4.4 Planning




I PVH for domUs







EFI intro

I Replacement for BIOS

I Includes “bootloader”-like functionality

I Boot-time services (similar to BIOS calls)

I Secure-boot services



EFI intro







EFI intro







EFI intro







EFI: Xen

(from Daniel Kiper @ Oracle)

I EFI support was introduced in Xen 4.2

I Only EFI loader is fully supported now

I GRUB/GRUB2 and Xen does not work on EFI platform

I But GRUB2 with fakebios and chainloader can be used to loadxen.efi on EFI platform (this option was not tested deeply)

I Daniel Kiper from Oracle works on full GRUB2 support

I Support for Secure Boot with GRUB2 will not be available atfirst

I GRUB2 requires some changes to support Secure Boot



EFI: Xen











EFI: Xen











EFI: Xen











EFI: Xen











EFI: Xen











EFI: Xen











EFI: Dom0 kernels


I Full Dom0 support is in SUSE Linux Kernel

I Upstream Linux Kernel does not have EFI support in Dom0yet

I Liang Tang from Oracle started working on it but laterdropped it

I There are some updates appearing from time to time onXen-devel but there is no WIP per se yet



EFI: Dom0 kernels








EFI: Dom0 kernels








EFI: Dom0 kernels








EFI: Guest support


I Open Virtual Machine Firmware (OVMF) was added to Xensource but it does not compile

I Framebuffer issue



EFI: Guest support


I Open Virtual Machine Firmware (OVMF) was added to Xensource but it does not compile

I Framebuffer issue



Network updates

(from Wei Liu @ Citrix)I Two new features during last cycle:

I 1:1 model netbackI split event channels

I Feature being worked on:

I IPv6 packet offload

I Planned new features:

I multiqueue supportI multipage ring support



Network updates


I 1:1 model netback

I split event channels







Network updates









Network updates









Network updates



I Feature being worked on:I IPv6 packet offload





Network updates








Network updates




I Planned new features:I multiqueue support

I multipage ring support



Network updates




I Planned new features:I multiqueue supportI multipage ring support



FreeBSD Xen status

(from Roger Pau Monne @ Citrix)Xen FreeBSD updates present in 10.0 (to be released by the end ofNovember):

I Rework of event channel implementation so it is sharedbetween PV and PVHVM guests.

I Implementation of the vector callback for HVM (allowsper-cpu event channel interrupts).

I PV event timer and time counter implementation for PVHVM.I PV IPIs for PVHVM.I Live Migration support for PVHVM guests.I Support for FLUSH/BARRIER in blkfront.I Full Xen support merged into GENERIC kernel (no need to

compile a Xen specific kernel for PVHVM).I Bug fixes for OSS Xen, XenServer and Amazon EC2.

This work has been done in conjunction between Spectra Logicand Citrix.



FreeBSD Xen status









FreeBSD Xen status




I PV event timer and time counter implementation for PVHVM.

I PV IPIs for PVHVM.I Live Migration support for PVHVM guests.I Support for FLUSH/BARRIER in blkfront.I Full Xen support merged into GENERIC kernel (no need to





FreeBSD Xen status




I PV event timer and time counter implementation for PVHVM.I PV IPIs for PVHVM.

I Live Migration support for PVHVM guests.I Support for FLUSH/BARRIER in blkfront.I Full Xen support merged into GENERIC kernel (no need to





FreeBSD Xen status




I PV event timer and time counter implementation for PVHVM.I PV IPIs for PVHVM.I Live Migration support for PVHVM guests.

I Support for FLUSH/BARRIER in blkfront.I Full Xen support merged into GENERIC kernel (no need to





FreeBSD Xen status




I PV event timer and time counter implementation for PVHVM.I PV IPIs for PVHVM.I Live Migration support for PVHVM guests.I Support for FLUSH/BARRIER in blkfront.

I Full Xen support merged into GENERIC kernel (no need tocompile a Xen specific kernel for PVHVM).

I Bug fixes for OSS Xen, XenServer and Amazon EC2.




FreeBSD Xen status





compile a Xen specific kernel for PVHVM).

I Bug fixes for OSS Xen, XenServer and Amazon EC2.




FreeBSD Xen status









FreeBSD Xen status









FreeBSD Xen status

(from Roger Pau Monne @ Citrix)What’s coming next:

I PVH guest support.

I PVH Dom0 support.



FreeBSD Xen status

(from Roger Pau Monne @ Citrix)What’s coming next:

I PVH guest support.

I PVH Dom0 support.



Block improvements

(from Roger Pau Monne @ Citrix)

I Persistent grants.

I Indirect descriptors.



Block improvements

(from Roger Pau Monne @ Citrix)

I Persistent grants.

I Indirect descriptors.



Libvirt

(from Dario Faggioli @ Citrix)

I Basic VM workflow

I Create, start, shutdown, reboot, destroy, pause, resume, save,restore, ... : working

I Migration, PCI passthrough: coming soon

I More advanced VM manipulation

I Scheduling, updating memory/vcpus, block attach, mediachange: working

I Vcpu pinning, NUMA node-affinity, network attach, native (xl)config file handling: missing

I Specific Libvirt stuff (for host and guests)

I Host and guest configuration and statistics reporting: missingI Block/network performance tuning and control: missing



Libvirt


I Basic VM workflowI Create, start, shutdown, reboot, destroy, pause, resume, save,

restore, ... : working

I Migration, PCI passthrough: coming soon








Libvirt



restore, ... : workingI Migration, PCI passthrough: coming soon








Libvirt











Libvirt




I More advanced VM manipulationI Scheduling, updating memory/vcpus, block attach, media

change: working






Libvirt





change: workingI Vcpu pinning, NUMA node-affinity, network attach, native (xl)

config file handling: missing





Libvirt











Libvirt







I Specific Libvirt stuff (for host and guests)I Host and guest configuration and statistics reporting: missing

I Block/network performance tuning and control: missing



Libvirt







I Specific Libvirt stuff (for host and guests)I Host and guest configuration and statistics reporting: missingI Block/network performance tuning and control: missing



NUMA


I NUMA aware automatic placementWhen a VM is created, allocate its memory from theminimum number and best possible node(s)Included in Xen 4.2

I NUMA aware scheduling for Credit1VMs can run everywhere, but prefer to run on the nodeswhere their memory isIncluded in Xen 4.3

I Per-VCPU NUMA aware schedulingEach VCPU in a VM can have a different NUMAnode-affinity: different VCPUs (may) prefer to run ondifferent nodes Targeting Xen 4.4 (status: probable)



NUMA







NUMA







NUMA, con’t


I Virtual NUMA topology for PV guestsVMs can be NUMA machines and have a virtual topology(number or nodes, amount of memory, distances, etc.)Targeting Xen 4.4 (status: probable)

I Memory migrationMoving the memory of a VM from a NUMA node to anotheron-lineTargeting Xen 4.4 (status: tentative)



NUMA, con’t


I Virtual NUMA topology for PV guestsVMs can be NUMA machines and have a virtual topology(number or nodes, amount of memory, distances, etc.)Targeting Xen 4.4 (status: probable)

I Memory migrationMoving the memory of a VM from a NUMA node to anotheron-lineTargeting Xen 4.4 (status: tentative)
