Xen 2 User Manual

8/13/2019 Xen 2 User Manual

1/56


2/56


3/56

Contents

I IntroductionandTutorial 11 Introduction 3

1.1 StructureofaXen-BasedSystem . . . . . . . . . . . . . . . . . . . .41.2 HardwareSupport . . . . . . . . . . . . . . . . . . . . . . . . . . . .41.3 History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5

2 Installation 72.1 Prerequisites. . . . . . . . . . . . ................... 72.2 InstallingfromBinaryTarball . .................... 82.3 InstallingfromSource . . . . . . ................... 8

2.3.1 ObtainingtheSource . . . ................... 82.3.2 BuildingfromSource. . . ................... 92.3.3 CustomXenLinuxBuilds ................... 102.3.4 InstallingtheBinaries . .................... 10

2.4 Configuration . . . . . . . . . . . ................... 102.4.1 GRUBConfiguration . . . ................... 112.4.2 SerialConsole(optional) .................... 112.4.3 TLSLibraries . . . . . . . ................... 12

2.5 BootingXen. . . . . . . . . . . . ................... 123 StartingAdditionalDomains 13

3.1 CreatingaDomainConfigurationFile . . . .............. 133.2 BootingtheDomain. . . . . . . . . . . . . .............. 143.3 Example:ttylinux . . . . . . . . . . . . . . .............. 143.4 Starting/StoppingDomainsAutomatically .............. 15

4 DomainManagementTools 174.1 Command-lineManagement . . . . . . ................ 17

4.1.1 BasicManagementCommands................. 174.1.2 xm list. . . . . . . . . . . . ................ 17

4.2 DomainSaveandRestore. . . . . . . . ................ 184.3 LiveMigration . . . . . . . . . . . . . ................ 19

i


4/56

4.4 ManagingDomainMemory . . . . . . . . . . . . . . . ....... 204.4.1 Settingmemoryfootprintsfromdom0 . . . . . . ....... 204.4.2 Settingmemoryfootprintsfromwithinadomain ....... 204.4.3 Settingmemorylimits . . . . . . . . . . . . . . ....... 20

5 DomainFilesystemStorage 215.1 ExportingPhysicalDevicesasVBDs.................. 215.2 UsingFile-backedVBDs . . . . . . . ................. 225.3 UsingLVM-backedVBDs . . . . . . ................. 235.4 UsingNFSRoot. . . . . . . . . . . . ................. 24

II UserReferenceDocumentation 256 ControlSoftware 27

6.1 Xend(nodecontroldaemon) . . . . . . . . . . . . . . . . . . . . . .276.2 Xm(commandlineinterface) . . . . . . . . . . . . . . . . . . . . . .286.3 Xensv(webcontrolinterface) . . . . . . . . . . . . . . . . . . . . .29

7 Domain onfiguration 317.1 ConfigurationFiles . . . . . . . . . . ................. 317.2 NetworkConfiguration . . . . . . . . ................. 32

7.2.1 Xenvirtualnetworktopology ................. 327.2.2 Xennetworkingscripts . . . . ................. 33

7.3 DriverDomainConfiguration . . . . . ................. 337.4 SchedulerConfiguration . . . . . . . ................. 34

7.4.1 BorrowedVirtualTime . . . . ................. 347.4.2 Atropos . . . . . . . . . . . . ................. 357.4.3 RoundRobin . . . . . . . . . ................. 35

8 Build,BootandDebugoptions 378.1 XenBuildOptions . . . . ....................... 378.2 XenBootOptions . . . . . ....................... 378.3 XenLinuxBootOptions. ........................ 398.4 Debugging. . . . . . . . . ....................... 40

9 FurtherSupport 419.1 OtherDocumentation . . . . . . . . . . . . . . . . . . . . . . . . . .419.2 OnlineReferences. . . . . . . . . . . . . . . . . . . . . . . . . . . .419.3 MailingLists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41

A InstallingXen/XenLinuxonDebian 43ii


5/56

B InstallingXen/XenLinuxonRedhatorFedoraCore 47C GlossaryofTerms 49

iii


6/56

iv


7/56

PartIIntroductionandTutorial

1


8/56


9/56

Chapter1

Introduction

Xenisaparavirtualisingvirtualmachinemonitor(VMM),orhypervisor,forthex86processorarchitecture.Xencansecurelyexecutemultiplevirtualmachinesonasinglephysical system with close-to-nativeperformance. The virtual machine technologyfacilitatesenterprise-gradefunctionality,including:

Virtualmachineswithperformanceclosetonativehardware. Livemigrationofrunningvirtualmachinesbetweenphysicalhosts. Excellenthardwaresupport(supportsmostLinuxdevicedrivers). Sandboxed,restartabledevicedrivers.

Paravirtualisationpermitsveryhighperformancevirtualisation, evenonarchitectureslikex86thataretraditionallyveryhardtovirtualise. ThedrawbackofthisapproachisthatitrequiresoperatingsystemstobeportedtorunonXen. PortinganOStorunonXenissimilartosupportinganewhardwareplatform,howevertheprocessissim-plifiedbecausetheparavirtualmachinearchitectureisverysimilartotheunderlyingnativehardware.EventhoughoperatingsystemkernelsmustexplicitlysupportXen,akeyfeatureisthatuserspaceapplicationsandlibrariesdonotrequiremodification.Xen support is available for increasingly many operating systems: right now, Linux2.4,Linux2.6andNetBSDareavailableforXen2.0. AFreeBSDportisundergoingtestingandwillbeincorporatedintothereleasesoon. OtherOSports,includingPlan9, are inprogress. We hope that that arch-xenpatcheswillbe incorporated into themainstreamreleasesoftheseoperatingsystemsinduecourse(ashasalreadyhappenedforNetBSD).PossibleusagescenariosforXeninclude:Kerneldevelopment. Testanddebugkernelmodificationsinasandboxedvirtualma-

chinenoneedforaseparatetestmachine.MultipleOSconfigurations Runmultipleoperatingsystemssimultaneously,forin-

stanceforcompatibilityorQApurposes.

3


10/56

Serverconsolidation. Movemultipleserversontoasinglephysicalhostwithperfor-manceandfaultisolationprovidedatvirtualmachineboundaries.

Clustercomputing. Management at VM granularity provides more flexibility thanseparately managing eachphysical host,but better control and isolation thansingle-systemimagesolutions,particularlybyusinglivemigrationforloadbal-ancing.

HardwaresupportforcustomOSes. AllowdevelopmentofnewOSeswhilebene-fitingfromthewide-ranginghardwaresupportofexistingOSessuchasLinux.

1.1 StructureofaXen-BasedSystemA Xen system has multiple layers, the lowest and mostprivileged of which is Xenitself.Xeninturnmayhostmultipleguestoperatingsystems, eachofwhichisexe-cutedwithinasecurevirtualmachine(inXenterminology, adomain). DomainsarescheduledbyXentomakeeffectiveuseoftheavailablephysicalCPUs. EachguestOSmanages its own applications,which includes responsibility for scheduling eachapplicationwithinthetimeallottedtotheVMbyXen.Thefirstdomain, domain0, iscreatedautomaticallywhenthesystembootsandhasspecial managementprivileges. Domain 0 builds other domains and manages theirvirtual devices. It alsoperforms administrative tasks such as suspending, resumingandmigratingothervirtualmachines.Withindomain0,aprocesscalledxendrunstomanagethesystem.Xendisresponsibleformanagingvirtualmachinesandprovidingaccesstotheirconsoles.CommandsareissuedtoxendoveranHTTPinterface,eitherfromacommand-linetoolorfromawebbrowser.1.2 HardwareSupportXencurrentlyrunsonlyonthex86architecture,requiringaP6ornewerprocessor(e.g.PentiumPro,Celeron,PentiumII,PentiumIII,PentiumIV,Xeon,AMDAthlon,AMDDuron).Multiprocessormachinesaresupported,andwealsohavebasicsupportforHyperThreading(SMT),althoughthisremainsatopicforongoingresearch.Aportspecificallyforx86/64isinprogress,althoughXenalreadyrunsonsuchsystemsin32-bitlegacymode.InadditionaporttotheIA64architectureisapproachingcompletion.WehopetoaddotherarchitecturessuchasPPCandARMinduecourse.Xencancurrentlyuseupto4GBofmemory.Itispossibleforx86machinestoaddressupto64GBofphysicalmemorybuttherearenocurrentplanstosupportthesesystems:Thex86/64portistheplannedroutetosupportinglargermemorysizes.XenoffloadsmostofthehardwaresupportissuestotheguestOSrunninginDomain0.

4


11/56

Xenitselfcontainsonlythecoderequiredtodetectandstartsecondaryprocessors,setupinterruptrouting, andperformPCIbusenumeration. DevicedriversrunwithinaprivilegedguestOSratherthanwithinXenitself. Thisapproachprovidescompatibil-itywiththemajorityofdevicehardwaresupportedbyLinux. ThedefaultXenLinuxbuildcontainssupportforrelativelymodernserver-classnetworkanddiskhardware,butyoucanaddsupportforotherhardwarebyconfiguringyourXenLinuxkernelinthenormalway.

1.3 HistoryXen was originally developedby the Systems Research Group at the University ofCambridge Computer Laboratory aspart of theXenoServersproject, fundedby theUK-EPSRC.XenoServersaimtoprovideapublicinfrastructureforglobaldistributedcomputing,andXenplaysakeypartinthat,allowingustoefficientlypartitionasin-glemachinetoenablemultipleindependentclientstoruntheiroperatingsystemsandapplicationsinanenvironmentprovidingprotection, resourceisolationandaccount-ing. Theprojectwebpagecontainsfurtherinformationalongwithpointerstopapersandtechnicalreports:http://www.cl.cam.ac.uk/xenoXenhassincegrownintoafully-fledgedprojectinitsownright,enablingustoinvesti-gateinterestingresearchissuesregardingthebesttechniquesforvirtualisingresourcessuchastheCPU,memory,diskandnetwork. Theprojecthasbeenbolsteredbysup-portfromIntelResearchCambridge,andHPLabs,whoarenowworkingcloselywithus.Xen was first described in apaperpresented at SOSP in 20031, and the firstpublicrelease(1.0)wasmadethatOctober. Sincethen,Xenhassignificantlymaturedandisnowusedinproductionscenariosonmanysites.Xen2.0featuresgreatlyenhancedhardwaresupport,configurationflexibility,usabilityandalargercomplementofsupportedoperatingsystems.ThislatestreleasetakesXenastepclosertobecomingthedefinitiveopensourcesolutionforvirtualisation.

1http://www.cl.cam.ac.uk/netos/papers/2003-xensosp.pdf 5


12/56

6


13/56

Thefollowingisafulllistofprerequisites. Itemsmarkedarerequiredbythexend

A

working

Linux

distribution

using

the

GRUB

bootloader

and

running

on

aP6-

AninstallationofTwistedv1.3orabove2.Theremaybeabinarypackageavail-

Chapter2

Installation

TheXendistributionincludesthreemaincomponents: Xenitself,portsofLinux2.4and 2.6 andNetBSD to run on Xen, and the user-space tools required to manage aXen-basedsystem.ThischapterdescribeshowtoinstalltheXen2.0distributionfromsource. Alternatively,theremaybepre-builtpackagesavailableaspartofyouroper-atingsystemdistribution.

2.1 Prerequisites

control tools, and hence required if youwant to runmore than one virtualmachine;itemsmarkedareonlyrequiredifyouwishtobuildfromsource.

class(ornewer)CPU. Theiproute2package. TheLinuxbridge-utils1(e.g.,/sbin/brctl)able for your distribution; alternatively it canbe installed by running makeinstall-twistedintherootoftheXensourcetree.

Buildtools(gccv3.2.xorv3.3.x,binutils,GNUmake). Developmentinstallationoflibcurl(e.g.,libcurl-devel) Developmentinstallationofzlib(e.g.,zlib-dev). DevelopmentinstallationofPythonv2.2orlater(e.g.,python-dev). LATEXandtransfigarerequiredtobuildthedocumentation.

1Availablefromhttp://bridge.sourceforge.net2Availablefromhttp://www.twistedmatrix.com

7


14/56

Onceyouhavesatisfiedtherelevantprerequisites,youcannowinstalleitherabinaryorsourcedistributionofXen.

2.2 InstallingfromBinaryTarballPre-builttarballsareavailablefordownloadfromtheXendownloadpage

http://xen.sf.netOnceyouvedownloadedthetarball,simplyunpackandinstall:# tar zxvf xen-2.0-install.tgz# cd xen-2.0-install# sh ./install.shOnceyouveinstalledthebinariesyouneedtoconfigureyoursystemasdescribedinSection2.4.

2.3 InstallingfromSourceThissectiondescribeshowtoobtain,build,andinstallXenfromsource.

2.3.1 ObtainingtheSourceTheXensourcetreeisavailableaseitheracompressedsourcetarballorasacloneofourmasterBitKeeperrepository.ObtainingtheSourceTarball

Stable versions (and daily snapshots) of the Xen source tree are available ascompressedtarballsfromtheXendownloadpage

http://xen.sf.netUsingBitKeeper

IfyouwishtoinstallXenfromacloneofourlatestBitKeeperrepositorythenyou will need to install the BitKeeper tools. Download instructions for Bit-Keepercanbeobtainedbyfillingouttheformat:

http://www.bitmover.com/cgi-bin/download.cgiThepublicmasterBKrepositoryforthe2.0releaselivesat:

bk://xen.bkbits.net/xen-2.0.bkYoucanuseBitKeepertodownloaditandkeepitupdatedwiththelatestfeaturesandfixes.Changetothedirectoryinwhichyouwanttoputthesourcecode,thenrun:

8


15/56

Download(ifnecessary)andunpacktheLinux2.6sourcecode,andpatchitfor BuildaLinuxkerneltouseindomain0andasmallerunprivilegedkernel,which

# bk clone bk://xen.bkbits.net/xen-2.0.bkUnder your current directory, a new directory named xen-2.0.bk hasbeencreated,whichcontainsallthesourcecodeforXen,theOSports,andthecontroltools. You can update your repository with the latest changes at any timebyrunning:# cd xen-2.0.bk # to change into the local repository# bk pull # to update the repository

2.3.2 BuildingfromSourceThetop-levelXenMakefileincludesatargetworldthatwilldothefollowing:

BuildXen Buildthecontroltools,includingxendusewithXencanoptionallybeusedforunprivilegedvirtualmachines.

After the build has completed you should have a top-level directory called dist/inwhichallresultingtargetswillbeplaced;ofparticularinterestarethetwokernelsXenLinuxkernelimages,onewitha-xen0extensionwhichcontainshardwaredevicedriversanddriversforXensvirtualdevices,andonewitha-xenUextensionthatjustcontainsthevirtualones. Thesearefoundindist/install/boot/alongwiththeimageforXenitselfandtheconfigurationfilesusedduringthebuild.TheNetBSDportcanbebuiltusing:

# make netbsd20NetBSDportisbuiltusingasnapshotofthenetbsd-2-0cvsbranch. Thesnapshotisdownloadedaspartofthebuildprocess,ifitisnotyetpresentintheNETBSD SRC PATHsearchpath.ThebuildprocessalsodownloadsatoolchainwhichincludesallthetoolsnecessarytobuildtheNetBSDkernelunderLinux.Tocustomizefurtherthesetofkernelsbuiltyouneedtoeditthetop-levelMakefile.Lookfortheline:

KERNELS ?= mk.linux-2.6-xen0 mk.linux-2.6-xenUYoucaneditthislinetoincludeanysetofoperatingsystemkernelswhichhavecon-figurationsinthetop-levelbuildconfigs/directory,forexamplemk.linux-2.4-xenUtobuildaLinux2.4kernelcontainingonlyvirtualdevicedrivers.

9


16/56

2.3.3 CustomXenLinuxBuildsIfyouwishtobuildacustomizedXenLinuxkernel(e.g. tosupportadditionaldevicesorenabledistribution-requiredfeatures),youcanusethestandardLinuxconfigurationmechanisms,specifyingthatthearchitecturebeingbuiltforisxen,e.g:

# cd linux-2.6.11-xen0# make ARCH=xen xconfig# cd ..# make

YoucanalsocopyanexistingLinuxconfiguration(.config)intolinux-2.6.11-xen0andexecute:

# make ARCH=xen oldconfigYoumaybepromptedwith someXen-specific options; we advise accepting the de-faultsfortheseoptions.NotethattheonlydifferencebetweenthetwotypesofLinuxkernelthatarebuiltisthe configuration file used for each. The U suffixed (unprivileged) versions dontcontain any of the physical hardware device drivers, leading to a 30% reduction insize; hence youmayprefer these for your non-privileged domains. The 0 suffixedprivilegedversionscanbeusedtobootthesystem, aswellasindriverdomainsandunprivilegeddomains.2.3.4 InstallingtheBinariesThefilesproducedbythebuildprocessarestoredunderthedist/install/direc-tory.Toinstallthemintheirdefaultlocations,do:

# make installAlternatively, users with special installation requirements may wish to install themmanuallybycopyingthefilestotheirappropriatedestinations.Thedist/install/bootdirectorywillalsocontaintheconfigfilesusedforbuild-ingtheXenLinuxkernels,andalsoversionsofXenandXenLinuxkernelsthatcontaindebugsymbols(xen-syms-2.0.6andvmlinux-syms-2.6.11.11-xen0)whichare essential for interpreting crash dumps. Retain these files as the developersmaywishtoseethemifyoupostonthemailinglist.

2.4 onfigurationOnce you havebuilt and installed the Xen distribution, it is simple toprepare themachineforbootingandrunningXen.

10


17/56

2.4.1 GRUB onfiguration Anentryshouldbeaddedtogrub.conf(oftenfoundunder/boot/or/boot/grub/)toallowXen/XenLinuxtoboot.Thisfileissometimescalledmenu.lst,dependingonyourdistribution.Theentryshouldlooksomethinglikethefollowing:title Xen 2.0 / XenLinux 2.6

kernel /boot/xen-2.0.gz dom0_mem=131072module /boot/vmlinuz-2.6-xen0 root=/dev/sda4 ro console=tty0

ThekernellinetellsGRUBwheretofindXenitselfandwhatbootparametersshouldbepassedtoit(inthiscase,settingdomain0smemoryallocationinkilobytesandthesettingsfortheserialport). FormoredetailsonthevariousXenbootparametersseeSection8.2.The module line of the configuration describes the location of the XenLinux kernelthatXenshouldstartandtheparametersthatshouldbepassedtoit(thesearestandardLinuxparameters, identifying the root device and specifying itbe initially mountedreadonlyandinstructingthatconsoleoutputbesenttothescreen).SomedistributionssuchasSuSEdonotrequiretheroparameter.Ifyouwanttouseaninitrd,justaddanothermodulelinetotheconfiguration,asusual:

module /boot/my_initrd.gzAs always when installing a new kernel, it is recommended that you do not deleteexistingmenuoptionsfrommenu.lstyoumaywanttobootyouroldLinuxkernelinfuture,particularlyifyouhaveproblems.2.4.2 SerialConsole(optional)InordertoconfigureXenserialconsoleoutput,itisnecessarytoaddanbootoptiontoyourGRUBconfig;e.g.replacetheabovekernellinewith:

kernel /boot/xen.gz dom0_mem=131072 com1=115200,8n1ThisconfiguresXentooutputonCOM1at115,200baud,8databits,1stopbitandnoparity.Modifytheseparametersforyoursetup.OnecanalsoconfigureXenLinuxtosharetheserialconsole; toachievethisappendconsole=ttyS0toyourmoduleline.IfyouwishtobeabletologinovertheXenLinuxserialconsoleitisnecessarytoaddalineinto/etc/inittab,justasperregularLinux.Simplyaddtheline:

c:2345:respawn:/sbin/mingetty ttyS0andyoushouldbeabletologin.NotethattosuccessfullyloginasrootovertheseriallinewillrequireaddingttyS0to/etc/securettyinmostmoderndistributions.

11


18/56

2.4.3 TLSLibrariesUsersoftheXenLinux2.6kernelshoulddisableThreadLocalStorage(e.g.bydoingamv /lib/tls /lib/tls.disabled)beforeattemptingtorunwithaXenLinuxkernel3. Youcanalwaysreenableitbyrestoringthedirectorytoitsoriginallocation(i.e.mv /lib/tls.disabled /lib/tls).ThereasonforthisisthatthecurrentTLSimplementationusessegmentationinawaythatisnotpermissibleunderXen. IfTLSisnotdisabled,anemulationmodeisusedwithinXenwhichreducesperformancesubstantially.WehopethatthisissuecanberesolvedbyworkingwithLinuxdistributionvendorstoimplementaminorbackward-compatiblechangetotheTLSlibrary.

2.5 BootingXenItshouldnowbepossibletorestartthesystemanduseXen.RebootasusualbutchoosethenewXenoptionwhentheGrubscreenappears.WhatfollowsshouldlookmuchlikeaconventionalLinuxboot. ThefirstportionoftheoutputcomesfromXenitself,supplyinglowlevelinformationaboutitselfandthemachineitisrunningon.ThefollowingportionoftheoutputcomesfromXenLinux.YoumayseesomeerrorsduringtheXenLinuxboot. Thesearenotnecessarilyany-thingtoworryabouttheymayresultfromkernelconfigurationdifferencesbetweenyourXenLinuxkernelandtheoneyouusuallyuse.Whenthebootcompletes,youshouldbeabletologintoyoursystemasusual. IfyouareunabletologintoyoursystemrunningXen,youshouldstillbeabletorebootwithyournormalLinuxkernel.

3IfyoubootwithoutfirstdisablingTLS,youwillgetawarningmessageduringthebootprocess.Inthiscase,simplyperformtherenameafterthemachineisupandthenrun/sbin/ldconfigtomakeittakeeffect.

12


19/56

/etc/xen/xmexample1 isasimpletemplateconfiguration filefordescribing /etc/xen/xmexample2 file is a template description that is intended tobe

Chapter3

StartingAdditionalDomainsThefirststepincreatinganewdomainistopreparearootfilesystemforittobootoff.Typically,thismightbestoredinanormalpartition,anLVMorothervolumemanagerpartition, adiskfileoronanNFSserver. AsimplewaytodothisissimplytobootfromyourstandardOSinstallCDandinstallthedistributionintoanotherpartitiononyourharddrive.Tostartthexendcontroldaemon,type

# xend startIfyouwishthedaemontostartautomatically,seetheinstructionsinSection6.1.Oncethedaemonisrunning,youcanusethexmtooltomonitorandmaintainthedomainsrunningonyoursystem. Thischapterprovidesonlyabrieftutorial: weprovidefulldetailsofthexmtoolinthenextchapter.

3.1 CreatingaDomain onfigurationFileBeforeyoucanstartanadditionaldomain, youmustcreateaconfigurationfile. Weprovidetwoexamplefileswhichyoucanuseasastartingpoint:

asingleVM.reusedformultiplevirtualmachines. Settingthevalueofthevmidvariableonthexmcommandlinefillsinpartsofthistemplate.

Copyoneofthesefilesandedititasappropriate. Typicalvaluesyoumaywishtoeditinclude:

kernel SetthistothepathofthekernelyoucompiledforusewithXen(e.g.kernel = /boot/vmlinuz-2.6-xenU)

13


20/56

memory Setthistothesizeofthedomainsmemoryinmegabytes(e.g.memory = 64)

disk Setthefirstentryinthislisttocalculatetheoffsetofthedomainsrootpartition,based on the domain ID. Set the second to the lo-cationof/usrifyouaresharingitbetweendomains(e.g.disk =[phy:your hard drive%d,sda1,w % (base partition number+ vmid), phy:your usr partition,sda6,r ]

dhcp Uncommentthedhcpvariable, sothatthedomainwillreceiveitsIPaddressfromaDHCPserver(e.g.dhcp=dhcp)

YoumayalsowanttoeditthevifvariableinordertochoosetheMACaddressofthevirtualethernetinterfaceyourself.Forexample:

vif = [mac=00:06:AA:F6:BB:B3]Ifyoudonotsetthisvariable,xendwillautomaticallygeneratearandomMACaddressfromanunusedrange.

3.2 BootingtheDomainThexmtoolprovidesavarietyofcommandsformanagingdomains. Usethecreatecommandtostartnewdomains.Assumingyouvecreatedaconfigurationfilemyvmconfbasedaround/etc/xen/xmexample2,tostartadomainwithvirtualmachineID1youshouldtype:

# xm create -c myvmconf vmid=1The-cswitchcausesxmtoturnintothedomainsconsoleaftercreation.Thevmid=1setsthevmidvariableusedinthemyvmconffile.You should see the consoleboot messages from the new domain appearing in theterminalinwhichyoutypedthecommand,culminatinginaloginprompt.

3.3 Example:ttylinuxTtylinuxisaverysmallLinuxdistribution,designedtorequireveryfewresources.WewilluseitasaconcreteexampleofhowtostartaXendomain.Mostuserswillproba-blywanttoinstallafull-featureddistributiononcetheyhavemasteredthebasics1.

1. Download and extract the ttylinux disk image from the Files section of theprojectsSourceForgesite(seehttp://sf.net/projects/xen/).

2. Createaconfigurationfilelikethefollowing:1ttylinuxismaintainedbyPascalSchmidt.Youcandownloadsourcepackagesfromthedistributions

homepage:http://www.minimalinux.org/ttylinux/

14


21/56

kernel = "/boot/vmlinuz-2.6-xenU"memory = 64name = "ttylinux"nics = 1ip = "1.2.3.4"disk = [file:/path/to/ttylinux/rootfs,sda1,w]root = "/dev/sda1 ro"

3. Nowstartthedomainandconnecttoitsconsole:xm create configfile -c

4. Loginasroot,passwordroot.

3.4 Starting/StoppingDomainsAutomaticallyItispossibletohavecertaindomainsstartautomaticallyatboottimeandtohavedom0waitforallrunningdomainstoshutdownbeforeitshutsdownthesystem.Tospecifyadomainistostartatboot-time,placeitsconfigurationfile(oralinktoit)under/etc/xen/auto/.ASys-VstyleinitscriptforRedHatandLSB-compliantsystemsisprovidedandwillbeautomaticallycopiedto/etc/init.d/duringinstall. Youcanthenenableitintheappropriatewayforyourdistribution.Forinstance,onRedHat:

# chkconfig --add xendomainsBydefault,thiswillstarttheboot-timedomainsinrunlevels3,4and5.Youcanalsousetheservicecommandtorunthisscriptmanually,e.g:

# service xendomains startStartsallthedomainswithconfigfilesunder/etc/xen/auto/.# service xendomains stopShutsdownALLrunningXendomains.

15


22/56

16


23/56

Chapter4

DomainManagementToolsThepreviouschapterdescribedasimpleexampleofhowtoconfigureandstartado-main.Thischaptersummarisesthetoolsavailabletomanagerunningdomains.

4.1 Command-lineManagementCommand linemanagement tasks are alsoperformed using the xm tool. For onlinehelpforthecommandsavailable,type:

# xm helpYoucanalsotypexm help formoreinformationonagivencommand.

4.1.1 BasicManagementCommandsThemostimportantxmcommandsare:

# xm list:Listsalldomainsrunning.# xm consoles:Givesinformationaboutthedomainconsoles.# xm console:Opensaconsoletoadomain(e.g.# xm console myVM

4.1.2 xm listTheoutputofxm listisinrowsofthefollowingformat:

name domid memory cpu state cputime consolename Thedescriptivenameofthevirtualmachine.domid ThenumberofthedomainIDthisvirtualmachineisrunningin.memory Memorysizeinmegabytes.cpu TheCPUthisdomainisrunningon.

17


24/56

state Domainstateconsistsof5fields:r runningbblockedppauseds shutdownc crashed

cputime HowmuchCPUtime(inseconds)thedomainhasusedsofar.console TCPportacceptingconnectionstothedomainsconsole.

The xm list command also supports a long output format when the -l switch isused.ThisoutputsthefullsdetailsoftherunningdomainsinxendsSXPconfigurationformat.Forexample,supposethesystemisrunningthettylinuxdomainasdescribedearlier.Thelistcommandshouldproduceoutputsomewhatlikethefollowing:# xm listName Id Mem(MB) CPU State Time(s) ConsoleDomain-0 0 251 0 r---- 172.2ttylinux 5 63 0 -b--- 3.0 9605Herewecanseethedetailsforthettylinuxdomain,aswellasfordomain0(which,of course, is always running). Note that the consoleport for the ttylinux domain is9605. ThiscanbeconnectedtobyTCPusingaterminalprogram(e.g. telnetor,better,xencons). Thesimplestwaytoconnectistousethexm consolecommand,specifyingthedomainnameorID.Toconnecttotheconsoleofthettylinuxdomain,wecoulduseanyofthefollowing:# xm console ttylinux# xm console 5# xencons localhost 9605

4.2 DomainSaveandRestoreTheadministratorofaXensystemmaysuspendavirtualmachinescurrentstateintoadiskfileindomain0,allowingittoberesumedatalatertime.The

ttylinux

domain

described

earlier

can

be

suspended

to

disk

using

the

command:

# xm save ttylinux ttylinux.xenThiswillstopthedomainnamedttylinuxandsaveitscurrentstateintoafilecalledttylinux.xen.Toresumeexecutionofthisdomain,usethexm restorecommand:

18


25/56

# xm restore ttylinux.xenThis will restore the state of the domain and restart it. The domain will carry onasbeforeandtheconsolemaybereconnectedusingthexm consolecommand, asabove.

4.3 LiveMigrationLivemigrationisusedtotransferadomainbetweenphysicalhostswhilstthatdomaincontinuestoperformitsusualactivitiesfromtheusersperspective,themigrationshouldbeimperceptible.Toperformalivemigration,bothhostsmustberunningXen/xendandthedestina-tionhostmusthavesufficientresources(e.g. memorycapacity)toaccommodatethedomainafterthemove. FurthermorewecurrentlyrequirebothsourceanddestinationmachinestobeonthesameL2subnet.Currently, there is no support forproviding automatic remote access to filesystemsstored on local disk when a domain is migrated. Administrators should choose anappropriatestoragesolution(i.e. SAN,NAS,etc.) toensurethatdomainfilesystemsarealsoavailableontheirdestinationnode. GNBDisagoodmethodforexportingavolumefromonemachinetoanother.iSCSIcandoasimilarjob,butismorecomplextosetup.Whenadomainmigrates,itsMACandIPaddressmovewithit,thusitisonlypossibletomigrateVMswithinthesamelayer-2networkandIPsubnet.Ifthedestinationnodeisonadifferentsubnet,theadministratorwouldneedtomanuallyconfigureasuitableetheriporIPtunnelinthedomain0oftheremotenode.Adomainmaybemigratedusingthexm migratecommand. Tolivemigrateado-maintoanothermachine,wewouldusethecommand:# xm migrate --live mydomain destination.ournetwork.comWithoutthe--liveflag,xendsimplystopsthedomainandcopiesthememoryimageover to the new node and restarts it. Since domains can have large allocations thiscanbequitetimeconsuming,evenonaGigabitnetwork. Withthe--liveflagxendattempts to keep the domain runningwhile themigration is inprogress, resulting intypicaldowntimesofjust60300ms.Fornowitwillbenecessarytoreconnecttothedomainsconsoleonthenewmachineusingthexm consolecommand. Ifamigrateddomainhasanyopennetworkcon-nectionsthentheywillbepreserved,soSSHconnectionsdonothavethislimitation.

19


26/56

4.4 ManagingDomainMemoryXenLinux domains have the ability to relinquish / reclaim machine memory at therequestoftheadministratorortheuserofthedomain.

4.4.1

Settingmemory

footprints

from

dom0

Themachineadministratorcanrequestthatadomainalteritsmemoryfootprintusingthexm set-mem command. For instance,we can request that our example ttylinuxdomainreduceitsmemoryfootprintto32megabytes.# xm set-mem ttylinux 32Wecannowseetheresultofthisintheoutputofxm list:# xm listName Id Mem(MB) CPU State Time(s) ConsoleDomain-0 0 251 0 r---- 172.2ttylinux 5 31 0 -b--- 4.3 9605ThedomainhasrespondedtotherequestbyreturningmemorytoXen.Wecanrestorethedomaintoitsoriginalsizeusingthecommandline:# xm set-mem ttylinux 64

4.4.2 SettingmemoryfootprintsfromwithinadomainThe virtual file /proc/xen/balloon allows the owner of a domain to adjust theirownmemoryfootprint. Readingthefile(e.g. cat /proc/xen/balloon)printsoutthecurrentmemoryfootprintofthedomain.Writingthefile(e.g.echo new target> /proc/xen/balloon)requeststhatthekerneladjustthedomainsmemoryfoot-printtoanewvalue.4.4.3 SettingmemorylimitsXenassociatesamemorysizelimitwitheachdomain. Bydefault,thisistheamountof memory the domain is originally started with,preventing the domain from evergrowingbeyondthissize. Topermitadomaintogrowbeyonditsoriginalallocationortopreventadomainyouveshrunkfromreclaimingthememoryitrelinquished,usethexm maxmemcommand.

20


27/56

Chapter5

DomainFilesystemStorageItispossibletodirectlyexportanyLinuxblockdeviceindom0toanotherdomain,orto export filesystems / devices to virtualmachines using standard networkprotocols(e.g.NBD,iSCSI,NFS,etc).Thischaptercoverssomeofthepossibilities.

5.1 ExportingPhysicalDevicesasVBDsOneofthesimplestconfigurationsistodirectlyexportindividualpartitionsfromdo-main 0 to other domains. To achieve this use the phy: specifier in your domainconfigurationfile.Forexamplealinelike

disk = [phy:hda3,sda1,w]specifiesthatthepartition/dev/hda3indomain0shouldbeexportedread-writetothe

new

domain

as

/dev/sda1;

one

could

equally

well

export

it

as

/dev/hda

or

/dev/sdb5shouldonewish.Inadditiontolocaldisksandpartitions,itispossibletoexportanydevicethatLinuxconsiderstobeadiskinthesamemanner. Forexample,ifyouhaveiSCSIdisksorGNBDvolumesimportedintodomain0youcanexportthesetootherdomainsusingthephy:disksyntax.E.g.:

disk = [phy:vg/lvm1,sda2,w]Warning: Block device sharing

Blockdevicesshouldtypicallyonlybesharedbetweendomainsinaread-only

fashion

otherwise

the

Linux

kernels

file

systems

will

get

very

con-

fused as the file system structure may change underneath them (havingthe same ext3partitionmounted rw twice is a sure fireway to cause ir-reparable damage)! Xend will attempt to prevent you from doing thisby checking that the device is notmounted read-write in domain 0, andhasntalreadybeenexportedread-writetoanotherdomain. Ifyouwant

21


28/56

read-write sharing, export the directory to other domains viaNFS fromdomain0(oruseaclusterfilesystemsuchasGFSorocfs2).

5.2 UsingFile-backedVBDsItisalsopossibletouseafileinDomain0astheprimarystorageforavirtualmachine.Aswellasbeingconvenient,thisalsohastheadvantagethatthevirtualblockdevicewillbesparsespacewillonlyreallybeallocatedaspartsofthefileareused. Soifavirtualmachineusesonlyhalfofitsdiskspacethenthefilereallytakesuphalfofthesizeallocated.For example, to create a 2GB sparse file-backed virtualblock device (actually onlyconsumes1KBofdisk):

# dd if=/dev/zero of=vm1disk bs=1k seek=2048k count=1Makeafilesysteminthediskfile:

# mkfs -t ext3 vm1disk(whenthetoolasksforconfirmation,answery)Populatethefilesysteme.g.bycopyingfromthecurrentroot:

# mount -o loop vm1disk /mnt# cp -ax /{root,dev,var,etc,usr,bin,sbin,lib} /mnt# mkdir /mnt/{proc,sys,home,tmp}

Tailor the file systemby editing /etc/fstab, /etc/hostname, etc (dont forgettoeditthefilesinthemountedfilesystem,insteadofyourdomain0filesystem,e.g.you would edit /mnt/etc/fstab instead of /etc/fstab ). For this exampleput/dev/sda1torootinfstab.Nowunmount(thisisimportant!):

# umount /mntIntheconfigurationfileset:

disk = [file:/full/path/to/vm1disk,sda1,w]Asthevirtualmachinewritestoitsdisk,thesparsefilewillbefilledinandconsumemorespaceuptotheoriginal2GB.Note that file b cked VBDs may not be appropriate for backing I/O-intensivedomains. File-backedVBDs are known to experience substantial slowdowns underheavy I/Oworkloads, due to the I/O handlingby the loopbackblock device used tosupportfile-backedVBDsindom0. BetterI/OperformancecanbeachievedbyusingeitherLVM-backedVBDs(Section5.3)orphysicaldevicesasVBDs(Section5.1).Linuxsupportsamaximumofeightfile-backedVBDsacrossalldomainsbydefault.

22


29/56

Thislimitcanbestaticallyincreasedbyusingthemax loopmoduleparameterifCON-FIG BLK DEV LOOPiscompiledasamoduleinthedom0kernel, orbyusingthemax loop=nbootoptionifCONFIG BLK DEV LOOPiscompileddirectlyintothedom0kernel.

5.3 UsingLVM-backedVBDsAparticularlyappealingsolutionistouseLVMvolumesasbackingfordomainfile-systemssincethisallowsdynamicgrowing/shrinkingofvolumesaswellassnapshotandotherfeatures.ToinitialiseapartitiontosupportLVMvolumes:

# pvcreate /dev/sda10Createavolumegroupnamedvgonthephysicalpartition:

# vgcreate vg /dev/sda10Createalogicalvolumeofsize4GBnamedmyvmdisk1:

# lvcreate -L4096M -n myvmdisk1 vgYoushouldnowseethatyouhavea/dev/vg/myvmdisk1Makeafilesystem,mountitandpopulateit,e.g.:

# mkfs -t ext3 /dev/vg/myvmdisk1# mount /dev/vg/myvmdisk1 /mnt# cp -ax / /mnt# umount /mnt

NowconfigureyourVMwiththefollowingdiskconfiguration:disk = [ phy:vg/myvmdisk1,sda1,w ]

LVM enables you to grow the size of logical volumes,but youll need to resize thecorrespondingfilesystemtomakeuseofthenewspace.Somefilesystems(e.g.ext3)nowsupporton-lineresize.SeetheLVMmanualsformoredetails.You can also use LVM for creating copy-on-write clones of LVM volumes (knownaswritablepersistentsnapshotsinLVMterminology). ThisfacilityisnewinLinux2.6.8,soisntasstableasonemighthope.Inparticular,usinglotsofCoWLVMdisksconsumes a lot of dom0 memory, and error conditions such as running out of diskspacearenothandledwell.Hopefullythiswillimproveinfuture.Tocreatetwocopy-on-writecloneoftheabovefilesystemyouwouldusethefollowingcommands:

# lvcreate -s -L1024M -n myclonedisk1 /dev/vg/myvmdisk1# lvcreate -s -L1024M -n myclonedisk2 /dev/vg/myvmdisk1

23


30/56

Eachofthesecangrowtohave1GBofdifferencesfromthemastervolume. Youcangrowtheamountofspaceforstoringthedifferencesusingthelvextendcommand,e.g.:

# lvextend +100M /dev/vg/myclonedisk1DontletthedifferencesvolumeeverfillupotherwiseLVMgetsratherconfused. Itmaybepossibletoautomatethegrowingprocessbyusingdmsetup waittospotthevolumegettingfullandthenissueanlvextend.Inprinciple,itispossibletocontinuewritingtothevolumethathasbeencloned(thechangeswillnotbevisibletotheclones),butwewouldntrecommendthis: havetheclonedvolumeasapristinefilesysteminstallthatisntmounteddirectlybyanyofthevirtualmachines.

5.4 UsingNFSRootFirst,populatearootfilesysteminadirectoryontheservermachine.Thiscanbeonadistinctphysicalmachine,orsimplyrunwithinavirtualmachineonthesamenode.NowconfiguretheNFSservertoexportthisfilesystemoverthenetworkbyaddingalineto/etc/exports,forinstance:

/export/vm1root 1.2.3.4/24 (rw,sync,no_root_squash)Finally,configurethedomaintouseNFSroot.Inadditiontothenormalvariables,youshouldmakesuretosetthefollowingvaluesinthedomainsconfigurationfile:

root = /dev/nfsnfs_server = 2.3.4.5 # substitute IP address of servernfs_root = /path/to/root # path to root FS on the server

Thedomain

will

need

network

access

at

boot

time,

so

either

statically

configure

an

IPaddress(Usingtheconfigvariablesip,netmask,gateway,hostname)orenableDHCP(dhcp=dhcp).Note that the LinuxNFS root implementation is known to have stability problemsunderhighload(thisisnotaXen-specificproblem),sothisconfigurationmaynotbeappropriateforcriticalservers.

24


31/56

PartIIUserReferenceDocumentation

25


32/56


33/56

Chapter6

ControlSoftwareThe Xen control software includes the xend node control daemon (which mustberunning),thexmcommandlinetools,andtheprototypexensvwebinterface.

6.1 Xend(nodecontroldaemon)The Xen Daemon (Xend)performs system management functions related to virtualmachines.ItformsacentralpointofcontrolforamachineandcanbecontrolledusinganHTTP-basedprotocol. Xendmustberunninginordertostartandmanagevirtualmachines.Xendmustbe run as rootbecause it needs access toprivileged systemmanagementfunctions.Asmallsetofcommandsmaybeissuedonthexendcommandline:# xend start startxend,ifnotalreadyrunning# xend stop stopxendifalreadyrunning# xend restart restartxendifrunning,otherwisestartit# xend status indicatesxendstatusbyitsreturncodeASysVinitscriptcalledxendisprovidedtostartxendatboottime.make installinstallsthisscriptin/etc/init.d. Toenableit,youhavetomakesymboliclinksintheappropriaterunleveldirectoriesorusethechkconfigtool,whereavailable.Once xend is running, more sophisticated administration canbe done using the xmtool(seeSection6.2)andtheexperimentalXensvwebinterface(seeSection6.3).As xend runs, events willbe logged to /var/log/xend.log and, if the migrationassistantdaemon(xfrd)hasbeenstarted, /var/log/xfrd.log. Thesemaybeofusefortroubleshootingproblems.

27


34/56

6.2 Xm(commandlineinterface)The xm tool is theprimary tool for managing Xen from the console. The generalformatofanxmcommandlineis:# xm command [switches] [arguments] [variables]The available switches and arguments are dependent on the command chosen. Thevariables maybe set using declarations of the form variable=value and com-mandlinedeclarationsoverrideanyofthevaluesintheconfigurationfilebeingused,including the standard variables described above and any custom variables (for in-stance,thexmdefconfigfileusesavmidvariable).Theavailablecommandsareasfollows:set-mem Requestadomaintoadjustitsmemoryfootprint.create Createanewdomain.destroy Killadomainimmediately. list Listrunningdomains.shutdown Askadomaintoshutdown.dmesg FetchtheXen(notLinux!)bootoutput.consoles Liststheavailableconsoles.console Connecttotheconsoleforadomain.help Gethelponxmcommands.save Suspendadomaintodisk.restore Restoreadomainfromdisk.pause Pauseadomainsexecution.unpause Unpauseadomain.pincpu PinadomaintoaCPU.bvt SetBVTschedulerparametersforadomain.bvt ctxallow SettheBVTcontextswitchingallowanceforthesystem.atropos Settheatroposparametersforadomain.rrobin Settheroundrobintimesliceforthesystem.info GetinformationabouttheXenhost.call CallaxendHTTPAPIfunctiondirectly.Foradetailedoverviewofswitches,argumentsandvariablestoeachcommandtry

# xm help command

28


35/56

6.3 Xensv(webcontrolinterface)XensvistheexperimentalwebcontrolinterfaceformanagingaXenmachine. Itcanbeusedtoperformsome(butnotyetall)ofthemanagementtasksthatcanbedoneusingthexmtool.Itcanbestartedusing:

# xensv startandstoppedusing:

# xensv stopBydefault,Xensvwillserveoutthewebinterfaceonport8080. Thiscanbechangedbyediting/usr/lib/python2.3/site-packages/xen/sv/params.py.OnceXensvisrunning,thewebinterfacecanbeusedtocreateandmanagerunningdomains.

29


36/56

30


37/56


38/56

dhcp SettodhcpifyouwanttouseDHCPtoconfigurenetworking.netmask ManuallyconfiguredIPnetmask.gateway ManuallyconfiguredIPgateway.hostname Setthehostnameforthevirtualmachine.root Specifytherootdeviceparameteronthekernelcommandline.nfs server IPaddressfortheNFSserver(ifany).nfs root PathoftherootfilesystemontheNFSserver(ifany).extra Extrastringtoappendtothekernelcommandline(ifany)restart Threepossibleoptions:

always Alwaysrestartthedomain,nomatterwhatitsexitcodeis.never Neverrestartthedomain.onreboot Restartthedomainiffitrequestsreboot.

Foradditionalflexibility,itisalsopossibletoincludePythonscriptingcommandsinconfiguration files. An example of this is the xmexample2 file, which uses Pythoncodetohandlethevmidvariable.

7.2 Network onfigurationFormanyusers, thedefaultinstallationshouldworkoutofthebox. Morecompli-cated network setups, for instance with multiple ethernet interfaces and/or existingbridgingsetupswillrequiresomespecialconfiguration.ThepurposeofthissectionistodescribethemechanismsprovidedbyxendtoallowaflexibleconfigurationforXensvirtualnetworking.7.2.1 XenvirtualnetworktopologyEach domain network interface is connected to a virtual network interface in dom0by apoint to point link (effectively a virtual crossover cable). These devices arenamedvif.(e.g. vif1.0forthefirstinterfaceindomain1,vif3.1forthesecondinterfaceindomain3).Trafficonthesevirtualinterfacesishandledindomain0usingstandardLinuxmech-anismsforbridging,routing,ratelimiting,etc. Xendcallsontwoshellscriptstoper-forminitialconfigurationofthenetworkandconfigurationofnewvirtualinterfaces.Bydefault,thesescriptsconfigureasinglebridgeforallthevirtualinterfaces. Arbi-traryrouting/bridgingconfigurationscanbeconfiguredbycustomisingthescripts,asdescribedinthefollowingsection.

32


39/56

SXPFormat:Includedeviceelementsoftheform:

7.2.2 XennetworkingscriptsXensvirtualnetworkingisconfiguredbytwoshellscripts(bydefaultnetworkandvif-bridge). These are called automaticallyby xend when certain events occur,withargumentstothescriptsprovidingfurthercontextualinformation. Thesescriptsarefoundbydefaultin/etc/xen/scripts. Thenamesandlocationsofthescriptscanbeconfiguredin/etc/xen/xend-config.sxp.network: Thisscriptiscalledwheneverxendisstartedorstoppedtorespectivelyini-

tialiseorteardowntheXenvirtualnetwork.Inthedefaultconfigurationinitial-isationcreatesthebridgexen-br0andmoveseth0ontothatbridge,modifyingtheroutingaccordingly.Whenxendexits,itdeletestheXenbridgeandremoveseth0,restoringthenormalIPandroutingconfiguration.

vif-bridge: Thisscriptiscalledforeverydomainvirtualinterfaceandcanconfigurefirewallingrulesandaddtheviftotheappropriatebridge. Bydefault,thisaddsandremovesVIFsonthedefaultXenbridge.

Formore

complex

network

setups

(e.g.

where

routing

is

required

or

integrate

with

existingbridges)thesescriptsmaybereplacedwithcustomisedvariantsforyoursitespreferredconfiguration.

7.3 DriverDomain onfigurationI/OprivilegescanbeassignedtoallowadomaintodirectlyaccessPCIdevicesitself.Thisisusedtosupportdriverdomains.SettingbackendprivilegesiscurrentlyonlysupportedinSXPformatconfigfiles. Toallow a domain to function as abackend for others, somewhere within the vm ele-ment of its configuration filemustbe a backend element of the form (backend(type)) where type maybe either netif or blkif, according to the type ofvirtualdevicethisdomainwillservice.Note that ablockbackend cannot currently import virtualblock devices from otherdomains, and a network backend cannot import virtual network devices from otherdomains. Thus (particularly in the case ofblockbackends, which cannot import avirtualblockdeviceastheirrootfilesystem),youmayneedtobootabackenddomainfromaramdiskoranetworkdevice.AccesstoPCIdevicesmaybeconfiguredonaper-devicebasis. Xenwillassigntheminimalsetofhardwareprivilegestoadomainthatarerequiredtocontrolitsdevices.Thiscanbeconfiguredineitherformatofconfigurationfile:

(device (pci (bus x) (dev y) (func z)))insidethetop-levelvmelement. Eachonespecifiestheaddressofadevicethis

33


40/56

FlatFormat:IncludealistofPCIdeviceaddressesoftheformat:

domainisallowedtoaccessthenumbersx,yandzmaybeineitherdecimalorhexadecimalformat.

pci = [x,y,z, ...]whereeachelementinthelistisastringspecifyingthecomponentsofthePCIdeviceaddress,separatedbycommas. Thecomponents(x,yandz)ofthelistmaybeformattedaseitherdecimalorhexadecimal.

7.4 Scheduler onfigurationXenoffersaboottimechoicebetweenmultipleschedulers.Toselectascheduler,passthebootparametersched=sched nametoXen,substitutingtheappropriateschedulername. Detailsoftheschedulersandtheirparametersareincludedbelow; futurever-sionsofthetoolswillprovideahigher-levelinterfacetothesetools.It is expected that system administrators configure their system to use the schedulermost appropriate to their needs. Currently, the BVT scheduler is the recommendedchoice.7.4.1 BorrowedVirtualTimesched=bvt(thedefault)BVTprovidesproportionalfairsharesoftheCPUtime. Ithasbeenobservedtope-nalise domains thatblock frequently (e.g. I/O intensive domains),but this canbecompensatedforbyusingwarping.GlobalParametersctx allow thecontextswitchallowanceissimilartothequantumintraditionalsched-

ulers. It is theminimum time that a scheduled domainwillbe allowed to runbeforebeingpre-empted.

Per-domainparametersmcuadv the MCU (Minimum Charging Unit) advance determines theproportional

shareoftheCPUthatadomainreceives. Itissetinverselyproportionallytoadomainssharingweight.

warp theamountofvirtualtimethedomainisallowedtowarpbackwardswarpl thewarplimitisthemaximumtimeadomaincanrunwarpedfor

34


41/56

warpu theunwarprequirementistheminimumtimeadomainmustrununwarpedforbeforeitcanwarpagain

7.4.2 Atropossched=atroposAtropos is a soft real time scheduler. It provides guarantees about absolute sharesoftheCPU,withafacilityforsharingslackCPUtimeonabest-effortbasis. Itcanprovidetimelinessguaranteesforlatency-sensitivedomains.Everydomainhasanassociatedperiodandslice. Thedomainshouldreceiveslicenanosecondseveryperiodnanoseconds. ThisallowstheadministratortoconfigureboththeabsoluteshareoftheCPUadomainreceivesandthefrequencywithwhichitisscheduled.Note: dontovercommittheCPUwhenusingAtropos(i.e. dontreservemoreCPUthanisavailabletheutilisationshouldbekepttoslightlylessthan100%inordertoensurepredictablebehaviour).Per-domainparametersperiod Theregulartimeintervalduringwhichadomainisguaranteedtoreceiveits

allocationofCPUtime.slice The length of time per period that a domain is guaranteed to run for (in the

absenceofvoluntaryyieldingoftheCPU).latency The latency hint is used to control how soon after waking up a domain it

shouldbescheduled.xtratime Thisisabooleanflagthatspecifieswhetheradomainshouldbealloweda

shareofthesystemslacktime.7.4.3 RoundRobinsched=rrobinThe round robin scheduler is included as a simple demonstration of Xens internalschedulerAPI.Itisnotintendedforproductionuse.

35


42/56

GlobalParametersrr slice Themaximumtimeeachdomainrunsbeforethenextschedulingdecisionis

made.

36


43/56

Chapter8

Build,BootandDebugoptionsThis chapter describes thebuild- andboot-time optionswhichmaybe used to tailoryourXensystem.

8.1 XenBuildOptionsXenprovides a number ofbuild-time options which shouldbe set as environmentvariablesorpassedonmakescommand-line.verbose=y Enable debuggingmessageswhenXen detects an unexpected condition.

Alsoenablesconsoleoutputfromalldomains.debug=y Enabledebugassertions. Impliesverbose=y. (Primarilyusefulfortracing

bugsinXen).debugger=y Enablethein-Xendebugger.ThiscanbeusedtodebugXen,guestOSes,

andapplications.perfc=y EnableperformancecountersforsignificanteventswithinXen. Thecounts

canberesetordisplayedonXensconsoleviaconsolecontrolkeys.trace=y Enableper-cputracebufferswhichlogarangeofeventswithinXenforcol-

lectionbycontrolsoftware.

8.2 XenBootOptionsTheseoptionsareusedtoconfigureXensbehaviouratruntime. Theyshouldbeap-pendedtoXenscommandline,eithermanuallyorbyeditinggrub.conf.noreboot Dontrebootthemachineautomaticallyonerrors. Thisisusefultocatch

debugoutputifyouarentcatchingconsolemessagesviatheserialline.nosmp DisableSMPsupport.Thisoptionisimpliedbyignorebiostables.

37


44/56

watchdog EnableNMIwatchdogwhichcanreportcertainfailures.noirqbalance DisablesoftwareIRQbalancingandaffinity.Thiscanbeusedonsys-

temssuchasDell1850/2850thathaveworkaroundsinhardwareforIRQ-routingissues.

badpage=,,... Specify a list ofpages not tobeallocatedforusebecausetheycontainbadbytes. Forexample,ifyourmemorytestersaysthatbyte0x12345678isbad, youwouldplacebadpage=0x12345onXenscommandline.

com1=,DPS,,com2=,DPS,,Xensupportsuptotwo16550-compatibleserialports.Forexample:com1=9600,8n1, 0x408, 5mapsCOM1toa9600-baudport, 8databits, noparity, 1stopbit, I/Oport base 0x408, IRQ 5. If some configuration options are standard(e.g.,I/ObaseandIRQ),thenonlyaprefixofthefullconfigurationstringneedbespecified.Ifthebaudrateispre-configured(e.g.,bythebootloader)thenyoucanspecifyautoinplaceofanumericbaudrate.

console= SpecifythedestinationforXenconsoleI/O.Thisisacomma-separatedlistof,forexample:vga useVGAconsoleandallowkeyboardinputcom1 useserialportcom1com2H use serialport com2. Transmitted chars will have the MSB set. Re-

ceivedcharsmusthaveMSBset.com2L use serial port com2. Transmitted chars will have the MSB cleared.

ReceivedcharsmusthaveMSBcleared.The

latter

two

examples

allow

asingle

port

to

be

shared

by

two

subsystems

(e.g. console and debugger). Sharing is controlledbyMSB of each transmit-ted/receivedcharacter.[NB.Defaultforthisoptioniscom1,vga]

sync console Force synchronous console output. This is useful if you system failsunexpectedlybeforeithassentallavailableoutputtotheconsole.InmostcasesXenwillautomaticallyentersynchronousmodewhenanexceptionaleventoc-curs,butthisoptionprovidesamanualfallback.

conswitch= Specifyhowtoswitchserial-consoleinputbetweenXenandDOM0.TherequiredsequenceisCTRL-pressedthreetimes. Specifyingthebacktickcharacterdisablesswitching. ThespecifieswhetherXenshouldauto-switchinputtoDOM0whenitbootsifitisxthenauto-switchingisdisabled. Anyothervalue,oromittingthecharacter,enablesauto-switching.[NB.defaultswitch-charisa]

nmi=xxx SpecifywhattodowithanNMIparityorI/Oerror.nmi=fatal:Xenprintsadiagnosticandthenhangs.

38


45/56

nmi=dom0:InformDOM0oftheNMI.nmi=ignore:IgnoretheNMI.

mem=xxx Set thephysical RAM address limit. Any RAM appearingbeyond thisphysical address in thememory map willbe ignored. Thisparameter maybespecifiedwithaB,K,MorGsuffix,representingbytes, kilobytes,megabytesandgigabytesrespectively.Thedefaultunit,ifnosuffixisspecified,isbytes.

dom0 mem=xxx Settheamountofmemorytobeallocatedtodomain0. InXen3.xtheparametermaybespecifiedwithaB,K,MorGsuffix,representingbytes,kilobytes, megabytes and gigabytes respectively; if no suffix is specified, theparameterdefaultstokilobytes. InpreviousversionsofXen,suffixeswerenotsupportedandthevalueisalwaysinterpretedaskilobytes.

tbuf size=xxx Setthesizeoftheper-cputracebuffers,inpages(default1).Notethatthe tracebuffersare only enabled in debugbuilds. Most users can ignore thisfeaturecompletely.

sched=xxx SelecttheCPUschedulerXenshoulduse. Thecurrentpossibilitiesarebvt(default),atroposandrrobin.FormoreinformationseeSection7.4.

apic verbosity=debug,verbose Print more detailed information about localAPICandIOAPICconfiguration.

lapic ForceuseoflocalAPICevenwhenleftdisabledbyuniprocessorBIOS.nolapic IgnorelocalAPICinauniprocessorsystem,evenifenabledbytheBIOS.apic=bigsmp,default,es7000,summit SpecifyNUMAplatform.Thiscanusuallybe

probedautomatically.Inaddition,thefollowingoptionsmaybespecifiedontheXencommandline. Sincedomain

0

shares

responsibility

for

booting

the

platform,

Xen

will

automatically

propa-

gatetheseoptionstoitscommandline.TheseoptionsaretakenfromLinuxscommand-linesyntaxwithunchangedsemantics.acpi=off,force,strict,ht,noirq,. .. ModifyhowXen(anddomain0)parsestheBIOS

ACPItables.acpi skip timer override InstructXen(anddomain0)toignoretimer-interruptover-

rideinstructionsspecifiedbytheBIOSACPItables.noapic InstructXen (and domain 0) to ignore any IOAPICs that arepresent in the

system,andinsteadcontinuetousethelegacyPIC.

8.3 XenLinuxBootOptionsInadditiontothestandardLinuxkernelbootoptions,wesupport:

39


46/56

xencons=xxx SpecifythedevicenodetowhichtheXenvirtualconsoledriverisat-tached.Thefollowingoptionsaresupported:

xencons=off:disablevirtualconsolexencons=tty:attachconsoleto/dev/tty1(tty0atboot-time)xencons=ttyS:attachconsoleto/dev/ttyS0

Thedefault

is

ttyS

for

dom0

and

tty

for

all

other

domains.

8.4 DebuggingXen has a set of debugging features that canbe useful to try and figure out whatsgoingon. Hithontheserialline(ifyouspecifiedabaudrateontheXencommandline)orScrollLock-honthekeyboardtogetalistofsupportedcommands.IfyouhaveacrashyoulllikelygetacrashdumpcontaininganEIP(PC)which,alongwithanobjdump -d image,canbeusefulinfiguringoutwhatshappened. DebugaXenlinux

image

just

as

you

would

any

other

Linux

kernel.

40


47/56

Chapter9

FurtherSupportIf you have questions that are not answeredby thismanual, the sources of informa-tion listedbelow maybe of interest to you. Note thatbug reports, suggestions andcontributionsrelatedtothesoftware(orthedocumentation)shouldbesenttotheXendevelopersmailinglist(addressbelow).

9.1 OtherDocumentationFordevelopersinterestedinportingoperatingsystemstoXen,theXenInterfaceMan-ualisdistributedinthedocs/directoryoftheXensourcedistribution.

9.2 OnlineReferencesTheofficialXenwebsiteisfoundat:

http://www.cl.cam.ac.uk/netos/xen/This contains links to the latest versions of all on-line documentation (including thelatesetversionoftheFAQ).

9.3 MailingListsTherearecurrentlyfourofficialXenmailinglists:[email protected] Usedfordevelopmentdiscussionsandbugreports.

Subscribeat:http://lists.xensource.com/xen-devel

[email protected] Usedforinstallationandusagediscussionsandre-

41


48/56

questsforhelp.Subscribeat:http://lists.xensource.com/xen-users

[email protected] Usedforannouncementsonly.Subscribeat:http://lists.xensource.com/xen-announce

[email protected] Changelogfeedfromtheunstableand2.0trees-developeroriented.Subscribeat:http://lists.xensource.com/xen-changelog

42


49/56


50/56

http://ftp..debian.org/debianYoucanuseanyotherDebianhttp/ftpmirroryouwant.

7. Whendebootstrapcompletessuccessfully,modifysettings:chroot /mnt/disk /bin/bashEditthefollowingfilesusingviornanoandmakeneededchanges:/etc/hostname/etc/hosts/etc/resolv.conf/etc/network/interfaces/etc/networksSetupaccesstotheservices,edit:/etc/hosts.deny/etc/hosts.allow/etc/inetd.confAddDebianmirrorto:/etc/apt/sources.listCreatefstablikethis:/dev/sda1 / ext3 errors=remount-ro 0 1/dev/sda2 none swap sw 0 0proc /proc proc defaults 0 0Logout

8. Unmountthediskimageumount /mnt/disk

9. CreateXen2.0configurationfileforthenewdomain.Youcanusetheexample-configurationscomingwithXenasatemplate.Makesureyouhavethefollowingsetup:disk = [ file:/path/diskimage,sda1,w, file:/path/swapimage,sda2,w ]root = "/dev/sda1 ro"

10. Startthenewdomainxm create -f domain_config_fileCheckthatthenewdomainisrunning:xm list

11. Attach to the console of the new domain. You should see something like thiswhenstartingthenewdomain:Started domain testdomain2, console on port 9626ThereyoucanseetheIDoftheconsole:26.Youcanalsolisttheconsoleswithxm consoles(IDisthelasttwodigitsoftheportnumber.)

44


51/56

Attachtotheconsole:xm console 26orby telnetting to theport 9626 of localhost (the xm consoleprogramworksbetter).

12. Loginandrunbase-configAsadefaulttheresnopasswordfortheroot.CheckthateverythinglooksOK,andthesystemstartedwithouterrors. Checkthattheswapisactive,andthenetworksettingsarecorrect.Run/usr/sbin/base-configtosetuptheDebiansettings.Setupthepasswordforrootusingpasswd.

13. Done.YoucanexittheconsolebypressingCtrl + ]Ifyouneedtocreatenewdomains,youcanjustcopythecontentsofthetemplate-imagetothenewdiskimages,eitherbymountingthetemplateandthenewimage,andusingcp -aortarorbysimplycopyingtheimagefile. Oncethisisdone,modifytheimage-specificsettings(hostname,networksettings,etc).

45


52/56

46


53/56

AppendixB

InstallingXen/XenLinuxonRedhatorFedoraCoreWhen usingXen /XenLinux on a standard Linux distribution there are a couple ofthingstowatchoutfor:Note that,because domains0 dont have anyprivileged access at all, certain com-mands in the defaultboot sequence will fail e.g. attempts to update the hwclock,change the console font, update the keytablemap, start apmd (power management),or gpm (mouse cursor). Either ignore the errors (they should be harmless), or re-move them from the startup scripts. Deleting the following links are a good start:S24pcmcia,S09isdn,S17keytable,S26apmd,S85gpm.Ifyouwanttouseasinglerootfilesystemthatworkscleanlyforbothdomain0andunprivilegeddomains,ausefultrickistousedifferentinitrunlevels. Forexample,userunlevel3fordomain0,andrunlevel4forotherdomains.Thisenablesdifferentstartup scripts tobe run in depending on the run level numberpassed on the kernelcommandline.If usingNFS root files systemsmounted either from an external server or from do-main0thereareacoupleofothergotchas.Thedefault/etc/sysconfig/iptablesrulesblockNFS,sopartwaythroughthebootsequencethingswillsuddenlygodead.If youreplanning on having a separateNFS /usrpartition, the RH9boot scriptsdontmakelifeeasy-theyattempttomountNFSfilesystemswaytolateinthebootprocess. TheeasiestwayIfoundtodothiswastohavea/linuxrcscriptrunaheadof/sbin/initthatmounts/usr:

#!/bin/bash/sbin/ipconfig lo 127.0.0.1/sbin/portmap/bin/mount /usrexec /sbin/init "$@" /dev/console 2>&1

47


54/56

The one slight complication with the above is that /sbin/portmap is dynamicallylinkedagainst/usr/lib/libwrap.so.0Sincethisisin/usr,itwontwork. Thiscanbesolvedbycopyingthefile(andlink)belowthe/usrmountpoint,andjustletthefilebecoveredwhenthemounthappens.Insomeinstallations,whereasharedread-only/usrisbeingused,itmaybedesirabletomoveotherlargedirectoriesoverintotheread-only/usr. Forexample,youmightreplace/bin, /liband/sbinwithlinksinto/usr/root/bin, /usr/root/liband/usr/root/sbinrespectively.Thiscreatesotherproblemsforrunningthe/linuxrcscript,requiringbash,portmap,mount,ifconfig,andahandfulofothersharedlibrariestobe copiedbelow the mountpoint a simple statically-linkedCprogram wouldsolvethisproblem.

48


55/56

AppendixC

GlossaryofTermsAtropos One of the CPU schedulersprovided by Xen. Atroposprovides domains

withabsolutesharesoftheCPU,withtimelinessguaranteesandamechanismforsharingoutslacktime.

BVT TheBVT schedulerisusedtogiveproportionalfairsharesoftheCPU todo-mains.

Exokernel Aminimalpieceofprivilegedcode,similartoamicrokernelbutprovid-ingamorehardware-likeinterfacetothetasksitmanages. ThisissimilartoaparavirtualisingVMMlikeXenbutwasdesignedasanewoperatingsystemstructure,ratherthanspecificallytorunmultipleconventionalOSs.

Domain Adomainistheexecutioncontextthatcontainsarunningvirtualmachine.The relationshipbetween virtual machines and domains on Xen is similar tothatbetweenprogramsandprocessesinanoperatingsystem: avirtualmachineisapersistententitythatresidesondisk(somewhatlikeaprogram). Whenitisloadedforexecution,itrunsinadomain.EachdomainhasadomainID.

Domain0 ThefirstdomaintobestartedonaXenmachine. Domain0isresponsibleformanagingthesystem.

DomainID Auniqueidentifierforadomain,analogoustoaprocessIDinanoperat-ingsystem.

Fullvirtualisation Anapproachtovirtualisationwhichrequiresnomodificationstothehostedoperatingsystem,providingtheillusionofacompletesystemofrealhardwaredevices.

Hypervisor AnalternativetermforVMM,usedbecauseitmeansbeyondsupervi-sor,sinceitisresponsibleformanagingmultiplesupervisorkernels.

Livemigration Atechniqueformovingarunningvirtualmachinetoanotherphysicalhost,withoutstoppingitortheservicesrunningonit.

Microkernel Asmallbaseofcoderunningatthehighesthardwareprivilegelevel.A49


56/56

microkernelisresponsibleforsharingCPUandmemory(andsometimesotherdevices)betweenlessprivilegedtasksrunningonthesystem.ThisissimilartoaVMM,particularlyaparavirtualisingVMMbuttypicallyaddressingadifferentproblemspaceandprovidingdifferentkindofinterface.

NetBSD/Xen AportofNetBSDtotheXenarchitecture.Paravirtualisation Anapproachtovirtualisationwhichrequiresmodificationstothe

operatingsysteminordertoruninavirtualmachine.Xenusesparavirtualisationbutpreservesbinarycompatibilityforuserspaceapplications.

Shadowpagetables A technique for hiding the layout of machine memory from avirtualmachinesoperatingsystem.UsedinsomeVMMstoprovidetheillusionofcontiguousphysicalmemory,inXenthisisusedduringlivemigration.

VirtualMachine Theenvironmentinwhichahostedoperatingsystemruns,provid-ingtheabstractionofadedicatedmachine. Avirtualmachinemaybeidenticaltotheunderlyinghardware(asinfullvirtualisation,oritmaydiffer,asinpar-avirtualisation.

VMM VirtualMachineMonitor-thesoftwarethatallowsmultiplevirtualmachinestobemultiplexedonasinglephysicalmachine.

Xen Xen is aparavirtualising virtual machine monitor, developedprimarilyby theSystemsResearchGroupattheUniversityofCambridgeComputerLaboratory.

XenLinux OfficialnamefortheportoftheLinuxkernelthatrunsonXen.

Date post:	04-Jun-2018
Category:	Documents
Upload:	najia-al-umri
View:	220 times
Download:	0 times

Xen 2 User Manual

Documents