Scheduling - University of North Carolina at Chapel Hill · 2016. 7. 29. · Scheduling Domains •...

CSE506:Opera.ngSystems

Scheduling

DonPorter

1


LogicalDiagram

MemoryManagement

CPUScheduler

User

Kernel

Hardware

BinaryFormats

Consistency

SystemCalls

Interrupts Disk Net

RCU FileSystem

DeviceDrivers

Networking Sync

MemoryAllocators Threads

Today’sLectureSwitchingtoCPU

scheduling

2


Lecturegoals•  Understandlow-levelbuildingblocksofascheduler•  UnderstandcompeLngpolicygoals•  UnderstandtheO(1)scheduler–  CFSnextlecture

•  FamiliaritywithstandardUnixschedulingAPIs

3


Undergradreview•  WhatiscooperaLvemulLtasking?–  ProcessesvoluntarilyyieldCPUwhentheyaredone

•  WhatispreempLvemulLtasking?–  OSonlyletstasksrunforalimitedLme,thenforciblycontextswitchestheCPU

•  Pros/cons?–  CooperaLvegivesmorecontrol;somuchthatonetaskcanhogtheCPUforever

–  PreempLvegivesOSmorecontrol,moreoverheads/complexity

4


Wherecanwepreemptaprocess?•  Inotherwords,whatarethelogicalpointsatwhichtheOScanregaincontroloftheCPU?

•  Systemcalls–  Before–  During(morenextLmeonthis)–  AXer

•  Interrupts–  Timerinterrupt–ensuresmaximumLmeslice

5


(Linux)Terminology•  mm_struct–representsanaddressspaceinkernel•  task–representsathreadinthekernel–  Ataskpointsto0or1mm_structs

•  Kernelthreadsjust“borrow”previoustask’smm,astheyonlyexecuteinkerneladdressspace

– Manytaskscanpointtothesamemm_struct•  MulL-threading

•  Quantum–CPULmeslice

6


Outline•  Policygoals•  Low-levelmechanisms•  O(1)Scheduler•  CPUtopologies•  Schedulinginterfaces

7


Policygoals•  Fairness–everythinggetsafairshareoftheCPU•  Real-Lmedeadlines–  CPULmebeforeadeadlinemorevaluablethanLmeaXer

•  Latencyvs.Throughput:Timeslicelengthmaeers!–  GUIprogramsshouldfeelresponsive–  CPU-boundjobswantlongLmeslices,beeerthroughput

•  UserprioriLes–  Virusscanningisnice,butIdon’twantitslowingthingsdown

8


NoperfectsoluLon•  OpLmizingmulLplevariables•  LikememoryallocaLon,thisisbest-effort–  Someworkloadsprefersomeschedulingstrategies

•  Nonetheless,somesoluLonsaregenerallybeeerthanothers

9


Contextswitching•  Whatisit?–  Swapouttheaddressspaceandrunningthread

•  Addressspace:–  Needtochangepagetables–  Updatecr3registeronx86–  SimplifiedbyconvenLonthatkernelisatsameaddressrangeinallprocesses

– Whatwouldbehardaboutmappingkernelindifferentplaces?

10


Othercontextswitchingtasks•  Swapoutotherregisterstate–  Segments,debuggingregisters,MMX,etc.

•  IfdeschedulingaprocessforthelastLme,reclaimitsmemory

•  Switchthreadstacks

11


Switchingthreads•  ProgrammingabstracLon:

/*Dosomework*/schedule();/*Somethingelseruns*//*Domorework*/

12


Howtoswitchstacks?•  Storeregisterstateonthestackinawell-definedformat

•  Carefullyupdatestackregisterstonewstack–  Tricky:can’tusestack-basedstorageforthisstep!

13


Example

Thread1(prev)

Thread2(next)

/* eax is next->thread_info.esp *//* push general-purpose regs*/push ebpmov esp, eaxpop ebp/* pop other regs */

ebp

esp

eax

regs

ebp

regs

ebp

14


Weirdcodetowrite•  Insideschedule(),youendupwithcodelike:switch_to(me, next, &last);/* possibly clean up last */•  Wheredoeslastcomefrom?–  Outputofswitch_to– Wrieenonmystackbypreviousthread(notme)!

15


Howtocodethis?•  Pickaregister(sayebx);beforecontextswitch,thisisapointertolast’slocaLononthestack

•  Pickasecondregister(sayeax)tostoresthepointertothecurrentlyrunningtask(me)

•  MakesuretopushebxaXereax•  AXerswitchingstacks:–  popebx/*eaxsLllpointstooldtask*/–  mov(ebx),eax/*storeeaxatthelocaLonebxpointsto*/

–  popeax/*Updateeaxtonewtask*/

16



17


Strawmanscheduler•  Organizeallprocessesasasimplelist•  Inschedule():–  Pickfirstoneonlisttorunnext–  Putsuspendedtaskattheendofthelist

•  Problem?–  Onlyallowsround-robinscheduling–  Can’tprioriLzetasks

18


Evenstraw-ierman•  NaïveapproachtoprioriLes:–  ScantheenLrelistoneachrun–  Orperiodicallyreshufflethelist

•  Problems:–  Forking–wheredoeschildgo?– Whataboutifyouonlyusepartofyourquantum?

•  E.g.,blockingI/O

19


O(1)scheduler•  Goal:decidewhotorunnext,independentofnumberofprocessesinsystem–  SLllmaintainabilitytoprioriLzetasks,handleparLallyunusedquanta,etc

20


O(1)Bookkeeping•  runqueue:alistofrunnableprocesses–  Blockedprocessesarenotonanyrunqueue–  ArunqueuebelongstoaspecificCPU–  Eachtaskisonexactlyonerunqueue

•  Taskonlyscheduledonrunqueue’sCPUunlessmigrated

•  2*40*#CPUsrunqueues–  40dynamicprioritylevels(morelater)–  2setsofrunqueues–oneacLveandoneexpired

21


O(1)DataStructuresAcLve Expired

139

138

137

100

101

...

139

138

137

100

101

...

22


O(1)IntuiLon•  Takethefirsttaskoffthelowest-numberedrunqueueonacLveset–  Confusingly:alowerpriorityvaluemeanshigherpriority

•  Whendone,putitonappropriaterunqueueonexpiredset

•  OnceacLveiscompletelyempty,swapwhichsetofrunqueuesisacLveandexpired

•  ConstantLme,sincefixednumberofqueuestocheck;onlytakefirstitemfromnon-emptyqueue

23


O(1)ExampleAcLve Expired

139

138

137

100

101

...

139

138

137

100

101

...

Pickfirst,highest

prioritytasktorun

Movetoexpiredqueuewhenquantumexpires

24


Whatnow?AcLve Expired

139

138

137

100

101

...

139

138

137

100

101

...

25


BlockedTasks•  WhatifaprogramblocksonI/O,sayforthedisk?–  ItsLllhaspartofitsquantumleX–  Notrunnable,sodon’twasteLmepuungitontheacLveorexpiredrunqueues

•  Weneeda“waitqueue”associatedwitheachblockableevent–  Disk,lock,pipe,networksocket,etc.

26


BlockingExampleAcLve Expired

139

138

137

100

101

...

139

138

137

100

101

...

Disk

Blockondisk!

Processgoesondiskwaitqueue

27


BlockedTasks,cont.•  AblockedtaskismovedtoawaitqueueunLltheexpectedeventhappens–  Nolongeronanyac.veorexpiredqueue!

•  Diskexample:–  AXerI/Ocompletes,interrupthandlermovestaskbacktoacLverunqueue

28


Timeslicetracking•  Ifaprocessblocksandthenbecomesrunnable,howdoweknowhowmuchLmeithadleX?

•  EachtasktracksLcksleXin‘Lme_slice’field–  OneachclockLck:current->time_slice--–  IfLmeslicegoestozero,movetoexpiredqueue

•  RefillLmeslice•  Schedulesomeoneelse

–  AnunblockedtaskcanusebalanceofLmeslice–  ForkinghalvesLmeslicewithchild

29


MoreonprioriLes•  100=highestpriority•  139=lowestpriority•  120=basepriority–  “nice”value:user-specifiedadjustmenttobasepriority–  Selfish(notnice)=-20(Iwanttogofirst)–  Reallynice=+19(Iwillgolast)

30


BaseLmeslice

•  “Higher”prioritytasksgetlongerLmeslices–  Andrunfirst

time =(140− prio)*20ms prio <120

(140− prio)*5ms prio ≥120

#

$%

&%

31


Goal:ResponsiveUIs•  MostGUIprogramsareI/Oboundontheuser–  UnlikelytouseenLreLmeslice

•  UsersgetannoyedwhentheytypeakeyandittakesalongLmetoappear

•  Idea:giveUIprogramsapriorityboost–  Gotofrontofline,runbriefly,blockonI/Oagain

•  WhichonesaretheUIprograms?

32


Idea:InferfromsleepLme•  BydefiniLon,I/OboundapplicaLonsspendmostoftheirLmewaiLngonI/O

•  WecanmonitorI/OwaitLmeandinferwhichprogramsareGUI(anddiskintensive)

•  GivetheseapplicaLonsapriorityboost•  Notethatthisbehaviorcanbedynamic–  Ex:GUIconfiguresDVDripping,thenitisCPU-bound–  Schedulingshouldmatchprogramphases

33


Dynamicprioritydynamicpriority=max(100,min(sta.cpriority−bonus+5,139))•  BonusiscalculatedbasedonsleepLme•  Dynamicprioritydeterminesatasks’runqueue•  ThisisaheurisLctobalancecompeLnggoalsofCPUthroughputandlatencyindealingwithinfrequentI/O– MaynotbeopLmal

34


DynamicPriorityinO(1)Scheduler•  Important:Therunqueueaprocessgoesinisdeterminedbythedynamicpriority,notthestaLcpriority–  DynamicpriorityismostlydeterminedbyLmespentwaiLng,toboostUIresponsiveness

•  Nicevaluesinfluencesta.cpriority(directly)–  StaLcpriorityisastarLngpointfordynamicpriority–  Nomaeerhow“nice”youare(oraren’t),youcan’tboostyour“bonus”withoutblockingonawaitqueue!

35


Rebalancingtasks•  Asdescribed,onceataskendsupinoneCPU’srunqueue,itstaysonthatCPUforever

36


RebalancingCPU0 CPU1

......

CPU1NeedsMoreWork!

37


Rebalancingtasks•  Asdescribed,onceataskendsupinoneCPU’srunqueue,itstaysonthatCPUforever

•  WhatifalltheprocessesonCPU0exit,andalloftheprocessesonCPU1forkmorechildren?

•  Weneedtoperiodicallyrebalance•  Balanceoverheadsagainstbenefits–  Figuringoutwheretomovetasksisn’tfree

38


Idea:IdleCPUsrebalance•  IfaCPUisoutofrunnabletasks,itshouldtakeloadfrombusyCPUs–  BusyCPUsshouldn’tloseLmefindingidleCPUstotaketheirworkifpossible

•  TheremaynotbeanyidleCPUs–  OverheadtofigureoutwhetherotheridleCPUsexist–  JusthavebusyCPUsrebalancemuchlessfrequently

39


Averageload•  HowdowemeasurehowbusyaCPUis?•  AveragenumberofrunnabletasksoverLme•  Availablein/proc/loadavg

40


Rebalancingstrategy•  ReadtheloadavgofeachCPU•  Findtheonewiththehighestloadavg•  (Handwaving)Figureouthowmanytaskswecouldtake–  Ifworthit,locktheCPU’srunqueuesandtakethem–  Ifnot,tryagainlater

41


Whynotrebalance?•  IntuiLon:IfthingsrunsloweronanotherCPU•  Whymightthishappen?–  NUMA(Non-UniformMemoryAccess)–  Hyper-threading– MulL-corecachebehavior

•  Vs:SymmetricMulL-Processor(SMP)–performanceonallCPUsisbasicallythesame

42


SMP

•  AllCPUssimilar,equally“close”tomemory

CPU0 CPU1 CPU2 CPU3

Memory

43


NUMA

•  WanttokeepexecuLonnearmemory;highermigraLoncosts

CPU0 CPU1 CPU2 CPU3

MemoryMemory

Node Node

44


SchedulingDomains•  GeneralabstracLonforCPUtopology•  “Tree”ofCPUs–  Eachleafnodecontainsagroupof“close”CPUs

•  WhenanidleCPUrebalances,itstartsatleafnodeandworksuptotheroot– Mostrebalancingwithintheleaf–  Higherthresholdtorebalanceacrossaparent

45


SMPSchedulingDomain

CPU0 CPU1 CPU2 CPU3

Flat,allCPUSequivalent!

46


NUMASchedulingDomains

CPU0 CPU1 CPU2 CPU3

CPU0startsrebalancingherefirst

Higherthresholdtomovetosibling/parent

47


Hyper-threading•  PrecursortomulL-core–  AfewmoretransistorsthanIntelknewwhattodowith,butnotenoughtobuildasecondcoreonachipyet

•  Duplicatearchitecturalstate(registers,etc),butnotexecuLonresources(ALU,floaLngpoint,etc)

•  OSview:2logicalCPUs•  CPU:pipelinebubbleinone“CPU”canbefilledwithoperaLonsfromanother;yieldinghigheruLlizaLon

48


Hyper-threadedscheduling•  Imagine2hyper-threadedCPUs–  4LogicalCPUs–  Butonly2CPUs-worthofpower

•  SupposeIhave2tasks–  Theywilldomuchbeeeron2differentphysicalCPUsthansharingonephysicalCPU

•  Theywillalsocontendforspaceinthecache–  Lessofaproblemforthreadsinsameprogram.Why?

49


NUMA+HyperthreadingDomains

CPU0

CPU1

NUMADOMAIN1 NUMADOMAIN1

CPU2

CPU3

CPU4

CPU5

CPU6

CPU7

LogicalCPU

PhysicalCPU

isascheddomain

50


MulL-core•  Morelevelsofcaches•  MigraLonamongCPUssharingacachepreferable– Why?– Morelikelytokeepdataincache

•  Schedulingdomainsbasedonsharedcaches–  E.g.,coresonsamechipareinonedomain

51



52


SeungprioriLes•  setpriority(which,who,niceval)andgetpriority()– Which:process,processgroup,oruserid–  PID,PGID,orUID–  Niceval:-20to+19(recallearlier)

•  nice(niceval)–  Historicalinterface(backwardscompaLble)–  Equivalentto:

•  setpriority(PRIO_PROCESS,getpid(),niceval)

53


SchedulerAffinity•  sched_setaffinityandsched_getaffinity•  CanspecifyabitmapofCPUsonwhichthiscanbescheduled–  Beeernotbe0!

•  Usefulforbenchmarking:ensureeachthreadonadedicatedCPU

54


yield•  Movesarunnabletasktotheexpiredrunqueue–  Unlessreal-Lme(morelater),thenjustmovetotheendoftheacLverunqueue

•  Severalotherreal-LmerelatedAPIs

55


Summary•  UnderstandcompeLngschedulinggoals•  Understandhowcontextswitchingimplemented•  UnderstandO(1)scheduler+rebalancing•  UnderstandvariousCPUtopologiesandschedulingdomains

•  Schedulingsystemcalls

56

Date post:	15-Aug-2021
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Scheduling - University of North Carolina at Chapel Hill · 2016. 7. 29. · Scheduling Domains •...

Documents