1

Energy Efficiency Op-miza-on with GEOPM

JonathanEastep[jonathan.m.eastep@intel.com]PrincipalEngineer,PhD

12November2017

h9p://geopm.github.io/geopm

2

§ GEOPMoverview,use-cases,andstatus§  Intel,LRZ,LLNL,ArgonnecollaboraMon§ Newexperimentalresults§ CollaboraMonnextsteps

Outline

IntelCorporaMon

3

§  RunMmeforin-bandpowermanagementandopMmizaMon§  On-the-flymonitoringofHWcounters&applicaMonprofiling§  Feedback-guidedopMmizaMonofHWcontrolknobse[ngs

§  Opensourceso\ware(flexibleBSDthreeclauselicense)

§  Extensiblethroughpluginarchitecture§  AddnewenergyopMmizaMonstrategies§  Addsupportfornewarchitecturesbeyondx86(trulyopen)

§  DesignedforholisMcopMmizaMon§  Job-wideglobalopMmizaMonofHWcontrolknobse[ngs§  ApplicaMon-awarenessformaxspeeduporenergysavings

§  Scalableviadistributedtree-hierarchicaldesign,algorithms

MPI Comms Overlay Shared Mem Region

Power-Aware RM / Scheduler

GEOPM Controller

SHM

GEOPM

GEOPMRoot

GEOPMAggregator

GEOPMAggregator

GEOPMLeaf

Msr-safe (or Other Drivers for Non-x86 PlaPorms)

MSR

MPI Ranks 0 to i-1

GEOPMLeaf

Processor

MPI Ranks i to j-1

Processor

MPI Ranks j to k-1

GEOPMLeaf

Processor

MPI Ranks k to n-1

GEOPMLeaf

Processor

Projecturl:hdp://geopm.github.io/geopmContact:jonathan.m.eastep@intel.com

4

§  Turn-key(requiresnoappannotaMon):§  AutomaMconlinejobprofiling

§  Node-level:tracesamplesofprocessorcountersandcorrelateHWacMvitytoeachOpenMPparallelregion

§  Job-level:aggregatetheenergycountersacrossalljobcomputenodestomonitoroveralljobpowerorenergy

§  AutomaMcofflineoronlineopMmizaMon§  Willtalkmoreaboutthistoday

§  OfflinevisualizaMonofprofiledata§  Pythonscriptsleveragingpandasfordataanalysis§  Helpfulfordebuggingnewpluginsorunderstanding

howtheyopMmizeenergyorrunMme§  Plottraceofplugindecisionsanddatathey’rebasedon

GEOPM Use Cases

IntelCorporaMon

§  Advanced(requiresusingGEOPMprofilingAPIforappannotaMon):§  AutomaMconlinerebalancing

ofpower&perfamongnodes§  Purpose:acceleratecriMcalpathnodes

inMPIbulk-synchronousapplicaMons§  RefertoISC’17paperonGEOPMby

Eastepetal.formoreinfo§  Note:workinprogresstomakethe

annotaMonautomaMc/turn-keytoo

5

GEOPM Community (1) Ins$tu$on PrincipalInves$gator Project

NameProjectScope Contribu$on

TypeTimeSpan

QualityLevel

Funded?

Argonne KalyanKumaranVitaliMorozov

CORAL 1.GEOPM1.0productdevelopment Sponsor Q2’15–Q4’17

Product Yes

IBMSTFC–Hartree

VadimElisseevMilosPuzovicNeilMorgan

1.GEOPMporttoPower8+NVLink2.IntegraMonofGEOPMwithEAS

Contributor Q4’16–TBD

Research Yes

LLNL BarryRountreeAniruddhaMarathe

CRADA 1.IntegraMonofGEOPMandConductorrunMmetech2.StudiestomoMvateGEOPM/HWcodesign

Contributor Q3’13–TBD

Research Yes

LLNLU.ofArizonaArgonne

TapasyaPatkiDaveLowenthalPeteBeckman

ECPPSECPArgo-GRM

1.ExascalepowerstackleveragingGEOPM2.IntegraMonofGEOPM+Caliperframework3.IntegraMonofGEOPMwithEAS4.PortofGEOPMtonon-x86architecture

Contributor Q1’17–Q4’19

Near-Product

Yes

LRZ DieterKranzlmüllerHerbertHuberTorstenWilde

1.EnergyopMmizaMonpluginforGEOPM1.02.PowerramplimiMngpluginforGEOPM1.x

Contributor Q3’17–Q4’20

Near-Product

Yes

Sandia JamesLarosRyanGrant

PowerAPI

1.GEOPMandPowerAPIxfacecompaMbility2.PowerAPIcommunityWGkickoffatIntel

User Q4’14-TBD

IndustryStandard

Yes

*

*

*=collaboratorwillbesharingtheirGEOPMusagesandexperiencesatSC17:BoFonPowerAPI,GEOPM,andRedfish

6

GEOPM Community (2) Ins$tu$on PrincipalInves$gator Project

NameProjectScope Contribu$on

TypeTimeSpan

QualityLevel

Funded?

Argonne KalyanKumaranVitaliMorozovKevinHarms

1.GEOPM>1.0featuredevelopment2.GEOPMenablementforsystempowercapping+EAS3.StudiestomoMvateGEOPM/hardwarecodesign

Sponsor Q1’18–Q4’21

Product WIP

CINECA CarloCavazzoni 1.SystemlevelrunMmeforpowercappingandpowerramplimiMngleveragingGEOPM

Contributor Q2’18–Q1’21

Near-Product

WIPꝉ

IT4I LubomirRiha 1.GEOPMportstoOpenPOWERandARM2.ExtensionstoGEOPMapplicaMonprofiler3.IntegraMonofGEOPMwithEAS

Contributor Q2’18–Q1’21

Near-Product

WIPꝉ

E4 FabrizioMagugliani 1.GEOPMporttoOpenPOWER Contributor Q2’18–Q1’21

Near-Product

WIPꝉ

PNNL LeonSong 1.GEOPMextensionstotunenewHWcontrolknobse[ngs2.GEOPMextensionsforcoordinatedtuningofSWparamsandHWcontrolknobse[ngs

Contributor Q1’19–Q4’20

Research WIPꝉ

ꝉ=lederofintentorequivalentin-hand(non-binding)

7

GEOPM Release Schedule

AlphaQ2’17

BetaQ2’18

v1.0Q4’18

Commitment:

AlphaQ2’17

BetaQ1’18

v1.0Q2’18

StretchGoal:

TOSS3.x

ISC’18 SC’18

IntelCorporaMon

Announcement:OpenHPCapplicaMonhasbeensubmided.UnderconsideraMon.

8

§  Presentfocus:§  DevelopnewtechniquesinGEOPMtoimproveenergy-to-soluMonw/modestimpacttoMme-to-soluMon§  WorktowardintegraMngGEOPMandthesetechniquesintoproducMonsystemsatLRZ,LLNL,andArgonne

§  EnergyopMmizaMonapproach:§  AutomaMcallyadaptprocessorcorefrequencybasedoncharacterisMcsofindividualapplicaMons§  Runthecoresslowertosaveenergyiftheappisbodleneckedbythememoryornetworksubsystems§  Adaptfrequencyatafine-grainedMmescale:differentfrequencyforeachcomputaMonalphaseintheapp§  It’scriMcaltoadapttophasessinceeachphase’srunMmecanhavewildlydifferentfrequencysensiMvity

§  InnovaMonvspriorart:§  Novelper-phase-adaptaMonenablesbiggerenergysavingsandlowerimpacttoMme-to-soluMon

Intel, LRZ, LLNL, and Argonne Collab

Bigwinsarepossible:upto16.5%energysavingsat0.3%increasein$me-to-solu$onStatus:trendingtocompletethisworkinMmeforGEOPMBetarelease

9

Two Techniques Under Development

IntelCorporaMon

§  Startsimple:offlineautoma*cfrequencyopMmizaMonviascriptsandDeciderplugin§  GEOPMplugintakesineachphase’sfrequencyas

aninput,appliestheinpudedfrequencyuponphaseentry

§  Offline,foragivenapplicaMon,scriptssweepoverpluginfrequencyconfiguraMonstocharacterizetheapp’sphases,idenMfybestfrequencies(minenergy@<10%execuMonMmeimpact)

§  EachMmeappislaunched,itrunswithbestphasefrequencyconfiguraMonsidenMfiedbythescripts

§  OfflineapproacheslikethishaveknownlimitaMons:§  TheybreakdownwhenphaseexecuMonMmevs

frequencyscalingdependsonrunMmefactors

§  Getfancy:onlineautoma*cfrequencyopMmizaMonviaDeciderplugin§  GEOPMplugincharacterizestheapplicaMon

onlineandtuneseachphase’sfrequencyduringaniniMal“learning”period

§  Whenfinishedlearning,pluginusesbestfrequencyforeachphaseandreapsenergybenefitsfortherestofexecuMon

§  Onlineapproacheslikethishavetradeoffs:§  They’rerobustagainstcaseswhenphase

execuMonMmevsfrequencyscalingdependsonrunMmefactorsandthebestphasefrequencycan’tbedeterminedreliablyoffline

§  Theybreakdownwhenlearningoverheadisanon-trivial%oftotalexecuMonMme(notcommoncaseinlong-runningHPCapps)

10

Experimental Setup: Measurement

IntelCorporaMon

§  CommonmeasurementmethodologyusedforevaluaMngbothtechniques

§  ProgrammersinstrumentapplicaMonphaseentry/exitwithaprofilingAPIprovidedwithGEOPM§  APIisdesignedtobelightweightandeasyforprogrammerstouse§  Nonetheless,workunderwaytoautomatephaseinstrumentaMonusingOMPT

§  GEOPMrunMmeperformsenergyandexecuMonMmemeasurements§  UsesthephaseinstrumentaMontotrackphaseentryandexitMmestamps§  Periodicallysamplesenergyusingprocessorcounters(orothermeans)§  IncorporatesaccounMnglogictotracktotalphaseenergyandexecuMonMmebasedonabove§  GEOPMreportfeatureoutputsthisper-phasedatatotextfileoneachnode

§  Alldatapointsinchartssharedtodayrepresentanaverageofatleast7trials§  Foreachtrial,resultsaveragedacrossnodessinceresultsvaryfordifferentnodes

11

Experimental Setup: Workloads

IntelCorporaMon

•  Studiedworkloadsincluding:•  Proxyapp:modelbulk-synchronousapplicaMonwithconfigurablebalanceofDGEMMand

STREAMcomputaMon(fromGEOPMtutorialsandintegraMontests)•  FT:adiscrete3-dFFTkernel(fromNASParallelBenchmarksuite)•  miniFE:finiteelementcode(fromCORALprocurementbenchmarks)•  Nekbone:thermalhydraulicscode(fromCORALprocurementbenchmarks)

•  AppliedthefollowingconvenMonswhenconfiguringworkloads:•  SizedtheproblemtofitwithinavailableDRAMonthenode•  TestedseveralconfigsofMPIranksandOpenMPthreadsperrank;usedtheconfigwithlowest

Mme-to-soluMon•  SetapplicaMonaffinitymaskstostayoffofCPU0tominimizeOSjidereffects•  SetGEOPMcontrollerprocessaffinitytoCPU1(applicaMonaffinitysettostayoffofthisCPU)•  Didnotusehyperthreads

12

Experimental Setup: Hardware

IntelCorporaMon

JLSEBroadwellXeonCluster(Argonne)

QuartzBroadwellXeonSystem(LLNL)

WorkloadconfiguraMonunderstudy

proxyapp:used1instancepernodewithnointer-nodecommunicaMon

FT,miniFE,Nekbone,Gadget:allusedmulM-nodeconfiguraMonswithMPIcommunicaMon

Processorandmemoryspecs

4x44-corenodes(dual-socket),Broadwellserverprocessors,128GBofDRAM

Upto8x36-corenodes(dual-socket),Broadwellserverprocessors,128GBofDRAM

Processorcorefrequencyrange

1.2GHzto2.2GHzsMcker,Turboisenabled:frequencymayexceedsMcker

1.2GHzto2.1GHzsMcker,Turboisenabled:frequencymayexceedsMcker

Networkhardware N/A IntelOmniPathHFIsandswitches

So\wareenvironment RHEL7.4Linuxdistro,IntelP-statedriverrunningsetto‘performance’withmin=max=sMcker,Intelcompilertoolchain,IntelMPIimplementaMon

RHEL7.3LinuxdistrowithIntelP-statedriverdisabled,legacygovernorsetto‘performance’,Intelcompilertoolchain,MVAPICH2MPIimplementaMon

13

Experimental Setup: 3 Inves-ga-ons

IntelCorporaMon

1.  OpportunityAnalysis•  Useproxyapp(parameterizedmodelapplicaMon)todetermineenvelopeofenergy-to-soluMonand

Mme-to-soluMonimpactwe’llseeoverthelandscapeofBSPapplicaMons•  Measureenergy-to-soluMondecreaseandMme-to-soluMontradeoffrela$vetorunningats$ckeron

theJLSEclusteratArgonne•  Comparetwodifferentuse-casesfortheofflinetechniquewedeveloped:

•  ‘OfflineautomaMcapplica*onbest-fit:’allphasesrunatcommonfrequency(best-fitacrossall)•  ‘OfflineautomaMcper-phasebestfit:’eachphaserunsatthebestfrequencyforit

2.  BenchmarkofflineenergyopMmizaMontechnique•  TargetFT,miniFE,andNekboneworkloads•  SameasabovebuttargetslesssyntheMcworkloadsandperformsexperimentsonLLNLQuartzsystem

3.  BenchmarkonlineenergyopMmizaMontechnique•  TargettheproxyappandperformexperimentsonJLSEclusteratArgonne•  Comparetheonlineandofflinetechniqueswedeveloped:

•  ‘OfflineautomaMcper-phasebest-fit:’scriptsidenMfybestfrequencyviaofflinecharacterizaMon•  ‘OnlineautomaMcper-phasebestfit:’GEOPMpluginperformscharacterizaMon/tuningonline

14

Results: Opportunity Analysis

BigenergysavingsarepossiblewithfrequencyopMmizaMoninGEOPMvsrunningworkloadsatsMcker:upto16.5%energysavingsat0.3%increasein$me-to-solu$on

Withper-phaseopMmizaMon,energysavingsincreasewithincreasein%Mmeinmemory-limitedphasePer-phaseopMmizaMonsimultaneouslyoffersbederenergy-to-soluMonANDMme-to-soluMonversus

opMmizingfrequencyacrosstheblendedcharacterisMcsofallapplicaMonphases

0

5

10

15

20

18% 32% 40% 49% 56% 64% 75%

%decreaseinene

rgy-to-soluM

on

%MmeinSTREAMphase

Energy-to-SoluMonDecreaseofflineautoapplicaMonbest-fitofflineautoper-phasebest-fit

-2

0

2

4

6

8

10

18% 32% 40% 49% 56% 64% 75%

%increaseto

Mme-to-soluM

on

%MmeinSTREAMphase

Time-to-SoluMonIncreaseofflineautoapplicaMonbestfitofflineautoper-phasebest-fit

1.1E+09

1.3E+09

1.5E+09

1.7E+09

1.9E+09

2.1E+09

2.3E+09

18% 32% 40% 49% 56% 64% 75%

best-fitfrequ

ency(H

z)

%MmeinSTREAMphase

OfflineAutoAppBest-FitFrequency

DGEMMBest-Fit

STREAMBest-Fit

15

Results: Offline App vs Per-Phase Best-Fit

IntelCorporaMon

Energy-to-SoluMonandTime-to-SoluMonComparisononQuartzOfflineAutomaMcApplica*onBest-Fit OfflineAutomaMcPer-PhaseBest-Fit

Workload EtSDecreasevsSMcker

TtSIncreasevsSMcker

EtSDecreasevsSMcker

TtSIncreasevsSMcker

FT 9.5% 6.8% 15.8% 4.8%

miniFE 8.5% 5.8% CollecMngdatanow CollecMngdatanow

Nekbone 7.9% 2.4% CollecMngdatanow CollecMngdatanow

ResultsstarMngtoconfirmthatGEOPMprovidesbenefitsforanumberofworkloadsbeyondourproxyapp

Moredataontheway,butdatastarMngtosuggestper-phasefrequencyopMmizaMonsimultaneouslyoffersbederenergy-to-soluMonANDMme-to-soluMonvsopMmizingfrequencyacrossblendedcharacterisMcsofwholeapp

16

Results: Online vs Offline Technique

Remember,offlineapproachisbridle.Thegoal:same(orbeder)resultsviamorerobustonlineapproachWethinkmuchoftheEtSandTtSgapcanbeclosedviaaddressingfrequencylatency&doinglongerrunsFine-tuningneeded,butalreadyseeingpromisingdecreasesinenergy-to-soluMonwithonlineapproach

ExplanaMonofEtSandTtSgaps:•  Runswereshorterthanrealapps

->noMceable“learning”overhead•  Reduced#samplesinlearning

periodtoreduceoverhead->morenoise-relatedcontrolerrors

•  Observedlatencybetweenfrequencychangerequestsandenactment(10sofmilliseconds)->notrunningatdesiredfrequencyimmediately,confusingalgorithm

0

2

4

6

8

10

12

14

16

18

18% 32% 40% 49% 56% 64% 75%%decreaseinene

rgy-to-soluM

on

%MmeinSTREAMphase

Energy-to-SoluMonDecrease

onlineautoper-phasebest-fitofflineautoper-phasebest-fit

-2

0

2

4

6

8

10

18% 32% 40% 49% 56% 64% 75%%increaseto

Mme-to-soluM

on

%MmeinSTREAMphase

Time-to-SoluMonIncrease

onlineautoper-phasebest-fitofflineautoper-phasebestfit

17

Next Steps

IntelCorporaMon

1.  Extendonlineautoper-phaseenergyopMmizaMonplugin•  LeverageGEOPMfeatureforautomaMcdetecMonofOpenMPparallelregion

entry/exittoremoveneedforprogrammertoinstrumentphaseswithAPI

2.  ExpandevaluaMonstoincludemorebenchmarks(e.g.Gadget@LRZ)andmorearchitectures(e.g.ThetaKnightsLandingsystem@Argonne)

3.  PolishtheenergyopMmizaMontechniquesdemonstratedtodayandincludetheminGEOPMBeta

4.  WorkwithLRZ,LLNL,andArgonnetowarddeploymentontheirproducMonsystems

18IntelCorporaMon

GEOPM Core Team Acknowledgements

HardwareTeam:•  ProcessorFirmware

•  RevathyRajasree

• HardwareArchitectureandDesign•  FedeArdanaz•  FuatKeceli•  KellyLivingston•  LowrenLawson

So\wareTeam:• GEOPMDevelopment

•  ChrisCantalupo•  DianaGudman•  BradGeltz•  BrandonBaker

•  Research•  SidJana•  AsmaAl-Rawi• MadhiasMaiterth

LeadArchitect:•  JonathanEastep,PrincipalEngineer

Backup Slides

20

Per-Phase Best-Fit Frequency Table for FT

IntelCorporaMon

Widerangeinbest-fitfrequencyacrossphases

CodeinspecMonconfirmsthephasesthattolerate1.2GHzarememory-limitedorMPIphases,asexpected

PhasestoleraMng1.2GHzuseroughlythesame%oftotalexecuMonMmeasphasesneeding2.0-2.1GHz

Runningallphasesat2.0-2.1GHzwouldwasteenergyinTRANSPOSE,MPI,andEVOLVEphaseswithlidlebenefittoexecuMonMme

Runningallphasesat1.2GHzwouldharmexecuMonMmeofFFTandINDEX_MAPphasesbymuchmorethan10%

Thesefactsillustratewhyitissub-opMmaltoapplythesamefrequencyacrossallphases

FTPhase BestFrequency

%TotalExecTimeatS$cker

Transpose_1 1.2GHz 06.7%

Transpose_2 1.2GHz 06.8%

MPI_All2All 1.2GHz 35.2%

EVOLVE 1.2GHz 09.9%

FFT_1_2 2.0GHz 13.1%

FFT_1 2.0GHz 13.1%

FFT_2 2.1GHz 16.0%

INDEX_MAP 2.1GHz 0.03%

21

Energy May Not Monotonically Increase •  Onlinealgorithmcan’tsimplywalkdownfrequencyunMlthere’sa10%TtSincrease(relaMvetosMcker)

•  Reason:reducingfrequencybelowsMckerincreases(notdecreases)energyforDGEMMphase;naïvealgorithmswouldchooselowerfrequencyforDGEMMthanisopMmalfromanenergyperspecMve!

166001680017000172001740017600178001800018200184001860018800

0

20

40

60

80

100

120

energy(J)

runM

me(s)

frequency(Hz)

DGEMMPhaseScaling

runMme

energy

22

1.   At-scaleloadimbalanceduetomanufacturingvaria$oninpower-cappedsystems.ThisproblemisdeemedoneofthekeyExascale-erapowerchallenges.DevelopingGEOPMandtechniquestoaddressthisproblemoverthepast6yearsmademeaPrincipalEngineeratIntel.

2.   Gapincommunityenergymanagementresearchtools.Therewaspreviouslynopla�ormforenergymanagementresearchthatwasopen,scalable,robust,flexible,portable(trulyopen),andbackedbyseriousengineeringresources.NowthecommunityisusingGEOPM,porMngtonon-x86architectures,integraMngtheiropMmizaMontechniquesintoit,andintegraMngitwithotherso\warecomponents.

3.   Gapinindustryserverpowermanagementroadmapsandtechnicaldirec$ons.Powermanagementwaspreviouslydonenode-locally.Techniqueswereoblivioustoapplica3on-levelinforma3onsuchasbodlenecksonremotenodesthatcouldlimitoverallperformanceandwereunabletoforecastwhatcomputaMonwasgoingtohappeninthefutureandopMmizepower-performancepolicyaccordingly.GEOPMaddsacriMcallayerofglobalopMmizaMonacrossnodes,applicaMonandapplicaMonphaseawareness,andforecasMngcapabiliMes.SeeISC’17paperfordemoofbenefits(upto30%speedup).

What Problems Does GEOPM Address?

IntelCorporaMon

23

GEOPM Interfaces and HPC Stack Integra-on

IntelCorporaMon

§  GEOPM=job-levelpowermanager§  CoordinatesopMmizaMonofhardwarecontrol

knobse[ngsacrossallcomputenodesinjob§  Userspaceso\ware;accesstocontrolknobs

facilitatedviaOSdriverlikemsr-safefromLLNL§  Supportedcontrolknobs:nodepowercaps,

processorfrequencycontrols,morecoming

§  RunGEOPMviajoblaunchwrappers§  Includeswrappersforsrunandaprunsofar§  Samesyntaxbutwithaddedflagstoconfigure

geopm;e.g.:powerbudgetandplugin§  AdmincanprovidedefaultsviaJSONconfigfile

§  IntegrateswithRMandscheduler§  Near-term:GEOPMcanstandalone§  Long-term:integrateswithemergingSystem

PowerManager(SPM)runMmecomponent§  GEOPMinterfacetosystempowermanager

§  Feedback:GEOPMreportsjobpowerconsumpMonandotherjobcharacterisMcs

§  Control:SPMdynamicallyreconfiguresjobpowerbudgetand/orGEOPMplugin

Power-Aware Resource Manager / Scheduler

GEOPM - Resource Manager Interface

Job Power Manager GEOPM

Processor PM and Perf Counter Interfaces

GEOPM Application Profiling Interface

(Optional)

Job Launch Wrappers

PM Interfaces for System-Level

Resources

3rd parties

Intel GEOPM Team

Intel PM Arch Team

Research / Future Work

System Power Manager Runtime

JSON Config File with Defaults

24

§  SeeGEOPMISC’17paperbyEastepetal.fordetailsofexperimentalsetupandfurtheranalysis§  Paperdemonstratespowerbalancingplugin:itleveragesannotaMonofapplicaMon’soutersynchronizaMonlooptodetect

criMcalpathnodesandthenreallocatespoweramongnodesinordertoequalizetheirMmetocompletealoopiteraMon§  ComparedoverallMme-to-soluMonwhencappingjobpoweron12-nodeKNLclusterwithpowerbalancerplug-invs.staMc

uniformpowerdivision(baseline);sweptoverarangeofdifferentjobpowercaps§  Regionofinterestinjobpowercaps:low-endofjobpowercapswasselectedtoavoidinefficientclockthrodlingandthehigh-

endofthejobpowercapsequalstheunconstrainedpowerconsumpMonoftheworkload§  Mainresult:upto30%improvementinMme-to-soluMonatlowendofcaps(miniFE,CoMD,AMG),withupto9-23%forthe

rest.Improvementgenerallyincreasesaspowerismoreconstrained

Results: Inter-Node Power Balancing Use Case

IntelCorporaMon

25

Results: Four Addi-onal Workloads

IntelCorporaMon

26

Take-awaypoints:•  Resultsdemonstraterobustnessofpower

balancingalgorithmagainstMme-varyingamountsofworkintheouterloopandsharpshi\sincomputaMonal-intensity(topgraphs)

•  Node8,withlowestpowerefficiencyinourKNLcluster,isallocatedmorepower(middlegraphs)

•  PowerbalancingalgorithmimprovescriMcalpathloopMmebyfindingthepowerallocaMonthatroughlyequalizesthefrequenciesofallnodes(bodomgraphs)

GEOPMSpeedupAnalysis(usingincludedGEOPMTraceandPythonVisualizaMonTools)

IntelCorporaMon

27

§  GEOPMprojectisnotjustaso\wareproject.ItalsodrivescodesignofthefeaturesinIntelhardwareforpower-performancemonitoringandcontrol

§  Goalsaretosignificantlyadvancethestate-of-the-artinHPCpowermanagementtechnologyandtoensureGEOPMrunsbestonIntel

§  Researchareas:§  Processor:improvementstogranularity,reacMonMme,andinterfacesforexisMngfeatures§  Processor:hooksforGEOPMtoguideallocaMonofTurboheadroomamongcores§  Memory:hooksforGEOPMtohinttomemcontrollerwhenit’sbesttoenterlow-powerstates§  Network:hooksforGEOPMtoesMmatepower,managetradeoffsbetweenpowerand

bandwidthinHFIandswitches,andhinttoHFIwhenit’sbesttoenterlow-powerstates

Research on GEOPM/HW/FW Codesign

IntelCorporaMon

28

§  GEOPMso\warepackageisopensource,providesarichfeaturesetfreeofcharge

§  IntentisforIntel’sfutureworkontheso\waretobeopensourceaswell§  3rdparMesareabletomakeproprietaryextensionsofGEOPM(BSD3-clauselicense)

§  EnablesintegratorslikeDell/Cray/HPEtodevelopcommercialfor-profitplugins(i.e.addpowermanagementsecretsaucetodifferenMateyoursystemsvsthecompeMMon)

§  GEOPMteamcanhelpintegratorswiththisinaconsulMngcapacity

§  Intelcanexploredevelopingcustomprocessorfirmwareenhancementsforcustomers§  EnablesprocessorpowermanagementfirmwareandGEOPMpluginstobeco-opMmizedfor

individualcustomerneeds§  Enablesmanagementofhardwarecontrolknobse[ngswhicharenot(yet)publicallyavailable§  ProvidingGEOPMNREfundinginasystemcontractisagoodwaytoestablishsuchanengagement

GEOPM New Business Opportuni-es

IntelCorporaMon

InquirewithJonathanEastepformoreinformaMon:jonathan.m.eastep@intel.com