ModernScientificSoftwareManagementusingEasyBuild&co
PRACE-VI-SEEM2017SpringSchool-SystemAdministrationTrackApril25th2017-TheCyprusInstitute
[email protected] http://hpcugent.github.io/easybuild/
http://www.ugent.be/hpc https://www.vscentrum.be
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 2
• partofcentralITdepartmentofGhentUniversity(Belgium)
• centralisedscientificcomputingservices,training&support
• forresearchersofUGent,industry&knowledgeinstitutes
• memberofFlemishSupercomputerCentre(VSC)https://www.vscentrum.be/
• corevalues:empowerment-centralisation-automation-collaboration
HPC-UGent
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 3
• Masters&PhDinComputerSciencefromGhentUniversity(Belgium)
• joinedHPC-UGentteaminOctober2010
• maintasks:usersupport&training,softwareinstallations
• inheritedmaintenanceofEasyBuildin2011
• slowlyalsobecameleaddeveloper&releasemanager
• bigfanofloudmusic&FOSS(Free&OpenSourceSoftware)
[email protected](GitHub,IRC)-@kehoste(Twitter)
whoami
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 4
• focusisontoolsforHPCsystems,mostlyrelatedtousersupport
• inparticular,installing(scientific)softwareforusersofHPCclusters
• onlyactivelymaintained,well-documentedopensourcesoftwareprojects
• importantthingsIwillnotbetalkingabout(indetail):
• configurationmanagement(tips:Ansible,Warewulf,Quattor)
• security,monitoring,cloudservices,...
• MPI,performancetuning,accelerators,...
• thelatesthype(BigData,DeepLearning,...)
Scope
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 5
• Birds-eyeviewoftools&projects
• EasyBuild,Lmod,Singularity,ClusterShell,XALT/OGRT,OpenHPC
• IntroductiontoEasyBuild&Lmod
• Scope,motivation,terminology&features
• Installationandconfiguration
• Typicalusage
• Advancedtopics
Outline
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 6
• EasyBuild
• Lmod
• Singularity
• ClusterShell
• XALT&OGRT
• OpenHPC
Birds-eyeviewoftools&projects
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 7
• significantpartofusersupportservices
• traditionalpackagemanagersarenotsufficient
• installationshouldbedoneinanoptimisedway(fromsource)
• usersoftenlackrequiredknowledgetohandlethisthemselves(efficiently)
• scientificsoftwareinparticularcanbe...'challenging'
• littlecollaborationbetweenHPCsites,eventhoughthisisacommonburden
Usecase:installing(scientific)software
artworkonVSCBrENIACTier-1system
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 8
frameworkforbuilding&installing(scientific)softwareonHPCclusters
http://hpcugent.github.io/easybuild/
willbecoveredindetaillater...
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 9
• lotsof(scientific)softwareistypicallyprovidedonHPCclusters
• usuallyinnon-standardlocations,differentversions&variants,...
• usersneedtoquerywhatsoftwareisreadilyavailableandgeteasyaccesstoit
• wellestablishedsolutiononHPCclusters:environmentmodules
• traditionalenvironmentmodulestoolisoutdated,nolongeractivelymaintained...
Usecase:providingeasyaccesstosoftware
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 10
amodernenvironmentmodulestool
https://www.tacc.utexas.edu/research-development/tacc-projects/lmod
willbecoveredindetaillater...
Lmod
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 11
• EasyBuild:getting(scientific)softwareinstalled
• Lmod:providingeasyaccesstosoftwareinstallations
• Singularity
• ClusterShell
• XALT&OGRT
• OpenHPC
Birds-eyeviewoftools&projects
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 12
• switchingtoHPCclusterusuallyrequiresuserstochangetheirworkflow
• differentenvironmenttoworkin(env.modules,etc.)
• requiredtoolsaremissingorinstalleddifferently
• maybealsodifferentversions,configuration,etc.
• confusingandtime-consumingforusers
• reproducingtheenvironmenttheyareusedtomayrequirealotofeffort
• usersmaybeunabletotakecareofitthemselves(lackpermissions&technicalexpertise,dependencyhell,...)
• supportteamsarebusy,maylackdomainknowledge
Usecase:sameworkflowonlocal&HPCsystems
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 13
• http://singularity.lbl.gov/-http://singularity.lbl.gov/user-guide
• containersolutionforHPCenvironments
• provides"mobilityofcompute":containerimagescanbeusedonany(Linux)system
• enablesreproduciblescience:containerimagesincludeeverythingthatisrequired
• dealswithsecurityconcernsofothercontainersolutionslikeDocker
• goodsupportforaccelerators(GPUs,...),MPI,Infiniband,etc.
Singularity
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 14
• developedbyHPCteamatLawrenceBerkeleyNationalLaboratory(LBNL)
• projectlead:GregoryM.Kurtzer(alsoknownfromCentOSLinux,Warewulf)
• firstpublicreleaseinApril2016,massiveuptakesince
• latestrelease:Singularityv2.2.1(Feb14th2017)
• downloadfrom:https://github.com/singularityware/singularity/releases
• tobeincludedinOpenHPCv1.3.1
Singularity
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute
Singularity ArchitectureSingularity on the other hand is not designed around the idea of micro-service process isolation. Therefore it uses the smallest number of namespaces necessary to achieve its primary design goals.
This gives Singularity a much lighter footprint, greater performance potential and easier integration than container platforms that are designed for full isolation.
15
Singularityarchitecture
• similartoothercontainersolutions
• limiteduseofnamespaces
• loweroverheadthanDocker&co
• easytointegrate
(figurecourtesyofGregoryM.Kurtzer(LBNL)
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 16
• regularusersmustnotbeabletogainadminprivilegesinanyway...
• sensitiveissuebecauseofsharedfilesystems&lotsofresources
• noprivilegeescalationallowedinSingularity
• callinguserismaintainedwithinthecontainer
• ifyouwanttoberootinsidecontainer,youmustberootoutsidethecontainer
• norootowneddaemonprocess(asopposedtoDocker)
• containersarestartedasuserprocesses
SecurityconcernswithcontainersonHPCsystems
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 17
• containerimagesaresinglefiles
• easytodistribute&share
• veryefficientonparallelfilesystems
• self-containedcontainerimages:"itjustworks"
• containerimagescanbeusedonanyLinuxsystem
• BringYourOwnEnvironment(BYOE)
• userisfamiliarwithenvironment
encapsulatedincontainer
Mobilityofcompute,reproduciblescience&BYOESingularity: Bootstrapping the container$ sudo singularity bootstrap /tmp/Centos-7.img centos.def
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 18
• Singularityisdesignedfromthegrounduptointegrateeasilywithavailableresources
• containersarelaunchedviaresourcemanager,justlikeregularjobs
• nochangesrequiredtosystemconfiguration,itjustworks
• accesstoaccelerators(GPU,XeonPhi,...)iseasytocomeby
• someeffortisrequiredtouseMPIandleverageinterconnectslikeInfiniband
• requiresnecessarylibrariestobeavailableinthecontainerimage...
• sharedfilesystemscanbeboundintothecontainerthroughSingularityconfiguration
• supportsmultiplearchitectures(x86_64,POWER,AARCH64,...)
Integrationwithavailableresources
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute
Singularity: Workflow./container.img
Kurtzer GM, Sochat V, Bauer MW. Singularity: Scientific Containers for Mobility of Compute (under review)
19
Singularityworkflow
adminprivilegesare(only)requiredtocreate
containerimages (andtoinstallSingularity)
(figurecourtesyofGregoryM.Kurtzer(LBNL)
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 20
• https://singularity-hub.org/
• createdbyVanessaSochat(Stanford'sResearchComputinggroup)
• platformtobuild&archiveSingularitycontainerimages
• designedtofacilitatereproducibleresearch,publications&peer-review
• integratedwithGitHubtotriggerbuildsandautomatecontainerisation
• containerimageshostedonSingularityHubcanbeuseddirectly:
singularity run shub://singularityhub/$USER/$CONTAINER:$TAG
SingularityHub
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 21
• benchmarkingexperimentsshowverylowoverheadcomparedtonativeexecution
• loweroverheadcomparedtoDocker&co
• sometimeseven(slightly)betterperformancethannative!(notescaleonY-axis,diff.issmall)
Singularity:performance
Containerized MPI Latency comparison over IB
● OpenMPI 2.0.1 with OSU Micro Benchmarks 5.3.2 http://mvapich.cse.ohio-state.edu/benchmarks/● Running over 40Gb/s Infiniband with Singularity v2.2
Note: Centos-7.2 uses libibverbs 1.1.8 and Centos-7.3 is 1.2.1, libmlx4 is (1.0.6 vs 1.2.1), among other OS differences which may account for the differences
Data provided by: Paulo Souza <[email protected]>
dataprovidedbyPauloSouza([email protected]),viaGregoryM.Kurtzer
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 22
• http://singularity.lbl.gov-https://github.com/singularityware/singularity
• presentations:
• FOSDEM'17(byMichaelBauer,UniversityofMichigan)
• https://fosdem.org/2017/schedule/event/singularity
• https://fosdem.org/2017/schedule/event/singularityhpc
• HPCACStanfordconference'17(byGregoryM.Kurtzer)
• http://insidehpc.com/2017/02/video-singularity-containers-science-reproducibility-hpc
• https://www.nextplatform.com/2017/04/10/singularity-containers-hpc-reproducibility-mobility
Singularity:moreinformation
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 23
• EasyBuild:getting(scientific)softwareinstalled
• Lmod:providingeasyaccesstosoftwareinstallations
• Singularity:containersforHPC
• ClusterShell
• XALT&OGRT
• OpenHPC
Birds-eyeviewoftools&projects
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 24
• HPCsystemadministratorsoftenneedtoruncommandsonremotesystems
• forexample:workernodesinanHPCcluster,loginnodes,managernodes,etc.
• poorsolution:for-looptorunacommandviaSSHoneachsystem
• veryslowforlargenumberofsystems(eventhoughitisembarrassinglyparallel...)
• hardtocomparecommandoutputs
• specifyinglistoftargetnodesmaybetricky(e.g.,excludingnodesthataredown)
• supportforrunningremotecommandsviaacustomPythonscriptwouldbeuseful
• pdsh:noPythonintegration,nointegratedCLI(pdsh,dshbak,pdcp),scalingissues,...
Usecase:runningcommandsonremotesystems
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 25
• website:http://cea-hpc.github.io/clustershell/
• documentation:http://clustershell.readthedocs.io/
• toolforrunningshellcommandsonremotesystems,inparallelandscalable
• supportformanagingnodesets,merging/diffingcommandoutputs,...
• canalsobeusedasaPythonlibrary(import ClusterShell)
• (originally)developedbyCEAHPCcenter(France)
• latestrelease:ClusterShellv1.7.3(Dec20th2016)
• downloadfrom:https://github.com/cea-hpc/clustershell/releases
ClusterShell
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 26
• toolforremoteexecutionofshellcommands,compatiblewithpdsh
• supportsSSH,RSH+customconnectionmodesusingPythonplugins
• targetnodesarespecifiedvianodesets(-w)orpredefined/dynamicnodegroups(-g)
• supportscopyingoffiles(inparallel,bothways)via-c/--copyand--rcopy
• smartdisplayofcommandresultsusing-b/-B/-Loptions
• integratedoutputgathering,sortingbynode,nodesetornodegroups
• degreeofparallelismcanbecontrolledvia-f/--fanout
• supportsconfigurable'treemode'forbetterscalability
ClusterShell:clushcommand
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 27
ClusterShell:nodesetandclubakcommands
• nodeset:toolformanipulationofnodesets,nodegroups
• expanding(-e),folding(-f),counting(-c)ofnodenames/sets/groups
• nodegroups:YAMLfiles,orviaexternalsources(e.g.SLURM'ssinfocommand)
• outputformatcanbetweakedaccordingtoneeds(-O)
• supportsnodesetarithmetic:joiningsets,excludingelements,etc.
• clubak:utilitytosummariseoutputandhighlightdifferences
• backwardscompatiblewithdshbakutilitythatisavailablewithpdsh
• leveragedbyclushwhen-b/-B/-Lisused
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 28
• nodesetmanipulation:
• commandexecution:
ClusterShellasPythonlibrary
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 29
ClusterShell:demo
# example usage of nodeset command$ nodeset -e 'node24[01-5]'node2401 node2402 node2403 node2404 node2405# predefined group of 200 workernodes in golett cluster$ nodeset -c @golett200
# output for 'module --version' on all golett workernodes (10 nodes in parallel)$ clush -B -f 10 -w @golett module --version---------------node[2400-2599] (200)---------------
Modules based on Lua: Version 6.6 2016-10-13 13:28 -05:00 by Robert McLay [email protected]
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 30
ClusterShell:moreinformation
• website:http://cea-hpc.github.io/clustershell/
• documentation:http://clustershell.readthedocs.io/
• wiki:https://github.com/cea-hpc/clustershell/wiki
• support:
• mailinglist:https://lists.sourceforge.net/lists/listinfo/clustershell-devel
• issuetracker:https://github.com/cea-hpc/clustershell/issues
• article:http://slashdot.org/journal/266980/A-Quick-Look-on-ClusterShell
• presentation:https://archive.fosdem.org/2016/schedule/event/hpc_bigdata_clustershell/
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 31
• EasyBuild:getting(scientific)softwareinstalled
• Lmod:providingeasyaccesstosoftwareinstallations
• Singularity:containersforHPC
• ClusterShell:executingcommandsonremotesystems
• XALT&OGRT
• OpenHPC
Birds-eyeviewoftools&projects
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 32
• knowingwhich(scientific)softwareisbeingusedisimportant
• whichscientificsoftwareapplicationsconsumemostofavailablecomputetime?
• providedoptimisedinstallationsvsavailableviaOSvsuser-installed
• areprovidedsoftwareinstallationsactuallyusedlong-term?
• areoldsoftwareinstallationsstillbeingused?
• justcheckingwhichmodulesarebeingloadedisnotreliable
• loadingmodulesdoesnotimplythesoftwaretheyprovidewillactuallybeused
• whatifmultiplemodulesareloadedinjobs?
• usagetrackingshouldbeverylight-weightandshouldnevercauseanyproblems...
Usecase:trackingsoftwareusageonHPCsystems
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 33
• https://github.com/Fahey-McLay/xalt-http://xalt.readthedocs.io
• lateststablerelease:XALTv1.1.2(Jan4th2017),XALTv2isunderactivedevelopment
• tooltotrackwhichprograms/librariesusersareusingonHPCsystems
• successorofALTD(http://www.nersc.gov/users/software/programming-libraries/altd/)
• XALTv2isimplementedinC/C++,developedbyRobertMcLay(TACC)
• light-weight,transparent&robust
• hijackslinker(ld)tofingerprintexecutablessotheycanbetrackedindetail
• runtimetrackingbyXALTtriggeredviainit/finiinELFheaderofbinaries
XALT
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 34
• https://github.com/georg-rath/ogrt
• latestrelease:OGRTv0.4.1(Sept28th2016)
• (another)tooltotrackwhichprograms/librariesusersareusingonHPCsystems
• implementedinC/Go,byGeorgRath(freelancer)
• light-weight(trackingallin-memory)andtransparent,resistanttooutsideinfluence
• noruntimedependencies,easytodeploy
• onlyworkswithdynamicbinaries,relieson$LD_PRELOAD
OGRT
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 35
OGRTv0.4.1
• implementedinC/Go
• backends:SPLUNK,ElasticSearch
• heritage:trackmanysmalljobs/executables
• real-timedatacollection,senttoOGRTserver(viaUDP)
• trackingvia$LD_PRELOAD
• tracksallMPIranks
• usedat:IMBA,<privatecompany>
OGRTvsXALTside-by-sidecomparisonXALTv1.6.1-devel
(pre-v2)
• implementedinC/C++
• backends:database,file,syslog
• heritage:tracklargeMPIjobs(alsotracksnon-MPI)
• datatofile/syslog&digestednightlytoDB,canalsododirect-to-DBinreal-time
• trackingviaELFinit/finior$LD_PRELOAD• onlytracksMPIrank0(ignoresotherranks)
• usedat:TACC,NICS,Univ.Florida,KAUST,...
• commercialsupportbyEllexus
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 36
• quotebyRobertMcLay:"BigDatasucks!"
• easytocollectalotofdataonsoftwarebeingusedinjobs
• bydefaultallexecutablesaretracked,incl.standardtoolslikels,cd,cp,etc.
• alsocapturesalldefinedenvironmentvariables(typically~250perevent)
• ~17KBofdataperevent,easily>1GB/dayintotal...
• filteringisrequiredtofocusoneventsofinterest:
• onlyexecutables&environmentvariablesthatyoucareabout
• onlyonhoststhatreallymatter
• extracareshouldbetakenwithMPIjobs(e.g.,XALTonlytracksrank0)
(Big)DatacollectionwithXALT/OGRT
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 37
Collecteddataopenupwholerangeofpotentialusecases:
• targetedoutreach
• findoutwhichusersareusingoldsoftwareversions,buggylibraries,etc.
• verifyinghowsoftwareisbeingused
• dojobsreallyusetherequestedresources(cores,memory,...)?
• issoftwarebeingruninthecorrectenvironment(e.g.,usedMPIlibraries)?
• detailedstatisticsonwhichsoftwareisconsumingmostofavailablecomputetime
• wheretofocusoptimisationefforts(e.g.,awkusecasewithOGRT)
• canhelpwithtargetedbenchmarkingfornewsystems
UsecasesfordatacollectedwithXALT/OGRT
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 38
ExampleanalysisofdatacollectedwithXALT
top20executables onDarter(NICS) October'14-June'15
source: XALTpaper@HUST'15
https://doi.org/10.1145/2834996.2835000
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 39
XALT/OGRT:moreinformation
• GitHub:https://github.com/Fahey-McLay/xalt-https://github.com/georg-rath/ogrt
• documentation:http://xalt.readthedocs.io/en/latest-https://github.com/georg-rath/ogrt
• XALTpresentations
• FOSDEM'16:https://archive.fosdem.org/2016/schedule/event/hpc_bigdata_xalt/
• EUM'17:http://hpcugent.github.io/easybuild/files/EUM17/20170208-5_XALT.pdf
• OGRTpresentations:
• EUM'16:http://goo.gl/zbvChr-http://youtu.be/3l0eJq0nrOU
• FOSDEM'17:https://fosdem.org/2017/schedule/event/ogrt/
• jointBirds-of-a-FeathersessionatISC'17onXALT&OGRT
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 40
• EasyBuild:getting(scientific)softwareinstalled
• Lmod:providingeasyaccesstosoftwareinstallations
• Singularity:containersforHPC
• ClusterShell:executingcommandsonremotesystems
• XALT&OGRT:trackwhichsoftwareisbeingused
• OpenHPC
Birds-eyeviewoftools&projects
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 41
• referencecollectionofopen-sourceHPCsoftwarecomponents
• includesSLURM,Warewulf,Lmod,EasyBuild,ClusterShell,etc.
(soonalsoSingularity)
• supportsbothCentOS&SLES,x86_64andAARCH64(currentlyintechpreview)
• pre-builtpackagesforcompilers,MPI,libraries&toolsforHPC,etc.
• alsoincludesbestpractices,guidelinesforconfiguration,etc.
• intendstolowerthebarriertoentryforHPC
• initialreleaseatSC'15,latestreleasev1.3.0(March'17)
OpenHPC
OpenHPC Community BOF
Karl W. Schulz, IntelDavid Brayford, Leibniz Supercomputing Centre
November 16 ! Salt Lake City, Utah
http://openhpc.community
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 42
• website:http://openhpc.community
• GitHubrepository:https://github.com/openhpc/ohpc
• presentationatFOSDEM'16(Feb'16) OpenHPC:CommunityBuildingBlocksforHPCSystemshttps://fosdem.org/2016/schedule/event/hpc_bigdata_openhpc/
• paperatSystemProfessionalsWorkshop(Nov'16) ClusterComputingwithOpenHPC https://github.com/openhpc/ohpc/files/619162/HPCSYSPROS16_OpenHPC.pdf
OpenHPC:moreinformation
OpenHPC Community BOF
Karl W. Schulz, IntelDavid Brayford, Leibniz Supercomputing Centre
November 16 ! Salt Lake City, Utah
http://openhpc.community
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 43
• GettingScientificSoftwareInstalledwithEasyBuild&Lmod
• Context&motivationfortools&automation
• IntroductiontoEasyBuild&Lmod
• Installation&configuration
• Usage&workflow
• Advancedtopics
Outline
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 44
• resolvingproblemsthatoccurwhenusingtheHPCsystem(s)
- loginproblems,crashingjobs,thingsthat"stoppedworking",...
• answeringquestions,fromsimpletoverytechnical
• installing(scientific)softwaretools/libraries/applications
• helpingusersimprovetheirworkflow(notnecessarilybyrequest)
• training:Linuxbasics,OpenMP,MPI,Python,etc.
• performanceanalysisandoptimisationoflargescientificapplications
• consultancyservicesw.r.t.developingscientificsoftware
TasksforHPCusersupportteams
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 45
InstallingscientificsoftwareforusersinHPCsystems
• byuserrequest:newsoftware,versionupdates,morevariants,...
• usuallyona(sharedNFS)filesystemavailableoneveryworkernode
• specificallytargetedtotheHPCclusteritwillbeusedon
• builtfromsource(ifpossible)
• separateinstallationper(typeof)cluster
• highlyoptimisedforsystemarchitecture
• rebuildwhenupdatesforcompilers/librariesbecomeavailable
• installationstypicallyremainavailableduringlifetimeofsystem
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 46
"Pleaseinstall<software>ontheHPC?"Themostcommontypeofsupportrequestfromusersistoinstall(scientific)software.
• over25%ofsupportticketsatHPC-UGent
• consumes(way)morethan25%oftimeofHPC-UGentsupportteam
Installing(lotsof)scientificsoftwareis:
• error-prone,trial-and-error
• tedious,hardtogetright
• repetitive&boring(well...)
• time-consuming(hours,days,evenweeks)
• frustrating(e.g.,dependencyhell)
• sometimessimplynotworththeeffort...
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 47
CommonissueswithscientificsoftwareResearchersfocusonthesciencebehindthesoftwaretheyimplement,andcarelittleabouttools,buildprocedure,portability,...
Scientistsarenotsoftwaredevelopersorsysadmins(norshouldtheybe).
“Ifwewouldknowwhatwearedoing,itwouldnotbecalled‘research’.”
Thisresultsin:
• useofnon-standardbuildtools(orbrokenones)
• incompletebuildprocedure,e.g.,noconfigureorinstallstep
• interactiveinstallationscripts
• hardcodedparameters(compilers,libraries,paths,...)
• poor/outdated/missing/incorrectdocumentation
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 48
Primeexample:WRFWeatherResearchandForecastingModel(http://www.wrf-model.org)
• dozendependencies:netCDF(C,Fortran),HDF5,tcsh,JasPer,...
• knownissuesinlastreleaseare(only)documentedonwebsite
• nopatchfileprovided,infrequentbugfixreleases
• interactive'configure'script,optionschangebetweenversions
• resultingconfigurationfile'configure.wrf'needswork:fixhardcodedsettings(compilers,libraries,...),tweakingofoptions
• custom‘compile’script(wrapsaround‘make’) buildinginparallelisbrokenwithoutfixingtheMakefile
• noactualinstallationstep
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 49
Houston,wehaveaproblem...
InstallationofscientificsoftwareisatremendousproblemforHPCsitesallaroundtheworld.
• hugeburdenonHPCusersupportteams
• researchersloselotsoftime(waiting)
• sitestypicallyresorttocrudein-housescripting
• verylittlecollaborationamongHPCsites
http://geekand
poke.ty
pepad.com/geekand
poke/2010/05/how
-to-be
come-invaluable.htm
l
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 50
Whataboutexistingsoftwareinstallationtools?ExistingtoolsarenotwellsuitedtoscientificsoftwareandHPCsystems.
• packagemanagers:yum(RPMs),apt-get(.deb),...
• Homebrew(MacOSX),http://brew.sh/;Linuxbrew,http://brew.sh/linuxbrew/
• Portage(Gentoo),http://wiki.gentoo.org/wiki/Project:Portage
• pkgsrc(NetBSD&(alot)more),http://pkgsrc.org/
• Nix,http://nixos.org/nix;GNUGuix,https://www.gnu.org/s/guix
Commonproblems:
• usuallypoorsupportforold/multipleversionsand/ortohavebuildsside-by-side
• notflexibleenoughtodealwithidiosyncrasiesofscientificsoftware
• littlesupportforscientificsoftware,non-GCCcompilers,MPI
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 51
• website:http://hpcugent.github.io/easybuild/
• documentation:http://easybuild.readthedocs.io
• frameworktobuild&installscientificsoftwareonHPCclusters
• implementedinPython2,leaddevelopmentbyHPC-UGent
• supportsdifferentcompilers&MPIlibraries,>1,150differentsoftwarepackages,...
• active&helpfulworldwidecommunity
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 52
• (in-housedevelopmentatHPC-UGentsince2009)
• firstpublicreleaseinApril2012(EasyBuildv0.5)
• initialintentionwastogetfeedback,butacommunityemergedratherquickly...
• firststablereleaseinNovember2012(EasyBuildv1.0)
• frequentstablereleasessincethen(bothfeatureandbugfix)
• latest:EasyBuildv3.1.2(March20th2017)
• developmentishighlycommunity-driven(bugreports,featurerequests,contributions)
5yearsofEasyBuild
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 53
Featurehighlights(1)
• fullyautonomouslybuildingandinstalling(scientific)software
- automaticdependencyresolution
- automaticgenerationofmodulefiles(TclorLuasyntax)
• thoroughloggingofexecutedbuild/installprocedure
• archivingofbuildspecifications
• highlyconfigurable,viaconfigfiles/environment/commandline
• dynamicallyextendablewithadditionaleasyblocks,toolchains,etc.
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 54
Featurehighlights(2)
• supportforcustommodulenamingschemes(incl.hierarchical)
• transparencyviasupportfor'dryrun'installation
• comprehensivelytested:lotsofunittests,regressiontesting,...
• activelydeveloped,frequentstablereleases
• collaborationbetweenvariousHPCsites
• worldwidecommunity
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 55
WhatEasyBuildis(not)
EasyBuildis:
• notYABT(YetAnotherBuildTool);itdoesnotreplacetoolslikemake,itwrapsaroundthem
• notareplacementforyourfavouritepackagemanager
• notamagicsolutiontoallyour(softwareinstallation)problems...
• auniforminterfacethatwrapsaroundsoftwarebuildprocedures
• ahugetime-saver,byautomatingtedious/boring/repetitivetasks
• awaytoprovideaconsistentsoftwarestacktoyourusers
• anexpertsystemforsoftwareinstallationonHPCsystems
• aplatformforcollaborationwithHPCsitesworld-wide
• awaytoempoweruserstoself-managetheirsoftwarestackonHPCsystems
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 56
Supportedsoftware
• EasyBuildv3.1.2supportsinstallingover1,150softwaretools&applications
• includingCP2K,NAMD,NWChem,OpenFOAM,QuantumESPRESSO,WRF,WPS,...
• inaddition:hundredsofRlibraries,Pythonpackages,Perlmodules,...
• diversetoolchainsupport:
• compilers:GCC,Intel,Clang,PGI,IBMXL,Cray
• MPIlibraries:OpenMPI,IntelMPI,MPICH,MPICH2,MVAPICH2,...
• BLAS/LAPACKlibraries:IntelMKL,OpenBLAS,ATLAS,ACML,CrayLibSci,...
http://easybuild.readthedocs.io/en/latest/version-specific/Supported_software.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 57
Terminology
• EasyBuildframework
- coreofEasyBuild:Pythonmodules&packages
- providessupportingfunctionalityforbuildingandinstallingsoftware
• easyblock
- aPythonmodulethatservesasabuildscript,'plugin'fortheEasyBuildframework
- implementsa(generic)softwarebuild/installprocedure
• easyconfigfile(*.eb):buildspecification;softwarename/version,compilertoolchain,etc.
• (compiler)toolchain:setofcompilerswithaccompanyinglibraries(MPI,BLAS/LAPACK,...)
• extensions:additionallibraries/packages/modulesforaparticularapplications(e.g.,Python,R)
http://easybuild.readthedocs.io/en/latest/Concepts_and_Terminology.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 58
InnumbersincludedinEasyBuildv3.1.2:
• about50kLoCofPython(incl.framework,easyblocks,andlogging/optionparsingsupport)
• about15kLoCmoreforunit/integrationtestsuites
• 1,182supportedsoftwareapplications(excl.extensions,softwareversions)
• 25differenttoolchaindefinitions(excl.toolchainversions)
• 181software-specificeasyblocks,30genericeasyblocks
• 6,718easyconfigfiles
• differentsoftwareversions,variants,usingdifferenttoolchains,...
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 59
Providing(easy)accesstosoftwareinstallations
• lotsof(scientific)softwareistypicallyinstalledonHPCclusters• potentiallywiderangeofscientificdomains
• multipledifferentversions,builds,variants,...
• installedwithdifferentgenerationsofcompilers/libraries
• usuallyinanon-standardlocation(sharedfilesystem)
• environmentneedstobemodifiedtousethesesoftwareinstallations
• $PATH,$LD_LIBRARY_PATH,etc.+customsoftware-specificenvironmentvariables
• howcanweeasilyletusersaccessthesesoftwareinstallations?
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 60
Environmentmodules• wellestablishedsolutioninHPCcommunity
• shell-agnostic'scripts'tosetupenvironment
• intuitiveuserinterfaceviamodulecommand
• usedonmostHPCsystems(>80%1),sincemid90's
• Tcl-basedenvironmentmodulessystemisubiquitous,but:
• lastreleaseofTcl/Cimplementation(v3.2.10)wasDec2012
• pureTclversionisactivelymaintained,but(alot)lessused
• lacksfeaturessupportedbymorerecentimplementations...
(1)http://hpcugent.github.io/easybuild/files/SC15_BoF_Getting_Scientific_Software_Installed.pdf
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 61
Environmentmodules:userinterface
• checkingwhichmodulesareavailableisdoneusingmodule avail
• toupdatetheenvironmentforaparticularsoftwareinstallation,
amoduleshouldbeloadedusingmodule load
• anoverviewofcurrentlyloadedmodulescanbeobtainedwithmodule list
• reversingthechangestotheenvironmentcanbedonewithmodule unload
• resettingtheenvironmentbyunloadingallloadedmodulescanbedone
usingmodule purge
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute
$ which gcc
/usr/bin/gcc
$ module avail GCC
--------------------- /apps/modules/all ---------------------
GCC/4.7.4 GCC/4.8.5 GCC/4.9.4 GCC/5.1.0 GCC/5.2.0 GCC/5.3.0
$ module load GCC/5.3.0
$ which gcc
/apps/software/GCC/5.3.0/bin/gcc
62
Environmentmodules:userinterface
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 63
• website:https://www.tacc.utexas.edu/research-development/tacc-projects/lmod
• documentation:http://lmod.readthedocs.io/
• latest:Lmodv7.4.4(April17th2017)-https://github.com/TACC/Lmod/releases
• implementedinLua,leaddevelopmentbydr.RobertMcLay(TACC)
• mostlydrop-inreplacementforTcl-basedenvironmentmodulestool('Tmod')
• activelymaintained,(very)frequentupdates,vibrant&helpfulcommunity
• slowlytakingoverthe(HPC)world...
Lmod:amodernenvironmentmodulestool
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 64
Lmod:featurehighlights(1)• samemoduleuserinterfaceasTmod
• consumesmodulefileswrittenineitherTclorLua,mixingofbothissupported
• modulefilesarecachedtomakeuserinterfacemoreresponsive
• Lmod'sspidercachemustbekeptup-to-date!(cronjob)
• alsoprovidesmlshorthandcommand('moudle'is(too)easytomistype)
• mlisthesameasmodule list• ml fooisthesameasmodule load foo• ml avisthesameasmodule available• ml -fooisthesameasmodule unload foo
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 65
Lmod:featurehighlights(2)
• someminorfunctionaldifferenceswithTmod:
• onenamerule:onlyonesinglemodulecanbeloadedforaparticularsoftware
• canalsobeenforcedinTmodvia'conflict'statementsinmodulefiles
• Lmodautomaticallyunloadspreviouslyloadedmoduleonnameclash(bydefault)
• correctorderingofsoftwareversions(5.6isolderthan5.10)
• propersupportforhierarchicalmodules
• cansearchthroughallexistingmodulesvia'module spider',eveniftheyarenot(yet)availableforloading
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 66
Lmod:featurehighlights(3)• supportforadditionalfeatures,including:
• definingmodulefamilies(e.g.MPI),onlyonemodulecanbeloadedperfamily
• moduleproperties,whichareshowninoutputof'module avail'
• stickymodules,whichdonotgetunloadedon'module purge'(unlessforced)
• user-definedmodulecollectionsvia'module save'and'modulerestore'
• deprecatingmodules,usercangetacustommessagewhenamoduleisloaded
• hookstocustomiseLmod'sbehaviour,e.g.,totrackwhichmodulesgetloaded
• easydebuggingofLmoditselfvia'-D'optiontomodule/mlcommands
• ...
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 67
Communityaspects• EasyBuild&Lmodcommunitieshavebeengrowingrapidlythelastcoupleofyears
• hundredsofHPCsitesworldwide,largeandsmall
• verywelcoming&supportivetonewcomers
• significantoverlapbetweenEasyBuild&Lmodcommunities
• activemailinglists:
• EasyBuild:https://lists.ugent.be/wws/info/easybuild
• Lmod:https://sourceforge.net/p/lmod/mailman/lmod-users/
• EasyBuildalsohasactiveIRCandSlackchannels
• usersalsostartcontributing:bugreports,featurerequests,codecontributions,...
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 68
EasyBuildmailinglist(subscribers)
(2013) +25
(2014) +30
(2015) +60
(2016) +55
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 69
EasyBuildmailinglist(traffic)
0
200
400
600
800
1000
1200
2012 2013 2014 2015 2016 2017
#messages/year
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute
0
20
40
60
80
100
120
140
Jan-12 Jul-12 Jan-13 Jul-13 Jan-14 Jul-14 Jan-15 Jul-15 Jan-16 Jul-16 Jan-17 Jul-17
#un
ique
contrib
utors
framework
easyblocks
easyconfigs
70
CodecontributionstoEasyBuild
(2013) +20
(2014) +20
(2015) +30
(2016) +40
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 71
• GettingScientificSoftwareInstalledwithEasyBuild&Lmod
• Context&motivationfortools&automation
• IntroductiontoEasyBuild&Lmod
• Installation&configuration
• Usage&workflow
• Advancedtopics
Outline
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 72
EasyBuildrequirements
• focusisonLinuxx86_64HPCsystems
• RedHat-based(CentOS,SL,...),alsoonDebianandSuSEderivatives
• also(kindof)worksonmacOS,butnotatargetplatform
• basicWindowssupportvianewWindowsSubsystemforLinux(WSL)
• ongoingeffortstoenhancesupportonAARCH64(ARM)andPowerLinux
• Python2.6ormorerecentPython2,incl.setuptools(noPython3supportyet)
• systemC/C++compiler(GCC),usedtokickstartinstallationoftoolchain
• anenvironmentmodulestool(Lmodishighlyrecommended!)
http://easybuild.readthedocs.io/en/latest/Installation.html#requirements
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 73
InstallingEasyBuild
• installingPythonpackagescanbemessy,cfr.plethoraofPythoninstallationtools
• bootstrapscriptforEasyBuildwascreatedoutoffrustration
• standardinstallationtoolslikeeasy_install,pip,etc.worktoo
• packagingeffortsunderway(seeOpenHPC;WIPinFedora)
# download bootstrap script from # https://raw.github.com/hpcugent/easybuild-framework/develop/easybuild/scripts/bootstrap_eb.py$ python bootstrap eb.py $PREFIX $ module use $PREFIX/modules/all $ module load EasyBuild
http://easybuild.readthedocs.io/en/latest/Installation.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 74
UpdatingEasyBuild
• newreleasesarefullybackwards-compatible(exceptfornewmajorversions)
• toupdateEasyBuild:
• re-bootstraptoobtainamoduleforlatestEasyBuildrelease
• oruse'eb --install-latest-eb-release'toinstallmoduleforlatestrelease
• installaspecificEasyBuildversionwithEasyBuildusinganeasyconfigfilehttps://github.com/hpcugent/easybuild-easyconfigs/tree/develop/easybuild/easyconfigs/e/EasyBuild
http://easybuild.readthedocs.io/en/latest/Installation.html#updating-an-existing-easybuild-installation
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 75
ConfiguringEasyBuild(1)
YoushouldconfigureEasyBuildtoyourpreferences,via:
• configurationfile(s):key-valuelines,textfiles(e.g.,prefix=/tmp)
• environmentvariables(e.g.,$EASYBUILD_PREFIXsetto/tmp)
• commandlineparameters(e.g., --prefix=/tmp)
Eachconfigurationoptioncanbesetoneachoftheseconfigurationlevels.
Priorityamongdifferentoptions:cmdline,envvars,configfile.Forexample:
• --prefixoverrules$EASYBUILD_PREFIX
• $EASYBUILD_PREFIXoverrulesprefixinconfigurationfile
http://easybuild.readthedocs.io/en/latest/Configuration.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 76
ConfiguringEasyBuild(2)
Bydefault,EasyBuildwill(ab)use$HOME/.local/easybuild. Use'eb --show-config'togetanoverviewofthecurrentconfiguration.
http://easybuild.readthedocs.io/en/latest/Configuration.html
$ EASYBUILD_PREFIX=/tmp eb --buildpath /dev/shm --show-config## Current EasyBuild configuration# (C: command line argument, D: default value, E: environment variable, F: configuration file)#buildpath (C) = /dev/shminstallpath (E) = /tmppackagepath (E) = /tmp/packagesprefix (E) = /tmprepositorypath (E) = /tmp/ebfiles_reporobot-paths (D) = /Users/kehoste/work/easybuild-easyconfigs/easybuild/easyconfigssourcepath (E) = /tmp/sources
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 77
InstallingLmod
• requirements:
• Lua5.1ornewer,incl.luaposixandluafilesystemlibraries
• luajsonandluatermarealsorecommended(butnotstrictlyrequired)
• Lmodinstallationprocedureisstandardconfigure-make-make install
• profilescriptsareincluded,requiredtodefinemodule/mlcommandsonlogin
• typicallyinstalledsystem-wideasanOSpackage,withsite-specificconfiguration
• .specfilefor(our)Lmodisavailableathttps://github.com/hpcugent/Lmod-UGent
http://lmod.readthedocs.io/en/latest/030_installing.htmlhttps://github.com/TACC/Lmod/blob/master/INSTALL
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 78
ConfiguringLmod(1)
• defaultLmodbehaviourcanbeconfiguredatbuildtimeusingconfigureoptions
• canalsobetweakedatruntimeusing$LMOD_*environmentvariables
• someaspectsareconfiguredvialmodrc.luafile
• locationofLmodspidercache,knownmoduleproperties,...
• hookscanbeimplementedinSitePackage.luafile
• activeLmodconfigurationcanbequeriedvia'module --config'
http://lmod.readthedocs.io/en/latest/155_lmodrc.html
http://lmod.readthedocs.io/en/latest/170_hooks.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 79
ConfiguringLmod(2)
• somenoteworthyconfigurationoptions:
• --with-redirect=yes:redirectmessagestostdout(otherthanwarnings/errors)
• --with-disableNameAutoSwap=yes:produceanerrorratherthan automaticallyunloadingmoduleonnameclash(onenamerule)
• --with-shortTime=<N>:withlargeN,avoidbuildingofuserspidercachefile
• --with-pinVersions=yes:alwayspinmoduleversionsincollections
• --with-cachedLoads=yes:alsousespidercachefor'module load'
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 80
Lmod:spidercache
• makingLmoduseanup-to-datesystemspidercachefileisveryimportant
• requiredforresponsive'module'/'ml'
• locationofsystemspidercachecanbecustomised:
• viaconfigureoptions:--with-spiderCacheDir,...
• viaactivelmodrc.luaconfigurationfile(seescDescriptTentry)
• verifyconfigurationvia'ml --config'
• useupdate_lmod_system_cache_filestoupdatesystemspidercacheviacron
• avoidbuildingofuserspidercachebyspecifyinglargevalueto--with-shortTime
http://lmod.readthedocs.io/en/latest/130_spider_cache.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 81
• GettingScientificSoftwareInstalledwithEasyBuild&Lmod
• Context&motivationfortools&automation
• IntroductiontoEasyBuild&Lmod
• Installation&configuration
• Usage&workflow
• Advancedtopics
Outline
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 82
Basicusage
• specifysoftwarename/versionandtoolchainto'eb'command
• commonlyviaeasyconfigfilename(s):
eb GCC-4.9.2.eb Clang-3.6.0-GCC-4.9.2.eb
• checkwhetherrequiredtoolchain&dependenciesareavailableusing--dry-run/-D:
eb Python-2.7.13-intel-2017a.eb -D
• enabledependencyresolutionvia--robot/-r:
eb WRF-3.8.0-intel-2016b-dmpar.eb -dr
http://easybuild.readthedocs.io/en/latest/Using_the_EasyBuild_command_line.html
http://easybuild.readthedocs.io/en/latest/Typical_workflow_example_with_WRF.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute
83
Step-wiseinstallationprocedure
IV: unpack sources
V: apply patches
VI: prepare
VII: configure
VIII: build
IX: test
X: install
XI: extensions
XII: sanity check
XIII: cleanup
XIV: env. module
III: check readiness XV: permissions
II: fetch sources XVI: packaging
I: parse easyconfig XVII: test cases
EasyBuildperformsastep-wiseinstallationprocedureforeachsoftware
• downloadsources(besteffort)
• setupbuilddirectory&environment
• unpacksources(&applypatches)
• loadmodulesfortoolchain&deps
• definetoolchain-relatedenvvars($CC,$CFLAGS,...)
• configure,build,(test),install,(extensions)
• performasimplesanitycheckoninstallation
• generatemodulefile
eachstepcanbecustomisedviaeasyconfigparametersoraneasyblock
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 84
Exampleoutputofusing'eb'command$ eb GCC-4.9.3.eb == temporary log file in case of crash /tmp/eb-GyvPHx/easybuild-U1TkEI.log == processing EasyBuild easyconfig GCC-4.9.3.eb == building and installing GCC/4.9.3... == fetching files... == creating build dir, resetting environment... == unpacking... == patching... == preparing... == configuring... == building... == testing... == installing... == taking care of extensions... == postprocessing... == sanity checking... == cleaning up... == creating module... == permissions... == packaging... == COMPLETED: Installation ended successfully == Results of the build can be found in the log file /opt/easybuild/software/GCC/4... == Build succeeded for 1 out of 1 == Temporary log file(s) /tmp/eb-GyvPHx/easybuild-U1TkEI.log* have been removed. == Temporary directory /tmp/eb-GyvPHx has been removed.
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 85
Easyconfigfilesasbuildspecifications
• simpletextfilesincludingasetofeasyconfigparameters(inPythonsyntax)
• somearemandatory:softwarename/version,toolchain,metadata(homepage,descr.)
• othercommonlyusedparameters:
• easyblocktouse
• listofsources&patches
• (build)dependencies
• optionsforconfigure/build/installcommands
• filesanddirectoriesthatshouldbepresent(sanitycheck)
http://easybuild.readthedocs.io/en/latest/Writing_easyconfig_files.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 86
Exampleeasyconfigfile
name = 'WRF'version = '3.8.0'
buildtype = 'dmpar' # custom parameter for WRFversionsuffix = '-' + buildtype # part of module name
homepage = 'http://www.wrf-model.org'description = "Weather Research and Forecasting (WRF) Model"
toolchain = {'name': 'intel', 'version': '2016b'}
source_urls = ['http://www.mmm.ucar.edu/wrf/src/']sources = ['%(name)sV%(version_major_minor)s.TAR.gz']patches = ['WRF-%(version)s_known_problems.patch']
builddependencies = [('tcsh', '6.20.00')]dependencies = [ ('JasPer', '2.0.10'), ('netCDF', '4.4.1'), ('netCDF-Fortran', '4.4.4'),]
softwarenameandversion
buildvariant(specifictoWRF)('dmpar':distributed,MPI)
softwaremetadata
sources&patches
toolchainname&version
listof(build)dependenciesnote:allversionsarefixed!
noeasyblockspecified,whichimpliesusingasoftware-specificeasyblock(EB_WRF)
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 87
• GettingScientificSoftwareInstalledwithEasyBuild&Lmod
• Context&motivationfortools&automation
• IntroductiontoEasyBuild&Lmod
• Installation&configuration
• Typicalusage&workflow
• Advancedtopics
Outline
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 88
Logfiles
• EasyBuildthoroughlylogsexecutedinstallationprocedure
• activeEasyBuildconfiguration
• easyconfigfilethatwasused
• modulesthatwereloaded+resultingchangestoenvironment
• definedenvironmentvariables
• outputofexecutedcommands
• informativelogmessagesproducedbyeasyblock
• logisfileiscopiedtosoftwareinstallationdirectoryforfuturereference
• canbeusedtodebugbuildproblemsorseehowinstallationwasperformedexactly
http://easybuild.readthedocs.io/en/latest/Logfiles.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 89
Logfiles:example== 2016-04-24 13:34:31,906 main.EB_HPL INFO This is EasyBuild 3.1.2 (framework: 3.1.2, easyblocks: 3.1.2) on host example....== 2016-04-24 13:34:35,503 main.EB_HPL INFO configuring...== 2016-04-24 13:34:48,817 main.EB_HPL INFO Starting configure step...== 2016-04-24 13:34:48,823 main.EB_HPL INFO Running method configure_step part of step configure...== 2016-04-24 13:34:48,823 main.run DEBUG run_cmd: running cmd /bin/bash make_generic (in /tmp/user/easybuild_build/HPL/2.0/goolf-1.4.10/hpl-2.0/setup)== 2016-04-24 13:34:48,823 main.run DEBUG run_cmd: Command output will be logged to /tmp/easybuild-W85p4r/easybuild-run_cmd-XoJwMY.log== 2016-04-24 13:34:48,849 main.run INFO cmd "/bin/bash make_generic" exited with exitcode 0 and output:...
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 90
Addingsupportforadditionalsoftware
• foreachsoftwareinstallation,aneasyconfigfileisrequired
• defineseasyconfigparametersthatspecifytoEasyBuildwhattoinstall,andhow
• existingeasyconfigfilescanserveasexamples
• forversionortoolchainupdates,atweakedeasyconfigcanbegeneratedviaeb --try-*
• seehttp://easybuild.readthedocs.io/en/latest/Writing_easyconfig_files.html
• for'standard'installationprocedures,agenericeasyblockcanbeused
• genericinstallationcanbecontrolledwhereneededviaeasyconfigparameters
• forcustominstallationprocedures,asoftware-specificeasyblockismustbeimplemented
• seehttp://easybuild.readthedocs.io/en/latest/Implementing-easyblocks.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 91
Commongenericeasyblocks
• ConfigureMake
standard'./configure'-'make'-'make install'installationprocedure
• CMakeMake
sameasConfigureMake,butusingCMakeforconfiguring
• PythonPackage
installingPythonpackages('pythonsetup.pyinstall','pip install',...)
• MakeCp no(standard)configurationstep,buildwith'make',installbycopyingbinaries/libraries
• Tarball:justunpacksourcesandcopyeverythingtoinstallationdirectory
• Binary:runbinaryinstaller(specifiedvia'install_cmd'easyconfigparameter)
http://easybuild.readthedocs.io/en/latest/version-specific/generic_easyblocks.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 92
Easyconfigfilesvseasyblocks
• thinlinebetweenusingcustomeasyblockand'fat'easyconfigwithgenericeasyblock
• customeasyblocksare"doonceandforget",centralsolutiontobuildpeculiarities
• reasonstoconsiderimplementingasoftware-specificeasyblockinclude:
• 'critical'valuesforeasyconfigparametersrequiredtomakeinstallationsucceed
• toolchain-specificaspectsofthebuildandinstallationprocedure(e.g.,configureoptions)
• interactivecommandsthatneedtoberun
• custom(configure)optionsfordependencies
• havingtocreateoradjustspecific(configuration)files
• 'hackish'usageofagenericeasyblock
http://easybuild.readthedocs.io/en/latest/Implementing-easyblocks.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 93
Commontoolchains
• intelandfoss1toolchainsaremostcommonlyusedinEasyBuildcommunity
• helpstofocuseffortsofHPCsitesusingoneorbothofthesetoolchains
• updatedtwiceayear,clearversioningscheme(2016b,2017a,...)
• latestversion:
• foss/2017a binutils2.27,GCC6.3.0,OpenMPI2.0.2,OpenBLAS0.2.19,LAPACK3.7.0,FFTW3.3.6
• intel/2017a binutils2.27+GCC6.3.0asbaseversion2017.1.132ofIntelcompilers,IntelMPIandIntelMKL
http://easybuild.readthedocs.io/en/latest/Common-toolchains.html
(1)FOSS:FreeandOpenSourceSoftware
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 94
Transparencyofperformedinstallprocedure
• 'eb --extended-dry-run','eb -x'revealsplannedinstallationprocedure
• runsinamatterofseconds
• showscommandsthatwillbeexecuted,buildenvironment,generatedmodulefile,...
• anyerrorsthatoccurinusedeasyblockareignoredbutclearlyreported
• not100%accuratesinceeasyblockmayrequirecertainfilestobepresent,etc.
• veryusefulwhendebuggingeasyblocks,instantfeedbackasafirstpass
• implementationmotivatedbyrequestsfromthecommunity
• helpstoavoidimpressionthatEasyBuildisamagicblackboxforinstallingsoftware
http://easybuild.readthedocs.io/en/latest/Extended_dry_run.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 95
$ eb WRF-3.8.0-intel-2016b-dmpar.eb -x
== temporary log file in case of crash /tmp/eb-Dh1wOp/easybuild-0bu9u9.log
== processing EasyBuild easyconfig /home/example/eb/easybuild-easyconfigs/easybuild/easyconfigs/w/WRF/WRF-3.8.0-intel-2016b-dmpar.eb
...
*** DRY RUN using 'EB_WRF' easyblock (easybuild.easyblocks.wrf @ /home/example/eb/easybuild-easyblocks/easybuild/easyblocks/w/wrf.py) ***
== building and installing WRF/3.8.0-intel-2016b-dmpar...fetching files... [DRY RUN]
[fetch_step method]Available download URLs for sources/patches: * http://www2.mmm.ucar.edu/wrf/src//$source * http://www.mmm.ucar.edu/wrf/src//$source
List of sources: * WRFV3.8.0.TAR.gz will be downloaded to /home/example/eb/sources/w/WRF/WRFV3.8.0.TAR.gz...
Exampleoutputof--extended-dry-run(1)
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 96
...
building... [DRY RUN]
[build_step method] running command "tcsh ./compile -j 4 wrf" (in /home/example/eb/software/WRF/3.8.0-intel-2016b-dmpar/WRF-3.8.0) running command "tcsh ./compile -j 4 em_real" (in /home/example/eb/software/WRF/3.8.0-intel-2016b-dmpar/WRF-3.8.0) running command "tcsh ./compile -j 4 em_b_wave" (in /home/example/eb/software/WRF/3.8.0-intel-2016b-dmpar/WRF-3.8.0)...
[sanity_check_step method]Sanity check paths - file ['files'] * WRFV3/main/libwrflib.a * WRFV3/main/real.exe * WRFV3/main/wrf.exeSanity check paths - (non-empty) directory ['dirs'] * WRFV3/main * WRFV3/runSanity check commands (none)
Exampleoutputof--extended-dry-run(2)
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 97
...
[make_module_step method]Generating module file /home/example/eb/modules/all/WRF/3.8.0-intel-2016b-dmpar, with contents:
#%Module proc ModulesHelp { } { puts stderr { The Weather Research and Forecasting (WRF) Model } } module-whatis {Description: WRF - Homepage: http://www.wrf-model.org}
set root /home/example/eb/software/WRF/3.8.0-intel-2016b-dmpar
conflict WRF
if { ![ is-loaded intel/2016b ] } { module load intel/2016b }
if { ![ is-loaded JasPer/1.900.1-intel-2016b ] } { module load JasPer/1.900.1-intel-2016b }
if { ![ is-loaded netCDF/4.4.1-intel-2016b ] } {
Exampleoutputof--extended-dry-run(3)
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 98
Usingacustommodulenamingscheme• acoupleofdifferentmodulenamingschemesareincludedinEasyBuild
• see--avail-module-naming-schemes
• specifyactivemodulenamingschemevia--module-naming-scheme
• default:EasyBuildMNS(<name>/<version>-<toolchain>-<versionsuffix>)
• youcanimplementyourownmodulenamingschemerelativelyeasily
• specifyhowtocomposemodulenameusingprovidedmetadata
• viaPythonmodulethatdefinescustomderivativeclassofModuleNamingScheme
• makeEasyBuildawareofitvia--include-module-naming-schemes
• decouplenamingofinstalldirsvsmodulesvia--fixed-installdir-naming-scheme
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 99
Flatvshierarchicalmodulenamingscheme• flatmodulenamingscheme:
• allexistingmodulesarealwaysavailableforloading
• downsides:
• long(confusing)modulenamestodistinguishbuildwithdifferenttoolchains
• userscaneasilyshootthemselvesinthefootby(trying)toloadincompatiblemodules
• hierarchicalmodulenamingscheme:
• modulesareorganisedinatree-likefashion
• a'core'modulemustbeloadedfirsttomakemore(compatible)modulesavailable
• typicallyusedwithdifferenthierarchylevels:Core-Compiler-MPI
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute
Hierarchicalmodulenamingschemeindetail
• initially,onlyCoremodulesareavailableforloading
• othermodulesareonlyvisiblevia'module spider'
OpenMPI/2.1.0OpenMPI/2.1.0OpenMPI/1.10.6
GCC/5.3.0 GCC/6.1.0
OpenMPI/1.10.6
FFTW/3.3.4 FFTW/3.3.5 FFTW/3.3.6 FFTW/3.3.4 FFTW/3.3.5 FFTW/3.3.6
(loaded)
(available)
(notavailable)
Corelevel
Compilerlevel (compiler-dependent)
MPIlevel (MPI-dependent)
legend
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute
Hierarchicalmodulenamingschemeindetail
• Coremodulesmayextend$MODULEPATHwithanadditionallocation
• loadingaCoremodulemaymakemoremodulesavailable
• inthisexample,loadingaGCCmodulemakesOpenMPImodulesavailable
OpenMPI/2.1.0OpenMPI/2.1.0OpenMPI/1.10.6
GCC/5.3.0 GCC/6.1.0
OpenMPI/1.10.6
FFTW/3.3.4 FFTW/3.3.5 FFTW/3.3.6 FFTW/3.3.4 FFTW/3.3.5 FFTW/3.3.6
(loaded)
(available)
(notavailable)
Corelevel
Compilerlevel (compiler-dependent)
MPIlevel (MPI-dependent)
legend
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute
Hierarchicalmodulenamingschemeindetail
OpenMPI/2.1.0OpenMPI/2.1.0OpenMPI/1.10.6
GCC/5.3.0 GCC/6.1.0
OpenMPI/1.10.6
FFTW/3.3.4 FFTW/3.3.5 FFTW/3.3.6 FFTW/3.3.4 FFTW/3.3.5 FFTW/3.3.6
(loaded)
(available)
(notavailable)
Corelevel
Compilerlevel (compiler-dependent)
MPIlevel (MPI-dependent)
legend
• evenmoremodulesmaybemadeavailablebyloadingothermodules
• forexample,loadinganOpenMPImodulesrevealsMPI-dependentmodules
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 103
IntegrationwithGitHub
• accesstoGitHubstraightfromEasyBuildcommandline(!)
• 'eb --from-pr'topullineasyconfigsfromaspecificpullrequest
• --upload-test-reporttoaddcommentwithtestreportintoapullrequest
• 'eb --new-pr'tocreateanewpullrequestforcontributing(changesto)easyconfigs
• 'eb --update-pr'toaddupdatestoexistingpullrequests
• significantlylowersthebarforcontributing,noneedtobefamiliarwithgit
• alsoturnsouttobeahugetimesaverforexperiencedcontributors
• fornowonlysupportedforeasyconfigfiles,soonalsoforeasyblocks&framework
http://easybuild.readthedocs.io/en/latest/Integration_with_GitHub.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 104
Otherfeaturesthatwerenotcovered...
• lettingusersmanagetheirsoftwarestackontopofcentrallyprovidedmodules
• installinghiddenmodules,hidingcertaindependencies&toolchains
• supportforusingRPATHlinking
• partialinstallations:only(re)generatemodulefile,installadditionalextensions
• dynamicallyextendingEasyBuildvia--include-*
• submittinginstallationsasjobstoanHPCclustervia--job
• creatingpackages(RPMs,...)forsoftwareinstallationsdonewithEasyBuild
• usingEasyBuildonCraysystems,integrationwithCrayProgrammingEnvironment
http://easybuild.readthedocs.io
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 105
ContributingtoEasyBuild&Lmod
BothEasyBuildandLmodhaveimprovedsignificantlythankstotheircommunities.
Youtoocancontributeback,by:
• sendingfeedback
• reportingbugs
• joiningthediscussion(mailinglists,EasyBuildconfcalls)
• sharingsuggestionsandideasforchanges&additionalfeatures
• contributingeasyconfigs,enhancingeasyblocks,addingsupportfornewsoftware...
• extending&enhancingdocumentation
http://easybuild.readthedocs.io/en/latest/Contributing.html
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 106
vs
• Spack(http://spack.io/)isapackagemanagerfocusedonscientificsoftwareandHPC
• leaddevelopmentbyToddGamblin(LLNL)
• significantgrowthinuserbaseinrecentmonths
• similaryetdifferentapproachcomparedEasyBuild
• veryflexiblecommandlineinterfacew.r.t.specifyingwhattoinstall
• concretisespartialspecification,dependencyversionsdon'tneedtobefixed
• focusismoreonsoftwaredevelopers
• someexperiencewithGit&Pythonisassumed/required
ModernScientificSoftwareManagementusingEasyBuild&co-PRACE-VI-SEEM2017SpringSchoolKennethHoste(HPC-UGent)April25th2017-TheCyprusInstitute 107
PapersonEasyBuild(&Lmod)
ModernScientificSoftwareManagementUsingEasyBuildandLmod
MarkusGeimer(JSC),KennethHoste(HPC-UGent),RobertMcLay(TACC)
http://hpcugent.github.io/easybuild/files/hust14_paper.pdf
MakingScientificSoftwareInstallationReproducibleOnCraySystemsUsingEasyBuild
PetarForai(IMP),GuilhermePeretti-Pezzi(CSCS),KennethHoste(HPC-UGent)
https://cug.org/proceedings/cug2016_proceedings/includes/files/pap145.pdf
ScientificSoftwareManagementinRealLife:DeploymentofEasyBuildonaLargeScaleSystem
DamianAlvarez,AlanO’Cais,MarkusGeimer(JSC),KennethHoste(HPC-UGent)
http://hpcugent.github.io/easybuild/files/eb-jsc-hust16.pdf
ModernScientificSoftwareManagementusingEasyBuild&co
PRACE-VI-SEEM2017SpringSchool-SystemAdministrationTrackApril25th2017-TheCyprusInstitute
[email protected] http://hpcugent.github.io/easybuild/
http://www.ugent.be/hpc https://www.vscentrum.be