COMPILING, BUILDING, AND INSTALLING PROGRAMS ON
THE CLUSTER
BUILDING COMPUTER PROGRAMS
• Theprocessofconver-ngahuman-readablefiletoamachine-readablefile.
#include <stdio.h> int main() { prin9(“Hello World!\n”); return 0; }
01010101000010101010010101110101010010100000010111010100101100011101101010101010101010101010001010101010111111001011110111000011101010101001111010101111010101010000110101010101
Cprogram(simpletextfilewri9eninCprogramminglanguage)
Binaryexecutablefile(asetofCPUinstruc-onsencodedin0’sand1’s)
Sophis-catedprograms(e.g.acompiler)areusedtoperformthismul--stepconversion.
HelloWorld!
THE BUILD PROCESS
Notalllanguagesarecompiledlanguages!Theprocesstothe
leIappliestoprogramswri9eninC,C++,andFortran.
Linker
Assembler
Compiler
Preprocessor
Expandedsourcecode
Sourcecode(e.g.Cprogram)
Assemblycode
Objectcode
Binaryexecutable
Externallibraries
Human-readableHigher-levellanguage
Lower-levellanguage
Machine-readable(i.e.canbeexecutedbyCPU)
PREPROCESSOR
• Expandsorremovesspeciallinesofcodepriortocompila-on.
.
. #include <stdio.h> . .
.
. #define PI 3.1415 . .
.
. #ifndef FOO_H #define FOO_H #include “myHeader.h” void myFunc(int); #endif . .
Includestatements: Definestatements: Headerguards:
• Copiescontentsofstdio.hintofile.
• ReplacesallinstancesofPIwithinfilewith3.1415.
• Preventsexpandingmul-plecopiesofthesameheaderfilebydefiningaunique“macro”foreachheaderfile.
InC,preprocessordirec-vesbeginwiththe#symbolandare
NOTconsideredCcode.
COMPILER
• Convertsexpandedsourcecodetoassemblycode.
#include <stdio.h> int main() { prin9(“Hello World!\n”); return 0; }
.
. main: .cfi_startproc pushq %rbp .cfi_def_cfa_offset 16 movq %rsp %rbp . .
• Assembly-levelinstruc-onsarespecifictoaprocessor’sInstruc-onSetArchitecture(ISA).
• ExampleISAsarex86,x86_64,andARM.MostmachinesinHPCtodaysupportx86_64.
Portabilityisanissuewithcompiledlanguagessinceassemblylanguagecontains
instruc-onsthatarespecifictoaCPU’sarchitecture.
ASSEMBLER AND LINKER
• Assembler:convertsassemblycodetoobjectcode.
• Linker:s-chestogetherallobjectfiles(includinganyexternallibraries)intothefinalbinaryexecutablefile.
• Objectcodeisinabinaryformatbutcannotbeexecutedbyacomputer’sOS.• ExternallibrariesareoIendistributedassharedobjectfilesthatareobjectcode.
• Hidesspecificimplementa-onsincethesefilesarenothumanreadable.• Noneedtoberecompiledforeachapplica-onthatusesthelibrary.• Storedefficientlyinbinaryformat.
• Manyapplica-onsoIencontainmul-plesourcefiles,eachofwhichneedtobeincludedinthefinalexecutablebinary.
• Thejobofthelinkeristocombinealltheseobjectfilestogetherintoafinalexecutablebinary(a.k.a.“executable”or“binary”)thatcanberun.
Executable
ObjectFile1
ObjectFile2
ExtLibA
USING COMPILERS ON THE CLUSTER (1/3)
IMPORTANTNOTE:Inprac-ce,thestepsperformedbythepreprocessor,compiler,assembler,andlinkeraregenerallyobscuredfromtheuserintoasinglestepusing(inLinux)asinglecommand.Inthenextseveralslides,wewillrefertothissinglecommandasacompiler,butnotethatwe’reactuallytalkingaboutatoolthatisapreprocessor+compiler+assembler+linker.
• GCC:GNUCompilerCollec<on• Freeandopensource• MostwidelyusedsetofcompilersinLinux• Ccompiler:gcc• C++compiler:g++• Fortrancompiler:gfortran
• IntelCompilerSuite• Licensedandclosedsource,butACCRE
purchasesalicense• OIenproducesfasterbinariesthanGCC• Occasionallymoredifficulttobuildcode
duetolackofcommunitytes-ng• Ccompiler:icc• C++compiler:icpc• Fortrancompiler:ifort
USING COMPILERS ON THE CLUSTER (2/3)
gcchello.c
• BuildsCprogramwiththeGCCCcompiler.• Producesabinarycalleda.outthatcanberunbytyping./a.out
gcc–ohellohello.c
• Producesabinarycalledhellothatcanberunbytyping./hello
Errormessagesresultwhenthebuildprocessfails.Thecompilershouldprovide
detailsaboutwhythebuildfailed.
Warningmessagesoccurwhenaprogram’ssyntaxisnot100%cleartothecompiler,butitmakesanassump-on
andcon-nuesthebuildprocess.
USING COMPILERS ON THE CLUSTER (3/3)
gcc–ohello-Wallhello.c
• -Wallwillshowallwarningmessages
gcc–ohello-ghello.c
• -gwillbuildthebinarywithdebugsymbols
gcc–Ehello.c
• Showexpandedsourcecode
gcc–Shello.c
• Createassemblyfilecalledhello.s
gcc–chello.c
• Createobjectfilecalledhello.o
gcc–ohello–O3hello.c
• -O3willbuildthebinarywithlevel3op-miza-ons• Levels0to3(mostaggressive)available• Canleadtofasterexecu-on-mes• Defaultis–O0inGCCand–O2inIntelsuite
Vectorizedloopexecu-onisenabledwith–O3forGCCand–O2forIntel.
icc–ohello–xHosthello.c• UseIntel’sCcompilertoaggressivelyop-mizeforthespecificCPU
microarchitecture
Usingthe–xHostop-onleadstopoorbinaryportability.Onlyusethisop-onifyouaresurethebinarywillalwaysbeexecutedonaspecificprocessortype.
EXTERNAL LIBRARIES (1/2)
• Sta<callyLinkedLibrary:namingconven-on:liblibraryname.a(e.g.libcurl.aisasta-ccurllibrary)
• DynamicallyLinkedLibrary:namingconven-on:liblibraryname.so(e.g.libcurl.soisadynamiccurllibrary)
• Linkercopiesalllibraryrou-nesintothefinalexecutable.• Requiresmorememoryanddiskspacethandynamiclinking.• Moreportablebecausethelibrarydoesnotneedtobeavailableatrun-me.
• Onlythenameofthelibrarycopiedintothefinalexecutable,notanyactualcode.• Atrun-me,theexecutablesearchestheLD_LIBRARY_PATHandstandardpathforthelibrary.• Requireslessmemoryanddiskspace;mul-plebinariescansharethesamedynamicallylinkedlibraryatonce.• Bydefault,alinkerlooksforadynamiclibraryratherthanasta-cone.
• DoNOTneedtospecifytheloca<onofalibraryatbuild<meifit’sinastandardloca<on(/lib64,/usr/lib64,/lib,/usr/lib).Forexample,libc.solivesin/lib64.
EXTERNAL LIBRARIES (2/2)
• Linkingtolibrariesinnon-standardloca<onsrequiresthefollowinginforma<onatbuild-<me:
• Nameoflibrary(specifiedwith–llibrarynameflag)• Loca-onoflibrary(specifiedwith–L/path/to/non/standard/loca-on/lib)• Loca-onofheaderfiles(specifiedwith–I/path/to/non/standard/loca-on/include)
gcc–L/usr/local/gsl/latest/x86_64/gcc46/nonet/lib–I/usr/local/gsl/latest/x86_64/gcc46/nonet/include–lgsl–lgslcblasbessel.c–Wall–O3–ocalc_bessel
• Inthisexample,twolibraries(gslandgslcblas)arelinkedtothefinalexecutable.• Alterna-vely,useLIBRARY_PATHandC_INCLUDE_PATHtospecifyloca-onsoflibrariesandheaders.
• ChecktheLD_LIBRARY_PATHandoutputofthelddcommandbeforerunningtheprogram:
• LD_LIBRARY_PATHshowslistofdirectoriesthatlinkersearchesfordynamicallylinkedlibraries• Runldd./my_progtoseethedynamicallylinkedlibrariesneededbyanexecutableandthecurrentpath
toeachlibrary
PORTABILITY
Supportforspecificvectoriza-onextensionsisalsorequiredfor
portability.Forexample,youcannotbuildaprogramwithAVX2onplaoormAandrunitonplaoormBifAVX2isnot
supportedbyplaoormB!
CanIbuildanexecutableoncomputerAandrunitoncomputerB?
• CPUinstruc-onsetarchitecture(e.g.x86_64)• Opera-ngsystem• Externallibraries
Itdepends!Arethepla/ormsthesame?
Pla/orm
• ThisiswhyyouoIenseedifferentinstallersfordifferentopera-ngsystems–theinstallerissimplycopyingapre-builtbinarytoyourmachine!
• DifferentCPUarchitecturesarepresentonthecluster,sobesuretocompilewithoutoverlyaggressiveop-miza-onsorspecifythetargetCPUarchitecture/familyinyourSLURMscript(e.g.#SBATCH--constrain=haswell)
OTHER COMPILER FUN FACTS
• Manydifferentcompilersexistbutnotallcompilersarecreatedequal!
• Performanceofprogramcanbeverycompiler-dependent!
• GCC,Intel,AbsoI,PortlandGroup(PGI),MicrosoIVisualStudio(MSVS),tonameafew.• Somearefree,othersarenot!• Itisnotunusual(especiallywithlargeprojects)forcompilerAtobuildaprogramwhilecompilerBfails.• Errormessagesandlevelsofverbositycanalsovarywidely.
• Thisisespeciallytrueinscien-ficandhigh-performancecompu-nginvolvingalotofnumericalprocessing.• Compilerop-miza-onsareespeciallytricky,some-mesthecompilerneedshelpfromtheprogrammer(e.g.
re-factoringcodesothecompilercanmakeeasier/saferdecisionsaboutwhentoop-mizecode).• Somecompilers(especiallyIntel’s)tendtooutperformtheircounterpartsbecausetheyhavemorein-mate/
nuancedinforma-onaboutaCPU’sarchitecture(whichareoIenIntel-based!).
AUTOMATING THE PROCESS: MAKEFILES (1/3)
• DefaultfileiscalledMakefileormakefile.• Allowsbuildprocesstobebrokenupintodiscreetsteps,ifdesired.Forexample,separaterulescanbe
definedfor(i)compiling+assembling,(ii)linking,(iii)tes-ng,and(iv)installingcode.• Makeanalyzesthe-mestampsofatargetandthattarget’sdependenciestodecidewhethertoexecutea
• TheMaketoolallowsaprogrammertodefinethedependenciesbetweensetsoffilesinprogrammingproject,andsetsofrulesforhowto(mosto_en)buildtheproject.
Automating the build process
• make)u2lity))– Provides)a)way)for)separate)compila2on))– Describe)the)dependencies)among)the)project)files))– Default)file)to)look)for)is)makefile-or)Makefile)
27)
compiler) assembler) linker)
.c
.h
.c
project1.c*
common.h*
project2.c*
.o project1.o*
.o project2.o*
executable*Bydefiningdependencies,youcan
avoidunnecessarilyrebuildingcertainfiles.Forexample,intheexampleontheright,project2.cdoesnotneedtobere-compiledifchangeshavebeen
madetoproject1.c.
AUTOMATING THE PROCESS: MAKEFILES (2/3)
• Makeanalyzesthe<mestampofatarget’slastmodifica<onandcomparesittothatofthetarget’sdependenciestodecidewhethertoexecutethecommand(s)definedforthattarget’srule.
target: dependencies # rule <tab> command1 # shell command <tab> command2 # shell command . .
MakefileTemplate
• A“target”isalabel/iden-fierforarule• OIenthetargetiseitherthenameofa
fileoraconven-onalrule(e.g.“install”)• Dependenciesarefilesthatthetarget
dependon• Commandsmustbeprecededbyatab
executable: project1.o proect2.o gcc –o executable project1.o project2.o
project1.o: project1.c common.h
gcc –c project1.c # generates project1.o project2.o: roject2.c common.h
gcc –c project2.c # generates project2.o
ExampleMakefile(seepreviousslide)
• ThereareoIenmul-plerulesdefinedperMakefile
• Byjusttyping“make”,thefirstruleinthefilewillbeexecuted
AUTOMATING THE PROCESS: MAKEFILES (3/3)
make
• Generallybuildstheen-reproject.
makeinstall
• GenerallyinstallsthesoIware.
maketest
• Generallyrunsunittests.
makeclean
• Deletesintermediatebuildfilestostartthebuildprocessfromscratch.
“makeinstall”generallyfailswith“permissiondenied”errorsifyoudonothaveadministra-veprivilegesorhavenotconfiguredthebuildto
installintoalocaldirectory.
TolearnmoreaboutMakefiles,checkoutthefollowingtutorial:
h9ps://swcarpentry.github.io/make-novice/
• No-cethatMakeissmartenoughtonotrebuildtheprogramifnofileshavebeenmodifiedsinceourlastbuild.
• Makeisalsosmartenoughtoonlyre-compileproject2.cwhenithasbeenchangedbutproject1.chasnot.
AUTOMATING THE PROCESS: CONFIGURE SCRIPTS (1/2)
• AconfigurescriptisanexecutablefileresponsibleforbuildingaMakefileforaproject.
• Determiningthedependenciesonagivensystemisdifficulttopredictandsubjecttoconstantchange–wri-ngaMakefilebyhandforeachsystem(orevenasubsetofrepresenta-vesystems)wouldbeanenormouschallengeandanadministra-vehassle.
• Instead,aconfigurescriptcanbeusedtoscanasysteminsearchofalltheneededdependencies(includingversionsofsoIware,loca-onsofexternallibraries),andbuildaMakefilethatisspecifictothatsystem.
• Configurescriptsareindispensibleforlargeprojectsespeciallywherethenumberofdependenciesislargeanddifficulttomanage/track.
• Alterna-vestotheconfigurescriptexist(cmakebeingthemostcommon).
./configuremakemaketestmakeinstall
• BuildingprojectsonLinuxat-mesthissimple.• Runonlyifyouhaveadministra-verightsonsystem.
./configure-–prefix=/my/local/dirmakemaketestmakeinstall
• --prefixop-onneededifinstallinginhomedirectoryonthecluster.
AUTOMATING THE PROCESS: CONFIGURE SCRIPTS (2/2)
• Manyconfigurescriptssupportanumberofdifferentop<onsforconfiguringyourbuild.
./configure--help
• Showcommandlineop-ons.
MAKE AND CONFIGURE MACROS
CC• Ccompilercommand(e.g.gcc)
• Thereareanumberof“macros”(thinkofasvariables)thathavestandardmeaningsinMakeandconfigurescripts.Thesemacroscangenerallybeexportedasenvironmentvariablestocustomizeyourbuild.
CFLAGS• Ccompilerflags(e.g.–Wall–O3)
CPP• Cpreprocessorcommand(e.g.gcc)
CXX• C++compilercommand(e.g.g++)
CXXFLAGS• C++compilerflags(e.g.–Wall–O3)
LDFLAGS• Linkerflags(e.g.–L/path/to/lib)
LIBS• Librarynames(e.g.–lcurl)
FC• Fortrancompilercommand(e.g.gfortran)
FFLAGS• Fortrancompilerflags(e.g.–O3)
MPICC• MPICcompilerwrappercommand(e.g.mpicc)
COMPILED VS. INTERPRETED LANGUAGES
Whataboutinterpretedlanguages?
CompiledLanguage• Fasterexecu-on-me• Slowerdevelopment-me• Lessportable• C,C++,Fortran
InterpretedLanguage• Slowerexecu-on-me• Fasterdevelopment-me• Moreportable• Python,Matlab,R,Ruby,Julia
ThetradeoffslistedtotheleIarenotuniversallytruebutin
generalapply.
Manypopularmodules/packages(e.g.NumPy,SciPy)loadedfrominterpretedlanguagesarecompiledsharedobjectfilesandoffercomparableperformancetopurecompiledlanguages.