1

JaimeE.CombarizaAssociateResearchProfessorDepartmentofChemistryDirectorMARCCJohnsHopkinsUniversity

MARCC:

Update FY17

2017IDIESAnnualSymposium

Configura9on2017

2

21,120cores(FY17+2Kcores)822nodes36condo-nodes900condo-cores

20condo-nodes+ 4condo-GPU-nodes+ 24nodes+ 23,360+cores

1.1PFLOPs

1.2PFLOPs

MetricsFY17

3

•ResearchGroups:290•Accounts:1500•CPU-cyclesused:130Mcore-hours•U9liza9on80%•Publica9ons>100•Classes:10•Scien9ficApplica9ons>300•HelpTickets>5000

U9liza9onFY17

4

U9liza9onperDepartment

5

6

TheGrayresearchgroup(ChemE)predictsanddesignsbimolecularstructures.MARCCiscri9caltoourworkaseachcalcula9onrequirestheenumera9onofvastnumbersofcandidatestructuresandsequences.Ourworkencompassesbothmethoddevelopmentandapplica9ons.(i)Protein-proteindockingwithflexiblebackbones.(ii)Newcomputa9onalmodelsoflipidmembranes.(iii)Resis9n-an9bodyinterac9onsforpulmonaryhypertension.(iv)Dualac9oninhibitorsofHistoneDeacetylaseandDemethylase

JasonEisner(CS):MARCChasbeenagodsendtoourlab.Ithasenabledustodocomputa9onallyintensivework.We'reverygratefultoJHUandthestateofMaryland.Inmyfield,researchershavebeenleavingacademiaandgotoindustriallabslikeGoogle,Facebook,Amazon,etc.Onereasontheyohengiveisthattheywillhaveaccesstovastcomputeclustersandwillbefreetobuildthemodels/runtheexperimentsthattheyreallywant.Providingthiskindofreasonablylarge-scale,GPU-enabled,professionallyadministeredcomputeinfrastructurecanhelpuniversi9esstayintheresearchgameandretaintheirfaculty.Trayanova research group (BME):The availability ofMARCC has been transformative for researchersworking inmylaboratory.Therun-timeswehaveseenonthistrulyimpressivehighperformancecomputing(HPC)platformrepresentaconsiderablespeedupcomparedtothosewewereabletoobtainusingpreviousclusters.Moreover,researchersinmylabhavereportedimpressivelyshortqueuetimesforexecutingtypicaljobs,meaningthatthetotalcomputationalthroughputofourlaboratoryhasincreasedsubstantially.C.Bennet (Phys): Weareengaged in trying toanswersomeof thebiggestandmost fundamentalquestionsaboutouruniverse.Almosteverythingwedoincosmologyrequiressubstantialcomputingpower.MARCCisanindispensablefacilitythathasenabledustodoresearchthatwecouldnothavedonewithoutit.ItprovidesacompetitiveadvantagetoMarylandscientists.Weare contributing toNASA'splanning fora future spacemission (WFIRST)andwearemembersofaNASAteamcontributing to aEuropean spacemission (Euclid). Thisplanningworkhas involved simulationsof the sky tobeobserved and analysis of the simulated data to discern what can be deduced from the observations and thus how tooptimizemissionplanning. WehaveusedMARCCforboththesimulationsandtheanalysisinsupportofthesetwospacemissions.

ImpactonResearch

Whatisnew

7

•“data”“longtermstorage"•Containers•OpenOnDemand(OOD)•Powerbackup•HIPAA

“Data”storage

8

•$HOME:20GB(backedup)•“scratch”/“work”:1TBdefaultTEMPORARYfilesnobackup•“data”directorieshavebeencreatedperPI.Storedataforlonger9me•Sharedwithallmembersofthegroup•ZFS(lowperformance)•Bydefault1TBbutitcanbeextendedto10TBperrequest.•Itisbackedupatanoff-siteloca9on•PLEASEdonotcompute(I/O)to“data”thesystemwillslowdownsignificantly.

Whatisnew

9

•“data”•Containers•OpenOnDemand(OOD)•Powerbackup•HIPAA

Containers

10

•Singularity:•Mobilityandreproducibility•Enablestheusertohavefullcontroloftheenvironment.Norootaccessneeded,onMARCC•Usercanbuildanimageonhis/herlocalmachine,uploadittoMARCCandthenimportitusingsingularity•Buildtheimagebasedonapar9cularpipeline,OS,data,‘local’scien9ficsohware•Compa9blewithdocker(canimportdockercontainers)

Whatisnew

11

•“data”•Containers•OpenOnDemand(OOD)•Powerbackup•HIPAA

OpenOnDemand

12

•Applica9ontoconnecttoMARCCusinga“browser”•Connec9ontocomputenodes•Interac9vework,classes,Pythonnotebooks•Mustfollowauthen9ca9onprocess•Matlab,R-Studio,Jupyternotebooks•Anynon-GUIwork•Somevisualiza9onapplica9onscanbeadded•upload/downloadfiles,checkjobs,...

Examples

13

Tunneling

14

•jupyter_session•rstudio_session

Whatisnew

15

•“data”•Containers•OpenOnDemand(OOD)•Powerbackup•HIPAA

Powerbackup

16

•RequestsfromPIsthattheycannotusethecoloca9onspacebecausethereisnopowerbackup•Needtoprotect“cri9cal”componentsfrompoweroutages(Networkswitches,storage)•Capabilityto“workoncomponents”withouttotaldown9me•ProvideabeuermoresecureHPCenvironment

Powerbackup

17

•Addgeneratorpower(ongoing)•Design:June2017(completed)•Bids:July-August2017(completed)•Budget:August2017(completed)•Acquisi9on,landprepara9on,(ongoing)•Projectedcomple9ondateSummer2018

Whatisnew

18

•“data”•Containers•OpenOnDemand(OOD)•Powerbackup•HIPAA

MARCCSecureEnvironment

19

•Con9nua9onfrom2016talk.•SysteminplaceatMARCC(trustedci.org)•“Island”withintheBluecrabcluster•24nodes•RunningSLURM•ZFSfilesystemwithACLsatthefilelevel•Firewallatloginnode•Similarauthen9ca9onprocessasMARCC•Authoritytooperate(10/25/17)

MSE

20

•Process•AddMARCCasacollaboratoronyourproposal•CoordinatewithMARCCforresourcesneeded,security,otherandallaspects•Submitproposaltothe“datatrustgroup”•Requestanalloca9on•Requestaccounts•CoordinatewithMARCCtomovedata.Mustbeencrypted•HIPAATraining•Followprocessesandguidelines

ReturnonInvestment(ROI)

21

•ROI:MARCChasallowedmanyinves9gatorstomovetheirresearchforward

•Researcherscannowdotheresearchtheyreallywantasopposedtotheresearchtheycan

•over45Mingrants

Publica9ons

22

•hups://www.marcc.jhu.edu/news-and-events/publica9ons/

•Publica9ons(FY17)thatusedMARCCforcompu9ng

Thanks

23

•marcc-help@marcc.jhu.edu

•hup://marcc.jhu.edu