1
JaimeE.CombarizaAssociateResearchProfessorDepartmentofChemistryDirectorMARCCJohnsHopkinsUniversity
MARCC:
Update FY17
2017IDIESAnnualSymposium
Configura9on2017
2
21,120cores(FY17+2Kcores)822nodes36condo-nodes900condo-cores
20condo-nodes+ 4condo-GPU-nodes+ 24nodes+ 23,360+cores
1.1PFLOPs
1.2PFLOPs
MetricsFY17
3
•ResearchGroups:290•Accounts:1500•CPU-cyclesused:130Mcore-hours•U9liza9on80%•Publica9ons>100•Classes:10•Scien9ficApplica9ons>300•HelpTickets>5000
U9liza9onFY17
4
U9liza9onperDepartment
5
6
TheGrayresearchgroup(ChemE)predictsanddesignsbimolecularstructures.MARCCiscri9caltoourworkaseachcalcula9onrequirestheenumera9onofvastnumbersofcandidatestructuresandsequences.Ourworkencompassesbothmethoddevelopmentandapplica9ons.(i)Protein-proteindockingwithflexiblebackbones.(ii)Newcomputa9onalmodelsoflipidmembranes.(iii)Resis9n-an9bodyinterac9onsforpulmonaryhypertension.(iv)Dualac9oninhibitorsofHistoneDeacetylaseandDemethylase
JasonEisner(CS):MARCChasbeenagodsendtoourlab.Ithasenabledustodocomputa9onallyintensivework.We'reverygratefultoJHUandthestateofMaryland.Inmyfield,researchershavebeenleavingacademiaandgotoindustriallabslikeGoogle,Facebook,Amazon,etc.Onereasontheyohengiveisthattheywillhaveaccesstovastcomputeclustersandwillbefreetobuildthemodels/runtheexperimentsthattheyreallywant.Providingthiskindofreasonablylarge-scale,GPU-enabled,professionallyadministeredcomputeinfrastructurecanhelpuniversi9esstayintheresearchgameandretaintheirfaculty.Trayanova research group (BME):The availability ofMARCC has been transformative for researchersworking inmylaboratory.Therun-timeswehaveseenonthistrulyimpressivehighperformancecomputing(HPC)platformrepresentaconsiderablespeedupcomparedtothosewewereabletoobtainusingpreviousclusters.Moreover,researchersinmylabhavereportedimpressivelyshortqueuetimesforexecutingtypicaljobs,meaningthatthetotalcomputationalthroughputofourlaboratoryhasincreasedsubstantially.C.Bennet (Phys): Weareengaged in trying toanswersomeof thebiggestandmost fundamentalquestionsaboutouruniverse.Almosteverythingwedoincosmologyrequiressubstantialcomputingpower.MARCCisanindispensablefacilitythathasenabledustodoresearchthatwecouldnothavedonewithoutit.ItprovidesacompetitiveadvantagetoMarylandscientists.Weare contributing toNASA'splanning fora future spacemission (WFIRST)andwearemembersofaNASAteamcontributing to aEuropean spacemission (Euclid). Thisplanningworkhas involved simulationsof the sky tobeobserved and analysis of the simulated data to discern what can be deduced from the observations and thus how tooptimizemissionplanning. WehaveusedMARCCforboththesimulationsandtheanalysisinsupportofthesetwospacemissions.
ImpactonResearch
Whatisnew
7
•“data”“longtermstorage"•Containers•OpenOnDemand(OOD)•Powerbackup•HIPAA
“Data”storage
8
•$HOME:20GB(backedup)•“scratch”/“work”:1TBdefaultTEMPORARYfilesnobackup•“data”directorieshavebeencreatedperPI.Storedataforlonger9me•Sharedwithallmembersofthegroup•ZFS(lowperformance)•Bydefault1TBbutitcanbeextendedto10TBperrequest.•Itisbackedupatanoff-siteloca9on•PLEASEdonotcompute(I/O)to“data”thesystemwillslowdownsignificantly.
Whatisnew
9
•“data”•Containers•OpenOnDemand(OOD)•Powerbackup•HIPAA
Containers
10
•Singularity:•Mobilityandreproducibility•Enablestheusertohavefullcontroloftheenvironment.Norootaccessneeded,onMARCC•Usercanbuildanimageonhis/herlocalmachine,uploadittoMARCCandthenimportitusingsingularity•Buildtheimagebasedonapar9cularpipeline,OS,data,‘local’scien9ficsohware•Compa9blewithdocker(canimportdockercontainers)
Whatisnew
11
•“data”•Containers•OpenOnDemand(OOD)•Powerbackup•HIPAA
OpenOnDemand
12
•Applica9ontoconnecttoMARCCusinga“browser”•Connec9ontocomputenodes•Interac9vework,classes,Pythonnotebooks•Mustfollowauthen9ca9onprocess•Matlab,R-Studio,Jupyternotebooks•Anynon-GUIwork•Somevisualiza9onapplica9onscanbeadded•upload/downloadfiles,checkjobs,...
Examples
13
Tunneling
14
•jupyter_session•rstudio_session
Whatisnew
15
•“data”•Containers•OpenOnDemand(OOD)•Powerbackup•HIPAA
Powerbackup
16
•RequestsfromPIsthattheycannotusethecoloca9onspacebecausethereisnopowerbackup•Needtoprotect“cri9cal”componentsfrompoweroutages(Networkswitches,storage)•Capabilityto“workoncomponents”withouttotaldown9me•ProvideabeuermoresecureHPCenvironment
Powerbackup
17
•Addgeneratorpower(ongoing)•Design:June2017(completed)•Bids:July-August2017(completed)•Budget:August2017(completed)•Acquisi9on,landprepara9on,(ongoing)•Projectedcomple9ondateSummer2018
Whatisnew
18
•“data”•Containers•OpenOnDemand(OOD)•Powerbackup•HIPAA
MARCCSecureEnvironment
19
•Con9nua9onfrom2016talk.•SysteminplaceatMARCC(trustedci.org)•“Island”withintheBluecrabcluster•24nodes•RunningSLURM•ZFSfilesystemwithACLsatthefilelevel•Firewallatloginnode•Similarauthen9ca9onprocessasMARCC•Authoritytooperate(10/25/17)
MSE
20
•Process•AddMARCCasacollaboratoronyourproposal•CoordinatewithMARCCforresourcesneeded,security,otherandallaspects•Submitproposaltothe“datatrustgroup”•Requestanalloca9on•Requestaccounts•CoordinatewithMARCCtomovedata.Mustbeencrypted•HIPAATraining•Followprocessesandguidelines
ReturnonInvestment(ROI)
21
•ROI:MARCChasallowedmanyinves9gatorstomovetheirresearchforward
•Researcherscannowdotheresearchtheyreallywantasopposedtotheresearchtheycan
•over45Mingrants
Publica9ons
22
•hups://www.marcc.jhu.edu/news-and-events/publica9ons/
•Publica9ons(FY17)thatusedMARCCforcompu9ng