Home >Documents >Leveraging GPGPU Computing in Grid and Cloud Environments€¦ · Leveraging GPGPU Computing in...

Leveraging GPGPU Computing in Grid and Cloud Environments€¦ · Leveraging GPGPU Computing in...

Date post:07-Oct-2020
Category:
View:1 times
Download:0 times
Share this document with a friend
Transcript:
  • www.egi.eu EGI-Engage isco-fundedbytheHorizon2020FrameworkProgrammeoftheEuropeanUnionundergrantnumber654142

    [email protected]

    LeveragingGPGPUComputinginGridandCloudEnvironments:

    FirstResultsfromtheMobrain CC

    AlexandreBonvin

  • 24/8/16

    Mainactivities

    • Task1:Usersupportandtraining 9PM• Task2:Cryo-EMinthecloud:bringingcloudstothedata 23.4PM• Task3:Increasing thethroughputefficiencyofWeNMR 0PM

    portalsviaDIRAC4EGI• Task4:CloudVMsforstructuralbiology 0PM• Task5:GPUportalsforbiomolecularsimulations 14PM• Task6:Integratingthemicro(WeNMR/INSTRUCT) and

    macroscopic (NeuGRID4you)VRCs 11PM

    • TOTALfundedeffort 57.4PM

  • 34/8/16

    mobrain.egi.eu§ Bringsthemicro(WeNMR)andmacro

    (N4U)worldstogetherintoonecompetencecenterunderEGIEngage:

    § Withactivitiestoward:§ Integratingthecommunities§ Makingbestuseofcloudresources§ Bringingdatatothecloud(cryo-EM)§ ExploitingGPGPUresources

    § Whilemaintainingthequalityofourcurrentservices!

  • 44/8/16

    WeNMRVRC (December2015)

    • Oneofthelargest(#users)VOinlifesciences• >720VOregisteredusers(36%outsideEU)• >2250VRCmembers(>60%outsideEU)• ~41sitesfor>142000CPUcoresviaEGIinfrastructure• User-friendlyaccesstoGridviawebportals

    www.wenmr.eu

    NMRSAXS

    Aworldwidee-Infrastructure forNMRandstructuralbiology

  • 54/8/16

    SustainedgrowthoftheWeNMRVRC

    0

    250

    500

    750

    1000

    1250

    1500

    1750

    2000

    2250

    2011

    -01

    2011

    -03

    2011

    -05

    2011

    -07

    2011

    -09

    2011

    -11

    2012

    -01

    2012

    -03

    2012

    -05

    2012

    -07

    2012

    -09

    2012

    -11

    2013

    -01

    2013

    -03

    2013

    -05

    2013

    -07

    2013

    -09

    2013

    -11

    2014

    -01

    2014

    -03

    2014

    -05

    2014

    -07

    2014

    -09

    2014

    -11

    2015

    -01

    2015

    -03

    2015

    -05

    2015

    -07

    2015

    -09

    2015

    -11

    End of WeNMR

    EU funding

  • 64/8/16

    Sustained#ofjobs

    End of WeNMR

    EU funding

  • 74/8/16

    • SupportforMoBrain/West-Lifeactivites

    • 75MCPUhours,50TBstoragefrom7sites– INFN-PADOVA(Italy)– RAL-LCG2(UK)– TW-NCHC (Taiwan)

    – SURFsara (TheNetherlands)– NCG-INGRID-PT (Portugal)– NIKHEF(TheNetherlands)– CESNET-MetaCloud (Czech

    Republic)

    NewSLAagreement

  • 84/8/16

    z

    x

    y

    6D cross-correlation

    search

    Automatic rigid body fitting of biomolecular structures into cryo-EM densities

    Exhaustive 6D search of 3 translational and 3 rotational degrees of freedom

    Calculate cross-correlation at every scanned position

    Python package

    Simple command-line program, able to run using single/multiple CPUs or GPU

    ExploringGPGPUresources:PowerFit

    vanZundertandBonvin.AIMSBiophysics 2,73-87(2015)www.github.com/haddocking/powerfit

  • 94/8/16

    Fast Fourier Transform for fast translational

    scans

    GPU acceleration

    Optimised rotation sets

    Resampling and trimming target

    z

    x

    y

    Speedingupthesearch

  • 104/8/16

    Required input:• PDB file• Cryo-EM density• Resolution

    Optional:• Rotational sampling interval• Laplace filter• Core-weighted correlation• Number of solutions to file• Number of processors to use• Offload to GPU

    PowerFitP

    rotocol

  • 114/8/16

    ExploringGPGPUresources:DisVis

    • Pythonpackagetovisualizeandquantifytheaccessibleinteractionspaceofdistancerestrainedbinarybiomolecularcomplexes.

    • Simplecommand-lineprogram,abletorunusingsingle/multipleCPUsorGPU

    vanZundertand Bonvin.Bioinformatics.31,3222-3224(2015)www.github.com/haddocking/disvis

  • 124/8/16

    PerformanceofCPUvsGPUinhouseresources

    • CPU:AMDOpteron6344usingFFTW3• GPU:NVIDIAGeForceGTX680usingclFFT

    PowerFit

    DisVisSystem Number of

    complexessampled

    Time CPU TimeGPU

    Speedup

    RNA-polymerase II 19 × 109 19h44m 56m 21x

    PRE5-PUP2 7× 109 7h 12m 15m 29x

    System Mapsize(voxels)

    Rotationssampled

    TimeCPU

    TimeGPU

    Speedup

    GroEL-GroES 90x72x72 70728 1h29m 4m9s 21x

    RsgA into ribosome 72x80x72 70728 1h16m 4m2s 19x

  • 134/8/16

    PerformanceofCPUvsGPUinhouseresources

    PowerFit

    DisVis

  • 144/8/16

    RequirementsforPowerFit andDisVis

    • Basic:– Python2.7– NumPy 1.8+– SciPy– GCC(oranotherC-compiler)

    • OptionalforfasterCPUversion:– FFTW3– pyFFTW

    • OptionalforGPUversion:– OpenCL1.1+– pyopencl– cIFFT– gpyfft

    Solutionforgridandcloudcomputing:

    Docker containersbuiltwithproperlibrariesandopencl support:

    Basedockerfile withopencl:https://github.com/indigo-dc/docker-opencl

    Dockerfile forDisVis:https://github.com/indigo-dc/docker-disvis

  • 154/8/16

    AccessingGPGPUresourcesviagrid

    • TestbedatCIRMMPinFlorence• Newjdl requirementsforGPUs:

    executable = "disvis.sh";inputSandbox = { ... };...GPUNumber=2;

    • DirectsubmissiontoFlorencetestbedCE:glite-ce-job-submit -o jobid.txt -a \-r cegpu.cerm.unifi.it:8443/cream-pbs-batch disvis.jdl

  • 164/8/16

    AccessingGPGPUresourcesviagrid

    • Exec script with docker command to access the GPU:docker run --device=/dev/nvidia0:/dev/nvidia0 \

    --device=/dev/nvidia1:/dev/nvidia1 \

    --device=/dev/nvidiactl:/dev/nvidiactl \

    -v $WDIR:/home opencl_disvis /bin/sh \

    -c 'export LD_LIBRARY_PATH=/usr/local/lib64; \

    disvis /home/O14250.pdb /home/Q9UT97.pdb /home/restrain

    • The docker container was built directly on the site with cuda5.5 and is thus site-specific (hardware dependencies)

  • 174/8/16

    AccessingGPGPUresourcesviacloud

    thankstoMarioDavid

    • Build a opencl docker container with the proper driver for the graphic card (i.e. site specific) (cuda 7.x)

    • https://github.com/indigo-dc/docker-opencl– Dockerfile based on ubuntu_nvidia_opencl + disvis

  • 184/8/16

    Baremetal vsgridvscloud

    ID Type GPU #CoresCPUtype Mem(GB)

    B-K20 Baremetal TeslaK20 24HT(12real) Intel(R)Xeon(R)[email protected] 32

    B-K40 Baremetal TeslaK40 48HT(24real) Intel(R)Xeon(R)[email protected] 512

    D-K20 Docker onK20 TeslaK20 24 Intel(R)Xeon(R)[email protected] 32

    K-K40 KVMonK40 TeslaK40 24 Intel(R)Xeon(R)[email protected] 32

    Case MachineTimeGPU(sec)

    TimeCPU1core CPU1/GPU

    B-K40 Baremetal 674 7928 11.8

    K-K40 KVM 671 7996 11.9

    B-K20 Baremetal 830 11839 14.3

    D- K20 Docker 837 11926 14.3

    Nolossofperformance

    CourtesyofM

    arioDavid

    INDIGO

  • 194/8/16

    Conclusions

    • GPGPU-enabledsoftwaresuccessfullyportedtobothgridandcloudenvironmentsusingdocker containers

    • Thedocker containersaresite-specific(becauseofthedriverdependencies)

    • SuccessfulgLite-basedsubmissiontotestbed

    • Webportalindevelopment

    • MoreGPGPUworkdone(refertoAntonioRosato’spresentationofWednesday)

  • 204/8/16

    Conclusions

  • www.egi.eu

    Thankyouforyourattention.

    Questions?

    EGI-Engage isco-fundedbytheHorizon2020FrameworkProgrammeoftheEuropeanUnionundergrantnumber654142

    Withthanksto:- MarcoVerlato (MoBrain/INDIGO)- AntonioRosato (MoBrain/INDIGO)- ZeynepKurkcuoglu (INDIGO)- MarioDavid(INDIGO)

Click here to load reader

Reader Image
Embed Size (px)
Recommended