Scientific Computing II
Molecular Dynamics Simulation – Introduction
Michael Bader – SCCSTechnical University of Munich
Summer 2017
The Simulation Pipeline – Revisitedphenomenon, process etc.
mathematical model?
modelling
numerical algorithm?
numerical treatment
simulation code?
implementation
results to interpret?
visualization
�����
HHHHj embedding
statement tool
-
-
-
validation
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 2
The Seven Dwarfs of HPC – Dwarf # 4
“dwarfs” = key algorithmic kernels in many scientific computingapplications
P. Colella (LBNL), 2004:1. dense linear algebra2. sparse linear algebra3. spectral methods4. N-body methods5. structured grids6. unstructured grids7. Monte Carlo
→ discuss simulation pipeline for molecular dynamics(as example for N-body methods)
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 3
Overview: Particle-Oriented Simulation Methods
General Modelling Approach:• “N-body problem”→ compute motion paths of many individual particles
• requires modelling and computation of inter-particle forces• typically leads to ODE for particle positions and velocities
Numerical Aspects:• how to discretize the resulting modelling equations?• efficient time stepping algorithms?
Implementation Aspects:• suitable data structures?• efficient algorithms to compute short- and long-range forces?• parallelisation?
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 4
Applications for Micro and Nano Simulations
Flow through a nanotube (where the assumptions of continuum mechanicsare no longer valid)
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 5
Applications for Micro and Nano Simulations
Protein simulation: human haemoglobin (light blue and purple: alpha chains;red and green: beta chains; yellow, black, and dark blue: docked stabilizers or
potential docking positions for oxygen)Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 6
Applications for Micro and Nano Simulations
Material science: hexagonal crystal grid of Bornitrid
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 7
HPC Example – Gordon Bell Prize 2005• Gordon-Bell-Prize 2005 (most important annual supercomputing award)• phenomenon studied: solidification processes in Tantalum and Uranium• method: 3D molecular dynamics, up to 524,000,000 atoms simulated• machine: IBM Blue Gene/L, 131,072 processors (world’s #1 in November
2005)• performance: more than 101 TeraFlops (almost 30% of the peak
performance)
(Streitz et al., 2005)
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 8
HPC Example – Millennium-XXL Project
(Springel, Angulo, et al., 2010)
• N-body simulation with N = 3 · 1011 “particles”• study gravitational forces
(each “particles” corresp. to ∼ 109 suns)• simulates the generation of galaxy clusters
served to “validate” the cold dark matter modelMichael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 9
Millennium-XXL Project (2)
Simulation Figures:• N-body simulation with N = 3 · 1011 particles• 10 TB RAM required only to store positions and velocities (single
precision)• entire memory requirements: 29 TB• JuRoPa Supercomputer (Jlich)• computation on 1536 nodes
(each 2x QuadCore, i.e., 12 288 cores)• hybrid parallelisation: MPI plus OpenMP/Posix threads• execution time: 9.3 days; ca. 300 CPU years
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 10
HPC Example – Gordon Bell Prize 2010
(Rahimian, . . . , Biros, 2010)
• direct simulation of blood flow• particulate flow simulation (coupled problem)• Stokes flow for blood plasma• red blood cells as immersed, deformable particles
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 11
HPC Example – Gordon Bell Prize 2010 (2)
Simulation – HPC-Related Data:• up to 260 Mio blood cells, up to 9 · 1010 unknowns• fast multipole method to compute Stokes flow
(octree-based; octree-level 4–24)• scalability: 327 CPU-GPU nodes on Keeneland cluster,
200,000 AMD cores on Jaguar (ORNL)• 0.7 Petaflops/s sustained performance on Jaguar• extensive use of GEMM routine (matrix multiplication)• runtime: ≈ 1 minute per time step
Article for Supercomputing conference:http://www.cc.gatech.edu/~gbiros/papers/sc10.pdf
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 12
Scales – an Important Issue
• length scales in simulations:• from 10−9m (atoms)• to 1023m (galaxy clusters)
• time scales in simulations:• from 10−15s• to 1017s
• mass scales in simulations:• from 10−24g (atoms)• to 1043g (galaxies)
• obviously impossible to take all scales into acount in an explicit andsimultaneous way
• first molecular dynamics simulations reported in 1957
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 13
Laws of Motion
• force on a molecule: ~Fi =∑
j 6=i~Fij
• leads to acceleration (Newton’s 2nd Law):
~̈ri =~Fi
mi=
∑j 6=i~Fij
mi= −
∑j 6=i
∂U(~ri ,~rj )∂|rij |
mi(1)
• system of dN ODE (2nd order)(N: number of molecules, d : dimension),
• reformulated into a system of 2dN 1st-order ODEs:
~pi := mi ~̇ri (2a)
~̇pi = ~Fi (2b)
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 14
Example: Hooke’s Law
i j
rij
• “harmonic potential”: Uharm(rij)= 1
2 k(rij − r0
)2
• potential energy of a spring of length r0 when extended or compressed tolength rij
• resulting force:
1D: Fij = −grad U(rij)= −∂U
∂rij= −k
(rij − r0
)2D, 3D: ~Fij = −k
(~rij −~r0
)Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 15
Example: Gravity
• attractive force due to the mass of two bodies (planets, etc.)• gravity potential: Ugrav
(rij)= −g mi mj
rij
• resulting force:
1D: Fij = −grad U(rij)= −g
mimj
r2ij
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 16
Example: Coulomb Potential
1q
2qr12
+ −
• attractive or repulsive force between charged particles• Coulomb potential: Ucol
(rij)= 1
4πε0
qi qjrij
• resulting force:
1D: Fij = −grad U(rij)=
14πε0
qiqj
r2ij
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 17
Example – Smoothed Particle Hydrodynamics
“Forces” result from discretisation of a PDE:• approximate functions using kernel functions W :
f (x) ≈∫V
f (r ′)W (|r − r ′|,h)dV ′
• for h→ 0: W → δ (Dirac function)• approximation of derivatives→ integration by parts:
∇f (x) ≈∫V
f (r ′)∇W (|r − r ′|,h)dV ′
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 18
Example – Smoothed Particle Hydrodynamics (2)
• approximate integrals at particle positions:
f (ri) ≈N∑
j=1
mj
ρ(rj)f (rj)W (|ri − rj |,h)
• in particular for the density
ρ(ri) ≈N∑
j=1
mjW (|ri − rj |,h)
• similar for derivatives:
∇f (ri) ≈N∑
j=1
mj
ρ(rj)f (rj)∇W (|ri − rj |,h)
• leads to N-body problem (based on Navier-Stokes equations, e.g.)
Michael Bader – SCCS | Scientific Computing II | Molecular Dynamics – Intro | Summer 2017 19