Gadget2 Science Gateway Development

transcript

Overview of GADGET-2Hydrodynamic GalacticSimulations

Bruce Becker | bbecker@csir.co.zaCoordinator, SAGrid

Meraka Institute, CSIR

Bruce Becker: Coordinator, SAGrid | bbecker@csir.co.za | http://www.sagrid.ac.za

Outline● What is GADGET and what does it do ? ● Application implementation and

dependencies● Execution model● Proposed development strategy● Current situation

GADGET: what is it ?http://www.mpa-garching.mpg.de/gadget/

● GAlaxies with Dark matter and Gas intEracT ● Cosmological N-body/Smoothed Particle Hydrodymanics

simulations (SPH)● SPH:

● Divides a fluid into discrete elements (particles)● Have a typical length, over which properties of the

constituents are ”smothed”● Smoothing length can be made adaptive, based on local

conditions● particularly suited to astrophysics simulations at all scales.

GADGET: What is it good for ?● Can be used for studies of isolated systems, or for simulations

that include the cosmological expansion of space, both with or without periodic boundary conditions.

● In all these types of simulations, GADGET follows the evolution of a self-gravitating collisionless N-body system, and allows gas dynamics to be optionally included.

● Both the force computation and the time stepping of GADGET are fully adaptive, with a dynamic range which is, in principle, unlimited.

● Can be used to address a wide array of astrophysically interesting problems, ranging from colliding and merging galaxies, to the formation of large-scale structure in the Universe

GADGET: What is it good for ?

Galactic sphere collissions

Cosmological galactic structure evolution

http://www.mpa-garching.mpg.de/galform/millennium

GADGET-2 : ”The Millenium Run””The Millennium Run used more than 10 billion particles to trace the evolution of the matter distribution in a cubic region of the Universe over 2 billion light-years on a side. It kept busy the principal supercomputer at the Max Planck Society's Supercomputing Centre in Garching, Germany for more than a month. By applying sophisticated modelling techniques to the 25 TB of stored output, Virgo scientists have been able to recreate evolutionary histories both for the 20 million or so galaxies which populate this enormous volume and for the supermassive black holes which occasionally power quasars at their hearts. By comparing such simulated data to large observational surveys, one can clarify the physical processes underlying the buildup of real galaxies and black holes.”

- http://www.mpa-garching.mpg.de/galform/virgo/millennium/

GADGET – Research or Teaching ?

● GADGET is a very serious research application, running on the most powerful computers in the world.

● For serious research, run parameters need to be properly validated...

● ...”But as a teaching tool I think this idea could be great.”

- Prof. Romeel DavéSARChI Chair in Cosmology, UWC/SAAO/AIMS

Application implementation and dependencies

● GADGET-2 is written in ANSI-C, with MPI-bindings

● Inherently parallel, but can also be executed on single CPU serially (good for testing)

● Requires:● GNU Science Libraries● FFTW● (optional) HDF5 library for I/O

Execution Model● GADGET requires a set of initial conditions and a parameter

file to execute – mostly defined in Makefile● Frequent recompilation necessary

● Typical workflow

1) Create initial condition file for specific problem with MUSIC*

2) Compile gadget for specific problem

3) Setup parameter input file

4) run the code and wait (monitor output log)

5) run analysis on the resulting data.

Execution Model

Create initial conditions with MUSICSim.conf → MUSIC → ics.dat

Research question - ”what would I measure

if the following scenariohappened”

Define simulation parametersMysim.param

Run the simulationGadget2 Mysim.param

Portlet – Must Haves● Required input:

● Makefile● Parameters file

● Require two-step portlet● Generate initial conditions● Run Gadget Simulation

● GADGET recompilation happens on the fly● All other dependencies deployed on the computing

infrastructure● Can be obtained from Jenkins or CVMFS at the sites

Portlet – Nice to Haves

● Simple ”wizard” to generate IC and param files● Parametric jobs (with certain parameter ranges)● Snooping of output to see state of the job