Distributed Geant4 simulations on NorduGrid

Andi Hektorandi.hektor@cern.ch

NICPB in Tallinn

(CERN, Estonian Grid)

2nd Geant4 Workshop at HIPJune 6-7, 2005

Outline

The problem: mobile gamma-ray spectrometry The detector system and detection limits Geant4 model and some examples Distributed MC simulations on the Grid Some results, conclusions and questions

A. Hektor, K. Kurvinen, R. Pöllänen and P. SmolanderRadiation and Nuclear Safety Authority (STUK)M. KettunenFinnish Defence Forces Technical Research CentreJ. LyytinenHelsinki University of Technology, Laboratory of Lightweight Structures

Funded by: SCIENTIFICFunded by: SCIENTIFIC ADVISORY BOARD FORADVISORY BOARD FOR DEFENCEDEFENCE

REAL TIME RADIATION SURVEILLANCE EQUIPMENT FOR THE UNMANNED AERIAL VEHICLE RANGER

In the frames of the project:

Mobile gamma-ray detector

Tracking of a release plume

• Time critical, decision aid for initial response, radiation safety of the air crew

Fallout mapping

• After the initial emergency, long term risk management

Locating of a point source

• Lost sources, smuggling and terrorism

Unmanned aerial vehicle (UAV) is Unmanned aerial vehicle (UAV) is an excellent choicean excellent choice!!

Unmanned aerial vehicle

Ranger Tactical UAV Operated by FDF Range: 150 km Endurance: 5 hrs Payload: 40 kg

Catapult take-off Landing on skids

Fully autonomous or remotely piloted

Geometry of the detector system

•Airworthiness above all

•Vibration attenuation

•Temperature control

•Modularity

•Easy installation

Light weight structure

Installation of the detector

The gamma-ray spectrometer

Physics in a gamma-ray scintillator

NaI(Tl)

Photoelectric absorption

Compton scattering

Pair production

Multiple scattering

Electron ionisation

Bremsstrahlung

Positron annihilation

Source

Detection limits of the detector

Different type of sources, long distances between the source and detector, high speed of the detector, energy resolution

1D

3D

2D

Constructing of our detector model in Geant4

Geometry

Physics List

Input parameters

Kernel

AnalysisVisualization

Visualization filesHistograms

Kernel logsTracks

CAD model

Analysis

Ideas for physicsand input data

Some examples of our MC simulations in Geant4 (1/4)

Energy of the gammas: 662 keV (Cs-137)Number of the gammas: 15

Example 1 Example

2

Some examples of the MC simulations in Geant4 (2/4)

Energy of the gammas: 662 keV (Cs-137)Number of the gammas: 15

Example 2

Some examples of the MC simulations in Geant4 (3/4)

Energy of the gammas: 662 keV (Cs-137)Number of the gammas: 15

Example 4

Some examples of the MC simulations in Geant4 (4/4)

Energy of the gammas: 662 keV (Cs-137)Number of the gammas: 15

Example 4

Distributed MC simulations

Inp

ut

Ou

tpu

t

Parallel comutings on thedifferent comuters

The Grid

Users Computing Resources

Data Storage Elements

Special Devices

Cert. Authority (PKI)

Information System

NorduGrid

Estonian Grid, Jan 2004

GIIS

First “PC-cluster”

Certification Authority

Phone of the local Registration Authority

Management of the “gridified” Gean4 simulations

Generator of Grid jobs (simple Perl script)

Simple job manager(Python script + MySQL)

Data collector (simple Perl script)

Data analysis tools (ROOT scripts)

Geant4 macro filesResource Specification files

ROOT data files

The Grid

Compiled ROOT data file

Management Politics

Logs

Our Grid jobs

Compiled binary of the Geant4 code (3.7 MB):Linux, glibc 2.6.9

Geant4 macro file with input parameters:/gun/energy 662 keV/run/beamOn 500000000, etc

In total 640 Grid jobs were submitted Computation time in total: 417 CPU days

Statistics of the Grid jobs

640 jobs and 2.6 GB of dataComputation time in total:

417 CPU days (on 2.4 GHz and 3.06 GHz Intel P4)

17 failed jobs (2.6%) during the computations and 16 post-processing errors (2.5%)

94%

3%

3%

0%Correct jobs

During comp

Post-proc

Other

NorduGrid middleware is surprisingly stable!

Some conclusions and open questions

It is easy to “gridify” Geant4 applications NorduGrid middleware is stable enough Is it possible to add “Grid functionality” to the Geant4

package? Other middleware packages: LCG2, gLite, UNICORE

etc

Thank you and…

STUKSTUK and and HIPHIP in Helsinki in Helsinki SCIENTIFIC ADVISORY BOARD FOR DEFENCE IN SCIENTIFIC ADVISORY BOARD FOR DEFENCE IN

FINLAND for fundingFINLAND for funding the initial work the initial work Other members of tOther members of technical coordination group of echnical coordination group of

the Ethe Estonian stonian GGridrid: : Mario Kadastik (NICPB), Lauri Anton, Mario Kadastik (NICPB), Lauri Anton, Hardi Teder (EENet), Konstantin Skaburskas (UT)Hardi Teder (EENet), Konstantin Skaburskas (UT)

NorduGrid core developersNorduGrid core developers for technical supportfor technical support Estonian Science Foundation:Estonian Science Foundation: the grants no. 5135, the grants no. 5135,

5935, 53165935, 5316 EU 6th FPEU 6th FP: the grant EC MC MERG-CT-2003-503626: the grant EC MC MERG-CT-2003-503626 Ministry of Education and Research of EstoniaMinistry of Education and Research of Estonia: the : the

support of the technical meetings of the Esupport of the technical meetings of the Estonian stonian GGridrid