Date post: | 13-Jan-2016 |
Category: |
Documents |
Upload: | edwin-gibbs |
View: | 217 times |
Download: | 0 times |
INFSO-RI-508833
Enabling Grids for E-sciencE
www.eu-egee.org
Geant4 Physics Validation: Use of the GRID Resources
Patricia Mendez Lorenzo
CERN (IT-GD) / CNAF
Geant4 Bio-Medical DevelopmentsGeant4 Physics Validation
INFN Genova, 13-20 July 2005
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Outlook
Introduction to the LCG
Geant4 in LCG
First Geant4 Productions
Results and Summary
Future Plans
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
What is the LCG?
The LHC: Generation of 40 million particle collisions (events) per second
at the center of each for experiments
Reduce by online computers that filter out
a few hundred good events per sec
Recorded on disk and magnetic tape
at 100-1000 MB/sec: 15 PB/year
Here it is where
the GRID environment
comes in
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
LCG Service Hierarchy
Tier-0 – the accelerator centreData acquisition and initial
processingDistribution of data to the different
Tier’s
Canada – Triumf (Vancouver)France – IN2P3 (Lyon)Germany – Forschunszentrum KarlsruheItaly – CNAF (Bologna)Netherlands – NIKHEF/SARA (Amsterdam)Nordic countries – distributed Tier-1
Spain – PIC (Barcelona)Taiwan – Academia SInica (Taipei)UK – CLRC (Oxford)US – FermiLab (Illinois) – Brookhaven (NY)
Tier-1 – “online” to the data acquisition process high availabilityManaged Mass Storage – grid-enabled data serviceData-heavy analysisNational, regional support
Tier-2 – ~100 centres in ~40 countriesSimulation
End-user analysis – batch and interactive
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Who is who in LCG?
physicsgroup
Tier2
Lab a
Uni a
Lab m
Lab b
Uni y
Uni x
Tier3physics
department
Desktop
Germany
Taiwan UK
FranceItaly
USA
Netherlands
NordicTier-1
CERN Tier 0
Spain
Canada
Uni b
Lab c
Uni n
regional group
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
LCG in the World
May 2005140 Grid sites
34 countries
12000 CPUs
8 PetaBytes
30 sites3200 cpus
25 Universities4 National Labs2800 CPUs
Grid3
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
How the LCG works?
UI UI
CE
RB/BDII
SE
WN
WN
WN
WN
WN
WN
LFC
Connections to UI
Resources Searching
Sent to th
e batc
h
system Distribution to CPUs
Ouputs copied to
Storage Resources
Catalogs getting track of the inputs
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Geant4 in LCG
◘ Electromagnetic and Hadronic physics are fundamental features to be properly simulated in Geant4, however they are extremely CPU demanding
▪ Number of events and energy depending: 1 event of 1GeV ~ 0.03 sec (2.4GHz machine)1 event of 300 Gev ~ 9-10 sec
◘ Goal during the Software Validation: Comparison some shower observables between the two different Geant4 versions and check statistical significant changes
◘ Applications in LCG: First application last December 2004Second application end of June 2005
▪ A total amount of about 3 years of CPU time (1GHz machine)
▪ Very small output for the whole production: 15-20 GB
GRID
GRID
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Geant4 in LCG
◘ Samplings: ▪ 7 simplified detectors
FeSci, CuSci, PbSci, CuLAr, PbLAr, WLAr, PbW04 ▪ 7 different particles (8 in the 2nd production)
e- (2nd production), pi+, pi-, k+, k-, k0L, p, n ▪ 23 different beam energies (GeV)
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 80, 100, 120, 150, 180, 200, 250, 300, 1000(never achieved)
▪ 5 physics list LHEP, QGSP, QGSC, QGSP_BIC, QGSP_BERT
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Geant4 in LCG
◘ Strategy:▪ First Production
◦ Comparison of 7.0.cand01 vs 6.2.p01◦ During the event production phase, 5635 had to be run for each Geant4 version: 11270 jobs to be run◦ Finally the statistical tests were applied to each pair of Geant4 version outputs
▪ Second Production◦ Comparison of 7.0.p01 vs 7.1.cand01◦ During the event production, 6440 jobs had to be run◦ This time each production job contained the production of both Geant4 versions and the statistical analysis
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Geant4 Production in LCG
◘ Stages: 1. Software installation: Installation of the Geant4 packages (with all the required
external additional packages: PI, AIDA, etc) ▪ Software provided via a tar file ▪ Installation through jobs using specific LCG tools ▪ Fundamental request for the sites: Shared area between
WNs and perfectly definition of the software installation region
2. Events production: ▪ Jobs sent by bunches of 1227 (1288 2nd production)
defined by each physics list ▪ 5000 events in each job were produced
3. Analysis: Statistical tests to perform the comparison between the two G4
versions
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Geant4 Production in LCG
◘ General Characteristics: ▪ VO:
◦ 1st Production: dteam (6 certificates, one as dteamsgm) ◦ 2nd Production: alice (2certificates, one as alicesgm)
▪ Sites and middleware operating system: ◦ 1st Production: RedHat7.3 ◦ 2nd Production: Scientific Linux
▪ Resources: ◦ 1st Production: Own RB+BDII+UI: lxb2006 at CERN ◦ 2nd Production: lxplus resources and 2 BDII
▪ All output: ◦ 1st Production: About 30 GB stored at CERN (lxn1183) ◦ 2nd Production: Comparable quantity stored at CERN (lxn1180)
afs Geant4 area at CERN was set to hold the outputs
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Framework developed for Geant4
◘ Generation of a general framework consisting of 3 major tools:
▪ Tool for general and automatic job submission ▪ Tool for events generation in all those sites where the
software has been installed ▪ Tool for data analysis (not needed during the 2nd Production)◘ First Part: Tool for job submissionMethodology:Methodology: ▪ Copy and registry of the Geant4 package
◦ A file containing the TURL is created and is passed to the WN ▪ Follow up of candidates able to admit Geant4 jobs ▪ Selection of long queues only ▪ Automatic built of the .jdl files for each long queue
◦ Built taking as base those proposed by the user joining the name of
the queue where to submit the job◦ Software Installation tools are used to perform the
installation ▪ Submission of these files to each queue
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Framework developed for Geant4
◘ Software Installation tool(Tool submitted in the first step to all sites to install the software) ▪ First step:
◦ The tar file is copied from the SE at CERN to the WN◦ It is untar and copied to the VO_DTEAM_SW_DIR area
▪ Second Step: Software Installation tool◦ Some Geant4 tests are performed to validate the installation◦ If succeeded a tag is published in the Information System
▪ Results:◦ The software installation was tried in 63 sites◦ 1st Production: 28 sites ◦ 2nd Production: 35 sites
▪ Main Problems:◦ Sites were having submission problems◦ Sites did not have defined the VO_DTEAM_SW_DIR area or did not have shared area among WNs
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Framework developed for Geant4
◘ Second Step: Tool for the ProductionStrategy:Strategy: ▪ Only long queues will be used to run the production ▪ All outputs (hbook files) will be stored at CERNMethodology:Methodology: ▪ Geant4 provides their own code to perform the
events production ▪ Python Script for each type of particle, energy,
physics list and calorimeter is created by the framework from one template provided by Geant4
▪ Generation of one jdl per job containing the code provided by Geant4 (the same for all jobs) + that script generated by the framework and changing for each job
▪ Submission of all jdl files to all sites containing the Geant4 installation
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Framework developed for Geant4
Results (First and Second Production):Results (First and Second Production): ▪ A hbook file containing 5000 event is created in the case the
production succeeded ▪The name of the file is created by the framework containing the type
of particles, the energy, the physics list and the calorimeter within the name (important to perform later the comparison)
▪ The hbook file is copied and registered to a disk at CERN
·During the 2nd production a tar file containing different files should have been created in the case the job succeeded. This file was retrieved to the afs area delivered for this aim and copied and registered to the grid.
Around 4508 jobs (two physics list for both Geant4 versions) were run in lest than 2 weeks in 28 sites with a efficiency of about 87%
And for the 2nd production the results are provided by Alberto Ribon
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Framework developed for Geant4
◘ Before the analysis stage the outputs should be checked (only for the 1st Production):
▪ Dealing with about 5000 outputs is not an easy task◦ Tool able to print in a file all the LFNs in the case the efficiency was 100% (reference file)◦ Output retrieve (only std.out files). If succeeded the file will contain the name of the LFN◦ A 2nd tool will check all std.out files looking for the succeeded jobs
- The corresponding GUID and LFN will be stored (test file) and compared with the information included in the reference file
▪ At this point it was more important for us to analyze the successful jobs than to understand the cause of the unsuccessful ones (BUT THIS PROCEDURE IS WRONG!)
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Framework developed for Geant4
◘ Third Step: Tool for the Analysis (Only for the 1st Production)
Methodology:Methodology:
▪ Search of common successful outputs in both Geant4 Versions
▪ Each couple of successful outputs are copied into a local area and analyzed with their own tools
▪ Finally the copy is removed from the local area
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Update of the Framework
▪ This framework covered the Geant4 requirements for its first production
▪ It’s not useful for larger production ◦ Difficult to deal with the output and visualize the results
◘ A new complete tool has been developed for large production
▪ Flexible enough to be used for any VO and any user application
▪ Most of the improvements mostly relative to the outputs handle
Documentation: “LCG2 User Guide”http://grid-deployment.web.cern.ch/grid-deployment/cgi-bin/
index.cgi?var=eis/docs
Download: http://goc.grid.sinica.edu.tw/gocwiki/User_tools
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Update of the Framework
◘ The new framework consists mainly of two tools:
▪ Tool to perform the automatic job submission ▪ Tool to retrieve and handle the corresponding output
1. Automatic job submissionOverview: Given an user’s jdl this tool performs the following actions: ◦ It lists all sites able to run the jdl provided by the user ◦ It creates automatically a jdl file based on that provided by
the user ◦ It submits the just created jdl containing the user
application(s) Moreover it creates a subdirectory (defined by the user)
containing a list of the sites where the jobs have been submitted, the corresponding jdls and the jobs IDs
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Update of the Framework
Additional Features: ◦ The user can define the queues where the jobs are submitted. These queues are checked to see whether it fixes the job requirements. ◦ Requested LFN files can be included. The corresponding TURLs are searched and included in a file passed in the InputSandbox to the WN
2. Retrieve and handle of the outputs◦ The 2nd tool checks the status of the jobs from the job IDs included in the directory given by the user◦ It provides the following output:
The job run in ramses.dcic.ups.es:2119/jobmanager-torque-dteam is in status: ScheduledThe job run in grid01.phy.ncu.edu.tw:2119/jobmanager-torque-dteam is in status: runningThe job run in scaic10.scai.frauhofer.de:2119/jobmanager-torque-dteam is in status: over
The user is queried to retrieve the output to the destination he has previously decided
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Update of the Framework
Additional Features:
• It is possible to visualize the outputs on the web• A html report is provided showing the filesdecided by the user
Genova 13th-20th July Patricia Mendez Lorenzo
Enabling Grids for E-sciencE
INFSO-RI-508833
Summary and Conclusions
◘ Satisfactory implementation of the Geant4 code in the LCG ▪ (Hopefully this is the beginning of a long friendship) ▪ Paper Geant4-LCG submitted to SC05 Conference in EEUU◘ The LCG deployment team is quite interested in the implementation of the Geant4 code in our own tests suites ▪ We hope to collaborate together to help us as well
◘ You cannot keep on working borrowing another VOs: ▪ We have to make you VO=Geant4 as soon as possible ▪ For the next Geant4 Production this should be done
◘ Please come before in each production! ▪ Two weeks is not enough to make a good production ▪ We have to understand in all cases the failed jobs and this need time