Exploiting the Grid to Exploiting the Grid to SimulateSimulate

& Design the LHCb & Design the LHCb ExperimentExperiment

Eric van Herwijnen (CERN) Glenn Patrick (Rutherford Appleton Laboratory)

National e-Science Centre25th April 2002

Moun

Yoke

Vertex

Shielding TrackerCalorimeters

RICH-2

Coil

Muon Detector

RICH-1

LHCb LHCb ExperimentExperiment

Weight = 4,270 tonnesMagnet alone weighs1500 tonnes!

18m x 12m x 12m

1.2 million electronicchannels.

Located 100munderground at the Large HadronCollider (LHC).

Due to startoperation in 2006.

Large Hadron Collider – CERN, Geneva7 TeV protons colliding with 7 TeV protons

27km circumference

8000 cryomagnets 8.3 Tesla magnetic field. 40,000 tons of metal cooled to -271°C 1.8 million Amperes without electrical dissipation 40 MWatts needed for cooling

LHC

Recreating the conditions10-12 seconds after the Big Bang

2835 bunches/beam1011 protons/bunch40MHz crossing rate

Netherlands

Brazil

France

Germany

Italy PRC Romania

Spain

Switzerland

Ukraine

UK

Poland Russia

Finland

Design and construction of LHCb undertaken by 500 scientists from 48 institutes in 14 countries around the world.

Studying the subtle differences between matter and anti-matter. Over 1000 million short-lived particles of matter and anti-matter called B and B-bar mesons will be studied at LHCb each year.

In order to design the detector and to understand the physics, many millions of simulated events also have to be produced.

b

d

B meson B mesonb

d

quarks

0

*0

D

DB

0B

10 mm

Typical decay of Bo meson and anti-Bo meson.

Lifetime is only 0.0000000000015 seconds.

Travel a few millimetres before decaying into other particles which have to be detected.

Design Simulation/Reconstruction

Complex interactions Understanding

Grid – A Single Grid – A Single ResourceResource

Peta Bytes of data storage

Many millions

of events

Many samples

Distributed resources

Many 1000s of computers required

GRIDA unified approach

Worldwide collaboration

Various conditions

Heterogeneous operating systems

DataGrid SitesDataGrid Sites

Dubna

Moscow

RAL

Lund

Lisboa

Santander

Madrid

Valencia

Barcelona

Paris

Berlin

LyonGrenoble

Marseille

BrnoPrague

Torino

Milano

BO-CNAFPD-LNL

Pisa

Roma

Catania

ESRIN

CERN

HEP sites

ESA sites

IPSL

Estec KNMI

(>40) Europe Status Testbed 1 Status

Updatebookkeepingdatabase

Transferdata tomass store

Data quality check

Submit jobs remotelyvia Web

Executeon farm

Analysis

Using Grid MiddlewareUsing Grid Middlewareand DataGrid Toolsand DataGrid Tools

1. Authenticationgrid-proxy-init

2. Job submission to DataGriddg-job-submit

3. Monitoring and controldg-job-statusdg-job-canceldg-job-get-output

4. Data publication and replicationglobus-url-copy, GDMP

5. Resource scheduling – use of CERN MSSJDL, sandboxes, storage elements

Job SubmissionJob Submission

dg-job-submit /home/evh/sicb/sicb/bbincl1600061.jdl -o /home/evh/logsub/bbincl1600061.jdl:#Executable = "script_prod";Arguments = "1600061,v235r4dst,v233r2";StdOutput = "file1600061.output";StdError = "file1600061.err";InputSandbox = {"/home/evhtbed/scripts/x509up_u149","/home/evhtbed/sicb/mcsend","/home/evhtbed/sicb/fsize","/home/evhtbed/sicb/cdispose.class","/home/evhtbed/v235r4dst.tar.gz","/home/evhtbed/sicb/sicb/bbincl1600061.sh","/home/evhtbed/script_prod","/home/evhtbed/sicb/sicb1600061.dat","/home/evhtbed/sicb/sicb1600062.dat","/home/evhtbed/sicb/sicb1600063.dat","/home/evhtbed/v233r2.tar.gz"};OutputSandbox = {"job1600061.txt","D1600063","file1600061.output","file1600061.err","job1600062.txt","job1600063.txt"};

Monitoring and Monitoring and ControlControl

Job

Local disk

Compute Element

globus-url-copy

ReplicaCatalogue

NIKHEF - Amsterdam

CERN TESTBED

REST-OF-GRID

JobStorage Element

replica-get

publish

register-local-file

Storage Element

mss

Data Data

Data

Data Publishing & Data Publishing & ReplicationReplication

Publish Data on Publish Data on Storage ElementStorage Element

Copy data file to storage element:globus-url-copy file:///${chemin}/L69999

gsiftp://lxshare0219.cern.ch/flatfiles/SE1/lhcb/L69999

Register stored data in the catalogue:/opt/globus/bin/globus-job-run lxshare0219.cern.ch /bin/bash -c "export

GDMP_CONFIG_FILE=/opt/edg/lhcb/etc/gdmp.conf;/opt/edg/bin/gdmp_register_local_file -d /flatfiles/SE1/lhcb"

Publish catalogue:/opt/globus/bin/globus-job-run lxshare0219.cern.ch /bin/bash -c "export

GDMP_CONFIG_FILE=/opt/edg/lhcb/etc/gdmp.conf; /opt/edg/bin/gdmp_publish_catalogue -n"

Resource Scheduling:Resource Scheduling:Use of CERN MSSUse of CERN MSS

Copy output to MSS:rfcp L1600061 /castor/cern.ch/lhcb/mc/L1600061

JDL to specify use of CERN MSS:Executable = "script_prod";Arguments = "1600061,v235r4dst,v233r2";StdOutput = "file1600061.output";StdError = "file1600061.err";

OutputSE = "lxshare0219.cern.ch";InputSandbox =

{"/home/evhtbed/scripts/x509up_u149","/home/evhtbed/sicb/mcsend","/home/evhtbed/sicb/fsize","/home/evhtbed/sicb/cdispose.class","/home/evhtbed/v235r4dst.tar.gz","/home/evhtbed/sicb/sicb/bbincl1600061.sh","/home/evhtbed/script_prod","/home/evhtbed/sicb/sicb1600061.dat","/home/evhtbed/sicb/sicb1600062.dat","/home/evhtbed/sicb/sicb1600063.dat","/home/evhtbed/v233r2.tar.gz"};

OutputSandbox = {"job1600061.txt","D1600063","file1600061.output","file1600061.err","job1600062.txt","job1600063.txt"};

Analysis Analysis Improved Improved DesignDesign

email :eric.van.herwijnen@cern.ch

g.n.patrick@rl.ac.uk