Proposal for the MEG Proposal for the MEG Offline SystemOffline System

Assisi 9/21/2004Corrado Gatto

General Architecture Computing Model

Organization & Responsibilities

Milestones

Dataflow and Reconstruction Dataflow and Reconstruction RequirementsRequirements

100 Hz L3 trigger

evt size : 1.2 MB

Raw Data throughput: (10+10)Hz 1.2Mb/Phys evt 0.1 + 80Hz 0.01 Mb/bkg evt = 3.5 Mb/s

<evt size> : 35 kB

Total raw data storage: 3.5Mb/s 107s = 35 TB/yr

Monarc Analysis Model Baseline: Event Monarc Analysis Model Baseline: Event Sizes and StorageSizes and Storage

Sizes– Raw data 1.2 MB/event– ESD 10 KB/event– AOD 2 KB/event– TAG or DPD 1 KB/event

Storage

– Raw data 35 TByte– Reconstructed data 13 Tbyte/reprocessing – MC generated events 40 TByte – MC reconstructed events 25 Tbyte /reprocessing

Assuming 109 generated MC events

Very Rough Estimate

Compare to the othersCompare to the others

Requirements for software architecture Requirements for software architecture or frameworkor framework

Geant3 compatible (at least at the beginning) Easy interface with existing packages:

– Geant3 , Geant4, external (fortran) event generators Scalability Simple structure to be used by non-computing experts Written and maintained by few people Portability Use a world-wide accepted framework

Use ROOT + An existing Offline Package as starting point

Project is startedProject is started5 existing Offline packages under evaluationAliRoot is winning: all aspects of the Offline are

implementedSupport offered by Carminati+Brun

Raw PerformancesRaw Performances

0.0

20.0

40.0

60.0

80.0

100.0

120.0

140.0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Number of event builders

Glo

bal t

hrou

ghpu

t (M

B/s

)

Pure Linux setup20 data sourcesFastEthernet local connection

MEG Computing Model: Work in ProgressMEG Computing Model: Work in Progress Analysis = Reconstruction -> Need O(1) farm per

analysis Analysis policy not yet established (1 centralized

analysis or several competing analyses) Very CPU demanding processing:

– MC generation: 0.5 Hz– Reconstruction: 1 Hz

Final design within few months

Manpower Estimate (framework only)Manpower Estimate (framework only)Activity Profile 2004 (4mo) 2005 2006 2007Off-line Coordination PH 0.8 0.8 0.8 0.8Off-line Framework Develop. PH/CE 1.2 1.2 1.2 1.2Collaborating Class Develop. PH 1.0 1.0 1.0Databases (Design & Maint) CE 1.0 1.0 0.5QA & Documentation PH/CE 0.5 0.5 1.0I/O Development CE/PH 1.0 1.0 1.0MDC CE 0.5 0.5Event Display & UI CE 1.0 1.0 0.5Syst. + web support CT 1.0 1.0 3.0Physics Tools 1.0 1.0 1.0Production PH/CE 1.0Total Needed 4.0 9.0 9.0 9.0

Profile Jul-04 2004 (4mo) 2005PH 0.2 1.2 1.2PostDoc 1.0Ph.D 1 2.0 2.0Student 1.0 1.0CT 1.0Total 1.2 4.2 6.2

Available at Lecce (Estimate)Available at Lecce (Estimate)

Responsibilities & Tasks (all Responsibilities & Tasks (all software)software)

Offline Framework Develop. PSI (Schneebeli) + LecceClass Development Lecce + Detector expertsDatabases (Design & Maint) Tokyo (Savada)Computing Organization & Support LecceSimulation Tokyio (Yamada) + Pisa (Cei)Online Interface PSI (Ritt)Event Display & UI PSI + LecceSyst. + web support PSI + LecceProduction To be decided

Detector experts:– LXe: Signorelli, Yamada, Savada– DC: Schneebeli (hit), Hajime (Pattern), Lecce– TC: Pavia/Genova– Trigger: Nicolo’ (Pisa)

Starting-up: Tasks and Objectives Starting-up: Tasks and Objectives

Setup a system to test the Offline code

DAQ

Prompt

Calib

Reco farm

Online Disk

Server

Staging to tape

Prototype the Main Program Container Classes Steering program FastMC

Test functionalities

& performance

Core Offline System

Development

RDBMS

MilestonesMilestones1. Start-up: September 2004

2. Choice of the prototype Offline system: End October 2004

3. Organize the reconstruction code. December 20041. per subdetector (simulation, part of reconstruction)2. central tasks (framework, global reconstruction, visualisation, geometry database …)

4. Start the development system HW: January 2005

5. Write down the Offline Structure (container classes, event class, etc…) : February 2005

6. MDC: 4th quarter 2005

7. Keep the existing MC in the Geant3 framework. Form a panel to decide if and how to migrate to ROOT: 4th quarter 2005

ConclusionsConclusionsMEG’s Offline project approved by the

collaborationOffline group is consolidating (mostly in Lecce)Work is startingSoftware framework and architecture have been

frozenComputing Model will be chosen soonMerging with existing software within 1 year

Proposed ArchitectureProposed Architecture Fully OO. Each detector executes a list of detector actions/tasks &

produces/posts its own data All the functionalities are implemented in the “Detector Class”

– Both sensitive modules (detectors) and non-sensitive ones are described by this base class. – supports the hit and digit trees produced by the simulation– supports the the objects produced by the reconstruction. – This class is also responsible for building the geometry of the detectors.

The Run Manager coordinates the detector classes– executes the detector objects in the order of the list– Global trigger, simulation and reconstruction are special services controlled by the Run

Manager class Ensure high level of modularity (for easy of maintenance)

– The structure of every detector package is designed so that static parameters (like geometry and detector response parameters) are stored in distinct objects

The data structure is build up as ROOT TTree-objects Offline services (Geometry browser, Event Display, RDBMS interface, Tape

interface, etc.) based on ROOT bult-in services

Computing ModelComputing ModelMulti-T ier S tructure

A n a lys is 1

T ie r 1P isa

M C P rod u c tionR e p roce ss.

A n a lys is 2

T ie r 1L e cce

M C P rod u c tionR e p roce ss.

A n a lys is 3

T ie r 1T o kyo

M C P rod u c tionR e p roce ss.

A n a lys is 4

T ie r 1P S I

M C P rod u c tionR e p roce s s.

T ie r-0P S I

P ro m pt C a lib .E v t R e co

What Does MEG Need?What Does MEG Need? Wan file access Parallel/remote Processing Robotic Tape Support RDBMS Connectivity Event Display GEANT3 Interface Geometric Modeler UI DQM Histogramming

What does ROOT OfferWhat does ROOT Offer Extensive CERN support

– Bonus for small collaborations Unprecedented Large contributing HEP Community

– Open Source project Multiplatforms Support multi-threading and asynchronous I/O

– Vital for a reconstruction farm

Optimised for different access granularity– Raw data, DST's, NTuple analysis

Experiments Using ROOT for the Experiments Using ROOT for the OfflineOffline

Experiment Max Evt size Evt rate DAQ out Tape Storage Subdetectors Collaborators

STAR 20 MB 1 Hz 20 MB/s 200 TB/yr 3 >400

Phobos 300 kB 100 Hz 30 MB/s 400 TB/yr 3 >100

Phenix 116 kB 200 Hz 17 MB/s 200 TB/yr 12 600

Hades 9 kB (compr.) 33 Hz 300 kB/s 1 TB/yr 5 17

Blast 0.5 kB 500 Hz 250 kB/s 5 55

Meg 1.2 MB 100 Hz 3.5 MB/s 70 TB/yr 3

From GEANT3 to ROOTFrom GEANT3 to ROOT

A conversion program from Geant3 to Root (g2root) exists to convert a GEANT RZ file into a ROOT C++ program– All the components are translated: geometry, materials, kinematics, etc.– However, need to write the output in the format required by the

reconstruction code.– The new code is integrated with the VMC schema and can be run with

any desired MC

Call the fortran code from a C++ program– The calls to Geant3 are intercepted and the ROOT components are fully

usable (geometry browser, geometric modeler, etc.)– However, need to interface the output in Zebra format with the TTree

format required by the ROOT based reconstruction code.

Migration schema #1Migration schema #1

Migration schema #2Migration schema #2

ROOT vs FortranROOT vs Fortran Several months start-up Investment for the future All HEP components built in a

unique framework All libraries still mantained

Plenty of compilers Built-in 3D full Event display with

navigator Interactive remote analysis Fully platform independent (can

even mix Unix and Windows) 3-D histos Automatic file compression

Immediate start-up Hard to actract younger people Need to merge many libraries

CERNLIB no longer mantained (last is for the Itanium)

Hard to find free compilers No Event display available in

fortran (naive in G3) No interactive remote analysis

No compression

•A ROOT file is 2-3 times smaller than ZEBRAA ROOT file is 2-3 times smaller than ZEBRA

ConclusionsConclusions ROOT has all the features MEG need already built-in Migration from Geant3 to ROOT is a non existent

problem Real issue are the reconstruction routines: they have to

be written in C++

The decision between an Offline in fortran or in C++ should be taken on the base of much reconstruction code rather than Montecarlo code has already been written