NA61 collaboration meeting, Zagreb, October 2011 1
Shine Offline Framework:Status and Plans
The Shine Offline Group
NA61 collaboration meeting, Zagreb, October 2011 2
Shine: Organization
• Framework: event (object persistence), detector & mag. field classes, central configuration, coordinate systems, data-source managers
• Event I/O: raw event (BOS), DS, SHOE (import/export)
• Modules: reading/writing, calibration, reconstruction, simulation
• NA49: clients, interfaces
• eventBrowser
• Validation: framework, core, event structures, modules
• Utilities: misc. helpers, STL (containers), Math, Geometry
• Documentation: SHOE, I/O, modules, reconstruction, physics, tracking
• Build system: CMakeModules, CMakeLists
NA61 collaboration meeting, Zagreb, October 2011 3
Overview
Run Control
Central ConfigManager Registry
NA61:
TPC
TOF
Trigger
PSD
NA61:
TPC
TOF
Trigger
PSD
Mag. Field
NA61 collaboration meeting, Zagreb, October 2011 4
Framework
• Event structures: SHOE
• Detector description
• strict separation of framework classes and (physics) code
• one class per header file
• non-deletable: processing only adds things, Has/Make/Get interfaces
• prevent accidental processing of a copy: private ctor/dtor
• data-source Managers: config defined queues of Managers; easy source
replacement, wrappers for MySQL, SQLite, Xerces
• central configuration: Detector, Managers, Modules
• Registry of Random number streams, seeding conf
NA61 collaboration meeting, Zagreb, October 2011 5
Framework: Data Managers
SQLiteMySQL Xerces
TPC
VTPC
MTPC
GTPC
TOF
BPD
Trigger
PSD
Magnetic field
Drift velocity
etc.
and now also TGeo
NA61 collaboration meeting, Zagreb, October 2011 6
Framework: Configuration
• XML
• run-time XML Schema validation: XSD
• automatic conversion of physical quantities into internal units
<energy unit=”GeV”> 137.23 </energy><position unit=”cm”> 14.3 </position>
• default configuration check-summed, modifications reported in run logs
• full configuration saved with data
NA61 collaboration meeting, Zagreb, October 2011 7
Event I/O
Currently supports:
• raw BOS reading
• DS reading
• SHOE read/write (ROOT persistent Event)
• (pending) reading of event generators UniGen, HepMC, ...
NA61 collaboration meeting, Zagreb, October 2011 8
SHOE
• output files (ROOT) are Event objects (no persistent/transient separation)
•mega/mini/micro SHOEswitching-off increasing level of Event details (branches)
• GUIevent/detector browser, run configuration inspector, simple selection tools
NA61 collaboration meeting, Zagreb, October 2011 9
SHOE
NA61 collaboration meeting, Zagreb, October 2011 10
Modules
• By detector: BPD, Trigger, TPCs, TOF, PSD etc.
• calibration, reconstruction, simulation (Geant4)
• readers/writers: native event/detector (SHOE)
• modules replaceableimplementing several alternative or competing approaches is easy
• (if needed) Modules communicate (only) through the Event structure
NA61 collaboration meeting, Zagreb, October 2011 11
Modules: Run Control
• rudimentary control-language implemented in XML
<sequenceFile>
<enableTiming/>
<moduleControl>
<loop numTimes="unbounded">
<module> EventFileReaderSG </module>
<module> EventCheckerSG </module>
<module> BPDCalibratorSG </module>
<module> BPDEventSelectorSG </module>
<module> TPCCalibratorSG </module>
<module> TOFCalibratorSG </module>
<module> TPCClusterFinderFG </module>
<module> TPCTrackFinderWG </module>
<module> TOFTimeFinderDG </module>
<module> TPCTrackPlotterSG </module>
<module> ShineFileExporterSG </module>
</loop>
</moduleControl>
</sequenceFile>
intermediate Event states can be streamed at any time by inserting Writers
NA61 collaboration meeting, Zagreb, October 2011 12
Modules: Interface
class ModuleName : public fwk::VModule {
public:
fwk::VModule::EResultFlag Init();
fwk::VModule::EResultFlag Process(evt::Event& event,
const utl::AttributeMap& attr);
fwk::VModule::EResultFlag Finish();
REGISTER_MODULE(”ModuleName”, ModuleName, ”$Id$”);
};
NA61 collaboration meeting, Zagreb, October 2011 13
NA49 Clients as Modules<sequenceFile>
<enableTiming/>
<moduleControl>
<loop numTimes="unbounded">
<module> EventFileReaderSG </module>
<module> ClientInitializerSG </module>
<module config="many"> MiniModuleSG </module>
<module config="vdinit"> MiniModuleSG </module>
<module> VdriftCorrModuleSG </module>
<module config="VTPCs"> Dipt256NewModuleSG </module>
<module config="MTPCs"> Dipt256NewModuleSG </module>
<module config="VT1"> EdistoModuleSG </module>
<module config="VT1"> VtNcalcModuleSG </module>
<module config="VT2"> EdistoModuleSG </module>
<module config="VT2"> VtNcalcModuleSG </module>
<module config="MTL"> EdistoModuleSG </module>
<module config="MTR"> EdistoModuleSG </module>
<module config="ALL"> ResCorrModuleSG </module>
<module config="TrckM1"> MtracModuleSG </module>
<module config="I2RTrck"> MtracModuleSG </module>
<module config="MTPCs"> MpatModuleSG </module>
<module config="VT-2"> MpatModuleSG </module>
<module> NA49ToShineConverterSG </module>
<module> ShineFileExporterSG </module>
</loop>
</moduleControl>
</sequenceFile>
current module sequence with wrapped clients
DSPACK assimilated into a shared library (DSHACK), runs in local mode (still needs shm)
NA61 collaboration meeting, Zagreb, October 2011 14
Client-Module Configuration
<ResCorrModuleSG>
<commandLine config="ALL">
-vt1_chris [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/vt1.corr
-vt2_chris [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/vt2.corr
-mtl_chris [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/mtl.corr
-mtr_chris [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/mtr.corr
-p [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/vdrift_2007.txt
</commandLine>
</ResCorrModuleSG>
example: XML configuration of RES_CORR_ZG client,run as Shine module
NA61 collaboration meeting, Zagreb, October 2011 15
Utilities
• XML Reader (Xerces library wrapper)
• SI units, internal units definition
• Mathematics, Statistics & Physics functions
• Traces, trace algorithms
• Linear Algebra, Geometry
• ODE Runge-Kutta Integrator, Mag. Field Tracker
• MD5 for configuration checksums
• custom exceptions
• logging, generic file I/O
• special template libraryspecialized event & detector containers, shadow pointer, Meyers singleton, counted objects, safe bool cast etc.
NA61 collaboration meeting, Zagreb, October 2011 16
Build System
• CMakeseems to be preferred by large projects, multiplatform
• Shapefor external dependencies and production releasessmall dependency resolution engine; pythonfor installation follow instructions in twiki pages
• buildbotsautomated compile, test, validation after each commitseveral linux flavors, email blame notifications
NA61 collaboration meeting, Zagreb, October 2011 17
Validation• Unit testing: CppUnit
• I/O validation (test ROOT schema evolution)
• detect changes in reconstructed quantities on example datasets
NA61 collaboration meeting, Zagreb, October 2011 18
Status of Client Porting
Roland & Oskar are mostly done with the technicalities of client porting and inclusion in Shine.
more in Roland’s and Oskar’s talk
To do:
• fix all bugs related to the operational difference: clients were single processes before, initialization/run/exit once per event (nobody cared about memory management); now initialization is done only once at Shine start, call client main() for each event in the chain(most clients written in a really unprofessional way -> a lot of bug fixes, will have to be ported back to old source)
• output verification (done for some)for some can be done only on statistical level, numerical differences (e.g patrec) mostly due to the pgf/gfortran differences in executable code
• event comparison
NA61 collaboration meeting, Zagreb, October 2011 19
Status of Client Portingmini ported module
vdrift_corrported module
dipt ported module
edisto ported module
vt_ncalc ported module
res_corr ported module
mtrac ported module
mpat ported module
patrec ported module
tpc_error ported module
r3d ported module
tpc_flight_time ported module
vtx ported module
ppoints ported module
gtpc_trackported module
reform ported module
ftof, tofl, tofr ported module
dedxna61 ported module
v0find ported module
v0fit ported module
NA61 collaboration meeting, Zagreb, October 2011 20
News
• Migration of NDST Online Monitor to Shine
• first native module (TriggerSimulatorSG )
• MC event fully implemented in SHOE with RecEvent SimEvent cross linking
• buildbot integration (mostly done)
• first tags and a release
• several physics analysis done with SHOE
NA61 collaboration meeting, Zagreb, October 2011 21
First Physics Analyses with SHOE
T61 – Tomek’s analysis
SHOE – Martin’s analysissame event selection
pC @ 31GeV 2007 (07H)h- analysis
NA61 collaboration meeting, Zagreb, October 2011 22
First Physics Analyses with SHOE
12 events out of ~200,000
already done: MC corrections; next: acceptance, feed down
NA61 collaboration meeting, Zagreb, October 2011 23
First Physics Analyses with SHOEAnd much more:
• Krypton calibration (Adam)
• S4 trigger bias studies with TriggerSimulatorSG (Michael H)
• T2 trigger probabilities with BOSToShine (Michael H)
• charged hadron analysis with SHOE (Martin)
• V0 analysis with SHOE (Michael U)
NA61 collaboration meeting, Zagreb, October 2011 24
First Physics Analyses with SHOE
And much more...
• short lived resonances (Tanja)
• PSD (Oleg)
• (Maja, Anar, ... ?)
NA61 collaboration meeting, Zagreb, October 2011 25
Development in Progress
• Shoe event: still thinking about alternative to ShoeLaces(current track/vertex linking implemented with indices)
• warning: SHOE still under development
• current production with beta release
• backward compatibility not guaranteed (yet)might affect files and user code!
NA61 collaboration meeting, Zagreb, October 2011 26
Monte Carlo Status
• see Marek’s talk
NA61 collaboration meeting, Zagreb, October 2011 27
Suggestions for Help
• detector description (especially “new” detectors)
• access to databases (currently DSPACK soon SQLite or ROOT)
• reconstruction modules for new detectors: PSD, LMPDthese Modules can run after the client-wrappers or, since they will not be ready soon, even in the DSToSHOE-phase:EventFileReaderSG (read and convert standard reconstruction output) -> PSDReconstructorSG -> LMPDReconstructorSG -> ShineFileExporterSG
• SHOE validationeverybody who runs old analysis on T61 or DSPACK:please re-write your progs for SHOE and check if physics results are the same
• utilities from old framework need to run natively:PadTrans: (x,y,z) given (pad,time) and vice versapotential point client: ideal for first moduleKalmanTracker?
• simulation modulesadvances with VMC, simHits are ready, need e.g. native MTSIM
NA61 collaboration meeting, Zagreb, October 2011 28
End