21
JANA2: Multi-threaded Event Reconstruction Saas Fee, Switzerland ACAT 2019
JANA2: Multi-threaded Event Reconstruction
Saas Fee, Switzerland
ACAT 2019
Jefferson Lab by the numbers:• 700 employees• FY2018 Budget: $162.4M• 169 acre site• 1,600 Active “User Scientists”• 27 Joint faculty• 608 PhDs granted to-date (211 in progress)• K-12 programs serve more than 13,000 students and 300
teachers annually
Overview of Jefferson Lab
2
• Department of Energy National Laboratory with research mission in Nuclear Physics
• In operation since 1995
• Managed for DOE by Jefferson Science Associates, LLC – Joint venture of Southeastern
Universities Research Association and PAE
• Our primary research tool is CEBAF (Continuous Electron Beam Accelerator Facility) – unique in the world
Aerial photon taken April 6, 2012
June 3, 2013
Exotic Hybrid Spectroscopy with GlueX - David Lawrence - JLab
3
Hall-D Complex
Electron beam accelerator • continuous-wave (1497MHz, 2ns bunch structure in halls)
• Polarized electron beam• Upgraded to 12GeV (from 6GeV)
• 70 μA max @ 12Gev (200μA max @ 6GeV)
Thomas Jefferson National Accelerator Facility (JLab)
Newport News, Virginia, USA
Aerial photon taken April 6, 2012
June 3, 2013
Exotic Hybrid Spectroscopy with GlueX - David Lawrence - JLab
4
Hall-D Complex
Electron beam accelerator • continuous-wave (1497MHz, 2ns bunch structure in halls)
• Polarized electron beam• Upgraded to 12GeV (from 6GeV)
• 70 μA max @ 12Gev (200μA max @ 6GeV)
DAQ obj
reconstruction algorithms
objobj
obj
objobj
obj
•
•
•
Factory Model
Multi-threadingo
o
o
Source(file) thread
Histograms/Trees(file)
eventeventevent
Queue
thread
entangled event
Queue
entangled event
entangled event
eventeventeventevent
thread
thread
read
parse
analyze
High event rate (100kHz) requires buffering in front end leading to entangled events “Event” changes meaning.
Source(file) thread
Histograms/Trees(file)
recon. evnt taskrecon. evnt taskrecon. evnt task
thread
entangled event task
Task Queue
entangled event task
entangled event task
recon. evnt taskrecon. evnt taskrecon. evnt taskrecon. evnt task
thread
thread
read
parse
analyze
Task Queue
std::packaged_task<> combines data and algorithm into single objects allowing threads to be generic
What the user needs to know:
auto tracks = jevent->Get<DTrack>();
for(auto t : tracks){ // ... do something with a track}
JANA process configured for full recon launch
Run 42513:Physics Production mode Trigger: FCAL_BCAL_PS_m9.confsetup: hd_all.tsg0/90 PERP 90JD70-100 58umTPOL Be 75umbeam looks stable
Plugins
EVIO file
JANA2
“Parsing Only” Job demonstrates more stable CPU usage with single flavor of
thread
JANA1 on KNL @ NERSCfull recon.
Full event reconstruction using JANA1 on KNL via container at NERSC
•––––
•––
••
•••••
Schedule for JANA2 LDRD Project at JLab
JANA2
“Parsing Only” Job demonstrates more stable CPU usage with single flavor of
thread
