Erik BlaufussUniversity of Maryland
Data Filtering and Software
IceCube Collaboration Meeting
Monday, March 21, 2005
March 21, 2005Data Filtering and Software – E. Blaufuss
Overview• Data Filtering and Software includes:
– Processing and Filtering (PnF) System at Pole– Production Processing software environment
• Offline software suite• IceTray software framework• Dataclasses event containers• Build system and tools
– Database design and implementation• Database server• Access tools
– Data visualization• Event Viewer• Root-based analysis tools
March 21, 2005Data Filtering and Software – E. Blaufuss
PnF System Overview
DAQ
Dispatch
P&F Server
Recon WorkerX 50-60?
4-15MB/s1500-2000Hz
Tape
Transient Buffer
DAQ Land
Raw Data
Online Analysis
Online Analysis
Online Analysis
Recon Monitor
Data Handling
Rec/FiltData
ALL: RAW+FILT+REC
FILT:RAW+FILT+REC
ALL:FILT+REC
Expt. Control / Logging
Offline Software/IceTray
March 21, 2005Data Filtering and Software – E. Blaufuss
PnF system at Pole• V1.0 of PnF system was deployed at Pole
this past season.– Running IceTray in Online environment– Parses and decodes the incoming DAQ stream– Currently performs no reconstruction.
• Currently not in the Main data stream.– Lack of global run information at Pole– Lack of system-wide integration testing
• No PnF output sent via satellite– TestDAQ/Monolith output tarfile only output sent
• New version of Monolith soon (April)– Real testing of data flow to PnF (SPTS -> SPS)
March 21, 2005Data Filtering and Software – E. Blaufuss
PnF Plans for next year
• With the “Real DAQ” in place:– PnF output IS the output stream for RAW event
data from IceCube DAQ.– All other non-event data will be handled by
databases.
• After this next deployment season:– Filtering at South Pole will be REQUIRED.– Will require reconstruction/filtering modules be
ready and fully tested in IceTray offline!
March 21, 2005Data Filtering and Software – E. Blaufuss
Offline Software
• Several incremental releases this past year– In use by reconstruction and simulation groups.
• Current release (new!):– OFFLINE-SOFTWARE V01-05-00
• Full details on release, and HEAD at:– http://glacier.lbl.gov/
• Plans for more functionality and tools over the next year.
• More details regarding “readiness” of software later….
March 21, 2005Data Filtering and Software – E. Blaufuss
Databases (Mons group)
• DOM assembly and testing database setup and in use for FAT testing.
• Geometry database for String 21/IceTop up to date with Kurt’s latest numbers.– Includes “nominal” locations for other strings
• Preliminary version of DOM calibration DB• IceTray module for DB access in place• In the next few months:
– Run configuration DB for data from current year– Continued integration of database into IceTray
March 21, 2005Data Filtering and Software – E. Blaufuss
Visualization
• Event viewer in place– Animations– Fits, features,
readouts
• Root analysis Tree tools in place– Fit/reconstruction
level analyses– Waveforms, features,
hits level analyses
• Plans for more over the next year.
March 21, 2005Data Filtering and Software – E. Blaufuss
Current State of Offline SW(Is it “analysis ready”?)
• IceTray software Framework– Has been released for ~1 year.– Can handle single events in a linear chain of
modules• More advanced module topologies planned this year
• Dataclasses event storage classes– Have been released for >9 months.– Designed for storage of
• IceTray DOM data• Amanda data (treat as stand-alone/together)• Reconstruction results• Geometry, calibration, monitoring structures, more…
March 21, 2005Data Filtering and Software – E. Blaufuss
State of Software (cont’d)
– More extensions planned as need dictates.– Can hold TestDAQ data from this year
• I/O routines for IceCube data– Monolith-reader can decode data from Pole.– Triggered events from South Pole in dataclasses– Basic functionality is there.
• Includes domcal information (Thanks Dima…)• Still some work to do (trigger ID, run ID, etc)
• Run “meta-data” – Run configuration info.– Data exists (files), but documentation and wider
understanding is minimal.– Need docs, fill run configuration database to
make properly available for analysis in IceTray.
March 21, 2005Data Filtering and Software – E. Blaufuss
State of Software (cont’d)
• Cleaning, feature extraction– Basic functionality in place and working
• Hit selection/cleaning• FeatureExtractor (Dima)
– Advanced cleaning needs to be implemented• Amanda specific: Cross talk, flare checking, etc.• Plans for that in next few weeks with Peter Steffen and
others
• Reconstruction– Basic algorithms are done (and reviewed!)
• First guess (linefit, dipole, c-first, TOI, JAMS)• Standard Log-likelihood for muon tracks
– Advanced (exotic PDFs, iterative fits) needed
March 21, 2005Data Filtering and Software – E. Blaufuss
State of Software (cont’d)
– Cut parameters/observables• Work has just started here and much remains.
• Filtering capabilities– Currently possible to use linear chain of modules
to reduce data files.– Branching of flow in modules planned for this
year.• Allow muon and cascade recos in same chain.• Needs a “decision” module
• Amanda– Full support for calibrated F2K files
• Includes TWR in F2K files
March 21, 2005Data Filtering and Software – E. Blaufuss
State of Software (cont’d)
– TWR readouts (non-F2K) support this year– Working towards being ready to merge
AMANDA/IceCube data this summer.• IceCube and Amanda data are both handled naturally
together in IceTray/dataclasses.
• Simulation(Significant driving force in the design and
requirements for IceTray.)• Merging input frames to overlay events
– Currently developing simulation releases against OFFLINE-SOFTWARE releases.
March 21, 2005Data Filtering and Software – E. Blaufuss
String 21 Data – A few plots
March 21, 2005Data Filtering and Software – E. Blaufuss
String 21 Data – A few plots
More like this in Gary’s talk
March 21, 2005Data Filtering and Software – E. Blaufuss
Status SummaryFor example…
Is…. Work provided by…
Core Software IceTray,
Dataclasses
I/O
V1 of all ready and being used
UMD, LBNL, UDel
Core Utilities DB access,
File readers,
eventviewer
80% done, all in a usable state
UMD, Mons, LBNL, Dortmund
Application Software
Reconst.,
Filtering, Calib, Sim.
Work is underway(some useable)
Entire Collaboration
March 21, 2005Data Filtering and Software – E. Blaufuss
Status Summary (2)• IceCube Offline software effort needs more:
– Users– Developers of applications– Feedback, bugs found, documentation readers,
etc.
• We’re providing the framework and some tools. – Real work of providing reconstructions,
analyses, simulations will be the work of the entire collaboration.
• Start using it. It’s ready.– There are many ways to help out in software.