CALIBRATION

C. ZampolliALICE Offline Week, 25-27 March 2013

25/03/2013 1C. Zampolli - Calibration - Offline Week

Ever tried. Ever failed. No matter. Try Again. Fail again. Fail better.

(S. Beckett)

ALICE current Calibration(s)

25/03/2013 C. Zampolli - Calibration - Offline Week 3

DataDAQ DA Shuttle OCDB

Data CPass0 OCDB

Data CPass1 OCDB

Manual Calibration

Data Reconstruction ESDs for analysis

C. Zampolli - Calibration - Offline Week 4

Alien job

Calibration strategy – CPass0 + CPass1

25/03/2013

Chunk 0

Alien job

Chunk 1

Alien job

Chunk N

Alien job

…

reconstruction

ESDs

Calib. train

OCDB Update

Merging + OCDB entires

CPass0

Alien job

Chunk 0

Alien job

Chunk 1

Alien job

Chunk N

Alien job

…

ESDs

Calib. train QA train

OCDB Update

OCDBsnapshot

reconstruction

snapshot

Merging + OCDB entires

CPass1

Used for 2012 and 2013 data taking

C. Zampolli - Calibration - Offline Week

2012 Calibration Scheme

5

cPass0

cPass1

Daily QA from detectorsSavannah tickets and communication between QA coordination and detectors

QA weekly meetingOnly Savannah tickets

discussed, to be then handled by offline + calibration groups

QA weekly meeting

Manual update (1w)

+7d

Calibration period, Δ

vPass

QA on vPass

~3 weeks

Max delay between end of data taking and validation of calibration ~3 weeks

Main QA25/03/2013

Used for 2012 and 2013 data taking (some ad-hoc

modifications for 2013 pA data)

2012-2013 summary

period CPass0 CPass1 VPass PPass

LHC13b

LHC13c

LHC13d

LHC13e

LHC13f

LHC13g

25/03/2013 C. Zampolli - Calibration - Offline Week 6

Completed

Ongoing

Not foreseen

Very soon

period CPass0 CPass1 VPass PPass

LHC12a

LHC12b

LHC12c

LHC12d

LHC12e

LHC12f

LHC12g

LHC12h

LHC12i

pA periods

Soon

Delayed to give priority to pA

When and why the timeline was not followed

• Resubmissions of (any) failed “parts”– Many reasons, e.g.:

• Trigger aliases problem• Failures of calibration for some detector which prevented CPass1 from

starting– Much better situation since we use the return code

» Negative: warning, calibration not ok, but expected, so please continue

» Positive: ERROR, calibration not ok, not expected, DO NOT continue» 0: Everything ok

• Some grid issues, e.g.:– Disks full– Problems with one site– OCDB problem

• CPass0 stopped (e.g. in June) to fix code, synchronize it

25/03/2013 C. Zampolli - Calibration - Offline Week 7

When and why the timeline was not followed

• Resubmission of an entire period– LHC12d (2x!), LHC12h, LHC12i– To account for detectors code bugs, unexpected behavior of calibration with

new data taking conditions• Too short period

– Need to be merged with previous or subsequent ones• Lack of feedback from detectors

– But people need holiday…– …and much better since Autumn 2012

25/03/2013 C. Zampolli - Calibration - Offline Week 8

A huge thanks to all the people who have worked (hard) over the last month (and year) to “turn” the ALICE data into the

input for Physics analyses!

When and why the timeline was not followed

• Resubmission of an entire period– LHC12d (2x!), LHC12h, LHC12i– To account for detectors code bugs, unexpected behavior of calibration with

new data taking conditions• Too short period

– Need to be merged with previous or subsequent ones• Detectors needing more than 1 week to assess calibration readiness• And in pA… unexpected problems… see later (unfortunately, not enough

space in one slide )

25/03/2013 C. Zampolli - Calibration - Offline Week 9

Issues during pA• Long runs: in principle not an issue, but…

– Online calibration– Reconstruction time– Merging time

• fixed very quickly, TPC limiting number of chunks to be merged for them• SPD hs misaligned in time at the end of run

– only good chunks reconstructed• Low reconstruction efficiency

– partially fixed in v5-03-Rev-17• TPC online calibration in rare periods

– recovered at CPass1, or need to re-run• HMPID not reconstructed in CPass1 for LHC13d, e, f

– fixed asap, but already in LHC13f• TPC HV trips, HV voltage changed during the run (30/140 runs) in LHC13d, e, f

– TPC code modified to take this into account in v5-03-Rev-15– Not sure how to treat this in MC…

25/03/2013 C. Zampolli - Calibration - Offline Week 10

Issues during pA – II

• Analysis: a source of precious information, which allowed to spot problems– First event in chunk not calibrated for TPC

• Related to the use of the HLT seeds, introduced since 2013 processing– Fixed in v5-03-Rev-15

– Chunks without any TPC information• Related to the way HLT used OCDB, and specifically to when an entry

was not there– Fixed in v5-03-Rev-15

25/03/2013 C. Zampolli - Calibration - Offline Week 11

Essentialities

• A lot of synergies within different “communities”– Offline

• Reconstruction• GRID (including job (re)submission)• MonALISA

– Detector experts• Tuning of calibration (especially TPC)• Fixes of code• Updates of code• But not only offline… (see TPC during LHC13d,e)

– QA• Fundamental to spot problems

– Run Coordination and Trigger Coordination• Constant communication

25/03/2013 C. Zampolli - Calibration - Offline Week 12

Flaws in the procedure?

• Absolutely not! But if we want to talk about room for improvement…– Reduce manual interventions during the processing

• More stable calibrations• Calibrations more robust wrt changes in the running conditions• More stable code

– Check 3x before committing code, test it 4x before using it in production– Reduce the time between data taking and calibrated data (i.e. Physics

Passes)• Increase fraction of online calibrations• Reduce inter-dependencies between detectors • Automatize QA as much as possible

25/03/2013 C. Zampolli - Calibration - Offline Week 13

Up to now…

25/03/2013 C. Zampolli - Calibration - Offline Week 14

Nothing ventured, nothing gained.

Its’ a tough job, but…

Ever tried. Ever failed. No matter. Try Again. Fail again. Fail better. (S. Beckett)

Really?

Something is “easier” than…

• Reprocessing– Using the same schema as up to now

• CPass0 + CPass1 + VPass + Ppass• Cannot (obviously) rely on Trigger Aliases• Some pre-requisites are necessary

– Memory consumption Fixed!

25/03/2013 C. Zampolli - Calibration - Offline Week 15

Something is “easier” than…

• Reprocessing– Using the same schema as up to now

• CPass0 + CPass1 + VPass + Ppass• Cannot (obviously) rely on Trigger Aliases• Some pre-requisites are necessary

– Memory consumption Fixed!– Some issues with calibration Fixed!– Detector simulation should match Data

• Could analysis be used to assess if something is missing?• Review of Savannah open bugs…

25/03/2013 C. Zampolli - Calibration - Offline Week 16

…something else

25/03/2013 C. Zampolli - Calibration - Offline Week 17

Pant!

Pant!

Run2…

Run3…

Preparation for Run2/Run3

See Document

Main goal of Upgrade

25/03/2013 C. Zampolli - Calibration - Offline Week 19

Run2• Detectors should try to move as much as possible online

– Detector algorithms• Due to the dependency on tracking (TPC), everyone should rely on an online TPC

calibration (within HLT) and an online reconstruction• The Shuttle will then continue to export the Condition Data to the OCDB• After the run, possibility to run (one single!) CPass, and perform manual

calibration– Then (likely) VPass, Ppass

• …but the data before CPass should be good enough for a first physics

25/03/2013 C. Zampolli - Calibration - Offline Week 20

DataOnline

Processing Shuttle OCDB

Data CPass0 OCDBManual

Calibration

Data Reconstruction ESDs for analysis

Run3• No more CPass-es possible

– Only second-order corrections possible• Use O2 farm to perform as much online calibration as possible, and online

reconstruction• As for Run2, due to the dependency on tracking (TPC), everyone should rely on

an online TPC calibration (within HLT) and an online reconstruction

25/03/2013 C. Zampolli - Calibration - Offline Week 21

Run3• No more CPass-es possible

– Only second-order corrections possible• Use O2 farm to perform as much online calibration as possible, and online

reconstruction• As for Run2, due to the dependency on tracking (TPC), everyone should rely on

an online TPC calibration (within HLT) and an online reconstruction

25/03/2013 C. Zampolli - Calibration - Offline Week 22

Data

Online Processing :

Calibration and Reconstruction

OCDB

ESDs for analysis

The TPC proposal

25/03/2013 C. Zampolli - Calibration - Offline Week 23

J. Th

eade

r at O

2 m

eetin

g

See talk this afternoon

The TPC proposal - consideration

25/03/2013 C. Zampolli - Calibration - Offline Week 24

J. Th

eade

r at O

2 m

eetin

g Not the standard OCDB, but something available online; standard OCDB updated at

the end of the run/fill

See talk this afternoon

The TPC proposal - continued

• Calibration performed in “Calibration intervals”• TPC would extrapolate the calibration obtained in the previous

calibration interval– Stability in time to be able to extrapolate– First events (not calibrated) stored anyway at Run2, flagged somehow at

Run3• Data would be TPC-calibrated when reconstructed

– All the other detectors may use TPC-calibrated data for their own calibration

25/03/2013 C. Zampolli - Calibration - Offline Week 25

See talk this afternoon

Implications for detectors

• Mainly detectors participating in CPass are affected– TPC will move online anyway– TRD – TOF– T0– MeanVertex– SDD

25/03/2013 C. Zampolli - Calibration - Offline Week 26

Conclusions

• Procedure used for calibrating 2012-2013 data (CPass0 CPass1 VPass PPass) subject to many manual interventions…– Sensitive to many parameters

• …but in general quite successful– To be used for re-processing for old data

• Preparation for Run2/Run3 relies on online calibrations– Basic ingredient: TPC – who already started planning for this

25/03/2013 C. Zampolli - Calibration - Offline Week 27