NGC 6217 in DSI mode with F658N Mosaic before new superbias why we don’t release images until after SMOV!

ACS CCD monitoring and pipeline calibration review

Max Mutchler15 July 2009

NGC 6217 in DSI mode with F658N Mosaic with new superbias

ACS CCD monitoring and pipeline calibration review

Max Mutchler15 July 2009

NGC 6217 with ACS / WFCMosaic 4-filter composite image

ACS bias and dark calibration• Homework: ACS Instrument Science Report 2004-07 and 2005 Calibration Workshop

talk• SMOV and CAL programs

– SMOV 11371: NGC 6217 illustration– SMOV 11369: CCD functional (all amp/readout modes)– 11465 CCD monitor (routine bias and dark frames during SMOV)) – 11879 CCD monitor (routine bias and dark frames throughout Cycle 17)– obtain only gain=2 bias frames?

• Near term: assemble best initial set of bias (all gains) and dark calibrations; put it somewhere online (jref) so we and early GOs can download and run CALACS

• By early August, we should get into regular production mode, with our best calibrations getting quickly delivered to the pipeline. Using original production methods, Tyler’s method, or acspipe?

• Cadence of anneals and ref file batches -- extend to more weeks? • Balance speed vs quality• How do we deal with striping? Can only ensure ref files don’t impose it on science data• new bias gradient – stable, a concern?• Adjust bad pixel flagging scheme: BPIXTAB, superbias DQ, superdark DQ

Philosophy and Practices

• “Best” calibration reference files typically available within 2-3 weeks after any observation: quality vs timeliness

• Automated reference file production• Production in native pipeline environment

(SunFire)• Consolidated deliveries to CDBS & archive• Reference file quality control, and CCD

trending

Max’s wishlist from 2005

• Two eras: make new flagging scheme retroactive. Monthly bad pixel tables for 2002-2004?

• More complete flagging of hot pixel CTE tails• Data associated more completely for MultiDrizzle.

Ingest asn tables for data already in the archive?• Add image registration (e.g. tweakshifts) to

pipeline. Ingest asn tables with shifts included?• Better handling of moving target (planetary)

observations in the pipeline: non-WCS image combination, single-image output

Evolution of ACS bias and dark features

• What does “normal” look like for WFC and HRC?• A deep look in 2003: UDF ultrabias and ultradark• Compare normal superbias and superdark from 2002

and 2007 (first and last ones)• Dark current and hot pixel trends (Lucas TIRs)• What do we get after SM4 -- after 29 months of

radiation damage and “annealing”?

WFC bias calibration

• Bias level is measured from each frame’s overscan (with some random variation!), and subtracted (by amplifier quadrant)

• Bias features are subtracted by the “superbias” reference file (BIASFILE)

• Bi-weekly superbias is a simple cleaned combination of 8 frames taken during each two-week period

• Bias structure (mainly bad columns) assigned flag 128 in each superbias data quality (DQ) array as of Oct 2004; was in a static bad pixel table (BPIXTAB); flags propagate to science data

superbias 2002

superbias 2007

ACS WFC superbiascalibration reference fileSMOV 11369 CCD functional8 July 2009

Monitoring bias trends, and superbias quality control

Bias structure and corresponding flagging (as of Oct 2004):5 sigma threshold; don’t want to over-flag!

WFC bad columns DQ flag 128

WFC dark calibration• Monthly CCD annealing• Four 1000-second dark frames now obtained every other

day (was every day until Oct 2004), and just before/after each CCD annealing

• Superdark (DARKFILE) is a hybrid combination of a 2-week “basedark” (has less noise) and a 4-frame “daydark” (has daily warm and hot pixels)

• Residual bias level correction, so superdarks don’t propagate one source of this problem

• Dark features (not just hot pixels anymore) flagged in superdark data quality (DQ) array. More later…

Superdark reference files are hybrids, made from a 2-week “basedark”, with warm and hot pixels from a 4-frame “daydark” added

superdark 2002

superdark 2007

ACS WFC superdarkcalibration reference fileSMOV 11369 CCD functional8 July 2009

WFC superdark and corresponding data quality flagging

SCI array DQ array

ACS / WFC dark current histogram 2002-2005

2005

2002

warmpixels

hotpixels

ACS data quality flagging• Starting Oct 2004, we redefined some redundant DQ flags for new use in reference file DQ

arrays• Use ref file DQ arrays more, use bad pixel table (BPIXTAB) less• Flag 128 now used in superbias DQ (better trending)• Flag 32 redefined as “hot pixel CTE tails” (just the first trailing pixel for now) in superdark

DQ• Flag 64 redefined as “warm pixels” between 0.02 and 0.08 e/sec (also gets more of the

CTE tails) in superdark DQ• New flags provide optional leverage: the pipeline ignores flags 32 and 64• Must set bits=96 to ignore new flags in standalone drizzling, or leave bits=0 to exclude

these pixels• The old 5-sigma threshold for identifying bad bias pixels with flag=128 still seems good. • Leave hot pixel flag=16 at 0.08 e/s • Raise warm pixel flagging threshold from 0.02 to 0.04 or higher   • Stop using flag=32 for trailing CTE pixel; new warm pixel flagging will handle this better• Consider removing bits=96 (=32+64) from pipeline (set to bits=0) so that the default is to

exclude warm pixels, which   include the growing CTE tails behind hot pixels?   The user can always reverse this by setting bits=64, but as the impact of CTE on our dark calibration grows, this flagging would help mitigate it