LIGO Data Quality Monitoring

Keith RilesUniversity of Michigan

2LIGO-G0600XX-00-Z

Outline*

Online data quality monitoring» Figures of merit

» Real-time DQ flagging and DQ database

Offline data quality monitoring» Teams / working groups

» Offline DQ flagging

» DQ infrastructure

Use of DQ information in astrophysical analysis

*Thanks to John Zweizig for some borrowed slides

3LIGO-G0600XX-00-Z

Figures of Merit – Spectra

Control room wall(updates ~1/second) S5 web page

4LIGO-G0600XX-00-Z

Figures of Merit – DMT

Most of the control room figures of merit for performance are produced by Data Monitoring Tool (DMT) programs running 24/7 in background

With infrastructure (and many monitoring programs) provided by John Zweizig (Caltech), the DMT environment allows many different LSC scientists to contribute to online data quality monitoring

Supports nearly real-time strip charts, spectra, and histograms (DMT viewer), archiving of second and minute trends (visible with the standard Data Viewer), triggers for the database, control room alarms, and web display of status and plots.

Figures of merit and other status/performance measures monitored by operators (24/7) and scientific monitors (scimons – 20/7)

5LIGO-G0600XX-00-Z

Figures of Merit – DMT

6LIGO-G0600XX-00-Z

Figures of Merit – DMT

aaaaa

Seismic band-limited RMS noise

Anthropogenic noise (car traffic)

Blue spikes from trucks

7LIGO-G0600XX-00-Z

Figures of Merit – DMT

State Vector: 0 - Down

1 - Mode cleaner locked

2 - Arms locked

3 - Full power

4 - Science mode

8LIGO-G0600XX-00-Z

Figures of Merit – DMT

Binary neutron star range

(SNR=8, averaged over location/orientation)

Livingston 4-km (L1)

Hanford 4-km (H1)

Hanford 2-km (H2)

9LIGO-G0600XX-00-Z

Figures of Merit – DMT

Calibration line strengths and unity gain frequencies

10LIGO-G0600XX-00-Z

Figures of Merit – DMT(FOM2 at LLO)

Glitch rates (4σ, 6σ)

Band-limited RMS

Pulsar injection

excitation

Histogram (high-passed)

Days to reach Crab pulsar spindown limit

11LIGO-G0600XX-00-Z

Figures of Merit – DMT(FOM3 at LHO)

“Pixel fraction”

(Canary)

hrss at 50% efficiency (sine-Gaussians)

Binary inspiral trigger SNR

Stochastic Ω sensitivity (1 minute)

12LIGO-G0600XX-00-Z

Figures of Merit – DMT(Strain RMS for L1)

Low freqs

Mid freqs II

Mid freqs I

High freqs

13LIGO-G0600XX-00-Z

Figures of Merit – Online Flagging

These online figures of merit allow rapid identification of problems (if operators & scimons are attentive)

They also allow early identification of time intervals to be flagged as having questionable data quality DQ Flags (E-Log)

Handful of S5 flags are automatically inserted in the database: (John Zweizig)

ASI_CORR_OVERFLOW Injection

H1_Not_Locked PD_Overflow

H2_Not_Locked Wind_Over_30MPH

14LIGO-G0600XX-00-Z

The S5 Web Page

The S5 web page: http://blue.ligo-wa.caltech.edu/scirun/S5 serves as a central ‘clearinghouse’ for run-related information:

• Links to web-accessible real-time figures of merit

• Links to archives of summary plots

• Scimon instructions and checklists

• Links to instructions for software tools and infrastructure

• Links to web pages of investigation teams

• Links to inspiral/burst triggers

15LIGO-G0600XX-00-Z

Figures of Merit – Online Flagging

Other DMT monitors provide raw information for DQ flags to be defined offline

• Minute trends

• Triggers

Recent aircraft passage at Hanford (microphone time/frequency trajectory of excess power)

Fitted parameters from PlaneMon

16LIGO-G0600XX-00-Z

Offline DQ Flagging

Many DQ issues require more investigation:

• Investigation Teams [ Committee/Working Groups as of mid 2006 ]

Calibration stability Interchannel correlations

Glitches Upconversion

Timing stability Environmental disturbances

Hardware injections Data reduction

Data quality

• Astrophysical Search Groups (“bottom line effects”)

Bursts Continuous-Wave

Inspirals Stochastic

17LIGO-G0600XX-00-Z

Example: Calibration Line Errors*

Calibration lines» Used to monitor IFO optical gain.» Inject three sinusoids (~50, ~550,

~1100Hz) into differential length control channel.

» Injected signals written to frames Several problem with injection

process discovered» Single sample drop-outs» 1-second dropouts» Repeated 1-second segments

Monitoring to detect future errors

» Calibrations notched out» 5σ excursions generate triggers» Trigger identified (offline script)

Segments produced to cover triggers

*Slide from J. Zweizig’s Elba DQ talk

18LIGO-G0600XX-00-Z

DQ Flags Storage

Summary information on DQ flags stored in segment “repositories”:

http://gallatin.physics.lsa.umich.edu/~keithr/S2DQ

http://gallatin.physics.lsa.umich.edu/~keithr/S3DQ

http://gallatin.physics.lsa.umich.edu/~keithr/S4DQ

http://gallatin.physics.lsa.umich.edu/~keithr/S5DQ

More detailed information on many investigations:

http://www.ligo.caltech.edu/%7Ejzweizig/S2_Data_Quality/index.html

http://www.ligo.caltech.edu/%7Ejzweizig/S3_Data_Quality/index.html

http://www.ligo.caltech.edu/%7Ejzweizig/S4_Data_Quality/index.html

http://www.ligo.caltech.edu/%7Ejzweizig/S4_Data_Quality/index.html

19LIGO-G0600XX-00-Z

Offline Feedback to Control Room

For ongoing data runs, operations can benefit from offline analysis

First success: Glitch team feedback in S4 (and pre-S4 eng. run)

For the S5 run, weekly S5 run coordination and detector characterization telecons have alerted commissioners and observatory scientists to problems to repair in weekly maintenance periods or 2-week commissioning periods.

Rapid feedback has been especially useful from

Glitch studies Line-finding Data quality studies

Environmental/upconversion studies

Burst and inspiral trigger studies

20LIGO-G0600XX-00-Z

Offline Feedback to Control Room

Very strong effort from the Glitch Group:

Large detchar team drawn from Burst and Inspiral Groups (Team led at different times in S5 by Laura Cadonati, Erik Katsavounidis, Alessandra Di Credico, Gaby Gonzalez, Shourov Chatterji)

Formal “shifts” using remote access to online / offline diagnostics, including “Event Display” (Shantanu Desai)and “Q Scan” (Shourov Chatterji)

Investigation of loud inspiral / burst triggers

Weekly telecons (initially twice-weekly) and reports

Archive of summary information and of daily/weekly/monthly reports (most recent example)

21LIGO-G0600XX-00-Z

Q-Scan Display (snapshot)*W

hite

ned

Spe

ctro

gram

s

Whi

tene

d T

ime

Ser

ies

*Slide from J. Zweizig’s Elba DQ talk

22LIGO-G0600XX-00-Z

How are flags recorded / retrieved?

Two recording schemes used to date:

S2-S4: Manual flag insertion with segments/flags on disk

S5: Automatic/manual flag insertion of segments/flags in

IBM DB2 database (Duncan Brown)

Two retrieval schemes used to date:

S2-S4: segwizard and segments utilities (Peter Shawhan)

S5: Ditto + LSCsegFind (Duncan)

23LIGO-G0600XX-00-Z

How are flags recorded / retrieved?

Previous scheme:

Aperiodic releases of new DQ versions all at once , with KR as common bottleneck (6 releases for S2, 6 for S3, and 10 for S4)

New scheme:

Flags have individual version numbers and can be updated at any time (KR still a bottleneck for manual updates, but

threshold for making changes much reduced; Chris Messenger serving corresponding role for GEO)

Handling of version numbers needs improvement (problems became apparent only recently)

24LIGO-G0600XX-00-Z

Epochs or Vetoes?*

In theory» Epochs used to handle exceptional conditions that are

– Long term several second to hours– Affect reliability or alter noise spectrum greatly– Disable analysis of data in time epoch.

» Vetoes used for transients (short term effects)– Analyse data, but reject any GW candidate.– Minimizes dead-time– Simplifies analysis job submission

In practice» Difficult to determine extent of effects (e.g. are signals really linear around

PD overflows?)» Epoch easier to use than vetoes (much better tools)» Most data quality flags used to define epochs (at discretion of analysis

groups)

*Slide from J. Zweizig’s Elba DQ talk

25LIGO-G0600XX-00-Z

Segment Database*

Database interfaces» LSCSegFind: Command line database query

» Text files– Available over web

– Used by SegWizard and automated analysis pipelines

» SegWizard GUI (or command line interface)– User selects single or multiple IFOs in science mode

– Remove any combination of data quality segments (click on segment name)

– Prints a list of time ranges to be analysed

Example segment types» IFO states, e.g. Science or Injection mode

» Environmental noise sources: Unusual seismic noise, High winds

» IFO conditions: PD saturation, ADC overflows, Calib line dropouts

*Slide from J. Zweizig’s Elba DQ talk

26LIGO-G0600XX-00-Z

Segment type H1 H2 L1ASC_Overflow 1758(0.01) 1481(0.01) 102366(1.14)Calib_Bad_Coefs 4625(0.04) 2509(0.02) 5125(0.06)Calib_Dropouts 8586(0.08) 448(0.00) 165(0.00)Checksum 225773(1.91) 253924(1.93) -Hx_Not_locked 434268(3.67) 758595(5.70) -Injections 52033(0.44) 78584(0.60) 17087(0.19)Light_Dip (5%) 824(0.01) 24666(0.19) 27752(0.31)PD_Overflow 88156(0.75) 3174(0.02) 11165(0.12)Out_of_Lock 596(0.01) 699(0.01) 994(0.01)Pre_LockLoss (30m) aa16336(0.14) 17943(0.14) 24074(0.31iWind > 30MPH 11877(0.10) 16699(0.13) -AS_Trigger 0 0 2665(0.03)Bad_Sensing - - 21500(0.24)OSEM Glitch 0 112(0.00) 0All 824047(6.97) 1131062(8.58)

11.9Ms 13.2Ms 8.9Ms

Data Quality Segment Types*

(not exhaustive list)

*Slide from J. Zweizig’s Elba DQ talk

27LIGO-G0600XX-00-Z

Use of Data Quality in Analyses*

Segments defined with no guarantees» No guarantee of efficacy

» Could cause some GW signals to self-veto

Analysis groups must» Decide which segments are appropriate

» Test segment safety (does it veto loud injections?)

» Decide whether to analyse data from segment, treat as a trigger veto or ignore.

*Slide from J. Zweizig’s Elba DQ talk

28LIGO-G0600XX-00-Z

Remarks

Data quality evaluation benefits from attacks on several fronts:» A priori investigation of generic “glitchiness” seen in online monitors» “Near-bottom-line” investigation of loud inspiral / burst triggers» Ditto for weaker triggers with time slides» Ditto for stochastic analysis correlations (identifying bad epochs)» Ditto for spectral lines found by pulsar analyses » But occasional surprises from “real bottom line” (infamous S2 plane)

Participation by astrophysics search groups is critical in identifying the problems that hurt us most

29LIGO-G0600XX-00-Z

Remarks

Instrumental expertise not necessary in order to identify problems Commissioners and observatory scientists truly welcome targeted diagnostics where astrophysics sensitivity can be improved and there is a “meter”

See example from last Friday’s Glitch Report

30LIGO-G0600XX-00-Z

Loud-glitch rate had been climbing – Now back to normal

H1 glitch improvement after recent pre-MC fix

(L. Cadonati)

Days in S5

31LIGO-G0600XX-00-Z

Remarks

As part of ongoing LSC reorganization, the LIGO detchar working groups are requesting official GEO liaisons:» Report on artifacts seen in GEO data and on useful GEO

characterization methods

» Report back to GEO analysts on LIGO problems and methods

We can learn much from each other!