Post on 19-Dec-2015
transcript
Nat
iona
l Dat
a B
uoy
Cen
ter
National Data Buoy Center
Presented to the
QARTOD III Workshop
November 2, 2005
Wave Data Quality Control
Chung-Chu Teng
Nat
iona
l Dat
a B
uoy
Cen
ter
NDBC Wave Measurement systems
96 buoys (35 directional waves) and 4 C-MAN stations
Nat
iona
l Dat
a B
uoy
Cen
ter
• Sensor Evaluations
• Individual Sensor Calibrations
• Payload Testing
• System Lab Testing
• Blue-tag System Testing
• Burn-In
• Data Evaluation at deployment
Data Quality Control:The Last Line of Defense, after…
Nat
iona
l Dat
a B
uoy
Cen
ter
FPSN7 vs 41013
• 3-m Discus moored next to FPSN7
• 3-m to replace FPSN7
• Interesting, but limited data comparison
Nat
iona
l Dat
a B
uoy
Cen
ter
• Performed On:– NDBC and Regional Observatory Data
• Consists Of:– Automated Real-Time Checks
– Man-Machine Mix
• Outputs:– Withhold, adjust real-time data
• NOTICE: Users don’t see flags
– Archive Data Set
Data Quality Control
Nat
iona
l Dat
a B
uoy
Cen
ter Quality Control Process Flow
• Real-Time:– Message Integrity – Automated Hard-Flag– Automated Soft-Flag
• Post-Release, Pre-Archive: Data Analyst
Nat
iona
l Dat
a B
uoy
Cen
ter Quality Control of Wave Data
• No QC Done Onboard the Buoy• Handbook of Automated Quality Control Checks and Procedures of
the National Data Buoy Center
http://www.ndbc.noaa.gov/handbook.pdf• Automated QC Done in Real-Time at NWSTG• Hard-flags:
– Stop the Release and Archive of Data unless Analyst Overrides.• Override in advance – Storm Limits• Remove before archive
– Hierarchical: Order of Precedence, Fail One and Stop
• Soft-flags: Warning
Nat
iona
l Dat
a B
uoy
Cen
ter
Hard-Flagsby hierarchy - highest to lowest:
• T Transmission parity error• M Missing sensor data• W wave message is short, checksum or parity errors.• E Calculation limits are exceeded or are in error • D Delete measurement• S Invalid statistical parameter• V Failed time continuity. • L Failed range limits. • R Related measurement has failed a hard QC check.
Nat
iona
l Dat
a B
uoy
Cen
ter Time Continuity Check
• Dependent on time since last valid observation.
• Max. allowed value = 0.58*Std.dev.*SQRT(TimeDiff)
• Std. dev. Chosen 50% higher climatic standard dev. within a month
• Only Applied to Wave Height
Nat
iona
l Dat
a B
uoy
Cen
ter
If data fails time continuity,it may still be released if:
• Wave Height: Current wind speed > 15 m/s
• Storm Limits: Allows Release of Failed Time Continuity with Approach of a Hurricane, or Storm of the Century
Nat
iona
l Dat
a B
uoy
Cen
ter
The tie between range and time continuity checks
• If WVHGT fails for two consecutive hours, no measurements will be sent until analyst reviews (D-flag)
• Measurements should still be sent if it fails a range check, unless it failed a time continuity check the previous hour.
Nat
iona
l Dat
a B
uoy
Cen
ter Soft-flags for Waves
• a: Measurement is above monthly, regional limit.• b: Measurement is below monthly, regional limit.• c: Measurement has been adjusted, or corrected. • f: Measurement failed hourly time continuity. • m: High frequency spikes detected in the wave spectrum.• p: Failed wave height to wave period comparison test.• q: Swell direction is from an improbable direction. • w: Failed wind direction verses wave direction check.• x: Wind wave energy is too high for prevailing wind
speed.• y: Wind wave energy is too low for prevailing wind speed.
Nat
iona
l Dat
a B
uoy
Cen
ter Post-Release Analysis
• Wind vs Wave Energy Scatterplots
• Ad-hoc Comparisons with NCEP WWIII
• Check-Ratio at 0.1, 02., and 03. Hz
Nat
iona
l Dat
a B
uoy
Cen
ter Time Series
• Mean Acceleration or Displacement within Limits (L-flag) and Not Outside of Max/Min (S-flag)
Nat
iona
l Dat
a B
uoy
Cen
ter
NDBC QC Table
TIME SERIES VALUES DIR BUOY NON-DIR
RANGE CHECK (TS): verify that values fall within established ranges.
SPECTRAL VALUES DIR BUOY NON-DIR
MESSAGE CHECK: verify checksums, parity of transmission.
RANGE CHECK (SP): check range of spectral wave density values.
TIME CONTINUITY (SP): check continuity of spectral values over 1 hour.
PARAMETER VALUES DIR BUOY NON-DIR
RANGE CHECK (PM): check range of primary and ancillary parameters.
TIME CONTINUITY (PM): check continuity of parameter values over time.
LOW ENERGY: set parameters to zero for very low energy readings.
HEIGHT VS. PERIOD: check that Hs does not exceed limits for the given Ta.
WIND/WAVE ENERGY: checks that wave energy corresponds to measured wind.
WIND/WAVE DIRECTION: checks that wave, wind directions correspond.
SWELL DIRECTION: verifies that swell comes from non-sheltered directions.
CHECK FACTORS: range test of directional wave check factors.
VISUAL INSPECTION: data analyst reviews data.
Nat
iona
l Dat
a B
uoy
Cen
ter
NDBC Stations and DataPeering into the Future…
• More stations– More buoys and C-MAN stations (96 and 57)– DART buoys (from 6 to 39)– TAO buoys (55 + 4)
• More measurements: directional waves, current (surface or profile), CT (surface or profile), and other oceanographic and climate data.
• Multipurpose systems and stations• Data Management and Communication (DMAC) –
including NDBC Data Assembly Center• New technology and systems