Post on 31-Dec-2015
description
transcript
GODAE Quality Control Pilot ProjectGODAE Quality Control Pilot Project
Status Report August 2007
IGST Meeting
Department Fisheries Ocean
St. John’s, New Foundland, Canada
Outline
1. New Data Sources
2. Output File and Reference Data
3. Plans and Issues
4. Argo FSI Float QC Outcomes
New Data Sources
• UKMO– long list form of UKMO QC output now available in real-time– UKMO back filled long list data to 2003 – thanks to Matt Martin– long list provides additional QC outcomes and both in situ and
potential temperatures (short list had only potential temperatures)– convention is to use in situ temperature in QC databases
• in situ calculated from potential when salinity reported• not always the case in the UKMO short list files
• BMRC– pushing QC data from BoM system to GODAE server– currently have data for 2005 and 2006 (nothing from 2007 yet) – working on BMRC data decoder
GODAE Quality Control Pilot ProjectGODAE Quality Control Pilot Project
New Output File Format and Reference Data
• Call sign database conversion to NetCDF format complete– based on Coriolis/Argo multiple profile NetCDF file structure – easily expanded with new QC variables or new QC data providers– sorted by observation time and data center for ease of use
• Navy GDEM climate and climate variability profiles added as reference profiles in call sign database– temperature and salinity climatology
– global, monthly, 0.25 degree resolution
– 78 vertical levels (47 in upper 1000 m of water column)
– climate mean and standard deviations interpolated to observed levels for month in which observation taken
GODAE Quality Control Pilot ProjectGODAE Quality Control Pilot Project
New MEDS QC Variable in Call Sign Database • decoded MEDS hex bit codes on QC tests performed
Meds QC Tests (binary string bit position from the RIGHT): Bit # - Test 0 - Platform Identification 1 - Impossible Date/Time 2 - Impossible Location 3 - Position on Land 4 - Impossible Speed 5 - Impossible Sounding 6 - Global Impossible Parameter Values 7 - Regional Impossible Parameter Values 8 - Increasing Depth 9 - Profile Envelop 10 - Constant Profile 11 - Freezing Point 12 – Spike 13 - Top and Bottom Spike14 – Gradient 15 - Density Inversion 16 - Levitus Seasonal Clim17 - Emery and Dewar Clim 18 - Asheville Clim 19 - Levitus Monthly Clim 20 – Waterfall 21 - Cruise Track 22 – Profiles23 – Bottom 24 - Temperature Inversion
• experience indicates only two tests flagged as typically performed– Levitus seasonal climate check and temperature inversion
GODAE Quality Control Pilot ProjectGODAE Quality Control Pilot Project
Plans • initialize call sign databases with 2003-2006 production run
– NetCDF call sign databases will be hosted on US GODAE server– target date for completion Oct 2007
Issues• US GODAE server hardware upgrade has never taken place
– IBM X-346 machines plus 30 TB storage purchased years ago by ONR– never installed due to security issues – completion date still unknown
• GODAE QC application cannot run on existing server hardware– cannot do real-time updates of call sign databases as planned– in process of purchasing additional disk space for personal Linux
workstation to support GODAE QC function– using my Linux workstation (at best) only monthly updates of NetCDF call
sign databases on GODAE server possible
GODAE Quality Control Pilot ProjectGODAE Quality Control Pilot Project
FSI Argo Floats Pressure Offset Errors
• discovered in February, 2007 that Argo profiles from SOLO floats with FSI CTD (Argo Program WHOI) have incorrect pressures – profile data may be offset upward by one or more pressure levels,
resulting in a significant cold bias for the data– a data denial list was posted on the Argo web site (287 WMO IDs)– data no longer distributed on GTS
• Argo science team asked if bad FSI float data had been accepted into operational assimilation systems
• FSI float QC outcomes from the NRL/FNMOC real-time QC system were evaluated from 2004 through 2006– instrumentation error checks– climate background field check– cross validation (buddy) check
GODAE Quality Control Pilot ProjectGODAE Quality Control Pilot Project
float track
error probability of entire profile
(reject if p > 0.95)
GDEM climate anomaly
cross validation anomaly
error probability of profile level
(reject if p > 0.95)
NRL FSI Float QC Outcomes Display
Summary NRL/FNMOC QC of FSI Argo Float Pressure Errors
• rejected 11% of the float temperature profiles
• rejected 5% of the float temperature levels
• 185 float profiles rejected due to inadequate vertical sampling as a result of the vertical level rejections (created gaps in profiles)
Positive feedback from Howard Freeland on call sign databases
• From: Freeland, Howard [mailto:FreelandHj@pac.dfo-mpo.gc.ca] Sent: Wednesday, August 01, 2007 2:58 PMTo: Cummings, James [cummings@nrlmry.navy.mil]Subject: Rejected floats
I think the call sign files are exactly what I'd like to look at. I've been doing some simple investigations about data being rejected by less sophisticated users than GODAE people and have some interesting results...
GODAE Quality Control Pilot ProjectGODAE Quality Control Pilot Project
END
dac source of data (FNMOC, NAVO, MEDS, Coriolis, UKMO, BMRC)
dtg observation time
dtg_qc quality observation time
lat, lon observation position
pos_qc quality observation position
n_lvl number levels
lvl observation depth levels
sal observed salinity
sal_fcst salinity forecast
s_fcst_err salinity forecast error
sal_clm salinity climatology
s_clm_err salinity climate variability
sal_qc salinity profile quality
Call Sign Database Structure Call Sign Database Structure
sal_qc_lvl salinity level quality
sal_flg salinity QC test failures
tmp observed temperature
tmp_fcst temperature forecast
t_fcst_err temp forecast error
tmp_clm temperature climate
t_clm_err temp climate variability
tmp_qc temperature profile quality
tmp_qc_lvl temperature level quality
tmp_flg temperature QC test failures
sign observation call sign
upd_dtg profile update time at center
qc_conflict flag indicating QC outcomes differ among data centers
NRL MEDS CoriolisUKMO
GODAE QC Application Program Interface (API) read different QC data formats, check for valid call sign, match
profiles (lat, lon, time, call sign), populate database structure with observation, QC codes, QC flags, and supplemental information, sort by observation time, update
NetCDF call sign database
1900305 Argo Float
A8ES9 Ship XBT
52007 TAO Mooring
Call Sign Databases
BMRC