Observations Preprocessing Carla Cardinali

ECMWFTraining course 2005 slide 1

Observations Preprocessing

Carla Cardinali


Observations preprocessing

MakeCMA

Screening

Minimization

MatchUp

FeedBack

Forecast

ECMA

CCMA

ECMA

BUFR

ODB

BUFR



ECMA/ODB

CCMA/ODB

Output BUFRs



What is ODB?

Developed software at ECMWF to manage large amounts of satellite data.

ODB/SQL language is a subset of ANSI/SQL query language. Compiler translates

ODB/SQL in C-codes files

How do use it?

Access to ODB database is through Fortran90 modules functions. A ECMA data

base contains table with column entries lat, lon, date, time observation type

which is used to locate observation at a given moment

Obs. Ident ( sat. id, lat, lon, st.alt., date, time)

Obs. V. (wind, temp.,…per pressure) and (radiances per ch., inst. type)

Various flag: active, blacklisted….

Departures (obs-background, obs-analysis)



Incoming Observation

The observations arrive at ECMWF through GTS and they are stored in a

decoded format in Report Data Base after some rudimentary quality control e.g.

observation format and position, climatological and hydrostatic limit and

temporal consistency. An observation file suitable for assimilation is created

6-hour Preprocessing Observation Array

Format conversions, change of some observed variables (relative humidity

from dry and wet bulb temperature) and assignment of observation error

statistics


Observations Screening

Select the “best” observation

At the first trajectory run the model counterparts for all the observations are

calculated through the nonlinear observation operators. For the observation

screening, the background errors are interpolated to the locations of the

observed variables. The "extended" observation array (ECMA) contains

observations complemented by the background departures and quality control

information for most of the observations.

This array is stored for later feedback.

After the screening a "compressed"

array is passed to the minimization

(CCMA).


Independent Observations Screening

The screening logic is to make first those decisions that are not

depending on any other ones

Preliminary checks: completeness of the report

Blacklist

Data Selection: which observation types, variables, vertical ranges will be used

in the assimilation.

Monthly selection: discarding stations that have been reporting excessively

noisy or biased compare to the background field.

Impact studies.


Independent Screening Decision: Conventional Observations

Background Quality Control

The BgQC is applied to all the variables that are intended to be used in the

assimilation

BgQC of wind observations is done simultaneously for both wind components. For wind direction the error limits of 60, 90 and 120 apply for flags 1,2 and 3

2 2

3

2

12

o b b

Hx y

uo

ub


Independent Screening Decision: Conventional Observations


When is not working


Independent Screening Decision: Satellite Observation

Bias Correction

Bias correction coefficients are recomputed from the past 2/4 weeks of

departure statistics. The feedback files are used for monitoring the performance

of the observing and assimilation system: e.g. ATOVS and SSMI radiances,

SCATT wind.

If the removed bias is a model forecast bias, the subsequent assimilation will enforce it. Usually, only half bias is removed.

Gross check Measured and background brightness temperatures are present for all required channels. Bias correction coefficients, satellite id, and scan position are all valid before proceeding. For all channels a cloud detection is performed

290K

230K


Dependent Screening Decision: Satellite Observation

HIRS

3

2

1

land +oceanChannels below rejected

SSMI

ocean: LWP=f(ch3/4) If LWP > cloud+rain


Dependent Screening Decision: Satellite Observation

ocean: LWP=f(ch1,ch2,ch3)If LWP > cloud+rainland: if (ch1-ch3) > rain because of large emissivity over land no cloud contamination detection

1 3

Tb

no cloud

cloud

rain

AMSU-A

12

3

4

5

6



Background Quality control: ATOVS, SSMI, METEOSAT

Background temperature, specific humidity and

ozone profiles are checked to make sure they are close to or within the range

for which the radiative transfer model is valid. Temperature is within the range

150-350 K, specific humidity is positive and not supersaturated and the ozone

is within climate extremes.

Radiance at the 2 extreme edge positions of the swath are not used in 4D-Var

O Rad RadT

b O HBH2






Dependent Screening Decision: Conventional Observation

Before performing the dependent screening decisions, the flag information

gathered so far is converted into a report status, namely active, passive,

rejected or blacklisted i.e the RDB datum flag.

Vertical consistency of multilevel reports

Duplicated levels are removed. If some consecutive layers are of suspicious

quality they are rejected and for geopotential obs. also the layer above

Removal of duplicated report

Search globally for duplicated (AIREP) data. Usually, there are aircraft reports

that have the same date/time, roughly the same location and, at least partially,

the same data. Usually, the station ID is slightly different.



Redundancy check

Removes redundant SYNOP/PAOB :

co-located reports with the same station ID are searched for and only the

closest active report to the analysis time is retained. For simultaneous

reports, the one with more active data is retained. The same is done for

reports that have equal time difference to the analysis time. Co-located

SYNOP (mass obs) is redundant to TEMP (geop.) that is within 50 hPa

Removes redundant SHIP/DRIBU:

moving platforms within a circle of 1 ° are considered as potentially

redundant: reports closest to the centre of the screening time with most

active date are retained.


Dependent Screening Decision: SYNOP

The effect of the observation screening on SYNOP surface pressure observations. Column height gives the number of observations available, while the shaded part displays those actually used in the assimilation.

4D-Var Screening for 4D-Var

3D-Var Screening for 3D/4D-Var



Level selection for TEMP/PILOT redundancies:

for one time window rejection in layers around std-levels according the

following priorities

maximum datum flag

time difference to analysis time

distance to the std-level

significant level (turning points of the sounding)

temp over pilot


Dependent Screening Decision: Thinning

AIREP thinning

horizontal thinning

from the same platform, a minimum distance between the nearby reports is

enforced: Box 125 62 km

vertical thinning

is performed around the model vertical levels: one aircraft measurement

per model level.



VERT increased number of AIREPfrom 15 to 60 vertical levels

+

+HOR also increased number of AIREP in horizontal


Screening Decision: Thinning

TEMP

GTSObs

thin



ATOVS thinning: a repeated scan is performed to get the

observation resolution of 70 km (it depends on sensors/channels)

a sea sounding is preferred to a land one

clear sounding to a cloudy one

closeness of observation time to the centre of the screening

time window

A second thinning takes place that selects one observation every

140km



The usage of ATOVS reports in the assimilation on the North Eastern Atlantic. Filled rings mark reports contain one or more channels used in the assimilation, whereas the empty rings denote rejected reports. Most of the rejections are due to the horizontal thinning and much less due to the quality reasons.Note that both edges of the swath are rejected.



SCAT:ERS-2

The process is defined with respect to the particular measurement geometry of

the instrument. The backscatter data are acquired within individual cells related

to a 450 km wide grid with a mesh of 25 km in the across and along track

directions. 19 measurement nodes are thus defined across the scatterometer´s

swath, numbered from 1 to 19 as the incidence angle increases, while 19 rows

are also considered in the along track direction to gather the data in squares of

19 by 19 points. The thinning is then achieved by keeping only every fourth

point within these squares. The data are thus used at a resolution of 100 km

instead of the original 25 km sampling distance.

Sea-ice contamination < 273K

High wind rejection test > 25m/s

Normalized distance to the cone or wind residual. During the assimilation this quantity is computed and data rejected if a large value is found.



Triplet obs

.. ..

.. ..

..

.. ..……....

.. 3 .. .. .. 7

check the distancebetween observations

1-step

2-step


QuikSCAT

50 km

Courtesy of Hans HersbachLars Isaksen Mark Leidner

Quadruplet obs



162

1

()(())OBSMOD

iii

MLE

uu

High resolution trajectory to choose only one



30°N30°N

40°N 40°N

50°N50°N

180°

180° 170°W

170°W

Obs$rv$d winds abov$ 1. m/sAmbig. clos$st to ana, from 2001 0518 0412 to 2001 0518 0733QSCAT 50km ($6dr): data cov$rag$

30°N30°N

40°N 40°N

50°N50°N

180°

180° 170°W

170°W

Rej: only 1 beam measur.active

Minima are not well defined

Var-QC

White areas: insuf. info for MLE inversion e.g rain contamination


Screening Statistics

281515286821701848026058052492792140

Reports Active Passive

SynopAirepSatobDribuTempPilotSatemPaobScat

Obs. Type

2421829969405001708602730558631380

Rejected

3933228991765183094375193416760

Blacklisted

00100699007154867760


After Screening

Compression of CMA-file

After screening only 15% of all observations are active

10-20% TOVS left

40% Conventional left

The observation are resorted among the processors for a more

optimal load balancing of the parallel computer


After Screening

Parallel computing environment

The observation screening should result in exactly the same selection of

observations when different number of processors are used. Independent

decision can be made at different processor fully in parallel. But global view of

observation array is needed when a dependent decision has to be taken which

implies that some communication between the processors is required. The

observation array is too large to be copied in each individual processor then

only a minimum necessary information is globally communicated

Global Array

Time

IDLocation 1

2

Date post:	13-Jan-2016
Category:	Documents
Upload:	brook
View:	27 times
Download:	0 times

Observations Preprocessing Carla Cardinali

Documents