Data Assimilation with Overlapping Windows · Data Assimilation with Overlapping Windows Yannick Tr...

Data Assimilation with Overlapping Windows

Yannick Trémolet, Elias Hólm

ECMWF

ISDA, Reading, July 2016

Y. Trémolet Overlapping Window 4D-Var

Long Window Weak Constraint 4D-Var

��

��

�

��

��

��

(1) Weak constraint 4D-Var

(2) Extended window

��

��

�

��

��

��

� ��

(3) Initial term has converged (4) Assimilation window is moved forward

• This implementation is an approximation of weak constraint 4D-Var with anassimilation window that extends (almost) indefinitely in the past...

• ...which is equivalent to a full rank Kalman smoother (Fisher et al., 2005,Ménard and Daley, 1996) that has been running indefinitely.

• In principle B is a problem of the past, only the error characteristics of thefundamental ingredients of the DA problem remain.

Y. Trémolet Overlapping Window 4D-Var 1 / 12


��

��

�

��

��

��

(1) Weak constraint 4D-Var (2) Extended window

��

��

�

��

��

��

� ��







��

��

�

��

��

��


��

��

�

��

��

��

� ��

(3) Initial term has converged

(4) Assimilation window is moved forward






��

��

�

��

��

��


��

��

�

��

��

��

� ��






We are not there yet...

Weak constraint 4D-Var is difficult to implement in the IFS (both for scientificand technical reasons).

1. Overlapping windows with strong constraint 4D-Var

2. Results in reanalysis context

3. Results in NWP context

4. Comments about re-use of observations


Operational System: Observations Delay

• Observations are not availableinstantly

• Operational systems work with along-ish cut-off

• Issue: not all observations that havearrived are used!

• An early-delivery assimilation is run

• When cycling, information from theearly delivery run is lost


Overlapping System: Observations Delay

• A shorter cut-off is used

• The next overlapping cycle will usethe observations that have arrived inbetween

• No observations are lost

• The cost is the same: 4 × 12h4D-Var vs. 2 × 12h 4D-Var + 4 ×6h 4D-Var

• It is possible to use newly arrivedobservations only

• This algorithm is a quasi-smoother


Future use of long windows

time

• The assimilation window can be moved in time frequently, for example by 15minutes every 15 minutes.

• Observations that have arrived in the last 15 minutes are added.– Very short cut-off time, without losing observations.

• Because of the large overlap, few inner iterations are needed at each stage– but many effective outer iterations.

• The assimilation becomes a service that runs continuously:– an up-to-date analysis is always available,– less pressure on the time critical path,– can be optimised for energy consumption rather than time to solution,– no peak in daily computer usage (smaller machine).


Overlapping Windows

• The assimilation needs updating several times daily: long assimilationwindows will overlap.

time

time

xb

– The background and observationerrors are uncorrelated

– The background is not the most upto date

– Two DA streams runningindependently

time

xb

– The background is the mostaccurate

– The background and observationerrors are correlated

• In practice, using the most recent background gives the best results.


Overlapping Windows

• The assimilation needs updating several times daily: long assimilationwindows will overlap.

time

xb

– The background and observationerrors are uncorrelated

– The background is not the most upto date

– Two DA streams runningindependently

time

xb

– The background is the mostaccurate

– The background and observationerrors are correlated

• In practice, using the most recent background gives the best results.


Ps-only Re-analysis

Background and Analysis fit to Observationsthroughout the assimilation window

2004-07-01 to 2005-04-09

0 3 6 9 12 15 18 21 24

Time in assimilation window (h)

0.5

1.0

1.5

2.0

Ps

obse

rvat

ion

fit(h

Pa)

Overlapping 24h 4D-Var 24h 4D-Var (ERA 20C) 12h 4D-Var

Paul PoliY. Trémolet Overlapping Window 4D-Var 7 / 12

Ps-only Re-analysis

Background and Analysis fit to Observationsthroughout the assimilation window

2004-07-01 to 2005-04-09

0 3 6 9 12 15 18 21 24

Time in assimilation window (h)

0.5

1.0

1.5

2.0

Ps

obse

rvat

ion

fit(h

Pa)

12h


Paul PoliY. Trémolet Overlapping Window 4D-Var 7 / 12

Ps-only Re-analysis

Forecast scores vs. operational analysisZ500, NH, 2004-07-01 to 2005-04-09

0 1 2 3 4 5 6 7 8 9 10

Forecast Range (days)

20

30

40

50

60

70

80

90

100

Ano

mal

yC

orre

latio

n(%

)


• Verification against independent (unused) observations:– confirms positive results with overlapping windows,– shows that 24h 4D-Var without overlap is slightly better than 12h 4D-Var.


Use of Observations (full system)

• Overlaping assimilation windows implies that observations are used twice.

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0Std. dev.

0

5

10

15

20

25

30

35

40

45

Heig

ht

(km

)

GPSRO North Hemis.

12h 4D-Var24h 4D-Var (12-24h)24h 4D-Var (0-12h)

– The background fit the firsttime observations are seen issimilar to the 12h 4D-Varcontrol

– The analysis fit is equal orbetter than in 12h 4D-Var thesecond time observations areused

– It is possible to fitobservations at least as welleven with more constraints


Forecast Performance

• For operational use: 12h4D-Var every 6 hours

• Flow dependent B from6-hour EDA

• In practice, re-usingobservations gives betterperformance

• Not re-using observationswould require replacing Bby an analysis errorcovariance matrix

−0.2

−0.1

0.0

0.1

0.2

Diffe

rence in a

nom

aly

corr

ela

tion n

orm

alis

ed b

y 1

−A

C

T+24

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

T+48

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

T+72

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

T+84

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

T+96

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

T+120

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

T+144

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

T+168

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

T+192

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

T+216

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

New obs. only - obs. re-use, JJA 2014


Forecast Performance

• For operational use: 12h4D-Var every 6 hours

• Flow dependent B from6-hour EDA

• Without any change in B,comparison betweenoverlapping windows andearly delivery is mixed

• B will be re-tuned for theoverlapping system

−0.2

−0.1

0.0

0.1

0.2

Diffe

rence in a

nom

aly

corr

ela

tion n

orm

alis

ed b

y 1

−A

C

T+24

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

T+48

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

T+72

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

T+84

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

T+96

−90 −60 −30 0 30 60 90Latitude

1000

700

400

100

10

1

Pre

ssure

, hP

a

T+120

Overlap - Early delivery, JJA 2014


Background-Observations Errors Correlation

• Results improve when re-using observations

• Other errors in B and R might be (much) larger than the missingcross-correlation term

• Observation errors could be adjusted (Cf. talk Marc Bocquet)?

• The state used as the background is in fact already an analysis– B should be replaced by an analysis error covariance matrix

(the EDA can be modified for this)– Keeping both B and the observations anchors the guess where it has already

been pushed: (Jb + Jo)min acts as a Ja weight– Keeping B but removing the observations at the start of the window creates

an information leak (not enough weight on an accurate prior)

• In the long term, with weak constraint 4D-Var, there is no B, only Q– No issue of cross-correlation– Viewing B as an approximation of Q, ignoring cross correlations is sensible


Date post:	13-Feb-2021
Category:	Documents
Upload:	others
View:	5 times
Download:	0 times

Data Assimilation with Overlapping Windows · Data Assimilation with Overlapping Windows Yannick Tr...

Documents