ROMS 4D-Var: Secrets Revealed

Andy Moore

UC Santa Cruz

ROMS Obsy, R

fb, Bf

bb, Bb

xb, B

, Q

Posterior

4D-Var

Priors &Hypotheses

ClippedAnalyses

Ensemble(SV, SO)

HypothesisTests

Forecast

dof Adjoint4D-Var

impact

Term balance,eigenmodes

UncertaintyAnalysis

error

ROMS 4D-VarEnsemble

4D-Var

ROMS 4D-Var• Incremental (linearize about a prior) (Courtier et al, 1994)

• Primal & dual formulations (Courtier 1997) • Primal – Incremental 4-Var (I4D-Var) • Dual – PSAS (4D-PSAS) & indirect representer (R4D-

Var) (Da Silva et al, 1995; Egbert et al, 1994)

• Strong and weak (dual only) constraint• Preconditioned, Lanczos formulation of conjugate

gradient (Lorenc, 2003; Tshimanga et al, 2008; Fisher, 1997)

• Diffusion operator model for prior covariances (Derber & Bouttier, 1999; Weaver & Courtier, 2001)

• Multivariate balance for prior covariance (Weaver et al, 2005)

• Physical and ecosystem components• Parallel (MPI)

The Secrets…

• The adjoint operator

• Matrix-less iterations

• Lanczos algorithm

What is an adjoint?

3 7x

Can you solve this equation?

7 / 3x

1 23 7x x 1 22 3x x

1 211/ 5 2 / 5x x

Can you solve these equations?

1 2 33 2 27x x x 1 2 32 4x x x

Can you solve these equations?

There are infinite possibilities!

But…

…there is a Natural Solution

1

2

3

x

x

x

x3 1 2

1 2 1

A27

4

y

y Ax

1 22 1s s

1 2 37 0 3x x x

Tx A s 1

2

s

s

sLook for solutions of the form:

Generatingvector

Two Spaces

A is a 2×3 matrix

y and s reside in “2-space”

x resides in “3-space”

A maps from 3-space to 2-space

AT maps from 2-space to 3-space

y Ax Tx A s Ty AA s

Identifies the part of 3-space that is activated by y

AT is a 3×2 matrix

This looks lot like the ocean state estimation problem…

y Ax

Oceanstate

OceanObservations

Insufficient observations to uniquely determine elements of x

Use AT to identify the natural solution

Think of x as correctionsto the prior/background

1

2

3

x

x

x

x3 1 2

1 2 1

A27

4

y

y Ax

1 2 37 0 3x x x

y activates only x1 and x2

TAThe Adjoint

T

Sx

u

ζ

Notation

Ocean state vector:

Data Assimilation: Recap

Prior (background) circulation estimate:bx

Observations: yPosterior (analysis) circulation estimate:

( ) b bax K y Hxx

×

HObservation

matrix

GainMatrix

Data Assimilation: Recap

Posterior (analysis) circulation estimate:

bax x Kd

bd y HxInnovation vector:

model 1N bx obs 1N y obs modelN N

Adjoint plays a key role in identifying K

,y R

Data Assimilation: Recap

bb(t), Bb

fb(t), Bf

xb(0), B

Model solutions depends on xb(0), fb(t), bb(t), (t)

time

x(t)

Obs, y

Prior, xb

Posterior,xa

Prior

More Notation & Nomenclature

T

S

x u

v

ζ

(0)

( )

( )

( )

t

t

t

x

fz

b

η

1

N

y

y

y

bd y Hx

Statevector

Controlvector

Observationvector

Innovationvector

×

HObservation

matrix

Prior

(t)=0 : Strong constraint

(t)≠0 : Weak constraint

(t) = Correction for model error

Variational Data Assimilation: Recap

T T1 1( ) ( ) ( ) ( ) ( )

2 2J 1 1

b bx z z D z z y Hx R y Hx

diag( , , , ) b fD B B B Q

Background error covariance

( | ) expP J z yConditional Probability:

Problem: Find z=za that minimizes J (i.e. maximizes P)

za is the “maximum likelihood” or “minimum variance”estimate.

J is called the “cost” or“penalty” function.

Observation error covariance

( (0), ( ), ( ), ( ))T T T Tt t t b fz x ε ε η

initialconditionincrement

boundaryconditionincrement

forcingincrement

corrections for model

error

bb(t), Bb

fb(t), Bf

xb(0), B

1 11 1

2 2TTJ z D z G z d R G z d

diag( , , , ) b fD B B B Q

Background error covariance

TangentLinear Modelsampled atobs points

ObsErrorCov.

Innovation

bd y Hx

Prior

Incremental Formulation

Two Spaces

G maps from model (primal) space to observation (dual) space

GT maps from observation (dual) space to model (primal) space

G samples the tangent linear model at observation points.

1 11 1

2 2TTJ z D z G z d R G z d

Variational Data Assimilation

( (0), ( ), ( ), ( ))T T T Tt t t b fz x ε ε η

initialconditionincrement

boundaryconditionincrement

forcingincrement

corrections for model

error

bb(t), Bb

fb(t), Bf

xb(0), B

0J

z

Find z that minimizes J:

using principles of variational calculus (“Var”)

Prior

The Solution!

a bz z Kd

T T -1( ) K DG GDG R

Analysis:

Gain (dual):

Gain (primal):

( )T T 1 1 1 1K D G R G G R

Two Spaces

T T -1( ) K DG GDG R

Gain (dual):

Gain (primal):

( )T T 1 1 1 1K D G R G G R

obs obsN N

model modelN Nobs modelN N

Two Spaces

G maps from model (primal) space to observation (dual) space

GT maps from observation (dual) space to model (primal) space

Recall the Solutions

a bz z Kd

T T -1( ) K DG GDG R

Analysis:

Gain (dual):

Gain (primal):

( )T T 1 1 1 1K D G R G G R

Equivalent Linear Systems

T

T

( )

GDG R u d

v DG u

Dual:

Primal:

( )T T 1 1 1D G R G w G R d

Iterative Solution of Linear Equations

As r 0

General form:

Find s that minimizes:

T T1

2J s As s r

At the minimum:

J s 0 As r 0

Contours of J

Conjugate Gradient (CG) Methods

Looks, feels & smells like matrix…

T

T

( )

GDG R u d

v DG u

Dual:

Primal:

( )T T 1 1 1D G R G w G R d

Matrix-less Operations

TGDG δThere are no matrix multiplications!

Zonal shear flow

Matrix-less Operations

There are no matrix multiplications!

Adjoint Model

TGDG δ

Zonal shear flow

Matrix-less Operations

There are no matrix multiplications!

Adjoint Model

TGDG δ

Zonal shear flow

Matrix-less Operations

There are no matrix multiplications!

Adjoint Model

TGDG δ

Zonal shear flow

Matrix-less Operations

There are no matrix multiplications!

Covariance

TGDG δ

Zonal shear flow

Matrix-less Operations

There are no matrix multiplications!

Covariance

TGDG δ

Zonal shear flow

Matrix-less Operations

There are no matrix multiplications!

Tangent Linear Model

TGDG δ

Zonal shear flow

Matrix-less Operations

There are no matrix multiplications!

Tangent Linear Model

TGDG δ

Zonal shear flow

TGDG δ

A covariance

= A representer

Green’s Function

Zonal shear flow

Matrix-less Operations

Matrix-less Operations

There are no matrix multiplications!

Tangent Linear Model

TGDG δ

Zonal shear flow

The Priors for ROMS CCS

30km, 10 km & 3 km grids, 30- 42 levels

Veneziani et al (2009)Broquet et al (2009)

COAMPSforcing

ECCO openboundaryconditions

fb(t), Bf

bb(t), Bb

xb(0), B

Previous assimilationcycle

Observations (y)

CalCOFI &GLOBEC

SST &SSH

Argo

TOPP Elephant Seals

Ingleby andHuddleston (2007)

Data from Dan Costa

Primal, strong

Dual, strong

Dual, weak

Jmin

4D-Var Performance

3-10 March, 2003(10km, 42 levels)

The Beauty of Lanczos

CorneliusLanczos

(1893-1974)

Contours of J

Conjugate Gradient (CG) Methods

1 1 Tm m mCG A Q T Q

Recall the Solutions

a bz z Kd

T 1 T d d dK DG Q T Q

Analysis:

Gain (dual):

Gain (primal):

1 T T 1p p pK Q T Q G R

Dual Lanczosvectors

Primal Lanczosvectors

Endless Possibilties?

• Posterior error• Posterior EOFs• Preconditioning• Degrees of freedom• Array modes• Observation impact• Observation sensitivity• Think outside the box…

ROMS Obsy, R

fb, Bf

bb, Bb

xb, B

, Q

Posterior

4D-Var

Priors &Hypotheses

ClippedAnalyses

Ensemble(SV, SO)

HypothesisTests

Forecast

dof Adjoint4D-Var

impact

Term balance,eigenmodes

UncertaintyAnalysis

error

ROMS 4D-VarEnsemble

4D-Var

ROMS Obsy, R

fb, Bf

bb, Bb

xb, B

, Q

Posterior

4D-Var

UncertaintyAnalysis

error

ROMS 4D-Var

Expected Posterior Error

( ) aE I KG DT 1 T

m m mK DG Q T Q

Matrix of Lanczosvectors

( ) / b a b

m=number of inner-loops

b = prior error std

a = posterior error std

3 March, 2003

(10km, 42 levels)

SSSZonal wind stress

Meridional wind stress Heat Flux

Ritz # 0 1 2 3 4 8

J 9173 3131 1877 1855 1833 1972

Preconditioning

J J J

1st levelpreconditioning

2nd levelpreconditioning

4X25

As=s

ROMS Obsy, R

fb, Bf

bb, Bb

xb, B

, Q

Posterior

4D-Var

Priors &Hypotheses

HypothesisTests

ROMS 4D-Var

degrees of freedomdegrees of reachabilityarray modes

Degrees of Freedom

1 11 1

2 2TTJ z D z G z d R G z d

Recall that the optimal increments minimize:

No. of dof in obs

No. of dof in prior

“dof” – degrees of freedom

min obs / 2J NTheoretical min:

min( ) Tr( ) / 2bJ KG

min obs( ) ( - Tr( )) / 2oJ N KG

(Bennett et al, 1993; Cardinali et al, 2004; Desroziers et al., 2009)

bJ oJ

Assimilation cycle (2003-2004)

Lo

g10

(J)

dofof

obs

(30km, 30 level, dual, strong, sequential, 7 day, 200 inner-loops)

• Less than 10% of all observations provide independent info• LOTS OF REDUNDANCY!

Degrees of Freedom

2obsN

1min

1( ) ( Tr( ))

2b obsJ N dT

Degrees of Freedom

min( ) Tr( ) / 2bJ KG

Degrees of freedom in the obs:

T 1 T d d dK DG Q T QBut:

So:

1min

1

1( )

2

obsN

b obs ii

J N

Degrees of Freedom

min( ) Tr( ) / 2bJ KG

Degrees of freedom in the obs:

T 1 T d d dK DG Q T QBut:

So:

Computed directly during 4D-Var

Computed from a curve fit

310log ( ) , 1.684, 2.7 10bi

i ae a b

Degrees of Freedom

Observation Impacts

Consider a scalar function of the ocean state vector:

( )I I x

( )I Ib bx ( )I Ia ax I I I a b

Prior Posterior Increment

T T T ( )I I d K M x

Innovations

Adjoint ofgain matrix

Adjoint model

Observation Impacts

But:T 1 T d d dK DG Q T Q

So:

T 1 T d d dK Q T Q GD

Observation Sensitivity

The gain matrix is actually a nonlinear function of d:

( ) a bx x dKConsider variations in the observation vector y:

ax yy

K

Relationship to obs impact:

Linearizationof 4D-Var

e.g.: 37N transport

T T

T

I I I

a b vd M h

y

KAdjoint of 4D-Var

Assimilation impacts on CC

No assim

PrimalStrong

Time meanAlongshore

Flow(10km, 42 lev)

CC

CC

CUC

CUC

CC = California CurrentCUC = California Under Current

37N Transport: Impact vs Sensitivity

Impact vs Sensitivity: 37N Transport

Impact

Sensitivity

Two Spaces: Obs Impact

maps from model (primal) space to observation (dual) space

maps from observation (dual) space to model (primal) space

Identifies the part of model space that controls 37N transportand that is activated by the observations

K

TK

Two Spaces: Obs Sensitivity

maps from model (primal) space to observation (dual) space

maps from observation (dual) space to model (primal) space

Identifies the part of model space that controls 37N transportand that is activated by the observations during 4D-Var

yK

T yK

Observing System Experiments (OSEs)

Change in 37N transportwhen Argo withheld

Change in 37N transportwhen SSH withheld

Summary

• The adjoint operator

• Matrix-less iterations

• Lanczos algorithm

Most of the power and utility of modern data assimilation systems is possible because of: