Ensemble prediction and strategiesEnsemble prediction and strategiesfor initialization: Tangent Linear andfor initialization: Tangent Linear and
Adjoint Models, Singular Vectors,Adjoint Models, Singular Vectors,Lyapunov Lyapunov vectorsvectors
Eugenia KalnayLecture 2
Alghero, May 2008
Elements of EnsembleElements of EnsembleForecastingForecasting
• It used to be that a single control forecast was integrated fromthe analysis (initial conditions)
• In ensemble forecasting several forecasts are run from slightlyperturbed initial conditions (or with different models)
• The spread among ensemble members gives information aboutthe forecast errors
• How to create slightly perturbed initial conditions?• Basically
– Singular Vectors– Bred Vectors– Ensembles of data assimilation (perturbed obs. EnKF)
In another fundamental paper, Lorenz (1965)introduced (without using their current names) theconcepts of:Tangent linear model, Adjoint model,Singular vectors, and Lyapunov vectorsfor a low order atmospheric model, and theirconsequences for ensemble forecasting.
He also introduced the concept of“errors of the day”: predictability is not constant:It depends on the stability of the evolvingatmospheric flow (the basic trajectory or referencestate).
When there is an instability, all perturbations convergetowards the fastest growing perturbation (leading
Lyapunov Vector). The LLV is computed applying thelinear tangent model on each perturbation of the nonlinear
trajectory Fig. 6.7: Schematic of how all perturbations will converge towards the leading Local Lyapunov Vector
trajectory
random initial
perturbations
leading local
Lyapunov vector
Tangent linear model (TLM) and adjointmodels
dx
dt= F(x), x =
x1
.
.
xn
!
"
####
$
%
&&&&
, F =
F1
.
.
Fn
!
"
####
$
%
&&&&
a nonlinear modeldiscretized inspace: ODE’s
x
n+1= x
n+ !tF(
xn+ x
n+1
2)
discretized in time: aset of nonlinearalgebraic equations
“Running the model” is integrating the model fromtime to to time t
This is thenonlinear model
x(t) = M (x(t0))
M (x(t
0) + y(t
0)) = M (x(t
0)) +
!M
!xy(t
0) + O(y(t
0)2 ) = x(t) + y(t) + O(y(t
0)2 )
We add a small perturbation y(to) and neglect terms O(y2)
y(t) = L(t
0,t)y(t
0) This is the tangent
linear model (TLM)
L(t
0,t) =
!M
!x
L is an nxn matrix that dependson x(t) but not on y(t)
Euclidean norm of a vector: inner product of avector with itself
Size of y (Euclidean norm)
Define adjoint of an operator K:
y2= y
Ty =< y,y >
< x,Ky >!< KTx, y >
In this case of a model with real variables, the adjointof the TLM is simply the transpose of the TLM
L
T (t0,t)
Now separate the interval (to,t) into two successive intervals
t0< t
1< t L(t
0,t) = L(t
1,t)L(t
0,t1)
Transposing we get the adjoint:the last segment is executed first!
LT(t0,t) = L
T(t0,t1)L
T(t1,t)
The adjoint of the model can also be separated into singletime steps, but they are executed backwards in time,starting from the last time step at t, and ending with thefirst time step at t0.
For low order models the tangent linear model and itsadjoint can be constructed by repeated integrations of thenonlinear model for small perturbations, as done by Lorenz(1965), and by Molteni and Palmer (1993) with a globalquasi-geostrophic model.
For large NWP models this approach is too timeconsuming, and instead it is customary to develop thelinear tangent and adjoint codes from the nonlinear modelcode following some rules discussed in Appendix B.
A very nice example of the TLM and ADJ code generationfor the Lorenz (1963) model is given by Shu-Chih Yang:
Advanced data assimilation methods withevolving forecast error covariance:
4D-VarExample of TLM and ADJ code with
Lorenz (1963) model
Shu-Chih Yang (with EK)
Example with the Lorenz 3-variable model
• The background state is needed in both L and LT
(need to save the model trajectory)• In a complex NWP model, it is impossible to write
explicitly this matrix form
dx1
dt= !px
1+ px
2
dx2
dt= rx
1! x
1x3! x
2
dx3
dt= x
1x2! bx
3
Nonlinear model x=[x1,x2,x3]
Tangent linear model δx=[δx1, δx2, δx3]
Adjoint model δx*=[δx*
1, δx*2, δx*
3]
L =!M
!x=!M
!xi
=
"p p 0
r " x3
"1 "x1
x2
x1
"b
#
$
%
%
%
&
'
(
(
(
x3 x1
x2 x1
LT
=!M
!xi
"
# $
%
& '
T
=
(p r ( x3
x2
p (1 x1
0 (x1
(b
"
#
$
$
$
%
&
'
'
' x1
x1
x2x3
In tangent linear model
forward in time
We will see that in the adjoint model the above line becomes
backward in time
Example of tangent linear and adjoint codes (1)
!x3(t + "t) # !x
3(t)
"t= x
2(t)!x
1(t) + x
1(t)!x
2(t) # b!x
3(t), or
!x3(t + "t) = !x
3(t) + [x
2(t)!x
1(t) + x
1(t)!x
2(t) # b!x
3(t)]"t
!x3
*(t) = !x
3
*(t) + (1" b#t)!x
3
*(t + #t)
!x2
*(t) = !x
2
*(t) + (x
1(t)#t)!x
3
*(t + #t)
!x1
*(t) = !x
1
*(t) + (x
2(t)#t)!x
3
*(t + #t)
!x3
*(t + #t) = 0
* Try an example in Appendix B (B.1.15)
use forward scheme to integrate in time
Tangent linear model,
forward in time
Example of tangent linear and adjoint codes (2)
!x3(t + "t) = !x
3(t) + [x
2(t)!x
1(t) + x
1(t)!x
2(t) # b!x
3(t)]"t
use forward scheme to integrate in time
!x3(t + "t)
!x1(t)
!x2(t)
!x3(t)
#
$
%%%%
&
'
((((
=
0 x2(t)"t x
1(t)"t 1) b"t( )
0 1 0 0
0 0 1 0
0 0 0 1
#
$
%%%%
&
'
((((
!x3(t + "t)
!x1(t)
!x2(t)
!x3(t)
#
$
%%%%
&
'
((((
We have to write for each statement all the “active” variables.
Then we transpose it to get the adjoint model
Tangent linear model,
forward in time
Example of tangent linear and adjoint codes (3)
!x3(t + "t) = !x
3(t) + [x
2(t)!x
1(t) + x
1(t)!x
2(t) # b!x
3(t)]"t
!x3(t + "t)
!x1(t)
!x2(t)
!x3(t)
#
$
%%%%
&
'
((((
=
0 x2(t)"t x
1(t)"t 1) b"t( )
0 1 0 0
0 0 1 0
0 0 0 1
#
$
%%%%
&
'
((((
!x3(t + "t)
!x1(t)
!x2(t)
!x3(t)
#
$
%%%%
&
'
((((
!x3
*(t + "t)
!x1
*(t)
!x2
*(t)
!x3
*(t)
#
$
%%%%
&
'
((((
=
0 0 0 0
x2(t)"t 1 0 0
x1(t)"t 0 1 0
1) b"t( ) 0 0 1
#
$
%%%%
&
'
((((
!x3
*(t + "t)
!x1
*(t)
!x2
*(t)
!x3
*(t)
#
$
%%%%
&
'
((((
Adjoint model: transpose of the linear tangent, backward in time
Execute in reverse order
Example of tangent linear and adjoint codes (4)!x
3(t + "t) = !x
3(t) + [x
2(t)!x
1(t) + x
1(t)!x
2(t) # b!x
3(t)]"t
!x3
*(t + "t)
!x1
*(t)
!x2
*(t)
!x3
*(t)
#
$
%%%%
&
'
((((
=
0 0 0 0
x2(t)"t 1 0 0
x1(t)"t 0 1 0
1) b"t( ) 0 0 1
#
$
%%%%
&
'
((((
!x3
*(t + "t)
!x1
*(t)
!x2
*(t)
!x3
*(t)
#
$
%%%%
&
'
((((
Adjoint model: transpose of the linear tangent, backward in time
Execute in reverse order
In adjoint model the line above becomes
backward in time
!x3
*(t) = !x
3
*(t) + (1" b#t)!x
3
*(t + #t)
!x2
*(t) = !x
2
*(t) + (x
1(t)#t)!x
3
*(t + #t)
!x1
*(t) = !x
1
*(t) + (x
2(t)#t)!x
3
*(t + #t)
!x3
*(t + #t) = 0
There are automatic compilers to create linearThere are automatic compilers to create lineartangent and adjoint models (e.g., TAMC)tangent and adjoint models (e.g., TAMC)
• It is still a long and painful process…• Creating and maintaining an adjoint model is the
main disadvantage of 4D-Var• On the other hand, the adjoint provides tremendous
possibilities:– 4D-Var– Sensitivity of the results to any parameter or initial conditions
(e.g., sensitivity of forecasts to observations, Langland andBaker 2004)
– And many others– Ensemble Kalman Filter allows doing the with the nonlinear
model ensemble, without the need for an adjoint
Singular VectorsSingular Vectors: the theory is too long, so we: the theory is too long, so we’’lllljust give a basic idea (see section 6.3 of book)just give a basic idea (see section 6.3 of book)
y(t
1) = L(t
0,t
1)y(t
0) Remember the TLM L(to, t1) evolves
the perturbation from to to t1
UTLV = S Singular Value Decomposition (SVD)
S =
!1
0 . 0
0 !2
. 0
. . . .
0 0 . !n
"
#
$$$$$
%
&
'''''
S is a diagonal matrix whoseelements are the singularvalues of L.
Singular Vectors: the theory is too long, so weSingular Vectors: the theory is too long, so we’’lllljust give a basic ideajust give a basic idea
y(t
1) = L(t
0,t
1)y(t
0) Remember the TLM L(to, t1) evolves
the perturbation from to to t1
UTLV = S Singular Value Decomposition (SVD)
S =
!1
0 . 0
0 !2
. 0
. . . .
0 0 . !n
"
#
$$$$$
%
&
'''''
S is a diagonal matrix whoseelements are the singularvalues of L.
and UUT= I; VV
T= I
Left multiply by U:
where vi and ui are the vector columns of V and Uvi are the initial singular vectorsui are the final singular vectors
The initial SV gets multiplied by thesingular value when L is appliedforward in time
LV = US, i.e., L(v
1,...,v
n) = (!
1u
1,...,!
nu
n)
Lvi= !
iui
Right multiply by VT:
The final SV gets multiplied by thesingular value when LT is appliedbackward in time
UTL = SV
T and transposing,
LTU = VS, i.e., L
T(u
1,...,u
n) = (!
1v1,...,!
nvn) so that
LTui= !
ivi
From these we obtain
so that vi are theeigenvectors of LTL
LTLv
i= !
iLTui= !
i
2vi
Conversely, the final SVs ui are the eigenvectors of LLT
LLTui= !
iLv
i= !
i
2ui
SV summary and extra propertiesSV summary and extra properties• In order to get SVs we need the TLM and the ADJ
models.• The leading SVs are obtained by the Lanczos
algorithm (expensive).• One can define an initial and a final norm (size), this
gives flexibility and arbitrariness(1).• The leading initial SV is the vector that will grow
fastest (starting with the smallest initial norm andending with the largest final norm).
• The leading SVs grow initially faster than theLyapunov vectors, but at the end of the period, theylook like LVs (and bred vectors~LVs).
• The initial SVs are very sensitive to the norm used.The final SVs look like LVs~BVs
Jon Jon Ahlquist Ahlquist theoremtheorem• One can define an initial and a final norm (size), this
gives flexibility and arbitrariness(1).• Given a linear operator L, a set of arbitrary vectorsxi and a set of arbitrary nonnegative numbers siarranged in decreasing order, Ahlquist (2000)showed how to define a norm such that si and the xiare the ith singular value and singular vector of L.
• “Because anything not in the null space can be asingular vector, even the leading singular vector, onecannot assign a physical meaning to a singular vectorsimply because it is a singular vector. Any physicalmeaning must come from an additional aspect of theproblem. Said in another way, nature evolves frominitial conditions without knowing which innerproducts and norms the user wants to use”
Comparison of SV and BV (LV) for a QG modelComparison of SV and BV (LV) for a QG model Fig. 6.7: Schematic of how all perturbations will converge towards the leading Local Lyapunov Vector
trajectory
random initial
perturbations
leading local
Lyapunov vector
All perturbations (including singular vectors) end up looking likelocal Lyapunov vectors (i.e., like Bred Vectors).These are the local instabilities that make forecast errors grow.
Two initial and final BV (24hr)contours: forecast errors, colors: BVs
Note that the BV (colors) have shapes similar to theforecast errors (contours)
Two initial and final SV (24hr, vorticity norm)contours: forecast errors, colors: SVs
With an enstrophy norm, the initial SVs have large scales(low vorticity). By the end of the”optimization” interval, thefinal SVs look like BVs (and LVs)
Example of initial and final singular vectors using enstrophy/stream function norms(QG model, 12 hour optimization, courtesy of Shu-Chih Yang)
The initialSVs areverysensitiveto thenorm
The finalSVs looklike bredvectors (orLyapunovvectors)
