Post on 17-Feb-2022
transcript
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 1
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Lecture 2Meteorological Data Assimilation
H. ElbernRhenish Institute for Environmental Research at the
University of Cologneand
Virt. Inst. for Inverse Modelling of Atmopheric Chemical Composition
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 2
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Objectives of this lecture
Presentation of• a review of the satellite observation data
set for meteorological applications
• numerical implementation measures for spatio-temporal assimilation algorithms
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 3
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Meteorological remote sensing observation suite (1)
• Atmospheric sounding channels from passive instruments: atmospheric temperature and humidity (Atmospheric sounding channels from the HIRS (High resolution Infrared Sounder) and AMSU (Advanced Microwave Sounding Unit) on board NOAA
• Surface sensing channels from passive instruments "imaging" channels: located in atmospheric "window" regions of the infra-red and microwave spectrum : surface temperature and cloud track information
• Surface sensing channels from active instruments: scatterometeremit microwave radiation for ocean wind retrievals. Some similar-class active instruments such as altimeters and SARS (Synthetic Aperture Radars) provide information on wave height and spectra.
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 4
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Meteorological remote sensing observation suite (2)
• visible (Lidars) or the microwave (radars) analyse the signal backscattered – molecules, aerosols, water droplets or ice particles. – penetration capability allow the derivation of information on cloud
base, cloud top, wind profiles (Lidars) or cloud and rain profiles
(radars).
• Radio-occulation technique using GPS (Global Positioning System). GPS receivers (e.g. METOP/GRAS) measure the Doppler shift of a GPS signal refracted along the atmospheric limb path.
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 5
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Observation systems (1)
Type and number of observations used to estimate the atmosphere initial conditions in a typical day. (Buizza, 2000)
dimobservation space= O(106)
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 6
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Observation systems (2): In-situ observations
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 7
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Observation systems (3): polar orbiting satellites (e.g. AMSU-A)
Data coverage for the NOAA-15 (red), NOAA-16 (cyan) and NOAA-17 (blue) AMSU-A instruments, for the four 6-hourperiods centred at 00, 06, 12 and 18 UTC 12 November 2002. The plots show the data used for AMSU-A channel 5, which is atemperature-sounding channel in the mid and lower troposphere.
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 8
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Observation systems (4): geostationary
Data coverage provided by the GOES satellites (cyan and orange) and the METEOSAT satellites (magenta and red) for 00 UTC 10 May 2003. The total number of observations was 266,878.
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 9
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Information source: numerical model
nowT799L91(~25 km)
dim O(108)
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 10
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
At which retrieval level to assimilate remote sensing data?
• Assimilation of retrieved products from space agencies or research institutes (“level 2”)– Pro: most simple for assimilation (assimilation “like in situ observations”)– Con: inexact as xb and B are inconsistent and mostly of poorer quality as
available at NWP centres• Locally produced or “1D-Var” or “Averaging Kernel” retrievals
– Pro: xb and B much better known (e.g. fronts featured and consistent with model)– Con: xb and B used twice with the subsequent assimilation: y and xb correlated
• Direct assimilation of radiances (“level 1”)– Pro: retrieval step is essentially incorporated within the main analysis by finding
the model variables that minimize a cost function measuring the departure between the analysed state and both the background and available observations.
– Con: Fast linearized radiative transfer scheme and its adjoint is needed.
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 11
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Example for averaging kernels
Averaging kernels (in K/K) for AIRS (left panel) and HIRS (right panel) instruments.
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 12
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Background Error Covariance Matrix
The BECM is of utmost importance, as it•weights the model error against the competing observation errors•spreads information from observations to the adjacent area•influences coupled parameters: temperature wind field, chemical constituents (i.e. multivariate correlation)•(serves for preconditioning in the case of variationalassimilation) 2 simple yet popular
covariance models:Gaussian
Balgovind
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 13
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Background Error Covariance Matrix B(2 pragmatic methods to estimate B)
( )( )∑=
−−=K
nj
nji
niij xxxx
KB
1
1Ensemble integration
K= # ensemble members;
i,j grid cells
1. Ensemble approach: (e.g. Evensen, 1994)
2. NMC method: (e.g. Parrish and Derber, 1992)
Difference column vector of model statesof two forecasts at some absolute time ti, one starting 24 hours earlier than the other,thus trying to capture the most sensitive errors.Covariances calculated by outer product.
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 14
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Novel multivariate covariance modelling (1)
B composition ofcross-correlationblock matrices
cascadic linear relationship between parameterswith separation in balanced B and unbalanced U component.
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 15
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Novel multivariate covariance modelling (2)
where
then,B amenable to factorization by
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 16
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Construction of the adjoint code
forward model(forward differential equation)
algorithm(solver)
code
backward model(backward differential equation)
adjoint algorithm(adjoint solver)
adjoint code
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 17
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Constructing the adjoint from forward code
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 18
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Constructing the adjoint from forward code
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 19
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Adjoint code verification
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 20
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Adjoint compilerSome examples• Odyssee• TAMC • Tapenade• ADOL-C and ADOL-F (includes ability for higher
order derivatives)• IMASFor much more comprehensive list see:www.autodiff.org/?module=Tools&submenu=&language=ALL
Also codes from adjoint compilers must be tested!!Algorithm and performance bugs occur!
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 21
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Adjoint integration Adjoint integration ““backward in timebackward in time””How to make the parameters of resolvents iM(ti-1,ti) available in reverseorder??
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 22
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
CONTRACEConvective Transport of Trace Gases into
the upper Troposphere over
Europe: Budget and Impact of Chemistry
Coord.: H. Huntrieser, DLR
flight path Nov. 14, 2001
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 23
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
CONTRACE Nov. 14, 2001 north (= home) bound
O3
H2O2CO
NO
1. guess assimilation result observations flight height [km]
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 24
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Isopleths of the cost function and transformed cost function and minimisation steps
Minimisation by mere gradients, quasi-Newon method L-BFGS(Large dimensional Broyden Fletcher Goldfarb Shanno),and preconditioned (transformed) L-BFGS application
concentration species 1 transformed species 1
conc
entra
tion
spec
ies
2tra
nsfo
rmed
spe
cies
2
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 25
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 26
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Computation of inverse B, square root B, inverse square root Bby (Sca)LAPACK eigenpair decomposition, but better choices available.
Transformed cost functiondefine
transformation
transformed cost function
transformed gradient of the cost function
Pro transformation:minimisation problem is better conditionedContra:strictly positive definite approximation to B required
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 27
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Kalman filter: basic equationsForecast equations
Analysis equations
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 28
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Reduced rank Kalman filter (basic idea)Appoximate covariance matrices Pb,a (n x n) by a product of suitably low ranked matrix Sb,a (n x p), and p << n . Same procedure for the system noise matrix Q with T (n x r).
The forecast step remains unchanged.
The forecast error covariance matrix rests on 2 x p model integrations only!
The enlargement of p by r enforces periodic reductions of columns (with lowest ranked eigenvalues)
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 29
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Reduced rank Kalman filter (calculus)
In practice, all calculations can be performed without actually calculating matrices P!
Positive semidefinitenes is maintained!
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 30
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Ensemble Ensemble KalmanKalman FilterFilterAssimilation system with different backgroundfields and perturbed observations, reflectingestimated error covariance matrices
After short prediction:
•Rank of matrices = number of elements orders of magnitude smaller (O(100)) than dimension of phase space=> analysis increment only within the sub space spanned by the ensemble Further measures: supression of spurious covariances
Random vectors do not provides optimal subspaces
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 31
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Implementing ensemble Kalman filter in practice
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 32
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
εi perturbation vector of potential optimisation parameters: initial values boundary values, emission rates, deposition velocities
C norm inducing pos. def., sym. operator at initial time t0 (Mahalanobis)M tangent linear modelE norm inducing pos. def., sym. operator at optimisation time t1(P projection operator, extinguishing areas or species outside focus)λ Lagrange parameter and generalised eigenvalues
unit constraint (scalar product):maximiseRaleigh quotient:
maximise⇒generalised
EV problem
Some advanced auxiliary calculations (1)Some advanced auxiliary calculations (1)singular value analysis singular value analysis
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 33
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Assumptions:• Gaussian error distribution assumption sufficiently valid• First guess not too far from “solution” (tangent-linear approximation
must hold)• A priori defined error covariances (background, observations)
Necessary condition for a posteriorivalidation:adjust B and R such that:
ExpectationVariance
Some advanced auxiliary calculations (2)a posteriori validation of data assimilation results
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 34
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Some advanced auxiliary calculations (3)a posteriori validation of data assimilation results
The partition of costs in terms of all observations and the background at the final analysis
How do the partial costs in the cost function divide?
partial costs of observations (Ip identity matrix in observation space with p observations)
partial costs of background:
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 35
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
The partition of costs in terms of individual classes of information sources zj
Day 5 Lecture 2 Data Assimilation Hendrik Elbern 36
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING
Summary
• Satellite data still not fully exploited, but within the scope of variational calculus
• 4Dvar is presently regarded as the most rewarding data assimilation method
• Kalman Filtering with suitably reduced complexity (square root and/or ensemble) feasible, but superiority yet to be proved