Post on 25-Mar-2020
transcript
Jasper A. Vrugt
University of California Irvine, CEE & EES
University of Amsterdam, CGE Email: jasper@uci.edu
LECTURE 4
DATA ASSIMILATION
true input
true response
observed input
simulated response
measurement
outp
ut
time f
parameters prior info
observed response
optimize parameters
TUNING THE PARAMETERS SO THAT CLOSEST FIT TO THE OBSERVED SYSTEM RESPONSE IS OBTAINED
ENVIRONMENTAL MODELING FRAMEWORK
MATHEMATICAL FORMULATION
LETS USE A STATE SPACE FORMULATION
THE MEASUREMENT OPERATOR
THE ERROR RESIDUAL
SINGLE LAYER CANOPY INTERCEPTION MODEL
Rainfall, P Evaporation, E
Drainage, D
Storage, S
MAIN MODEL EQUATIONS
Vrugt et al., WRR, (2003)
INTERCEPTION MODELING
True rainfall
Rainfall data
Time [hours]
Sto
rage
[m
m]
Inte
rcept
ion
Dra
inag
e
Eva
pora
tion
Max
imum
S
tora
ge
INTERCEPTION MODELING (continued)
True rainfall
Rainfall data
Time [hours]
Sto
rage
[m
m]
Simulation
Unable to fit
DATA ASSIMILATION
True rainfall
Rainfall data
Time [hours]
Sto
rage
[m
m]
Simulation
REMEMBER
Blow up at time t
X
?
t +1
DATA ASSIMILATION (continued)
True rainfall
Rainfall data
Time [hours]
Sto
rage
[m
m]
Data assimilation
REMEMBER
DATA ASSIMILATION REMOVES PERSISTENT BIAS BY UPDATING STATE VARIABLES
Simulation
HOW TO DETERMINE SIZE STATE UPDATES?
?
t +1
X
SIZE OF STATE UPDATES DEPENDS DIRECTLY ON SIZE OF MODEL AND MEASUREMENT ERROR
t
f
t
a
t yyy ~22
2
22
2
REMEMBER
3
2
X = measurement
= model
1
x
t
x
t+1
= updated
Cmax
0
bexp Alpha
(1-Alpha)
Rq Rq Rq
Rs
ANOTHER CONCEPTUAL EXPLANATION USING ANOTHER MODEL
Vrugt et al., WRR, (2005)
Y(t)
Forcing (Input Variables)
System invariants (Parameters)
Output (Diagnostic Variables)
f p(Yt)
U(t)
X(t)
Observations
p(Ot)
Update rule
DREAM p(M)
p(Ut)
State (Prognostic Variables)
p(Xt)
Ensemble Kalman Filter
Vrugt et al., WRR, (2005); Vrugt et al., GRL, (2005); Vrugt et al., JHM, (2006)
SIMULTANEOUS OPTIMIZATION AND DATA ASSIMILATION
Parameter and
State Estimation
Parameter Estimation
POSTERIOR MODEL PREDICTION RANGES
Vrugt et al., WRR, (2005)
-0.2
0
0.2
0.4
0.6
Au
toc
orr
ela
tio
n
(A) SCEM-UA
0 5 10 15 20 25
-0.2
0
0.2
0.4
0.6
Au
toc
orr
ela
tio
n
Lag [d]
(B) SODA
Significantly less auto-correlation between residuals with recursive state updating
AUTOCORRELATION BETWEEN RESIDUALS
Vrugt et al., WRR, (2005)
0
0.05
0.1
0.15
0.2
0.25
0.3(A)
SC
EM
-UA
Marg
inal
po
ste
rio
r d
en
sit
y (B) (C) (D) (E)
250 300 350 400 4500
0.05
0.1
0.15
0.2
0.25
0.3(F)
Cmax
SO
DA
Marg
inal
po
ste
rio
r d
en
sit
y
0.4 0.6 0.8 1 1.2 1.4
(G)
bexp
0.8 0.85 0.9 0.95
(H)
Alpha
0.02 0.04 0.06 0.08
(I)
Rs
0.38 0.4 0.42 0.44 0.46
(J)
Rq
[mm] [-] [-] [d] [d]
MARGINAL POSTERIOR PARAMETER DISTRIBUTIONS
Vrugt et al., WRR, (2005)
0 100 200 300 400-150
-100
-50
0
50
100
150(A) DRIVEN
Me
an
en
se
mb
le o
utp
ut
inn
ov
ati
on
[
m3/s
]
0 50 100 150 200 250-150
-100
-50
0
50
100
150(B) NONDRIVEN QUICK
1 1.5 2 2.5 3-1.5
-1
-0.5
0
0.5
1
1.5(C) NONDRIVEN SLOW
Mean ensemble streamflow prediction [m3/s]
Byproduct of Data Assimilation is the time series of output/state innovations: info about model structural errors?
INSIGHTS INTO MODEL STRUCTURAL ERRORS?
Vrugt et al., WRR, (2005)
0 20 40 60 80 100 1200
0.5
1
1.5
2
2.5
3
3.5
Time [days]
Nor
malized
Tra
cer
Con
c. (x
10
3)
Unplanned 14 hr
flow interruption
Planned 7-day
flow interruption Planned 14-day
flow interruption
Bromide
Pentafluorobenzoate
Lithium
YUCCA MOUNTAIN SUBSURFACE FLOW AND TRANSPORT MODEL
Vrugt et al., GRL, (2005)
x = 30 meters
Injection well Production well
0 20 40 60 80 100 1200
0.5
1
1.5
2
2.5
3
3.5
Time [days]
No
rm
alized
Tracer C
on
c. (x
10
3)
Unplanned 14 hr
flow interruption
Planned 7-day
flow interruption Planned 14-day
flow interruption
Bromide
Pentafluorobenzoate
Lithium
DATA COLLECTION AND TRACERS
Vrugt et al., GRL, (2005)
Injection well
Modeling forced-gradient cross-hole tracer experiments
Define volume size nodes
Define exit fluxes (qi)
Solve advection – dispersion equation
time
Con
c. Production well
- ×
N
i
i
N
i
i i t
out t
q
q C
C
1
1
,
, ) (
CONCEPTUAL MODEL: RESIDENCE TIME DISTRIBUTION
Vrugt et al., GRL, (2005)
Nodal concentration update according to:
i
N
i
iouttouttit
itititq
qCCZ
KCC-
-
1
,,,
,,,
)~
)((
Parameter estimation using adaptive MCMC
The Shuffled Complex Evolution Metropolis (SCEM-UA) algorithm
HOW TO DO PARAMETER AND STATE ESTIMATION?
Vrugt et al., GRL, (2005)
0 25 50 75 100 1250
0.5
1
0.25
0.75
Nor
maliz
ed L
ithium
C
onc.
(x 1
03)
Time [days]
(B) SCEM-UA -- No state updating
Time [days]
Time [days]
0 25 50 75 100 1250
0.5
1
0.25
0.75
Nor
maliz
ed L
ithium
C
onc.
(x 1
03) (A) SODA -- State updating
Kf = 0.20 – 0.24
n = 0.63 – 0.64
Kf = 0.01 – 0.08
n = 0.64 – 0.68
Which parameter values to use for transport predictions?
MODEL PREDICTION UNCERTAINTY RANGES
Vrugt et al., GRL, (2005)
0
0.2
0.4
0.6
0.8
1
Norm
aliz
ed P
ara
mete
r R
ange
sx sy
DB
r
DP
FB
A
DLi
Kf n 0.2
0.30.4
0.5
0
0.2
0.40.05
0.1
0.15
0.2
0.25
f Li
fBr
fPFBA
Nonsorbing tracers show similar parameter values. Most trade-off appears between the fitting of the sorbing and nonsorbing tracers
► Sorbing and nonsorbing tracers provide conflicting information
PARETO SOLUTION SET (WITH AMALGAM)
Vrugt et al., VZJ, (2008)
0
1
2
3
No
rmali
zed
Bro
mid
e
C
on
c.
(x 1
03)
(A) Tracer - Bromide
0
1
2
3
4
No
rmali
zed
PF
BA
Co
nc.
(x 1
03)
(B) Tracer - Pentafluorobenzoate
0 25 50 75 100 1250
0.2
0.4
0.6
0.8
1
Time [days]
No
rmali
zed
Lit
hiu
m
C
on
c.
(x 1
03)
(C) Tracer - Lithium
MODEL PREDICTION UNCERTAINTY RANGES
Vrugt et al., VZJ, (2008)
-1
0
1(A) No State Updating
Tracer - Bromide
-1
0
1
Au
toco
rr.
Resid
uals
(B) SODA
-1
0
1(A) No State Updating
Tracer - Lithium
0 5 10 15 20 25-1
0
1
Au
toco
rr.
Resid
uals
(B) SODA
Lag
PARAMETER ESTIMATION .VS. DATA ASSIMILATION
Vrugt et al., VZJ, (2008)
RECENT DEVELOPMENTS
PARTICLE-MARKOV CHAIN MONTE CARLO
Vrugt et al., AWR, (2012)
X = measurement = model = updated
x t
x
t+1
Particle-DREAM Joint Parameter and State Estimation
f
BAYESIAN ANALYIS
Bayes, Thomas (1763). "An Essay towards solving a Problem in the Doctrine of Chances.“, Philosophical Transactions of the Royal Society of London, 53, 370–418.
P(A|B)
PRIOR, LIKELIHOOD, EVIDENCE, POSTERIOR
= P(A) P(B|A)
P(B)
PRIOR CONDITIONAL PROBABILITY (= LIKELIHOOD)
EVIDENCE POSTERIOR
IN OUR CASE WE USE THE FOLLOWING NOTATION
IMAGINE WE HAVE SOME DATA “B” AND WE LIKE TO ESTIMATE “B”
BAYES LAW TELLS US TO DO THE FOLLOWING
SEQUENTIAL BAYES LAW
DERIVATION OF SEQUENTIAL BAYES LAW
Doucet and Johansen, 2011; Vrugt et al., AWR, (2012)
X = measurement = model = updated
x t
x
t+1
Particle-DREAM
f
SEQUENTIAL BAYES LAW
SEQUENTIAL BAYES LAW (PARAMETERS ASSUMED KNOWN!)
Doucet and Johansen, 2011; Vrugt et al., AWR, (2012)
SEQUENTIAL MONTE CARLO (SMC) METHODS
Doucet and Johansen, 2011; Vrugt et al., AWR, (2012)
CRUX OF SMC: RESAMPLING
Vrugt et al., AWR, (2012)
RESAMPLING WITH DREAM AT (t-1)
WITH METROPOLIS ACCEPTANCE PROBABILITY
SCHEMATIC ILLUSTRATION OF RESAMPLING
Vrugt et al., AWR, (2012)
PSEUDO-CODE OF PARTICLE-DREAM
BUT WHAT TO DO WITH PARAMETERS?
TWO DIFFERENT POSSIBILITIES
P-DREAM(VP) STATE AUGMENTATION
P-DREAM(IP) OUTSIDE DREAM LOOP
VARIABLE PARAMETERS (STATE AUGMENTATION) NOT RECOMMENDED!!!!
CAUTIONARY NOTE
Vrugt et al., AWR, (2012)
PARTICLE - DREAM
BENCHMARK STUDY: LORENZ MODEL
Vrugt et al., AWR, (2012)
PARAMETERS ADDED TO STATE VECTOR – NOT RECOMMENDED!!
PARTICLE-DREAM WITH INVARIANT PARAMETERS
DREAM + PARTICLE-DREAM
PSEUDO-CODE OF P-DREAM(IP)
Vrugt et al., AWR, (2012)
CASE STUDY: SIMPLE LORENZ MODEL
Vrugt et al., AWR, (2012)
Vrugt et al., AWR, (2012)
CASE STUDY: HYDROLOGIC MODEL
FINAL REMARK ABOUT EFFICIENCY
DIRECTED UPDATED TOWARDS OBSERVATIONS
CAUTION: DETAILED BALANCE!!!
SOFTWARE: FACULTY.SITES.UCI.EDU/JASPER