Use of time-dependent parameters for improvement and uncertainty estimation of dynamic models

Eawag: Swiss Federal Institute of Aquatic Science and Technology

Use of time-dependent parameters for improvement and uncertainty estimation of dynamic models

Peter Reichert

Eawag Dübendorf and ETH Zürich

SAMSI meetingNov. 6, 2006

Contents

Motivation

References

Approach

Preliminary Results

Problems/Challenges

Motivation

References

Approach

Concept

Implementation

Preliminary Results for a Simple Hydrologic Model

Problems / Challenges


Motivation

Motivation

Motivation

References

Approach

Preliminary Results

Problems/Challenges


Motivation

Fundamental Objectives: Improve understanding of mechanisms

governing the behaviour of the system described by the model.

Estimate realistic uncertainty bounds / decrease the width of uncertainty bounds of model predictions.

Technical Objectives: Improve the formulation of the deterministic

model component. Make the stochastic component of the model

more realistic.

Motivation

References

Approach

Preliminary Results

Problems/Challenges


Motivation

Achieve these objectives by:

1. Improving the input error model.

2. Allowing model parameters to vary (e.g. in time) to address model structure error.

3. Improving the output error model (by addressing bias explicitly).

In particular:

Search for statistical model components that cannot be rejected by the data.

Try to „explain“ the bias by input and/or model structure error („trace“ the causes of the bias).

Motivation

References

Approach

Preliminary Results

Problems/Challenges


References

References

Motivation

References

Approach

Preliminary Results

Problems/Challenges


References

The idea of using time-dependent parameters for model structure deficit evaluation is very old (e.g. review by Beck, 1987).

Our work applies this idea to continuous time models and provides algorithms to apply it to nonlinear dynamic systems.

Motivation

References

Approach

Preliminary Results

Problems/Challenges This talk is based on:

Brun, PhD dissertation, 2002: First trials with filtering algorithm.

Buser, Masters thesis, 2003: Smoothing, MCMC algorithm.

Tomassini, Reichert, Künsch, Buser, Borsuk, 2007:Estimation of process parameters, cross-validation.


Approach

Approach

Motivation

References

Approach

Preliminary Results

Problems/Challenges


Notation (according to Bayarri et al. 2005)

Input: Field data = reality plus measurement error:Motivation

References

Approach

Preliminary Results

Problems/Challenges

xxx RF

An ideal model describes reality:

*RRR idealideal,)( MM xyxy

A realistic model approximates an ideal model:

***

approxapproxapproxapproxidealideal,,, MMMMMM xbxyxy

yMxMMxM xbxyy *F

*FF approxapproxapproxapprox

,,

Output: Field data = reality plus measurement error:

yyy RF

All terms together:

inputerror

meas.error

effect of model structure error


Problem

Motivation

References

Approach

Preliminary Results

Problems/Challenges

yMxMMxM xbxyy *F


,,

inputerror

meas.error


Problem:

The bias term describes the effect, but not the cause of the model structure error. This leads to a satisfying statistical description of the past, but is hard to extra-polate into the future.

For uncertainty reduction and extrapolation it would be better to reduce the bias by improving the mechanistic description of the system. In particular, trends must be described by the model, not by the bias term.

How can statistical procedures support this?


Concept

Motivation

References

Approach

Preliminary Results

Problems/Challenges

yMxMMxM xbxyy *F


,,

inputerror

meas.error


Concept:

1. Allow model parameters to vary. Add parameters where appropriate (input, output).

2. Try to reduce the bias by finding an adequate behaviour of these parameters.

3. Explore dependency of parameter variability on external or model variables. If successful (from a statistical and physical point of view), modify the model structure to reflect this dependency.

4. Redo the analysis with improved model structure and reduced bias.


Use for dynamic models

Motivation

References

Approach

Preliminary Results

Problems/Challenges

yMxMMxM xbxyy *F


,,

inputerror

meas.error


xt : Correction accounting for input error.

t : Model-internal correction of model structure error.

yt : Model-external correction for remaining effect of

model structure error.In the ideal case, this error could be neglected as it would be accounted for by the internal correction.

yty

tM

txM xyy *

FF approxapprox,

Formulation for time dependent models:


Approach

Motivation

References

Approach

Preliminary Results

Problems/Challenges

1. Fit model with constant parameters, identify presence of bias. If bias exists:

2. Identify, separately or jointly, time-dependent input variation (x

t ) parameter variation (

t ) output variation (y

t )

3. Identify dependences of time-dependent parameters on external or model variables.

4. Improve the model structure by deterministic or sto-chastic elements (according to statistical and physi-cal considerations), try to avoid output error (y

t ).

5. Use the extended model for understanding and prediction.

yty

tM

txM xyy *

FF approxapprox,


Implementation

Motivation

References

Approach

Preliminary Results

Problems/Challenges

This has the advantage that we can use the analytical solution:

)(d)()(d tWtt W

The time dependent parameter is modelled by a mean-reverting Ornstein Uhlenbeck process:

)(2

2)(

st 12,)(N~ stWst

ss ee

or, after reparameterization:

stst

ss ee2

2st 1,)(N~

2,

1 22 W


Implementation

Motivation

References

Approach

Preliminary Results

Problems/Challenges

We combine the estimation of

constant model parameters, , with

state estimation of the time-dependent parameter(s), t, and with

the estimation of (constant) parameters of the Ornstein-Uhlenbeck process(es) of the time dependent parameter(s), =(,, ,to).


Conceptual Framework

Motivation

References

Approach

Preliminary Results

Problems/Challenges

xF

Xx

tx

QM

X

tQ

xmod Q

M,mod

YMdet

YMbias

YMmeas

ty

Xy

Ey

Model extended by input- and output-parameter and measurement error

Original deterministic model


Simplified Framework

Motivation

References

Approach

Preliminary Results

Problems/Challenges

QM

X

t

YM

Simplifications:

Omit representation of given measured input, xF.

Add parameter to input to represent input uncertainty by parameter uncertainty.

Add parameter to output to represent output uncertainty by parameter uncertainty.


Numerical Implementation (1)

Motivation

References

Approach

Preliminary Results

Problems/Challenges

QM

X

t

YM

Gibbs sampling for the three different types of parameters. Conditional distributions:

ttt yffyθfyθf ,)(,,,

ttt fffyθf )(,,

ttt yffyθf ,,,

Ornstein-Uhlenbeck process (cheap)

simulation model (expensive)

simulation model (expensive)

Ornstein-Uhlenbeck process (cheap)


Numerical Implementation (2)

Motivation

References

Approach

Preliminary Results

Problems/Challenges

Metropolis-Hastings sampling for each type of parameter:

ttt yffyθfyθf ,)(,,,

ttt fffyθf )(,,

ttt yffyθf ,,,

Multivariate normal jump distributions for the parameters and . This requires one simulation to be performed per suggested new value of .

The discretized Ornstein-Uhlenbeck parameter, t, is split into subintervals for which OU-process realizations conditional on initial and end points are sampled. This requires the number of subintervals simulations per complete new time series of t.


Estimation of Hyperparametersby Cross - Validation

Motivation

References

Approach

Preliminary Results

Problems/Challenges

Due to identifiability problems we selected the hyperparameters, , in a previous application (Tomassini et al., 2006) alternatively by cross-validation:

ξy max(log i

iiyfpsl


Preliminary Results

Preliminary Results for a Simple Hydrologic Model

Motivation

References

Approach

Preliminary Results

Problems/Challenges Model

Model Application Preliminary Results

(based on Markov chains of insufficient length)


Model

A Simple Hydrologic Watershed Model (1):

gwlatetrunoffrains )(d

dqqqqq

t

h

bfgwgw

d

dqq

t

h

rbflatrunoffr

d

dqqqq

t

h

Kuczera et al. 2006

Motivation

References

Approach

Preliminary Results

Problems/Challenges


Model


)(rainrain trainfq

)(rainsatrunoff trainffq

)()exp(1 petet tpetfhkq set

maxlat,satlat qfq

gwbfbf hkq

maxgw,satgw qfq

rrr hkq

QbqAQ rwr

100

1

)exp()99(1

1

FssFsat

shks

f

Kuczera et al. 2006

Motivation

References

Approach

Preliminary Results

Problems/Challenges

1

2

3 4

5

6

7

A

B

7 model parameters3 initial conditions1 standard dev. of meas. err.3 „modification parameters“

C


Model


0 500 1000 1500

0.0

0.2

0.4

0.6

0.8

1.0

hs [mm]

f sat

[-]

ks=0.02/mm, sF=2300ks=0.01/mm, sF=2300ks=0.04/mm, sF=2300ks=0.02/mm, sF=1150ks=0.02/mm, sF=4600

100

1

)exp()99(1

1

FssFsat

shks

f

Kuczera et al. 2006

Motivation

References

Approach

Preliminary Results

Problems/Challenges


Model Application

Model application:

Data set of Abercrombie watershed, New South Wales, Australia (2770 km2), kindly provided by George Kuczera (Kuczera et al. 2006).

Box-Cox transformation applied to model and data to decrease heteroscedasticity of residuals.

Step function input to account for input data in the form of daily sums of precipitation and potential evapotranspiration.

Daily averaged output to account for output data in the form of daily average discharge.

Motivation

References

Approach

Preliminary Results

Problems/Challenges


Model Application

Prior distribution:

Estimation of constant parameters:

Independent uniform distributions for the loga-rithms of all parameters (7+3+1=11), keeping correction factors (frain, fpet) equal to unity and bias (bQ) equal to zero.

Estimation of time-dependent parameters:

Ornstein-Uhlenbeck process applied to log of the parameter (with the exception of bQ). Hyper-parameters: = 5d, fixed, only estimation of initial value and mean (0 for frain, fpet, bQ). Constant parameters as above.

Motivation

References

Approach

Preliminary Results

Problems/Challenges


Preliminary Results (MC of insufficient length)

Posterior marginals: Motivation

References

Approach

Preliminary Results

Problems/Challenges

0.10 0.25

13

57

h_s_ini

4 6 8

0.1

0.3

0.5

h_gw _ini

0.0 1.0

0.2

0.8

h_r_ini

0.015 0.025

50

15

0

k_s

0 100 250

0.0

05

0.0

15

s_F

0.010 0.018

50

15

0

k_et

1.0 2.0

0.5

1.5

q_lat_max

3 4 5 6

0.2

0.6

q_gw _max

0 e+00 3 e-04

10

00

50

00

k_bf

0.5 0.7 0.9

13

5

k_r

1.00 1.15 1.30

26

10

sd_Q_trans



Max. post. simulation with constant parameters: Motivation

References

Approach

Preliminary Results

Problems/Challenges

400 500 600 700 800 900 1000 1100

05

01

00

15

02

00

t [days since 31/12/1971]

Q [m

3/s

]



Residuals of max. post. sim. with const. pars.: Motivation

References

Approach

Preliminary Results

Problems/Challenges 500 600 700 800 900 1000

-20

00

20

0


resi

du

als

[m3

/s]

NS = 0.63se = 21

500 600 700 800 900 1000

-6-2

26


resi

d. o

f tr.

ou

tpu

t

NS = 0.63se = 1.1



Residual analysis, max. post., constant parameters Motivation

References

Approach

Preliminary Results

Problems/Challenges

Residual analysis, max. post., q_gw_max time-dependent

500 700 900

-6-2

26


resi

d. o

f tr.

ou

tpu

tNS = 0.63se = 1.1

0 5 10 15 20 25

0.0

0.4

0.8

Lag

AC

F

500 700 900

-50

5


resi

d. o

f tr.

ou

tpu

t

NS = 0.67se = 1.0

0 5 10 15 20 25

0.0

0.4

0.8

Lag

AC

F



Residual analysis, max. post., s_F time-dependent Motivation

References

Approach

Preliminary Results

Problems/Challenges

Residual analysis, max. post., f_rain time-dependent

500 700 900

-6-2

26


resi

d. o

f tr.

ou

tpu

tNS = 0.76se = 0.8

0 5 10 15 20 25

0.0

0.4

0.8

Lag

AC

F

500 700 900

-20

2


resi

d. o

f tr.

ou

tpu

t

NS = 0.9se = 0.67

0 5 10 15 20 25

0.0

0.4

0.8

Lag

AC

F



Time-dependent parameters

Motivation

References

Approach

Preliminary Results

Problems/Challenges

400 500 600 700 800 900 1000 11002

61

0

time [days since 31/12/1971]

qg

wm

ax

400 500 600 700 800 900 1000 1100

20

08

00


sF

400 500 600 700 800 900 1000 1100

0.6

1.2

1.8


fra

in




(= Working Group Opportunities)

Motivation

References

Approach

Preliminary Results

Problems/Challenges




1. Other formulations of time-dependent parameters?

2. Dependence on other factors than time.

3. How to estimate hyperparameters? (Reduction in correlation time always improves the fit.)

4. How to avoid modelling physical processes with the bias term?

5. Learn from more applications.

6. Compare results with methodology by Bayarri et al. (2005). Combine/extend the two methodologies?

7. ?

Motivation

References

Approach

Preliminary Results

Problems/Challenges




(= Working Group Opportunities)

Discussion slides from talk at Oct. 16.

Motivation

References

Approach

Preliminary Results

Problems/Challenges



Research Questions / Options for Projects (1)

1. Compare results when making different model parameters stochastic and time-dependent. (Ongoing with a postdoc in Switzerland extending earlier work with continuous-time stochastic parameters.)

2. Develop a better statistical description of rainfall uncertainty.(Option for a collaboration with climate/weather working groups.)

3. Explore alternative options for making parameters time-dependent.(Suggestions so far: storm-dependent parameters, time-dependent parameter as an Ornstein-Uhlenbeck process.)

Motivation

References

Approach

Preliminary Results

Problems/Challenges




4. Investigate how to learn from state estimation of stochastic hydrological models.(Can the pattern of state adaptations lead to insights of model structure deficits or input errors?)

5. Develop uncertainty estimates when using multi-objective optimization.(How to use information on Pareto set for uncertainty estimation of parameters and results?)

6. Analyse differences in results of suggested approaches when using different models.(Is there a generic behaviour of different techniques when they are applied to different models/data sets?)

Motivation

References

Approach

Preliminary Results

Problems/Challenges




7. Improve the efficientcy of posterior maximisation and posterior sampling.(Efficiency becomes important when having complex watershed models in mind. Efficient global optimizers and sampling from multi-modal posterior distributions becomes then important.)

8. More questions will come up during discussions.

Motivation

References

Approach

Preliminary Results

Problems/Challenges

Eawag: Swiss Federal Institute of Aquatic Science and Technology

Thank you for your attention

Date post:	16-Jan-2016
Category:	Documents
Upload:	imelda
View:	35 times
Download:	0 times

Use of time-dependent parameters for improvement and uncertainty estimation of dynamic models

Documents