Nonlinear Bayesian State Estimation: A Review of Recent … › dept › chemeng › control ›...

Nonlinear Bayesian EState Estimation:

A Review of Recent Developmentsp

Sachin C. PatawardhanDepartment of Chemical EngineeringD partm nt of h m ca Eng n r ng

I.I.T. BombayEmail: [email protected] [email protected]

Automation LabIIT BombayOutline

Motivation and Origin Motivation and Origin

Nonlinear State Estimation

Extended Kalman Filter

Deterministic Derivative-free estimatorsDeterministic Derivative free estimators

Particle Filters

Constrained State Estimation

Estimation under Model-Plant Mismatch Estimation under Model Plant Mismatch

On-line Model Maintenance

May, 12 UBC - UofA Workshop 2

Future research directions

Automation LabIIT BombayNotation

DynamicsState

du,x,fxdtd

),(

DynamicsState

Mechanistic Model

xy H

dt

Modelt Measuremen

y

Assumptions

1kfor )((t)constant piecewise and inputs dManipulate

kk Tttttkuu

mean value of odneighborho in the nsfluctuatio randomconstat piecewise as modelled are esdisturbanc Unmeasured


1kfor )( ktttk(t) wdd

Automation LabIIT BombayNotation

θwduxfxx ))()()(()()(1

dkkttkt

θwd,u,xfxx

:)1(

)),()()(()()(

1

1

TimeSamplingTTktkTt

dkkttk

kk

tkk

wd,u,xfxx )),()()(()()1(1

dkkkkk

k

t

t

θwux ),(),(),( kkkF

Control Relevant Discrete Time Representation

)()(

),(),(),()1(kHk

kkkFkxy

θwuxx


)()( kHk xy

Automation LabIIT BombayBayesian Formulation y

Models: mechanistic models of the form

(k) H (k) (k)

θwuxx ),(),(),()1( kkkFk

w(k): uncertainty in states due to unknown inputs (k): m s m nt s (n is )

(k) H (k) (k) y x v

v(k): measurement errors (noise) (stationary random processes with known statistical properties)

ObjectiveObjectiveFind the conditional probability density function (PDF),

kp (k)| x Y

Yk :set of all the available measurements up to time instant k.

p (k)| x Y


up to time instant k.

Automation LabIIT BombayNonlinear Bayesian Estimation y

Alternative ApproachesAlternative Approaches Sequential Unconstrained Estimation: Methods

that obtain the conditional density function by that obtain the conditional density function by application of Bayes’ rule, and then obtain the estimate using one of the optimization criteria

Direct Optimization: Methods that assume a suitable form for the prior probability density f ti d t th ti ti bl function and convert the estimation problem directly into an optimization problem.

Sequential constrained estimators Sequential constrained estimators Moving horizon estimator

5/31/2012 State Estimation 6

Automation LabIIT BombaySequential Bayesian Estimationq y

Prediction step: posterior density at previous ti t i t d i t t ti t time step is propagated into next time step through state transition density to compute prior

k 1

k 1

k 1

p (k) |

p (k) | (k 1) p (k 1) | d (k 1)

x Y

x x x Y x

Update step: Computation of posterior density from the prior

k k 1k 1

p (k) | (k)p (k) | p (k) |

p (k) |

y xx Y x Y

y Yp (k) | y Y

The Posterior Density function constitutes the complete solution to the sequential estimation problem


solution to the sequential estimation problem.

Automation LabIIT BombayBayesian Estimation y

P dicti n nd upd t st t p vid s n ptim l Prediction and update strategy provides an optimal solution to the state estimation problem

involves high dimensional integration involves high-dimensional integration. exact analytical solution to the recursive propagation of

the posterior density is difficult obtain p y ff

Linear state estimation: possible to compute analytical solution ana yt ca so ut on

Nonlinear filtering techniques: develop approximate and computationally tractable sub-approximate and computationally tractable suboptimal (local) solutions to the sequential Bayesian estimation problem


y p

Automation LabIIT BombayApproximation Approaches pp pp

P di ti st Prediction step Taylor series approximation

D l b d Deterministic sampling based approximations Stochastic sampling (Monte Carlo) based

i ti approximations

U d Update step Statistical linear regression or linear minimum

i i mean square estimation Monte Carlo sampling based approximations


Automation LabIIT BombayExtended Kalman Filter (EKF)

Most popular and widely used Nonlinear Bayesian Filter Propagation step: Predicted Mean

1|)()1|(ˆ kYkEkk xx

Propagation step: Predicted Mean

1|)1(),1(),1( kYkkkFE wux

inionapproximatseriesTaylorUsing

0)1()1|1(ˆ)1()1()1(0),1(),1|1(ˆ)( of nbhd the

in ion approximatseriesTaylor Using

kkkFkkkFkkk

uxwuxux

)1()1|1(

0),1(),1|1()1(),1(),1(

)()(

kFkkFkkkFkkkF

wd

εx

uxwux

)()(

0),1(),1|1(ˆ|)1(),1(),1( 1 kkkFYkkkFE k uxwux


0),1(,|)1(

),(),|(|)(),(),(1 kYkEF k ux

Automation LabIIT BombayExtended Kalman Filter (EKF)

)()1|1()1|(

Covariance Predicted

TT FFFkkFkk QPP

Update Step: Updated Mean computation using

)(

)()()(

)1|1()1|(

kkkk

dQ

dxP

xP

)()()( 1kkk PPL

p p p p gStatistical Linear Regression

)()()1|(ˆ)|(ˆ)1|(ˆ)()(

)()()( 1

kkkkkkkkHkk

kkk

LxyePPL eeεe

)()()1|()|( kkkkkk eLxx

RPP

T

eeHkkHk )1|()(

T

eHkkk )1|()(

PP


xx ee

)()(

)|()( e)(

)|()(

x

Automation LabIIT BombayEKF: Update Step p p

Covariance Updated

H

( )| k ( )| k 1

)1|()()|()(

kkHkkk Px

LIP

Approximates p[x(k)|Yk] and p[x(k)|Yk-1] to be Gaussian i.e.

))1|(),1|(ˆ|)( 1 kkkkkp k PxYx

))|(),|(ˆ|)( and

kkkkkp k PxYx

Gaussian approximation: simplest method to approximate numerical integration problem due to its analytical tractability

Local asymptotic convergence of estimation error (in absence of the state and the measurement noise) has been established

i L ’ d th d (R if t l 1999)


using Lyapunov’s second method (Reif at al. 1999)

Automation LabIIT BombayEKF : Plug Flow (Tubular) Reactor (PFR)EKF : Plug Flow (Tubular) Reactor (PFR)

Steam, TjoTj-1, TR-1 Tj-2, TR-2

CA(1,t), CB(1,t)

T T T Tj-5, TR-5

A B C

CAo, TRoCC(1,t), TR(1,t)

(Endothermic Reaction)

Tj(0,t)

State Estimation Problem Estimate concentration profile inside the reactor using

f t t t l th l th few temperature measurements along the length

Automation LabIIT BombayFixed Bed Reactor

Material Balances (Distributed Parameter System)

1 rE / RTA Al 10 A

C Cv k e Ct z

……..Reactant A

Energy Balances

1 r 2 rE / RT E / RTB Bl 10 A 20 B

C Cv k e C k e Ct z

……..Product B

Energy Balances

1 rr1 E / RTr r

l 10 A

HT Tv k e Ct z C

……..Reactor Temp.

2 r

m pm

r2 E / RT w20 B j r

t z C

H U k e C T TC C V

……..Reactor Temp.

jm pm m pm rC C V

j j wjj

T T Uu T T

……..Jacket Temp. r j

mj pmj j

u T Tt z C V

p

Automation LabIIT BombayPDE To ODE Model (Finite Differencing)PDE To ODE Model (Finite Differencing)

0 1 2 N N + 1

Plant Model

No. of internal discretization points 19 4p

No. of states 80 20

No of jacket side temp measurements 3 3No. of jacket side temp. measurements 3 3

No. of reactor side temp. measurements 3 3

Automation LabIIT BombayState Estimation using EKFState Estimation using EKF

Simulation Parameters

Variable Nominal Value Fluctuations added

F d Fl 1 / 0 01 /Feed Flow 1 m/min 0.01 m/min

Feed Concentration 4 mol/lit 0.14 mol/lit

T 0 4 KTemperature measurements - 0.4 K

Steam flow rate 1 m/min -Steam flow rate 1 m/min

Performance of EKF under the effect of feed flow and feed concentration fluctuations was studied

The estimated concentration approaches the true concentration within 5 minutesconcentration within 5 minutes

Automation LabIIT BombayFluctuations in Feed Flow and Feed

C iConcentration

Automation LabIIT Bombay

Actual and Estimated Exit Concentration of BConcentration of B

Automation LabIIT BombaySimulation Result: Concentration profiles

of product B at different time instants

Automation LabIIT BombayState and Parameter Estimation

Estimation of deterministic changes in

)()()()()()1()1(

kdkkkkTk

wθUXFXX

unmeasured disturbances / model parameters

)()()1(

)()(),(),()()1(

kkk

kdkkkkkT

wθθ

wθUXFXX

θ

)()()( kkXk vHY

Augment the model with fictitious discrete evolution equation

states with estimated be to parameters / esdisturbanc unmeasured containing Vector :)(kθ

states with estimated be to parameters


Prediction step:

ˆ

)1|1(),1(),()1|1(ˆ

)1|(ˆ)1|(ˆ

0

kk

dtkkkkkkkkk

T

θUXFXθX

r ct on st p

)1|1()1|( kkkk

θθ

Correction Step:

)1|(ˆ)()()1|1(ˆ)1|1(ˆ

)|(ˆ)|(ˆ

kkkkkkkk

kkkk

xCyLθx

θx

Covariance Update FF

))1|1()1()1|1(()(

)(;)()()(

kkkkk

kk

θUXθFB

XFA θ

))1|1(),1(),1|1(()( kkkkk θUX


Covariance Update : using augmented matrices

)()(exp)(;)(exp)(0

dkBkAkkTAkT

o ar anc Up at us ng augm nt matr c s

]0[]0[)()(

)(

0

kkka

QQ

]0[]0[

:Covarance Noise State ][

Tuning Parameter

Fast changing parameter / disturbance : usehigh values of co-variance high values of co-variance

Automation LabIIT BombayExperiment: Combined State and

Parameter Estimation on Heater Mixer SetupParameter Estimation on Heater-Mixer Setup

CV-1

Cold Water Flow 3-15 psiInput

CV-1CV-2

Cold Water Flow

Tank - 1TT

LTTank - 2

ThyristerControl Unit

4-20 mA Input Signal

TTT

Automation LabIIT BombayExample: Stirred Tank Heater-MixerExample: Stirred Tank Heater Mixer

)()( 111 IQTTFdT estimatedbetoParameter

)(1

)(

2

111

1

FIFFdh

CVTT

Vdt pi

factor loss-Heat :statesusly with simultaneo

estimatedbetoParameter

)()()(1

)(

22

2212

TTUATTFTTFdT

FIFFAdt

atm

0073.0989.0979.7)(

)()(

31

2111

22221122

IIIIQ

CTTFTTF

Ahdt pi

)(;/5139

0093.071.0279.3)(

)(

02

32

22222

1111

hhkhFKsmJU

IIIIF

Q

)(;/5.139 2 hhkhFKsmJU

controller power thyrister to input current % :1Ivalve control to input current%:I2


Estimation of states and parameters using EKF using EKF

T k 1 t m t d h t l ss Tank 1 temperature and heat loss parameter are to be estimated using EKF

k d l l Tank 2 temperature and level are measured

The system is kept in perturbed state by perturbing the inputs (heater input and p g p ( ptank 2 inlet flow)

The flow to tank 1 is kept constant. The flow to tank 1 is kept constant. The heat-loss parameter (β)is initialized

with a value of 0 8with a value of 0.8


Experimental result: Tank 1 temperature and heat loss parameter estimatesand heat loss parameter estimates


Motivation and Origin Motivation and Origin Linear State Estimation

Kalman filter Nonlinear State Estimation

Extended Kalman FilterD t i isti D i ti f sti t s Deterministic Derivative-free estimators

Particle Filters Constrained State Estimation Constrained State Estimation Estimation under Model-Plant Mismatch

Robustness On-line Model Maintenance



Automation LabIIT BombayLimitations of EKFLimitations of EKF

Covariance update: Using local linearization p g(Taylor series approximation) of nonlinear system equations

l f b h Requires evaluation of Jacobian at each time step

Smoothness requirement on system dynamics: Smoothness requirement on system dynamics: discontinuities not permitted

Computationally expensive for large dimension p y psystems

Propagation step assumes

( ) ( )

(mean) Function Nonlinear ) Function Nonlinear Mean(


E g(x) g E(x)


Example: Autonomous Hybrid SystemExample: Autonomous Hybrid System

u1Pump 1 Pump 2

u1

Valve u2 u6

Valveu4

h1

h2

h3q2

q3

Valveu3

u8

q3

qd q5Valve

Discontinuities in state dynamics: EKF cannot be used


y

Automation LabIIT BombayExample: Autonomous Hybrid SystemExample Autonomous Hybr d System

dh1 q z k | (h1' h2 ') | umax 1 2 3 6

dh1A1 q u q q q (32)dtdh2A2 q q q q q (33)

2 1 2 2q z k | (h1 h2 ) | u ' '

T Th1 h1 h ; h2 h2 h 2 3 4 7 5

max 5 4 7

A2 q q q q q (33)dt

dh3A3 q u q q (34)d

7 2 7 7q z k | h2 ' h3' | u

max 5 4 7q q q ( )dt '

T Th2 ' h2 h ;h3 h3 h

T T

T T

0 (h1 h )AND(h2 h )

(h1 h )AND(h2 h ) ORz 1

State

Dependent

1 ' 'T T

T T

z 1(h1 h )AND(h2 h ) AND(h1 h2 )

(h1 h )AND(h2 h ) OR1

DependentDiscrete Variables


' 'T T

1(h1 h )AND(h2 h ) OR(h1 h2 )

Automation LabIIT BombayAutonomous Hybrid Systemy y

Autonomous Hybrid System: An example of class of systems ith di ti iti i th d i

The state vector consists of continuous as well as discrete state variables, which can take only integer values.

with discontinuities in the dynamics

state variables, which can take only integer values. )1()1(

)(),(),(),()1(

kGk

kkkkFkxξ

wξuxx

can andstatesdiscretethedenotes:)(

)()()()1()1(

kkkHk

kGk

ξvxy

xξ

1.or 1,0- assuch luesinteger vaonly take)(

ξ

The function G( ) is expressed as a combination of logic variables The function G(.) is expressed as a combination of logic variables such as OR, AND, XOR, IF..THEN ..ELSE etc.

Difficulty: Jacobian of F[.] cannot be computed due


Difficulty Jacobian of F[.] cannot be computed dueto discontinuities introduced by the logic variables

Automation LabIIT BombayDerivative Free Filters

Basic idea: Better estimates of the moments of a di t ib ti b bt i d i l th th i distribution can be obtained using samples rather than using the Taylor series approximation of the nonlinear function (that transforms a random variable)

Statistical linear regression is used instead of Taylor series approximation

Derivative Free Filters

Deterministic Sampling Unsc nt d K lm n Filt r Stochastic Sampling Unscented Kalman Filter Divided Difference Filter Gauss-Hermite filter

Stochastic Sampling Particle Filters Ensemble Kalman Filter


Central difference filter

Automation LabIIT BombayStatistical Linear Regressiong

nonlinear a and , vector, variablerandom aGiven e

ofion approximatlinear aapproach ion linearizatlstatisticaby )F( say vector,random theoffunction eε

eeEminimizingbydconstructeis )F(

ppppy

T

bAeeε

)( respect towith

eeEminimizingby dconstructeisbA,

The optimum is reached for the following choices of (A,b)


Automation LabIIT BombayStatistical Linear Regressiong

For these choice of optimal parameters we haveFor these choice of optimal parameters, we have

TEminimizingbyderivedbecan )( optimalfor sexpression Identical

eeb A,

In the literature this linear approximation is also

Eminimizingby derivedbecan ee

In the literature, this linear approximation is also referred to as linear least mean square

(LLMS) estimation.


( )

Automation LabIIT BombayStatistical Linearization Based Filters

Sample generation: Uses a deterministic Sample generation: Uses a deterministic sampling technique to select a finite set of sample pointssample points

)()(ˆ )(

i ii ...2,1:)()(),1()(),1|1(ˆ )(

N

i Nikikikk vwx

1such that weightsassociated define and1i

ii

Prediction: Propagate these samples through the system dynamics to compute a cloud of transformed points


dynamics to compute a cloud of transformed points


)1()1()1|1(ˆ)1|(ˆ )()()( kkkkFkk jjj )1(),1(),1|1()1|( )()()( kkkkFkk jjj wuxx

)()1|(ˆ)( )()()( kkkHk jjj

h j 1 2 N

)()1|()( )()()( kkkHk jjj vxy

where j = 1,2,…N

Statistics of nonlinearly transformed pointsy p

Sample Means

N

j

ji kkkk

1

)( )1|(ˆ)1|(ˆ xx N

j

ji kkkk

1

)( )1|(ˆ)1|(ˆ yy


j 1 j 1


Statistical Linearization Based Filters Statistical Linearization Based Filters Sample Covariance

N

i

Tiiie kkk

1

)()(, )()()( eεP

N

i

Tiiiee kkk

1

)()(, )()()( eeP

(i) (i)ˆ ˆ(k) (k | k 1) (k | k 1) ε x x (i) (i)ˆ ˆ(k) (k | k 1) (k | k 1) e y y

K l G i U d t

i 1

1L(k) P (k) P (k)

Kalman Gain Update

,e e,eL(k) P (k) P (k)

Updated Mean and Covariance

)1(ˆ)()()1|(ˆ)|(ˆ k|kkkkkkk yyLxxTP(k | k) P(k | k 1) L(k)P (k)L(k)


e,eP(k | k) P(k | k 1) L(k)P (k)L(k)

Automation LabIIT BombayMethods for Drawing Samples g p

P follows as , matrix, covariance andmean its , vector,augmented Define aa

00x

vwx)1|1(ˆ)1|1(ˆ

)()1()1()1(

kkkkkkkk

TTT

TTTT

RQPP )1|1()1|1( kkDiagBlockkka

Sigma Point Generation

kkkkM

)1|1(ˆ)1|1()dim(M wheregenerated are samples 12

)1(

Sigma Point Generation

kkkkkk

kkkk

jMj

ja

j )1|1()1|1(ˆ)1|1(

)1|1(ˆ)1|1(

)()1(

)()1(

)1(

ζP

Mjkkkkkk j

aMj

,....,2,1)1|1()1|1(ˆ)1|1( )()1(

ζP

)( j


0toequalrest and1 toequalelement th j'r with Unit vecto:)( jζ

Automation LabIIT BombayMethods for Drawing Samples g p

Unscented Kalman Filter (UKF)

parameter tuninga is where M

)1|1(equalsmatrixcovariancesampleweightedthat theirsuch chosen been have weightsassociated and points sample These kkaP

Divided Difference Kalman Filter (DDKF)

)1|1(equalsmatrix covariancesample weightedthat their kkP

Covariance estimate computed using Stirling’s multi dimensional Covariance estimate computed using Stirling s multi-dimensional polynomial interpolation.

The first and second order terms in Taylor series approximation are i t d i t l diff th d ith t i ‘h’approximated using central difference method with step-size ‘h’



UKF results in approximations accurate to third order UKF results in approximations accurate to third order for Gaussian inputs for all nonlinearities.

For non-Gaussian inputs approximations are accurate For non-Gaussian inputs, approximations are accurate to at least the second-order. Accuracy of third and higher order moments determined by the choice of g ytuning parameters

Sampling based filters be applied for state estimation in systems with discontinuous nonlinear transformations such as autonomous hybrid systems

Limitation: Do not work well when the conditional densities of states are skewed, Multi-modal, non-Gaussian


Gaussian

Automation LabIIT BombayExample: Autonomous Hybrid Systemp y y

State Estimation using UKF

0.3

0.4

0.5

evel

(h1)

True

Estimated

0 50 100 150 200 250 300 3500.1

0.2

Sampling Instants

Le

0.5

0.2

0.3

0.4

Leve

l(h2)

True

Estimated

0 50 100 150 200 250 300 3500.1

Sampling Instants

0 6

0.8True

E ti t d

0

0.2

0.4

0.6

Leve

l(h3)

Estimated


0 50 100 150 200 250 300 3500

Sampling Instants

Automation LabIIT BombayExample: Autonomous Hybrid Systemp y y

Discrete State Estimates

0.5

1

ble(

z1)

0.5

1

ble(

z2)

True

-0.5

0

Dis

cret

e V

aria

b

-0.5

0

Dis

cret

e V

aria

b

True

0 100 200 300 400-1

Sampling Instants0 100 200 300 400

-1

Sampling Instants

1

0

0.5

1

Varia

ble(

z1)

0

0.5

1

aria

ble(

z2)

1

-0.5

0

Dis

cret

e V

1

-0.5

0D

iscr

ete

Va

Estimated


0 100 200 300 400-1

Sampling Instants0 100 200 300 400

-1

Sampling Instants

Automation LabIIT BombayMotivating Example g p

System with skewed, Multi-modal, non-Gaussian yconditional densities of states

Measurement noise covariance (R) = 1

St t i i (Q) 10State noise covariance (Q) = 10

Initial state : x(0|0) = 0 and P(0|0)=10.


Automation LabIIT BombayMotivating Exampleg p

Histogram of particle filter generated samples at f li i t t


few sampling instnat.

Automation LabIIT BombayMotivating Exampleg p

Hist m i di t s th t th diti l Histogram indicates that the conditional density of the states are multi-modal and tim intime varying

Estimators, such as EKF or sigma point f l h h l l d l filters, which implicitly assume uni-modal conditional densities of states, may not be bl able to generate accurate state estimates

This simple example underscores the need p pto develop better estimation methods for dealing with such pathological systems g p g y


Automation LabIIT BombayGaussian Sum Filters

Underlying assumption: any arbitrary PDF can be y g p y yapproximated by a convex combination of Gaussian distributions (Alspach and Sorenson, 1972)

M l i l EKF i ll l

wN(i)

ii 1

p[w(k)] N w,Q

Multiple EKFs are run in parallel Updated state estimate: convex combination of

individual estimatesindividual estimatesxN

(i)i

i 1

ˆ ˆ(k | k) (k) (k | k) x x

Weights recursively updated by application of Bayes’ rule and assuming that innovations of individual EKFs

i 1


m g f Khave Gaussian distributions.

Automation LabIIT BombayGaussian Sum Filters

The conditional densities are approximated via a ppconvex combination of multiple Gaussian densities, i.e.

The Gaussian sum assumption implies that

Weights are recursively updated by application of Bayes rule


Automation LabIIT BombayParticle Filters (PF)

Can deal with state estimation problems arising p gfrom multimodal and non-Gaussian distributions

Excellent reviews: Arulampalam et al., (2002), p , ( ),Chen, Z. (2003), Bakshi and Rawlings, (2006)

PF approximates multi-dimensional integration pp ginvolved in propagation and update steps using Monte Carlo sampling. p g

XX

XdPXfdXXpXf )()()()(Integral:

Approximated as

N

i

iN Xf

Nf

1

1 )(


{X(1), X(2),…X(N)}: i.i.d. particles drawn from P(X)

Automation LabIIT BombayEnsemble Kalman Filter

Proposed by Evensen (1993) and is based on random p ysampling of the state and the measurement noise from their respective distributions

Can work with arbitrary distributions of the state disturbance and the measurement noise

Good combination of stochastic sampling and statistical linearization based filtering: uses only g yfirst and second order moments, which are generated using ensemble propagation and update

Number of samples necessary for generating good estimates can be large


Automation LabIIT BombayEnsemble Kalman Filter

Computation in prediction and observer gain calculation steps are similar to Statistical Linearization Based Filters. Significant differences are as follows:

1)-(kinstant at step update thefrom propagated are )1( of Samples .)( and )1(for only drawn are Samples 1. kkk xvw

Ni /1 i.e. weights,equal assigned are samples All 2.

iii k|kkkkkkk yyLxx )1(ˆ)()()1|(ˆ)|(ˆnscomputatiosubsequentfor samplesgeneratetousedis step Update.3

)()()(

k

N

i

ii kkN

kk xx )|(ˆ1)|(ˆ1

)()(


kk Yx |)(pofnatureabout themadebetohasassumption no Thus,


Particle Filter based on Sequential Importance SamplingImportance Sampling

Difficulty: PDF p(x) of x(k) is unknown D ff cu ty DF p( ) of ( ) s un nown

)())(())(())(( kdkpkfkfE xxxx

Solution: Select importance density q(x)

)())(())(())(())(())(( xx

xxxx kdkq

kqkpkfkfE

Weighting Function

)(~)(ondistributi importancefrom Draw

)( xx qki

Function

Draw samples from proposal (importance) distribution Weight them according to how they fit the original

di ib i


distribution

Automation LabIIT BombayImportance Weightsp g

Particles are weighted by importance weights.

N

iiN kf

Nkf 1 )())(( )(xx

Importance weights are updated using Baye’s Rule

iN 1

(i) k

i (i) k

p (1: k) |(k)

q (1: k) |

x Y

x Y(i) (i) (i)

i(i) (i) k

q (1: k) |

p (k) | (k) p (k) | (k 1)(k 1)

q (k) | (k 1)

x Y

y x x x

x x Y

ii N

q (k) | (k 1),

(k)(k)(k)

x x Y


jj 1

(k)

Automation LabIIT BombayImportance Sampling using EKF p p g g

For example, when EKF is used to generate the importance di t ib ti t i l d i i t li f ll distribution, steps involved in importance sampling are as follows

,...2,1 where)1|1( particleeach For (i)(i)(i)

(i) Nikkx

)|(and)|(),1|(estimateandEKFImplement 1. (i)(i)(i) kkkkkk Pxx

a draw and)|(),|( asdensity importanceConstruct 2. (i)(i) kkkk PxN )|(),|(~)|(i.e. on,distributi thisfrom sample new (i)(i)(i) kkkkkk PxNx

using ~ weight dunnormaize theCompute .3 i Requires running

),1|()(|)(,)|()(|)(

(i)(i)

(i)(i)

kkkkpkkkkp

QxNxyRxhNxy

Requires runningN EKFs in parallel: Computationally

Expensive


)|(),|(),1(|)( (i)(i)(i)(i) kkkkkkp k PxNYxx Expensive

Automation LabIIT BombayParticle Filter Algorithmg

Initi liz ti n st p: C t p ticl s s s mpl s f m Initialization step: Create particles as samples from the initial state distribution.

A kth i t tAs kth instant: Sample each particle from a proposal distribution

EKF as proposal UKF as proposal

C t i ht f h ti l i th Compute weights for each particle using the observation value.R l l l Resample particles generating new particles according to importance weights


Automation LabIIT BombayPF: Schematic Diagram g

PF with importance sampling and re-samplingF w th mportanc samp ng an r samp ng


(Figure taken from Chen, Z., 2005)

Automation LabIIT BombayMotivating Example (Contd )Motivating Example (Contd.)

Comparison of Mean Sum Squared Estimation ompar son of M an Sum Squar Est mat on Error (SSEE) values over 25 trials



Motivating Example (Contd )Motivating Example (Contd.)

Comparison of true states and


states estimated using SIR PF

Automation LabIIT BombayPF: Advantages & Limitations g

PF can in principle deal with arbitrary probability PF can, in principle, deal with arbitrary probability distributions of state propagation error

It suffers from ‘curse of dimensionality’ like most It suffers from curse of dimensionality like most other nonlinear filters developed under Bayesian framework framework

Successful when EKF / UKF can be used to generate proposal density If EKF/UKF diverge generate proposal density. If EKF/UKF diverge, then, most of the samples will be mostly not useful.

As proposed it cannot deal with constraints on As proposed, it cannot deal with constraints on states / parameters



Motivation and Origin Motivation and Origin Linear State Estimation

Kalman filter Nonlinear State Estimation

Extended Kalman FilterD t i isti D i ti f sti t s Deterministic Derivative-free estimators

Particle Filters Constrained State EstimationConstrained State Estimation Estimation under Model-Plant Mismatch

Robustness On-line Model Maintenance



Automation LabIIT BombayConstrained State estimation

In most physical systems, states / parameters are p y y , pbounded, which introduces constraints on state / parameter estimates. Moving horizon estimation (MHE) Constrained Recursive Formulations

Moving horizon estimation (MHE) (Liebman et al. 1992 , Raoand Rawlings, 2002): State estimation formulated as constrained nonlinear

optimization problem over a moving window [k-N:k]B d t t / t th l b i Bounds on states/parameters or any other algebraic constraints can be handled


Automation LabIIT BombayRecursive Constrained Estimation

MHE formulation Easy to handle multi-rate and delayed

measurements. R qui s l dim nsi n l n nlin ptimiz ti n Requires a large dimensional nonlinear optimization problem to be solved at each time step

Recursive constrained formulations Based on the premise that the constraint violations

tl i th d t t occur mostly in the update step Combines computational advantages of recursive

estimation while handling constraints m g Constrained optimization problem solved over single

time step, which make them attractive from the viewpoint of online computations


viewpoint of online computations.

Automation LabIIT BombayRecursive Constrained EstimatorsRecursive Constrained Estimators

Constrained EKF or Recursive Dynamic data Constrained EKF or Recursive Dynamic data Reconciliation (RNDDR or C-EKF)(Vachhani et al AIChEJ 2004)(Vachhani et al., AIChEJ, 2004)

Constrained-UKF (C-UKF) or URNDDR( ) (Vachhani et al., Journal of Process Control, 2006)

Constrained Ensemble Kalman Filter (C-EnKF)(Prakash et. al, I.EC.R., 2010)

Constrained Particle Filter (C-PF)Constrained Particle Filter (C PF)(Prakash et. al, JPC, 2011)


Automation LabIIT BombayConstrained EKF

RNDDR (or C-EKF)Prediction Step: State and covariance propagation

steps identical to that of EKF

Update Step: solving constrained optimization problem over [k-1:k] problem over [k 1 k]

)()()1|()1|()1|()(

min)|(ˆ 11 kkkkkkkk

kkk TT eReεPεx

)1|(ˆ)()1|(

)()()|()|()|()(

)|(

kkkkkk

xxεx

)()()( kHkk xye

k xxx )(Subject to


HL k xxx )(Subject to

Automation LabIIT BombayConstrained EnKF

Prediction Step: Ensemble prediction identical to Prediction Step: Ensemble prediction identical to that of unconstrained EnKF

Update Step: solving constrained N optimization problems over [k-1:k]

)()()1|()1|()1|()(

)|(ˆ )(1)()(1)()( kkkkkkkkk

Minkk iTiiTii eReεPε

xx

.

)(

)()()()(;)1|()()1|( )()()()( kkHkkkkkkk iiic

i vxyexxε

Subject to constraints: UL k xxx )(

N i kkNkk )( )|(ˆ)/1()|(


ii kkNkk

1)( )|()/1()|( xx

Automation LabIIT BombayExample: Gas-Phase Reactionp

Bench-mark Problem: Gas-phase irreversible reaction pin a well mixed, constant volume, isothermal batch reactor (Haseltine and Rawlings, 2003)

12A B k 0.6

2Adp 2k36 0

P(0 | 0)

2A1 A

2B

p 2k pdt

dp k

( | )0 36

(0 | 0) 3 1x2B1 A

p k pdt

p

(0 | 0) 3 1x

ˆ(0 | 0) 0 1 4 5x A

B

pP 1 1

p

(0 | 0) 0.1 4.5x


Note that the partial pressures should not be negative.

Automation LabIIT BombayUnconstrained EnKF

3True

0

1

2

essu

re o

f A

TrueEstimated(N=100)Estimated(N=10)

-4

-3

-2

-1Pa

rtia

l Pre

10 20 30 40 50 60 70 80 90 100-4

Sampling Instants

7True

4

5

6

ress

ure

of B

TrueEstimated(N=100)Estimated(N=10)

10 20 30 40 50 60 70 80 90 1001

2

3

Part

ial P


10 20 30 40 50 60 70 80 90 100

Sampling Instants

Automation LabIIT BombayConstrained EnKFConstrained-EnKF

3 T

1.5

2

2.5

Pres

sure

of A

TrueEstimated

10 20 30 40 50 60 70 80 90 1000

0.5

1

Part

ial P

10 20 30 40 50 60 70 80 90 100

Sampling Instants

4

4.5

B

TrueEstimated

2.5

3

3.5

al P

ress

ure

of Estimated

10 20 30 40 50 60 70 80 90 1001

1.5

2

Sampling Instants

Part

ia


Sampling Instants

(Prakash, Patwardhan and Shah. I.E.C.R., 2010)

Automation LabIIT BombayMoving Horizon Estimationg

Formulate a sequence of optimization problems over

NkV x )(

p pa moving window [k-N,k]

k

Nkj

Tk

Nkj

T

Nk

jjjj

NkV

kNk vRvwQw

x

xx

)()()()(

)(

)(),....,( min

11

1

NkjNkj

jtoSubject

uxFxw

)(,(j)1)(j(j)

HL j

j

xxxxHyv

)( :state on Bounds(j)-(j)(j)

)(,(j))(j(j)

k)|(kˆk),|1-(kˆk),.....,|(1ˆk),|N-(kˆi.e.,estimates, statecurrent and smoothed yieldsSolution

xxxx


states) on the (bounds sconstraint under the)|(),|(), ,|(),|(,

Automation LabIIT BombayMoving Horizon Estimationg

Cost Arrival:)( NkV Nk x

NkNkNk NkV

xxx

P)0|0(ˆ)0(

)( 11

2

)0|0( 1

Nk

j

T

j

TNk

Nkp

jjjj

Yx

vRvwQw

|)(log

)()()()()(

0

1

1

1

Nkp Yx |)(log

Important to construct reasonably accurate estimates of the Arrival Cost: estimates of the Arrival Cost:

an open issue in MHE literature

casenonlineardconstrainein the estimate todifficult :density lConditiona


case.nonlinear dconstrainein the

Automation LabIIT BombayArrival Cost Estimation

error typesquaredmeantheusetodecideweSuppose

)|(ˆ)()(

i.e. cost, arrival for theion approximat error typesquaredmean theusetodecide weSuppose

2NkNkNkNkV ?consraints of presence in the )|( estimate tohow then,

)|()()()|( 1

NkNk

NkNkNkNkVNkNkNk

P

xxxP

Constrained (sampling based) Recursive Bayesian estimators, such as C-EnKF or C-PF, are better suited for arrival cost estimation

Covariance estimate generated from the constrained l i d f i ti th i l t samples is used for approximating the arrival cost.


(López-Negrete et al., JPC, 2011)

Automation LabIIT BombayArrival Cost Estimation: Case Studyy

CSTR System with constraints on states

If arrival cost is estimated with a constrained recursive filterinstead of EKF, can it reduce the window size and,


in turn, reduce the on-line computations?


MSE as a function of horizon length when using constrained filters for the arrival cost approximation for the CSTR example


filters for the arrival cost approximation for the CSTR example


EKF based approximations of the arrival cost introduce EKF based approximations of the arrival cost introduce unwanted errors, which require the choice of longer horizon lengths and a larger optimization problem to be g g p psolved on-line.

Particle-based filters can approximate arrival cost dist ib ti s si s mpl s d th s i f distributions using samples, and thus require few assumptions on the type of distribution. Moreover, CEnKF and constrained C-PF handle bounds on the states, and thus provide a more consistent approximation of the arrival cost.R lti i t i th i l t Resulting improvements in the arrival cost approximation allow us to use a smaller horizon window for MHE, and a smaller NLP can be solved on-linef r MHE, an a ma r NL can n n



Motivation Motivation Linear State Estimation

Kalman filterKalman filter Nonlinear State Estimation

Extended Kalman Filter Deterministic Derivative-free estimators Particle Filters

Constrained State Estimation Robust Estimation and On-Line Model

MMaintenance Future research directions


Automation LabIIT BombayEstimation with M-P-MEstimation with M P M

Model-Plant-Mismatch Model Plant Mismatch Parameter drifts / abrupt changes

Equipment fouling Equipment fouling Catalyst degradation

Leaks Leaks Sensor / actuator biases / failures

Is the state estimator “robust” to MPM? Is estimation error bounded if MPM is

bounded? Can we find which part of the model is bad ?


p

Automation LabIIT BombayRobustness of EKF

Extended the nominal convergence proof of by Extended the nominal convergence proof of by Reif et al. (1999) to show“If MPM is restricted to a compact set, then the If MPM is restricted to a compact set, then the observer errors are bounded (i.e. input to state (ISS) stable)”

Using a EKF in Nonlinear MPC for offset free control: “If observer is ISS and NMPC is nominally stable, then closed loop system obtained by combining the observer with the NMPC is Input to State the observer with the NMPC is Input-to-State practically Stable (ISpS)”


(Huang, Patwardhan, Biegler, JPC, 2011)

Automation LabIIT BombaySimultaneous State & Parameter

Esti ti Estimation

Direct approach (state augmentation) Direct approach (state augmentation) Augment state vector with extra states corresponding to

faults Simultaneously estimate state and ‘fault states’

Advantages Arbitrary type of fault behavior (step/slow drift) can be

tracked Magnitude estimate of the fault is available and can be Magnitude estimate of the fault is available and can be

used for achieving fault tolerance Limitation

Number of extra states which can be estimated cannot exceed number of measurements.


Automation LabIIT BombayActive Model Maintenance

Fault Diagnosis: Sophisticated schemes for one-ti b l b h i id tifi ti

Disturbances O t t

Faults

time abnormal behavior identification

ProcessInputs OutputsModel

Corrections

Fault-free Dynamic

d l

Set of Active Faults

Model

Plant-model mismatchModel Based

Identified Faults

Diagnosis

Fault Models


(Deshpande at al., JPC, 2009)

Automation LabIIT BombayActive Model Maintenance Active Model Maintenance

ObjectivesO j ct s Online detection of multiple abrupt changes

occurring sequentially in time g q y On-line model correction based on diagnosis

Approach: GLR Methodpp Diagnosis: Generalized Likelihood Ratio method Innovation sequences generated by KF / EKF q g y

carry signature of change Exploits the pattern of innovation to identify p p y

fault magnitude fault type Fault that corresponds to maximum value of


likelihood ratio is identified as fault

Automation LabIIT BombayIssues in State Estimation

Robustness to plant model mismatch: Model accuracy is Robustness to plant-model mismatch: Model accuracy is critical to state estimation

Noise Model Parameters: Measurement and state noise Noise Model Parameters Measurement and state noise co-variances are difficult to estimate. These matrices are often treated as tuning parameters

f ( d d / Number of extra states (unmeasured disturbances / parameters) estimated cannot exceed number of measurementsmeasurements

Computationally efficient methods for irregularly sampled multi-rate measurement scenario

Conditional density and arrival cost estimation in presence of constraints on states


Automation LabIIT BombayResearch Directions Research Directions

Nonlinear state estimation: rich and highly active research areag y State estimation of systems governed by DAE in Bayesian

framework S li b d filt ff f th f Sampling based filters suffer from the curse of dimensionality: handling computational complexities that can arise in large scale systems g y

Optimal state estimation in the presence of inequality constraints I i f f l di i h i f li d l Integration of fault diagnosis techniques for on-line model maintenance: isolation of active set of changing parameters and dealing with structural MPM g

Estimation in presence of irregularly sampled and delayed measurements

i l d i i


Simultaneous state and parameter estimation

Automation LabIIT BombayAcknowledgements g

Collaborators Prof. Sirish Shah (Univ. of Alberta, Canada) Prof. S. Narasimhan (IIT, Madras) Prof Lorenz Biegler (CMU) Prof. Lorenz Biegler (CMU)

Graduate Students Dr. J. Prakash (IIT Madras) Dr. Seema Manuja (IIT Madras) Dr Anjali Deshpande (IIT Bombay)Dr. Anjali Deshpande (IIT Bombay) Dr. Abhijit Badwe (IIT Bombay) Dr. Vinay Bavdekar (IIT Bombay)

D R i H (CMU) Dr. Rui Huang (CMU) Dr. Rodrigo López-Negrete (CMU)


Automation LabIIT BombayAcknowledgements g

Board of Research for Nuclear Sciences, Govt. of India NSERC-Matrikon-Suncore-Icore Industrial Research

Chair program in Computer Process Control at University of Alberta

Indo-U.S. Science and Technology Forum (IUSTF) travel t grant

Thank You !

Questions ?


Automation LabIIT BombayReferences

Review Articles / Books Arulampalam S Maskell N Gordon and T Clapp (2002) A tutorial on particle Arulampalam, S. Maskell, N. Gordon and T. Clapp, (2002). A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking, IEEE Transactions on Signal Processing, 50(2), pp. 174–188.

Burgers, G, Leeuwen, P. J. V., Evensen G. (1998). Analysis Scheme in the Burgers, G, Leeuwen, P. J. V., Evensen G. (1998). Analysis Scheme in the Ensemble Kalman Filter, Monthly Weather Review, 126, 1719-1724.

Chen, Z. (2003). Bayesian filtering: from Kalman filters to particle filters, and beyond, Technical Report, Adaptive Syst. Lab., McMaster University,

lHamilton, ON, Canada. Daum, F. (2005). Nonlinear filters: Beyond the Kalman filter, IEEE A&E

Systems Magazin, 20, (8), 57–69.J li S J d J K Uhl (2004) U s t d filt i d li Julier S.J. and J.K. Uhlmann (2004), Unscented filtering and nonlinear estimation, Proceedings of the IEEE, 92 (3), pp. 401–422.

Muske, K.R. and T.F. Edgar (1997). Nonlinear state estimation. In Nonlinear Process Control (M.A. Henson and D.E. Seborg, Eds.), Prentice-Hall.Process Control (M.A. Henson and D.E. Seborg, Eds.), Prentice Hall.

Soderstorm,T.(2002), Discrete-time stochastic systems, Advanced Textbooks in Control and Signal Processing, Springer.

Sorenson, H. (1985). Kalman Filtering: Theory and Applications, IEEE Press,


( ) g y ppNew York, 1985.


Research Papers in referred in the slides Alspach, D. L., Sorenson, H. W. (1972). Nonlinear Bayesian Estimation Using

Gaussian Sum Approximations, IEEE Transactions on Automatic Control, 4, 439-448.Bavdekar V A Deshpande A P Patwardhan S C (2011a) "Identification of Bavdekar, V. A., Deshpande, A. P., Patwardhan, S. C. (2011a). Identification of process and measurement noise covariances for state and parameter estimation using extended Kalman filter", Journal of Process Control, 21, 585-601.

Deshpande, A., Patwardhan, S. C., Narasimhan, S. Intelligent State Estimation p , , , , , gfor Fault Tolerant Nonlinear Model Predictive Control, Journal of Process Control, 19, 187–204, 2009.

Huang, R., Patwardhan, S. C., Biegler, L. T. (2010). Stability of a class of di i li i b J f P C l 20 1150discrete-time nonlinear recursive observers, J. of Process Control, 20, 1150--1160.

López-Negrete, R.; Patwardhan, S.C.; and Biegler, L.T., (2011). Constrained particle filter approach to approximate the arrival cost in Moving Horizon particle filter approach to approximate the arrival cost in Moving Horizon Estimation. Journal of Process Control, 21, pp. 909-919.

Prakash, J., Patwardhan, S. C., Shah, S. L On the choice of importance distributions for unconstrained and constrained state estimation using particle


g pfilter, Journal of Process Control, 21, pp. 119-129.


Patwardhan S C Narasimhan S Prakash J Gopaluni B Shah S L (2012) Patwardhan, S. C., Narasimhan, S., Prakash, J., Gopaluni, B., Shah, S. L (2012) Nonlinear Bayesian State Estimation: A Review Of Recent Developments”. Accepted for publication in Control Engineering Practice, 2012.

Prakash, J., S. C. Patwardhan and S. L. Shah (2010) “Constrained State Estimation Using the Ensemble Kalman Filter”,, Ind. Eng. Chem. Res., 49 (5), pp 2242-2253.

Rawlings, J. B., and B R. Bakshi, Particle filtering and moving horizon ti ti C t d Ch i l E i i 30 10 12 1529 1541 2006estimation, Computers and Chemical Engineering, 30, 10-12, 1529-1541, 2006.

Reif, K., Gunther, S., Yaz, E., Unbehauen, R. (1999). Stochastoc stability of the discrete-time extended Kalman filter. IEEE Trans. on Automatic Control, 44,4, 714-728714 728.

Sorenson, H. (1970). Least square estimation from Gauss to Kalman. IEEE Spectrum, pp 63-68, July, 1970.

Vachhani, P., R. Rengaswamy, V. Gangwal and S. Narasimhan (2004), Recursive , , g y, g ( ),estimation in constrained nonlinear dynamical systems, AIChE J., 946–959.

Vachhani, P., Narasimhan, S., Rengaswamy, R. (2006). Robust and reliable estimation via Unscented Recursive Nonlinear Dynamic Data Reconciliation. J l f P C t l 16 1075 1086


Journal of Process Control, 16, 1075-1086.

Date post:	07-Jun-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Nonlinear Bayesian State Estimation: A Review of Recent … › dept › chemeng › control ›...

Documents