Feedback Particle Filterand its Application to Coupled Oscillators
Presentation atUniversity of Maryland, College Park, MD
Prashant Mehta
Dept. of Mechanical Science and Engineeringand the Coordinated Science Laboratory
University of Illinois at Urbana-Champaign
May 1, 2015
Bayesian Inference/FilteringMathematics of prediction: Bayes’ rule
Signal (hidden): X X ∼ P(X), (prior, known)
Feedback Particle Filter Prashant Mehta 2 / 34 Prashant Mehta
Bayesian Inference/FilteringMathematics of prediction: Bayes’ rule
Signal (hidden): X X ∼ P(X), (prior, known)
Observation: Y (known)
Feedback Particle Filter Prashant Mehta 2 / 34 Prashant Mehta
Bayesian Inference/FilteringMathematics of prediction: Bayes’ rule
Signal (hidden): X X ∼ P(X), (prior, known)
Observation: Y (known)
Observation model: P(Y|X) (known)
Feedback Particle Filter Prashant Mehta 2 / 34 Prashant Mehta
Bayesian Inference/FilteringMathematics of prediction: Bayes’ rule
Signal (hidden): X X ∼ P(X), (prior, known)
Observation: Y (known)
Observation model: P(Y|X) (known)
Problem: What is X ?
Feedback Particle Filter Prashant Mehta 2 / 34 Prashant Mehta
Bayesian Inference/FilteringMathematics of prediction: Bayes’ rule
Signal (hidden): X X ∼ P(X), (prior, known)
Observation: Y (known)
Observation model: P(Y|X) (known)
Problem: What is X ?
Solution
Bayes’ rule: P(X|Y)︸ ︷︷ ︸Posterior
∝ P(Y|X)P(X)︸ ︷︷ ︸Prior
Feedback Particle Filter Prashant Mehta 2 / 34 Prashant Mehta
Bayesian Inference/FilteringMathematics of prediction: Bayes’ rule
Signal (hidden): X X ∼ P(X), (prior, known)
Observation: Y (known)
Observation model: P(Y|X) (known)
Problem: What is X ?
Solution
Bayes’ rule: P(X|Y)︸ ︷︷ ︸Posterior
∝ P(Y|X)P(X)︸ ︷︷ ︸Prior
This talk is about implementing Bayes’ rule indynamic, nonlinear, non-Gaussian settings!
Feedback Particle Filter Prashant Mehta 2 / 34 Prashant Mehta
ApplicationsTarget state estimation
Feedback Particle Filter Prashant Mehta 3 / 34 Prashant Mehta
ApplicationsTarget state estimation
Feedback Particle Filter Prashant Mehta 3 / 34 Prashant Mehta
ApplicationsTarget state estimation
Feedback Particle Filter Prashant Mehta 3 / 34 Prashant Mehta
ApplicationsTarget state estimation
Feedback Particle Filter Prashant Mehta 3 / 34 Prashant Mehta
ApplicationsBayesian model of sensory signal processing
Feedback Particle Filter Prashant Mehta 3 / 34 Prashant Mehta
Nonlinear FilteringMathematical Problem
Signal model: dXt = a(Xt)dt+ dBt, X0 ∼ p∗0(·)
Posterior is an information state
P(Xt ∈ A|Z t) =∫
Ap∗(x, t)dx
E(Xt|Z t) =∫R
xp∗(x, t)dx
A. Bain and D. Crisan, Fundamentals of Stochastic Filtering. Springer, 2010.
Feedback Particle Filter Prashant Mehta 4 / 34 Prashant Mehta
Nonlinear FilteringMathematical Problem
Signal model: dXt = a(Xt)dt+ dBt, X0 ∼ p∗0(·)
Observation model: dZt = h(Xt)dt+ dWt
Posterior is an information state
P(Xt ∈ A|Z t) =∫
Ap∗(x, t)dx
E(Xt|Z t) =∫R
xp∗(x, t)dx
A. Bain and D. Crisan, Fundamentals of Stochastic Filtering. Springer, 2010.
Feedback Particle Filter Prashant Mehta 4 / 34 Prashant Mehta
Nonlinear FilteringMathematical Problem
Signal model: dXt = a(Xt)dt+ dBt, X0 ∼ p∗0(·)
Observation model: dZt = h(Xt)dt+ dWt
Problem: What is Xt ? given obs. till time t =: Z t
Posterior is an information state
P(Xt ∈ A|Z t) =∫
Ap∗(x, t)dx
E(Xt|Z t) =∫R
xp∗(x, t)dx
A. Bain and D. Crisan, Fundamentals of Stochastic Filtering. Springer, 2010.
Feedback Particle Filter Prashant Mehta 4 / 34 Prashant Mehta
Nonlinear FilteringMathematical Problem
Signal model: dXt = a(Xt)dt+ dBt, X0 ∼ p∗0(·)
Observation model: dZt = h(Xt)dt+ dWt
Problem: What is Xt ? given obs. till time t =: Z t
Answer in terms of posterior: P(Xt|Z t) =: p∗(x, t).
Posterior is an information state
P(Xt ∈ A|Z t) =∫
Ap∗(x, t)dx
E(Xt|Z t) =∫R
xp∗(x, t)dx
A. Bain and D. Crisan, Fundamentals of Stochastic Filtering. Springer, 2010.
Feedback Particle Filter Prashant Mehta 4 / 34 Prashant Mehta
Nonlinear FilteringMathematical Problem
Signal model: dXt = a(Xt)dt+ dBt, X0 ∼ p∗0(·)
Observation model: dZt = h(Xt)dt+ dWt
Problem: What is Xt ? given obs. till time t =: Z t
Answer in terms of posterior: P(Xt|Z t) =: p∗(x, t).
Posterior is an information state
P(Xt ∈ A|Z t) =∫
Ap∗(x, t)dx
E(Xt|Z t) =∫R
xp∗(x, t)dx
A. Bain and D. Crisan, Fundamentals of Stochastic Filtering. Springer, 2010.
Feedback Particle Filter Prashant Mehta 4 / 34 Prashant Mehta
Nonlinear FilteringMathematical Problem
Signal model: dXt = a(Xt)dt+ dBt, X0 ∼ p∗0(·)
Observation model: dZt = h(Xt)dt+ dWt
Problem: What is Xt ? given obs. till time t =: Z t
Answer in terms of posterior: P(Xt|Z t) =: p∗(x, t).
Posterior is an information state
P(Xt ∈ A|Z t) =∫
Ap∗(x, t)dx
E(Xt|Z t) =∫R
xp∗(x, t)dx
A. Bain and D. Crisan, Fundamentals of Stochastic Filtering. Springer, 2010.
Feedback Particle Filter Prashant Mehta 4 / 34 Prashant Mehta
Kalman filterSolution in linear Gaussian settings
dXt = αXt dt+ dBt (1)
dZt = γXt dt+ dWt (2)
Kalman filter: p∗ = N(Xt,Σt)
dXt = αXt dt + K(dZt− γXt dt)︸ ︷︷ ︸Update
Kalman Filter
Observation: dZt = γXt dt+ dWt
Prediction: dZt = γXt dt
Innov. error: dIt = dZt− dZt= dZt− γXt dt
Control: dUt = K dIt
Gain: Kalman gain
R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961);R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961).
Feedback Particle Filter Prashant Mehta 5 / 34 Prashant Mehta
Kalman filterSolution in linear Gaussian settings
dXt = αXt dt+ dBt (1)
dZt = γXt dt+ dWt (2)
Kalman filter: p∗ = N(Xt,Σt)
dXt = αXt dt + K(dZt− γXt dt)︸ ︷︷ ︸Update
Kalman Filter
Observation: dZt = γXt dt+ dWt
Prediction: dZt = γXt dt
Innov. error: dIt = dZt− dZt= dZt− γXt dt
Control: dUt = K dIt
Gain: Kalman gain
R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961);R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961).
Feedback Particle Filter Prashant Mehta 5 / 34 Prashant Mehta
Kalman filterSolution in linear Gaussian settings
dXt = αXt dt+ dBt (1)
dZt = γXt dt+ dWt (2)Kalman filter: p∗ = N(Xt,Σt)
dXt = αXt dt + K(dZt− γXt dt)︸ ︷︷ ︸Update
Kalman Filter
-
+
Kalman Filter
Observation: dZt = γXt dt+ dWt
Prediction: dZt = γXt dt
Innov. error: dIt = dZt− dZt= dZt− γXt dt
Control: dUt = K dIt
Gain: Kalman gain
R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961);R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961).
Feedback Particle Filter Prashant Mehta 5 / 34 Prashant Mehta
Kalman filterSolution in linear Gaussian settings
dXt = αXt dt+ dBt (1)
dZt = γXt dt+ dWt (2)Kalman filter: p∗ = N(Xt,Σt)
dXt = αXt dt + K(dZt− γXt dt)︸ ︷︷ ︸Update
Kalman Filter
-
+
Kalman Filter
Observation: dZt = γXt dt+ dWt
Prediction: dZt = γXt dt
Innov. error: dIt = dZt− dZt= dZt− γXt dt
Control: dUt = K dIt
Gain: Kalman gain
R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961);R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961).
Feedback Particle Filter Prashant Mehta 5 / 34 Prashant Mehta
Kalman filterSolution in linear Gaussian settings
dXt = αXt dt+ dBt (1)
dZt = γXt dt+ dWt (2)Kalman filter: p∗ = N(Xt,Σt)
dXt = αXt dt + K(dZt− γXt dt)︸ ︷︷ ︸Update
Kalman Filter
-
+
Kalman Filter
Observation: dZt = γXt dt+ dWt
Prediction: dZt = γXt dt
Innov. error: dIt = dZt− dZt= dZt− γXt dt
Control: dUt = K dIt
Gain: Kalman gain
R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961);R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961).
Feedback Particle Filter Prashant Mehta 5 / 34 Prashant Mehta
Kalman filterSolution in linear Gaussian settings
dXt = αXt dt+ dBt (1)
dZt = γXt dt+ dWt (2)Kalman filter: p∗ = N(Xt,Σt)
dXt = αXt dt + K(dZt− γXt dt)︸ ︷︷ ︸Update
Kalman Filter
-
+
Kalman Filter
Observation: dZt = γXt dt+ dWt
Prediction: dZt = γXt dt
Innov. error: dIt = dZt− dZt= dZt− γXt dt
Control: dUt = K dIt
Gain: Kalman gain
R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961);R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961).
Feedback Particle Filter Prashant Mehta 5 / 34 Prashant Mehta
Kalman filterSolution in linear Gaussian settings
dXt = αXt dt+ dBt (1)
dZt = γXt dt+ dWt (2)Kalman filter: p∗ = N(Xt,Σt)
dXt = αXt dt + K(dZt− γXt dt)︸ ︷︷ ︸Update
Kalman Filter
-
+
Kalman Filter
Observation: dZt = γXt dt+ dWt
Prediction: dZt = γXt dt
Innov. error: dIt = dZt− dZt= dZt− γXt dt
Control: dUt = K dIt
Gain: Kalman gain
R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961);R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961).
Feedback Particle Filter Prashant Mehta 5 / 34 Prashant Mehta
Kalman filterSolution in linear Gaussian settings
dXt = αXt dt+ dBt (1)
dZt = γXt dt+ dWt (2)Kalman filter: p∗ = N(Xt,Σt)
dXt = αXt dt + K(dZt− γXt dt)︸ ︷︷ ︸Update
Kalman Filter
-
+
Kalman Filter
Observation: dZt = γXt dt+ dWt
Prediction: dZt = γXt dt
Innov. error: dIt = dZt− dZt= dZt− γXt dt
Control: dUt = K dIt
Gain: Kalman gain
R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961);R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961).
Feedback Particle Filter Prashant Mehta 5 / 34 Prashant Mehta
Kalman filter
dXt = αXt dt︸ ︷︷ ︸Prediction
+ K(dZt− γXt dt)︸ ︷︷ ︸Update
Simple enough to be included in the first undergraduate course on control!
Feedback Particle Filter Prashant Mehta 6 / 34 Prashant Mehta
Kalman filter
dXt = αXt dt︸ ︷︷ ︸Prediction
+ K(dZt− γXt dt)︸ ︷︷ ︸Update
Kalman Filter
-
+
Simple enough to be included in the first undergraduate course on control!
Feedback Particle Filter Prashant Mehta 6 / 34 Prashant Mehta
Kalman filter
dXt = αXt dt︸ ︷︷ ︸Prediction
+ K(dZt− γXt dt)︸ ︷︷ ︸Update
Kalman Filter
-
+
This illustrates the key features of feedback control:
1 Use error to obtain control (dUt = K dIt)
2 Negative gain feedback serves to reduce error (K =γ
σ2W︸︷︷︸
SNR
Σt)
Simple enough to be included in the first undergraduate course on control!
Feedback Particle Filter Prashant Mehta 6 / 34 Prashant Mehta
Pretty Formulae in MathematicsMore often than not, these are simply stated
Euler’s identity
eiπ =−1
Euler’s formula
v− e+ f = 2
Pythagoras theorem
x2 + y2 = z2
Kenneth Chang. What Makes an Equation Beautiful? in The New York Times on October 24, 2004
Feedback Particle Filter Prashant Mehta 7 / 34 Prashant Mehta
Filtering ProblemNonlinear Model: Kushner-Stratonovich PDE
Signal & Observations dXt = a(Xt)dt+ dBt, (1)
dZt = h(Xt)dt+ dWt (2)
Posterior distribution p∗ is a solution of a stochastic PDE:
dp∗ = L †(p∗)dt+1
σ2W(h− h)(dZt− hdt)p∗
where h = E[h(Xt)|Zt] =∫
h(x)p∗(x, t)dx
L †(p∗) =− ∂ (p∗ ·a(x))∂x
+12
∂ 2p∗
∂x2
R. L. Stratonovich. Conditional Markov Processes. Theory Probab. Appl. (1960);H. J. Kushner. On the differential equations satisfied by conditional probability densities of Markov processes. SIAM J. Control (1964).
Feedback Particle Filter Prashant Mehta 8 / 34 Prashant Mehta
Filtering ProblemNonlinear Model: Kushner-Stratonovich PDE
Signal & Observations dXt = a(Xt)dt+ dBt, (1)
dZt = h(Xt)dt+ dWt (2)
Posterior distribution p∗ is a solution of a stochastic PDE:
dp∗ = L †(p∗)dt+1
σ2W(h− h)(dZt− hdt)p∗
where h = E[h(Xt)|Zt] =∫
h(x)p∗(x, t)dx
L †(p∗) =− ∂ (p∗ ·a(x))∂x
+12
∂ 2p∗
∂x2
R. L. Stratonovich. Conditional Markov Processes. Theory Probab. Appl. (1960);H. J. Kushner. On the differential equations satisfied by conditional probability densities of Markov processes. SIAM J. Control (1964).
Feedback Particle Filter Prashant Mehta 8 / 34 Prashant Mehta
Filtering ProblemNonlinear Model: Kushner-Stratonovich PDE
Signal & Observations dXt = a(Xt)dt+ dBt, (1)
dZt = h(Xt)dt+ dWt (2)
Posterior distribution p∗ is a solution of a stochastic PDE:
dp∗ = L †(p∗)dt+1
σ2W(h− h)(dZt− hdt)p∗
where h = E[h(Xt)|Zt] =∫
h(x)p∗(x, t)dx
L †(p∗) =− ∂ (p∗ ·a(x))∂x
+12
∂ 2p∗
∂x2
No closed-form solution in general. Closure problem.
R. L. Stratonovich. Conditional Markov Processes. Theory Probab. Appl. (1960);H. J. Kushner. On the differential equations satisfied by conditional probability densities of Markov processes. SIAM J. Control (1964).
Feedback Particle Filter Prashant Mehta 8 / 34 Prashant Mehta
Particle FilterAn algorithm to solve nonlinear filtering problem
Approximate posterior in terms of particles p∗(x, t) =1N
N
∑i=1
δXit(x)
Algorithm outline
1 Initialization at time 0: Xi0 ∼ p∗0(·)
2 At each discrete time step:Importance sampling (Bayes update step)Resampling (for variance reduction)
J. E. Handschin and D. Q. Mayne. Monte-Carlo techniques to estimate conditional expectation in nonlinear filtering. Int. J. Control (1969);N. Gordon, D. Salmond, A. Smith. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. F Radar Signal Process (1993);J. Xiong. Particle approximation to filtering problems in continuous time. The Oxford handbook of nonlinear filtering (2011);A. Budhiraja, L. Chen, C. Lee. A survey of numerical methods for nonlinear filtering problems. Physica D (2007).
Feedback Particle Filter Prashant Mehta 9 / 34 Prashant Mehta
Particle FilterAn algorithm to solve nonlinear filtering problem
Approximate posterior in terms of particles p∗(x, t) =1N
N
∑i=1
δXit(x)
Algorithm outline
1 Initialization at time 0: Xi0 ∼ p∗0(·)
2 At each discrete time step:Importance sampling (Bayes update step)Resampling (for variance reduction)
J. E. Handschin and D. Q. Mayne. Monte-Carlo techniques to estimate conditional expectation in nonlinear filtering. Int. J. Control (1969);N. Gordon, D. Salmond, A. Smith. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. F Radar Signal Process (1993);J. Xiong. Particle approximation to filtering problems in continuous time. The Oxford handbook of nonlinear filtering (2011);A. Budhiraja, L. Chen, C. Lee. A survey of numerical methods for nonlinear filtering problems. Physica D (2007).
Feedback Particle Filter Prashant Mehta 9 / 34 Prashant Mehta
Particle FilterAn algorithm to solve nonlinear filtering problem
Approximate posterior in terms of particles p∗(x, t) =1N
N
∑i=1
δXit(x)
Algorithm outline
1 Initialization at time 0: Xi0 ∼ p∗0(·)
2 At each discrete time step:Importance sampling (Bayes update step)Resampling (for variance reduction)
J. E. Handschin and D. Q. Mayne. Monte-Carlo techniques to estimate conditional expectation in nonlinear filtering. Int. J. Control (1969);N. Gordon, D. Salmond, A. Smith. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. F Radar Signal Process (1993);J. Xiong. Particle approximation to filtering problems in continuous time. The Oxford handbook of nonlinear filtering (2011);A. Budhiraja, L. Chen, C. Lee. A survey of numerical methods for nonlinear filtering problems. Physica D (2007).
Feedback Particle Filter Prashant Mehta 9 / 34 Prashant Mehta
Particle FilterAn algorithm to solve nonlinear filtering problem
Approximate posterior in terms of particles p∗(x, t) =1N
N
∑i=1
δXit(x)
Algorithm outline
1 Initialization at time 0: Xi0 ∼ p∗0(·)
2 At each discrete time step:Importance sampling (Bayes update step)Resampling (for variance reduction)
e.g. dZt = Xt dt+ small noise
J. E. Handschin and D. Q. Mayne. Monte-Carlo techniques to estimate conditional expectation in nonlinear filtering. Int. J. Control (1969);N. Gordon, D. Salmond, A. Smith. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. F Radar Signal Process (1993);J. Xiong. Particle approximation to filtering problems in continuous time. The Oxford handbook of nonlinear filtering (2011);A. Budhiraja, L. Chen, C. Lee. A survey of numerical methods for nonlinear filtering problems. Physica D (2007).
Feedback Particle Filter Prashant Mehta 9 / 34 Prashant Mehta
Particle FilterAn algorithm to solve nonlinear filtering problem
Approximate posterior in terms of particles p∗(x, t) =1N
N
∑i=1
δXit(x)
Algorithm outline
1 Initialization at time 0: Xi0 ∼ p∗0(·)
2 At each discrete time step:Importance sampling (Bayes update step)Resampling (for variance reduction)
J. E. Handschin and D. Q. Mayne. Monte-Carlo techniques to estimate conditional expectation in nonlinear filtering. Int. J. Control (1969);N. Gordon, D. Salmond, A. Smith. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. F Radar Signal Process (1993);J. Xiong. Particle approximation to filtering problems in continuous time. The Oxford handbook of nonlinear filtering (2011);A. Budhiraja, L. Chen, C. Lee. A survey of numerical methods for nonlinear filtering problems. Physica D (2007).
Feedback Particle Filter Prashant Mehta 9 / 34 Prashant Mehta
Particle FilterAn algorithm to solve nonlinear filtering problem
Approximate posterior in terms of particles p∗(x, t) =1N
N
∑i=1
δXit(x)
Algorithm outline
1 Initialization at time 0: Xi0 ∼ p∗0(·)
2 At each discrete time step:Importance sampling (Bayes update step)Resampling (for variance reduction)
J. E. Handschin and D. Q. Mayne. Monte-Carlo techniques to estimate conditional expectation in nonlinear filtering. Int. J. Control (1969);N. Gordon, D. Salmond, A. Smith. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. F Radar Signal Process (1993);J. Xiong. Particle approximation to filtering problems in continuous time. The Oxford handbook of nonlinear filtering (2011);A. Budhiraja, L. Chen, C. Lee. A survey of numerical methods for nonlinear filtering problems. Physica D (2007).
Feedback Particle Filter Prashant Mehta 9 / 34 Prashant Mehta
Particle FilterAn algorithm to solve nonlinear filtering problem
Approximate posterior in terms of particles p∗(x, t) =1N
N
∑i=1
δXit(x)
Algorithm outline
1 Initialization at time 0: Xi0 ∼ p∗0(·)
2 At each discrete time step:Importance sampling (Bayes update step)Resampling (for variance reduction)
Innovation error, feedback? And most importantly, is this pretty?
J. E. Handschin and D. Q. Mayne. Monte-Carlo techniques to estimate conditional expectation in nonlinear filtering. Int. J. Control (1969);N. Gordon, D. Salmond, A. Smith. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. F Radar Signal Process (1993);J. Xiong. Particle approximation to filtering problems in continuous time. The Oxford handbook of nonlinear filtering (2011);A. Budhiraja, L. Chen, C. Lee. A survey of numerical methods for nonlinear filtering problems. Physica D (2007).
Feedback Particle Filter Prashant Mehta 9 / 34 Prashant Mehta
Feedback Particle FilterA control-oriented approach
Signal & Observations dXt = a(Xt)dt+ dBt (1)
dZt = h(Xt)dt+ dWt (2)
Motivation: Work of Huang, Caines and Malhame on Mean-field games (IEEE TAC 2007).Related approaches: D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs. Stochastics (2009);S. K. Mitter and N. J. Newton. A variational approach to nonlinear estimation. SIAM J. Control Optimiz. (2003);F. Daum and J. Huang. Generalized particle flow for nonlinear filters. Proc. SPIE (2010);S. Reich, A dynamical systems framework for intermittent data assimilation. BIT Numer. Math. (2011).
Feedback Particle Filter Prashant Mehta 10 / 34 Prashant Mehta
Feedback Particle FilterA control-oriented approach
Signal & Observations dXt = a(Xt)dt+ dBt (1)
dZt = h(Xt)dt+ dWt (2)
Controlled system (N particles):
dXit = a(Xi
t)dt+ dBit + dUi
t︸︷︷︸mean-field control
, i = 1, ...,N (3)
BitN
i=1 are ind. standard white noises.
Motivation: Work of Huang, Caines and Malhame on Mean-field games (IEEE TAC 2007).Related approaches: D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs. Stochastics (2009);S. K. Mitter and N. J. Newton. A variational approach to nonlinear estimation. SIAM J. Control Optimiz. (2003);F. Daum and J. Huang. Generalized particle flow for nonlinear filters. Proc. SPIE (2010);S. Reich, A dynamical systems framework for intermittent data assimilation. BIT Numer. Math. (2011).
Feedback Particle Filter Prashant Mehta 10 / 34 Prashant Mehta
Feedback Particle FilterA control-oriented approach
Signal & Observations dXt = a(Xt)dt+ dBt (1)
dZt = h(Xt)dt+ dWt (2)
Controlled system (N particles):
dXit = a(Xi
t)dt+ dBit + dUi
t︸︷︷︸mean-field control
, i = 1, ...,N (3)
BitN
i=1 are ind. standard white noises.
Variational approach:
1. Gradient flow construction:
Nonlinear filter is shown to be a gradient flow (steepest descent)
2. Optimal transport:
Derivation of the feedback particle filter
Motivation: Work of Huang, Caines and Malhame on Mean-field games (IEEE TAC 2007).Related approaches: D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs. Stochastics (2009);S. K. Mitter and N. J. Newton. A variational approach to nonlinear estimation. SIAM J. Control Optimiz. (2003);F. Daum and J. Huang. Generalized particle flow for nonlinear filters. Proc. SPIE (2010);S. Reich, A dynamical systems framework for intermittent data assimilation. BIT Numer. Math. (2011).
Feedback Particle Filter Prashant Mehta 10 / 34 Prashant Mehta
Update StepHow does feedback particle filter implement Bayes’ rule?
Feedback particle filter Kalman filter
Observation: dZt = h(Xt)dt+ dWt dZt = γXt dt+ dWt
Prediction: dZit =
h(Xit )+h2 dt dZt = γXt dt
h = 1N ∑
Ni=1 h(Xi
t)
Innov. error: dIit = dZt− dZi
t dIt = dZt− dZt
= dZt− h(Xit )+h2 dt = dZt− γXt dt
Control: dUit = K(Xi
t) dIit dUt = K dIt
Gain: K is a solution of a linear BVP K is the Kalman gain
Main Result: FPF is an exact algorithm (in the mean-field, N→ ∞, limit).
Yang, Mehta and Meyn. Feedback Particle Filter. IEEE TAC (2013).
Feedback Particle Filter Prashant Mehta 11 / 34 Prashant Mehta
Update StepHow does feedback particle filter implement Bayes’ rule?
Feedback particle filter Kalman filter
Observation: dZt = h(Xt)dt+ dWt dZt = γXt dt+ dWt
Prediction: dZit =
h(Xit )+h2 dt dZt = γXt dt
h = 1N ∑
Ni=1 h(Xi
t)
Innov. error: dIit = dZt− dZi
t dIt = dZt− dZt
= dZt− h(Xit )+h2 dt = dZt− γXt dt
Control: dUit = K(Xi
t) dIit dUt = K dIt
Gain: K is a solution of a linear BVP K is the Kalman gain
Main Result: FPF is an exact algorithm (in the mean-field, N→ ∞, limit).
Yang, Mehta and Meyn. Feedback Particle Filter. IEEE TAC (2013).
Feedback Particle Filter Prashant Mehta 11 / 34 Prashant Mehta
Update StepHow does feedback particle filter implement Bayes’ rule?
Feedback particle filter Kalman filter
Observation: dZt = h(Xt)dt+ dWt dZt = γXt dt+ dWt
Prediction: dZit =
h(Xit )+h2 dt dZt = γXt dt
h = 1N ∑
Ni=1 h(Xi
t)
Innov. error: dIit = dZt− dZi
t dIt = dZt− dZt
= dZt− h(Xit )+h2 dt = dZt− γXt dt
Control: dUit = K(Xi
t) dIit dUt = K dIt
Gain: K is a solution of a linear BVP K is the Kalman gain
Main Result: FPF is an exact algorithm (in the mean-field, N→ ∞, limit).
Yang, Mehta and Meyn. Feedback Particle Filter. IEEE TAC (2013).
Feedback Particle Filter Prashant Mehta 11 / 34 Prashant Mehta
Update StepHow does feedback particle filter implement Bayes’ rule?
Feedback particle filter Kalman filter
Observation: dZt = h(Xt)dt+ dWt dZt = γXt dt+ dWt
Prediction: dZit =
h(Xit )+h2 dt dZt = γXt dt
h = 1N ∑
Ni=1 h(Xi
t)
Innov. error: dIit = dZt− dZi
t dIt = dZt− dZt
= dZt− h(Xit )+h2 dt = dZt− γXt dt
Control: dUit = K(Xi
t) dIit dUt = K dIt
Gain: K is a solution of a linear BVP K is the Kalman gain
Main Result: FPF is an exact algorithm (in the mean-field, N→ ∞, limit).
Yang, Mehta and Meyn. Feedback Particle Filter. IEEE TAC (2013).
Feedback Particle Filter Prashant Mehta 11 / 34 Prashant Mehta
Update StepHow does feedback particle filter implement Bayes’ rule?
Feedback particle filter Kalman filter
Observation: dZt = h(Xt)dt+ dWt dZt = γXt dt+ dWt
Prediction: dZit =
h(Xit )+h2 dt dZt = γXt dt
h = 1N ∑
Ni=1 h(Xi
t)
Innov. error: dIit = dZt− dZi
t dIt = dZt− dZt
= dZt− h(Xit )+h2 dt = dZt− γXt dt
Control: dUit = K(Xi
t) dIit dUt = K dIt
Gain: K is a solution of a linear BVP K is the Kalman gain
Main Result: FPF is an exact algorithm (in the mean-field, N→ ∞, limit).
Yang, Mehta and Meyn. Feedback Particle Filter. IEEE TAC (2013).
Feedback Particle Filter Prashant Mehta 11 / 34 Prashant Mehta
Variance ReductionFiltering for a linear model.
Mean-square error:1T
∫ T
0
(Σ(N)t −Σt
Σt
)2
dt
102 103
10−3
10−2
10−1
N (number of particles)
Bootstrap (BPF)
Feedback (FPF)
MSE
Feedback Particle Filter Prashant Mehta 12 / 34 Prashant Mehta
The Next Few SlidesResults 1 and 2
1. Gradient flow construction:
1 Nonlinear filter is shown to be a gradient flow
2. Optimal transport:
1 Derivation of feedback particle filter
Poisson’s equation is central to both 1 and 2
Details appear in: Laugesen, Mehta, Meyn and Raginsky. Poisson’s equation in nonlinear filtering. SIAM J. Control Optimiz. (2015);Also see: Yang, Mehta and Meyn. Feedback particle filter. IEEE Trans. Automat. Control (2013);Yang, Laugesen, Mehta and Meyn. Multivariable feedback particle filter. Automatica (To Appear).
Feedback Particle Filter Prashant Mehta 13 / 34 Prashant Mehta
The Next Few SlidesResults 1 and 2
1. Gradient flow construction:
1 Nonlinear filter is shown to be a gradient flow
2. Optimal transport:
1 Derivation of feedback particle filter
Poisson’s equation is central to both 1 and 2
Details appear in: Laugesen, Mehta, Meyn and Raginsky. Poisson’s equation in nonlinear filtering. SIAM J. Control Optimiz. (2015);Also see: Yang, Mehta and Meyn. Feedback particle filter. IEEE Trans. Automat. Control (2013);Yang, Laugesen, Mehta and Meyn. Multivariable feedback particle filter. Automatica (To Appear).
Feedback Particle Filter Prashant Mehta 13 / 34 Prashant Mehta
Poisson’s EquationReview of various mathematical forms
Strong form: − ∇ ·∇︸︷︷︸Laplacian
.= ∇2
φ(x) = h(x)− c on domain Ω
Feedback Particle Filter Prashant Mehta 14 / 34 Prashant Mehta
Poisson’s EquationReview of various mathematical forms
Strong form: − ∇ ·∇︸︷︷︸Laplacian
.= ∇2
φ(x) = h(x)− c on domain Ω
Weak form:∫
∇φ ·∇ψ dx =∫(h− c)ψ dx ∀ test fns. ψ ∈ H1
Feedback Particle Filter Prashant Mehta 14 / 34 Prashant Mehta
Poisson’s EquationReview of various mathematical forms
Strong form: − ∇ ·∇︸︷︷︸Laplacian
.= ∇2
φ(x) = h(x)− c on domain Ω
Weak form:∫
∇φ ·∇ψ dx =∫(h− c)ψ dx ∀ test fns. ψ ∈ H1
Generalization:∫
∇φ ·∇ψ p(x)dx =∫(h− c)ψ p(x)dx ∀ψ ∈ H1
Feedback Particle Filter Prashant Mehta 14 / 34 Prashant Mehta
Poisson’s EquationReview of various mathematical forms
Strong form: − ∇ ·∇︸︷︷︸Laplacian
.= ∇2
φ(x) = h(x)− c on domain Ω
Weak form:∫
∇φ ·∇ψ dx =∫(h− c)ψ dx ∀ test fns. ψ ∈ H1
Generalization:∫
∇φ ·∇ψ p(x)dx =∫(h− c)ψ p(x)dx ∀ψ ∈ H1
or: Ep[∇φ ·∇ψ] = Ep[(h− h)ψ] ∀ψ ∈ H1
Feedback Particle Filter Prashant Mehta 14 / 34 Prashant Mehta
Poisson’s EquationReview of various mathematical forms
Strong form: − ∇ ·∇︸︷︷︸Laplacian
.= ∇2
φ(x) = h(x)− c on domain Ω
Weak form:∫
∇φ ·∇ψ dx =∫(h− c)ψ dx ∀ test fns. ψ ∈ H1
Generalization:∫
∇φ ·∇ψ p(x)dx =∫(h− c)ψ p(x)dx ∀ψ ∈ H1
or: Ep[∇φ ·∇ψ] = Ep[(h− h)ψ] ∀ψ ∈ H1
where h = Ep[h] =∫
h(x)p(x)dx
Feedback Particle Filter Prashant Mehta 14 / 34 Prashant Mehta
Poisson’s EquationReview of various mathematical forms
Strong form: − ∇ ·∇︸︷︷︸Laplacian
.= ∇2
φ(x) = h(x)− c on domain Ω
Weak form:∫
∇φ ·∇ψ dx =∫(h− c)ψ dx ∀ test fns. ψ ∈ H1
Generalization:∫
∇φ ·∇ψ p(x)dx =∫(h− c)ψ p(x)dx ∀ψ ∈ H1
or: Ep[∇φ ·∇ψ] = Ep[(h− h)ψ] ∀ψ ∈ H1
where h = Ep[h] =∫
h(x)p(x)dx
Strong form: −∇ · (p(x)∇φ)(x) = (h(x)− c)p(x)
Feedback Particle Filter Prashant Mehta 14 / 34 Prashant Mehta
Poisson’s Equation in PhysicsThis equation is fundamental to many fields!
Electric potential: − 14π
∇2φ = ρ ← charge density
Gravitational potential:1
4πG∇
2φ = ρ ←mass density
Temperature: κ∇2φ = q← heat-flux density
Walter Strauss. Partial Differential Equations. Wiley (1992).
Feedback Particle Filter Prashant Mehta 15 / 34 Prashant Mehta
Gradient flowAn elementary example
Time stepping procedure
x∗(t+∆t) = arg miny
12|y− x∗(t)|2 +∆t h(y)
Feedback Particle Filter Prashant Mehta 16 / 34 Prashant Mehta
Gradient flowAn elementary example
Time stepping procedure
x∗(t+∆t) = arg miny
12|y− x∗(t)|2 +∆t h(y)
Calculus 101: x∗(t+∆t) = x∗(t)−∆t ∇h(x∗(t+∆t))
Feedback Particle Filter Prashant Mehta 16 / 34 Prashant Mehta
Gradient flowAn elementary example
Time stepping procedure
x∗(t+∆t) = arg miny
12|y− x∗(t)|2 +∆t h(y)
Calculus 101: x∗(t+∆t) = x∗(t)−∆t ∇h(x∗(t+∆t))
Cont. limit:dx∗
dt=−∇h(x∗).
Feedback Particle Filter Prashant Mehta 16 / 34 Prashant Mehta
Gradient flowAn elementary example
Time stepping procedure
x∗(t+∆t) = arg miny
12|y− x∗(t)|2︸ ︷︷ ︸
metric
+ ∆t h(y)︸︷︷︸min
Feedback Particle Filter Prashant Mehta 17 / 34 Prashant Mehta
Gradient flowAn elementary example
Time stepping procedure
x∗(t+∆t) = arg miny
12|y− x∗(t)|2︸ ︷︷ ︸
metric
+ ∆t h(y)︸︷︷︸min
Calc. of Variation: 〈x∗(t+∆t),ψ〉= 〈x∗(t),ψ〉−∆t 〈∇h(x∗(t+∆t)),ψ〉
Feedback Particle Filter Prashant Mehta 17 / 34 Prashant Mehta
Gradient flowAn elementary example
Time stepping procedure
x∗(t+∆t) = arg miny
12|y− x∗(t)|2︸ ︷︷ ︸
metric
+ ∆t h(y)︸︷︷︸min
Calc. of Variation: 〈x∗(t+∆t),ψ〉= 〈x∗(t),ψ〉−∆t 〈∇h(x∗(t+∆t)),ψ〉
Cont. limit: 〈x∗(t),ψ〉= 〈x∗(0),ψ〉−∫ t
0〈∇h(x∗(s)),ψ〉ds.
Feedback Particle Filter Prashant Mehta 17 / 34 Prashant Mehta
Gradient Flow Interpretation of Heat Equation1998 paper of Jordon, Kinderlehrer and Otto
Time stepping procedure
p∗t+∆t = arg minρ∈P
12
W22 (ρ,p
∗t )+∆t
∫ρ(x) lnρ(x)dx
Jordon, Kinderlehrer and Otto. The variational formulation of the Fokker-Planck equation. SIAM J. Math. Anal., 29 (1998)
Feedback Particle Filter Prashant Mehta 18 / 34 Prashant Mehta
Gradient Flow Interpretation of Heat Equation1998 paper of Jordon, Kinderlehrer and Otto
Time stepping procedure
p∗t+∆t = arg minρ∈P
12
W22 (ρ,p
∗t )+∆t
∫ρ(x) lnρ(x)dx
E-L equation: . . .
Jordon, Kinderlehrer and Otto. The variational formulation of the Fokker-Planck equation. SIAM J. Math. Anal., 29 (1998)
Feedback Particle Filter Prashant Mehta 18 / 34 Prashant Mehta
Gradient Flow Interpretation of Heat Equation1998 paper of Jordon, Kinderlehrer and Otto
Time stepping procedure
p∗t+∆t = arg minρ∈P
12
W22 (ρ,p
∗t )+∆t
∫ρ(x) lnρ(x)dx
E-L equation: . . .
Cont. limit:∂p∗
∂ t= ∇
2p∗
Jordon, Kinderlehrer and Otto. The variational formulation of the Fokker-Planck equation. SIAM J. Math. Anal., 29 (1998)
Feedback Particle Filter Prashant Mehta 18 / 34 Prashant Mehta
Gradient Flow for Nonlinear FilterConstruction via a time-stepping procedure
Signal & Observations dXt = 0,
dZt = h(Xt)dt+ dWt
Time stepping procedure
p∗t+∆t = arg minρ∈P
D(ρ | p∗t )+∆t2
∫ρ(x)(Yt−h(x))2 dx,
where Yt :=Zt+∆t−Zt
∆t.
Laugesen, Mehta, Meyn and Raginsky. Poisson’s Equation in Nonlinear Filtering. SIAM J. Control Optim (2015)
Feedback Particle Filter Prashant Mehta 19 / 34 Prashant Mehta
Gradient Flow Interpretation of Nonlinear FilterConstruction via a time-stepping procedure
Time stepping procedure
p∗t+∆t = arg minρ∈P
D(ρ | p∗t )+∆t2
∫ρ(x)(Yt−h(x))2 dx,
S. K. Mitter and N. J. Newton. A variational approach to nonlinear estimation. SIAM J. Control Optimiz. (2003);
Feedback Particle Filter Prashant Mehta 20 / 34 Prashant Mehta
Gradient Flow Interpretation of Nonlinear FilterConstruction via a time-stepping procedure
Time stepping procedure
p∗t+∆t = arg minρ∈P
D(ρ | p∗t )+∆t2
∫ρ(x)(Yt−h(x))2 dx,
E-L equation: . . .
S. K. Mitter and N. J. Newton. A variational approach to nonlinear estimation. SIAM J. Control Optimiz. (2003);
Feedback Particle Filter Prashant Mehta 20 / 34 Prashant Mehta
Gradient Flow Interpretation of Nonlinear FilterConstruction via a time-stepping procedure
Time stepping procedure
p∗t+∆t = arg minρ∈P
D(ρ | p∗t )+∆t2
∫ρ(x)(Yt−h(x))2 dx,
E-L equation: . . .
Cont. limit: dp∗ = (h− h)(dZt− hdt)p∗
S. K. Mitter and N. J. Newton. A variational approach to nonlinear estimation. SIAM J. Control Optimiz. (2003);
Feedback Particle Filter Prashant Mehta 20 / 34 Prashant Mehta
Gradient Flow Interpretation of Nonlinear FilterPoisson’s equation?
E-L equation: Ep∗t+∆t[ψ] = Ep∗t [ψ]+Ep∗t+∆t
[(∆Zt−h∆t)∇h ·∇ς ]
Feedback Particle Filter Prashant Mehta 21 / 34 Prashant Mehta
Gradient Flow Interpretation of Nonlinear FilterPoisson’s equation?
E-L equation: Ep∗t+∆t[ψ] = Ep∗t [ψ]+Ep∗t+∆t
[(∆Zt−h∆t)∇h ·∇ς ]
Poisson’s equation: ∇ · (p∗t (x)∇ς(x)) =−(ψ(x)− ψt)p∗t (x)
Feedback Particle Filter Prashant Mehta 21 / 34 Prashant Mehta
Gradient Flow Interpretation of Nonlinear FilterPoisson’s equation?
E-L equation: Ep∗t+∆t[ψ] = Ep∗t [ψ]+Ep∗t+∆t
[(∆Zt−h∆t)∇h ·∇ς ]
Poisson’s equation: ∇ · (p∗t (x)∇ς(x)) =−(ψ(x)− ψt)p∗t (x)
Assumption: Spectral gap
For some λ0 > 0, and for all functions ψ ∈ H1 with Ep∗0[ψ] = 0,
∫|ψ(x)|2p∗0(x)dx≤ 1
λ0
∫|∇ψ(x)|2p∗0(x)dx. [PI(λ0)]
Feedback Particle Filter Prashant Mehta 21 / 34 Prashant Mehta
Gradient Flow Interpretation of Nonlinear FilterResult 1: Derivation of nonlinear filter
Result 1
Certain Technical conditions. The density p∗ is a weak solution of the nonlinear filter with priorp∗0. That is, for any test function ψ ∈ Cc(Rd),
〈ψ,p∗t 〉= 〈ψ,p∗0〉+∫ t
0〈(h− hs)(dZs− hs ds)ψ,p∗s 〉,
where 〈ψ,p∗t 〉.=∫
ψ(x)p∗(x, t)dx.
Feedback Particle Filter Prashant Mehta 22 / 34 Prashant Mehta
Optimal TransportResult 2: Derivation of feedback particle filter
Optimization problem
J(N)(s∗) .= min
s ∑tn
(Itn (stn
#(p∗tn ))−∆t2
Y2tn
),
Feedback Particle Filter Prashant Mehta 23 / 34 Prashant Mehta
Optimal TransportResult 2: Derivation of feedback particle filter
Optimization problem
J(N)(s∗) .= min
s ∑tn
(Itn (stn
#(p∗tn ))−∆t2
Y2tn
),
It(ρ).= D(ρ | p∗t )+
∆t2
∫ρ(x)(Yt−h(x))2 dx
Feedback Particle Filter Prashant Mehta 23 / 34 Prashant Mehta
Optimal TransportResult 2: Derivation of feedback particle filter
Optimization problem
J(N)(s∗) .= min
s ∑tn
(Itn (stn
#(p∗tn ))−∆t2
Y2tn
),
It(ρ).= D(ρ | p∗t )+
∆t2
∫ρ(x)(Yt−h(x))2 dx
st# : Optimal transport
Feedback Particle Filter Prashant Mehta 23 / 34 Prashant Mehta
Optimal TransportResult 2: Derivation of feedback particle filter
Optimization problem
J(N)(s∗) .= min
s ∑tn
(Itn (stn
#(p∗tn ))−∆t2
Y2tn
),
It(ρ).= D(ρ | p∗t )+
∆t2
∫ρ(x)(Yt−h(x))2 dx
st# : Optimal transport
st : dXit = u(Xi
t , t)dt+K(Xit , t)dZt
Feedback Particle Filter Prashant Mehta 23 / 34 Prashant Mehta
Feedback Particle FilterAlgorithm summary
Signal: dXt = a(Xt)dt+ dBt
Observations: dZt = h(Xt)dt+ dWt
Feedback Particle Filter Prashant Mehta 24 / 34 Prashant Mehta
Feedback Particle FilterAlgorithm summary
Signal: dXt = a(Xt)dt+ dBt
Observations: dZt = h(Xt)dt+ dWt
Problem: Approximate the posterior distribution p∗(x, t).
Feedback Particle Filter Prashant Mehta 24 / 34 Prashant Mehta
Feedback Particle FilterAlgorithm summary
Signal: dXt = a(Xt)dt+ dBt
Observations: dZt = h(Xt)dt+ dWt
Problem: Approximate the posterior distribution p∗(x, t).
FPF Algo.: dXit = a(Xi
t)dt+ dBit
+K(Xi, t)(
dZt−12(h(Xi
t)+ ht)dt)
︸ ︷︷ ︸FPF control
Feedback Particle Filter Prashant Mehta 24 / 34 Prashant Mehta
Boundary Value ProblemEuler-Lagrange equation for the variational problem
Multi-dimensional boundary value problem
Gain Fn.: K = ∇φ
∇ · (p∇φ) =−(h− h)p
solved at each time-step.
Feedback Particle Filter Prashant Mehta 25 / 34 Prashant Mehta
Boundary Value ProblemEuler-Lagrange equation for the variational problem
Multi-dimensional boundary value problem
Gain Fn.: K = ∇φ
∇ · (p∇φ) =−(h− h)p
solved at each time-step.
Linear case:
Feedback Particle Filter Prashant Mehta 25 / 34 Prashant Mehta
Boundary Value ProblemEuler-Lagrange equation for the variational problem
Multi-dimensional boundary value problem
Gain Fn.: K = ∇φ
∇ · (p∇φ) =−(h− h)p
solved at each time-step.
Linear case:
Feedback Particle Filter Prashant Mehta 25 / 34 Prashant Mehta
Boundary Value ProblemEuler-Lagrange equation for the variational problem
Multi-dimensional boundary value problem
Gain Fn.: K = ∇φ
∇ · (p∇φ) =−(h− h)p
solved at each time-step.
Linear case: Nonlinear case:
Feedback Particle Filter Prashant Mehta 25 / 34 Prashant Mehta
Boundary Value ProblemEuler-Lagrange equation for the variational problem
Multi-dimensional boundary value problem
Gain Fn.: K = ∇φ
∇ · (p∇φ) =−(h− h)p
solved at each time-step.
Linear case: Nonlinear case:
Feedback Particle Filter Prashant Mehta 25 / 34 Prashant Mehta
Boundary Value ProblemEuler-Lagrange equation for the variational problem
Multi-dimensional boundary value problem
Gain Fn.: K = ∇φ
∇ · (p∇φ) =−(h− h)p
solved at each time-step.
Linear case: Nonlinear case:
Feedback Particle Filter Prashant Mehta 25 / 34 Prashant Mehta
Summary
Kalman Filter
Kalman Filter
-
+
Innovation Error:
dIt = dZt−h(X)dt
Gain Function:
K = Kalman Gain
Feedback Particle Filter
Feedback Particle Filter
-
+
Innovation Error:
dIit = dZt−
12(h(Xi
t)+ ht)
dt
Gain Function:
K is solution of a linear BVP.
Yang, Mehta and Meyn. Feedback Particle Filter. IEEE TAC (2013).
Feedback Particle Filter Prashant Mehta 26 / 34 Prashant Mehta
Summary
Kalman Filter
Kalman Filter
-
+
Innovation Error:
dIt = dZt−h(X)dt
Gain Function:
K = Kalman Gain
Feedback Particle Filter
Feedback Particle Filter
-
+
Innovation Error:
dIit = dZt−
12(h(Xi
t)+ ht)
dt
Gain Function:
K is solution of a linear BVP.
Yang, Mehta and Meyn. Feedback Particle Filter. IEEE TAC (2013).
Feedback Particle Filter Prashant Mehta 26 / 34 Prashant Mehta
Summary
Kalman Filter
Kalman Filter
-
+
Innovation Error:
dIt = dZt−h(X)dt
Gain Function:
K = Kalman Gain
Feedback Particle Filter
Feedback Particle Filter
-
+
Innovation Error:
dIit = dZt−
12(h(Xi
t)+ ht)
dt
Gain Function:
K is solution of a linear BVP.
Yang, Mehta and Meyn. Feedback Particle Filter. IEEE TAC (2013).
Feedback Particle Filter Prashant Mehta 26 / 34 Prashant Mehta
Coupled OscillatorsKuramoto model
dθit =
(ωi +
κ
N
N
∑j=1
sin(θ jt −θ
it )
)dt+σ dξ
it , i = 1, . . . ,N
ωi taken from distribution g(ω) over [1− γ,1+ γ]
γ — measures the heterogeneity of the population
κ — measures the strength of coupling
Y. Kuramoto. Self-entrainment of a population of coupled nonlinear oscillators (1975);Strogatz and Mirollo. Stability of incoherence in a population of coupled oscillators. J. Stat. Phy. (1991);N. Kopell and G. B. Ermentrout. Symmetry and phaselocking in chains of weakly coupled oscillators. Commun. Pure Appl. Math. (1986)
Feedback Particle Filter Prashant Mehta 27 / 34 Prashant Mehta
Coupled OscillatorsKuramoto model
dθit =
(ωi +
κ
N
N
∑j=1
sin(θ jt −θ
it )
)dt+σ dξ
it , i = 1, . . . ,N
ωi taken from distribution g(ω) over [1− γ,1+ γ]
γ — measures the heterogeneity of the population
κ — measures the strength of coupling 1- 1+1
Y. Kuramoto. Self-entrainment of a population of coupled nonlinear oscillators (1975);Strogatz and Mirollo. Stability of incoherence in a population of coupled oscillators. J. Stat. Phy. (1991);N. Kopell and G. B. Ermentrout. Symmetry and phaselocking in chains of weakly coupled oscillators. Commun. Pure Appl. Math. (1986)
Feedback Particle Filter Prashant Mehta 27 / 34 Prashant Mehta
Coupled OscillatorsKuramoto model
dθit =
(ωi +
κ
N
N
∑j=1
sin(θ jt −θ
it )
)dt+σ dξ
it , i = 1, . . . ,N
ωi taken from distribution g(ω) over [1− γ,1+ γ]
γ — measures the heterogeneity of the population
κ — measures the strength of coupling
Y. Kuramoto. Self-entrainment of a population of coupled nonlinear oscillators (1975);Strogatz and Mirollo. Stability of incoherence in a population of coupled oscillators. J. Stat. Phy. (1991);N. Kopell and G. B. Ermentrout. Symmetry and phaselocking in chains of weakly coupled oscillators. Commun. Pure Appl. Math. (1986)
Feedback Particle Filter Prashant Mehta 27 / 34 Prashant Mehta
Coupled OscillatorsKuramoto model
dθit =
(ωi +
κ
N
N
∑j=1
sin(θ jt −θ
it )
)dt+σ dξ
it , i = 1, . . . ,N
ωi taken from distribution g(ω) over [1− γ,1+ γ]
γ — measures the heterogeneity of the population
κ — measures the strength of coupling
0 0.1 0.20.1
0.2
0.3
Incoherence
Synchrony
Y. Kuramoto. Self-entrainment of a population of coupled nonlinear oscillators (1975);Strogatz and Mirollo. Stability of incoherence in a population of coupled oscillators. J. Stat. Phy. (1991);N. Kopell and G. B. Ermentrout. Symmetry and phaselocking in chains of weakly coupled oscillators. Commun. Pure Appl. Math. (1986)
Feedback Particle Filter Prashant Mehta 27 / 34 Prashant Mehta
Hodgkin-Huxley type Neuron modelNormal form reduction
CdVdt
=−gT ·m2∞(V) ·h · (V−ET )
−gh · r · (V−Eh)− . . . . . .
dhdt
=h∞(V)−h
τh(V)
drdt
=r∞(V)− r
τr(V)2000 2200 2400 2600 2800 3000 3200 3400 3600 3800 4000
−150
−100
−50
0
50
100
Voltage
time
Neural spike train
J. Guckenheimer. Isochrons and phaseless sets. J. Math. Biol. (1975);Brown, Moehlis and Holmes. On the phase reduction and response dynamics of neural oscillator populations. Neural Computation (2004);E. M. Izhikevich. Dynamical Systems in Neuroscience. in Chapter 10. The MIT Press (2006).
Feedback Particle Filter Prashant Mehta 28 / 34 Prashant Mehta
Hodgkin-Huxley type Neuron modelNormal form reduction
CdVdt
=−gT ·m2∞(V) ·h · (V−ET )
−gh · r · (V−Eh)− . . . . . .
dhdt
=h∞(V)−h
τh(V)
drdt
=r∞(V)− r
τr(V)2000 2200 2400 2600 2800 3000 3200 3400 3600 3800 4000
−150
−100
−50
0
50
100
Voltage
time
Neural spike train
J. Guckenheimer. Isochrons and phaseless sets. J. Math. Biol. (1975);Brown, Moehlis and Holmes. On the phase reduction and response dynamics of neural oscillator populations. Neural Computation (2004);E. M. Izhikevich. Dynamical Systems in Neuroscience. in Chapter 10. The MIT Press (2006).
Feedback Particle Filter Prashant Mehta 28 / 34 Prashant Mehta
Hodgkin-Huxley type Neuron modelNormal form reduction
CdVdt
=−gT ·m2∞(V) ·h · (V−ET )
−gh · r · (V−Eh)− . . . . . .
dhdt
=h∞(V)−h
τh(V)
drdt
=r∞(V)− r
τr(V)2000 2200 2400 2600 2800 3000 3200 3400 3600 3800 4000
−150
−100
−50
0
50
100
Voltage
time
Neural spike train
−100
−50
0
50
100
0
0.2
0.4
0.6
0.8
10
0.1
0.2
0.3
0.4
Vh
r
Limit cyle
r
h v
J. Guckenheimer. Isochrons and phaseless sets. J. Math. Biol. (1975);Brown, Moehlis and Holmes. On the phase reduction and response dynamics of neural oscillator populations. Neural Computation (2004);E. M. Izhikevich. Dynamical Systems in Neuroscience. in Chapter 10. The MIT Press (2006).
Feedback Particle Filter Prashant Mehta 28 / 34 Prashant Mehta
Hodgkin-Huxley type Neuron modelNormal form reduction
CdVdt
=−gT ·m2∞(V) ·h · (V−ET )
−gh · r · (V−Eh)− . . . . . .
dhdt
=h∞(V)−h
τh(V)
drdt
=r∞(V)− r
τr(V)2000 2200 2400 2600 2800 3000 3200 3400 3600 3800 4000
−150
−100
−50
0
50
100
Voltage
time
Neural spike train
−100
−50
0
50
100
0
0.2
0.4
0.6
0.8
10
0.1
0.2
0.3
0.4
Vh
r
Limit cyle
r
h v
Normal form reduction−−−−−−−−−−−−−→
θi = ωi +ui ·Φ(θi)
J. Guckenheimer. Isochrons and phaseless sets. J. Math. Biol. (1975);Brown, Moehlis and Holmes. On the phase reduction and response dynamics of neural oscillator populations. Neural Computation (2004);E. M. Izhikevich. Dynamical Systems in Neuroscience. in Chapter 10. The MIT Press (2006).
Feedback Particle Filter Prashant Mehta 28 / 34 Prashant Mehta
Gait CycleSignal model
Stance phase Swing phase
Model (Noisy oscillator)
dθt = ω0 dt︸︷︷︸natural frequency
+ noise
Feedback Particle Filter Prashant Mehta 29 / 34 Prashant Mehta
Gait CycleSignal model
Stance phase Swing phase
Model (Noisy oscillator)
dθt = ω0 dt︸︷︷︸natural frequency
+ noise
Feedback Particle Filter Prashant Mehta 29 / 34 Prashant Mehta
Gait CycleSignal model
Stance phase Swing phase
Model (Noisy oscillator)
dθt = ω0 dt︸︷︷︸natural frequency
+ noise
Feedback Particle Filter Prashant Mehta 29 / 34 Prashant Mehta
Gait CycleSignal model
Stance phase Swing phase
Model (Noisy oscillator)
dθt = ω0 dt︸︷︷︸natural frequency
+ noise
Feedback Particle Filter Prashant Mehta 29 / 34 Prashant Mehta
Gait CycleSignal model
Stance phase Swing phase
Model (Noisy oscillator)
dθt = ω0 dt︸︷︷︸natural frequency
+ noise
Feedback Particle Filter Prashant Mehta 29 / 34 Prashant Mehta
Gait CycleSignal model
Stance phase Swing phase
Model (Noisy oscillator)
dθt = ω0 dt︸︷︷︸natural frequency
+ noise
Feedback Particle Filter Prashant Mehta 29 / 34 Prashant Mehta
Gait CycleSignal model
Stance phase Swing phase
Model (Noisy oscillator)
dθt = ω0 dt︸︷︷︸natural frequency
+ noise
Feedback Particle Filter Prashant Mehta 29 / 34 Prashant Mehta
Gait CycleSignal model
Stance phase Swing phase
Model (Noisy oscillator)
dθt = ω0 dt︸︷︷︸natural frequency
+ noise
Feedback Particle Filter Prashant Mehta 29 / 34 Prashant Mehta
Simulation ResultsSolution of the Estimation of Gait Cycle Problem
feedback particle filter
dynamics estimate
noisymeasurements
[Click to play the movie]
Tilton, Hsiao-Wecksler and Mehta. Filtering with rhythms: Application to estimation of gait cycle. American Control Conference (2012).
Feedback Particle Filter Prashant Mehta 30 / 34 Prashant Mehta
Geometric ControlLocomotion Systems
shape variables: x1,x2group variable: ψ
3-body system
P. S. Krishnaprasad. Geometric phases and optimal reconfiguration for multibody systems. UM Tech. Report (1990);P. S. Krishnaprasad. Motion control and coupled oscillators. Procs. of Symp. on Motion, Control & Geometry. Natl. Acad. Sciences (1995);R. Brockett. Pattern generation and the control of nonlinear systems. IEEE TAC (2003);S. Kelly and R. Murray. Geometric phases and robotic locomotion. J. Robotic Systems (1995); R. Murray and S. Sastry. Nonholonomic motion planning:Steering using sinusoids. IEEE TAC (1993); J. Blair and T. Iwasaki. Optimal gaits for mechanical rectifier systems. IEEE TAC (2011);
Feedback Particle Filter Prashant Mehta 31 / 34 Prashant Mehta
Geometric ControlLocomotion Systems
shape variables: x1,x2group variable: ψ
3-body system
P. S. Krishnaprasad. Geometric phases and optimal reconfiguration for multibody systems. UM Tech. Report (1990);P. S. Krishnaprasad. Motion control and coupled oscillators. Procs. of Symp. on Motion, Control & Geometry. Natl. Acad. Sciences (1995);R. Brockett. Pattern generation and the control of nonlinear systems. IEEE TAC (2003);S. Kelly and R. Murray. Geometric phases and robotic locomotion. J. Robotic Systems (1995); R. Murray and S. Sastry. Nonholonomic motion planning:Steering using sinusoids. IEEE TAC (1993); J. Blair and T. Iwasaki. Optimal gaits for mechanical rectifier systems. IEEE TAC (2011);
Feedback Particle Filter Prashant Mehta 31 / 34 Prashant Mehta
Geometric ControlLocomotion Systems
shape variables: x1,x2group variable: ψ
3-body system
P. S. Krishnaprasad. Geometric phases and optimal reconfiguration for multibody systems. UM Tech. Report (1990);P. S. Krishnaprasad. Motion control and coupled oscillators. Procs. of Symp. on Motion, Control & Geometry. Natl. Acad. Sciences (1995);R. Brockett. Pattern generation and the control of nonlinear systems. IEEE TAC (2003);S. Kelly and R. Murray. Geometric phases and robotic locomotion. J. Robotic Systems (1995); R. Murray and S. Sastry. Nonholonomic motion planning:Steering using sinusoids. IEEE TAC (1993); J. Blair and T. Iwasaki. Optimal gaits for mechanical rectifier systems. IEEE TAC (2011);
Feedback Particle Filter Prashant Mehta 31 / 34 Prashant Mehta
Geometric ControlLocomotion Systems
shape variables: x1,x2group variable: ψ
3-body system
x = f (x, x,τ)
τ: torque input
P. S. Krishnaprasad. Geometric phases and optimal reconfiguration for multibody systems. UM Tech. Report (1990);P. S. Krishnaprasad. Motion control and coupled oscillators. Procs. of Symp. on Motion, Control & Geometry. Natl. Acad. Sciences (1995);R. Brockett. Pattern generation and the control of nonlinear systems. IEEE TAC (2003);S. Kelly and R. Murray. Geometric phases and robotic locomotion. J. Robotic Systems (1995); R. Murray and S. Sastry. Nonholonomic motion planning:Steering using sinusoids. IEEE TAC (1993); J. Blair and T. Iwasaki. Optimal gaits for mechanical rectifier systems. IEEE TAC (2011);
Feedback Particle Filter Prashant Mehta 31 / 34 Prashant Mehta
Geometric ControlLocomotion Systems
shape variables: x1,x2group variable: ψ
3-body system
x = f (x, x,τ)
τ: torque input
ψ = a1(x)x1 +a2(x)x2
Reconstruction equation
P. S. Krishnaprasad. Geometric phases and optimal reconfiguration for multibody systems. UM Tech. Report (1990);P. S. Krishnaprasad. Motion control and coupled oscillators. Procs. of Symp. on Motion, Control & Geometry. Natl. Acad. Sciences (1995);R. Brockett. Pattern generation and the control of nonlinear systems. IEEE TAC (2003);S. Kelly and R. Murray. Geometric phases and robotic locomotion. J. Robotic Systems (1995); R. Murray and S. Sastry. Nonholonomic motion planning:Steering using sinusoids. IEEE TAC (1993); J. Blair and T. Iwasaki. Optimal gaits for mechanical rectifier systems. IEEE TAC (2011);
Feedback Particle Filter Prashant Mehta 31 / 34 Prashant Mehta
Control of Locomotion Gaits2-body System
ψ = f (x)x
A. Taghvaei, S. Hutchinson and P. G. Mehta. A coupled-oscillators-based control architecture for locomotory gaits. IEEE CDC (2014).
Feedback Particle Filter Prashant Mehta 32 / 34 Prashant Mehta
Control of Locomotion Gaits2-body System
ψ = f (x)x = f (θ)
A. Taghvaei, S. Hutchinson and P. G. Mehta. A coupled-oscillators-based control architecture for locomotory gaits. IEEE CDC (2014).
Feedback Particle Filter Prashant Mehta 32 / 34 Prashant Mehta
Control of Locomotion Gaits2-body System
ψ = f (x)x = f (θ ,u)
A. Taghvaei, S. Hutchinson and P. G. Mehta. A coupled-oscillators-based control architecture for locomotory gaits. IEEE CDC (2014).
Feedback Particle Filter Prashant Mehta 32 / 34 Prashant Mehta
Control of Locomotion Gaits2-body System
ψ = f (x)x = f (θ ,u)
minu[0,T]
E[ψ(T)−ψ(0)︸ ︷︷ ︸
Geom. phase
+1
2ε
∫ T
0u(t)2dt
]
A. Taghvaei, S. Hutchinson and P. G. Mehta. A coupled-oscillators-based control architecture for locomotory gaits. IEEE CDC (2014).
Feedback Particle Filter Prashant Mehta 32 / 34 Prashant Mehta
2-body system, Simulation Result
Particles True Phase
t
−π3
0
π3
x
x(t) Observation: y(t)
0 10 20 30 40 50 60 70 80t
−1.0
−0.8
−0.6
−0.4
−0.2
0.0
0.2
q1
open loop: q1(t) close loop: q1(t)
↑
→
→
↓
[Click to play the movie]
A. Taghvaei, S. Hutchinson and P. G. Mehta. A coupled-oscillators-based control architecture for locomotory gaits. IEEE CDC (2014).
Feedback Particle Filter Prashant Mehta 33 / 34 Prashant Mehta
AcknowledgementStudents in red
1. Feedback particle filter:
Tao Yang, Rick Laugesen, Sean Meyn, Max Raginsky
2. Coupled oscillators for estimation:
Adam Tilton, Shane Ghiotto, Liz Hsiao-Wecksler
3. Coupled oscillators for control:
Amirhossein Taghvaei, Seth Hutchinson
Research supported by NSF
Feedback Particle Filter Prashant Mehta 34 / 34 Prashant Mehta