Kalman Filters

transcript

Kalman Filters ELE 774 - Adaptive Signal Processing 1

Kalman Filters

ELE 774 - Adaptive Signal Processing 2Kalman Filters

Introduction

Mathematical formulation is described by state space concepts.

Solution is computed recursively. Both stationary and also non-stationary environments.

Each updated estimate of the state computed from The previous estimate, and The new input data (innovation).

A unifying framework for the family of recursive least-squares (RLS) filters.

Recursive MMS Estimation for Scalar RVs

Assume a complete set of observed r.v.s upto time n-1y(1), y(2), ..., y(n-1)

Let the minimum mean-square estimate of the zero mean x(n-1) be

where is the space spanned by the observations y(1) ... y(n-1).

Let there be a new observation y(n), We estimate x(n) using the observations y(1), y(2),...,y(n-1), y(n)

Do this either by storing y(1), y(2),...,y(n-1) and redo the whole problem, or

Exploit and use the new observation y(n), i.e. use a recursive estimation procedure.

What is new in the new observation y(n)? Innovations!

Define the forward prediction error

Prediction order (n-1) increases linearly with n.

According to principle of orthogonality, fn-1(n) is orthogonal to y(1), y(2), ..., y(n-1).

fn-1(n) is a measure of the new information in y(n) ═> innovations!

Information provided by y(n) is composed of two parts One that in not new, contained in One that is new, contained in

one-step prediction of y(n)

Refer to the prediction error as innovations, and define

Properties of the innovation (n):

Property 1: (n) is orthogonal to the observations y(1), y(2), ...,y(n-1)

Follows from the principle of orthogonality.

Property 2: (1), (2), ..., (n) are orthogonal to each other

Innovations process is white.

Follow from

Property 3: There is one to one correspondence between

One sequence may be obtained from the other by means of a causal and causally invertible filter without any loss of information.

To show this use Gram-Schmidt orthogonalization procedure:

Collecting all terms together

kth row of the matrix gives the coefficients of the forward prediction-error filter of order k-1.

The innovations can be calculated from the observations, or The observations can be calculated from the innovations

There is no loss of information in this transformation.

or, equivalently

Clearly,

Recalling that innovations are orthogonal to each other, and

choosing bk to minimize the mean-square estimation error

we get

Now rewrite

Adding a correction term bn(n) to the previous estimate gives the updated estimate , can be calculated recursively.

Predictor – Corrector structure The use of observations to compute a forward prediction error – innovations, The use of the innovation to update (correct) the minimum mean-square

estimate of a r.v. related linearly to the observations

Discrete-Time Dynamical System

A linear discrete-time dynamical system can be characterized by

Process Equation

Measurement Equation

The state vector, x(n), is the minimal set of data that is sufficient to uniquely describe the unforced dynamical behaviour of the system. fewest data on the past behaviour needed to predict the future one. Dimension M.

The observation vector, y(n), is the set of observed data. Dimension N.

The process equation:

models an unknown physical stochastic phenomenon denoted by the state x(n) as the output of a linear dynamical system excited by the white noise 1(n).

Properties of the transition matrix 1. Product rule

2. Inverse rule

Corollary

Transition matrix

The measurement equation

gives the relation between the state x(n) and the output y(n), with zero-mean white measurement noise (disturbance) 2(n)

Initial value of the state : x(0) uncorrelated with both 1(n) and 2(n),

Noise vectors 1(n) and 2(n) are statistically independent

Kalman Filtering

We need to solve these eqn.s jointly to find the state x(n)

Use the entire observed data, consisting of the observations y(1), y(2), ..., y(n) to find the minimum mean-square estimate of the state x(i), n ≥ 1

i = n, filtering, i > n, prediction, i < n, smoothing.

Process Equation

Measurement Equation

The Innovations Process

Let the MMS estimate of y(n) be Span of the vector space is y(1), y(2), ..., y(n-1)

The innovations process associated with y(n)

(Similar to the scalar case)

the Mx1 vector (n) represents the new information in the observed data y(n).

Properties of the innovations process: Property I: (n) is orthogonal to all past observations y(1), ..., y(n-1)

Property II: The innovations process consists of a sequence of vector random

variables that are orthogonal to each other

Property III: There is a one-to-one correspondence between the observations and

the observation process. One sequence may be obtained from the other by means of linear stable operators, without loss of information.

Correlation Matrix of the Innovations Process Starting from initial state (n=0), we write

i.e. x(k) is a linear combination of x(0), 1(1), ..., 1(k-1)

We know that , then 1.

Recall that

Then, given the past decisions , i.e. , the MMS estimate of y(n)

Predicted state-error vector at time n using data up to time n-1.

Autocorrelation of the innovation process (n)

where the predicted state-error correlation matrix is

statistical description of the error in the predicted estimate

Estimation of the State

Minimum mean-square estimation of the state x(n)

Estimate may be expressed as a linear combination of the sequence of innovations process:

Using principle of orthogonality

Minimum mean-square estimate of x(n)

Let i=n+1

Summation in is

Kalman Gain Define

Convenient way to calculate the Kalman gain

Hence,Kalman gain becomes

Rewriting the estimate

have to be calculated at each iteration! Let’s make it recursive.

Kalman gain computer

Riccati Equation: Predicted state-error vector:

After substitution and manipulations

We want to find K(n,n-1), hence from previous slide

Then using (1) and (2), we get the Riccati difference equation

Riccati equation solver

Kalman’s one-step prediction algorithm:

One step prediction algorithm

Filtering

Compute the filtered estimate by using the one-step prediction algorithm.

The state x(n) and the process noise 1(n) are independent of each other.

The MMSE estimate x(n+1) given the observations upto time n is

where second line follows from the fact that y(n) and 1(n) are

independent of each other.

Filtering Filtered Estimation Error and Conversion Factor Define the filtered estimation error vector

We know that

Conversionfactor

Filtering

Substituting

Filtered State-Error Correlation Matrix Define filtered state-error vector

Filtering

Manipulations give us

Initial Conditions: Initial state of the process equation is not precisely known

In the absence of any observed data at n=0, let

andand

if x is zero mean

Filtering

Kalman filter based on one-step prediction

Summary

Kalman Filters

Documents