American Institute of Aeronautics and Astronautics

1

Nonlinear Stochastic Control Part I: A Moment-based Approach

Yunjun Xu1

Department of Mechanical, Materials, and Aerospace Engineering University of Central Florida, Orlando, FL, 32816

Prakash Vedula2 School of Aerospace and Mechanical Engineering, University of Oklahoma, Norman, OK, 73019

This paper describes a new stochastic control methodology for nonlinear affine systems subject to parametric and functional uncertainties, with random excitations. The primary objective of this method is to control the statistical nature of the state of a nonlinear system to designed (attainable) statistical properties (e.g. moments). This methodology involves a constrained optimization problem for obtaining the undetermined control parameters, where the norm of the error between the desired and actual stationary moments of state/output responses is minimized subject to constraints on moments corresponding to a stationary distribution. To overcome the difficulties in solving the associated Fokker-Planck equation, generally experienced in nonlinear stochastic control and filtering problems, an approximation using the direct quadrature method of moments is proposed. In this innovative approach, the state probability density function is expressed in terms of a finite collection of Dirac delta functions, with the associated weights and locations determined by moment equations. The advantages of the proposed method are: (1) robustness with respect to parametric and functional uncertainties; (2) ability to control any specified stationary moments of the states/output probability density function; and (3) the state process can be Non-Gaussian. A numerical simulation is used to demonstrate the capability of the proposed nonlinear stochastic control method.

I. Introduction ost applications of dynamical systems have intrinsic uncertainties such as environmental disturbance, random noise, and approximation in mathematical modeling. Control of a stochastic dynamical system is a more

challenging problem as compared to its deterministic counterpart. Monte Carlo simulation approaches, which are commonly used in analysis of statistical dynamic systems, can be used to find proper control parameters, if a desired statistic of the closed loop system performance is required. However, these become intractable when the system is large which may result in a prohibitive number of control design iterations (through trial and error) and CPU time. To reduce the computational cost and provide a systematic design methodology, extensive research efforts were spent for both linear and nonlinear stochastic systems, as discussed below.

On one hand, the stochastic control for linear systems has been well studied such that desired statistical behavior of state variables (e.g. mean and variance) [1-6] is approximately ensured. Just to list a few, an LQG type regulator is formulated by Davis and Vinter in [1], and an observer based covariance control method has been proposed by Iwasaki [2] for linear systems. Sinha and Miller [3] designed an optimal sliding mode regulator for a linear stochastic system where the linear dynamics is assumed to perfectly known. Bratus et al. [5] proposed an optimal control (in the nonlinear form) for linear systems excited by a zero-mean Gaussian white noise forces through the Hamilton-Jacobi-Bellman equation. Grigoriadis and Skelton [4] designed the minimum energy covariance control for linear systems with a zero mean white noise process. The shortcomings of the above mentioned linear stochastic control methods are obvious as (1) one might want to attain a state with a desired mean and covariance instead of a

1 Assistant Professor, The Department of Mechanical, Materials, and Aerospace Engineering, University of

Central Florida, Email: yunjunxu@mail.ucf.edu, AIAA Senior Member 2 Assistant Professor, School of Aerospace and Mechanical Engineering, University of Oklahoma, Email:

pvedula@ou.edu.

M

AIAA Guidance, Navigation, and Control Conference10 - 13 August 2009, Chicago, Illinois

AIAA 2009-5627

Copyright © 2009 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved.

American Institute of Aeronautics and Astronautics

2

scalar weight performance index (e.g. quadratic form index) [6] and (2) the approximation as linear system is only valid in a small region around the trim point.

On the other hand, the nonlinear robust control of stochastic systems is always a challenging task and has attracted extensive interests. The methods investigated to date can be broadly put into three categories.

The first method used in handling nonlinear stochastic control problems is to linearize the system so that linear stochastic control methods [7] could be applied. However, this technique suffers from the same drawbacks as LQG type control and only works in a small region around the linearization points [8].

The second approach to solve nonlinear stochastic control problems is via the solution of the associated Fokker-Planck Equation once the structure of the closed loop system is known. In order to evaluate the expectation or covariance, one must have knowledge of the response PDF [9]. However for nonlinear systems, the exact PDF is typically unknown and just like the case in the FPE and Bayes’ based nonlinear filtering, solving the associated Fokker-Planck Equation (FPE) [10-14] make this problem cumbersome or unsolvable with few special exceptions [15] due to the curse of dimensionality. To mitigate the computational cost, methods such as path integral method [16], cell-mapping method [17] and adaptive grids methods [18-20] have been tried. However, the computational cost is still high even for low dimension nonlinear control problems.

Many interests have been attracted to the somewhat promising third category, consisting of approximation methods [21-25], through which the pre-specified response mean and covariance in steady state can be achieved. The Gaussian Closure method investigated by Sun and Hsu [21] is one of them, in which the first and second order moments are controlled through the proposed sliding mode control. In this approach, the moments of order higher than two are approximated in terms of the first and second order moments through the Gaussian Closure method.

However the expectation of nonlinear terms, such as 31[ ( )]E x sign s , have to be integrated over a domain big enough

such that majority of the probability can be captured. In the approach proposed by Sobczy and Wojtkiewicz in [8, 22], a maximum entropy method has been applied thus a system of nonlinear equation can be formed for user-specified response moments. The availability of the response PDF makes the evaluation of higher order response moments fairly effective, even though the PDF is approximate. In the paper by Chang [23], describing functions were applied to study the covariance control for a nonlinear system. Kim and Rock [24] used dual properties in designing a suboptimal stochastic control where the feedback control structure is pre-defined and the mean value of the performance is controlled. However, the weighting matrix for the error covariance needs to be diagonal and no equality or inequality constraints can be included. Forbes [25] used Gram-Charlier expansions as the PDF basis functions and obtained an approximately parameterized response PDF to track the target PDF in steady state, however, without a guaranteed stability.

In brief, most of the methods mentioned above have one or several of the following limitations: (a) the difficulty in solving the FPE, (b) some expectations of nonlinear terms cannot be obtained effectively, (c) the form of the response PDF has to be know a priori (d) it is not easy to handle inequality and equality constraints and (e) stability is not guaranteed.

In the present work, these limitations for nonlinear stochastic control problems will be addressed based on the novel quadrature based moment approach (referred as the direct quadrature method of moments - DQMOM), along with an asymptotically stable nonlinear tracking control. This approach involves representing the state PDF in terms of a finite summation of Dirac delta functions, whose weights and locations (abscissas) are determined based on moments constraints. Using a small number of scalars, the method is able to efficiently and accurately model stochastic processes described by the multidimensional FPE through a set of ordinary differential equations (ODEs). Together with the DQMOM approach [26, 42], a nonlinear controller is designed here based on the concepts of sliding manifold and input-output feedback linearization with guaranteed asymptotic tracking stability. Different from the commonly used sliding mode control (SMC) [27-36], the high speed switching (discontinuous) function shown in the SMC has been removed to satisfy the continuity requirement in the partial derivatives of the FPE, and in addition the inherent chattering problem experienced in the commonly used SMC can be eliminated.

The main contributions of the paper can be summarized as follow. First, the controller proposed is robust to bounded parametric as well functional uncertainties. The existence of a finite moment index implies that the control system is stable up to the highest order of moments included in the design. Second, selected order moments of the variables can be controlled accurately in steady state. Third, the state process need not be a Gaussian.

The rest of this paper is organized as follows. First, the system model of an affine nonlinear stochastic system is described along with the control objectives. Second, a nonlinear robust control method is proposed based on the concepts of input-output feedback linearization and sliding manifold. Following this, the governing equations of the weights and abscissas, which are used to represent the state PDF, are derived from the FPE using the proposed quadrature based moment approach (DQMOM). Next, the undetermined control parameters, weights and abscissas

American Institute of Aeronautics and Astronautics

3

are optimized offline through a constrained nonlinear optimization. Finally, a numerical example is illustrated followed by the conclusion.

II. The System Model and Control Objectives Let us consider the following affine nonlinear stochastic differential equation (SDE) with additive noise

( )1 1 1

1 1

( ,..., , ) ( ,..., ) ( ,..., ) ( ), 1,...,w

i

Nmn

i i n ij n j ij n jj j

x f t b u g w t i n= =

= + + = x x x x x x (1)

and an output model to be

1( ,..., ), 1,...,i i ny h i p= =x x (2)

where ( 1),..., i iTn n

i i ix x − = ∈ℜ x and ( 1) 1 1/i i in n ni ix d x dt− − − are states with up to 1in − derivatives.

1= [ ,..., ]T mmu u ∈ℜu is the control input and 1( ,..., ) n m

ij nb × = ∈ℜ B x x is the input matrix with linearly

independent vector fields. 1= [ ,..., ]T nnf f ∈ℜf is the nonlinear state function. The relative degree for the output

1[ ,..., ]T ppy y= = ∈ℜy h is 1[ ,..., ]T p

pr r= ∈ℜr . To avoid numerical errors in the pseudo inverse associated with

the proposed controller, p m≤ is required. The case when p m> , singular perturbation or multi-time scale

decomposition methods can be used [37]. Furthermore, in this model, ( ) wNt ∈ℜw is assumed to be a Weiner

process [10] with a zero-mean and a covariance matrix of ( )tQ , and 1( ,..., ) wn Nij ng × = ∈ℜ G x x is the associated

diffusion matrix. The control objectives are (1) to stabilize the system while being capable of tracking the desired trajectory

, , 1,...,i dy i p= (the subscript d denotes the desired signal), and (2) to achieve desired stationary moments of the

states/output probability density function (PDF). For example, the desired stationary PDF distribution can be characterized by (but not limited to) the stationary values of mean, variance, and high order moments.

The nonlinear robust controller will be designed based on the following nominal system

( )1 1

1

ˆ ˆ( ,..., , ) ( ,..., ) , 1,...,i

mn

i i n ij n jj

x f t b u i n=

= + = xx x x (3)

with a nominal output to be

1ˆ ( ,..., ), 1,...,i i ny h i p= =x x (4)

where ∧ represents the nominal information, 1ˆ ˆˆ [ ,..., ]T n

nf f= ∈ℜf and 1ˆˆ ( ,..., ) n mij nb × = ∈ℜ B x x . The parametric

uncertainties of the input matrix are bounded by , , 1,...,ij ijD i j pΔ ≤ = as

1 1ˆ ˆ

ˆ( ) ( ) ( ) , p pL L L L+

− − × + = ∈ℜ r r

B f B fI Δ h x h x Δ (5)

where I is an identity matrix with the proper dimension. B and Β̂ are assumed to satisfy the matching condition, i.e. the maximum eigenvalue of the matrix D satisfy max ( ) 1λ <D . “ L ” and “ + ” are used to denote the Lie

derivative and the pseudo inverse, respectively. The error between the nominal and actual state functions is bounded by 1[ ,..., ]T p

pF F= ∈ℜF as

ˆˆ , 1,...,i ir r

i i iF L h L h i p= − + =f f (6)

Note that the bounds of the noise are not included which will be handled by the proposed nonlinear stochastic control later.

III. Nonlinear Robust Control In this section, a nonlinear robust controller will be proposed. Unlike a commonly used SMC approach [27-30],

there is no discontinuous function involved. The later fact is preferred by the FPE based approach because of continuity requirements in the partial derivatives. Let us define the sliding manifold 1[ ]T p

ps ,...,s ∈ℜs = as 2

( 1)( )1, ,

0

, 1,...,i

i

rrk

i i i k i i ik

s e dt e e i pλ λ−

−−

=

= + + = (7)

American Institute of Aeronautics and Astronautics

4

where , 0, 1,..., 2, 1,...k i ik r i pλ > = − − = can be any positive number and the error signal is defined to be

, , 1,...,i i d ie y y i p= − = .

For a general nonlinear system (1) with bounded parametric and functional uncertainties (5) and (6), the proposed MIMO feedback control scheme

21 ( 1)

ˆ ˆ ˆ 10

ˆ ˆ( ) ( ) kdk

k

dL L L

dt

−+− +

−=

= − + ⋅ + ⋅ + ⋅

r rr r

rB f f

yu h x h x λ e λ e k s (8)

guarantees that the closed-loop system is globally asymptotically stable for tracking desired signal ,i dy . Note that

the element-wise multiplication 1 1[ ,..., ]Tp pa b a b⋅a b = is applied.

The time varying feedback gain 1[ ,..., ]T ppk k= ∈ℜk can be uniquely solved from

2( 1)

ˆ 10

ˆ( ) ( )kdk

k

dL

dt

−+

−=

+ − + ⋅ + ⋅ + ⋅ = − ⋅r r

rr f

yF D h x λ e λ e η s I D k s (9)

for any positive numbers λ and η . In case of 0is → , the magnitude of i ik s ( i i ik sς = ) instead of ik will be

calculated using (9) because the proposed controller (8) only uses i ik s . The sign of i ik s is determined by is since

0ik > . The stability proof of Eq. (8) and Eq. (9) can be found in [40, 41].

Note that the state tracking control =y x is a special of the proposed robust nonlinear control.

IV. Nonlinear Stochastic Control based upon DQMOM Now, the state equation of the closed loop system SDE (Eqs. 1) can be rewritten as

( )1 1 1 1

1 1

( ,..., , ) ( ,..., ) ( ,..., , , ) ( ,..., ) ( ), 1,...,w

i

Nmn

i i n ij n j n ij n jj j

x = f t b u g w t i n= =

+ + = x x x x x x x xλ η (10)

which is then converted into the form of the Itô SDE as

1 11

( ,..., , ) ( ,..., ) ( ), 1,...,wN

i i n ij n j sj

dx = f ,t dt g d t i Nβ=

+ = x x η x xλ, (11)

where ( ) ~ ( ) /j jw t d t dtβ and n

s ii

N n= is the number of states ix

in the first order system. Also in this model,

( ) (0, )jd t dtβ , [ ] s wN Nijg ×= ∈

G , and ( ) [ ] sNit x= ∈ x .

If the process described by the SDE is a Markovian diffusion process, the probability density function characterizing this process is governed by the FPE [10-12] as

( )2

1 1 1

1

2

s s s

TN N N

i ij

i i ji i j

ppfp

t x x x= = =

∂ ∂ ∂ = − +∂ ∂ ∂ ∂

GQG

(12)

where ( )( )p p t= x is the state PDF. The first term on the right hand side (RHS) of the FPE is the drift term

whereas the second one is the diffusion term and = 1/ 2 T

D GQG is defined.

However, the central issue associated with solving the FPE is the high computational cost, which explains the reason of why this type of nonlinear stochastic control and nonlinear filtering has not been widely used. Here a new quadrature based moment approach is proposed for solving the FPE. This method involves the representation of the state PDF in terms of a finite summation of Dirac delta functions as

( )( )1 1

( ) [ ]sNN

j jj

p t w t x xα αα

δ= =

= − ∏ x (13)

where N is the number of nodes, ( )w w tα α= denotes the corresponding weight for node α , 1,..., Nα = , and

( ), 1,...,j j sx x t j Nα α

= = represents the property vector of node α (called “abscissas”). The weights and

abscissas are determined based on the constraints from the evolution of moments. Using a small number of scalars

American Institute of Aeronautics and Astronautics

5

(in the Dirac delta function), this method is able to efficiently and accurately model stochastic processes described by the multivariable FPE through a set of ordinary differential equations (ODEs) as demonstrated below.

The dynamics of the abscissas jxα

and weights wα are governed by the following differential algebraic

equations

1

1

,...1 1 1 11 1,

1s ss s

jk k

Ns

N NN NN Nkk k

j k j j k j k kj jk k k j

k x a k x x b Sα αα ααα α

−

= = = == = ≠

− + =

∏ ∏ (14)

with the definitions of / , 1,...,dw dt a Nα α α = (15)

and / , 1,..., ; 1,...,j j sd dt b j N Nα ας α= = (16)

where the weighted abscissas j jw xα α ας is introduced. The 1 2, ,...,

sNk k k moment constraint is derived as

1 1 1

1 2,... ,... ,...N N Ns s s

k k k k k kS S S+ , in which

1 11

1

11,... 1 1 1 1

1 1

( ) ( ,... )s

Nsi i i

N s ss

N Nkk k kk

k k i i i i N i Ni

S k w t x x x x x f x xα α αα α α α αα

− +−− +

= =

= ⋅⋅ ⋅ ⋅ ⋅ ⋅

(17)

when i j≠ ,

11

2,... ,...

1 1 1 1

/ [ ( )]ss s

k

Ns Ns

NN N Nk

k k i j k i j ij x xi j k

S w k k x x xα α

α α α αα= = = =

= ⋅

∏ D x (18)

whereas when i j= ,

11

22,... ,...

1 1

( 1) / [ ( )]s

k

Ns Ns

NNk

k k i i k i ij x xk

S w k k x xα α

α α αα = =

= − ⋅

∏ D x (19)

The derivation of Eq. (17) through Eq. (19) is shown in Appendix A. Equation (14) can be rewritten in the matrix form as

=Aμ s (20)

where , 1,..., ; 1,...,T

j sa b N j Nα α α = = = μ . Once the abscissas and weights are calculated, any selected

moment of the state PDF can be found from

1 2 ...

1 1

sNs

NNkk k k

jj

M w xα

α αα = = ∏ (21)

where 1 2, ,...,sNk k k are nonnegative integers.

For any selected nonnegative integer 1 2, ,...,sNk k k , the corresponding stationary moments of the PDF is

governed by

1,... 0Ns

k kS = (22)

Proof: In steady state, the abscissas and weights of the moments will not change in time; therefore, based on Eq. 14, the corresponding RHS is zero.

V. Offline Nonlinear Constrained Optimization One of the control objectives is to find the control parameters λ and η for getting desired stationary moments

by minimizing the p-norm (e.g. 2-norm) of the error between the desired and actual stationary 1 2, ,...,sNk k k

moments as 1 21 2 ...... NN ss

k k kk k kd

pJ M M= − (23)

American Institute of Aeronautics and Astronautics

6

in which the PDF is characterized by the weights wα and abscissas jxα

. This problem can be formulated as an

NLP where the objective function is described as Eq. (23). Equation (22) for selected choice of 1 2, ,...,sNk k k and

1

1N

wαα =

= (property of the PDF) are the equality constraints, whereas 0η > (stability requirement), 0>λ

(stability requirement), and 0wα > (property of the PDF) are inequality constraints. The parameters to be optimized

are control parameters λ and η , and the weights wα and abscissas jxα

. Note that due to the flexibility of the

NLP approach used here, the performance index can be extended to a more general form including the quadratic type index.

The method is summarized and illustrated in Fig. 1. First, the desired stationary moments 1 2 ... Nsk k kdM are

specified. After that an NLP optimal control algorithm will be used to obtain desired optimal control parameters λ and η through the DQMOM approach. Finally, the obtained controller parameters will be implemented in the

nonlinear robust control.

Fig. 1 Outline of the stochastic robust control algorithm

VI. A Simple Numerical Example To demonstrate the effectiveness of the proposed algorithm, a numerical example modified from [33] is shown

below. 2

1 2 2 2 1; cos3x x x ax x u w= = − + + (24)

where the mean and variance values of the noise applied here are E[ ] 0w = and 2 2E[ ]w σ= respectively. The

uncertainty parameter a is bounded by 1 2a≤ ≤ and the nominal value is ˆ 1.5a = . The output is 1 2[ , ]T= = x xy x .

According to Eq. (8), the nonlinear robust controller is designed as 22 1 2, 2 2,ˆ cos3 ( )d du ax x x ks x xλ= + − − − (25)

where the uncertainty bound is: 20.5 cos3F x x= . The sliding surface is defined as 2 2, 1 1,( ) ( )d ds x x x xλ= − + −

where subscript d denotes the desired values. When 0s > , ks F sη= + ; when 0s < , ks F sη= − + ; whereas

0ks = when 0s = . The Itô stochastic differential equation of the system is

212

2 2 1

0, (0, )

ˆ cos3

xdxdt dw dt

dx dwax x u σ

= + − + (26)

where the control u (Eq. 26) is a function of states and control parameters λ and η . The corresponding FPE

governing the state PDF is derived as

( )22 2 2

1 1 1

( )i ij

i i ji i j

p Dpfp

t x x x= = =

∂∂∂ = − +∂ ∂ ∂ ∂

x

(27)

American Institute of Aeronautics and Astronautics

7

where 11 12 21 0D D D= = = , 22D Qdt= , 1 2f x=

, and 22 2 1ˆ cos3f ax x u= − +

. In this example, the state PDF is

modeled as

( )2

1 21 1

, ( ) [ ]N

j jj

p x x w t x xα αα

δ= =

= − ∏ (28)

and the 1 2,k k moments of the state PDF is derived as

21 2

2

1 21 1

ji

Nkk kk k

jj

M x x w xα αα = =

= ∏ (29)

The objectives of the controller are to (1) track the desired output 1 1y = and 2 0y = and (2) achieve a desired

tracking performance statistic distribution under different noise level as shown in Table 1. In this simulation, the

performance index in the NLP optimization is modeled as dJ = −M M , where 10 01 20 02[ , , , ]T= M M M MM . The

MATLAB® function fminsearch is used for the NLP optimization. The moments to be constrained in Eq. (22) are selected to be 1 [1,0, 2,1,0,3,2,1,0,4,3,2]k = and 1 [0,1,0,1,2,0,1,2,3,0,1, 2]k = for the case when 4N = . The actual

statistics is achieved through two thousand Monte Carlo runs, and after that the control parameter is found in the optimization. The results are compared with the case, where arbitrarily selected parameters 1λ = and 1η = is used

(without the offline stochastic optimization). To simplify the description, here the proposed stochastic robust controller is denoted as “method 1”, and the nonlinear robust controller method using arbitrarily selected control parameters is represented as “method 2”. In Table 1, the subscript d , a , and t denote the desired, actual value found from “method 1”, and actual value found through “method 2”. The residuals of the performance index (Eq. 23) in optimization for all the simulation cases and the achieved control parameters aη and aλ are listed as well.

The small value of 610− indicates the converged solution is obtained in the optimization. Since similar conclusions can be drawn from all the four cases shown in Table 1, only the figures found from

Case 1 and Case 4 are illustrated. The mean values of the position ( 1x ) and velocity states ( 2x ) are successfully controlled to the desired values in

both methods as shown in Figs. 2 and 3. Also as shown in Table 1, the desired stationary mean value needs to be unity for the position and zero for the velocity. The values achieved by method 1 are one for the position and zero for the velocity, whereas those values from method 2 are 0.99 and zero. Therefore, in terms of mean value control, there are no significant difference between method 2 and the proposed method 1. The same conclusion can be made for other cases. For example, in Figs. 6 and 7 (case 4 in Table 1), the mean values of the position and velocity found through both methods are one and zero respectively, which satisfy the requirement.

The significance and advantage of method 1 can be easily seen from Figs. 4, 5, 8 and 9. Method 1 has successfully controlled the moments 20

aM and 02aM to the desired value 1.01 and 0.105 in case 1. However in

method 2, the actual stationary moments 20aM and 02

aM are 1.4 and 0.45, which is quite different from the desired

statistic. In case 4, the desired moments are 20 1.2dM = and 02 0.5dM = . Through method 1, the accomplished

stationary moments are 1.19 and 0.5 whereas in method 2, the corresponding values are 1.86 and 0.88.

0 10 20 30 400

0.2

0.4

0.6

0.8

1

1.2

1.4

Time (s)

Mea

n V

alue

of t

he P

ositi

on

Method 1

Method 2

Fig. 2(a) Mean value of the position (case 1)

10 15 20 25 30 35

0.94

0.96

0.98

1

1.02

1.04

Time (s)

Mea

n V

alue

of t

he P

ositi

on

Method 1

Method 2

Fig. 2(b) Stationary mean value of the position (case 1)

American Institute of Aeronautics and Astronautics

8

0 10 20 30 40-0.2

0

0.2

0.4

0.6

0.8

Time (s)

Mea

n V

alue

of t

he V

eloc

ity

Method 1

Method 2

Fig. 3(a) Mean value of the velocity (case 1)

10 20 30 40-0.06

-0.04

-0.02

0

0.02

0.04

0.06

Time (s)

Mea

n V

alue

of t

he V

eloc

ity

Method 1

Method 2

Fig. 3(b) Stationary mean value of the velocity (case 1)

0 10 20 30 400

0.5

1

1.5

Time (s)

Mom

ent 2

0

Method 2

Method 1

Fig. 4 20

aM and 20tM of the output PDF (case 1)

0 10 20 30 400

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Time (s)

Mom

ent 0

2

Method 2

Method 1

Fig. 5 02

aM and 02tM of the output PDF (case 1)

Table 1. Stochastic Control Scenario and Results

Cases, Noise 10M 01M 20M 02M10dM 10

aM 10tM 01

dM 01aM 01

aM 20dM 20

aM 20tM 02

dM 02aM 02

tM

1, 1Q = 1.0 1.0 0.99 0 0 0 1.01 1.01 1.4 0.1 0.105 0.45

2, 1Q = 1.0 1.0 0.99 0 0 0 1.02 1.05 1.4 0.2 0.21 0.44

3, 2Q = 1.0 1.0 1.0 0 0 0 1.02 1.02 1.77 0.2 0.22 0.84

4, 2Q = 1.0 1.0 1.0 0 0 0 1.2 1.19 1.86 0.5 0.5 0.88

Cases, Noise 1, 1Q = 2, 1Q = 3, 2Q = 4, 2Q =

Residual of J 610− 610− 610− 610− 1aλ = =8.817704aη =3.721468aη =8.237894aη =2.696682aη

0 10 20 30 400

0.2

0.4

0.6

0.8

1

1.2

1.4

Time (s)

Mea

n V

alue

of t

he P

ositi

on

Method 1

Method 2

10 20 30 40

0.94

0.96

0.98

1

1.02

1.04

1.06

Time (s)

Mea

n V

alue

of t

he P

ositi

on

Method 1

Method 2

American Institute of Aeronautics and Astronautics

9

Fig. 6(a) Mean value of the position (case 4) Fig. 6(b) Stationary mean value of the position (case 4)

0 10 20 30 40-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

Time (s)

Mea

n V

alue

of t

he V

eloc

ity

Method 1

Method 2

Fig. 7(a) Mean value of the velocity (case 4)

10 20 30 40

-0.06

-0.04

-0.02

0

0.02

0.04

0.06

Time (s)

Mea

n V

alue

of t

he V

eloc

ity

Method 1

Method 2

Fig. 7(b) Stationary mean value of the velocity (case 4)

0 10 20 30 400

0.5

1

1.5

2

Time (s)

Mom

ent 2

0

Method 2

Method 1

Fig. 8 20

aM and 20tM of the output PDF (case 4)

0 10 20 30 400

0.2

0.4

0.6

0.8

1

Time (s)

Mom

ent 0

2

Method 2

Method 1

Fig. 9 02

aM and 02tM of the output PDF (case 4)

VII. Conclusion This paper describes an innovative robust stochastic control methodology for nonlinear affine systems subject to

parametric and functional uncertainties with random excitations. The method is of interest because the process is not necessary to be Gaussian and any specified set of (attainable) stationary moments can be precisely controlled. Two particular attributes of the methodology are: (1) the deterministic part of the controller is robust with respect to parametric and functional uncertainties without discontinuous functions involved; and (2) the state PDF is expressed in terms of a finite collection of Dirac delta functions, and the associated weights and locations are governed by moment constraints. In this approach, the associated Fokker-Planck equation need not be solved. The advantages of the method are (1) ability to control the distribution of any specified stationary moments of the states/output probability density function (PDF), and (2) robustness with respect to parametric and functional uncertainties. Our numerical simulations have successfully demonstrated the capability of the proposed nonlinear stochastic control method.

Appendix A – Moments Equation in DQMOM To show the derivation, the FPE of the state PDF (Eq. 12) is repeated here as

( )2

1 1 1

1

2

s s s

TN N N

i ij

i i ji i j

ppfp

t x x x= = =

∂ ∂ ∂ = − +∂ ∂ ∂ ∂

GQG

(A-1)

The DQMOM method was initially developed by Marchisio and Fox [39] to address the population balance problems in multiphase flows [39]. Based on this method, we developed a novel extension of this method with applications in nonlinear estimation and controls. In our approach, the state PDF is written as a summation of a multi-dimensional Dirac delta function

American Institute of Aeronautics and Astronautics

10

( )( )1 1

( ) [ ( )]sNN

j jj

p t w t x x tα αα

δ= =

= − ∏ x (A-2)

Substituting Eq. (A-2) into Eq. (A-1), the left hand side (LHS) of Eq. (A-1) becomes

1 1 11 1,

'

1 1 11 1,

[ ] [ ]s ss

s ss

N NNN Njj

j j k kjjj k k j

N NNN Nj

j k jjj k k j

xwpx x w x x

t t tx

xww

t t

αα αα αα

α α α

α αα α α α

α α

δδ δ

δ δ δ

= = == = ≠

= = == = ≠

∂∂∂∂ = − − − ∂ ∂ ∂∂

∂∂ = − ∂ ∂

∏ ∏

∏ ∏

(A-3)

where [ ]j j jx xα αδ δ − and ' /j j jxα α α

δ δ∂ ∂ .

If the weighted abscissas j jw xα α ας is introduced, after some manipulations, Eq. (A-3) can be written as

' '

1 1 1 11 1, 1,

s s ss sN N NN NN Nj

j j k j k jj jj k k j k k j

w wpx

t t t tαα α

α α α α ααα α

ςδ δ δ δ δ

= = = == = ≠ = ≠

∂∂ ∂∂ = + − ∂ ∂ ∂ ∂ ∏ ∏ ∏ (A-4)

Notice that wα , jας , and jαδ are functions of time, thus the partial derivatives of the functions can be written as

total derivatives.

' '

1 1 1 11 1, 1,

s s ss sN N NN NN Nj

j j k j k jj jj k k j k k j

ddw dwpx

t dt dt dtαα α

α α α α ααα α

ςδ δ δ δ δ

= = = == = ≠ = ≠

∂ = + − ∂ ∏ ∏ ∏ (A-5)

With the definitions / , 1,...,dw dt a Nα α α = (A-6)

and / , 1,..., ; 1,...,j j sd dt b j N Nα ας α= = (A-7)

Eq. (A-5) (LHS of Eq. (A-1)) can be further simplified as

' '

1 1 1 11 1, 1,

s s ss sN N NN NN N

j j k j k j jj jj k k j k k j

px a b

t α α α α α α ααα α

δ δ δ δ δ= = = == = ≠ = ≠

∂ = + −∂

∏ ∏ ∏ (A-8)

The RHS of Eq. (A-1) is now given by the expression 2

1 1 1

[1/ 2 ( ) ]( )

s s s TN N Niji

i i ji i j

ppfS

x x x= = =

∂∂= − +

∂ ∂ ∂ x

GQG

x (A-9)

The FPE can be written in terms of the multi-variable Dirac delta function as

' '

1 1 1 1 11 1, 1,

( )s s ss sN N NN NN N N

j j k j k j jj jj k k j k k j

a x a b Sα α α α α α α ααα α α

δ δ δ δ δ= = = = == = ≠ = ≠

+ − =

∏ ∏ ∏x x (A-10)

There are total (1 )sN N+ parameters which need to be found in order to construct the conditional PDF

( )( )p tx : aα and , 1,..., , 1,...,j sb j N Nα α= = . The DQMOM method applies an independent set of user defined

moment constraints to construct (1 )sN N+ algebraic ordinary differential equations (ODEs).

Given the following three Dirac delta function properties

( )kkx x x dx xα αδ

+∞

−∞− =

(A-11)

1' ( )kkx x x dx k xα αδ

+∞ −

−∞− = −

(A-12)

and 2"( ) ( 1)

kkx x x dx k k xα αδ+∞ −

−∞− = −

(A-13)

The 1 2, ,...,sNk k k moments of Eq. (A-10) can be written as followed

American Institute of Aeronautics and Astronautics

11

1 1

1

'1 1

1 1 11 1 1, 1

'1

1 1 1,

... ...

...

s s s ssN Ns s

s s

ssNs

s

N N N NNN Nk kk k

j l j k j lN Njj l k k j l

NNNkk

k j jNj k k j

x x a dx x x x a dx

x x b

α α α α ααα α

α α αα

δ δ δ

δ δ

+∞ +∞ +∞ +∞

−∞ −∞ −∞ −∞= = == = = ≠ =

= = = ≠

⋅ ⋅ ⋅ + ⋅⋅ ⋅

− ⋅⋅⋅

∏ ∏ ∏ ∏

∏

[ ]11

1 1

... ( )s s

Ns

s

N Nkk

l lNl l

dx x x S dx+∞ +∞ +∞ +∞

−∞ −∞ −∞ −∞= =

= ⋅⋅ ⋅ ∏ ∏ x

x

(A-14)

After rearranging and simplifying Eq. (A-14), the (1 )sN N+ unknown parameters can be found in

1

1

,...1 1 1 11 1,

1s ss s

jk k

Ns

N NN NN Nkk k

j k j j k j k kj jk k k j

k x a k x x b Sα αα ααα α

−

= = = == = ≠

− + =

∏ ∏ (A-15)

where 1 1 1

1 2,... ,... ,...N N Ns s s

k k k k k kS S S+ . The first term in the LHS of Eq. (A-15) can be simplified as

111

1 11 1

... ...s s

jNs

ss

N NN Nkkk

j N jNj j

x x a dx dx x aα α ααα α

δ+∞ +∞

−∞ −∞= == =

⋅ ⋅ ⋅ = ∏ ∏

(A-16)

whereas the second term in the LHS of Eq. (A-15) is

'1

1 1 1 11 1, 1

... ...s s ss s

km

s

N N NN NN Nkk

m j k j N j kj jm k k j k

x x a dx dx k x aα α α αααα α

δ δ+∞ +∞

−∞ −∞= = = == = ≠ =

= −

∏ ∏ ∏ (A-17)

In the same way, the third term of the LHS in Eq. (A-15) is

11'

111 1 1 11, 1,

...s ss s

jN ks

ss

N NN NN Nkk kk

j j k N j j k jNj jk k j k k j

b x x dx dx k x x bα α α αααα α

δ δ+∞ −

−∞= = = == ≠ = ≠

− ⋅⋅ ⋅ =

∏ ∏ (A-18)

The 1,...,sNk k moments of the RHS of Eq. (A-15) are then derived to be

1 1

1

1

1

,... 1 11 11

21

1 ,...11 1

... ( ) ... ...

[ ( )1...

2

N Ns s

N s ss ss

s sNs

s Ns s

nk kk k i

k k N NN Nii

TN Nk ijk

N k kNi ji j

pfS x x S dx dx x x dx dx

x

px x dx dx S

x x

+∞ +∞ ∞

−∞ −∞ −∞=

∞

−∞= =

∂⋅⋅⋅ = − ⋅⋅ ⋅ ∂

∂ + ⋅⋅⋅ =

∂ ∂

x

x

GQG1

2,... Ns

k kS+

(A-19)

where

1 111

1

11,... 1 1 1 11

1 1 1

1

... ( )

( ,... )

s sN i i is

N sss

Ns

s s

N N Nk k k kkk i

k k N i i i iNii i

k

N i N

pfS x x dx dx k w t x x x x

x

x f x x

α α α α αα

αα α

− +∞ −

− +−∞

= = =

∂= − ⋅⋅ ⋅ = ⋅⋅ ⋅ ⋅ ⋅ ⋅ ∂

(A-20)

When i j≠ ,

1

1

1

22

,... 111 1

,...1 1 1 1

[ ( )1...

2

/ [ ( )]

s sNs

N sss

ss sk

Ns

TN Nk ijk

k k NNi ji j

NN N Nk

i j k i j ij x xi j k

pS x x dx dx

x x

w k k x x xα α

α α α αα

∞

−∞= =

= = = =

∂ = ⋅⋅⋅

∂ ∂ = ⋅

∏

GQG

D x

(A-21)

whereas when i j= ,

1

1

1

22

,... 11 2

2

,...1 1

[ ( )]...

( 1) / [ ( )]

Ns

N sss

sk

Ns

kk iik k NN

i

NNk

i i k i ij x xk

f DS x x dx dx

x

w k k x x Dα α

α α αα

∞

−∞

= =

∂= ⋅⋅ ⋅

∂

= −

∏

x

x (A-22)

American Institute of Aeronautics and Astronautics

12

Thus, the (1 )sN N+ ODEs can be constructed using a set of independent moment constraints 1,...,sNk k as shown in

Eq. (20).

References 1Davis, M. H. A., and Vinter, R. B., Stochastic Modeling and Control, Chapman and Hall, London, 1985. 2Iwasaki, T., and Skelton, R. E., “On the observer-based structure of covariance controllers,” System Control Letter, 22(1),

1994, 17-25. 3Sinha, A., and Miller, D. W., “Optimal sliding-mode control of a flexible spacecraft under stochastic disturbances,” Journal

of Guidance, Control, and Dynamics, 18(3), 1995, 486-492. 4Grigoriadis, K. M. and Skelton, R. E., “Minimum-energy covariance controllers,” Automatica, 33, 1997, 569-578. 5Bratus, A., Dimentberg, M., and Iourtchenko, D., “Optimal bounded response control for a second-order system under a

white-noise excitation,” Journal of Vibration and Control, 6(5), 2000, 741–755. 6Hotz, A., and Skelton, R. E., “A covariance control theory,” International Journal of Control, 46(1), 1987, 13-32. 7Young, G. E., and Chang, R. J., “Optimal control of stochastic parametrically and externally excited nonlinear control

systems,” ASME Journal of Dynamical System, 110, 1988, 114-119. 8Wojtkiewicz, S. F., and Bergman, L. A., “A moment specification algorithm for control of nonlinear system driven by

Gaussian white noise,” Nonlinear Dynamics, 24(1), 2001, 17-30. 9Elbeyli, O., and Sun, J. Q., “Covariance control of nonlinear dynamic systems via exact stationary probability density

function,” Journal of Vibration and Acoustic, 126(1), 2004, 71-76. 10Jazwinski, A., Stochastic Process and Filtering Theory, New York, NY, Academic Press, 2007 (re-publication of the

version of 1970) pp. 72-74. 11Fokker, A., Annalen der Physik, 43(810), 1940. 12Planck, M., Sitzungsber. Preuss. Akad. Wissens., 1917, 324. 13Zhou, Y. and Chirikjian, G. S., “Probabilistic models of dead-reckoning error in nonholonomic mobile robots,” Proceedings

of IEEE Int.Conference on Robotics and Automation, 2, 2003, 1594-1599. 14Spencer Jr., B. F., and Bergman, L. A., “On the numerical solution of the Fokker equations for nonlinear stochastic

systems,” Nonlinear Dynamics, 4, 1993, 357–372. 15Daum, F., “Nonlinear filters: beyond the Kalman filter,” IEEE Transaction on Aerospace and Electronic Systems Magazine,

20(8), Part 2, 2005, 57-69. 16Naess, A., and Johnson, J. M., “Response statistics of nonlinear dynamics systems by path integration,” IUTAM Symposium

on Nonlinear Stochastic Mechanics, 1992, 401–414. 17Sun, J. Q., and Hsu, C. S., “The generalized cell mapping method in nonlinear random vibration based upon short-time

Gaussian approximation,” ASME Journal of Applied Mechanic, 57, 1990, 1018–1025. 18Challa, S., and Bar-Shalom, Y., “Nonlinear filter design using Fokker-Planck-Kolmogorov probability density evolutions,”

IEEE Transaction on Aerospace Electronics System, 36(1), 2000, 309-315. 19Musick, S., Greenswald, J., Kreucher, C., and Kastella, K., “Comparison of particle method and finite difference nonlinear

filters for low SNR target tracking,” The 2001 Defense Applications of Signal Processing Workshop, 2001. 20Yoon, J., and Xu, Y., “Relative position estimation using Fokker-Planck and Bayes’ equations,” 2007 AIAA Guidance,

Control, and Dynamics Conference, August 20-23, Hilton Head, South Carolina, 2007. 21Sun, J. Q., and Hsu, C. S., “Cumulant-neglect closure method for asymmetric nonlinear systems driven by Gaussian white

noise,” Journal of Sound and Vibration, 135(2), 1989, 338-345. 22Sobczy, K., and Trebicki, J., “Maximum entropy principle in stochastic dynamics,” Probabilistic Engineering Mechanics,

3(5), 1990, 102-110. 23Chang, K. Y., Wang, W. J., and Chang, W. J., “Constrained control for stochastic multivariable systems with hysteresis

nonlinearity,” International Journal of Systems Science, 28(7), 1997, 731-736. 24Kim, J., and Rock, S., “Stochastic feedback controller design considering the dual effect,” AIAA Guidance, Navigation, and

Control Conference and Exhibit, Keystone, Colorado, Aug. 21-24 2006. 25Forbes, M. G., Forbes, J. F., and Guay, M., “Regulatory control design for stochastic processes: shaping the probability

density function,” American Control Conference, Denver, Colorado, June 4-6, 2003, 3998-4003. 26Attar, P., and Vedula P., “Direct quadrature method of moments solution of the Fokker–Planck equation,” Journal of Sound

and Vibration, 317, 2008, 265-272. 27Utkin, V. I., “Variable structure systems with sliding modes,” IEEE Transaction on Automatic Control, 22(2), 1977, 212-

222 28Bartolini, G., Punta, E., and Zolezzi, T., “Simplex methods for nonlinear uncertain sliding mode control,” IEEE Transaction

on Automatic Control, 49(6), 2004, 922-933. 29Hung, J. Y., Gao, W., and Hung, J. C., “Variable structure control: a survey,” IEEE Transaction on Industrial Electronics,

40(1), 1993, 2-22. 30Bartolini, G., Ferrara, A., Usai, E., and Utkin, V. I., “On multi-input chattering-free second order sliding mode control,”

IEEE Transaction on Automatic Control, 45(9), 2000, 1711-1717 31Perruquetti, W., and Barbot, J. P., Sliding Mode Control in Engineering, New York: Marcel Dekker, Inc., 2002, pp. 12-14.

American Institute of Aeronautics and Astronautics

13

32Lee, J. G., Park, C. G., and Park, H. W., “Sliding-mode controller design for spacecraft attitude tracking maneuvers,” IEEE Transaction on Aerospace and Electronic System, 29(4), 1993, 1328-1333.

33Slotine, J. E., and Li, W., Applied Nonlinear Control, New Jersey: Prentice Hall, 1990, pp. 267-307. 34Buffington, J. M., and Shtessel, Y. B., “Saturation protection for feedback linearizable systems using sliding mode theory,”

American Control Conference, Philadelphia, PA, 1998, 1028-1032. 35Phuah, J., Lu, J., and Yahagi, T., “Chattering free sliding mode control in magnetic levitation system,” IEEJ Transactions

on Electronics, Information and Systems, 125(4), 2005, 600-606. 36Yao, B., and Tomizuka, M., “Smooth robust adaptive sliding mode control of manipulators with guaranteed transient

performance,” American Control Conference, Baltimore, MD, 1994, 1176-1180. 37Kokotovic, P., Khalil, H. K., and O’Reilly, J., “Singular perturbation methods in control, analysis and design,” Society for

Industrial and Applied Mathematics, Philadelphia, 1999, 1-45. 38LaSalle, J. P., “Some extensions of Lyapunov’s second method,” IRE Transaction on Circuit Theory, CT-7, 1960, 520-527. 39Marchisio, D. L., and Fox, R. O., “Solution of population balance equations using the direct quadrature method of

moments,” Journal Aerosol Science, 36, 2005, 43-73. 40Xu, Y. “Chattering free robust control for nonlinear systems,” IEEE Transactions on Control Systems Technology, 16(6),

2008, pp. 1352-1359. 41Hopkins, R., and Xu, Y., “Position tracking control for a simulated miniature helicopter,” 2008 AIAA Guidance, Navigation,

and Control Conference and Exhibit, Honolulu, Hawaii, August 18-21, 2008. 42Yunjun Xu and Prakash Vedula, “A Moment-based Approach for Nonlinear Stochastic Tracking Control,” Submitted for

publication in Nonlinear Dynamics.