6

H∞-Like Control for Nonlinear Stochastic

Systems

6.1 Introduction

We consider the system (1.11). The problem of state-feedback is formulated inthe following obvious way. Given an output which is to be regulated (entitled,in this chapter, controlled output):

zt= zt(xt, ut) =

[h(xt, t)ut

](6.1)

where h : Rn × [0,∞) → Rr is a Borel measurable function with h(0, t) =0, ∀t ≥ 0. The objective is to synthesize a controller ut = u(xt, t) such that,for a given γ > 0, the following H∞ criterion is satisfied.

E{∫ T

0

‖zt‖2 dt} ≤ γ2E{‖x0‖2 +∫ T

0

(‖vt‖2)dt} (6.2)

for all T ≥ 0 and for all disturbances vt in Au (provided Au is nonempty).Whenever the system (1.11) satisfies the above inequality, it is said to havethe L2-gain property, and we also write L2-gain≤ γ.

The above SF control problem may be also treated within the contextof stochastic game theory, analogously to the utilization of game theory inthe deterministic H∞ control and estimation (see, e.g. [5]). In fact, thereis a significant volume of research work regarding this approach, which isbased on what is called risk-sensitive control, see, e.g., [29] and the referencestherein. The motivation for investigating risk-sensitive problems stems partlyfrom the duality relation that exists between risk-sensitive stochastic controlproblems and stochastic differential games, which allows to obtain solutionsto the stochastic differential games problems, in particular, robustH∞ controlproblems (see, e.g. [29], [76]). The work presented in [17] takes a more directapproach to the minimax stochastic games as it utilizes the information stateconcept and establishes a Hamilton -Jacobi-Bellman equation which is drivenby the system’s measurement process while this process is taken to be (for

E. Gershon & U. Shaked: Advanced Topics in Control & Estimation, LNCIS 439, pp. 115–128.DOI: 10.1007/978-1-4471-5070-1_6 c© Springer-Verlag London 2013

116 6 H∞-Like Control for Nonlinear Stochastic Systems

technical reasons) a cylindrical stochastic process. This work differ from ours(aside from dealing with output feedback), first in its approach, as ours basedon the notion of dissipativity and establishes Hamiltonn, Jacobi inequalitiesrather than equalities, as done in [17], and second in that we deal with infinitetime horizon and therefore consider stochastic stability.

We adopt the approach that is based on the notion of stochastic dissipa-tion. In Section 6.3 we state and prove some kind of a bounded real lemma fornon linear stochastic systems. In particular, we introduce a certain Hamilton-Jacobi inequality (HJI for short) and we establish necessary and sufficientconditions for the HJI to guarantee a dissipation of the underlying system,which in turn implies the L2 − gain property of the system.

In Section 6.4 we discuss the stability of the closed-loop system whichis subject to the H∞ controller. In particular, we establish sufficient condi-tions under which stability in some mean square sense and in probability isguaranteed.

In Section 6.5 we discuss the stationary case with nonlinear bounded un-certainties. Thus we are led to deal with the algebraic HJI. We introduce acertain LMI the solution to which, if exists, solves also the algebraic HJI; this,in turn, implies that the closed-loop system is stable in the mean-square senseand the underlying nonlinear system has the L2 − gain property.

Applying the definitions of Chapter 1 we obtain the following.

Theorem 6.1.1 The function Va of the above definition is a storage functionfor the system of (1.11) (or equivalently, the system (1.11) is dissipative withrespect to the supply rate S) iff E{Va(x, t)} is finite for all t ∈ [0,∞) and forany Ft measurable Rn valued random variable x with E{||x||2} <∞.

Proof: Assume E{Va} is finite for some u and for all v ∈ Au in the sense ofthe theorem. It is obvious that Va(x, t) ≥ 0 P − a.e. as 0 is a member ofthe set over which the supremum in (1.18) is taken (take T = t). Let now0 ≤ t0 ≤ t1 be arbitrarily fixed. Then

E{Va(xt0 , t0)} = E{ supT ≥ t0, v ∈ Au

[−E{[∫ T

t0

S(vs, zs)ds]/xt0}]}

≥ E [−E{[∫ T

t0

S(vs, zs)ds]/xt0}] ∀T ≥ t0, v ∈ Au

from which we have

E{Va(xt0 , t0)}≥E [−E{[∫ t1t0S(vs, zs)ds]/Fxt0

}]+E [−E{[∫ T

t1S(vs, zs)ds]/Fxt0

}],

∀T ≥ t1, v ∈ Au.

Using now elementary properties of conditional expectation this last inequalityimplies

6.2 Stochastic H∞ SF Control 117

E{Va(xt0 , t0)}≥−E{∫ t1t0S(vs, zs)ds}+ E [−E{[

∫ T

t1S(vs, zs)ds]/xt1}],

∀T ≥ t1, v ∈ Au

which leads to

E{Va(xt0 , t0)}≥−E{∫ t1t0S(vs, zs)ds}

+E{ supT ≥ t1, v ∈ Au

[−E{[∫ T

t1S(vs, zs)ds]/xt1}]},

= −E{∫ t1t0S(vs, zs)ds}+ E{Va(xt1 , t1)}, ∀v ∈ Au

(6.3)

so that the system (1.11) is dissipative, that is Va is a storage function of thissystem.

Suppose now that the system (1.11) is dissipative with respect to thesupply rate S, that is, it possesses some storage function V that satisfies

0 ≤ E{V (xT , T )} ≤ E{V (xt0 , t0)}+ E{∫ T

t0

S(vs, zs)ds} (6.4)

for any 0 ≤ t0 ≤ T . The last inequality implies −E{∫ T

t0S(vs, zs)ds} ≤

E{V (xt0 , t0)} <∞ for all admissible v and for all T ≥ t0. Since

− E{∫ T

t0

S(vs, zs)ds} = E [−E{[∫ T

t0

S(vs, zs)ds]/xt0}] (6.5)

it follows that

E{Va(xt0 , t0)} = E{ supT ≥ t0, v ∈ Au

[−E{[∫ T

t0

S(vs, zs)ds]/Fxt0}]} <∞.

(6.6)This completes the proof. ��

6.2 Stochastic H∞ SF Control

In this section we consider the stochastic system (1.11). The H∞ SF controlobjective is to synthesize a state-feedback ut = u(xt, t) such that for a pre-scribed γ > 0 the controlled output (6.1) satisfies (6.2) for all T ≥ 0 andfor all v ∈ Au where h : Rn × [0, T ) → Rr is a continuous function in botharguments. In this case, we say that the closed-loop system has an L2-gain≤ γ.

As has been mentioned before, we take an approach that is based on thedissipation of stochastic systems, and establish a connection between the dis-sipation of a stochastic system and a certain HJI. In fact, it will be shown thata necessary and sufficient condition for a stochastic system to be dissipativewith respect to a prescribed supply rate is that there exists a storage function

118 6 H∞-Like Control for Nonlinear Stochastic Systems

which satisfies this HJI. Also, it will be shown that dissipation implies thedesired objective, i.e. L2-gain≤ γ. We then consider the case of infinite timehorizon and establish conditions under which the H∞ controller is a stabiliz-ing one. We also consider the time-invariant case, which yields an algebraicHJI which, in turn, yields a stabilizing time-invariant controller. Stability willbe established in both probability and mean square sense.

We first state the following trivial fact.

Lemma 6.2.1 Suppose there is a controller ut = u(xt, t) such that the system

(1.11) is dissipative with respect to the supply rate S(v, z) = γ2 ‖v‖2−‖z‖2 and

assume that the associated storage function satisfies E{V (x0, t)} ≤ γ2E ‖x0‖2for all t ≥ 0. Then, the closed-loop system (1.11) has an L2-gain≤ γ.

Utilizing now the of stochastic dissipation concept, we prove the following:

Theorem 6.2.1 Consider the system described by (1.11) with the controlled

output of (6.1), and the supply rate S(v, z) = γ2 ‖v‖2 − ‖z‖2. Then the fol-lowing hold:

A. Let γ > 0 be fixed, and let V (x, t) ∈ C2,1 be a positive function satisfying

U = γ2I − 1

2U(x, t) ≥ αI

for some α > 0, and for all x, t, where U(x, t) is defined by

U(x, t) = [g2(x, t)]TVxx(x, t)g2(x, t). (6.7)

Let alsoD(x, t) = I + 1

2 gT (x, t)Vxx(x, t)g(x, t).

Assume the following HJI is satisfied

Vt(x, t) + Vx(x, t)f(x, t) − 14Vx(x, t)g(x, t)D

−1(x, t)gT (x, t)V Tx (x, t)

+ 14Vx(x, t)g1(x, t)U

−1(x, t)gT1 (x, t)VTx (x, t)

+ 12Tr{GT (x, t)Vxx(x, t)G(x, t)} + hT (x, t)h(x, t) ≤ 0 ∀x ∈ Rn, ∀t ≥ 0

(6.8)

Then, for ut = − 12D

−1(x, t)gT (x, t)V Tx (x, t) the system (1.11) is dissipative

with respect to the supply rate S(v, z) (provided Au is nonempty).

B. Assume that for some control ut = l(xt, t) the system (1.11) is dissipativewith respect to the supply rate S(v, z) for some storage function V ∈ C2,1

which is assumed to satisfy U ≥ αI for all x, t. Assume also that vt(x) =12 U

−1(x, t)gT1 (x, t)VTx (x, t) ∈ Au. Then V (x, t) satisfies the HJI for all x ∈ Rn

and for all t ≥ 0.

6.2 Stochastic H∞ SF Control 119

Proof:A. Assume there is a positive function V (x, t) that satisfies the HJI, and let

ut(x) = −1

2D−1(x, t)gT (x, t)V T

x (x, t).

We first show that the following inequality implies dissipation, and then com-plete the proof of part A by showing that this inequality is implied by theHJI with the storage function V . Assume

LV (x, t) + ‖z(x, t)‖2 − γ2 ‖v‖2 ≤ 0 (6.9)

for all x ∈ Rn, for all v ∈ Rm1 , and for all t ≥ 0, where LV is the infinitesimalgenerator of the system (1.11) with respect to V , that is

LV (x, t) = Vt(x, t) + Vx(x, t)[f(x, t)) + g(x, t)u+ g1(x, t)v]

+ 12Tr{GT (x, t)Vxx(x, t)G(x, t)}+ 1

2vT {[g2(x, t)]TVxx(x, t)g2(x, t)}v

+ 12u

T {[g(x, t)]TVxx(x, t)g(x, t)}u.(6.10)

Using now the Ito formula in the derivation of the stochastic differential ofV (xt, t), where xt is the solution of the differential equation of (1.11), onefinds that

dV (xt, t) = LV (xt, t)dt+ Vx(xt, t)G(xt, t)dW1t

+Vx(xt, t)g2(xt, t)vtdW2t Vx(xt, t)g(xt, t)utdWt, P -a.e.,

which may be written in the following integral form:

V (xt, t)=V (x0, 0) +∫ t

0 LV (xs, s)ds+∫ t

0 Vx(xs, s)G(xs, s)dW1s

+∫ t

0 Vx(xs, s)g2(xs, s)vsdW2s +

∫ t

0 Vx(xt, t)g(xt, t)utdWt P−a.e.(6.11)

where v ∈ Au (Au is assumed to be nonempty) and xt is the solution to(1.11). Using the inequality of (6.9) in (6.11) it is found that:

V (xt, t) ≤ V (x0, 0)+∫ t

0 (γ2 ‖vs‖2 − ‖zs‖2)ds+

∫ t

0 Vx(xs, s)G(xs, s)dW1s

+∫ t

0Vx(xs, s)g2(xs, s)vsdW

2s +

∫ t

0Vx(xt, t)g(xt, t)utdWt P−a.e.

Taking expectation of both sides of the last inequality and recall that

E{∫ t

0Vx(xs, s)G(xs, s)dW

1s +

∫ t

0Vx(xs, s)g2(xs, s)vsdW

2s

+∫ t

0Vx(xt, t)g(xt, t)utdWt} = 0,

we then arrive at

E{V (xt, t)} ≤ E{V (x0, 0)}+ E{∫ t

0

(γ2 ‖vσ‖2 − ‖zσ‖2)dσ} (6.12)

120 6 H∞-Like Control for Nonlinear Stochastic Systems

for all v ∈ Au, and for all t ≥ 0.Thus, the system is dissipative for ut(x) = − 1

2D−1(x, t)gT (x, t)V T

x (x, t)(which yields the inequality (6.9)). Completing the left hand side of the in-equality (6.9) to squares, yields

LVt(x, t)+Vx[f(x, t)) + g1(x, t)v+g(x, t)u]+‖z‖2−γ2 ‖v‖2

+ 12Tr{GT (x, t)Vxx(x, t)G(x, t)} + 1

2vT {[g2(x, t)]TVxx(x, t)g2(x, t)}vT

+ 12u

T {[g(x, t)]T Vxx(x, t)g(x, t)}uT

=Vt(x, t)+Vx(x, t)f(x, t)−u∗TD(x, t)u∗+v∗T U(x, t)v∗ + h(x, t)Th(x, t)

+12Tr{GT (x, t)Vxx(x, t)G(x, t)}+ (u− u∗)TD(x, t)(u − u∗)

+(v−v∗)T U(x, t)(v−v∗)

where:

u∗ = −1

2D−1(x, t)gT (x, t)V T

x (x, t), and v∗ = −1

2U(x, t)gT1 (x, t)V

Tx (x, t)

Using u = −u∗ together with the hypothesis on the HJI, yields the inequality(6.9), which in turn, implies the dissipation of the system for this u.

B. Assume u = l(x, t) renders the system of (1.11) dissipative with respect

to the supply rate S(v, z) = γ2 ‖v‖2 − ‖z‖2 for some continuous function l,and for a storage function V ∈ C2,1, i.e.

E{V (xt, t)} ≤ E{V (xs, s)}+ E{∫ t

s

(γ2 ‖vσ‖2 − ‖zσ‖2)dσ} (6.13)

for all xs which is Fs- measurable, square-integrable, and for all t ≥ s andv ∈ Au. Fix now x ∈ Rn, take xs = x ∈ Rn, and choose v ∈ Au to be anarbitrary constant function in Rm1 . In view of the last inequality the followingholds for all t > s:

1

t− sE{V (xt, t)− V (xs, s)} ≤ 1

t− sE{∫ t

s

(γ2 ‖vσ‖2 − ‖zσ‖2)dσ}. (6.14)

Application of the Ito formula to V (xt, t) yields

1

t− sE{V (xs, s) +∫ t

s

(LV )(xσ , σ)dσ+

∫ t

s

Vx(xσ , σ)G(xσ , σ)dW1σ

+∫ t

s Vx(xσ, σ)g2(xσ , σ)vσdW2σ +

∫ t

s

Vx(xt, t)g(xt, t)utdWt − V (xs, s)}

≤ 1

t− sE{∫ t

s

(γ2 ‖vσ‖2 − ‖zσ‖2)dσ.(6.15)

By the assumed nature of v (being a constant), by the continuity of u (in xand t) and the P−a.s. continuity of xt (recall: xt is a solution to (1.11)), andby the Fubini theorem ([37]), together with the following fact:

6.3 The Infinite-Time Horizon Case: A Stabilizing Controller 121

E{∫ t

sVx(xσ , σ)G(xσ , σ)dW

1σ +

∫ t

sVx(xσ , σ)g2(xσ , σ)vσdW

2σ

+∫ t

s Vx(xt, t)g(xt, t)utdWt} = 0,(6.16)

the last inequality yields

1

t− s

∫ t

s

E{(LV )(xσ , σ)}dσ ≤ 1

t− s

∫ t

s

E{(γ2 ‖vσ‖2 − ‖zσ‖2)dσ ∀t>s.(6.17)

This inequality implies, as t ↓ s:

E{LV (xs, s)} ≤ E{γ2||vs||2 − ||zs||2} = E{γ2||vs||2 − ||zs(xs, us(xs))||2}.

As xs and vs are taken to be arbitrary deterministic vectors in Rn and Rm1 ,respectively, the latter inequality yields (LV )(x, s) ≤ γ2 ‖v‖2 − ‖zs‖2 , for allx ∈ Rn and for all v ∈ Rm1 . Completion to squares then yields

Vt(x, t)+Vx(x, t)f(x, t)− 14Vx(x, t)g(x, t)g

T (x, t)V Tx (x, t) + v∗T U(x, t)v∗

12Tr{GT (x, t)Vxx(x, t)G(x, t)} +hT (x, t)h(x, t)+ (u− u∗)TD(x, t)(u − u∗)+(v−v∗)T U(x, t)(v−v∗) ≤ 0.

(6.18)Introducing vt(x) = v

∗ into the last inequality yields the HJI. ��

6.3 The Infinite-Time Horizon Case: A StabilizingController

In this section we consider the infinite-time horizon case. The problem ofsynthesizing an H∞ controller that renders a stable closed-loop system is in-vestigated. Conditions under which the closed-loop system is asymptoticallystable, in both the probability sense and the mean-square sense, will be dis-cussed.

We first recall few facts from the theory of stochastic stability (see e.g.[67]). We remark that in what follows we consider only global stability. Obvi-ously, local stability results may also be achieved, in a similar way.

Definition 6.3.1 Consider the stochastic system

dxt = f(xt, t)dt+G(xt, t)dWt (6.19)

with f(0, t) = G(0, t) = 0 for all t ≥ 0, and assume that f,G satisfy theusual Lipschitz conditions that guarantee a unique strong solution relative tothe filtered probability space (Ω,F, {Ft}t≥0, P ) where {Ft} is generated by theWiener Process. The solution xt is said to be stable in probability if for anyε > 0 lim

x→0P{sup

t≥0‖xt‖ > ε} = 0.

122 6 H∞-Like Control for Nonlinear Stochastic Systems

Definition 6.3.2 The solution xt of (6.19) is said to be globally asymptoti-cally stable in probability if it is stable in probability, and if P{ lim

t→∞xt = 0} = 1

for any initial state x ∈ Rn.

A sufficient condition for a global stability in probability is given by the the-orem below.

Theorem 6.3.1 ([67])Assume there exists a positive function V (x, t) ∈ C2,1,with V (0, t) = 0, so that (LV )(x, t) < 0 for all x ∈ Rn and for all t ≥ 0.Assume also that inf

t>0V (x, t) → ∞ as ‖x‖ → ∞. Then, the system of (6.19)

is globally asymptotically stable in probability.

Definition 6.3.3 [stability in the mean square sense] The system of (6.19) issaid to be globally exponentially stable in the mean square sense if E{||xt||2} ≤kE{‖xs‖2} exp{−α(t− s)} for all 0 ≤ s ≤ t, and for some positive numbersk and α.

Theorem 6.3.2 ([67])Assume there exists a positive function V (x, t) ∈ C2,1,with V (0, t) = 0. Then the system of (6.19) is globally exponentially stable ifthere are positive numbers k1, k2, k3 such that the following hold.

k1||x||2 ≤ V (x, t) ≤ k2||x||2, (LV )(x, t) ≤ −k3||x||2 for all t ≥ 0. (6.20)

We now state and prove two lemmas regarding the stability of the closed-loopsystem.

Lemma 6.3.1 [Stability in probability] Assume there exists a positive func-tion V (x, t) ∈ C2,1 such that inf

t>0V (x, t) → ∞ as ‖x‖ → ∞, and as-

sume V (x, t) satisfies the HJI (6.8), and hT (x, t)hT (x, t) > 0 for all xand for all t ≥ 0. Then, the homogeneous closed-loop system (v = 0) withu = − 1

2gT (x, t)V T

x (x, t) is globally asymptotically stable in probability.

Proof: Note that for v = 0 the HJI (6.8) reduces to

Vt(x, t) + Vx(x, t)f(x, t) − 14Vx(x, t)g(x, t)D

−1(x, t)gT (x, t)V Tx (x, t)

+ 12Tr{GT

1 (x, t)Vxx(x, t)G1(x, t)}+ hT (x, t)h(x, t) ≤ 0 ∀x ∈ Rn, ∀t≥0.(6.21)

Thus, we have (LV )(x, t) ≤ −hT (x, t)h(x, t) < 0. ��

Remark 6.1. Assume there exists a solution V to (6.21). A sufficient conditionfor V to be positive is the zero state observability of the system with respectto z, which is defined as follows. The system (6.19) is zero state observable ata time s if for v = 0, z(t) = 0 for all s ≤ t implies x(t) = 0 for all s ≤ t. Thatthis implies the desired positiveness of V is readily seen from the dissipationinequality (1.17). In fact, the dissipation inequality implies

E{∫ t

s

||z(σ)||2dσ} ≤ V (x, s) for all t ≥ s and for all x ∈ Rn. (6.22)

6.3 The Infinite-Time Horizon Case: A Stabilizing Controller 123

Assume now that for some s > 0 and x, V (x, s) = 0. Then E{∫ t

s||z(σ)||2dσ} =

0 for all t ≥ s, which implies z(t) = 0 P − a.e. for all t ≥ s. The zero stateobservability implies now that x(t) = 0 for all t ≥ s, and in particular x = 0(x = x(s)). Therefore, we conclude that V (x, s) = 0 implies x = 0, so that Vis positive.

The next lemma establishes the stability of the closed-loop system in the meansquare sense.

Lemma 6.3.2 Assume there exists a positive function V (x, t) ∈ C2,1, withV (0, t) = 0 for all t ≥ 0, which satisfies the HJI of (6.8) for some γ > 0.Inaddition, let V satisfy

k1||x||2 ≤ V (x, t) ≤ k2||x||2 (6.23)

for all t ≥ 0 and for some positive numbers k1, k2. Furthermore, assumethat for some k3, h

T (x, t)h(x, t) ≥ k3||x||2 for all t ≥ 0. Then the closed-loop system of (6.19) with v = 0 and u = − 1

2D−1(x, t)gT (x, t)V T

x (x, t) isexponentially stable in the mean square sense, and has the property of L2 −gain ≤ γ, that is

E{∫ ∞

0

‖zt‖2 dt} ≤ γ2E{‖x0‖2 +∫ ∞

0

(‖vt‖2)dt} (6.24)

for all nonanticipative stochastic processes v that satisfy E{∫∞0

(‖vt‖2)dt} <∞, and whenever x0 satisfies E{V (x0, 0)} ≤ γ2E{||x0||2}.

Proof: By Theorem 6.2.1 the existence of the HJI (6.8) implies the dissipationinequality

E{V (xt, t)} ≤ E{V (x0, 0)}+ E{∫ t

0

(γ2 ‖vσ‖2 − ‖zσ‖2)dσ} (6.25)

for all t ≥ 0, which impliesE{∫∞0

‖zt‖2 dt} ≤ γ2E{‖x0‖2 +∫∞0

(‖vt‖2)dt},provided E{

∫∞0

(‖vt‖2)dt} < ∞ and E{V (x0, 0)} ≤ γ2E{||x0||2}. Now, forv = 0, the HJI reduces to the one used in the proof of Lemma 6.3.1which implies that (LV )(x, t) ≤ −hT (x, t)h(x, t) ≤ −k3||x||2 , where LVis the infinitesimal generator of the homogeneous closed-loop system withu=− 1

2D−1(x, t)gT (x, t)V T

x (x, t), that is

(LV)(x, t)=Vt(x, t) + Vx(x, t)f(x, t)

−12Vx(x, t)g(x, t)D

−1(x, t)gT(x, t)V Tx (x, t)+ 1

2Tr{GT1(x, t)Vxx(x, t)G1(x, t)}.

(6.26)

124 6 H∞-Like Control for Nonlinear Stochastic Systems

By the last inequality and by (6.23) it follows that the system of (6.19) isexponentially stable in the mean-square sense. ��We consider now the time invariant case:

dxt=f(xt)dt+g(xt)utdt+ g(xt)utdWt+g1(xt)vtdt+g2(xt)vtdW2t +G(xt)dW

1t .(6.27)

It is easy now to prove a time invariant analog of Theorem 6.2.1 as a simplecorollary. However, in order to facilitate the development of what follows inthe next section, we choose to adopt a slightly different point of view. In fact,the following lemma which provides a sufficient condition for the existence ofthe L2−gain property, may be proved.

Lemma 6.3.3 Consider the system (6.27) with the controlled output

z(x, u(x)) =

[h(x)u(x)

], where u(x) = l(x), l : Rn → Rm is a continuous func-

tion. Suppose there is a positive function V (x) ∈ C2 which satisfies U(x) ≥ αIfor some α > 0, for some γ > 0, and for all x. Assume the following HJI issatisfied

Vx(x)[f(x) + g(x)l(x)] +14Vx(x)g1(x)U

−1gT1 (x)Vx(x)

+ 12Tr{GT (x)Vxx(x)G(x)} + hT (x)h(x) ≤ 0 ∀x ∈ Rn.

(6.28)

Assume also 12 U

−1(x)gT1 (x)VTx (x) ∈ Al. Then, the system of (6.27) is L2 −

gain ≤ γ, that is

E{∫ t

0

‖zs‖2 ds} ≤ γ2E{‖x0‖2 +∫ t

0

(‖vs‖2)ds} (6.29)

for all t ∈ [0,∞), for all v ∈ Al, and provided E{‖x0‖2} ≥ E{V (x0)}. More-over, assume also that V (x) > 0 for all x �= 0 and V (0) = 0. Then, thesystem (6.27) is asymptotically stable in probability. In addition, if V satis-fies: β1||x||2 ≤ V (x) ≤ β2||x||2 for some positive β1 and β2, the system (6.27)is also exponentially stable in the mean square sense.

An example is now in order. We introduce the following one which, althoughis one dimensional, it is fairly important as it includes, as a special case, thefamily of bilinear systems.

6.3.1 Example

Consider the following scalar system:

dxt = [−ax+ g(x)u + bv]dt+ g(x)udWt +GxdW1t

where z = col{cx, u} and where a is a positive scalar and Wt, W1t are

independent scalar Wiener processes. We assume that: G2 < 2a. ChoosingV (x) = x2p where p is a positive scalar, the resulting HJI is:

6.4 Norm-Bounded Uncertainty in the Stationary Case 125

−2apx2 − p2g(x)2x2

1 + px2g(x)2+ γ−2p2x2b2 + x2G2p+ c2x2 ≤ 0.

The requirement that there exists p > 0 that satisfies γ−2b2p2+(G−2a)p+c2 ≤0 will then be a sufficient condition for the existence of a state feedbackcontrol input u = − pg(x)

1+pg(x)2x that achieves L2-gain < γ whenever v ∈ Au

which contains at least all the functions v which satisfy ||v|| ≤ M ||x|| forsome M > 0. Internal exponential stability in the mean square sense is alsoachieved by this controller (see the remark below).

The latter inequality has a real solution if (2a − G2)2 > 4b2c2γ−2. The

minimum value of γ is thus2bc

2a−G2. For this value of γ the resulting p is

p =2c2

2a−G2.

Remark 6.2. There is a question as to whether the above differential equationhas a solution which satisfies E|xt|2 <∞. The answer for this is the followingone.

The HJI above implies: LV (x) − γ2|v|2 + |u(x)|2 + c2|x|2 ≤ 0 for allv : |v| ≤ M for some M > 0, ∀x ∈ R, and for the u introduced above.

If, for examplepg2(x)

1 + pg(x)2is bounded, one has, for some M > 0: LV (x) ≤

Mpx2 = MV (x) ∀x ∈ R. It is also easy to show that the above differentialequation coefficients satisfies sufficient conditions for the regularity, provided,for example, that g(x), g(x) are locally Lipschitz (see, e.g. [67]). These im-ply that the above system possesses a unique strong solution which satisfiesE|xt|2 < ∞ for all t ≥ 0, provided E|x0|2 < ∞. The exponential stability inthe mean square is a consequence of Theorem 6.3.2.

6.4 Norm-Bounded Uncertainty in the Stationary Case

As an application of the above theory, we consider now the special case oftime-invariant stochastic system with norm bounded uncertainties:

dxt = f(xt)xtdt+G(xt)xtdWt + g1(xt)vtdt+ g(xt)utdt (6.30)

where f(x) = (A+HF (x)E1), G(x) = (A1 +HF (x)E3), g1(x) = B1, g(x) =B2 +HF (x)E2. A,A1, B1 and B2 are matrices of appropriate dimensions.

We view the nonlinear part F (x) as an uncertainty which is to be presentin the stochastic model of (6.30). The nonlinear part F (x) is assumed to bebounded, namely

F (x)TF (x) ≤ I (6.31)

The sufficient condition, in terms of the appropriate HJI, for (6.30) to satisfyL2-gain ≤ γ is

126 6 H∞-Like Control for Nonlinear Stochastic Systems

Vx(x)f(x) − 14Vx(x)g(x)g

T (x)V Tx (x) + 1

4γ2Vx(x)g1(x)gT1 (x)Vx(x)

+ 12Tr{GT

1 (x)Vxx(x)G1(x)} + hT (x)h(x) ≤ 0, ∀x ∈ Rn(6.32)

In what follows we seek a state-feedback control u = Kx, where K is aconstant matrix of the appropriate dimensions and we consider the case wherez(x) = C1x+DKx. We construct a certain linear matrix inequality, a solutionto which will be shown to satisfy (6.32). Applying V (x) = xTPx, where P isa positive definite matrix in Rn×n, it readily follows from (6.32) that

xT[P (A+B2K +HF (x)(E1 + E2K)) + (AT +KTBT

2 + (ET1 +KTET

2 )

F (x)THT )P +γ−2PB1BT1 P + (AT

1 + ET3 F

T (x)HT )P (A1 +HF (x)E3)

+ (C1 +KTDT )(C1 +DK)

]x ≤ 0

(6.33)In a matrix inequality form we obtain, using Schur complement formula ([15]),the following:

⎡⎢⎢⎣Γ11 QA

T1 +QE

T3 F

THT B1 QCT1 +Y TDT

∗ −Q 0 0∗ ∗ −γ2I 0∗ ∗ ∗ −I

⎤⎥⎥⎦ ≤ 0 (6.34)

where:

Γ11Δ=QAT+AQ+B2Y +Y TBT

2 +HF (E1Q+E2Y )+(QET1 +Y TET

2 )FTHT ,

QΔ= P−1 and Y

Δ= KQ.

The latter inequality can also be written as:

Γ + Φ1FΦT2 + Φ2F

TΦT1 + Φ3FΦ

T4 + Φ4F

TΦT3 ≤ 0 (6.35)

where

Φ1 =[HT 0 0 0

]T, Φ2 =

[E1Q+ E2Y 0 0 0

]T, Φ3 =

[0 HT 0 0

]T,

Φ4 =[E3Q 0 0 0

]T,

(6.36)

and where

Γ =

⎡⎢⎢⎣QAT +AQ +B2Y + Y TBT

2 QAT1 B1 QCT

1 +Y TDT

∗ −Q 0 0∗ ∗ −γ2I 0∗ ∗ ∗ −I

⎤⎥⎥⎦ . (6.37)

Using the fact that for any two matrices α and β of compatible dimensions andfor any positive scalar ε the following holds: αβT + βαT ≤ εααT + ε−1ββT ,together with the bound of ( 6.31), we obtain the following:

Γ1 + ε1Φ1ΦT1 + ε−1

1 Φ2ΦT2 + ε2Φ3Φ

T3 + ε−1

2 Φ4ΦT4 ≤ 0 (6.38)

for some positive scalars ε1 and ε2. We thus obtain the following.

6.4 Norm-Bounded Uncertainty in the Stationary Case 127

Theorem 6.4.1 Consider the system of (6.30). Given the scalar 0 < γ, thereexists a state-feedback controller u = Kx that globally stabilizes the closed-loopexponentially, in the mean-square sense, and achieves L2 − gain ≤ γ if thereexist 0 < Q in Rn×n, Y in Rp×n and positive scalars ε1 and ε2 so that thefollowing LMI is satisfied:

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

Γ11 QET3 QA

T1 B1 QCT

1 +Y TDT ε1H QET1 +Y TET

2 0∗ −ε2I 0 0 0 0 0 0∗ ∗ −Q 0 0 0 0 ε2H∗ ∗ ∗ −γ2I 0 0 0 0∗ ∗ ∗ ∗ −I 0 0 0∗ ∗ ∗ ∗ ∗ −ε1I 0 0∗ ∗ ∗ ∗ ∗ ∗ −ε1I 0∗ ∗ ∗ ∗ ∗ ∗ ∗ −ε2I

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

≤ 0.

(6.39)

where Γ11Δ= QAT+AQ+B2Y +Y TBT

2 . If a solution to this LMI exists, thestate-feedback gain is given by

K=Y Q−1. (6.40)

6.4.1 Example

We consider the following model for a single degree of freedom inverted pen-dulum with the the multiplicative white noise ω:

ml2θ −mgl sin(θ) + (ς + ω)θ + kθ = u+ 2v (6.41)

where k is the spring coefficient and ς is damping coefficient. The signal vis a deterministic disturbance acting on the control input u and ω is thestochastic uncertainty in the damping. In this model, θ is the inclinationangle of the pendulum, l and m are its length and mass, respectively, and gis the gravitation coefficient. The state space representation of this model isgiven by:

⎡⎣ x1x2

⎤⎦=

⎡⎣ 0 1

ml2

mglsin(x1)x1 − k − ς

ml2

⎤⎦⎡⎣x1x2

⎤⎦+

⎡⎣0

1

⎤⎦ u+

⎡⎣0

2

⎤⎦ v +

[0

− x2ml2

]ω

where θ =[1 0

] [x1x2

]. The Ito formalism of this is:

dx = [A(x) +HF (x)E1]xdt+

[02

]vdt+

[0

− x2ml2

]dWt +

[01

]udt (6.42)

where H =[0 mgl

]T, E1 =

[1 0

]and F (x) = x−1

1 sin(x1). In the notationsof (6.30) we have:

128 6 H∞-Like Control for Nonlinear Stochastic Systems

A1 =

[0 00 − 1

ml2

], B1 =

[02

], B2 =

[01

], E2 = 0. (6.43)

For m = 0.5kg, l = 0.7meter, k = 0.5Newton/meter and ς = 0.25 we solve

(6.39). Requiring γ = 0.35, and choosing: C1 =

[1 00 0

], D =

[00.1

], we find

that the LMI of Theorem 6.4.1 is satisfied by

Q =

[0.830 −0.381−0.381 1.943

], Y = −

[0.313 112.98

]and ρ = 2.5. (6.44)

The latter leads to K = −[29.74 63.97

]. The corresponding function V =

xTQ−1x will then satisfy the HJI of (6.32).

6.5 Conclusions

H∞ control theory has been extended, via the concept of dissipative stochas-tic systems, to accommodate nonlinear stochastic systems. In particular, thestate-feedback problem is solved for this type of systems. In addition, a par-ticular class of systems is considered which consists of norm-bounded nonlin-earities. For this class, a certain LMI is introduced so that its solution satisfiesthe HJI of (6.8), and therefore provides a state-feedback controller that sta-bilizes the stochastic closed-loop system, exponentially in the mean squaresense, and also renders a L2 − gain system.

This theory of H∞ control for nonlinear stochastic systems is closely re-lated to the control and estimation problems of uncertain systems when theunderlying system is linear with uncertainties modelled as multiplicative noise(almost invariably assumed to be a Wiener process for technical reasons).However, there are instances when uncertainties of more general nature wouldbe more appropriate. One common approach to treat such uncertainties is toconsider the noise as the output of a linear system driven by a Wiener pro-cess. Augmenting the system state to accommodate both systems, a nonlin-ear system is obtained which can be treated by the theory developed in thischapter.