1

Introduction

1.1 Stochastic State-Multiplicative Time Delay Systems

The field of stochastic state-multiplicative control and filtering has greatlymatured since, its emergence in the 60’s of the last century [72], [91], [92],[93], [97], [100], [102], [121], [122], [124], [125], (see also [16], [53], [80] forextensive review). The linear quadratic optimization problems (stochasticH2)that were treated in the first two decades cleared the way, in the mid 80s, tothe H∞ worst case control strategy, resulting in a great expansion of researcheffort, aimed at the solution of related problems such as state-feedback control,estimation, dynamic output-feedback control, preview tracking control andzero-order control among other problems, for various types of nominal anduncertain systems [13], [24]-[28], [44]-[54], [94], [95], [99], [115] (see also [53]for extensive review). In the last decade a great deal of the research in thisfield has been centered around time-delay systems for both: linear systemsand nonlinear ones, where solutions have been obtained for various problemsin the H∞ context. These problems mainly concern continuous-time delayedsystems and include various stability issues, state-feedback control, estimationand dynamic output-feedback control [18]- [20], [31], [32], [55]-[64], [81], [85],[88], [89], [116], [117], [119], [120], [126], [127], [128], [130], [134], [138], similarlyto the ones encountered in the delay-free stochastic systems and indeed, inthe deterministic delayed systems counterpart [12], [33]-[36], [71], [96], [111].The discrete-time setting has been also tackled but to a lesser extent (see [39],[38], [118] and the references therein).

The stability, and control of deterministic delayed systems of various types(i.e., constant time delay, slow and fast varying delay, etc.) has been a centralfield within the system theory sciences. In the last two decades, systems withuncertain time-delay have been a subject of recurring interest, especially dueto the emergence of the H∞ control theory in the early 80s. Most of the re-search, within the control and estimation community, is based on applicationof different types of Lyapunov Krasovskii Functionals (LKFs) (see for exam-ple, [33], [79]). Also continuous-time systems with fast varying delays (i.e.,

E. Gershon & U. Shaked: Advanced Topics in Control & Estimation, LNCIS 439, pp. 1–19.DOI: 10.1007/978-1-4471-5070-1_1 c© Springer-Verlag London 2013

2 1 Introduction

without any constraints on the delay derivative) were treated via descriptortype LKF [33], where the derivative of the LKF along the trajectories of thesystem depends on the state and the state derivative. The results that havebeen obtained for the stability of deterministic retarded systems have beennaturally applied also to stochastic systems by large and in the last decade tostate-multiplicative noisy systems.

In the this monograph we extend our previous results, concerning linearstochastic systems that are summed up in [53] to linear-delayed stochastic sys-tems. We apply an input–output approach [77] for mostly delay-dependentsolutions of various control and the filtering stochastic problems. By this ap-proach, the system is replaced by one with norm-bounded operators withoutdelays, which is treated by the standard theory of non-retarded systems withstate-multiplicative noise [53]. In our system we allow for a slowly varying de-lay, (i.e. delays with the derivative less than 1) where the uncertain stochasticparameters multiply both the delayed and the non delayed states in the statespace model of the system. In the filtering and dynamic output-feedback prob-lems of Chapter 2, we incorporate also a stochastic multiplicative noise in themeasurement equation. We address first the stability issue of the stochasticstate-multiplicative delayed systems and we then formulate and find the con-ditions for the corresponding Bounded Real Lemma (BRL). Based on theBRL we then solve the various control and the filtering problems in Chapters2–5. We also demonstrate the applicability and tractability of our results viamany examples, some of which are given in the latter chapters and some,the more advanced ones, appear in Chapter 13 (Applications). We start thismonograph by introducing first, in the following section, the input–outputapproach for the the design of control and estimation of linear time-delayedstochastic systems.

1.2 The Input–Output Approach for Delayed Systems

In Chapters 2–5 we apply the input-output approach to linear time invariantdelayed systems. This approach transforms a given delayed system to a norm-bounded uncertain system that can be treated, in the stochastic context, bythe various solutions methods that can be found in [53]. The major advantageof the input-output approach lies in its simplicity of use such that the resultinginequalities that emerge are relatively tractable and simple, for both: delay-dependent and delay-independent solutions. However, this technique entailssome degree of conservatism that can be compensated by a clever choice of theLyapunov function that is involved in the solution method. In the followingtwo subsections we introduce the input-output approach for continuous-time,and discrete-time stochastic systems.

1.2 The Input–Output Approach for Delayed Systems 3

1.2.1 Continuous-Time Case

We consider the following system:

dx(t) = [A0x(t) + A1x(t− τ(t))]dt +Hx(t− τ(t))dζ(t)+Gx(t)dβ(t), x(θ) = 0, θ ≤ 0,

(1.1)

where x(t) ∈ Rn is the state vector and A0, A1 and G, H are time invariantmatrices and where β(t), ζ(t) are zero-mean real scalar Wiener processessatisfying:

E{β(t)β(s)} =min(t, s), E{ζ(t)ζ(s)} =min(t, s),

E{β(t)ζ(s)}= α ·min(t, s), |α| ≤ 1.

In (1.1), τ(t) is an unknown time-delay which satisfies:

0 ≤ τ(t) ≤ h, τ(t) ≤ d < 1. (1.2)

In the input–output approach, we use the following operators:

(Δ1g)(t)Δ= g(t− τ(t)), (Δ2g)(t)

Δ=

∫ t

t−τ(t)

g(s)ds. (1.3)

In what follows we use the fact that the induced L2-norm of Δ1 is boundedby 1√

1−d, and, similarly to [77], the fact that the induced L2-norm of Δ2 is

bounded by h. Using the above operator notations, the system (1.1) becomesa special case of the following system:

dx(t) = [A0 +m]x(t)dt + (A1 −m)w1(t)dt−mw2(t)dt+Gx(t)dβ(t)+Hw1(t)dζ(t) − Γβdt− Γζdt,y(t) = [A0 +m]x(t) + (A1 −m)w1(t)−mw2(t)− Γβ − Γζ ,

(1.4)

where

Γβ = m

∫ t

t−τ

Gx(s)dβ(s), and Γζ = m

∫ t

t−τ

Hw1(s)dζ(s), (1.5)

and wherew1(t) = (Δ1x)(t), and w2(t) = (Δ2y)(t). (1.6)

Remark 1.1. The dynamics of (1.1) is a special case of that of (1.4) as follows:Noting (1.6) and applying the operators of (1.3), Equation (1.4a) can be writ-ten as:

dx(t) = [A0 +m]x(t)dt + (A1 −m)w1(t)dt−m{∫ t

t−τ

y(t,)dt,}dt

+Gx(t)dβ(t) +Hw1(t)dζ, w1(t) = x(t− τ(t)) .

4 1 Introduction

Now, recalling y of (1.4b) one can write:

dx(t) = y(t)dt+Gx(t)dβ(t) +Hw1(t)dζ(t)

and therefore

y(t,)dt, = dx(t,)−Gx(t,)dβ(t,)−Hw1(t,)dζ(t,).

Hence,

−mw2(t)Δ= −m

∫ t

t−τ

y(t,)dt, = −m∫ t

t−τ

{dx(t,)−Gx(t,)dβ(t,)−Hw1(t,)dζ(t,)}

= −mx(t) +mx(t− τ) + Γβ + Γζ = −mx(t) +mw1(t) + Γβ + Γζ ,

where Γβ and Γζ are defined in (1.5a,b) respectively. Replacing the right handside of the latter equation for −mw2(t) in (1.4a), the dynamics of (1.1a) isrecovered.

We note that the matrix m is a n × n unknown constant matrix to be de-termined. This matrix is introduced into the dynamics of (1.4) in order toachieve additional degree of freedom in the design of the various controllers inall the linear continuous-time delayed systems considered in this book. Usingthe fact that ||Δ1||∞ ≤ 1√

1−dand ||Δ2||∞ ≤ h, (1.4) may be cast into what

is entitled: the norm-bounded uncertain model, by introducing into (1.1) theabove new variables of (1.6) where ||Δ1||∞ ≤ 1√

1−dand ||Δ2||∞ ≤ h are

diagonal operators having identical scalar operators on the main diagonal.

1.2.2 Discrete-Time Case

We consider the following linear retarded system:

xk+1 = (A0 +Dνk)xk + (A1 + Fμk)xk−τ(k),xl = 0, l ≤ 0,

(1.7)

where xk ∈ Rn is the system state vector and where the time delay is denotedby the integer τk and it is assumed that 0 ≤ τk ≤ h, ∀k. The variables {μk}and {νk} are zero-mean real scalar white-noise sequences that satisfy:

E{νkνj} = δkj , E{μkμj} = δkj ,

E{μkνj} = 0, ∀k, j ≥ 0.

The matrices in (1.7) are constant matrices of appropriate dimensions.In order to tackle the stability (and hence the BRL) of the retarded

discrete-time system, we introduce the following scalar operators which areneeded, in the sequel, for transforming the delayed system to an equivalentnorm-bounded nominal system:

1.3 Non Linear Control of Stochastic State-Multiplicative Systems 5

Δ1(gk) = gk−h, Δ2(gk) =

k−1∑j=k−h

gj . (1.8)

Denoting

yk = xk+1 − xkand using the fact that

Δ2(yk) = xk − xk−h,

the following state space description of the system is obtained:

xk+1 = (A0 +Dνk +M)xk + (A1 −M + Fμk)Δ1(xk)−MΔ2(yk)

xl = 0, l ≤ 0,

where the matrix M is a free decision variable to be determined. Similarly tothe continuous-time case, this matrix is introduced into the dynamics of (1.7)in order to achieve additional degree of freedom in the design of the state-feedback controller and the estimator of Chapter 5 in this book. We considerthen the following auxiliary system:

xk+1 = (A0 +Dνk +M)xk + (A1 −M + Fμk)w1,k −Mw2,k (1.9)

with the feedback

w1,k = Δ1(xk), w2,k = Δ2(yk). (1.10)

In Chapter 5, it is shown how the norm-boundness of the operators of (1.8)together with the auxiliary system of (1.9) enable the solution of the stabilityissue of discrete-time stochastic retarded systems.

1.3 Non Linear Control of StochasticState-Multiplicative Systems

1.3.1 The Continuous-Time Case

In recent years there has been a growing interest, as reflects from the variouspublished research works, in the extension of H∞ control and estimation the-ory to accommodate stochastic systems [13], [24]-[28], [44]-[54] (see also [53]for extensive review).

The main thrust for these efforts stems from the attempt to model sys-tem uncertainties as a stochastic process, in particular, as a white noise, orformally as a Wiener process. This has led to the development of an H∞theory for stochastic linear systems with multiplicative noise. There has beenalso some work done in the direction of extending the linear stochastic H∞

6 1 Introduction

control theory to accommodate stochastic nonlinear systems (see, e.g., [5],[6], [7], [17] and [137]). As [6], [7], [137] have developed theories for the statefeedback case, [17] and [5] considered the output feedback case. The latteradopted a differential game point of view, and stressed the connection to therisk-sensitive control (see, e.g. [29] and [76]), while the former took the differ-ential game approach and considered (for technical reasons) finitely additivemeasure space as a foundation for the measurement process structure. Theobjective of the present chapter is to develop an H∞-like control theory fornonlinear stochastic systems with output feedback, exhibiting a combinationof deterministic and stochastic uncertainties, together with a presence of whatis called norm bounded uncertainty. In particular, we consider the followingnonlinear stochastic system.

dxt = f(xt,t)dt+g(xt, t)utdt+g1(xt,t)vtdt+ g(xt,t)utdWt

+g2(xt,t)vtdW2t +G(xt,t)dW

1t

(1.11)

dyt = h2(xt, t)dt+ g3(xt, t)vtdt+G2(xt, t)dW3t (1.12)

where {xt}t≥0 is a solution to (1.11) with: an initial condition x0, an exogenousdisturbance {vt}t≥0, a control signal {ut}t≥0, and Wiener processes {Wt}t≥0,{W 1

t }t≥0, {W 2t }t≥0, {W 3

t }t≥0. Also, yt is an Rp-valued observation vectorwhich is corrupted by noise (Wiener processes {W 3

t }t≥0), and contains anuncertain component (a stochastic process {vt}t≥0).

This type of system may be viewed, on one hand, as an extension of thelinear case, which has been extensively treated in the literature (see, e.g., [53],and the references therein, or [69] where a motivation for considering suchsystems is provided), and on the other hand it may be considered as a resultof an expansion of a general nonlinear stochastic system in terms of its statesand its uncertain variables up to the second order terms, or Volterra typestochastic systems.

The following will be assumed to hold throughout this work.1. Let (Ω,F, {Ft}t≥0, P ) be a filtered probability space where {Ft}t≥0

is the family of sub σ- algebras generated by {Wt}t≥0, {W 1t }t≥0, {W 2

t }t≥0,{W 3

t }t≥0, which are all Wiener processes taken to be R1- valued, Rl- valued,R1-valued, Rm3-valued, respectively.

2. All the functions below are assumed to be Borel measurable on Rn ×[0,∞). f : Rn × [0,∞) → Rn, g : Rn × [0,∞) → Rn×m, g1 : Rn × [0,∞) →Rn×m1 , g2 : Rn × [0,∞) → Rn×m2 , G : Rn × [0,∞) → Rn×l. In aadditionG2 : Rn× [0,∞) → Rp×m3 , h2 : Rn× [0,∞) → Rp, g3 : R

n× [0,∞) → Rp×m1 .It is also assumed that f(0, t) = 0, G(0, t) = 0, h2(0, t) = 0, G2(0, t) = 0 forall t ≥ 0.

3. {vt}t≥0 is a non-anticipative Rm1 -valued stochastic process defined on

(Ω,F, {Ft}t≥0, P ), which satisfies E{∫ t

0 ‖vs‖2 ds} <∞ for all t ∈ [0,∞), whereE denotes expectation: E{x} =

∫Ωx(ω)dP (ω).

4. {ut}t≥0 is a non-anticipative Rm-valued stochastic process defined on(Ω,F, {Ft}t≥0, P ).

1.3 Non Linear Control of Stochastic State-Multiplicative Systems 7

5. x0 is assumed to be F0-measurable, and to satisfy E{‖x0‖2} <∞In this chapter we consider the case for which the solution xt satisfies

E{||xt||2} < ∞, ∀t ≥ 0. For the pertaining conditions which guarantee thissee e.g. [65].

Definition 1.3.1 The pair {ut, vt}t∈[(0,∞), or in short {u, v}, is said to beadmissible if the stochastic differential equation (1.11) possesses a uniquestrong solution relative to the filtered probability space (Ω,F, {Ft}t≥0, P ) so

that E{‖xt‖2} <∞ for all t ∈ [(0,∞).

Remark 1.2. The family of all admissible pairs {u, v} will be denoted by A.The notation Au will be used for all admissible pairs {u, v} with fixed u. Wenote that Au may be empty for some non-anticipative u.

Let α(·, ·) be positive Borel function on Rn × [0,∞) (where Rn × [0,∞) isendowed with the Borel σ-algebra). In what follows it will be assumed thatE{α(x, t)} <∞ for all t ∈ [0,∞) and for all F -measurable, Rn-valued randomvariables which satisfy E{||x||2} <∞. The control objective is now stated asfollows. Consider the controlled output:

zt =

[h(xt, t)ut

], t ∈ [0,∞) (1.13)

where h : Rn× [0,∞) → Rr is a Borel measurable function, and let Yt = {ys :s ≤ t}. Find an output-feedback controller ut = u(Yt, t) such that, for a givenγ > 0, the following H∞ criterion is satisfied.

E{∫ t2

t1

‖zt‖2 dt} ≤ γ2E{α(xt1 , t1) +∫ t2

t1

‖vt‖2 dt} (1.14)

for all 0 ≤ t1 < t2, for all F0-measurable x0 with E{||x0||2} < ∞, and forall disturbances vt in Au (provided Au is nonempty). Whenever the system(1.11) satisfies the above inequality, it is said to possess an L2-gain that isless than or equal to γ. Note that for the infinite time-horizon t2 = ∞, and itis required that v satisfies:

∫∞0 ||vt||2dt <∞.

As we adopt the stochastic dissipativity point of view in dealing with thiscontrol problem, we first recall the concept of stochastic dissipative systemsand then discuss some properties of these systems. In addition, we introducewhat we call the Bounded Real Lemma (BRL) for nonlinear stochastic sys-tems which was first developed in [6]. This is done in Section 2. In Section 3,we develop the H∞ output-feedback control for nonlinear stochastic systemsof the type described by (1.11). We introduce there a pair of Hamilton-JacobiInequalities (HJI), the solution of which yields an output-feedback controllerthat renders the underlying closed-loop system L2-gain≤ γ. The approachtaken here is analogous to the one introduced by Isidori ([73]) in the frame-work of the deterministic counterpart. Section 4 deals with the special case ofsystems with norm-bounded uncertainties where we introduce certain matrix

8 1 Introduction

inequalities whose solution yields a robust output-feedback. In Section 5, weconsider the infinite-time horizon where we introduce sufficient conditions forthe synthesis of a stabilizing (in both mean square and in probability senses)output-feedback controller. This section is concluded with a simple exampleof a single-degree-of-freedom inverted pendulum.

1.3.2 Stability

We recall few facts from the theory of stochastic stability (see e.g. [67]). Weremark that in what follows global stability is considered. Obviously, localstability results may also be achieved, in a similar way.

Definition 1.3.2 Consider the stochastic system

dxt = f(xt, t)dt+G(xt, t)dWt (1.15)

with f(0, t) = G(0, t) = 0 for all t ≥ 0, and assume that f and G are such that(1.15) possesses a unique strong solution for all t ≥ 0 relative to the filteredprobability space (Ω,F, {Ft}t≥0, P ), where {Ft} is generated by the Wienerprocess {Wt}t≥0. The solution xt is said to be stable in probability if for anyε > 0 lim

x→0P{sup

t≥0‖xt‖ > ε} = 0.

Definition 1.3.3 The solution xt of (1.15) is said to be globally asymptoti-cally stable in probability if it is stable in probability, and if P{ lim

t→∞xt = 0} = 1

for any initial state x0 ∈ Rn.

A sufficient condition for a global stability in probability is given by the fol-lowing theorem.

Theorem 1.3.1 ([67]) Assume there exists a positive storage function V (x, t) ∈C2,1, with V (0, t) = 0. Let L(x, t) be the infinitesimal generator of the processxt, that is

LV (x, t) = Vx(x, t)f(x, t) +1

2Tr{GT (x, t)Vxx(x, t)G(x, t)},

so that (LV )(x, t) < 0 for all x ∈ Rn and for all t ≥ 0. Assume also thatinft>0

V (x, t) → ∞ as ‖x‖ → ∞. Then, the system of (1.15) is globally asymp-

totically stable in probability.

Definition 1.3.4 [stability in the mean-square sense]The system (1.15) is said to be globally exponentially stable in the mean-square

sense if E{||xt||2} ≤ kE{‖xs‖2} exp{−α(t − s)} for all 0 ≤ s ≤ t, and forsome positive numbers k and α.

Theorem 1.3.2 ([67]) Assume there exists a positive function V (x, t) ∈ C2,1,with V (0, t) = 0. Then, the system of (1.15) is globally exponentially stable ifthere are positive numbers k1, k2, k3 such that the following hold.

k1||x||2 ≤ V (x, t) ≤ k2||x||2, (LV )(x, t) ≤ −k3||x||2 for all t ≥ 0. (1.16)

1.3 Non Linear Control of Stochastic State-Multiplicative Systems 9

1.3.3 Dissipative Stochastic Systems

In this section we summarize various results pertaining what is called Stochas-tic Bounded Real Lemma (SBRL) which serve as a basis for the theory to bedeveloped in the sequel. A full account of what follows may be found in ([113],[7], and [11]).

Let S : Rm × Rr+m1 → R, be a Borel measurable function which will beentitled supply rate.

Definition 1.3.5 Consider the system (1.11) together with the controlled out-put z(t) as defined in (1.13), and let S be a supply rate as defined above. Let ube such that Au is nonempty. Then, the system (1.11) is said to be dissipativewith respect to the supply rate S if there is a function V : Rn × [0,∞) → R,with V (x, t) ≥ 0 for all x ∈ Rn and t ∈ [0,∞), so that V (0, t) = 0 ∀t ∈ [0,∞)satisfies E{V (x, t)} < ∞ for all t and for all F -measurable r.v.s satisfyingE{||x||2} <∞, such that:

E{V (xt, t)} ≤ E{V (xs, s)}+ E{∫ t

s

S(vσ, zσ)dσ} (1.17)

for all t ≥ s ≥ 0 and for all admissible disturbances {vt}t≥0 in Au, where xtis the solution to the differential equation (1.11). V is then called a storagefunction for the system (1.11).

Similar to the deterministic theory of dissipative systems, the theorem be-low establishes conditions under which the system (1.11) possesses a storagefunction. First we introduce a candidate for a storage function.

Definition 1.3.6 Consider the system (1.11) and let x be an Rn valued ran-

dom variable defined on the probability space (Ω,F, P ) with E{‖x‖2} < ∞.Assume also that x is Ft measurable. Let u be such that Au is nonempty.Given t ∈ [0,∞) and let xt = x. Define

Va(x, t) = supT ≥ t, v ∈ Au

[−E{[∫ T

t

S(vs, zs)ds]/x}]. (1.18)

1.3.4 The Discrete-Time-Time Case

H∞ control for discrete-time deterministic nonlinear systems has been con-sidered by numerous researchers, see e.g. [87], which utilize, in part, the dis-sipativity concept. In order to develop an analogous theory for the stochasticcounterpart, we setup, in the present paper, some theory of stochastic dissi-pativity. A related topic is the risk sensitive control problems (see, e.g. [29]and references therein) which, in general, deal with optimization of stochasticsystems where the cost function involves an exponential.

In Chapter 9 an effort is made to extend the work presented in [6], inorder to include discrete-time nonlinear stochastic systems. The contribution

10 1 Introduction

of this paper is in that it provides means (the Bounded Real Lemma-BRL)for synthesizing a state-feedback H∞ controller for a large class of nonlinearstochastic systems. In addition, the BRL facilitates, in a natural way, theutilization of the Linear Matrix Inequality (LMI) techniques to achieve l2-gain≤ γ for a large class of uncertain nonlinear systems with norm boundeduncertainties. Another contribution of this paper is the introduction of whatwe call stochastic dissipation, which serves as a basis for the H∞ controltheory developed in the sequel.

We consider the following stochastic system

xk+1 = fk(xk, vk, uk, ωk) (1.19)

where {xk}k≥0 is a solution to (1.19), with: an initial condition x0, an ex-ogenous disturbances {vk}k≥0, a control signal {uk}k≥0, and a white noisesequence ω = {ωk}k≥0 defined on a probability space (Ω,F, P ). In the sequelω = {ωk}k≥0 describes both exogenous random inputs and parameter uncer-tainty of the system. The following will be assumed to hold throughout thiswork.

1. Let (Ω,F, {Fk}k≥0, P ) be a filtered probability space where {Fk}k≥0

is the family of sub σ-algebras of F generated by {ωk}k≥0, where ωk areassumed to be Rl-valued. In fact, each Fk is assumed to be the minimal σ-algebra generated by {ωi}0≤i≤k−1 while F0 is assumed to be some given subσ-algebra of F , independent of Fi for all i > 0.

2. {uk}k≥0 and {vk}k≥0 are non-anticipative (that is uk, vk are independentof {Fi, i > k}), Rm and Rm1 valued, respectively, stochastic processes definedon (Ω,F, {Fk}k≥0, P ). The vectors xk are Rn valued, and for each k, fk :Rn×Rm1×Rm×Rl → Rn is assumed to be continuous onRn×Rm1×Rm×Rl.

3. {vk}k≥0 satisfies E{∑N

k=0 ||vk||2} <∞ ∀N ≥ 0, where E stands for theexpectation operation.

4. x0 is assumed to be F0-measurable and to satisfy E{||x0||2} <∞.5. The following notation will be used in the sequel. Let X,Y be Rn1 and

Rn2-valued random variables defined on (Ω,F, P ) and let V : Rn1 × Rn2 →R. Let Py be the probability distribution of Y . Then the random variableEy{V (X,Y )} is defined by Ey{V (X,Y )} .=

∫Rn2

V (X, y)dPy(y).In this work we investigate the problem of stochastic H∞ state-feedback

control which is formulated as follows. Given a controlled output:

zk(x, u) =

[hk(x)u

](1.20)

where hk : Rn → Rr, synthesize a controller uk = Kk(xk) such that, for agiven γ > 0, the following H∞ criterion is satisfied.

E{∑k−1

i=j ||zi(xi, ui)||2} ≤ E{βj(xj)}+∑k−1

i=j γ2E{||vi||2} (1.21)

1.3 Non Linear Control of Stochastic State-Multiplicative Systems 11

for all 0 ≤ j ≤ k, for all {vk}k≥0 such that∑k

i=0 E{||vi||2} < ∞, whereβj : Rn → R+ are positive functions satisfying E{βj(x)} < ∞ whenever xis an F -measurable random variable satisfying E{||x||2} <∞. A system thatsatisfies this property is said to be l2-gain≤ γ. We also denote the family of allexogenous disturbances v which satisfy

∑Ni=0 E{||vi||2} < ∞, and such that

(1.19) possesses a solution {xk}k≥0 with E{||xk||2 < ∞} for some {uk}k≥0,by Au.

1.3.5 Stability

Consider the following nonlinear stochastic system

xk+1 = fk(xk, wk) (1.22)

where xk ∈ Rn and {wk}k is an Rl-valued stochastic process defined on theprobability space (Ω,F, {Fk}k≥0, P ) as defined above. It will be assumed, inwhat follows, that (1.22) has a unique solution for all k ≥ 0. The following isstandard.

Definition 1.3.7 The (1.22) is said to be globally exponential mean squarestable if there is a positive constant a < 1 so that the solution of (1.22) satisfies

E{||xk||2} ≤ akE{||x0||2} ∀k ≥ 0, x0 ∈ L2(Ω,F0, P,Rn)

The following sufficient conditions are obtained for the system (1.22) to beexponentially mean square stable; these are a straightforward adjustment ofthe continuous-time counterparts.

Theorem 1.3.3 Consider the system (1.22). Let {Vk(x)}k≥0 be a family ofpositive functions defined on Rn such that the following conditions are satis-fied. There are positive constants k1, k2, k3 so that:

1. k1||x||2 ≤ Vk(x) ≤ k2||x||2 ∀k ≥ 0 ∀x ∈ Rn

2. Ewk{Vk+1(fk(x,wk))} − Vk(x) ≤ −k3||x||2 ∀k ≥ 0 ∀x ∈ Rn.

Then, the system (1.22) is exponentially mean square stable.

1.3.6 Dissipative Discrete-Time Nonlinear StochasticSystems

In analogy to the continuous-time case, we define the notion of stochasticdissipative system.

Definition 1.3.8 Consider the discrete-time stochastic system (1.19) definedon the discrete time interval [0,∞), with the associated controlled output(1.20) and let S : Rr+m × Rm1 → R be a Borel measurable function onRr+m × Rm1 . Then, the system (1.19) is said to be dissipative with respect

12 1 Introduction

to S if, for some admissible control sequence {uk}k, there exists a family offunctions Vk : (Rn × N+) → R with Vk(x) ≥ 0 ∀x ∈ Rn, k ∈ N+ (N+

denotes the positive integers), so that Vk(0) = 0 ∀k ∈ N+, E{V0(x0)} <∞,and

E{Vk(xk)} ≤ E{Vj(xj)}+ E{k−1∑i=j

S(zi, vi)} (1.23)

for all k : k ≥ j ≥ 0, with the convention:∑l−1

i=j S(zi, vi) = 0 whenever l = j,

and for all non-anticipative v = {vi}i≥0 with E{∑∞

i=0 ||vi||2} <∞. We call Ssupply rate (see e.g. Willems [122] for the deterministic case), and the familyV = {Vk}k is said to be a storage function of the system.

We have now the following trivial result which connects the l2-gain propertyto the notion of stochastic dissipation.

Lemma 1.3.1 Consider the system (1.19) and let the supply rate be S(z, v) =γ2||v||2 − ||z||2. Assume the system possesses a storage function V so that thesystem is dissipative with respect to this S. Then, the system has l2-gain ≤ γ.

1.4 Stochastic Processes – Short Survey

Stochastic processes are a family of random variables parameterized by timet ∈ T . Namely, at each instant t, x(t) is a random variable. When t is con-tinuous (namely T = R), we say that x(t) is a continuous-time stochasticprocess, and if t is discrete (namely T = {1, 2, ....}), we say that x(t) is adiscrete-time variable. For any finite set of {t1, t2, ...tn} ∈ T , we can define thejoint distribution F (x(t1), x(t2), ..., x(tn)) and the corresponding joint densityp(x(t1), x(t2), ..., x(tn)).

The first and the second order distribution functions, p(x(t)) and p(x(t),x(τ)), respectively, play an important role in our discussion. Also the mean

mx(t)Δ= E{x(t)} and the autocorrelation γx(t, τ)

Δ= E{x(t)x(τ)} are use-

ful characteristics of the stochastic process x(t). When x(t) is vector valued,the autocorrelation is generalized to be Γx(t, τ) = E{x(t)x(τ)T }. The co-variance matrix of a vector valued stochastic process x(t) is a measure of

its perturbations with respect to its mean value and is defined by Px(t)Δ=

E{(x(t)−mx(t))(x(t) −mx(t))T }.

A process x(t) is said to be stationary if

p(x(t1), x(t2), ..., x(tn)) = p(x(t1 + τ), x(t2 + τ), ..., x(tn + τ))

for all n and τ . If the latter is true only for n = 1, then the process x(t)is said to be stationary of order 1 and then p(x(t)) does not depend on t.Consequently, the mean mx(t) is constant and p(x(t), x(τ)) depends only ont − τ . Also in such a case, the autocorrelation function of two time instantsdepends only on the time difference, namely γx(t, t− τ) = γx(τ).

1.5 Mean Square Calculus 13

An important class of stochastic processes is one of Markov processes. Astochastic process x(t) is called a Markov process if for any finite set of timeinstants t1 < t2 < ... < tn−1 < tn and for any real λ it satisfies

Pr{x(tn) < λ|x(t1), x(t2), ..., x(tn−1), x(tn)} = Pr{x(tn)|x(tn−1}.

Stochastic processes convergence properties of a process x(t) to a limit xcan be analyzed using different definitions. The common definitions are al-most sure or with probability 1 convergence (namely x(t) → x almost surely,meaning that this is satisfied except for an event with a zero probability), con-vergence in probability (namely for all ε > 0, the probability of |x(t)− x| ≥ εgoes to zero), and mean square convergence, where given that E{x(t)2} andE{x2} are both finite, E{(x(t) − x)2} → 0. In general, almost sure conver-gence neither implies nor it is implied by mean square convergence, but bothimply convergence in probability. In the present book we adopt the notion ofmean square convergence and the corresponding measure of stability, namelymean square stability.

1.5 Mean Square Calculus

Working with continuous-time stochastic processes in terms of differentia-tion, integration, etc. is similar to the analysis of deterministic functions, butit requires some extra care in evaluation of limits. One of the most usefulapproaches to calculus of stochastic processes is the so called mean squarecalculus where mean square convergence is used when evaluating limits.

The full scope of mean square calculus is covered in [75] and [105] but webring here only a few results that are useful to our discussion.

The notions of mean square continuity and differentiability are key issuesin our discussion. A process x(t) is said to be mean square continuous iflimh→0x(t + h) = x(t). It is easy to see that if γx(t, τ) is continuous at (t, t)then also x(t) is mean square continuous. Since the converse is also true,then mean square continuity of x(t) is equivalent to continuity of γ(t, τ) in(t, t). Defining mean square derivative by the mean square limit as h → 0of (x(t + h) − x(t))/h, then it is similarly obtained that x(t) is mean squaredifferentiable (i.e. its derivative exists in the mean square sense) if and only ifγx(t, τ) is differentiable at (t, t). A stochastic process is said to be mean square

integrable, whenever∑n−1

i=0 x(τi)(ti+1 − ti) is mean square convergent wherea = t0 < t1 < ... < tn = b, where τi ∈ [ti, ti+1] and where |ti+1 − ti| → 0. In

such a case, the resulting limit is denoted by∫ b

a x(t)dt. It is important to knowthat x(t) is mean square integrable on [a, b] if and only if γx(t, τ) is integrableon [a, b]× [a, b]. The fundamental theorem of mean square calculus then statesthat if x(t) is mean square integrable on [a, b], then for any t ∈ [a, b], we have

x(t) − x(a) =∫ t

a

x(τ)dτ.

14 1 Introduction

The reader is referred to [75] for a more comprehensive coverage of meansquare calculus.

1.6 White Noise Sequences and Wiener Process

In this section we consider both: discrete-time white noise type stochasticprocesses (Chapters 5,9,10) and continuous-time Wiener-type stochastic pro-cesses (Chapters 2–4, 6–8, 11). We start with the description of the Wienertype stochastic processes.

1.6.1 Wiener Process

A process β(t) is said to be a Wiener Process (also referred to as Wiener-Levy process or Brownian motion) if it has the initial value of β(0) = 0with probability 1, has stationary independent increments, and is normallydistributed with zero mean for all t ≥ 0. The Wiener process has then thefollowing properties : β(t)− β(τ) is normally distributed with zero mean andvariance σ2(t−τ) for t > τ where σ2 is an empirical positive constant. Considernow for t > τ the autocorrelation

γβ(t, τ) = E{βtβτ )} = E{(β(t) − β(τ) + β(τ))β(τ)}= E{(β(t)− β(τ))β(τ)} + E{β2(τ)}.

Since the first term is zero, due to the independent increments property ofthe Wiener process, it is readily obtained that γβ(t, τ) = σ2τ . Since we haveassumed that t > τ we have in fact that γβ(t, τ) = σ2min(t, τ). Since thelatter is obviously continuous at (t, t), it follows that β(t) is mean square con-tinuous. However, a direct calculation (see [75]) of the second order derivativeof γβ(t, τ), with respect to t and τ at (t, t), shows that

min(t+ h, t+ h′)−min(t, t)hh′

= 1/max(h, h′)

which is clearly unbounded as h and h′ tend to zero. Therefore, γβ(t, τ) isnot differentiable at any (t, t) and consequently β(t) is not mean square dif-ferentiable anywhere. It is, therefore, concluded that the Wiener process iscontinuous but not differentiable in the mean square sense. In fact, it canbe shown that the latter conclusion holds also in the sense of almost sureconvergence.

1.6.2 White Noise Sequences

A discrete-time process is said to be white if it is a Markov process and if allx(k) are mutually independent. Such a process is said to be a white Gaus-sian noise if, additionally, its samples are normally distributed. The mutual

1.6 White Noise Sequences and Wiener Process 15

independence property leads, in the vector valued case, to E{x(n)xT (m)} =Qnδn,m where δn,m is the Kronecker delta function (1 for equal arguments andzero otherwise) and where Qn ≥ 0. The discrete-time white noise is a usefulapproximation of measurement noise in many practical cases. Its continuous-time analog also appears to be useful. Consider a stationary process x(t) whosesamples are mutually independent, taken at large enough intervals. Namely,

γ(τ) = E{x(t+ τ)x(t)} = σ2ρ

2e−ρ|τ

where ρ >> 1. As ρ tends to infinity γ(τ) rapidly decays as a function of τ , andtherefore the samples of x(t) become virtually independent and the processbecomes white. Noting that for ρ that tends to infinity, ρ

2e−ρ|τ → δ(τ) where δ

is the Dirac delta function [75], a vector valued white process x(t) is formallyconsidered to have the autocorrelation of γ(τ) = Q(t)δ(τ) where Q(t) ≥ 0.Namely, E{x(t)x(τ)} = Q(t)δ(t− τ). Defining the spectral density of x(t) bythe Fourier transform of its autocorrelation, namely by

f(ω) =

∫ ∞

−∞e−iτωσ2

ρ

2e−ρ|τ |dτ =

σ2

1 + ω2/ρ2

we see that this spectral density is constant and has the value of σ2 up toabout the frequency ρ where it starts dropping to zero. Namely, the spectrumof x(t) is nearly flat independently of the frequency, which is the source ofthe name “white” noise, in analogy to white light including all frequenciesor wavelengths. We note that for finite ρ >> 1, x(t) is said to be a wide-band noise (where 1 may represent the measured process bandwidth and ρ themeasurement noise bandwidth). In such a case, modelling x(t) as a white noiseis a reasonable approximation. We note, however, that constant f(ω) or whitespectrum corresponds to infinite energy by Parseval’s theorem . Alternatively,looking at the autocorrelation at τ = 0, we see that

γ(0) = E{x2(t)} =1

2π

∫ ∞

−∞f(ω)dω → ∞.

Therefore, white noise is not physically realizable but is an approximationto wide band noise. To allow mathematical manipulations of white noise, werelate it to Wiener processes which are well defined. To this end we recall thatthe autocorrelation of a Wiener process β(t) is given by

γ(t, τ) = E{x(t)x(τ)} = σ2min(t, τ).

Since expectation and derivatives can be interchanged, namely

E{dβ(t)dt

dβ(τ)

dτ} =

d2

dtdτE{βtβτ},

it follows that the autocorrelation of ˙β(t) is given by σ2 ddτ [

ddtmin(t, τ)]. How-

ever, min(t, τ) is τ for τ < t and t otherwise; therefore, its partial derivative

16 1 Introduction

with respect to t is a step function of τ rising from 0 to 1 at τ = t. Con-sequently, the partial derivative of this step function is just σ2δ(t − τ). Theautocorrelation of β(t) is thus σ2δ(t− τ), just as the autocorrelation of whitenoise, and we may, therefore, formally conclude that white noise is the deriva-tive, with respect to time, of a Wiener process.

1.7 Stochastic Differential Equations

Many stochastic processes are formally described (see [75] and [105]) by theLangevin’s equation:

dx

dt= f(x(t), t) + g(x(t), t)β(t)

where β(t) is a white noise process. For example, in this monograph, theso called state-multiplicative process is obtained when f(x(t), t) = Ax andg(x(t), t) = Dx leading to

dx

dt= Ax+Dxβ(t).

When we write the latter in terms of differentials rather than in terms ofderivatives, we obtain the following equation

dx = Axdt+Dxdβ

where the physically unrealizable β(t) no longer appears but instead the dif-ferential dβ of β(t) drives the equation. Note that

dβ(t)Δ= β(t)− β(t− dt)

is normally distributed with zero mean and σ2(t − (t − dt)) = σ2dt. Whenσ2 = 1 we say that the Wiener process β(t) and the corresponding white noiseprocess β(t) are standard. Back to Langevin’s equation, we may realize thatit can also be written in terms of differentials as

dx(t) = f(x(t), t)dt + g(x(t), t)dβ(t).

This equation is, in fact, interpreted by

x(t)− x(t0) =∫ t

t0

f(x(τ, τ)dτ +

∫ t

t0

g(x(τ, τ)dβ(τ),

where the first term is a Lebesgue-Stieltges integral and the second term isan Ito integral with respect to the Wiener process β(t). Namely, this integralis defined via approximation by the sum:

1.8 Ito Lemma 17

n−1∑i=0

gti [β(ti+1)− β(ti)], where a = t0 < t1 < ... < tn = b,

where gti is a random variable at the fixed time ti which is Fti measurable(see Section 1.4.1). It is assumed that gti is independent on future incrementsβ(tk)−β(tl) for ti ≤ tl ≤ tk ≤ b of β(t). The stochastic integral is then definedby choosing a series gnof piecewise step functions which converge to g, in thesense that the mean square of the integral of gn−g tends to zero as n tends toinfinity. Whenever gt is mean square integrable and is independent of futureincrements of β(t), the stochastic Ito sense integral exists. Furthermore, itsatisfies two useful identities:

E{∫gtdβ(t)} = 0, and E{

∫gtdβ(t)

∫ftdβ(t)} = σ2

∫E{gtft}dt.

In fact, we have defined in the above the first order stochastic integral. Thesecond order stochastic integral in the Ito sense,

∫ t

0 gtdβ2(t), is similarly de-

fined by taking the limit of n to infinity in∑n−1

i=0 gti [β(ti+1) − β(ti)]. It can

be shown [75] that the latter converges, in mean square, to just∫ t

t0σ2gtdt.

We consider next

dx(t) = x(t)dβ(t) +1

2x(t)dβ2(t)

where x(0) = 1 almost surely. Integrating the latter yields

x(t) − 1 =

∫ t

0

x(t)dβ(t) +1

2

∫ t

0

x(t)dβ2(t) =

∫ t

0

x(t)dβ(t) +σ2

2

∫ t

0

x(t)dt.

The latter is simply the integral form in the Ito sense of:

dx(t) = x(t)dβ(t) +σ2

2x(t)dt,

meaning that in stochastic differential equations, dβ2(t) can be replaced inthe mean square sense by σ2dt.

1.8 Ito Lemma

In this monograph, Ito lemma is a key lemma which is widely used in thevarious chapters to evaluate differentials of nonlinear scalar valued functionsϕ(x(t)) of solutions x(t) of Ito type stochastic differential equations.

Consider a scalar process x(t) which satisfies

dx

dt= f(x(t), t) + g(x(t), t)β(t).

18 1 Introduction

Then, using Taylor expansion, we have

dϕ = ϕtdt+ ϕxdx+1

2ϕxxdx

2 +1

3ϕxxxdx

3 + ....

Discarding terms of the order o(dt), recalling that dβ2(t) is of the order of dt,and substituting for dx in the above Taylor expansion, it is found that [75]:

dϕ = ϕtdt+ ϕxdx+1

2ϕxxg

2dβ2(t).

Substituting σ2dt for dβ2(t) we obtain

dϕ = ϕtdt+ ϕxdx+σ2

2ϕxxg

2dt.

For vector valued x(t), where Qdt = E{dβdβT }, the latter result reads:

dϕ = ϕtdt+ ϕxdx+1

2Tr{gQgTϕxx}dt

where ϕxx is the Hessian of ϕ with respect to x.

1.9 Nomenclature

(·)T matrix transposition.Rn the n dimensional Euclidean space.Rn×m the set of all n×m real matrices.P >0, (P ≥0) the matrix P ∈ Rn×n is symmetric

and positive definite (respectively, semi-definite).

||.||22 the standard l2-norm:||d||22 = (ΣN−1k=0 d

Tk dk).

l2[0 N − 1] the space of square summable functions over [0 N − 1].||fk||2R the product fTk Rfk.‖f‖ the Euclidean norm of f.Ev{·} the expectation operator with respect to v.

[Qk]+ the causal part of a sequence {Qi, i = 1, 2, ..., N}.[Qk]− the anti causal part of a sequence {Qi, i = 1, 2, ..., N}.T r{·} the trace of a matrix.δij the Kronecker delta function.δ(t) the Dirac delta function.N the set of natural numbers.Ω the sample space.F σ−algebra of subsets of Ω called events.P the probability measure on F .P r(·) probability of (·).l2(Ω,Rn) the space of square-summable Rn− valued functions.

1.10 Abbreviations 19

on the probability space (Ω,F ,P).(Fk)k∈N an increasing family of σ−algebras Fk ⊂ F .l2([0, N ];Rn) the space of nonanticipative stochastic processes.

{fk}={fk}k∈[0,N ] in Rn with respect to (Fk)k∈[0,N) satisfying

||fk||2l2 = E{∑N

0 ||fk||2} =∑N

0 E{||fk||2} <∞, fk ∈ l2([0,∞);Rn).

l2([0,∞);Rn) the above space for N → ∞L2([0, T );Rk) the space of non anticipative stochastic processes.

f(·) = (f(t))t∈[0,T ] in Rk with respect to (Ft)t∈[0,T ) satisfying

||f(·)||2L2

= E{∫ T

0 ||f(t)||2dt} =∫ T

0 E{||f(t)||2}dt <∞.[P R∗ Q

]=

[P ∗RT Q

],for symmetric P and Q, is the matrix

[P RRT Q

].

diag{A, B} the block diagonal matrix

[A 00 B

].

col{a, b} the matrix (vector)

[ab

].

1.10 Abbreviations

BRL Bonded Real LemmaBLS Bilinear SystemBLSS Bilinear Stochastic SystemBRL Bounded Real LemmaDBLS Deterministic Bilinear SystemDRE Difference Riccati EquationGBM Geometrical Brownian MotionLF Lyapunov FunctionLKF Lyapunov Krasovskii FunctionalLMI Linear Matrix InequalityLPD Lyapunov Parameter DependentLTI Linear Time InvariantLTV Linear Time VariantOF Output FeedbackP2P Peak to PeakSDN State Dependent NoiseSF State FeedbackSNR Signal to Noise RatioSOF Static Output-Feedback