Fast suboptimal state-space self-tuner forlinear stochastic multivariable systems

Prof. L.S. Shieh, Ph.D., C.T. Wang. M.Sc. and Y.T. Tsay, M.Sc.

Indexing terms:methods

Multivariable control systems, Optimal control, Algorithms, Stochastic control, State-space

Abstract: A fast state-space self-tuner is developed for suboptimal control of linear stochastic multivariablesystems. The suboptimal self-tuner is determined by utilising both the standard recursive-extended-least-squaresparameter estimation algorithm and the recently developed matrix sign algorithm, which gives a fast solution ofthe steady-state discrete Riccati equation. The developed suboptimal state-space self-tuner can be applied to aclass of stable/unstable and minimum/non-minimum phase linear stochastic multivariable systems, in which thepair (A, C) is block observable and the pair (A, B) is stablisable. Also, the pair (A, Q) with QQT = Q isdetectable where A, B and C are system, input and output matrices, respectively, and Q is a weighting matrix ina quadratic performance index.

1 Introduction

Many control strategies, based on the certainty-equivalence principle [1], have been developed for self-tuning control of linear SISO and/or MIMO stochasticsystems. Recent methodologies include minimum varianceconcepts [2, 3], minimisation of a finite-step cost function[4, 5], pole placement methods [6, 7] and state-spaceapproaches [8, 9]. Many of the above methods have beensuccessfully applied to industrial processes [1]. To improvethe quality control and energy conservation, the finite-stage receding-horizon law [4] and the A-controller ofClarke and Hastings-James [10] have been successfullyapplied to suboptimal control of non-minimum phasesystems.

Recently, several authors [11, 12, 13, 14] have employedoptimal control theory for determining the adaptivecontrol laws of deterministic and stochastic systems.Samson and Fuchs [11] proposed an adaptive regulatorobtained by iterating the Riccati equation once at eachtime step for deterministic adapative regulator problems,but not for stochastic adaptive regulator problems. Lam[12] and Salcudean and Belanger [13] proposed theiradaptive controllers, obtained by iterating the Riccatiequation once or a few times per time step for stochasticadaptive control problems. Moreover, El-Sherief andSinha [14] proposed a constant adaptive controllerobtained by solving the infinite-time (steady-state) Riccatiequation [15] via a backward iterative procedure for sto-chastic adaptive control problems.

The use of a one-step optimisation scheme [11, 12, 13]significantly reduces the computational load of solving thesteady-state Riccati equation. However, the obtainedadaptive controller or regulator is suboptimal in the sensethat the scheme does not minimise overall quadratic per-formance index [15]. This fact implies that the quality ofcontrol and energy conservation at each time step may notbe optimal and good transient behaviour is not alwaysassured. On the other hand, the use of a few step (nearsteady-state) [12, 13] and infinite-step (steady-state) [14]optimisation schemes often results in both optimal feed-back controllers and better transient behaviour at theexpense of computational load. However, the computa-tional load can be reduced by using advanced digital hard-

Paper 253OD, received 13th January 1983

The authors are with the Department of Electrical Engineering, University ofHouston, Central Campus, Houston, Texas 77004, USA

ware and/or by using the quick algorithm proposed in thispaper.

In this paper, we formulate a linear stochastic multi-variable system with unknown parameters and correlatedsystem disturbances into a state-space innovations form[16] and an associated ARM AX model [17] suitable forboth parameter identification and state estimation. Thestandard recursive extended-least-squares estimation algo-rithm [18] is used to identify the parameters. As a result,the Kalman gain matrix and states can be estimatedwithout utilising the solution of the discrete Riccati equa-tion in the standard state estimation algorithm. Then, asuboptimal control law is developed by a fast solution of adiscrete Riccati equation, via the recently developed matrixsign algorithm (Reference 19 and Appendix 8). Theobtained suboptimal control law is finally implemented atthe estimated states for the state feedback control of themultivariable stochastic systems. The suboptimal control-ler will approach the optimal controller when the identifiedparameters approach their true values.

2 Multivariable state-space self-tuners

Consider an m-input p-output stochastic system describedby the following discrete-time state-space equations:

X0(k = A0 X0(k) + Bo U(k) + co0(k) (la)

= Co*o(*) + «>o(*)where Ao e RnXn, Bo e RnXm, and Coe RpXn are system,input and output matrices, respectively. X0(k) e R"x\U(k) e Rmxl, and Y(k) e RpXl are state, input and outputvectors, respectively. a>0(k) e RnXl and vo{k) e Rpxl arezero-mean white vector processes with covariance matrix

Qo>0; Ro>0 (lc)

The superscript T in eqn. \c designates the transpose of amatrix or vector.

The system in eqn. 1 is called the block observablesystem if the rank of the following observability test matrix@(A0,C0) is n:

Q, Co) = [(Co A'o~ T , (Co A'o- 2 ) T , . . . ,

{C0A0)\ClY (2)

IEE PROCEEDINGS, Vol. 130, Pi. D, No. 4, JULY 1983 143

Note that the observability index of {%(A0, Co) is r (wherer = n/p, an integer). The constraints in eqn. 2 imply thatthe Kronecker indices of the system in eqn. 1 are all r'sand n = rm. When the system in eqn. 1 is block observable,it can be transformed into the following observable blockcompanion form [20]:

X(k + 1) = AX(k) + BU(k) + co(k)

Y(k) = CX(k) + v(k)

where

X(k) = To 1X0(k); co(k) = To lco0(k)

1Q = \_AQ (}Q y AQ C[Q , . . . , AQ (JQ ? C[Q\

A =

~At I, 0 p ••• 0p

-A2 0p Ip . . . 0p

- U r o p o p ••• o p

B = To Bo =

Br

X(k) =X2(k)

— C T — VI 0 0 1

(3e)

(3/)

5{T ~l) ~

= V

- 1 ) {3g)

(3h)

/ p and Op denote p x p identity and null matrices, respec-tively. The system in eqn. 3 can be represented by thestate-space innovations model [16] as follows:

X(k + 1) = AX(k) + BU(k) + K(k)e(k) (4a)

Y(k) = CX(k) + e(k) (4b)

where K(k) is the Kalman gain which can be computed bythe following algorithm [21]:

K(k) = [AP(k)CT + S-][CP(k)CT + RY1 (5a)

P(k + 1) = \_A - K(k)C-]P(k)[A - K(k)C]T

+ K(k)RKT(k) - SKT(k) - KT(k)ST + Q

= ElX(k + l)XT(k + 1)]

P(0) = E{lX(0) -

(5b)

(5c)

where X(k) is the optimal estimate of X(k) giveny(k - 1) = \_y(i), i = 0, 1, . . . , k - 1]; X(k) ^ X(k) - X(k) isthe state estimate error; e(k) = Y(k) - CX(k) = CX(k)+ v(k) is the zero-mean white process (called the innova-

tions process) with Re ^ E[e(k)eT(k)~] = CP(k)CT + R.

If the pair (A, C) is detectable _ and the pair(A - BSR~lC, BQo) with QoQl = Q - SR" 1ST is stabilis-able [21], then P(k)-+P (the stationary error covariancematrix) and K(k)-+ K (the stationary Kalman gain). More-over, the eigenvalues of A — KC are inside the unit circle.

Let z" 1 denote the backward shift operator and

(3a)

(3b)

(3c)

(3d) where

then the input-output relationship of the steady-state inno-vations representation of eqn. 4 can be written as

(6a)

(6b)

x lBz~lU(k) + Kz-ie(k)\ + e(k)

~l +B2z-i +

• A2z

- 2 i

- 2 _

+ Brz

.z~r

-Arz-'— r

(6c)

(6d)

(6e)

and

(6/)

The zeros of det [/)(z-1)] are inside the unit circle.Eqn. 6b is the multivariable ARMAX model. If the par-

ameter matrices Aif Bt and C, for i = 1, 2, . . . , r in eqn. 3are known, and the covariance matrix in eqn. 3g is avail-able, then the recursive algorithm in eqn. 5 can be appliedto determine the Kalman gain K(k). Thus, the state X(k)can be optimally estimated using eqn. 4a. When the par-ameter matrices A(, Bt and C, are unknown, and thecovariance matrix in eqn. 3g is not available, we can usethe innovations form of eqn. 4a in conjunction with theextended least-squares-parameter estimation algorithm[18] to determine the adaptive Kalman gain K(k) and theestimated state X(k), using the estimate parameter matricesAi(k), B((k) and D,(/c) of the respective true parametermatrices At, B, and Dt for i = 1, . . . , r. The estimatedKalman gain K{(k) becomes

i= (la)

and the estimated state vector in the block observableinnovations form is

X(k + 1) = A(k)X(k) + B(k)U(k) + K(k)e(k)

e(k) = Y(k) - CX(k)

(7b)

(7c)

where A(k), B(k) and K(k) contain the estimated parametermatrices A((k), B^k) and K((k) for i = 1, . . . , r. When theestimated parameter matrices converge to the true par-ameter matrices, the X(k) and e(k) converge to the optimalestimate state X(k) and innovations process e(k), respec-tively.

To estimate the parameter matrices At, B( and D(, thestandard recursive extended-least-squares parameter esti-mation algorithm [18] can be applied to the model equa-tion of the following form:

Y(k) = s(k) (8a)

or

144

yi(k) = 6j(k - l)(f>(k) + 8t(k), i=\,2,...,p (Sb)

where O^k) e @{2n+rm)xl is the ith column vector of the

IEE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983

parameter matrix defined by

0(k - 1) = [&(* - 1), 62{k - 1), • • •,

= [A^k - I), ..., A,(k - I),

- 1)]

- 1), ...,

(8c)

- yp{k- r),Ul{k -

8p(k-l),...,ep(k-r)Y

y,(/c) G 0t is the ith component of the output vector Y(k)and £,(/c) G is the ith component of the residual vectore(k) obtained from eqn. 8a.

The identified parameter matrices A^k), B^k) and D,{k),i = 1, . . . , r, are then substituted into ecms. 7a and 76 todetermine the estimated Kalman gain K(k) and the esti-mated state X(k). Applying the certainty equivalence prin-ciple, the suboptimal state-feedback controller derived inthe following Section can be implemented at the estimatedstate X(k) for the self-tuning control of linear stochasticmulti-input multi-output (MIMO) systems.

3 Fast suboptimal state-space self tuners

In order to improve the quality of control and energyconservation of a linear stochastic MIMO system, anoptimal controller is often desired and developed by mini-misation of the infinite-time quadratic cost function [15].However, for the real-time optimal control of the multi-variable systems, the computational load of solving theinfinite-time (steady-state) Riccati equation to determinethe optimal controller may become severe. A suboptimalcontroller obtained by using the approximate solution ofthe steady-state Riccati equation is more practical than theoptimal controller. To determine the suboptimal control-ler, a fast convergent iterative procedure via the recentlydeveloped matrix sign algorithm [19, 22] is proposed for aquick and approximate solution to the steady-state Riccatiequation.

The matrix sign algorithm, originated by Roberts [22]and improved by Hoskins et al. [23] and Balzer [24], is aNewton-Raphson type algorithm with quadratic con-vergence in the neighborhood of the solution. It is a robustand efficient algorithm [24] for solving the eigenvectorsrelated problems [25], including the spectral factorisationof a polynomial matrix [26] in systems theory [25]. Forsolving the Riccati-type problems, the comparisons withother methods [27, 23, 24] in computational time [27], inprogram storage and workspace [23], and in convergencerate [24] have been performed. The results tend to favourthe matrix sign algorithm [24]. Anderson [28] has com-mented that the matrix-sign-function method may be thequickest of all currently available methods of solving thesteady-state Riccati equation. For completeness, the matrixsign algorithm [19, 22] is reviewed and given inAppendix 8.

From eqn. 7, we observe that the estimated states areavailable due to the results of the innovations form and theestimated parameter matrices. Therefore, we can apply theseparation theorem and certainty-equivalence principle [1]to determine the optimal controller from the noise-freeversion of the estimated system in eqn. 4 as follows:

Let the noise-free version of the estimated system ineqn. 4 be

1EE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983

X(k + 1) = AX(k) + BU(k)

Y(k) = CX(k)

(9a)

(96)

where A and B contain the estimated parameter matricesAj(k) and B^k), respectively. The infinite-time performanceindex is given by

J = lim E j l £ [_XT(i + l)QX(i + 1) + UT(i)RU(mN-oo UV i = 0

(9c)where Q( = QQT) is a symmetric non-negative matrix andR is a symmetric positive definite matrix. The steady-statefeedback optimal control law [15] becomes

U(k) = -{R + BTP$)-lBTPAX(k) (10a)

where the non-negative definite constant matrix P is thesolution of the following algebraic nonlinear discreteRiccati equation at time step k [15]:

P = Q + ATPA - ATPB(R + BTPB)-lBTPA (106)

To ensure that the resultant feedback control system isasymptotically stable, the pair (A, Q) is detectable(sufficient condition) and the pair (A, B) is stablisable(necessary condition). In this paper, the matrix sign algo-rithm (References 19, 22 and Appendix 8) is proposed tosolve the steady-state Riccati gain P of eqn. 106. When theestimated parameter matrices A and B have not yet con-verged to the true values, the resulting optimal control lawof eqn. 10a becomes a suboptimal control law. The pro-cedure for determining the suboptimal control law isdescribed as follows:

Define a 2n x 2n Hamiltonian matrix [29]

\_QA~1 AT (lla)

The modal matrix of G at time step k and its inversion aredefined as

=22 21 22

The block diagonal decomposition of G, using the modalmatrix, becomes

M~lGM = block diag [A, A (lie)

where A(A r) is the Jordan block corresponding to theeigenvalues_ of G outside (inside) the unit circle. M, i eCXn and Mi} e C*";i,j = 1, 2 are the partitioned matri-ces of M and M"1, respectively. The non-negative definitesolution of eqn. 106 is given [15, 29] by

P = M2lM;ll (12)

The direct computations of the eigenvectors of G at eachtime step for the solution P in eqn. 12 are not suitable forthe online self-tuning control. To overcome the computa-tional difficulties, we use the matrix sign algorithm [19] tocompute the matrix sign function [19, 22] of G. The matrixsign function preserves the eigenstructure of G at time stepk so that the matrix P at time step k and its approximationcan be easily determined. The matrix sign function [19, 22]of G is defined as

sign (G) = M • block diag [sign (A),

sign(A"T)] • M"1

= M • block diag [/„, - / „ ] • M~l

1 (13a)

145

/ £ block diag [/„, - / „ ] (136)

where sign (A) = /„ and sign (A~T) = —In.Defining a new matrix W at time step k, and performing

some algebraic manipulations [15, 22, 29] on the matrix,yields

W * sign (G) + r= M[TM'1 + M""1/]

= M • block diag [ 2 M n , - 2 M 2 2 ]

12

Thus, the constant matrix P at time step /c of eqn. 12 canbe determined from eqn. 14a as

P = W2l W~lxl = (2M21 A?11)(2M11 A?!,)"1

= M21M1"11 (14b)

The matrix sign algorithm [19] for computing the matrixsign function of G is

sign (G) = lim (& - I2n)(Gj + l2nY

l

J-*oo

= lim(G^ + / 2 n ) (G J - / 2 n ) - 1

^ Gj for a finite j at time step k

(15a)

(15b)

(15c)

The choice of the finite number j depends on the accuracyof the approximate matrix sign function desired. Ingeneral, the matrix-sign-algorithm iterations can be ter-minated, or the j value can be determined, when

| trace [ ( G / ] - 2n | /2n < e, (15d)

where e, is a desired error tolerance £see Appendix 8). Forthe real-time self-tuning control G,, the approximatematrix sign function of G at time step k in eqn. 15c can beused to determine the approximate Riccati gain matrix P(defined as P, at time step k) as follows:

Substituting G, of eqn. 15c into eqn. 14a yields anapproximate W (defined as Wj at time step k) as

V ) . (Wi2)~)

v ) (w ) {l6a)V2l)j \yy22)jJ

where T= block diag [/„, — /„].Then, substituting the block elements, (Wll)j€CnXn

and (W2l)j e C"x" of eqn. 16a, into eqn. 146 results in theapproximate Riccati gain matrix Pj as

(166)

Thus the suboptimal control law at time step k can beobtained from eqn. 10a as

U(k)= -FX(k) (16c)

where

F = (R + BTPj B) ~i BTPj A (16d)

Note that the feedback gain F obtained by using the finitej in eqn. 15c is different from that obtained by iterating theRiccati equation j steps per time step. Thus, the sub-optimal regulated stochastic system becomes

X(k + 1) = (A - BF)X(k) + Ke(k)

Y(k) = CX(k) + e(k)

(lla)

(lib)

For an m-input m-output tracking system, the suboptimalcontrol law can be written as

U(k) = Hr(k) - FX(k) (18)

where r(k) e RmXi is a reference input vector, and H eRmXm is an input gain matrix at time step k.

The closed-loop suboptimal controlled system using theU(k) of eqn. 18 becomes

X(k + 1) = (A - BF)X(k) + BHr(k) + Ke(k) (19a)

Y(k) = CX(k) + e(k) (19b)

For a constant reference input vector r(k), the mean of thesteady-state output can be adjusted to be equal to the r(k),or

lim E[Y(/c)] = r(k) (20a)fc->0O

The mean of the steady-state output can be determinedfrom eqn. 19 as follows:

lim E[Y(k)] = lim {C(zln - A + BF)~lBHr(k)

+ [/„ + C(zln -A

= C(In- A + BF)~lBHr(k)

(20b)

Solving eqn. 20b for H to meet the requirement of eqn. 20agives

H = [C(/n -A + BF)-l8yl (20c)

Thus the suboptimal control law of eqn. 19 can bedescribed as

U(k) = [C(/n - A + BFy'BT1^) - FX(k) (21)

4 Illustrative example

Consider an unstable and non-minimum phase systemdescribed by the ARM AX model:

(I2 + A i Z ~ l +A2z'2)Y(k) = (BlZ~l + B2z~2)U(k)

D,z~l + D2z-2)e(k) (22a)

where

R -

).4l

i.oj-1 .2 - 0 . 4 1 I 0.4 -0 .2

-0 .6 — l o j 2~

0.4 l.O

0.2

0.8 - l .oj

0.11 _T-0.48 - 0 . 2 ]0.2 J D2~l 0.2 -0.24J

and e(k) is a white-noise vector sequence with zero meanand covariance:

Z]Z (22b)

The equivalent state-space innovations representation ofthe system becomes

146

1 _ ' \e(k) (23a)

,, 02]X(/c) + e(k) (236)

1EE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983

10 Or

12.5 -

- 5 . 0

Fig. 1 Reference input rx{k) and output Yx{k)

40 50 60 70t i me step x 10

80 90 100

15. 0 r

1 2 .5

1 0 . 0

7 . 5

5.0

2 . 5

- 2 . 5 .

- 5 . 00 10 20 30 40

Fig. 2 Reference input r2(k) and output Y2(k)

IEE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983

50 60

t i m e s t e p , x10

70 80 90 100

147

- 0 . 2

- 1 . 610 20 30 40 50 60 70

t i m e s t e p . x 1080 90 100

Fig. 3 Estimated parameter matrix /4,(/c)real parameter matrix A,

1 . 4

1 . 1

- 0 . 7

- 1 . 00 10 20 30 40 50 60

time s t e p , x 1 0

80 90 100

Fig. 4 Estimated parameter matrix A2(k)real parameter matrix A,

148 IEE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983

1. 50

1 . 2 5 -

1 .00

0. 75

ID

am

ei

oa.

0.

0 .

5 0

25

- 0 . 2 5

- 0 . 5 010 20 30 40 50 60 70

t i me step , x1 0

80 90 100

Fig. 5 Estimated parameter matrix B{(k)

real parameter matrix B,

- 0 . 2 .

- 0 . S - -

- 1 4 .

- 2 . 010 20 30 40 50 60

t i m e s t e p x 1070 80 90 100

Fig. 6 Estimated parameter matrix B2(k)

real parameter matrix B,

lEE PROCEEDINGS, Vol. 130, Pi. D, No. 4, JULY 1983 149

0.6

0 .4 -

- 0 . 6

- 0 .8

-1 .OL-20 30 40 50 60

t i m e ste p , x 1 0

70 80 90 100

Fig. 7 Estimated parameter matrix D^

real parameter matrix Di

-0.810 20 30 40 50 60

t i me ste p , x 1 070 60 90 100

Fig. 8 Estimated parameter matrix D2(k)real parameter matrix D2

150 IEE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983

1.10

0.95

0.60

iT 0.65-

0 . 5 0 .

0 . 3 5

0 . 20

0 .05

- 0 . 1 0

Fig. 9 Estimated feedback gain F,(/c)real feedback gain

0.5

10 20 30 40 50 60 70 80 90 100

- 0 . 2 ,

- 0 . 30 10 20 30

Fig. 10 Estimated feedback gain F2(k)real feedback gain

50 60 70

t i m e s t e p , x 10

80 90 100

IEE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983 151

The performance criterion to be minimised is assumed tobe

1000J = E

(23c)

In addition, the relationship of the steady-state output anda periodic square-wave reference input vector r(k) ought tobe

Ely(ky] - r(k) = 02xl (23d)

where Q = / 4 and R = I2.The steady-state optimal gain matrix, using the exact

parameters in eqn. 22a, can be computed as

F =).852).135

0.1450.364

0.3620.187

0.325"0.140

(24)

The proposed matrix sign algorithm of eqn. 15c withy = 2,4, 6 has been used to compute the fast suboptimal self-tuners. The error tolerances e, in eqn. 15d for j = 2, 4, and6 at k = 1000 are obtained as £2 = 0.08, £4 = 0.02, ande6 = 0.01, respectively, j = 4 is chosen for the simulation.The choice of j = 4 is based on the observations that £4 =2% and | e4 — e21 ^ | e . — e61 - I* might be interesting tonote that the corresponding error tolerances Sj for j = 2, 4,and 6, obtained by using the true parameters in eqn. 22,are s2 = 0.085, e4 = 0.02 and E6 = 0.01, respectively. Thesimulation results with j = 4 are shown in Figs. 1-10. Thesuboptimal controlled outputs and reference inputs areshown in Figs. 1 and 2. The convergences of the systemparameters and steady-state feedback gains are shown inFigs. 3-10. After 1000 iterations or the time step k = 1000,the estimated parameter matrices are

-1 .16 -

r~" 1

[0.98|_0.42

0.20

-0.38]-1.00J

l0.461.06 ..-[

-0 .010.110.11

0.38-0.50

-0.590.80

-0.470.24

-0.220.80

2.04-0.99

-0.20-0.21

1 n - I " " 0 - 4 7 -0-20]j 2 ~ [ °-24 -0.21J

/? —r =

From

0.837[0.131Figs. 1

0.1400.364

and 2,

0.3570.178

0.3300.144

we observe

(25a)

The estimated suboptimal gain matrix at the time stepk = 1000 is

(25b)

that the closed-loopoutput Y(k) closely follows the reference input r(k). Also,Figs. 3-10 show that the estimated parameter matrices andfeedback gains converge to the true values in a nicesmooth way. The comparison between the estimated sub-optimal gain F at k = 1000 in eqn. 256 with the optimalgain F in eqn. 24 shows that the estimation error( | | F - F | | 7 | | F | | 2 ) is less than 0.1%. Therefore, from theabove facts, we conclude that the suboptimal controlledresults are quite satisfactory.

5 Conclusions

A fast suboptimal state-space self-tuner has been devel-oped for a class of linear stochastic multivariable systems,which can be described by an equivalent Kalman-filterrepresentation in a block companion form. The standardrecursive extended-least-squares parameter estimation

algorithm and the fast matrix sign algorithm have beenused to develop the fast suboptimal state-space self-tuner.The simulation results show that the proposed fast self-tuner gives a satisfactory suboptimal control of anunstable and non-minimum-phase stochastic multivariablesystem in both transient and steady state.

Comparing the proposed method with the pole-assignment methods [7, 30] for the self-tuning control ofmultivariable stochastic systems, we conclude:

(a) All three methods deal with the MIMO stochasticsystem in the block observable form or the equivalentARMA or ARMAX model with a monic or regular charac-teristic >l-matrix A(z~x) in eqn. 6c.

(b) The pole-assignment methods [7, 30] involve eithera resultant matrix [7] or a similarity transformationmatrix [30] for determining the self-tuning controllers.Notwithstanding the computational load above, theobtained self-tuning controllers may not be optimal. Inother words, the self-tuning controlled system has pre-scribed closed-loop poles; however, the state-feedback self-tuners may not be optimal [31],

(c) The proposed method in this paper gives optimal orsuboptimal self-tuning controllers, and the obtained con-trollers can be immediately implemented at the sequen-tially estimated states for online adaptive control ofmultivariable stochastic systems.

6 Acknowledgments

The authors wish to express their gratitude for the valu-able remarks and suggestions made by the referees and byDr. J. Chandra, Director of Mathematics Division, USArmy Research Office.

This work was supported in part by the US ArmyResearch Office, under research grant DAAG-29-80-K-0077, and the US Army Missile Research and Develop-ment Command, under contract DAAHO1-83-C-A084.

7 References

1 HARRIS, C.J., and BILLINGS, S.A.: 'Self-tuning and adaptivecontrol: Theory and Applications' (Peter Peregrinus, 1981)

2 ASTROM, K.J, and WITTENMARK, B.: 'On self-tuning regulators',Automatica, 1973, 9, pp. 185-199

3 BORISON, U.: 'Self-tuning regulators for a class of multivariablesystems', ibid., 1979, 15, pp. 209-215

4 CLARKE, D.W., and GAWTHROP, P.J.: 'Self-tuning controller',Proc. IEE, 1975, 122, (9), pp. 929-934

5 KOIVO, H.H.: 'A Multivariable self-tuning controller', Automatica,1980, 16, pp.351-366

6 WELLSTEAD, P.E., and ZANKER, P.: 'Pole assignment self-tuningregulator', Proc. IEE, 1979, 126, (8), pp. 781-787

7 PRAGER, D.L., and WELLSTEAD, P.E.: 'Multivariable pole-assignment Self-Tuning regulators', IEE Proc. D, Control Theory andAppi, 1981,128,(1), pp. 9-18

8 TSAY, Y.T., and SHIEH, L.S.: 'State-space approach for self-tuningfeedback control with Pole Assignment', ibid., 1981, 128, (3), pp.93-101

9 WARWICK, K.: 'Self-Tuning regulators-a state-space approach', Int.J. Control, 1981, 33, pp. 839-858

10 CLARKE, D.W., and HASTINGS-JAMES, R.: 'Design of digitalcontrollers for randomly disturbed systems', Proc. IEE, 1971, 118, pp.1503-1506

11 SAMSON, C, and FUCHS, J.J.: 'Discrete adaptive regulation of not-necessarily minimum-phase systems', IEE Proc. D, Control Theory &Appl. 1981,128, pp. 102-108

12 LAM, K.P.: 'Implicit and Explicit Self-tuning controllers'. Ph.D.thesis, University of Oxford, 1980, Report No. 1334

13 SALCUDEAN, S., and BELANGER, P.R.: 'An indirect adaptiveregulator computational algorithms and simulation results'. SixthIFAC Symposium on identification and systems parameter estima-tion, Washington, D.C., 1982, pp. 198-203

14 EL-SHERIEF, H., and SINHA, N.K.: 'Suboptimal control of linearstochastic multivariable systems with unknown parameters', Auto-matica, 1982, 18, pp. 101-105

152 IEE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983

15 KUO, B.C.: 'Digital control systems' (Holt, Rinehart and Winston,New York, 1980)

16 KAILATH, T.: 'An innovations approach to least-squares estimation.Part I: Linear Filtering in Additive White Noise', IEEE Trans., 1968,AC-13, pp. 646-655

17 ASTROM, K.J.: 'Introduction to stochastic control theory'(Academic Press, New York, 1970)

18 LJUNG, L.: 'Convergence analysis of parametric identificationmethods', IEEE Trans., 1978, AC-23, pp. 770-783

19 SHIEH, L.S., TSAY, Y.T., and YATES, R.E.: 'Some properties ofmatrix sign functions derived from continued fractions', IEE Proc. D,Control Theory and Appl., 1983, 130, (3), pp. 111-118

20 SHIEH, L.S., and TSAY, Y.T.: 'Transformations of a Class of Multi-variable control systems to block companion forms', IEEE Trans.1982, AC-27, pp. 199-203

21 ANDERSON, B.D.O., and MOORE, J.B.: 'Optimal filtering'(Prentice-Hall, New Jersey, 1979)

22 ROBERTS, J.D.: 'Linear model reduction and solution of the alge-braic Riccati equation by use of the sign function', Int. J. Control,1980, 32, pp. 677-687

23 HOSKINS, W.D., MEEK, D.S., and WALTON, D.J.: 'The numericalsolution of X = A, X + SA2 + D, X(0) = C , IEEE Trans., 1977,AC-22, pp. 881-882

24 BALZER, L.A.: 'Accelerated convergence of the matrix sign functionmethod of solving Lyapunov Riccati and other matrix equations', Int.J. Control, 1982, 32, pp. 1057-1078

25 DENMAN, E.D., and BEAVERS, A.N.: 'The matrix sign functionand computations in systems', Appl. Math. & Comput., 1976, 2, pp.63-94

26 KUCERA, V.: 'New results in state estimation and regulation', Auto-matica, 1981, 17, pp. 745-748

27 BEAVERS, A.N., and DENMAN, E.D.: 'A new solution method forthe Lyapunov matrix equation', SIAM J. Appl. Math., 1975, 29, pp.416-421

28 ANDERSON, B.D.O.: 'Second-order convergent algorithms for thesteady-state Riccati equation', Int. J. Control, 1978, 28, pp. 295-306

29 POTTER, J.E.: 'Matrix quadratic solutions', SIAM J. Appl. Math.,1966, 14, pp. 496-501

30 SHIEH, L.S., WANG, C.T., and TSAY, Y.T.: 'Multivariable state-feedback self-tuning controllers'. Sixth IFAC Symposium: Identifica-tion and System Parameter Estimation, Washington, DC, 1982, pp.1243-1248

31 WEI, Y.J., and SHIEH, L.S.: 'Synthesis of optimal block controllersfor multivariable control systems and its inverse optimal-controlProblem', Proc. IEE, 1979, 126, (5), pp. 449-456

8 Appendix

The scalar sign function [22] can be defined over thedomain Re (A) ^ 0 by

sign (A) =+1 if A e C+ (i.e. Re (A) > 0) (26a)- 1 if XGC~ (i.e. Re (A) < 0) (26b)

The matrix sign function [22] of a matrix A e C"x" withspectrum (A,) c C+ u C~ is defined by

sign (/I) = 2 sign+(/ l)- / , ,

where

(27a)

(21b)

and S+ is a simple closed contour in the right-half planeand enclosed all the eigenvalues with Re (A,) > 0.

The matrix A is equivalent via a similarity transform-ation to its Jordan normal form. Thus

(28a)A = MJM~l

where

J = block diag [J+ , J _ ] = J + © J _

and J+ e C 1 * " 1 and J_ e C 2 * " 2 are the collections ofJordan blocks with spectrum a(J+) a C+ and with spec-trum o(J_)<^C~, respectively. MeC"x" is a modalmatrix of A. The matrix sign function of a matrix can bedefined in terms of the Jordan canonical form of a matrix.

Following the definition of eqns. 27, it is easy to show thatthe matrix sign function of A as

S ^ sign (A) = M[sign (J+)©sign (J_)]M

(29)

A recursive scheme for computing the matrix sign function,given by Roberts [22] and improved by Balzer [24], isdescribed as follows:

k + 1 — afc Pk S0 = (30)

where <xk + (lk = I and

l i m <xfc = l im 0k =k-* oo k — ao

if spectrum (A,) c= C+ KJ C .

The algorithm is known as a Newton-Raphson type withquadratic convergence in the neighborhood of the solution[22, 24] and used as a standard algorithm for computingthe matrix sign function [25].

The scalar sign function defined in eqns. 26 can be re-written in an alternative form [19] as

sign (A) =if Re (?) > 0

if Re (A) < 0

where

(31a)

(31b)

(31c)A if Re (A) > 0-A if Re(A)>0

Note that Re (A) = 0 is not included in the definition.When A is a real value, say a, then g(A) in eqns. 31 isobviously an absolute value function, or

(32)

(33)

with proper selection of branch cut to match the definitionof eqns. 32 and 33. In Reference 19, it has been shown thatthe continued fraction of g(k) is given by

g(a) = Ja2, a e R

when A is a complex value, g(X) becomes

A) = J A ~ \ X e C

0W = = 1 +A 2 - l

2 +k2 - 1

A e C1 (34)

2 +

where €' = C — I and / is the entire imaginary axis.Also, in Reference 19, it has been shown that g}{A), the

yth truncation of the continued fraction expansion of g(X),can be written as

gtf) = * + + (i - Ayfor j=\, 2, . . . . (35)

Substituting eqn. 35 into eqns. 31 gives an alternate ofscalar sign function as

where

sign (A) = ^ = lim ^ = lim sign/A) (36a

(i + iy + (i - ky(36b)

The scalar sign function in eqns. 36 can be extended to thematrix sign function as follows.

IEE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983 153

Theorem 1Given a matrix A e C"x" with its spectrum o(A) e C. Thematrix sign function of A can be described by a matrixcontinued fraction as

sign (A) = A\ln + (A2 - In){2In + (A2 - /„)

- / „ ) { • (37)

Corollary 1The matrix sign function of A defined in eqn. 37 can beapproximated by

sign/X) = [(/„ + AY + (/„ - AY]

x Wn + AY - (/„ - AY'] -l (38)

Note that sxgnJ^A) for j = 2 in eqn. 38 is the standardmatrix sign algorithm [22] in eqn. 30 with ak = pk = \.The recursive algorithm for the matrix sign function ineqn. 38 is stated as follows.

Theorem 2The recursive algorithm for computing the matrix sign

function of A with spectrum o(A) e C is

signn;+l(y4) = signy.[signn.(y4)]; signx(/4) = A (39a)

where

ni + l= fr Hi for i = 1, 2, ...,fi> 1 and nt ^ 1 (39b)

Remark JThe standard recursive matrix sign algorithm in eqn. 30 isin fact a special case of the matrix sign algorithm derivedherein by choosing/) = 2 and ak = fik = j .

Remark 2For discrete-time system problems, the imaginary axishaving Re (X) = 0 in the continuous-time domain can be

mapped onto the unit circle with z e C and \z\ = 1 in thediscrete-time domain by the following bilinear transform-ation:

z - 1

z+ 1(40)

The relationship between the continuous-time matrix Aand the transformed discrete-time matrix G becomes

A = [G - 7J[G + / J (41)

Thus, the corresponding matrix sign algorithm in thediscrete-time domain [19] can be written as

sign/G) = if | o(G) \ (42)

and the approximate matrix sign function (defined as G}

for a finite;) is

sign (G) = lim sign/G) = lim (Gj + IJGj - /„)"

~ Gj for a finite j and if | o(G) \ ^ 1 (43)

It has been shown [25] that the continuous-time matrixsign algorithm iterations in eqns. 39 can be terminatedwhen

| trace {[sign (/I)]2} — n \ < s (44)

where e is a desired error tolerance. In a similar manner,the discrete-time matrix sign algorithm iterations ineqn. 43 can be terminated, or the) value in eqn. 43 can bedetermined, when

|trace [(G,-)2] - n\/n < 8j

where Ej is a desired error tolerance.

(45)

Remark 3For a large j value, the truncation errors due to directcomputations of Gj in eqn. 43 may occur. To reduce thetruncation errors, the recursive algorithm shown ineqns. 39 with A = (G - In)(G + / J " 1 can be applied todetermine G7 in eqn. 43 where Gj = signj(A).

154 1EE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983