Realization of the Markov parameter sequences using thesingular value decomposition of the Hankel matrixCitation for published version (APA):Hajdasinski, A. K., & Damen, A. A. H. (1979). Realization of the Markov parameter sequences using the singularvalue decomposition of the Hankel matrix. (EUT report. E, Fac. of Electrical Engineering; Vol. 79-E-095).Technische Hogeschool Eindhoven.

Document status and date:Published: 01/01/1979

Document Version:Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)

Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can beimportant differences between the submitted version and the official published version of record. Peopleinterested in the research are advised to contact the author for the final version of the publication, or visit theDOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publication

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:www.tue.nl/taverne

Take down policyIf you believe that this document breaches copyright please contact us at:openaccess@tue.nlproviding details and we will investigate your claim.

Download date: 13. Jun. 2021

https://research.tue.nl/en/publications/realization-of-the-markov-parameter-sequences-using-the-singular-value-decomposition-of-the-hankel-matrix(4d0c695f-1b3c-4669-883b-2d7e043937fc).html

Realization of the Markov parameter sequences

using the singular value decomposition of the

Hankel matrix

by

A. K, Hajdasinski and A. A. H. Damen

j

E I N D H 0 V E NUN I V E R SIT Y 0 F T E C H N 0 LOG Y

Department of Electrical Engineering

Eindhoven The Netherlands

REALIZATION OF THE MARKOV PARAMETER

SEQUENCES USING THE SINGULAR VALUE

DECOMPOSITION OF THE HANKEL MATRIX

by

A.K. Hajdasinski

and

A.A.H. Damen

TH-Report 79-E-95

ISBN 90-6144-095-5

Eindhoven

May 1979

CONTENTS

Abstract

1. Introduction

1.1 Remarks about the desired type of identification

1.2 The degree of complexity of the model:

the order definition of a MIMO-system

1.3 Gauss-Markov estimation of Markov parameters

2. Description and properties of the Ho-Kalman Algorithm

3. Description and properties of the Singular Value Decomposition

3.1 Existence of the S.V.D.

3.2 The least squares fit on a matrix

3.3 Some properties of the S.V.D.

4. Derivation of the realization algorithm using the S.V.D.

4.1 The noise free case

4.2 Estimation of the realization for the noisy case

4.2.1. Estimation of the system order fi 0

4.2.2. Estimation of the Hankel matrix of rank fi 0

4.2.3. Estimation of the shifted Hankel matrix

5. Results of the simulation-examples

6. Conel us ions

Appendix: Some remarks about the limitations of S.V.D. realization

References

2

2

3

12

17

20

20

21

22

23

23

25

25

21)

27

30

43

46

49

-1--

REALI ZATION OF THE MAHKOV PAHAMETEH SEQUENCES USING THE SINGULAH VALUE DECOMPOSITION OF THE HANKEL MATRIX

Abstract

Identification of multi input/multi output systems is the topic of this study. A

crucial problem is the degree of complexity of the model used to estimate

tile original system. In mathematical terms tllis is defined by the order or

the dimension of the system and these important notions for multivariable

systems are redefined. Once these notions have been strictly defined and

Markov parameters have heen estimated [rom input/output sequences, it is

shown, that a clear estimate of the system dimension can be obtained by means

of a singular value decomposition of the Hankel matrix. which is built up

by all estimated Markov parameters. With the results of that same singular

value decomposition it is possible to derive a realization, which turns out

to improve the Ho-Kalman algorithm Ln the case of additive independent noise

on the output signals of the system. Especially in low order, long Markov

parameters sequences the presented algorithm seemS to be preferable-It offers

us the possibility to incorporate all estimated Markov parameters into the

estimation of a realization of the system to be identified. In the noisefree

case hoth tllgorithms are equivalent.

Addresses of the Authors:

A. K. Hajdasinski, Central Mining and Designing Office, Plae Grunwaldski 10/8, KATOWICE, Poland

A. A. H. Damen, Group Measurement and Control, Dcpartn.ent of Eledrical En~ineerin~, E indhuvcn L1ni vcrsity uf TcchnuloKY, P.O. Box Gl:l, [,(iOO Mil EINDIIOVEN, The Ndht,,'lands

-2-

1. INTRODUCTION

1.1. Remarks about the desired type of identification

In this study we are solely interested in the identification of multi

input / multi output (MIMO) relations in the mathematical sense. No

physical interpretation whatsoever will be pursued and the characte-

ristics of the models will dominantly be determined by our limited,

mathematical ahility, which defines characteristics as linearity, inde-

pendence o[ disturhances, (inite dimensionality etc. Nevertheless the

first aim is application to real processes, which puts strong restric-

tions on possible models and methods. This last

-)-

seem to be able now to supply a prac:tiCi"l]]Y relevant algori l:llm, wC'

merely confined ourselves to practical tests by means of model-to-

model adjustments.

The above adstructed degree of complexity corresponds to mathemati-

cal terms as dimension and order. These notions, which are rather

complex in MIMO-systems will be defined and elucidated in the next

paragraph.

2. Once the degree of complexity has been determined or the mathematical

cquivalent,the order estimation has been perforrr:.erl a set of parameters

may he defined, which uniquely describes the model of that special

order. This set of parame_ters may be identified then in the sense, that

a residual (some error between process- and modeloutput) is minimized.

However, various sets of parameters, are possible and one set may be

transformed into another set.

The set of minimal number of parameters, given a certain order of the

model, can be denoted as fundamental, because all other sets necessari-

ly show interdependence of the different parameters.

The interrelations between the different parameter sets will be commen-

ted upon in the next paragraph, especially the "induced sequence" M: ,

denoted as Markov parameters, that can be supposed to be the multidi-

mensional impulse response on the one hand and the "realization" A, B

and C, which is generally known as respectively the system-. input-

and outputmatrix, on the other hand.

1.2. The degree of complexity of the model: the order definition of MIHO-

systems

The structure of the models under discussion ~s defined by the following

adjectives:

- linear

- multivariable (MIMO multi input / multi output)

- time invariant

- finite dynamical

- eli scretc

-4-

Onc,e this structure has been given, the complexity has to be limited.

Thi" is known as the model order determination or as the system order

es timation problem. While for S1S0 systems the notion of the system

ord,er is very well defined and extensively worked out, for M1MO-systems

the term "order" causes a number of misunderstandings and ambiguities.

lIowever the order definition of the MIMO-system is even more important

tl!:ll1 [or S[SO-systems. 1\ reasonahle reduction of the state space dimen-

sion, closely related with the order, is extremely important for the

sake of the modelling and computational simplicity.

In the sequel of this report there will be made an attempt towards un~

fication and the precize definition of the multivariable dynamical system

ord,er. Very useful then will occur to be the so called H-model of the

lin,ear multi variable dynamical system, but also some other definitions

must be remembered here in order to make clear all possible equivocal

pas.sages. There will also be printed out the equivalency of different

typ,es of models used for identification of the multi variable system.

See~{ing for thie equivalence was a necessary condition for the develop-

ment of the realization theory i.e. methods of finding the state space

description given a transfer matrix or input/output data.

Definition I: For the mllltivariahlc, linear. dynamical system having p

inputs u 1 (k) '" up (k) and q outputs y I (k) ••. y q (k) there

is defined the qxp matrix K(Z) called the transfer matrix

(being considered the rational matrix of the argument z)

fulfilling the following condition

1. (z)

where y I (z)

1. (z) =

y (z) q

K(z)u(z) :

.':: (z)

u (z) p

and y(z) ,u(z) are the "z" transforms of y(k) and u(k)

under zero initial conditions. 14 I Ii 111121

-'1-

iJefinition 2: The characleristic polynomi31 W(?) of the strictly proper

or proper tran.fer matrix K(Z) is defined 35 the least

Conunon iJenominCltor of 311 minors in K(z). having hy the

greatc.t power of Z the coefficient equal to one. I I 11 1121

Definition 3: The degree "IK(z)1 o[ the strictly proper or proper transfer

matrix K(z) is defLned as the degree of its characteristic

polynomial. (Practically it is the smallest number of shifting

elements necessary to model the dynamics of this system).1111

Definition 4: For the multivariable, linear, time invariant, dynamical sys-

tems, the state of the system at an arbitrary time instant

K = ko 1S defined as a minimal set of such numbers YI(ko)'

'J2 (k ) ... x (I< ) the knowledge of which together with the o n 0

knowledge of the system model and inputs for k ). k "

x(k ) = - 0

x (k ) n 0

is sufficient for determination

of the sys tern behaviour for k ~ k o

is called the state vector, and

members xI (ko )" ·yu(ko ) are called state variables.p IIh21

Definition 5: The set of difference equations

~:k+l) = ~(k) + B ~(k)

where x(k) is a (nxl) state vector

~(k) is a (pxl) input vector

is called the state equation, while the set

y(k) = C x(k) - -"here Z(k) 1S a (qxl) output vector

is called the output equation.1 4 11la II~

--------------------------------

-h-

Definition 6: The number n of state variables Ln the state equation 1.8

defined as the dimension of the state vector or the state

space and also denoted as the dimension of the complete

system.

Definition 7: The triplet of matrices I A, E, C I is defined as the reali-zation of the dynamical, linear, time invariant, multi va-

riable system.

Definition 8: Any polynomial f(:I.)

Lermna I:

f (z) k k-I 2

~ z + Clz + ... + Ck_2z + Ck_lz + Ck [or which holds

f (A) ~ Ak C Ak- I 2 Ck_1A + C A

O ~ ri + + ... + Ck_2A + I k I

1S called the annihilating polynomial of the A matrix.

The characteristic polynomial of the A matrix - WA(z) is

one of the annihilating polynomials for A (according to

Cayley-Hamilton).

Defl_nition 9: The polynomial f(z) of the smallest, nonequal zero degree k,

fulfilling definition 8 1S called the minimal polynomial

of the A matrix. II IJ 1121 [9]

Definition 10: The matrix coefficient '\ ~ C Ak B for k ~ 0, 1,2, ..... is referred to as the k-th Markov parameter of the system

defined by the realization lA, B, CI .19 III~

(i.e. the inverse liZ" transform of the transfer matrix)

Definition II: The following description of the multivariable dynamical

system is referred to as the H-model of this system.

1711811121

u (i) o for i < 0

where

-7-

r'J ~"(O)] y " i (I) ; 1I ':- ~(I) ; /; - properly dimensioned 1. (2) u(2) block vector containing the initial conditions

[MO MI ~-I " " "J Hk~ MI M2 ~ ... Generalized Hankel Matrix M2 M3 ~+I

1 Generalized Toer1itz Matrix

and M - for k = 0, I, 2 ••• are the Markov parameters of k

the considered system.

Now it is necessary to present two theorems being fundamental for fur-

ther considerations.

Theorem I:

Remark:

The sequence of Markov parameters {~} for k = 0, 1, 2, 3 ...

has a finite dimensional realization {A, B, C) if and only if there are an integer r and constants a. such that:

M . r+J

r

= "' /. i=1

a. M . . for all J > 0 L r+]-l

~

where r ~s the degree of the minimal polynomial of the state

matrix A (assuming we consider only minimal realizations).

Theorem 1 is called the realizability criterion and the r

is called the realizability index.

The proof of this theorem ~s to be found in 1 7 1 19 1 II 21

Theorem '2:

Remark:

-8-

If the Narkov parallleters sequence {Mk} [or k = 0, 1, 2 ••.

has a finite dimensional realization {A, B, C), \rlith reali-

zability index r,

state space (also

fulfils

then the rr.inimal dimension n of the o

of the realization) for this realization

where

and

H = r

rank III I = n r 0

n - minima] state-space dimension o

I Mil - qxp n r x mln (p,q)

0

M MI M r-I 0

MI M2 M r

LS the Hankel matrix (see definition II).

(fhe proof of this theorem is given in 17119 1112)

From the linear dependence of Markov parameters follows lhat

that rank II N = rank II r+ r

n for all N~O. o

Hith the aid of these II definitions and two theorems it is possible now

to generalize the meaning of the system "order" for different types of

multivariable system descriptions. Before it will be made formally, it

can be of some use to present a simple scheme sharing relations between

the three defined types of models: the transfer matrix, the state equa-

tions and the H-mode 1. (See fig. I)

From this scheme we learn that while from the state space description

there is a straiglltforward way to get the transfer matrix K(z), the re-

verse procedure viz. realization theory is a lot mnTC' complicated. On the

contrary knowing the. Markov narameters it is equally easy to get any re-

quired form of description. For the sake of modelling, Markov parameters

can be derived as easily from the state space description as from the

tranl-lfer matrix. Obviously Markov parameters are also used in the H-rnodel.

-')-

other

rC;lli~ations

.. lin-Kalman state space

II, B, e K(z) realization description

+ Markov

parameters

• t II-model

fig. 1. Interdependence of different type models.

considering all pro's and contra's it seems desirable to express .11."1

structural invariants of multivariahle dynamical systems in terms or Markov parameters.

Definition 12: The order of the multivariable system will be defined

as the minimal number of Markov parameters necessary

and sufficient to reconstruct the entire realizable

sequence of Markov parameters according to the theorem l.

In other words the multivariable system order is equal

to the realizability index r.

Alternatively fur the state space description, the multivariahle system

order C;ln he defined ;ts tllC degree of tile minimal polynomial of the

stilte matrix 1\. This follows Jirectly from the proof of theorem I. For

the transfer matrix descri ption llOwcver, the order defini tion in the

general Cilse is not possible.

-10-

Defin:ction 13: The dimension of the multivariable dynamical system is

defined as the number n being equal to the rank of the o

H -Hankel matrix for this system, where r is the reali-r

zability index (order of the system).

Alternatively for the state space description it is the dimension of the

state matrix A. And again for the transfer matrix description there does

not exist a unique definition of the system dimension. Only in the case

when

L ... _

-11-

Theorclll I mui? lngl'tlier wiLh dl·finitiolls I? and '"3 ;lrc ('ssl'Iltinl for lhe

estimation of the system order. As it was pointed out, the complexity de-

finitiort for the multivariable dynamic system can be most flexible and

most general while using the Markov parameters description. The task of

the complexi ty identification will thus he to determine the order rand

tile (minimal) dimension n of the system being considered. ~~~~~~~~~~~o~~~~~~~~~~~~~~

Such .1. posing of the problem is possible only for some strictly conceptual

systems having both finite order and finite dimension. In such a case

looking for exact rand n makes sense. However in real systems identifi-o cation problem one cannot search for any exact rand n because usually

o those are systems of infinite order and dimension. The only goal .!hich we

can aim at is to find a reasonably simple approximation of the real system

(i.e. estimates of the rand n ). o

As an illustration can be given the following example:

Example I:

K(d +3

-(z-2)

(z+ I) 2

2 (z+4) 2

(2 + I)

The Markov parameters are:

M {)

rank

112

f 112

jll I f

0

3

=f III 2

-I 4

2 4

4 -2 -7

4 -3 -10

defCree I K (z) f (there are no

in K(z».

= 2 " n o distinct poles

-12-

So thl~ dimen:-;ion of titP system J S

n = 3 0 One of possible realizations is:

+ 2

-J [: ;} c=[~ A -2 1 ; B = -2 The characteristic polynomial of 1\ ~s:

3 (z+ I)

Ilut the minimal polynomial of /\ is:

"1

:J 0

2 11/\ (z) = (z+l) as can easily be checked, so the order of the system

is, i11deed H = H • 2 r

1.3. Gauss-Markov estimation of Markov parameters

This .mbjeet has b('en broadly treated en I R I , and here will be presented

only main features o[ the method. Starting with the definition 12 of the

II-model it is straightforward to derive the following equation:

T T '1' Y = Hk Sm + M S 00 ( 1 )

where

'! T -"- [ Y (I) Y (I + 1 ) ••• y (I +m) ] (2)

and Z(l) I-th measurement of the output vector

Z (1 +m) I +m-th measurement o[ the output vector

1\ " [M(O) M( I) ••. M(k)] - block vector cont.qining first k+1

Markov parameters of the multi- (3)

variable system.

-13-

T N " [H(k+l) ~f(k+:~) ... ~I(I;:-t·llI) ... ~I(I+III-k)]- hlne],; Vt't'lnl" Ill" I il1il4'

2::(1-1) 2:: (1-1 +m)

S " 2::(1-2) 2:: (1-2+m) m=

u(l-I-k) .. 2:: (l-I-k+m)

2:: (1-k-2) .. ~(1-2-k+m)

S A u(1-k-3) .. !!(1-3-k+m) 00

2:: (0)

0

o 0 .•. 0 2::(0)

lec~tll cnntaininJ~ r('-(4 )

maining Markov para-

meters of the considered

system.

finite dimensional matrix

of the input samples

finite dimensional matrix

of the input initial con-

ditions

(5 )

Relation (I) represents the real system equation, while the model equation

IS assumed to be:

(6)

l' -1' N stands for the estimate of the ~ and Y for the estimate of the

Considering that the multivariable dynamical system is corrupted by the

rnultivariable noise E, relation (I) may he written as:

Y (7)

where E is defined as:

(8)

The equation (7) refers to the model shown in fig. 2.

-I ',-

noise

fi lter [-

H

y

fig. 2. The block diagram of the multivariable dynamical system described

in terms of Markov parameters.

The multivariable no~se E is considered to be an output of a colouring

filter [or the white multivariable noise H

Assuming that {.':!.(i)} and{~(j)} are for all i and j mutually uncorrelated and that expected value of the {!:(i)}.' \!;.(i)} = Q.iln estimate of the first k+l Markov parameters is found minimizing the following loss function:

v w

whe:re W is a nonnegative weighting matrix and

Solution of this problem results in the following expression for the

Markov parameters estimate:

N T -I

(S W S) S W Y m m m

Expressing N in terms of the E

N

(9)

(10)

(J I)

Conditions under which the N is an asymptotically unbiased estimate of

tlw ~ are discussed in 15 118 I

-\5-

UStln11y, due to decre~lsinfl nnttIre of tile {H} sequence [()r staille systems T

for k .1nd m great enough the term S M can be neglected and the part of 'I' -\ ,n tlte hi ns (B W S) S W E assymptotically vanishes when assUming m m '!11

I L~(j)} = 0 and there IS no correlation between samples of E and S . ill

Because those are the only cases to be handled,. the following expression

is ~lssurned to deliver the ;rsymptotical1y umbiased estimate N:

N

-I Choosing as the weighting matrix W _ R , where

it can be shown that such a choice leads to minimization of the

As the following inequal ity

II' 1 (Mk -N) (Mk -N) 'I'f w II :> II, 1 (~-N) (~-N) T f R II

-\ can be proved for all W I R •

At las t

( \2)

( 13)

(I 4)

(15)

and this estimation is equivalent with the Gauss-Markov procedure for the

single input/single output system 1411d. [3]

There remains only to find an appropriat~estimate of the R matrix. This task is completed with the aid of the composite noise notion and again

the realization theory. The estimate achieved on this way is called fur-

ther the Gauss-Markov Estimate with Realization of the Covariance Matrix

(named further C.M.R.E.).

_~Appropriate in the sense, that the profit of the use of R instead of the identity matri~ is not overcompensated in the negative sense by the devia-

tions in the estimate of R.

-)(,-

This estimate has hC(.'11 inlrotiuct'd in I H I and also lll(.'rt' are discussed

further properties of the C.M.R.E. estimation.

From this moment, for the sake of the continuation of the re~ort, it can be assumed that we are having the C.M.R.E. estimates of the identified

system Narkov parame ters ~

-17-

2. DESCRIPTION AND PROPERTIES OF TIlE HO -KALMAN ALCORITHM

The original version of the lIo-Kalman algorithm has been derived for

noise-free systems. Since the time it was first published r 9 I a number of .1.1gori thros has been proposed basing upon the Ho-Kalman ver-

sion and attempting to give solutions for cases where Markov parameters

were estimated from the operating records. However ever since has not

been proposed any, giving a satisfactory simple approximation of the

realization without a high complexity to attack this problem. So~e

facts about the Realization Algorithm of B.L. Ho and R.E. Kalmanl 9 11121 are reviewed.

Theorem 3: 191 114/

There 1S

El -=-k

For an arbitrary, finite dimensional, linear, dynamical sys-

tem given the input-output map the canonical realization

exists in tIle following form:

-p- number of inputs

-q- number of outputs

-n- dimension of the realization

defined the following matrix

k x 1 . [k matr1x ~ l-~ ~ if k < 1 [II ] k x 1 matrix o~ if k > 1 -'1

-18-

In 0 pr-n

-n P II = En E

pr --r !l -qr -n

n pr-n 0 0 -qr-n -qr-n

where H - is the !Iankel matrix for the considered system r

!I -1'

M !'!c l ••• !'!cr-I ] M2 ••• M - -r

• • • • • •• !'!c2r- 2

3. A canonical realization of the considered system is given by:

A = Eqr P (aH ) Q En -n - -r - -pr

B = Eqr P H EP -n - -r -pr

C Eqr H En -q -r -pr

where (YH 1S the shifted Hankel matrix -r

oil -r

MZ ••• M ] - -r • • • • • • •• M I -r+ • • • • • • •• M

-2r-

The .proof of this realization theorem can be found in reference [II.].

Also there can be found the following theorem which will be useful

late:r.

Theol:em 4: The Penrose-Moore pseudoinverse of the Hankel matrix H is ------ -r

r,iven the following:

where H+ stands [or the lJenrose-Moore pseudoinverse of the H -r -r

(without proof here).

-19-

Bee,luse for the n01 sy case the realizability criterion (theorem I and 3)

will never be fulfilled, (for the linear dependence is lost in case of

estimated Markov parameters) it is necessary to optimize the multi-

variable system model order r another way. Some solutions of this pro-

blem are suggested 1n 15 II 71. Also thc 1I0-Kalman algorithm does not fit to the no1SY case, because

using not all Markov parameters estimated (only the number which comes

out from the order test 2r-1 see ref. I 7 I), it truncates the informa-tion contained in all noisy data estimated.

.!J Supposing the estimate r of the system order r has been already estima-

ted, it is to be seen that! and Q matrices are to be evaluated basing "-

on the H~ matrix, which is already truncated to qr x pr dimension. Thus -r

if there are estimated L > 2r-1 Markov parameters necessary to produce '" ~ It? and CTHr , then information contained in remaining L-2r+ I Markov para-meters is lost.

*)

Ren~ember , tllat for the noise free case the order r and the dimension n o

have to fulfill the following conditions:

r 1 ) det {H } = 0 M r+j L a. M .. J ). 0 r

i= 1 1 r+J-1

det {H H T}= 0 if p .1< q r r

2) rank{1l } rank {Il } n for all N > o. r r+n 0

3) > n r 0

min (q, p)

In the noisy case

smallest integer,

minant will not lIe

n can he estimated from 2), while r will be the o

that satisfies 3) and 1) approximately, as the deter-

zero exactly due to the noise.

-"0-

J. DESCRIPTION AND PROPERTIES OF THE SINCULAR VALUE DECOMPOSITION

The Sjngular Value Decomposition (S.V.D.) will he introduced ill a very

compact form by means of few most important theorems and definitions.

Host vigorous and formal material dealing with this subject is to be

found in121, 16 1,1101.

3. I. Existence of the S.V.D.

Theore:n 5: For any m x n matrix A, the S. V.D. exists, given by:

where

U - 1.5 a m x II matrix consisting of f1 orthonormal colunms U. -J

so:

T U U If'

f' - 1S the rank of the matrix A

D - 1S the f' x f' diagonal matrix:

D diag ("I' "Z' "f')

(11 ~ (fZ';y. •• ~ rTf} > 0

v - ~s a n x I} matrix consisting ofporthonormal columns v. so: -J

The (T. are called singular values. J

The proof of the theorem 5 can be found 1n many references among

them in 1101.

-21-

3.~. The le.:lst sqU.:lre_s fit on :l. m~ltrix

For the sake of the realization algorithm it will be desirable to limit

the rank of a Hankel matrix. This also must be done with a minimal effect

on this matrix. Expressing this problem in categories of m x n matrices

(rectangular): if A is the original m x n matrix with the ranklAI =1',

the task will be to find the m x n matrix B with the rank{B I < I' in such

a way that the euklidean norm from A-B is minimal. The norm of this type

is given the definition:

Definition 16: The norm of a matrix A will be defined as:

Theorem·6:

In other words it is necessary to find a B with a limited

rank and minimized norm of difference between A and B. [2] [10]

1611101 Given the S.V.D. for a m x n matrix A:

A diag (aI' a2 ••• al')

IT J ? If 2 ~ (I 3 ~ ... ~ (" > 0

The m x n matrix B of a rank k.,;!' and such that IA-BI m1n, 1S given the following:

f ' 1 Dk I 0

vT vT B U ---:---- Uk Dk k o I 0

I

where Uk contains the first k columns of U V -

k contains the first k columns of V

Die = diag. (lf1 , ('2 '" a k)

Remark: So the B matrix 1S found by setting the smallest

r-k singular values equal to zero in the S.V.D.

-22-

It u; also possible to evaluate the error made during such a fit:[IOl

absolute error: IA-BI = lEI =

relative error:

,/~ ~

('

J. j=k+1

/' J.

j=1

Jf 2: ? fT~ "" = J 1,j j=k+1 J 2

IT. J

2 IT.

J

This theorem will be the basic tool for the system order determination and

approximation of the Hankel matrix.

3.3 Some properties of the S.V.D. (10(.

Property I: If the s.v.n. of A is ~lven by A A + of A wi 11 be (2 ( , (I () ( :

and the singular values of A+ are:

u n vT, th~ pseudo lnverse

-I -I (J IT

fJ ' P-I'

Property 2: From the S.V.D. and orthonormality of the columns of V it

follows that

T tr(A A)

/' ;::

j=1

2 fT.

J

•

-23-

4. DERIVATION OF THE REALIZATION ALGORITHM USING THE S.V.D.

it wiH be demonstrated that the S.V.Il. delivers in a noise frel'! case an

exact realization algorithm which is equivalent to the Ho-Kalman algorithm

but intuitively simpler and sav1ng some computational efforts. For the

noisy case the S.V.D. will deliver for a chosen r the best approximation

in the least squares sense of the realization.

4. I. The noise free case

The whole procedure will base upon the S.V.D. of the Hankel matrix ~.

Having exact Markov parameters it is always possible to find .k,~ r the

system order. For such a case referring to the Theorems 2, 3, 4, 5 and

Property I we have:

where [D J according

n = n o

= n x n o 0

to theorem 2 the rank will be

(16)

( 17)

(loS)

Comparison of (17) and (18) may lead to the following equivalence (infinite

possibilities are available however)

(19)

V (20)

With the aid of (19) and (20) the HO-Kalman algorithm equations can be re-

written the following way:

A D -I T

U o-H k V (21 )

-I U

T IT D VT P VI EP (22) B D Epk = pk

C EqkU D VT V Eqk U D (23) q q

where

-:~)-

P f\ Q ["~~\~!] (Uqk) T Un D (Vn ) T [vn vPk- n] ; qk qk n pk pk pk fJ I qk-n

~-" J n I a EPk EPk ---;--- E" D t,n qk n n ·qk n o ,I k , q -n

This last completes the devivation of the realization algorithm which

c.an be used for further consideration.

Actually this shows especially for the n01se free case that P and Q

contain too many degrees of freedom, while U~k and v~k contain strictly

sufficient parameters necessary to construct the realization from the

Hankel matrix.

4.2. Estimation of the realization for the noisy case

In the noisy case there can be performed an easy test which singular

values are substantial and which can be neglected comparing their rate

of decrease.

4.2.1. Estimation of the system order n "

It is assumed that there are estimated 2k Markov parameters. From those

Markov parameters it is possible to construct the following Hankel and

shifted Hankel matrices:

Hk ; aHk

Performing the S.V.D. there also 1S found a vector of singular values

°13-°2;' 30 . (where s = k * min. (p, q). Comparison of singular values s gives a solution to the order test. Neglecting

values we determine no as the dimension of the

s-n singular values. o

smallest s-n singular o

realization. Consequently

we omit the smallest

A criterion deciding which singular values are sufficiently small is very

problem-dependent. But in all investigated cases (model to model) there

always existed very sudden changes in singular values for an increasing

position index. This situation is illustrated in fig. 3.

o

0' J

•

\

2

\

\

\

'--

3

-20-

~/A

-:'7

Because the Markov parameters have been found as an consistent L.S.

estimate (also efficient) of ideal ones, the S.V.D. appea~

one more filtration of the noisy data in the L.S. sense. The

approximation of the realization takes the following form:

A n- I 'l' (61'k ) V V n n n

B -I

VT

V VT EP VT

EP D D n n n n n pk n pk

C Eqk V D VT

V Eqk

V D q n n n n q n n

to be

(Z7)

(Z8)

(Z9)

still remains one more problem to solve, the estimate of the shifted

Hankel matrix aHk

. To solve this it is necessary to take a deeper look ~ ~

into the structure of Hk and aHk

•

4.Z.3. Estimation of the shifted Hankel matrix

~

Let us remember once again the structure of the Hk and crHk matrices:

M MI MZ ~-I a

MI MZ M3 ~ H =

k

~-I ~ ........ MZk- Z

MI MZ Y7Y M2 M3 / /Y /~+I

"II = M3 M4 / ~+I/ k / / / / / / /MZk- Z

/ / ,/ - - - - . - / - .-~t ~+I

/ MZk-ziMZk-1

From equations (30) and (31) it 1S seen that only one element in aHk

differs from elements of Hk

, and this is the last Markov parameter

estimated for the dynamical system MZk_ I '

(30)

(31 )

-28-

"" Huwever, after the least squares fit on the 1\ we have:

Ilk

1JJ. where 11, r

II 12 110 111

ZI 22 111 I1Z

11 32 I1Z 113

kl kZ 11 k-I 11k

13 112

Z3 po

J

33 1'4

k3 I1k+ I

Ik 11k - I

Zk 11k

,3k l'k+ I

kk 112k -k

J; i, J I • Z •••• k

{ 0, I ..• Zk-Z,

(12)

whic',l means that the block synunetry property in the 1\ matrix is lost. It s,~ems like Hk matrix was the Hankel matrix of a vari linear system.

Taking this under consideration there can be proposed two structures

of the aHk matrix:

1,

aHlk =

IZ 13 PI )lZ

22 Z3 I1Z 113

3Z 33 113 114

kZ k3 \lk \lk+1

Ik Zk Pk-I 11k

Zk/3k 11k I1k+1

3k/· I1k+1

/~k (33) I1Zk-Z

kk / - ----

I1Zk-Z : ,

The last column of the shifted matrix is constructed copy~ng elements

of the last column of Ilk ,which is due~to the assumption about the vari-

linear nature of the system described Hko Making such a choice we

possibly cormnit the smallest nonaccuracy in aHk estimation.

-29-

kk Al so it is proposed to take [or 1l

2k-

1 the c:orrcBponding real vaJ lie of

the estimate M2k

- l • As it has been experimentally checked, the

realization (27), (28) and (29) is not very sensitive to changes of

jl ~~_I' However, M2k_ 1 is taken from a different matrix space, which has the realization of the higher order then estimated r, as it

incorporates the noise.

2. 21 22 23 2k ]1 I ]12 113 Ilk

31 32 33 3k 112 113 114 Il k+ I

a H2k

kl k2 k3 kk Ilk-I Ilk Ilk+1

kZ/ k3/ • Ilk Ilk+1 Il

• IlZk-Z /' ---

kk : IlZk-Z: t

MZk- 1

The last row of the shifted matrix ~s constructed cory~ng elements of

the last row of Hk , which again is due to the assumption about a vari-

linear nature of system described Hk

. And again it is proposed to take kk

for PZk-1 the corresponding real value of the estimate MZk_l~

There are many other possibilities of the solution for the aRk matrix

estimation. However, having in mind also a simplicity of the algorithm,

which in case of multivariable and multidimensional systems can be even

more important than a slight sophistication of the formalism , one or

two above mentioned methods is proposed as a solution.

In numerical experiments there was chosen the realization algorithm

incorporating the allik shifted Hankel matrix.

As the complete algorithm for the noisy case is explained now and its

profits will be illustrated by means of a number of examples, some

remarks should be made of the drawbacks as well and more especially

about the heuristical property of the algorithm or the impossibility till

noW to prove the algorithm in some mathematical sense. Therefore some

remarks will be made in the appendix in order not to disturb the progress

of the explanation of the algorithm.

-30-

5. RESULTS OF THE SIMULATION-EXAMPLES

Example 2.

ConGider the two input, two output system given the following block

diagram - fig. 4.

1I1(z)

U (z) K (z) o

"I (z) 1: 2 (z) ,.._~_-L_~

+

J 1 (z)

J L (z)

Fig. 4. the transfer function model of the identified system.

The

the

K (Z) o

[

(Z - 0.8)

0.0

0.2 ]

(z _-_0_._8.:..) _(_Z_- O. 6 )

(z - 0.6)

*) fact that Ko(Z) = K~(Z) means that this system is equivalent to

system with an additive white input noise, if we omit initial

conditions.

The realization of the K (Z) 0

transfer function is given by:

A [o.s 0.2J [1.0 o.o];c= [1.0 0.0 ] B 0.0 0.6 0.0 1.0 0.0 1.0

and Markov parameters are:

[1.0 o·t = [0.8 0.2] . ~.64 0.28J M = MI M = 0 0.0 0.0 ' 0.0 0.6 ' 2 0.0 0.36 t· 512 0.296] ; M = [0.4096 0.28 J M = 4 . . . . 3 0.0 0.216 0.0 0.129

*) Tlle poles may be distinct or cornmon. As we neglect the initial

c·:mditions~ the poles can always be looked upon as cotmllon.

-'11-

Eigenvalues of the state m.:1trix /\ are:

Case a: ,n order to show how the modified lio-Kalman algorithm works

for the sake of the ideal systems modelling, the exact Markov

parameters were taken as an input to the algorithm.

The singular value decomposition of the H4

singular values:

°1 2.5701194

°2 1.4734138

°3 8.6947604 10- 12

~ 0

°4 1.5247013 10- 12 - 0

°5 I. 5134382 10-

12 - 0

H4 delivered following

It is quite clear that the system dimension n = 2. o

The estimate H4 used for the realization is exactly (i.e. within

accuracy of the computer) equal H4 , such that the system is exactly

realizable. Also oH4

can be evaluated exactly.

~0.834575 -0.068194) A =

-0.011896 0.565417

~o. 568066 -0.259609] B

-0;288067 0.772964

C = r 1 .504 1 70 -0.560574

-0.505194]

1.105450

This realization generates exactly ideal Markov parameters i.e. - -; - .

M. = CAB = C A'B. , Eirr,en valueI)' of the A matrix are:

0.79999 ...

0.59999 ..•

0.8

0.6

which shows practically exact ness of the proposed algorithm.

-32-

CaBe b: An additive coloured noise sequence is generated on the output

of the systC'm. The i npllt is gC'IlC'r:1 tC'd ;ts tIle wll i tC' nOJ s('

having a rectangular density function between (-I, 1)

while the noise filter input 1S generated

as the white gaussian no~se with the standard deviation 0.1.

The sequence of 10 Markov parameters was estimated in 6 runs

based on 100 samples in every run, and averaged over those 6

runs. The Markov parameters estimated this way with the G.M.R.E.

method are:

M=~I.OIO 0.019J M = [0.746 0.239"] ; [0.637 M = o -0.005 0.995 I 0.024 0.558 2 0.009

[0.481 0.294] M = [0.439 0.315 ] [0.362 M = M = 3 0.030 0.201 4 0.001 0.084 5 0.030

~O. 315 O. 188 ] M = [0.237 0.168 J M = [0. 102 M = 6 0.036 0.038 7 0.032 0.019 8 0.072

r· 027 0.083] M = 9 0.083 0.018 The S.V.D. of the H4 delivered following singular values.

°1

°2

°3

°4

2.0

1.0

2.591692

1.473426

0.231209

0.199619

2 o J

°5 0.146380

°b 0.088079

°7 0.039385

°8 0.021803

4 5 (, 7

Fi~. 5. singular values if the H4 matrix.

8

0.260 J 0.385

0.243J

0.304

0.14

J 0.034

-33-

Again it i:-; ohvious th.1t the .... dimension no :::: 2 will be the best

approximation. The estimate H4 used for the realization is not any

more e~uAl H4 , but the deviation is really very small.

As shown before the overall relative error will be: R

\I ii-l! \I { U(Dk-Dn) vT

V (DltDn) UT}~ [ 2 trace i~3 o· 1 .014 ~

Ill! II {u Dk VT V Dk uT 1 8 trace 2 [ i=1 o· 1

So the relative error 1n each element of H will be in the region of

~~ .12. This overall relative error of 12% is accomplished by an error of -4%

for the big numbers -I and by an error up to -100% for the small numbers

-.01. As the estimates of the Markov parameters have been given before,

the differences between H, Ii and H can be checked given the following found H:

0.975 0.024 I 0.766 0.222 I 0.638 0.272 0.520 0.268 -0. 00 2 0.990 I 0.025 0.600 1 0.021 0.408 0.039 0.276 I + . .. - I 0.766 0.244 0.607 0.300 0.505 0.298 0.417 0.282 0.014 0.599 0.027 0.366 1 0.023 0.251 0.032 0.17l

-I . , . l! ~ 4 0.637 0.268 0.507 0.298 I 0.423 0.281 0.350 0.257

0.003 0.409 0.01/, 0.248 0.011 0.170 0.018 O. liS I I . 0.517 0.293 1 0.413 0.288 0.344 0.260 I 0.287 0.230

-0.000 0.284 I 0.007 0.172 0.006 0.117 ,

0.011 0.080 ,

The matrix H4 has "almost" a block symmetric structure and assumptions

about a slight varilinearity of the system modelled H4 for the sake of

the aH4 reconstruction seems to be quite reasonable.

The second order approximation of the realization is found using the

simplest method for estimation of the aH4 i.e. filling M7 for the

lacking element of the oH4'

The approximation of

A [0.847094

-0.106906

B [

-0.539030

0.345924

the realization

0.053035·1 ;

0.576929

-0.299207 J -0.742041

takes form:

_ [-1.45438 C -

-0.677517

0.55351J

-1.06114

I

I

-34-

ThlS realization generates .1. very r,ood sequence of Markov pnr,amcters

es'~imates, being very close to original ones:

(fDr five first Markov parameters)

Ideal Markov Markov parameters Markov parameters

parameter generated via HO- generated via S.V.D.

Kalman realization realization - -

{M. } 1

{M. } 1

{M. } 1

1.0 0.0 1.0 0.00151 -- 0.97543 0.24434 M M M

0 0.0 0

-0.0041 1.0 0

-0.001871 0.99013 1.0

0.8 0.2 - 0.77918 0.20118 - 0.7797 0.20660 MI

0.0 0.6 MI

-0.005519 0.59000 MI

0.'124006 0.618721

0.64 0.28 0.6066 - 0.27492 - 0.62817 0.28213 H2 0.0 0.36 M2 -0.005671 0.34747 M2 0.03511 0.3915

0.512 0.296 0.47206 - 0.28393 .. 0.50902 0.29902 M3 0.0 0.216 M3 -0.0052270 0.20416 M3 0.003813 0.25172

0.4096 0.28 0.36716 0.26230 - 0.41430 0.28718 -M4 0.0 O. 1296 M4 -0.0045473 0.11957 M4 0.036940 0.164871

Th,~ minimal polynomial coefficients for Ho-Kalman and S. V. D. real izations

co:nparing wi th ideal ones are:

I

-35-

I I Minimal polynomial Minimal polynomial Minimal polynomial , 1 I coefficient coefficient coefficient I ideal system Ho-Kalman S.V.D. I realization realization I

- -a l -0.48 at -0.46 at -0.49 - -a 2 ' .4 a2 1.37 a2 1.42

Comparison of elgen values for state matrices of the Ho-Kalman reali.za-

tion and S.V.D. realization shows the following:

~ , 0.777 } HO-Kalman realization

A2 0.593

A, = 0.7906 } S.V.D. realization A2 0.6035

While ideal elgen valuES are A, = 0.8; A2 = 0.6.

It also occurs to be a very interesting experlence to observe the

norm of the following matrices:

II (Mk - ~ ) II 110 and

II (~ - ~ ) II S. V. D.

where T ~= (M 0 M, M2 ~)

T -~= (M M, MZ ~) 0 - T ~= (M 0 M, M2 ~)

-)6-

For k=2, which is the sufficient index to construct the He-Kalman

approximation of the realization there is:

II (~ ~) _ 0.0015

thus for k=2

II ~! - t\ II < II~ - ~II II S.V.D. I: 0 [or k=4

II ~ - ~II H 0.0216 0

II ~ - ~II S.V.D. = 0.02149 so

II ~ - ~IIH 0 II~ - ~II S. V.D. 0

but for k >4 for example k=IO

II ~\ - ~111l O. 1976 0

II ~\ - ~II = 0.03544 S.V.D. II ~ ~II > II~ - ~II S.V.D. II

0

Which induces the conclusion that while Ho-Kalman approximation of the

realization gives slightly better modelling of the syst€r.t in the

transient state, the S.V.D. approximation gives almost equally good

approximation in the transient and steady state, which means that it

gives a better overall fit.

This phenomenon is caused by the fact that S. V.D. incorporates more

available information containe d in noisy data than the H crKalman

algorithm.

Case c. The additive coloured noise 1S generated at the output of the

system. The input is generated as the white nOl.se having a

r~ctangular density function between (-I, I)

while the noise filter input is generated as the

white gal1Bsion noise with the standard deviation 0.5.

-37-

The Harkov parameters estimated under such conditiong with th£'

C.M.R.E. method arc:

(10 Markov parameters - 100 samples)

M = ~ I. II 0.2 ] M=(0.439 0.465J M = [ 0.t,39 a -0,112 I. 09_ I 0.0601 0.490 ~ -0.0247

M = l 0.101 0.22~1 M {0.294 0.32~; M = U· 377 3 0.110 0.189 4 0.0318 -0.102 5 0.073 M = [ 0.622 -0.022~] M =[0.677 0.124]. M = [0.342 6 0.0912 -0.0362 7 0.0423 -0.0287 ' 8 0.235

M = [ 0 .298 0.268 J 9 0.277 0.0038 The S.V.D. of the H5 delivered following singular values:

(JI

"2

= 2.0950862

= 1.3087709

~ 6. 3 1 L

2

°3

"4

2

= 6.8433805.10- 1 °5

= 5.5023061.10- 1 °6

°7

°8

3 4 5 6 7

Fig. 6. singular values of the 114 matrix.

= 5.0606823.10- 1

2.5954151.10- 1

1.326579 .10- 1

3.036047 .10-2

i 8

0.144J; 0.571

0.044j; -0.191

0.32~} 0.070

The result of the dimension test 1S less prOnOllnCJ.ng 1.n this case,

bllt still it LS possible to decide for n ~2. o

The estimate 114 used for the realization certainly suffers deviation

fn)m the block symmetry property, but also this deviation is rather

small considering the noise level.

Again the second order approximation of the realization is found using

the simplest method of the a1l4 estimation.

The approximation of the realization takes form:

A~ [ O. 741028 -0.0183221J - [-1. 12064 -1.01055 J c--0. I 10 I 77 0.33448 0.776194 -0.865162

- [-0.432248 -0.56872

J B~ 0.662446 -0.609601 Considering the noise level 50%, this realization is also a very good

api')roximation of the original one,havirig eigen values of the A:

, ~ 0.74593 ::1 A 2 =: 0.32957

Example 3

(0.8)

(0.6)

The following system, having a structure presented on the fig. 4. was

simulated under following conditions.

wh!:!:re

and

where

r·' 0 0 A ~ 0 0.4 'J B ~ 0 0 o. K (7)

0 C (17 - /1)-1 B

/I_~ rO• 1 0 ] LO 0.7

B ~ E,

K (I.) !;

C (II - /I ) - I B r, r; r,

l: -:] [~ 0 -~] c -I

-39-

Markov parameters COl" this system are:

H o

=CB=[-l,OOJ

1.0 1.0

[

-0.64 0.6-]

0.4 0.4

and 50 on.

CAB r-O. 8 L 0.4

0.61 0.4

[

-0.4096

0.0154

0.408J

0.0154

The intensi ty of the simul;lted noise "as 10% of the output signal

amplitude,

"i~envalues of the A matrix ),1=0.8, ),2=0.4, ),3=0.2.

Estimates of Markov parameters derived via C.M.R.E. method are:

to.994 -0.003] [-0.788 0.586]

H = MI 0

1.0 I .0 0.403 0.397

H2 [-0.611

0.162

0.s7

J 0.166 H) [ -0.469

0.069

0.47IJ

0.068

M4 [-0.347

0.028

0.376]

0.028 Ms

[ -0.246

0.015 O.27~J ; ....... 0.010

Singular values of the Hs matrix for such a set of Markov parameters

are:

°1 2.7706702

(J2 I. 7184590

l13 3.582625.10-1

"4 3.7090322.10-2

"5 8.1052389.10-3

-)

°6 3.819628.10 -3 "7 2.842466.10

°8 1.0787837. 10-3

-40-

0. L

1

.,

2 3 4 5 7 8 i

Fig. 7. singular values of the 114 matrix.

Tn this case the more pronouncing will be testing of the ra.tio:

o. 1

x

10

3 \ / \ \

r, \

\ 4 2

"-~ .•.. //

2 3 4 5 6 7 8 i

Fig. 8. ratios of sinr,ular values of the H4

which apparpntly stands for no 3.

-1,1-

After approximation of the 114 with the 114 calculated from the S.V.Il.

and applying the realization alf;orithm, the approx.imation of the

realization will be:

[ ,. '",''' A = 0.0852291

-0.547107

t ,.,m" B = -0.458556

0.510278

c = [-I . 12M, 7 0.459472

0.175944

0.429488

-0.421211

-,. ""'''] -0.775701. -0.566931

0.217971

-1.47049

0.""""] 0.0104386 0.205312

0.269546 J 0.0988497

ICigen values of the A matrix are:

0.7325

0.4075

0.2875

which again is a very good estimate of original ones.

Example 4

This case deals with the no~se free, two input/two output system

being of the first order and having the dimension I.

This example is "'eant to show that in such case also the S.V.D. aided

realization nlgorithm is capable to deliver correct results.

U 1 (z)

~L __ K_(_Z) ____ ~ ____ Y~I_(7_') ______ : -1 ... Y2(z) -Fig. Y. the block diagram of the considered system.

-42-

-1.0 1.0 (z 0.5) (z - 0.5)

K( ~)

2.0 -2.0

(z - D.5) (2 - 0.5)

The realization of this system can he:

A = 0.5 B=(I,-I)

The Harkov parameters are:

M o [

-I 11 2 -2

~1. I = 0.5 M. 1+ 1

c

[

-0.5 0.5J

I -I

The singular values of the H4 are:

"I I,. 1999

-II "2

1.4551915.10

"3 0

a4 a

M2 = [-0.25

0.5

and all a. 1

o for i = 3, 4, 5, 6, 7, 8, 9, 10.

Thi~: glves an exact answer for n = 1. o _

The realization computed from the 114 there is:

A 0.5

0.25·J -0.5

B ( -0.613572 0.613572 ) ( - 1.0 I .0 ) * 0.6 I 3572

Then

C

via

A

Il

C

[ 1.6298 J 1.6298 [-~J -3,2596 tIle similarity relation for T = 0.613572 we have

'1'-1 ~ T

'1'-1 R C T

1.6298+0.5.>'-0.613572 = 0.5

1.6298 ..-0.16572 ;, (-1.0 1.0) (-1.0 1.0)

I.6298H.I('357U[_~ J= [_~] which leads to the origi.nal reali.zalion.

(,. CONCLUS TONS

Along this report there has been made an attempt to unify and redefine

some notions important for the multivariable systems theory. Also

another version of the realization algorithm has been proposed. This

algorithm, called the Singular Value Decomposition Realization, shows

some important and jnteresting properties:

I. For the noise free case the dimension test is an intrinsic

part of the S.V.Il. realization.

2. For the no~sy case the S.V.Il. clgorithm delivers an excellent

and convincing dimension test, which can be implemented

automatically.

3. For the noise free case the S.V.D. realization ~s equivalent

to the Ho-Kalman realization (see 4.1.) and in the noisy case

the equivalency exists as well, if the minimal amount of

Markov parameters is used to constitute a Hankel matrix H. 4. For the noisy case the S.V.Il. algorithm lead" to an

approximate realization, which is hased upon all estimated

Markov parameters improving an overall fit of the model to

a g~ven system.

The whole procedure illustrating the identification of the multivariable

system in terms of the Markov parameters may be represented in the

following graph:

g

er 10th

! re.a I

lizations ~

i bas L.--

:Lng on Hk

-44-

~

- -- .. ~-----.

System _r ~ C.M. R.E.

method

{N.j .v

Ilk Order

Test

L ."-

S.V.D. -'\

Order test

4. ~ , Estimate llk

S.V.D.R. I-- uses (realizations)

~nly 1I~

~ r

[~i1

+ + .y Comparison

lIses only -J I!~ r Ho-Kalman

Realization

{Mil

, 1

, Conclusions!

I ,

- I

· I, ').

The conclltsions may he, that. the lIo-Kalman alr,orithm gives it better

fi t to till' first M~rkov p;lr;lllletcrs, while the s.v.n. alr,orithm provides a hetter overall fit especially for long Markov sequences.

This can be explained in the following 'Way:

In order to obtain a good, unbiased estimation of Markov sequences

from input/output signals, it is necessary to estimate all Markov

parameters above the noise level. If some time-constants of the

system are large compared to the sample time, quite a lot of Markov

parameters have to be taken into account.

The Ho-Kalman algorithm can only use a small number of Markov

parameters, determined by the order of the system, to constitute a

Hankel matrix and a consequent realization. In that way all information

contained in the remaining, estimated Markov parameters, which are not

used, is lost. Some errors in the first Markov parameters may lead to

great deviations, as they are not compensated by the information in

the latter estimated Markov parnmeters.

The algorithm, discussed in this report, uses all estimated Harkov

parameters, so it shows not the drawbacks of Ho-Kalman algorithm.

Nevertheless it is not ideal, as not all Markov parameters are weighted

equally in the least squares sense (see appendix). The Markov parameters

in the middle of the used sequence have much more influence on the

realization, than the first and latter Markov parameters. Nevertheless

it shows to be an excellent and comparatively simple tool to determine

the dimension of a system and producing a good estimate of the realiza-

tion of that system.

APPENDIX

Short remarks about the limitations of the Singular Value

Del~omposi tiOll Realization

Ci ven a lIankel matrix 113 as an example of the more general case:

• pk - ...... P .... 1 M MI 112 r M3 i a qk MI M2 M3 •

1 G M4 qk k ~ r

1 H2 ~13 Ill, Me 1 J t f t t ~I ~2 ~3 ~ '~

II

Lml,:icitly we were trying to obtain a solution for this equation In

the sense, that the 'variability' of R is minimal i.e.r dep,rees of

fre€~dom, where .0. ought to he:

T Il

(\ , I' I '2

o -p------ --pk -----

To t·.oat purpose we performed a Singular Value Decomposition on Hand

here we made (at least) four 'mistakes':

I. There -is also \lOI.se on~, so much more a A.S.V.D. should be performed

on H and ~ tor,ether like in the example of Golub and Reinsch. ::t)

M: the noj se on each M. is expected to be of equal level, we should ~

also equally wei!~ht ~.

(1 n the example of Colub and Reinsch k 1).

*) Go]uh and Hcinsch, "S.V.D. and L.S. solutions", Hand hook Series

Linear Algehra, Numer. Math. 14,1970, example 2J •. on page 408.

11 Even if W(' incorporate ~ and perform an S.V.D., we will

obtain a pragmatic solution, where not all (lM. have equally 1

III

been weighted. In the above example (lM2

has been weighted 3

times, compared with 6M1 and 6M3 2 times and 6Mo and (lM

4 only

once.

No necessary

H by H = V n n

restrictions were T

D V the general n n

il.= V 1)-1 VT V n n n ~ + s-n Y

s = m~n (pk. qk)

and Y is a (s-n) x p matrix.

made upon~. As we approximate

solution for ~ becomes:

In theory we are only able to use (s-n) 1< p degrees of freedom

to adjust il. to the restrictions i.e.:

a. Each pxq block contains equal

b. All off-diagonal elements are

s· 's on the main diagonal. 1

equal to zero.

c. The first s-r B. 's can be defined as zero, as the minimal 1

IV

-48-

T = U D V is not 'block-symmetric': n n n

The Hankel matrix H n

we cannot distinguish uniquely the respective Markov parameters.

This implies, that we cannot apply a shift operator, which is

used in the proof of the lIo-Kalman algorithm. Although not all

conditions for the IIn-K.oIman algorithm are fulfilled we use the

resulting formula to define the realization.

Ln practical applications it turns out to be profitable, but

there is sti 11 a lack of proof.

-49-

References

(I) Andree, R.V. C:OHI'UTATTON OF TilE 1 NVERSE OF A HATRTX.Ameri C;1n Hathcmatical Honthly, Vol. ')8(19",1), p. 87-92.

(2) Albert, A. RECRESS roN AND TilE HOOlm-l'lmROSE I'SEIIDO- I NVICHSE • New York: Ac'ldcmic: Press, 1C)72. ~tathem

-50-

(13) T;I11II0\l, .1.1.. "1'1' RIIX I flATE \l I: A II S S -fiA RKIIV I': ST I fIATIII{S AN II I{I': I ,A'I'I': II :: I :111': m::: • Eindhoven IIllivcrsi ty or 'I'(~clm()logy, 1971. -""I'~rttnent of Electric

E1NDIIOVEN UNIVU{SITY OF TH'IINOLOGY THE NETlIERLANIlS DEPARTMENT OF ELECTI{lCAL EN(;INFEI{IN(;

Reports:

I) Dijk, j" M. Jcuken and EJ. Ma'"Hlers AN ANTENNA FOR A SATH.LlTI' COMMUNICATION (;ROUNIl STATIUN (PROVISIONAL ELECTRICAL m'SI(;N). TII-Re'port (,X-E-Ol. 19C1X. ISBN '10-Cl I 44-00 1-7

2) Veefkiml, A., 1.11. Illom and L.H.Th. Rietjens TIII'OKETICAL AND I'XPERIMENTAL INVl'S"nl;ATION OF A NUN-EQUILIBRIUM PLASMA IN A MilD CIIANNEL.. Suhmilled 10 the SYIllPosium on Magnctohydrodynamic Ekclrical Power (;cneralion, Warsaw, Poland, 24-30 luly, 1'I6I~. TII-KqHlrt hX-E-02. 19hX. ISBN 90-61 44-0o:!-5

31 Boom, A,l.W. van den ,,,,

l'INIIiIOVI'N IINIVHlSITY 01' TH 'IINOIO(;Y 1m, NLTIIHllANIlS DEPARTME':NT OF ELH"I'RIl'AI. I'N(;INI'TIHN(;

ReporlS:

14)

15 )

16)

17)

18 )

19 )

21 )

23)

24)

25)

21»

27)

Lorelldll, M, AUTOMATic METEOR REFLECTIONS RECORDIN(; EQUIPMENT, TII-Rel'ort 70-10-14, 1970, ISBN 90-6144-014-9

SlIlcts, A,S, '1'111' INSTRUMENTAL VARIABLE METIIOD AND RELATED IDENTII'ICATION SCIIEMES, 'I'll-Report 70-1'-15,1970, ISBN 90-(,144-015-7

White, Jr" R,C. A SURVEY OF RANDOM METHODS FOR PARAMETER OPTIMIZATION, 'I'll-Report 70-1'-16,1971. ISBN 90-6144-016-5

Talmoll, J, I.. APPROXIMATED GAUSS-MARKOV ESTIMATORS AND RELATED SCHEMES, TH- Report 71-E-I7, 1971. ISBN 90-6144-017-3

v Kalasek, V, MEASUREMENT OF TIME CONSTANTS ON CASCADE D,C, ARC IN NITROGEN, HI-Report 71-1'-18,1971, ISBN 90-6144-018-1

lIossekt, L.M,L.F, OZONHILDUNG MrrTELS I;LEKTRISCIII;R ENTLADUNGEN, TII-RL'port 71-1'-19,1971. ISBN 90-1>144-019-X

Arts, M,G,J, ON '1'111; I NSTANTANI;OllS M I;ASU REM ENT OF BLOODFLOW BY UL TRASON IC MEANS, TII-Rc'pllrt 71-1'-20,1971, ISBN 90-6144-020-3

Roer, 11'h,G, van de NON-ISO TIIERMAL ANALYSIS OF CARRIER WAVES IN A SEMICONDUCTOR, 'III-Report 71-1'-21. 1971. ISBN 90-1>144-021-1

Jell!.cn, P,J" C. Hllhcr and CE, Mlliders SENSIN(; INERTIAL ROTATION WITII TUNING FORKS, 'I'll-Report 71-1'-22,1971, ISBN 90-6 I 44-022-X

Dijk, J., J,M. Berends and E.J. M,mnders APERTURE BLOCKAGE IN DUAL REFLECTOR ANTENNA SYSTEMS - A REVIEW, TH-Report 71-1'-23,1971, ISBN 90-6144-023-8

KrcgtiOlg, J. al1144-024-1>

Damen, A,A,II. and H,A,L. I'iceni Till' MliLTIPLE DIPOLE MODEL OF TilL; VI':NTRICULAR DEPOLARISATION, 'I'll-Report 71-1'-25, 1971, ISBN 90-1> 144-025-4

Bremmer, II, A MATHEMATICAL TIIU)I{Y CONNECTIN(; SCATTERIN(; AND DIFFRACTION PIIENOMENA, INCLUDINC BRAGG-TYPE INTERFERENCES, TII-Rcport 71-1'-26,1971. ISBN 90-6144-026-2

Bokho"en, W,M.G. van METIIODS AND ASPECTS OF ACTIVE R(,-FILTI'RS SYNTHESIS. 'I'll-Report 71-1;-27, 1970. ISBN 90-6144-027-0

Boes(>holcll. F. TWO I'LlIl()S MODEL RI;I'XAMINI'I) FOR A COLLISiON LESS PLASMA IN THE STATIONARY STATF. TII-Rl'J,ort 72-E-28, 1972, ISBN 90-h 144-028-9

ElNDIIOVI'N UNIVI-:!{SITY OF T1TIINOLO(;Y THE NETHeRLANDS DEPARTMENT OF ELHTI{ICAL ENWNEERINC

Reports:

~l) RI-PORT ON 1'111,: CI.OSI:D CYCLI: MilD SPECIALIST MH:TlN(;. Working group of the joint l'!I;t,A/IAEA Inll'rnalional MilD Liaison Group. EindhovL'Il, The Netherlands, September 20-22, 1971. Euited by L.H.Th. Rietjells. Til-Report 72-1'-2'1.1'172. ISBN 90-6144-029-7

30) Kessel, C.C.M. vall and J.W.M.A. Houben LOSS MECIIANISMS IN AN MilD GENERATOR. 'I'll-Report 72-1'-30. I (n~. ISBN 90-6144-030-0

3 I) Veefkind, A. CONDUCTION GRIDS TO STABI L1ZE MilD GENERATOR PLASMAS AGAINST IONIZATION INSTABILl'rll'S. 'I'll Report 72-E-31. 1972. ISBN '10-6144-031-9

32) Daalder, J.E., and C.W.M. Vos DISTRIBUTION FUNCTIONS OF TilE SPOT DIAMETER FOR SINGLE- AND MULTI-CATIIO])[: DISCIIAR(;]':S IN VACUUM. '1'11- Reporl 73-1'-32. I (J73. ISBN 90-6144-032-7

33) Daalder, J.E. JOULI: III,ATIN(; ANI) DIAMETI·:R OF TIll: ('ATIIODE SPOT IN A VACUUM ARC'. TII-Reporl 73-1:-33. I (!73. ISBN 90-6 I 44-03J-5

341 Huber, C. BEHAVIOUR OF Till: SPINNIN(; GYRO ROTOR. TH-Report 73-1'-34.1973. ISBN 90-6144-034-3

35) Bastiall, C'. et al. TilE VACUUM ARC AS A FACILITY FOR I{I':U,VANT I:XPI':RIMI'NTS IN FUSION RESI'ARCII. Annual Reporl 1972. EURATOM-T.lI.E. Croup 'Rotating Plasma'. I'll-Report 73+:-35.1973. ISBN 90-6144-035-1

31> I Blom, J. A. ANALYSIS OF PIIYSIOLO(;ICAL SYSTEMS BY PARAMETI'.R I':STIMATION TH'IINH)lII'.S. '1'11- Report 73-1'-3(>. 1973. ISBN '10-6 I 44-036-X

37 I Ca ncelleu

3111 Andriessen, F.J., W. Boerman and I.F.E.M. Holtz CALCULATION OF RADIATION LOSSES IN ('YUNDER SYMMETRIC HIGH PRESSURE DISCHARGES BY MEANS OF A DIGITAL COMPUTER. Til-Report 73-E-31l. 1973. ISBN 90-6 I 44-031l-6

39) Dijk, J., C.T.W. vall Diepellileck, E.J. Maandcrs amclI, A.A.H. A (,OMI'ARATIVI' ANALYSIS OF SEVI'RAL MODELS OF TilE VENTRICULAR DEPOLARIZATION; INTRODUCTION Or: A STRING-MODFL. 'I'll-Report 73-1'-41. 1973. ISBN 90-6144-041-6

ElNDIIOVEN UNIVERSITY OF T1'l'IINOLOGY TilE NETIlERLANJ)S J)EpAlnMENT OF ELElTRI('AL I':N(;INEEI~IN(;

Reports:

421 Oijk, 'G, H.M. van THEORY or (;YRO WITII ROTATIN(; (;IMBAL AND I''LLXURAL PIVOTS. '1'11- R"port 73-1'.-42. 1973. ISBN 9()-h 144-042-4

43 I Breimer, A.J, ON TilE I DENTI FICATION OF CONTINOUS LI NLA R PROCESSES. 1'11- Report 74-10-43. 1974. ISBN 90-6144-043-2

441 Lier, M,e. van 'Illd R.H.J.M, Otten CAD OF MASKS AND WIRIN(;. 'I'II-Repurt 74-E-44. 1974. ISBN 90-6144-044-0

4S1 Bastian. C et al. FXpl'RIMI'NTS WITII A LAI{(;E SIZED 1I0LLOW CATIIODE DISCHARDE FED WITII AR(;ON. Annual Report 197.1. EURATOM-T.II.E. Group 'Rotating Plasma'. Til-Report 74-1'-45. 1974. ISlIN 90-6144-045-9

4(, I Roer. TII.G. van de ANALYTICAL SMALL-SI(;NAL TIIEORY OF BARITT DIODES. TII-Rt·pmt74-E-46.1974.ISBN90-6144-046-7

471 Leliveld. W.H. TilL DESI(;N OF A MOCK CIRCULATION SYSTEM. TII-R"port 74-E-47. 1'174. ISBN 90-hI 44-047-S

4X I Damen. A.A.H, SOMI' NOTES ON '1'111' INVERSE PROBLEM IN ELECTRO CARDIOGRAPIIY. TII- RC'port 74-E-4X. 1974. ISBN 90-h I 44-04X-3

491 Meehcrg. L van de A VITERBI DECODER. Tlt-Rvport 74-1'-49. 1'174. ISBN 90-(,144-049-1

501 Poel, A,p.M. van der A COMPUTI'R SEAI{(,II FOR (;OOD CONVOLUTIONAL CODES. Til-Report 74-E-50. 1974. ISBN 90-6144-050-5

5 I I Sam"i.c, G. THE BIT ERROR PROBABILITY AS A FUNCTION PATH REGISTER LENGTH IN TilE VITEIWI DECODER. TII-Repml 74-E-51. 1974. ISBN 90-6144-051-3

Sci Scllalkwijk. J.P.M. CODIN(; FOR A COMPUTI'R NETWORK. TII- Repurt 74-1'-52. 1'174. ISBN 90-6144-052-1

53 I Stappc-r, M. MEASUREMENT OF TilE INTENSITY OF PROGRESSIVE ULTRASONIC WAVES BY MEAI\S OF RAMAN-NATII DIFRACTION. 'I'll-Report 74-10-53. 1'174. ISBN 90-6 I 44-053-X

54) Scllall;wijk, J.P,M. and A,J. Vinck SYN DROM F DECODI N(; OF CONVOLUTIONA L CODES. 'I'll-Report 74-E-54. 1974. ISBN ')0-6 I 44-054-X

55) Y"kimov. A, FLUCTUATIONS IN IMPATT-DIODE OSCILLATORS WITH LOW Q-FACTORS. 'I'll-Report 74-1'.-55. 1'174. ISBN 90-6 I 44-055-h

HNIlIIOVt:N lINIVI':I~SITY (W TlTIlNOIO(;Y THE NETIII'I{IANIlS DEPARTMI'NI' OF I'LHTRI('AL t:N(;INH':RIN(;

Reporls:

56) Plaals, J. van del' ANALYSIS OF TIIRU': CONDUCTOR COAXIAL SYSTEMS. COlllplller·Hltleti tlclcnninalion of the fl"l'qllCIlCY ciJaracicrislics and the impulse and step response of a two-pori t:ullsisting of a syslL'1ll of Ihrl'l' l'o;!\ialcOlldudofS terminating ill IUlllped impedances. TlI·RL'J'o!'1 75-1'-51>. l'n5. ISBN

UNIlI/OVII'N IINIVI'IISPERSION RATIOS IN NONLINEAR SYSTEM IDioNTI FICATI

EINDHOVEN UNIVEI{SITY OF TH'IINOLO(;Y THE NETlIERLANDS DEPARTMENT OF ELECTRICAL I':N(;INEERIN(;

Reporls:

XI, Kalll, J.J, v:llliler alld A.A.II. Dallll'lI (IIISI,:RVABII'TY OF I':U'CTRI('AL III'ART ACTIVITY STUDIES WI'I'lIllll': SINGULAR VALli I' IJFCOMPOSITION 'III-Report 7!l-E-XI. I'nx. ISBN '10-hI44-0XI·5

82) Jallsell, J. alld J.F. Barrell ON Till' TIIEORY OF MAXIMUM LlKFLIIIOOD ESTIMATION OF STRUCTURAL RI-:! .. ATIONS. Pari 2: Mulli-dilllellsiollal case. TII-Reporl 7S-E-X2. 1'I7X. ISBN 90-(, I 44-0X2-3

X3) Ellell, W. vall allti E. de J illig OPTIMUM TAPPED DELAY LINES FOR TilE E()lIALIZATION OF MULTIPLE CHANNEL SYSTEMS. 'I'll-Report 78-l'-!U. I 'nx. ISBN 90-(,1 44-0X3-1

li31 Villck, A.J. MAXIMUM LlKELlIIOOD SYNDROME DECODING OF LINEAR BLOCK CODES. TII-Rq)orl 7X-I'-X4. 1'I7K ISBN 90-(d44-0X4-X

X5) Spnoil, W.P. A DI(;!TAL LOW FRH)UENCY SPECTRUM ANALYZER. lISIN(; A PROGRAMMABLE POCKI'T CAL( ·lILATOR. TlI-Reporl 7X-E-XS. I97X. ISBN 90-hI44-0!i5-X

8(») Beneken, J.l'.W .• '1 al TREND PREDICTION AS A BASIS FOR OPTIMAL TIIERAPY. TII-Rl'port 7X-I'-Xh. I97X. ISBN 90-1> I 44-0XI>-()

X7) Ceus, LA.M. alld J. Dijk ('A LCli LATION OF API'.RTli RE AND FA R-FI ELD DISTRI BUT ION "'ROM M EASU REM ENTS IN Till'. IRI:SNI'.L ZONI' OF LAI{(;E REFLHTOR ANTENNAS. 'I'll-Report 7X-L-X7. 1'I7X. ISBN 90-1> I 44-0X7-4

!lX) lIajtiasinski, A. K. TilE (;ALJSS-MARKOV API'ROXIMATED SCIIEME FOR IDENTIFICATION OF MliLTIVARIABELE DYNAMICAL SYSTEMS VIA TilE REALIZATION THEORY. An Explicit Approach. TII-Reporl 78-1'-88.1'178, ISBN 90-6144-088-2

89) NicdcriillSki, A. THE GLOBAL ERROR APPROACII TO THE CONVERGENDE OF CLOSED-LOOP IDENTIFICATION. SELF-TUNINC REWJLATORS AND SELF-TUNING PREDICTORS. 'I'll-Report 78-E-X9. 1'178, ISBN 90-1> I 44-0X9-0

90) Vi lick, A.J. alld A.J.p, dc Pacpc REDUCING Till' NUMBFR OF COMPliTATIONS IN STACK DECODING OF CONVOLUTIONAL CODES BY 1':Xpl,OITING SYMME I'RIES OF THE ENCODER. TII-Reporl 78-1'.-90. 1'178. ISBN lJO-h 144-090-4

9 I) Gculjcs. A.J. and \).J. Klcyn A PARAMETRIC STU\)Y OF 1000 MWe COMBINED CLOSED CYCLE MHD/STEAM ELECTRICAL POWER GENERATING PLANTS. Til-Report 78-E-91. 1'178. ISBN 90-hI44-091-2

92) Massec, 1'. Till' DISPERSION RELATION OF ELECTROTHERMAL WAVES IN A NONEQUILIllRIUM MilD PLASMA. Til-Report 7X-E-92. 1'I7X. ISBN 'IO-h 144-092-0

UNIlIIOVEN UNIVEHSfTY OF TlTIINOLO(;Y TIlE NEIIIERLANUS DEPARTMENT OF ELHTRICAL ENGINEERING

~JJ) Delln, C.A. van

1J.l POLE SCl\'l"l'EHING OF ":U:Cl'HOMAGNE'l'IC WAVES PHOI'A(;ATION THHOUGH A RAIN

MEDll:M. '1'11- Heport }