8/11/2019 475_46ce89a7

1/5

I EEE

TRANSACTIONS ON SIGNAL PROCESSING. VOL. 41. NO. 3. MARCH

transforms for improving performance

of

transform domain normal-

ized LMS algorithm, Proc. Inst. Elec. Eng., pt. F, vol. 139, pp.

327-335, Oct. 1992.

[ 6 ] D. L. Duttweiler, Adaptive filter performance with nonlinearities in

the correlation multipliers, IEEE

Trans. Acoust.

, Speech,

Signal

Processing, vol. ASSP-30, pp. 578-586, Aug. 1982.

[7] E. Eweda, Analysis and design of a signed regressor LMS algorithm

for stationary and nonstationary adaptive filtering with correlated

Gaussian data,

IEEE

Trans. Circuits S y s t . , vol. 37, pp. 1367-1374,

Nov. 1990.

[8]

R. Price, A useful theorem for nonlinear devices having Gaussian

inputs, IRE

Trans. Infarm.

Theory, vol. IT-4, pp. 69-72, June 1958.

[9] S . S . Reddi, A time-domain adaptive algorithm for rapid conver-

gence,

Proc.

IEEE, vol. 72, pp. 533-535, Apr. 1984.

[ lo] B. Farhang-Boroujeny, An efficient quasi-LMSiNewton algorithm:

Analysis and simulation results, Tech. Rep. 01-10-91, Commun.

Division, Dept. Elec. Eng., National Univ. Singapore, Oct. 1991.

1111 D.

F.

Marshall and W . K . Jenkins, A fast quasi-Newton adaptive

filtering algorithm, in

Proc.

1988

ICASSP

(New York, NY), Apr.

11-14, pp. 1377-1380.

[I21 D. F. Marshall and

W. K.

Jenkins, A fast quasi-Newton adaptive

filtering algorithm, IEEE Trans. Signal Processing, vol. 40, pp.

1652-1662, July 1992.

A Unified Square-Root-Free Approach for

QRD-

Based Recursive Least Squares Estimation

S .

F .

Hsieh , K . J .

R.

Liu, and K . Y a o

Abstract-Givens rotation is the most commonly used method in per-

forming the

QR

decomposition QRD) updating. The generic formula

for these rotations requires explicit square-root sqrt) computations

which constitute a computational bottleneck and are quite undesirable

from the practical VLSI circuit design point of view. So far, there has

been more than ten known sqrt-free algorithms. In this correspon-

dence, we provide a unified systematic approach for the sqrt-free Giv-

ens rotation. By properly choosing two parameters,

p

and

v,

all pre-

viously known sqrt-free,

as

well as new methods, are included in our

unified approach. This unified treatment is also extended to the QRD-

based recursive least squares RLS) problem for optimum residual ac-

quisition without sqrt operations.

I. INTRODUCTION

The Giv ens ro ta t ion , which requires a square- root ( sqrt ) opera-

tion in the generic formulation, is a versatile method in performing

many s ignal processing algor i thms involv ing matr ix computat ions ,

such as the

Q R

decompos i t ion

(QRD),

the s ingular value decom-

pos i t ion , and the e igendecompos i tion [7] . W hile many researchers

have worked on reformulat ing a lgori thms su i tab le for paralle l com -

put ing and V LSI archi tectures , current V LSI archi tectures s t i l l d is-

approve if not prohibit sophisticated computations. A noticeable

example is the sqr t operat ion , which may occupy much area in a

VLSI chip or may also require many cycles to accomplish such

Manuscript received July 10, 1991; revised March 31, 1992. This work

was supported in pan by the National Science Council of

t he

Republic of

China under Grant NSC80-E-SP-009-01A , the NSF Engineering Center

Grant ECD-8803012 and Minta Martin Award of the University of Mary-

land, the NSF Grant NCR-8814407, and a UC MICRO grant.

S . F.

Hsieh is with the Departm ent of Communication Engineering, Na-

tional Chiao Tung University, Hsinchu, Taiwan 30039, R epublic of China.

K . J . R . Liu is with the Department

of

Electrical Engineering, Systems

Research Center, University

of

Maryland, College Park, MD 20742.

K . Yao is with the Department of Electrical Engineering, University of

California, Los Angeles, CA 90024- 1594.

IEEE Log Number 9206004.

1993 1405

computat ion . In addi t ion , a recent s imulat ion s tudy presented in

[

141 by Proudler

et

al. showed that a finite-precision implementa-

tion of a sqrt-free lattice algorithm achieved better numerical re-

su l ts than that us ing the convent ional Giv ens ro ta t ion method.

Thus , much ef for t has been spent

on

minimizing or everl elimi-

nat ing the sqr t operat ion f rom these a lgor i thms . One wel l-known

example is the sqr t - free Given s ro ta t ion fi r st proposed by Gentle-

man [5]. Hammarling generalized his results briefly [9]. Later,

o ther vers ions of the sqr t - f ree Given s ro ta t ions were a lso proposed

[ l ] , [ 2 ] , [ 8 ] .

All

of the above algor i thms only focus

on

the sqrt-

free Givens rotation itself andlor its applications in solving a least

squares (LS) problem. M cWh ir ter (131 was the f i rs t to apply the

sqr t- f ree Givens ro ta t ion to recurs ive LS (RLS) problems in com -

put ing the des ired opt imum res idual without so lv ing expl ic i t ly for

the LS coefficients. Closely related to the Givens rotation method

is the modif ied Gram-Schmidt (MGS) or thogonal izat ion , which is

another approach in per forming the QRD. Ling et al. [ l l ] , 1121

and Kalson and Yao

[lo]

independent ly developed the sqr t - f ree

MG S methods for the R LS f i l ter ing problems . A rank-one updat ing

of Cholesky factor izat ion without sqr t s has a lso been repor ted in

the l i tera ture 131. Recent ly , Chen and Yao [4] summarized the

works done on the sqrt-free RLS filtering and proposed another

more efficient sqrt-free method. So far , there has been more than

ten known sqrt-free algorithms 111, [2], [4], 151, [ 8 ] - [ l l ] . Ho w-

ever , a l l of the previous ly known der ivat ions were based

on

heu-

r is tic approaches . Th ere is

no

known sys tematic way of generat ing

the sqr t - f ree a lgor i thms . Motivated by these works , we wish to

understand the fundamental re la t ionships among these sqr t - f ree a l-

gor i thms .

One

of the contr ibut ions of th is correspondence is that

these fundamental re la t ionships are character ized in s imp le man-

ners through only two parameters .

The pro to types of general ized sqr t- f ree a lgor ithms are g iven in

Section 11, where all of the sqrt-free algorithms are found by the

selection of two parameters . W e proceed in Sect ion I to seek a

sqr t-f ree opt im um res idual of the RLS f i l ter ing problem. A brief

conclus ions is g iven in Sect ion IV .

11. THEpv F A M I L YF SQ U A R E-R O O T-FR EELGORITHMS

A Givens ro ta t ion matr ix as g iven by

is used to premult ip ly a two-row matr ix

1

Y1

C Y 2 a p

P

02 . . 0

a;

f f ff;

to zero out the e lement a t the

(2,

I ) locat ion such that i t becomes

where

c = a,/-, a n d s = PI/- (1)

1053-587X/93 03.00 993 IEEE

8/11/2019 475_46ce89a7

2/5

I406

IEEE rRANSACTIONS ON SIGNAL PROCESSING, VOL. 41, NO.

3.

MARCH 1993

In VLSI circuit design, sqrt operation is expensive, because i t

takes up much area

or

i s s low (due to many i t e ra t ions) . Therefore ,

it i s advantageous to avoid or minimize sqrt operations.

By taking out a sca l ing fac tor f rom each row, the two row s under

considera t ion before and a f te r the Givens or thogonal t ransforma-

tions is denoted by

s expressions in (18) are not explici t ly needed

in

the computa t ion

of ( l3)- (17 ) . Th e use of the rota t ion parameter c in 18) (with one

sqrt operation) will be further considered in Section

I11

when the

optimum residual e i s des i red. F urthermore , Sec t ion 111 will show

that it is possible to obtain

e

without any sqrt operation and the

explici t computation of the rota t ion parameter c can be bypassed.

To avoid repe t i t ive computa t ions and take the advantage of previ -

ously com puted resul t s, (14) .

16),

and (18) use the newly updated

k:

of (13) . As s ta ted ear l i e r , we a re free to choose those two pa-

rameters and

U.

Different choices of

p

and

U

will affect the num-

ber of multiplications and divisions, as well as the numerical sta-

bil i ty and parallel ism of these computations.

It can be easily shown that this unified view can generate al l of

the previously known sqrt-free algori thms via a proper choice of p

and U . In fact , there has been more than ten sqrt-free algori thms

known so fa r . Among them are Gent leman [ 5 ] , Hammarl ing [ 9 ] ,

Bare iss [ I ] , Kalson and Yao

[ I O ] ,

Ling er al. [ I l l [ 1 2 ] , Barlow

and Ispen [2], Chen and Yao 141, Gotze and Schwiege lsohn [ 8 ] .

For example , i f we choose p =

and

U = 1 , i t becomes the sqrt-

free algori thm proposed by Gent leman in [5] and can be upda ted

as fol lows:

k6 = k,,a:

+

k,bt

19)

ki,

= k,,ki,/kA 20)

a;

=

(21)

a, = (k,a,a, +khb1b,)/kl,

b,

=

-bla, a,b,.

j

= 2 ,

. P

(22)

and

where

6)

,,

k h , k6,

and kA are the scaling factors result ing in sqrt-free

- .

operations, and

a:

and p: are the upda ted

a,

and 0 when

PI

is

zeroed out.

Now, our task is to find the expressions for, k:, k i , a; , {(a,

b,), = 2. , p } , n te rms of k, , k h , { (a , ,b l ) , j =

1,

. , p } ,

such that

no

sqrt operation is actually needed. The sqrt expressions

of

a,a,4.

nd

4

n

( 5 )

and

6)

are used for representa-

t ional purposes only and are not actually performed.

Replacing a =

&a,, 0

= a b , , ai = U,, 6

=

b, , j

= 1, , p , in (1)-(4) l eads to

a ; = J(k,at + kbbt)/kL

4 k,a: kbb:

a;

=

[koala,+ kbblb,]

j = 2 ,

. 3

P .

To check that these results are correct , we find that

k u a l a ] + khblb ,

a; =

4 a ;

=

4

ala, + PIP

m

c ,a / + SIP]

and

To avoid sqrt computa t ion, we need to de te rmine k: and ki such

that ai, a,, and b, will not require sqrt operation. It is clear that if

we choose

k, ,

and k i as

ku kb

k

-

v2(k,a: kbb:)

.IO, -

Pia,

(24)

-sa cp,

which are consistent with the results in (3) and (4 ) . For the sys tol ic

array implementation described in

[13],

we choose a , =

1

and de-

fine the generalized rotat ional parameters

where

p

and U are parameters that wil l be determined later to be

any sqrt-free function of k,, kb,

a , ,

and b , , hen @ - I O ) can be

computed wi thout sqr t opera t ion. We then have the fol lowing up-

dating formulas without sqrt operation:

k: = (koa: kbb:) /p2

(13)

C = k , /k : , and = k,b,/k, , . (25)

Then we have

a, = Cu, Sb,

b, = b, bla,.

(27)

(28)

b,

=

u[-b,a, a,b,]

(17)

c = ( a l / p ) a ,

and

= ( b , / p ) J k h / k : , . 18)

Not ice tha t the sqrt opera t ions di sappear in our formulas of (13)-

(17) , whi le they a re needed in the Givens rota t ions . Also, the

c

and

These results are consistent with the works by Gentleman [ 5 ] and

McWhirte r

[

131. The details of the systolic implementation can be

found in [13].

In Table I, we list various sqrt-free algori thms and the corre-

sponding choices of p and

U.

Hence , thi s c lass of sqr t -free a lgo-

8/11/2019 475_46ce89a7

3/5

I E E E T R A N S A C T I O N S O N S I G N A L P R O C E S S I N G , V O L .

41,

N O .

3, MARCH 1993

I407

TABLE I

SOME

KNOWNQRT-FREEL G O R I T H M S

F THE

pv F A M I L Y

Ir V

Authors (Year) Remark

1

a , k ,b :

1

Gentleman (1973)

a , =

Hammarling (1974)

Bareiss (1982)

Ling (1989), KalsoniYao (1985)

k , ,a ,

=

1

CheniYao (1988) k ,a , =

GotzeiSchwiegelshohn (1989)

BarlowiIspen (1 987) Scaled

New algorithm

TABLE

I1

COMPARISONS

O F

COMPUTATIONAL COMPLEXITY

OF

SOME MEMBERSN + V F A M I L Y

~

Square Root

ultiplication Division Addition

1

2p - 1 1

Givens Rotation

4P

p = l , u = l 4p + 3 2p - 0

2p

+

6

2

2p

- 1 0

2p 6

2

2p - 0

4p 4 2 2p

- 1 0

1

a ,

p = l , v = -

4 p

5

2p - 2

0

koa: khb:

u =

1 4p + 6 2 2p - 0

Cc=

ko kh

4p

+

4 1 2p - 1 0

koa:

+

khb:

p = koa: + k,b:,

v

=

rithms is called the

pv

family of sqrt-free Givens rotation algo-

1

v

r i thms .

koa:

kbb:

not

Only

can

we

generate

those

known

sqr t- f ree a lgo- then

we

can readily verify that this is a new sqrt-free algorithm. In

f l t h m s , but we are

choos ing

new

pairs Of

( p ,

parameters .

to f ind new sqr t- f ree

by

let us

fact , the new sqf l- f ree a lgor i thm is among the bes t in the l is t of

Table

I

in terms

of

number of d iv is ions , e .g . , i t on ly requires one

n

division and no square root. In principle, there are unlimited choice s

of

p

and

v

for sqr t - f ree a lgor i thms . Table I shows compar isons

of

computat ional complexi ty of some algor i thms l is ted in Table I .

choose

p

=

koa: +

kbb:

8/11/2019 475_46ce89a7

4/5

1408

IEEE

TRANSACTIONS ON SIGNAL PROCESSING,

VOL. 41, NO.

3,

M ARCH 1993

111. SQRT-FREET R I A N G U L A RR R A Y P D A T I N GND O P T I M U M

R E S I D U A L C Q U I S I T I O N

Solving a full rank LS problem Aw = b ( A R m X n n

without the sqrt operation can be easily achieved [5]. Let the QRD

of

A

be

QTA

=

R,

here

R

s an upper triangular matrix, then

and the optimum weight vector can be obtained by solving RO =

U. Now, start ing with a

ful l

dense augmented mat rix

&[A

b] ,

ser ies of sqr t -free rota t ions can be appl ied to zero ou t the subvec tor

below the main diagonal of the underlying matrix to obtain

[ R 0 ; 1

where = diag 4,

' ,

4)nd

R

=

SF, U

=

U.

Since the explici t comp utation of s not required, the optimum

weight vector can be obtained without the sqrt operation by

solving R O =

U .

In the following, we will apply the developed prototypes of sqrt-

free rota t ions deve loped before to the Q RD-based RLS estimation

problem where we are only interested in the optimum residu al . How

to obtain the optimum residual by using the systolic array [131 has

been well known. To be specific, we are interested

in

updat ing

from

R u

R U

L T

to [o v 1

(30)

It has been shown [13] that the p X p upper t r i angular mat r ix R

can be obta ined through a sequence of p Givens rota t ions , and the

opt imum res idua l

e for

the newly appended da ta

[x y ]

is given

by

with c, represent ing the cosine va lue of the th rotat ion angle.

agona l mat r ix l eads (30) to the form of

Fac toring out the sca l ing constants into the premul t iplying di -

Unl ike the previously deve loped formula , where we a re only in-

terested

in

updat ing k , ,

a,,

, o k,'

,

U, , and zeroing out al l the b , s ,

this t ime we also need to know the cosine values explici t ly as re-

quired in the opt imum res idual given in (31) .

After the first rotat ion, b , will be zeroed out and we have

(33)

(34)

with (p l , v I ) being the parameter pair which are st i l l free to be

chosen later. Note the close analogy of (33)-(38) to those of (13)-

(18) . Simi la r ly, a f te r the i th rota t ion (1