IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 35, NO. 5, MAY 1988 603

Now V I > N I N \

\ i = l I

(3.23) Al soVl~{ l ; . . ,N} , s ince K I = k I

d i m ( K )

j = N + l

’ [ fi K s I ) ’”( K s j - i =1 i = l

Compare (3.24) with tke right side of (3.7) equated to zero. Then identifying K,, with k , , and using the fact th:t K , = k , V i E

{l,..., N } one can see that (3.24) aL/ak=O. Since by hypothesis K,, K , > 0 V i E {l,. . ., N}, Lemma 3.1 yields that V i E { l ; . . , N } K , , = k , . Then (3.23) implies that K , , = K , V I > N . W

Note, u.a.s. requires that KSl; . . , KsN always have the same sign as k, ; . ., k,, respectively. We now show how this can be achieved.

Consider the case where the bounds on each k , are available viz. rn, and M,ViE{ l ; . . ,N}areknownsuchthatm,dk ,d M I . Consider now a linear translation in each parameter k , b-y a pcsitive amount A , . %us the new unknown vector is k k [ k,; . * , such that k , = k , + A , . Suppose m, + A, > 0. Then with obvious definitions of I?( t ) , Fo ( t ) , L, ( t ) and I?( t ) we can reformulate the adaptive law as

Then Theorem 3.2 ensures the conJergence of K , to I? if K,, is restricted to the same orthant as k , . Due to (3.21), L , ( t ) never increases. Choose initial K, , ( t , ) in a way that m, + A, G K, , ( t , ) < M , + A , V i € ( l ; . . ,N} and V ~ > d i m ( k ) K s , ( t o ) = ~ , , ( f o ) . Then one can find large enough A , , such that if K , , ( t ) = 0 for some iE{l; . . ,N},whiletheotherK,, , i ~ { l , . . . , N } arestill positive, then L, ( t ) > L,( t , ) . For such A , , K s , ( t ) L , ( t ) > 0; V i E {l; . -, N } . Details of this technique of finding A, are omitted, but an illustration for the multilinear case can be found in [9].

IV. CONCLUSION We have presented two uniformly asymptotically convergent

algorithms for the on line identification of systems with a poly- nomial dependence on the unknown parameters. Such para- meterizations occur when a wide class of physical components of the system are the unknown parameters.

ACKNOWLEDGMENT

Many of the ideas in this paper crystallized through discus- sions with Professor B. D. 0. Anderson and Dr. R. J. Kaye.

REFERENCES G. Kreisselmeier, “Adaptive observers with arbitrary exponential rates of convergence,” IEEE Trans Auromat. Contr., vol. AC-21, pp. 2-8, Jan. 1977. R. L. Carroll and D. P. Lindorff, “An adaptive observer for SISO linear systems,” IEEE Trans. Automat. Contr., vol. AC-18, p. 428, 1973. G. Luders and K. S . Narendra, “Stable adaptive schemes for estimation and identification of linear systems,” IEEE Trans. Automat. Contr. vol. AC-19, p. 841, 1974. K. S . Narendra and P. Kudva, “Stable adaptive schemes for system identification and control: Parts I and 11,” IEEE Trans. on Syst. Man Cybern., vol. SMC-4, p. 542, 1974. B. D. 0. Anderson, “An approach to multivariable system identifica- tion,” Automatica, vol. 13, p. 401, 1977. S . Dasgupta and B. D. 0. Anderson, “Physically based parameteha- tions for designing adaptive algorithms,” Automatica, vol. 23, p. 469, 1987. S . Dasgupta, B. D. 0. Anderson and R. J. Kay, “Robust identification of partially known systems,’’ in Proc. 22nd C D C , San Antonio TX, p. 1510, 1983. -, “Output error identification methods for partially known sys- tems,” Int . J. Contr., p. 177, Jan. 1986. -, “Identification of physical parameters in structured systems,’’ Automatica, Mar. 1988. S. Dasgupta, B. D. 0. Anderson and R. J. Kaye, “Persistence of excitation conditions for partially known systems,” Automatica, to ap- pear.

MVSE Adaptive Filtering Subject to a Constraint on MSE JERRY D. GIBSON AND STEPHEN D. GRAY

Abstract -A new adaptive filtering algorithm which minimizes the vari- ance of the squared error subject to a constraint on the mean squared error (MSE) is proposed and studied. For the adaptive linear combiner signal processing configuration, this algorithm, called the LVCMS algorithm for /east oariance subject to a constraint on mean squared error, has a gradient that is a weighted linear combination of the least mean square (LMS) and least mean fourth (LMF) algorithm gradients. Theoretical expressions are derived for the LVCMS algorithm convergence factor and misadjustment, and comparisons are made with the LMS and LMF adaptive rules for Gaussian, Laplacian, and uniform plant noise and driving term distributions. Simulation studies reveal that the theoretical values are quite close to experimental results. Performance comparisons of the LVCMS, LMS, and LMF algorithms based upon Monte Carlo simulation studies indicate that the LVCMS adaptation rule not only can yield a small variance of the squared error, but it also produces very favorable values of the mean squared error and the mean squared coefficient error.

I. INTRODUCTION Since the introduction of the Widrow-Hoff least mean square

(LMS) algorithm over twenty-five years ago [ l ] , adaptive filters have proven useful in a wide variety of applications [2]-[7]. Today, there is still intense research interest in adaptive filtering

Manuscript received August 25, 1986; revised April 10,1987 and December 30, 1987. This work was supported in part by the Statistics and Probability Program in the Office of Naval Research under Contract N00014-81-K-0299. This paper was recommended by W. B. Mikhael and S . Steams, Guest Editors for IEEE Transactions on Circuits and Systems, July 1987.

J. D. Gibson is with the Department of Electrical Engineering, Texas A&M University, College Station, TX 77843.

S . D. Gray was with Sandia National Laboratories, Albuquerque, NM 87185. He is now with E-Systems. Inc., Dallas, TX 75266.

IEEE Log Number 8819867.

OO98-4094/88/0500-0603$01 .OO 01988 IEEE

604 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 35, NO. 5, MAY 1988

because of the need for systems to perform adequately in un- known and/or time-varying environments. Virtually all of this past work has considered adaptive filters which minimize or approximately minimize the mean squared value of some ap- propriately chosen error measure. Recently, however, Walach and Widrow [8], [9] investigated adaptive filtering algorithms which approximately minimize the mean 2 Kth moment ( K > 1) of the error. The present research is in this same vein, but it was motivated by the work of Sain and his students in stochastic control theory [lo], [ll]. In particular, Sain examined the prob- lem of minimizing the variance of the performance index for a linear control system subject to a constraint on the mean value of the performance index.

The work presented here is concerned with designing an adap- tive filtering algorithm which minimizes the variance of the squared error subject to a constraint on the maximum allowable mean squared error (MSE), which is, of course, greater than the minimum MSE attainable using an adaptive filter designed solely for the standard MSE criterion. Thus with this new adaptive filtering algorithm, we can guarantee that our filter performance does not deviate too much from the acceptable distortion (MSE) value. As we shall see, this approach yields some very interesting and useful results. The adaptive signal processing structure con- sidered here is the adaptive linear combiner with real-valued input data [7], [8], which is described in the following section.

11. THE ADAPTIVE LINEAR COMBINER The particular adaptive signal processing configuration of in-

terest here is that shown in Fig. 1 where it is desired to determine the model of an unknown system based upon measurements of the system's input and output [3], [7], [8]. We reduce the problem to one of parameter estimation by letting P(z) = w: + wzz-' + ... +w,$z-~+' , where the {wl*, i=1 ,2 , . - . ,N} are the un- @own system weights or coefficients, and by ass*ng that P (z ) = w1 + w2z-' + . . . + w ~ z - ~ " . Thus P(z) and P(z) have the same order, and we must adapGvely determine the coeffi- cients (wl, i=1,2;..,7V} to make P ( z ) close to P(z) in some sense. The polynomial P ( z ) represents a nonrecursive adaptive filter, often called the transversal form of the adaptive linear combiner. The output of the unknown system (Plant) in Fig. 1 cannot be observed directly, but is contaminated by an additive white noise sequence (n,} which has a symmetric distribution about zero. Additionally, the input sequence ( xI } is assumed to be independent of nJ , symmetrically distributed about zero, and uncorrelated, so E ( x , x , ) = 0, i # j .

The error signal at the j t h time instant can be written as [8]

WIJX,-l+ 1 = d, - yTX, N

Z, = d, - 1 =1

= d, - X T y = n, +( W* - y)TX, = n, - yTX, (1)

where ( W * )== [ w: , w? ,. . . , w$ ] is the vector of optimum filter coefficients, yT= [ y , , w,,,.. ., wN,] is the vector of adaptive filter coefficients at time instant j , XT = [ x,, x,- ',. . . , xIp N +

is the N-vector of past inputs, and 5 = - W * is the coeffi- cient error vector at time j .

111. THE 2 KTH MOMENT ALGORITHMS

Gradient algorithms have been proposed and studied which approximately minimize E { € ; " } , K > 1 , which is the 2Kth mo- ment of the error given by (1) for the adaptive linear combiner

Plant Noise

I "' output

I I 4 P(Z) input 2,

I I

/T' Model

Fig. 1. Adaptive signal processing configuration

[7], [8]. For K =1, we desire to minimize the MSE,

E { E;} = E { df} - 2 E { d,XT} U: + yTE{ = E { df} - 2 P T y + yTRT (2)

where it should be clear that we have defined E { d,X, } P and E { X , X T } = R . The popular Widrow-Hoff LMS algorithm em- ploys an instantaneous estimate of the gradient of E { cf } in (2) to obtain the coefficient adaptation rule [7]

y+* = + p ( - 0 , c f ) = y + 2 p c / x , (3)

where the parameter p, often called the convergence factor, determines the algorithm rate of convergence and stability. Un- der the assumption that the input vectors ( X , ) are uncorrelated, it can be shown that the mean coefficient error vector converges to the optimal solution if and only if 0 < p < 1/ymm, where ym, is the largest eigenvalue of the R matrix. If the eigenvalues of R are not known, then a sufficient condition for the convergence of the mean coefficient error and a necessary and sufficient condi- tion for MSE convergence is that 0 < p < l / trR where trR =

trace of R [3]. The error in the gradient estimate generates a "noise" in the coefficient vectors which results in the MSE being greater than the minimum possible (optimal) MSE. Hence, another important indicator of adaptive filter performance is the misadjustment, denoted by M , which is defined as the ratio of the average excess MSE to the minimum MSE (MMSE) obtained by the optimal coefficient vector. Under the previous assump- tions and the assumption that 7 is close to the optimal weight vector W * , the misadjustment for the LMS algorithm can be shown to be [3], [7]

M = p t r R . ( 4)

For any K 2 1, the coefficient adaptation rule which approxi- mately minimizes E { c/2 " } is [ 81

?+' = 7 +2pKc,2"-'X1. ( 5 )

Under assumptions similar to those used for the LMS algorithm, the mean of the coefficient error vector can be shown to converge if [8]

1

K(2K- l )E{ nfK-'} trR O < p <

1

K(2K-1)E(nfKp2}y,,,, < . (6)

Further, the misadjustment for the 2 Ktb moment algorithm can be obtained as [8]

p K E { n;"-'} trR

( 2 K - - l ) E { n ; } E { n/2"-'} '

M ( K ) = ( 7)

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 35, NO. 5, MAY 1988 605

IV. THE MINIMUM VARIANCE ALGORITHM guaranteed if we choose To develop our new algorithm, we consider the problem of

minimizing the variance of the squared error given by

var( E,’) = E ( [E,’ - E ( E ; ) ] ’ ) subject to a constraint on the MSE, specified by

E( E,’) = U;. ( 9) Appending the constraint in (9) to (8) with a Lagrange multiplier, we obtain the functional H ( W , A ) = E { [ € ; - E(C,’)]~} + A{ U: - E(€; ) } which is to be minimized with respect to the coefficient vector ”: and the undetermined multiplier A. Taking the partial derivative of H( W , A) with respect to the coefficient vector and A yields

and

Normally, we would set the result in (10) to zero, solve for the optimum weight vector in terms of A, and then use the constraint in (11) to find A. However, for our current application, we obtain an estimate for the gradient in (10) by dropping the expectations, so that the coefficient adaptation rule that approximately mini- mizes H( W , A ) is

y+l = ”: + 4 4 x , - 2 p ( 2 4 + A)EJXJ (12)

However, this condition on p also guarantees that (15) is globally asymptotically stable or stable in the large [13] so that E[?+,] -+ 0 as j + 00, and hence, the LVCMS algorithm in (12) pro- vides an asymptotically unbiased estimate of w * . Since the eigenvalues may not be known, we can again use the fact that ym, < t rR, and write the bound on p in (16) as

1

[6E( ni) -2.0’ - A ] trR ’ O < p < (17)

To complete the development, we need an expression for the misadjustment of the LVCMS algorithm. The misadjustment is given by [8]

The evaluation of the numerator in (18) requires the calculation of E { y+lyTl}, which is a straightforward but somewhat lengthy and tedious process (see [14]). The final result for the misadjust- ment is

M’ =

[ 4 E ( n , 6 ) - 4 ( 2 u i + X ) E ( n f ) + ( 2 ~ 0 ” + A ) * E ( n , ’ ) ] p N E ( x ~ )

[6E(nf)-(2u:+A)]E(n;)

where A is now a parameter to be selected. As long as A is (19) negative, H ( W , is a linear With positive weights This expression is evaluated for a few representative noise densi- of two convex functions, so that H ( W , A) is also convex [12]. It follows that H ( W , A) has a minimum (not a maximum) and that

ties and to M(2) in Section v. it cannot have local minima. Equation (12) is, therefore, the new adaptive filtering algorithm which (approximately) minimizes the variance of the squared error subject to a constraint on the MSE. We shall denote the adaptation rule in (12) as the least uariance subject to a constraint on mean squared error algorithm, or just the LVCMS algorithm for brevity. Note that we can also generate an adaptation rule which minimizes the unconstrained variance of the squared error problem by letting A = 0 in H( W , A).

To determine bounds on the parameter p , we first form a recursion for y ,

y+l = y + 4 p + 5 -2p(2u0” + A)c,,X, (13) and then substitute for E , from (1) to obtain

Now, if we assume that is close to W * so that 7 is small, we can neglect higher order terms in 7, so that taking expectations on both sides of (14) yields

E ( y + I } = [ - 2p { 6E( n;) - 20: - A } Jt ] E { 7 } (15)

where we have employed the assumptions that n, is zero mean and independent of both XJ and 5. We know that for the recursion in (15) to be asymptotically stable, the eigenvalues of the bracketed term must have a magnitude less than one. This is

V. SIMULATION RESULTS AND COMPARISONS Simulation results are presented in this section which compare

the performance of the LMS, LMF, and LVCMS algorithms. For ease of comparison, the system to be modeled is chosen to be the same as one investigated in Walach and Widrow [8], namely P(z) = 0.1+0.2z-’ + O . ~ Z - ~ + 0 . 4 z - 3 + 0 . 5 ~ - ~ + O . ~ Z Y ’ + O . ~ Z - ~ +O.2z-’+O.lz-*. The polynomial P ( z ) is used in the $pal processing configuration of Fig. 1, and the coefficients of P(z) are adaptively computed to minimize the particular error criterion of interest. In order to obtain valid results when making comparisons between adaptive filtering algorithms, either the convergence rate must be held fixed and the misadjustment contrasted or the convergence rates can be compared for the same misadjustment. Here, we take the former approach (as in [SI) and choose a common convergence rate. The convergence rate is specified by the time constants of the different modes of the dynamic equation describing the evolution of E( vJ+ which in turn, depends on the eigenvalues of R = E{ XJXT}. For the present work, R = E ( x f ) Z for all j , so the eigenvalues are all the same. The time constant for the LMS algorithm is thus given by [3], [7] r =1/2py, while for the LMF algorithm, the time con- stant is [8] r =1/(12pE(nf)y) where y is the Nth-order eigen- value of R. The time constant for the LVCMS algorithm can be discerned from (16) to be

1

p[6E( n f ) -2.0’ - A ] y ’

7 =

606 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 35, NO. 5 , MAY 1988

5.0 : 15.0 -

h

U 0 U U W

I

I

~

2.5-

I

0.0 L . LVCMS 0 500 1000 1500 2000 2500

U 1 8 6 10.0 -

0.0 0 500 1000 1500 2000 2500

SAMPLE

Fig, 3. Mean squared coefficient error for Laplacianly distributed plant noise. SAMPLE

Mean squared coefficient error for uniformly distributed plant noise. Fig. 2.

SAMPLE The parameters in the various algorithms were selected to fix the time constants at 555 samples.

Simulations both with and without plant noise for Gaussian, Laplacian, and uniformly distributed input sequences { x,} were performed. For those simulations with plant noise, the plant noise distribution is chosen to be the same as the input sequence distribution to :educe the number of cases to be studied. The coefficients in P ( z ) were initialized by adding to each of the coefficients a zero mean, Gaussian random variate with standard deviation 0.75.

Performance with Plant Noise

The additive plant noise had zero mean and E( n;) = 100 for all j . In comparing the three algorithms, we are interested in the three quantities, mean squared coefficient error, the mean squared

quantities were generated by averaging twenty independent se- quences, each 2500 samples in length. An examination of these plots for d l three noise densities reveals that the three algorithms achieve almost the same mean squared error and the same variance of the squared error and hence, these plots are not

0

error, and the variance of the squared error. Plots of these LVCMS

I Fig. 4. Variance of the squared error for LMS, LMF, and LVCMS al-

gorithms with uniform density and no plant noise.

reproduced here. The reason for this behavior is that the noise TABLE I power is so large that its moments dominate these two quantities.

However, the mean squared coefficient error yields some im- portant insights into the performance of the three algorithms. For the uniform noise density results in Fig. 2, the relationship between the LMS and LMF curves are as previously demon- strated by Walach and Widrow [SI, with the LMF performance the better of the two. Interestingly, the LVCMS algorithm is able to reduce the mean squared coefficient error even more than the LMF algorithm. The theoretical and actual misadjustments for the three adaptation rules and three noise densities, uniform, Gaussian, and Laplacian, are presented in Table I. The actual misadjustments were obtained by averaging over the last 125 samples for each curve, and there is good agreement between the theoretical and experimental values for all three noise densities. The mean squared coefficient errors with Gaussian plant noise and driving sequence are not plotted here, but qualitatively, the LMF algorithm performs the poorest followed in order by the LVCMS and the LMS. The mean squared coefficient error for a Laplacian noise density and input sequence shown in Fig. 3 exhibits some very interesting behavior. The LMS algorithm

MISADJUSTMENT VALUES FOR THE THREE ALGORITHMS

L a - LMF LVCMS pdf Theory Actual Theory Actual Theory Actual

Uniform 0.00810 0.00890 0.00340 0.00415 0.00252 0.00303

Gaussian 0 00810 0.00905 0.0135 0.0159 0.00925 0 0102

Laplacian 0.00810 0.00912 0.0810 0.101 0.00255 0.003U8

performs best here followed in order by the LVCMS and the LMF algorithms. The striking thing in Fig. 3 is the difficulty encountered by the LMF algorithm, which actually increases the mean squared coefficient error over the initial condition. Of course, the LMF algorithm should be more sensitive to the heavier tailed densities than the LMS algorithm, and this behav- ior is evident for the Gaussian and Laplacian results. The LVCMS algorithm has a gradient which is a weighted linear combination of the LMS and LMF gradients, and thus it maintains good performance for all three noise densities. This characteristic could be advantageous in unknown or time-varying environ- ments.

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 35, NO. 5 , MAY 1988

1.0 -- 1 .o

0.6

a g U

W

a 3 ' 0.4 z U

0.6

a

!i 0.2

0.0 I

100 200 300 400 500 SAMPLE

(a)

1.0 -;

607

0.0 1 I

(b)

0 100 200 300 400 500

SAMPLE

SAMPLE (4

input density and no plant noise. (c) MSE for LVCMS algorithm with Laplacian input density and no plant noise. Fig. 5. (a) MSE for LMS algorithm with Laplacian input density and no plant noise. (b) MSE for LMF algorithm with Laplacian

Performance without Plant Noise

In order for a reduction in the MSE by the LMS algorithm and a reduction in the variance of the squared error by the LVCMS algorithm to be more noticeable, the plant noise variance cannot be so large as to dominate these quantities. We report here on the extreme but instructive case of no additive plant noise at the output of P ( z ) . Unlike the preceding cases with plant noise, plots of al l three performance indicators are of interest in the no noise case. Simulations were performed with three different den- sities, uniformL Gaussian, and Laplacian, for the input sequence to P ( z ) and P ( t ) . It must also be noted that for the no noise case, the bounds on the convergence factor for the LMF and LVCMS algorithms in (6) and (17) are no longer useful, and so one is left only with simulations to aid in the selection of these quantities. In an attempt to obtain a fair comparison, we chose

the parameters of each algorithm such that the algorithm mini- mizes its own individual error criterion. Thus using simulations, the LMS algorithm convergence factor was selected to minimize the MSE, the LMF convergence factor was chosen to minimize the fourth moment of the error, and the LVCMS parameters were selected to minimize the variance of the squared error. To con- serve space, only selected results are included here (see [14]).

With a uniform density on { x, }, the variance of the squared error for the three algorithms is plotted in Fig. 4 (note the dB scale). The variance of the squared error for the LVCMS adapta- tion rule decreases quite rapidly to its minimum and flattens out. The LMS algorithm variance of the squared error decreases more slowly and is more erratic, but it still decreases relatively quickly to its minimum value near that produced by the LVCMS al- gorithm. The variance of the squared error for the LMF adapta- tion rule has a rapid early response, but soon settles into a much

608 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 35, NO. 5 , MAY 1988

slower decrease at a value considerably higher than the other two algorithms. The variance of thesquared error for the LVCMS algorithm tends to combine the best features of the LMS and LMF curves with a rapid early reduction of variance and a continued substantial decrease as more samples are processed. For a Gaussian input sequence, the MSE, the variance of the squared error, and the mean squared coefficient errors do not provide insights not available in other cases and hence are not included here. Fig. 5(a)-(c) contains the MSE for the LMS, LMF, and LVCMS algorithms, respectively, when the driving sequence is Laplacian. The LMS and LVCMS curves become small rapidly, but the LMF algorithm has a strikingly different MSE curve, and it clearly performs considerably poorer than the other two. The variance of the squared error curves are not presented, and the mean squared coefficient errors are smoother, but qualitatively the same as the MSE plots in Fig. 5(a)-(c).

VI. CONCLUSIONS An adaptive filtering algorithm has been proposed which ap-

proximately minimizes the variance of the squared error subject to a constraint on the MSE. A bound on the convergence parameter p was derived, as well as an expression for the mis- adjustment. Simulation results using uniform, Gaussian, and Laplacian distributed plant input sequences and uniform, Gauss- ian, and Laplacian plant noises illustrated that the LVCMS algorithm provides consistently small values of the performance indicators, namely, the MSE, the variance of the squared error, and the mean squared coefficient error. In those instances where the LVCMS algorithm does not yield the best performance of the three adaptation rules, its performance is close to the best. It never performs poorer than both the LMS and LMF algorithms for the cases considered. The heavy-tailed density causes difficul- ties for the LMF algorithm, while the “no-tailed” uniform den- sity compromises LMS performance. The theoretical and actual

(simulations-based) misadjustment values show close agreement. The generally excellent performance of the LVCMS algorithm is a consequence of the fact that its gradient is a weighted linear combination of the LMS and LMF algorithm gradients. In fact, it can be shown that the LMS, LMF, and LVCMS adaptation rules are all special cases (but probably the most important) of generalized weighted-moment-minimizing rules.

REFERENCES B. Widrow and M. E. Hoff, “Adaptive switching circuits,” in 1960 WESCON C o w . Rec., pt. 4, pp. 96-104. B. Widrow, P. Mantey, L. Griffiths, and B. Goode, “Adaptive antenna systems,” Proc. IEEE, vol. 55, pp. 2143-2159, Dec. 1967. B. Widrow, “Adaptive filters,” in Aspects o/ Network and System Theory, R. Kalman and N. DeClaris, Eds., New York: Holt, Rinehart, and Winston, 1971, pp. 563-587. B. Widrow et al., “Adaptive noise cancelling: Principles and applica- tions,’’ Proc. IEEE, vol. 63, pp. 1692-1716, Dec. 1975. J. D. Gibson, “Adaptive prediction in speech differential encoding systems,” Proc. IEEE, vol. 68, pp. 488-525, Apr. 1980. S. U. H. Qureshi, “Adaptive equalization,” Proc. IEEE, vol. 73, pp. 1349-1387, Sept. 1985. B. Widrow and S. D. Steams, Adaptive Signal Processing. E n g l e w d Cliffs, NJ: Prentice-Hall, 1985. E. Walach and B. Widrow, “The least mean fourth algorithm and its family,” IEEE Trans. Inform. Theory, vol. IT-30, pp. 275-283, Mar. 1984. E. Walach, “On higher-order error criteria for system identification,” IEEE Trans. Acourt., Speech, and Signal Processing, vol. ASSP-33, pp. 1634-1635, Dec. 1985. M. K. Sain, “Relative costs of mean and variance control for a class of linear, noisy systems,” in Proc. Third Annuol Allerton Con/. on Circuit and System Theory, Monticello, IL, pp. 121-129, 1965. M. K. Sain, “Control of linear systems according to the minimal vari- ance criterion-A new approach to the disturbance problem,” IEEE Trans. Automat. Control, vol. AC-11, pp. 118-122, Jan. 1966. D. G. Luenberger, Optimization by Vector Space Methods. New York: Wiley, 1969. W. L. Broean. Modern Control Theorv. Enelewood Cliffs. NJ: Prentice-

, I

Hall, 2nd edition, 1985. J. D. Gibson and S . D. Gray, “MVSE adaptive filtering subject to a constraint on MSE,” TCSL Res. Rep. no. 86-05-01, Dept. Elec. Eng., Texas A&M Univ., Aug. 5, 1986.