2562 IEEE TRANSACTIONS ON SIGNAL … IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 59, NO. 6, JUNE...

2562 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 59, NO. 6, JUNE 2011

Convergence Properties of AdaptiveEqualizer Algorithms

Markus Rupp, Senior Member, IEEE

Abstract—In this paper, we provide a thorough stability analysisof two well known adaptive algorithms for equalization based on anovel least squares reference model that allows to treat the equal-izer problem equivalently as system identification problem. Whilenot surprising the adaptive minimum mean-square error (MMSE)equalizer algorithm behaves �–stable for a wide range of step-sizes, the even older zero-forcing (ZF) algorithm however behavesvery differently. We prove that the ZF algorithm generally doesnot belong to the class of robust algorithms but can be convergentin the mean square sense. We furthermore provide conditions onthe upper step-size bound to guarantee such mean squares conver-gence. We specifically show how noise variance of added channelnoise and the channel impulse response influences this bound. Sim-ulation examples validate our findings.

Index Terms—Adaptive gradient type filters, error bounds,�-stability, mean-square-convergence, mismatch, robustness,

zero forcing.

I. INTRODUCTION

M ODERN digital receivers in wireless and cable-basedsystems are not considerable without equalizers in

some form. The first mentioning of digital equalizers was byLucky [1], [2] in 1965 and 1966 at the Bell System Tech-nical Journal, who also coined the expression “zero forcing”(ZF). Correspondingly, the MMSE formulation was providedby Gersho [3]. Further milestone papers are by Forney [4],Cioffi et al. [5], Al-Dhahir et al. [6] as well as Treichler et al.[7]. Good overviews in adaptive equalization are provided in[8]–[11] and [12].

Consider the following transmission over a time dispersive(frequency selective) channel model:

(1)

Manuscript received July 15, 2010; revised October 09, 2010 and December22, 2010; accepted February 18, 2011. Date of publication March 03, 2011; dateof current version May 18, 2011. The associate editor coordinating the reviewof this manuscript and approving it for publication was Prof. Isao Yamada. Thiswork has been funded by the NFN SISE project S10609 (National ResearchNetwork Signal and Information Processing in Science and Engineering). Sec-tion IV of the paper appears as a conference paper entitled “On Gradient TypeAdaptive Filters with Non-Symmetric Matrix Step-Sizes” in the Proceedingsof the International Conference on Acoustics, Speech and Signal Processing(ICASSP), Prague, Czech Republic, May 22–27, 2011.

The author is with the Vienna University of Technology, Institute of Telecom-munications, 1040 Vienna, Austria (e-mail: [email protected]).

This paper has supplementary downloadable multimedia material availableat http://ieeexplore.ieee.org provided by the authors. This includes all Matlabcode, which shows the simulation examples. This material is 11 kB in size.

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TSP.2011.2121905

TABLE IOVERVIEW OF MOST COMMON VARIABLES

Here, vector consistsof the current and past symbols according to the span

of channel which is considered here tobe of Toeplitz form as shown in (2). Received vector

. Let the transmission be dis-turbed by additive noise being of the same dimension as .

.... . .

. . .. . .

......

(2)

Throughout this paper, we will assume that transmit signalshave unit energy that is , and the noise vari-

ance is given by without loss of generality. Notethat for a Toeplitz form channel we have whichrefers to a Single-Input Single-Output (SISO) case. This con-tribution only focuses on the SISO model in (1); extensions to-wards multiantenna (MIMO) and/or multiuser (MU) scenarioswill be treated elsewhere. A linear equalizer applies an FIR filter

on received signal so that is an estimate offor some delayed version of . The optimal selection of

will not be treated here.Table I provides an overview of the most common variables

and their dimensions. In Section II we introduce a referencemodel for linear equalizers based on a deterministic leastsquares approach. We thus avoid the commonly used MSE ap-proach. Section III formulates the problem in terms of adaptive

1053-587X/$26.00 © 2011 IEEE

RUPP: CONVERGENCE PROPERTIES OF ADAPTIVE EQUALIZER ALGORITHMS 2563

equalizers. While the MMSE adaptive equalizer is straightfor-ward to analyze, Lucky’s classical adaptive ZF equalizer turnsout to be more difficult. Here, we have to solve two problemsthat is adaptive filters with arbitrary matrices as step-sizesand parameter error terms appearing in the additive noise termwhich is addressed in Sections IV and V, respectively. With thisnew knowledge we return to the original problem and finallyanalyze the adaptive ZF equalizer algorithm in Section VI.Simulation results support our findings. A conclusion is pro-vided in Section VII.

II. A REFERENCE MODEL FOR EQUALIZATION

A. ZF Equalizer

A solution to the ZF equalizer problem is equivalently givenby the following least-squares formulation:

(3)

with indicating a unit vector with a single one entry at posi-tion , thus . The resulting LS solution is calledthe ZF solution . Note that this form of derivationdoes not require signal or noise information but focuses only onproperties of linear time-invariant systems of finite length (FIR);it thus ignores the presence of noise entirely. This is identical tothe original formulations by Lucky [1], [2] where system prop-erties were the focus.

In a SISO scenario, we have and the solution to thisproblem is obviously given by

(4)

As the ZF solution leads to ISI for finite length vectors, wepropose the following reference model for ZF equalizers

(5)

with modeling noise

(6)

Due to its projection properties we find that the outcome ofthe reference model lays in the range of with an additiveterm from its orthogonal complement with the followingproperties:

(7)

(8)

(9)

The last term can be interpreted as the energy of the modelingerror but equally describes the remaining ISI power.

How does a received signal look after such ZF-equalization?We apply on the observation vector and obtain

(10)

(11)

Such relation serves as SISO ZF reference model as we willapply it to adaptive algorithms further ahead. Note that often ISIas well as additive noise is treated equivalently as a compoundnoise as indicated in (11).

B. MMSE Equalizer

Correspondingly, the well-known MMSE solutioncan be obtained including noise variance from a spec-

trally white noise

(12)

as a solution of the weighted LS problem

(13)

Correspondingly to the reference model for ZF equalizers in (5),we can now also define a reference model for MMSE equalizers

(14)

with modeling noise

(15)

Note, however, that different to the ZF solution the mod-eling error is not orthogonal to the MMSE solution, that is

. Multiplying the signal vector withwe obtain

(16)

How does a received signal look after such MMSE-equal-ization? We apply on the observation vector and obtainanalogously to (11):

(17)

Note that compound noise is now different when com-pared to the ZF solution in (11).

III. ADAPTIVE EQUALIZERS

A. Recursive MMSE Algorithm

We start with the classical adaptive MMSE equalizer as it ismuch easier to analyze than its ZF counterpart. Such algorithm


is also known under the name least-mean-square (LMS) algo-rithm for equalization (see, for example, Rappaport [8]). Recur-sive algorithms try to improve their estimation in the presenceof new data that has not been applied before. Due to this prop-erty they typically behave adaptively as well. This allows thereinterpretation of the algorithm in terms of a recursive proce-dure in which new data are being processed at every time in-stant . Starting with some initial value , the equalizer esti-mate reads

(18)

In order to perform a stability analysis we introduce parametererror vector as well as the reference model(17) and obtain

(19)

The description of the MMSE equalizer is thus identical to aclassical system identification problem. As we know the be-havior of a standard LMS algorithm for such circumstances wecan deduce immediately the results from there for our MMSEequalizer problem. While results concerning the classical con-vergence in the mean square sense are already known [3], newresults are possible concerning the adaptive filter misadjustmentand relative system mismatch due to the knowledge of com-pound noise term . We will show this further ahead inSection VI with some simulation results.

Entirely new is the convergence of these adaptive equalizersin the –sense. According to [13], [14], and [15, Ch. 17],the LMS algorithm can be described in terms of robustness,showing –stability. Different to the classic approaches inwhich the driving signals are stochastic processes, a robustnessdescription does not require any statistic for the signals ornoise. In fact, the –stability is guaranteed for any drivingsignal and noise sequence. With this new interpretation of anadaptive MMSE equalizer as system identification problem, wecan thus directly adapt the results from literature and state thefollowing for the adaptive MMSE equalizer algorithm.

Theorem 3.1: The adaptive MMSE equalizer withUpdate (18) is -stable from its input uncertain-ties to its output errors

for a constant step-size if

(20)

and for a time-variant step-size if

(21)

We like to note that the derivation of the theorem can be foundin [13] in which the small gain theorem was applied. The boundsare thus conservative and not necessarily tight. A recent discus-sion on this is presented in [16]. Further note that the statementin Theorem 3.1 relates to the convergence of the undistorted apriori error . If the noise energy is bounded, so is thea priori error energy and thus for the a priori errorneeds to converge to zero (Cauchy series). In order to concludethat parameter error converges to zero, a further persistent ex-citation argument on the observed data is required [15], [17].

B. Recursive ZF Algorithm

The original ZF algorithm [2] is given by the following updateequation, starting with some initial value to estimate

(22)

Matrix is required to shorten the longvector in (1) from to its length . We have deliberatelyselected a step-size as will become clear later.

Applying the same method as before, introducing parametererror vector and the ZF reference model (11), wenow obtain

(23)

Here, we abbreviated

(24)

(25)

an algorithmic compound noise that contains an additional com-ponent when compared of the corresponding ZF value in(11), depending on the parameter error vector itself. We split ex-citation vector into two parts:

(26)

(27)

. . .. . .

. . .. . .

. . . (28)


as well as the Toeplitz matrix [compare with (2)] shown in (28)at the bottom of the previous page for .

(29)

(30)

We can thus also split the inner vector product

(31)

By this splitting operation, we obtain an upper triangular matrixas shown in (28). We assume that the cursor index is selected

in such a way that the main diagonal is not filled with zeros. Aslong as this property is satisfied, the matrix is regular and itsinverse exists.

With such reformulation we find now the update equation tobe

(32)

The compound noise now takes on an additional component thatis also dependent on the parameter error vector.

Comparing the algorithm with a standard LMS algorithmwith matrix step-size [17]–[20]

(33)

(34)

we identify and . As is an uppertriangular matrix we can expect it to be regular, provided thatthe cursor position is chosen correctly and thus .The parameter error vector is simply given by . Thedistorted a priori error term

(35)

comprises of an undistorted a priori term andadditive noise . We follow here mostly the notationof [15].

Alternatively, we can premultiply (32) by from the leftand use the substitution , resulting in an updateform in rather than in . In this case we identifyand . This alternative form will be even more usefulas typically driving process is white and thus correspondingterms become much easier to compute. Details will follow fur-ther ahead in Section VI.

We have now reformulated Lucky’s original ZF algorithminto an LMS algorithm with two unusual features:

1) a nonsymmetric matrix step-size ;2) a noise term that depends on the parameter error

vector itself.In order to proceed with the algorithmic analysis, we first haveto address both effects.

IV. ADAPTIVE GRADIENT ALGORITHMS WITH NONSYMMETRIC

MATRIX STEP-SIZE

As the content of this section can be treated independentlyof the equalizer context, we will present the results in differentnotation. Later, we simply substitute the terms as indicated inthe last discussion of the previous section. We thus will delib-erately select now as parameter error vector (and not ) aslater linearly transformed versions will be applied to describethe ZF algorithm leading to as well as the true compoundnoise rather than the additive noise .

We start with an LMS algorithm with matrix step-sizein the context of system identification for which we as-

sume a reference system exists with additively disturbed output

(36)

denoting the observed output of the reference ,and being its input(or sometimes called regression) vector, the driving se-quence and additive noise. We subtract the true solution

from its estimate and use only the parameter error vectorfrom now on:

(37)

(38)

While there exists convergence results in the mean-squaresense [19], [20] and in the sense [17] for symmetric matrixstep-sizes, there is none for nonsymmetric matrix step-sizes.In this section, we will address this issue with various novelideas (Method A,B,C). We follow here a classical analysispath [13]–[15], [17] and show some weak so-called localrobustness properties when considering the adaptation fromtime instant to . However, it will turn out that suchalgorithms are not as robust as the LMS algorithm with sym-metric matrix step-size. We will therefore at a certain point ofthe analysis have to leave this path of robustness and employstatistical properties of the input signals and noise. We thuswill not be able to provide strict stability conditions for thealgorithm but instead convergence in the mean square sense.Note that MSE convergence could also be shown with simplertechniques (for example, [21]) but it would be tedious, requirea lot more of simplifying assumptions, and only provide loosestep-size bounds, while our analysis is much more rigorous andwill provide very tight bounds as we will show by simulationexamples. We will even prove further ahead in the followingsection that the ZF equalizer algorithm indeed does not belongto the class of robust algorithms.

A. Analysis Method A

We introduce an additional square matrix thatwe multiply from the left to obtain a modified update equation

(39)


A straightforward idea is now to compute parameter vector errorenergy in the light of such matrix that is

(40)

This weighted square Euclidean norm requires that isbeing positive definite or in this case equivalently is beingof full rank, which is a first condition and restriction on . Wethus obtain

(41)

where we employed the following notation:

(42)

(43)

We introduce a proportionality factor such that

(44)

which allows the simplification of previous (41) into

(45)

Next to additive noise we can now form a new variable :

(46)

(47)

(48)

which allows to reformulate (45)

(49)

We can further bound by

(50)

(51)

(52)

for some positive value which, in turn, allows now towrite

(53)

(54)

(55)

If term is negative or equivalently, the last term in (54) can simply

be dropped and we obtain a first local stability condition relatingthe update from time instant to :

Lemma 4.1: The adaptive gradient type algorithm withUpdate (38) exhibits the following local robustness proper-ties from its inputs to its outputs

:

(56)

as long as can be selected so thatfor some , and .

Such a local robustness property however is only useful if itcan be extended towards a global property. To this end we sumup the energy terms over a finite horizon fromand compute norms:

(57)

The expression makes sense as long as . However wecan extend the result even for . To show this property,we start with summing up (54) under the condition that

, remembering that and obtain

(58)

(59)

for which both terms and remainpositive and bounded. We thus can conclude on global robust-ness:

Lemma 4.2: The adaptive gradient type algorithmwith Update (38) exhibits a global robustness from ini-tial uncertainties and additive noise energy se-quence to its a priori errorsequence if the normalized step-size

for some ,and .

While such statement ensures the LMS algorithm with non-symmetric matrix step-size to be –stable, it actually is basedon the condition that . This brings us back to thechoice of which we will have to analyze further. Recall thatwe defined that is we relateand . As these inner vector products,defining as well as , can take on every arbitrary value,independent of each other, there is no relation in form of a bound


from one to the other and as a consequence a strict stabilityanalysis must end here. Note however, if the relations of the pre-vious lemma hold for any signal they also hold for random pro-cesses following some statistics. Thus, placing the expectationoperation over all energy terms results in correct statements eventhough somewhat restricted now by the imposed statistics. Notefurther that even if and is hard to be related for generalsignals, from a statistical point of view the two signals are re-lated. This can be seen when we compute their average energy,that is

Starting with (41), taking expectations on both side andsolving for steady-state, that is

, we find

(60)

(61)

where we applied the independence assumption [15, Ch.9] on regression vectors with autocorrelation matrix

of driving process and correspondingparameter error vectors . The so defined can beinterpreted as the mean of . The term takes ona particular simple form ( ) when a normalized step-size isapplied: . The steady-state solution can be a meansfor defining a step-size bound: . As is typicallyunknown, it would be difficult to evaluate . A conservativebound however is simple to derive by the Rayleigh factor of aHermitian matrix1:

Let us summarize the previous considerations in the followingtheorem.

Theorem 4.2: The adaptive filter with Update (38) with non-symmetric step-size matrix , some square matrix that satis-fies the condition , and normalized step-sizeguarantees convergence in the mean square sense of its param-eter error vector if the step-size

(62)

under the independence assumption of regression vectorswith .

If the minimum Rayleigh factor is negative we cannotconclude convergence. If the step-size is larger than weexpect divergence.

1We use the short notation � � �� . Similarly� � �� and� � �� . Moreoverfor positive definite Hermitian matrices, we use� � � to denote positivenessand � � � � � � � .

Example A: Let us use and . In this case wefind

(63)

and convergence in the mean square sense for.

B. Analysis Method B

We now modify the previous method by the following idea.Let us assume again an additional matrix that is multipliedfrom the left. However, now we will not compute the norm in

but the inner vector product including only. We repeatthe process with and obtain so the conjugate complex of thefirst part. Adding both terms results in the following:

(64)

with the new abbreviations

(65)

(66)

(67)

(68)

From here, the derivation follows the same path as before, wethus will present the important highlights so that the reader canfollow easily. Note that the norm in which we require conver-

gence of the parameter error vector is in which makesMethod B distinctively different to the previous one.

As in Method A we employ the same method and arrive at

(69)

(70)

This allows for a first local stability condition:Lemma 4.3: The adaptive gradient type algorithm with

Update (38) exhibits the following local robustness proper-ties from its inputs to its outputs

:

(71)

as long as can be selected so thatfor , and .

Following the same method as before, we find the followingglobal statement:


Lemma 4.4: The adaptive gradient type algorithm with Up-date (38) exhibits a global robustness from initial uncertaintyand additive noise sequence to itsa priori error sequence if the normalizedstep-size forand .

This lemma offers similar properties than Lemma 4.2 ofMethod A and thus the problem of the in general unknown

. We thus also follow the steady-state computation as inthe previous A and find

(72)

Theorem 4.3: The adaptive filter with Update (38) with non-symmetric step-size matrix , some square matrix that sat-isfies the condition , and normalized step-size

guarantees convergence in the mean square senseof its parameter error vector if the step-size

(73)

under the independence assumption of regression vectorswith .

If the minimum Rayleigh factor is negative we cannotconclude convergence. If the step-size is larger than weexpect divergence.

1) Example B: Let us use and . In this case,we find

(74)

and convergence for . Thus, forthis choice methods A and B coincide (compare to Example A).

C. Analysis Method C

We now continue in a similar way as in previous Method Bbut assume that exists. We find the following innervector product:

which we complement by its conjugate complex part just as inprevious Method B. However, now some terms compensate as

. We now introduce

(75)

(76)

(77)

(78)

(79)

Note that now takes a slightly different form compared tothe values in Methods A and B, leading to much tighter bounds.

Lemma 4.5: The adaptive gradient type algorithm withUpdate (38) exhibits the following local robustness propertiesfrom its input values to

its output values

as long as can be selected so thatfor some and as long as the matrix

is positive definite.Summing up the energy terms and computing norms we ob-

tain the global robustness property:Lemma 4.6: The adaptive gradient type algorithm with

Update (38) exhibits a global robustness from initial un-certainties and additive noiseenergy sequence to its apriori error energy sequence if

for someand .

Note that this analysis method compared to the previoustwo methods delivers a stronger argument when compared toMethods A and B. Here the step-size bound could becomepositive and it might be even possible to guarantee –stabilityin some scenarios.

Following the stochastic approach as before, we compute thesteady-state to be

(80)

We find the mean of to be bounded by

Theorem 4.4: The adaptive filter with Update (38) with non-symmetric step-size matrix , satisfying and nor-malized step-size guarantees convergence in the meansquare sense of its parameter error vector if the step-size

(81)

under the independence assumption of regression vectorswith . Alternatively, the so normalized algo-rithm also converges if the matrix is negative definite.

Note that due to the normalization of the step-size by termsin , replacing by causes a positive definite matrix

to become negative definite so that the productremains positive. Also due to the products effects com-pensate each other. The positive upper bound for the normal-ized step-size is thus not changed by this. The derivation simplyrequires in this case to define

to be a norm.


TABLE IIVARIOUS ALGORITHMIC NORMALIZATIONS BASED ON THE PROPOSED METHODS A, B, C WITH CORRESPONDING CONDITION �� (ALGORITHM 2:

�� ) FOR MEAN-SQUARE SENSE CONVERGENCE UNDER WHITE EXCITATION PROCESSES.� � � STANDS FOR POSITIVE DEFINITENESS

Fig. 1. Convergence bound � over parameter �.

D. Consequences

A further consequence worth stating is:Corollary 4.1: Consider the three update equations:

(82)

(83)

(84)

with and . Allthree algorithms converge in the mean square sense as long as

is positive definite for sufficiently small step-size. Note that this can even include that is negative

definite.Furthermore, the steady-state of such algorithms can also be

computed. Starting from (41) we compute the expectation ofthe energy terms considering a fixed start value as well asrandom excitation and additive noise . For steady-state,we find that , and weobtain

(85)

which immediately leads to the desired result for normalizedstep-sizes :

(86)

The only difference to other LMS algorithms shows in the valueof that takes on the value two in a standard NLMS. However,

the actual value of is difficult to compute. For white drivingprocesses , its bounds are

E. Validation

In a Monte Carlo experiment, we run simulations (20 runsfor each parameter setup) for filter order with a noisevariance of . Excitation signals are white symbolsfrom a QPSK alphabet. The experiment applies the matrix

...(87)

where we vary from zero to one.2 Independent of the valuethe matrix is always regular. We are interested in correctness

and the precision of our derived bounds. We thus use the nor-malized step-sizes and normalize them w.r.t. their bounds, thatis . We thus expect to find converging algorithmsfor . Table II depicts a list of choices. Fig. 1 exhibits theobserved bounds for from Algorithm 1 to 6 when ranging

. Compare Algorithm 2 and Algorithm 5, being iden-tical but with different bounds, the bound of Algorithm 2 beingabout twice as large as that of Algorithm 5. Algorithm 1 and Al-gorithm 3 as well as Algorithm 4 and Algorithm 6 show almostidentical behavior, respectively. Above all, only Algorithm 3 isof practical interest if matrix is not known beforehand.

V. GRADIENT ALGORITHMS WITH VARIABLE NOISE

We now return to the second problem defined at the end ofSection III – that is adaptive algorithms in which the noise parthas a component depending on the parameter error vector itself.We thus consider the following update form (noisy LMS algo-rithm):

(88)

(89)

2Note that for all examples and simulations the corresponding Matlab code isavailable online at https://www.nt.tuwien.ac.at/downloads/featured-downloads.


Fig. 2. System mismatch over step-size � for classical NLMS versus noisyNLMS.

in which we incorporated classical noise , a constant and fi-nally a term depending on the parameter error vector. Assumingwhite noise sequences with variance and , respec-tively, we find

(90)

where we assumed noise and to be uncorre-lated. Although in later considerations this is not true, we willneglect the correlation term as it is typically small. For whitedriving processes and a normalized step-size with normaliza-tion (NLMS), it is well known [11], [15] that therelative system mismatch at time instant is given by

(91)

Factor accounts for the correlation in driving sequence .For uncorrelated processes, . Substituting the noise vari-ance we obtain at equilibrium

(92)

Now the relative system mismatch is no longer proportional tobut also impacts the stability bound. We find now a reduced

stability bound at

(93)

that is the higher the noise variance, the smaller the step-sizebound. Fig. 2 with , , depicts thedependency of the relative system mismatch on the normalizedstep-size . Only for small step-sizes we approximately find theprevious behavior.

A. Interpretation

Note that such Update (88) with noise term (89) can alsobe interpreted differently. We can equivalently formulate thesource for the parameter error vector dependent noise as partof the gradient term:

(94)

Fig. 3. System mismatch over adaptation steps for various values of � in noisyNLMS algorithm.

(95)

Thus, we can interpret such algorithm equivalently as an algo-rithm with a disturbed gradient. In [16], such algorithm is provento be non –stable although it can behave well in average. Wethus can conclude from here that algorithms with such propertyare guaranteed not to be robust but can behave convergent in themean square sense as long as the step-size is sufficiently small.

B. Validation

Simulation runs for such scenario with white excitationshowed that this description is indeed very accurate. Fig. 3shows typical simulation runs with various step-sizes com-paring the steady-state with the predicted values according to(92) at high noise level , . The agreementis obviously excellent.

VI. ADAPTIVE ZF GRADIENT ALGORITHM

We are now ready to analyze our original problem, theadaptive ZF algorithm. We briefly summarize the previoussteps [(22) and (34)]:

(96)

(97)

Note that in literature [1], [10], it is argued that a steepestdescent algorithm of this form must converge to a global min-imum. It is further conjectured that noise may have an impacton the convergence of the algorithm. In the following we willshow that there is indeed conditions on the channel required forglobal stability. Moreover, we will specify the qualitative as wellas quantitative impact of noise on step-size choice, convergencecondition and finally steady-state behavior.


A. Analysis

In order to analyze the behavior of Update (97), we premul-tiply by first, , and obtain

(98)

We thus recognize our adaptive algorithm of the previous sec-tion with nonsymmetric step-size matrix . In the nextSection VI-B we will apply our derived conditions for typicalexample channels and test whether we can satisfy the requiredconditions and thus find step-size bounds for mean square con-vergence. Based on the previous analysis we can now derive theconvergence of the algorithm for a normalization

and a normalized step-size .The second problem we encounter is the noise being depen-

dent on the parameter error vector. Let us thus now investigatethe noise term. We can write

(99)

The noise variance at time instant of this can be computed to

(100)

under the condition that we have white noise as well as awhite excitation signal and that both random processes areuncorrelated. Note that we have selected not to apply the ex-pectation operators (that is, and ) in theequation above as we are considering a given thus fixed channel.In case a set of channels with random selection is considered,the expectation operator may be applied. We obviously obtain anoise variance that depends on time and on the state of the adap-tive filter. At steady-state the movement of the adaptive filter isexpected to have reached an equilibrium (in the mean square).We then obtain

(101)

Problematic is the last term that we denote by

Parameter is bounded from below by the smallesteigenvalue (which can be as small as zero) and from above bythe largest eigenvalue of which canfurther be bounded by .

Applying the considerations from the previous section we cannow compute the relative system mismatch to be

(102)

and the upper step-size bound is eventually

(103)

Note that not only can have a wide range but also. Different to MMSE estimation, we did not include a

correction factor to reflect the correlation of driving sequenceas we expect white data sequences only.If we approximate , we can turn around

the last equation and obtain

(104)

This formulation clearly shows that we expect an upper limit fornoise when employing a fixed normalized step-size . Once thislimit is exceeded, the algorithm can become unstable.

B. Simulation Examples

In this section we provide some simulation results to verifyour findings. For this purpose, we consider a set of seven channelimpulse responses of finite length

We select the length of the channel to be for which thefirst four impulse responses have decayed considerably. In allsimulations relatively strong noise was added of . Wewill show all graphic results based on channel and report forwhich cases we found significant changes. Note that channels

and perfectly fit to the example in (87) forand , respectively.

Figs. 4 and 5 depict the ZF and MMSE solutions of bothchannels as well the corresponding convolutions of channel andequalizer, respectively. Both channels appear to be relativelysimilar in terms of ZF and MMSE performance but can beclearly differentiated due to the relatively strong noise term

. We selected the cursor position to be in the middle ofthe equalizer, well knowing that this may not be the optimalposition. As the resulting convoluted systems are rather sym-metrical it is not expected that other positions change the resultsdramatically. As we average the Monte Carlo runs only overdifferent noise and transmit symbols, 20 runs were performedwhich already provide sufficiently smooth and accurate curves.

1) MMSE Equalizer: In our first equalizer experiment weshow the classical adaptive MMSE equalizer for channel .Fig. 6 depicts the norm of the parameter error vector (relativesystem distance) over iteration numbers for various normalizedstep-sizes following (21). As expected, the sta-bility is guaranteed for . The learning behavior is


Fig. 4. Upper: ZF and MMSE equalizer impulse response for � . Lower:Corresponding convolution of channel and equalizer for � .

Fig. 5. Upper: ZF and MMSE equalizer impulse response for � . Lower:Corresponding convolution of channel and equalizer for � .

very much as expected. If we compare the theoretically derivedvalues

(105)

(106)

with the simulations in Fig. 6, we find excellent agreement. Asexpected the MMSE equalizer does not differentiate between

Fig. 6. MMSE equalizer with normalized step-size on channel � .

various channels and behaves perfectly robust. The step-sizebound for is tight and holds for various sequences in-dependent of the channel and the noise. For all seven channelswe obtained very similar results.

2) ZF Equalizer: We will pick now Algorithm 3 as it is theonly algorithm that can work without knowing the channel inform of the triangular matrix . We identify . Forthis algorithm on the other hand we have to find the smallesteigenvalue of in order tofind the upper step-size bound which is practically impossiblewithout knowing the matrix.

(107)

(108)

We apply the adaptive ZF equalizer on channel . We againemployed the normalized step-size with tospeed up convergence. Note that due to the QPSK symbols for

the norm is constant, and the algorithm can also beinterpreted as a fixed step-size algorithm. The results are dis-played in Fig. 7. Compared to the adaptive MMSE filter we finda significantly higher convergence speed which is certainly dueto the fact that the MMSE algorithm is driven by a strongly cor-related signal while the ZF equalizer only ”sees” the white datasequence. According to the theoretical derivation the algorithmcannot be guaranteed to converge as the smallest eigenvalue be-comes slightly negative (see Table III). However, the derivedbound is not tight and the stability bound is found for largerstep-sizes at around 0.50. For all other channels, convergence inthe mean square sense was expected as and the de-rived step-size bounds showed to be typically conservative.

Using relation (103), we can also derive a step-size boundbased on and .

(109)

We used a least-squares fit of the plots for various step-sizesto fit to the two unknown values and . They are typically


TABLE IIIVARIOUS CHARACTERISTICS FOR THE SEVEN CHANNELS UNDER CONSIDERATION

Fig. 7. ZF equalizer with normalized step-size on channel � .

around one ( ) and 0.5 ( ) and thus the corresponding boundaround 0.65 which is a good agreement with our observations.

It is worth studying the entries of Table III. On the secondcolumn we filled in the step-size bound that we found experi-mentally. This is to compare with the third column which pro-vides a conservative step-size bound, the fourth column whichprovides a bound for which we certainly expect divergence aswell as the fifth column with an extreme conservative bound andthe sixth column with a bound derived from our least squares fit.The later is tedious and difficult to be used in algorithmic designbut rather accurate. For the first five channels we find as goodagreement as we had in the adaptive MMSE algorithm. For thelast two channels however, prediction turns out to be more dif-ficult. Even our conservative bounds from the third column turnout to be a bit too high. An explanation can be found in the ma-trix that is very ill conditioned. In the last column ofTable III, we list the distortion measure

being the convolution of the adaptive filter and the channelimpulse response. In [10] it is argued that the adaptive ZF algo-rithm only works for values smaller than one. What we foundindeed is that the algorithm still works however, loses a lot ofthe estimation quality that we find in other channels. The qualityof the adaptive ZF equalizer is thus very much dependent on theactual channel.

Remember that the adaptive ZF algorithm is not robust. Whenswitching from random QPSK data sequences to worst caseQPSK sequences, we found the algorithm to be non convergent,

Fig. 8. ZF equalizer with fixed normalized step-size on channel � whenvarying additive noise variance � .

as expected. We could not find a step-size small enough to en-sure convergence under such worst case sequences for any ofthe channels.

We repeated the experiment for a fixed normalized step-sizebut varied the noise variance. The result is shown in

Fig. 8. As predicted in (104), the stability bound varies with thenoise and in our example for noise variances larger than one, thealgorithm indeed became unstable.

VII. CONCLUSION

The adaptive MMSE equalizer has shown to be robust, guar-anteeing –stability for a fixed range of step-sizes independentof additive noise or the channel itself. Its steady-state qualitywas derived analytically and showed excellent agreement withthe simulation examples.

Novel criteria have been found to ensure convergence of awell-known adaptive ZF receiver. Different to the general beliefthese criteria strongly depend on the channel that is to be equal-ized as well as on the additive noise that is present. Simulationresults verified the correctness of these findings. We were able toderive explicit steady-state results of adaptive ZF receivers. Asa side result, a class of adaptive gradient type algorithms witharbitrary but time-invariant matrix step-size can now be treatedin terms of mean square stability. Note however that the condi-tions we need to apply cannot be satisfied for all kind of chan-nels. According to Table III, certain criteria on the channel are tobe satisfied. We observed that although the adaptive ZF equal-izer algorithm behaves stable under random sequences, a strict

–stability does not hold and worst case sequences under which


the algorithm become unstable even under very small step-sizescan be found.

ACKNOWLEDGMENT

The author would like to thank the anonymous reviewers fortheir constructive help and S. Caban for preparing the figures aswell as R. Dallinger for providing a Matlab code framework.

REFERENCES

[1] R. W. Lucky, “Automatic equalization for digital communication,” BellSyst. Tech. J., vol. 44, pp. 547–588, Apr. 1965.

[2] R. W. Lucky, “Techniques for adaptive equalization of digital commu-nication systems,” Bell Syst. Tech. J., vol. 45, pp. 255–286, Apr. 1966.

[3] A. Gersho, “Adaptive equalization in highly dispersive channels fordata transmission,” Bell Syst. Tech, J., vol. 48, pp. 55–70, 1969.

[4] G. Forney, “Maximum likelihood sequence estimation of digital se-quences in the presence of intersymbol interference,” IEEE Trans. Inf.Theory, vol. 18, no. 3, pp. 363–378, 1972.

[5] J. M. Cioffi, G. Dudevoir, M. Eyuboglu, and G. D. F. Jr, “MMSE deci-sion feedback equalization and coding-Part I,” IEEE Trans. Commun.,vol. 43, pp. 2582–2594, Oct. 1995.

[6] N. Al-Dhahir and J. M. Cioffi, “MMSE decision feedback equalizers:Finite length results,” IEEE Trans. Info. Theory, vol. 41, pp. 961–975,Jul. 1995.

[7] J. R. Treichler, I. Fijalkow, and C. R. Johnson Jr., “Fractionally spacedequalizer. How long should they really be?,” IEEE Signal Process.Mag., vol. 13, pp. 65–81, May 1996.

[8] T. Rappaport, Wireless Communications. Englewood Cliffs, NJ:Prentice-Hall, 1996.

[9] E. A. Lee and D. G. Messerschmitt, Digital Communications. Nor-well, MA: Kluwer, 1994.

[10] J. Proakis, Digital Communications. New York: McGraw-Hill, 2000.[11] S. Haykin, Adaptive Filter Theory, 4 ed. Englewood Cliffs, NJ: Pren-

tice-Hall, 2002.[12] M. Rupp and A. Burg, “Algorithms for equalization in wireless ap-

plications,” in Adaptive Signal Processing: Application to Real-WorldProblems. New York: Springer, 2003.

[13] M. Rupp and A. H. Sayed, “A time-domain feedback analysis of fil-tered-error adaptive gradient algorithms,” IEEE Trans. Signal Process.,vol. 44, pp. 1428–1440, Jun. 1996.

[14] A. H. Sayed and M. Rupp, “Robustness issues in adaptive filters,” inThe DSP Handbook. Boca Raton, FL: CRC Press, 1998.

[15] A. H. Sayed, Fundamentals of Adaptive Filtering. New York: Wiley,2003.

[16] R. Dallinger and M. Rupp, “A strict stability limit for adaptive gradienttype algorithms,” in Conf. Rec. 43rd Asilomar Conf. Signals, Syst.,Comput., Pacific Grove, CA, Nov. 2009, pp. 1370–1374.

[17] A. H. Sayed and M. Rupp, “Error-energy bounds for adaptive gra-dient algorithms,” IEEE Trans. Signal Process., vol. 44, no. 8, pp.1982–1989, Aug. 1996.

[18] M. Rupp and J. Cezanne, “Robustness conditions of the LMS algorithmwith time-variant matrix step-size,” Signal Process., vol. 80, no. 9, pp.1787–1794, Sep. 2000.

[19] S. Makino, Y. Kaneda, and N. Koizumi, “Exponentially weighted step-size NLMS adaptive filter based on the statistics of a room impulseresponse,” IEEE Trans. Speech Audio Process., vol. 1, pp. 101–108,Jan. 1993.

[20] D. L. Duttweiler, “Proportionate normalized least mean square adap-tation in echo cancellers,” IEEE Trans. Speech Audio Process., vol. 8,pp. 508–518, Sep. 2000.

[21] A. Feuer and E. Weinstein, “Convergence analysis of LMS filters withuncorrelated Gaussian data,” IEEE Trans. Acoust., Speech, SignalProcess., vol. ASSP-33, pp. 222–230, Feb. 1985.

Markus Rupp (M’03–SM’06) received theDipl.-Ing. degree from the University of Saar-brücken, Saarbrücken, Germany, in 1988 and theDr.Ing. degree from Technische Universität Darm-stadt, Darmstadt, Germany, in 1993, where heworked with E. Hänsler on designing new algorithmsfor acoustical and electrical echo compensation.

From November 1993 to July 1995, he held, withS. Mitra, a postdoctoral position with the Universityof California, Santa Barbara, where he worked withA. H. Sayed on a robustness description of adaptive

filters with impact on neural networks and active noise control. From October1995 to August 2001, he was a Member of Technical Staff with the WirelessTechnology Research Department, Bell Laboratories, Crawford Hill, NJ, wherehe worked on various topics related to adaptive equalization and rapid imple-mentation for IS-136, 802.11, and the Universal Mobile TelecommunicationsSystem. Since October 2001, he has been a Full Professor of digital signal pro-cessing in mobile communications with the Vienna University of Technology,where he founded the Christian Doppler Laboratory for Design Methodology ofSignal Processing Algorithms, at the Institute of Communications and Radio-Frequency Engineering, in 2002. He served as Dean from 2005 to 2007. He isthe author or a coauthor of more than 350 papers and is the holder of 15 patentson adaptive filtering, wireless communications, and rapid prototyping, as wellas automatic design methods.

Dr. Rupp is currently an Associate Editor for the EURASIP Journal ofAdvances in Signal Processing and the EURASIP Journal on EmbeddedSystems. He was an Associate Editor for the IEEE TRANSACTIONS ON SIGNAL

PROCESSING from 2002 to 2005. He has been an Administrative CommitteeMember of EURASIP since 2004 and served as the President of EURASIPfrom 2009 to 2010.

Date post:	24-Mar-2018
Category:	Documents
Upload:	duongdiep
View:	225 times
Download:	3 times

2562 IEEE TRANSACTIONS ON SIGNAL … IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 59, NO. 6, JUNE...

Documents