MAIN DOCUMENTATION.docx

8/7/2019 MAIN DOCUMENTATION.docx

1/83

1

A Project report

On

A Time-Varying Convergence Parameter for the

LMS Algorithm in the Presence of White Gaussian

NoiseSubmitted in partial fulfillment of the

Requirements for the award of the degree ofBACHELOR OF TECHNOLOGY

IN

ELECTRONICS & COMMUNICATION ENGINEERINGBY

N.KIRAN KUMAR REDDY (06681A0437)

D.ARAVIND

M.S.SAI ADWAITH (06681A0427)

D.S.PRADEEP (0

Under the guidance of

Mr. B.NAVEEN KUMAR

Assistant Professor

Department of

Electronics & Communication Engineering

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING

CHRISTU JYOTI INSTITUTE OF TECHNOLOGY & SCIENCE

(Affiliated to JNTU, Hyderabad)

COLOMBONAGAR, YESHWANTHAPUR,JANGAON(MDL), WARANGAL(DIST),

ANDHRA PRADESH.

(2009-10)


2/83

2

CERTIFICATE

This is to certify that the major project entitled A Time-Varying Convergence Parameter for the

LMS Algorithm in the Presence of White Gaussian Noise being submitted by

N.KIRAN KUMAR REDDY (06681A0437)

D.ARAVIND

M.S. SAI ADWAITH (06681A0427)

D.S.PRADEEP (066

In partial fulfillment for the award of the degree of Bachelor of Technology in

Electronics & Communication to Christu Jyothi Institute of Technology & Science, Jangaon

is a bonafide work carried by them under our guidance and supervision.

The results embodied in this project have not been submitted to any other

University/Institute for the award of any degree or diploma.

HEAD OF THE DEPARTMENT

Mr. G.THOMAS REDDY

Associate Professor

EXTERNAL EXAMINER


3/83

3

Acknowledgements

We are very grateful to our DirectorRev.Fr.Vijay Kumar Reddy, our Principal sir, Dr.A.Anjaneyulu who have given us the opportunity to take up our project work and for givingtheir kind cooperation.

Our sincere and heartfelt thanks to our Head of Department Mr.G.Thomas Reddy for allowing

our thoughts to become a reality. Although we have achieved very little in this project we havelearnt a lot and we are sure that this would be a very helpful tool in all our endeavors.

Our thanks to our guide Mr.DAVID, who has the experience of working at KRESTTECHNOLOGIES Hyderabad, for igniting our minds with the necessary input that made us to

work hard and put our heart and soul.

Our heartfelt thanks to our internal guide Mr.B.NAVEEN KUMAR for helping & guiding usthroughout the entire project.

As a group we need to thank each other as we stayed together, put our best effort in all the

difficulties we got in the process of making of this major project.

We thank the management for granting us the permission to do this project in a very renowned

institute.


4/83

4

ABSTRACT

A novel approach for the least-mean-square (LMS) estimation algorithm is proposed. Rather

than using a fixed convergence parameter , this approach utilizes a time-varying LMS

parameter. This technique leads to faster convergence and provides reduced mean-squarederror compared to the conventional fixed parameter LMS algorithm. The algorithm has been

tested for noise reduction and estimation in narrow-band FM signals corrupted by additive white

Gaussian noise.

For the LMS algorithm in a white Gaussian noise environment. A general power decaying law

has been studied, however, other time-varying laws could also be applicable. The main idea is to

set the convergence parameter to a large value in the initial state in order to speed up the

algorithm convergence. The modified algorithm has been tested for noise reduction and

estimation in linear frequency-modulated (LFM) narrowband signals corrupted by additive white

Gaussian noise.


5/83

5

CONTENTS PAGE NO.

1.INTRODUCTION 2.ADAPTIVE FILTER

2.1BLOCK DIAGRAM2.2APPLICATIONS2.3LEAST MEAN SQUARES

2.4FORMULATION2.5SIMPLIFICATIONS

3. LEAST MEAN SQUARE ALGORITHM

3.1 OPTIMAL LEARNING RATE

3.2 PROOF4. MEAN SQUARED ERROR

4.1DEFINATIONS AND BASIC PROPARTIES4.2ALTERNATIVE USAGE

5. ADDITIVE WHITE GUASSIAN NOISE 6. THE PROPOSED ALGORITHM

7.ADDAPTIVE ECHO CANCELLERS 8.APPROACH TO DEVELOP LINEAR ADAPTIVE FILTER

9.IMPLEMENTATION OF LMS ALGORITHM 10 .SPEECH SIGNAL

11. FILTER DESIGN 12. METHODOLOGY

13. IMPULSE RESPONSE 13.1MOVING AVERAGE EXAMPLE

14. INFINITE IMPULSE RESPONSE 15. DESCRIPTION OF BLOCK DIAGRAM

16. SIMULATION RESULTS 17. CONCLUSION


6/83

6

CHAPTER-1

INTRODUCTION


7/83

7

1.INTRODUCTIONThe LMS algorithm is a well-known adaptive estimation and prediction technique. It has been

extensively studied in the literature and widely used in a variety of applications. The

performance of the LMS algorithm is highly dependent on the selected convergence parameter and the signal condition. A larger convergence parameter value leads to faster convergence of

the LMS algorithm, i.e., convergence of the filter coefficients to their optimal values. After

coefficients converge to their optimal value, the convergence parameter should be small for

better estimation accuracy .

In this project, we propose a time-varying convergence parameter for the LMS algorithm in a

white Gaussian noise environment. A general power decaying law has been studied, however,

other time-varying laws could also be applicable. The main idea is to set the convergence

parameter to a large value in the initial state in order to speed up the algorithm convergence. As

time passes, the parameter will be adjusted to a smaller value so that the adaptive filter will have

a smaller mean-squared error. The modified algorithm has been tested for noise reduction and

estimation in linear frequency-modulated (LFM) narrowband signals corrupted by additive white

Gaussian noise. Simulation results have shown that the modified LMS algorithm has better

performance in terms of convergence speed than the conventional LMS algorithm with a

constant convergence parameter and the least-mean-squares error close is to the optimal value.


8/83

8

CHAPTER-2

ADAPTIVE FILTERS


9/83

9

2.ADAPTIVE FILTERAn adaptive filter is a filter that self-adjusts its transfer function according to an optimizing

algorithm. Because of the complexity of the optimizing algorithms, most adaptive filters are

digital filters that perform digital signal processing and adapt their performance based on the

input signal. By way of contrast, a non-adaptive filter has static filter coefficients (which

collectively form the transfer function).

For some applications, adaptive coefficients are required since some parameters of the desired

processing operation (for instance, the properties of some noise signal) are not known in

advance. In these situations it is common to employ an adaptive filter, which uses feedback to

refine the values of the filter coefficients and hence its frequency response.

Generally speaking, the adapting process involves the use of a cost function, which is a criterion

for optimum performance of the filter (for example, minimizing the noise component of the

input), to feed an algorithm, which determines how to modify the filter coefficients to minimize

the cost on the next iteration.

As the power of digital signal processors has increased, adaptive filters have become much more

common and are now routinely used in devices such as mobile phones and other communicationdevices, camcorders and digital cameras, and medical monitoring equipment.

EXAMPLE

Suppose a hospital is recording a heart beat (an ECG), which is being corrupted by a 50 Hz noise

(the frequency coming from the power supply in many countries).

One way to remove the noise is to filter the signal with a notch filter at 50 Hz. However, due to

slight variations in the power supply to the hospital, the exact frequency of the power supply

might (hypothetically) wander between 47 Hz and 53 Hz. A static filter would need to remove all


10/83

10

the frequencies between 47 and 53 Hz, which could excessively degrade the quality of the ECG

since the heart beat would also likely have frequency components in the rejected range.

To circumvent this potential loss of information, an adaptive filter could be used. The adaptive

filter would take input both from the patient and from the power supply directly and would thus

be able to track the actual frequency of the noise as it fluctuates. Such an adaptive technique

generally allows for a filter with a smaller rejection range, which means, in our case, that the

quality of the output signal is more accurate for medical diagnoses.

2.1BLOCK DIAGRAM

The block diagram, shown in the following figure, serves as a foundation for particular adaptivefilter realisations, such as Least Mean Squares (LMS) and Recursive Least Squares (RLS). The

idea behind the block diagram is that a variable filter extracts an estimate of the desired signal.

To start the discussion of the block diagram we take the following assumptions:

The input signal is the sum of a desired signal d(n) and interfering noise v(n)

x(n) = d(n) + v(n)

y The variable filter has a Finite Impulse Response (FIR) structure. For such structures the

impulse response is equal to the filter coefficients. The coefficients for a filter of orderp

are defined as


11/83

11

.

y The error signal or cost function is the difference between the desired and the estimated

signal

The variable filter estimates the desired signal by convolving the input signal with the

impulse response. In vector notation this is expressed as

where

is an input signal vector. Moreover, the variable filter updates the filter coefficients

at every time instant

where is a correction factor for the filter coefficients. The adaptive algorithm generates

this correction factor based on the input and error signals. LMS and RLS define two different

coefficient update algorithms.

2.2 APPLICATIONS OF ADAPTIVE FILTERS

y Noise cancellation

y Signal prediction

y Adaptive feedback cancellation


12/83

12

2.3LEAST MEAN SQUARES (LMS)y Least mean squares (LMS) algorithms is a type of adaptive filter used to mimic a desired

filter by finding the filter coefficients that relate to producing the least mean squares of

the error signal (difference between the desired and the actual signal). It is a stochastic

gradient descent method in that the filter is only adapted based on the error at the current

time. It was invented in 1960 by Stanford University professor Bernard Widrow and his

first Ph.D. student, Ted Hoff.

2.4 PROBLEM FORMULATION

Most linear adaptive filtering problems can be formulated using the block diagram above. That

is, an unknown system is to be identified and the adaptive filter attempts to adapt the filter

to make it as close as possible to , while using only observable signals x(n), d(n)

and e(n); but y(n), v(n) and h(n) are not directly observable. Its solution is closely related to theWiener filter.

The idea behind LMS filters is to use steepest descent to find filter weights which

minimize a cost function. We start by defining the cost function as


13/83

13

where e(n) is the error at the current sample 'n' and E{.} denotes the expected value. Applying

steepest descent means to take the partial derivatives with respect to the individual entries of the

filter coefficient (weight) vector

where is the gradient operator. With

and it follows

Now, is a vector which points towards the steepest ascent of the cost function. To find

the minimum of the cost function we need to take a step in the opposite direction of .

To express that in mathematical terms

where is the step size. That means we have found a sequential update algorithm which

minimizes the cost function. Unfortunately, this algorithm is not realizable until we know

.


14/83

14

2.4SIMPLIFICATIONSFor most systems the expectation function must be approximated. This

can be done with the following unbiased estimator

where N indicates the number of samples we use for that estimate. The simplest case is N = 1

For that simple case the update algorithm follows as

Indeed this constitutes the update algorithm for the LMS filter.

LMS ALGORITHM

The LMS algorithm for a pth order algorithm can be summarized as

Parameters: p = filter order

= step size

Initialisation:

Computation: For n = 0,1,2,...

where denotes the Hermitian transpose of .


15/83

15

CHAPTER-3

LEAST MEAN SQUARE ALGORITHM


16/83

16

3.LEAST MEAN SQUARE ALGORITHMLeast mean squares (LMS) algorithms are used in adaptive filters to find the filter coefficients

that relate to producing the least mean squares of the error signal (difference between the desired

and the actual signal). It is a stochastic gradient descent method in which the filter is adaptive

based on the error at the current time. It was invented in 1960 by Stanford University professor

Bernard Widrow and his first Ph.D. student, Ted Hoff. The adaptive linear combiner output, is a

linear combination of the input samples. The error in measurement is given by

where is the transpose vector of input samples.


17/83

17

To develop an adaptive algorithm ,it is required to estimate the gradient of =E[] by taking

differences between short term averages of .Instead, to develop the LMS algorithm process,is

taken as the estimate of Thus at each iteration in the adaptive process a gradient estimate form is

as follows

With this simple estimate the steepest descent type of adaptive algorithm is specified as

This is the LMS algorithm. Where is the gain constant that regulates the speed and stability

of adaptation. Since the weight changes at each iteration are based on imperfect gradient

estimates, the adaptive process is expected to be noisy. The LMS algorithm can be implemented

without squaring, averaging or differentiation and is simple and efficient process.

CONVERGENCE OF WEIGHT VECTORAs with all adaptive Algorithms, the primary

concern with the LMS Algorithm is its convergence to the weight vector solution, where error E

[] is minimized.

NORMALISED LEAST MEAN SQUARES FILTER (NLMS)

main drawback of the "pure" LMS algorithm is that it is sensitive to the scaling of its input x(n).

This makes it very hard (if not impossible) to choose a learning rate that guarantees stability of

the algorithm.. The Normalised least mean squares filter (NLMS) is a variant of the LMS


18/83

18

algorithm that solves this problem by normalising with the power of the input. The NLMS

algorithm can be summarised as:

Parameters: p = filter order

= step size

Initialization:

Computation: For n = 0,1,2,...

3.1 OPTIMAL LEARNING RATE

It can be shown that if there is no interference (v(n) = 0), then the optimal learning rate for the

NLMS algorithm is

opt = 1

and is independent of the input x(n) and the real (unknown) impulse response . In the

general case with interference ( ), the optimal learning rate is

The results above assume that the signals v(n) and x(n) are uncorrelated to each other, which is

generally the case in practice.


19/83

19

3.2 PROOF

Let the filter misalignment be defined as , we can derive the

expected misalignment for the next sample as:

Let and

Assuming independence, we have:

The optimal learning rate is found at , which leads to:


20/83

20


21/83

21

CHAPTER-4

MEAN SQUARE ERROR


22/83

22

4.MEAN SQUARED ERRORIn statistics, the mean square error or MSE of an estimator is one of many ways to quantify the

difference between an estimator and the true value of the quantity being estimated. MSE is a risk

function, corresponding to the expected value of the squared error loss or quadratic loss. MSE

measures the average of the square of the "error." The error is the amount by which the estimator

differs from the quantity to be estimated. The difference occurs because of randomness or

because the estimator doesn't account for information that could produce a more accurate

estimate.

The MSE is the second moment (about the origin) of the error, and thus incorporates both the

variance of the estimator and its bias. For an unbiased estimator, the MSE is the variance. Likethe variance, MSE has the same unit of measurement as the square of the quantity being

estimated. In an analogy to standard deviation, taking the square root of MSE yields the root

mean squared error or RMSE, which has the same units as the quantity being estimated; for an

unbiased estimator, the RMSE is the square root of the variance, known as the standard error.

4.1 DEFINITION AND BASIC PROPERTIES

The MSE of an estimator with respect to the estimated parameter is defined as

The MSE is equal to the sum of the variance and the squared bias of the estimator

The MSE thus assesses the quality of an estimator in terms of its variation and unbiasedness.

Note that the MSE is not equivalent to the expected value of the absolute error.


23/83

23

Since MSE is an expectation, it is a scalar, and not a random variable. It may be a function of the

unknown parameter, but it does not depend on any random quantities. However, when MSE is

computed for a particular estimator of the true value of which is not known, it will be subject

to estimation error. In a Bayesian sense, this means that there are cases in which it may be treated

as a random variable.

4.2 ALTERNATIVE USAGE

The term mean squared error is sometimes used to refer to residual sum of squares, divided by

the number of observations. This is an observed quantity, whereas the definition above is a

function of an unknown parameter. For more details, see errors and residuals in statistics.

EXAMPLES

Suppose we have a random sample of size n from an identically distributed population,

.

Some commonly-used estimators of the true parameters of the population, and 2, are shown in

the following table (see notes for distribution requirements for the MSEs in the table related to

variance estimators).

True

valu

e

Estimator Mean squared error

=

= the unbiased estimator of the

population mean,


24/83

24

=

2

= the unbiased estimator of the

population variance,

=

2

= the biased estimator of the


=

2

= the biased estimator of the


1.The MSEs shown for the variance estimators assume i.i.d. so that

. The result for follows easily from the variance

that is 2n 2.

2.The general MSE expression for the unbiased variance estimator, without distribution

assumptions, is , where 4 is the fourth central

moment.[3]

3.Unbiased estimators may not produce estimates with the smallest total variation (as

measured by MSE): 's MSE is larger than 's MSE.


25/83

25

4.Estimators with the smallest total variation may produce biased estimates: typically

underestimates 2

by

INTERPRETATION

An MSE of zero, meaning that the estimator predicts observations of the parameter with

perfect accuracy, is the ideal and forms the basis for the least squares method of regression

analysis.

While particular values of MSE other than zero are meaningless in and of themselves, they may

be used for comparative purposes. Two or more statistical models may be compared using their

MSEs as a measure of how well they explain a given set of observations: The unbiased model

with the smallest MSE is generally interpreted as best explaining the variability in the

observations.

Both linear regression techniques such as analysis of variance estimate the MSE as part of the

analysis and use the estimated MSE to determine the statistical significance of the factors or

predictors under study. The goal of experimental design is to construct experiments in such a

way that when the observations are analyzed, the MSE is close to zero relative to the magnitude

of at least one of the estimated treatment effects.

MSE is also used in several stepwise regression techniques as part of the determination as to how

many predictors from a candidate set to include in a model for a given set of observations.

APPLICATIONS

y Minimizing MSE is a key criterion in selection estimators. Among unbiased estimators,

the minimal MSE is equivalent to minimizing the variance, and is obtained by the

MVUE. However, a biased estimator may have lower MSE; see estimator bias.


26/83

26

y In statistical modeling, the MSE is defined as the difference between the actual

observations and the response predicted by the model and is used to determine whether

the model does not fit the data or whether the model can be simplified by removing

terms.

NOISE

In common use, the word noise means any unwanted sound. In both analog and digital

electronics, noise is an unwanted perturbation to a wanted signal; it is called noise as a

generalization of the audible noise heard when listening to a weak radio transmission. Signal

noise is heard as acoustic noise if played through a loudspeaker; it manifests as 'snow' on a

television or video image. In signal processing or computing it can be considered unwanted data

without meaning; that is, data that is not being used to transmit a signal, but is simply produced

as an unwanted by-product of other activities. In Information Theory, however, noise is still

considered to be information. In a broader sense, film grain or even advertisements encountered

while looking for something else can be considered noise. In biology, noise can describe the

variability of a measurement around the mean, for example transcriptional noise describes the

variability in gene activity between cells in a population.

Noise can block, distort, change or interfere with the meaning of a message in both human and

electronic communication.

In many of these areas, the special case of thermal noise arises, which sets a fundamental lower

limit to what can be measured or signaled and is related to basic physical processes at the

molecular level described by well-established thermodynamics considerations, some of which

are expressible by simple formulae.

GAUSSIAN NOISE

Gaussian noise is statistical noise that has a probability density function (abbreviated pdf) of the

normal distribution (also known as Gaussian distribution). In other words, the values that the

noise can take on are Gaussian-distributed. It is most commonly used as additive white noise to

yield additive white Gaussian noise (AWGN).


27/83

27

CHAPTER-5

ADDITIVE WHITE GUASSIAN NOISE


28/83

28

5.ADDITIVE WHITE GAUSSIAN NOISE (AWGN)Additive white Gaussian noise (AWGN) is a channel model in which the only impairment to

communication is a linear addition of wideband or white noise with a constant spectral density

(expressed as watts per hertz of bandwidth) and a Gaussian distribution of amplitude. The model

does not account for fading, frequency selectivity, interference, nonlinearity or dispersion.

However, it produces simple and tractable mathematical models.

which are useful for gaining insight into the underlying behavior of a system before these other

phenomena are considered.

Wideband Gaussian noise comes from many natural sources, such as the thermal vibrations of

atoms in conductors (referred to as thermal noise or Johnson-Nyquist noise), shot noise, black

body radiation from the earth and other warm objects, and from celestial sources such as the Sun.

The AWGN channel is a good model for many satellite and deep space communication links. It

is not a good model for most terrestrial links because of multipath, terrain blocking, interference,

etc. However, for terrestrial path modeling, AWGN is commonly used to simulate background

noise of the channel under study, in addition to multipath, terrain blocking, interference, ground

clutter and self interference that modern radio systems encounter in terrestrial operation.

LINEAR FREQUENCY MODULATION (FM)

y Till now we've seen signals that do not change in frequency over time. How do we

modify the signal to obtain a time-varying frequency?

y A chirp signal is one that sweeps linearly from a low to a high frequency.

y Can we create such a signal by concatenating small sequences, each with a frequency that

is higher than the last?

y This approach will likely lead to problems lining up the phase of each segment so that

discontinuities aren't introduced in the resulting waveform (as seen below).


29/83

29

Figure : A signal made by concatenating sinusoids of different frequencies will result in

discontinuities if care is not taken to match the initial phase.


30/83


31/83

31

6.THE PROPOSED ALGORITHMIn the conventional LMS algorithm, the weight vector coefficients w(n) for the FIR filter are

updated according tothe formula.

w(n) = w(n -1) + e(n)y(n)where w(n) = [wo(n) w1(n):::wM(n)] (M+1 being the filter length), is the convergence

parameter (sometimes referred to as step-size), e(n) = d(n) - z(n) is the output error (z(n) being

the filter output), and d(n) is the reference signal). Note that z(n) = w(n-1)y T (n) = where

is the original signal and y(n) = [y(n) y(n - 1):::y(n - M)] is the input signal to the filter.

For the algorithm to be useful for a range of FM signals with different bandwidths (including

single-tone sinusoids), we first specify the centre frequency fm in the spectrum of interest. The

conventional LMS algorithm is then used (with a singletone of frequency fm) to find an optimal

value of at that frequency. This optimal value is used to update the timevarying convergence

parameter according to the following formula

(2)

whereis a decaying factor. We will consider the following decaying law:

(3)

where C, a, b are positive constants that will determine the magnitude and the rate of decrease

for. According to the above law, C has to be a positive number larger than 1. When C = 1,

will be equal to 1 and the new algorithm will be the same as the conventional LMS algorithm.

A summary of the time-varying LMS algorithm is shown below:

Z(n)=W(n-1)(n) (4)

e(n)=d(n)-z(n) (5)

(6)

(7)

W(n)=W(n-1)+ (8)


32/83

32

Fig. 1. Spectrum for LFM narrowband signal with fo = 100 Hz, Bandwidth = 100 Hz and Ts =

0.001.


33/83

33

CHAPTER-7

ADAPTIVE ECHO CANCELLERS


34/83

34

7.ADAPTIVE ECHO CANCELLERSAdaptive filtering techniques to reduce this unwanted echo, thus increasing communication

quality. These echoes can be very annoying to callers. A widely used technique to suppress

echoes is to employ adaptive echo cancellers.A technique to remove or cancel echoes is shown in Figure. The echo canceller mimics the

transfer function of the echo path (or room acoustic) to synthesize a replica of the echo, and then

subtracts that replica from the combined echo and near-end speech (or disturbance) signal to

obtain the near end signal alone. However, the transfer function is unknown in practice, and so it

must be identified. The solution to this problem is to use an adaptive filter the method used to

cancel the echo signal is known as adaptive filtering.

Adaptive filters are dynamic filters which iteratively alter their characteristics in order to achieve

an optimal desired output. An adaptive filter algorithmically alters its parameters in order to

minimize a unction of the difference between the desired output d(n) and its actual output y(n).

This function is known as the cost function of the adaptive algorithm. Figure shows a block

diagram of the adaptive echo cancellation system implemented throughout this thesis. Here the

filter H(n) represents the impulse response of the acoustic environment, W(n) represents the

adaptive filter used to cancel the echo signal. The adaptive filter aims to equate its output y(n) to

the desired output d(n) (the signal reverberated within the acoustic environment). At each

iteration the error signal, e(n)=d(n)-y(n), is fed back into the filter, where the filter characteristis.


35/83

35

CHOICE OF ALGORITHM

A wide variety of recursive algorithms have been developed in the literature for the operation of

linear adaptive filters, In the final analysis, the choice of one algorithm over another is

determined by one or more of the following factors

RATE OF CONVERGENCE

This is defined as the number of iterations required for the algorithm, in response to stationary

inputs, to converge close enough to the optimum wiener solution in the mean-square error

sense. A fast rate of convergence allows the algorithm to adapt rapidly to a stationary

environment of unknown statistics.

MISS ADJUSTMENT

For an algorithm of interest, this parameter provides a quantitative measure of the amount

which the final value of the mean-square error, averaged over an ensemble of adaptive filters,

deviates from the minimum mean-square error produced by the Wiener filter.

TRACKING

When an adaptive filtering algorithm operates in a non-stationary environment. The algorithm is

required to track statistical variations in the environment. Two contradictory features, however,

influence the tracking performance of the algorithm

(1) Rate of convergence, and

(2) steady-state fluctuation due to algorithm noise.

ROBUSTNESS

For an adaptive filter to be robust, small disturbances (I.e., disturbances with small energy) can

only result in small estimation errors. The disturbances may arise from a variety of factors,

internal or external to the filter.

COMPUTATIONAL REQUIREMENTS

Here the issues of concern include

(a) The number of operations (i.e., multiplications, divisions, and additions/ subtractions)

Required to make one complete iteration of the algorithm.

(b) The size of memory locations required to store the data and the program,

(c) The investment required to program the algorithm on a computer.


36/83

36

CHAPTER-8

APPROACH TO DEVELOP LINEARADAPTIVE FILTER STOCHASTIC

GRADIENT APPROACH


37/83

37

8. APPROACH TO DEVELOP LINEAR ADAPTIVE FILTER

STOCHASTIC GRADIENT APPROACH

The stochastic gradient approach uses a tapped-delay line, or transversal filter, as the structural

basis for implementing the linear adaptive filter. For the case of stationary inputs, the cost

function, also referred to as the index of performance, is defined as the mean square error (i.e.,

the mean square value of the difference between the desired response and the transversal filter

output). This cost function is precisely a second order function of the tap weights in the

transversal filter.

To develop a recursive algorithm for updating the tap weights of the adaptive transversal filter,

we proceed in two stages, First, we use an iterative procedure to solve the Wiener Hopfequations (i.e., the Matrix equation defining the optimum Wiener solution); the iterative

procedure is based on the method of steepest descent, which is a well known technique in

optimization theory. This method required the use of a gradient vector, the value of which

depends on two parameters: the correlation Matrix of the tap inputs in the transversal filter and

the cross correlation vector between the desired response and the same tap inputs. Next, we use

instantaneous values for this correlation, so as to derive an estimate for the gradient vector,

making it assume a stochastic character in general.

The resulting algorithm is widely known as the least mean square (LMS) algorithm, , the essence

of which for the case of a transversal filter operating on real valued data may be described as

Where the error signal is defined as the difference between some desired response and the actual

response of the transversal filter produced by the tap input vector.


38/83

38

LEAST MEAN SQUARE (LMS) ALGORITHM

The Least Mean Square (LMS) algorithm was first developed by Widrow and Hoff in1959

through their studies of pattern recognition. From there it has become one of the most widely

used algorithms in adaptive filtering. The LMS algorithm is an important member of the family

of stochastic gradient-based algorithms as it utilizes the gradient vector of the filter tap weights

to converge on the optimal wiener solution. It is well known and widely used due to its

computational simplicity. It is this simplicity that has made it the benchmark against which all

other adaptive filtering algorithms are judged.

The LMS algorithm is a linear adaptive filter algorithm, which in general consists of two basic

processes.

1. A filter process: This involves

a. Computing the output of a linear filter in response to an input signal.

b. Generating an estimation error by comparing this output with a desired response.

2. An adaptive process which involves the automatic adjustment of the

Parameters of the filter in accordance with the estimation error.

The combination of these two processes working together constitutes a feedback loop; First, we

have a transversal filter, around which the LMS algorithm is built. This component is responsible

for performing the filtering process. Second, we have a mechanism for performing the adaptive

control process on the tap weights of the transversal filter. With each iteration of the LMS

algorithm, the filter tap weights of the adaptive filter are updated according to the following

formula (Farhang-Boroujeny 1999).

w (n +1) = w(n) + 2ex(n)

Here x(n) is the input vector of time delayed input values,

x(n) = [x(n) x(n-1) x(n-2) .x(n- N+1)]


39/83

39

The vector represents the coefficients of the adaptive FIR filter tap weight vector at time n. The

parameter is known as the step size parameter and is a small positive constant. This step size

parameter controls the influence of the updating factor.

Selection of a suitable value for is imperative to the performance of the LMS algorithm, if the

value is too small the time the adaptive filter takes to converge on the optimal solution will be

too long; if is too large the adaptive filter becomes unstable and its output diverges. Is the

simplest to implement and is stable when the step size parameter is selected appropriately. This

requires prior knowledge of the input signal which is not feasible for the echo cancellation

system.

DERIVATION OF THE LMS ALGORITHM

The derivation of the LMS algorithm builds upon the theory of the wiener solution for the

optimal filter tap weights, Wo . It also depends on the steepest-descent algorithm. This is a

formula which updates the filter coefficients using the current tap weight vector and the current

gradient of the cost function with respect to the filter tap weight coefficient vector ,

As the negative gradient vector points in the direction of steepest descent for the N-dimensionalquadratic cost function, each recursion shifts the value of the filter coefficients closer toward

their optimum value, which corresponds to the minimum achievable value of the cost function,

(n).The LMS algorithm is a random process implementation of the steepest descent algorithm.

Here the expectation for the error signal is not known so the instantaneous value is used as an

estimate. The steepest descent algorithm then becomes

The gradient of the cost function,(n), can alternatively be expressed in the following

.


40/83

40

Substituting this into the steepest descent algorithm of Eqn , we arrive at the recursion for the

LMS adaptive algorithm.

w (n +1) = w(n) + 2ex(n)


41/83

41

CHAPTER-9

IMPLEMENTATION OF THE LMS

ALGORITHM


42/83

42

9.IMPLEMENTATION OF THE LMS ALGORITHM

Each iteration of the LMS algorithm requires 3 distinct steps in this order:

1.The output of the FIR filter, y(n) is calculated using equation

2.The value of the error estimation is calculated using equation

3.The tap weights of the FIR vector are updated in preparation for the next iteration,

by

The main reason for the LMS algorithms popularity in adaptive filtering is its

Computational simplicity, making it easier to implement than all other commonly use adaptive

algorithms. For each iteration the LMS algorithm requires 2Nadditions and 2N+1 multiplications

(N for calculating the output, y(n), one for 2e(n) and an additional N for the scalar by vector

multiplication).


43/83

43

CHAPTER-10

SPEECH SIGNALS


44/83

44

10. SPEECH SIGNALS

A speech signal consists of three classes of sounds. They are voice, fricative and plosive sounds.

Voiced sounds are caused by excitation of the vocal tract with quasi-periodic pulses of airflow.

Fricative sounds are formed by constricting the vocal tract and passing air through it, causing

Turbulence that result in a noise-like sound. Plosive sounds are created by closing up the vocal

tract, building up air behind it then suddenly releasing it; this is heard in the sound made by the

letter Figure shows a discrete time representation of a speech signal.

By looking at it as a whole we can tell that it is non-stationary. That is, its mean values vary

with time and cannot be predicted using the above mathematical models for random processes.

However, a speech signal can be considered as a linear composite of the above three classes of

sound, each of these sounds are stationary and remain fairly constant over intervals of the orderof 30 to 40 ms. The theory behind the derivations of many adaptive filtering algorithms usually

requires the input signal to be stationary. Although speech is non-stationary for all time, it is an

assumption that the short term stationary behavior outlined above will prove adequate for the

adaptive filters to function as desired

Representation of Speech Signal

SPEECH GENERATION

Speech generation and recognition are used to communicate between humans and machines.

Rather than using your hands and eyes, you use your mouth and ears. This is very convenient

when your hands and eyes should be doing something else, such as: driving a car, performing

surgery, or (unfortunately) firing your weapons at the enemy. Two approaches are used for


45/83

45

computer generated speech: digital recording and vocal tract simulation. In digital recording, the

voice of a human speaker is digitized and stored, usually in a compressed form.

During playback, the stored data are uncompressed and converted back into an analog signal. An

entire hour of recorded speech requires only about three megabytes of storage, well within the

capabilities of even small computer systems. This is the most common method of digital speech

generation used today. Vocal tract simulators are more complicated, trying to mimic the physical

mechanisms by which humans create speech. The human vocal tract is an acoustic cavity with

resonate frequencies determined by the size and shape of the chambers. Sound originates in the

vocal tract in one of two basic ways, called voiced and fricative sounds. With voiced sounds,

vocal cord vibration produces near periodic pulses of air into the vocal cavities. In comparison,

fricative sounds originate from the noisy air turbulence at narrow constrictions, such as the teeth

and lips. Vocal tract simulators operate by generating digital signals that resemble these two

types of excitation. The characteristics of the resonate chamber are simulated by passing the

excitation signal through a digital filter with similar resonances. This approach was used in one

of the very early DSP success stories, the Speak & Spell, a widely sold electronic learning aid for

children.

SPEECH PRODUCTION

Speech is produced when air is forced from the lungs through the vocal cords and along the

vocal tract. The vocal tract extends from the opening in the vocal cords (called the glottis) to the

mouth, and in an average man is about 17 cm long. It introduces short-term correlations (of the

order of 1 ms) into the speech signal, and can be thought of as a filter with broad resonances

called formants. The frequencies of these formants are controlled by varying the shape of the

tract, for example by moving the position of the tongue. An important part of many speech

codecs is the modeling of the vocal tract as a short-term filter. As the shape of the vocal tract

varies relatively slowly, the transfer function of its modeling filter needs to be updated only

relatively infrequently (typically every 20 ms or so).

The vocal tract filter is excited by air forced into it through the vocal cords. Speech sounds can

be broken into three classes depending on their mode of excitation.

y Voiced sounds are produced when the vocal cords vibrate open and closed, thus

interrupting the flow of air from the lungs to the vocal tract and producing quasi-periodic


46/83

46

pulses of air as the excitation. The rate of the opening and closing gives the pitch of the

sound. Varying the shape of, and the tension in, the vocal cords, and the pressure of the

air behind them can adjust this. Voiced sounds show a high degree of periodicity at the

pitch period, which is typically between 2 and 20 ms. This long-term periodicity can be

seen in Figure 1 which shows a segment of voiced speech sampled at 8 kHz. Here the

pitch period is about 8 ms or 64 samples.

y Unvoiced sounds result when the excitation is a noise-like turbulence produced by

forcing air at high velocities through a constriction in the vocal tract while the glottis is

held open. Such sounds show little long-term periodicity as can be seen from Figures 3

and 4 although short-term correlations due to the vocal tract are still present.

y Plosive sounds result when a complete closure is made in the vocal tract, and air pressure

is built up behind this closure and released suddenly.

Some sounds cannot be considered to fall into any one of the three classes above, but are a

mixture. For example voiced fricatives result when both vocal cord vibration and a constriction

in the vocal tract are present.

Although there are many possible speech sounds which can be produced, the shape of the vocal

tract and its mode of excitation change relatively slowly, and so speech can be considered to be

quasi-stationary over short periods of time (of the order of 20 ms). Speech signals show a high

degree of predictability, due sometimes to the quasi-periodic vibrations of the vocal cords and

also due to the resonances of the vocal tract. Speech coders attempt to exploit this predictability

in order to reduce the data rate necessary for good quality voice transmission

From the technical, signal-oriented point of view, the production of speech is widely described

as a two-level process. In the first stage the sound is initiated and in the second stage it is filtered

on the second level. This distinction between phases has its orgin in the source-filter model of

speech production.


47/83

47

Fig: Source Filter Model of Speech Production

The basic assumption of the model is that the source signal produced at the glottal level is

linearly filtered through the vocal tract. The resulting sound is emitted to the surrounding air

through radiation loading (lips). The model assumes that source and filter are independent of

each other. Although recent findings show some interaction between the vocal tract and a glottal

source (Rothenberg 1981; Fant 1986), Fant's theory of speech production is still used as a

framework for the description of the human voice, especially as far as the articulation of vowels

is concerned.

SPEECH PROCESSING

The term speech processing basically refers to the scientific discipline concerning the analysis

and processing of speech signals in order to achieve the best benefit in various practical

scenarios. The field of speech processing is, at present, undergoing a rapid growth in terms of

both performance and applications. The advances being made in the field of microelectronics,

computation and algorithm design stimulate this. Nevertheless, speech processing still covers an

extremely broad area, which relates to the following three engineering applications:

Speech Coding and transmission that is mainly concerned with man-to man voice

communication;

Speech Synthesis which deals with machine-to-man communications;

Speech Recognition relating to man-to machine communication.

Speech Coding: Speech coding or compression is the field concerned with compact digital

representations of speech signals for the purpose of efficient transmission or storage. The central

objective is to represent a signal with a minimum number of bits while maintaining perceptual

quality. Current applications for speech and audio coding algorithms include cellular and

personal communications networks (PCNs), teleconferencing, desktop multi-media systems, and

secure communications.

SPEECH SYNTHESIS

The process that involves the conversion of a command sequence or input text (words or

sentences) into speech waveform using algorithms and previously coded speech data is known as


48/83

48

speech synthesis. The inputting of text can be processed through by keyboard, optical character

recognition, or from a previously stored database. A speech synthesizer can be characterized by

the size of the speech units they concatenate to yield the output speech as well as by the method

used to code, store and synthesize the speech. If large speech units are involved, such as phrases

and sentences, high-quality output speech (with large memory requirements) can be achieved.

On the contrary, efficient coding methods can be used for reducing memory needs, but these

usually degrade speech quality.

NOISE SOURCES

Sources of noise exist throughout the environment. One type of noise is due to turbulence and is

therefore totally random and impossible to predict. Engineers like to look at signals, noise

included, in the frequency domain. That is, "How is the noise energy distributed as a function of

frequency?"

These turbulent noises tend to distribute their energy evenly across the frequency bands and are

therefore referred to as "Broadband Noise". Very commonly we come across a word white

noise white noise comes under the category of Broadband Noise. White Noise is a noise having

a frequency spectrum that is continuous and uniform over a specified frequency band.

Note: White noise has equal power per hertz over the specified frequency band. (Synonym

additive white Gaussian noise) Examples of broadband noise are the low frequency noise from

jet planes and the impulse noise of an explosion.

A large number of environmental noises are different. These "Narrow Band Noises" concentrate

most of their noise energy at specific frequencies. When the source of the noise is a rotating or

repetitive machine, the noise frequencies are all multiples of a basic "Noise Cycle" and the noise

is approximately periodic. This "Tonal Noise" is common in the environment as manmade

machinery tends to generate it (along with a smaller amount of broadband noise) at increasingly

high levels.


49/83

49

Chapter-11

Filter design


50/83

50

11. Filter design

Filter design is the process of designing a filter (in the sense in which the term is used in signal

processing, statistics, and applied mathematics), often a linear shift-invariant filter, which

satisfies a set of requirements, some of which are contradictory. The problem is to find a

realization of the filter which meets each of the requirements to a sufficient degree to make it

useful.

The filter design process can be described as an optimization problem where each requirement

contributes with a term to an error function which should be minimized. Certain parts of the

design process can be automated, but normally an experienced electrical engineer is needed to

get a good result.

Typical design requirements

Typical requirements which are considered in the design process are:

y The filter should have a specific frequency response

y The filter should have a specific impulse response

y The filter should be causal

y The filter should be stable

y The filter should be localized

y The computational complexity of the filter should be low

y The filter should be implemented in particular hardware or software

The frequency function

Typical examples of frequency function are"

y A low-pass filter is used to cut unwanted high-frequency signals.

y A high-pass filter passes high frequencies fairly well; it is helpful as a filter to cut any unwanted

low frequency components.

y A band-pass filter passes a limited range of frequencies.


51/83

51

y A band-stop filter passes frequencies above and below a certain range. A very narrow band-stop

filter is known as a notch filter.

y A low-shelf filter passes all frequencies, but increasing or reducing frequencies below the cutoff

frequency by specified amount.

y A high-shelf filter passes all frequencies, but increasing or reducing frequencies above the cutoff

frequency by specified amount.

y A peak EQ filter makes a peak or a dip in the frequency response, commonly used in graphic

equalizers.

y An all-pass filter passes through all frequencies unchanged, but changes the phase of the signal.

This is a filter commonly used in phaser effects.

An important parameter is the required frequency response. In particular, the steepness and

complexity of the response curve is a deciding factor for the filter order and feasibility.

A first order recursive filter will only have a single frequency-dependent component. This means

that the slope of the frequency response is limited to 6 dB per octave. For many purposes, this is

not sufficient. To achieve steeper slopes, higher order filters are required.

In relation to the desired frequency function, there may also be an accompanying weighting

function which describes, for each frequency, how important it is that the resulting frequency

function approximates the desired one. The larger weight, the more important is a closeapproximation.

The impulse response

There is a direct correspondence between the filter's frequency function and its impulse response,

the former is the Fourier transform of the latter. This means that any requirement on the

frequency function is a requirement on the impulse response, and vice versa.

However, in certain applications it may be the filter's impulse response which is explicit and the

design process then aims at producing as close an approximation as possible to the requested

impulse response given all other requirements.


52/83

52

In some cases it may even be relevant to consider a frequency function and impulse response of

the filter which are chosen independently from each other. For example, we may both want a

specific frequency function of the filter and that the resulting filter have a small effective width

in the signal domain as possible. The latter condition can be realized by considering a very

narrow function as the wanted impulse response of the filter even though this function has no

relation to the desired frequency function. The goal of the design process is then to realize a filter

which tries to meet both these contradicting design goals as much as possible.

Causality

In order to be implementable, any time-dependent filter must be causal: the filter response only

depends on the current and past inputs. A standard approach is to leave this requirement until the

final step. If the resulting filter is not causal, it can be made causal by introducing an appropriate

time-shift (or delay). If the filter is a part of a larger system (which it normally is) these types of

delays have to be introduced with care since they affect the operation of the entire system.

Stability

A stable filter assures that every limited input signal produces a limited filter response. A filter

which does not meet this requirement may in some situations prove useless or even harmful.

Certain design approaches can guarantee stability, for example by using only feed-forward

circuits such as an FIR filter. On the other hand, filter based on feedback circuits have other

advantages and may therefore be preferred, even if this class of filters include unstable filters. In

this case, the filters must be carefully designed in order to avoid instability.

Locality

In certain applications we have to deal with signals which contain components which can bedescribed as local phenomena, for example pulses or steps, which have certain time duration. A

consequence of applying a filter to a signal is, in intuitive terms, that the duration of the local

phenomena is extended by the width of the filter. This implies that it is sometimes important to

keep the width of the filter's impulse response function as short as possible.


53/83

53

According to the uncertainty relation of the Fourier transform, the product of the width of the

filter's impulse response function and the width of its frequency function must exceed a certain

constant. This means that any requirement on the filter's locality also implies a bound on its

frequency function's width. Consequently, it may not be possible to simultaneously meet

requirements on the locality of the filter's impulse response function as well as on its frequency

function. This is a typical example of contradicting requirements.

Computational complexity

A general desire in any design is that the number of operations (additions and multiplications)

needed to compute the filter response is as low as possible. In certain applications, this desire is a

strict requirement, for example due to limited computational resources, limited power resources,

or limited time. The last limitation is typical in real-time applications.

There are several ways in which a filter can have different computational complexity. For

example, the order of a filter is more or less proportional to the number of operations. This

means that by choosing a low order filter, the computation time can be reduced.

For discrete filters the computational complexity is more or less proportional to the number of

filter coefficients. If the filter has many coefficients, for example in the case of multidimensional

signals such as tomography data, it may be relevant to reduce the number of coefficients by

removing those which are sufficiently close to zero.

Another issue related to computational complexity is separability, that is, if and how a filter can

be written as a convolution of two or more simpler filters. In particular, this issue is of

importance for multidimensional filters, e.g., 2D filter which are used in image processing. In

this case, a significant reduction in computational complexity can be obtained if the filter can be

separated as the convolution of one 1D filter in the horizontal direction and one 1D filter in thevertical direction. A result of the filter design process may, e.g., be to approximate some desired

filter as a separable filter or as a sum of separable filters.


54/83

54

Other considerations

It must also be decided how the filter is going to be implemented:

y Analog filtery Analog sampled filter

y Digital filter

y Mechanical filter

Analog filters

The design of linear analog filters is for the most part covered in the linear filter section.

Digital filters

Digital filters are classified into one of two basic forms, according to how they respond to an unit

impulse:

y Finite impulse response, orFIR, filters express each output sample as a weighted sum of

the last N inputs, where N is the order of the filter. Since they do not use feedback, they

are inherently stable. If the coefficients are symmetrical (the usual case), then such a filter

is linear phase, so it delays signals of all frequencies equally. This is important in many

applications. It is also straightforward to avoid overflow in an FIR filter. The main

disadvantage is that they may require significantly more processing and memory

resources than cleverly designed IIR variants. FIR filters are generally easier to design

than IIR filters - the Remez exchange algorithm is one suitable method for designing

quite good filters semi-automatically. (See Methodology.)

y Infinite impulse response, orIIR, filters are the digital counterpart to analog filters. Such

a filter contains internal state, and the output and the next internal state are determined by

a linear combination of the previous inputs and outputs (in other words, they use

feedback, which FIR filters normally do not). In theory, the impulse response of such a

filter never dies out completely, hence the name IIR, though in practice, this is not true

given the finite resolution of computer arithmetic. IIR filters normally require less


55/83

55

computing resources than an FIR filter of similar performance. However, due to the

feedback, high order IIR filters may have problems with instability, arithmetic overflow,

and limit cycles, and require careful design to avoid such pitfalls. Additionally, since the

phase shift is inherently a non-linear function of frequency, the time delay through such a

filter is frequency-dependent, which can be a problem in many situations. 2nd order IIR

filters are often called 'biquads' and a common implementation of higher order filters is to

cascade biquads. A useful reference for computing biquad coefficients is the RBJ Audio

EQ Cookbook.

Sample rate

Unless the sample rate is fixed by some outside constraint, selecting a suitable sample rate is an

important design decision. A high rate will require more in terms of computational resources, but

less in terms of anti-aliasing filters. Interference[disambiguation needed] and beating with other signals

in the system may also be an issue.

Anti-aliasing

For any digital filter design, it is crucial to analyze and avoid aliasing effects. Often, this is done

by adding analog anti-aliasing filters at the input and output, thus avoiding any frequency

component above the Nyquist frequency. The complexity (i.e., steepness) of such filters depends

on the required signal to noise ratio and the ratio between the sampling rate and the highest

frequency of the signal.

Theoretical basis

Parts of the design problem relate to the fact that certain requirements are described in the

frequency domain while others are expressed in the signal domain and that these may contradict.

For example, it is not possible to obtain a filter which has both an arbitrary impulse response and

arbitrary frequency function. Other effects which refer to relations between the signal and

frequency domain are

y The uncertainty principle between the signal and frequency domains


56/83

56

y The variance extension theorem

y The asymptotic behaviour of one domain versus discontinuities in the other

The uncertainty principle

As stated in the uncertainty principle, the product of the width of the frequency function and the

width of the impulse response cannot be smaller than a specific constant. This implies that if a

specific frequency function is requested, corresponding to a specific frequency width, the

minimum width of the filter in the signal domain is set. Vice versa, if the maximum width of the

response is given, this determines the smallest possible width in the frequency. This is a typical

example of contradicting requirements where the filter design process may try to find a useful

compromise.

The variance extension theorem

Let be the variance of the input signal and let be the variance of the filter. The variance

of the filter response, , is then given by

= +

This means that r> fand implies that the localization of various features such as pulses or

steps in the filter response is limited by the filter width in the signal domain. If a precise

localization is requested, we need a filter of small width in the signal domain and, via the

uncertainty principle, its width in the frequency domain cannot be arbitrary small.

Discontinuities versus asymptotic behaviour

Let f(t) be a function and let F() be its Fourier transform. There is a theorem which states that if

the first derivative of F which is discontinuous has order , then f has an asymptotic decay

like t n 1

.

A consequence of this theorem is that the frequency function of a filter should be as smooth as

possible to allow its impulse response to have a fast decay, and thereby a short width.


57/83

57

Chapter-12

METHODOLOGY


58/83


59/83

59

Simultaneous optimization in both domains

The previous method can be extended to include an additional error term related to a desired

filter impulse response in the signal domain, with a corresponding weighting function. The ideal

impulse response can be chosen independently of the ideal frequency function and is in practice

used to limit the effective width and to remove ringing effects of the resulting filter in the signal

domain. This is done by choosing a narrow ideal filter impulse response function, e.g., an

impulse, and a weighting function which grows fast with the distance from the origin, e.g., the

distance squared. The optimal filter can still be calculated by solving a simple least squares

problem and the resulting filter is then a "compromise" which has a total optimal fit to the ideal

functions in both domains. An important parameter is the relative strength of the two weighting

functions which determines in which domain it is more important to have a good fit relative tothe ideal function.

FINITE IMPULSE RESPONSE:

y A finite impulse response (FIR) filter is a type of a digital filter. The impulse response,

the filter's response to a Kronecker delta input, is finite because it settles to zero in a finite

number of sample intervals. This is in contrast to infinite impulse response (IIR) filters,

which have internal feedback and may continue to respond indefinitely. The impulseresponse of an Nth-order FIR filter lasts for N+1 samples, and then dies to zero.

The difference equation that defines the output of an FIR filter in terms of its input is:

where:

y x[n] is the input signal,

y y[n] is the output signal,

y bi are the filter coefficients, and

y Nis the filter order an Nth-order filter has (N+ 1) terms on the right-hand side; these

are commonly referred to as taps.


60/83

60

This equation can also be expressed as a convolution of the coefficient sequence bi with the input

signal:

That is, the filter output is a weighted sum of the current and a finite number of previous values

of the input.

An FIR filter has a number of useful properties which sometimes make it preferable to an infinite

impulse response (IIR) filter. FIR filters:

1.Are inherently stable. This is due to the fact that all the poles are located at the origin and

thus are located within the unit circle.

2.Require no feedback. This means that any rounding errors are not compounded by summed

iterations. The same relative error occurs in each calculation. This also makes implementation

simpler.

3.They can easily be designed to be linear phase by making the coefficient sequence

symmetric; linear phase, or phase change proportional to frequency, corresponds to equal delayat all frequencies. This property is sometimes desired for phase-sensitive applications, for

example crossover filters, and mastering.

The main disadvantage of FIR filters is that considerably more computation power is required

compared to an IIR filter with similar sharpness or selectivity, especially when low frequencies

(relative to the sample rate) cutoffs are needed.


61/83

61

CHAPTER-13

IMPULSE RESPONSE


62/83


63/83

63

filter specifications refer to the frequency response of the filter. There are different methods to

find the coefficients from frequency specifications:

1.Window design method

2.Frequency Sampling method

3.Weighted least squares design

4.Parks-McClelland method (also known as the Equiripple, Optimal, or Minimax method).

The Remez exchange algorithm is commonly used to find an optimal equiripple set of

coefficients. Here the user specifies a desired frequency response, a weighting function

for errors from this response, and a filter orderN. The algorithm then finds the set of (N+

1) coefficients that minimize the maximum deviation from the ideal. Intuitively, this finds

the filter that is as close as you can get to the desired response given that you can useonly (N+ 1) coefficients. This method is particularly easy in practice since at least one

text[1] includes a program that takes the desired filter and N, and returns the optimum

coefficients.

Software packages like MATLAB, GNU Octave, Scilab, and SciPy provide convenient ways to

apply these different methods.

Some of the time, the filter specifications refer to the time-domain shape of the input signal thefilter is expected to "recognize". The optimum matched filter is to sample that shape and use

those samples directly as the coefficients of the filter -- giving the filter an impulse response that

is the time-reverse of the expected input signal.

WINDOW DESIGN METHOD

In the Window Design Method, one designs an ideal IIR filter, then applies a window function to

it in the time domain, multiplying the infinite impulse by the window function. This results in

the frequency response of the IIR being convolved with the frequency response of the window

function[2]

thus the imperfections of the FIR filter (compared to the ideal IIR filter) can be

understood in terms of the frequency response of the window function.


64/83

64

The ideal frequency response of a window is a Dirac delta function, as that results in the

frequency response of the FIR filter being identical to that of the IIR filter, but this is not

attainable for finite windows, and deviations from this yield differences between the FIR

response and the IIR response.

13.1 MOVING AVERAGE EXAMPLE

Block diagram of a simple FIR filter (2nd-order/3-tap filter in this case, implementing a moving

average)

A moving average filter is a very simple FIR filter. The filter coefficients are found via the

following equation:

for

To provide a more specific example, we select the filter order:

The impulse response of the resulting filter is:

The following figure shows the block diagram of such a 2nd-order moving-average filter.

To discuss stability and spectral topics we take the z-transform of the impulse response:


65/83

65

The following figure shows the pole-zero diagram of the filter. Two poles are located at the

origin, and two zeros are located at ,

The frequency response, for frequency in radians per sample, is:

The following figure shows the absolute value of the frequency response. Clearly, the moving-

average filter passes low frequencies with a gain near 1, and attenuates high frequencies. This is


66/83


67/83

67


68/83

68

CHAPTER-14

INFINITE IMPULSE RESPONCE


69/83

69

14. INFINITE IMPULSE RESPONSE

Infinite impulse response (IIR) is a property of signal processing systems. Systems with this property

are known as IIR systems or, when dealing with filter systems, as IIR filters. IIR systems have an impulse

response function that is non-zero over an infinite length of time. This is in contrast to finite impulse

response filters (FIR), which have fixed-duration impulse responses. The simplest analog IIR filter is an

RC filter made up of a single resistor (R) feeding into a node shared with a single capacitor (C). This filter

has an exponential impulse response characterized by an RC time constant.IIR filters may be

implemented as either analog or digital filters. In digital IIR filters, the output feedback is immediately

apparent in the equations defining the output. Note that unlike with FIR filters, in designing IIR filters it

is necessary to carefully consider "time zero" case in which the outputs of the filter have not yet been

clearly defined.Design of digital IIR filters is heavily dependent on that of their analog counterparts

because there are plenty of resources, works and straightforward design methods concerning analog

feedback filter design while there are hardly any for digital IIR filters. As a result, usually, when a digital

IIR filter is going to be implemented, an analog filter (e.g. Chebyshev filter, Butterworth filter, Elliptic

filter) is first designed and then is converted to a digital filter by applying discretization techniques such

as Bilinear transform or Impulse invariance.

y Example IIR filters include the Chebyshev filter, Butterworth filter, and the Bessel filter.

TRANSFER FUNCTION DERIVATION

Digitals filters are often described and implemented in terms of the difference equation that

defines how the output signal is related to the input signal:

where:

y is the feedforward filter order

y are the feedforward filter coefficients


70/83

70

y is the feedback filter order

y are the feedback filter coefficients

y is the input signal

y is the output signal.

A more condensed form of the difference equation is:

which, when rearranged, becomes:

To find the transfer function of the filter, we first take the Z-transform of each side of the above

equation, where we use the time-shift property to obtain:

We define the transfer function to be:

Considering that in most IIR filter designs coefficient is 1, the IIR filter transfer function takes

the more traditional form:


71/83

71

CHAPTER-15

DESCRIPTION OF BLOCK DIAGRAM


72/83

72

15. DESCRIPTION OF BLOCK DIAGRAM

Simple IIR filter block diagram

A typical block diagram of an IIR filter looks like the following. The z 1

block is a unit delay.

The coefficients and number of feedback/feedforward paths are implementation-dependent.

Stability

The transfer function allows us to judge whether or not a system is bounded-input, bounded-

output (BIBO) stable. To be specific, the BIBO stability criteria requires the ROC of the system

include the unit circle. For example, for a causal system, all poles of the transfer function have tohave an absolute value smaller than one. In other words, all poles must be located within a unit

circle in the z-plane.

The poles are defined as the values ofzwhich make the denominator ofH(z) equal to 0:

Clearly, if then the poles are not located at the origin of the z-plane. This is in contrast to

the FIR filter where all poles are located at the origin, and is therefore always stable.


73/83

73

IIR filters are sometimes preferred over FIR filters because an IIR filter can achieve a much

sharper transition region roll-off than FIR filter of the same order.

Example

Let the transfer function of a filterHbe

with ROC a < | z| and 0 < a < 1

which has a pole at a, is stable and causal. The time-domain impulse response is

h(n) = anu(n)

which is non-zero forn > = 0.


74/83

74

CHAPTER-16

SIMULATION RESULTS


75/83

75

16. SIMULATION RESULTS

In this section, we shall present simulation results to evaluate the performance of the proposed

algorithm using Matlab. In this simulation, the input signal for both algorithms has the

form y(t) = x(t) + n(t), n(t)being white Gaussian noise with 1dB power and x(t)is the original

signal assumed to be a finite-length LFM signal of the form

X(t)=cos(

(9)

where = 2 fo is a constant (initial frequency), taken here as 100 Hz, T is the signal duration,

andEis the modulation index which will determine the bandwidth of LFM signal.

Fig. (1) shows the spectrum for an LFM narrowband signal that will be used in the simulation,

where fm is its mean frequency. The bandwidth BW of this LFM signal can be adjusted by

varying the parameter . Increasing Ewill increase the signal bandwidth, as can be numerically

shown using the relationships

dw (10)

BW=

(11)

where X(f)is the Fourier transform ofx(t).

The mean squared error (MSE) for each convergence parameter is calculated as follows:

MSE=

(12)

where x(n)is the original signal and is the filter output, which represents an estimate of the

input signal.

Fig. (2) shows the MSE for different LFM signals using the conventional and the time varying

LMS algorithms. The performance of the conventional LMS algorithm varies depending

on the LFM signal bandwidth. However, the optimal value forin the case of LFM is still

located in the lower region, within the range of 0.0001 to 0.0005. As a result, if the algorithm is

to be used with LFM signals with the same centre frequency (but probably with different

bandwidths), we should choose from this range for the time-varying LMS algorithm.


76/83

76

Fig. 2. MSE for conventional LMS algorithm (filter order = 100, Ts

= 0.001, SNR = 1 dB, fo = 100 Hz).


77/83


78/83

78

signals are different. Theoptimal for a single-tone sinusoid is in the range 0.15*to

0.2*, and for an LFM narrowband signal of 50 Hz isaround 0.4*

Fig. (5) shows the estimation curve when the time-varyingLMS algorithm is used for noise

reduction in narrowband FMsignals. The curve in Fig. (5) is the estimated output errore(n)

from equation (5). In general, the time-varying LMSalgorithm provides faster convergence than

the conventionalLMS algorithm (C = 1). Fig. (5) also shows the effect of the parameterCas a

convergence controlling factor. Larger C willprovide faster convergence.

Fig. (6), Fig. (7), and Fig. (8) show the mean-squared errorversus the number of samples N for

LFM signals with different bandwidths and different values ofC. These figures showthat the

time-varying LMS algorithm provides better MSE.It can be concluded that the time-varying

LMS algorithmprovides better MSE performance for a larger bandwidth.

Fig. (9) and Fig. (10) show the convergence of thetime-varying LMS algorithm to a limit of

MSE = 0.05 usingLFM signal with a bandwidth of 50 Hz and MSE = 0.08 foranother LFM

signal with a bandwidth of 100 Hz. Fig. (9)and Fig. (10) also show that the time-varying LMS

algorithmprovides faster convergence for larger C.

Fig. 4. MSE performance comparison for narrowband LFM of 50 Hz and signal-tone at 125 Hz

for both conventional LMS and timevarying LMS algorithm (filter order = 100, Ts = 0.001, SNR

= 1 dB).


79/83

79

Fig. 5. The effect of parameterCon estimation error curve using the time-varying LMS for noise

reduction in narrowband signals (filter order = 10, _o = 0.001, SNR = 2 dB, fo = 100 Hz, and

LFM bandwidth = 50 Hz).

Fig. 6. MSE vs number of samples for different Cvalues (filter order = 100, _o = 0.0002, SNR =

1 dB , single-tone at 125 Hz, a = 0.01 , and b =0.7).


80/83

80

Fig. 7. MSE vs Number of sample for differenre Cused (filter order = 100, _o = 0.0002, SNR =

1 dB , fo = 100, LFM bandwidth = 50 Hz, a = 0.01, b = 0.7).


81/83

81

CHAPTER-17

CONCLUSION & REFRENCES


82/83

82

17. CONCLUSIONS

A new structure for the LMS algorithm with a decaying, time-varying law for the convergence

parameter has been proposed. In a stationary white Gaussian noise environment, simulations

show that the time-varying LMS algorithm provides faster convergence than the conventional

LMS algorithm and a smaller mean-squared error (MSE) which is close to the optimal value. The

algorithm is based on selecting the optimal value of the convergence parameter using a single

tone sinusoid with a frequency that equals the centre frequency of expected LFM signals,

assuming they are narrowband. The best decay controlling factor is bandwith-dependent. Further

study of different decay laws is needed to extend the algorithm to deal with non-linear FM

signals.


83/83

REFERENCES[1] S. Haykin, Adaptive filter theory,Prentice Hall, 1986.

[2] Monson H. Hayes, Statistical digital signal processing and modeling

John Wiley & Sons, 1996.

[3] L. Cohen, Time-Frequency Anaysis, Prentice-Hall, 1995.

Date post:	08-Apr-2018
Category:	Documents
Upload:	pawan-kalyan-kalyan
View:	226 times
Download:	0 times

MAIN DOCUMENTATION.docx

Documents