Date post: | 06-Apr-2018 |
Category: |
Documents |
Upload: | krupesh-shah |
View: | 214 times |
Download: | 0 times |
of 40
8/3/2019 GCET-EC-MAA-20-44-48
1/40
MULTIRATE ADAPTIVE FILTERING
A PROJECT REPORT
Submitted by
NEERAJ LAD 080110111020
PRITESH PRAJAPATI 080110111044
KRUPESH SHAH 080110111048
In fulfillment for the award of the degree
of
BACHELOR OF ENGINEERING
in
ELECTRONICS AND COMMUNICATION
G. H. Patel College of Engineering And
Technology, VallabhVidyanagar
Gujarat Technological University,Ahmedabad
DECEMBER2011
8/3/2019 GCET-EC-MAA-20-44-48
2/40
G. H. Patel College of Engineering and Technology
Department of Electronics and Communication
2011
CERTIFICATE
Date:
This is to certify that the dissertation entitled MULTIRATE ADAPTIVE
FILTERING has been carried out by NEERAJ LAD 0801110111020, PRITESH
PRAJAPATI 080110111044, KRUPESH SHAH 080110111048 under my guidance in
fulfillment of the degree of Bachelor of Engineering in Electronics and
Communication (7th Semester / 8th Semester) of Gujarat Technological
University,Ahmedabad during the academic year 2011-12.
Guide:
Mr. Mayank Ardeshana
Assistant Professor
G. H. Patel College of Engineering and Technology
Vallabh Vidyanagar
Head of the Department
8/3/2019 GCET-EC-MAA-20-44-48
3/40
ACKNOWLEDGEMENT
We take immense pleasure in thanking Dr. Chintan Modi, Head - EC Department,
GCET College for having permitted us to carry out this project work.
We take this opportunity to express our regards and sincere thanks to my advisor and
guide Assistant Prof. Mayank Ardeshana, without whose support, this project would not
have been possible. His constant encouragement and moral support gave us the motivation to
carry out the project successfully.
I am also indebted to Mr. Darshak Bhatt for his valuable and timely guidance.
The discussions with him helped a lot in developing an in-depth understanding of the topics
involved.
Finally, yet importantly, we would like to express our heartfelt thanks to our beloved
parents for their blessings, our friends/classmates for their help and wishes for the successful
accomplishment of this part of the project.
ABSTRACT
8/3/2019 GCET-EC-MAA-20-44-48
4/40
8/3/2019 GCET-EC-MAA-20-44-48
5/40
Figure No Figure Description Page No
Figure 2.1 Block diagram of a digital signal processing 2
Figure 2.2 Block diagram of Convolution 3
Figure 2.3 Relationship between Impulse response and frequency 6
Figure 2.4 Ideal LowPass filter response 10
Figure 2.5 MATLAB implementation of LowPass Filter response 11
Figure 2.6 Ideal HighPass filter response 11
Figure 2.7 MATLAB implementation of HighPass Filter response 12
Figure 2.8 Ideal BandPass filter response 12
Figure 2.9 MATLAB implementation of BandPass Filter response 13
Figure 2.10 Ideal Bandstop filter Characteristic 13
Figure 2.11 MATLAB implementation of Bandstop Filter response 14
Figure 3.1 Basic block diagram of Adaptive Filter 18
Figure 3.2 Block diagram of System Identification 21
Figure 3.3 Block diagram of Linear Prediction 22
Figure 3.4 Block diagram of Noise Cancellation 23
Figure 4.1 Least Mean Square Filter using Adaptive Algorithm 24
Figure 4.2 Block diagram of Transversal Filter 26
Figure 4.3 Signal flow graph representation of LMS 29
Figure 4.4 Summary of LMS algorithm using Flow Chart 31
8/3/2019 GCET-EC-MAA-20-44-48
6/40
TABLE OF CONTENTS
Acknowledgement i
Abstract ii
List ofFigures iii
Table ofContents iv
Chapter : 1 Introduction
1.1 Motivation 1
1.2 Project specification 1
Chapter : 2 Theoretical Background
2.1 Fundamentals of signals and systems processing 2
2.2 The filtering problem 8
2.3 Implementation of filters 10
Chapter : 3 Adaptive filters
3.1 Introduction of adaptive filters 15
3.2 Adaptive filters using block diagram 17
3.3 Approaches to the development of linear
adaptive filtering algorithms 19
3.4 How to Choose an Adaptive Filter 20
3.5 Application of adaptive filters 20
Chapter : 4 LMS algorithm
4.1 Introduction to LMS algorithm 24
4.2 Detail description of the algorithm 24
4.3 Least-mean-square adaptation algorithm 25
4.4 Summary of the LMS algorithm 28
4.5 Summary of the LMS algorithm 31
Future implementation 32
Conclusion 32
References 33
8/3/2019 GCET-EC-MAA-20-44-48
7/40
8/3/2019 GCET-EC-MAA-20-44-48
8/40
domain adaptive filtering. Time domain adaptive filtering will not be replaced entirely as
it is far more efficient for systems with short impulse responses.
Chapter 2
Theoretical Background
2.1. Fundamentals of signals and systems processing
2.1.1. Signals and Systems
A signal is a description of how one parameter varies with another parameter. A
system is any process that produces an output signal in response to an input signal.
Signal processing is the operation/s done on signals. The general block diagram of such
a process is shown [1]
2.1.2. Linearity
A system is called linear if it has two mathematical properties: homogeneity
and additivity. If you can show that a system has both properties, then you have
proven that the system is linear. Likewise, if you can show that a system doesn't have
one or both properties, you have proven that it isn't linear. A third property, shift
invariance, is not a strict requirement for linearity, but it is a mandatory property for
most DSP techniques. When you see the term linear system used in DSP, you should
assume it includes shift invariance unless you have reason to believe otherwise. These
Fig2.1 Block diagram of a digital signal processing
8/3/2019 GCET-EC-MAA-20-44-48
9/40
three properties form the mathematics of how linear system theory is defined and
used. [1]
2.1.3. Convolution
Convolution is a formal mathematical operation, just as multiplication,
addition, and integration. Addition takes two numbers and produces a third number,
while convolution takes two signals and produces a third signal. Convolution is used
in the mathematics of many fields, such as probability and statistics. In linear systems,it is used to describe the relationship between three signals of interest: the input
signal, the impulse response, and the output signal.
An input signal, x[n], enters a linear system with an impulse response, h[n],
resulting in an output signal, y[n]. In equation form: x[n] * h[n] = y[n]. Expressed in
words, the input signal convolved with the impulse response is equal to the output
signal. Just as addition is represented by the plus, +, and multiplication by the cross,
, convolution is represented by the star, *.
The equation that represents the convolution is:
Fig2.2Block diagram of Convolution
8/3/2019 GCET-EC-MAA-20-44-48
10/40
The system output y[n] is the convolution of the input signal x[n] which
convolves with the system impulse response h[n]. [1]
2.1.4. Correlation
Correlation is a mathematical operation that is very similar to convolution.
Just as with convolution, correlation uses two signals to produce a third signal. This
third signal is called the cross-correlation of the two input signals. If a signal is
correlated with itself, the resulting signal is instead called the autocorrelation.
Correlation is the optimal technique for detecting a known waveform in
random noise. That is, the peak is higher above the noise using correlation than can be
produced by any other linear system. Using correlation to detect a known waveform is
frequently called matched filtering. [1]
2.1.5. The Discrete Fourier Transform (DFT)
Fourier analysis is a family of mathematical techniques, all based on
decomposing signals into sinusoids. The Discrete Fourier Transform is the family
member used with digitized signals. Before we get started on the DFT, let's look for a
moment at the Fourier transform (FT) and explain why we are not talking about it
instead. The Fourier transform of a continuous-time signal x(t) may be defined as:
The DFT, on the other hand, replaces the infinite integral with a finite sum:
8/3/2019 GCET-EC-MAA-20-44-48
11/40
A signal can be either continuous or discrete, and it can be either periodic or
aperiodic. The combination of these two features generates the four categories:
1) Aperiodic-Continuous
This includes, for example, decaying exponentials and the Gaussian curve.
These signals extend to both positive and negative infinity without repeating in a
periodic pattern. The Fourier Transform for this type of signal is simply called the
Fourier Transform. [1]
2) Periodic-Continuous
Here the examples include: sine waves, square waves, and any waveform thatrepeats itself in a regular pattern from negative to positive infinity. This version of the
Fourier transform is called the Fourier series. [1]
3) Aperiodic-Discrete
These signals are only defined at discrete points between positive and negative
infinity, and do not repeat themselves in a periodic fashion. This type of Fourier
transform is called the Discrete Time Fourier Transform. [1]
4) Periodic-Discrete
These are discrete signals that repeat themselves in a periodic fashion from
negative to positive infinity. This class of Fourier Transform is sometimes called the
Discrete Fourier Series, but is most often called the Discrete Fourier Transform. [1]
2.1.6. Frequency Response of Systems
Systems are analyzed in the time domain by using convolution. A similar
analysis can be done in the frequency domain. Using the Fourier transform, every
input signal can be represented as a group of cosine waves, each with a specified
amplitude and phase shift. Likewise, the DFT can be used to represent every output
signal in a similar form. This means that any linear system can be completely
described by how it changes the amplitude and phase of cosine waves passing through
it. This information is called the system's frequency response. Since both the impulseresponse and the frequency response contain complete information about the system,
8/3/2019 GCET-EC-MAA-20-44-48
12/40
there must be a one-to-one correspondence between the two. Given one, you can
calculate the other. The relationship between the impulse response and the frequency
response is one of the foundations of signal processing: A system's frequency
response is the Fourier Transform of its impulse response. [1]
Keeping with standard DSP notation, impulse responses use lower case
variables, while the corresponding frequency responses are upper case. Since h[ ] is
the common symbol for the impulse response, H[ ] is used for the frequency response.
Systems are described in the time domain by convolution, that is: x[n] * h[n] = y[n].
In the frequency domain, the input spectrum is multiplied by the frequency response,
resulting in the output spectrum. As an equation: X[f ] H[f ] = Y[f ]. That is,
convolution in the time domain corresponds to multiplication in the frequency
domain. And convolution in the frequency domain corresponds to multiplication in
the time domain. The next figure illustrates the relationship between the impulse
response and the frequency response:
2.1.7. Inverse DFT
The inverse DFT (the IDFT) is given by:
Fig2.3Relationship between Impulse response and
8/3/2019 GCET-EC-MAA-20-44-48
13/40
In summary, the DFT is proportional to the set of coefficients of projection
onto the sinusoidal basis set, and the IDFT is the reconstruction of the original signal
as a superposition of its sinusoidal projections. This basic architecture extends to all
linear orthogonal transforms, including wavelets, Fourier transforms, Fourier series,
the discrete-time Fourier transform (DTFT), and certain short-time Fourier transforms
(STFT).
2.1.8. The Fast Fourier Transform (FFT)
The Fast Fourier Transform is another method for calculating the DFT. While
it produces the same result as the other approaches, it is incredibly more efficient,
often reducing the computation time by hundreds. This is the same improvement as
flying in a jet aircraft versus walking! The FFT requires a few dozen lines of code,
and it is one of the most complicated algorithms in DSP. [1]
2.1.8.1. How the FFT works
By making use of periodicities in the sines that are multiplied to do the
transforms, the FFT greatly reduces the amount of calculation required. Here's a little
overview. Functionally, the FFT decomposes the set of data to be transformed into a
series of smaller data sets to be transformed. Then, it decomposes those smaller sets
into even smaller sets. At each stage of processing, the results of the previous stage
are combined in special way. Finally, it calculates the DFT of each small data set. For
example, an FFT of size 32 is broken into 2 FFT's of size 16, which are broken into 4
FFT's of size 8, which are broken into 8 FFT's of size 4, which are broken into 16
FFT's of size 2. Calculating a DFT of size 2 is trivial.
Here is a slightly more rigorous explanation: It turns out that it is possible to
take the DFT of the first N/2 points and combine them in a special way with the DFT
of the second N/2 points to produce a single N-point DFT. Each of these N/2-point
DFTs can be calculated using smaller DFTs in the same way. One (radix-2) FFT
begins, therefore, by calculating N/2 2-point DFTs. These are combined to form N/4
4-point DFTs. The next stage produces N/8 8-point DFTs, and so on, until a single
N-point DFT is produced. [1]
8/3/2019 GCET-EC-MAA-20-44-48
14/40
2.1.8.2. Efficiency of the FFT
The DFT takes N^2 operations for N points. Since at any stage the
computation required to combine smaller DFTs into larger DFTs is proportional to
N, and there are log2(N) stages (for radix 2), the total computation is proportional to
N * log2(N). Therefore, the ratio between a DFT computation and an FFT
computation for the same N is proportional to N / log2(n). In cases where N is small
this ratio is not very significant, but when N becomes large, this ratio gets very large.
(Every time you double N, the numerator doubles, but the denominator only increases
by 1.)
2.1.8.3. Some Terminology used in FFT
The radix is the size of FFT decomposition. For single-radix FFT's, the
transform size must be a power of the radix. FFT's can be decomposed using DFT's of
even and odd points, which is called a Decimation-In-Time (DIT) FFT, or they can be
decomposed using a first-half / second-half approach, which is called a Decimation-
In-Frequency (DIF) FFT. Generally, the user does not need to worry which type is
being used.
2.1.8.4. Implementation of FFT
Except as a learning exercise, you generally will never have to. Many good FFT
implementations are available in C, FORTRAN and other languages, and microprocessor
manufacturers generally provide free optimized FFT implementations in their processors'
assembly code, therefore, it is not so important to understand how the FFT really works,
as it is to understand how to use it.[1]
2.2 The filtering problem
Filtering is a process of changing signals spectral content. The change is usually the
reduction of certain frequencies in the signal while allowing the other frequencies to pass.
A fundamental aspect of signal processing is filtering. Filtering involves the
manipulation of the spectrum of a signal by passing or blocking certain portions of the
spectrum, depending on the frequency of those portions. Filters are designed according to
what kind of manipulation of the signal is required for a particular application. Digital filters
8/3/2019 GCET-EC-MAA-20-44-48
15/40
are implemented using three fundamental building blocks: an adder, a multiplier, and a delay
element.
The design process of a digital filter is long and tedious if done by hand. With the aid
of computer programs performing filter design algorithms, designing and optimizing filters
can be done relatively quickly. We will use MATLAB, a mathematical software package, to
design, manipulate, and analyze digital filters. The design options in MATLAB allow the
user to either create a code for designing filters that calls built-in functions.
The term filter is often used to describe a device in the form of a piece of physical
hardware or software that is applied to a set of noisy data in order to extract information
about a prescribed quantity of interest. The noise may arise from a variety of sources. For
example, the data 'may have been derived by means of noisy sensors or may represent a
useful signal component that has been corrupted by transmission through a communication
channel. In any event, we may use a filter to perform three basic information-processing
tasks: [2]
1. Filtering is an operation that involves the extraction of information about a quantity
of interest at time t by using data measured up to and including time t.
2. Smoothing is different from filtering, in that information about the quantity of
interest need not be available at time t, and data measured later than time t can be used in
obtaining this information. This means that in the case of smoothing there is a delay in
producing the result of interest. Since in the smoothing process we are able to use data
obtained not only up to time t but also data obtained after time t, we would expect smoothing
to be more accurate in some sense than filtering.
3. Prediction is the forecasting side of information processing. The aim here is to
derive information about what the quantity of interest will be like at some time t + in the
future, for some > 0, by using data measured up to and including time t.
We may classify filters into linear and nonlinear. A filter is said to be linear if the
filtered, smoothed, or predicted quantity at the output of the device is a linear function of the
observations applied to the filter input. Otherwise, the filter is nonlinear.
8/3/2019 GCET-EC-MAA-20-44-48
16/40
Filters used for direct filtering can be either Fixed or Adaptive. [3]
1. Fixed filters - The design of fixed filters requires a priori knowledge of
both the signal and the noise, i.e. if we know the signal and noise
beforehand; we can design a filter that passes frequencies contained in the
signal and rejects the frequency band occupied by the noise. Filter can be
divided into several types which are lowpass filter, highpass filter, bandpass
filter and bandstop filter.
2. Adaptive filters - Adaptive filters, on the other hand, have the ability to
adjust their impulse response to filter out the correlated signal in the input.
They require little or no a priori knowledge of the signal and noise
characteristics. (If the signal is narrowband and noise broadband, which is
usually the case, or vice versa, no a priori information is needed; otherwise
they require a signal (desired response) that is correlated in some sense to
the signal to be estimated.) Moreover adaptive filters have the capability of
adaptively tracking the signal under non-stationary conditions.
2.3 Implementation of filters
2.3.1Lowpass filter:
The ideal lowpass filter with cutoff frequency is shown. Also the practical
filter is shown having cutoff frequency 5K Hz, and sampling frequency of 30K Hz is
used.
8/3/2019 GCET-EC-MAA-20-44-48
17/40
2.3.2 Highpass filter:
Fig2.4Ideal LowPass filter response
X axis Frequency(Hz)
Y axisMagnitude(dB)
Cut off freq5 KHz
X axis Frequency(Hz)
Y axisPhase(degrees)
Cut off freq5 KHz
Fig2.5 MATLAB implementation of LowPass Filter response
8/3/2019 GCET-EC-MAA-20-44-48
18/40
The ideal highpass filter with cutoff frequency is shown. Also the
practical filter is shown having cutoff frequency 5K Hz, and sampling frequency of
30K Hz is used.
2.3.3 Bandpass filter:
Fig2.6Ideal HighPass filter response
X axis Frequency(Hz)
Y axisMagnitude(dB)
Cut off freq5 KHz
X axis Frequency(Hz)
Y axisPhase(degrees)
Cut off freq5 KHz
Fig2.7 MATLAB implementation of HighPass Filter
response
8/3/2019 GCET-EC-MAA-20-44-48
19/40
The ideal bandpass filter with allowable bandwidth is shown.
Also the practical filter is shown having lower cutoff frequency of 1200 and higher
cutoff frequency of 1800, and sampling frequency of 8K Hz is used.
Fi 2.8 Ideal BandPass ilter res onse
X axis Frequency(Hz)
Y axisPhase(degrees)
Cut off freq1200Hz
X axis Frequency(Hz)
Y axisPhase(degrees)
Cut off freq1800Hz
Fig2.9 MATLAB implementation of BandPass Filter response
8/3/2019 GCET-EC-MAA-20-44-48
20/40
2.3.4 Bandstop filter:
The ideal bandstop filter with frequencies is shown. Also the practical
filter is shown having lower cutoff frequency of 1200 and higher cutoff frequency of
1800, and sampling frequency of 8K Hz is used.
Fig2.10Ideal Bandstop filter Characteristic
X axis Frequency(Hz)
Y axisPhase(degrees)
Cut off freq2800Hz
X axis Frequency(Hz)
Y axisMagnitude(dB)
Cut off freq1800Hz
Fig2.11 MATLAB implementation of Bandstop Filter response
8/3/2019 GCET-EC-MAA-20-44-48
21/40
Chapter 3
Adaptive filters
3.1 Introduction
In normal filters, a straight forward approach is used where the "estimate and plug"
procedure is implemented. This is a two-stage process whereby the filter first "estimates" the
statistical parameters of the relevant signals and then "plugs" the results so obtained into a
nonrecursive formula for computing the filter parameters. For real-time operation, this
procedure has the disadvantage of requiring excessively elaborate and costly hardware. A
more efficient method is to use an adaptive filter.
By such a device we mean one that is self-designing in that the adaptive filter relies
for its operation on a recursive algorithm, which makes it possible for the filter to perform
satisfactorily in an environment where complete knowledge of the relevant signal
characteristics is not available. The algorithm starts from some predetermined set of initial
conditions, representing whatever we know about the environment. Yet, in a stationary
environment, we find that after successive iterations of the algorithm it converges to the
optimum Wiener solution in some statistical sense. In a nonstationary environment, the
8/3/2019 GCET-EC-MAA-20-44-48
22/40
algorithm offers a tracking capability, in that it can track time variations in the statistics of the
input data, provided that the variations are sufficiently slow.
As a direct consequence of the application of a recursive algorithm whereby the
parameters of an adaptive filter are updated from iteration to iteration, the parameters become
data dependent. This, therefore, means that an adaptive filter is in reality a nonlinear device,
in the sense that it does not obey the principle of superposition. Notwithstanding this
property, adaptive filters are commonly classified as linear or nonlinear. An adaptive filter is
said to be linear if the estimate of a quantity of interest is computed adaptively (at the output
of the filter) as a linear combination of the available set of observations applied to the filter
input. Otherwise, the adaptive filter is said to be nonlinear.
A wide variety of recursive algorithms has been developed in the literature for the
operation of linear adaptive filters. In the final analysis, the choice of one algorithm over
another is determined by one or more of the following factors:
Rate of convergence: This is defined as the number of iterations required for the
algorithm, in response to stationary inputs, to converge "close enough" to the
optimum Wiener solution in the mean-square sense. A fast rate of convergence allows
the algorithm to adapt rapidly to a stationary environment of unknown statistics.
Misadjustment: For an algorithm of interest, this parameter provides a quantitative
measure of the amount by which the final value of the mean-squared error, averaged
over an ensemble of adaptive filters, deviates from the minimum mean-squared error
that is produced by the Wiener filter.
Tracking: When an adaptive filtering algorithm operates in a nonstationary
environment, the algorithm is required to track statistical variations in the
environment. The tracking performance of the algorithm, however, is influenced by
two contradictory features: (1) rate of convergence, and (b) steady-state fluctuation
due to algorithm noise.
Robustness: For an adaptive filter to be robust, small disturbances (i.e., disturbances
with small energy) can only result in small estimation errors. The disturbances may
arise from a variety of factors, internal or external to the filter.
Computational requirements. Here the issues of concern include (a) the number of
operations (i.e., multiplications, divisions, and additions/subtractions) required to
make one complete iteration of the algorithm, (b) the size of memory location
8/3/2019 GCET-EC-MAA-20-44-48
23/40
required to store the data and the program, and (c) the investment required So
program the algorithm on a computer.
Structure: This refers to the structure of information flow in the algorithm,
determining the manner in which it is implemented in hardware form. For example,an algorithm whose structure exhibits high modularity, parallelism, or concurrency is
well suited for implementation using very large-scale integration (VLSI).
Numerical properties. When an algorithm is implemented numerically, inaccuracies
are produced due to quantization errors. The quantization errors are due to analog-to-
digital conversion of the input data and digital representation of internal calculations.
Ordinarily, it is the latter source of quantization errors that poses a serious design
problem. In particular, there are two basic issues of concern: numerical stability and
numerical accuracy. Numerical stability is an inherent characteristic of an adaptive
filtering algorithm. Numerical accuracy, on the other hand, is determined by the
number of bits (i.e., binary digits) used in the numerical representation of data
samples and filter coefficients. An adaptive filtering algorithm is said to be
numerically robust when it is insensitive to variations in the wordlength used in its
digital implementation. [2]
3.2 Adaptive filters using block diagram
An adaptive filter is a computational device that attempts to model the relationship
between two signals in real time in an iterative manner. Adaptive filters are often realized
either as a set of program instructions running on an arithmetical processing device such as a
microprocessor or DSP chip, or as a set of logic operations implemented in a field-
programmable gate array (FPGA) or in a semicustom or custom VLSI integrated circuit.
However, ignoring any errors introduced by numerical precision effects in these
implementations, the fundamental operation of an adaptive filter can be characterized
independently of the specific physical realization that it takes. For this reason, we shall focus
on the mathematical forms of adaptive filters as opposed to their specific realizations in
software or hardware. Descriptions of adaptive filters as implemented on DSP chips and on a
dedicated integrated circuit can be found as follows:
An adaptive filter is defined by four aspects:
1. The signals being processed by the filter.
8/3/2019 GCET-EC-MAA-20-44-48
24/40
2. The structure that defines how the output signal of the filter is computed
from its input signal.
3. The parameters within this structure that can be iteratively changed to alter
the filters input-output relationship.
4. The adaptive algorithm that describes how the parameters are adjusted from
one time instant to the next.
By choosing a particular adaptive filter structure, one specifies the number and type of
parameters that can be adjusted. The adaptive algorithm used to update the parameter values
of the system can take on a myriad of forms and is often derived as a form of optimization
procedure that minimizes an error criterion that is useful for the task at hand.
In this section, we present the general adaptive filtering problem and introduce the
mathematical notation for representing the form and operation of the adaptive filter. We then
discuss several different structures that have been proven to be useful in practical
applications. We provide an overview of the many and varied applications in which adaptive
filters have been successfully used. Finally, we give a simple derivation of the least-mean-
square (LMS) algorithm, which is perhaps the most popular method for adjusting the
coefficients of an adaptive filter, and we discuss some of this algorithms properties.
As for the mathematical notation used throughout this section, all quantities are
assumed to be real-valued. Scalar and vector quantities shall be indicated by lowercase (e.g.,
x) and uppercase-bold (e.g., X) letters, respectively. We represent scalar and vector
sequences or signals as x(n) and X(n), respectively, where n denotes the discrete time or
discrete spatial index, depending on the application [2]
8/3/2019 GCET-EC-MAA-20-44-48
25/40
Figure shows a block diagram in which a sample from a digital input signal x(n) is fed
into a device, called an adaptive filter, that computes a corresponding output signal sample
y(n) at time n. For the moment, the structure of the adaptive filter is not important, except for
the fact that it contains adjustable parameters whose values affect how y(n) is computed. The
output signal is compared to a second signal d(n), called the desired response signal, by
subtracting the two samples at time n. This difference signal, given by: e(n) = d(n) y(n) is
known as the error signal. The error signal is fed into a procedure which alters or adapts theparameters of the filter from time n to time (n + 1) in a well-defined manner. This process of
adaptation is represented by the oblique arrow that pierces the adaptive filter block in the
figure. As the time index n is incremented, it is hoped that the output of the adaptive filter
becomes a better and better match to the desired response signal through this adaptation
process, such that the magnitude of e(n) decreases over time. In this context, what is meant
by better is specified by the form of the adaptive algorithm used to adjust the parameters of
the adaptive filter.
In the adaptive filtering task, adaptation refers to the method by which the parameters
of the system are changed from time index n to time index (n + 1). The number and types of
parameters within this system depend on the computational structure chosen for the system.
[2]
3.3 Approaches to the development of linear adaptive filtering algorithms
Fig3.1Basic block diagram of Adaptive Filter
8/3/2019 GCET-EC-MAA-20-44-48
26/40
8/3/2019 GCET-EC-MAA-20-44-48
27/40
3.4 How to Choose an Adaptive Filter
Given the wide variety of adaptive filters available it is a choice to be made for an
application of interest. The filter used should be based on project type and its computational
cost, performance, and robustness are the issues which are to be considered while designing
adaptive filters.
The use of computer simulation provides a good first step in undertaking a detailed
investigation of these issues. We begin by using the LMS algorithm as an adaptive filtering
tool for the project. The LMS algorithm is relatively simple to implement. Yet it is powerful
enough to evaluate the practical benefits that may result from the application of adaptivity to
the problem at hand. Moreover, it provides a practical frame of reference for assessing any
further improvement that may be attained through the use of more sophisticated adaptive
filtering algorithms.
Practical applications of adaptive filtering are very diverse, with each application
having peculiarities of its own. The solution for one application may not be suitable for
another. Nevertheless, to be successful we have to develop a physical understanding of the
environment in which the filter has to operate and thereby relate to the realities of the
application of interest.
3.5 Applications of adaptive filters
1) System Identification
This is used to identify the response of system. The adaptive filters try to
determine the best linear model that describes the input-output relationship of the unknown
system.
8/3/2019 GCET-EC-MAA-20-44-48
28/40
Let d(n) represent the output of the unknown system with x(n) as its input.
Then, the desired response signal in this model is d(n)= d(n)+x(n). Here, the task of the
adaptive filter is to accurately represent the signal d(n) at its output. If y(n)= d(n) then the
adaptive filter has accurately modeled or identified the portion of the unknown system that is
driven by x(n).
Since the model typically chosen for the adaptive filter is a linear filter, the
practical goal of the adaptive filter is to determine the best linear model that describes the
input-output relationship of the unknown system. Such a procedure makes the most sense
when the unknown system is also a linear model of the same structure as the adaptive filter,
as it is possible that y(n)= d(n) for some set of adaptive filter parameters.
The system identification task is at the heart of numerous adaptive filtering
applications. We list some of its applications here.
Channel Identification
In communication systems, useful information is transmitted from one point to
another across a medium such as an electrical wire, an optical fiber, or a wireless radio link.
Nonidealities of the transmission medium or channel distort the fidelity of the transmitted
signals, making the deciphering of the received information difficult. In such cases, an
adaptive filter can be used to model the effects of the channel ISI for purposes of deciphering
the received information in an optimal manner. In this problem scenario, the transmitter sends
to the receiver a sample sequence x(n) that is known to both the transmitter and receiver. The
Fig3.2Block diagram of System Identification
8/3/2019 GCET-EC-MAA-20-44-48
29/40
receiver then attempts to model the received signal d(n)using an adaptive filter whose input is
the known transmitted sequence x(n) After a suitable period of adaptation, the parameters of
the adaptive filter in W(n) are fixed and then used in a procedure to decode future signals
transmitted across the channel.
Plant Identification
In many control tasks, knowledge of the transfer function of a linear plant is
required by the physical controller so that a suitable control signal can be calculated and
applied. In such cases, we can characterize the transfer function of the plant by exciting it
with a known signal x(n) and then attempting to match the output of the plant d(n) with a
linear adaptive filter. After a suitable period of adaptation, the system has been adequately
modeled, and the resulting adaptive filter coefficients in W(n) can be used in a control
scheme to enable the overall closed-loop system to behave in the desired manner. [2]
2) Linear Prediction
In this system, the input signal x(n) is derived from the desired response signal
as x(n)= d(n ), Where is an integer value of delay. In effect, the input signal serves as the
desired response signal, and for this reason it is always available. In such cases, the linear
adaptive filter attempts to predict future values of the input signal using past samples, giving
rise to the name linear prediction for this task.
If an estimate of the signal x(n + ) at time n is desired, a copy of the
adaptive filter whose input is the current sample x(n) can be employed to compute this
quantity. However, linear prediction has a number of uses besides the obvious application
of forecasting future events, as described in the following two applications. [2]
Fig3.3Block diagram of Linear Prediction
8/3/2019 GCET-EC-MAA-20-44-48
30/40
3) Noise cancellation
A signal of interest is linearly mixed with other extraneous noises which
introduce unacceptable errors in the measurements. Noise cancellation is thus used to
cancel this interference from a primary signal.
Adaptive noise cancelling has been used for several applications. One of the first
was a medical application that enabled the electroencephalogram (EEG) of the fetal
heartbeat of an unborn child to be cleanly extracted from the much-stronger
interfering EEG of the maternal heartbeat signal [2]
4) echo cancellation
The term echo cancellation is used in telephony to describe the process of
removing echo from a voice communication in order to improve voice quality on
a telephone call. In addition to improving subjective quality, this process increases the
capacity achieved through silence suppression by preventing echo from traveling
across a network.
Echo cancellation involves first recognizing the originally transmitted signal that
re-appears, with some delay, in the transmitted or received signal. Once the echo is
recognized, it can be removed by 'subtracting' it from the transmitted or received
signal. This technique is generally implemented using a digital signal
processor(DSP), but can also be implemented in software. Echo cancellation is done
using eitherecho suppressors or echo cancellers, or in some cases both. [2]
Fig3.4Block diagram of Noise Cancellation
http://en.wikipedia.org/wiki/Telephonyhttp://en.wikipedia.org/wiki/Echo_(phenomenon)http://en.wikipedia.org/wiki/Telephone_callhttp://en.wikipedia.org/wiki/Silence_suppressionhttp://en.wikipedia.org/wiki/Telecommunications_networkhttp://en.wikipedia.org/wiki/Digital_signal_processorhttp://en.wikipedia.org/wiki/Digital_signal_processorhttp://en.wikipedia.org/wiki/Softwarehttp://en.wikipedia.org/wiki/Echo_suppressorhttp://en.wikipedia.org/wiki/Telephonyhttp://en.wikipedia.org/wiki/Echo_(phenomenon)http://en.wikipedia.org/wiki/Telephone_callhttp://en.wikipedia.org/wiki/Silence_suppressionhttp://en.wikipedia.org/wiki/Telecommunications_networkhttp://en.wikipedia.org/wiki/Digital_signal_processorhttp://en.wikipedia.org/wiki/Digital_signal_processorhttp://en.wikipedia.org/wiki/Softwarehttp://en.wikipedia.org/wiki/Echo_suppressor8/3/2019 GCET-EC-MAA-20-44-48
31/40
Chapter 4
LMS algorithm
4.1 Introduction to LMS algorithm
Least mean squares (LMS) algorithms are a class ofadaptive filter used to mimic a
desired filter by finding the filter coefficients that relate to producing the least mean squares
of the error signal (difference between the desired and the actual signal). It was invented in
1960 by Stanford University professor Bernard Widrow and his first Ph.D. student, Ted Hoff.
The LMS algorithm is an important member of the family of stochastic gradient
algorithms. The term "stochastic gradient" is intended to distinguish the LMS algorithm from
the method of steepest descent that uses a deterministic gradient in a recursive computation ofthe Wiener filter for stochastic inputs. A significant feature of the LMS algorithm is its
simplicity.
Moreover, it does not require measurements of the pertinent correlation functions, nor
does it require matrix inversion. Indeed, it is the simplicity of the LMS algorithm that has
made it the standard against which other adaptive filtering algorithms are benchmarked. [6]
4.2 Problem formulation
Most linear adaptive filtering problems can be formulated using the block diagram
above. That is, an unknown system is to be identified and the adaptive filter attempts
Fig4.1Least Mean Square Filter using Adaptive Algorithm
http://en.wikipedia.org/wiki/File:Lms_filter.png8/3/2019 GCET-EC-MAA-20-44-48
32/40
to adapt the filter to make it as close as possible to , while using only observable
signals x(n), d(n) and e(n); but y(n), v(n) and h(n) are not directly observable. Its solution is
closely related to theWiener filter.
Definition of symbols
d (n) = y (n) + (n)
Idea
The idea behind LMS filters is to usesteepest descentto find filter weightswhich minimize a cost function.[6]
4.3 Overview of the least-mean-square algorithm
The least-mean-square (LMS) algorithm is a linear adaptive filtering algorithm that
consists of two basic processes:
1. A filtering process, which involves (a) computing the output of a transversal filter
produced by a set of tap inputs, and (b) generating an estimation error by comparing this
output to a desired response.
2. An adaptive process, which involves the automatic adjustment of the tap weights
of the filter in accordance with the estimation error.
Thus, the combination of these two processes working together constitutes a feedback
loop around the LMS algorithm, as illustrated in the block diagram of Figure. First, we have a
transversal filter, around which the LMS algorithm is built; this component is responsible for
performing the filtering process. Second, we have a mechanism for performing the adaptive
control process on the tap weights of the transversal filter, hence the designation "adaptive
weight-control mechanism" in Figure (a). Details of the transversal filter component are
presented in Figure (b). The tap inputs u(n), u(n - 1) ....., u(n - M + 1) form the elements of
the M-by-1 tap-input vector u(n), where M - 1 is the number of delay elements; these tap
inputs span a multidimensional space denoted by . Correspondingly, the tap weights
form the elements of the M-by- 1 tap-weight vector W(n). The
value computed for the tap-weight vector W(n) using the LMS algorithm represents an
http://en.wikipedia.org/wiki/Wiener_filterhttp://en.wikipedia.org/wiki/Wiener_filterhttp://en.wikipedia.org/wiki/Wiener_filterhttp://en.wikipedia.org/wiki/Steepest_descenthttp://en.wikipedia.org/wiki/Steepest_descenthttp://en.wikipedia.org/wiki/Cost_functionhttp://en.wikipedia.org/wiki/Cost_functionhttp://en.wikipedia.org/wiki/Wiener_filterhttp://en.wikipedia.org/wiki/Steepest_descenthttp://en.wikipedia.org/wiki/Cost_function8/3/2019 GCET-EC-MAA-20-44-48
33/40
estimate whose expected value approaches the Wiener solution W0 (for a wide-sense
stationary environment) as the number of iterations n approaches infinity.
During the filtering process the desired response d(n) is supplied for processing,
alongside the tap-input vector u(n). Given this input, the transversal filter produces an output
used as an estimate of the desired response d(n). Accordingly, we may define an
estimation error e(n) as the difference between the desired response and the actual filter
output, as indicated in the output end of Figure (b). The estimation error e(n) and the tap-
Fig4.2Block diagram of Transversal Filter
8/3/2019 GCET-EC-MAA-20-44-48
34/40
input vector u(n) are applied to the control mechanism, and the feedback loop around the tap
weights is thereby closed.
A scalar version of the inner product of the estimation error e(n) and the tap input u(n
- k) is computed for k = 0, 1, 2 ..... M - 2, M - 1. The result so obtained defines the correction
Wk(n) applied to the tap weight Wk(n) at iteration n + 1. The scaling factor used in this
computation is denoted by , is called the step-size parameter.
The LMS algorithm uses the product u(n - k)e*(k) as an estimate of element k in the
gradient vector J(n) that characterizes the method of steepest descent. In other words, the
expectation operator is missed out from all the paths. Accordingly, the recursive computation
of each tap weight in the LMS algorithm suffers from a gradient noise.
Earlier we stated that the LMS algorithm involves feedback in its operation, which
therefore raises the related issue of stability. In this context, a meaningful criterion is to
require that
J(n) J( ) as n ,
Where J(n) is the mean-squared error produced by the LMS algorithm at time n, and its final
value j( ) is a constant. An algorithm that satisfies this requirement is said to be convergent
in the mean square. For the LMS algorithm to satisfy this criterion, the step-size parameter
has to satisfy a certain condition related to the Eigen structure of the correlation matrix of thetap inputs.
The difference between the final value J( ) and the minimum value attained by
the Wiener solution is called the excess mean-squared error . This difference
represents the price paid for using the adaptive (stochastic) mechanism to control the tap
weights in the LMS algorithm in place of a deterministic approach as in the method of
steepest descent. The ratio of to is called the misadjustment, which is a measure
of how far the steady-state solution computed by the LMS algorithm is away from the Wiener
solution. It is important to realize, however, that the Misadjustment, M is under the designer's
control. In particular, the feedback loop acting around the tap weights behaves like a low-pass
filter, whose "average" time constant is inversely proportional to the stepsize parameter .
Hence, by assigning a small value to the adaptive process is made to progress slowly, and
the effects of gradient noise on the tap weights ate largely filtered out. This, in turn, has the
effect of reducing the Misadjustment.
8/3/2019 GCET-EC-MAA-20-44-48
35/40
Therefore the LMS algorithm is simple in implementation, yet capable of delivering
high performance by adapting to its external environment. To do so, however, we have to pay
particular attention to the choice of a suitable value for the step-size parameter . [6]
4.4 Least-mean-square adaptation algorithm
If it were possible to make exact measurements of the gradient vector J(n) at each
iteration n, and if the step-size parameter is suitably chosen, then the tap-weight vector
computed by using the steepest-descent algorithm would indeed converge to the optimum
Wiener solution. In reality, however, exact measurements of the gradient vector are not
possible since this would require prior knowledge of both the correlation matrix R of the tap
inputs and the cross-correlation vector P between the tap inputs and the desired response.
Consequently, the gradient vector must be estimated from the available data
To develop an estimate of the gradient vector J(n), the most obvious strategy is to
substitute estimates of the correlation matrix R and the cross-correlation vector P in the
formula:
The simplest choice of estimators for R and p is to use instantaneous estimates that
are based on sample values of the tap-input vector and desired response, as defined by,
respectively,
And
Correspondingly, the instantaneous estimate of the gradient vector is
J(n) = - 2u(n)d*(n) + 2u(n) (n)W(n)
Generally speaking, this estimate is biased because the tap-weight estimate vector
W(n) is a random vector that depends on the tap-input vector u(n). Note that the estimate
J(n) may also be viewed as the gradient operator applied to the instantaneous squared error
Substituting the estimate of above equation, for the gradient vector J(n) in the
steepest descent algorithm, we get a new recursive relation for updating the tap-weight
vector:
W(n + 1) =W(n) + u(n)[d*(n) - (n)W(n)]
8/3/2019 GCET-EC-MAA-20-44-48
36/40
Here we have used a hat over the symbol for the tap-weight vector to distinguish it
from the value obtained by using the steepest-descent algorithm. Equivalently, we may write
the result in the form of three basic relations as follows:
1. Filter output:
2. Estimation error:
3. Tap-weight adaptation:
The (1) and (2) equations define the estimation error e(n), the computation of which is
based on the current estimate of the tap-weight vector, W(n). Note also that the second term, u(n)e*(n), on the right-hand side of Eq. (3) represents the correction that is applied to the
current estimate of the tap-weight vector, W(n). The iterative procedure is started with an
initial guess W(0).
The algorithm described by Equations (1) to (3) is the complex form of the adaptive
least-mean-square (LMS) algorithm. At each iteration or time update, it also requires
knowledge of the most recent values: u(n), d(n), and W(n). The LMS algorithm is a member
of the family of stochastic gradient algorithms. In particular, when the LMS algorithmoperates on stochastic inputs, the allowed set of directions along which we "step" from one
iteration cycle to the next is quite random and cannot therefore be thought of as being true
gradient directions.
8/3/2019 GCET-EC-MAA-20-44-48
37/40
The above figure shows a signal-flow graph representation of the LMS algorithm in the
form of a feedback model. This model bears a close resemblance to the feedback model of
the steepest-descent algorithm. The signal-flow graph of above figure clearly illustrates the
simplicity of the LMS algorithm. In particular, we see from this figure that the LMS
algorithm requires only 2M + 1 complex multiplications and 2M complex additions per
iteration, where M is the number of tap weights used in the adaptive transversal filter. In
other words, the computational complexity of the LMS algorithm is O(M).
The instantaneous estimates of R and P have relatively large variances. At first sight,
it may therefore seem that the LMS algorithm is incapable of good performance since the
algorithm uses these instantaneous estimates. However, we must remember that the LMS
algorithm is recursive in nature, with the result that the algorithm itself effectively averages
these estimates, in some sense, during the course of adaptation. [6]
Fig4.3 Signal flow graph representation of LMS algorithm
8/3/2019 GCET-EC-MAA-20-44-48
38/40
8/3/2019 GCET-EC-MAA-20-44-48
39/40
FUTURE IMPLEMENTATION
We are going to implement the adaptive filters in MATLAB, which can process onsound and then we will go for image.
Then we are going to implement it on hardware (DSP kit TI-DSK 6713).
We will try to implement this adaptive filter on some real time application.
CONCLUSION
Through this project we learnt various concepts of Digital Signal Processing. We
also learnt in detail about adaptive filters and its application. Adaptive filters are those who
adapt themselves according to required situation.
Adaptive filters use LMS algorithm for error detection. LMS algorithm is simple to
understand and easy to implement.
Thus we have satisfactorily completed our first part of project.
8/3/2019 GCET-EC-MAA-20-44-48
40/40
REFERENCES
Books:
1. Proakis, John G. and Manolakis, Dimitris G. Digital Signal Processing: Principles,Algorithms, and Applications, 3rd Edition. Prentice Hall. Upper Saddle River,
NJ,1996
2. Vijay K. Madisetti and Douglas B. Williams Boca Raton (1999)
Introduction to Adaptive FiltersDigital Signal Processing Handbook
3. Oppeheim, Schafer, Buck Pearson education publication, 2nd Edition (2003)
Discrete Time Signal Processing
4. IEEE Transactions on Signal Processing, Vol. 39,No. 10, (1991)
5. E.Ifeachor and B.Jervis, 2nd edition Prentice Hall (1998) Digital Signal Processing: A
practical Approach
6. Simon Hayking, 3rd Edition Filter Theory
Papers:
1. John J. Shynk, IEEE SP Magazine Frequency Domain and multirate Adaptive
Filtering
2. Christian Feldbauer, Franz Pernkopf, and Erhard Rank , Signal Processing and Speech
Communication Laboratory, Inffeldgasse 16c Adaptive Filters Tutorial