Post on 14-Apr-2017
transcript
EECS0712 Adaptive Signal Processing1
Introduction to Adaptive SignalProcessing
EECS0712 Adaptive Signal Processing1
Introduction to Adaptive SignalProcessing
Assoc. Prof. Dr. Peerapol YuvapoositanonDept. of Electronic Engineering
CESdSP ASP1-1EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Course Outline
• Introduction to Adaptive Signal Processing• Adaptive Algorithms Families:• Newton’s Method and Steepest Descent• Least Mean Squared (LMS)• Recursive Least Squares (RLS)• Kalman Filtering• Applications of Adaptive Signal Processing in
Communications and Blind Equalization
• Introduction to Adaptive Signal Processing• Adaptive Algorithms Families:• Newton’s Method and Steepest Descent• Least Mean Squared (LMS)• Recursive Least Squares (RLS)• Kalman Filtering• Applications of Adaptive Signal Processing in
Communications and Blind EqualizationCESdSP ASP1-2
EECS0712 Adaptive Signal Processinghttp://embedsigproc.wordpress.com/eecs0712
Assoc. Prof. Dr. P.Yuvapoositanon
Evaluation
• Assignment= 20 %• Midterm = 30 %• Final = 50 %
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-3
Textbooks
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-4
http://embedsigproc.wordpress.com/eecs0712-adaptive-signal-processing/
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-5
QR code
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-6
Adaptive Signal Processing
• Definition: Adaptive signal processing is thedesign of adaptive systems for signal-processing applications.
[http://encyclopedia2.thefreedictionary.com/adaptive+signal+processing]
• Definition: Adaptive signal processing is thedesign of adaptive systems for signal-processing applications.
[http://encyclopedia2.thefreedictionary.com/adaptive+signal+processing]
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-7
System Identification
• Let’s consider a system called “plant”• We need to know its characteristics, i.e., The
impulse response of the system
CESdSP ASP1-8EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Plant Comparison
CESdSP ASP1-9EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Error of Plant Outputs
CESdSP ASP1-10EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Error of Estimation
• Error of estimation is represented by thesignal energy of error
2 2
2 2
( )
2
e d y
d dy y
CESdSP ASP1-11EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
2 2
2 2
( )
2
e d y
d dy y
Adaptive System
• We can do it adaptively
CESdSP ASP1-12EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
• Adjust the weight for minimum error e
One-weight
CESdSP ASP1-13EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
2 2
2 2
2 20 0 0 0
( )
2
( ) 2( )( ) ( )I I
e d y
d dy y
w x w x w x w x
CESdSP
2 2
2 2
2 20 0 0 0
( )
2
( ) 2( )( ) ( )I I
e d y
d dy y
w x w x w x w x
ASP1-14EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Error Curve
• Parabola equation
CESdSP ASP1-15EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Partial diff. and set to zero
• Partial differentiation
• Set to zero
• Result:
22 2
0 0 0 00 0
2 20 0
( ) 2( )( ) ( )
2 2
I II I
I
ew x w x w x w x
w w
w x w x
• Partial differentiation
• Set to zero
• Result:
CESdSP
22 2
0 0 0 00 0
2 20 0
( ) 2( )( ) ( )
2 2
I II I
I
ew x w x w x w x
w w
w x w x
2 20 00 2 2 Iw x w x
0 0Iw w
ASP1-16EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Multiple Weight Plants
• We calculate the weight adaptively• Questions:
– What is the type of signal “x” to be used, e.g.Sine, Cosine or Random signals ?
– If there is more than one weight w0 , i.e., w0….wN-
1, how do we calculate the solution?
• We calculate the weight adaptively• Questions:
– What is the type of signal “x” to be used, e.g.Sine, Cosine or Random signals ?
– If there is more than one weight w0 , i.e., w0….wN-
1, how do we calculate the solution?
CESdSP ASP1-17EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Plants with Multiple Weight
• If we have multiple weights
CESdSP
10 1w w z w
ASP1-18EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
• In the case of two-weight
Two-weight
CESdSP ASP1-19EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Input
• From
• We construct the x as vector with firstelement is the most recent
(3), (2), (1), (0), ( 1), ( 2),...x x x x x x
• From
• We construct the x as vector with firstelement is the most recent
CESdSP
[ (3) (2) (1) (0)...]Tx x x xx
ASP1-20EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Plants with Multiple Weight(aka “Transversal Filter”)
• If we have multiple weights( )x n ( 1)x n
CESdSP
0 ( )w x n0 ( 1)w x n
0 0( ) ( ) ( 1)y n w x n w x n
ASP1-21EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Regression input signal vector
• If the current time is n, we have “Regressioninput signal vector”
[ ( ) ( 1) ( 2) ( 3)...]Tx n x n x n x n x
CESdSP
[ ( ) ( 1) ( 2) ( 3)...]Tx n x n x n x n x
ASP1-22EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
00 1
1[ ]T
ww ww
w
CESdSP
00 1
1[ ]T
ww ww
w
00 1
1
ˆ [ ]
II I T
I
ww w
w
w
ASP1-23EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Convolution
• Output of plant is a convolution
• Ex For N=2
1
1
( ) ( )N
kk
y n w x n k
• Output of plant is a convolution
• Ex For N=2
CESdSP
1
1
( ) ( )N
kk
y n w x n k
0 0( ) ( 0) ( 1)y n w x n w x n
ASP1-24EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
0 1
0 1
0 1
0 1
0 1
(3) (3) (2)
(2) (2) (1)
(1) (1) (0)
(0) (0) ( 1)
( 1) ( 1) ( 2)
y w x w x
y w x w x
y w x w x
y w x w x
y w x w x
CESdSP
0 1
0 1
0 1
0 1
0 1
(3) (3) (2)
(2) (2) (1)
(1) (1) (0)
(0) (0) ( 1)
( 1) ( 1) ( 2)
y w x w x
y w x w x
y w x w x
y w x w x
y w x w x
ASP1-25EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
• We can use a vector-matrix multiplication• For example, for n=3 we construct y(3) as
• For example, for n=1 we construct y(1) as
0 1 0 1
(3)(3) (3) (2) [ ] (3)
(2)T
xy w x w x w w
x
w x
• We can use a vector-matrix multiplication• For example, for n=3 we construct y(3) as
• For example, for n=1 we construct y(1) as
CESdSP
0 1 0 1
(3)(3) (3) (2) [ ] (3)
(2)T
xy w x w x w w
x
w x
0 1 0 1
(1)(1) (1) (0) [ ] (1)
(0)T
xy w x w x w w
x
w x
ASP1-26EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
0 1 0 1
0 1 0 1
0 1 0 1
0 1 0 1
(3)(3) (3) (2) [ ] (3)
(2)
(2)(2) (2) (1) [ ] (2)
(1)
(1)(1) (1) (0) [ ] (1)
(0)
(2)(0) (0) ( 1) [ ] (0
(1)
T
T
T
T
xy w x w x w w
x
xy w x w x w w
x
xy w x w x w w
x
xy w x w x w w
x
w x
w x
w x
w x )
CESdSP
0 1 0 1
0 1 0 1
0 1 0 1
0 1 0 1
(3)(3) (3) (2) [ ] (3)
(2)
(2)(2) (2) (1) [ ] (2)
(1)
(1)(1) (1) (0) [ ] (1)
(0)
(2)(0) (0) ( 1) [ ] (0
(1)
T
T
T
T
xy w x w x w w
x
xy w x w x w w
x
xy w x w x w w
x
xy w x w x w w
x
w x
w x
w x
w x )
ASP1-27EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
• The error squared is
• Let us stop there to consider Random signaltheory first.
2 2
2 2
2 2
( )
2
ˆ ˆ( ) 2( )( ) ( )T T T T
e d y
d dy y
w x w x w x w x
• The error squared is
• Let us stop there to consider Random signaltheory first.
CESdSP
2 2
2 2
2 2
( )
2
ˆ ˆ( ) 2( )( ) ( )T T T T
e d y
d dy y
w x w x w x w x
ASP1-28EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Review of Random Signals
CESdSP ASP1-29EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Wireless Transmissions
• Ideal signal transmission
11 00 11 00 11 0011 11 11 000011
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP2-30
11 00 11 00 11 0011 11 11 000011
Information
Information is Random
Random variable
CESdSP ASP1-31EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Random Variable
• Random variable is a function• For a single time Coin Tossing
1,( )
-1,
x HX x
x T
• Random variable is a function• For a single time Coin Tossing
CESdSP
1,( )
-1,
x HX x
x T
ASP1-32EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Our signal x(n) is a RandomVariable
• For a series of Coin Tossing
1,( )
-1,
i
ii
x HX x
x T
• For a series of Coin Tossing
CESdSP
1,( )
-1,
i
ii
x HX x
x T
0 1 2 3 4{ , , , , ,....}x x x x x x
ASP1-33EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Coin tossing and Random Variable
• If random
• We have random variable X0 1 2 3 4
{ , , , , }
{ , , , , }
x H H T H T
x x x x x
CESdSP
• If random
• We have random variable X
0 1 2 3 4( ) { ( ), ( ), ( ), ( ), ( )}
{ ( ), ( ), ( ), ( ), ( )}
{1,1, 1,1, 1}
iX x X x X x X x X x X x
X H X H X T X H X T
ASP1-34EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Random Digital Signal
• If the random variable is a function of time, itis called a stochastic process
CESdSP ASP1-35EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Probability Mass Function
• We need also to define the probability of eachrandom variable
( ) { ( ), ( ), ( ), ( ), ( )}
{1,1, 1,1, 1}
X x X H X H X T X H X T
CESdSP
( ) { ( ), ( ), ( ), ( ), ( )}
{1,1, 1,1, 1}
X x X H X H X T X H X T
ASP1-36EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Probability Mass Function
• PMF is for Discrete distribution function
CESdSP ASP1-37EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Time and Emsemble
CESdSP ASP1-38EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Probability of X(2)
CESdSP ASP1-39EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Probability Density Function
• PDF is for Continuous Distribution Function
CESdSP ASP1-40EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
CESdSP ASP1-41EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Probability Density Function
• PDF values can be > 1 as long as its area undercurve is 1
2
CESdSP
1/2
2
1
1
ASP1-42EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Cumulative Distribution Function
CESdSP
( ( )) Pr[ ( )]P x n X x n x
ASP1-43EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
( )
( ( )) ( )x n
P x n p z dz
x x
CESdSP
( )
( ( )) ( )x n
P x n p z dz
x x
ASP1-44EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Expectation Operator
{}E
CESdSP
{}E
ASP1-45EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Expected Value
• Expected value is known as the “Mean”
{ } ( )X XE x xp x dx
CESdSP
{ } ( )X XE x xp x dx
ASP1-46EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Example of Expected Value(Discrete)
• We toss a die N times and get a set ofoutcomes
• Suppose we roll a die with N=6, we might get
{ ( )} { (1), (2), (3),..., ( )}X i X X X X N
• We toss a die N times and get a set ofoutcomes
• Suppose we roll a die with N=6, we might get
CESdSP
{ ( )} { (1), (2), (3),..., ( )}X i X X X X N
{ ( )} {2,3,6,3,1,1}X i
ASP1-47EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Example of Expected Value(Discrete)
• But, empirically we have Empirical (MonteCarlo) estimate as Expected Value
6
1
{ } ( )Pr( ( ))
1 1 1 11 2 3 6
3 6 3 62.67
Xi
E x X i X X i
CESdSP
6
1
{ } ( )Pr( ( ))
1 1 1 11 2 3 6
3 6 3 62.67
Xi
E x X i X X i
ASP1-48EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Theoretical Expected Value
• But in theory, for a die
6
1
{ } ( )Pr( ( ))
1 1 1 1 1 11 2 3 4 5 6
6 6 6 6 6 63.5
Xi
E X X i X X i
1Pr( ( ))
6X X i
CESdSP
6
1
{ } ( )Pr( ( ))
1 1 1 1 1 11 2 3 4 5 6
6 6 6 6 6 63.5
Xi
E X X i X X i
ASP1-49EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Ensemble Average
i ensembles
1 1 2 2Ensemble Average of (1) (1)Pr[ (1)] (1)Pr[ (1)]
(1)Pr[ (1)]N N
x x x x x
x x
1 ensemble
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-50
i ensembles
Ensemble Average
{ ( )}E x n
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-51
{ ( )} ( ) ( ( )) ( )E x n x n p x n dx n
x
{ ( )}E x n
• I) Linearity
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-52
{ ( ) ( )} { ( )} { ( )}E ax n by n aE x n bE y n
• II)
{ ( ) ( )} { ( )} { ( )}E x n y n E x n E y n
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-53
{ ( ) ( )} { ( )} { ( )}E x n y n E x n E y n
• III)
{ ( )} ( ( )) ( ( )) ( )E y n g x n p x n dx n
x
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-54
{ ( )} ( ( )) ( ( )) ( )E y n g x n p x n dx n
x
Autocorrelation
1 1( , ) { ( ) ( )}r n m E x n x mxx
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-55
1 11 1 1 1 1 1( , ) ( ) ( ) ( ( ), ( )) ( ) ( )r n m x n x m p x n x m dx n x m
xx x x
1 1(1, 4) { (1) (4)}r E x xxx
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-56
Autocorrelation
• n=m
2( , ) ( , ) { ( )}r n m r n n E x n xx xx
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-57
2( , ) ( , ) { ( )}r n m r n n E x n xx xx
Autocorrelation Matrix
(0,0) (0,1) (0, 1)
(1,0) (1,1) (1, 1)
( 1,0) ( 1,1) ( 1, 1)
r r r N
r r r N
r N r N r N N
xx xx xx
xx xx xxxx
xx xx xx
R
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-58
(0,0) (0,1) (0, 1)
(1,0) (1,1) (1, 1)
( 1,0) ( 1,1) ( 1, 1)
r r r N
r r r N
r N r N r N N
xx xx xx
xx xx xxxx
xx xx xx
R
Covariance
( , ) {[ ( ) ( )][ ( ) ( )]}c n m E x n n x m m xx
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-59
( , ) {[ ( ) ( )][ ( ) ( )]}c n m E x n n x m m xx
Stationarity (I)
• I)
{ ( )} { ( )}E x n E x m n1
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-60
n2
Stationarity (II)
• II)
( , ) { ( ) ( )}r n n m E x n x n m xx
1 1 1 1( , ) { ( ) ( )}r n n m E x n x n m xx
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-61
1 1 1 1( , ) { ( ) ( )}r n n m E x n x n m xx
Expected Value of Error Energy
• Let’s take the expected value of error energy
2 2 2ˆ ˆ{ } {( ) 2( )( ) ( ) }
ˆ ˆ ˆ{( )( )} 2 {( )( )} {( )( )}
ˆ ˆ ˆ{ } 2 {( )( )} { }
ˆ ˆ ˆ2 {( )( )}
T T T T
T T T T T T
T T T T T T
T T T T
E e E
E E E
E E E
E
w x w x w x w x
w x x w x w w x w x x w
w xx w x w x w w xx w
w Rw x w x w w Rw
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-62
2 2 2ˆ ˆ{ } {( ) 2( )( ) ( ) }
ˆ ˆ ˆ{( )( )} 2 {( )( )} {( )( )}
ˆ ˆ ˆ{ } 2 {( )( )} { }
ˆ ˆ ˆ2 {( )( )}
T T T T
T T T T T T
T T T T T T
T T T T
E e E
E E E
E E E
E
w x w x w x w x
w x x w x w w x w x x w
w xx w x w x w w xx w
w Rw x w x w w Rw
Vector-Matrix Differentiation
ˆI)ˆ
ˆ ˆ ˆII) 2ˆ
T
T T
w x xw
w xx w Rww
CESdSP
ˆI)ˆ
ˆ ˆ ˆII) 2ˆ
T
T T
w x xw
w xx w Rww
ASP1-63EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Partial diff. and set to zero
• Differentiation
• Result:
ˆ0 2 {( ) } 2ˆ
ˆ2 { } 2
ˆ2 2
TE
E d
w x x Rww
x Rw
r Rw
• Differentiation
• Result:
CESdSP
ˆ0 2 {( ) } 2ˆ
ˆ2 { } 2
ˆ2 2
TE
E d
w x x Rww
x Rw
r Rw
1ˆ w R rASP1-64
EECS0712 Adaptive Signal Processinghttp://embedsigproc.wordpress.com/eecs0712
Assoc. Prof. Dr. P.Yuvapoositanon
2-D Error surface
CESdSP
1ˆ w R r
ASP1-65EECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
Four Basic Classes of AdaptiveSignal Processing
• I) Identification• II) Inverse Modelling• III) Prediction• IV) Interference Cancelling
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-66
• I) Identification• II) Inverse Modelling• III) Prediction• IV) Interference Cancelling
The Four Classes of AdaptiveFiltering
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-67
System Identification
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP2-68
Inverse Modelling
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP2-69
Prediction
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP2-70
Interference Canceller
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP2-71
What are we looking for inAdaptive Systems?
• Rate of Convergence• Misadjustment• Tracking• Robustness• Computational Complexity• Numerical Properties
• Rate of Convergence• Misadjustment• Tracking• Robustness• Computational Complexity• Numerical Properties
CESdSPEECS0712 Adaptive Signal Processing
http://embedsigproc.wordpress.com/eecs0712Assoc. Prof. Dr. P.Yuvapoositanon
ASP1-72