22
Adaptive Filters and SIMO System Identification Patrick A. Naylor Imperial College London Co-workers: Nick Gaubitch, Uttachai Manmontri, Andy Khong, Rehan Ahmad, Jimi Wen, Xiang (Shawn) Lin
Adaptive Filters and SIMO System Identification
Patrick A. NaylorImperial College London
Co-workers: Nick Gaubitch, Uttachai Manmontri, Andy Khong, Rehan Ahmad, Jimi Wen, Xiang (Shawn) Lin
Adaptive Filters and SIMO System Identification 2
Contents
Adaptive signal processing (a DSP perspective)Overview and simple applicationClasses of techniques
Supervised algorithmsSingle channel system identification
Unsupervised algorithmsSingle channelMultichannel
Channel Inversion
Adaptive Filters and SIMO System Identification 3
Problem Formulation – system identification
Given input x(n) and output d(n) of an unknown system H(z), estimate a system such that e(n) is minimized.The system is often assumedto be FIR
Can’t go unstable
The taps of theadaptive filter areadjusted by theadaptive algorithm
H(z)
H(z)
++-
x(n) e(n)
AdaptiveAlgorithm
d(n)
)(ˆ zH
Adaptive Filters and SIMO System Identification 4
Notation
)(ˆ zH
Adaptive Filters and SIMO System Identification 5
Steepest Descent
Write the error as:
Form the cost function:
Modify the coefficients so as to reduce the cost:
$1
0( ) ( ) ( ) ( )
L
kk
e n d n h n x n k−
=
= − −∑
2 ( )J E e n⎡ ⎤= ⎣ ⎦
$ $$
$1 ( )( 1) ( )2
Jn n µ ∂+ = −
∂hh h
hGradients in all the dimensions of h
Step-size
J(h)
h0
h1
.x
2-tap example
Adaptive Filters and SIMO System Identification 6
Optimal Filtering
Steepest descent is an iterative (adaptive) algorithm to find the minimum point in the cost functionSteepest descent ideally attains the optimal solution given by the Wiener-Hopf equations
where is the autocorrelation matrixand is the cross-correlation vector
$ 1opt
−=h R p
( ) ( )TE n n⎡ ⎤= ⎣ ⎦R x x[ ]( ) ( )E n d n=p x
Adaptive Filters and SIMO System Identification 7
The Least-Mean-Square Adaptive Algorithm
Least-Mean-Square is an approximation to the steepest gradient approach.
True gradient
Instantaneous approximation uses the product
The expectation has been removed – gradient noise is expected
Tap Update Equation:
[ ]( ) 2 ( ) ( )k
k
J E x n k e nh
∂= − −
∂h
( ) ( )x n k e n−
$ $( 1) ( ) ( ) ( )n n n e nµ+ = +h h x
Adaptive Filters and SIMO System Identification 8
The Normalized NLMS ‘Workhorse’
The Normalized LMS (NLMS) is popular in practicePerformance is less dependent on data properties
This is the workhorse of adaptive DSP• Low complexity• Can be run using fixed point arithmetic• Noise robust
$ $2( 1) ( ) ( ) ( )
( )n n n e n
nµ
δ+ = +
+h h x
x
Adaptive Filters and SIMO System Identification 9
Classes of Algorithms
SupervisedA reference signal used to generate an error that can be minimised
Single ChannelOne observationIdentifiy a single channel system
BlindNo reference signal available
MultichannelMore than one observationIdentify several systems
How much information is available? HarderEasier
How many assumptions must we make?
Adaptive Filters and SIMO System Identification 10
Single Channel System Identification
Example: echo cancellation
H0(z)
-+
x(n)
e(n)
H0(z)
Adaptive Filters and SIMO System Identification 11
Performance Evaluation
Segmental mean square error (MSE)Measures the output errorDivide error signal into blocks of chosen lengthCompute MSE in each blockOften expressed in dB as a power ratio between d(n) and e(n)
System MisalignmentMeasures the error in the system identification
dB
Where is the sum of the squared elements of the vector
$2
210 2
2
( )10log
nE⎛ ⎞−⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠
h h
h
2
2
Adaptive Filters and SIMO System Identification 12
AEC Performance – White noise input
0 2000 4000 6000 8000 10000 12000 14000 16000
-50
0
50
Near End
AEC P erformance
0 2000 4000 6000 8000 10000 12000 14000 16000
-50
0
50
Error
0 10 20 30 40 50 60 700
5
10
15
ERLE (dB)
Adaptive Filters and SIMO System Identification 13
AEC Performance – speech input
0 1 2 3 4 5 6 7 8 9
x 104
-0.2
0
0.2
Near End
AEC P erformance
0 1 2 3 4 5 6 7 8 9
x 104
-0.2
0
0.2
Error
50 100 150 200 250 300 350
0
10
20
ERLE (dB)
Adaptive Filters and SIMO System Identification 14
Unsupervised Single Channel Technique
Blind Signal ExtractionTo extract a particular signal from a mixture of signals and noise‘Blind’ because we don’t know the signals and we don’t know how they have been mixed
Consider N signals in a vectorand an MxN mixing matrix A
where x(n) are the observed mixturesNow consider demixed signals y(n) and a demixing matrix W applied to x
1 2( ) [ ( ), ( ), , ( )]TNn s n s n s n=s K
( ) ( )n n=x As
( ) ( )n n=y Wx
Adaptive Filters and SIMO System Identification 15
Approach
AssumptionsA is an MxN matrix of rank Nand M≤ΝThe signal to be extracted was generated by a nonwhite source signal generated by an AR process
)(1 nx M
)(11 nx
)(12 nx
11w
12w )(1 ny1−Z
1−Z
1−Z
11~w
12~w
11~
Γw
)(ˆ1 ny)( 1 ne
)1(1 −ny
)2(1 −ny
)( 11 Γ−ny
∑ ∑
Mw1
∑
Blind Signal Extraction Mechanism
Adaptive Filters and SIMO System Identification 16
Results
better
worse
Adaptive Filters and SIMO System Identification 17
Blind Multichannel System Identification
The aim is to estimate the channels from the observations alone.
Adaptive Filters and SIMO System Identification 18
Approach
The only fact that is known is that all the observations originate from the same source. An error function can therefore be written:
( ) ( ) ( ) , 1, 2,...,T Ti j j ie n n n i j M= − =x h x h
Adaptive Filters and SIMO System Identification 19
Performance Evaluation
worse
better
Variable step-size LMS-type adaptationa) state-of-the-art; b) optimal variable step-size; c) theoretical performance bound
Adaptive Filters and SIMO System Identification 20
Inversion
Inversion aims to recover the source signal Convolve the observation with the inverse system estimate
ConsiderationsNon-minimum phase systems result in unstable inverses
• Zeros outside the unit circle in z give rise to unstable poles when invertedMulti-channel systems can be exactly inverted using the MINT methodNote that exact inverse of a system estimate is usually worthless
• Need the system estimate to be perfect• Current research into approximate inverse techniques
Adaptive Filters and SIMO System Identification 21
Summary and Conclusions
Adaptive system identification is a powerful techniqueSupervised algorithms require a reference signalUnsupervised algorithms require other information in lieu of the reference
Assumptions on the nature of the signal or channelMultichannel observations
Open research questions in unsupervised multichannel blind methods are many
Step-sizeNoise robustnessOrder estimation
Adaptive Filters and SIMO System Identification 22
References
Haykin, S., Adaptive Filter Theory, 4th Ed. Prentice Hall, 2002.Morgan, D.R.; Benesty, J.; Sondhi, M.M., “On the evaluation of estimated impulse
responses,” Signal Processing Letters, IEEE , vol.5, no.7 pp.174-176, Jul 1998 Amari, S., Cichocki, A. and Yang, H. H., “Blind signal separation and extraction:
neural and information-theoretic approaches,” in Unsupervised Adaptive Filter, Vol. I: Blind Source Separation, S. Haykin, Ed., pp. 63-138, John Wiley, 2000.
Gaubitch, N. D., Hasan, Md. K. and Naylor, P. A., “Generalized Optimal Step-Size for Blind Multichannel LMS System Identification,” Signal Processing Letters, IEEE , to appear.
Miyoshi, M.; Kaneda, Y., “Inverse filtering of room acoustics,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol.36, no.2 pp.145-152, Feb 1988.
