Date post: | 13-Apr-2017 |
Category: |
Documents |
Upload: | naveena-vemulapalli |
View: | 153 times |
Download: | 1 times |
EE-4253 RTSP PROJECT
BLIND SOURCE SEPARATION USING
ICA & SFA
By
Naveena Vemulapalli
Ali Aghdaei
I. INTRODUCTION
Blind Source Separation also called as the Blind Signal Separation is the method used to
separate the set of source signals from the mixed signals. Blind source separation has many
methods but in this project we mainly concentrate on two methods that is Independent
Component Analysis (ICA) and the Signal Fraction Analysis (SFA). In general when we have
a large amount of data for analysis it becomes very important to reduce this data, scientists in
many fields collect the measurements and observe different techniques to describe observe and
analyze the data that is collected. Selecting an appropriate technique is very important to
segregate this data
Some of the popular data analysis techniques for the data compression is Independent
Component Analysis (ICA), Principal Component Analysis (PCA) and the Signal Fraction
Analysis (SFA). Independent Component Analysis is a technique that is for the data sets that
are Non-Gaussian and are independent to each other. Principal Component Analysis is based
on the orthogonal transformation of the signals. It converts the correlated signals into
uncorrelated signal to the correlated signal.
II. INDEPENDENT COMPONENT ANALYSIS
Independent Component Analysis is mainly based on the assumption of having non Gaussian
signals which are independent of each other. These two conditions should always be true to
provide a good results. Although having some disadvantages this method is one of the best
method in the Blind Source Separation. This method is used to separate the signals which are
mixed together with the unknown mixing matrix. The algorithm for the Independent
Component Analysis is given as follows.
Let 𝑆𝑇 = [𝑠1, 𝑠2, 𝑠3, … … … … … . . 𝑠𝑛] be a source vector and 𝑋𝑇 = [𝑥1, 𝑥2, … … … . 𝑥𝑛] be the
source mixture random vector. The mixing matrix is considered to be A. using the matrix
notation we have equation (1)
𝑋 = 𝐴𝑆 (1)
In this above equation we know only X. we have no information about the A matrix and the S
matrix. We know that the elements of S are non-gaussian and statistically independent.
Therefore let us denote the linear combination of elements of X by y as shown in equation (2)
𝑦 = 𝑊𝑇𝑋 (2)
From equation 1 we can write the equation 3
𝑦 = 𝑊𝑇𝑋 = 𝑊𝑇𝐴𝑆 = 𝑍𝑇𝑆 (3)
The main matrix that is to be determined is the matrix W. From the equation (3) we know that
y is a linear combination of the original independent non-gaussian signals. According the
central limit theorem we know that the sum of all the non-gaussian independent signals leads
to the gaussian signal. Hence we should measure the amount of non-Gaussianity to apply on y
to find the separated independent sources. One of the popular method to measure the non-
Gaussianity is called the Kurtosis. In the Independent Component Analysis we need to have
several estimations to estimate the maximum likelihood which leads to the estimation of higher
order cumulants. This is called the kurtosis. The kurtosis of y is defined as shown in the
equation (4)
𝑘𝑢𝑟𝑡(𝑦) = 𝐸(𝑦4) − 3[𝐸(𝑦2)]2 (4)
Let us assume that y has a unity variance then the equation (4) becomes
𝑘𝑢𝑟𝑡(𝑦) = 𝐸(𝑦4) − 3 (5)
This kurtosis generally has three kinds of values which are shown below
Kurtosis>0 Sup-Gaussian
Kurtosis=0 Gaussian
Kurtosis >0 Sup-Gaussian
For the gaussian random variables the shape of the random variable is bell shape. For the sub-
Gaussian random variables the shape is flatter than the Gaussian. For the Sup-Gaussian the
shape is sharper and has a long tail. Applying fixed point algorithm to optimize the kurtosis
determines the matrix W through which we can determine the mixing matrix A.ICA also solves
the non-linear optimization problem which needs more complicated algorithms.
III. SIGNAL FRACTION ANALYSIS
In this section we intend to describe SFA algorithm which is one of Blind Source Separation
techniques. We assume that received signal is composed of two additive components. Let, X;
n > p be a multivariate data set, an addition of the desired data (signal) S, and undesired signal
(noise) N. Therefore, the observed data can be represented in equation (6)
𝑋 = 𝑆 + 𝑁 (6)
From the above equation, we conclude that:
𝑋𝑇𝑋 = 𝑆𝑇𝑆 + 𝑁𝑇𝑁 + 𝑆𝑇𝑁 + 𝑁𝑇𝑆 (7)
Then we define the signal to noise ratio (SNR), the ration of the variance of the signal to
variance of the noise is given in equation (8)
𝑆𝑁𝑅 = 𝑣𝑎𝑟 (𝑆)
𝑣𝑎𝑟 (𝑁) (8)
In addition, noise fraction is defined as the ratio of noise variance to the total variance. That is
given in equation (9),
𝑢 = 𝑣𝑎𝑟 (𝑁)
𝑣𝑎𝑟 (𝑋) (9)
The maximum signal fraction transformation is a linear transformation defined in equation (10)
𝑌 = 𝑋𝑎 = 𝑆𝑎 + 𝑁𝑎 (10)
Where a is a p by 1 vector. Here, the problem is to determine a in order to maximize the signal
to noise ratio for Y. The SNR for Y is represented by equation (11)
𝑆𝑁𝑅 = 𝑣𝑎𝑟 (𝑆𝑎)
𝑣𝑎𝑟 (𝑁𝑎) (12)
Thus,
𝑆𝑁𝑅 = 𝐸(𝑎𝑇𝑆𝑇𝑆𝑎) − 𝐸(𝑎𝑇𝑆𝑇)𝐸(𝑠𝑎)
𝐸(𝑎𝑇𝑁𝑇𝑁𝑎) − 𝐸(𝑎𝑇𝑁𝑇)𝐸(𝑁𝑎)
Considering this condition that all the data is zero mean, therefore:
𝑆𝑁𝑅 = 𝐸(𝑎𝑇𝑆𝑇𝑆𝑎)
𝐸(𝑎𝑇𝑁𝑇𝑁𝑎) (13)
From the equation (13) we get equation 14
𝑎𝑇𝑋𝑇𝑋𝑎 = 𝑎𝑇𝑆𝑇𝑆𝑎 + 𝑎𝑇𝑁𝑇𝑁𝑎 + 𝑎𝑇𝑆𝑇𝑁𝑎 + 𝑎𝑇𝑁𝑇𝑆𝑎 (14)
Therefore we will get equation 15
𝑆𝑁𝑅 = 𝐸(𝑎𝑇𝑋𝑇𝑋𝑎) − 𝐸(𝑎𝑇𝑆𝑇𝑁𝑎) − 𝐸(𝑎𝑇𝑁𝑇𝑆𝑎 − 𝐸(𝑎𝑇𝑁𝑇𝑁𝑎)
𝐸(𝑎𝑇𝑁𝑇𝑁𝑎) (15)
We assume that signal and noise are independent of each other, thus the second term in the
nominator of the above equation will be zero. The equation will be simplified to equation (16)
𝑆𝑁𝑅 = 𝐸(𝑎𝑇𝑋𝑇𝑋𝑎)
𝐸(𝑎𝑇𝑁𝑇𝑁𝑎) -1 (16)
Since we don't have access to the statistical information, we must use the estimation of the
covariance matrices to compute the SNR. Assuming n large, we can estimate the statistical
means via their sample mean. In this case, we have:
𝑆𝑁𝑅 = (𝑎𝑇𝑋𝑇𝑋𝑎)
(𝑎𝑇𝑁𝑇𝑁𝑎) -1
The SFA technique is based on maximizing the signal to noise ratio powers. Therefore, to
maximize the SNR we have to maximize the term
𝐷(𝑎) = (𝑎𝑇𝑋𝑇𝑋𝑎)
(𝑎𝑇𝑁𝑇𝑁𝑎) -1
This optimization problem is solved by taking the derivative of D(a)with respect to a and
setting it to zero
𝜕𝐷(𝑎)
𝜕𝑎=
2𝑋𝑇𝑋𝑎(𝑎𝑇𝑁𝑇𝑁𝑎) − 2𝑎𝑇𝑁𝑇𝑁𝑎(𝑎𝑇𝑋𝑇𝑋𝑎)
𝑎𝑇𝑁𝑇𝑁𝑎 (17)
XTXa(𝑎𝑇𝑁𝑇𝑁𝑎) = 𝑁𝑇𝑁𝑎(𝑎𝑇XTXa)
Here we define 𝛼 and 𝛽 as:
𝛼 = 𝑎𝑇𝑋𝑇𝑋𝑎
𝛽 = 𝑎𝑇𝑁𝑇𝑁𝑎
Thus we will get equation (18)
β𝑋𝑇𝑋𝑎 = α𝑁𝑇𝑁𝑎 (18) Now, defining
𝜆 =𝛽
𝛼 for 𝛽 ≠ 0
Thus:
𝜆 = 𝑆𝑁𝑅 + 1
And finally we have:
𝑋𝑇𝑋𝑎 = 𝜆𝑁𝑇𝑁𝑎
In order to solve above equation and calculate a we use eig( ) command in MATLAB while a
is the generalized singular vector and 𝜆 is the generalized singular value associated to a that
shows the connection between SFA and GSVD.
Therefore, given matrix X we can recover S by applying SFA under the following
Conditions:
1. n must be large enough (i.e. is a tall matrix) in order to estimate the statistical mean via
the sample mean.
2. E(S) = E (N) = E(X) = 0 (i.e., the columns of S, N and X are assumed to have zero mean).
3. Signals are independent from the noise, (i.e., E(STN) = E(NTS) = 0).
The generalized singular vectors (ai 's) are orthonormal with respect to matrix NTN,
and are orthogonal with respect to matrix XTX. FFA algorithm is shown in the below figure in
a nutshell:
Estimation of noise covariance matrix:
In SFA algorithm noise covariance matrix should be estimated. There are many methods in
order to calculate that, but we are going to introduce differencing method here.
As we explained before, the observed data can be represented by X = S +N. We denote the
difference matrix as dX in which each row i (i.e., dX(i; :)) is the difference between the (i + 1)-
th row of X (i.e., X(i + 1; :)) and the i-th row of X (i.e., X(i; :))).Suppose that we can estimate
the noise covariance using the difference matrix as: 𝑁𝐻𝑁 =𝑑𝑋𝐻𝑑𝑋
2.
Therefore:
Then,
Where 𝑋 ∈ 𝑅𝑛×𝑝 and𝑑𝑋 ∈ 𝑅(𝑛−1)×𝑝. Here we assumed we have the additive noise signal as:
𝑥𝑚(𝑡𝑖) = 𝑠𝑚(𝑡𝑖) + 𝑛𝑚(𝑡𝑖)
Therefore,
𝑥𝑚(𝑡𝑖) − 𝑥𝑚(𝑡𝑖−1) = 𝑠𝑚(𝑡𝑖) − 𝑠𝑚(𝑡𝑖−1) + 𝑛𝑚(𝑡𝑖) − 𝑛𝑚(𝑡𝑖−1)
To simplify, we calculate matrix dX including elements of 𝑥𝑚(𝑡𝑖) − 𝑥𝑚(𝑡𝑖−1) and compute
matrices dS and dN based on elements 𝑠𝑚(𝑡𝑖) − 𝑠𝑚(𝑡𝑖−1) and 𝑛𝑚(𝑡𝑖) − 𝑛𝑚(𝑡𝑖−1) respectively
when m = 1,...p and i = 1,…n.
𝑑𝑋 = 𝑑𝑆 + 𝑑𝑁
Which leads to:
𝑑𝑋𝑇𝑑𝑋 = 𝑑𝑆𝑇𝑑𝑆 + 𝑑𝑁𝑇𝑑𝑁
Here using 𝑑𝑁 = 𝑁𝑡 − 𝑁𝑡−1 , 𝑁𝑡−1𝑇 𝑁𝑡−1 = 𝑁𝑡
𝑇𝑁𝑡 and 𝑁𝑡−1𝑇 𝑁𝑡 = 𝑁𝑡
𝑇𝑁𝑡−1 = 0 , thus:
𝑑𝑋𝑇𝑑𝑋 = 𝑑𝑆𝑇𝑑𝑆 + 2𝑁𝑡𝑇𝑁𝑡
Assuming the sampling rate is high enough and therefore it leads to 𝑑𝑆 ≈ 0
𝑑𝑋𝑇𝑑𝑋 = 2𝑁𝑡𝑇𝑁𝑡
This means that we could use the differencing idea for the observed data and apply it to
estimate the noise covariance matrix.
IV PERFORMANCE EVALUATION:
The performance of the ICA and the SFA techniques is evaluated using the Normalized Mean
Square Error (NMSE). The equation for NMSE is shown in equation (19). NMSE is difference
between the observed value and the predicted value. The performance of the filter is better if
this value is as low as possible.
𝑚𝑠𝑒 =∑ |𝑠(𝑛) − 𝑠(𝑛)̂|
2𝑁𝑛=1
∑ |𝑠(𝑛)|2𝑁𝑛=1
(19)
The second performance metric that is used for the performance evaluation is the correlation.
Correlation is defined as the dependence of the signals on each other. The equation for
correlation is shown in equation (20). The measured signal changes as the original signal
changes.
𝑐𝑜𝑟𝑟 =1
𝑁∑ 𝑠(𝑛)̂ − 𝑠(𝑛) (20)𝑛
𝑛=1
V. SIMULATION RESULTS FOR ICA:
The simulation result for the ICA is shown below. The first signal is the voice signal and the
second signal is the music signal. These signals are selected because the voice and the music
is always independent of each other. For proper reconstruction of the signals the signals should
be independent of each other. The signals are first converted to a .wav file using the software
Audacity. The signals are sampled at the rate of 44100Hz. The length of both the signals is
made equal so as avoid any irregularities. The noise is adding to the signals and here we
consider 2 observed signals.
Figure1: The original Voice and Music Signal
Figure 2: The observed Voice and music signal
Figure 3: The reconstructed Voice and the music signal
SNR
NMSE
MUSIC VOICE S1 S2
52.488 9.1261 0.0040 0.0531
5.2488 0.9126 0.0028 0.0525
0.5248 0.09126 0.0030 0.0432
0.0525 0.00913 0.0030 0.0438
Table 1: Effect of the SNR on NMSE in ICA process
The NMSE for the independent Component Analysis decreases as the SNR increases but only
up to a certain value and then the NMSE increases. This performance is somewhat similar to
the SFA. But here in the ICA the breaking threshold is less than that of SFA. Therefore we can
say that the performance of SFA is better than that of the ICA
VI. SIMULATION RESULTS FOR SFA:
The simulation is done by taking two source signals. The first signal is the voice signal and the
second signal is the music signal. These signals are selected because the voice and the music
is always independent of each other. For proper reconstruction of the signals the signals should
be independent of each other. The signals are first converted to a .wav file using the software
Audacity. The signals are sampled at the rate of 44100Hz. The length of both the signals is
made equal so as avoid any irregularities. The noise is adding to the signals and here we
consider 2 observed signals.
Figure 4: The original Voice and the music signal
Figure 5: The observed voice and music signal
Figure 6: The separated voice and the music signal
EFFECT OF ALPHAS ON THE MSE & CORRELATION:
ALPHAS NMSE CORRELATION
𝜶𝟏 𝜶𝟐 𝜶𝟑 𝜶𝟒 S1(MUSIC) S2(VOICE) S1(MUSIC) S2(VOICE)
1 1 1 1 0.0713 0.0123 -0.0866 877.524
1 7 1 7 0.0713 0.0110 -0.5171 14399
7 1 7 1 0.0715 0.0139 0.0319 157.12
7 1 1 1 0.0715 0.0142 11.697 10.6281
7 1 1 7 0.0715 0.0142 6.812 6.9005
1 7 7 1 0.0715 0.0145 6.8271 -6.9113
1 1 1 7 0.0716 0.0141 12.3663 11.0153
Table 2: Effect of the alphas on NMSE in SFA process
From the table that is shown above we can make some interesting observations based on the
alpha values. The alphas that are shown here are nothing but the unknown mixing matrix. This
matrix is generally random and is unknown. As we need to evaluate the performance based on
these alphas we have taken some deterministic values. As the number of observed signals is
taken as 2 we have considered 4 different kinds of alphas. The Table (2) shows all the different
kinds of alphas and there performance on the mean square error and the correlation. From the
table we can observe that the correlation is very high when the values of
𝜶𝟏,𝜶𝟑 and𝜶𝟐,𝜶𝟒 are equal. When these values are equal the correlation between the signals
is high and they cannot be separated properly. When the reconstruction fails in the voice signal
we get the reconstructed signal as the complete noise signal and the music signal is the
observed signal.
EFFECT OF SNR ON THE NMSE:
SNR
VARIANCE
NMSE
CORRELATION
MUSIC VOICE S1 S2 S1 S2
0.00713 0.00137 10 0.0713 0.0137 -0.3991 1.6019
0.0713 0.0137 1 0.0713 0.0137 2.4484 16.1612
0.713 0.137 0.1 0.0711 0.0136 24.504 136.48
7.13 1.37 0.01 0.0710 0.0138 148.75 186.93
71.3 13.7 0.001 0.0715 0.0142 15.98 13.87
Table 3: Effect of the SNR on NMSE in SFA process
Again the noise is also random in nature. In order to evaluate the effect of noise on the
performance of the reconstruction we have considered some deterministic values of the noise.
Firstly, we know that as the SNR increases the MSE decreases. The Table (3) shows the values
of the NMSE for different values of the noise. From the table we can conclude that the up to
the variance the means square error decreases as the signal to noise ratio increases as the Signal
to Noise Ratio increases. If we decrease the variance beyond 0.1 MSE decreases which means
that we can consider the variance 0.1 as the breakpoint and after this the reconstruction of the
signals fail completely. We cannot separate the source signals at all.
VII. CONCLUSION:
Blind Source Separation method is mainly used to separate the source signals that are mixed
together by an unknown mixing matrix. We can conclude that the Signal Fraction analysis is a
better method than the Independent Component Analysis. They are many reasons to explain
this deduction. Firstly, the observed signal always contains the noise and moreover the noise
is also added during the transmission. As we know that in ICA the components should be non-
gaussian but in the presence of the noise ICA cannot distinguish properly from the noise.
Secondly, SFA algorithm is comparatively easy than the ICA algorithm. As ICA requires
several estimations like estimating the maximum likelihood and also kurtosis to measure the
amount of non-Gaussianity leads to the estimation of higher order cumulants. ICA also solves
the nonlinear optimization problem. SFA just needs to solve the GSVD problem.
Using the MATLAB SFA algorithm can be programmed easily and moreover we can evaluate
the effects of performance of different parameters like the alphas, noise variances on the
reconstructed signal is done above. We have clearly observed the cases where the
reconstruction fails and how the separated source signal is changing based on the SFA
algorithm.
VIII. REFERENCES:
[1]DISSERTATION-Signal Fraction Analysis For Subspace Processing Of High Dimensional Data by Fatemeh Emdad Department of Mathematics
IX. APPENDIX:
1) The code for the Independent Component Analysis is shown below
% SFA and ICA Computation and Comparison
close all; clear all;
% reading the Audio Signals
[inp fs] = wavread('H:\best.wav');
[inp2 fs2] = wavread('H:\ChillingMusic.wav');
inp = inp';
inp2 = inp2';
% Giving the Noise variances
Sigma = [0.001 0.01 0.1 1 0.001 0.01 0.1 1];
inp = inp(1,1:50000); VAR1 = var(inp);
inp2 = inp2(1,1:50000); VAR2 = var(inp2);
SNR = [VAR1 ./ Sigma(1,1:4) VAR2 ./ Sigma(1,5:8)]
vect = [inp;inp2];
NN=2;
std=0.1;
% Noisa Generation and adding it to the original signal;
for loop =1:8
for i=1:NN
for j=1:NN
ron(i,j)=1-0.1*abs(i-j);
end
end
R=chol(ron);
noi=randn(2,50000)'*std*R;
AA=rand(2,2)
Phi=[10 20];
for p=1:2;
teta(1:2,p)=(p-1)*pi*cos(Phi(p)*pi/180);
end
obs=AA*vect + Sigma(1,loop)*randn(size(vect))
% ICA Part
[E, D] = pcamat(obs);
[nv, wm, dwm] = whitenv(obs, E, D);
[A, W] = fpica(nv, wm, dwm);
x=W*obs;
figure
for i=1:2;
subplot(2,1,i);hold on
plot(vect(i,:));
title('The original signal');
end
% plotting the figures
figure
for i=1:2;
subplot(2,1,i);hold on
plot(obs(i,:));
title('The observed signal');
end
figure
for i=1:2;
subplot(2,1,i);hold on
plot(x(i,:));
title('The Seperated signals using ICA');
end
if loop <= 4
MSE_voice(loop) = ((sum((x(1,:) - vect(1,:)).^2)) / (sum(vect(1,:)).^2))/(50000)
else
MSE_music(loop) = ((sum((x(2,:) - vect(2,:)).^2)) / (sum(vect(2,:)).^2))/(50000)
end
end
2) The code for the Signal Fraction Analysis is shown below
% SFA and ICA Computation and Comparison
clear all
clc
close all
% Reading the audio signals
[inp fs] = wavread('H:\best.wav');
[inp2 fs2] = wavread('H:\ChillingMusic.wav');
inp = inp';
inp2 = inp2';
% Adding the noise to the signal
Sigma = [0.001 0.01 0.1 1 0.001 0.01 0.1 1];
inp = inp(1,1:50000); VAR1 = var(inp);
inp2 = inp2(1,1:50000); VAR2 = var(inp2);
SNR = [VAR1 ./ Sigma(1,1:4) VAR2 ./ Sigma(1,5:8)]
vect = [inp;inp2];
NN = 2;
Sigma = 0.01;
obs = zeros(2,50000);
AA = abs(rand(2,2));
% generating observed signal
for loop = 1:8
for j = 1:2
obs(j,1:end) = (AA(1,j) .* vect(1,:)) + (AA(2,j) .* vect(2,:)) +...
+ (Sigma .* randn(1,50000));
end
deltaX = zeros(2,50000);
% estimating covariance matrix using differencing method
deltaX(1,1:end) = abs(obs(1+1,1:end) - obs(1,1:end));
deltaX(2,1:length(obs)) = abs(obs(1,1:end) - obs(2,1:end));
noise_cov = deltaX * deltaX';
XTX = obs * obs';
% estimating filter coefficients
[a , lambda] = eig(XTX , (noise_cov/2));
% reconstructing the signal
for i =1:2
s_hat(i,:) = a(:,i)' * obs;
end
% plot original, observed and reconstructed signals
figure
for i=1:2;
subplot(2,1,i);hold on
plot(vect(i,:));
title('The original signal');
end
figure
for i=1:2;
subplot(2,1,i);hold on
plot(obs(i,:));
title('The observed signal');
end
figure
for i=1:2;
subplot(2,1,i);hold on
plot(s_hat(i,:));
title('The Seperated signals using SFA');
end
if loop <= 4
MSE_voice(loop) = ((sum((s_hat(1,:) - vect(1,:)).^2)) / (sum(vect(1,:)).^2))/(50000)
else
MSE_music(loop) = ((sum((s_hat(2,:) - vect(2,:)).^2)) / (sum(vect(2,:)).^2))/(50000)
end
end