DSP Lab Part1

7/29/2019 DSP Lab Part1

1/29

Lab Experiment-5

Discrete Cosine transform/Discrete Fourier transform

Aim:

To find a transformation matrix to convert the given sequence of input into Discrete cosine

transform/Discrete Fourier transform, using change of basis method.

Theory:

DFT

A discrete Fourier transform performs a transformation operation on the given function and

convert it to its frequency domain representation. To have a transformation matrix that performs

DFT operation, it is enough to convert the standard basis of given dimension using the DFT

equation which is given as

DFT has many applications and is used in spectral analysis, Data compression etc.

DCT

A discrete cosine transform (DCT) expresses a sequence of finitely many data points in terms of

sum of cosine functions oscillating at different frequencies. To have a transformation matrix that

perform DCT operation it is enough to convert the standard basis of given dimension using theDCT equation which is given as..

DCT also has many applications in mp3 audio compression, image compression etc


2/29

Matlab code

%Program to find the transformation matrix corresponding to DFT and DCT

%computing the transformation matrix of N point DFT

clcclear

N=input('Enter the dimension of the Signal space for finding the DFT transformation matrix :')

I=eye(N);

%implementing the DFT equationfor k=1:N

for n=1:N

x_n=I(:,k);X_k1(n,k)=sum(x_n.*exp(-1i*2*(pi/N)*(k-1)*(n-1)));

end

end

%Implementing the DCT transformation matrix

for k=1:N

for n=1:N

x_n=I(:,k);X_k2(n,k)=sum(x_n.*cos((pi/N)*((n-1)+0.5)*(k-1)));

end

enddisp('The DFT transformation matrix for the dimension that you entered is:')

X_k1

disp('The DCT transformation matrix for the dimension that you entered is:')

X_k2

Results

Enter the dimension of the Signal space for finding the DFT transformation matrix :

N =4

The DFT transformtion matrix for the dimension that you entered is:

X_k1 = 1.0000 1.0000 1.0000 1.0000

1.0000 0.0000 - 1.0000i -1.0000 - i -0.0000 + 1.0000i

1.0000 -1.0000 - 0.0000i 1.0000 + 0.0000i -1.0000 - 0.0000i

1.0000 -0.0000 + 1.0000i -1.0000 - 0.0000i 0.0000 - 1.0000i

The DCT transformtion matrix for the dimension that you entered is:


3/29

X_k2 =

1.0000 0.9239 0.7071 0.3827

1.0000 0.3827 -0.7071 -0.9239

1.0000 -0.3827 -0.7071 0.9239

1.0000 -0.9239 0.7071 -0.3827

Discussion

We have sucessfully implemented DCT/DFT transformation matrix which would perform

transformation operation on any vector of a given finite dimensional vector space. We have

implimented it by tranforming the standard basis for the given dimension.


4/29

Lab Experiment-6

Eigen values and Eigen vectors

Aim:

To find the Eigen values and Eigen vectors of any given m x m matrix

Theory:

An eigenvector of square matrix is a vector which, when, multiplied by the matrix, gives the

result as a scaled version of the original vector

. Ax=x

where is the scaling factor and is called the eigen value. The associated vector X is called eigen

vector of A, where A is a matrix of .

Implementation:

Finding eigen values

To find eigen values, we find the values of which satisfy the characteristic equation of the

matrix A, namely those values of for which

det(A I) = 0

where I is the mm identity matrix.

Finding eigen vectors

Once the eigen values of a matrix (A) have been found, we can find the eigenvectors by finding

the null space of (AI)

Eigen values and eigen vectors

Matlab code

clear

clc

eigvec_mat=[];A=input('Enter the matrix whose Eigen values are to found:')

[m,n]=size(A);

if((m~=n))%||(det(A)==0))

disp('eigen values cannot be found for this matrix')else

I=eye(n);


5/29

y=sym('lambda');

d=det(A-(y.*I));

c=fliplr(coeffs(d));

for i=1:length(c)

c1(i)=double(c(i));end% finding the eigen values

disp('the eigen values of the entered matrix are :')

e_values=double(roots(c1))end

%finding the eigen vectors of the matrix A

for i=1:length(e_values)A_lambda_I=A-e_values(i).*I;

e1=e_values(i);

eigvec=null(A_lambda_I);eigvec_mat=[eigvec_mat eigvec];

%sprintf('the orthonormal eigen vector corresponding to eigen value=%d is %d',e1,eigvec')

End

eigvec_mat

[ ch_eigen_vec, ch_eigen_val]=eig(A)

Results

Enter the matrix whose eigen values are to found:

A = 1 -2 1

-4 2 3

1 6 3

the eigen values of the entered matrix are :

e_values = 6.8898,

-3.6178

2.7281

eigvec_mat =

-0.0446 0.4172 0.6102

0.5487 0.6410 -0.1371

0.8349 -0.6442 0.7803


6/29

Check of program

[ ch_eigen_vec, ch_eigen_val]=eig(A)

ch_eigen_vec = 0.4172 0.6102 -0.0446

0.6410 -0.1371 0.5487

-0.6442 0.7803 0.8349

ch_eigen_val = -3.6178 0 0

0 2.7281 0

0 0 6.8898

Discussion

We have sucessfully implemented the program for finding Eigen values, Eigen vectors for anygiven m x m matrix and also verified our program by using built in command.


7/29

Lab Experiment 8

Applications of Eigen values and Eigen Vectors

Aim

To explore some of the applications of eigen values and eigen vectors of a matrix, i.e finding

out the solutions of a set of differential equations characterizing a continuous time system using

the eigen values and eigen vectors of the system matrix and diagonalization of a matrix.

Theory

Diagonalization: Suppose we have an n x n matrix A has n independent eigen vectors x1,x2, ...

xn. A matrix formed with x1,x2 x3... xn as columns be S. Then S-1

AS is the diagonal eigen value

matrix D.

D =

for a 3 x 3 matrix A.

Solution of differential equations: Consider a system of linear differential equations given by

= Au

Here A is a constant matrix. Hence the solutions will be of the form x

Thus the steps for solving the difference equations are :

1. Find the eigen values is and the n independent eigen vectors xi of A.2. Write the initial vector u(0) as a combination cx1+c2x2... of the eigen vectors

Then u(t) = c1x1 + c2x2+ ....... is the desired solution.

Enter the matrix to be diagonalized:[1 3 4;3 4 5;6 7 8]

The diagonal representation of A S^(-1)AS :

14.6520 -0.0000 0

0.0000 -1.5170 -0.0000


8/29

0.0000 -0.0000 -0.1350

The diagonal eigen value matrix :

14.6520 0 0

0 -1.5170 0

0 0 -0.1350

Enter the coefficients of a system of differential equations row-wise: [-2 1;1 -2]

Enter the initial state:[1 0]

Solution of the given differential equation is :

1/2*(1/2*2^(1/2)*exp(-3*t)+1/2*2^(1/2))*2^(1/2)+1/2*(1/2*2^(1/2)+1/2*2^(1/2)*exp(-

t))*2^(1/2)

1/2*(-1/2*2^(1/2)*exp(-3*t)+1/2*2^(1/2))*2^(1/2)+1/2*(-1/2*2^(1/2)+1/2*2^(1/2)*exp(-

t))*2^(1/2)


9/29

Lab Experiment-7

Karhunen-Loeve TransformAim:

To apply Karhunen-Loeve transformation to given image and also show that the image can be

reconstructed with fewer number of eigen values and eigen vectors.

Theory:

Karhunen-Loeve transformation is a transformation in which the transform kernal is not fixed.In

KLT the transform kernal is dependent on the data or we can say the kernal is derived from data .

The data is represented in the form of vectors. we define the transformation as.

Y=A(Xi-x)

Y is the transformed matrix, A is the eigen vector matrix of the covariance matrix of dataarranged according to desending order of magnitude of its corresponding eigen values , x is the

mean value (row sum) vector of the input data.

The covariance matrix is obtained from the given equation

Cx=E { (Xi- x) x (Xi- x)T

}

We can easly reconstruct our image back by applying inverse operation

Xi=ATY+ x

KLT closely related to the Principal Component Analysis (PCA) and widely used in data

analysis in many fields.


10/29

Karhunen-Loeve Transform

Matlab code

% TO implement Karhunen Loeve Transformclc

clear

X=imread('lena_128.jpg');X=double(X);

[m,n]=size(X);

sum_cov_mat=zeros(m,m);for i=1:m

r(i)=mean(X(i,:));

end

mean_vec=r';for i=1:n

diff_mean_vec(:,i)=X(:,i)-mean_vec;

end

for i=1:n

temp=diff_mean_vec(:,i);

temp_cov_mat=temp*temp';sum_cov_mat=sum_cov_mat+temp_cov_mat;

end

avg_cov_mat=(1/n).*sum_cov_mat;eig_vec=eig(avg_cov_mat);

[eigen_vec,eigen_values]=eig(avg_cov_mat);

eigen_vec=fliplr(eigen_vec);%arranging the eigen vectors as columns corresponding to%decreasig order of magnitude of eigen values

eigen_vec=eigen_vec';

A=eigen_vec;%--------------------------------------------------------------------------

%% Uncomment this part and comment the next part to see perfect reconstruction

%%start of perfect reconstruction procedure% for i=1:length(mean_vec)

% Y(:,i)=A*(X(:,i)-mean_vec);

% end

%

% for i=1:length(mean_vec)% Reconstructed_image(:,i)=A'*Y(:,i)+mean_vec;

% end

% Reconstructed_image=uint8(Reconstructed_image);% imshow(Reconstructed_image)

%%End of Perfect reconstruction procedure

%--------------------------------------------------------------------------% In the above part,we have acheived Perfect reconstruction since we have


11/29

% considered all the eigen values

%--------------------------------------------------------------------------

%Now we are going for approximate reconstruction so that we will get%compression

n_values=input ('enter the no of largest eigen values to be considered: ')

for i=1:n_valuesA_k(i,:)=A(i,:);end

for i=1:length(A_k)Y_k(:,i)=A_k*(X(:,i)-mean_vec);

end

for i=1:length(A_k)Reconstructed_image(:,i)=A_k'*Y_k(:,i)+mean_vec;

end

%--------------------------------------------------------------------------%Calculating the mean square error

%%WORK IN PROGRESS

% for i=1:n_values

% mod_eig(i)=eig_vec(i);% end

for i=1:mfor j=1:n

error_reconstruction(i,j)=X(i,j)-Reconstructed_image(i,j);

square_error(i,j)=(error_reconstruction(i,j))^2;

endend

disp('The Mean square error between Original image and reconstructed image is: ')

MSE=sum(sum(square_error))/(m*n)disp('PSNR in dB')

PSNR=20*log10(max(max(X))/sqrt(MSE))

%%-------------------------------------------------------------------------Reconstructed_image=uint8(Reconstructed_image);

figure

imshow(Reconstructed_image)

title('Reconstructed image')figure

imshow(uint8(X))

title('Original image')


12/29

Results

Test 1

enter the no of largest eigen values to be considered: 10

n_values =10

The Mean square error between Original image and reconstructed image is:

MSE =346.6516

PSNR in dB

PSNR =22.1326

Test 2

enter the no of largest eigen values to be considered:

n_values =30


MSE =65.8073

PSNR in dB

PSNR =29.3488

Reconstructed image Original image


13/29

Test3

enter the no of largest eigen values to be considered: 50

n_values =50


MSE =17.1151

PSNR in dB

PSNR =35.1978

DiscussionWe have sucessfully implemented a program for finding KLT of the given image and have also

studied the difference in the represantation of the image with different number of eigen values of

the covariance matrix. As the number of eigen values and eigen vectors considered for

reconstruction increases, we will get better reconstruction of the image. This technique can be

effectively used for compression of images. The disadvantage is the calculation of eigen values

and eigen vectores for large matrices.

Reconstructed imageOriginal image

Reconstructed image

Original image


14/29

Lab Experiment-8

Singular value decomposition

Aim:

To perform singular value decomposition (SVD) on any given m x n matrix

Theory:

The singular value decomposition (SVD) is a factorization of any real or complex matix of the

order m x n. Any m x n matrix can be decomposed in the form

A=U V* (here * operation indicates conjugate-transpose operation)

The left-singular vectors U are eigenvectors ofAA*.

The right-singular vectors of V are eigenvectors of A*A.

is a diagoanl matrix containing the square roots of the eigenvalues of AA* or A*A, depending

upon whichever is larger

By applying SVD we can get a set of basis functions for the four important subspaces of a given

dimensional vector space.

1 Orthonormal basis for Row space A

2 Orthonormal basis for column space A

3 Orthonormal basis for null space A

4 Orthonormal basis for null space of AT


15/29

SVD

Matlab code

% to implement singular value decomposition

clc

clearA=input('Enter the Matrix: ');

[m n]=size(A);

A_symm_V=A*A';

A_symm_U=A'*A;[eig_vec_V eig_val_V ]=eig(A_symm_V);

[eig_vec_U eig_val_U ]=eig(A_symm_U);

n1=max(size(A_symm_U));n2=max(size(A_symm_V));

temp1=zeros(n1,n1);temp2=zeros(n2,n2);

for i=1:n1

temp1(i,i)=eig_val_U(n1+1-i,n1+1-i);

endA_symm_U=sqrt(temp1);

eig_vec_U=fliplr(eig_vec_U);

for i=1:n2

temp2(i,i)=eig_val_V(n2+1-i,n2+1-i);end

A_symm_V=sqrt(temp2);eig_vec_V=fliplr(eig_vec_V);

sig=zeros(m,n);

for i=1:mfor j=1:n

if (n1>=n2)

sig(i,j)=A_symm_U(i,j);

else

sig(i,j)=A_symm_V(i,j);end

endend

% values to be returned in the case of a function

V=eig_vec_Usigma=sig

U=eig_vec_V


16/29

Results

V =

-0.8944 0.4472

-0.4472 -0.8944

sigma =

3.1623 0

0 2.2361

0 0

U =

0.5657 0.6000 -0.5657

-0.4243 0.8000 0.4243

0.7071 0 0.7071

A =

1 2

-2 1

2 1

Rec_A =

-1 -2

2 -1

-2 -1

Cross check

[U,sigma,V]=svd(A)

U = 0.5657 0.6000 -0.5657

- 0.4243 0.8000 0.4243

0.7071 0.0000 0.7071


17/29

sigma = 3.1623 0

0 2.2361

V = 0.8944 -0.4472

0.4472 0.8944

Discussion

We have successfully implemented a program for performing SVD for any given m x n matrix.


18/29

Lab Experiment-9-a

Generation of Random variables

Aim: To be familiar pdf/pmf and CDF of different probability distributions.

Theory:

Exponential distribution

The exponential distributiondescribes the arrival time of a randomly recurring independent event

sequence. is the mean waiting time for the next event recurrence. Its probability densit

function is

CDF is given as

Poisson distribution

The Poisson distributionis the probability distribution of independent event occurrences in an

interval. Ifis the mean occurrence per interval, then the probability of having x occurrences

within a given interval is:

K=0, 1, 2, 3,

Binomial distributionThe binomial distributionis a discrete probability distribution. It describes the outcome o

nindependent trials in an experiment. Each trial is assumed to have only two outcomes, either

success or failure. If the probability of a success in a trial isp, then the probability ohaving ksuccessful outcomes in an experiment ofn independent trials is as follows.

k=0, 1, 2, 3,


19/29

Normal distribution

The normal distributionis defined by the following probability density function, where is

the population mean and 2

is the variance.

If a random variableXfollows the normal distribution, then we write:

In particular, the normal distribution with = 0 and = 1 is called the standard normal

distribution, and is denoted as N(0,1).

Bernoulli distribution

The Bernoulli distribution is a discrete distribution having two possible outcomes labeledby n=0 and n=1 in which n=1 ("success") occurs with probability P and n=0 ("failure") occurs

with probability q=1-p, where 0


20/29

Probability distributions

Matlab code for continuous Random variables

% Generation of continuous random variables using Inverse Transform

% Generating U(0,1)

clc;

clear all;

close all;U = rand(1,1000);

% Uniform distribution U(a,b)

a = 1;

b = 100;

X = a+(b-a).*U;

[y,axis_x]=hist(X);

fx = y/1000;

subplot(2,1,1),bar(axis_x,y);

title('Histogram');

subplot(2,1,2),plot(axis_x,fx);title('Uniform density function');

axis([0 100 0 1]);

% Exponential distribution exp(lambda)

lambda = 5;

X = -lambda*log(U);

[y,axis_x]=hist(X);

fx = y/1000;

figure


title('Histogram');

subplot(2,1,2),plot(axis_x,fx);

title('Exponential density function');

axis([0 100 0 1]);

% Normal distribution N(m,sigma)

m = 5;

sigma = 0.5;


21/29

U1 = rand(1,1000);

U2 = rand(1,1000);

X1 = sqrt(-2.*log(U1)).*cos(2*pi*U2);

X = m + sigma.*X1;

[y,axis_x]=hist(X);

fx = y/1000;

figure


title('Histogram');


title('Normal density function');

axis([0 10 0 0.5]);

% Laplacian distribution with mean 0 and variance 2

U1 = rand(1,1000);

U2 = rand(1,1000);for n = 1:1000

if U1(n)


22/29

axis([0 25 0 1]);

Matlab code for discrete Random variables

clc;

clear all;

close all;

U = rand(1,1000);

% Bernoulli distribution of parameter p

p = 0.60;

for n = 1:1000

if U(n)


23/29

fx = y/1000;

figure


title('Histogram');

subplot(2,1,2),stem(axis_x,fx);

title('Binomial probability mass function');

axis([0 10 0 1])

Results


24/29


25/29


26/29

Discussion

We have successfully generated random variables like Uniform, Exponential, Rayleigh and

Laplace distributions from uniformly distributed random variable U in (0,1) using inverse

transform method. Normal distribution was generated using Box-Muller method. Bernoulli

random variable was generated by modeling an experiment which generates it and Binomialrandom variable was generated as a sum of Bernoulli random variables.


27/29

Lab Experiment-9-b

Verification of Central Limit Theorem

Aim:

To visualize Central Limit Theorem

Thoery:The central limit theorem is one of the most remarkable results in probability theory.Let

X1, X2, X3.. be a sequence of independent, identically distributed random variables each

with mean and variance 2. Then the distribution of

tends to the standard normal distribution as n . That is

n

This theorem holds for any distribution of Xi

Central Limit Theorem

Matlab code

% Verification of central limit theorem%By Generating n iid random variables following exponential distributionclc;

clearall;

size=10000;U = rand(1,size);

lambda = 5;

n = 1000;X = zeros(1,size);

X_exp=-lambda*log(rand(1,size));

[y_,axis_x_exp]=hist(X_exp);

fx_ = y_/size;subplot(2,1,1),bar(axis_x_exp,y_);

title('Histogram');

subplot(2,1,2),plot(axis_x_exp,fx_);title('pdf of exponential random variable')

fori = 1:n

X1= -lambda*log(rand(1,size));

n

nn

XXX

....21

dx

a 22

xe

2

1a}

n

nn

X....2

X1

X

P{


28/29

X = X + X1; % generating a sum 'n'of exponential random variables

end

X = (X-mean(X))./sqrt(var(X));[y,axis_x]=hist(X);

fx = y/size;

figuresubplot(2,1,1),bar(axis_x,y);title('Histogram');


title('pdf of the normalised sum random variable')

Results

0 5 10 15 20 25 30 35 40 450

2000

4000

6000Histogram

0 5 10 15 20 25 30 35 40 450

0.2

0.4

0.6

0.8pdf of exponential random variable


29/29

Discussion

We have verified the central limit theorem by generating a normalized sum of n iid random

variables (we have taken iid random variables as the exponential random variables in the

program) and observing that this resulting random variable has a Gaussian distribution.

-4 -3 -2 -1 0 1 2 3 40

1000

2000

3000Histogram

-4 -3 -2 -1 0 1 2 30

0.1

0.2

0.3

0.4pdf of the normalised sum random variable

Date post:	04-Apr-2018
Category:	Documents
Upload:	vineeth-kumar
View:	237 times
Download:	0 times

DSP Lab Part1

Documents