ECE 484 Digital Image Processing Lec 16 - Transform Domain ......ECE 484 Digital Image Processing,...

ECE 484 Digital Image Processing Lec 16 - Transform Domain Image Processing I

Dimension Reduction

Zhu LiDept of CSEE, UMKC

Office: FH560E, Email: [email protected], Ph: x 2346.http://l.web.umkc.edu/lizhu

ECE 484 Digital Image Processing, 2019 p.1

slides created with WPS Office Linux and EqualX equation editor

Outline

Recap: Image Processing tricks so far...

Linear Algebra Refresher SVD Principal Component Analysis (PCA) Laplacian Eigen Map (LEM)


Recap of DIP tricks so far

Point Operation (1 x 1)

Linear Filtering (m x m)

Non-Linear Fitlering (m x m)

Freq Domain Filtering (W x H)


(much hyped) Deep Leanring

Classificaiton: Pixels to Label

Image Denoising/Enhancement


HW-4: Deep Learning Denoising

4 types of noisy images: PSF = Gaussian Blur, Motion Blur Additive Gaussian Noise:

Data Set: NWPU Aerial Image Data Set:


HW-4

Task 1: Denoising with VDSR like network prediction: up to you, baseline can just use a gaussian

smoothing residual learning network:

o each conv layer 64 3x3 filterso need to train with the NWPU data set, or any other data set

Task 2: bonus points, Target Decomposition wavelet decomposition (will have a tutorial on this) optimize the wavelet decomposition structure to see which

one works out the best.


Image Recognition Problem - Transform

Image Tagging given pixels in Rhxw

predict its label: {airplane, automobiles, bird, cat, dog,....}

Can we find a projection A, s.t.


Y AX

=

Outline

Recap: Image Processing tricks so far...



Vector and Matrix Notations

Vector

Matrix

p.9ECE 484 Digital Image Processing, 2019

Vector Products

Inner Product

Outer Product


Matrix-Vector Product

y=Ax

So, y is a linear combination of basis {ak} with weights from x


Matrix Product

C=AB

Associative: ABC = (AB)C = A(BC)

Distributive: A(B+C) = AB + AC

p.12

A: nxp B: pxm A: nxm=

ECE 484 Digital Image Processing, 2019

Outer Product/Kron

Vector outer product:

Example


Matrix Transpose

Transpose


Matrix Trace and Determinant

Trace:Tr(A): only for nxn square matrix

Determinant: Det(A): The size of volumes spanned by A, All possible linear combinations of a1 and a2

p.15

Det(A) = |2-9| = 7;


Eigen Values and Eigen Vectors

Definition: for nxn matrix A:

In Matlab: [P, V]=eig(A);


Eigen Vectors of Symmetric Matrix


SVD for non square matrix: A mxn:


SVD as Signal Expansion

p.19

A(mxn) = U(mxm) S(mxn)

V(nxn)

The 1st order SVD approx. of A is:


SVD approximation of an image

Very easy…function [x]=svd_approx(x0, k)dbg=0;if dbg x0= fix(100*randn(4,6)); k=2;end[u, s, v]=svd(x0);[m, n]=size(s);x = zeros(m, n); sgm = diag(s);for j=1:k x = x + sgm(j)*u(:,j)*v(:,j)'; end


SVD for Separable Filtering

Take LoG filter for eg. h=fspecial('LoG', 11, 2.0); [u,s,v]=svd(h); h1=s(1,1)*u(:,1)*v(:,1)';


h1 is 1-SVD approx of LoG

Many implications forDeep networks acceleration !

NormVector Norm: Length of the vector Euclidean Norm (L2 Norm): norm(x, 2)

Lp norm:

Matrix Norm: Forbenius Norm


Quadratic Form

Quadratic form f(x)=xTAx in R:

Positive Definite (PD): For non-zero x, xTAx > 0

Positive Semi-Definite (PSD): For non-zero x, xTAx >= 0

Indefinite: Exists x1, x2 non zero, but x1

TAx1 >0, while x2TAx2 < 0;


Matrix Calculus

Gradient of f(A):

Matrix Gradient Properties


Hessian of f(X)


Outline

Recap: About HW-2 Quiz-1

Linear Algebra Refresher SVD Principal Component Analysis (PCA)


PCA -Dimension Reduction in Retrieval

A typical image retrieval pipeline


Image Formation

Feature Computing

Feature Aggregation Classification

Knowledge/Data Base

e.g, dense SIFT: 12000 x 128 e.g, Fisher Vector: k=64, d=128

Rd -> Rp

Principal Component Analysis

The formulation: for data points {x1, x2,…, } in Rn, find a lower dimensional

representation in Rm, via a projection W,: mxn, s.t., the energy of the data is preserved


PCA solution

Take the Lagrangian of the problem

Take the derivative w.r.t. to w, and KKT condition gives us,

This is an Eigen problem, finding projection s.t. it is just a scaling along the scatter matrix eigen vectors.


PCA – how to compute

PCA via SVD on the Covariance matrix


S: covariance, nxn

2d Data


Principal Components

1st principal vector

2nd principal vector

Gives best axis to project Minimum RMS

errorPrincipal vectors

are orthogonal


PCA on HoGs

Matlab Implementation of PCA: [A, s, eig_values]=princomp(hogs);


HoG basis function

PCA Application in Aggregation

SIFT aggregation Usually a PCA is

done on SIFT features, to reduce the dimension from 128 to say 24, 32. Then a GMM is

trained in R32 space, for FV encoding

Homework-2 Aggregation Fisher Vector

Aggregation of SIFT


load../../dataset/cdvs_sift_aggregation_test_data.mat;

[n_sift, kd_sift]=size(gd_sift_cdvs);offs = randperm(n_sift); offs = offs(1:200*2^10);% PCA[A1, s1, lat1]=princomp(double(gd_sift_cdvs(offs,:)));

figure(41); hold on; grid on; stem(lat1, '.'); title('sift pca eigen values');

SIFT PCA

Eigen values

That is why we have kd=[24, 32 48] for SIFT GMM in FV aggregation


SIFT PCA Basis Functions

Capturing max variation directions


Visualizing SIFT in lower dimensional space

Project SIFTs from 2 images to 2D space


Outline

Recap: Part I and Quiz-1



Laplacian Eigen Map

Directly compute an embedding {yk} from input {xk} without the explicit projection model A, s.t. Y=AX Objective function

where the nxn affinity matrix W reflects the data points relationship in the original space X.


M. Belkin and P. Niyogi. Laplacian Eigenmaps and spectral techniques for embedding and clustering. In Advances in Neural Information Processing Systems, volume 14, pages 585–591, Cambridge, MA, USA, 2002. The MIT Press

Graph Laplacian

Graph Laplacian: L= D - W


Laplacian Eigenmap

Minimizing the following

Is equivalent to

Where D is the degree matrix (diagonal) with


Laplacian Eigen Map Solution

Numerically, solve the eigen problem:

where the first d smallest eigen values’ corresponding eigenvectors, will form a d-dimensional feature of {yk}


43

Locally Linear Embedding (LLE) History: S.T. Roweis and L.K. Saul. Nonlinear dimensionality reduction by Locally Linear

Embedding.Science, 290(5500):2323–2326, 2000Procedure

1. Identify the neighbors of each data point2. Compute weights that best linearly reconstruct the point from its

neighbors

3. Find the low-dimensional embedding vector {yi} which is best reconstructed by the weights determined in Step 2

In matrix form:


LLE Solution

Similar to Laplacian Eigenmap, the first d eigenvectors provides the best local embedding.

Many successful applications in embedding


2d embedding with Laplacian Eigenmap 2d embedding with LLE

Summary

SVD and PCA SVD – non-square matrix decomposition, left transform and

right transform, with scaling in between SVD – as an image decomposition, linear combination of

outer-product basis PCA – eigen values indicate amount of info/energy in each

dimension, PCA – basis are eigen vectors to the covariance matrix LEM – direct data embedding without explicit projection

from graph relationship (expressed as Laplacian) Many applications


Date post:	12-Jan-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

ECE 484 Digital Image Processing Lec 16 - Transform Domain ......ECE 484 Digital Image Processing,...

Documents