Thomas Wachtler, Te-Won Lee and Terrence J. Sejnowski

presented by Andrew RabinovichOctober 17th, 2002

Goal: Find efficient representation of chromatic sensory information such that its redundancy is reduced

Solution:

12211 niniiifor all i=1,…,n

N

i ii1

• The independent components are assumed statistically independent

• The independent components must have Non-Gaussian distributions

Two random variables yi and yj are said to be independent if information of yi does not give any information on value of yj for i?j

In terms of probability densities:p(y1,y2,…,yn)=p1(y1) p2(y2)… p3(y3)

and expected values:E[g(y1)h(y2)]=E[g(y1)]E[h(y2)]

Two random variables x1 and x2 are uncorrelated if:cov(x1,x2)=E[x1,x2]-E[x1]E[x2]=0

For mx=0(centered data):corr(x1,x2)=E[x1,x2]

Higher-order(past 2nd) cumulants are zero for Gaussian distributions, which is essential for estimation of the ICA model.

4th order cumulant that is a classical measure of Non Gaussianity

kurt(y)=E[y4]-3(E[y2])2

assuming unit variance:kurt(y)=E[y4]-3

Kurtosis for Non Gaussian random variables:

superGaussian(leptokurtic)(+) subGaussian(platykurtic)(-)

Whitening the data, reduces the search space of the unmixing matrix

A’=VA (A’ is orthogonal)E[z1z1

T]=A’E[s1s1T]A’T=A’A’T=I

Whiteness of a zero-mean random vector y1 guarantees uncorrelatedness and unit variance of its components

whitening(sphering)

I. Begin with a uniform distribution:

II. Mix the independent component with A

III. Whiten the data

IV. Perform ICA to extract the rotation

I. II. III. IV.

5 1010 2=

Original Distribution

Mixed Distribution

Gaussian Distribution is rotationally symmetric

No information on the directions of the columns of the mixing matrix A

The variances(energies) of the independent components cannot be determined.

The order of independent components cannot be determined

i iii

i

asaa

x **

• Maximization of Non-Gaussianity- Kurtosis- Negentropy

• Maximization of Likelihood Estimation- InfoMax- Gradient

• Minimization of Mutual Information• Tensorial Methods • Nonlinear Decorrelation & Nonlinear PCA

Non-Gaussian Distribution is chosen, since Gaussian pdf is Completely described by mean and variance(PCA)

Principal Component Analysis (PCA) finds directions of maximal variance in Non-Gaussian data (second-order statistics).

Independent Component Analysis(ICA) finds directions of maximal independence in non-Gaussian data (higher-order statistics).

Mixed Signal: Independent Components:

Mixture 1

Mixture 2

Mixed Signal: Independent Components:

Find efficient representation of chromatic sensory information such that its redundancy is reduced

8 hyperspectral images of natural scenes

RGB vs. Hyperspectral

ICA PCA

Hyperspectral

HCEV

East/WestNorth/South

LMS – Opponent color space

S

L-M

Pixels

(Long, Medium, Short), not Least Means Squares

• Broadband and broad peaked basis functions

• Achromatic and color-opponent basis functions with non-orthogonal opponencydirection

• More accurate and efficient than PCA

Te-Won Lee, Serge Belongie and Terrence J. Sejnowski

• Aapo Hyvarinen, Juha Karhunen and ErkkiOja. Independent Component Analysis. Wiley-Interscience Publication, 2001

• Te-Won Lee, Thomas Wachtler and Terrence J. Sejnowski. Color Opponencyis an efficient representation of spectral properties in natural scenes. Vision Research 42 (2002)

This document was created with Win2PDF available at http://www.daneprairie.com.The unregistered version of Win2PDF is for evaluation or non-commercial use only.