Olshausen’s Demo

1. The Training set ?

Natural Images (Olhausen’s database)

How much do we learn ?

face database and car database

2. The Sparseness term ?

Prior steepness

Sparseness function

3. Natural encoding or hacking?

Whitening the data

Non-stationary hypothesis

How Important Is:

Training with Natural Images

Training: 10 images (512x512)

10,000 presentations

Batch size: 100

Basis Function: 16x16

Face Database

Training: 100 images (100x100)

10,000 presentations

Batch size: 100

Basis Function: 16x16

Encoding Properties

Original

50 basis

10 basis

30 basis 40 basis

20 basis

Car Database

Training: 200 images (128x128)

10,000 presentations

Batch size: 100

Basis Function: 16x16

Comments

1. The algorithm seems to capture the structure of the images (cf car):

Learning is experience-dependent

2. Basis functions found in good agreement with properties of neurons in visual cortex:

Receptive fields are localized, oriented, bandpass

1. The Training set ? Background, face and car databases

2. The Sparseness term ? Prior steepness

Sparseness function

3. Natural encoding or hacking?

Whitening the data

Non-stationary hypothesis

How Important Is:

Prior Steepness

Steepness 2.2

Steepness 10

Steepness 5

Steepness 100

i

i

iii

aSxaIE

2

)(

Prior Steepness

Steepness 2.2 Steepness 1.5

Steepness 0.2

i

i

iii

aSxaIE

2

)(

Sparseness Function

Sparseness Function

Sparseness Function

S(x)=|x| S(x)=log(1+x^2)

Sparseness Function

• batch of 100 samples:

• Mean Error: abs=.471 / log = .504

1. The Training set ? Background, face and car databases

2. The Sparseness term ? Prior steepness

Sparseness function

3. Natural encoding or hacking?

Whitening the data

Non-stationary hypothesis

How Important Is:

Whitening the Data

Data are filtered with whitening/low-pass filter:

picture / cycles 200,)( 0)/( 4

0 ffefR ff

• How important is it for the convergence of the algorithm?

• The question is to know whether it is just a speed-up or is it required for convergence?

Non-preprocessed Car Images

Training: 100 images (100x100)

30,000 presentations

Batch size: 100

Basis Function: 16x16

Non-stationary Hypothesis:Encoding the Full Face

After few iterations…

Code + images available:

http://web.mit.edu/serre/www/