1

Nonlinear models for Natural Images

Urs Köster & Aapo HyvärinenUniversity of Helsinki

2

Overview

1.Limitations of linear models

2.A hierarchical model learns Complex Cell pooling

3.A Horizonal product model for Contrast Gain Control

4.A Markov Random Field generalizes ICA to large images

2

1. Limitations of ICA image models

Natural images have complex structure, cannot be modeled as superpositions of basis functions

Linear models ignore much of the rich interactions between units

Modeling the dependencies leads to more abstract representations

Variance dependencies are particularly obvious structure not captured by ICA

Model with (complex cell) pooling of filter outputs - hierarchical models

Alternative: Model dependencies by gain control on the pixel level

Schwartz & Simoncelli 2001

5

A hierarchical model estimated with Score Matching learns Complex Cell

receptive fields

6

2. Two Layer Model estimated with Score Matching

Define an energy based model of the form

Squaring the outputs of linear filters

Second layer linear transform v

Nonlinearity that leads to a super-gaussian pdf.

Cannot be normalized in closed form. Estimation with Score Matching makes learning possible without need for Monte Carlo methods or approximations

7

Results

Some pooling patterns

The second layer learns to pool over units with similar location and orientation, but different spatial phase

Following the energy model of Complex Cells without any assumptions on the pooling

Estimating W and V simultaneously leads to a better optimum and more phase invariance of the higher order units

8

Learning to perform Gain Control with a Horizontal Product model

9

Multiplicative interactions Data is described by element-wise

multiplying outputs of sub-models

Can implement highly nonlinear (discontinuous) functions

Combine aspects of a stimulus generated by separate mechanisms

Horizontal layers Two parallel streams or layers on

one level of the hierarchy

Unrelated aspects of the stimulus are generated separately

Observed data is generated by combining all the sub-models

A horizontal network model:

10

The model

Definition of the model:

Likelihood:

Constraints: B and t are non-negative, W invertible g(.) is a log-cosh nonlinearity (logistic distribution) t has a Laplacian sparseness prior

10

Results

First layer W 4 units in B

First layer W 16 units in B

Second Layer: Contrast Gain Control

Reconstruction from As only

True image patches

Modulation from Bt

Emergence of a contrast map in the second layer

It performs Contrast Gain Control on the LGN level (rather than on filter outputs)

Similar effect to performing divisive normalization as preprocessing

The model can be written as

Something impossible to do with hierarchical models

13

The “big” picture: A Markov Random Field generalizes ICA to

arbitrary size images

14

4. Markov Random Field

Goal: Define probabilities for whole images rather than small patches

A MRF uses a convolution to analyze large images with small filters

Estimating the optimal filters in an ICA framework is difficult, the model cannot be normalized

Energy based optimization using Score Matching

15

Model estimation

The energy (neg. log pdf) is

We can rewrite the convolution

where xi are all possible patches from the image, wk are the different filters

We can use score matching just like in an overcomplete ICA model

The MRF is equivalent to overcomplete ICA with filters that are smaller than the patch and copied in all possible locations.

16

Results We can estimate MRF filters of size 12x12 pixels

(much larger than previous work, e.g. 5x5)

This is possible from 23x23 pixel ‘images’, but the filters generalize to images of arbitrary size

This is possible because all possible overlaps are accounted for in the (2 n -1) size image

Filters similar to ICA, but less localized (since they need to explain more of the surrounding patch)

Possible applications in denoising and filling-in

16