ter Haar Romeny, FEV

Scale-time

ter Haar Romeny, FEV

Time measurements can essentially be processed in two ways: • as pre-recorded frames or instances, • or real-time.

Humans are real-time, they continuously perform a temporal analysis with their senses. The scale-space treatment of these two categories will turn out to be essentially different.

The prerecorded sequences are treated as regularly sampled points,leading to a regular causal time-scale-space, with temporalapertures and Gaussian temporal differential operators.

ter Haar Romeny, FEV

We can never reach the present moment:

A measurement aperture (operator) with infinitesimally short duration will be needed.

In the real-time measurement and analysis of temporal data we have a serious problem: the time axis is only a half axis: the past. There is a sharp and unavoidable boundary on the time axis: the present moment.

2 1

time 0 present

ter Haar Romeny, FEV

1 0.5 0.5 1t

1

1

2

3

4

s

Koenderink proposed to reparametrize the time axis in a logarithmicFashion, thus maintaining causality. The log time axis is now a full axis, allowing diffusion, i.e. causal operators.

present

ter Haar Romeny, FEV

t0 t

Dimensionless elapsed time

1 0.5 0.5 1t

1

1

2

3

4

s

present

s

1

s c1 ln lnt0 t

ter Haar Romeny, FEV

Kt, t0; 12 e

122 lnt0t

2The receptive fieldsbecome skewed:

0.4 0.3 0.2 0.1

0.5

1

1.5

2temporal order 0

time

0.4 0.3 0.2 0.1

302010

1020

temporal order 1

time 0.4 0.3 0.2 0.1

1000

500

500

1000temporal order 2

time

time

ter Haar Romeny, FEV

It is interesting to study if the temporal receptive field (RF) sensitivity profiles match the predicted skewness for the different differential order.

Many neurophysiology labs have recorded time sequences of neural RF’s, e.g. by the reverse correlation technique.

ter Haar Romeny, FEVDeAngelis, Ohzawa, Freeman, 1991

Cat V1 simple cell, 5 ms interframe interval

ter Haar Romeny, FEV

De Valois, Cottaris, Mahon, Elfar and Wilson, 2000

space

time

A clear skewness is observed in thetime direction.

ter Haar Romeny, FEV

15105 0 5 10 15

space

0

0.2

0.4

0.6

0.8

emit

nspace2,ntime 0

15105 0 5 10 15

space

0

0.2

0.4

0.6

0.8

emit

nspace2,ntime 1

15105 0 5 10 15

space

0

0.2

0.4

0.6

0.8

emit

nspace 2,n time 2

15105 0 5 10 15

space

0

0.2

0.4

0.6

0.8

emit

nspace1,ntime 0

15105 0 5 10 15

space

0

0.2

0.4

0.6

0.8

emit

nspace1,ntime 1

15105 0 5 10 15

space

0

0.2

0.4

0.6

0.8

emit

nspace 1,n time 2

15105 0 5 10 15

space

0

0.2

0.4

0.6

0.8

emit

nspace0,ntime 0

15105 0 5 10 15

space

0

0.2

0.4

0.6

0.8

emit

nspace0,ntime 1

15105 0 5 10 15

space

0

0.2

0.4

0.6

0.8

emit

nspace 0,n time 2

A time-scale model ofskewed receptive fieldprofiles on a logarithmicaxis.

ter Haar Romeny, FEV

From group theory:

Any half-axis should be parameterized logarithmically to enablelinear addition of a multiplication property for the measurementconvolution.

Examples:

• The scale axis (Powers of Ten: ‘orders of magnitude’);• The intensity axis (retina log mapping, gamma-corrections);• The time axis;• The sound pressure axis (decibels);• Etc.