Attributes of a neural code for perceptual/cognitive computation

Presented Feb 12, 2003Mathematical Biosciences Institute

Ohio State University

David W. Arathorn

Copyright(c)2003 David W. Arathorn

Unless otherwise indicated all diagrams from

“Map-Seeking Circuits in Visual Cognition”

David W. ArathornStanford University Press

M ap-S eek in g C ircu its In V isu al C ogn ition

David W . A ra thorn

from Stanford University Press

A neural code must be able to support a

sufficiently rich computational repertoire

• It must encode a sufficient range of values with sufficient accuracy and discriminability

• It must close under a sufficient set of operations implementable with neural mechanics

Two alternative approaches to problem, not mutually exclusive

• Try to deduce the code from inputs

• Try to deduce the code, or at least specific characteristics of the code, from computational requirements

Starting with the second approach

• What are the operations necessary to support a realistically complex computation?

– combine, – match, – map, inverse map, – compete, – attenuate, – scale – a nonlinearity (if not inherent)

A circuit composed of the following operations…

is capable of a variety of visual functions, computing inverse kinematics, and other cognitive computations

s s

2

f

w1

g1

g2

g3

q1

q2

q3

b

b

w

m w11

m w22

forward backward

r

f

d1( )r d2( )r d3( )r

i ig d b r

map, attenuate, combine

s s

2

f

w1

g1

g2

g3

q1

q2

q3

b

b

w

m w11

m w22

forward backward

r

f

d1( )r d2( )r d3( )r

k km f b w

k k k

k

m f b w b w w�� attenuate, combine

match, non-linearity

s s

2

f

w1

g1

g2

g3

q1

q2

q3

b

b

w

m w11

m w22

forward backward

r

f

d1( )r d2( )r d3( )r

i iq d b r�

comp , g g q

match, map

competition

Mapping module for multilayer circuits

s

d 1( )r

g 1g 2

g 3

q 1

q 2

q 3

b b

forw ard backw ard

r

s

d 1 ( )b-1

r

d 2 ( )b-1 d 3 ( )b

-1

d 2( )r d 3( )r

FORWARD BACKWARD

input image

layer 1

layer 2

memory

algebraic versions of the basic operations

i ig d b r

k km f b w

k k k k

k k

m f b w b w w��

i iq d b r�

comp , g g q

1) map, attenuate, combine: b forward, b and r backward superpositions

2) match, non-linearity: memory response and forward/backward match

3) competition: mapping coefficient competition

1i ig d r b

��

Role of the Ordering Property

In two places there is a match between a superposition and a pattern (which itself may be a superposition from another layer in multilayer circuits.)

si S s are components of superposition si

r S r is not component of superposition si

Ordering property is

P(sj si > r si ) >> P(sj si < r si )

for sj S

Depends on sparsity of si and number of components of si

s1

s2

sr

Edge-filtered images are sparse high dimensional vectors

During convergence only a small excess needed: sj si > r si

r is composed of parts of various si and is dismantled during convergence

FORWARD BACKWARD

input image

layer 1

layer 2

memory

First, search concurrently for locating patterns anywhere…

Then, search seqentially for confirming feature patternsin locations predicted by head model….

lynx image courtesy U.S Fish and Wildlife Service

iter 1

iter 25

(next slide)

iter 1 iter 20

iter 40

iter 60 iter 80 iters 20+40+60+80

pattern x-loc y-loc orient yscale xscaleeyes 110 69 7.500 0.850 0.850 ------snout 109 48 7.500 0.850 0.800------left ear 83 108 12.500 1.000 0.900------right ear 144 99 27.500 0.900 0.850

Pattern location – related to tracking

antelope image courtesy U.S Fish and Wildlife Service

Training model, 3D normals

Signal encoding has to “interoperate” with plausible synaptic encoding... e.g. encoding of 3D normals

1

2

3

41

2

3

4

00.250.5

0.75

1

1

2

3

4

0

50

100

1500

50

100

150

0

0.5

1

0

50

100

150φ

ψ

k - encode( <ψ,φ>) encode(<90,90>)

16 synapse weights (wi = e -| v-ci | 2 )

per <x,y,z> encode spherical orientation of normal

determine if viewpoint orientation perpendicular to model normal

Input image

0 50 100 150 200

0

50

100

150

200

3d models courtesy www.3Dcafe.com

0 50 100 150 200

0

50

100

150

200

3D projection

rotation in plane

scale/aspect

translation

iter 1 iter 25

25, 0.01, 0.15, 10, F, -10.0, -5.0, 2.5, 1.0, 1.0, 0.05, 1.0, 1.0, 0.05, 0.7, 1.4, 0.1, 1.3, 1.3, 0.1, 1.4, 1.4, 0.1, -75.0, -15.0, 5.0, 10.0, 45.0, 5.0 /1, 1, 25 pig.dat1, 50 / testsamples, interval pig1dis4_h.datpig1dis4_v.datpig1dis4_l.datpig1dis4_r.dat

----------------- 40000 25667929 1 17708 108 111 1.000 2 17709 109 111 0.533

------------------

1 1 25667929 1 2 -5.000 1.000 1.000 1.000 2 1 -7.500 1.000 1.000 0.363------------------

1 1 25667929 1 3 1.000 1.300 1.400 1.000 2 4 1.100 1.300 1.400 0.911 3 2 0.900 1.300 1.400 0.409 4 5 1.200 1.300 1.400 0.283------------------

13 8 25667929 1 67 -35.000 25.000 1.000 2 66 -35.000 20.000 0.823 3 59 -40.000 25.000 0.730 4 65 -35.000 15.000 0.289 5 58 -40.000 20.000 0.137 6 75 -30.000 25.000 0.065

translation

rotation in plane

scaling/aspect

3D projection

another view

25, 0.01, 0.15, 10, F, -30.0, 0.0, 3.0, 1.0, 1.0, 0.05, 1.0, 1.0, 0.05, 0.7, 1.4, 0.1, 1.3, 1.3, 0.1, 1.4, 1.4, 0.1, -75.0, -15.0, 5.0, 10.0, 45.0, 5.0 /1, 1, 25 ------

1 1 25661176 1 8 -6.000 1.000 1.000 1.000 2 7 -9.000 1.000 1.000 0.902 3 9 -3.000 1.000 1.000 0.681 4 10 0.000 1.000 1.000 0.335 5 6 -12.000 1.000 1.000 0.195------

1 1 25661176 1 5 1.200 1.300 1.400 1.000------

13 8 25661176 1 17 -65.000 15.000 1.000 2 27 -60.000 25.000 0.591 3 16 -65.000 10.000 0.526 4 26 -60.000 20.000 0.294

rotation in plane

scaling/aspect

3D projection

iter 1

iter 25

layer 1 layer 2 layer 3

translation rotation scaling/perspect

mapping convergence

iter 1

iter 25

Recharacterization: mappings as data

-10 -5 0 5 10-15

-10

-5

0

5

10

15

-10 -5 0 5 10-15

-10

-5

0

5

10

15

memory pattern 2 components input pattern

locations of activemappings at endof convergence

Concurrently active translational mappings pop out figure defined by repeated pattern.

Another composed inverse mapping problem:inverse kinematics

1

2

3

p0

p3

seg 1

seg 2

seg 3

constr3(F2)

constr2(F1)

Convergence to inverse kinematic solution

0 20 40 60 800

20

40

60

80

0 20 40 60 800

20

40

60

80

0 20 40 60 800

20

40

60

80

10 20 30 40 50 60 70 80

10

20

30

40

50

60

70

80

Layer 3/Segment 3 Layer 2/Segment 2 Layer 1/Segment 1

iter 1

iter 2

iter 5

0 20 40 60 800

20

40

60

80

activate end effector targetlocation

Can the basic operations be implemented with neural or other analog mechanisms?

– combine, – match, – map, inverse map, – compete (involves a max( ) operation) – attenuate, – scale – a nonlinearity (if not inherent)

Constraints imposed by neural mechanics

• non-linearity

• limited accuracy and reliability

• limited amplitude range

• signal warp– pulse spread in dendrite means signal

interactions at distance from synapse of origin cannot depend directly on temporal fine structure seen in the axonal signal

Neuronal circuit is constructed from cell pairs which implement reciprocal pathways with temporal or phase encoding

stage i Fi

stage i+1 Fi+1

Bi

Bi+1

inhibitorysynapse

excitatorysynapse

inhibitoryinter-cell

forward backward

V

t

V

t

DC 1.0 vs. DC 0.7

DC 1.0 vs. DC 0.5

• Vectors are self “normalizing• Amplitude encoding range is not limited to cell

dynamic range• Capacitance (pulse spread) is friendly

– Increases temporal differentiation– Decreases effect of spatial location of synapse

• Only monotonicity required in combination to implement addition-like operation

• Devil-take-the-hindmost competition implemented by inhibitory signal derived from signal front –keeps meaningful signals in narrow range

Some advantages of phase encoding

target 1

target 2

target 3

no targets

left

• Dendritic trees are non-homogeneous. i.e. local structure determines local signal interactions, global structure determines combination of local results

• All computation takes place in dendritic trees (in idealized cell).

• Signal interactions evolve primarily during rise time of dendritic pulse

• State can be held over between cycles to amplify signal differences

• Pulse length is long compared to propagation time: some location independence

Intra-dendritic computation using phase in real neurons

From SR Williams, GJ Stuart, 2002, “Dependence of EPSP Efficacy on Synapse Location in Neocortical Pyramidal Neurons”, Science 295:1907-10

signal propagation

sEPSPs

From SR Williams, GJ Stuart, 2002, “Dependence of EPSP Efficacy on Synapse Location in Neocortical Pyramidal Neurons”, Science 295:1907-10

amplitude - rise time relation

From SR Williams, GJ Stuart, 2002, “Dependence of EPSP Efficacy on Synapse Location in Neocortical Pyramidal Neurons”, Science 295:1907-10

phase difference – response amplitude function(also known as “coincidence detection”)

somaticsynapses

distal (620 um)synapses

distal (620 um)synapses

From SR Williams, GJ Stuart, 2002, “Dependence of EPSP Efficacy on Synapse Location in Neocortical Pyramidal Neurons”, Science 295:1907-10

bbcol

stage b

stage m

stage r

input field

rm = r-match ri = r-intersect

rbkwd rfwd

bfwd bbkwd

mbkwd mfwd

mapping competition

1 2

3 4

6 5

7 8

9

10

rmcli

rm ri

11 subtract

1r

1r�

q1

g1

1b

1b�

maxb�

max q

max iqq

(1)

(2)

(3)

(4)

(5)

(6)

(9)

(10)

(11)

Signal equivalentsin algorithmic circuit

(7)

(8)

1m

1m�

bbcol

stage r

input field

rbkwdrfwd

bfwd

bbkwd

mapping competition

1

2

6 5

9

10

rmcli

rm

ri

11 subtract

1r

1r�

q1

g1

1b

1b�

maxb�

max q

max iqq

(1)

(2)

(3)

(4)

(5)

(6)

(9)

(10)

(11)

Signal equivalentsin algorithmic circuit

3

4

Signal relationships in neuronal circuit

0 5u 10u 15u 20u 25u 30u 0 600m 1.2 1.8 2.4 3

3.6

rm ri

bbcol

0 5u 10u 15u 20u 25u 30u 0 600m 1.2 1.8 2.4 3

3.6 bbcol

rm ri

recognition state

non-recognition state

bbcol inhibits late rm signalsthus blocking paired ri signal

Is phase encoding the only way to satisfy the original requirements?

Encoding must support a sufficiently rich repertoire of computation– It must encode a sufficient range of values

with sufficient accuracy and discriminability– It must close under a sufficient set of

operations implementable with neural mechanics

Map-seeking circuits will work with any encoding that supports the operations listed earlier.

• Encodings must have started simple to have co-evolved with useful, robust mechanisms to implement operations on them.

An evolutionary caveat from engineering: complex and/or delicate systems generally do not work at first...but an evolutionary step has to work in “alpha release.”