Van Essen et al., 1992

F unctionA rchitectureC onnectionsT opography

Adelson & Bergen, 1990

PIT

V1 V2 V3

PIP

V3a

MDP

MIP

PO

MT

V4

VIP

LIP

MST

FST7a

STPp

CIT

STPa

AIT

OV

T

DP

Wallisch & Movshon, 2008

after Felleman & Van Essen, 1991

Ungerleider & Mishkin, 1982

Object discriminationLandmark discrimination

Macaque visual cortex

V1

V2

MT

FST

AITv

CITv

7a

STP

LIP, VIP

MST

DP

V4

PITd

PITvCITd

AITd

VOTVP

FEF

Visualinput

Record

Hubel and Wiesel, 1968

V1

0% coherence 50% coherence 100% coherence

Coherence controls visibility

Newsome and Paré, 1988

Downing and Movshon, 1989

V1

V2

MT

FST

AITv

CITv

7a

STP

LIP, VIP

MST

DP

V4

PITd

PITvCITd

AITd

VOTVP

FEF

Visualinput

Record

Maunsell and Van Essen, 1983

MT

Hubel and Wiesel, 1968

V1

0% coherence 50% coherence 100% coherence

Visualstimulus

Neuronalresponse

Behavioraljudgement

FixationPoint

Pref target

Null target

10 deg

Receptive field

DotsAperture

Fix Pt

Dots

Targets

1 sec

Britten, Shadlen, Newsome & Movshon, 1992

Barlow, Levick & Yoon, 1971

Britten, Shadlen, Newsome & Movshon, 1992

Britten, Shadlen, Newsome & Movshon, 1992

0

20

40

60

Num

ber

of c

ells

0.1 1 10Threshold ratio (neuron/behavior)

Britten, Shadlen, Newsome & Movshon, 1992

Neuronal threshold, choice (%)

Neu

rona

l thr

esho

ld, f

ixat

ion

(%)

1 10 1001

10

100

Britten, Shadlen, Newsome & Movshon, 1992

Visualstimulus

Neuronalresponse

Behavioraljudgement

Neurometric function Psychometric function

Visualstimulus

Neuronalresponse

Behavioraljudgement

Neurometric function Psychometric function

?

0.1 1 10 100

Motion strength (% coherence)10 20 30 40 50 60 70 80

Trial number

0

5

10

15

20

25

30

35

40

45

50

Res

pons

e (im

puls

es/s

ec)

0.4

0.5

0.6

0.7

0.8

0.9

1

Pro

port

ion

corr

ect

NeuronBehavior

Correct trialsError trials

Shadlen, Britten, Newsome & Movshon, 1996

Britten, Newsome, Shadlen, Celebrini & Movshon, 1996

Britten, Newsome, Shadlen, Celebrini & Movshon, 1996

010

2030

40

0

20

40

Pro

ba

bili

ty

PREF resp

NULL resp

0.3 1 3 10

Response ratio ("preferred"/"null")

0.0

0.5

1.0

Choic

e p

robabili

ty

0.0

0.2P

roport

ion o

f tr

ials

1.066

0.0 0.2

Proportion of trials

0.548

C

Britten, Newsome, Shadlen, Celebrini & Movshon, 1996

0 500 1000 1500 2000

Time (msec)

0.0

0.2

0.4

Mean n

orm

aliz

ed r

esponse

0 500 1000 1500 2000

Time (msec)

-0.1

0.0

0.1

Diffe

rence in n

orm

aliz

ed r

esponse

Britten, Newsome, Shadlen, Celebrini & Movshon, 1996

Britten, Newsome, Shadlen, Celebrini & Movshon, 1996

Visualstimulus

Neuronalresponse

Behavioraljudgement

Neurometric function Psychometric function

Choice probability

Albright, 1984

V1

V2

MT

FST

AITv

CITv

7a

STP

LIP, VIP

MST

DP

V4

PITd

PITvCITd

AITd

VOTVP

FEF

Visualinput

Stimulate

FixationPoint

Pref target

Null target

10 deg

Receptive field

DotsAperture

Fix Pt

Dots

Targets

1 sec

ElectStim

Salzman, Murasugi, Britten and Newsome, 1992

Salzman, Murasugi, Britten and Newsome, 1992

Salzman, Murasugi, Britten and Newsome, 1992

Salzman, Murasugi, Britten and Newsome, 1992

Visualstimulus

Neuronalresponse

Behavioraljudgement

Neurometric function Psychometric function

Choice probability

0 90 180

Difference in preferred direction (deg)

0.0

0.5

Inte

rneuro

nal corr

ela

tion

MT neuron pairs (Zohary et al.)

Zohary, Shadlen and Newsome, 1994

Xup

1

Xup

2

Xup

3

!

!

!

Xup

N

Pooled MT Signal

! X "up

Xdown

1

Xdown

2

Xdown

3

!

!

!

Xdown

N

! X "

Pooled MT Signal

down

Decisionup

! X " ! X "down

>

Shadlen, Britten, Newsome & Movshon, 1996

1 10 100 1

10

Thr

esho

ld (

% c

oher

ence

)

1 10 1000.5

0.6

0.7

0.8

Number of neurons

Cho

ice

prob

abili

ty

1 100.5

0.6

0.7

0.8

Threshold (% coherence)

Cho

ice

prob

abili

ty

4

16256

16

4

!" Decision

Pooled "up" signal

Pooled "down" signal

MT neuron pool

r = 0

r = 0.18

r = 0

r = 0.18

Shadlen, Britten, Newsome & Movshon, 1996

1 10 100 1

10

Thr

esho

ld (

% c

oher

ence

)

1 10 1000.5

0.6

0.7

0.8

Number of neurons

Cho

ice

prob

abili

ty

1 100.5

0.6

0.7

0.8

Threshold (% coherence)

Cho

ice

prob

abili

ty

4

16256

16

4

!" Decision

Pooled "up" signal

Pooled "down" signal

MT neuron pool

r = 0

r = 0.18

r = 0

r = 0.18

!" Decision

Pooled "up" signal

Pooled "down" signal

MT neuron pool

+

+

Pooling noise

Pooling noise

1 100.5

0.55

0.6

0.65

0.7

Cho

ice

prob

abili

ty

Threshold (% coherence)

A

B

C

D

Jazayeri & Movshon, 2006

Computing the likelihood of each direction

log L ( )

log L ( )

= 180o

MT

population

activity

Pooling

connections

Saccade Vector Map

RightwardLeftward

Direction of Motion Map

● ●

✙

● ●

✙

● ●

Fixate350 msec

Targets appear 500 msec

Random dot motion2 sec

Delay500-1000 msec

Saccade

✙

● ●Target 2Target 1

✙

task_panels_vert_noMF.isl

● ●Target 2Target 1

✙ FP

A

B

Shadlen and Newsome, 2001

0 1 20

50

-1 0 0 1 20

50

-1 0

0 1 20

50

-1 0 0 1 20

50

-1 0

0 1 20

50

-1 0 0 1 20

50

-1 0

Time (s)

sp/s

51.2%

12.8%

0%

Shadlen and Newsome, 2001

-0.5 0 0.5 1

10

15

20

25

30

35

40

45

50

-0.5 0 0.5

10

15

20

25

30

35

40

45

50

Time (s)

motion onset saccadeM

ean

resp

onse

(sp

/s)

51.2%25.6%12.8%6.4%0%

N=106

Mike Shadlen

-0.5 0 0.5 10.5

0.6

0.7

0.8

0.9

1

Time from motion onset (s)

Pro

babi

lity

(mea

n)

-0.5 0 0.50.5

0.6

0.7

0.8

0.9

1

Time from saccade

51.2%25.6%12.8%6.4%0%

N=106Correct choicesMike Shadlen

Responses in a reaction-time version of the direction discrimination task

Average LIP activity in a reaction-time task shows evidence of integration to a “decision boundary”

Roitman & Shadlen (2002)

choose Tin

choose Tout

High motion strength

High motio

n strength

Low motion strength

Time

~1 secStimulus

onStimulus

off

Spikes/s

Time

~1 secStimulus

onStimulus

off

Spikes/s

Low motion strength

MT neurons represent ongoing motion, while LIP neurons seem to represent the output of a neural integrator that accumulates evidence for decision making

MT – sensory evidenceMotion energy “step”

LIP – decision formationAccumulation of evidence “ramp”

Threshold

Mike Shadlen

Momentary evidencee.g.,

∆Spike rate:MTRight– MTLeft

Accumulated evidencefor Rightward

andagainst Leftward

Criterion to answer “Right”

Criterion to answer “Left”

Diffusion to bound model

Palmer et al (2005) Shadlen et al (2006)

�

µ = kC

C is motion strength (coherence)

P =1

1+ e−2k C B

�

t(C) = BkCtanh(BkC) + tnd

Responses in a reaction-time version of the direction discrimination taskare well described by the “race” model of integration to a decision boundary