A View from the Bottom
Peter DayanGatsby Computational Neuroscience Unit
Neural Decision Making
• bewilderingly vast topic • models playing a central role
– so beware of self-confirmation + battles
3
• Ethology/Economics(?)– optimality– logic of the approach
• Psychology– economic choices – instrumental/Pavlovian conditioning
• Computation
• Algorithm
• Implementation/Neurobiologyneuromodulators; amygdala; prefrontal cortex
nucleus accumbens; dorsal striatum
prediction: of important eventscontrol: in the light of those predictions
Neural Decision Making
Imprecision & Noise
• computation– Bayesian sensory inference– Kalman filtering and optimal learning– metacognition
– exploration/exploitation
– game theory
Imprecision & Noise
• algorithm– multiple methods of choice
• instrumental: model-based; model-free– (note influence on RTs)
• Pavlovian: evolutionary programming– uncertainty-modulated inference and learning
– DFT/drift diffusion decision-making
– MCMC methods for inference
Imprecision & Noise
• implementation– (where does the noise come from?)
– evidence accumulation– Q-learning and dopamine– metacognition and the PFC– acetylcholine/norepinephrine and uncertainty-
sensitive inference and learning
Diffusion to Bound
Britten et al, 1992
Diffusion to Bound• expected reward, priors affect
starting point• some evidence for urgency
signal• works for discrete evidence
(WPT)• less data on >2 options• micro-stimulation works as
expected• decision via striatum/superior
colliculus/etc?• choice probability for single
neuronsGold & Shadlen, 2007
9
dopamine and prediction error
no prediction prediction, reward prediction, no reward
TD error
Vt
R
RL
tttt VVr 1
)(t
Probability and Magnitude
Tobler et al, 2005
Fior
illo
et a
l, 20
03
Risk Processing
< 1 sec
0.5 sec
You won40 cents
5 secISI
19 subjects (dropped 3 non learners, N=16)3T scanner, TR=2sec, interleaved234 trials: 130 choice, 104 single stimulusrandomly ordered and counterbalanced
2-5secITI
5 stimuli:
40¢20¢
0/40¢0¢0¢
Neural results: Prediction errorswhat would a prediction error look like (in BOLD)?
Neural results I: Prediction errors in NAC
unbiased anatomical ROI in nucleus accumbens (marked per
subject*)
* thanks to Laura deSouza
raw BOLD(avg over all
subjects)
Value Independent of Choice CauuvrECQ tttt ,1|)(),1( 1
**
),1(),2(max),1(),1( CQaQrCQCQ at
Roesch et al, 2007
Metacognition
• Fleming et al, 2010
• contrast staircase for performance; type II ROC for confidence
Structural Correlate
• also associated white matter (connections)
Discussion
• what can economics do for us?– theoretical, experimental ideas– experimental methods– like behaviorism…
• what can we do for economics?– large range of constraints– objects of experimental inquiry precisely aligned
with economic notions– grounding/excuse for complexity…