Virginia Tech Carilion Research InstituteDepartment of Physics, Virginia Tech2 Riverside Circle, Roanoke, VA 24014

Computational Psychiatry and the Mentalization of Others

Wellcome Trust Centre for NeuroimagingUniversity College London, 12 Queen Square, London, UK, WC1N 3BG

P. Read Montague

2

Did (lack of) free trade contribute to Neanderthal downfall?

Shogren, J, Horan, R, Bulte, E (2005) How Trade Saved Humanity from Biological Exclusion: An Economic Theory of Neanderthal Extinction.” Journal of Economic Behavior and Organization.

1. Response to ‘fair’ reciprocity (social norms)2. Depth-of-thought (excellent for ASD)3. Sensitivity to horizon (planning)4. Sensitivity to history of play (working memory)5. Ability to learn - to model other subject

Reciprocation exhibits patterns and requires lots of cognition

BA BA BA

i

r

i

Alan Turing

R1

0 01 0$

*0

Turing Machine

Hardware (physical interactions)

Patterns of information processing

Computation Computational Theory of Mind (CTOM)

Mind is equivalent to information processing supported by brains.

How does one get ‘mind-like’ stuff from ‘stuff-like’ stuff ?

Turing proposed a solution to an age-old problem

Growing body of cellular & molecular data

Behavior, thoughts,moods, etc.

ComputationalModels and methods

The prevailing model (ambition) of computational neuroscience

complex behaviors, thoughts, moods, etc ascomputations.

neuronal interactions as computations.

Computational Psychiatry

http://computationalpsychiatry.org/

translating computational neuroscience

1. Biochemical networks, synapses, cells

2. Neuronal networks

3. System-level brain responses (fMRI, fNIRS, MEG, EEG)

4. Cognitive variables and their dynamics (behavior, perception)

5. Groups, social interactions, interactions-with-culture

Complication 1 – many levels

but there are other big complications…

Complication 2: Validation in model organisms?

e.g. is rodent empathy a model for human empathy?

human

We can profit from reversing model organism pipeline.

hypothesis: There is a software gap

Use well-defined economic probes - normative solutions

What are we doing?

Other minds task battery

Uncertainty and rewardtask battery

Ultimatum game - simple probe of mentalizing

BA

Offer split ($60 : $40) ?

accept or reject

What is fair? What is fair?

What does bluethink I think is fair?

What does redthink I think is fair?2nd order beliefs

What does bluethink is fair?

What does redthink is fair?1st order beliefs

Why use economic games?

1. possess natural learning signals (deviations)

2. equipped with concept of optimal play

3. new variables and dynamics

economic variables and their dynamics

‘standard’ cognitive

constructs

RBS version 1 ‐Ultimatum‐ proposer role‐Dictator‐ proposer role‐Trust‐ investor role‐Risk Aversion‐Temporal Discounting‐Passive film viewing‐Structural scan‐Resting state scan

RBS version 2 ‐Ultimatum‐ responder role‐Trust‐ investor role‐Risk Aversion‐Temporal Discounting‐Filmotyping‐Structural scan‐Resting state scan

Reinforcement Learning US‐Go/No‐Go‐Two‐Step‐Skewness‐High/Low Card‐Structural scan‐Resting state scan

Social Exchange US‐Ultimatum‐ responder role‐Trust‐ investor role‐Bargaining‐ buyer and seller role‐Social hierarchy‐Structural scan‐Resting state scan

Social Exchange UK‐Ultimatum‐ responder role‐Trust‐ investor role‐Bargaining‐ buyer and seller role‐Social hierarchy‐Structural scan‐Resting state scan

Other minds task battery between UK and US

Borderline PD (BPD)Anti-social PD (ASPD)

Investor Trustee

$20

Measuring reciprocity and model-building with a10-round ‘trust’ game

repay

pay (x3)

investperiod

8 s 8 s10 s 10 s 10 s8 s8 s4 s4 s

cue to invest

repaymentrevealed toboth brains

Gave

13

Kept

29

investmentrevealed toboth brains

Kept Gave

6 14

repayperiod

cue to repay

delayperiod

delayperiod

delayperiod

delayperiod

investment phase repayment phase

Totals

2919

totalsrevealed toboth brains

inter-rounddelay period

investperiod

8 s 8 s10 s 10 s 10 s8 s8 s4 s4 s

cue to invest

repaymentrevealed toboth brains

Gave

13

Kept

29

Gave

13

Kept

29

investmentrevealed toboth brains

Kept Gave

6 14

Kept Gave

6 14

repayperiod

cue to repay

delayperiod

delayperiod

delayperiod

delayperiod

investment phase repayment phase

Totals

2919

Totals

2919

totalsrevealed toboth brains

inter-rounddelay period

Structure of a round

The game engages prediction systems consistent with mesostriatal dopaminergic responses

*

time (sec)-8 0 10

-0.2

0

0.2

Trustee ‘intention to increase trust’ shifts with reputation building

Reputationdevelops across

rounds

Trustee will increasetrust on next move

Trustee will decreasetrust on next move

submit reveal *

increases or decreasesin future trust by trustee

-0.2

0

0.2

time (sec)

Signal nowanticipatesoutcome

reciprocity modulatedvoxels

Adapted from King-Casas et al., 2005 Science 308:78-83

Why is this temporal shift provocative?

Midbrain dopamine neurons

burst

pause

R timenaive

R timeAfter learning

Pause, burst, and ‘no change’ responses representreward prediction errors

Cooperation always breaks…

Application to Borderline Personality Disorder

Borderline PD subjectsinduce a break in cooperation

Adapted from King-Casas et al., 2008 Science 321:806-810

BPD trustee

Healthy trustee

Borderline PD anterior insula response does not differentiate offer levels (input insensitivity)

unfair fair

unfair fair

Adapted from King-Casas et al., 2008 Science 321:806-810

receivesignal

sendsignal

BPD anterior insula response occurs only for ‘input’

control BPD

1. Behavioral signal – sustained cooperation breakdown

In BPD subjects, social exchange game exposes:

3. Selectivity of insula signal – social sensing deficit?

2. Neural signal – insular response unmodulated by fairness

All the effects disappear in proposer role.

Application across a range of psychopathology groups

Biosensor approach to psychopathology classification

R

I

proposer responder

Psychopathology group always played responder role

MDD, ADHD, ASD, BPD medicated, BPD unmedicated, controls

The idea:  Humans are sensitive detectors of interpersonal exchange patterns –exploit this capacity as a kind of device.

BA BA BA

i

r

i

20 numbers

Koshelev et al. (2010). PLoS Comput Biol 6(10), e1000966.

Biosensor approach to psychopathology classification

regress preceding I’s and r’s onto the next investment, cluster on the regression coefficients

over-represented in cluster

under-represented in cluster

Biosensor approach to psychopathology classification

Koshelev et al. (2010). PLoS Comput Biol 6(10), e1000966.

Theory-of-mind model based classification of behavior

Use observed exchanges and computational model to classify investors using depth-of-thought and inequality aversion

investor trustee

10 rounds$20

multi-round trust game

n = 195 pairs

Examine 1st and 2nd orderinterpersonal prediction errors in brains for each depth-of-thought level

investor trustee

Level 0

Level 1

Level 2

Strategy

30Ray D, et al. Bayesian model of behaviour in economic games. NIPS (2008)

How to determine a player’s type and depth-of-thought?

31

• Compute the probability of taking certain actions

• Fit model to actual behavior

• Maximize the likelihood of observing the actual behavior given all possible types and levels of depth-of-thought

050

100150200250300

Level 0 Level 1 Level 2

Tota

l ear

ning

s (m

onet

ary

units

)

Depth-of-thought of investor

*

ns

0%

20%

40%

60%

80%

100%

1 2 3 4 5 6 7 8 9 10

Inve

stm

ent r

atio

Round number

investor trustee

0%10%20%30%40%50%60%

Level 0 Level 1 Level 2

Perc

enta

ge

Depth-of-thought of investor

Depth-of-thought classification captures distinct behavioral characteristics

Xiang T, Ray D, et al. (2012). Computational Phenotyping of Two-Person Interactions Reveals Differential Neural Response to Depth-of-Thought. PLoS Comput Biol 8(12).

Depth‐of‐thought (investor)

level 0 level 2level 1

Y = 0

70corrected p < 0.01

Y = 0

Y = 0

Y = 0

p < 0.0001p < 0.001uncorrected

1st orderInterpersonal prediction error

R - mi (R)

Depth‐of‐thought (investor)

level 0 level 2level 1

70

corrected p < 0.01

X = ‐48

X = ‐48

X = ‐48X = ‐48

p < 0.0001p < 0.001uncorrected

2nd orderInterpersonal prediction error

I - mi (mj (I))

Distinct neural signals that emerge after depth-of thought phenotyping

R

I

player i player j

Computational phenotyping with reciprocation games

1. Shows promise neurally and behaviorally

2. Requires lots of normative data

3. Tools and methods are now the pressing need

Collaborators

Virginia Tech - Kevin Hill, Terry Lohrenz, Ken Kishida, Amin Kayali, Ann Harvey, Justin King, Meghana Bhatt, Rosalyn Moran

University College London - Peter Dayan, Karl Friston, Xiaosi Gu, Andreas Hula, Peter Fonagy, Ray Dolan, Tobi Nolte, Sarah Carr, and UCL interns

Baylor College of Medicine - James Lu, Richard Gibbs, Josepheen Cruz

Acknowledgments: The Wellcome Trust, The Kane Family Foundation, NIDA, NIMH, NIA, DARPA, MacArthur Foundation, The Dana Foundation

Baylor College of MedicinePearl Chiu

Xu Ciu (Stanford U)Brooks King-Casas

Josepheen CruzKen Kishida

Misha KoshelevJian Li (PKU)Terry Lohrenz Damon Tomlin

Ting XiangDongni Yang

CaltechCedric Anen

Colin CamererSteve Quartz

Antonio RangelDeb Ray

Emory UniversityGreg Berns

University of HoustonAmin Kayali

UCLPeter DayanPeter Fonagy

Tobi NolteXiaosi Gu

Andreas Hula