+ All Categories
Home > Documents > Seeking Consistent Stories

Seeking Consistent Stories

Date post: 19-Jan-2016
Category:
Upload: flavio
View: 34 times
Download: 0 times
Share this document with a friend
Description:
Seeking Consistent Stories. By Reinterpreting or Discounting Evidence: An Agent-Based Model^. CSCN Presentation. Agenda. The Phenomenon Agent-Based Model (ABM) Primer The Model Sample Run Experiments. Research Interests. Goal: psychology, law, computational modeling - PowerPoint PPT Presentation
57
Lydia L. Chen Instructor: Robert Axelrod PS793 – Complexity Theory Jun 23, 2022 06/23/22 Lydia L. Chen <[email protected]> Seeking Consistent Stories Seeking Consistent Stories By Reinterpreting or Discounting By Reinterpreting or Discounting Evidence: Evidence: An Agent-Based Model^ An Agent-Based Model^ CSCN Presentation
Transcript
Page 1: Seeking Consistent Stories

Lydia L. ChenInstructor: Robert AxelrodPS793 – Complexity Theory

Apr 21, 2023

04/21/23 Lydia L. Chen <[email protected]>

Seeking Consistent StoriesSeeking Consistent StoriesBy Reinterpreting or Discounting Evidence: By Reinterpreting or Discounting Evidence:

An Agent-Based Model^An Agent-Based Model^

CSCN Presentation

Page 2: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 2

AgendaAgenda

1. The Phenomenon

2. Agent-Based Model (ABM) Primer

3. The Model

4. Sample Run

5. Experiments

Page 3: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 3

Research InterestsResearch Interests

Goal: psychology, law, computational modeling

persuasion and decision making

law classes => persuasion techniques storytelling narrative coherence metaphors

Story model of jury decision making (Pennington & Hastie, 1986)

Page 4: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 4

Real-Life Scenario: Bench TrialReal-Life Scenario: Bench TrialProsecution () Defense Lawyer

“Guilty!

”“Innocen

t!”

Page 5: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 5

Sequential Evidence – What’s NormativeSequential Evidence – What’s Normative

Evidence 1

Evidence 2

Evidence 3

Evidence N

Official Deliberatio

n

Story/ Verdict

Page 6: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 6

Empirical Literature on JDMEmpirical Literature on JDM Form “coherent” stories that support option/verdict

(Pennington & Hastie, 1988)

Confidence in option/verdict increases with “coherence” (Glockner et al., under review)

Decision threshold: “sufficiently strong” (supported by many consistent evidence) or “sufficiently stronger” than other stories (review by Hastie, 1993

Narrative coherence: consistency causality completeness

“Consistency” aspect of “good” stories consistency between evidences in a story consistency of evidence with favored story

Page 7: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 7

Example Case (Pennington & Hastie, 1988)Example Case (Pennington & Hastie, 1988)

Scenario: Defendant Frank Johnson stabbed and killed Alan Caldwell

Evidence := facts or arguments given in support of a story/verdict Facts— “Johnson took knife with him” ; “Johnson pulled out his knife” Arguments—”Johnson pulled out knife because he wanted revenge” vs. “Johson

pulled out knife because he was afraid”

Story := set of evidence supporting a given verdict

Same evidence can be framed to support multiple verdicts/stories!

P D

Desired Verdict

“Guilty” “Innocent”

Story “Premeditated murder”

“Self-defense”

Interpretation of Evid

“Johnson was angry”

“Johnson was afraid”

Page 8: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 8

Sequential Evidence – More DescriptiveSequential Evidence – More Descriptive

Evidence 1

Compare, Deliberate, Interpret

Evidence 2

Evidence 3

Evidence N

Official Deliberatio

n

Story/ Verdict

Premature Story/

Verdict @ Evid n < N

(Brownstein, 2003; Russo et al., 2000)

Page 9: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 9

How People Deal with Incoming EvidenceHow People Deal with Incoming Evidence

People don’t just take evid @ face value, but are selective!

Possible Reactions to New Evid in Light of Old:

“reinforce” each other

“reinterpret” less plausible one (Russo et al., etc.); e.g., misremember the info

“discount” less plausible one (Winston)

actively “seek” more evidence (not modeled here)

Existing evidence A: “Johnson was not carrying a knife.”

New evidence B1: “Johnson is nonviolent.”

Inconsistent new evid B2: “Johnson pulled a knife.”

Reinterpret B2: “Johnson grabbed a knife from Caldwell.” (i.e., explain it was Caldwell’s knife, not Johnson’s)

Discount B2: “Witness must be mistaken.”

Judge asks layers follow-up questions

Page 10: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 10

AgendaAgenda

1. The Phenomenon

2. Agent-Based Model (ABM) Primer

3. The Model

4. Sample Run

5. Experiments

Page 11: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 11

What is Agent-Based Modeling?What is Agent-Based Modeling?

agents + interactions^

start simple; build up^

Key terminology agents system dynamics

agent births and deaths interactions/competitions

parameters

Page 12: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 12

Contributions ABMs Can MakeContributions ABMs Can Make

Symbiotic relationship:

Behavioral Experiments

ABM

parsimony:

demo emergence of seemingly complex

phenomen from small set of simple rulespredictions:

new observations/predictions

understanding:

study processes in detail

Test

Inputdescription:

informs base assumptions

Page 13: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 13

(Other Models (Hastie, 1993))(Other Models (Hastie, 1993))

Page 14: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 14

(Contrast with Bayesian & Algebraic Models)(Contrast with Bayesian & Algebraic Models)

Algebraic (additive), Bayesian (multiplicative): “single meter” of overall plausibility

ABM allows: revisiting and reconsidering previously-

processed evidence interaction/competition between new

evidence and individual pieces of previous evidence (not just conglomerate)

Page 15: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 15

Contrast with Story Model & ECHOContrast with Story Model & ECHO Explanatory Coherence Model := Thagard’s

Theory of Explanatory Coherence (TEC) + Story Model

Only implemented discounting, not reinterpretation

ABM enforces lower evidence-agent-level consistency, as opposed to higher story-level consistency

Unlike previous ABMs, model agents within individual as system

Page 16: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 16

MotivationsMotivations

1. Study emergence of consistent stories via reinterpretation, discounting, reinforcement mechanisms

2. Adaptive? aid consistency? speed-accuracy tradeoff? avoid indecisiveness

increases convergence rates? order effects hurt accuracy?

Page 17: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 17

AgendaAgenda

1. The Phenomenon

2. Agent-Based Model (ABM) Primer

3. The Model

4. Sample Run

5. Experiments

Page 18: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 18

GoalGoal

Model consistency-seeking process in story formation

Present evidence-agents to judge-system

Judge-system compares evidence-agents => keep, reinterpret, or discount evidence (agents “interact” & “compete”)

Until sufficiently strong/stronger story emerges

Page 19: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 19

Evidence-agents, composed of “features”

Operationalize consistency /b/ agents: “similarity” in abstract features (Axelrod’s culture ABM,

1997) “inverse Hamming distance” := % feature matches

AgentsAgents

G

x

x

89%

Evid 1

N

x

y

34%

Evid 2

Verdict (“G”,”N”,…)

Abstract features (binary)

Plausibility index (0%-

100%)

N

x

x

14%

Evid 1

Page 20: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 20

ParametersParametersParameter Variabl

eBase

# agents to present

- Initial A0 3

- Additional A 20

# possible verdicts V 2

# features/evidence (excluding verdict) F 2

% feature matches for:

Sufficiently consistent C 100%

Close enough for reinterpretation N < C 50%

# interactions /b/ evidence-agent births I (A0+A -1)

Change rate of agent plausibility (increase upon winning, decrease upon losing)

D 0.1

Rule(s) and threshold(s) for winning story-- “sufficiently strong” and/or “sufficiently stronger”

S, Sd Both — 0.5, 0.2

Page 21: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 21

(Java GUI)(Java GUI)

Page 22: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 22

System & Agent BirthsSystem & Agent Births

Judge-system represents judge’s mind

Evidence presentation = “agent birth”:

Initialization of N0 Agents: Set up randomly-generated agents, OR…

uniform distributions—even for plausibility index due to prior beliefs (Kunda, 1990) or knowledge (Klein, 1993)

User-specified

Page 23: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 23

(Topology)(Topology)

results\sysout_081118_0455.log

Printing 8 agents:

01 02 03 04 05 06 07 08

N G I G N I N N

y y y y y y y y

y y y y y y y y

100% 100% 100% 100% 100% 100% 100% 80%

Dead/Rejected Evidence--Printing 9 agents:

01 02 03 04 05 06 07 08 09

I G G I G I N N N

y y x x y x x x y

y x x x y x x y y

00% 00% 00% 00% 00% 00% 00% 00% 00%

To keep track of evidence-agents and their order of presentation:

lists of agents in order of birth/presentation to the system.

latest-born agent always appears at the end of a list.

no geographical “topology”

Page 24: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 24

StoriesStories:= {evidence-agents supporting a verdict}

Consistency = inverse Hamming distance amongst evidence (:= evidencePairs # feature matches / # evidencePairs / F )

Plausibility Index (“Strength”) = average of plausibility indices

G

x

x

89%

Evid 1N

x

y

51%

Evid 2

N

x

x

14%

Evid 1

Story promoting

“Guilty” verdict

Story promoting “Innocent”

verdict

strength = 89%

strength = 99% / 3

= 33%

N

y

y

34%

Evid 3

Page 25: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 25

Births & InteractionsBirths & Interactions1. If time period t = k * I, where k is some constant, birth new

agent.

2. Random selection of agent to compare with newest-born.

3. Compare agents and compute consistency. If consistency… = C, both agents are "winners“; increase both agents' plausibility

indices by D < C, "inconsistency conflict“ => competition.

higher plausibility index => "winner" (“draw” if indices identical). If consistency…

= N, "loser" is salvageable by “reinterpreting” one of its inconsistent features to match winner’s

< N, discount (decrease plausibility by D) Agents with plausibility = 0% => death & removal from system

4. Gather stories in system. Check strengths.

“Winning story found” if 1 ! story with strength >= S and/or |strength-strength | >= Sd for all competing stories; stop run early.

Page 26: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 26

Operationalizing Consistency--ExamplesOperationalizing Consistency--Examples

Possible Reactions to New Evid :

Completely consistent (e.g., 100% features match) => “reinforce” both; “reward consistency”

Inconsistent…

but salvageable (50% features match) => “reinterpret” less plausible “loser”; “increase inconsistency”

not salvageable (0% features match) => “discount” “loser”; “punish inconsistency”

=> plaus(Evid1) > plaus(Evid2) => Evid2 “loser”

x

y

51%

Evid 1

x

y

34%

Evid 2

x

y

61%

Evid 1

x

y

44%

Evid 2

x

y

51%

Evid 1

y

y

34%

Evid 2

x

y

51%

Evid 1

x

y

34%

Evid 2

x

x

51%

Evid 1

y

y

34%

Evid 2

x

x

51%

Evid 1

x

y

24%

Evid 2

Page 27: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 27

AgendaAgenda

1. The Phenomenon

2. Agent-Based Model (ABM) Primer

3. The Model

4. Sample Run

5. Experiments

Page 28: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 28

Sample Output^Sample Output^ Live Evidence--Printing 8 agents:

01 02 03 04 05 06 07 08

N G I G N I N N

y y y y y y y y

y y y y y y y y

100% 100% 100% 100% 100% 100% 100% 80%

Dead/Rejected Evidence--Printing 9 agents:

01 02 03 04 05 06 07 08 09

I G G I G I N N N

y y x x y x x x y

y x x x y x x y y

00% 00% 00% 00% 00% 00% 00% 00% 00%

3 stories found:

-Verdict N supported by 4 evidence, with 1.00 consistency => 0.48 strength

-Verdict G supported by 2 evidence, with 1.00 consistency => 0.25 strength

-Verdict I supported by 2 evidence, with 1.00 consistency => 0.25 strength

*** Found winning story! Verdict N supported by 4 evidence, with 1.00 consistency => 0.48 strength

Page 29: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 29

Judge-System can get Stuck…Judge-System can get Stuck…results\sysout_STUCK.log

Live Evidence--Printing 17 agents:

01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17

N G G I G I I I I N I G G N N N G

x x x x x x x x x x x x x x x x x

y y y x y y y y y y y y y y y y y

100% 100% 100% 90% 100% 100% 100% 100% 100% 100% 84% 100% 79% 100% 100% 98% 100%

Dead/Rejected Evidence--Printing 10 agents:

01 02 03 04 05 06 07 08 09 10

I I I I N I I G N G

y x y y y y y y y y

y x x x x x x y x x

00% 00% 00% 00% 00% 00% 00% 00% 00% 00%

3 stories found:

-Verdict N supported by 5 evidence, with 1.00 consistency => 0.29 strength

-Verdict G supported by 6 evidence, with 1.00 consistency => 0.34 strength

-Verdict I supported by 6 evidence, with 1.00 consistency => 0.34 strength

No winning story found.

Page 30: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 30

Output of a Run^Output of a Run^

GGG GGGGGGGG GGGGGGG GG

5 10 15 20

04

8

N N N N N N N N N N N N N N N N N N N N

5 10 15 20

04

8

Supporting Evidence (#)

Time (presentation of agents)

GGGGGGG GGGGGGG GGGGGG

5 10 15 20

0.0

0.6

N NN

N N N N N N N N N N N N N N N N N

5 10 15 20

0.0

0.6

Plausibility (%)

Time (presentation of agents)

GGGGG

GGGG

G GGGGGGG GG

5 10 15 20

0.0

0.6

N NN N N

N N N N N N N NN N

N NN N N

5 10 15 20

0.0

0.6

Consistency (%)

Time (presentation of agents)

GGGGGGG GGGGGGG GGGGGG

5 10 15 20

0.0

0.4

0.8

N N NN N N N N N N N N N N N N N N N N

5 10 15 20

0.0

0.4

0.8

Strength (%)

Time (presentation of agents)

Page 31: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 31

AgendaAgenda

1. The Phenomenon

2. Agent-Based Model (ABM) Primer

3. The Model

4. Sample Run

5. Experiments

Page 32: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 32

5 Experiments5 Experiments

Experiment 1: Emergence of consistency

Decision Speed Experiment 2: Speedup Experiment 3: Accuracy tradeoff

Decision Accuracy—Order Effects Experiment 4: Emergence of order effects Experiment 5: Extending deliberation

Page 33: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 33

Obtaining Consistent Stories – Q1Obtaining Consistent Stories – Q1

Q1: Evidence-level consistency sufficient? Which of the 3 mechanisms?

Implementation: No rules

DV: Consistency of stories

Page 34: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 34

Obtaining Consistent Stories – Q1 ResultsObtaining Consistent Stories – Q1 Results

Reinforce Reinterpret Discount Median consistency

0.9

0.8

0.7

0.6

0.6

0.6

0.5

Reinterpret > Discount > Reinforcement

Page 35: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 35

Speed-Accuracy Tradeoff – Q2Speed-Accuracy Tradeoff – Q2

Q2: Reinterpretation & Discounting increase speed?

Prediction: Reinterpretation & Discounting allow much faster convergence

DV: Time to Converge, Max nEvid (Max Consistency)

Implementation: Both rules; all cases have reinforce

Page 36: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 36

Speed-Accuracy Tradeoff – Q2 ResultsSpeed-Accuracy Tradeoff – Q2 Results

Results of 10 Runs—Time to First Convergence, Maximum Number of Evidence,

Maximum Story Consistency

Run1 2 3 4 5 6 7 8 9 10

W/o Reinterp, W/o Discount results/sysout_081215_1938.log

NoC160.5

NoC130.5

NoC120.5

NoC120.5

NoC120.5

NoC140.5

NoC150.5

NoC120.5

NoC120.5

NoC130.5

W/o Reinterp, W/ Discountresults/sysout_081215_1937.log

NoC9

0.8

64

1.0

18110.6

NoC9

0.6

NoC100.5

NoC140.5

NoC110.6

NoC110.7

178

1.0

NoC9

0.5

W/ Reinterp, W/o Discountresults/sysout_081215_1935.log

NoC131.0

NoC120.8

NoC140.6

54

1.0

86

1.0

NoC120.6

NoC130.8

NoC120.9

21140.9

54

1.0

W/ Reinterp, W/ Discountresults/sysout_081215_1946.log

NoC100.8

NoC131.0

43

1.0

NoC8

0.8

126

1.0

43

1.0

NoC8

0.8

86

1.0

NoC9

0.8

19101.0

Page 37: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 37

Speed-Accuracy Tradeoff – Q2 ResultsSpeed-Accuracy Tradeoff – Q2 Results

Medians of 10 Runs—Time to First Convergence, Maximum Number of Evidence,

Maximum Story Consistency

Reinterpret > Discount > Reinforcement only

W/o Discounting W/ Discounting

W/o Re-interpretation Converged 0% of runs nEvid = 12.5

(0.5)

Converged 30% of runs, time = 17,

8(1.0)

W/ Re-interpretation Converged 40% of runs, time = 6.5, nEvid = 5

(1.0)

Converged 50% of runs, time = 8

6 (1.0)

Page 38: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 38

Speed-Accuracy Tradeoff – Q3Speed-Accuracy Tradeoff – Q3 Q3: What would happen if allow process to continue even after having

found winner? Any point to "holding off judgment" until all evidence presented?

DV: Which story wins? (Strength)

Prediction: Leader will only be strengthened; competing stories never get a foothold.

Implementation: Allow system to continue running even if found winner

Run 1 2 3 4 5 6 7 8 9 10

Winner once conditions met (G/N)

Winner after additional evidence presented (G/N)

Changed winner? (Y/N)

Page 39: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 39

Speed-Accuracy Tradeoff – Q3 Results^Speed-Accuracy Tradeoff – Q3 Results^

Over 20 runs, # runs same winner:# runs different winner = 15:1

=> Good heuristic to stop deliberation, for less time & effort

GGG GGGGGGGG GGGGGGG GG

5 10 15 20

04

8

N N N N N N N N N N N N N N N N N N N N

5 10 15 20

04

8

Supporting Evidence (#)

Time (presentation of agents)

GGGGGGG GGGGGGG GGGGGG

5 10 15 20

0.0

0.6

N NN

N N N N N N N N N N N N N N N N N

5 10 15 20

0.0

0.6

Plausibility (%)

Time (presentation of agents)

GGGGG

GGGG

G GGGGGGG GG

5 10 15 20

0.0

0.6

N NN N N

N N N N N N N NN N

N NN N N

5 10 15 20

0.0

0.6

Consistency (%)

Time (presentation of agents)

GGGGGGG GGGGGGG GGGGGG

5 10 15 20

0.0

0.4

0.8

N N NN N N N N N N N N N N N N N N N N

5 10 15 20

0.0

0.4

0.8

Strength (%)

Time (presentation of agents)

Figure 2. Example run where winner switches

Page 40: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 40

Order Effects – Q4Order Effects – Q4

Heuristic may be ok only if randomized evidence…what if biased evidence?

Q4: Is there an Order Effect?

Took {20 randomly-generated evidence} and then “doctored” it; % D win = “accuracy”

IV: Presentation order--P goes first, followed by D vs. interwoven evidence

Prediction: earlier, weaker side (e.g., P) beats out later, stronger side (e.g., D); "Accuracy" of D…P… > PDPDPD… > P…D…

DV: Time to Converge, (Projected) Winner

Page 41: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 41

Order Effects – Q4 ResultsOrder Effects – Q4 Results

GGG GGGGGGGG GGGGGGG GG

5 10 15 20

04

8

N N N N N N N N N N N N N N N N N N N N

5 10 15 20

04

8

Supporting Evidence (#)

Time (presentation of agents)

GGGGGGG G

GGGGGG GGGGGG

5 10 15 20

0.0

0.6

N N N N N N N N N N N N N N N N N N N N

5 10 15 20

0.0

0.6

Plausibility (%)

Time (presentation of agents)

G

GG GGGGGG

GG GGGGGGG GG

5 10 15 20

0.0

0.6 N N N

N N N NN N N N N N N N N N N

N N

5 10 15 20

0.0

0.6

Consistency (%)

Time (presentation of agents)

GGGGGGG GGGGGGG GGGGGG

5 10 15 20

0.0

0.4

0.8

N NN N N N N N N N N N N N

N N N N N N

5 10 15 20

0.0

0.4

0.8

Strength (%)

Time (presentation of agents)

Figure 3. Sample random presentation order run.

Page 42: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 42

GGG GGGGGGGG GGGGGGG GG

5 10 15 20

04

8

N N N N N N N N N N N N N N N N N N N N

5 10 15 20

04

8

Supporting Evidence (#)

Time (presentation of agents)

GGGGGGG GGGGGG

G GGGGGG

5 10 15 20

0.0

0.6

N N N N N N N N N N N NN N N N N N N N

5 10 15 20

0.0

0.6

Plausibility (%)

Time (presentation of agents)

GGG GGGGGGGG GGGGG

GG GG

5 10 15 20

0.0

0.6

N N

N N N N N N N N N N N N N N N N N N

5 10 15 20

0.0

0.6

Consistency (%)

Time (presentation of agents)

GGGGGGG GGGGGGG GGGGGG

5 10 15 20

0.0

0.4

0.8

NN

N N N N N N N N N NN N N N N N N N

5 10 15 20

0.0

0.4

0.8

Strength (%)

Time (presentation of agents)

Order Effects – Q4 ResultsOrder Effects – Q4 Results

Figure 4. Sample D…P… biased presentation order run.

Page 43: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 43

Order Effects – Q4 ResultsOrder Effects – Q4 Results

Figure 5. Sample P…D… biased presentation order run.

GGG GGGGGGGG GGGGGGG GG

5 10 15 20

06

12

N N N N N N N N N N N N N N N N N N N N

5 10 15 20

06

12

Supporting Evidence (#)

Time (presentation of agents)

GGGG

GGG GGGGGGG GGGGGG

5 10 15 20

0.0

0.6

NN N N N N N N N N N N N N N N N N N N

5 10 15 20

0.0

0.6

Plausibility (%)

Time (presentation of agents)

GGGG

GGGGGGG GGGGGGG GG

5 10 15 20

0.0

0.6 N

N N N N N N

N NN N N N N N N N

N N N

5 10 15 20

0.0

0.6

Consistency (%)

Time (presentation of agents)

GGG

GGGG GGGGGGG GGGGGG

5 10 15 20

0.0

0.4

0.8

N N N N N N N N N N N N N N N N N N N N

5 10 15 20

0.0

0.4

0.8

Strength (%)

Time (presentation of agents)

Page 44: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 44

Order Effects – Q4 ResultsOrder Effects – Q4 Results

Table 4. Results of 10 Runs Varying Presentation Order of Evidence

Presentation Order

Median Time to Converge% Runs that Defense Won

% Runs Resulting in Switch in Winner

PDPDPD… 7D won 90%, NoC 10%

0%

D… P… 5D won 90%, NoC 10%

0%

P... D… 8P won 80%, NoC 20%

30%

• Strong primacy effect

• Exper3 conclusion no longer holds; longer deliberation DOES help!

Page 45: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 45

Increasing Deliberation – Q5Increasing Deliberation – Q5

Q5: Can deliberating more often between births reduce order effects (i.e., increase "accuracy")?

Implementation: Use P…D… model from Exper4

IV: Varied I (I = 0 => wait till end to deliberate)

DV: % runs that P wins

Page 46: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 46

Increasing Deliberation – Q5 ResultsIncreasing Deliberation – Q5 Results

Run 1 2 3 4 5 6 7 8 9 10 % P Wins

I = 0.8 * (# agents – 1)

60%

I = 1.0 * 80%

I = 1.2 * 100%

Too much lag time during trials can be detrimental!

Page 47: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 47

Summary of Key FindingsSummary of Key Findings Why reinterpret and discount evidence?

1. Maximizes consistency (Experiment 1)2. Hastens convergence on decision (Experiment 2)

Reinterpret > Discount > Reinforcement

Speed-accuracy tradeoff? Yes… Accuracy ok if evidence balanced (Experiment 3) Not ok/primacy effect if biased (Experiment 4) => Important to interweave evidence, like in real trials!

Can reduce primacy effect by decreasing premature deliberation (Experiment 5)

All achieved by modeling consistency @ evidence level, not story level

Page 48: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 48

(Future Expansions)(Future Expansions) Q6: What happens when introduce bias toward

certain verdicts? Prediction: Verdict-driven process takes less time to

converge DV: Time to Converge (Consistency of Stories) Implementation: Add favoredVerdict

Q7: In general, what conditions lead to indecision?

Page 49: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 49

BACKUPS

Page 50: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 50

AgendaAgenda

1. The Phenomenon

2. Agent-Based Model (ABM) Primer

3. The Model

4. Sample Run

5. Experiments

Page 51: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 51

Sequential Stream of Evidence…Sequential Stream of Evidence…

Evidence 1

Compare, Deliberate, Interpret

Evidence 2

Evidence 3

Evidence N

Official Deliberatio

n

Story/ Verdict

Premature Story/

Verdict @ Evid n < N

Page 52: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 52

A Variant of “Biased Predecisional Processing”^A Variant of “Biased Predecisional Processing”^

“leader” / “favored option”

Pre-Decisional Spreading

A

B,C,D,...switch to A DECISIONn

eg

ati

vep

osi

tive

Post-Decisional Spreading

Con

fid

en

ce o

r Perc

eiv

ed

Su

pp

ort

A

B,C,D,...

DECISIONneg

ati

vep

osi

tive

Time

Page 53: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 53

Tests (DV: Time to converge; stories’ consistency)

Model 0 P Model 1

Reinterpretation Speeds Process: W/o interpretation/

discounting> With

reinterpretation/

discounting

Primacy Order Effects:Weaker stories presented earlier beat stronger stories presented later

E.g., Defense is disadvantaged since it presents evidence after Prosecution!

>

for

Introduce Bias Toward Certain Verdicts (Kunda motivated reasoning, 1999; Thagard emotional coh, 2003)Lifetime of features and agents that differ from those initially (& arbitrarily) associated with favored verdict:

Data-driven process (stories emerge and compete w/o intervention)

> Verdict-driven process (e.g., “defendants tend to be guilty”):

Tests & PredictionsTests & Predictions

Page 54: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 54

(2 Levels of Plausiblity & Consistency)(2 Levels of Plausiblity & Consistency)

Plausibility1. Plausibility of evidence2. Plausibility (“strength”) of story

(composed of evidence)

Consistency1. Consistency /b/ 2 evidence2. Consistency among evidence @ story

level

Page 55: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 55

(Notes)(Notes)

“fast learning” with constant D with every winner/loser

100% & 0% are absorbing states for plausibility indices

Does not take into account innoculation effects as predicted by cross-cultural differences (Nisbett et al.)

Page 56: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 56

Other Simplifying AssumptionsOther Simplifying Assumptions Features

Any combination of abstract features can be framed (e.g., by the lawyers) to support a verdict. All evidence are automatically categorizable into a verdict.

Interactions Interactions only take place between the newest agent and another agent

(as opposed to having two older agents interact and compete).

Computing Consistency Verdict feature neither considered nor included in interactions

Comparing Stories judge forms 1 story / verdict If only 1 story is found at time t, use the "sufficiently strong" rule as

opposed to the "sufficiently stronger" rule. (i.e., no “holding out” by judge) Which layer drives consistency-seeking?

consistency at the individual agent interaction level, AND/OR consistency at the story level

Page 57: Seeking Consistent Stories

04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 57

ReferencesReferencesAronson, E. (1999). Dissonance, hypocrisy, and the self-concept. In E. Harmon-Jones & J. Mills (Eds.),

Cognitive dissonance: Progress on a pivotal theory in social psychology (pp. 103–126). Washington, DC: American Psychological Association.

Beckmann, J., & Kuhl, J. (1984). Altering information to gain action control: Functional aspects of human information processing in decision making. Journal of Research in Personality, 18, 224–237.Brownstein, A. (2003). Biased predecision processing. Psychological Bulletin, 129, 545-568.

Cacioppo, J. T., Petty, R. E., & Kao, C. E. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 48, 306-307.

Cialdini, R. B. (1993). Influence: The psychology of persuasion. New York: Morrow.Curley, S. P., Yates, J. F., & Abrams, R. A. (1986). Psychological sources of ambiguity avoidance. Organizational

Behavior and Human Decision Processes, 38, 230-256.Feather, N. T. (1982). Expectations and actions: Expectancy-value models in psychology. Hillsdale, NJ:

Lawrence Erlbaum.Festinger, L. (1957). A theory of cognitive dissonance. Evanston, IL: Row, Peterson.Gollwitzer, P. M., Heckhausen, H., & Ratajczak, H. (1990). From weighing to willing—Approaching a change

decision through pre- or postdecisional mentation. Organizational Behavior and Human Decision Processes, 45, 41-65.

Harmon-Jones, E. (2000). An update on dissonance theory, with a focus on the self. In A. Tesser, R. Felson, & J. Suis (Eds.), Psychological perspectives on self and identity. Washington, DC: American Psychological Association.

Kuhl, J. (1994). Motivation and volition. In G. d'Ydevalle, P. Bertelson, & P. Eelen (Eds.), Current advances in psychological science: An international perspective (pp. 311–340). Hillsdale, NJ: Erlbaum.

Kuhl, J. (1984). Volitional aspects of achievement motivation and learned helplessness: Toward a comprehensive theory of action control. In B. A. Maher (Ed.), Progress in experimental personality research (Vol. 13, pp. 99-171). New York: Academic Press.

McCaul, K. D., Ployhart, R. E., Hinsz, V. B., McCaul, H. S. (1995). Appraisals of a consistent versus a similar politician: Voter preferences and intuitive judgments. Journal of Personality and Social Psychology, 68, 292-299.

Peng, K., & Nisbett, R. E. (1999). Culture, dialectics, and reasoning about contradiction. American Psychologist, 54, 741-754.

Simon, D., Krawczyk, D., & Holyoak, K.J. (2004). Construction of preferences by constraint satisfaction. Psychological Science, 15, 331-336.


Recommended