Lydia L. ChenInstructor: Robert AxelrodPS793 – Complexity Theory
Apr 21, 2023
04/21/23 Lydia L. Chen <[email protected]>
Seeking Consistent StoriesSeeking Consistent StoriesBy Reinterpreting or Discounting Evidence: By Reinterpreting or Discounting Evidence:
An Agent-Based Model^An Agent-Based Model^
CSCN Presentation
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 2
AgendaAgenda
1. The Phenomenon
2. Agent-Based Model (ABM) Primer
3. The Model
4. Sample Run
5. Experiments
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 3
Research InterestsResearch Interests
Goal: psychology, law, computational modeling
persuasion and decision making
law classes => persuasion techniques storytelling narrative coherence metaphors
Story model of jury decision making (Pennington & Hastie, 1986)
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 4
Real-Life Scenario: Bench TrialReal-Life Scenario: Bench TrialProsecution () Defense Lawyer
“Guilty!
”“Innocen
t!”
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 5
Sequential Evidence – What’s NormativeSequential Evidence – What’s Normative
Evidence 1
…
Evidence 2
Evidence 3
Evidence N
Official Deliberatio
n
Story/ Verdict
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 6
Empirical Literature on JDMEmpirical Literature on JDM Form “coherent” stories that support option/verdict
(Pennington & Hastie, 1988)
Confidence in option/verdict increases with “coherence” (Glockner et al., under review)
Decision threshold: “sufficiently strong” (supported by many consistent evidence) or “sufficiently stronger” than other stories (review by Hastie, 1993
Narrative coherence: consistency causality completeness
“Consistency” aspect of “good” stories consistency between evidences in a story consistency of evidence with favored story
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 7
Example Case (Pennington & Hastie, 1988)Example Case (Pennington & Hastie, 1988)
Scenario: Defendant Frank Johnson stabbed and killed Alan Caldwell
Evidence := facts or arguments given in support of a story/verdict Facts— “Johnson took knife with him” ; “Johnson pulled out his knife” Arguments—”Johnson pulled out knife because he wanted revenge” vs. “Johson
pulled out knife because he was afraid”
Story := set of evidence supporting a given verdict
Same evidence can be framed to support multiple verdicts/stories!
P D
Desired Verdict
“Guilty” “Innocent”
Story “Premeditated murder”
“Self-defense”
Interpretation of Evid
“Johnson was angry”
“Johnson was afraid”
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 8
Sequential Evidence – More DescriptiveSequential Evidence – More Descriptive
Evidence 1
Compare, Deliberate, Interpret
…
Evidence 2
Evidence 3
Evidence N
Official Deliberatio
n
Story/ Verdict
Premature Story/
Verdict @ Evid n < N
(Brownstein, 2003; Russo et al., 2000)
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 9
How People Deal with Incoming EvidenceHow People Deal with Incoming Evidence
People don’t just take evid @ face value, but are selective!
Possible Reactions to New Evid in Light of Old:
“reinforce” each other
“reinterpret” less plausible one (Russo et al., etc.); e.g., misremember the info
“discount” less plausible one (Winston)
actively “seek” more evidence (not modeled here)
Existing evidence A: “Johnson was not carrying a knife.”
New evidence B1: “Johnson is nonviolent.”
Inconsistent new evid B2: “Johnson pulled a knife.”
Reinterpret B2: “Johnson grabbed a knife from Caldwell.” (i.e., explain it was Caldwell’s knife, not Johnson’s)
Discount B2: “Witness must be mistaken.”
Judge asks layers follow-up questions
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 10
AgendaAgenda
1. The Phenomenon
2. Agent-Based Model (ABM) Primer
3. The Model
4. Sample Run
5. Experiments
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 11
What is Agent-Based Modeling?What is Agent-Based Modeling?
agents + interactions^
start simple; build up^
Key terminology agents system dynamics
agent births and deaths interactions/competitions
parameters
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 12
Contributions ABMs Can MakeContributions ABMs Can Make
Symbiotic relationship:
Behavioral Experiments
ABM
parsimony:
demo emergence of seemingly complex
phenomen from small set of simple rulespredictions:
new observations/predictions
understanding:
study processes in detail
Test
Inputdescription:
informs base assumptions
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 13
(Other Models (Hastie, 1993))(Other Models (Hastie, 1993))
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 14
(Contrast with Bayesian & Algebraic Models)(Contrast with Bayesian & Algebraic Models)
Algebraic (additive), Bayesian (multiplicative): “single meter” of overall plausibility
ABM allows: revisiting and reconsidering previously-
processed evidence interaction/competition between new
evidence and individual pieces of previous evidence (not just conglomerate)
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 15
Contrast with Story Model & ECHOContrast with Story Model & ECHO Explanatory Coherence Model := Thagard’s
Theory of Explanatory Coherence (TEC) + Story Model
Only implemented discounting, not reinterpretation
ABM enforces lower evidence-agent-level consistency, as opposed to higher story-level consistency
Unlike previous ABMs, model agents within individual as system
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 16
MotivationsMotivations
1. Study emergence of consistent stories via reinterpretation, discounting, reinforcement mechanisms
2. Adaptive? aid consistency? speed-accuracy tradeoff? avoid indecisiveness
increases convergence rates? order effects hurt accuracy?
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 17
AgendaAgenda
1. The Phenomenon
2. Agent-Based Model (ABM) Primer
3. The Model
4. Sample Run
5. Experiments
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 18
GoalGoal
Model consistency-seeking process in story formation
Present evidence-agents to judge-system
Judge-system compares evidence-agents => keep, reinterpret, or discount evidence (agents “interact” & “compete”)
Until sufficiently strong/stronger story emerges
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 19
Evidence-agents, composed of “features”
Operationalize consistency /b/ agents: “similarity” in abstract features (Axelrod’s culture ABM,
1997) “inverse Hamming distance” := % feature matches
AgentsAgents
G
x
x
89%
Evid 1
N
x
y
34%
Evid 2
…
Verdict (“G”,”N”,…)
Abstract features (binary)
Plausibility index (0%-
100%)
N
x
x
14%
Evid 1
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 20
ParametersParametersParameter Variabl
eBase
# agents to present
- Initial A0 3
- Additional A 20
# possible verdicts V 2
# features/evidence (excluding verdict) F 2
% feature matches for:
Sufficiently consistent C 100%
Close enough for reinterpretation N < C 50%
# interactions /b/ evidence-agent births I (A0+A -1)
Change rate of agent plausibility (increase upon winning, decrease upon losing)
D 0.1
Rule(s) and threshold(s) for winning story-- “sufficiently strong” and/or “sufficiently stronger”
S, Sd Both — 0.5, 0.2
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 22
System & Agent BirthsSystem & Agent Births
Judge-system represents judge’s mind
Evidence presentation = “agent birth”:
Initialization of N0 Agents: Set up randomly-generated agents, OR…
uniform distributions—even for plausibility index due to prior beliefs (Kunda, 1990) or knowledge (Klein, 1993)
User-specified
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 23
(Topology)(Topology)
results\sysout_081118_0455.log
Printing 8 agents:
01 02 03 04 05 06 07 08
N G I G N I N N
y y y y y y y y
y y y y y y y y
100% 100% 100% 100% 100% 100% 100% 80%
Dead/Rejected Evidence--Printing 9 agents:
01 02 03 04 05 06 07 08 09
I G G I G I N N N
y y x x y x x x y
y x x x y x x y y
00% 00% 00% 00% 00% 00% 00% 00% 00%
To keep track of evidence-agents and their order of presentation:
lists of agents in order of birth/presentation to the system.
latest-born agent always appears at the end of a list.
no geographical “topology”
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 24
StoriesStories:= {evidence-agents supporting a verdict}
Consistency = inverse Hamming distance amongst evidence (:= evidencePairs # feature matches / # evidencePairs / F )
Plausibility Index (“Strength”) = average of plausibility indices
G
x
x
89%
Evid 1N
x
y
51%
Evid 2
N
x
x
14%
Evid 1
Story promoting
“Guilty” verdict
Story promoting “Innocent”
verdict
strength = 89%
strength = 99% / 3
= 33%
N
y
y
34%
Evid 3
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 25
Births & InteractionsBirths & Interactions1. If time period t = k * I, where k is some constant, birth new
agent.
2. Random selection of agent to compare with newest-born.
3. Compare agents and compute consistency. If consistency… = C, both agents are "winners“; increase both agents' plausibility
indices by D < C, "inconsistency conflict“ => competition.
higher plausibility index => "winner" (“draw” if indices identical). If consistency…
= N, "loser" is salvageable by “reinterpreting” one of its inconsistent features to match winner’s
< N, discount (decrease plausibility by D) Agents with plausibility = 0% => death & removal from system
4. Gather stories in system. Check strengths.
“Winning story found” if 1 ! story with strength >= S and/or |strength-strength | >= Sd for all competing stories; stop run early.
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 26
Operationalizing Consistency--ExamplesOperationalizing Consistency--Examples
Possible Reactions to New Evid :
Completely consistent (e.g., 100% features match) => “reinforce” both; “reward consistency”
Inconsistent…
but salvageable (50% features match) => “reinterpret” less plausible “loser”; “increase inconsistency”
not salvageable (0% features match) => “discount” “loser”; “punish inconsistency”
=> plaus(Evid1) > plaus(Evid2) => Evid2 “loser”
x
y
51%
Evid 1
x
y
34%
Evid 2
x
y
61%
Evid 1
x
y
44%
Evid 2
x
y
51%
Evid 1
y
y
34%
Evid 2
x
y
51%
Evid 1
x
y
34%
Evid 2
x
x
51%
Evid 1
y
y
34%
Evid 2
x
x
51%
Evid 1
x
y
24%
Evid 2
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 27
AgendaAgenda
1. The Phenomenon
2. Agent-Based Model (ABM) Primer
3. The Model
4. Sample Run
5. Experiments
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 28
Sample Output^Sample Output^ Live Evidence--Printing 8 agents:
01 02 03 04 05 06 07 08
N G I G N I N N
y y y y y y y y
y y y y y y y y
100% 100% 100% 100% 100% 100% 100% 80%
Dead/Rejected Evidence--Printing 9 agents:
01 02 03 04 05 06 07 08 09
I G G I G I N N N
y y x x y x x x y
y x x x y x x y y
00% 00% 00% 00% 00% 00% 00% 00% 00%
3 stories found:
-Verdict N supported by 4 evidence, with 1.00 consistency => 0.48 strength
-Verdict G supported by 2 evidence, with 1.00 consistency => 0.25 strength
-Verdict I supported by 2 evidence, with 1.00 consistency => 0.25 strength
*** Found winning story! Verdict N supported by 4 evidence, with 1.00 consistency => 0.48 strength
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 29
Judge-System can get Stuck…Judge-System can get Stuck…results\sysout_STUCK.log
Live Evidence--Printing 17 agents:
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17
N G G I G I I I I N I G G N N N G
x x x x x x x x x x x x x x x x x
y y y x y y y y y y y y y y y y y
100% 100% 100% 90% 100% 100% 100% 100% 100% 100% 84% 100% 79% 100% 100% 98% 100%
Dead/Rejected Evidence--Printing 10 agents:
01 02 03 04 05 06 07 08 09 10
I I I I N I I G N G
y x y y y y y y y y
y x x x x x x y x x
00% 00% 00% 00% 00% 00% 00% 00% 00% 00%
3 stories found:
-Verdict N supported by 5 evidence, with 1.00 consistency => 0.29 strength
-Verdict G supported by 6 evidence, with 1.00 consistency => 0.34 strength
-Verdict I supported by 6 evidence, with 1.00 consistency => 0.34 strength
No winning story found.
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 30
Output of a Run^Output of a Run^
GGG GGGGGGGG GGGGGGG GG
5 10 15 20
04
8
N N N N N N N N N N N N N N N N N N N N
5 10 15 20
04
8
Supporting Evidence (#)
Time (presentation of agents)
GGGGGGG GGGGGGG GGGGGG
5 10 15 20
0.0
0.6
N NN
N N N N N N N N N N N N N N N N N
5 10 15 20
0.0
0.6
Plausibility (%)
Time (presentation of agents)
GGGGG
GGGG
G GGGGGGG GG
5 10 15 20
0.0
0.6
N NN N N
N N N N N N N NN N
N NN N N
5 10 15 20
0.0
0.6
Consistency (%)
Time (presentation of agents)
GGGGGGG GGGGGGG GGGGGG
5 10 15 20
0.0
0.4
0.8
N N NN N N N N N N N N N N N N N N N N
5 10 15 20
0.0
0.4
0.8
Strength (%)
Time (presentation of agents)
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 31
AgendaAgenda
1. The Phenomenon
2. Agent-Based Model (ABM) Primer
3. The Model
4. Sample Run
5. Experiments
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 32
5 Experiments5 Experiments
Experiment 1: Emergence of consistency
Decision Speed Experiment 2: Speedup Experiment 3: Accuracy tradeoff
Decision Accuracy—Order Effects Experiment 4: Emergence of order effects Experiment 5: Extending deliberation
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 33
Obtaining Consistent Stories – Q1Obtaining Consistent Stories – Q1
Q1: Evidence-level consistency sufficient? Which of the 3 mechanisms?
Implementation: No rules
DV: Consistency of stories
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 34
Obtaining Consistent Stories – Q1 ResultsObtaining Consistent Stories – Q1 Results
Reinforce Reinterpret Discount Median consistency
0.9
0.8
0.7
0.6
0.6
0.6
0.5
Reinterpret > Discount > Reinforcement
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 35
Speed-Accuracy Tradeoff – Q2Speed-Accuracy Tradeoff – Q2
Q2: Reinterpretation & Discounting increase speed?
Prediction: Reinterpretation & Discounting allow much faster convergence
DV: Time to Converge, Max nEvid (Max Consistency)
Implementation: Both rules; all cases have reinforce
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 36
Speed-Accuracy Tradeoff – Q2 ResultsSpeed-Accuracy Tradeoff – Q2 Results
Results of 10 Runs—Time to First Convergence, Maximum Number of Evidence,
Maximum Story Consistency
Run1 2 3 4 5 6 7 8 9 10
W/o Reinterp, W/o Discount results/sysout_081215_1938.log
NoC160.5
NoC130.5
NoC120.5
NoC120.5
NoC120.5
NoC140.5
NoC150.5
NoC120.5
NoC120.5
NoC130.5
W/o Reinterp, W/ Discountresults/sysout_081215_1937.log
NoC9
0.8
64
1.0
18110.6
NoC9
0.6
NoC100.5
NoC140.5
NoC110.6
NoC110.7
178
1.0
NoC9
0.5
W/ Reinterp, W/o Discountresults/sysout_081215_1935.log
NoC131.0
NoC120.8
NoC140.6
54
1.0
86
1.0
NoC120.6
NoC130.8
NoC120.9
21140.9
54
1.0
W/ Reinterp, W/ Discountresults/sysout_081215_1946.log
NoC100.8
NoC131.0
43
1.0
NoC8
0.8
126
1.0
43
1.0
NoC8
0.8
86
1.0
NoC9
0.8
19101.0
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 37
Speed-Accuracy Tradeoff – Q2 ResultsSpeed-Accuracy Tradeoff – Q2 Results
Medians of 10 Runs—Time to First Convergence, Maximum Number of Evidence,
Maximum Story Consistency
Reinterpret > Discount > Reinforcement only
W/o Discounting W/ Discounting
W/o Re-interpretation Converged 0% of runs nEvid = 12.5
(0.5)
Converged 30% of runs, time = 17,
8(1.0)
W/ Re-interpretation Converged 40% of runs, time = 6.5, nEvid = 5
(1.0)
Converged 50% of runs, time = 8
6 (1.0)
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 38
Speed-Accuracy Tradeoff – Q3Speed-Accuracy Tradeoff – Q3 Q3: What would happen if allow process to continue even after having
found winner? Any point to "holding off judgment" until all evidence presented?
DV: Which story wins? (Strength)
Prediction: Leader will only be strengthened; competing stories never get a foothold.
Implementation: Allow system to continue running even if found winner
Run 1 2 3 4 5 6 7 8 9 10
Winner once conditions met (G/N)
Winner after additional evidence presented (G/N)
Changed winner? (Y/N)
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 39
Speed-Accuracy Tradeoff – Q3 Results^Speed-Accuracy Tradeoff – Q3 Results^
Over 20 runs, # runs same winner:# runs different winner = 15:1
=> Good heuristic to stop deliberation, for less time & effort
GGG GGGGGGGG GGGGGGG GG
5 10 15 20
04
8
N N N N N N N N N N N N N N N N N N N N
5 10 15 20
04
8
Supporting Evidence (#)
Time (presentation of agents)
GGGGGGG GGGGGGG GGGGGG
5 10 15 20
0.0
0.6
N NN
N N N N N N N N N N N N N N N N N
5 10 15 20
0.0
0.6
Plausibility (%)
Time (presentation of agents)
GGGGG
GGGG
G GGGGGGG GG
5 10 15 20
0.0
0.6
N NN N N
N N N N N N N NN N
N NN N N
5 10 15 20
0.0
0.6
Consistency (%)
Time (presentation of agents)
GGGGGGG GGGGGGG GGGGGG
5 10 15 20
0.0
0.4
0.8
N N NN N N N N N N N N N N N N N N N N
5 10 15 20
0.0
0.4
0.8
Strength (%)
Time (presentation of agents)
Figure 2. Example run where winner switches
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 40
Order Effects – Q4Order Effects – Q4
Heuristic may be ok only if randomized evidence…what if biased evidence?
Q4: Is there an Order Effect?
Took {20 randomly-generated evidence} and then “doctored” it; % D win = “accuracy”
IV: Presentation order--P goes first, followed by D vs. interwoven evidence
Prediction: earlier, weaker side (e.g., P) beats out later, stronger side (e.g., D); "Accuracy" of D…P… > PDPDPD… > P…D…
DV: Time to Converge, (Projected) Winner
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 41
Order Effects – Q4 ResultsOrder Effects – Q4 Results
GGG GGGGGGGG GGGGGGG GG
5 10 15 20
04
8
N N N N N N N N N N N N N N N N N N N N
5 10 15 20
04
8
Supporting Evidence (#)
Time (presentation of agents)
GGGGGGG G
GGGGGG GGGGGG
5 10 15 20
0.0
0.6
N N N N N N N N N N N N N N N N N N N N
5 10 15 20
0.0
0.6
Plausibility (%)
Time (presentation of agents)
G
GG GGGGGG
GG GGGGGGG GG
5 10 15 20
0.0
0.6 N N N
N N N NN N N N N N N N N N N
N N
5 10 15 20
0.0
0.6
Consistency (%)
Time (presentation of agents)
GGGGGGG GGGGGGG GGGGGG
5 10 15 20
0.0
0.4
0.8
N NN N N N N N N N N N N N
N N N N N N
5 10 15 20
0.0
0.4
0.8
Strength (%)
Time (presentation of agents)
Figure 3. Sample random presentation order run.
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 42
GGG GGGGGGGG GGGGGGG GG
5 10 15 20
04
8
N N N N N N N N N N N N N N N N N N N N
5 10 15 20
04
8
Supporting Evidence (#)
Time (presentation of agents)
GGGGGGG GGGGGG
G GGGGGG
5 10 15 20
0.0
0.6
N N N N N N N N N N N NN N N N N N N N
5 10 15 20
0.0
0.6
Plausibility (%)
Time (presentation of agents)
GGG GGGGGGGG GGGGG
GG GG
5 10 15 20
0.0
0.6
N N
N N N N N N N N N N N N N N N N N N
5 10 15 20
0.0
0.6
Consistency (%)
Time (presentation of agents)
GGGGGGG GGGGGGG GGGGGG
5 10 15 20
0.0
0.4
0.8
NN
N N N N N N N N N NN N N N N N N N
5 10 15 20
0.0
0.4
0.8
Strength (%)
Time (presentation of agents)
Order Effects – Q4 ResultsOrder Effects – Q4 Results
Figure 4. Sample D…P… biased presentation order run.
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 43
Order Effects – Q4 ResultsOrder Effects – Q4 Results
Figure 5. Sample P…D… biased presentation order run.
GGG GGGGGGGG GGGGGGG GG
5 10 15 20
06
12
N N N N N N N N N N N N N N N N N N N N
5 10 15 20
06
12
Supporting Evidence (#)
Time (presentation of agents)
GGGG
GGG GGGGGGG GGGGGG
5 10 15 20
0.0
0.6
NN N N N N N N N N N N N N N N N N N N
5 10 15 20
0.0
0.6
Plausibility (%)
Time (presentation of agents)
GGGG
GGGGGGG GGGGGGG GG
5 10 15 20
0.0
0.6 N
N N N N N N
N NN N N N N N N N
N N N
5 10 15 20
0.0
0.6
Consistency (%)
Time (presentation of agents)
GGG
GGGG GGGGGGG GGGGGG
5 10 15 20
0.0
0.4
0.8
N N N N N N N N N N N N N N N N N N N N
5 10 15 20
0.0
0.4
0.8
Strength (%)
Time (presentation of agents)
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 44
Order Effects – Q4 ResultsOrder Effects – Q4 Results
Table 4. Results of 10 Runs Varying Presentation Order of Evidence
Presentation Order
Median Time to Converge% Runs that Defense Won
% Runs Resulting in Switch in Winner
PDPDPD… 7D won 90%, NoC 10%
0%
D… P… 5D won 90%, NoC 10%
0%
P... D… 8P won 80%, NoC 20%
30%
• Strong primacy effect
• Exper3 conclusion no longer holds; longer deliberation DOES help!
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 45
Increasing Deliberation – Q5Increasing Deliberation – Q5
Q5: Can deliberating more often between births reduce order effects (i.e., increase "accuracy")?
Implementation: Use P…D… model from Exper4
IV: Varied I (I = 0 => wait till end to deliberate)
DV: % runs that P wins
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 46
Increasing Deliberation – Q5 ResultsIncreasing Deliberation – Q5 Results
Run 1 2 3 4 5 6 7 8 9 10 % P Wins
I = 0.8 * (# agents – 1)
60%
I = 1.0 * 80%
I = 1.2 * 100%
Too much lag time during trials can be detrimental!
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 47
Summary of Key FindingsSummary of Key Findings Why reinterpret and discount evidence?
1. Maximizes consistency (Experiment 1)2. Hastens convergence on decision (Experiment 2)
Reinterpret > Discount > Reinforcement
Speed-accuracy tradeoff? Yes… Accuracy ok if evidence balanced (Experiment 3) Not ok/primacy effect if biased (Experiment 4) => Important to interweave evidence, like in real trials!
Can reduce primacy effect by decreasing premature deliberation (Experiment 5)
All achieved by modeling consistency @ evidence level, not story level
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 48
(Future Expansions)(Future Expansions) Q6: What happens when introduce bias toward
certain verdicts? Prediction: Verdict-driven process takes less time to
converge DV: Time to Converge (Consistency of Stories) Implementation: Add favoredVerdict
Q7: In general, what conditions lead to indecision?
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 50
AgendaAgenda
1. The Phenomenon
2. Agent-Based Model (ABM) Primer
3. The Model
4. Sample Run
5. Experiments
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 51
Sequential Stream of Evidence…Sequential Stream of Evidence…
Evidence 1
Compare, Deliberate, Interpret
…
Evidence 2
Evidence 3
Evidence N
Official Deliberatio
n
Story/ Verdict
Premature Story/
Verdict @ Evid n < N
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 52
A Variant of “Biased Predecisional Processing”^A Variant of “Biased Predecisional Processing”^
“leader” / “favored option”
Pre-Decisional Spreading
A
B,C,D,...switch to A DECISIONn
eg
ati
vep
osi
tive
Post-Decisional Spreading
Con
fid
en
ce o
r Perc
eiv
ed
Su
pp
ort
A
B,C,D,...
DECISIONneg
ati
vep
osi
tive
Time
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 53
Tests (DV: Time to converge; stories’ consistency)
Model 0 P Model 1
Reinterpretation Speeds Process: W/o interpretation/
discounting> With
reinterpretation/
discounting
Primacy Order Effects:Weaker stories presented earlier beat stronger stories presented later
E.g., Defense is disadvantaged since it presents evidence after Prosecution!
>
for
Introduce Bias Toward Certain Verdicts (Kunda motivated reasoning, 1999; Thagard emotional coh, 2003)Lifetime of features and agents that differ from those initially (& arbitrarily) associated with favored verdict:
Data-driven process (stories emerge and compete w/o intervention)
> Verdict-driven process (e.g., “defendants tend to be guilty”):
Tests & PredictionsTests & Predictions
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 54
(2 Levels of Plausiblity & Consistency)(2 Levels of Plausiblity & Consistency)
Plausibility1. Plausibility of evidence2. Plausibility (“strength”) of story
(composed of evidence)
Consistency1. Consistency /b/ 2 evidence2. Consistency among evidence @ story
level
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 55
(Notes)(Notes)
“fast learning” with constant D with every winner/loser
100% & 0% are absorbing states for plausibility indices
Does not take into account innoculation effects as predicted by cross-cultural differences (Nisbett et al.)
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 56
Other Simplifying AssumptionsOther Simplifying Assumptions Features
Any combination of abstract features can be framed (e.g., by the lawyers) to support a verdict. All evidence are automatically categorizable into a verdict.
Interactions Interactions only take place between the newest agent and another agent
(as opposed to having two older agents interact and compete).
Computing Consistency Verdict feature neither considered nor included in interactions
Comparing Stories judge forms 1 story / verdict If only 1 story is found at time t, use the "sufficiently strong" rule as
opposed to the "sufficiently stronger" rule. (i.e., no “holding out” by judge) Which layer drives consistency-seeking?
consistency at the individual agent interaction level, AND/OR consistency at the story level
04/21/23 Lydia L. Chen <[email protected]>“Seeking Consistency” - 57
ReferencesReferencesAronson, E. (1999). Dissonance, hypocrisy, and the self-concept. In E. Harmon-Jones & J. Mills (Eds.),
Cognitive dissonance: Progress on a pivotal theory in social psychology (pp. 103–126). Washington, DC: American Psychological Association.
Beckmann, J., & Kuhl, J. (1984). Altering information to gain action control: Functional aspects of human information processing in decision making. Journal of Research in Personality, 18, 224–237.Brownstein, A. (2003). Biased predecision processing. Psychological Bulletin, 129, 545-568.
Cacioppo, J. T., Petty, R. E., & Kao, C. E. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 48, 306-307.
Cialdini, R. B. (1993). Influence: The psychology of persuasion. New York: Morrow.Curley, S. P., Yates, J. F., & Abrams, R. A. (1986). Psychological sources of ambiguity avoidance. Organizational
Behavior and Human Decision Processes, 38, 230-256.Feather, N. T. (1982). Expectations and actions: Expectancy-value models in psychology. Hillsdale, NJ:
Lawrence Erlbaum.Festinger, L. (1957). A theory of cognitive dissonance. Evanston, IL: Row, Peterson.Gollwitzer, P. M., Heckhausen, H., & Ratajczak, H. (1990). From weighing to willing—Approaching a change
decision through pre- or postdecisional mentation. Organizational Behavior and Human Decision Processes, 45, 41-65.
Harmon-Jones, E. (2000). An update on dissonance theory, with a focus on the self. In A. Tesser, R. Felson, & J. Suis (Eds.), Psychological perspectives on self and identity. Washington, DC: American Psychological Association.
Kuhl, J. (1994). Motivation and volition. In G. d'Ydevalle, P. Bertelson, & P. Eelen (Eds.), Current advances in psychological science: An international perspective (pp. 311–340). Hillsdale, NJ: Erlbaum.
Kuhl, J. (1984). Volitional aspects of achievement motivation and learned helplessness: Toward a comprehensive theory of action control. In B. A. Maher (Ed.), Progress in experimental personality research (Vol. 13, pp. 99-171). New York: Academic Press.
McCaul, K. D., Ployhart, R. E., Hinsz, V. B., McCaul, H. S. (1995). Appraisals of a consistent versus a similar politician: Voter preferences and intuitive judgments. Journal of Personality and Social Psychology, 68, 292-299.
Peng, K., & Nisbett, R. E. (1999). Culture, dialectics, and reasoning about contradiction. American Psychologist, 54, 741-754.
Simon, D., Krawczyk, D., & Holyoak, K.J. (2004). Construction of preferences by constraint satisfaction. Psychological Science, 15, 331-336.