Date post: | 29-Dec-2015 |
Category: |
Documents |
Upload: | imogen-summers |
View: | 223 times |
Download: | 1 times |
Expectation, Surprise and the Hindsight Bias: The Effect of Information and Outcome Congruency on
Retrospective Judgments.
Ivan K. Ash, Ph.D.Assistant Professor
Psychology Department
Old Dominion University
Outline
• Introduction to Retrospective Judgment Making and the Hindsight Bias
• Present the Theories of Retrospective Judgment Making and Hindsight Bias
• Discuss the Role of Expectation and Surprise in Retrospective Judgment Making
• Present Two Studies that Test the the Predictions of These Different Theories
Predictive Judgments
• Secretary of Defense Donald Rumsfeld, 9/19/02 Testimony to the Senate Armed Services Committee regarding Iraq
• “No terrorist state poses a greater or more immediate threat to the security of our people and the stability of the world than the regime of Saddam Hussein in Iraq.”
Retrospective Judgment
• 3/14/04, Face the Nation, CBS News • Asked: “why … did they [Hussein’s
regime] pose an immediate threat to us, to this country?
• Rumsfeld: “Well, you're the--you and a few other critics are the only people I've heard use the phrase ‘immediate threat.’ I didn't. The president didn't. And it's become kind of folklore that that's--that's what's happened.”
Predictive Judgments• Predictive Judgment – assessment of the
likelihood of different possible future outcomes to a situation or event
• Guide choices and decisions we make in life
• Determine what actions we will take to solve our problems
• Help us set goals and make plans• Serve as hypotheses that can be used to test
the validity of our current knowledge or beliefs
Retrospective Judgments• Retrospective Judgments- after-the-fact
assessments of how predictable you would have judged an outcome before it had occurred
• Judging the inevitability of an outcome
• Used as a post-hoc test of the validity of our beliefs and knowledge
• Source for belief revision, behavioral modification, or theory building
Hindsight Bias
• Tendency for people’s retrospective judgments influenced by the actual outcome of a situation or event
• In retrospective judgment making you are supposed to ignore the outcome and attempt to remember was your predictive opinion.
• Hindsight bias is the tendency for people behave as if they “knew it all along”
Hindsight Bias can be Harmful• Carli (1999) – HSB and Victim Degradation
– Gave participants a text describing a couple going out on their first date
– Rate the likelihood that the date ended in the woman getting sexually assaulted by the man
– Rate their approval of the women’s behavior on the date
– Rate their opinion on the how favorable attributes of the women’s character and personality were
• Prediction Group (story ended with woman agreeing to return to man’s apartment)
• Retrospection Group (told that the man assaulting woman)
Carli (1999) resultsLikelihood of Rape
0
2
4
6
8
10
P R
not a
t all
likel
y
extr
emel
y lik
ely
Approval of Behavoir
0
2
4
6
8
10
P R
extr
eme
disa
prov
al
e
xtre
me
apro
val
Ratings of Charater
0
2
4
6
8
10
P R
Unf
avor
able
Fav
orab
le
HSB is a Reliable Effect• Meta-analysis of 122 studies (N = 7359)
(Christensen-Szalanski & Willham, 1991)– They put all the measured hindsight effects on a
comparable scale by Calculating the effect size for each study: Cohen’s d (1988, 1992)
– d = (MR– MP)/SD
– Average d = 0.34, 95% CI > 0• Small effect 0.20
• Medium effect 0.50
• Large effect 0.80
Summary
• People are unable to ignore outcome information when making retrospective judgments
• HSB can be harmful
• HSB is a reliable effect
• Important to explain how people make retrospective judgments and why these processes lead to HSB
Information AvailabilityPredictive Judgments
Prior Knowledge& Beliefs
Prediction (P)
Retrospective Judgments
Prior Knowledge& Beliefs P
EventOut-come
Retro-spection (R)SR
SubjectiveReaction
Information AvailabilityPredictive Judgments:Likelihood of outcome A, given possible outcomes A or B
Prior Knowledge& Beliefs P
Pre-outcomeInformation
Support ASupport B
PredictiveJudgment
FavorA or B
P (A) = Support A / (Support A + Support B)
Information AvailabilityRetrospective Judgments: Likelihood of outcome A, given possible outcomes A or B
Prior Knowledge& Beliefs P
EventOut-come
R
Support ASupport B
OutcomeInformation
A or B
Pre-outcomeInformation
SubjectiveReaction
SR
Surprising,Ambivalent,
orExpected
PredictiveJudgment
FavorA or B
RetrospectiveJudgment
Theories of Retrospective Judgment and HSB
• Direct Recall Theories• Cognitive Reconstruction Theories
– Anchoring and Adjusting – Re-judging After Integration of Outcome – Estimation Based on Metacognitive Cues – Re-judging After Surprise Cued Sense-
making
Direct Recall Theories
• Predictions:– Those given an outcome may recall the outcome
instead of their predictive judgment will pull the mean judgments toward the “true outcome”
– That those who do not receive outcome information will be more accurate at remembering their predictive judgments
Prior Knowledge& Beliefs P
EventOut-come
R
Direct Recall Theories
• This can be tested in a Within Subjects HSB design
• By comparing the number of “hits” or correctly recalled predictive judgments between– Those that receive an outcome (retrospective)– Those that do not receive an outcome (recall)
• “No outcome” recall is not more accurate
• Likely to err in either direction(Dehn & Erdfelder, 1998; Erdefeld & Buchner, 1998; Stahlberg, Eller, Maass, & Frey, 1995; Schwarz and Stahlberg, 2003)
Theories of Retrospective Judgment and HSB
• Direct Recall Theories• Cognitive Reconstruction Theories
– Anchoring and Adjusting – Re-judging After Integration of Outcome – Estimation Based on Metacognitive Cues – Re-judging After Surprise Cued Sense-
making
Cognitive Reconstruction Theories
• At the point of the retrospective judgment people attempt to re-judge the situation in order to reconstruct what they would have thought
• The outcome information in some way biases this re-judgment process
• Theory of Re-judgment process:– How are people reconstructing their predictive
judgments?
– What cues and processes do people use?
– How do these processes lead to Hindsight Bias?
Anchoring and Adjusting Schwarz and Stahlberg (2003)
• People use the given outcome information as metric by which to reproduce their initial judgment
• Begin with the actual outcome and then adjust their estimate away from this value
• Use metacognitive beliefs about their level of knowledge in the content domain to make this adjustment
Anchoring and Adjusting Schwarz and Stahlberg (2003)
• Knowledgeable = small adjustment from the given value
• Not knowledgeable = larger adjustment to compensate
• People are often over optimistic about their expertise
• This leads to the hindsight bias
Anchoring and AdjustingSchwarz and Stahlberg (2003)
Expected Ambivalent Surprising
PredictiveRetrospective
FavorGiven
FavorAlternative
Type of Outcome in the Same Domain
Anchor
Adjustment
Anchoring and AdjustingSchwarz and Stahlberg (2003)
Expected Ambivalent Surprising
PredictiveRetrospective
FavorGiven
FavorAlternative
Type of Outcome in the Same Domain
Anchor
Adjustment
HSB Predictions for different types of outcomes
Other
Theory Expected Ambivalent Surprising
Anchor &
Adjust
Little, no, or reverse
Less HSB Most HSB Equal R judg.
• Outcome information is automatically integrated into one’s memory representation
• Information in memory associated with or supportive of this outcome is more available
• Both predictive and retrospective judgments are made based on the current accessibility information in memory that supports an potential outcome (availability heuristic)
Re-judging after outcome integrationFischhoff (1975) , Hawkins& Hastie (1990), Wasserman, Lempert, & Hastie(1991)
Re-judging after outcome integrationFischhoff (1975) , Hawkins& Hastie (1990), Wasserman,
Lempert, & Hastie(1991)
Expected Ambivalent Surprising
PredictiveRetrospective
FavorGiven
FavorAlternative
Type of Outcome in the Same Domain
HSB Predictions for different types of outcomes
Other
Theory Expected Ambivalent Surprising
Anchor &
Adjust
Little, no, or reverse
Less HSB Most HSB Equal R judg.
Re-judge Integration
HSB HSB HSB
Estimation Based on Metacognitive Cues (Hoch and Loewenstein,1989; Ofir & Mazursky, 1990, 1997)
• People reconstruct there original judgment by using how “surprising” they found the outcome as a cue to their original opinion
• If you were not surprised by an outcome you think, “I would have known that!” – This leads to an overestimate of their predictive accuracy
(HBS)
• If you were surprised by an outcome you think “I would have never known that!” – This leads to either a more accurate (unbiased)
reconstruction or even a shift away from the given outcome
Estimation Based on Metacognitive Cues (Hoch and Loewenstein,1989; Ofir & Mazursky, 1990, 1997)
Expected Ambivalent Surprising
PredictiveRetrospective
FavorGiven
FavorAlternative
Type of Outcome in the Same Domain
HSB Predictions for different types of outcomes
Other
Theory Expected Ambivalent Surprising
Anchor &
Adjust
Little, no, or reverse
Less HSB Most HSB Equal R jdgmts
Re-judge Integration
HSB HSB HSB
Metacog.
Cue
Most HSB Less HSB No or reverse
- r
(sp, hsb)
Surprise cued “sense-making” Pezzo (2003)
• When a outcome is incongruent with prior expectations (surprising), individuals try to make sense of the outcome– Successful sense-making leads to new opinions and
beliefs about the situation– Or processing the outcome supporting information may
render that information more accessible in memory– A person may give more weight to the information that
supported the surprising outcome or think of new reasons that it occurred
– These new opinions and beliefs bias the reconstruction of one’s predictive judgment
Surprise cued “sense-making” Pezzo (2003)
Expected Ambivalent Surprising
PredictiveRetrospective
FavorGiven
FavorAlternative
Type of Outcome in the Same Domain
HSB Predictions for different types of outcomes
Other
Theory Expected Ambivalent Surprising
Anchor &
Adjust
Little, no, or reverse
Less HSB Most HSB Equal R jdgmts
Re-judge Integration
HSB HSB HSB
Metacog.
Cue
Most HSB Less HSB No or reverse
- r
(sp, hsb)
Sense-Making
No HSB Less HSB Most HSB + r
(sp, hsb)
Surprise and HSB
• Ofir & Mazursky (1997)
• Gave students a scenario of a man having a surgical procedure that had a 2% mortality rate
• Rate the probability of the man surviving the surgery
• Outcome group was told that man died
• Test for HSB on a surprising outcome
Ofir & Mazursky (1997)
80
82
84
86
88
90
92
94
96
98
100
No outcome Man Died
Pro
babi
lity
of L
ivin
g
Surprise and HSB
• Pezzo (2003)• Gave participant
descriptions of Psych studies with “surprising” outcomes
• “looks are equally important to men and women”
• Between subjects design
"Looks are equally important to men and women"
0
10
20
30
40
50
60
70
80
90
100
No Outcome Surprising Outcome
Pro
babi
lity
of S
urpr
isin
g O
utco
me
Problems with the DataConfounds and Incomplete Designs
• Unsystematic Manipulation– Do not look for HSB on the expected or ambivalent
outcomes– Vary materials and domain with their proposed surprise
manipulation
• Treatments Based on Intuition– “Surprise” manipulations simply tell people that one
outcome is extremely rare– Or depend on biases and beliefs that participants bring into
the lab
• Between Subjects Designs – do not actually have measures of expectation in the Retrospective Groups
Experiment 1:Effects of Pre-outcome Information on HSB
• All of the theories make different claims about effect that expectation and surprise should have on HSB
• Cannot manipulate expectation or surprise• Can manipulate the amount of pre-outcome
information that supports different outcomes
• Can manipulate whether this information is congruent or incongruent with the given outcome
Pre-outcome Information
• Text describing an upcoming Championship Tennis Match between two players or an ancient battle between two armies.
• Compare and contrast: where the supporting information for opponent was matched
• Completely fictional • “Bias” pre-outcome information by
manipulating the ratio of supporting information for each player
Pre-outcome Information Tennis Equated for Either Outcome
• Mark Krause– Experienced
– Trains hard
– Past Champion
– Top Ranked
– Great Serve
– Patient Style
– No Wind
– Over the hill
– Style Backfires
• Nathan Mitchell– Ambitious
– Natural Athlete
– Olympic Champ
– Top Ranked
– Great Backhand
– Aggressive Style
– Hot Day
– Inexperienced
– Style Backfires
Pre-outcome Information Tennis Supporting Mitchell Wins Outcome
• Mark Krause– Experienced
– Trains hard
– Past Champion
– Top Ranked
– Great Serve
– Patient Style
– No Wind
– Over the hill
– Style Backfires
• Nathan Mitchell– Ambitious
– Natural Athlete
– Olympic Champ
– Top Ranked
– Great Backhand
– Aggressive Style
– Hot Day
– Inexperienced
– Style Backfires
Pre-outcome Information Tennis Supporting Krause Wins Outcome
• Mark Krause– Experienced
– Trains hard
– Past Champion
– Top Ranked
– Great Serve
– Patient Style
– No Wind
– Over the hill
– Style Backfires
• Nathan Mitchell– Ambitious
– Natural Athlete
– Olympic Champ
– Top Ranked
– Great Backhand
– Aggressive Style
– Hot Day
– Inexperienced
– Style Backfires
Outcome Information
• Text that describes the “true” outcome of the match.
• It only tells who won the match
• Does not describe the event or justify the outcome in any way
• Allows for the manipulation of the congruency of the outcome with the pre-outcome information
Pre-outcome Information/Outcome Congruency
• Congruent Condition (Expected) n =31– Krause Supporting Text with Krause Wins outcome– Mitchell Supporting Text with Mitchell Wins outcome
• Equated Condition (Ambivalent) n = 29– Equal Text with Krause Wins outcome– Equal Text with Mitchell Wins outcome
• Incongruent Condition (Surprising) n =31– Krause Supporting Text with Mitchell Wins outcome– Mitchell Supporting Text with Krause Wins outcome
Experiment 1: General Procedure
• Read the pre-outcome text • Rated the likelihood of the possible outcomes
(Predictive Judgment)• Rated how relevant or supportive each sentence was
to the possible outcomes (Evidence Relevance Rating)• Read the outcome text• Rate the how surprising they found the outcome
(Surprise Rating)• One week interval• Attempted to remember their original ratings
(Retrospective Judgment)
Results: Surprise RatingsExperiment 1: Effect of Outcome Congruence on Surprise Ratings
0
10
20
30
40
50
60
70
80
Congruent Equated Incongruent
Text/Outcome Congruency Condition
Sur
pris
e R
atin
g
F (2, 88) = 26.37, p < 0.001, η2 = 0.38
HSB Predictions for Pre-outcome information/outcome congruency
Other
Theory Congruent(Expected)
Equated(Ambivalent)
Incongruent(Surprising)
Anchor &
Adjust
Little, no, or reverse
Less HSB Most HSB Equal R jdgmts
Re-judge Integration
HSB HSB HSB
Metacog.
Cue
Most HSB Less HSB No or reverse
- r
(sp, hsb)
Sense-Making
No HSB Less HSB Most HSB + r
(sp, hsb)
Results: Hindsight BiasExperiment 1: Hindsight Bias as a Function of Outcome Congrency
-40
-30
-20
-10
0
10
20
30
40
Congurent Equated Incongruent
Text/outcome condition
Jud
gm
en
t
Prediction Retrospection
d = -0.01 d = 0.54 d = 0.41
HSB main effect: F (1, 88) = 8.37, p < 0.01, η2 = 0.09HSB X condition: F (2, 88) = 2.36, p = 0.10, η2 = 0.05
HSB Predictions for Pre-outcome information/outcome congruency
Other
Theory Congruent(Expected)
Equated(Ambivalent)
Incongruent(Surprising)
Anchor &
Adjust
Little, no, or reverse
Less HSB Most HSB Equal R jdgmts
Re-judge Integration
HSB HSB HSB
Metacog.
Cue
Most HSB Less HSB No or reverse
- r
(sp, hsb)
Sense-Making
No HSB Less HSB Most HSB + r
(sp, hsb)
Experiment 1: Conclusions
• Congruent outcomes = very little HSB• Equated outcomes = usual HSB• Incongruent outcomes = usual HSB• Somewhat consistent with the surprise cued sense-
making account– Power < 50% detecting interaction– Are memory effects driving the HSB?– Is surprise mediating the HSB effect?
• Small sample for judging correlation predictions
Experiment 2
• Replicate the findings of Experiment 1
• Increased sample size for power to test interaction and relationship between surprise and HSB
• Added a memory test to see if memory effects are leading to the HSB
Pre-outcome Information/Outcome Congruency
• Congruent Condition (Expected) n =67
– Krause Supporting Text with Krause Wins outcome
– Mitchell Supporting Text with Mitchell Wins outcome
• Equated Condition (Ambivalent) n = 61
– Equal Text with Krause Wins outcome
– Equal Text with Mitchell Wins outcome
• Incongruent Condition (Surprising) n =74
– Krause Supporting Text with Mitchell Wins outcome
– Mitchell Supporting Text with Krause Wins outcome
Experiment 2: General Procedure• Read the pre-outcome text • Rated the likelihood of the possible outcomes
(Predictive Judgment)• Rated how relevant or supportive each sentence was to
the possible outcomes (Evidence Relevance Rating)• Read the outcome text• Rate the how surprising they found the outcome
(Surprise Rating)• One week interval• Attempted to remember their original ratings
(Retrospective Judgment) • Attempted to remember as much of the story as possible
(Neutral, Mitchell Supporting, Krause Supporting)
Results: SurpriseExperiment 2: Effect of Outcome Congruence on Surprise Ratings
0
10
20
30
40
50
60
70
80
Congruent Equated Incongruent
Text/outcome congruence
Sur
pris
e R
atin
g
F (2, 199) = 45.11, p < 0.001, η2 = 0.31
Results: Hindsight BiasExperiment 2: Hindsight Bias as a Function of Outcome Congrency
-40
-30
-20
-10
0
10
20
30
40
Congurent Equated Incongruent
Text/outcome condition
Jud
gm
en
t
Prediction Retrospection
HSB main effect: F (1, 199) = 14.74, p < 0.001, η2 = 0.07HSB X condition: F (2, 199) = 5.05, p < 0.01, η2 = 0.05
d = -0.06 d = 0.38 d = 0.41
Memory Results
• Divided the story up into “information units”– 17 outcome neutral items– Equated Story = 21 for each player– Biased Stories = 21 for one player, 13 for other
• Two independent raters coded all recall sheets
• High inter-rater reliability ≈ 0.9
Results: Recall Neutral Items (17 possible)
Experiment 2: Memory for Nuetral Items
0
1
2
3
4
5
6
Congruent Equated Incongruent
Text/Outcome Congruency
Nu
mb
er
of I
tem
s R
eca
lled
F (2, 199) = 1.40, p = 0.25, η2 = 0.01
Results: Recall in Equated Text Condition
Equated Group (21 possible items each)
0
1
2
3
4
5
6
Support Given Support Alternative
Type of Information
Nu
mb
er
of
Ite
ms
Re
calle
d
F (1, 66) = 43.40, p < 0.001, η2 = 0.40
d = 0.81
Experiment 2: Recall in Baised Text Conditions as a function of Outcome Congruence
0
1
2
3
4
5
6
21 Items Avalible 13 Items Avalible
Type of Information
Num
ber
of It
ems
Rec
alle
d
CongruentIncongruent
Results: Recall in Bias Story Conditions
d = 0.52
d = 0.54
Information main effect: F (1, 133) = 59.00, p < 0.001, η2 = 0.31Info X congruency: F (1, 133) = 32.67, p < 0.001, η2 = 0.20
Results: Surprise & HSBBoth Experiments (n = 293)
• No Correlation• Congruent
r = -0.16, p = 0.13• Equated
r = 0.11, p = 0.30• Incongruent
r = 0.03, p = 0.77
Summary of Experiment 2
• Congruent outcomes = very little HSB• Equated outcomes = usual HSB• Incongruent outcomes = usual HSB• Recalled more outcome supporting information in
all conditions• Surprise level did not predict the magnitude of the
hindsight bias
Surprise & Sense-Making
Info/OutcomeCongruency
Surprise SenseMaking
HSB
Pezzo (2003)
Info/OutcomeCongruency
Surprise
SenseMaking
HSB
Current Studies
Sense-Making & Retrospection
SenseMaking
Sense-making & Memory
ProcessOutcomeSupporting Information
StrongerMemoryOutcomeSupporting
RetrospectiveJudgment
SenseMaking
More Weight to Supporting
DiscountNon-supporting
New Representation
Sense-making & Beliefs
RetrospectiveJudgment
Take Home Message on HSB• “Those who forget the past are doomed to repeat it”• HSB was more prevalent when people were unsure
or just plain wrong in foresight!• This makes it an extremely maladaptive judgment
making bias. • Future directions
– Generalize these results to more “real world” judgment situations.
– Investigate whether the HSB may play a role in hindering learning in tasks that involve predictive and retrospective judgment making.
I bet you already knew all of this!!
Thanks UIC– Dr. Jennifer Wiley – Dr. Susan Goldman– Dr. Joe Magliano– Dr. Gary Raney– Dr. Keith Thiede– Mindy Jensen– Jimmy Wong
– University Fellowship
Thanks ODU– Clinton Comer
– Luke Lin
– Martin Smith-Rodden
– Jennifer McMaster
– Amanda Fletcher
– Jen Taylor
– Dave Finch
Contact: [email protected] Web: www.odu.edu/~iash