9/29/2015 John W Payne BA925 1
Probabilistic reasoning II: Heuristics and biases
Review of Heuristics and Biases: Tversky & Kahneman (1974) - 1
• Basic ideas: – People rely on a limited number of heuristic cues and processes that
reduce the complex task of assessing probabilities to one that is manageable. Related to the idea of “Bounded Rationality”.
– The heuristics are generally useful, but can lead to systemic errors (biases). Further, “errors of judgment are often systematic rather than random, manifesting bias rather than confusion” – Kahneman & Tversky (1979)
– The biases in forecasts are due to cognitive factors, not just motivational factors.
– Biases will be found with experts, e.g. doctors and lawyers, dealing with important problems as well as laypersons.
– Note, the idea is that error in judgment is not simply due to motivational effects. It is not an argument that motivational biases do not exist.
2
Heuristics and Biases: Tversky & Kahneman (1974) - 2
• Features of Heuristics:
– The heuristic processes are related to common psychological processes, e.g., similarity judgments. – How is this the same or different from Shah and Oppenheimer (2008) conception?
– The heuristics for probability judgments resemble subjective assessments of physical quantities like distance or size. That is, the processes are generally neither conscious or controllable. More System 1 type of thinking, see class 1 notes.
– The use of heuristics means that judgments may be a) insensitive to factors that should matter from a normative perspective, and b) sensitive to factors that shouldn’t matter. A research approach that uses the presence of biased responses to infer heuristic use. Cognitive illusions and visual illusions.
3 9/29/2015 John W Payne BA925
The Three Classic Heuristics
• Representativeness
• Availability
• Anchoring and (Insufficient) Adjustment
• The newer “Affect” Heuristic
9/29/2015 John W Payne BA925 4
Representativeness Heuristic and Biases: Tversky & Kahneman (1974)
• What is the Representativeness heuristic? – Used to make judgments about whether a target object belongs to a
given class of objects. – Judgment made on the basis of the similarity of the target object to
the prototypical instance of the class. Attribute or judgment substitution of an easier to make judgment, e.g. similarity, for a more difficult judgment such as probability. • Assumed that similarity is an easy-to-access form of information. • Is this another form of cognitive simplification?
• Some common biases
– Under-use of base-rates: The Tom W. example. • High correlation between mean similarity rank and mean
likelihood rank
– Discounting (Ignoring) sample sizes – The hospital problem and the Law of Small Numbers. “Fooled by Randomness”
• Relationship between representativeness and the frequency of events?
5 9/29/2015 John W Payne BA925
Classic Example of Representativeness: The Tom W. Problem
• Tom W. is on high intelligence, although lacking in true creativity. He has a need for order and clarity, and for neat an tidy systems in which every detail finds its appropriate place. His writing is rather dull and mechanical, occasionally enlivened by somewhat corny puns and flashes of imagination of the sci-fi type. He has a strong drive for competence. He seems to have little feel and little sympathy for other people and does not enjoy interacting with others. Self-centered, he nonetheless has a deep moral sense.
• The personality sketch of Tom W. given above was written during Tom’s senior year in high school by a psychologist on the basis of projective tests. Tom W. is currently a graduate student. Please rank the following nine field of graduate specialization in order of the likelihood that Tom W. is now a graduate student in each of these fields. (Use 1 for the most likely and 9 for the least likely.)
• A) Business Administration
• B) Computer Science
• C) Engineering
• D) Humanities and Education
• E) Law
• F) Library Science
• G) Medicine
• H) Physical and Life Sciences
• I) Social Science and Social Work
6 9/29/2015 John W Payne BA925
The Classic Tom W. Results
7
A) Business Administration
B) Computer Science
C) Engineering
D) Humanities and Education
E) Law
F) Library Science
G) Medicine
H) Physical and Life Sciences
I) Social Science and Social Work
Likelihood
Rank
Judged %
Base-rate
Similarity
Rank
4.3
2.5
2.6
7.6
5.2
4.7
5.8
4.3
8.0
3.9
2.1
2.9
7.2
5.9
4.2
5.9
4.5
8.2
15
7
9
20
9
3
8
12
17
An example from finance: How likely is an IPO to make (lose) money? Focus on
the case at hand and neglect of the base-rate of success for IPOs.
9/29/2015 John W Payne BA925
Example of a recent (2012) study: Does the use of base-rate versus case information vary by psychological distance of the
target?
• Task: Judge the likelihood that a person has a disease given some information on the base-rate of the disease, e.g., it is either rare or quite common, and given some information on the person, she has either one or four symptoms such as cough and/or fever.
• Vary the closeness of the target “Chris Chan” or Chris Smith (subjects were all of Chinese ethnicity living in Hong Kong).
• Assess the target’s likelihood of having the disease on a 7-point scale (1 = very unlikely; 7 = very likely).
8
See Yan & Sengupta, JCR, 2012
9/29/2015 John W Payne BA925
Results
9
Low Diag. High Diag. Diff. Score due to Diag
BR Low 2.31 3.61 1.30
BR High 2.69 4.07 1.38
Diff. Score due to BR
.38 .46
Low Diag. High Diag. Diff. Score
BR Low 2.40 2.94 .54
BR High 3.42 3.90 .48
Diff. Score due to BR
1.02 .96
Close
Far
Base-rate information had more of an effect when judging people more distant
psychologically. (Relate to “outside” versus “inside” perspective.
9/29/2015 John W Payne BA925
Law of Small Numbers?
• A fundamental concept in statistics is the law of large numbers. Loosely speaking, the law of large numbers says that as the size of a sample increases, the salient features of that sample (e.g., its mean) will more and more resemble the salient features of the population from which the sample is drawn.
• People often act as if they follow a “law of small numbers” in which even small samples are thought to be highly representative (resemble) the population from which the sample is drawn.
– Consequently, people will tend to over infer from small samples or short
sequences of information. Is a financial analyst with four good stock picks talented? The ‘hot hand” phenomenon. Will housing prices always go up?
– For events drawn from clearly “random” processes, people will believe that long streaks will need “balancing” if the underlying system is not assumed to have changed. “Gambler’s Fallacy”.
10 9/29/2015 John W Payne BA925
Example of Confidence in Small Samples
•Your firm has two plants, one large and one small, which mass produce a standard computer chip. Other than the amount they produce, the two plants are identical in all essential regards. Both use the same technology to produce the same product. When properly functioning, this particular technology produces one percent (1%) defective items.
•Whenever the number of defective items from one day’s production exceeds two percent (2%), a special note is made in the quality control log to “flag” the problem. At the end of the quarter, which plant would you expect to have more “flagged” days in its quality control log? Please mark one.
•
•A) The small plant
•B) The large plant
•C) The same number on average
11 9/29/2015 John W Payne BA925
Property of System 1: A bias of confidence over doubt
9/29/2015 John W Payne BA925 12
In a telephone poll of 300 seniors, 60% support the president.
“If you had to summarize the message of this sentence in exactly three words, what
would they be? Almost certainly you would choose “elderly support president.” These
words provide the gist of the story. The omitted details of the poll, that it was done on
the phone with a sample of 300, are of no interest in themselves; they provide
background information that attracts little attention.
Your summary would be the same if the sample size had been different. Of course, a
completely absurd number would draw your attention (“ a telephone poll of 6 [or 60
million] elderly voters…”). Unless you are a professional, however, you may not
react very differently to a sample of 150 and to a sample of 3,000. That is the
meaning of the statement that “people are not adequately sensitive to sample size.”
Kahneman, Daniel (2011-10-25). Thinking, Fast and Slow (pp. 113-114). Macmillan.
Kindle Edition.
Roulette Anyone?
9/29/2015 John W Payne BA925 13
Roulette is a casino game where a wheel and a ball are spun in opposite directions and eventually, the ball falls into a space. Bets can be placed on whether the ball will fall into a red, black, or green space.
There are 18 red spaces,
18 black, and 1 green
Question:
9/29/2015 John W Payne BA925 14
You’re in Vegas and walk up to two roulette tables with the 5 previous spins posted as:
Which table (1 or 2) would you feel more comfortable placing a bet on “black”?
Table 1 Table 2
2 9
27 36
31 7
26 5
12 9
Representativeness Heuristic - 1
• The Gambler’s Fallacy says:
– 5 “reds” in a row in a fair game…black is due!
– Therefore bet on Table2.
– “We were due for one,” Cook said. “We missed our first nine. It's just the law of averages.” http://www.newsobserver.com/2015/01/31/4521195/decock.html#storylink=cpy
• Representativeness Heuristic
– People also think that the recent past is indicative of the future. Table 2 only produces red.
– Therefore bet on Table 1.
• “Good” probability assessment? It doesn’t matter.
9/29/2015 John W Payne BA925 15
Another example of Representativeness: What Will Google’s Price Be In The Future?
9/29/2015 John W Payne BA925 16
$0
$200
$400
$600
$800
Dec-04 Jun-05 Dec-05 Jun-06 Dec-06 Jun-07 Dec-07
Google Stock Price Dec-04 to Dec-07
Excessive Extrapolation: 3 years of increases…it’s bound to
increase again! Graph from Benartzi 2012
The Hot Hand
17
There is a belief in basketball that a player is more likely to hit than miss
following a streak of successful hits. “The hot hand” belief.
Do you agree?
9/29/2015 John W Payne BA925
The illusion of “hot hands” in Basketball
• Gilovich, Vallone, and Tversky (1985) analyzed the shooting of a professional (NBA) team over the course of a season and found that the probability of success on any given shot was essentially independent of the outcomes of previous shots.
• There seems to be no hand hot in reality!
• Much debate about this result.
• The search for the “clutch” player.
• Also, does it impact how players and coaches conduct the game?
• Attali (2013) used an entire NBA season’s worth of data to explore that latter question.
18 9/29/2015 John W Payne BA925
Probability of players’ taking the next team shot following an initial shot as a
function of that shot’s success (hit or miss) and distance from the basket.
19
Attali Y Psychological Science 2013;24:1151-1156
Copyright © by Association for Psychological Science
Effect was even stronger when the outcomes
of the prior two shots were considered.
9/29/2015 John W Payne BA925
Probability that the last shooting player was substituted as a function of the shot’s
success (hit or miss) and distance from the basket.
20
Attali Y Psychological Science 2013;24:1151-1156
Copyright © by Association for Psychological Science
9/29/2015 John W Payne BA925
More likely to be substituted for
if you shot and missed.
Regression Toward the Mean
• Extreme scores or performances measured at any one point in time will probably be less extreme the next time they are measured.
• But this seems counter intuitive due to the representativeness heuristic.
– Why should similarity (fit) vary across time?
9/29/2015 John W Payne BA925 21
Question:
9/29/2015 John W Payne BA925 22
What would you predict these 3 students’ college GPAs to be based on their high school GPAs?
High School GPA
3.80
3.00
2.20
Predicted College GPA
3.46
2.77
2.03
Actual College GPA
3.30
2.93
2.70
Student
A
B
C
Graph from Benartzi 2012
Conclusion: People do not regress enough to the mean.
A form of overconfidence?
Summary “Sins” of Representativeness and Attribute Substitution (System 1 thinking)
• First, it is important to realize that the intuitive impressions that representativeness produces are often more accurate than chance guesses would be. Thus, the use of this heuristic is NOT completely unreasonable. It may reflect a tradeoff between accuracy and effort (ease of use).
• Second, the use of representativeness leads to predictable traps that can, at least, be mitigated such as the neglect of base rate information, overweighting or small samples, and failure to appreciate regression to the mean.
9/29/2015 John W Payne BA925 23
Simple Keys (tricks) to Bayesian Reasoning: You don’t have to be a statistician- Kahneman 2011.
• Anchor you judgment of probability of an outcome on a plausible base rate. – Remember that base rates matter, even in the presence of vivid
evidence about the case at hand unless the evidence is 100% predictive.
• Always question the diagnosticity of your evidence. – Remember that intuitive impressions of the diagnosticity of
evidence are often exaggerated.
• For example, people give to much importance to the perceived strength of evidence, e.g., a good interview impression, and not enough importance on the weight of the evidence, e.g., the size of a sample or the reliability of a interviews.
Question:
9/29/2015 John W Payne BA925 25
Which is the more likely cause of death?
Falling Plane Part Shark Attack
Death by falling plane parts is 30x MORE likely Source: http://www.learner.org/discoveringpsychology/11/e11expand.html
Graph taken from Benartzi 2012,
UCLA
Availability Heuristic
9/29/2015 John W Payne BA925 26
Readily available (easy to retrieve) events in memory affect the judgment of frequency.
Adapted from Benartzi 2012
This is a heuristic that seems impossible to ignore
Availability Heuristic and Biases: Tversky & Kahneman (1974)
• What is the Availability heuristic?
– Process vs. content of thought
– A process for generating a probability?
• Relationship to the frequency of events?
– Events with higher frequency of occurrence are more likely to be recalled. Thus, again, the heuristic is NOT unreasonable.
– However, what else might impact recall likelihood?
• Examples of biases
– Vividness effects and recent occurrences
– Fault-trees and other presentation effects – WYSIATI. 27 9/29/2015 John W Payne BA925
Another Example of Availability in Memory
28
Company
AMR
Cisco Systems*
Coca Cola*
Ingram Micro
McDonald’s*
Oracle*
Owens Corning*
Southwest Airlines*
Supervalu
TechData
United Technologies
Visteon
Median Estimate
18.0
20.0
24.5
10.0
22.0
18.4
15.0
13.5
9.0
9.6
18.0
10.4
Actual
19.0
22.3
20.1
25.2
14.9
10.9
4.8
5.6
23.2
17.2
27.9
17.8
Estimate the sales revenues of the following Fortune 500 firms for the year 2001. The median value on this list is $18.4 billion and the extremes values are $4.8 billion and $27.9 billion.
9/29/2015 John W Payne BA925
Availability – Reasoning by Recall (cont.)
High Name-Recognition
Low Name-Recognition
9/29/2015 29 John W Payne BA925
•I am afraid of spiders.
“73%” “40%”
What’s your best guess of the percentage of your
peers who are afraid?
21% 79%
YES NO
Availability and False Consensus: You know what
YOU believe!
9/29/2015 30 John W Payne BA925
Home Investment Percentages From Around The World
9/29/2015 John W Payne BA925 31
94% 82% 98%
Adapted from Benartzi 2012
Cognitive Ease and Probabilities
• Who perceives a higher personal risk of heart disease? – Man A who is asked to think of three risk-inducing behaviors he
engages in, or – Man B who is asked to think of eight risk-inducing behaviors he
engages in?
• One might assume that Man B who thinks of more risky behaviors would perceive a higher risk, however, it is the opposite.
• Why? The explanation (Schwarz, 2005) is that bringing to mind three risky behaviors is mentally easier than dredging up eights, so the individual thinks, in the former case, “Well, I must be at risk.”
• Implications for assessing risk?
9/29/2015 John W Payne BA925 32
9/29/2015 John W Payne BA925 33
A nice picture from the Visit Michigan website
Failures of System 1 together with System 2
9/29/2015 John W Payne BA925 34
• How many murders occur in the state of Michigan in one year?
• How many murder occur in the city of Detroit in one year?
• Kahneman, Daniel (2011-10-25). Thinking, Fast and Slow (p. 45). Macmillan. Kindle Edition.
9/29/2015 John W Payne BA925 35
• Facts that we know do not always come to mind when we need them. (Kahneman, 2011, p. 46).
9/29/2015 John W Payne BA925 36
“Memory Bias”
Too much weight given to recent history Kahneman and Tversky (1973)
“This time is different”
Carmen Reinhart and Ken Rogoff book
•
A Third Heuristic: Anchoring and Adjustment
• Estimation is a process of anchoring on a salient number and adjusting up or down.
• Very reasonable procedure but the anchor can be very, very heavy.
9/29/2015 John W Payne BA925 37
The Anchoring and Adjustment Heuristic and Biases: Tversky & Kahneman (1974)
• Examples of related Biases – Irrelevant anchors such as your SS number.
– Conjunctive versus disjunctive judgments
– Over tight confidence intervals.
• Applies to value (price) judgments as well as probability judgments.
• What determines the anchor value?
38 9/29/2015 John W Payne BA925
8 x 7 x 6 x 5 x 4 x 3 x 2 x 1
1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 median guess = 512
median guess = 2250
correct answer = 40,320
Anchoring by Task Order
9/29/2015 39 John W Payne BA925
if asked $0.80: $0.94
if asked $2.00: $1.47
What is your best estimate of the
exchange rate two years from now?
Do you believe that two years from now the
US$/Euro exchange rate will be above or
below $0.80 per €? ($2.00 per €)?
Anchoring: Exchange Rates
9/29/2015 40 John W Payne BA925
Anchoring as accessibility and adjustment
• A variety of procedures are used in anchoring studies – Anchors provided by an external source – usually involves a “comparative judgment” –
“was Aristotle born before or after AD 1825?” followed by an absolute estimate “In what year was Aristotle born?”
– Anchors generated by the participants themselves – “When was George Washington elected President of the United States? (Initial answer might be 1776, which is then adjusted. Initial anchor on the strength or extremity of evidence and then adjust for its validity (Griffin & Tversky, 1992).
– Anchors generated randomly.
• Grice (1975) conversational norm – a person might expect a stated anchor to matter. However, note, clearly irrelevant anchors do matter, e.g. your SS#.
• Multiple mechanisms underlying anchoring as an effect – Accessibility of anchor-consistent information – Insufficient adjustment – search for the first satisfactory answer
• Not all biases result from a single process.
• Why a bias occurs has implications for debiasing.
41 9/29/2015 John W Payne BA925
Tversky & Kahneman (1983): Extensional versus intuitive reasoning – 1*
• Basic idea: Judgments of probability vary in the degree to which they follow an extensional (decompositional, frequentistic) logic or a more singular, holistic, intuitive logic.
• Note, the idea of two different processes – System 1 vs. System 2?
• The probability calculus as a standard against which to compare intuitive
probability judgments, e.g., Bayes Theorem or the conjunction rule: If (A B) then P(A) P(B). Note, a coherence principle. Compare later to a correspondence standard used in Lens Model studies.
• Heuristics based on “natural” assessment processes like similarity and memory retrieval.
42 9/29/2015 John W Payne BA925
Tversky & Kahneman (1983): Extensional versus intuitive reasoning - 2
• Which reasoning process will be used to assess probabilities will depend upon -
– the nature of the evidence, e.g., how information is presented, e.g. probabilities, e.g. .10 versus frequencies 1 in 10 or 10 in a 100. Note, sometimes people will prefer to bet on an event that occurs 9 times out of 100 rather than an event that occurs 1 time out of 10. More generally, the formulation of the question matters.
– the transparency of the event structure, see Linda results
– appeal of the heuristic, including prior use
– sophistication of the respondent, see Linda results
– But do you sometimes need both sophistication and transparency?
• The use of intuitive processes, however, will be common, and therefore biases will be frequent and sizable.
43 9/29/2015 John W Payne BA925
“Classic’ Example of Type 1 Thinking and An Error in Judgment: The Famous Linda Problem
9/29/2015 John W. Payne 44
• Linda is 31 years old, single, outspoken and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.
• Below are eight statements about Linda. Rank all eight statements in terms of their probability of being true. Use 1 for the most probable and 8 for the least probable.
• A) Linda is a teacher in elementary school. • B) Linda works in a bookstore and takes Yoga classes. • C) Linda is active in the feminist movement. • D) Linda is a psychiatric social worker. • E) Linda is a member of the League of Women Voters. • F) Linda is a bank teller. • G) Linda is an insurance salesperson. • H) Linda is a bank teller and is active in the feminist movement.
Impact of Task and Statistical Sophistication on the Linda Problem Solving: Sometimes both task changes and knowledge
are needed!
9/29/2015 John W. Payne 45
• First, Linda is a bank teller
• Next, make the relationship less transparent, e.g., Linda is an insurance salesperson- such filler statements serve to make the relationship less transparent.
• Third, Linda is a bank teller and is active in the feminist movement.
% of
violations
8 items 2 items
Transparent
Naïve
undergrad 89 85
Sophisticat
ed Ph.D.
students
85 36
Tversky & Kahneman (1983): Extensional versus intuitive reasoning - 3
• Betting on a string of events
– RGRRR vs. GRGRRR (selected) vs. GRRRRR • That is adding an additional uncertain event G to RGRRR makes
that possibility more likely?
– Impact of real payoffs? • Answer – not much.
• Implications for Scenario Thinking - good stories can increase the perceived likelihood of an event by adding extra features. – Politicians know this well.
46 9/29/2015 John W Payne BA925
Tversky & Kahneman (1983): Extensional versus intuitive reasoning - 5
• Conclusion
– When will a logical principle be applied even when understood?
– Coherence versus correspondence views of good judgment?
– The role of “cognitive illusions” in judgment research?
47 9/29/2015 John W Payne BA925
Heuristic and Biases Reviewed
• Heuristics are highly economical and usually effective methods for dealing with judgments under certainty that are based on “natural” psychological assessments , but they can lead to systematic and predictable errors.
• Heuristics can be thought of as (intelligent and adaptive) strategies for searching through a problem space which simplify the process by disregarding some elements of the problem space.
• Judgments of probability vary in the degree to which they follow an extensional (decompositional, frequentistic, distributional) logic or a more singular, holistic, intuitive logic. Both types of reasoning occur when judging probabilities.
• Which reasoning process will be used to assess probabilities will depend upon the nature of the evidence, the formulation of the question, the transparency of the event structure, appeal of the heuristic, prior use, goals, and the sophistication of the respondent. However, we do not have a good theory of when certain heuristics will be more or less likely to be used.
48 9/29/2015 John W Payne BA925
Summary Flowcharts for the Three Heuristics (Adapted from Hastie & Dawes 2001)
49
Question?
Probe
Memory
Retrieve item(s)
that match or are
associated with probe
Assess ease
of recall
Respond
Availability Heuristic
Question?
Case or instance
description (feature
list).
Similarity Match:
How well does the
case feature list match
the category list?
Retrieve Category
representation (feature
list)
Estimate certainty of
category membership
based on similarity
Respond
Representativeness Heuristic
Question?
Search memory or
environment for evidence
Select most important
(remaining) evidence
Extract
Information
Is this first
item? Anchor on
information Adjust
Estimate
Yes No
Is there more
evidence?
Respond
No
Yes
Anchoring
9/29/2015 John W Payne BA925
Criticisms of the “heuristics and biases” viewpoint include:
• Over emphasis on error – – “If we are so dumb how did we reach the moon?” – Effort (Speed) tradeoffs?
• Lack of specification of the heuristics
• More needed on when and who
• Need for new methods such as response times and
task manipulations such as cognitive load that go beyond answers to simple word questions.
50 9/29/2015 John W Payne BA925
A newer area of study:
Emotions and Probabilistic Thought Do Interact
9/29/2015 John W Payne BA925 51
The Affect Heuristic
9/29/2015 John W Payne BA925 52
An emotional evaluation that occurs before conscious reasoning
“The stock market is more likely to go up on sunny than on cloudy days…”
Source: S. Benartzi UCLA
9/29/2015 John W Payne BA925 53
“He likes the project, so he thinks its costs are low
and its benefits are high. Nice example of the
affect heuristic.”
Kahneman, Daniel (2011-10-25). Thinking, Fast and Slow
(p. 104). Macmillan. Kindle Edition.
Emotions and the Use of Probabilities
• Cass Sunstein has argued “that when intense emotions are engaged, people tend to focus on the adverse outcome, not on its likelihood.” That is, they are not closely attuned to the probability that harm will occur. They exhibit “probably neglect.”
• Obvious implications for public policy, what about business decisions?
9/29/2015 John W Payne BA925 54
Questions?
55 9/29/2015 John W Payne BA925