Bounded Rationality: Thinking Is Costly
A baseball and bat together cost $11. The bat costs $10 more than the ball. How much does the ball cost?
Write down your answer.
Half of Harvard students said $1, which is the intuitive answer but wrong!
Correct answer is 50 cents: $10.50-$.50 = $10.00
People tend to use “intuitive thinking” or rules of thumb
CONCURRENT SCHEDULES OF REINFORCEMENTStudy of choice centers on
how operant behavior is affected its reinforcement history and by reinforcement history of other operant behaviors.
Research uses concurrent schedules: two or more schedules that operate simultaneously and independently, each for a different response.
Concurrent schedules
of reinforcement
Two schedules are in effect at the same time and the subject is free to switch from one response alternative to the other
Key A Key B
Schedule AVI 60 s
Schedule BFR 10
Choice Behaviorand the
Matching Law
Relative rate of responding on a particular lever equals the relative rate of reinforcement on that lever
The Matching Law is a mathematical statement describingthe relationship between the rate of responding and the rate of reward
developed by Herrstein
Matching Law
RA/(RA + RB) = rA/(rA + rB) orRA /rA = RB /rB
RA = Responses to ARB = Responses to BrA = Reinforcers to ArB = Reinforcers to B
Matching Law
Provides accurate description of behavior of many organisms in many different choice situations.Different speciesAppetitive and aversive stimuli Frequency, magnitude, and delay of
reinforcement
People Aren’t Always RationalStudies find that people make systematic mistakes: People are overconfident.
People give too much weight to a small number of vivid observations.
People are reluctant to change their minds.
Even though people are not always rational, the assumption that they are is usually a good approximation for economic modeling.
Deviations from Matching
Impatience: the desire for instant gratificationRead, Loewenstein & Kalyanaraman (1999)
Choose among 24 movie videosSome are “low brow”: My Cousin VinnySome are “high brow”: Schindler’s List
Picking for tonight: 56% of subjects choose low brow. Picking for next Thursday: 37% choose low brow. Picking for second Thursday: 29% choose low brow.
Tonight I want sugar-coated entertainment… next week I want things that are good for me.
Choice with Commitment
In a standard concurrent schedule of reinforcement, two (or more) response alternatives are available at the same time and the subject is free to switch from one to the other at any time
However, in some (real-life) situations, choosing one alternative makes other alternatives unavailable
In these cases, the choice may involve assessing complex,long-range goals
Can study these types of situations in the lab using aConcurrent-chain schedule of reinforcment
Terminal link
BA
Time
Choice link
Reinforcementschedule A
(VR 10)
Reinforcementschedule B
(FR 10)
Pecking the left key in the choice link puts into effect reinforcement schedule A in the terminal link. Pecking the right key in the choice link puts into effect reinforcement schedule B in the terminal link.
Concurrent-chain schedule
BA
Time
Smallreward
Largereward
Delay
A B
Smallreward
Largereward
DelayDirect-choice procedurePigeon chooses immediate,small reward
Concurrent-chain procedurePigeon chooses the schedulewith the delayed, larger reward
Self-Control
Concurrent chain schedules have been used to study ‘self-control’ in the lab
e.g., choosing a large delayed reward over animmediate small reward
With direct choice procedures, animals often lackself-control. That is, they choose the immediate, butsmaller reward
With concurrent-chain procedures, animals do showself-control. That is, they choose the larger, but delayedreward
Choosing fruit vs. chocolate
TimeChoosing Today Eating Next Week
If you were deciding today,would you choosefruit or chocolatefor next week?
Patient choices for the future:
TimeChoosing Today Eating Next Week
Today, subjectstypically choosefruit for next week.
74%choosefruit
Impatient choices for today:
Time
Choosing and Eating
Simultaneously
If you were deciding today,would you choosefruit or chocolatefor today?
Time Inconsistent Preferences:
Time
Choosing and Eating
Simultaneously
70%choose chocolate
Rachlin and Green (1972)If you delay the reinforcer for both a larger and smaller reinforcer and then you give organism a choice, it may well choose the larger delayed reinforcer
10 second delay
10 second delay
2 seconds foodimmediate
4 seconds foodMust wait 4seconds
4 seconds foodMust wait 4seconds
FR - 15
FR - 15
Self-Control
Why does anyone choose a smaller reward part of the time? Animals and people typically choose a
small immediate reward over a larger delayed reward.
Large rewards are selected when: The choice is made in advance of
reward. Reinforcers are not visible or reward
is already present (pleasurable activity).
What mechanism may account for this short temporal horizon? Two Brain System Theory
of Addiction Increased activity of the
Impulsive System (motivational)
Extented Amygdala Ventral Striatum
Decreased activity of Executive System (rationality)
Ventromedial Prefrontal Cortex (VMPC)
Dorsolaterial Prefrontal Cortex (DLPC)
Self-Control
Temporal Issue Lack of self-control arises from the fact that our behavior
is more heavily influenced by immediate consequences as opposed to delayed consequences.
Immediate Consequence Delayed Consequence
quitting withdrawal Improved health
smoking Nicotine high Deterioration of health
Self-control – preference for larger later reward
Impulsiveness – preference for smaller sooner reward
Which do you prefer
$500 now or $1,000 in two years$500 in four years or $2,000 in six years
People Are Inconsistent Over TimePeople tend to prefer instant gratification,
even when delaying would increase the gratification.
Result: People fail to follow through on plans to do things that are dreary, take effort, or cause discomfort. e.g., people often save less than they plan
To help follow through, people look for ways to commit themselves to their plans. e.g., worker has money taken out of paycheck
before he ever sees it
Choice and Foraging
Laboratory paradigms are often criticized because they do not capture essence of natural contingencies.
More recent research has tried to bring natural contingencies of reinforcement into laboratory.
This move has been prompted by optimal foraging theory.
Choice and Foraging
Optimal foraging theory: feeding behavior is sensitive to relation between amount of energy expended in finding, securing, and consuming food, and amount of energy or nutrition of food.
Patterns of foraging optimize relation between energy gain and energy expenditure.
Optimal foraging theory: feeding behavior is sensitive to relation between amount of energy expended in finding, securing, and consuming food, and amount of energy or nutrition of food. Patterns of foraging optimize relation between energy gain and energy expenditure
Behavioral Economics
Behavioral economics: a new field in which economists apply basic insights from psychology
People aren’t always as rational as traditional economic models assume.
Herbert Simon viewed humans as satisficers,people who make choices that are merely “good enough” rather than optimal.
Other economists have suggested that people are only “near rational” or exhibit “bounded rationality.”
OPERANT BEHAVIOR AND ECONOMICSIt is not surprising that behavior theorists
have applied economic concepts to their own domain.
Matching Law is viewed by some theorists as special case of general economic principles.
OPERANT BEHAVIOR AND ECONOMICS: BASICSDemand (amount purchased at given
price): Elastic (luxuries) Inelastic (necessities)
Commodities: Substitutes (more of one, less of other) Complements (more of one, more of other)
Income (money or responses required to purchase commodities at given price)
OPERANT BEHAVIOR AND ECONOMICS: BASICSDemand (amount purchased at given
price): Elastic (luxuries) Inelastic (necessities)
Commodities: Substitutes (more of one, less of other) Complements (more of one, more of other)
Income (money or responses required to purchase commodities at given price)
Rodents = Bar press
OPERANT BEHAVIOR AND ECONOMICS: BASICSEconomy (relationship between
commodities and income: Closed (fixed income) Open (extra income): most operant studies
Deprivation
People Care About Fairness
People’s choices are sometimes influenced more by their sense of fairness than self-interest.
Example: the ultimatum game
The rules Two players who do not know each
other have a chance to share a prize of $100.
Player A decides what portion of the prize to give to player B.
B must accept the split or both get nothing.
People Care About FairnessPredicted outcome if both players rational A would propose a 99-1 split and B would accept,
because $1 is better than nothing.
Actual outcome from experiments with real people B usually rejects lopsided splits like 99-1
as wildly unfair. Expecting this, A usually proposes giving
$30 or $40 to B. B views this as unfair, but not so much as to
abandon his self-interest, so B accepts.
People Care About Fairness
The results of the ultimatum game apply in other situations.
Example: A firm may pay above-equilibrium wages during profitable years to be fair, or to avoid appearing unfair and risking retaliation from workers.
OPERANT BEHAVIOR AND ECONOMICS Evidence suggests behavior of animals in
conditioning experiments conforms to what economic theory says people will do when confronted with similar choices.
Implies that operant behavior involves a kind of economic decision making.
Animals must decide how to allocate scarce behavioral resources; rules by which they do so are like those people use.
Economic theory and behavior theory may jointly explain human and animal choice.