• This demo is adapted from a teaching exercise developed by K. H. Grobman that is available at http://www.devpsy.org/teaching/method/confirmation_bias.html

• Start show with 3rd slide

Sequence Instructor’s judgment that my sequence is correct

My guess is that the instructor’s rule is …

How sure am I that my rule is correct?

2, 4, 6 %

__, __, ___ %

__, __, ___ %

__, __, ___ %

__, __, ___ %

__, __, ___ %

__, __, ___ %

• I have a rule in mind that makes sequences of 3 numbers. In a sequence, order matters. Your goal is to figure out the rule in my head! But you can’t simply ask me. Instead, to determine my rule, write a sequence of 3 numbers in the sequence column.

• I will then walk around the room putting either a “check” (meaning the sequence fits my rule) or a “0”(meaning the sequence does NOT fit my rule.

• After I have given you your check (or 0), guess my rule. Write the rule out as a sentence or a phrase.

• Then, estimate how sure you are of your guess by writing a percent (100% for completely sure and 0% for completely unsure) in the “How sure” column.

• To get you started, imagine that your first guess was 2, 4, 6. That sequence fits my rule, so I put a check on your paper. So, 2, 4, 6 fits my rule. Now, you should guess my rule and write down your guess as a sentence or phrase in the “Guess the instructor’s rule” column. Then, estimate how sure you are that your guess really is the rule in my mind.

• Now, let’s get started on the second row. Guess a sequence and wait until I stop by. Once I put a check or a zero, you can complete the rest of the row and make another guess.

• Once you reach 100% as how sure you are, please turn over your paper so I can move quickly past you while we wait for everyone else to be as sure as you are.

What was my rule?

My rule was

• “the numbers increase”

Different patterns of guessing

• In the next few slides, I’ll present different patterns of guessing. While I present these patterns, see

1. Which of these patterns is most similar to how you approached the problem.

2. Which pattern is best.

The danger of playing it safe: Not making wrong guesses doesn’t make you right. Sequence Fits

ruleRule How sure?

2, 4, 6 √ Count up by 2’s 50%

6, 8, 10 √ Count up by 2’s 60%

12, 14, 16 √ Count up by 2’s 70%

18, 20, 22 √ Count up by 2’s 80%

24, 26, 28 √ Count up by 2’s 90%

30, 32, 34 √ Count up by 2’s 100%

Finding more and more evidence that the earth is flat makes you more sure that you are right—but not more right.Sequence Fits

ruleRule How sure?

2, 4, 6 √ Count up by 2’s 50%

6, 8, 10 √ Count up by 2’s 60%

20, 22, 24 √ Count up by 2’s 70%

3, 5, 7 √ Count up by 2’s 80%

25, 27, 29 √ Count up by 2’s 90%

200, 202, 204

√ Count up by 2’s 100%

A Scientific Pattern of Guessing

Sequence Fits rule

Rule How sure?

2, 4, 6 √ Count up by 2’s 50%

10, 20, 30 √ Count up by multiples

60%

Asking “Could another rule also fit” leads to an insight

Sequence Fits rule

Rule How sure?

2, 4, 6 √ Count up by 2’s 50%

10, 20, 30 √ Count up by multiples

60%

100, 500, 894

√ Count up with all even numbers

70%

Asking “Could I be wrong?” leads to more insights

Sequence Fits rule

Rule How sure?

2, 4, 6 √ Count up by 2’s 50%

10, 20, 30 √ Count up by multiples

60%

100, 500, 894

√ Count up with all even numbers

70%

1, 9, 20 √ Count up 80%

Sequence Fits rule

Rule How sure?

2, 4, 6 √ Count up by 2’s 50%

10, 20, 30 √ Count up by multiples

60%

100, 500, 894

√ Count up with all even numbers

70%

27, 13, 4 0 Count up 80%

1,9, 20 √ Count up 90%

We learn as much—if not more– from predictions

that were wrong than from ones that were right.

Sequence Fits rule

Rule How sure?

2, 4, 6 √ Count up by 2’s 50%

10, 20, 30 √ Count up by multiples

60%

100, 500, 894

√ Count up with all even numbers

70%

1, 9, 20 √ Count up 80%

27, 13, 4 0 Count up 90%

55, 2, 999 0 Count up 99%

In science, we are almost never 100% sure.

Concluding Thoughts I

• Being sure you are right is different from being right.

• “Smart” (high IQ) people are just as likely to make the confirmation bias as anyone else, but we would expect wise people to make it less.

Concluding Thoughts II

• Can you think of real life situations in which people would be better off asking “Am I right or wrong?” rather than just asking “Am I right?” – Some possibilities …. (see next slides)

Need for more scientific testing

– People whose first impressions are “always right.”

– People who believe their horoscopes.– Prejudiced people.– People who have strong political beliefs and

watch a lot of news (but all from the same source).

– You—the last time you were in an argument (and you were “completely right”)?

With this evidence, do we know that the treatment worked?

• 200 people who got the treatment improved.

Adapted from pages 91-92 of Stanovich, K.E. (2007). How to think straight about psychology (8th ed.). New York: Allyn and Bacon.

With this evidence, do we know that the treatment worked?

• 200 people who got the treatment improved.• No, for example, we do not know how

many who got the treatment did not improve.

Adapted from pages 91-92 of Stanovich, K.E. (2007). How to think straight about psychology (8th ed.). New York: Allyn and Bacon.

With this evidience, do we know the treatment worked?

• 200 people who got the treatment improved.• 75 people who got the treatment did not

improve.

Adapted from pages 91-92 of Stanovich, K.E. (2007). How to think straight about psychology (8th ed.). New York: Allyn and Bacon.

With this evidence, do we know the treatment worked?

• 200 people who got the treatment improved.• 75 people who got the treatment did not

improve.• No, for example, we don’t know how many

people improved without the treatment.

With this evidence,do we know the treatment worked?

• 200 people who got the treatment improved.• 75 people who got the treatment did not

improve.• 100 people who did not get the treatment

improved.• 30 people who did not get the treatment did

not improve.

With this evidence, do we know the treatment worked?

• 200 people who got the treatment improved.• 75 people who got the treatment did not improve.• 100 people who did not get the treatment improved.• 30 people who did not get the treatment did not

improve. • No, there is no evidence that the treatment

worked*.

– Adapted from pages 91-92 of Stanovich, K.E. (2007). How to think straight about psychology (8th ed.). New York: Allyn and Bacon.

With this evidence, do we know the treatment worked?

• 200 people who got the treatment improved.• 75 people who got the treatment did not improve.• 100 people who did not get the treatment improved.• 30 people who did not get the treatment did not improve. • No, there is no evidence that the treatment worked. To see

why, consider what probably would have happened if we had had doubled the number of people in the no-treatment group.

• Treatment Group No treatment group• 200 improved 200 improved• 75 did not improve 60 did not improve

– Adapted from pages 91-92 of Stanovich, K.E. (2007). How to think straight about psychology (8th ed.). New York: Allyn and Bacon.