Cognitive Biases in Testing

Srikanth S KrishnanSr. Director QAOracle India

What is a Cognitive Bias ?

It is the human tendency to make systematic errors in certain circumstances based on cognitive factors rather than evidence.

Heuristics

Statistical mis-estimates

Memory errors

Social Constructs

Tester’s Delusion Spots

Test Planning

Test Design

Test Advocacy

Anchoring

Guess the percentage of African Nations in the

UN.

Anchoring

Will you be happier if you get a 100% raise?

Bidding after stating the last 2 digits of social security number

Why are black pearls more expensive?

occurs when people consider a particular value for an unknown quantity before estimating that quantity

Effect on Estimation

New Feature vs. regression testing of existing features

Effect on Negotiation

Relativity

Would you mind walking 5 min to save $10 on a $ 50

pen?

Would you mind walking 5 min to save $10 on a $ 1000 suit?

Decoy Effect or Asymmetric dominance effect

Option A Option B

Coverage 400 300

Cost 3800 2000

Option A

Option B

Option C

Coverage

400 300 350

Cost 3800 2000 4500

People not only compare things, but also compare things that are easily comparable.

The Price of Ownership

Endowment Effect

How much would you pay for a Coffee Mug ?

Applies to ideas as well as objects.

The more effort you put into something the more ownership you feel. IKEA Effect

Ex. Manager from Finance

Test repository clean up

Which one will you get rid of?

(assume similar coverage)

Suite1 Created with 100 man days effort

Suite2 Created with 30 man days effort

Effect of Expectation

Confirmation bias: Self-Fulfilling Prophecy

How do you get more women in the orchestra?

Positive vs. negative Test cases.

Seeing patterns where none exist.

exist.

exist.

20

12

4

31

Influence of great UI.

Negativity Bias

More attention is given to negative than positive experiences

The lingering effect of a nasty bug.

Sunk Costs

Concorde Fallacy

“I have spent so much time doing the setups.”

Concorde Fallacy

Why test a build that has been obsoleted?

Paradox of Choice

Concorde Fallacy

More is not always better

Burn your bridges

Concorde Fallacy

Why don’t we keep this server just in case?

Procrastination

Concorde Fallacy

Choose your own deadline

Concorde Fallacy

The hockey-stick effort curve

Concorde Fallacy

Milestones and distributed deadlines

Resurrection to Rationality

overcoming

channeling

References and Acknowledgements

Predictably Irrational – Dan ArielyNudge – Thaler & SunsteinBlink – Malcolm GladwellSway – Brafman (Ori & Rom)Wikipedia

Concorde Fallacy

Thank You

Twitter: sriks7

sriks7@gmail.com