8/9/2019 Literature on Heuristics Ahmednagar
1/24
A Sample of the Literature
on Heuristics* and Biases
8/9/2019 Literature on Heuristics Ahmednagar
2/24
What is this lecture about?
This lecture is a Sample of theLiterature on Heuristics* andBiases(Shortcuts in Thinking* andBiases Related To Them!the law ofsmall numbers (and its relatedheuristic: representativeness)and errors in statistical thinking
related to it dem"sti#ed$What is all this? % &ill present this
slide again in the end and ask "ou toe' lain it$
8/9/2019 Literature on Heuristics Ahmednagar
3/24
The la& of small numbersdem"sti#ed
A stud" of the incidence of kidne" cancerin ounties of )nited States$
idne" ancer is lo&est in counties &hichare mostl" rural and sparsel" populated
and located in traditionall" RepublicanStates$
+ou probabl" ignored the Republican ,art$
-id "ou focus on the Rural ,art?
-id "ou think that rural lifest"le leads tolo& kidne" cancer? ,robabl"$
8/9/2019 Literature on Heuristics Ahmednagar
4/24
.o& comes the surprise/
The counties in the )nited States &hichha0e the highest incidence of kidne"cancer also tend to be in mostl" rural1sparsel" populated counties and
Republican States/2a"be the po0ert" of the rural lifest"le
caused this/
Something is &rong here/ RuralLifest"le cannot e'plain both high andlo& incidence of kidne" cancer/
8/9/2019 Literature on Heuristics Ahmednagar
5/24
What is the solution to this riddle?!lets begin to think$
Well1 &hat could be the solution to thisriddle?1 Lets atleast begin the processof e'ploration$
%nfact1 the ke" factor is not that the
counties &ere rural or predominantl"Republican1 it is that rural countiesha0e small populations$
2ore about this e'planation later*1 #rstlets in0estigate certain features of thisconondrum$
8/9/2019 Literature on Heuristics Ahmednagar
6/24
The ,roblem is %n fact3 4rrors inStatistical Reasoning!
5rom &hate0er % told "ou abo0e "oumust ha0e thought 6 2" 7od/ Here1&e ha0e !another 8a0erage9 lectureon some academic topic1 but thelecture % am about to gi0e "ou is notacademic at all1 infact it is a lecture&hose basis is on the di:cult
relationship bet&een our mind andstatistics$
8/9/2019 Literature on Heuristics Ahmednagar
7/24
What &as the shortcut that "our mind gottrapped in at the beginning of this stor"?
We automaticall" and e;ortlessl" identif"causal connections bet&een e0ents1sometimes e0en &hen the connection is
spurious$What &as the shortcut that "our mind got
into? When told about the high incidencecounties1 "ou immediatel" assume that
these counties are di;erent from othercounties for a reason1 that there must bea cause that e'plains this di;erence$
8/9/2019 Literature on Heuristics Ahmednagar
8/24
6ur 2ind and Statistics$
6ur mind1 speciall" the intuiti0e partof our mind is inept &hen faced &ithmerel" statistical facts1 whichchange the probability of outcomesbut do not cause them to happen.How can we justify the above? Herewe go-
As % had promised earlier there is ane'planation to this stor" of idne"ancer$ What is it?
8/9/2019 Literature on Heuristics Ahmednagar
9/24
The 4'planation3
%magine the population of the )nitedStates as marbles in a giant urn$ Somemarbles are marked 1 for kidne"cancer$ +ou dra& samples of marbles
and populate each count" in turn$ RuralSamples are smaller than othersamples$ 4'treme 6utcomes (0er" highand
8/9/2019 Literature on Heuristics Ahmednagar
10/24
7oing -eeper into the 4'planation$
%magine a large urn #lled &ith marbles$Half the marbles are red1 half are &hite$.e't1 dra& = marbles from the urn1 recordthe number of red balls in the sample1
thro& the balls back into the urn1 and thendo it again1 man" times$ %f "ou summari>ethe results1 "ou &ill #nd that the outcome@ red1 @ &hite occurs (almost e'actl"
times as often as the outcome = red or= &hite$ This relationship is amathematical fact$
8/9/2019 Literature on Heuristics Ahmednagar
11/24
A related statistical fact
A related statistical fact is rele0antto the cancer e'ample$ 5rom thesame urn1 ack dra&s = marbles oneach trial1 ill dra&s C$ The" bothrecord each time the" obser0e ahomogeneous sampleDall &hite orall red$ %f the" go on long enough1
ack &ill obser0e such e'tremeoutcomes more often than illDb" afactor of E$
8/9/2019 Literature on Heuristics Ahmednagar
12/24
ust an accident of sampling
The small population of a count" neithercauses nor pre0ents cancerF it merel"allo&s the incidence of cancer to bemuch higher (or much lo&er than in the
larger!population$ The deeper truth isthat there is nothing to e'plain$ Theincidence of cancer is not trul" lo&er orhigher than normal in a count0 &ith a
small population it Gust appears to be soin a particular "ear because of anaccident of sampling$
8/9/2019 Literature on Heuristics Ahmednagar
13/24
La& of large and small numbers
large samples! deser0e more trust than smaller samples$
"ou ma" #nd that the follo&ing statements appl" to "ou3
sparsel" populated did not immediatel" stand out asrele0ant &hen "ou read the kidne" cancer stor"$
+ou &ere at least mildl" surprised b" the si>e of the di;erence bet&een
samples of = and samples of C$
The follo&ing t&o statements mean e'actl" the same thing3
Large samples are more precise than small samples$
Small samples "ield e'treme results more often than large samples do$
The rst statement has a clear ring of truth1 but until the second 0ersionmakes intuiti0e sense1 "ou ha0e not trul" understood the #rst$
The bottom line3 "es1 "ou did kno& that the results of large samples aremore precise1 but "ou ma" no& reali>e that "ou did not kno& it 0er" &ell$
8/9/2019 Literature on Heuristics Ahmednagar
14/24
Sampling ariation
"ou &ish to con#rm the h"pothesis that the0ocabular" of the a0erage si'!"ear!old girlis larger than the 0ocabular" of an a0eragebo" of the same age$ The h"pothesis is truein the populationF 7irls and bo"s 0ar" agreat deal1 ho&e0er1 and b" the luck of thedra& "ou could select a sample in &hichthe di;erence is inconclusi0e1 or e0en onein &hich bo"s actuall" score higher$ )sing asu:cientl" large sample is the onl" &a" toreduce the risk$ Researchers &ho pick toosmall a sample lea0e themsel0es at themerc" of sampling
8/9/2019 Literature on Heuristics Ahmednagar
15/24
ontent 0ersus reliabilit"
8%n a poll of IJJ seniors JK supportthe ,resident9
summari>e in e'actl" three &ords1
"ou &ould choose 8The elderl" support
the ,resident$ These &ords pro0ide thegist of the stor"$ +our summar" &ouldbe the same if the sample si>e hadbeen di;erent$ 6f course1 a completel"absurd number of sample si>e1 small orbig1 &ould dra& "our attention$
8/9/2019 Literature on Heuristics Ahmednagar
16/24
Halo 4;ect
belie0ing that small samples closel"resemble the population from &hich the"are dra&n implies3 &e are prone toe'aggerate the consistenc" and
coherence of &hat &e see1 Thee'aggerated faith of researchers in &hatcan be learned from a fe& obser0ations isclosel" related to the halo e;ect1 the
sense &e often get that &e kno& andunderstand a person about &hom &eactuall" kno& 0er" little$
8/9/2019 Literature on Heuristics Ahmednagar
17/24
There has to be a reason for e0er"thing9
Take the se' of si' babies born in seMuence at a hospital$ TheseMuence of bo"s and girls is ob0iousl" randomF the e0ents
are independent of each other1 and the number of bo"s andgirls &ho &ere born in the hospital in the last fe& hours hasno e;ect &hatsoe0er on the se' of the ne't$ Ho&e0er &e
&ill not consider the fact that if in the seMuence all e0ents
are independent and outcome 8bo"9 and 8 girl9 areappro'imatel" eMuall" likel"1 then an" possible seMuence ofsi' births is as likel" as an" other$ We are pattern seekers1belie0ers in a coherent &orld1 in &hich regularities (such as a
seMuence of si' girls appear as a result of someonesintention$ Lions ma" appear on the plain at random times1
but it &ould be safer to notice and respond to an apparentincrease in the rate of appearance of prides of lions1 e0en if itis actuall" due to the Nuctuations of a random process$
8/9/2019 Literature on Heuristics Ahmednagar
18/24
Search for certaint"1 search forcausalit"! a &aste of time$
War broke out in OPCI$ SMuadrons N"ing from thesame base1 one of &hich had lost four planes&hile the other had lost none$ An inMuir" &asinitiated$ There &as no prior reason to belie0e thatone of the sMuadrons &as more e;ecti0e than the
other1 and no operational di;erences &ere found1but of course the li0es of the pilots di;ered inman" random &a"s1 including1 ho& often the"&ent home bet&een missions$$ Rationall" thecommand should accept that the di; erent
outcomes &ere due to blind luck1 a randomsearch for a non ob0ious cause &as hopeless$
8/9/2019 Literature on Heuristics Ahmednagar
19/24
5inding ,atterns in Sports
The assumed hot hand in sports is0er" usual1 if in basketball a pla"ersinks three or four baskets in a ro&1defense starts guarding him more1
his pla"ers start passing more tohim1 e0en his coach thinks he has atemporar" hot hand1 &e are too
Muick to percei0e order and causalit"in randomness$
8/9/2019 Literature on Heuristics Ahmednagar
20/24
%n a small sample there &ill be more e'treme results$
A research stud" sho&ed that more small
schools had done &ell1 so the 7ates foundationstarted funding small schools and a causal stor"can easil" be linked to this sa"ing that attentionto students is more in small schools1 actuall"larger schools empiricall"1 if an"thing1 do better
possibl" because of greater curriculum options $And so )nfortunatel" the causal anal"sis is&rong1 the actual fact is &hich could ha0e beenpointed out had the 7ates foundation takenstatistics seriousl" is that more small schoolshad also done badl"$ learl" on a0erage smallschools are not better Gust more 0ariable$
8/9/2019 Literature on Heuristics Ahmednagar
21/24
ournal Articles
%n further readingO if "ou read the article % ha0e gi0en "ou
Belief in the la& of small numbers$T0ersk"1 AmosFahneman1 -aniel1 ,s"chological Bulletin1 ol C(@1Aug OPCO1 OJQ!OOJ$
This article gi0es man" e'amples &hich sho& thatthe fact that e'treme outcomes result from smallsamples more often than large samples is astatistical fact and can lead to a bias or statisticalerror in misinterpreting1 &hat is a result of sample
si>e1 as a factor related to the content of the stor"$)nfortunatel" &e focus more on content thanreliabilit"$
8/9/2019 Literature on Heuristics Ahmednagar
22/24
ournal Articles
%n further readingO if "ou read the article % ha0e gi0en "ou
6n the ps"cholog" of prediction$ahneman1
-anielF T0ersk"1 Amos ,s"chological Re0ie&1 ol
EJ(=1 ul OPCI1 @IC!@QO$
+ou &ill #nd e'amples of ho& people see patterns
&here none e'ist$ ,eople Gudge e0en e'tremeand rare outcomes as more probable if the
content of the stor" indicates that a particular
outcome is more representati0e1 e0en if thatoutcome is e'treme or rare$ This is the heuristicof representativeness.
8/9/2019 Literature on Heuristics Ahmednagar
23/24
ournal Articles%n further reading
O if "ou read the article % ha0e gi0en "ou How to MakeCognitive Illusions Disappear: Beyond Heuristics and Biases,Gerd Gigerener, European Review of Social Psychology,Volume2, Issue 1,1991
Special Issue: European Review of Social Psychology .
+ou &ill #nd that this statistical error of not consideringthe sample si>e and focussing on the content of thestor"1 itself has limitations1 as the author e'plains3 b"considering a more detailed 0ie& of statistics ie b"considering1 for e'ample3 relati0e freMuenc" rather thansingle freMuenc" case1 and also1 &hether or not the
obser0ation &as randoml" selected or self selecteditselfF this kind of a nuanced technical 0ie& of statisticscould possibl" remo0e this statistical error itself$
http://www.tandfonline.com/loi/pers20?open=2http://www.tandfonline.com/loi/pers20?open=2http://www.tandfonline.com/toc/pers20/2/1http://www.tandfonline.com/toc/pers20/2/1http://www.tandfonline.com/toc/pers20/2/1http://www.tandfonline.com/toc/pers20/2/1http://www.tandfonline.com/loi/pers20?open=2http://www.tandfonline.com/loi/pers20?open=28/9/2019 Literature on Heuristics Ahmednagar
24/24
What &as this lecture about?!4'plain belo& statement!
This lecture is a Sample of theLiterature on Heuristics* and
Biases(Shortcuts in Thinking* andBiases Related To Them!the law ofsmall numbers (and its relatedheuristic: representativeness)and errors in statistical thinkingrelated to it dem"sti#ed$