1
RevealedIndifference:UsingResponseTimestoInferPreferences*
ArkadyKonovalov1andIanKrajbich2
April23,2017
Abstract
Revealedpreferenceisthedominantapproachforinferringpreferences,butitrelieson
discrete,stochasticchoices.Thechoiceprocessalsoproducesresponsetimes(RTs)which
arecontinuousandcanoftenbeobservedintheabsenceofinformativechoiceoutcomes.
Moreover, there is a consistent relationship between RTs and strength-of-preference,
namelythatpeoplemakeslowerdecisionsastheyapproachindifference.Thisrelationship
arises from optimal solutions to sequential information sampling problems. Here, we
investigateseveralwaysinwhichthisrelationshipcanbeusedtoinferpreferenceswhen
choiceoutcomesareuninformativeorunavailable. WeshowthatRTsfromasingletwo-
alternativechoiceproblemcanbeenoughtousefullyrankpeopleaccordingtotheirdegree
oflossaversion.Usingalargenumberofchoiceproblems,wearefurtherabletorecover
individual utility-function parameters from RTs alone (no choice outcomes) in three
differentchoicedomains.Finally,weareabletouselongRTstopredictwhichchoicesare
inconsistentwithasubject’sutilityfunctionandlikelytolaterbereversed. Theseresults
provideaproofofconceptforanovel“methodofrevealedindifference”.JELCodes:C91;D01;D03;D87;D81;D90.
*TheauthorsthankPaulHealy,LucasCoffman,RyanWebb,DanLevin,JohnKagel,KirbyNielsen,JeevantRampal,andPujaBhattacharyafortheirhelpfulcommentsandconversationsandYosukeMorishima,ErnstFehr,ToddHare,ShabnamHakimi,andAntonioRangelforsharingtheirdata.1DepartmentofEconomics,TheOhioStateUniversity;[email protected],DepartmentofEconomics,[email protected].
2
1Introduction
When inferring a person’s preferences, businesses,managers, economists, and policy
makerstypicallyrelyontheirchoices. Thisisthestandardrevealedpreferenceapproach
(Samuelson, 1938). In this paperwe challenge this fundamental notion, by arguing that
individualpreferencescanbeinferredwithoutobservingchoiceoutcomes.
Choice itself is not the only output of the choice process. We are also often able to
observeotherfeaturessuchasresponsetimes(RT).Moreover,RTsarecontinuousandso
may carry more information than discrete stochastic choice outcomes (Loomes, 2005;
Webb,2013)3. Thetrickliesindiscoveringhowtheinteractionbetweenpreferencesand
choiceoptionsproducesRTs. Ifwecouldcharacterizeandtheninvertthisfunction,then
wecouldinferpreferencesfromRTs.
Considerthefollowingexample.Supposeweareattemptingtodeterminewhichoftwo
agentsismoreimpatient.Weaskeachofthemthesamequestion:“wouldyouratherhave
$25 todayor$40 in twoweeks?” Suppose thatbothagents take the$40. With just this
informationthereisnowaytodistinguishbetweentheagentswithoutfurtherquestioning.
Now suppose Agent A made his choice in 5 seconds, while Agent B made hers in 10
seconds.Whoislikelymorepatient?
Weargue,usingdatafromseveralchoiceexperimentsindifferentpreferencedomains,
thattheanswerisAgentA.Whilestandardeconomictheoryissilentinthissituation,there
arerelevanttheoreticalframeworksforansweringthisquestion.Onerelevantframework
of particular interest, due to its success in cognitivepsychology andneuroscience, is the
3ForareviewontheuseofRTsineconomics,seeClithero(2016a)andSpiliopoulosandOrtmann(2014).
3
class of sequential sampling models (SSM) (also known as drift-diffusion or evidence-
accumulationmodels). Thatframeworkviewsdecisionslikethisoneasamentaltug-of-
war between the options. For options that are similar in strength (utility) it takesmore
timetodeterminethewinner,andinmanycasestheweakersidemayprevail.
Toput itanotherway,rather thanbasingtheirdecisionsondeterministicutilities for
eachoption,agentsinsteadbasetheirdecisionsonstochasticutilities,likeinrandomutility
theory(RUT). AswithRUT,SSMsareagnosticaboutwheretheaverageutilitiesornoise
originate,theyaresimplytheresultofsomeevaluationprocess4.WhereSSMsdepartfrom
standardRUTisbyassumingthatagentsrepeatedlyestimateor“sample”thesestochastic
utilities, averaging out the noise across samples until one option emerges as the clear
winner. This evaluationprocess takes an amount of time that dependson thedifference
betweentheaverageutilities,i.e.thestrength-of-preference.
As a result of this process, a relationship emerges between strength-of-preference,
choiceprobability, andRT. Returning toourexample,weknow that sinceAgentAwas
relativelyfasterthanAgentB,itismorelikelythatAgentAfacedaneasierdecision.Since
itwas easier for Agent A to choose the later option,we can infer that he is likelymore
patient.
Of course, other independent factors may influence RT, but if there is a statistically
reliable relationshipbetweendecision speedandproximity to indifference, this couldbe
used toeconomists’andpolicymakers’advantage inanumberofways. First,onecould
design simpler and shorter preference-elicitation mechanisms such as the single-trial
4Theassumptionisthatthisevaluationprocess,followingthenotionofutilitytheoryasan“asif”model,doesnottypicallyinvolveexplicitutilitycalculation.OtherwisetheutilitieswouldbecomedeterministicandRTswouldnotdependonutilitydifference,astheyoftendoempirically.
4
binary-choiceexampleabove. Second,onecouldrecoverutility functions incaseswhere
agents’ preferences fall outside of the range of a given elicitation procedure,where it is
currentlyonlypossible toput a lowerorupperboundonpreferences. Third, one could
estimateutility-functionparametersinthecompleteabsenceofchoiceoutcomesorwhen
theseoutcomesarenot informative(i.e. thesame). Byaskingavarietyofquestions,one
couldseewhichoneselicittheslowestresponsesandinferthatthosequestionsmadethe
person roughly indifferent. This last point is to highlight that it is still possible to do
preference analysis in the absence of informative choice outcomes. This suggests, for
example, that in order to protect their private information, agents must consider
observabilityofboththeirchoicesandtheirRTs.
Here, we demonstrate that inferring preferences from RTs is indeed a promising
approach,usingexperimentaldata from threeprominent choicedomains: risk, time, and
social preferences. In Section 5.3 we show that single-trial RTs can be used to rank
subjects according to their degree of loss aversion. In Section5.4we show thatRTs on
“extreme” trials (wheremost subjects choose the sameoption) can alsobeused to rank
subjectsaccordingtotheirloss,time,andsocialpreferences.FinallyinSections5.5and5.6
we examine RTs from the full datasets and use each subject’s RT data to estimate their
utility-function parameters. In every case these rankings/parameters significantly align
with those estimated from subjects’ choices over the full datasets. Taken together, these
results serve as a proof of concept that individual preferences can be inferred fromRTs
alone.
2Model
5
2.1.Background
Aseconomists,weoftendefendexpectedutilitytheorybyarguingthatpeoplemerely
behave “as if” they are calculating andmaximizing utility. But if people do not explicitly
calculateutilities,howdotheymakedecisions? Thishasbeenatopicof intensestudyin
thefieldsofjudgmentanddecisionmaking,andmorerecently,neuroeconomics.Fromthis
research there is emerging consensus that people make decisions by accumulating and
comparing evidence for each of the alternatives, until the evidence for one option
sufficiently outweighs the other. In some cases this process may involve explicitly
collecting information from the environment, e.g. reading reviewsbeforedecidingwhich
movietowatch. Inothercases, itmay involve introspection:collecting internalevidence
basedonpastexperiencesandone’scurrentstate,e.g.decidingwhichofyourtwofavorite
movies to re-watch. Economists have traditionally treated these two types of decisions
separately,referringtotheformeras“perceptibility”andthelatteras“desirability”(Block
&Marschak,1960).
Whilethesemayseemlikeverydifferenttypesofdecisions,thereisnowoverwhelming
empiricalevidencethattheycanbeexplainedbyacommonmathematicalframework.The
precisemechanismbywhichthisevidenceaccumulationandcomparisonoccursisstillan
activetopicofstudyinneuroscience5,butfornowwetreatSSMsinthesamewaythatwe
treatexpectedutility: as “as if”models. That is,wedon’t yetknowexactlywhatorhow
evidence is being accumulated and compared in the brain, but thismodeling framework
5Theexactmechanismbehindnoisysampling(e.g.,samplingfrommemory,shiftsinattention,ornoiseinneuralnetworks)remainsasubjectofongoingdebateintheliterature(Shadlen&Shohamy,2016)andisoutsideofthescopeofourstudy.
6
allowsustoaccountforchoiceandRTdata(aswellaseye-trackingandneuraldata)that
othermodelscan’t.
TheideaofapplyingSSMstodecisionsthatarebasedsolelyoninternalevidenceiswell
establishedincognitivepsychology6.RogerRatcliff’s(1978)seminalworkonthediffusion
modelwasasatheoryofmemoryretrieval. Subjectswouldinitiallystudyalistofwords
andthenlaterbepresentedwitha“probe”word. Theirtaskwastoindicatewhetherthe
probewasawordthathadappearedonthestudylist.Evidenceinthismodelwassimply
aninternalcomparisonbetweentheprobewordandthesubject’simperfectmemoryofthe
study words. Similarly, economic choice relies on a comparison between imperfect
memoriesand/orrepresentationsofthechoicealternatives.
TheideaofapplyingSSMstoeconomicchoicewasfirstintroducedbyBusemeyerand
colleagues in the 1980s (Busemeyer, 1985; Busemeyer&Diederich, 2002; Busemeyer&
Rapoport, 1988;Busemeyer&Townsend, 1993; Johnson&Busemeyer, 2005;Rieskamp,
2008;Roe,Busemeyer,&Townsend,2001). Recentyearshaveseenrenewedinterestin
thisworkduetotheabilityofthesemodelstosimultaneouslyaccountforchoices,RTs,eye
movements, and brain activity in many individual preference domains such as risk and
uncertainty(Busemeyer,1985;Busemeyer&Townsend,1993;Fiedler&Glöckner,2012;
Hunt et al., 2012; Stewart, Hermens, & Matthews, 2015), intertemporal choice (Dai &
Busemeyer,2014;Rodriguez,Turner,&McClure,2014a),socialpreferences(Hutchersonet
al.2015;Krajbichetal.2015a;Krajbichetal.2015b),aswellasfoodandconsumerchoice
(DeMartino,Fleming,Garret,&Dolan,2013;Krajbich,Armel,&Rangel,2010;Krajbich,Lu,
6Forover50years,cognitivepsychologistshavebeenusingSSMstoexplainindividualbehaviorinsimpleperceptionandmemorytasks(BrownandHeathcote2008;Laming1979;Link1975;Ratcliff1978;RatcliffandMcKoon2008;Stone1960;UsherandMcClelland2001).
7
Camerer,&Rangel,2012;Krajbich&Rangel,2011;Milosavljevic,Malmaud,Huth,Koch,&
Rangel, 2010; Philiastides & Ratcliff, 2013; Towal, Mormann, & Koch, 2013), and more
complex decision problems (Caplin & Martin, 2016). Our approach builds on this
literature,withoneimportantchange:insteadoffittingthemodeltochoicesandRTs,wefit
themodeltoonlytheRTs.Thisletsustesthowusefulthemodelcanbewithoutobserving
choiceoutcomes.
Beyondtheirdescriptivepower,SSMscanalsobenormative. Wheneverydecision is
equallydifficultandequallyvaluable,theconstant-thresholddriftdiffusionmodel(DDM)is
optimal(Wald,1945), in thesense that foradesiredaccuracy, theDDMminimizesmean
RT. Whendecisions vary in difficulty or value, the optimal SSMs aremore complex and
involve collapsing thresholds (Busemeyer & Rapoport, 1988; Frazier & Yu, 2007;
Fudenberg,Strack,&Strzalecki,2015;Tajima,Drugowitsch,&Pouget,2016;Webb,2013b;
Woodford,2014).Itisworthnotingthatrecentworkindecisiontheoryhasfocusedonthe
problemofoptimal samplingand stochastic choice (Che&Mierendorff, 2016;Matejka&
McKay,2014;Sims,2003;Steiner,Stewart,&Matějka,2017;Woodford,2016).
In thispaperwedeliberatelyavoid the issueofwhetheragentsareusing theoptimal
SSM. Insteadwe rely on the simplest, most robust SSM variant, which is the DDMwith
constantthresholds.Therearemanypapersthathaveempiricallyinvestigatedoptimality,
withlittleevidenceforcollapsingthresholds(Hawkins,Forstmann,Wagenmakers,Ratcliff,
&Brown,2015;Oudetal.,2016),andourstudyisnotdesignedtoaddressit(suchstudies
typicallyinvolvehundredsoreventhousandsofchoicetrials).Admittedlythismayhinder
our ability to infer preferences, but the simple DDM provides a proof-of-concept and a
8
lowerboundontheusefulnessofRTsineconomicanalysis. Laterinthetextweexamine
othermodelsandstatisticaltechniquesforinferringpreferencesfromRTs.
Onekeythingtonoticewiththesemodels is thattheagentchoosesherstoppingrule
(thresholds)priortothesamplingprocessandsowhatdeterminesheractualchoiceand
RT, conditional on that stopping rule, is the sequenceof evidence that she accumulates7.
That sequence is stochastic, but does depend on the agent’s subjective evaluation of the
alternatives. More specifically, following previous literature, we assume that the net
evidence is the difference between the average utilities of the two options (from a RUT
perspective)8. Therefore the speed and accuracy of the decision are monotonically
increasinginthisdifference.Thisquantityisoftenreferredtoasthe“driftrate”,andinour
economicsettingitrepresentsstrength-of-preference,i.e.thedifferenceincardinalutility.
Thus,whenapersonisclosetoindifference,thedriftrateapproacheszero,causingadelay
inthechoice(Chabris,Morris,Taubinsky,Laibson,&Schuldt,2009;Dickhaut,Rustichini,&
Smith,2009;Moffatt,2005;Mosteller&Nogee,1951).
2.2.Model
Hereweuseastandarddrift-diffusionmodel(DDM)(Ratcliff1978)toestablishthelink
betweenRTsandtheunderlyingutilitiesinsimplebinarychoicetasks.Asnotedabove,this
model is a standard tool in cognitive psychology for simultaneously capturing choice
probabilitiesandRTdistributions. Ithasseenmostofitsuseinperceptionandmemory,7Thisisastandardassumption,withveryfewexceptions.Thisrelatesbacktothenormativeissuesdiscussedaboveandisdiscussedatlengthinthosearticles.8Again,weassumethatsubjectsarenotexplicitlycalculatingexpectedvaluesorotherutilities.Ifthatwasthecase,themaindeterminantofdifficultywouldbethecomplexityofthecalculations.Instead,weassumethatsubjectsestimatehowgoodeachalternativeis,anditistherelativeattractivenessofthetwooptionsthatdeterminesthedifficulty.
9
but it has alsomore recentlybeenapplied to economic choice, e.g. intertemporal choice,
performingbetterthancompetingmodelssuchaslogisticchoice(Dai&Busemeyer,2014).
Letusassumethatanagentobservesasetofalternatives𝑗 ∈ {1,2}.Thechoiceprocess
involves two components: a constant boundary threshold𝑏 and a decision variable𝑦(𝑡)
thatevolvesovertimeaccordingtothefollowingdifferentialequation:
𝑑𝑦 𝑡 = 𝑣 ∙ 𝑑𝑡 + 𝜎 ∙ 𝑑𝑊 (1)
where𝑦(𝑡) isaccumulatedevidencetowardsoption1(with𝑦(0) = 0),𝑣 is thedriftrate,
whichisassumedtobealinearfunctionoftheutilitydifference:
𝑣 ≡ 𝑧 ⋅ (𝑢9 𝜃 − 𝑢< 𝜃 ), (2)
where𝑧 ∈ 𝑅>isascalingparameter,𝑢?(⋅)istheutilityofthegivenalternative,and𝜃isthe
agent-specificutilityfunctionparameter.Finally,𝜎 ∙ 𝑑𝑊isaWienerprocess(i.e.Brownian
motion)thatrepresentsGaussianwhitenoisewithvariance𝜎<.Withoutlossofgenerality,
wenormalize𝜎 = 1asitcanonlybeidentifieduptoscale(duetothearbitraryunitsony).
We define the response time 𝑅𝑇 as the first time that the absolute value of the
decisionvariablereachesaboundary𝑏 ∈ 𝑅>,plusanon-stochasticcomponentknownas
non-decisiontime(𝜏 ∈ 𝑅>,typicallyinterpretedasthetimethatasubjectneedstoprocess
theinformationonthescreen):
𝑅𝑇 = min 𝑡: 𝑦 𝑡 ≥ 𝑏 + 𝜏. (3)
Thechoiceoutcome𝑎 ∈ {1,2}isdefinedasfollows:
𝑎 = 1𝑖𝑓𝑦 𝑅𝑇 = 𝑏2𝑖𝑓𝑦 𝑅𝑇 = −𝑏 (4)
10
Now, assuming without loss of generality 𝑣 ≥ 0, we can calculate the choice
probabilities𝑝(𝑎 = 𝑗)9,theexpectedRT,andanapproximatebivariateprobabilitydensity
function (PDF) for theRTs (minus 𝜏) as follows (Blurton, Kesselmeier,&Gondan, 2012;
Navarro&Fuss,2009;Ratcliff,1978;Srivastava,Feng,Cohen,Leonard,&Shenhav,2015;
Wabersich&Vandekerckhove,2014):
𝑝(𝑎 = 1) =NOPQR9
NOPQRNSOPQ𝑖𝑓𝑣 > 0
9<𝑖𝑓𝑣 = 0
(5)
𝐸 𝑅𝑇 =VW1 − <
NOPQRNSOPQ+ 𝜏𝑖𝑓𝑣 > 0
𝑏< + 𝜏𝑖𝑓𝑣 = 0 (6)
𝑓(𝑡) = XYVO
𝑒RPO[O −1 \R9𝑘 ⋅ 𝑒R
^O_O[`QOa
\b9 sin \X<
(𝑒WV + 𝑒RWV) (7)
Typically, if choice data is available, one can use equation (5) to estimate
parameterspurelyfromchoices,maximizinglog-likelihood:
𝐿𝐿 = (1 𝑎e = 1 ⋅ log 𝑝 𝑎e = 1 𝑏, 𝑣e +e (1 𝑎e = 2 ⋅ log 1 − 𝑝 𝑎e = 1 𝑏, 𝑣e ,(8)
where𝑛denotestrialnumber,and1(⋅)istheindicatorfunction.
Here,unlikepreviousstudies,wewillassumethatonlyRTdataisavailableandwilluse
theRTprobabilitydensitiestoestimatetheutilityparameterforeachsubject𝑖(𝜃j)given
theempiricaldistributionofRTsbymaximizingthefollowinglikelihoodfunction:
𝐿𝐿 = (log(𝑓 𝑅𝑇e, 𝑎e = 1 𝑏, 𝜏, 𝑣e ) +e log(𝑓 𝑅𝑇e, 𝑎e = 2 𝑏, 𝜏, 𝑣e )). (9)
9Previousworkhasshownthatthestandardrandomutilitymodelthatimpliesalogitchoicerulecanbederivedfromthedrift-diffusionmodel(Webb,2013b).
11
Figure 1. Example simulation of the drift-diffusion model (DDM). Response times(RTs)asafunctionofthedifferenceinutilitiesbetweentwooptionsin900simulatedtrials.Thegraydotsshowindividualtrials, theblackcirclesdenoteaverageswithbinsofwidth10.Theparametersusedforthesimulationcorrespondtotheparametersestimatedatthegroup level in the time discounting experiment (𝑏 = 1.33, 𝑧 = 0.09, 𝜏 = 0.11). Utilitydifferencesaresampledfromauniformdistributionbetween-20and20.
2.3.Modelpredictions
There are several important qualitative predictions of themodel that allow usmake
inferencesbeyondstructuralestimation.
First, the expected RTs decrease as the utility difference between the two options
becomes larger (lm(no)lW
< 0, see Figure 1 for a simulation example). If the drift scaling
parameter and non-decision time are the same (or similar) across subjects, we should
observe a group-level correlation between the RT and the utility function parameter of
interest, conditionalon thesubjectsmaking thesamechoice. Thusweshouldbeable to
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●● ●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●● ●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
● ●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●● ●●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
● ●
●
●
●●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
● ●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
● ●
● ●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●●
−20 −10 0 10 20
01
23
45
utility difference
RT
[s]
● ●
●
●●
●●
● ●
12
rank subjects according to their individual preferences using RTs from a single decision
problem.WeexplorethispredictioninSection5.3.
More generally, this relationship can be used to rank subjects using sets of choice
problemswheretheyallmadethesamechoice,andsochoicesareuninformative.Consider
anintertemporalchoiceexamplewithtwoindividuals:apatientpersonwhoisindifferent
between$10todayand$12tomorrow,andanimpatientpersonwhoisindifferentbetween
$10todayand$19tomorrow. Thepatientindividualwill likelytakesometimetodecide
between$10todayand$11tomorrow,butchooseveryquicklybetween$10todayand$20
tomorrow.Ontheotherhand,theimpatientindividualwillstruggletochoosebetween$10
todayand$20tomorrow,butchooseveryquicklybetween$10todayand$11tomorrow.
Notice that in thisexampleboth individualswouldmost likelychoose$10todayover
$11tomorrowand$20tomorrowover$10today,basedontheirindifferencepoints.Thus
arevealedpreferenceapproachwouldbeunlikelytomakeadistinctionbetweenthesetwo
individuals based only on these choices. In contrast, the revealed indifference approach
would tell us that a slow choice between $10 today and $11 tomorrow is indicative of
patience, while a slow choice between $10 today and $20 tomorrow is indicative of
impatience.WeconsiderthecaseofuninformativechoiceoutcomesinSection5.4.
Second, long RTs are considerably more informative than short RTs. Sequential
samplingmodelscorrectlypredictthatshortRTscanoccuratanylevelofchoicedifficulty
(i.e.strength-of-preference),butlongRTsalmostexclusivelyoccurfordifficultchoices(i.e.
nearindifference;again,seeFigure1foranexamplesimulation).Theintuitionhereisthat
there is a lotofnoise in thedecisionprocess and so thequickest choices are thosewith
noise pointed in the same direction (Ratcliff, Philiastides, & Sajda, 2009; Ratcliff &
13
Tuerlinckx,2002). Because these sequencesof alignednoiseoccur independentlyof the
driftrate,thereistypicallyno(oraveryweak)correlationbetweentheshortestRTsand
difficulty (Ratcliff & McKoon 2008). However, the slowest choices are ones where the
combinationofdriftplusnoiseleadstoroughlyzeronetevidenceaccumulation.Giventhat
thesamplednoiseismean-zero,net-zeroevidenceaccumulationisfarmorelikelytooccur
withsmalldrift(difficultproblems)thanwithlargedrift(easyproblems).Thisproducesa
correlationbetweenthelongestRTsanddifficulty.
By taking advantage of this knowledge, we can improve the RT-based estimation of
utility-function parameters (relative to structural estimation, Section 5.5), either by
focusingonthelongestRTsorbyestimatingthepeakinRTsusingthefullsubjectdataset.
ThesemethodsareexploredinSection5.6.
Finally, themodelpredicts thatmost “errors”, or choices inconsistentwith theutility
function,shouldhappenwhenthesubject iscloseto indifferenceandthustendstomake
slowchoices.WetestthishypothesisinSection5.7.
2.4.Modellimitations
WithRTs,“slow” isarelativeconceptwhereaschoice isabsolute. Inordertousethe
revealed indifference approach, we must therefore develop methods to identify slow
decisions for a given individual. Another potential issue is that even though RTs are
continuous,theycanbeinfluencedbyotherfactorsasidefromstrength-of-preferenceand
thusmayappeartobequitenoisy(Krajbichetal.2015a).Forinstance,othershaveargued
thatlongRTsreflectmoreeffortordeliberativethinkingonthepartofthesubject(Gabaix,
14
Laibson, Moloche, & Weinberg, 2006; Hey, 1995; Rubinstein, 2007; Wilcox, 1993). This
couldinterferewithourabilitytouseRTstoinferpreferences.
There are certain characteristics of the choice environment that we believe may
facilitateestimatingpreferencesfromRTs.First,RTsshouldberecordedwithmillisecond
precisionsincemanysimplebinarychoicestakeonlyafewseconds.Second,choicesshould
bemadeusingakeyboardbuttonpressratherthanusingamousetoselectanoption.This
issimplytominimizenoise inRTsduetohandmovements. Third,we limitourselves to
binary decisions since indifference is not straightforward with multiple alternatives.
Fourth,forthelastexercise,whereweusetherevealedindifferenceapproachandallthe
trials to estimateutility functions, the rangeof indifferencepoints impliedby the choice
problemsintheexperimentshouldcovertherangeofparametervaluesinthepopulation.
It shouldbementioned thatSSMshavebeensuccessfullyapplied todatasets thatviolate
someoftheserequirements(e.g.Krajbichetal.2015b;KrajbichandRangel2011),butsuch
datasetsarenotidealforourcurrentgoals.
Inadditiontotheserequirements,thereareotherfactorsthatmayaffectourabilityto
successfully infer indifferencepoints fromRTs. Onequestion iswhetherthereshouldbe
constraints on RTs. RT restrictions are common in binary choice tasks in order to keep
subjects focused, but overly restrictive cutoffs may attenuate the effect of strength-of-
preferenceonRTs(Fudenbergetal.,2015).Hereweexaminedatasetswithvaryingtime
constraints(3s,10s,andunlimited)inordertoexploretherangeofsituationsinwhichwe
canemploythemethodofrevealedindifference.Giventheveryfewmissedtrials,andthe
robustness of our results across the datasets, it is unlikely that the constraints were
binding.Anotherquestion iswhether themethodof revealed indifferencecanbeused in
15
situations where the utility function contains multiple parameters. This will obviously
complicatematters,butifoneparameterisrelativelymoreimportantthantheothers,the
methodmaystillwork,aswedemonstrateintheriskychoicetask.
3Relevantliterature
It is important toacknowledge thatwearenot the firsteconomists touseRTdata to
predictbehavior. Inadditiontotheworkmentionedabove,SchotterandTrevino(2014)
and Chabris et al. (2009) have used RTs to predict strategic behavior and aggregate
intertemporalpreferences,respectively.
Schotter and Trevino (2014) use the longest (and second longest) RTs to estimate
threshold strategies in a global game; theRT-based estimates are able to explainout-of-
samplechoicebetterthantheequilibriumprediction.Thispaperissimilarinspirittoours,
but studies strategic situationswhilewe focus on individual preference elicitation. The
authorsarguethatlongRTsintheirsettingreflecteitherdiscoveringtheoptimalthreshold
strategyorexplicitcalculation.Theseconsiderationsandexplanationsarequiteseparate
fromtheideasandmodelsexploredinourwork.
In the preference domain, Chabris et al. (2009) and an earlier unpublished working
paperbythesameauthors(Chabris,Laibson,Morris,Schuldt,&Taubinsky,2008)useda
methodinfluencedbythedirectedcognitionmodel(Gabaix&Laibson,2005;Gabaixetal.,
2006)andfullRT-distributiondataatthegroupleveltoestimateaggregatedintertemporal
preferences using a structural estimation approach. Thismethod aims to identify utility
parameters using the fact that RTs increase with choice difficulty. Their study finds a
correlation between the RT-based estimates and choice-based predictions for three
16
separatelargegroupsofsubjects,butdoesnotattemptanyanalysisofindividualsubjects’
preferences.Intheappendix,weprovideacomparisonbetweenthismethodandourown.
Finally, Clithero (2016b) used a DDM to augment choice data with RT information,
producingbetterout-of-samplepredictionsinfoodchoice.Whilethisworkisveryrelated
toourown,weshowthatRTinformationalonecanbeusedtopredictdecisions.
Ourwork also relates to a growing literature on incomplete preferences (Agranov&
Ortoleva, 2015; Cettolin & Riedl, 2015; Danan, Ziegelmeyer, & others, 2006; Eliaz & Ok,
2006;Mandler,2005;Ok,Ortoleva,&Riella,2012). This literaturehasshownthatwhen
given the choicebetween twoalternatives, individualsmayprefera lotteryover the two
alternatives.Thisproducesprobabilisticchoiceoutcomes,justlikewithSSMs.Moreover,
DananandZiegelmeyer(2006)report thatamajorityof thesubjects in theirexperiment
postponesimplechoicesbetweenlotteries,eventhoughpostponementiscostly.Whilenot
exactlyameasureofRT,thispostponementbehavioris inlinewiththeslow-indifference
phenomenonthatwereport.
Our work contrasts with the work of Rubinstein, Wilcox, and others (Chen &
Fischbacher,2015;Hey,1995;Recalde,Riedl,&Vesterlund,2014;Rubinstein,2007,2013,
2014;Wilcox, 1993),where longRTs are associatedwithdeliberative thought and short
RTsareassociatedwith intuition. Thesepapershaveprimarilystudiedstrategicsettings
where effort is likely to vary more substantially across subjects and so dominate the
strength-of-preferenceeffectsthatdriveourresults.Whilethiscanbeaccountedforwith
varying levelsof𝑏 in theDDMthatwehavediscussed, itmayalsobethatotherdecision
processesareinvolvedinstrategicdecisions.
17
4ExperimentalDesign
We analyze four separate datasets: the last two were collected with other research
goalsinmind,butincludedprecisemeasurementsofRTs,whilethefirsttwowerecollected
specificallyforthisstudy.
4.1.Loss-aversionexperiments
These experiments were conducted at The Ohio State University. In each round,
subjects chose between a sure amount of money and a 50/50 lottery that included a
positive amount and a loss (which in some roundswas equal to $0).The set of decision
problemswasadaptedfromSokol-Hessneretal.(2009).Subjects’RTswerenotrestricted.
Intheadaptiveexperiment,eachsubject’schoicedefinedthenexttrial’soptionsusinga
Bayesian procedure (DOSE, seeWang et al. 2010) to ensure an accurate estimate of the
subject’sriskandlossaversionwithinalimitednumberofrounds.Eachsubjectcompleted
the same three unpaid practice trials followed by 30 paid trials. Importantly, every
subject’sfirstpaidtrialwasidentical.Eachsubjectreceivedtheoutcomeofonerandomly
selectedtrial.61subjectsparticipatedinthisversionoftheexperiment,earning$17-20on
average10.
In the non-adaptive experiment, each subject first completed a three-trial practice
followedby276paid trials. These trialswerepresented in twoblocks of the same138
choice problems, each presented in random order without any pause between the two
blocks. Subjects were endowed with $17 and additionally earned the outcome of one
10Inordertocoveranypotentiallossesincurredduringthistask,subjectsfirstcompletedanunrelatedtaskthatendowedthemwithenoughmoneytocoverforanypotentiallosses.
18
randomly selected trial (in case of a loss it was subtracted from the endowment). 39
subjectsparticipatedinthisexperiment,earning$18onaverage.Allsubjectsgavewritten
consent,andthestudieswereapprovedbytheOSUInstitutionalReviewBoard.
For both experiments we assumed a standard prospect theory value function
(Kahneman&Tversky,1979):
𝑈 𝑊, 𝐿, 𝑆 = 0.5𝑊t − 0.5𝜆 −𝐿 t
𝑆t,
where𝑊 is the gain in the lottery, L is the loss, S is the sure amount,𝜌 reflects risk
aversion,and𝜆captureslossaversion.Inthenon-adaptiveexperiment,theutilityfunctions
wereestimatedusingastandardMLEapproachwithalogitchoicefunction.
Similartopriorworkusingthistask,wefoundthatriskaversionplaysaminimalrolein
this task relative to loss aversion, with 𝜌estimates typically close to 1. Therefore,
acknowledging that varying levels of risk aversion could add noise to the RTs, for the
analysesbelow (both choice- andRT-based)weassumed riskneutrality (𝜌 = 1). For the
non-adaptiveexperiment,weusedonlytrialswithnon-zerolosses(specifically,112outof
138 decision problems) for both estimation and out-of-sample prediction. Two subjects
withoutlyingestimatesof𝜆(beyondthreestandarddeviationsofthemean)wereremoved
fromtheanalysis.Thesameexclusioncriterionwasusedfortheotherdatasets.
4.2.Temporaldiscountingexperiment
This experiment was conducted while subjects underwent functional magnetic
resonanceimaging(fMRI)attheCaliforniaInstituteofTechnology(Hare,Hakimi,&Rangel,
2014). Ineachround,subjectschosebetweengetting$25rightafter theexperimentora
largeramount(upto$54)atalaterdate(7to200days).Therewere108uniquedecision
19
problemsandsubjectsencounteredeachproblemtwice. All216trialswerepresentedin
randomorder. Each trial, the amountwas first presentedon the screen, followedby the
delay,andsubjectswereaskedtopressoneoftwobuttonstoacceptorrejecttheoffer.The
decisionwas followedbya feedback screen showing “Yes” (if theofferwasaccepted)or
“No”(otherwise).Thedecisiontimewaslimitedto3seconds,andifasubjectfailedtogive
a response, the feedback screen contained the text “No decision received”. These trials
(2.6%acrosssubjects)wereexcludedfromtheanalysis.Trialswereseparatedbyrandom
intervals(2-6seconds).
41subjectsparticipatedinthisexperiment,earninga$50show-upfeeandtheamount
fromone randomly selected choice. Thepaymentsweremadeusingprepaiddebit cards
thatwereactivatedatthechosendelayeddate.Allsubjectsgavewrittenconsent,andthe
experimentwasapprovedbyCaltech’sInternalReviewBoard.
For this experiment, we used a hyperbolic discounting utility function (Herrnstein,
1981;Laibson,1997):
𝑈 𝑥, 𝑡 =𝑥
1 + 𝑘𝑡,
where𝑥isthedelayedamount,𝑘isthediscountfactor(higherismoreimpatient),and𝑡
is thedelayperiod indays.One subject that chose $25nowon every trialwas removed
from the analysis. The utility functionswere estimated using a standardMLE approach
withalogitchoicefunction.Twosubjectswithoutlyingestimatesof𝑘wereremovedfrom
theanalysis.
4.3.Binarydictatorgameexperiment
20
This dataset was collected while subjects underwent fMRI at the Social and Neural
Systems laboratory, University of Zurich (Krajbich et al. 2015a). Subjects made choices
between two allocations, X and Y, which specified their own payoff and an anonymous
receiver’s payoff. The payoffs were displayed in experimental currency units, and 120
predeterminedallocationswerepresentedinrandomorder.Eachallocationhadatradeoff
betweenafairoption(moreequaldivision)andaselfishoption(withhigherpayofftothe
dictator).72outof120decisionproblemspersubjecthadhigherpayofftothedictatorin
bothoptionsXandY(to identifyadvantageous inequalityaversion),whiletherestofthe
problems (48/120) had higher payoffs to the receiver in both options (to identify
disadvantageous inequalityaversion). Ineachround,subjectsobservedadecisionscreen
that included the twooptions,andhad tomakeachoicewitha two-buttonbox.Subjects
were required tomake their decisionswithin 10 seconds; if a subject failed to respond
under this time limit, that trialwas excluded from the analysis (4 trialswere excluded).
Intertrialintervalswererandomizeduniformlyfrom3to7seconds.
Subjects read written instructions before the experiment, and were tested for
comprehension with a control questionnaire. All subjects passed the questionnaire and
understoodtheanonymousnatureofthegame.Intotal,30subjectswererecruitedforthe
experiment. They received a show-up fee of 25 CHF and a payment from 6 randomly
chosenrounds,averagingatabout65CHF.Allsubjectsprovidedwrittenconsent,andthe
ethicscommitteeoftheCantonofZurichapprovedthestudy.
To fit choices in this experiment, we used a standard Fehr-Schmidt other-regarding
preferencemodel(Fehr&Schmidt,1999):
𝑈 𝑥j, 𝑥? = 𝑥j − 𝛼 ⋅ max 𝑥? − 𝑥j, 0 − 𝛽 ⋅ max 𝑥j − 𝑥?, 0 ,
21
where𝑥j isthedictator’spayoff,𝑥? isthereceiver’spayoff,𝛼reflectsdisadvantageous
inequalityaversion,and𝛽 reflectsadvantageous inequalityaversion.Asdescribedabove,
wewereabletoseparatetrialsthatidentified𝛼and𝛽,sothesechoicesweretreatedastwo
separate datasets. The utility functions were estimated using a standard MLE approach
withalogitchoicefunction.Onesubjectwithanoutlyingestimateof𝛼wasremovedfrom
theanalysis.
5Results
5.1.Choice-basedestimations
Thethreeutilityfunctionsweselectedtomodelsubjects’choicesperformedwellabove
chance. To examine the number of choices that were consistent with the estimated
parameter values, we used standard MLE estimates of logit choice functions (for the
estimation procedure details see Appendix A) to identify the “preferred” alternatives in
every trial and compared those to the actual choice outcomes. More specifically, we
calculated utilities using parameters estimated purely from choices, and in every trial
predicted that the alternativewith the higher utilitywouldbe chosenwith certainty.All
subjectswereveryconsistentintheirchoiceseveninthedatasetswithalargenumberof
trials:socialchoice𝛼:94%,socialchoice𝛽:93%,intertemporalchoice:83%,non-adaptive
riskychoice:89%(seeAppendixBforthesubject-levelchoicepredictions).
5.2.Responsetimesreflectchoicedifficulty
ThemethodofrevealedindifferenceispredicatedontheDDM-basedideathatasubject
willtakemoretimetodecideasthedecisionbecomesmoredifficult.Forthefollowing
22
Figure2.RTspeakatindifference.MeanRTinsecondsasafunctionofthedistancebetweentheindividualsubject’sutilityfunctionparameterandtheindifferencepointonaparticular trial; data are aggregated into bins ofwidth 0.02 (top row), 0.01 (bottom leftpanel), and 1 (bottom right panel), which are truncated and centered for illustrationpurposes.Binswith fewer than10 subjects arenot shown.Barsdenote standard errors,clusteredatthesubjectlevel.
analysis, our measure of difficulty was the difference between the subject’s utility
functionparameter(estimatedpurelyfromthesubject’schoices)andtheparametervalue
thatwouldmakeapersonindifferentbetweenthetwoalternativesinthattrial(wereferto
this as the “indifference point”). When a subject’s parameter value is equal to the
indifferencepointofa trial,wesay that thesubject is indifferenton that trialandso the
choiceismaximallydifficult.
●
●
●
●
●
●
●
−0.6 −0.4 −0.2 0.0 0.2 0.4 0.6
1.5
2.0
2.5
3.0
3.5
Social choice (α)
subject α − indifference α
RT [s
]
●
●
●
●
●
●
●●
●●
●
●
−1.0 −0.5 0.0 0.5 1.0
1.0
1.5
2.0
2.5
3.0
3.5
Social choice (β)
subject β − indifference β
RT [s
]●
●
●●
●
●
●
−0.03 −0.02 −0.01 0.00 0.01 0.02 0.03
0.6
0.8
1.0
1.2
1.4
Intertemporal choice
subject k − indifference k
RT [s
]
●● ●
●
●
●
●
●
●
−4 −2 0 2 4
1.0
1.5
2.0
2.5
Risk choice
subject λ − indifference λ
RT [s
]
23
Letusillustratethisconceptwithasimpleexample.Supposethatintheintertemporal-
choicetaskasubjecthastochoosebetween$25todayand$40in30days.Ifthesubjecthas
ahyperbolicdiscountingutilityfunction(asweassume),theywouldbeindifferentwithan
individualdiscountrate𝑘 that is thesolutionto theequation25 = 40/(1 − 30𝑘),ork=
0.0125. Thiswouldbetheindifferencepointofthisparticulartrial. Asubjectwiththisk
valuewouldbe indifferentonthis trial,asubjectwitha lowerkwould favorthedelayed
option,andasubjectwithahigherkwouldfavortheimmediateoption.
The bigger the absolute difference between the subject’s parameter and the trial’s
indifferencepoint, the easier thedecision, and the shorter the averageRT. Indeed, this
effectisobservedinallofourdatasets,withRTspeakingatindifference(Figure2).Mixed-
effects regressionmodels (treating subjects as random effects) show strong, statistically
significant effects of the absolute distance between the indifference point and subjects’
individualutilityparameterson log(RTs)11 forall thedatasets(fixedeffectofdistanceon
RT:dictatorgame𝛼:t=-7.5,p<0.001;dictatorgame𝛽:t=-9.1,p<0.001;intertemporal
choice𝑘:t=-9.9,p<0.001;non-adaptiveriskychoice𝜆:t=-9.6,p<0.001,adaptiverisky
choice𝜆:t=-4.4,p<0.001).
5.3.One-trialpreferenceranking
Intheadaptiveriskychoiceexperiment(Section4.1),allsubjectsfacedthesamechoice
problem in the first trial. They had to choose between a 50/50 lottery with a positive
amount($12)andaloss($7.5),andasureamount($0).Assumingriskneutrality,asubject
11 A key feature of RTs generated by sequential sampling models is that they are roughly log-normally distributed. Thus it is typical to transform RTs with a natural logarithm beforeperformingstatisticalanalyses.
24
Figure3.Preferencerankcanbeinferredfromasingledecisionproblem.RTsinthefirst roundof the adaptive risk experiment as a function of the individual subject’s loss-aversioncoefficient fromthewholeexperiment;Spearmancorrelationsdisplayed. In thisround,eachsubjectwaspresentedwithabinarychoicebetweenalotterythatincludeda50%chanceofwinning$12andlosing$7.5,andasureoptionof$0.Theleftpaneldisplayssubjectswhochosethesafeoption,andtherightpanelshowsthosewhochosetheriskyoption.ThesolidblacklinesareOLSfits.
with a loss aversion coefficient of 𝜆 = 1.6 should be indifferent between these two
options,withmore loss-averse subjects picking the safe option, and the rest picking the
riskyoption.
Because themean loss aversion in our samplewas 2.5 (median = 2.46),most of the
subjects(44outof61)pickedthesafeoptioninthisfirsttrial.Now,ifwehadtorestrictour
experimenttojustthisonetrial,theonlywaywecouldclassifysubjects’preferenceswould
betodividethemintotwogroups:thosewith𝜆 ≥ 1.6andthosewith𝜆 ≤ 1.6.Withineach
groupwewouldnotbeabletosayanythingabouteachindividual’slossaversion.
ByobservingRTswecanestablisharankingofthesubjectsineachgroup.Specifically,
theDDMpredicts that subjectswith longerRTswouldexhibit lossaversioncloser to1.6
(assuming that the threshold, drift, andnon-decision timeparameters are similar across
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
● ●
●●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●
●
1.5 2.0 2.5 3.0 3.5 4.0 4.5
24
68
1012
Safe option chosen
λ estimated from choices
RT [s
]
r = −0.43p = 0.004
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
0.0 0.5 1.0 1.5 2.0 2.5 3.0
24
68
1012
Risky option chosen
λ estimated from choices
RT [s
]
r = 0.41
p < 0.1
25
subjects). We ranked subjects in each group according to theirRTs and then compared
thoserankingstothe“true”lossaversionparametersestimatedfromthe30choicesinthe
fulldataset(seeFigure3).
In linewith the results of the previous section, RTs peaked around the indifference
point(𝜆 = 1.6). Therewasasignificantrank-based(Spearman)correlationbetweenRTs
andlossseekinginthe“safeoption”group(r=0.43,p=0.004)andbetweenRTsandloss
aversion in the “risky option” group (r = 0.41, p = 0.1). Thus the single-trial RT-based
rankingsalignedquitewellwiththe30-trialchoice-basedrankings.
As mentioned in the introduction, this approach could have great appeal since
economistsandpractitionersareoftenconstrainedwhencollectingdata,hencetheappeal
of adaptive experiments (Blais &Weber, 2006; Cavagnaro, Myung, Pitt, & Kujala, 2010;
Kim,Pitt,Lu,Steyvers,&Myung,2014;Rodriguez,Turner,&McClure,2014b;Wangetal.,
2010,andothers).Ourresultssuggestthat in factveryfewtrialsmaybeneededtogeta
reliableestimateofsubjects’preferences12.
5.4.Uninformativechoices
Another possible use of RT-based inference is the case where an experiment (or
questionnaire) is flawed in a such way that most subjects give the same answer to the
choice problems (or onemight consider a situationwherepeople feel social pressure to
give a certain answer, even if it contradicts their true preference). This is similar to the
situationfromthelastsection,exceptthatsubjects’choicesmaybeevenlessinformative,
andwithmultipletrials,theremaybeevenmoretogainfromexaminingtheRTs.12PredictivepowermaybefurtherimprovedbyaccountingforotherpotentialcontributorstoRTs,suchasdemographics,baselinemotorresponsetime,etc.
26
To model this situation, for each dataset (non-adaptive risk choice, intertemporal
choice, and social choice) we isolated trials with the most extreme indifference point,
wheremostsubjectschosethesameoption(e.g.,thelotteries),andlimitedouranalysisto
those subjects who picked this most popular option. In some instances, this involved
several trials since some of the choice problems were repeated or shared the same
indifferencepoint.
We found that the RTs on these trials were strongly correlated with subjects’
preferenceparametersinallfourdomains(riskychoice:r=0.55,p<0.001;intertemporal
choice:r=0.43,p=0.007;socialchoice𝛼:r=0.4,p=0.03;socialchoice𝛽:r=0.68,p<
0.001,Spearmancorrelations).Againweseethatitispossibletosomewhatreliablyrank
subjectsaccordingtotheirpreferencesintheabsenceofdistinguishingchoicedata.Similar
to theprevioussection, thismethodcouldbeused tobolsterdatasets thatare limited in
scopeandsounabletorecoverallsubjects’preferences.
5.5.DDM-basedutilityestimationfromRTs
TheresultsdescribedintheprevioussectionsdemonstratethatwecanuseRTstorank
subjectsaccordingtotheirpreferenceswithoutinformativechoicedata.Inthissection,we
exploreways to estimate individual subjects’ utility-function parameters from their RTs
acrossmultiplechoiceproblems.
The DDM predicts more than just a simple linear relationship between strength-of-
preferenceandmeanRT;itpredictsentireRTdistributions.AsdescribedinSection2,the
DDMapproachassumesthatineachtrial,subjectsmakedecisionsusingaWienerrandom
processthatproducesadistributionofRTsgiventhreefreeparameters,whichcanbe
27
Figure4.TheDDMestimatesofsubjects’utilityfunctionparameters,estimatingDDMparametersat thegroup level.Subject-levelcorrelation(Pearson)betweenparametersestimated fromchoicedataandRTdatausing thedrift-diffusionmodel (DDM).Thesolidlinesare45degreelines.
estimated for each individual subject (Wabersich and Vandekerckhove 2014, see
AppendixAfortheestimationdetails).Inparticular,thedriftrateinthemodelisalinear
functionoftheutilitydifferenceandsobyestimatingdriftrateswecanidentifythelatent
utility-functionparameters.
●● ●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
−1.0 −0.5 0.0 0.5 1.0
−1.0
−0.5
0.0
0.5
1.0
Social choice (α)
α estimated from choices
α e
stim
ated
from
RTs r = 0.39
p = 0.04●●
●●
●
●●
●
●
●
●
●●
●● ●
●
●
●
●
●
●●●
●
●●● ●
●
−1.0 −0.5 0.0 0.5 1.0 1.5
−1.0
−0.5
0.0
0.5
1.0
1.5
Social choice (β)
β estimated from choices
β es
timat
ed fr
om R
Ts
r = 0.52p = 0.003
●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
0.00 0.01 0.02 0.03 0.04 0.05
0.00
0.01
0.02
0.03
0.04
0.05
Intertemporal choice
k estimated from choices
k es
timat
ed fr
om R
Ts
r = 0.57
p < 0.000
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
0 2 4 6 8
02
46
8
Risk choice
λ estimated from choices
λ es
timat
ed fr
om R
Tsr = 0.36
p = 0.03
28
5.5.1.Group-levelDDMestimation
First,we estimated theDDM, assuming that the boundary parameter𝑏, non-decision
time𝜏, anddrift rate𝑧 are fixedacross subjects (seeSection2.2andAppendixA for the
modelandestimationdetails).Thisisacommonlyusedtechniquetohelpwithparameter
identificationwithlimiteddatasets,thoughinSection5.5.2.werelaxthisassumption.The
identificationproblemtypicallyoccurswhenRTspeakattheboundaryofanexperiment’s
indifference points. In these cases, it is impossible to say whether the RTs would have
continuedtorisebeyondtheboundaryandsowecannotknowthetruelocationoftheRT
peak. By fixing theotherparameters in themodelwe constrain thedistributionofRTs,
allowingus to estimate the expectedRTat indifference. Take for example the risk task,
wheremeanRTspeakatapproximately2.1sandthemaximumindifferencelis5.33.Ifwe
observedasubject’sRTsincreasingwiththeindifferencelbutonlyaveraging1.5satl=
5.33,wemightinferthatthesubject’slisnot5.33,butrather7.33(giventherelationship
inFig.2d).
Ineachdataset13,wefoundthat individualutility-functionparametersestimatedfrom
RTsalone(usingthestructuralDDMmodel,withoutchoicedata)werecorrelatedwiththe
sameparametersestimatedfromthechoicedata(socialchoice𝛼:r=0.39,p=0.04;social
choice𝛽:r=0.52,p=0.003;intertemporalchoice𝑘:r=0.57,p<0.001;riskychoice𝜆:r=
0.36, p = 0.03; Pearson correlations; Figure 4,). These results showed an improvement
overthedirectedcognitionmodelsuggestedbyChabrisetal.(2009)(seeAppendixAfor
estimationdetailsandAppendixEforthemodelcomparisonresults).
13Wedonotusetheadaptiveriskdatasetinthissectionsincetheadaptivenatureofthetaskcreatesautocorrelationsinthedatathatwilllikelyinterferewithourestimationprocedure.
29
Anotherway to assess the goodness-of-fit is to examine the number of choices that are
consistentwith theestimatedparametervalues,as inSection5.1. Todoso,weusedthe
RT-estimated parameters to identify the “preferred” alternatives in every trial and
compared those to the actual choice outcomes. RT-estimated parameters were able to
explainahighproportionofchoicesinthedatasets(socialchoice𝛼:79%respectively(p<
0.001); social choice𝛽: 80% (p < 0.001); intertemporal choice: 76% (p < 0.001); risky
choice:77%(p<0.001);p-valuesdenotetwo-sidedWilcoxonsignedranktestsignificance
at the subject level, comparing these proportions to chance). For a stricter test, we
calculated an average of all indifference points for each experiment, which roughly
corresponds to the mean of the experimenter’s prior parameter distribution, andmade
choice predictions for each subject using this single value. TheDDMaccuracy rates beat
thisbaselineintwooutoffourcases(fortheintertemporalchoiceandsocialchoice𝛽,p<
0.05,two-sidedWilcoxonsignedranktest).
5.5.2.Subject-levelDDMestimation
In addition, we estimated the DDM for each subject separately, assuming individual
variabilityintheboundaryparameter𝑏,non-decisiontime𝜏,anddriftrate𝑧.Asdescribed
intheprevioussection,thiscausesidentificationproblemsforcertainsubjects.Therefore,
inthissectionweexcludesubjectswhoseutility-functionparameterestimatesfelloutside
the range of indifference points employed in each experiment. After excluding these
subjects (2/30 and 2/30 in the social choice dataset, 16/39 in the intertemporal choice
dataset,and7/37intheriskchoicedataset)weobtainedcorrelationsthatshowed
30
Figure 5. The DDM estimates of subjects’ utility function parameters, fitting themodel to each subject individually. Subject-level correlation (Pearson) betweenparametersestimatedfromchoicedataandRTdatausingthedrift-diffusionmodel(DDM),including only the subjects with parameters estimated within the range of indifferencepointsintheexperiment(reddottedlines).Thesolidlinesare45degreelines.
improvementoverthepreviousDDMestimationinthreeoutof4cases,withcorrelations
ashighas0.74(Figure5,p<0.01).
Forthe𝛼parameterinthesocialchoicetask,thecorrelationwasinsignificant(Figure5,
p = 0.65), most likely due to the small number of trials (only 48 trials to estimate 4
parameters)andthetightdistributionofsubjects’indifferencepointsinthattask.
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
−1.0 −0.5 0.0 0.5 1.0 1.5
−1.0
−0.5
0.0
0.5
1.0
1.5
Social choice (α)
α estimated from choices
α e
stim
ated
from
RTs
r = 0.09p = 0.65
●●
●
●
●
●
●●
●
●
●
●●
●● ●
●
●
●
●
●
●●●
●
●●●●
●
−1.0 −0.5 0.0 0.5 1.0 1.5
−1.0
−0.5
0.0
0.5
1.0
1.5
Social choice (β)
β estimated from choices
β es
timat
ed fr
om R
Ts r = 0.74p < 0.001
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
● ●
0.00 0.01 0.02 0.03 0.04 0.05 0.06
0.00
0.01
0.02
0.03
0.04
0.05
0.06
Intertemporal choice
k estimated from choices
k es
timat
ed fr
om R
Ts
r = 0.63p < 0.000
●
●
●
●
●
●●●
●
●
● ●
●●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●●●●
●
●
●
●●
0 2 4 6 8
02
46
8
Risk choice
λ estimated from choices
λ es
timat
ed fr
om R
Ts
r = 0.53
p = 0.002
31
Theseresultshighlighttheimportanceoftaskdesign. Ifthetask’s indifferencepoints
span the range of indifference points in the population then the DDMmethod can yield
quiteaccurateutility-functionparameters.
5.6.AlternativeapproachestoutilityestimationfromRTs
The DDM may seem optimal for parameter recovery if that is indeed the data
generatingprocess.However,severalfactorslikelylimititsusefulnessinthissetting.The
DDMhasseveral freeparameters thatare identifiedusing featuresof choice-conditioned
RTdistributions. Identification thus typically reliesonmany trials andobserving choice
outcomes. Withoutmeeting these tworequirements, theDDMapproachmaystruggle to
identifyparametersaccurately.Belowweexplorealternativeapproachestoanalyzingthe
RTs.
5.6.1.TopRTdecilemethod
One alternative approach is to focus on the longest RTs: for instance, Schotter and
Trevino (2014) successfully use the longest RT over a number of trials to identify a
decisionthresholdinasimpleglobalgame.AslongRTsindicateindifference,theycouldbe
usedtoidentifytrialswherethesubjectiveutilitiesofthetwooptionswereequal,andthus
obtainanestimateof theutility functionparameter fromthe indifferencepoints in those
trials.Hereweexplorewhetherthismethodworksinourdatasets.
32
Figure6.Single longestRTsareanoisypredictorof indifference.RTinsecondsasafunctionofthedistancebetweentheindividualsubjectutilityfunctionparameterandtheindifference point on a particular trial; gray dots denote individual trials. Red trianglesdenotetrialswiththehighestRTforeachindividualsubject.
Figure 6 plots RTs as a function of strength-of-preference for every trial in the
experiments. It is easy to see thateven though formanysubjects the single slowest trial
providesagoodsignalof“true”preference(asdefinedbythechoice-basedestimation),for
others the longestRT is far from indifference.This is especially true for the risky choice
data (possibly due to the two-parameter utility function or the unrestricted RTs). This
suggestedtousthatitmaybebettertousemorethanthesingleslowesttrial.
●
●
●
● ●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
● ●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
● ● ●●
● ●
● ●
●
●
●
●
●● ●
●●
●
● ●●
●
●
●
●●● ●
●●●
●
●●
●
●
●
●
●
●
●●
●● ●
●
●
●● ●
●
● ●●● ●
●
●●
●
● ●●
● ●● ●● ●
●●
●●●
●
●
●
●● ●●
●
●
●
●
●
●
●●
●● ●
●
●●
●
●
●●
●●●
●
●
●
●●
●
●
●●●
●
●
●
●
●●
●●
●
●
●●
●● ● ●
●●
●
●
●
●
●● ●
●
●
●●
●
● ● ●●
●●
●
●
●
●● ●
●
●●
●● ●●
●
●
●
●
● ●●● ● ●●
● ●
●●● ●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●● ●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●●●
●●
●●
●●●
● ●●●
●●
●
●
●
●●●
● ●●
●●
●
●● ●
●● ● ●
●
● ●
●●● ●
●
●
●●
●
●
●●●
●
●
●●
●
●●
●
●
●●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●●
●
● ●
●
●● ● ●
●
●●
●
●●
●●
●● ●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
● ●
●●
●
●
● ●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
● ●●●
●
●
●
●
● ●
●
●●
●
●●
●
●● ●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
● ●
●
●
●●●
●
●
●●
●●
●● ●
●●
●● ●●
●
●
●●
●
●●
●● ●●
●●
●
● ●
●
●●●
●
●
●● ●
●
●
●
●
●
● ●●●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●● ●
●●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●●
●●● ●
● ●●
●
●
●●
●●
●●
●
●
●●
●●●
●
● ●●
●●
●
●
●
●
●
● ●● ●●
●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●●
●●●
●●
●
●
●
●
●
●
●
●●
●
●●●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●●
●
●
●●
●
●●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●● ●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5
24
68
10
Social choice (α)
subject α − indifference α
RT [s
]
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
● ●
●●
●
●●
●
●
●●
●●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●●
●
●
●
● ●
●
●
●
●●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●●
●
●
●
●
●
● ●●
●
●●
●●
●
●
●
●●
●●
●● ●
●
●
●
●
●
●
●●
●
●●●
●●
●
●●●
●
●
●●
●
●●●● ●
●
●
● ●
● ●●
●
● ●
●●
●
●●
●
●
●
●
●
●
●●●
●●●
● ●
●
●
●
●●
●
●
●●●
●● ●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●● ●
●●
●
●
●
●● ●
● ● ●●
●
●●●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●●●
●●●
●
● ●
●
●●
● ●
●
●
●
●
●
●
●●●
●
●
●
●●
●●
●
●
● ●● ●
●
●
●
●●
●
●
●
●
●
●● ●●
●
●● ●●
●
●
●
●●
●●
●
● ● ●●
●●
●● ●●
● ● ●
●●
●●●
●●
●
●●
●● ●●●
● ●●
●
●●
● ●●
● ●●●
●●●
●●●
● ●● ●
●
●
●
●●
●
●●●●
●
●
●
●
●●●
●
●●
●
●
● ●●
●
● ●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●● ●
●
●
●
●
● ●●
●
●
●
● ●●●
●●
●
●
●
●
●
●
●
●
●●
●●
● ●●
●
●
●
●
●●
●
● ●●
●
●●
●
●
●●
●●●
●
●
●●
●
●
●
●
●●●
●
●
●
●●●
●●
●●● ●
●
●
● ●
●
●
● ●●
●●
●
●
●●
●
●
●●
● ●●
●
● ●
●●
●
●
●
●●
●● ●
●●
●
●
●
●●
●
●
●●
●
●
●●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●●
●
●
●
●●
●
●●
●
●
●●
●
●
●●
●●
●●●
●
● ●●
●
●
●
●●
●●
●
● ●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
● ●
●
●●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●●
●
●
●●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
● ●
●
● ●
● ●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
● ●
●●
●●●
●●
●
●●
●
●
●●
●●●
●
●
●
●
●
●● ●
●
●
●
●●
●
● ●
●●●
●
●●
●●
●
●●
● ●● ●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●●
●●
●●
●●
●●●
●
●
●
●●
●
●●
●
●
● ●
●
● ●
●
●
●●
●
●●●
●
●
●
● ●●
●●
●●
●
●● ●● ●
● ●
●
●●
●●
●●
●
●●●
● ●● ●
● ●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●●●
●
●●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
●●
●● ●
●● ●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
● ●
●
●●
●●●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●●
●
●
●
●●●
●●
●●
●
●●
●
●
●● ●
●●
●●
●
●
●
●●
●
●●
●
● ●●
●●●
●
●●
●
●
●●
●
● ●●●
●●●
●
●
●●●
●
●
●
●
●●
●●●●
●
●●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●●
● ●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
● ●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●●
●
●
● ●●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●●
● ●
●
●
●●
●
●
● ●
●
●
●
●●
● ●●
●
●
●
●
●
●
●
●●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●●●
●●
●
●●
●
●
●
●●
●●
●● ●
●●
●●
●
●●
● ●
●
●●
●
●
●●
●●●
●
●
●
●
●●●
●●
● ●
●
●
●●●● ●
●●
●●
●
●●
●
−2 −1 0 1 2
24
68
10
Social choice (β)
subject β − indifference β
RT [s
]●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●● ●
● ●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●●
●
●
●●●
●
●●
●●
●
●
●
●
●
●●
●●●
● ●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
● ●
● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●● ●
●
●●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●●
●●
●
● ●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
● ●
● ●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●●●●
●
● ●
●●
●
●●
●
●
●
●● ●
●● ●
●
●
●●
● ●
●●
●
●
● ●●
●●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●●
● ●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
● ●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●●
●
●
●●
●●
●
●
●
●
●
● ●
● ● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●●●● ●
●
●
●
●●●
●
●●
●
●
●
●●
●●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●● ●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
● ●
●
●●
●● ●
●
●
●
● ●
●
●
●●
●
●
●
●●
●● ●●
●●
●
●●
●●●
●
●
●
●
●●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●●
●
●
●
●●
●
●●
●
●●
● ●●●
● ●●
●
●
●●
●
●
●●
●
●● ●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
● ●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●● ●
●●
●
●
●
●
●●
●● ●
● ●●
●
●
●
●
●
●●
● ●●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●● ●
●
● ●
●
●●●
●
●
●●
●
●
●
●●
●●
●
●●
●
●●
●
●
● ●
●●
●●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
● ● ●●
●
●
●
●
●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●●●
●
●
●●
●●●
●●
●●
● ● ●
●●
●
●
●●
●
●
●●
●
●
●
●●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●●●
●●
●
●●
●
●●
● ●●
●● ●● ●
● ●●● ●●● ●
●●
●●●● ●
●
●
●●
●
●●●●
●●
●●
●●
●●●
●
●●
●●
●
●
●
● ●
●●
●
●
●
●
●
● ●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●●
●●●●
●●
●
●●●
●
●
●●
●●●
●
●●
●
● ●
●
●●●
●●
●
● ●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●●
●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●●
● ●●
● ●
● ●●
●
●●
●●
●●●
●
●●
●
●
●
●
●●
●
●●
●
● ●●
●●
●
●
●
●●
●●
●
●●
● ●● ●
●
●
●
●
●
●●
●
●
●● ●●
●
●●
●●
● ●
●●
●
●
●
●
●
●
●
●
●●●
●●●
● ●
●
●●
●●●
●
●●
●● ●●
●●●
●
●
●●
●●
●
● ●●
●
●
●
●
●●
●
●
●●
●
●●●
●
●
●●
●
●●
●
● ●
●
●●
●
●
●
●
●
●● ●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●
●
● ●●
●
●●●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
● ●
●
●
●●
●
●
●●
●●
●
●●●
●●
●
●
●● ●● ●
●
●
●
●
●
●
●●●
●
●
●●
●●
●
●●
●●
●
●●
●
●●●
●
●
●
●
●●●
●
●
●●
●●
●
●●
●●
●
●
●
●
●●
●
●●●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●
● ●
●
●
●
●●
● ●
●●
●● ●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●●
●
●●●
●
●
●●●
● ● ●●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●●
●●●
●
●
●
●
●●
● ●
●
●
●●
●●
●●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●●●
●
●●
●● ●
●●
●
●
●●
●
●●●
● ●
●●
●
●
●
●
● ●
●
●
● ●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●● ●
●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●●●
●
●
●
●
●
●●
● ●
●
●●
● ●
●
●
●
●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●●
●
●
●
●
● ●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●●
●●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●●
●
● ●
●
●
●
●
●
●
●●
● ●
●
● ●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●●
●
●
● ●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
● ●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
● ●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●●
●
●
●●
●
●
●●
● ● ●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
● ●
●●
●
●
●
●●
●
●
●●
●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●●
●
●
●●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
● ●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
● ●
●●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●●
●
● ●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●●
●●
●
●
●
●
●
●
●
●●●●
●
●●●●
● ●●
●
●
●
●
●
●
●
●
●● ●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●● ●
● ●
●●
●
●●
●
●●
●
●
●●
●
●
●
● ●
●● ●
●●
●
●
● ●● ●
●
●
●
●●
●
●●
●
●●●
●
●
●
●●●
●
● ●
●
●●
●
●
●
●
●
●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●●
●
●
●
●
● ●
●
●●●
●
●
●
●●●
●
●●
●
●
●● ●●
●●● ●
●●●
●●
●●
●
●
●
● ●●
●
●
●●●
●
●●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●●●●
●
●
●
●●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
● ●
●
● ●
●
●●
●
●
● ●
●
●
●●●
●
●●
●
●●
●
●
●
●
●●
●
●
● ●●
●
●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●●
● ●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●● ●
●
● ●
●
●●
●
●
●
●●
●●
●
●●
●
●
● ●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
●●
●
●
●
●●
●
● ●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
● ● ●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●● ●
●
●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●
● ●
●
●
●
●
●●
●
●
●
●
● ●●
●
●●
●●●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
●
●
● ●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●●
●
●
●
●
● ●●
●●●
●
●
●
●
●
●
●
●●
●● ●● ●●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
● ●●
●
●● ●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●●
●●
●
● ●●
●
●
●
●●
●●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
● ●
●
●
●
●
●●
●
●
●
● ●
●
● ●
● ●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●● ●
●
●
●
●●
●●
●
●●
●
●●
●
●
●●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●●
●
●
●
●
●
●●
●
●
●
● ●
●●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●●
●
●
●●●
●
●●
●
●
●●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
● ●
●●
● ●
●
●
●
●
●
●●●
●
●
●
●
●
●●●●
● ●
●
●
●
●●
●
●
●
●
●●●
●
●●
●
●●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
−0.04 −0.02 0.00 0.02 0.04
0.5
1.0
1.5
2.0
2.5
3.0
Intertemporal choice
subject k − indifference k
RT [s
]
●
●
●●
●●●
●●
●● ●
●●
●
●● ● ●
●
●● ● ●● ● ●●
● ● ●● ●●
● ●●● ●●●●●
●
●
●● ● ●● ●●●●●●●
●● ● ●● ●●●●●●
●
● ●●
●
●
●
●● ●
● ●●
●
●●
●● ●
●
●●
●
●● ●●● ●● ●●
●
●
●
● ●●●● ●●
●● ●●
●
●●● ●●●●● ●● ●●●
● ●●●
● ● ●●● ●●● ●● ●● ●●
●● ●●● ●●● ●●● ●● ●● ● ●● ●● ● ●●●● ●● ●●● ●●● ●●●●● ● ●●●●● ●●●●●● ● ●●
●● ●●● ●● ●
●●●● ● ● ●●● ● ●● ● ● ●●●
●●
●●●
●●
●
●
●●
●●●
● ●●●●● ●●●
●
●
●
●●●
●●●
●
●
●
● ●●
●●●●
●● ● ●●
● ●●●
●●●
●● ● ●● ●
●● ●
●●
●
●●●●●
●● ●●●
●●● ●
●
● ●● ●●● ●●●●●● ●
●
●●● ●●●●
●
● ●●●
●
● ●● ●●● ● ●●●
●●
●
●
●
●●
●
● ●● ●●●●●
●●●● ●●●● ●●●●
●
●
●●
●
●
●
●● ●●●
●● ● ●●
● ●●
●
●
●● ● ●● ●●●● ●● ● ●● ●●● ●●● ●●●
● ●● ●● ●●●
●● ● ●●●●
●
● ●●
●
●●● ●● ●●
●●●
●
● ●
●●
●
●
●
●
●● ●
●
●
●●
●
●●
●
●●
●
●
●● ●● ●
●
● ●
●
●●● ●● ●● ●
●●
●●●
●
●
●
●● ● ●
●
●●●
●●●
●●● ●●
● ●
●
● ●
●
●●●
●●●
●
● ● ●
● ●
● ● ●●
●●●
● ●●
●
●
●● ●
●●●●
●●●
●
● ●●●
●●
●●
●●● ●● ●●
●
● ●●
●
●
●●
● ●●
●
●●
●
●
●
●●●● ●●
●
●●
●
●
●●●●
●
● ●●● ●
●
●
●●●
● ●●
●
●
●
●● ●● ●
●
●●●
●●●
● ●●
●
●●●● ● ●
● ● ●●●
●
●●●
●
●● ●● ●●
●
●● ●●● ●●●● ●●● ●● ● ●● ●
●
●●●● ●●●
●● ● ●
●
●●
● ●●●
●
●●
●
●● ●●● ●● ●● ●●
● ●●
●
● ● ●●●●
●● ●● ●● ●●
● ●●●●● ● ●
●●●● ●●●●
●● ●●●●● ●● ●● ●
●● ●●● ●●
●●
●● ●●● ●● ●●●
●●● ●●
●●
● ●●●● ● ●●●● ●●
●●● ●●●
●● ●●
●● ●● ● ●● ●● ● ●●●● ●
●●●●● ●●●
●●● ●● ●●● ●●● ●●● ●
●●●●
●● ●●●● ●●●●● ●
●●● ●● ●● ●
●●●● ●●
●● ●
●● ●●● ●●
●● ● ●●● ●●● ● ●●●
●●●●●
●
●●
●
● ●
●
●● ●●●
●●●●
●●● ●●
●●● ●● ●●●●●● ●● ●●● ●
●●●●
●● ●● ● ●● ●
●
●●●
● ● ●●●● ●● ●● ● ●●
●●● ●● ●
●●
●
● ●● ● ●●●● ●●● ●
●
●●● ●●
● ●●●
● ●●
●
●●
●●● ● ●● ●●
● ●● ●● ●
●
●● ●●
●●
● ●● ●
●●
●● ● ●●●●● ●●●● ●●● ●
●●
●●●●●● ●●
●
●● ●●●●●●● ●●
●●●● ●● ● ●●● ●●
●●●
●●
●●●●● ●●
● ●●●
●
● ●●● ●● ●● ● ●● ● ●● ●●●● ●●●● ●●● ●
●
●
●
●
●
●
●●●
● ●●
●
●
●●●
●●●
●●●
●
●●●
●
●●
● ●
●
●●●●●●
●● ●●●
● ●●●●
●
●●● ●● ●●●●
●●
●●
●
●
●
●
●●● ● ●● ● ●
●
●
●
●
●●
●● ●
●
●
●
● ●●
●
●
●● ●●● ● ●●●
●
● ●●
●
●
●●
●● ●●
●
●
●●●
●● ● ● ●● ●●●
●
●
●●●● ●● ● ●● ●●
● ●●●●● ● ●
●● ●●
● ●● ●
●●● ●●●●
●
●● ●
●●
● ●● ●● ●
●●●● ●●
●
●●● ● ●●● ●●● ●● ●
● ●●● ●● ● ● ●●● ● ●● ● ●●● ●●● ● ● ●●●●●
● ●
●
●●
●
●
●●●
● ●●
●
●●●●
●
●
●
●
●
● ●● ●
●
●●
●●
●
●
●
●
●
●●● ●●●
●
●●
● ●
●
●●
●
●● ●●
●●
●● ●● ● ●●
●
●●●
●●
●
●
●●●
●● ●
●●
●
●●
●
●
●● ●● ●●● ●●●
●
● ●
●
●● ●
●
●
●● ●●
●
● ●
●●●
●●●●●
●●●●●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ● ●●● ●●●● ●● ●●
●
●
●
●
●●
● ●●
●
●● ●● ●●●
●
●
●●
●
●● ● ● ●
●● ●● ●
●
●
● ●●● ●
●●●
● ●● ● ●●●● ● ●
●
●● ● ●
●●●
●
●●●●●
●
●
●
●
●
●
● ●
●
●
●●
●● ●
●●
●
●
●●
●●
●
●
●●
●
● ● ●●●
●
●
●
●●
●
●● ●●● ●●● ●●
●
●
●●●●
● ● ●●● ●
●
●●●●●● ●●● ●●
●
●
●
● ●
●
●●●●● ● ●●
●●
●●
●●
●
●● ●
●
●
●● ●● ●● ●●
●●●
● ●
●
●●
●
●●● ●●● ●
● ●
● ●
●
●
●
●●
●
●
●●
●●
●●● ●
●
●
●
●●
●
●●
●
● ●● ●●●● ●●
●●
●●
● ● ●●
●
●
●
●●
●●●●
●●● ●●
●●
●● ●●●
●
●
●●●
●
●●● ● ●
●
●
● ●● ●●
●● ●
●
●
●●
●●● ● ●● ●
●
●
●
●
●
●●
●
●
● ●● ●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●●
● ●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
● ●
●
●●●
●
●
●●
● ● ●
●
●●●
●
●
●
●●●
●●●
●●
●
●●●●●●● ● ● ●
●●● ●
●
●●● ●
●
● ●● ●●●●
●● ●
● ●●
● ● ●
●●●
●
●● ●●● ●● ● ● ●●●●● ●●●
●
●
●
●
●● ●● ● ●●●●●●
●
●●●
●●●● ●● ●● ●● ●●●● ●
●●
●
●
●●●
●
●
● ●● ●●
● ●
●
● ●● ●●●●
● ●●● ● ●●●● ●● ●●
●
●●● ●● ●●●● ● ●●●●● ●●● ●
●●● ●● ●
●
●●
● ●●●
●●●
● ●● ●● ●●● ●
●●● ●●●●●●●
●●●● ●● ●●●●●●●●● ●
●
●
●●●●●
●●●
● ● ●● ●●●● ●● ●●
● ●● ●●● ●●
● ●●● ●● ●●●
●● ●●●● ● ●
●●●● ●● ●● ●●● ●●● ● ●●●●
●●●●●
● ●● ●●●● ●●● ●
●●
●● ●● ●● ●●
●●●●● ●●
●●● ●●●●● ●●●● ●
●●
●
●●●● ●●●● ●
●●●●● ●●
●● ●●● ● ●● ●●
●●●● ●
● ● ●●
●●●●
●●●●●● ●●●●●
●
●
●●
●
●
●
●●
●
●
●●●
●
●
●
●● ● ●
●
● ●●
●
●
●
●
●
● ●●
●
●● ●● ●●●●●
●●
●●
●
● ●●
●●
●
●
●
● ●
●●
●● ●● ●
●●● ●● ●
●
●
●●
●
●
●●●●
●
●
●●●
●●
●
●●
●●
●
●
●
●
●
● ●●●
●
●●
●
●●
●
● ●● ●●● ●
● ●●
●
●
●●●
●● ●
●● ● ●●
●
●● ●
●
●
●
●
● ● ● ●●● ●
●
●●●
●
●
●●● ●
●
●
●
●●
●
●●
●●
● ●●●
● ●
●
●● ●● ● ●●
●●●●
●
●
●
●●●
●●
●●
●
●
● ●
●
●● ●● ●
●●
●
● ● ●●
●● ●●
●
● ●
●●●
●
●●●
●●
●
●●
●●
●●● ●
● ●●●●●
● ●●
●● ●●● ●
●
●●
●
●
●●●
●
●
● ●●
●●
●
●●●
●
●●
●
●
●●●
●●
●
●● ●●● ●
●
●● ● ●
●
● ●●●●
●
● ●
●
●●
● ●
●
●●●
●
●
●
●
●
●
●●●● ●
● ●● ●
●
●●
●
●
●
●
●●
●
● ●●●
● ●
●
●● ●●
●●●
●●
●
●
●
● ●●● ●
●●
●
●●
●
●
● ●
●●
●●●●●●●
● ●
●
●●
●● ●
●
●
●●●
●● ●●● ●●●●●
●● ●
●
● ●●
●
●●
●●
●● ●●
●●
●
●● ●●
●●
●● ● ●
●●●●●
●
●●
●● ●● ●● ●
●●
●●
●
●●
●
●
●
●
●
●
●●
●
●●●
●
●
● ●●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●● ●●
●●
●
● ●● ●● ●
●
●
●
●
●
●
●
●
●● ● ●●●
●
●●●● ●●
●
● ● ●
●
●● ●
●
● ●●
●●
●●
●● ●● ●
●
●
●
● ●●●
●
●●● ● ●●
● ●
●
●●
●
●
●●●
●
●
●●● ●●●
●
●●
●●
● ●
●
●● ●●
●
●
●●● ●●
●
●●
●
●
●
●
●●●
●●●
●
●
●
●●●
●●
●
●●●●
● ●●●
● ● ● ●●
● ●● ●
●
● ● ●
●
● ●
●
●
●
● ●●●● ●
●
●
●●
●●
●
● ● ●●
●
●●
●●●
●
●
●
●
●
●
●●●●
●●
●●
●
●
●●●●
● ●●
●
● ●
● ●
●
●●
●
●
●
●
●
●●● ● ●
●
● ●
●●
● ●● ●
●
●
●●
●● ●
●●
●● ●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●●● ● ●
●●
●●
●●
● ●●
●●
●●
● ●●
●
●●●
●
●●
●
●
●●
●
●●● ●
●
●● ●
●
●
●●
●
● ●
●
●●
●●
●
●● ●
●
●
●
●
●
●
●
●●● ●● ●
●
●●● ● ●
●●
●
●● ●●
●●
●●●●
●
●●● ● ●
●
● ●
●
●●
●●
●● ●●● ●●●●● ●●
●
● ●●●● ●●● ●●●● ●● ●●
● ●●● ●●● ●●● ●●●●●● ●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●●●
●●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●●
●●
●
● ●
●
●●
●●
●
●
●
●●
●
●
●
●●●●
●
● ●
●
●
●●
● ●
●●●
●●
●●
●●●
●●
● ●●
●●
●● ●●●
●●
●
●
● ●● ●● ●●
●
●
●●●●
●●●
●●
●
●●
●
● ●
●
●●
● ●● ●●
●●● ●●
●● ●●●
●●
●
● ●●●●● ●● ●●● ●● ●● ●● ● ●● ●●●● ●●●● ●●●
● ● ●●● ● ●● ●● ● ●● ●●● ●● ●
●●●● ●● ● ●● ●●●● ●● ●
●
●●
●
●●
●
●
●●
●
●●
●
●
●
●●●● ●● ●●● ●●
●
● ●●●●
● ●●
●
●●●●●● ●
●●●● ●●
●●● ●●● ●● ●
●●
●
●●●
●●
●● ●●● ●●●
●● ●●
●
●●●●●
●● ● ● ●●● ●● ●●
●●● ● ●●●● ●● ●●
● ●●
●●●
●
● ●●●
●
●● ●
● ●
●
●
●
●●
● ●●●
●●●●● ●●●
●●● ●●
●
●●● ●●●● ●●●● ●● ●●●
●● ●● ● ●●●●
●●●●●● ●● ●● ●● ●● ●●●● ●●●● ●● ●●
●
●● ● ● ●●●● ●●● ●●● ● ●●●●
●
●●● ●
●
●
●● ●●
●
●●●
●●●●
●● ●● ●
●●●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●●●●
●
●●
●
●● ●●● ● ●
● ●●● ●● ●●
●
●
●●
● ●●● ●●●
●●●
●
●●● ●
●
●
● ●●
● ●● ● ● ●●
●● ●
●
●● ●● ●●●
●
●● ●
●
● ●●●●● ●●
●● ●● ●● ●●
●
●● ● ● ●●
● ●
●●
●●●●● ●
●
●● ●●●●●
●
●● ●● ●● ●●● ●● ●●● ●●
●
●●●● ●
●
●● ●●●
● ●
●●
● ●● ●● ●●●
●●
●●
●●
●● ●●
●●● ●
●●●● ●●
● ● ●● ● ●●● ●●●
●●
●●
●
●
●
●
●●
● ●
●
●
●
●
●● ●●
●
●
●
●● ●
●●●
●
● ●
●
●
● ● ● ●●
●
● ●
●
●
●
● ●●●
●●
●
●●
●●
●●
●●
●●●
● ●
●
●
●
● ●●●●●
●●
●●
●●●
●● ●
●
●●
●●
●
●●
●
●
● ●●●
●
●●
●
●●
●
●
●●
●
●●●
● ●●
●
●●
●●
●●●●● ●
●
●● ●
●
●
● ●●
●
●
●
●● ●● ●●
●
●●
●
●●●
●
●● ● ●●
●●●
●
●●●
●
●
●●●
●
● ●
● ●
●
●
●●
●● ●●●● ●●
●●
●
●● ●●● ●
●●
●
●●
● ●●
● ●●●
● ● ●● ●
●
● ●●
●●
●●
●●●●●
●●●
●●
●●●
●●
●●●
●
●●
●●●
●●
●
●●
●● ●● ● ●●●
●● ●● ●
●● ●
●
●● ●● ●●
● ● ●●
●●
●
●● ● ●●
●
●● ●● ● ●●●
●● ●●
●
●●
●●●●● ● ●
●
●● ●● ●● ●●
●● ●● ●● ●●
●
●
●
●
● ●
●
● ●
●●
●●
●
● ●● ● ● ●●● ●●●● ● ●
●● ●●
●
●● ●● ●●●
●● ●●●●●●
●● ●●
●●●
● ●
● ●●● ●● ●
●●
●
●● ●● ●●● ●
●●
●●●
●
● ●●●●
●●
●
● ●●● ●●● ●●● ● ●
●● ● ●● ●●
●● ●●● ● ●● ● ●●● ● ●●● ● ●● ●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●●
●
● ●
●
●
●
●
● ●●
●
●
●●
●
●
●●
●●
●
●
●●●
●
●●
● ●
●
●
●●
●●
●
●
● ●
●
●
●
● ●● ●●●●
●
●●
●
● ●●
●
●
●
●
●●●● ●
●
●●●
●
●●●●●●●
●●
● ●●● ●
●●●
●
●●●
●●● ●
●●
●
● ● ●● ●●●
● ●●●●
●●●●●
●
●
●●
●
●● ●● ●●●● ●● ● ●●●● ●● ●●
●●
● ●● ●●
●
●● ●● ●● ● ●●●● ●
●
●
● ● ●● ●●
● ●●● ●●●●
● ● ●● ● ●● ●●● ●●●●● ● ●● ●● ● ●●●●● ● ●●●● ●● ● ●●
●
●
●
●●● ● ●●●●● ● ●● ●● ●● ●●● ●
●● ●● ●●●
●● ●●● ● ●● ●●
●
● ●●●●● ●● ●●●●● ● ●
●
● ●●
●
●●● ● ●●●●● ●● ●●●
●●
●● ●●●● ●●● ●●●●
● ●
●
● ●●●● ●●●● ●
●
●● ●●● ●●
●●● ●●●● ●●
●
●● ● ●● ●
●●
●
●
●
●
●
●
●●● ● ●●●
●● ● ●
●●
●
●
●
● ●●
●●●
●●
● ●●
●●
●
●
● ●
●
●●
●
●●
●
●●
● ●
●
●●●
●●
●
●
●
●
●
●
●●
●
●
●
● ●●●
●
●●
●
●● ●
●●● ●●●
●
●●
●
●
●
●● ●●●
●●
●● ●
●●
● ●● ●●● ●
●●
● ●
●●
● ●
●●●● ●●
●●
● ●●●
●●
● ●
●●● ●
●
●
●●
●●
●● ● ●
●●● ●
●
●●● ●
●
● ●
●
●● ●
●●
●● ●
●●
●●
●
●
●
●●● ● ●●●
●●●
●●
●●●●
●
●●
●
● ●●●
● ●● ●●
●
●
●●
●●
● ●●● ● ● ●●
●
●●●●
●
●●
●
● ● ●●●
●
●
●● ●●●●●● ●●
●
●● ●●● ●● ● ●●
●
● ●●●●
●
●● ●●●
●
●
● ● ●● ●●●●
● ●●
●
●● ●● ●●
●● ●●
●
●
●●● ●●● ●●● ●●● ●●
●●● ●
●● ● ●● ●●● ● ●
● ●
●
● ●● ●
●●●● ●●●●● ● ●●● ●●● ●● ●● ●● ●● ● ●●●●
● ●●● ●● ●● ●● ●●
● ●●● ●● ●●● ●● ●●●
● ●●● ●●
●
●
●
● ● ●●● ●●● ●● ●●
●●
●
●
●●●
●
●●
●
●●● ●●● ●
●●●● ●●
●● ●
●● ● ●
●●●● ●● ●●● ●●
●
●
●
●
●●
●●●●
●
●
●●
●●
●
● ●
●
●●● ●
●● ●
●
●
●●
●●
●
●●
●
●
●●●
●
●●● ●
●
●
●
●
●
●
●
●● ●●
●
●●●
●●
●
●●
●
●●
●●
●●
● ●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●●●
●
●●
●
●
●
●●● ●
●
●● ●●
● ●●● ●
●
● ●●
●
●●
●●● ●
● ●● ●● ●●●
●●
● ●●●● ●●●
●
●
●
●●
●
● ●●
● ●
●
●● ●● ● ●
●
●● ●
●
● ●●●
●
●●
● ● ●●
●
●
●●●
● ●
●●
●●
●
●
●
●●
● ●●
●
●
●●
●
●
● ●●●● ●
●●
●
● ● ●●● ●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●
●●
●●●
●●
●
●●
●
●●
●
●
●
●
●●
●
● ●
●
● ●
●
●
●
●
●●
●●
●●
●●●●
●
●
●
●
●
● ●●
●
●
●●
●
●●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
● ●● ●●
● ●●
●
●
●
● ●● ●●
●
●
●●
●●
●● ● ●● ●●
●
●● ●●
● ●●● ●●● ●● ●
●
● ●●●
●●●
●●
● ● ●●
●●
●
●● ● ●● ●●
● ● ●●●●● ●●●● ●●●
●
●
●●● ● ●
●●
●●
●
●●
●●● ●
●
● ●●
●
●
●●
●
● ●● ●●
●
●●
●●
●●●
●●
●●
●●
●● ●
●
●●●
●●
●●
●● ●●● ●●●● ●
●
●● ●●
●
●
●
●●●
●
●
●
●
●●
●● ● ●
●● ●●●●● ● ●● ●
● ●●●● ●
●●
●●●
●
●●
●●●● ●
●
●
●●●
●
●●●●●●● ●●
●●●●●● ●●●
● ●●● ●●● ● ● ●●● ●● ●● ● ●● ●● ●
●
● ●●●
●
●●● ● ●●●● ●
●●●●●● ● ●
●●
●
●
●●● ●●●●
● ●
●
●●●●●●
●● ●● ● ●● ●● ●●
●●●
●
● ●●●
●●● ●●● ● ●● ● ●● ●●
●
●
●● ●
●●●
●
●●
●●
●
●● ●●●
● ● ●● ●●● ● ●●● ●●
●
●
●●
●
● ●●●● ● ●●●
●●
●
●● ● ●
●●● ●●●
● ●●
● ●● ●
●
● ●● ●● ●●●
●
● ●
●
● ● ●●●
●
● ●●●●●●● ●●
●
● ●●
●
● ●●●● ●● ●
● ●● ● ●
●
●● ●●
●
● ●●
●● ●●
●
●●● ●●●● ● ●● ●●
●
●● ●● ●●● ●
●
●● ●● ●●
● ●
●
●
●
●●●●● ● ● ●●●●● ●●● ●●
●
●
●●
●●● ●●● ●●●●
● ●
●
●● ● ●●
●
●● ●●
●
●●
●● ●●●
●●●
● ●●
● ● ●● ●●●
●
●● ●● ●
●
●●● ● ●
●●
●●●
●●
●● ●●●● ●
●
● ●●
●
●●●● ●
●
●
●●●
●●
●●●● ●
● ●●
●
● ●● ● ●●
●
●●
● ●●● ●●● ●
●
●●●
● ●●●●
●● ●●●●●●
●●● ● ●
●
●
●●●●●
●
●
●● ● ●●●
●
●
●
● ●●
●● ●
●
●
●●● ●●● ●●● ●
●●●
●● ●● ●●● ● ●
●●●
● ●●
●●● ●
●
●
●
●●
●●
●
●
●
●● ●● ●●
●
●
●●●●
●●● ●●
●●
●●● ●
●●
●●● ●● ●● ● ●● ●●
●●●●● ●● ●●● ●
●●
●●●●● ● ●●
●
●●●●●
●
●● ● ●●●●●●● ●● ●
●●
●●
●●
●●
●●
●● ● ● ●
●●
●●
●
●
●●
●
●
●●
●●● ● ●
●
●●
●
●
● ●
●● ●●●
● ●●
● ● ●●● ●
●●●
●●●●●●●
●
●● ●
●●●● ●●●● ●● ●● ●●● ●● ●●
●● ●● ●●●
●●●
●
●●
●● ●●●●
●● ●● ● ●● ●● ●● ●● ●●● ● ● ●● ●● ● ●●● ●●● ●●● ● ●●●● ●● ● ●●● ●● ●
●●●● ●● ●●● ●● ●● ●● ●●
●●●● ●● ● ●●● ●●● ●●●●●●● ●● ●● ●● ● ●●●● ● ●●●● ● ●●● ●●● ● ●●● ●●● ●●●● ●●● ● ●
● ●●● ● ●●●● ●●●● ●
●
●
●●
●●
●
●
●●
●
●●
●
●●●●
●
●● ●●
●
●●
●
●● ●● ●●●●
●●●
● ● ●
●
●●
●
●●●
●
●●●●
● ●● ● ●
●
● ●● ●●
● ●
●●
● ●● ●
●
● ●
●
●
●
●
●
●
●
●●
●
●●●
●●
●●
● ●
●
●●
●●
●
●
●●
●
● ● ● ●
●●
●●
●●
●
●●
●●
●
●
●●●● ●
●●●●
●
●●
●●●●● ●
●
●
●
● ●
●
●●●●
●
● ● ●●● ●● ●● ● ●●●●●●
●
●●●
●●●● ●●● ●●●●● ● ●●●● ● ●●● ●● ●● ●●●
●
●
●● ●●●●
● ●●● ●●●
●●● ●
●●● ●
●●●●
●
●
●
●
●●
● ●●●
●
●
● ●
●
●●
●
●●●●● ●●
●
●●● ●
●
●●
●●● ●
●
●● ●●●●
●
●
●● ●●●
●
● ● ●
●
● ●●●●
●●● ●
●
●●●
●● ● ●
●●●●
●
●●
●●●
● ●
●
●●●
●
●
●●
●●
●●●
●
●
●● ●
●
● ●●●●●
●●●● ●●●● ●●● ●●
●●●
● ●
●
●● ●●●●●
●●
●● ● ● ●●
●● ● ●● ●●●
●
● ● ●● ●● ●● ●●●● ●
●● ●
●● ●●●●●● ● ●●● ●● ●● ● ●●● ● ●
●●
●●
●
●
●
● ●●● ●
●●
●
●●
●
●
●● ●● ●● ● ●●● ● ●●
●
●
●●
●●
●
●
●●●
●● ●●
●
●
●●
●●
●●
●●
●
●●
●
●●
●
●●
●
● ● ●
●●● ●
●
●●
●
●
●● ●●● ● ●●● ●●
●● ●
●●● ●● ●
●
●
● ● ●● ●● ●●●
●●●● ●● ● ●● ●● ●●●● ● ●●
● ●●
● ●● ●●
●
●● ●
● ●●
●●●●● ●● ●●● ● ●● ●● ●●
● ● ●●●● ● ● ●● ●●
●●
● ●●● ●● ●●●● ●●●● ●
●●● ● ●● ●● ●●●
●●●●
● ● ●●
●● ● ● ●
●● ●● ●●● ●●● ● ●●● ● ●●● ●
●●● ●●●
●● ●● ●●●● ●●●● ●● ●● ●●
●●
●
●●●
●● ●
●
●
●●●
●
●
●
●
●
●
●●
●●●
●
●
●
●● ●●
●
●●
●●
●
● ● ●
●
●● ●● ●● ● ●
●
●●
●●
●
●●●●●● ● ●●
●
●●
●●●
●
●
●
●● ●
●
●
●●
●
●●
●●
●
●
●
●●
● ●● ●
●
●● ● ●●●
● ● ●●
●
●●
●
●
●
●
●
●●
●
●●●
●●
●
●●
●● ●● ●●●
●●● ●●●●
●●
●
●●
●
● ●●●●
●
● ●●●
●
●● ●
●
●
●
●●
●
●
●●
● ●● ●● ●●
●
●
●●● ●
●●●●●
● ● ●
●
● ●●●● ●●●
●
●● ●● ●●
● ●● ●●● ●●
●
● ●●● ●● ● ●● ●
●
●
●●
●
●●●●
●●●●
●
●
●● ●
●
●● ●●●●
●
●
●
● ●●●●
●●
●●
●● ●●
●
●
●
●
●●●● ●
●
●●●
●●
●
●
●
●●●
●
●●
●
●●
●
●●●
●
● ●
●
● ●
●
●●
●●●
●
● ●
●
●●
●●
●
●
●
●●●
●
●
●●
●●●
●
●
●
●
● ●●
●
●
●
●●
●
●●●
●
●●● ●●
●●● ● ●● ●● ●
●
●
●
●●●● ●●●●
●
●●● ●●
●
●
●
●●●●●● ●● ●
●
●
●●●
●
●●
●
● ●● ●●
●● ●●● ●●
●●
●
●●● ●● ●●
●● ●●
●●
●● ● ●
●
●
●
●●●● ●●● ● ● ●
●●●●●
●
●
●●● ●●
● ●●●
●
●
● ●●● ●
●●●
●●●●
●
● ●
●● ●
●
●● ●●● ●● ●●
●
●●●
●
● ●● ●●● ●● ●
●●● ●● ●● ●● ● ●●
● ●
●
●
● ●
●
●●
●● ●●● ●● ●●●
● ●● ●● ●●
●
●●●● ● ●
●●● ●● ● ●●●●●
●
●●● ● ●●● ● ●● ●● ●●
●
●
●
●●
●●●●● ●● ● ●●
● ● ●●●● ● ●● ●●● ●● ● ●● ●● ●● ●●
●● ●● ● ●●● ●●●
●●● ● ●●
●●● ●
●● ●●●
●●● ●●● ● ●●● ● ●● ●●●
● ●●● ● ●●●● ●● ● ●
●
●● ●●
●
●●
●
●●
●
● ●
●●
●●
●
●
●
●
● ● ●●
● ●● ●●●
●
●
●
●●
●
●
●●●
●●●
●
●●●
●
●●
●
●●
●
●
●
●
●
●●
●●
●● ●
●
●● ●●●● ●
●●
●
● ● ●
●
●●
●
●●
●
●
●
●
●
●
● ●●
●●
● ●
●
● ●
●
●
●
● ●●●●
●
●●
●● ●
●●
● ●●●●
●
●
●●●●● ●
●
●
●
●● ● ●
●●
●
●●
●
●●
●
● ●●●●
●●●● ●
●
●●
●●
●
●●
●●●● ●
●
●●
●
●● ●●● ●● ●●
● ●●●
●
●
●●●● ●
●
● ●
●
●●●●●
●
●●●●●●● ●●
●● ●
●
●
●
● ●
●●●●●●
● ●
●
●
●●
●
●
● ●
●●
●●●
●
●
●
●●●●
●
● ●●
●
●
●
●
● ●●● ●● ●●
●
● ●●
●
●●
●● ●
● ●
●●
●
●●●●
●
●●
●
●● ●●
●
● ●● ●●●
●
●●●
●● ●● ●
●
● ●●●●● ● ●●●● ●●●
●●● ● ●
●
●
●
● ●●●●● ●
●● ●
● ●● ●
●●
●
●● ●
●
●●● ●●●
●● ● ●● ●●●● ●●
●●●
●●
●
●●●●
● ●●● ●●●
●●
● ● ●●●●●●● ●●●● ●●● ●●
●
● ●●● ●●●● ● ●●●
●●●● ●●●●
●● ●● ●●● ●
●●●● ●●●● ●●
●●● ● ●●
●
−6 −4 −2 0 2 4 6
010
2030
Risk choice
subject λ − indifference λ
RT [s
]
33
Figure7.Exampleofanindividualsubject’sRT-basedparameterestimation.Theplotshows RTs in all trials as a function of the indifference parameter value on that trial.ObservationsinthetopRTdecileareshowninred.TheredtriangleshowsthelongestRTfor the subject. The solid vertical red line shows the subject’s choice-based parameterestimate.ThedottedverticalredlineshowstheaverageindifferencevalueforthetopRTdecileapproach.Thedottedgrey lineshows the local regression fit (LOWESS, smoothingparameter=0.5).
Withthese facts inmind,wesetaboutconstructingamethodforusingRTsto infera
subject’sindifferencepoint.Clearly,focusingontheslowesttrialswouldyieldlessbiased
estimates of subjects’ indifference points. However, using too few slow trials would
increase the variance of those estimates. We settled on a simplemethod that uses the
slowest10%ofasubject’schoices,thoughwealsoexploredothercutoffs(AppendixD).
Inshort,ourestimationalgorithmforanindividualsubjectincludesthefollowingsteps:
(1) identify trialswith RTs in the upper 10% (the slowest decile); (2) for each of these
trials, calculate the value of the utility-function parameter that would make the subject
indifferentbetweenthetwoalternatives;(3)averagethesevaluestogettheestimateofthe
subject’s utility-function parameter (see Figure 7 for an example and Appendix A for
estimation details). It is important to note that this method puts bounds on possible
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−1.0 −0.5 0.0 0.5 1.0
1.0
1.5
2.0
indifference β
RT
[s]
●
●
●
●
●
●
●●
34
parameterestimates:theaverageofthehighest10%ofallpossible indifferencevaluesis
the upper bound, while the average of the lowest 10% of the indifference values is the
lower bound. Thus it is crucial to choose choice problems with enough range in their
indifferencevaluestomorethanspantherangeofvaluesinthepopulation.
Again, the parameters estimated using this method were correlated with the same
parameters estimated purely from the choice data, providing a better fit than the DDM
approach (social choice 𝛼: r = 0.44, p = 0.02; social choice 𝛽: r = 0.56, p = 0.001;
intertemporal choice𝑘: r = 0.71, p < 0.001; risky choice𝜆: r = 0.64, p < 0.001; Pearson
correlations; Figure 8, see Appendix A for estimation details for both methods).
Furthermore, these parameters provided prediction accuracy that was better than an
informedbaseline(seeSection5.5)inthreeoutoffourcases(excludingsocialchoice𝛼).In
all cases, a random10%sampleof trialsproducedestimates thatwerenotameaningful
predictorof thechoice-basedparametervalues (since theseestimatesare justameanof
10% random indifference points).14 As noted previously, the RT-based estimations have
upperandlowerboundsduetoaveragingovera10-percentsampleoftrialsandthusare
notabletocapturesomeoutliers(e.g.seeFigure8,bottomleftpanel).Furthermore, the
numberof“extreme”indifferencepointsinthedatasetsthatweconsideredislow,biasing
theRT-basedestimatestowardsthemiddle.
14Wedrewarandom10%sample1000times foreach individual ineachparameterdatasetandestimated the correlation between the average indifference point and the true parameter value(socialchoice𝛼:meanPearsonr=-0.01;socialchoice𝛽:r=0.01;intertemporalchoice𝑘:r=-0.02;riskychoice𝜆:r=-0.003).
35
Figure 8. The top RT decile estimates of subjects’ utility function parameters..Subject-level correlation (Pearson) between parameters estimated from choice data andresponse time (RT) data using trials with RTs in the top decile. The solid lines are 45degrees.ThedottedredlinesshowtheboundsonRTparameterestimations.
5.6.2.Localregressionmethod
Asthetop10%approachonlyutilizespartofthedata,wedevelopedanotherapproach
basedon thepeaks inRTs, this timeusingall theavailableRTdata.The local regression
(LOWESS) method also uses the “revealed indifference” approach: for each individual
subject,werunalocalpolynomialregressionofRTsontheindifferenceparametervalues
and use that regression to identify the indifference value that produces the highest
predictedRT(seeFigure7foranexampleandAppendixAfordetails).Asthepeakofthis
●●
●
●●
●●●
●●●●
●
●
●●
●
●●
●●●
●
●●
●
●
●
●
−1.0 −0.5 0.0 0.5 1.0
−1.0
−0.5
0.0
0.5
1.0
Social choice (α)
α estimated from choices
α e
stim
ated
from
RTs
r = 0.44
p = 0.02 ●●
●
●
●●
●●
●
●
●
●●
●●
●
●
●●
●
●
●●●
●
●●
● ●
●
−1.0 −0.5 0.0 0.5 1.0
−1.0
−0.5
0.0
0.5
1.0
Social choice (β)
β estimated from choices
β es
timat
ed fr
om R
Ts
r = 0.56p = 0.001
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
0.00 0.01 0.02 0.03 0.04
0.00
0.01
0.02
0.03
0.04
Intertemporal choice
k estimated from choices
k es
timat
ed fr
om R
Ts
r = 0.71
p < 0.000
●
●●
●
●● ●●
●
●● ●●
●
●
●
●●
● ●●
●●
●
●
● ●●
●●●●
●●
● ●
0 1 2 3 4 5 6
01
23
45
6
Risk choice
λ estimated from choices
λ es
timat
ed fr
om R
Ts
r = 0.64p < 0.000
36
line is typically close to the choice-basedestimateand theobservationswith thehighest
RTs, this method is quite similar in its predictions to the top-RT-decile method. This
methodrequirestheresearchertochoosethelocalregressionsmoothingparameter.Here
weuseavalueof0.5asitproducesthebestresultsacrossallofthedatasets(thoughother
valuescanworkbetterforspecificdatasets;seeAppendixC).
Althoughthisapproachgenerallyproducesresultssimilartothetopdecileapproach,it
can be affected by outliers (e.g. sparse data and unusually high RTs around extreme
indifference points) and thus sometimes misestimates individual subject parameters,
producing correlations that are in some casesworse than thoseproducedby the topRT
decile approach: social choice𝛼: r = 0.12, p = 0.52; social choice𝛽: r = 0.6, p = 0.001;
intertemporal choice 𝑘: r = 0.44, p = 0.01; risky choice 𝜆: r = 0.88, p < 0.001; Pearson
correlations.Thiscanbemitigatedbyusingaspecificsmoothingparameterforeachdata
set,butourgoalwastoidentifyamethodthatworkswellacrossallthedatasets.
5.7.Choicereversals
Finally, we wish to explore one additional set of predictions from the revealed
indifference approach. We know that when subjects are closer to indifference, their
choicesbecomelesspredictable.Wealsoknowthatsubjectsgenerallychoosemoreslowly
theclosertheyaretoindifference.Therefore,wehypothesizedthatslowerchoicesareless
predictableandthereforealsolesslikelytoberepeated.
Inallthreedatasets,thechoice-estimatedutilitymodelwassignificantlylessconsistent
withlong-RTchoicesthanwithshort-RTchoices(basedonamediansplitwithinsubject):
80%vs 89% (p <0.001) in the risky choice experiment, 71%vs 79% (p < 0.001) in the
37
intertemporalchoiceexperiment,88%vs94%(p=0.008)and90%vs96%(p<0.001)in
thedictatorgameexperiment;p-valuesdenoteWilcoxonsignedrank test significanceon
thesubjectlevel.
Asecond,morenuancedfeatureofSSMsisthatthey(mostly)predictslowerrors,even
conditioningondifficulty.Whiletherearesomeexceptions(seeRatcliff&McKoon2008),
the“slowerror”phenomenoniscommonlyobservedinexperimentsandaccountingforit
wasabreakthroughintheliterature(Ratcliff1978).Totestforslowerrorsinourdata,we
ran mixed-effects regressions of choice consistency on the RTs and the absolute utility
differencebetween the twooptions. In all caseswe founda strongnegative relationship
betweentheRTsandthechoiceconsistency(slowerchoice=lessconsistent)(fixedeffects
of RTs: social choice 𝛼: z = -2.62, p = 0.009; social choice 𝛽: z = -3.3, p < 0.001,
intertemporalchoice:z=-5.28,p<0.001,riskchoice:z=-5.35,p<0.001).Thusweindeed
observeevidenceforslowerrorsinallofourtasks.
In two of the datasets (intertemporal choice and non-adaptive risk choice) subjects
faced thesamesetofdecisionproblems twice.Thisallowedus toperformamoredirect
testof theslowerrorhypothesisbyseeingwhetherslowdecisions in the firstencounter
weremorelikelytobereversedonthesecondencounter.
In the intertemporal choice experiment, themedian RT for a later-reversed decision
was1.36 s, compared to1.17 s for a later-repeateddecision. Amixed-effects regression
effect of first-choiceRTon choice reversal, controlling for the absoluteutility difference,
washighly significant (z=4.04,p<0.001).Thedifferencewaseven stronger in the risk
choice experiment: subsequently reversed choices took 2.36 s versus only 1.4 s for
subsequently repeated choices.Again, amixed-effects regression revealed thatRTwas a
38
significant predictor of subsequent choice reversals (controlling for absolute utility
difference,z=5.2,p<0.001).
6Discussion
Herewehavedemonstratedaproof-of-conceptforthemethodofrevealedindifference.
The method of revealed indifference contrasts with the standard method of revealed
preference,byusingresponse times (RTs) rather thanchoices to inferpreferences. This
newmethodreliesonthefactthatpeoplegenerallytakelongertodecideastheyapproach
indifference. Using datasets from three different choice domains (risk, temporal, and
social)weestablishedthatpreferencesarehighlypredictablefromRTsalone.Finally,we
also found that longRTs are predictive of choicemistakes, as capturedby inconsistency
withtheestimatedutilityfunctionandlaterpreferencereversals.
Throughout the paper we have highlighted ways in which we think RTs may be
important foreconomists. First,usingRTsmayallowus toestimateagents’preferences
usingvery shortandsimpledecision tasks, evena singlebinary-choiceproblem(Section
5.3). For example, if youwant to knowwhether peoplewill buy your product for $50,
knowingthattheywouldbuyitfor$30isnotveryusefulinformation.However,ifyoualso
knowthat theywereveryquick tosayyesat$30,youmight reasonably infer thatmany
wouldstillpurchasetheproductfor$50.
Second,usingRTscanhelpustorecoverpreferenceswhen,evenwithmultiplechoice
problems,someagentsalwaysgive thesameresponse,meaning thatwecouldotherwise
onlyputboundsontheirpreferences(Section5.4).
39
Third,thefactthatRTscanbeusedtoinferpreferenceswhenchoicesareunobservable
oruninformative (Sections5.5&5.6) isan importantpoint for thosewhoareconcerned
about private information, mechanism design, etc. For instance, while voters are very
concernedabouttheconfidentialityoftheirchoices,theymaynotbethinkingaboutwhat
their time in thevotingboothmight conveyabout them. Inanelectionwheremostofa
community’s voters strongly favor one candidate, a long stop in the voting booth may
signal dissent. Another famous example from outside of economics is the implicit
associationtest(IAT),wheresubjects’RTsareusedtoinferpersonalitytraits(e.g.racism)
thatthesubjectswouldotherwiseneveradmittoorevenbeawareof(Greenwald,McGhee,
&Schwartz,1998).Thusprotectingprivacymayinvolvemorethansimplymaskingchoice
outcomes.
Fourth,ourworkhighlightsamethodfordetectingchoiceerrors. Whilethestandard
revealed preference approach must equate preferences and choices, the revealed
indifferenceapproachallowsustoidentifychoicesthataremorelikelytohavebeenmade
bymistake,orattheveryleast,withverylowconfidence.ThusRTsmayplayanimportant
normativeroleinestablishinghowconfidentlywecansaythatagivenchoicetrulyreveals
thatagent’sunderlyingpreference.
InthispaperwehavedescribedaframeworkthatmathematicallylinksRTsandchoices
tounderlyingpreferences.Thisframeworkisnotaheuristic;infact,itarisesastheoptimal
solutiontoasequentialsamplingproblem,wherewhatissampledarestochasticsignalsof
the underlying true preference. In addition to their normative appeal, these SSMs have
enjoyedmuchempirical success incapturingchoiceprobabilitiesandRTdistributions in
many domains, including economic choice (Fudenberg et al. 2015; Krajbich et al. 2010,
40
2014; Ratcliff and McKoon 2008; Webb 2013; Woodford 2014). Moreover, they are
appealing from a neuroscience perspective, as SSMs are biologically plausible and align
wellwithneuralrecordingsinbothhumansandotheranimals(Bogaczetal.2009;Polania
etal.2014).However,itisimportanttonotethatSSMsarenotuniqueintheirprediction
ofslowdecisionscorrespondingwithindifference.Forexample,thisisalsoafeatureofthe
directedcognitionmodel(Gabaix&Laibson,2005),thoughthatmodelperformedslightly
worseinourmodelcomparison.
OnequestionthatarisesfromourresultsiswhatistheoptimalwaytomakeuseofRTs
inordertoinfersubjects’preferences?Ofthemethodswetested,thetop10%ruleseems
toworkverywellacrossourdatasetsandisaneasymethodtouseinpractice.However,
this cutoffwill generally depend on the number of trials in a particular experiment: the
fewerthetrials,thelargerthetoppercentileneedstobe.Foreachofthethreedatasets,we
calculatedtheoptimalpercentilecutoffandfoundthatthetop10-20%RTsgeneratedthe
bestpredictions.Astheindividual-trialRTinformationisnoisy(fasttrialscanbebotheasy
anddifficult,butslowtrialsarealmostalwaysdifficult),usingfewertrialsproduceshigher
variancepredictions,whilemoretrialsmayintroducebias.
ThereareofcourselimitationstousingRTstoinferindifference.Itimportanttokeep
inmindthatotherfactorsmayinfluenceRTsinadditiontostrength-of-preference,suchas
complexity,stakesize,andtrialnumber(Moffatt2005;Krajbichetal.2015b).Aswithany
analysis, it is important to control or account for these factors in order tomaximize the
chanceofsuccess.
A second potential criticism of these findings is that we have focused on repeated
decisions which are made quite quickly (1-3 seconds on average) and so may not be
41
representativeof“realworld”decisionsormaybebeingmadeusingsimpleheuristics.We
haveseveralresponsestothiscriticism.First,theseareallmulti-attributechoiceproblems
and so it is unclear what simple heuristics subjects could be using. Second, the use of
simpleheuristicswouldonlyimpairourabilitytoestimatepreferencesfromRTs,sincein
thosecasesthereshouldbenorelationshipbetweenstrength-of-preferenceandRT. The
less people can rely on heuristics and instead have to evaluate the alternatives to
determinewhichisthebest,themoreeffectivethemethodofrevealedindifferenceshould
be.Third,wewouldarguethatmany,ifnotmost,realworlddecisionsareminorvariants
of otherdecisions thatwemake repeatedly over the courseof our lives. Sowhile these
tasksmaynot,forexample,fullycapturetheprocessofbuyingahouse,theymaybevery
representativeofroutineeconomicdecisions.Finally,whatresearchhasbeendoneonRTs
inone-shot,slowdecisions,issofarconsistentwiththeSSMpredictions.
For example,Krajbich et al. 2015b studya voluntary contributionpublic goods game
experimentwheresubjectsmadeonlythreedecisionsandtookonaverage43.5stodecide
eachtime.Inthatexperiment,slowdecisionstendedtofavorthelessattractivealternative,
consistentwithbeingclosertoindifference. Inparticular,withalow-benefitpublicgood,
slow contributions tended to be higher, while with a high-benefit public good, slow
contributionstendedtobelower.
More careful analysis is required to distinguish between the SSM and alternative
interpretationsbyRubinsteinandothers(Chen&Fischbacher,2015;Hey,1995;Recaldeet
al.,2014;Rubinstein,2007,2013,2014,2016),where longRTsareassociatedwithmore
carefulordeliberativethoughtandshortRTsareassociatedwithintuition. Itmayinfact
bethecasethat insomeinstancespeopledousea logic-basedapproach, inwhichcasea
42
longRTmaybemoreindicativeofcarefulthought,whileinotherinstancestheyrelyona
SSM approach, inwhich case a long RT likely indicates indifference. This could lead to
contradictoryconclusions fromthesameRTdata; forexampleoneresearchermayseea
longRTandassumethesubject isverywell informed,whileanotherresearchermaysee
thatsameRTandassumethesubjecthasnoevidenceonewayortheother.Moreresearch
isrequiredtotestwhetherSSMs,whicharedesignedtoteaseapartsuchexplanations,can
besuccessfullyappliedincomplexeconomicdecisions.
43
References
1. Agranov, M., & Ortoleva, P. (2015). Stochastic choice and preferences for
randomization.AvailableatSSRN2644784.
2. Blais,A.-R.,&Weber, E.U. (2006).Adomain-specific risk-taking (DOSPERT) scale
foradultpopulations.JudgmentandDecisionMaking,1(1).
3. Blurton,S.P.,Kesselmeier,M.,&Gondan,M.(2012).Fastandaccuratecalculations
forcumulative first-passage timedistributions inWienerdiffusionmodels. Journal
ofMathematicalPsychology,56(6),470–475.
4. Brown, S. D., & Heathcote, A. (2008). The simplest complete model of choice
responsetime:Linearballisticaccumulation.CognitivePsychology,57(3),153–178.
https://doi.org/10.1016/j.cogpsych.2007.12.002
5. Busemeyer,J.R.(1985).Decisionmakingunderuncertainty:Acomparisonofsimple
scalability, fixed-sample, and sequential-samplingmodels. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 11(3), 538–564.
https://doi.org/10.1037/0278-7393.11.3.538
6. Busemeyer, J. R., & Diederich, A. (2002). Survey of decision field theory.
MathematicalSocialSciences,43,345–370.
7. Busemeyer, J.R.,&Rapoport,A. (1988).Psychologicalmodelsofdeferreddecision
making.JournalofMathematicalPsychology,32,91–134.
8. Busemeyer, J. R., & Townsend, J. T. (1993). Decision field theory: A dynamic-
cognitiveapproach todecisionmaking inanuncertainenvironment.Psychological
Review,100(3),432–459.
44
9. Caplin, A., & Martin, D. (2016). THE DUAL-PROCESS DRIFT DIFFUSION MODEL:
EVIDENCE FROM RESPONSE TIMES. Economic Inquiry, 54(2), 1274–1282.
https://doi.org/10.1111/ecin.12294
10. Cavagnaro, D. R., Myung, J. I., Pitt, M. A., & Kujala, J. V. (2010). Adaptive design
optimization: A mutual information-based approach to model discrimination in
cognitivescience.NeuralComputation,22(4),887–905.
11. Cettolin,E.,&Riedl,A.(2015).Revealedincompletepreferencesunderuncertainty.
12. Chabris, C. F., Laibson, D., Morris, C. L., Schuldt, J. P., & Taubinsky, D. (2008).
Measuring intertemporal preferences using response times. National Bureau of
EconomicResearch.
13. Chabris, C. F.,Morris, C. L., Taubinsky,D., Laibson,D., & Schuldt, J. P. (2009). The
Allocation of Time in Decision-Making. Journal of the European Economic
Association,7(2–3),628–637.
14. Che, Y.-K., & Mierendorff, K. (2016). Optimal Sequential Decision with Limited
Attention.Unpublished,ColumbiaUniversity.
15. Chen, F., & Fischbacher, U. (2015). Response Time and Click Position: Cheap
IndicatorsofPreferences.ThurgauerWirtschaftsinstitut,Universit�tKonstanz.
16. Cleveland, W. S. (1979). Robust locally weighted regression and smoothing
scatterplots.JournaloftheAmericanStatisticalAssociation,74(368),829–836.
17. Clithero,J.A.(2016a).ImprovingOut-of-SamplePredictionsUsingResponseTimes
andaModeloftheDecisionProcess.AvailableatSSRN2798459.
18. Clithero,J.A.(2016b).ResponseTimesinEconomics:LookingThroughtheLensof
SequentialSamplingModels.AvailableatSSRN.
45
19. Dai,J.,&Busemeyer,J.R.(2014).Aprobabilistic,dynamic,andattribute-wisemodel
ofintertemporalchoice.JournalofExperimentalPsychology:General,143(4),1489–
1514.https://doi.org/10.1037/a0035976
20. Danan, E., Ziegelmeyer, A., & others. (2006). Are Preferences Complete?: An
Experimental Measurement of Indecisiveness Under Risk. Max Planck Inst. of
Economics, Strategic Interaction Group. Retrieved from
https://www.researchgate.net/profile/Anthony_Ziegelmeyer/publication/5018150
_Are_preferences_complete_An_experimental_measurement_of_indecisiveness_unde
r_risk/links/0c960516d8217911fe000000.pdf
21. DeMartino,B.,Fleming,S.M.,Garret,N.,&Dolan,R.J.(2013).Confidenceinvalue-
basedchoice.NatureNeuroscience,16(1),105–110.
22. Dickhaut, J., Rustichini, A., & Smith, V. (2009). A neuroeconomic theory of the
decisionprocess.ProceedingsoftheNationalAcademyofSciences,106(52),22145–
22150.
23. Eliaz, K., & Ok, E. A. (2006). Indifference or indecisiveness? Choice-theoretic
foundationsofincompletepreferences.GamesandEconomicBehavior,56(1),61–86.
https://doi.org/10.1016/j.geb.2005.06.007
24. Fehr,E.,&Schmidt,K.M.(1999).Atheoryoffairness,competition,andcooperation.
TheQuarterlyJournalofEconomics,114(3),817–868.
25. Fiedler,S.,&Glöckner,A.(2012).TheDynamicsofDecisionMakinginRiskyChoice:
An Eye-Tracking Analysis. Frontiers in Psychology, 3.
https://doi.org/10.3389/fpsyg.2012.00335
46
26. Frazier,P.,&Yu,A.(2007).Sequentialhypothesistestingunderstochasticdeadlines
(pp.465–472).PresentedattheAdvancesinneuralinformationprocessingsystems.
27. Fudenberg, D., Strack, P., & Strzalecki, T. (2015). Stochastic Choice and Optimal
SequentialSampling.arXivPreprintarXiv:1505.03342.
28. Gabaix, X., & Laibson, D. (2005). Bounded rationality and directed cognition.
Harvard University. Retrieved from
http://pages.stern.nyu.edu/~xgabaix/papers/boundedRationality.pdf
29. Gabaix, X., Laibson, D., Moloche, G., & Weinberg, S. (2006). Costly Information
Acquisition: Experimental Analysis of a Boundedly Rational Model. American
EconomicReview,96(4),1043–1068.https://doi.org/10.1257/aer.96.4.1043
30. Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. (1998). Measuring individual
differencesinimplicitcognition:theimplicitassociationtest.JournalofPersonality
andSocialPsychology,74(6),1464.
31. Hare, T. A., Hakimi, S., & Rangel, A. (2014). Activity in dlPFC and its effective
connectivity to vmPFC are associated with temporal discounting. Frontiers in
Neuroscience,8.https://doi.org/10.3389/fnins.2014.00050
32. Hawkins, G. E., Forstmann, B. U., Wagenmakers, E.-J., Ratcliff, R., & Brown, S. D.
(2015). Revisiting the Evidence for Collapsing Boundaries andUrgency Signals in
Perceptual Decision-Making. The Journal of Neuroscience, 35(6), 2476–2484.
https://doi.org/10.1523/JNEUROSCI.2410-14.2015
33. Herrnstein,R. J.(1981).Self-controlasresponsestrength.QuantificationofSteady-
StateOperantBehavior,3–20.
47
34. Hey, J. D. (1995). Experimental investigations of errors in decisionmaking under
risk.EuropeanEconomicReview,39(3),633–640.
35. Hunt,L.T.,Kolling,N.,Soltani,A.,Woolrich,M.W.,Rushworth,M.F.S.,&Behrens,T.
E. (2012). Mechanisms underlying cortical activity during value-guided choice.
NatureNeuroscience,15,470–476.
36. Hutcherson,C.A.,Bushong,B.,&Rangel,A.(2015).ANeurocomputationalModelof
AltruisticChoiceandItsImplications.Neuron,87(2),451–462.
37. Johnson,J.G.,&Busemeyer,J.R.(2005).Adynamic,stochastic,computationalmodel
ofpreferencereversalphenomena.PsychologicalReview,112(4),841–861.
38. Kahneman,D.,&Tversky,A.(1979).Prospecttheory:ananalysisofdecisionunder
risk.Econometrica,4,263–291.
39. Kim, W., Pitt, M. A., Lu, Z.-L., Steyvers, M., & Myung, J. I. (2014). A hierarchical
adaptiveapproachtooptimalexperimentaldesign.NeuralComputation.
40. Krajbich, I.,Armel,K.C.,&Rangel,A. (2010).Visual fixationsand thecomputation
andcomparisonofvalueinsimplechoice.NatureNeuroscience,13(10),1292–1298.
41. Krajbich,I.,Bartling,B.,Hare,T.,&Fehr,E.(2015).Rethinkingfastandslowbased
on a critique of reaction-time reverse inference.Nature Communications,6, 7455.
https://doi.org/10.1038/ncomms8455
42. Krajbich, I., Hare, T., Bartling, B., Morishima, Y., & Fehr, E. (2015). A Common
MechanismUnderlyingFoodChoiceandSocialDecisions.PLoSComputBiol,11(10),
e1004371.
48
43. Krajbich,I.,Lu,D.,Camerer,C.,&Rangel,A.(2012).TheAttentionalDrift-Diffusion
Model Extends to Simple Purchasing Decisions. Frontiers in Psychology, 3.
https://doi.org/10.3389/fpsyg.2012.00193
44. Krajbich, I., Oud, B., & Fehr, E. (2014). Benefits ofNeuroeconomicModeling:New
Policy Interventions and Predictors of Preference †. American Economic Review,
104(5),501–506.https://doi.org/10.1257/aer.104.5.501
45. Krajbich, I.,&Rangel,A.(2011).Multialternativedrift-diffusionmodelpredictsthe
relationship between visual fixations and choice in value-based decisions.
ProceedingsoftheNationalAcademyofSciences,108(33),13852–13857.
46. Laibson,D.(1997).Goldeneggsandhyperbolicdiscounting.TheQuarterlyJournalof
Economics,443–477.
47. Laming, D. (1979). A critical comparison of two random-walk models for choice
reactiontime.ActaPsychologica,43,431–453.
48. Link, S. W. (1975). The relative judgment theory of two choice response time.
JournalofMathematicalPsychology,12(1),114–135.
49. Loomes, G. (2005). Modelling the stochastic component of behaviour in
experiments:Someissuesfortheinterpretationofdata.ExperimentalEconomics,8,
301–323.
50. Mandler, M. (2005). Incomplete preferences and rational intransitivity of choice.
Games and Economic Behavior, 50(2), 255–277.
https://doi.org/10.1016/j.geb.2004.02.007
49
51. Matejka, F., & McKay, A. (2014). Rational inattention to discrete choices: A new
foundationforthemultinomiallogitmodel.TheAmericanEconomicReview,105(1),
272–298.
52. Milosavljevic, M., Malmaud, J., Huth, A., Koch, C., & Rangel, A. (2010). The drift
diffusion model can account for the accuracy and reaction time of value-based
choicesunderhighandlowtimepressure.JudgmentandDecisionMaking,5(6),437–
449.
53. Moffatt, P. G. (2005). Stochastic choice and the allocation of cognitive effort.
ExperimentalEconomics,8(4),369–388.
54. Mosteller,F.,&Nogee,P.(1951).AnExperimentalMeasurementofUtility.Journalof
PoliticalEconomy,59(5),371–404.
55. Navarro,D. J.,&Fuss, I.G. (2009). Fast andaccurate calculations for first-passage
times inWienerdiffusionmodels. JournalofMathematicalPsychology,53(4),222–
230.
56. Ok,E.A.,Ortoleva,P.,&Riella,G.(2012).Incompletepreferencesunderuncertainty:
Indecisivenessinbeliefsversustastes.Econometrica,80(4),1791–1808.
57. Oud, B., Krajbich, I., Miller, K., Cheong, J. H., Botvinick, M., & Fehr, E. (2016).
Irrational time allocation in decision-making. Proceedings of the Royal Society B:
BiologicalSciences,283(1822),20151439.https://doi.org/10.1098/rspb.2015.1439
58. Philiastides,M.G.,&Ratcliff,R. (2013). Influenceofbrandingonpreference-based
decisionmaking.PsychologicalScience,24(7),1208–1215.
59. Ratcliff,R.(1978).Atheoryofmemoryretrieval.PsychologicalReview,85,59–108.
50
60. Ratcliff,R.,&McKoon,G.(2008).Thediffusiondecisionmodel:theoryanddatafor
two-choicedecisiontasks.NeuralComputation,20(4),873–922.
61. Ratcliff,R.,Philiastides,M.G.,&Sajda,P.(2009).Qualityofevidenceforperceptual
decisionmakingisindexedbytrial-to-trialvariabilityoftheEEG.Proceedingsofthe
NationalAcademyofSciences,106(16),6539–6544.
62. Ratcliff,R.,&Tuerlinckx, F. (2002).Estimatingparametersof thediffusionmodel:
Approaches todealingwithcontaminantreaction timesandparametervariability.
PsychonomicBulletin&Review,9(3),438–481.
63. Recalde, M. P., Riedl, A., & Vesterlund, L. (2014). Error prone inference from
response time: The case of intuitive generosity. Retrieved from
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2507723
64. Rieskamp, J. (2008). The probabilistic nature of preferential choice. Journal of
ExperimentalPsychology:Learning,Memory,andCognition,34(6),1446.
65. Rodriguez, C. A., Turner, B.M., &McClure, S.M. (2014a). Intertemporal Choice as
DiscountedValueAccumulation.PloSOne,9(2),e90138.
66. Rodriguez, C. A., Turner, B.M., &McClure, S.M. (2014b). Intertemporal choice as
discountedvalueaccumulation.PLoS.ONE,9,e90138.
67. Roe,R.M.,Busemeyer,J.R.,&Townsend,J.T.(2001).Multialternativedecisionfield
theory: A dynamic connectionist model of decision making. Psychological Review,
108(2),370–392.
68. Rubinstein, A. (2007). Instinctive and Cognitive Reasoning: A Study of Response
Times*.TheEconomicJournal,117(523),1243–1259.
51
69. Rubinstein,A.(2013).Responsetimeanddecisionmaking:Anexperimentalstudy.
JudgmentandDecisionMaking,8(5),540–551.
70. Rubinstein, A. (2014). A Typology of Players: between Instinctive and
Contemplative.Retrievedfromhttp://arielrubinstein.tau.ac.il/papers/Typology.pdf
71. Samuelson, P. A. (1938). A note on the pure theory of consumer’s behaviour.
Economica,5(17),61–71.
72. Schotter, A., & Trevino, I. (2014). Is response time predictive of choice? An
experimental study of threshold strategies. WZB Discussion Paper. Retrieved from
http://www.econstor.eu/handle/10419/98843
73. Shadlen, M. N., & Shohamy, D. (2016). Decision Making and Sequential Sampling
from Memory. Neuron, 90(5), 927–939.
https://doi.org/10.1016/j.neuron.2016.04.036
74. Sims, C. A. (2003). Implications of rational inattention. Journal of Monetary
Economics,50(3),665–690.
75. Sokol-Hessner,P.,Hsu,M.,Curley,N.G.,Delgado,M.R.,Camerer,C.F.,&Phelps,E.A.
(2009). Thinking like a trader selectively reduces individuals’ loss aversion.
ProceedingsoftheNationalAcademyofSciences,106(13),5035–5040.
76. Spiliopoulos, L., & Ortmann, A. (2014). The BCD of response time analysis in
experimental economics. Available at SSRN 2401325. Retrieved from
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2401325
77. Srivastava, V., Feng, S. F., Cohen, J. D., Leonard, N. E., & Shenhav, A. (2015). First
PassageTimePropertiesforTime-varyingDiffusionModels:AMartingaleApproach.
arXivPreprintarXiv:1508.03373.
52
78. Steiner, J.,Stewart,C.,&Matějka,F. (2017).Rational InattentionDynamics: Inertia
andDelayinDecision-Making.Econometrica,85(2),521–553.
79. Stewart,N.,Hermens,F.,&Matthews,W.J.(2015).EyeMovementsinRiskyChoice:
Eye Movements in Risky Choice. Journal of Behavioral Decision Making, n/a-n/a.
https://doi.org/10.1002/bdm.1854
80. Stone,M.(1960).Modelsforchoice-reactiontime.Psychometrika,25(3),251–260.
81. Tajima, S., Drugowitsch, J., & Pouget, A. (2016). Optimal policy for value-based
decision-making. Nature Communications, 7, 12400.
https://doi.org/10.1038/ncomms12400
82. Towal, R. B., Mormann, M., & Koch, C. (2013). Simultaneous modeling of visual
saliency and value computation improves predictions of economic choice.
ProceedingsoftheNationalAcademyofSciences,110(40),E3858–E3867.
83. Usher,M.,&McClelland, J.(2001).Thetimecourseofperceptualchoice:theleaky,
competingaccumulatormodel.PsychologicalReview,108(3),550–592.
84. Wabersich, D., & Vandekerckhove, J. (2014). TheRWiener package: AnR package
providingdistributionfunctionsfortheWienerdiffusionmodel.RJournal,6(1).
85. Wald, A. (1945). Sequential tests of statistical hypotheses. The Annals of
MathematicalStatistics,16(2),117–186.
86. Wang, S.W., Filiba,M.,&Camerer, C. F. (2010).Dynamicallyoptimized sequential
experimentation (DOSE) for estimating economic preference parameters.
ManuscriptSubmittedforPublication.
87. Webb,R.(2013a).DynamicConstraintsontheDistributionofStochasticChoice:Drift
DiffusionImpliesRandomUtility(SSRNScholarlyPaperNo.ID2226018).Rochester,
53
NY: Social Science Research Network. Retrieved from
http://papers.ssrn.com/abstract=2226018
88. Webb,R.(2013b).Dynamicconstraintsonthedistributionofstochasticchoice:Drift
Diffusion implies Random Utility. Available at SSRN 2226018. Retrieved from
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2226018
89. Wilcox,N.T. (1993).Lotterychoice: Incentives, complexityanddecision time.The
EconomicJournal,1397–1417.
90. Woodford, M. (2014). Stochastic Choice: An Optimizing Neuroeconomic Model.
American Economic Review, 104(5), 495–500.
https://doi.org/10.1257/aer.104.5.495
91. Woodford,M.(2016).OptimalEvidenceAccumulationandStochasticChoice.
54
ONLINEAPPENDIX
A. Parameterestimationmethodology
Choice-basedmethod
Weestimateeachindividualutilityfunction𝑢(⋅ |𝜃),where𝜃isasubject-specificparameter
described in Section 3, in the standardway as follows.We assume that for each pair of
options and choice 𝑎 = 1,2 the error terms in utilities follow the type I extreme value
distribution,sotheprobabilityofchoosingoption1isalogisticfunction
𝑝(𝑎j = 1) =1
1 + 𝑒R� ��(⋅|�)R�O(⋅|� ),
where 𝜇 and 𝜃 are free parameters that can be estimated for each subject individually
maximizingalikelihoodfunction
𝐿𝐿 = (log 𝑝(𝑎e = 1) ⋅ 1 𝑎� = 1 + log 1 − 𝑝 𝑎e = 2 ⋅ 1(𝑎� = 2)),e
where𝑛 isthetrialnumber,𝑎e isthechoicemadebythesubjectonthattrial,and1(⋅) is
theindicatorfunction.
Appendix B shows the subject level correlations between the predicted and the actual
choices.
Top10%responsetimemethod
Foreachdecisionproblemoneach trial𝑛,wecalculate the indifferenceparametervalue
𝜃eje� asasolutiontotheequation
𝑢9 ⋅ 𝜃e = 𝑢< ⋅ 𝜃e .
55
ThenweaveragetheindifferencevaluesonthetrialsinthetopRTdeciletoobtainthefinal
parameterestimate:
𝜃 =(𝜃eje�e ⋅ 1(𝐹(𝑅𝑇e) ≥ 0.9))
1(𝐹(𝑅𝑇e) ≥ 0.9)e,
where𝑅𝑇e is the response time on trial𝑛,𝐹(⋅) is the empirical RT distribution for the
specificsubject,and1 ⋅ istheindicatorfunction.
Localregression(LOWESS)method
As in the previous method, for each decision problem on each trial 𝑛, we estimate the
indifferenceparametervalue𝜃eje� solvingtheequation
𝑢9 ⋅ 𝜃e = 𝑢< ⋅ 𝜃e .
For each individual subject, we regress response time log(𝑅𝑇) in every trial 𝑛 on the
corresponding indifference parameter value 𝜃eje� using a local polynomial regression
(LOWESS,Cleveland1979)intheRpackagestats:
𝑅𝑇 = 𝑓 𝜃eje� + 𝜀e.
We set the smoothing parameter to 0.5 as it has provided the best prediction accuracy
acrossallfourdatasets(seeAppendixC).
Then we obtain the parameter estimate 𝜃 by inverting the fitted regression line at the
maximumpredictedresponsetime𝑅𝑇:
𝜃 = 𝑓R9 max 𝑅𝑇 .
56
Drift-diffusionmodel(DDM)method
In theDDM (see Section2) a latent decision variable evolves over timewith an average
drift rate plus Gaussian noise (the Wiener diffusion) until it reaches one of two pre-
determinedboundaries,whichcorrespondtothetwochoiceoptions..Giventheboundary
separation parameter, the drift rate, the non-decision time (the component of RT not
attributable to thedecisionprocess itself, e.g.movingone’s hand to indicate the choice),
andthevarianceof theGaussiannoise, it ispossible tocalculatechoiceprobabilitiesand
choice-contingentRTdistributions.
Inourparticularcaseweassumethatchoicesareunknown,andsowecanonlyuse
thecombined(summed)RTdistributiontoestimatethefreeparametersofthemodel.We
assumethatallsubjectssharethesameconstantboundaryparameter𝑏,non-decisiontime
𝜏,anddriftrateparameter𝑧,whichmultipliestheutilitydifferenceoneverysingletrial:
𝑣 ≡ 𝑧 ⋅ 𝑢9 ⋅ 𝜃 − 𝑢< ⋅ 𝜃 .
We use a density function of the Wiener distribution from the RWiener R package
(WabersichandVandekerckhove2014)toestimatethelikelihood(9)fortheobservedRT
on every given trial assuming a set of parameters (𝑏, 𝜏, 𝑧, 𝜽), where 𝜽 is a vector of
individualsubjects’parameters.Essentially,theidentificationoftheindividualparameters
ispossibleduetothefact thatRTsarepredictedtovaryastheutilitydifference𝑣varies
acrosstrialsandsubjects.
Chabrisetal.(2009)method
Here we follow the method suggested by Chabris et al. (2009), which uses the full RT
distributiontoestimatetheutilityfunctionparameters.
57
Letthedifferenceinthetwoutilitiesbe
𝛥� ≡ 𝑢9� ⋅ 𝜃 − 𝑢<� ⋅ 𝜃 .
Assumethatthedecisiondifficultyisaconvexanddecreasingfunctionofthisdifference:
𝛤 𝛥� ≡2
1 + 𝑒��[,
where𝜔isafreeparameter.
The response times are then modeled as a function of the trial number and the choice
difficulty:
𝑅𝑇� = 𝛽� + 𝛽9𝑡 + 𝛽<𝛤 𝛥� + 𝜀�.
To estimate the set of parameters (𝜃, 𝜔, 𝛽�, 𝛽9, 𝛽<) we follow the original paper and
minimizetheerrorfunction
(𝑅𝑇� − 𝛽� − 𝛽9𝑡 − 𝛽<𝛤 𝛥� ).�
UnlikeChabrisetal.(2009),weestimatethismodelforeveryindividualsubject.
58
B. Choice-basedfits
FigureB1.Subject-levelcorrelation(Pearson)betweenchoiceproportionsinthedataand
aspredictedby choice-estimatedutility functions (seeAppendixA fordetails). The solid
linesare45degrees.
●
●
● ●●● ●●
●
●●●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
0.0 0.1 0.2 0.3 0.4 0.5 0.6
0.0
0.1
0.2
0.3
0.4
0.5
0.6
Social choice (α)
P(fair) in data
P(fa
ir) e
stim
ated
from
cho
ices
r = 0.91
p < 0.001
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
0.0 0.2 0.4 0.6 0.8 1.00.
00.
20.
40.
60.
81.
0
Social choice (β)
P(fair) in data
P(fa
ir) e
stim
ated
from
cho
ices
r = 0.99
p < 0.000
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Intertemporal choice
P(later) in data
P(la
ter)
estim
ated
from
cho
ices
r = 0.97
p < 0.000
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Risk choice
P(safe) in data
P(sa
fe) e
stim
ated
from
cho
ices
r = 0.99
p < 0.000
59
C. Local regression (LOWESS) method choice prediction accuracy for various
levelsofthesmoothingparameter
Figure C1. Choice prediction accuracy as a function of the smoothing parameter of the
LOWESSregressionmodel.Thesolidblack linesdenotemeanpredictionaccuracyacross
subjects,theshadedareasshowstandarderrorsatthesubjectlevel.
0.0 0.2 0.4 0.6 0.8 1.0
0.80
0.85
0.90
0.95
Social choice (α)
smoothing parameter
pred
ictio
n ac
cura
cy
0.0 0.2 0.4 0.6 0.8 1.0
0.80
0.85
0.90
Social choice (β)
smoothing parameter
pred
ictio
n ac
cura
cy
0.0 0.2 0.4 0.6 0.8 1.0
0.70
0.75
0.80
0.85
Intertemporal choice
smoothing parameter
pred
ictio
n ac
cura
cy
0.0 0.2 0.4 0.6 0.8 1.0
0.75
0.80
0.85
0.90
Risk choice
smoothing parameter
pred
ictio
n ac
cura
cy
60
D. ExploringtheoptimalcutoffforthetopRTestimationmethod
Figure D1. Averaging the longest 10-20% RT trials provides the best choice
predictionaccuracy.Choicepredictionaccuracyasafunctionofthepercentageofslowest
trialsusedintheparameterestimationfrom1to100%.Thesolidblacklinesdenotemean
predictionaccuracyacrosssubjects,theshadedareasshowstandarderrorsatthesubject
level,theredlinesabovethegraphsindicatesignificantdifferencefromthebaselineatthe
p=0.05 level (Wilcoxonsignedrank test).Thebaseline is theaverageofall indifference
0 20 40 60 80 100
0.80
0.85
0.90
0.95
Social choice (α)
top percentile
pred
ictio
n ac
cura
cy
0 20 40 60 80 100
0.75
0.80
0.85
0.90
Social choice (β)
top percentilepr
edic
tion
accu
racy
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0 20 40 60 80 100
0.65
0.70
0.75
0.80
Intertemporal choice
top percentile
pred
ictio
n ac
cura
cy
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●
0 20 40 60 80 100
0.76
0.78
0.80
0.82
0.84
Risk choice
top percentile
pred
ictio
n ac
cura
cy
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●
61
pointsacrosstrials.Itisimportanttoemphasizethatthebaselinetowhichwecomparethe
predictive power is not chance (50%) as almost any experimenter uses some prior
knowledgeoftheparameterdistributioninthepopulationtoselecttheirchoiceproblems.
For example, an experimenter studying intertemporal choicemight select a set of choice
problemssothattheaveragesubjectwouldchoosetheimmediateoptionhalfofthetime
andthedelayedoptiontheotherhalfofthetime.Soifyouweretoaveragetheindifference
pointsfromthetrialsinsuchanexperiment,youwouldbeabletopredictbehaviorquite
accurately,onaverage.Insuchanexperiment,behaviorintrialswithextremeindifference
pointswillbeverypredictable.Thatis,onatrialdesignedtomakeaverypatientsubject
indifferent, most subjects will have a strong preference for the immediate option.
Similarly,ona trialdesigned tomakean impatientperson indifferent,most subjectswill
haveastrongpreferenceforthedelayedoption.Thusbehaviorinmanyofanexperiment’s
trials isquiteeasytopredictbecausethosetrialsareonly includedto identifyparameter
values for extreme subjects. For instance, a single loss-aversion coefficient of𝜆 = 2 can
predictabout75%ofchoicesinourrisky-choicedataset.
62
E. Modelcomparisonresults
DDM Chabrisetal(2009) Top10%RT LOWESS(0.5)
Socialchoice(𝛼) 0.39 0.22 0.44 0.12
Socialchoice(𝛽) 0.52 0.53 0.56 0.6
Intertemporalchoice 0.57 0.59 0.71 0.44
Riskchoice 0.36 0.16 0.64 0.88
PearsoncorrelationbetweenparametersestimatedfromchoicesandRTs,atthesubject
level.Thebestperformingmethodforeachdatasetisshowninbold.Forestimation
methodsdetailsseeAppendixA.
63
F. Instructionsfortheriskexperiment
InstructionsThankyouforparticipatingintoday’sstudy.Pleasecarefullyreadthematerialonthefollowingpagestounderstand
• Therules• Thedecisionsyouwillbemakingtoday
Ifyouhaveanyquestionsafterreadingtheseinstructionsorduringtheexperiment,please
askthembeforetheexperimentorduringthedesignatedbreaks.Therules
• Pleasechecknowtoensurethatyourmobilephoneisonsilentmodeandputitinyourbagorpocket.
• Pleasedonottalkduringtheexperiment.ThestudyTodayyouwillbemakingaseriesofchoices,andyour finalpaymentwilldependonlyon
yourownchoicesandchance.PaymentYourpaymentwillconsistoftwoamounts:
• Afixedendowmentof34experimentalcurrencyunits(ECUs)thatyouaregivenatthebeginningofthestudy.
• Yourearnings fromone randomly selected choice round. Youmayearnadditionalmoneybeyondthe34ECUs,oryoumay losesomeofthat34ECUs,dependingonyourchoicesandonchance.Theminimumamountofmoneyyoucanearntodayis10ECUs=$5.
• All the amounts in today’s studywill be shown in ECUs andwill be converted todollarsattheendofthestudyatarateof2ECUs=$1.
64
Yourchoices
In each round of the experiment you will be asked to make a choice between one of twooptions. Option one consists of two possible amounts, each onewith a probability of 50%.Optiontwoconsistsofoneamount,withaprobabilityof100%.
Belowisanexampledecisionscreen.InthisroundOptionone(ontheleft)consistsofagain(ingreen)of30ECUsandaloss(inred)of10ECUs.Ifyoupickthisoption,thecomputerflipsafairdigital coin (chances are 50-50). In case of heads, you would earn 30 ECUs on top of yourendowment. In caseof tails, youwould lose10 ECUs,whichwould thenbe subtracted fromyourendowment.Tochoosethisoption,youwouldpress1.Optiontwo(ontheright)isasuregainof15ECUs.Ifyoupickthisoption,youwouldearn15ECUsontopofyourendowment.Tochoosethisoption,youwouldpress2.
Yourfinalearningswillonlydependononeofyourchoices:attheendofthestudy,onlyoneoftheroundswillberandomlyselectedforpayment.Theoutcomefromthisroundwillbeaddedorsubtractedfromyourinitialendowment.
Ifyouhaveanyquestions,pleaseraiseyourhandnow.
30
-10
15vs
1 2