Failure of intuition when presented with a choice between investing in a single goal or splitting resources between two goals A. D. F. Clarke ⇤ and A. R. Hunt † School of Psychology, University of Aberdeen, UK September 23, 2015 Abstract In a series of related experiments, we ask peo- ple to choose whether to split their attention be- tween two equally likely potential tasks, or priori- tise one task at the expense of the other. When the tasks are easy, the best strategy is to prepare for both of them. As difficulty increases beyond the point where participants can perform both tasks accurately, they should switch strategy and focus on one task at the expense of the other. Across three di↵erent modalities (target detection, throw- ing, and memory), none of the participants switch strategy at the correct point. Moreover, the ma- jority consistently fail to modify their behaviour with task difficulty at all. This failure may be re- lated to uncertainty about the trial outcome, be- cause in a version of the experiment in which there is no uncertainty, participants were uniformly op- timal. Keywords: decision making, optimal be- haviour 1 Introduction A goalie can choose to stand in the middle of the net or to stand to one side or the other. A student can choose to study all the course material, or to focus on learning a subset more deeply. A funding body can choose to divide resources across a large number of projects, or to focus resources on one or two especially promising ones. At its simplest, the choice that will lead to the best outcome in each of these scenarios depends on the likelihood of suc- cess given constraints of time and ability: Is it possible to achieve multiple goals given these con- ⇤ [email protected]† [email protected]straints? If so, it makes sense to try and achieve them all. Otherwise, we are better o↵ focusing our resources on only one task or goal at the expense of the others. Here we report results from three ex- periments that – despite using very di↵erent meth- ods – all converge on the conclusion that humans are surprisingly deficient at achieving optimal out- comes. When presented with a choice between di- viding available resources between two goals ver- sus investing all their resources in one goal, the participants were poor at choosing the best strat- egy even though the factors they need to take into account are relatively stable and limited. A def- inition of optimal decisions is those that achieve the best possible outcome while minimising en- ergy expenditure and risk. There are many exam- ples of optimal or near-optimal decisions in hu- mans (e.g. Kibbe and Kowler, 2011, K¨ ording and Wolpert, 2004, Najemnik and Geisler, 2005, Oru¸c, Maloney, and Landy, 2003). Wolpert and Landy (2012), for example, have argued that motor con- trol can be viewed as a decision-making problem of maximising movement outcome dependent on task, motor and sensory uncertainty. However, others have demonstrated human failures to max- imise expected gain in more deliberative human decisions Gardner (1959), Kahneman and Tver- sky (1984), Morvan and Maloney (2012), Vulkan (2000), Zhang, Morvan, Etezad-Heydari, and Mal- oney (2012). Our interest in optimal decision-making be- gan with an intriguing contradiction in the visual search literature. One influential model of search (Najemnik and Geisler, 2005) proposes that each eye movement during search is directed to the lo- cation that decreases uncertainty about the target location by the maximum amount possible. How- 1
Transcript
Failure of intuition when presented with a choice between investing in
a single goal or splitting resources between two goals
A. D. F. Clarke⇤ and A. R. Hunt†
School of Psychology, University of Aberdeen, UK
September 23, 2015
Abstract
In a series of related experiments, we ask peo-ple to choose whether to split their attention be-tween two equally likely potential tasks, or priori-tise one task at the expense of the other. When thetasks are easy, the best strategy is to prepare forboth of them. As di�culty increases beyond thepoint where participants can perform both tasksaccurately, they should switch strategy and focuson one task at the expense of the other. Acrossthree di↵erent modalities (target detection, throw-ing, and memory), none of the participants switchstrategy at the correct point. Moreover, the ma-jority consistently fail to modify their behaviourwith task di�culty at all. This failure may be re-lated to uncertainty about the trial outcome, be-cause in a version of the experiment in which thereis no uncertainty, participants were uniformly op-timal. Keywords: decision making, optimal be-haviour
1 Introduction
A goalie can choose to stand in the middle of thenet or to stand to one side or the other. A studentcan choose to study all the course material, or tofocus on learning a subset more deeply. A fundingbody can choose to divide resources across a largenumber of projects, or to focus resources on one ortwo especially promising ones. At its simplest, thechoice that will lead to the best outcome in each ofthese scenarios depends on the likelihood of suc-cess given constraints of time and ability: Is itpossible to achieve multiple goals given these con-
straints? If so, it makes sense to try and achievethem all. Otherwise, we are better o↵ focusing ourresources on only one task or goal at the expense ofthe others. Here we report results from three ex-periments that – despite using very di↵erent meth-ods – all converge on the conclusion that humansare surprisingly deficient at achieving optimal out-comes. When presented with a choice between di-viding available resources between two goals ver-sus investing all their resources in one goal, theparticipants were poor at choosing the best strat-egy even though the factors they need to take intoaccount are relatively stable and limited. A def-inition of optimal decisions is those that achievethe best possible outcome while minimising en-ergy expenditure and risk. There are many exam-ples of optimal or near-optimal decisions in hu-mans (e.g. Kibbe and Kowler, 2011, Kording andWolpert, 2004, Najemnik and Geisler, 2005, Oruc,Maloney, and Landy, 2003). Wolpert and Landy(2012), for example, have argued that motor con-trol can be viewed as a decision-making problemof maximising movement outcome dependent ontask, motor and sensory uncertainty. However,others have demonstrated human failures to max-imise expected gain in more deliberative humandecisions Gardner (1959), Kahneman and Tver-sky (1984), Morvan and Maloney (2012), Vulkan(2000), Zhang, Morvan, Etezad-Heydari, and Mal-oney (2012).
Our interest in optimal decision-making be-gan with an intriguing contradiction in the visualsearch literature. One influential model of search(Najemnik and Geisler, 2005) proposes that eacheye movement during search is directed to the lo-cation that decreases uncertainty about the targetlocation by the maximum amount possible. How-
1
Clarke & Hunt (2015)
ever, Morvan and Maloney (2012) recently pro-vided striking evidence that human observers donot reliably fixate locations that maximise theirprobability of detecting a target. In their study,observers had to choose where to fixate, and thena low-contrast discrimination target would appearinside one of two boxes. If humans take the prob-ability of detecting the target at a given eccentric-ity into account when deciding where to fixate,they should fixate a location in between the twoboxes when they are relatively close together. Asthe boxes move further apart, they will reach aneccentricity at which it is no longer possible to dis-criminate the target in either box above chance.At this point, observers should switch to a strat-egy of fixating one or the other box, since this willyield accuracy close to 100% if the target happensto appear inside the fixated box, and an at-chanceaccuracy if it does not. Surprisingly, not only didall four of the observers fail to maximise their tar-get discrimination performance, their tendency tofixate between versus on the target boxes did notvary with the distance between the boxes at all.Morvan and Maloney (2012) propose that saccadetarget selection is based largely on heuristics, suchas a tendency to saccade in particular directions,rather than taking visual sensitivity and uncer-tainty into account.
The failure to adjust fixation strategy in re-sponse to this very simple change in spatial con-figuration is surprising, and di�cult to reconcilewith models of fixation behaviour that dependon a mechanism that maximises information gain(Hayhoe and Ballard, 2014, Najemnik and Geisler,2005). Our first experiment therefore replicatedMorvan and Maloney (2012) in a larger sample.We then were interested in establishing whetherthis failure of optimal behaviour could be consid-ered specific to the context of eye movements anddetection of targets, or a larger problem pervadinghuman decisions in general. Saccadic eye move-ments are rapid, energy-e�cient, and frequentlynot under voluntary control, so the decisional pro-cesses involved may not generalize to other modal-ities. Indeed, there is precedent for rapid motorresponses to achieve di↵erent outcomes than de-liberative decisions (Hunt and Klein, 2002, Wu,Delgado, and Maloney, 2009). The aim of ourstudy is investigate whether participants exhibitmore strategic decision-making in tasks involvingdeliberate, high-stakes decisions that have more
tangible outcomes.
We carried out a series of four related experi-ments. While each experiment varied in terms oftask and modality (detection, throwing, memoryand reaching), crucially they all involved the samedecision-making paradigm: to experimentally cre-ate a point at which, to achieve the best possibleoutcome, it is necessary to switch between divid-ing available resources across two goals versus in-vesting all resources in one goal. All experimentswere conducted in two sessions. In the first sessionparticipants performed the task with only one tar-get/goal. The purpose of this session was to bothcharacterise each participant’s performance acrossdi�culty, as well as to facilitate the participant’sawareness of their own level of skill across di�-culty. In the second session, participants repeatedthe task, but this time there were two potentialtargets, and the participant had to make a de-cision about whether to divide possible successevenly between the two targets, or to abandonone target in favour of the other. Each partici-pants’ choices can be compared to individualizedestimates of what they should have chosen.
2 Method
The motivation and logic of the four experimentswere similar, so we report the methods and resultsfor all four together.
2.1 Participants
Forty-eight undergraduates at the University ofAberdeen were recruited to participate, twelve foreach experiment. All participants were recruitedvia word of mouth and were naive to the aims ofthe experiment. All gave informed consent to par-ticipate in the experiments, which were reviewedand approved by the School of Psychology ethicscommittee.
2.2 Materials and Procedures
Stimuli and setup: Stimuli and layout for eachof the four experiments are shown in Figure 1.Specific details for each of the four experimentsare described below.
Accepted for publication in Psychological Science 2
Clarke & Hunt (2015)
Figure 1: Example stimuli and set-up of the second session of each experiment. A The detection experiment:participants start the trial by fixating the cross. After they shift their gaze to one of the three boxes, the target(a small white circle) appears inside the left or right box. Eccentricity of the boxes varied. B The throwingexperiment: participants were told what color their target hoop will be. They choose a place to stand andare then told which of the two hoops of that color is their target. C The memory experiment: Example of atrial with 5-digit numbers. Participants do not know which number they will have to report. D The reachingexperiment: Participants are told which color beanbag they will need to pick up, and asked to select a chair.
2.2.1 Detection Experiment
Morvan and Maloney (2012) found choice behav-ior that was idiosyncratic but clearly not optimalin the four observers they tested. We expected toreplicate this pattern, but wanted to ensure it heldtrue in a larger sample. Given that each observeris compared against an individualized estimate oftheir own optimal strategy, however, a very largesample is unnecessary, so we decided on a samplesize of 12 for this and all subsequent experiments.An Eyelink 1000 (SR Research, Canada) was usedto record eye movements. The aim of the Session1 of the experiment was to obtain a psychometricfunction for each participant for target detection,and for the participant to gain practice with thedetection task and familiarity with their ownlevel of performance. The stimulus consistedof two grey squares (length = 1.8�, measuredin degrees of visual angle), equidistant (� 2{2.9�, 5.7�, 7.1�, 8.3�, 9.4�, 10.5�, 11.7�, 12.8�})from a central fixation cross. After the partic-ipant had maintained a stable central fixationfor 1 second, the target (a small white circle)was presented at the top or bottom of one ofthe two squares. The stimulus was displayedfor 500ms, after which a blank grey screen wasdisplayed. The trial was immediately cancelledif the participant broke central fixation. Partici-pants responded via keypress (up/down arrows)as to whether the target had been presented atthe top or bottom of one of the two squares, withthe instruction to just guess if they were not sure.There were four blocks of 96 trials (384 trials in
total). Within each block, trials were presentedin order of increasing �. No feedback was given.Individual performance was modelled in R usinga generalised linear model with the mafc-probitlink function from the psyphy package. Let �(�)be this function, where � is the distance from thefixation point to the target.
In the second session, which took place about aweek later, participants fixated a crosshair abovethree boxes and were instructed to choose one boxto fixate. The crosshair was presented above thetargets and positioned so that it was equidistantfrom the central and rightmost grey square (seeFigure 1A for an illustration of the trial setup).This meant that the cross’s position varied withthe separation. The same eight values of � asabove were used, with 48 repetitions of each. Thisgave a total of 384 trials, which were presented in arandom order. After a fixation was detected insideone of the three boxes, the target was presented ineither the left or right box and, as in Part 1, theparticipant had to simply report whether the tar-get had appeared up or down. Participants weretold the target would never appear in the centerbox. We use � from Session 1 to derive each partic-ipant’s optimal strategy and predicted accuracy.When the separation between the boxes is small,participants can direct their saccade towards thecentral box and have a good chance of detectingthe target in either location. Once � increases tothe point where �(�) < 0.75 the participant shouldswitch strategies and fixate either the left or rightbox. When fixating the left or right box, there is a50% chance that the target will appear at fixation,
Accepted for publication in Psychological Science 3
Clarke & Hunt (2015)
giving them ⇡ 100% chance of correctly respond-ing up or down, and a 50% chance that the targetwill appear at the other box, in which case theywill be correct 50% of the time by guessing. To-gether this gives an expected accuracy of 75%.
2.2.2 Throwing Experiment
This experiment is analogous to the detection ex-periment, except the task is to get a beanbag intoone of two hoops. It is not known which hoopwill be designated as the target, and participantsare asked to choose a standing position. For twohoops close to one another, the ideal position tostand is halfway between them. If the distance be-tween hoops is too large to throw accurately fromthe center, however, the optimal behaviour is tostand close to one hoop, giving a success rate of50%.
The experiment took place in a sheltered areaof concrete slabs (see Figure 1B). Each slab was0.46⇥0.61m, making them useful markers for plac-ing hoops and recording standing positions. Inthe first session, participants stood in the centerslab of the area, which was marked with blacktape, and flat hoops with a diameter of 0.40m wereplaced at six di↵erent distances from them (1.88,3.22, 4.14, 5.06, 6.90, and 8.74m). Each partici-pant tossed 12 beanbags for each hoop distance,in order of increasing distance, with each bean-bag cleared from the area after each toss. Theparticipant then completed the same set of dis-tances but tossing in the opposite direction (thiswas counterbalanced), for a total of 144 trials. Atrial was recorded as ”correct” if the final rest-ing place of the beanbag was inside or touchingthe specified hoop. No di↵erences between direc-tion were found so we ignored this factor in subse-quent analyses. Each participant’s accuracy wasmodelled using logistic regression with a fixed in-tercept of (0, 0.99). That is, we assume that par-ticipants are 99% accurate if they stand right nextto the hoop. Each participant’s curve (modelledthe same way as the detection experiment) wasused to select six slabs on which to place hoopsin Session 2. We based these around the slab atwhich participants were closest to 50% accurate(i.e. where �(�) = 0.50, which we will call slabM . This is the point where, to maximise accu-racy, participants should switch between standingin the centre to standing close to one or the otherhoop.
For the first block of Session 2, six hoops weretaped down, three in each direction: at slabsM+1M � 1 and on the slab with expected accuracy of90% (relative to a centre point, which was un-marked). The second block was the same config-uration but with hoops taped down on slabs M ,M +2, and on the slab with expected accuracy of10%. Red hoops were always closest, yellow in themiddle, and blue furthest away. Participants weretold:
“You will be given a beanbag. Yourtask is to get the beanbag into one of thetwo hoops of the same color. For exam-ple, if you are handed a yellow beanbag,this means you will have to get the bean-bag into one of the two yellow hoops. Iam not going to tell you which hoop yet.First, you need to select a place to stand.You can choose anywhere you like withinthe paved area, but remember your taskis to get the beanbag in the hoop of thespecified color. Once you are in position,tell the experimenter you are ready.”
Participants received one practice trial, and thencompleted 48 decisions/throws in each block (16trials for each distance condition in a random or-der). The main experimenter stood on the grassto the side of the paved area and the participantreturned to them after each trial to receive a newbeanbag, while the other experimenter cleared thebeanbags and recorded accuracy and standing po-sition (the numbers 1 to 40 had been chalked onthe edge of the paved area from one end to theother, for quick and subtle recording of standingposition). The order of colour and direction ofthrow was randomised separately for each partic-ipant.
2.2.3 Memory Experiment
In this experiment participants were shown twonumbers, and later asked to report only one ofthem. At the time of presentation, the partici-pant did not know which number would have tobe reported. If the two numbers have a small num-ber of digits, and are therefore easy to remember,the ideal behaviour is to look at, and memorise,both numbers. However, as the number of dig-its increases to the limits of your digit span, theoptimal behaviour is to focus on just one of thenumbers and ignore the other one.
Accepted for publication in Psychological Science 4
Clarke & Hunt (2015)
In Session 1 we measured each participant’sdigit span. Stimuli consisted of a randomly-generated sequence of between 2 and 12 digits.On each trial this digit sequence was displayed onthe left or the right hand side of the screen. Thetwo halves of the screen had a di↵erent colouredbackground, with the colours swapping randomlyon a trial to trial basis. The number was dis-played for five seconds, after which it was re-placed with a grey screen for three seconds. Fi-nally, a response screen was presented, consistingof the same coloured background as earlier, withthe prompt ”please enter the number” displayedat the location where the sequence of digits hadoriginally been placed. Participants then typedthe number in using the keypad. A response wasconsidered correct if all the digits were typed inthe correct order. There were nine repetitions ateach number length, giving 99 trials in total. Tri-als were presented in a random order and partic-ipants were given a break halfway through. Aswith the earlier experiments, we model each par-ticipant’s accuracy using logistic regression, giving�(n) as the probability of remembering an n-digitnumber.
The second session of the experiment was sim-ilar to the first. The main di↵erences were thatparticipants were eye-tracked while carrying outthe task (with an Eyelink 1000 as in Experiment1 above), and they were presented with two se-quences of digits to memorize (Figure 1C). Thetwo sequences of digits were equal in length. Whenthe response screen was shown, participants wereprompted to report either the left or the rightnumber. The coloured background was used asan additional prompt. There were 165 trials: 15repetitions for each value of n (2 to 12, as in Ses-sion 1). Trials were presented in random order.
Eye-tracking data were analysed by assigningfixations to one of two 14� ⇥ 2.8� areas of inter-est centred on the two numbers. Fixations fallingoutside of these areas were discarded. Attentionalsplit was then defined as the proportion of timespent fixating the area of interest that receivedthe most attention. So a value of 0.5 indicatesthe participant spent equal time looking at bothnumbers, while a value of 1.0 tells us that par-ticipants spent all their time fixating one of thenumbers. Unlike the previous two experiments, itis not as straightforward to derive the predictedaccuracy given an optimal strategy. We can esti-
mate the probability of remembering both num-bers as �(2n), but the data show that this un-derestimates performance for small n (see sup-plementary materials), presumably due to chunk-ing (Miller, 1956). We assume that for small n,our participants were memorising both numbers.However, as n increases the task becomes increas-ingly di�cult, participants should change strategyand only attempt to remember one number, with aprobability of them getting the trial correct being0.5�(n).
2.2.4 Reaching Experiment
To foreshadow, the results of the above three ex-periments indicated consistent failures in strategicthinking. In the final experiment we took the ba-sic choice we asked of participants in the previousexperiments to a trivially simple level, to ensureour results are not a consequence of participantsfailing to fully understand the decision we wereasking them to make. Six beanbags were placedon a long table (Figure 1D), with two red beanbagsnear the centre, two green beanbags each placedhalfway to the end, and a blue beanbag at eachend. Participants were first asked to sit in a chairplaced by the middle of the table and asked totry and reach, with their back still touching theseat, the red, green, and blue beanbags (demon-strating their own reach span, as in Session 1 ofthe previous experiments). Participants were thenasked to stand, and the experimenter asked themto choose one of three chairs to sit in to pick upa beanbag of a specified color. As in the throwingexperiment, they were not told which of the twobeanbags of this color they would have to pick upuntil after they selected a chair. Participants se-lected a chair once for each of the three colours.The order of colors and which was to be picked upwas randomised for each participant.
3 Results
A typical individual’s data from Session 1 of eachof the saccade, throwing, and memory experi-ments are shown in the top row of Figure 2. Thefull set of data from all participants can be foundin the supplementary information. In the bottomrow of Figure 2, the same participant’s actual be-haviour in the second session is compared to theoptimal strategy derived from their Session 1 per-formance (the blue line).
Accepted for publication in Psychological Science 5
Clarke & Hunt (2015)
●
●
●
●
● ●
●●
0.000.00
0.25
0.50
0.75
1.00
0 5 10 15delta (visual degrees)
prop
ortio
n co
rrect
●
●
●●
●
●
0.00
0.25
0.50
0.75
1.00
0.0 2.5 5.0 7.5 10.0delta (metres)
prop
ortio
n co
rrect
●
●
● ●
● ●
●
●
● ● ●0.00
0.25
0.50
0.75
1.00
0 5 10number of digits
prop
ortio
n co
rrect
●
●
● ● ●●
●
●
0.00
0.25
0.50
0.75
1.00
0 5 10 15delta (visual degrees)pr
opor
tion
of s
acca
des
to s
ide
squa
re
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
● ●
●●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
0.00
0.25
0.50
0.75
1.00
3 4 5 6delta (metres)
norm
alis
ed p
artic
ipan
t pos
ition
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●● ●
● ●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
0.5
0.6
0.7
0.8
0.9
1.0
2 4 6 8 10 12number of digits
atte
ntio
nal s
plit
Figure 2: Results from one participant in each of the detection, throwing and memory experiments. Resultsfrom other participants were similar and are shown in the supplementary materials. Top row shows Session 1accuracy. Accuracy decreased as the task di�culty increased, either by increasing distance (�) or, in the caseof the memory task, the number of digits. Error bars show 95% binomial proportion confidence intervals. Asthe detection task is two-alternative forced choice, chance performance is 50%. The bottom row shows decisionbehaviour from the same participants in Session 2 of the experiments. For the detection task, the participant’schoice is binary and hence each dot represents the proportion of trials on which the participant fixated the sidebox in each condition. In the other two tasks, each dot represents behaviour on a single trial. For the throwingexperiment, position “0” is the center and “1” is the distance to the target color on that trial. The blue lineillustrates the optimal strategy that the participant should adopt, based on their performance in Part 1.
Figure 3 shows the decision behaviour of allindividuals in all four experiments. In the firstthree experiments, the overwhelming majority ofparticipants failed to systematically change theirbehaviour with increasing task di�culty. In thedetection experiment, as in Morvan and Maloney(2012), each participant selects their own individ-ual strategy, and they tend not to vary this strat-egy as di�culty increases. Only one of the 12 par-ticipants exhibited behaviour that approached anoptimal strategy1. For the throwing experiment,there are fewer trials and individual strategies areless consistent; nonetheless the participants stood,in aggregate, just as close to the centre whenthrowing to hoops that were far away as when thehoops were close together. For this experiment wealso examined sequence e↵ects at the trial level tosee if participants had a tendency to learn or to
ipant 3. This participant was the only member of our lab
to participate in any of the studies, although she was naive
to the aims of the experiment.
persist over time with one strategy over another.There was no consistent pattern here (see Sup-plementary Information, Figure SupMat.4). Forthe memory experiment, they were as likely tofixate both digit sequences when they were longas when they were short, although several par-ticipants came closer to adopting a strategy thatwas optimal in this experiment, a detail we willreturn to in the discussion. In the reaching task,designed to check that participants could correctlyunderstand the instructions used in the precedingstudies, participants were uniformly optimal.
Each participant’s choice behaviour can bemodelled by fitting a step function [y = c1 for allx <= s, y = c2 for all x > s, where s is the point atwhich the participant switch strategies (e.g. froma centre to a side strategy). A linear model wouldnot be appropriate, given the nature of the opti-mal strategy as depicted in Figure 2. We fit s,c1, and c2 to the data using least-squares. Fromthis model fit, four patterns of behaviour can beroughly categorized:
Accepted for publication in Psychological Science 6
Clarke & Hunt (2015)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●0.00
0.25
0.50
0.75
1.00
2.5 5.0 7.5 10.0 12.5distance (visual degrees)
prop
ortio
n of
sac
cade
s to
sid
e bo
x
●
●
●●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●0.00
0.25
0.50
0.75
1.00
2 4 6distance (metres)
norm
alis
ed p
artic
ipan
t pos
ition
●
●
●
●●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●●●
●●
●
●
●
●●
●
●●
●
●
●●
●
●●
●●●
●
●
●
●
●●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
0.5
0.6
0.7
0.8
0.9
1.0
2.5 5.0 7.5 10.0 12.5number of digits
med
ian
atte
ntio
nal s
plit
● ●
●
central
side
close armslength fardistance
chai
r sel
ecte
d
●
●
●
●
● ●
●●
●
●●
−1.0
−0.5
0.0
0.5
1.0
−2.5 0.0 2.5 5.0switch point error (degrees)
step
size
● ●●●
●
●
●
●
●
●
●
●
−1.0
−0.5
0.0
0.5
1.0
−2 −1 0 1 2switch point error (m)
step
size ●
●
●
●
●●●
●● ●●
●
−0.50
−0.25
0.00
0.25
0.50
0 2 4 6switch point error (digits)
step
size
●
−1.0
−0.5
0.0
0.5
1.0
switch point error
step
size
Figure 3: First Row: Choice behaviour for the four experiments. In the first three panels, each colouredline shows a di↵erent participant, while in the fourth panel, all twelve participants behaved in the same way,as illustrated by the black line. In each experiment, as task di�culty increases (moving along the x-axis),and optimal participant would exhibit a step function similar to that seen in the bottom row of Figure 2,from maximising their performance on both targets, to maximising their performance for one of the two targets.Participants generally failed to adopt this behaviour, except in the reaching task, where all participants behavedoptimally. Second row: Results of analysis described in text in which step functions were fitted to the individualdata shown above. The x-axis shows the position of the step relative to the optimal location, and the y-axisshows the size of the step. Optimal behaviour in each dimension is represented by the two black lines on eachplot. Filled dots: participants’ model fit R2
> .1; Open dots: R2< .1.
A Perfectly rational behaviour would lead to c1 =0, c2 = 1 [c1 = 0.5 in the memory study], with s
equal to the point predicted by the performanceof each individual.
B A participant may behave rationally, but witha biased or noisy estimate of their own ability.This would lead to c1 ⇡ 0 and c2 ⇡ 1, butwith a value of s that does not match the opti-mal switch point and/or unexplained variance(a low R
2).
C If participants fail to behave rationally, but stillmodify strategy with task di�culty, c2 shouldbe larger than c1, but fall short of the maximumstep size. For example, in the detection study,a step with c1 = 0 and a c2 = 0.5 (leading toa step size of 0.5) would mean the participantalways fixated the central box when the boxeswere close together, and fixated the side box onhalf the trials with a large separation.
D Participants may not modify behaviour at all
or do so in the wrong direction, leading to nostep or a reversed step (further indicated by anR
2 close to 0).
All the model fits from this analysis are givenin Table 1 of the supplementary information, andsummarized in Figure 3 (second row). We focusin this figure on the size of the step, which shouldbe 1 (0.5 in memory study) and the position ofthe step relative to its predicted location underan optimal strategy. As can be seen in this fig-ure, all participants in the reaching experimentare perfectly described by the step function (Cat-egory A of the scheme laid out above). Of thethree other studies, no participants could be de-scribed as being in Categories A and B. Only inthe memory experiment was the step function areasonable model for participant behaviour (9 par-ticipants had an R2 > .1 as indicated by the filledcircles, falling into Category C). In the other twostudies, step size, direction, and location were gen-erally not consistent with a choice behaviour thatwas modified by task di�culty (Category D), with
Accepted for publication in Psychological Science 7
Clarke & Hunt (2015)
the possible exception of a few participants.In Figure 4 we compare overall accuracy in Ses-
sion 2 (observed) to accuracy expected if an opti-mal strategy, as well a simple reference strategy,had been adopted by each participant. Accuracyof the optimal strategy was calculated for eachdistance (or number of digits) as described in themethods section above. This value was calculatedindividually for each participant based on theirpsychometric curve. Expected optimal accuracyas shown in Figure 4 represents the mean over alldistances/di�culties. For saccades, throwing, andreaching, we used expected performance from thecentral location as a reference (“central”), and inthe memory task the reference is expected per-formance from only looking at one number (“sin-gle”). We can see that for the saccade, throwingand memory tasks participants manage to out-perform the reference, but the majority of themfail to achieve optimal performance. This dif-ference is significant when evaluated in a pairedt-test comparing “observed” accuracy to “opti-mal” (detection task: t(11) = �2.45, p = 0.017;throwing task: t(11) = �3.65, p = 0.002; mem-ory task: t(11) = �3.98, p = 0.001). It shouldbe noted that this way of illustrating the mag-nitude of the observed-optimal di↵erences down-plays our e↵ect compared to if we had excludedconditions in which the reference and optimal be-haviours are identical (such as very short distancesin the throwing task where the optimal behaviouris to stand halfway between the targets).
For one participant in the detection task, therewas no di↵erence in their central and optimalstrategy; this occurred because their visual acuitywas good enough to perform above chance evenat the largest eccentricity. Similarly, across de-tection, throwing, and memory tasks, several par-ticipants achieved accuracy in Session 2 that washigher than our predictions based on their Session1 performance, likely due to practice e↵ects. Thissuggests our estimate of optimal accuracy is con-servative.
4 Discussion
We observed a striking failure to make optimal de-cisions in three of the four tasks presented above.In the fourth, where the task was to choose a seatfrom which to reach one of two beanbags, peoplewere all able to select a chair close to one or the
other beanbag when the two beanbags were toofar away to reach from the central chair. This re-sult demonstrates that our participants are able tounderstand the instructions and the constraints ontheir decision well enough to behave sensibly whenthe task is trivial. Why are they seemingly unableto make this decision in the other situations? Thepossible explanations fall into three general cate-gories.
First, participants may fail to estimate theirown performance accurately. A di↵erence betweenthe reaching task and the others is that in theformer, the choice depends on the length of one’sarm, while the others involve learning and remem-bering the limitations of one’s own visual acu-ity, throwing skill, and memory - arguably moreabstract and di�cult to estimate. An inabilityto estimate performance seems unlikely, however,given the extensive practice participants had inthe first session with the range of distance and dif-ficulty levels presented in the second. Performancechanges across these manipulations were stableand systematic (this is clear from the individualcurves presented in the supplementary informa-tion), and people have been previously shown tobe able to accurately estimate and make decisionsbased on expected performance (e.g. Barthelmeand Mamassian, 2009, Paunonen and Hong, 2010).Moreover, if people were estimating performanceincorrectly or imprecisely, we would expect thereto be a switch in strategies with increasing taskdi�culty at some point, but participants wouldnot switch consistently or at the optimal level ofdi�culty (i.e. we would expect to see some evi-dence of bounded rationality, Simon, 1991). Thisdescribes what we denoted as Category B be-haviour in the results of the model fit. No partic-ipants were well described by this category, sug-gesting a more global failure.
Second, participants may fail to frame the deci-sion correctly. Achieving optimality requires theparticipants to make a logical decision (whetherto invest in one option or both), followed some-times by an entirely arbitrary decision (which tar-get/option to invest in). Participants are able tomake this pair of decisions e↵ectively in the reach-ing task, demonstrating that they are capable ofunderstanding the decision and its outcome in thisvery simple context. Perhaps the additional per-formance demands in the detection, throwing, andmemory tasks distract participants from framing
Accepted for publication in Psychological Science 8
Clarke & Hunt (2015)
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●●●
●
●
0.50
0.75
1.00
central observed optimal
prop
ortio
n co
rrect
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
0.25
0.50
0.75
central observed optimal
prop
ortio
n co
rrect
●
●
●●●●●●●●●●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
0.10
0.35
0.60
single observed optimal
prop
ortio
n co
rrect
●
●
●
0.50
0.75
1.00
central observed optimal
prop
ortio
n of
cor
rect
tria
ls
Figure 4: Accuracy of each participant in each experiment (“observed”) relative to how accurate wewould expect each observer to be given a simple reference strategy and an optimal strategy. Note:these figures take the average performance over a range of stimuli, including cases in which there isno di↵erence in central/single and optimal performance. Hence these figures under-estimate the sizeof the e↵ect. In the reaching task, the observed accuracy is higher than optimal because eight out oftwelve participants happened to choose the seat in front of the randomly-selected reach target (withinthe variation we would expect based on chance)
the task appropriately, or trial-to-trial changes intask di�culty prevent participants from setting asingle threshold at which to switch between strate-gies. During debriefing, we asked participants inthe throwing experiment to tell us how they ar-rived at their decisions, and some were arbitrary,while others became focused on finding some pat-tern in the order of targets selected (even thoughthey were told this was random). More partic-ipants fell into Category C in the memory taskthan in the detection and throwing tasks (i.e. theymodified their behaviour with di�culty). Unlikein the other experiments, in which decisions werediscrete, behaviour in the memory task unfoldsover a 5-second interval, so the participants mayhave been able to learn from the cumulative e↵ectof their choice behaviour more e↵ectively.
A third possible explanation is that participantsare prioritising something other than accuracy inthe task. Most (but not all) participants were bi-ased towards investing in both potential targetsrather than focusing on one. Performing the taskin more di�cult circumstances may be seen as achallenge or an opportunity for learning, while se-lecting one or the other option takes away the chal-lenge and puts the outcome in the hands of chance,which may be seen as a failure to be responsiblefor the outcome. Relatedly, it may be a partic-ularly unpleasant experience to guess incorrectlyand have the non-selected option turn out to bethe target, and participants who happen to experi-ence this loss on a series of trials in a row may havebeen discouraged from investing in a single option
on subsequent trials. It should be noted that inthe detection experiment of Morvan and Maloney(2012) participants were given substantial mone-tary rewards for accuracy, which does not seem tohave made them any more likely to adopt an op-timal strategy. Nonetheless, our participants werenot explicitly rewarded for accuracy, so we cannotrule out that some of them may have decided toprioritise their own interest/pleasure in the taskover accuracy.
There are many ways to be sub-optimal, andthe fact that no single explanation from thoselisted above can account for all the results sug-gests that they all play a possible role to someextent and in some individuals. Nonetheless, onlyin the reaching experiment do participants demon-strate behaviour that could be classified as opti-mal. It is easy to imagine many scenarios in whichthe decision to invest all one’s resources in onegoal versus to divide resources between two goalswould have serious consequences for an organ-ism’s survival (e.g. o↵spring investment; forag-ing). Given this, why is our ability to make a logi-cal choice under these circumstances so easily dis-rupted? As situations become more complex, withincreasing numbers of tasks and goals and decreas-ingly reliable ways of estimating likely success, thecomputations involved in determining the opti-mal strategy become more resource-intensive andtime-consuming, and the potential pay-o↵ dimin-ishes (e.g. DeMiguel, Garlappi, and Uppal, 2009).It is our suggestion that, as a consequence of thecomplexity involved in deciding between multiple
Accepted for publication in Psychological Science 9
Clarke & Hunt (2015)
goals in most situations, people in general fail toemploy a sensible strategy even when the requiredcomputations are extremely simple. This leads tothe paradoxical conclusion that people’s choicesabout how to allocate resources across multipletasks are probably not optimal in principle, butthey are usually adequate for complex situations.It is only as the tasks become fewer and the situ-ation simpler that the failure to adopt a sensiblestrategy becomes both more apparent, and alsomore detrimental.
Author Contributions
ADFC and ARH developed the study conceptand design. ADFC performed the data analysis.ADFC and ARH co-wrote the manuscript and ap-proved the final version for submission.
Acknowledgments
The authors thank Alex Irvine, Melissa Spilioti,and a group of third-year students for their helpwith data collection, and Peter Neri, Rob McIn-tosh, Sandrina Ritzmann, and James Sheils forvaluable input. This work was supported by aJames S McDonnell Foundation Scholar Award(ARH).
References
Simon Barthelme and Pascal Mamassian. Evaluationof objective uncertainty in the visual system. PLoSComput Biol, 5(9):e1000504–e1000504, 2009.
Victor DeMiguel, Lorenzo Garlappi, and Raman Up-pal. Optimal versus naive diversification: How in-e�cient is the 1/n portfolio strategy? Review ofFinancial Studies, 22(5):1915–1953, 2009.
Martin Gardner. Matheatical games. Scientific Amer-ican, October:180–182, 1959.
M. Hayhoe and D. Ballard. Modeling task controlof eye movements. Current Biology, 24(13):R622–R628, 2014.
Amelia R Hunt and Raymond M Klein. Eliminatingthe cost of task set reconfiguration. Memory & cog-nition, 30(4):529–539, 2002.
Daniel Kahneman and Amos Tversky. Choices, values,and frames. American psychologist, 39(4):341, 1984.
Melissa M Kibbe and Eileen Kowler. Visual searchfor category sets: Tradeo↵s between exploration andmemory. Journal of vision, 11(3):14, 2011.
Konrad P Kording and Daniel M Wolpert. Bayesianintegration in sensorimotor learning. Nature, 427(6971):244–247, 2004.
George A Miller. The magical number seven, plus orminus two: some limits on our capacity for pro-cessing information. Psychological review, 63(2):81,1956.
Camille Morvan and Laurence T. Maloney. Humanvisual search does not maximize the post-saccadicprobability of identifying targets. PLOS Computa-tional Biology, 8(2), 2012.
J. Najemnik and W. S. Geisler. Optimal eye movement strategies in visual search. Nature, 434:387–391, 2005.
Ipek Oruc, Laurence T Maloney, and Michael S Landy.Weighted linear cue combination with possibly cor-related error. Vision research, 43(23):2451–2468,2003.
Sampo V Paunonen and Ryan Y Hong. Self-e�cacyand the prediction of domain-specific cognitive abil-ities. Journal of personality, 78(1):339–360, 2010.
Herbert A Simon. Bounded rationality and organiza-tional learning. Organization science, 2(1):125–134,1991.
Nir Vulkan. An economists perspective on probabilitymatching. Journal of economic surveys, 14(1):101–118, 2000.
Daniel M Wolpert and Michael S Landy. Motor controlis decision-making. Current opinion in neurobiology,22(6):996–1003, 2012.
Shih-Wei Wu, Mauricio R Delgado, and Laurence TMaloney. Economic decision-making compared withan equivalent motor task. Proceedings of theNational Academy of Sciences, 106(15):6088–6093,2009.
Hang Zhang, Camille Morvan, Louis-AlexandreEtezad-Heydari, and Laurence TMaloney. Very slowsearch and reach: Failure to maximize expected gainin an eye-hand coordination task. PLoS computa-tional biology, 8(10):e1002718, 2012.
Supplementary Materials
See figures below.
Accepted for publication in Psychological Science 10
Figure SupMat.1: The complete results from the 12 participants in the detection experiment. Top set: resultsfrom Session 1. The black line is the psychometric curve fit to their data. Middle set: Proportion of saccadesto the side square in Session 2 (black dots) compared to an estimate of optimal fixation behaviour based onSession 1 performance (blue line). Bottom set: Detection accuracy (black dots) compared to predicted accuracyunder a baseline (blue line) and optimal (red line) strategy
Accepted for publication in Psychological Science 11
Figure SupMat.2: The complete results from the 12 participants in the throwing experiment. Top set: resultsfrom Session 1. The black line is the psychometric curve fit to their data. Middle set: standing position at eachof six distances in Session 2 (black dots) compared to an estimate of optimal standing position based on Session1 performance (blue line). Y-axis range has been restricted to 0-1, which removed data points for participants7, 8, and 9, who sometimes stood outside the range of the hoops. Bottom set: Throwing accuracy (black dots)compared to predicted accuracy under an optimal (red line) strategy.
Accepted for publication in Psychological Science 12
Figure SupMat.3: The complete results from the 12 participants in the memory experiment. Top set: resultsfrom Session 1. The black line is the psychometric curve fit to their data, giving �(n) as the probability ofremembering an n-digit number. Middle set: Attentional split (relative proportion of time spent on one digit,black dots) in Session 2. Bottom set: Memory accuracy in Session 2 (black dots) compared to predicted accuracyunder an optimal strategy, calculated as the maximum of the red (�(n)) and orange (0.5�(n)) lines
Accepted for publication in Psychological Science 13
Clarke & Hunt (2015)
1 2
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
12
34
56
78
910
1112
0 10 20 30 40 50 0 10 20 30 40 50Trial Number
Nor
mal
ised
Sub
ject
Pos
ition
Figure SupMat.4: This figure shows trial-to-trial sequential variation in participant standing position (thered line) in the throwing experiment. The first column shows the first block, the second column shows thesecond block. Each row corresponds to an individual participant. The black bars indicate the trials in whichthe optimal strategy is to stand by the hoop, otherwise they should stand at the mid-point. Discontinuities inthe red line correspond to trials in which the participant stood further than 1.5 normalized distance units fromthe center (Y axis range has been restricted to 1.5 to improve visibility).
Accepted for publication in Psychological Science 14
= predicted value for s based on participants performance curve;c1 and c2 = value of the fitted model before and after the step. Minimum c1 in the memory studyis 0.5. Participant 11 in the detection experiment had unusually good peripheral vision, leading tono point at which they should switch from looking in the center (hence no S
o
pt). In the reachingexperiment, for all participants, s = s
o
pt, c1 = 0; c2 = 1 and R
2 = 1.
Accepted for publication in Psychological Science 15
