Acta Psychologica 68 (1988) 183-196 North-Holland

183

DECISION DIFFICULTY AND IMPRECISE PREFERENCES *

David BUTLER and Graham LOOMES University of York, Heslington, UK

Conventional utility analysis assumes that individuals can give ‘certainty equivalent’ valuations of risky prospects. Our experimental evidence suggests that many people do not find this task easy. This paper considers evidence from more than 100 participants in a ‘payment-by-results’ experi- ment. The subjects were asked to place certainty equivalent valuations on some simple gambles, after having made a short series of choices in the vicinity of those valuations. At each decision point they were asked to record how confident they felt about these decisions. We discuss the light this sheds on the ‘sphere of haziness’ in which individuals often need to operate.

Introduction

When constructing a model of human decision making, it is appeal- ing and convenient to suppose that individuals’ preferences are basi- cally well-behaved. For example, Savage (1954: 17-18) assumes that for any two acts, f and g, either f is not preferred to g, or g is not preferred to f, or both.

We can write the above proposition as:

Eitherfsg, orgsf, orflg andglf, i.e. f-g. (1)

This rules out any individual simultaneously preferring f to g and g to f; it also rules out the possibility that neither f 5 g nor g I f, i.e. that f and g are simply incommensurable. Savage (ibid. p. 21) consid- ers the possibility of ‘admitting that some pairs of acts are incompara- ble. This would give expression to introspective sensations of indecision

* The experimental work in this paper was funded by Economic and Social Research Council Awards B 00 23 2127 and B 00 23 2163.

Our thanks to Suky Thompson for the computer programming, and to Norman Spivey for assisting with analysis of the data.

Requests for reprints should be sent to G. Loomes, Centre for Experimental Economics, University of York, Heslington, York YOl 5DD, UK.

OOOl-6918/88/$3.50 0 1988, Elsevier Science. Publishers B.V. (North-Holland)

184 D. Butler, G. Loomes / Decision difficulty

or vacillation which we may be reluctant to identify with indifference.’ But he conjectures that this ‘would prove a blind alley losing much in power and advancing little, if at all, in realism; but only an enthusiastic exploration could shed real light on the question.’

This is a point to which we shall return. But let us first examine certain conventional assumptions a little further. In particular, consider the notion of betweenness: when there are three acts, f, g and h, g is saidtobebetweenf andhiff<g<horfrgzh.Supposefrgsh and f-c h. Savage’s Theorem 4 (ibid. p. 73) states that under these circumstances there is one and only one p such that pf + (1 - p) h - g.

If f, g and h are sums of money within most people’s daily experience, such that f-c g -c h, it may seem plausible that there should not be more than one probability p such that pf + (1 - p)h - g; and it then also seems plausible that the certainty of some amount of money g* > g should be strictly preferred to the lottery pf + (1 - p)h.

However, as this paper will argue, the majority of individuals do not find it easy to state with high precision their certainty equivalent valuation for lotteries, even when there are only two possible conse- quences involving familiar sums of money and quite straightforward probabilities.

Most people would willingly accept that if 0 < p < 1, the certainty equivalent g should lie between f and h; and that some other value g* cannot simultaneously be that certainty equivalent. Yet many individu- als cannot state with complete confidence whether their certainty equivalent is g or g*. We shall suggest that such imprecision, and the ‘errors’ that result, may be of considerable importance to the testing of competing theories under risk and uncertainty. In this paper we report and discuss a pilot study using ‘Difficulty Scores’ to shed some light on this issue. We shall outline one approach to modelling the phenom- enon, and present some evidence bearing on the model.

Why does it matter?

Modem decision analysis is largely founded upon sets of axioms or postulates such as those proposed by von Neumann and Morgenstem (1947) and Savage (1954). However, during the past 40 years a substan- tial amount of evidence has accumulated which casts doubt on the

D. Builer, G, Laomes / Decision difficulty 185

descriptive power of some - indeed, most - of those basic axioms (for a survey and discussion, see Machina 1987). This has stimulated the development of a number of other models which attempt to explain various of the observed ‘ violations’, and we now have the task of trying to discriminate between the rival models. In particular, we must try to decide to what extent the departures from the conventional axioms are due to imprecision and simple human error, and to what extent they constitute systematic behaviour predicted by an alternative theory.

The point can be illustrated with reference to the ‘preference rever- sal’ phenomenon, first reported by Lichtenstein and Slavic (1971) and Lindman (1971). Subjects were presented with two lotteries: a ‘$-bet’, which offered a small probability of a large win, and a larger probabil- ity of a small loss; and a ‘P-bet’, which offered a smaller probability of loss, and a larger probability of a moderate win. They showed that a substantial proportion of subjects who chose the P-bet in a straight choice also placed a higher certainty equivalent valuation on the $-bet, apparently violating the principle that individuals’ preference orderings are stable (in the short run, at least) and transitive. The opposite ‘ violation’ - those who chose the $-bet while placing a higher valuation on the P-bet - was also observed, but much less frequently.

Preference reversals have been reproduced a number of times since, often by researchers who were initially sceptical about the results, and the phenomenon appears to be real and resilient. But is it evidence of some alternative model of decision making, or simply the product of imprecision and error ? In a recent paper, MacCrimmon and Smith (1986) have shown how preference reversals may be due to imprecision. Essentially, their argument is that rather than having precise points of indifference between alternatives, individuals actually have ‘equiv- alence intervals’.

In their experiments, they used a P-bet offering a 0.96 chance of $2.50 and a $-bet giving a 0.24 chance of $10.00 (in both cases, the other consequence was zero). While the equivalence interval for the P-bet was limited by the upper consequence of $2.50, they argued that the equivalence interval for the $-bet might include values considerably above $2.50, even for individuals who would choose the P-bet in a pairwise choice. If at least some of these individuals state a value for the $-bet drawn from the upper end of the interval, a standard preference reversal will be observed.

186 D. Butler, G. Loomes / Decision difficuIty

It is not obvious exactly how such equivalence intervals should be modelled. One possibility is that there is some ‘inner range’ of values such that any value in that range might be considered equivalent to the lottery being evaluated, but where an individual may not be able to say that one value is a more likely candidate than another. There might also be some ‘outer range’ which an individual may consider to be less likely to contain the certainty equivalent, or which may contain values that, after sufficient deliberation, the individual is able to rule out. Beyond that are values which can be disregarded almost instantly - the most obvious of these being values which transparently dominate, or are dominated by, the lottery.

Our objective, then, is to gain some insights into the nature and extent of any such intervals and consider what factors might affect them, in order to assess how far they may account for the behaviour we observe and enable us to make appropriate allowances when trying to discriminate between competing theories. In the next section we de- scribe the approach we used.

Experimental design and working hypotheses

The experiment reported in this paper was concerned with what should, in theory, be one of the simplest kinds of problem: establishing an individual’s certainty equiv- alent for a two-consequence monetary lottery. We shall be concerned with four lotteries, all with the same expected value, as follows:

Al: 0.2 chance of E30.00; 0.8 chance of 0. A2: 0.4 chance of E15.00; 0.6 chance of 0. A3: 0.6 chance of flO.OO; 0.4 chance of 0. A4: 0.8 chance of E7.50: 0.2 chance of 0.

These four lotteries were the subject of an earlier paper-Loomes (1988)-which compared three different procedures for eliciting certainty equivalents. One of the patterns that emerged very clearly from that paper was the tendency for the majority of individuals to round their stated certainty equivalents to the nearest pound or 5Op, even though they were invited to state their equivalents to the nearest penny, and despite the fact that real money incentives were involved. This may be taken as prima facie evidence that individuals do not have psychologically costless access to clearly defined preferences, even for the relatively straightforward lotteries under considera- tion.

In this paper we focus on one of the three procedures described in Loomes (1988) - the form of question known as ICV (Iterative Choice and Valuation). To explain how these questions worked, it is necessary to describe briefly the experimental context.

D. Butler, G. L.oomes / Decision difficult 187

___________-l______-_-_-________---------;;~:61 _-------_---_-_________ 10

A 10.00 0.00 _______-_-_~m______-____ih___________~__-________________________~-~_~~

B 3.70 3.70 ____________________---__ --__~_-__----_--~_____---------------~-~-~_~~-

M 40

Heas.e chose the alternative you prefer by pressing either A or B, and pressing the key labelled RETURN.

Fig. 1. ICV question initial display.

Subjects were recruited from among conference/summer school visitors to the campus of the University of York. They were randomly allocated to sit at separate computer terminals, and, before the experiment began, were asked to choose at random two sealed envelopes, one brown, one white. Each brown envelope contained a cloakroom ticket numbered from 1 to 100 inclusive, while the white envelope contained the number of the question which, at the end of the session, would be played out for real. It was explained that an individual’s payment for taking part in the experiment would depend entirely on how their decision in that particular question worked out.

At the beginning of each ICV question, participants were presented with the kind of display shown in fig. 1.

The numbers inside the boxes of the grid represent sums of money in pounds and pence, and the numbers set into the top row of hyphens refer to the cloakroom tickets. Thus any individual who chose alternative A stood to receive E10.00 if their ticket was numbered from 1 to 60, and zero if the ticket was anything from 61 to 100. The numbers at the base of each column showed at a glance the chances out of a hundred of receiving any sum. Alternative B, of course, offered the certainty of E3.70, and initially participants were simply asked to choose which alternative they preferred.

When they had made this choice and confirmed it, the grid, together with their decision, remained in the upper half of screen, the the display shown in fig. 2 appeared in the lower half.

Subjects were requested to move an illuminated cursor (initially located in the centre of the box) to whichever position they thought best reflected how difficult they felt the choice between options A and B had been. The cursor could be moved to any one of 51 positions; and although no numbers were displayed on the screen, the final

Confident Slightly unsure

I I Very _____-_------------------------------ Very

confident ( 1 unsure i,,d& ___--_-___-_--_--_----- ________ -- _____ indeed

I I Fairly confident Unsure

Fig. 2. Difficulty Score display,

188 D. Butler, G. Lmmes / Decision diffiadty

position of the cursor was registered by assigning numbers rising from 0.2 (least difficult) to 10.2 (most difficult) by increments of 0.2. That is, we imputed numerical scores to the subjective expressions of decision difficulty. For expositional convenience, we shall average these scores for particular subsets of participants, but we do not claim to be working with much more than an ordinal scale.

Having confirmed a decision, and registered a Difficulty Score, the display in fig. 1 changed. Depending on the choice made, the amounts offered by alternative B would alter. For example, if a participant stated a preference. for A over B in the initial choice, the 3.70 would be replaced by 7.10, and the participant would be asked to choose between A and this higher certainty - this new choice being followed by a request for a fresh expression of decision difficulty.

Some ICV questions began with alternative B offering 3.70, while others initially offered the certainty of 8.30. Depending on the starting value and the sequence of choices made subsequently, a participant was presented with up to four different certainties, and then finally was asked to type in the smallest sum, within the range determined by those previous choices, which he/she would be just willing to accept as an alternative to the lottery.

For instance, an individual presented initially with the display in fig. 1 might record the sequence of choices A, B, B, A in response to certainties of 3.70, 7.10, 5.40 and 4.55 being offered as alternative B. He/she would then be reminded of the smallest certainty accepted in preference to A (i.e. 5.40) and the largest amount rejected in favour of A (i.e. 4.55) and would be asked to type in the smallest amount in that range which would be just acceptable. The question was complete when this valuation was confirmed, together with a final Difficulty Score.

The device used to ensure that participants answered the questions as truthfully and precisely as possible was essentially the one proposed by Becker, DeGroot and Marschak (1964). Participants were told that if a question involving a valuation was selected to be played for real, an offer would be drawn at random from a distribution of offers, and the following rule applied: if the offer was equal to or greater than the final valuation they had typed in, they would be paid the full amount of the offer and would give up the lottery. However, if the offer was less than the amount they had stated to be the minimum acceptable, they would play out the lottery and be paid according to the cloakroom number in their brown envelope.

Individuals who are dynamically consistent will be induced by this device to reveal their true certainty equivalent valuations. And since, in the ICV form of question, participants are not permitted by the computer to enter any value outside the range determined by their earlier sequence of choices, individuals should also be induced to make each choice consistent with their underlying true preferences.

We hoped that the Difficulty Scores would provide various kinds of information about the equivalence intervals. For example, consider an individual presented with the question involving lottery A3, as in fig. 1. Taking the earlier example, suppose that the individual’s sequence of choices is A, B, B, A, and his/her final stated valuation is 5.00. the various certainties offered by the computer were, in ascending order, 3.70,‘4.55, 5.40 and 7.10. If the equivalence interval is narrow, we might expect low Difficulty Scores for all four values, whereas if the ‘inner range’ is sufficiently wide to cause the individual to round to the nearest pound, the scores at 4.55 and 5.40 might both be

D. Butler, G. Loomes / Decision difficulty 189

relatively high. Markedly different scores at values which are approximately equidistant from the final valuation may suggest some ‘skewness’ in the shape of the equivalence interval. Also, comparisons of patterns of scores across lotteries may give some indication of the way in which the width and shape of intervals are affected by the lottery parameters.

We emphasise the exploratory nature of our study, but will organise our results around the following working hypotheses:

(i) The ‘skewness’ of equivalence intervals tend to change with the parameters of the lottery: since all lotteries in this experiment have the same lower consequence and the same expected value, we would expect equivalence intervals to stretch further and further above final valuations as we move from A4 to Al, with the pattern of Difficulty Scores reflecting these changes.

(ii) The bigger the variance of the lottery, the broader the equivalence interval, reflected by higher scores persisting over a wider range of certainty values.

A third hypothesis is put forward rather more tentatively:

(iii) Individuals who employ simple, fii rules will express greater confidence in their decisions. This hypothesis is advanced more tentatively because the experiment was not designed to provide a thorough test, and our purpose will, at most, be to see whether there is circumstantial evidence which provides indications for firmer hypotheses that may be tested more rigorously in the future.

Results and Discussion

Tables Al-A4 show the average Difficulty Scores at each certain value for all those whose choices involved the same set of certainties. For example, in table Al, the row BAB shows the mean scores of all those who chose B when B offered 8.30, then chose A when B offered 4.90, chose B again when the value was raised to 6.60, and were finally presented with a choice between Al and the certainty of 5.75 (We do not distinguish in these tables between those who chose A and those who chose B at this final stage, although we have broken the data down in that way and can provide more detailed tables on request together with some graphs constructed from these tables. Note also that subjects who gave final valuations below the lowest value which the computer was programmed to offer, or above the highest offered value, have been omitted due to the lack of data points.)

In tables Al-A4 the columns are spaced in approximate proportion to the distances between values, and in each row we have marked with a cross the location of the mean certainty equivalent valuation given by participants faced with that sequence of choices. We have also underlined the highest average Difficulty Score in each row.

A priori we would have expected the highest Difficulty Score to be associated with the value closest to the mean. Inspection of tables Al-A4 shows this to be frequently the case. However, there also appears to be some tendency for the values with the

D. Butler, G. Lames / Decision difficul@

Tab

les

A3&

A4

3.20

3.

70

4.05

4.

55

4.90

5.

40

5.75

6.

25

6.60

7.

10

7.45

7.

95

8.30

8.

80

n

Mea

n

I I

I I

I I

I I

I I

I I

I I

CE

A3:

B

BA

AB

B

BA

B

AB

A

BA

A

AA

B

A4:

B

BA

AB

B

BA

B

AB

A

BA

A

AA

B

4.72

-

4.60

X

4.29

5.

09

- X

3.14

2.45

2.80

4.13

3.35

4.

03

2.98

X

2.51

3.

58

- X

3.39

2.50

2.20

2.20

4.62

1.

46

36

4.73

3.95

1.

71

X

3.78

3.

92

X

3.36

-

1.41

2.05

.b

13

5.83

b s 3

29

6.16

,-

r

2.52

X

4.67

X

1.56

5

7.50

1

2.80

3

7.47

t

2.55

0.

91

3.04

1.

61

X

4.40

2.

81

X

8.40

-

Y

1.38

13

6.

19

0.20

3.95

X

1.24

5

3.99

2.87

0.50

8

14

4.34

g:

3

0.75

35

4.44

e K

s

16

5.65

9

0.20

1

6.65

0.85

0.

75

4 7.

49

192 D. Butler, G. Laomes / Decision difficulty

highest Scores to be below the mean more often than they are above. This tendency seems to‘operate across the range of sequences and across all four lotteries.

It is possible that this may be due, in part at least, to some kind of reference point effect combined with the wording of the final element of the ICV question, where participants are asked to type in the smallest amount (within the range determined by their previous choices) which would make them just willing to accept B rather than A. Throughout each question, alternative A is the fixed element, the thing that appears as given. If there is uncertainty about the exact point of indifference, asking for a value which makes them willing to give up the reference lottery and accept B instead may induce participants who ‘want to be on the safe side’ to state a value above the ‘true’ point of indifference: hence amounts just below the stated value will tend to be closer to the ‘true’ point of indifference than amounts at or just above the stated value.

As we move from A4 to Al, we see a tendency for the overall mean valuation to rise, and correspondingly for more observations and higher Difficulty Scores to appear in the bottom right hand comers of the tables. By itself, this doesn’t tell us whether raising the prize has stretched the equivalence interval to the right and thereby induced higher mean valuations, or whether the certainty equivalents are higher for reasons other than increased imprecision, e.g. considerations of regret. Indeed, the two things are not mutually exclusive: if an increase in the range of the lottery increases uncertainty about whether the stated value is the right one, and if the costs of making the ‘wrong’ decision in one direction are increased relative to the costs of making the opposite ‘wrong’ decision, the combination of anticipated regret and uncertainty aversion may well reinforce each other to produce high& stated certainty equivalents.

Although the evidence so far does not appear to be inconsistent with the working hypotheses proposed in the previous section, more detailed examination of the tables indicates some patterns which may allow us to elaborate upon those provisional hypotheses.

First, we note that the patterns of Difficulty Scores for a given lottery vary with the different levels of final valuations. For this reason, it does not seem appropriate to talk about patterns of scores for each lottery as a whole: more useful comparisons across lotteries may be made by comparing the scores for each particular sequence of certain values.

In making such comparisons, we find, for example, that the scores for each value in each row of table Al are generally higher than for the corresponding values in table A4; indeed, if we exclude sequence BAA in table A4 (where n = l), the scores in table Al are higher in 19 out of 20 cases. This may seem like strong evidence in favour of the hypothesis that difficulty rises with the range of the lottery. However, a comparison between Al and A2 shows a much less clear difference. Despite the fact that the upper consequence of Al is double that of A2, 9 of the 24 scores are higher for A2 than Al, and a tenth is equal. It is also noticeable that these 9 tend to be clustered between 4.05 and 6.60, whereas Al scores are higher at the two ends of the tables. This may indicate that although the wider range of Al tends to increase the extent of imprecision, the more even balance of the probabilities in A2 (0.4 : 0.6 as compared with 0.2 : 0.8) may increase the intensity of difficulty over an intermediate range of values.

This suggests that a more even balance of probabilities combined with a modest increase in the range of the lottery may raise the degree of difficulty almost as much as

D. Butler, G. Loomes / Decision difficulty 193

increasing the range of the lottery much more while keeping the same balance of probabilities. That, at least, is one interpretation of the evidence from the comparison between A3 and A4, where A3 produces higher Difficulty Scores in 17 out of 20 cases (again excluding BAA).

Focussing attention on the right hand end of the tables, the evidence also seems consistent with the idea that increasing the upper consequence while holding the expected value constant may tend to increase the right hand tail of most individuals’ subjective probability distributions and lead to correspondingly higher Difficulty Scores. If we look at all rows where n > 5, scores at 7.10 and above fit very well with this suggestion, with the ranking at each value being perfectly correlated with the range of the lottery, even for sequences where this correlation has not been preserved at lower values.

So although the patterns of Difficulty Scores may be somewhat more complex than suggested in hypotheses (i) and (ii), those hypotheses are broadly supported by the data. There is also some evidence favouring hypothesis (iii). Without a more specialised study aimed at discovering in more detail what rules of thumb, if any, people used to make their decisions, what we can say here is limited. However, there were a minority of participants whose answers across these and other questions indicated that they were consistently using an expected value rule to guide their choices and valuations. Hypothesis (iii) suggests that those who employ clear rules of this kind are likely to report less difficulty in their decisions. In tables Al-A4, the scores of people using an expected value rule were included in the averages for sequences BAB and ABA, and we notice in those tables a tendency for these scores to be lower than comparable scores from adjacent sequences. Examination of 17 participants believed to be using the expected value rule show that their Difficulty Scores were indeed lower than the average for those sequences, and often formed a distinctive pattern-very low and flat for all values, except perhaps for the value closest to 6.00, where the score rose somewhat. In other words, it seems that use of this rule greatly reduced the dispersion of the subjective probability distribution of certainty equivalents.

Conclusion

The weight of evidence from this experiment and elsewhere suggests that even for simple lotteries involving just two monetary consequences well within normal experience and straightforward probabilities, many people find it difficult to be precise about their certainty equivalent valuation. Although most individuals would probably be willing to acknowledge that the same lottery cannot simultaneously be worth both g and some greater amount g* , there is good reason to believe that if g and g* are within a certain range of each other, many individuals will be unable to tell which precise value is their ‘true’ certainty equivalent.

194 D. Butler, G. Loomes / Decision difficulty

This shows up directly in the patterns of Difficulty Scores, and indirectly in the marked pattern of rounding reported in Loomes (1988). That earlier study showed that although the ICV type of question tended to reduce the degree of rounding by comparison with other methods of elicitation, about two-thirds of all final valuations were still expressed as some multiple of 5Op, with the degree of rounding particularly pronounced for valuations of Al and A2. This appears consistent with the suggestion by Beach et al. (1974) of a subjectively acceptable error, which they also refer to in terms of an ‘equivalence interval’, proportional in size to the magnitude of the correct answer.

However, although there is a tendency for the Difficulty Scores to stay higher over a wider interval as we move from A4 to Al, increasing the range of a lottery may not be the only significant factor affecting the degree of decision difficulty: the results also suggest that there may be some interactive effect from the balance of the probabilities of good and bad consequences.

Altogether, the evidence suggests that even in relatively simple problems there is what von Neumann and Morgenstem called a ‘sphere of haziness’ which is not insignificant. Although it might be tempting to try to model this as some kind of well-behaved error term, the evidence cautions against that solution, since it does not appear to be indepen- dent of the particular characteristics of different decision problems. Although the present study has not sought to investigate the extent to which this haziness or imprecision can account for various ‘ unconven- tional’ patterns of behaviour, a number of possibilities would seem to deserve serious consideration. The evidence in this paper gives some credibility to the ‘imprecise equivalences’ explanation for at least some preference reversals. Uncertainty about ‘true’ values may also help to explain the disparities between buying and selling prices for lotteries (e.g. MacCrimmon and Smith 1987), and between willingness to pay and compensation demanded (Knetsch and Sinden 1984). It also seems possible that the difficulty of accessing a clear and precise preference ordering may give rise to the widely recognised phenomenon of intran- sitive indifference, (see Fishburn 1970) where A - B, and B - C, but A > C. One explanation for this is that the differences between the neighbouring pairs A&B, and B&C are each too small to be worth worrying about, whereas between the more distant pair A&C they are not. Also relevant here may be notions of the ‘cost of thinking’ and

D. Butler, G. L.oomes / Decision difficulty 195

trade-offs between the costs and benefits of cognitive effort (in this case, the costs and benefits of trying to penetrate the sphere of haziness) explored variously by Marschak (1968), Shugan (1980) and Johnson and Payne (1985).

Whether imprecision can account for most, or even some, of the observed violations of conventional choice axioms is, of course, still an open question. Nor will it be easy to answer, since the very instruments we try to use, such as Difficulty Scores, are themselves vulnerable to imprecision and error-especially in the more difficult problems, as the rather more erratic patterns for Al indicate. Nevertheless, we do not consider further research into these issues to be ‘a blind alley advanc- ing little, if at all, in realism’. Imprecision about preferences is a real factor, and our ignorance about its nature and extent seriously com- plicates the problem of testing predictions and discriminating between alternative theories. We believe that this is an important issue for decision theorists and experimenters to address, and we think that the Difficulty Score approach may be able to make a contribution, al- though we hope it will be only one of a number of instruments developed in the course of ‘an enthusiastic exploration (which) could shed real light on the question’.

References

Beach, L.R., B. Beach, W. Carter and S. Barclay, 1974. Five studies of subjective equivalence. Grganisational Behaviour and Human Performance 12, 351-371.

Becker, G.M., M.H. DeGroot and J. Marschak, 1964. Measuring utility by a single-response sequential method. Behavioral Science 9, 226-32.

Fishbum, P.C., 1970. Intransitive indifference in preference theory: a survey. Operations Research 18,207-228.

Johnson, E. and J.W. Payne, 1985. Effort and accuracy in choice. Management Science 31, 395-414.

Knetsch, J.L. and J.A. Smden, 1984. Willingness to pay and compensation demanded: experimen- tal evidence of an unexpected disparity in measures of value. Quarterly Journal of Economics 99.507-21.

Lichtenstein, S. and P. Slavic, 1971. Reversals of preference between bids and choices in gambling decisions. Journal of Experimental Psychology 89.46-55.

Lindman, H., 1971. Inconsistent preferences among gambles. Journal of Experimental Psychology 89, 390-397.

Loomes, G., 1988. Different experimental procedures for obtaining valuations of risky actions: implications for utility theory. Theory and Decision 25, l-23.

MacCrimmon, K.R. and M. Smith, 1986. Imprecise equivalences: preference reversals in money and probability. Working paper 1211, University of British Columbia.

MacCrimmon, K.R. and M. Smith, 1987. Transaction reversals: buying and selling lotteries. Working paper 1230, University of British Columbia.

196 D. Butler, G. Lames / Decision difficulty

Machina, M.J., 1987. Choice under uncertainty: problems solved and unsolved. Journal of Economic Perspectives 1, 121-154.

Marschak, J., 1968. Economics of inquiring, communicating, deciding. American Economic Review Proceedings 5&l-18.

Neumann, J. van and 0. Morgenstem, 1947. Theory of games and economic behavior. Princeton, NJ: Princeton University Press.

Savage, L.J., 1954. The foundations of statistics. New York: Wiley. Shugan, S., 1980. The cost of thinking. Journal of Consumer Research 7, 99-111.