A Multi-Stage Test for a Normal Mean

A Multi-Stage Test for a Normal MeanAuthor(s): A. ScottSource: Journal of the Royal Statistical Society. Series B (Methodological), Vol. 30, No. 3(1968), pp. 461-468Published by: Wiley for the Royal Statistical SocietyStable URL: http://www.jstor.org/stable/2984248 .

Accessed: 28/06/2014 12:40

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Wiley and Royal Statistical Society are collaborating with JSTOR to digitize, preserve and extend access toJournal of the Royal Statistical Society. Series B (Methodological).

http://www.jstor.org

This content downloaded from 92.63.101.146 on Sat, 28 Jun 2014 12:41:00 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=black

http://www.jstor.org/action/showPublisher?publisherCode=rss

http://www.jstor.org/stable/2984248?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


1968] 461

A Multi-stage Test for a Normal Mean

By A. SCOTT London School of Economics and Political Science

[Received July 1967. Revised September 1967]

SUMMARY Grundy, Healy and Rees (1956) have examined the problem of deciding how much experimentation to carry out when trying to choose the best of two processes. This paper extends their results to situations in which more than one additional stage of observations can be taken.

1. INTRODUCTION

THIS paper deals with the problem of deciding how much testing to carry out in a simple experiment to choose the best of two possible actions, say A1 and A2. The experiment is to be conducted in several stages, and at the end of each stage the experimenter will decide whether or not to continue sampling. If he decides to continue he must choose the number of observations to be taken at the next stage.

Examples are common in technological and agricultural application. For instance, an agricultural research station might want to decide which of two varieties of grain is best suited for growing in a certain region. A natural procedure is to try out the two varieties in a trial plot in the region for a number of seasons and choose the one with the best performance. The station must then decide the size and duration of the trials.

In general, suppose that 6i is the amount gained if action Ai is taken (i= 1,2), and let 0 = - 02* Usually, of course, 0 is not known, but we suppose that it is possible to obtain independent observations, normally distributed about 0 with known variance a2, at a constant cost, c, per observation. In addition, suppose that the prior information about 0 can be represented by a normal prior distribution with mean xo and variance o2/no. A distribution of this form would arise naturally if most of the information about 0 comes from a pilot stage of experimentation, and it is convenient to express the prior in terms of no equivalent observations whatever its source. The observations can be taken in M distinct stages, and the problem is to decide at each stage if it is worthwhile to continue sampling, and, if so, how many observations to take at the next stage.

In applications to agriculture, we might suppose that the gain is proportional to the true yield and that the variance of the observed yield is inversely proportional to the area of land used (Finney, 1957). The total area of land used would then take the place of the number of observations and would not have to be integral. Whatever the application, the restriction of discreteness has been ignored in this paper.

A mathematically equivalent formulation of this problem has been treated by Grundy, Healy and Rees (1956) (from a fiducial viewpoint) when just a single additional stage of observations can be taken. Raiffa and Schlaifer (1961) have considered a different formulation of the same problem. This paper extends the



462 SCOTT - Multi-stage Test for Normal Mean [No. 3,

results of Grundy, Healy and Rees to situations in which two or three extra stages of observations can be taken. If the observations can be taken in small stages, approximations to a full sequential solution can be derived by considering a similar problem for the mean drift of a Wiener process (Chernoff, 1965). Often the stages are necessarily large; in agriculture, for example, each stage would usually take one year to complete. The truncated sequential approach used here should be of use in these situations, as well as when there is a limit on the time available for the experiment.

In the next section we describe the optimum M-stage strategy, and compare the expected gains for M = 1, 2, 3. Details of the derivation are given in Section 3. An attempt is made in Section 4 to help choose the size of a pilot stage, and a few extensions are sketched in the conclusion. The "optimum strategy", of course, means the best under our idealized set of assumptions. In real experimental situations there are many other factors to consider (for instance, the need to replicate over a variety of weather conditions in an agricultural experiment). However the so-called optimum strategy is fairly flexible and it is hoped that the results will give at least a very rough guide to the economic amount of testing in real experiments.

As far as possible, the notation is consistent with that of Grundy, Healy and Rees, and we suppose throughout that the gain (or loss) is measured from the value 02 (that is, the gain is 0 if A1 is chosen and zero otherwise).

2. OUTLINE OF PROCEDURE Suppose that the experiment is to be carried out in M stages, and let xi and

o2/nI denote the mean and variance of the posterior distribution of 0 after the ith stage. Thus, by the usual normal theory application of Bayes's theorem, if n observations have been taken in the first i stages and if x is the mean of all n observations, then x= (nO xo + nx)/(nO + n), ni = nO + n. (The prior distribution is treated as the posterior distribution from zeroth stage.) It is convenient to define the normalized quantities

Xi = 1(n*) x*/l and Ai = ony /c.

Then the procedure specified below leads to the maximum expected gain among all M-stage procedures. It is determined by two sequences of functions K*(@) and N&(.) (i = 1, ..., M) which do not depend on M for i ? M. Values can be found from Table 1.

After the ith stage (i = 0, ..., M- 1), calculate Xi and Ai. If I Xj KM-i(A)), stop sampling and choose A1 if Xi , KM-i(Ai), A2 if Xi <- KM-i(Ai). If I X I < KM-i(Ai)I continue sampling by taking a further ni NM-i(I Xi j, Ai) observations at the next stage. After the final stage calculate the posterior mean XM and choose A1 if XM > 0, A2 if XM< 0. (The expected gain is not affected by the choice if XM = 0.)

More detailed results for i = M- 1 (that is, when only one stage remains) can be found in the paper of Grundy, Healy and Rees. We note that the expected gain is not very sensitive to quite large changes in the value of N*, especially in the initial stages of a multi-stage scheme, so there is room for much flexibility in the design. This means that the rather coarse grid used in Table 1 should be fine enough for most purposes. Log N* is roughly linear in log Ai and Xi, and using this relation leads to more accurate interpolated values if needed.

So far, the number of stages has been assumed fixed in advance. Table 2, which compares the expected gains from the best one-, two- and three-stage schemes, is



1968] SCOTT - Multi-stage Test for Normal Mean 463

given to aid the choice of M when this is not dictated by external considerations. The values are measured from the expected gain when the decision is based on the prior distribution alone (in the terminology of Raiffa and Schlaifer, the table gives the expected value of sample information), and no account has been taken of costs that might be incurred when extra stages are taken.

3. DERIVATION

In principle there is no difficulty in obtaining the procedure that maximizes the unconditional expected gain (that is, the Bayes procedure) for a truncated sequential decision problem; proceed by "backward induction" (see, for example, Raiffa and Schlaifer, 1961).

After all M stages have been completed, 0 has a normal distribution with mean XM. Thus, the terminal decision rule that maximizes the expected gain is the natural one of choosing A1 when the posterior mean, XM, is positive and A2 when XM iS negative.

Now consider the situation immediately before the last stage had been taken. This is essentially the situation considered by Grundy, Healy and Rees, and some of their results are summarized in Theorem 1 below. The distribution of 0 is normal with mean xM-l and variance a2InM-1. Let Gl(n) be the expected gain if n observations are taken at the final stage and the best terminal decision rule is used when the outcome of this final stage is known. Then

Gl(n) = - cn + {1(nna I)T2Ur 2} f0 exp [(- 2U2)-1 {nM_ (6 - XM_1)2 + n(x - 0)2}] dx dO flM-1:XM--1+flX:> 0

= (aInM_) {XM-1 + [q{XM_l(1 + Nj')}]/V(1 + Nj1)

-| XM-l(I -I XM-1(1 + N1')} - (N1!AM)}, (1)

where N1 =/Mnl, X+ = X if X> 0 but X+ = 0 if X< 0, (D is the normal distribution function, and b is the density function. If we set G'(n) equal to zero, we obtain

AM-, q{XM_j(l + N1')} = 2N(1 + N1)l. (2)

Theorem 1. For each AM-, there is a critical value of XM-1, say Kl(AM-) such that: (1) If XMl J>K1, G1(O) > Gl(n) for all n >0. (If the first inequality is strict, then so is the second.) (2) If j XM-l < K, G1(0) < Sup G1(n).

n>O In the latter case, Sup Gl(n) is attained uniquely at n = nM-l N1(| XM- 1, AM-), where

n>O N1 is the unique solution of equation (2) when XM_l = 0, and it is the larger of the two solutions when XM-l74 0.

Suppose now that M- i stages have been completed, so that 0 has a normal distribution with mean XM-i and variance U2!nM-i. Let G*(n) be the expected gain if n observations are taken at the next stage and the best procedure used thereafter, and let N* = n1nM_j. Lemma. {l(nM-1)Io} G%(n) = XM+-i +f*(I XM-i , AM_, N*) - N!AM-, where f* has the following properties:




(1) fi( X, A, N) is a continuous decreasing function of j Xi, (2) fi( X , A, N) is a continuous increasing function of N for n > 0, with

fi(j Xj, A,oo) = b(X)-I XIj(-I XI). Proof. For i = 1, the lemma follows from equation (1). Suppose the lemma is true for i-1. Then

G*(n) = E[Sup G*_1(m)] - cn mzO

= / + E [XM-X+1 + Supf%1{( I XM-X+1 j, AM-,+, M) - M/AM-i+1} - Cn

=(/nM_.) (XM ? [{b( I XM_ i)1(1 + N*-1)}/V(1 + N-1)]

- XM-iI |{- I XM_* I 1(1 + N1)} - (N*/AM-i)

+ {1(1 + N*)}-' E [Sup{fi.1( l XM-+1| AM-*+1,M) - M/AM-+1}])

since XMN_*+l is normally distributed about 1+ N1 XM-i with variance N*. It follows from the properties of fi-, that G*(n) can be put in the form

(1n31,ri/ao) Gj(n) = XMl +fi(I XM-i 1, AM-i, N*) - NiIAM-i, with fA a continuous function of I XM-i I and N*, decreasing as I XM_i I increases and converging to

b(j XMiI-I Xn_ij)4D(-I XkIiI) as N*+oo. The fact thatfi is an increasing function of N* follows directly from the definition

of G*(n). For

fi(I XM-i |, A)m- NM) = (1nM-iIa) {G.i(n) + cn} -XM-

and G*(n) - cn is the expected gain from taking n observations without cost at the next stage and then using the best procedure for the remaining stages.

Theorem 2 follows directly from the lemma. Theorem 2. For each AM-i, there is a critical value of XM-i, say Ki(AM-i) such that: (1) If XM-i >I K*, G*(O) = Sup G*(n).

ntO (2) If | XM-*| < K, Gi(O) < Sup G*(n).

n>O In this latter case, there is a value N* satisfying

0 < N* < AM-i{,(XM-) - I XM-i I -( I XN-* I and depending only on IXM-i I and AM-i, such that Sup G*(n) is attained at n = nM-i N.

n>O Theorem 2 provides the justification for the form of the procedure outlined in

the preceding section. Thus, although the number of observations to be taken at the (i+ 1)st stage depends on five parameters, M, o, xi, ni and c, it can be tabulated in terms of two, namely I XM_* I and Ai. The problem of determining K* and N* remains.



1968] SCOTT - Multi-stage Test for Normal Mean 465

Unfortunately, it does not seem possible to obtain an explicit expression for G (n) with i> 1, but it is not difficult to evaluate numerically. Values of Ki and Ni can then be determined empirically, and Table 1 and Fig. 1 were obtained in this way.

TABLE 1

Sampling constants (a) Values of N1 and K1

Ixl log1o A K1

0.0 0 5 1.0 1.5 2.0 2 5

1.0 0 75 0 57 1.5 1 81 1P56 0 93 2.0 3-74 3 39 2-42 1 30 2-5 7 21 6 65 5 16 3 00 1P66 3 0 13-38 12 48 9.99 6 64 3 00 2.01 3 5 24 37 22 79 18-55 13 94 7 32 2-34 4 0 43 91 41 15 33 78 24 10 14 60 6-55 2 65

(b) Values of N2 and K2

I xi log1o A K2

0.0 0 5 1.0 1.5 2-0 2 5

1.0 0.59 0 40 0.72 1.5 1 18 0-93 0 38 1 18 2.0 2 27 1P89 1.12 0-52 1 64 2 5 4 00 3.53 2.30 1 29 0 46 2 08 3*0 6-79 5 94 4.19 2.52 1.31 2.47 3.5 11419 9.82 7 05 4 34 2 45 1.19 2 85 4*0 17 91 16 14 11 61 7416 4 10 2-18 3 20

(c) Values of N3 and K3

xi

log1o A K3 0.0 0 5 1.0 1.5 2.0 2 5

1P0 0-33 0.26 0 78 1P5 0 70 0.56 0 27 1 30 2.0 1-56 1P26 0-76 0 41 1'78 2 5 2 68 2 24 1 49 0 89 0 35 2 23 3 0 4'36 3-72 2*52 1.59 0 88 0 29 2 66 3.5 6 56 5 52 3.65 2.46 1.56 0-74 3 05 4 0 9 20 7 94 5.25 3 46 2 40 1P30 3 40




4. THE PRIOR DISTRIBUTION The results given so far depend on the assumption of a normal prior distribution.

If no pilot stage has been taken, and there is reluctance to translate a value feeling about the likely magnitude of 0 into a formal prior distribution, a natural procedure

08,

07-

0-6-

z 0-5-

0-4

0

04

U0-31-\

z

0-2

0 1

0 11

FIG. 1. I: nO= 10-1,g-2/3 c-2/3. II: nO= 10-2/3,g2/3 c-213. III: no 10-4/3 0r2/3 c-2/3. In units of C113 Uy2/3.

is to take an initial stage of no observations. Then, if the mean of these observations is xo, and the amount of prior knowledge is small compared to the information contained in a single observation, the posterior distribution of 0 will be approximately normal with mean xo, variance a2/no, and the procedure given earlier can be used for the remaining stages. How large should this initial stage be ? Unfortunately the use of a device such as a uniform prior distribution for 0 does not give reasonable answers in this type of design problem.

Fig. 1 is given to guide the choice of no. For a few values of no, and with suitable normalization, it shows the expected difference, as a function of 0, between the gain from an immediate correct decision and the gain if the scheme given here is followed



1968] Scorr - Multi-stage Test for Normal Mean 467

for two stages after the initial stage. A value of no can be chosen so that this difference is small in a specified region. This approach was suggested by remarks of Miss R. J. Maurice in the discussion of the paper of Grundy, Healy and Rees (1956).

5. EXTENSIONS

It would not be difficult to extend Table 1 and Figure 1 to deal with more than three stages, though it is clear from Table 2 that a substantial relative increase in the

TABLE 2

Expected gain in units of nO-

loA No. of xOi logA stages 0.0 0.5 1.0 l5 2-0 2-5

1 1 0-1862 0-0118 2 0-2023 0.0278 3 0-2111 0-0356

2 1 0-3170 0*1240 0-0241 2 0-3297 041367 0-0360 0-0032 3 0-3342 041412 0-0401 0-0056

3 1 0.3714 041721 0-0623 0.0140 0*0001 2 0.3775 041783 0-0684 0-0195 0-0030 3 0.3799 0-1806 0-0705 0.0209 0-0041 0-0002

4 1 0-3901 0.1901 0*0765 0-0241 0.0054 0-0004 2 0-3929 0 1929 0-0792 0.0265 0-0070 0-0013 3 0-3939 0-1938 0-0799 0-0270 0-0074 0-0015

expected gain would only result when 1(no) xo/ I a j is large. (It should be emphasized that relative increases will depend very much on where the gain is measured from. The values in Table 2 are measured from the expected gain when a decision is based on the prior distribution alone.) Some caution should be exercised for large values of I Xo , however, since a brief study of the effects of non-normality suggests that, while the results are not very sensitive to the shape of the prior distribution when Xo I is small, they become increasingly sensitive as I Xo increases.

The variance has been assumed known throughout. Schleifer (1961) has extended the results of Grundy, Healy and Rees to the case in which a2 is not known, but has a prior distribution of convenient form (inverted gamma). The results are similar to those for known a2, with the Student t-distribution taking over the role played by the normal. Extensive tables of the values needed to carry out the procedure are given in Bracken and Schleifer (1964). The results could be extended in the same way to more than one stage, but we have not done so here.

Finally, the results of this paper can be applied directly to a simple situation involving more than two processes. Suppose that we must decide whether to accept or reject each of K processes iT1, ..., 17k* If aiT is accepted, an amount 6i (possibly negative) is gained, and we can take observations, each costing an amount ci, that are normally distributed about Oi. Assume that 01, Sk have independent normal

17



468 Scorr - Multi-stage Test for Normal Mean [No. 3,

prior distributions (arising, perhaps, from a pilot study). We want to decide how many observations to take from each aiq in order to maximize the expected gain. Now the independence of the prior distributions and the additivity of the gains and costs mean that each aiq can be treated separately. The largest expected gain will be obtained by using the procedure given here for each of the K processes.

ACKNOWLEDGEMENTS This paper is based on part of a Ph.D. thesis at the University of Chicago (1965),

and the research was sponsored by the Army Research Office, Office of Naval Research, and Air Force Office of Scientific Research by Contract No. Nonr-2121 (23), NR 342-043. I would like to thank Professor D. L. Wallace for his help during the course of the work.

REFERENCES

BRACKEN, J. and SCHLEIFER, A. (1964). Tables for Normal Sampling with Unknown Variance. Boston: Harvard University.

CHERNOFF, H. (1965). Sequential tests for the mean of a normal distribution. IV (Discrete case), Ann. Math. Statist., 36, 57-68.

FINNEY, D. J. (1957). Statistical problems of plant selection. Bull. Inst. Internat. Statist., 36, 242-268.

GRUNDY, P. M., HEALY, M. J. R. and REES, D. (1956). Economic choice of the amount of experimentation. J. R. Statist. Soc. B, 18, 32-55.

RAIFFA, H. and SCHLAIFER, R. (1961). Applied Statistical Decision Theory. Boston: Harvard University.

SCHLEIFER, A., Jr. (1961). Studies in the Economics of Sample Size. D.B.A. thesis, Harvard University.



Date post:	27-Jan-2017
Category:	Documents
Upload:	a-scott
View:	213 times
Download:	1 times

A Multi-Stage Test for a Normal Mean

Documents