Sequential Analysis Explained

7/30/2019 Sequential Analysis Explained

1/33

Sequential Analysis Explained

Site links :HomeAboutIndexSearchPlotsFAQContact

Related link :

Sequential Probability Ratio Test explainedSequential paired comparison explainedSequential unpaired comparison explained

Table of Contents :IntroductionBrief historyMethodologyReferences

Introduction

Statistics in the late 19th and early 20th century was involved with agriculture andindustry, and analysis was usually carried out on batches of data already collected. Theearly methods were therefore focused around a single examination of the data,probability estimates were based on a single evaluation of the data.

A disciplined protocol was therefore developed, where a fixed sample size necessaryfor making statistical decisions was estimated at the planning stage, and data are notexamined until data collection is completed. In fact, when the placebo effect wasidentified, and that bias from researchers and subjects need to be controlled, it becameimportant that both blinding of all concerned to the treatment and enforced ignorance of

the results were observed until data collection is completed.

The fixed sample size approach however incurs considerable economic and ethicaldisadvantages. On many occasions, the effects of treatments are much more obviousthan envisaged during planning, and to implement the full sample size incursunnecessary costs and risks to research subjects.

In clinical research, subjects are recruited and data collected in a sequential manner,and a resaerch project may extend over a long period, and the nature of the data ismore suitable for sequential examination than a single analysis at the end of the longperiod.

Back to Top

Brief history

Sequential analysis began as mathematical games and concepts, of analysing how aset of complex probabilities behaved when it is applid repeatedly over time. It remainsan esoteric field of mathematics until the second world war.
http://www.stattools.net/index.phphttp://www.stattools.net/index.phphttp://www.stattools.net/StatToolsAbout.phphttp://www.stattools.net/StatToolsAbout.phphttp://www.stattools.net/StatToolsIndex.phphttp://www.stattools.net/StatToolsIndex.phphttp://www.stattools.net/StatToolsKeySearch.phphttp://www.stattools.net/StatToolsKeySearch.phphttp://www.stattools.net/StatToolsPlot.phphttp://www.stattools.net/StatToolsPlot.phphttp://www.stattools.net/StatToolsFAQ.phphttp://www.stattools.net/StatToolsFAQ.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/SeqSPRT_Exp.phphttp://www.stattools.net/SeqSPRT_Exp.phphttp://www.stattools.net/SeqPaired_Exp.phphttp://www.stattools.net/SeqPaired_Exp.phphttp://www.stattools.net/SeqUnpaired_Exp.phphttp://www.stattools.net/SeqUnpaired_Exp.phphttp://www.stattools.net/Seq_Exp.php#Introductionhttp://www.stattools.net/Seq_Exp.php#Introductionhttp://www.stattools.net/Seq_Exp.php#Brief%20historyhttp://www.stattools.net/Seq_Exp.php#Brief%20historyhttp://www.stattools.net/Seq_Exp.php#Methodologyhttp://www.stattools.net/Seq_Exp.php#Methodologyhttp://www.stattools.net/Seq_Exp.php#Referenceshttp://www.stattools.net/Seq_Exp.php#Referenceshttp://www.stattools.net/Seq_Exp.php#tophttp://www.stattools.net/Seq_Exp.php#tophttp://www.stattools.net/Seq_Exp.php#tophttp://www.stattools.net/Seq_Exp.php#Referenceshttp://www.stattools.net/Seq_Exp.php#Methodologyhttp://www.stattools.net/Seq_Exp.php#Brief%20historyhttp://www.stattools.net/Seq_Exp.php#Introductionhttp://www.stattools.net/SeqUnpaired_Exp.phphttp://www.stattools.net/SeqPaired_Exp.phphttp://www.stattools.net/SeqSPRT_Exp.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/StatToolsFAQ.phphttp://www.stattools.net/StatToolsPlot.phphttp://www.stattools.net/StatToolsKeySearch.phphttp://www.stattools.net/StatToolsIndex.phphttp://www.stattools.net/StatToolsAbout.phphttp://www.stattools.net/index.php


2/33

In the late 1930s and early 1940s there were massive increase in industrial productionof war materials, and there was a need to ensure that products, particularly munitions,are reliable to the users. The testing of these products are not only expensive, but attime destructive (as bullets can only be tested by firing, thus destroying the sample).There was therefore a need to develop a method of minimal testing, yet produce reliable

results.

Wald, and his group, in the Ministry of Supplies in USA developed the theoretical basisand methodologies of the Sequential Probability Ratio Test. Instead of defining asample size, a pair of statistical borders are drawn, one to decide the rejection of thenull hypothesis and the other to accept. Data are then obtained sequentially, and plottedagainst these borders. After each measurement, one of 3 decisions can be made.These are to stop further testing and reject the null hypothesis, to stop further testingand accept the null hypothesis, or to continue testing. The successes of thismethodology greatly reduced the costs and increase the reliability of supplies, and themethod was classified until after the war.

In the meantime, another group, under Barnard and Armitage, also developed theconcept of sequential analysis. The focus was not only on industrial production, but alsotesting the efficacy of new products. This resulted in the much quoted book by Armitageon sequential medical trials, focusing on how the methods can be used in drug trials.The focus here is onpaired comparisons, where a paired observation is made, and thedifference between the pair tested sequentially against the null hypothesis.

Sequential analysis has developed extensively since the mid sixties, and the medicaland statistical literature contains many models, some specific to particular researchmodels, and some even developed just to service a particular research project.

In the 1990s Whitehead published the theoretical backgrounds and methodologies ofunpaired group sequential analysis. This uses the two group controlled trial model, butthe statistics was adapted to allow periodic review of the results.

Back to Top

Methodology

Instead of prescribing a fixed sample size, sequential analysis calculates borders whichconstrain statistical decisions. Although the shape and size of these borders varies in
http://www.stattools.net/SeqSPRT_Exp.phphttp://www.stattools.net/SeqSPRT_Exp.phphttp://www.stattools.net/SeqPaired_Exp.phphttp://www.stattools.net/SeqPaired_Exp.phphttp://www.stattools.net/SeqPaired_Exp.phphttp://www.stattools.net/SeqUnpaired_Exp.phphttp://www.stattools.net/SeqUnpaired_Exp.phphttp://www.stattools.net/Seq_Exp.php#tophttp://www.stattools.net/Seq_Exp.php#tophttp://www.stattools.net/Seq_Exp.php#tophttp://www.stattools.net/SeqUnpaired_Exp.phphttp://www.stattools.net/SeqPaired_Exp.phphttp://www.stattools.net/SeqSPRT_Exp.php


3/33

different models, they conform to a basic pattern as shown below and to the left.

There are usually 2 borders, one to reject the null hypothesis (the result is not null), andone to accept the null hypothesis (the result is null). The relationship between thenumber of samples taken (the x axis) and some measure of effect size (the y axis) arethen plotted on the graph as the information is sampled sequentially. When the plotcrosses the rejection border, sampling can stop and a decision to reject the null

hypothesis (a difference exists) can be made. If the plot crosses the acceptance border,sampling can stop and a decision to accept the null hypothesis (no significant differenceexists) can be made. While the plot remains between the borders, no statistical decisionis made other than to continue sampling.

In most models, the rejection and acceptance borders converge, or that a third, stoppingborder is introduced. In these cases the maximum number of samples required isknown. In some, particularly the earlier models, the borders are parallel, and in theorysampling can go on forever until one of the borders is crossed.

In most cases, sequential analysis is applied to a one sided (one tail) test, whethergroup 1 is bigger than group 2. When necessary, the sequential analysis can alsoproduce borders for the two tail test (is there a difference, either way), and in thesecases the basic pattern is two overlapping triangles as shown to the right.

When the plot crosses the upper rejection border, a significant positive difference (grp 1- grp 2 significantly > null) can be concluded. When the plot crosses the lower rejectionborder, a significant negative difference (grp 1 - grp 2 significantly < null) can beconcluded. When the plot crosses both acceptance borders, the null hypothesis can beaccepted (the two groups are not significantly different). Where the plot has not as yetcrossed any borders, no statistical decision can be made, and further sampling isrequired.


4/33

Where the plot crosses the acceptance border of one triangle but remains within theborders of the other triangle, a partial decision can be made.

If the plot crosses the acceptance border of the lower triangle but remains withinthe borders of the upper traingle, then the conclusion that grp 2 is not greater

than grp 1 can be made, but whether grp 1 is greater than grp 2 remainsundecided and requires further sampling. If the plot crosses the acceptance border of the upper triangle but remains within

the borders of the lower traingle, then the conclusion that grp 1 is not greaterthan grp 2 can be made, but whether grp 2 is greater than grp 1 remainsundecided and requires further sampling.

Please note : that repeated sequential sampling has a statistical cost, in terms ofreduced statistical power. In the planning stage, for the same effect size, the maximumsample size calculated for sequential analysis is larger than that for the fixed samplesize model. However, the effect size in the data collected is often much larger or much

smaller than that envisaged during planning. When this happens, sequential analysisallows an earlier conclusion to reject or accept the null hypothesis and terminate thestudy, so the sample size actually used is often smaller than that planned, and smallerthan that required in the fixed sample size model.

Back to Top

References

Wald A (1947) Sequential Analysis. John Wiley and Son, Inc, New York.

Armitage P. Sequential Medial Trials (1975) Blackwell Scientific Publications. ISBN 0-632-08790-0

Whitehead John (1992). The Design and Analysis of Sequential Clinical Trials (Revised2nd. Edition) . John Wiley & Sons Ltd., Chichester, ISBN 0 47197550 8. p. 48-50

Please note that the books by Wald and Armitage are now out of print, although theycan be found in most university libraries, or obtained (as I did) from interlibrary loans.

Sequential Probability Ratio Test

by WaldSite links :HomeAboutIndexSearchPlotsFAQContact

Related link :Sequential analysis explainedProgram for SPRT for proportionsProgram for SPRT for means
http://www.stattools.net/Seq_Exp.php#tophttp://www.stattools.net/Seq_Exp.php#tophttp://www.stattools.net/index.phphttp://www.stattools.net/index.phphttp://www.stattools.net/StatToolsAbout.phphttp://www.stattools.net/StatToolsAbout.phphttp://www.stattools.net/StatToolsIndex.phphttp://www.stattools.net/StatToolsIndex.phphttp://www.stattools.net/StatToolsKeySearch.phphttp://www.stattools.net/StatToolsKeySearch.phphttp://www.stattools.net/StatToolsPlot.phphttp://www.stattools.net/StatToolsPlot.phphttp://www.stattools.net/StatToolsFAQ.phphttp://www.stattools.net/StatToolsFAQ.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/SeqProp_Pgm.phphttp://www.stattools.net/SeqProp_Pgm.phphttp://www.stattools.net/SeqProp_Pgm.phphttp://www.stattools.net/SeqMean_Pgm.phphttp://www.stattools.net/SeqMean_Pgm.phphttp://www.stattools.net/SeqMean_Pgm.phphttp://www.stattools.net/SeqProp_Pgm.phphttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/StatToolsFAQ.phphttp://www.stattools.net/StatToolsPlot.phphttp://www.stattools.net/StatToolsKeySearch.phphttp://www.stattools.net/StatToolsIndex.phphttp://www.stattools.net/StatToolsAbout.phphttp://www.stattools.net/index.phphttp://www.stattools.net/Seq_Exp.php#top


5/33

Table of Contents :IntroductionSPRT for proportionsSPRT for meansReferences

Introduction

General discussions onsequential analysisandquality controlare presented in aseparate page and will not be repeated here.

This page discusses the earliest sequential methods as developed by Wald and hisgroup in the early 1940. These methods were initially developed as a method ofquality control, and they form the basis of many subsequent and more sophisticateddevelopments in sequential and quality control methodologies.

The model aims to determine the quality of a batch of products by minimal sampling.The idea is to sample the batch sequentially until a decision can be made whetherthe batch conforms to specification and can be accepted or that it should berejected.

These pages offer 2 models, for proportion and mean.

Back to Top

SPRT for proportions

InSPRT for proportions, the aim is to test a group of subjects or a batch of products,and to determine whether the proportion of those with the indexed attribute (positivecases, defective items) exceeds 0 (null hypothesis). The proportion of zero isdefined by an upper bench mark (pn), above which the null hypothesis (it is zero) isrejected, and a lower bench mark (p0) below which the null hypothesis is accepted(it is zero). Members are sequentially sampled and examined whether it has theindexed attribute (positive cases). A decision is made after each evaluation whetherto reject the null hypothesis (bench mark exceeded), to accept the null hypothesis(bench mark not exceeded), or to continue sampling.

The parameters and data used for SPRT for proportions are as follows

Probability of type 1 error alpha (), usually 0.05. Power (1 - ). For most clinical studies this is set to 0.8. The proportion pn which defines the rejection of the null hypothesis, and the

lower proportion (p0) which defines the acceptance of the null hypothesis.

The data to be entered is a single column array, where the value 1 represent
http://www.stattools.net/SeqSPRT_Exp.php#Introductionhttp://www.stattools.net/SeqSPRT_Exp.php#Introductionhttp://www.stattools.net/SeqSPRT_Exp.php#SPRT%20for%20proportionshttp://www.stattools.net/SeqSPRT_Exp.php#SPRT%20for%20proportionshttp://www.stattools.net/SeqSPRT_Exp.php#SPRT%20for%20meanshttp://www.stattools.net/SeqSPRT_Exp.php#SPRT%20for%20meanshttp://www.stattools.net/SeqSPRT_Exp.php#Referenceshttp://www.stattools.net/SeqSPRT_Exp.php#Referenceshttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/quality_Exp.phphttp://www.stattools.net/quality_Exp.phphttp://www.stattools.net/quality_Exp.phphttp://www.stattools.net/SeqSPRT_Exp.php#tophttp://www.stattools.net/SeqSPRT_Exp.php#tophttp://www.stattools.net/SeqProp.phphttp://www.stattools.net/SeqProp.phphttp://www.stattools.net/SeqProp.phphttp://www.stattools.net/SeqProp.phphttp://www.stattools.net/SeqSPRT_Exp.php#tophttp://www.stattools.net/quality_Exp.phphttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/SeqSPRT_Exp.php#Referenceshttp://www.stattools.net/SeqSPRT_Exp.php#SPRT%20for%20meanshttp://www.stattools.net/SeqSPRT_Exp.php#SPRT%20for%20proportionshttp://www.stattools.net/SeqSPRT_Exp.php#Introduction


6/33

positive cases, and 0 negative cases.

The interpretations of the outcome are best demonstrated by the following example.

Example of SPRT for proportions

The transport department noticed that less than half of teenagers pass their drivingtest in their first attempt, a distressing and wasteful situation. A suggestion wasmade that a training course be offered before the test, but this requires organizationand resources, so there is a need to pre-test the course.

0

1

1

1

1

0

1

1

0

1

1

1

1

It was agreed that the course should result in 80% (0.8) of the candidate to pass thedriving test at the first attempt, and that it should be considered useless if the

passing rate remained the current 40% (0.4). We also set our statistical parametersso that alpha = 0.05, power = 0.9.

Effect size = 1.79Rejection border y = 1.5474 + 0.6131 nAcceptance border y = -0.8696 + 0.6131 nMax n = 18


7/33

The borders are shown to the right. The border to decide rejection of the nullhypothesis (that the pass rate is 80% or better) is y = 1.5474 + 0.6131 n, and toaccept the null hypothesis (that the pass rate is not better than 80%) y = -0.8696 +0.6131 n, n being the number of candidates reviewed, and y is the cumulative

number of passes.

The data is shown above and to the left, the value 1 for those who passed the test,and 0 those failed (the first, sixth, and nineth candidate).

The plot is shown to the right. By the 12th candidate, the border for rejecting the nullhypothesis has been reached. The study can at this stage be terminated and theconclusion drawn that the pass rate is 80% or better.

Back to Top

SPRT for means

InSPRT for means, the aim is to test a group of subjects or a batch of products, todetermine whether the mean measurements in particular parameter exceeds thezero value (null hypothesis). The null value is defined by an effect size (mean / sd)bench mark, above which the null hypothesis is rejected, and another bench markbelow which the null hypothesis is accepted. Members are sequentially sampled andmeasured, and a decision is made after each measurements whether to reject thenull hypothesis (bench mark exceeded), to accept the null hypothesis (bench marknot exceeded), or to continue sampling.

The parameters that needs to be set are as follows.

Probability of type 1 error alpha (), usually 0.05. Power (1 - ). For most clinical studies this is set to 0.8. The expected standard deviation of the measurement. The mean value above which the null hypothesis is to be rejected The mean value below which the null hypothesis is to be rejected.

The data is presented as a single column array of measurements.

Borders for rejection and acceptance of the null hypothesis are drawn from the
http://www.stattools.net/SeqSPRT_Exp.php#tophttp://www.stattools.net/SeqSPRT_Exp.php#tophttp://www.stattools.net/SeqMean.phphttp://www.stattools.net/SeqMean.phphttp://www.stattools.net/SeqMean.phphttp://www.stattools.net/SeqMean.phphttp://www.stattools.net/SeqSPRT_Exp.php#top


8/33

parameters, and the sum of the values are plotted against the number ofmeasurements sequentially.

The results are best followd by the following examples.

Example of SPRT for means

(5-wt)

1

-1

1

0

1

0

1

2

0

-1

1

1

-1

1

0

We are the buying department of a grocery chain, and purchases eggs from thefarmers. We expect the standard deviation of the weight of eggs to be 1g, and westipulated that the mean weight of the eggs we purchased must not be more than 1SD (1g) less than the average of 5gs. (x = 5 - wt of egg). For every batch of eggdelivered we wish to measure the eggs until we are satisfied that this bench mark ismet.


9/33

Rejection Line is y = 2.7726 + 0.5 nAcceptance Line is y = -1.5581 + 0.5 nExpected sample size n = 5Truncation sample size = 15

We set the parameters as alpha () = 0.05, power=0.8, expected standard deviation= 1g, bench mark for rejection of null hypothesis (eggs too small) 1 (bench mark =1g less than mean of 5), and the bench mark for accepting the null hypothesis (eggsnot too small) 0 (not less than 5 g)

The borders are as shown above and to the right. The border for rejection of nullhypothesis is y = 2.7726 + 0.5 n, and for acceptance of null hypothesis -1.5581 + 0.5n, where n = number sampled, and y = sum of x (x = 5-wt of egg). if the borders arenot crossed after 15 measurements, the null hypothesis is accepted (eggs not too

small)

The plot remained between the rejection and acceptance line, until the maximumnumber of samples was reached after 15 measurements, and at that stage thedecisions to stop further measurements and accept the null hypothesis (eggs not too

small) were made.

Back to Top

References

Wald A (1947) Sequential Analysis. John Wiley and Son, Inc, New York.

Proportions : p. 95-105 means : p. 121 - 124

Sequential paired comparisonsby Armitage


Related link :
http://www.stattools.net/SeqSPRT_Exp.php#tophttp://www.stattools.net/SeqSPRT_Exp.php#tophttp://www.stattools.net/index.phphttp://www.stattools.net/index.phphttp://www.stattools.net/StatToolsAbout.phphttp://www.stattools.net/StatToolsAbout.phphttp://www.stattools.net/StatToolsIndex.phphttp://www.stattools.net/StatToolsIndex.phphttp://www.stattools.net/StatToolsKeySearch.phphttp://www.stattools.net/StatToolsKeySearch.phphttp://www.stattools.net/StatToolsPlot.phphttp://www.stattools.net/StatToolsPlot.phphttp://www.stattools.net/StatToolsFAQ.phphttp://www.stattools.net/StatToolsFAQ.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/StatToolsFAQ.phphttp://www.stattools.net/StatToolsPlot.phphttp://www.stattools.net/StatToolsKeySearch.phphttp://www.stattools.net/StatToolsIndex.phphttp://www.stattools.net/StatToolsAbout.phphttp://www.stattools.net/index.phphttp://www.stattools.net/SeqSPRT_Exp.php#top


10/33

Sequential analysis explainedProgram for sequential comparison of paired preferencesProgram for sequential comparison of paired differences

Table of Contents :

IntroductionPaired preferencesPaired differencesReferences

Introduction

General discussions on sequential analysis are presented ina separate pageandwill not be repeated here.

This page discusses the development of paired sequential comparisons that were

developed in the late 1950s and 1960s by Armitage. The methods were particularlysuitable to support medical research comparing efficacies of different treatment ormedications.

The model used is that of paired comparisons, where two treatments areadministered to either the same individual or a paired of matched subjects, and thedifferences between the pair is then used for analysis.

The data is analysed after the results from each pair is available, and one of 3decisions are made. These are to conclude the study and reject the null hypothesis(significant difference exists), to accept the null hypothesis (significant difference

does not exist), or to defer any decision and collect more data.

In his book, Armitage presented 3 models, the paired preference, the paireddifference, and the paired follow up (survival). StatTools presents two of thesemodels, that of preference and paired differences.

Back to Top

Paired preferences

Insequential paired preference comparison, two treatments are given to the

subjects, and the responses required are treatment 1 is better, treatment 2 isbetter, or no difference. These are entered into the data box, where a positive value(1) represents a subject prefering treatment 1, a negative value (-1) represents asubject prefering treatment 2, and zero (0) representting no preference.

The parameters are then set, and these are as follows.

Probability of type 1 error alpha (), usually 0.05.
http://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/SeqPairedPref_Pgm.phphttp://www.stattools.net/SeqPairedPref_Pgm.phphttp://www.stattools.net/SeqPairedDiff_Pgm.phphttp://www.stattools.net/SeqPairedDiff_Pgm.phphttp://www.stattools.net/SeqPaired_Exp.php#Introductionhttp://www.stattools.net/SeqPaired_Exp.php#Introductionhttp://www.stattools.net/SeqPaired_Exp.php#Paired%20preferenceshttp://www.stattools.net/SeqPaired_Exp.php#Paired%20preferenceshttp://www.stattools.net/SeqPaired_Exp.php#Paired%20differenceshttp://www.stattools.net/SeqPaired_Exp.php#Paired%20differenceshttp://www.stattools.net/SeqPaired_Exp.php#Referenceshttp://www.stattools.net/SeqPaired_Exp.php#Referenceshttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/SeqPaired_Exp.php#tophttp://www.stattools.net/SeqPaired_Exp.php#tophttp://www.stattools.net/SeqPairedPref.phphttp://www.stattools.net/SeqPairedPref.phphttp://www.stattools.net/SeqPairedPref.phphttp://www.stattools.net/SeqPairedPref.phphttp://www.stattools.net/SeqPaired_Exp.php#tophttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/SeqPaired_Exp.php#Referenceshttp://www.stattools.net/SeqPaired_Exp.php#Paired%20differenceshttp://www.stattools.net/SeqPaired_Exp.php#Paired%20preferenceshttp://www.stattools.net/SeqPaired_Exp.php#Introductionhttp://www.stattools.net/SeqPairedDiff_Pgm.phphttp://www.stattools.net/SeqPairedPref_Pgm.phphttp://www.stattools.net/Seq_Exp.php


11/33

Power (1 - ). For most clinical studies this is set to 0.8, but most of theexamples set in Armitage's book set this at 0.9 or 0.95.

The expected preferences for the two treatments. This defines the effectsize of the model. The smaller the effect size, the larger will the sample sizebe before a decision is likely. The proportions are expressed as a number

between 0 and 1, so that 25% is represented by 0.25.

The interpretations of the outcome are best demonstrated by the followingexample.

Example of paired preference

1

1

1

-1

1

1

1

1

-1

1

1

1

1

We wish to compare two headache pills (A and B). We set our parameters so thatalpha = 0.05, power = 0.9, The model should be able to detect a difference wherepreference for pill A is 70% (0.7) and preference for pill B is 30% (0.3).

Effect size = 0.8448y = 3.4113 + 0.3809 xMax x (N) = 21


12/33

The border to decide rejection of the null hypothesis is shown to the right, being y =3.4113 + 0.3809 x, and if this border is not crossed after 21 pairs, then the nullhypothesis will be accepted (no significant difference).

As each subject report his preference, the no preference response is ignored(either not included into the data array or, if included will be ignored by theprogram). Those preferring pill A will be scored 1 and those preferring pill B willscore -1, and the scores are summed and plotted on the graph.

Although a decision is made after each subject, the data presented above and tothe left represents that obtained after 13 subjects, and the plot at that time is asshown to the right.

It can be seen that, the border is crossed after 12 subjects have been provided

their preference, and the null hypothesis can at that point be rejected (significantdifference shown). Also that, being a restricted model, the study is truncated at 21pairs, and the null hypothesis is accepted (no significant difference) if the border isnot crossed after 21 subjects.

Back to Top

Paired differences

Insequential paired difference, the difference between paired measurements areevaluated sequentially, and a decision is made after each pair of measurementswhether to reject the null hypothesis (significant difference exists), to accept thenull hypothesis (no significant difference), or to continue to collect data.

Unlike paired preferences however, the paired differences are evaluated on theassumption that the differences are normally distributed, so the borders are drawndifferently.

There are two models that can be used. The first assumes that the variance
http://www.stattools.net/SeqPaired_Exp.php#tophttp://www.stattools.net/SeqPaired_Exp.php#tophttp://www.stattools.net/SeqPairedDiff.phphttp://www.stattools.net/SeqPairedDiff.phphttp://www.stattools.net/SeqPairedDiff.phphttp://www.stattools.net/SeqPairedDiff.phphttp://www.stattools.net/SeqPaired_Exp.php#top


13/33

(standard deviation) is known, and this is incorporated into the parameters. Thesecond accepts that the variance is not known, and the mean and standarddeviations assigned are merely used to define the effect size (difference / standarddeviation) that the model can work with.

The parameters that needs to be set are as follows.

Probability of type 1 error alpha (), usually 0.05.

Power (1 - ). For most clinical studies this is set to 0.8, but most of theexamples set in Armitage's book set this at 0.9 or 0.95.

The difference and standard deviation of that difference. This defines theeffect size of the model. The smaller the effect size, the larger will thesample size be before a decision is likely.

The data is presented as a two column matrix, the two columns are pairedmeasurements, and data from each pair in a row.

The results are best followd by the following two examples, one assuming a knownvariance, the other does not.

Example 1. Paired differences assuming known variance

Border y = a + bxb = -4.0a (reject null hypothesis) a = -22.18a (accept null hypothesis) a = 12.47Maximum n for restricted model = 8

We wish to evaluate two methods of measuring blood pressure, the mercurymanometer(A) and the electronic (B), and we suspect that the method A measureslower than method B.

We set the parameters as alpha () = 0.05, power=0.8, expected mean paireddifference = 8mms (mean(A-B) = -8), and standard deviation of mean difference =8mms. As we are only interested in whether A measures less than B, we used theone tail test.

80 95

120 124

75 72

75 92


14/33

84 94

100 106

88 96

92 98

The borders are as shown above and to the right. The border for rejection of nullhypothesis is y = -22.18 - 4x, and for acceptance of null hypothesis y = 12.47 - 4x,where x = sequential pairs, and y = sum of differences. if the borders are notcrossed after 8 subjects, the null hypothesis is accepted (no significant difference)

The data are shown above and to the left. Column 1 the BP as measured using themercury manometer(A) and column 2 electronically (B).

Although the data is evaluated after each subject, the graph to the right shows allthe results after 8 subjects are evaluated.

The borders for rejection and acceptance of the null hypothesis, and the truncationafter 9 pairs, can be seen. The slopes are downwards because measurement incolumn 1 is expected to be less than that of column 2.

It can be seen that the study could and should have been stopped after 5 subjects,when the border for rejection of null hypothesis has been crossed. A conclusionthat BP as measured by the mercury manometer is significantly lower than thatmeasured electronically could have been made at that time.

Example 2. Paired differences assuming unknown variance

We will use the same data and setting as that of the previous example, with thefollowing modifications.


15/33

The model will be a two tail model regardless of the setting.

The difference and standard deviations do not necessarily reflect that from thedata, as the variance is assumed to be unknown. Rather the ratio (difference / SD)is used to define the effect size of the model, and it is just as easy to assign 1 for

standard deviation, and the difference reflecting the scale of the difference todetect. For example, an effect size (difference/SD) of 0.3 or less is considered asmall effect, rarely used in sequential studies. An effect size of 0.7 or more isconsidered large (an obvious difference) and this is most commonly used insequential studies and in drug trials. In between 0.3 and 0.7 or moderate size,common in clinical situations, and can be (but not that commonly) used insequential analysis.

n v1 v2 diff(d) Sum(d) Sum(d2) z

1 80 95 -15 -15 225 1

2 120 124 -4 -19 241 1.5

3 75 72 3 -16 250 1.0

4 75 92 -17 -33 539 2.

5 84 94 -10 -43 639 2.9

6 100 106 -6 -49 675 3.6

7 88 96 -8 -57 739 4.4

8 92 98 -6 -63 775 5.1

We have left the difference and standard deviation both at 8, therefore assigning alarge effect size of 1. The trucation point is calculated to be after 7.9 (rounded to 8)pairs of measurements. The border for rejecting null hypothesis is not a straight lineand will be calculated on the run. The results are shown to the left, and the graphso produced shown to the right.

The table above and to the left shows how the z value is calculated. With each pairof measurements, the difference between them is d. The sum of d and d 2 of allcases are calculated, and z = sum(d)

2/ sum(d

2)


16/33

It can be seen from the graph the border used to reject null hypothesis, and thetruncation line after 8 pairs, where null hypothesis will be accepted if the border hasnot been crossed by then.

The graph also shows that the rejection border has been crossed by the seventhpair, and at that time the conclusion that a significant paired difference exists couldbe drawn. From the table, the sum difference is a negative value, indicating thatmeasurements from the first column (A) are on the whole smaller than that fromclolumn 2 (B).

Back to Top

References

Armitage P. Sequential Medial Trials (1975) Blackwell Scientific Publications. ISBN

0-632-08790-0

Armitage P. Restricted sequential procedures(1957) Biometrika 44:p 9-26.

Sneiderman MA, Armitage P. Closed sequential t test(1952) Biometrika 49:p 359-366.

Back to Top

Sequential unpaired comparisonsby Whitehead


Related link :Sequential analysis explainedProgram for sequential comparison of 2 unpairedcounts
http://www.stattools.net/SeqPaired_Exp.php#tophttp://www.stattools.net/SeqPaired_Exp.php#tophttp://www.stattools.net/SeqPaired_Exp.php#tophttp://www.stattools.net/SeqPaired_Exp.php#tophttp://www.stattools.net/index.phphttp://www.stattools.net/index.phphttp://www.stattools.net/StatToolsAbout.phphttp://www.stattools.net/StatToolsAbout.phphttp://www.stattools.net/StatToolsIndex.phphttp://www.stattools.net/StatToolsIndex.phphttp://www.stattools.net/StatToolsKeySearch.phphttp://www.stattools.net/StatToolsKeySearch.phphttp://www.stattools.net/StatToolsPlot.phphttp://www.stattools.net/StatToolsPlot.phphttp://www.stattools.net/StatToolsFAQ.phphttp://www.stattools.net/StatToolsFAQ.phphttp://www.stattools.net/StatToolsFAQ.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/SeqUnpairedCount_Pgm.phphttp://www.stattools.net/SeqUnpairedCount_Pgm.phphttp://www.stattools.net/SeqUnpairedCount_Pgm.phphttp://www.stattools.net/SeqUnpairedCount_Pgm.phphttp://www.stattools.net/SeqUnpairedCount_Pgm.phphttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/Send_an_Email.phphttp://www.stattools.net/StatToolsFAQ.phphttp://www.stattools.net/StatToolsPlot.phphttp://www.stattools.net/StatToolsKeySearch.phphttp://www.stattools.net/StatToolsIndex.phphttp://www.stattools.net/StatToolsAbout.phphttp://www.stattools.net/index.phphttp://www.stattools.net/SeqPaired_Exp.php#tophttp://www.stattools.net/SeqPaired_Exp.php#top


17/33

Program for sequential comparison of 2 unpairedmeansProgram for sequential comparison of 2 unpairedordinal arraysProgram for sequential comparison of 2 unpaired

proportionsProgram for sequential comparison of 2 unpairedsurvival rates

Table of Contents :IntroductionUnpaired countsUnpaired meansUnpaired ordinal arraysUnpaired proportionsUnpaired survivals

References

Introduction

General discussions on sequential analysis arepresented ina separate pageand will not be repeatedhere.

This page briefly explains the Triangular Test that wasdeveloped in the late 1980s and 1990s by Whitehead.For those interested in obtaining a full understanding of

the theories and methodologies of the Triangular Test,Whitehead's text book (see reference) is highlyrecommended.

The Triangular test is a sequential statistical method forcomparing two groups, based on the relationshipbetween Fisher's information V (expression of thequantity of data) and the efficiency score Z (expressionof the effect size). The calculation of V and Z dependson the nature of the measurements concerned, but theinterpretations of their relationship are the same. V and

Z can be calculated at any time during the study.

Statistical borders are drawn that allows the researcherto make one of 3 decision whenever the data isreviewed. These are to continue with the experiment, toreject the null hypothesis and stop the experiment, or toaccept the null hypothesis and stop the experiment.
http://www.stattools.net/SeqUnpairedMean_Pgm.phphttp://www.stattools.net/SeqUnpairedMean_Pgm.phphttp://www.stattools.net/SeqUnpairedMean_Pgm.phphttp://www.stattools.net/SeqUnpairedOrd_Pgm.phphttp://www.stattools.net/SeqUnpairedOrd_Pgm.phphttp://www.stattools.net/SeqUnpairedOrd_Pgm.phphttp://www.stattools.net/SeqUnpairedProp_Pgm.phphttp://www.stattools.net/SeqUnpairedProp_Pgm.phphttp://www.stattools.net/SeqUnpairedProp_Pgm.phphttp://www.stattools.net/SeqUnpairedSurv_Pgm.phphttp://www.stattools.net/SeqUnpairedSurv_Pgm.phphttp://www.stattools.net/SeqUnpairedSurv_Pgm.phphttp://www.stattools.net/SeqUnpaired_Exp.php#Introductionhttp://www.stattools.net/SeqUnpaired_Exp.php#Introductionhttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20countshttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20countshttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20meanshttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20meanshttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20ordinal%20arrayshttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20ordinal%20arrayshttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20proportionshttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20proportionshttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20survivalshttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20survivalshttp://www.stattools.net/SeqUnpaired_Exp.php#Referenceshttp://www.stattools.net/SeqUnpaired_Exp.php#Referenceshttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/Seq_Exp.phphttp://www.stattools.net/SeqUnpaired_Exp.php#Referenceshttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20survivalshttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20proportionshttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20ordinal%20arrayshttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20meanshttp://www.stattools.net/SeqUnpaired_Exp.php#Unpaired%20countshttp://www.stattools.net/SeqUnpaired_Exp.php#Introductionhttp://www.stattools.net/SeqUnpairedSurv_Pgm.phphttp://www.stattools.net/SeqUnpairedSurv_Pgm.phphttp://www.stattools.net/SeqUnpairedProp_Pgm.phphttp://www.stattools.net/SeqUnpairedProp_Pgm.phphttp://www.stattools.net/SeqUnpairedOrd_Pgm.phphttp://www.stattools.net/SeqUnpairedOrd_Pgm.phphttp://www.stattools.net/SeqUnpairedMean_Pgm.phphttp://www.stattools.net/SeqUnpairedMean_Pgm.php


18/33

The baseline and two borders forms a triangle. Whilethe V / Z plot remains within the triangle, no decisionshould be made other than to collect more data. If theplot crosses the outter border, then the null hypothesiscan be rejected (significant difference exists). If the plot

crosses the inner border than the null hypothesis canbe accepted (no significant difference).

The primary straight line borders are calculated on theassumption that the data will be reviewed after everycase. These borders are narrowed to become morepowerful if the data are reviewed less frequently, theextent of the narrowing depending on the sample sizebetween the reviews. The final borders, with periodicnarrowing, looks like a christmas tree.

The methods of calculating the effects size for differenttypes of data, and how the borders are defined, will notbe explained in details in these pages. These are welldescribed in Whitehead's book, and the formalstatistical software available in a computer programPEST 3 published by the Readings University.

Please note that the coordinates defined in these pagesare based on a 2 sided test (detecting a difference ineither direction). If a one sided test is to be used (onegroup more than the other but not interested if it is the

other way around) then the type I Error (Alpha) usedshould be doubled (eg. 0.1 instead of 0.05).

At the end of the analysis, a termination test can bedone by calculating the final effect size ThetaT=Z/sqrt(V). For the null hypothesis, T0=0 andSD(T0)=sqrt(V). The normalized z test can therefore beused to test whether T deviates from null.

In his book, Whitehead presented 5 models, fornormally distributed means, Poisson distributed counts,

binomially distributed proportions, survival rates, andnon-parametric ordinal arrays. These are discussedindiidually as in the following sections.

Back to Top

Unpaired counts
http://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpaired_Exp.php#top


19/33

Poisson distribution concerns the number of events,and these are measured as Lambda (), the number ofevents per case. Common events may occur more thanonce per individual so is >1. Uncommon events do nothappen to everyone, so when averaged out it may be

less than 1. can therefore be any number between 0and infinity. Examples of are number of falls in anursing home per month, the number of cells seen in aparticular size grid under the microscope, number ofasthma attacks in per 100 children per month, and soon.

If the expected experimental group =L1, and controlgroup =L2, then the effect size Theta = abs(-log(L1/L2)).

For theTriangular Test for counts, the parameters to beset are as follows

Probability of Type I error, usually set to 0.05 Power (1 - ), usually set to 0.8 The two averaged counts (s), L1 and L2, the

ratio of which will form the effect size. The anticipated ratio between the two groups

(n1/n2). This is usually 1 for equal size samples.

The interpretation of the output is best explained by the

following example

Example of sequential comparison of 2unpaired counts

= 0.6931Maximum number of subjects = 679Border (significant difference) Z = 6.1778 + 0.2425VBorder (no significant difference) Z = -6.1778 + 0.7274V

We wish to compare a special breathing exercise

against no exercise as they affect the frequency ofasthma attacks in children. We will conclude that theexercise reduces asthma attack if it can half thefrequency. We therefore set the two averaged countsas 20 asthma attacks per 100 children over 1 months(L1=0.2) for the control group, and 10 (L2=0.1) for theexercise group. We designate =0.05 and power=0.8,and will allocate the case into two equal size groups
http://www.stattools.net/SeqUnpairedCount_Pgm.phphttp://www.stattools.net/SeqUnpairedCount_Pgm.phphttp://www.stattools.net/SeqUnpairedCount_Pgm.phphttp://www.stattools.net/SeqUnpairedCount_Pgm.php


20/33

(r=1). The borders as calculated are shown above andto the right.

Control Tmt

Review cases count cases count

2 month 50 10 53 8

4 month 100 18 99 15

6 month 150 28 160 25

We recruited the research subjects, and randomisedthem into two groups to receive or not receive theexercise. After completion of the treatment (or

equivalent time if not treated), the child is followed upfor 1 month, and any asthmatic attack is recorded. Theplan is to review the data available every 2 months untilthe study is completed.

The results at the end of 6 months (third reviews) arepresented in the table to the left, and the Triangular plot

shown to the right.

As the test is 2 sided, two triangles, one for group1>group 2, and the other for group 2>group 1.

The straight line borders are for use if the data isreviewed after every case, and the inward christmas

tree like narrowings are border adjustments when thereare more than 1 case added between reviews, theextent of the narrowing depends on the sample sizebetween reviews.

The relationship between V and Z are shown in the linejoining the dots.


21/33

At the first review at 2 months, there were 50children in the control group who had betweenthem 10 asthma attacks (=10/50=0.2), and 53children who received the exercise who had 8asthma attacks between them (=8/53=0.15).

The resulting plot was within the borders, and thedecision was to continue to collect data.

At the second review at 4 months, the number ofcompleted cases have increased. There werenow 100 children in the control group who hadbetween them 18 asthma attacks(=18/100=0.18), and 99 children who receivedthe exercise who had 15 asthma attacksbetween them (=15/99=0.15). As the resultingplot remained within the borders, the decisionwas to continue to collect data.

At the third review at 6 months, there were now150 children in the control group who hadbetween them 28 asthma attacks(=28/150=0.19), and 160 children who receivedthe exercise who had 25 asthma attacksbetween them (=25/160=0.16). The plot nowtraversed both inner borders, and the study couldbe terminated, and the conclusion drawn that thethe incidents of asthma in the two groups werenot significantly different.

The terminal analysis showed effectsize(Z/sqrt(V))=0.6473, p=0.2587. The sample size forthe two groups combined was 310, less than half of themaximum number of cases required for the TriagularTest, and well short of the 436 cases sample size if thestudy was to be a fixed sample one. This is because thedifference between the two groups in reality was verymuch less than anticipated, and the null hypothesiscould be accepted earlier.

Back to Top

Unpaired means

For theTriangular Test for comparing two means, theparameters to be set are as follows

Probability of Type I error, usually set to 0.05 Power (1 - ), usually set to 0.8
http://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpairedMean_Pgm.phphttp://www.stattools.net/SeqUnpairedMean_Pgm.phphttp://www.stattools.net/SeqUnpairedMean_Pgm.phphttp://www.stattools.net/SeqUnpairedMean_Pgm.phphttp://www.stattools.net/SeqUnpaired_Exp.php#top


22/33

The expected difference between the means ofthe two groups, and the within group standarddeviation. The effect size used is the ratiobetween the difference and standard deviation(es = diff / sd). If the standard deviation is

unknown, set it to 1, and choose the differenceaccording to the effect the model should detect.

An effects size of 0.3 or less is considered small,one 0.7 or more considered large, and in mostclinical circumstances somewhere between 0.3and 0.7.

The anticipated ratio between the two groups(n1/n2). This is usually 1 for equal size samples.

The interpretation of the output is best explained by thefollowing example

Example of sequential comparison of 2unpaired means

Effect size (diff / sd) = 0.6931Maximum number of subjects = 110Border (significant difference) Z = 6.4232 + 0.23325VBorder (no significant difference) Z = -6.4232 + 0.6996V

We wish to compare the at term birthweight of babies,between boys and girls. We assume that birthweights

are normally distributed, with a standard deviation of0.3Kgs. We would be happy to decide that boysweighed more than girls if the difference is 0.2kg ormore. We designate =0.05 and power=0.8, and willanticipate that the number of boys and girls deliveredare roughly the same so we set the ratio to 1. Theborders as calculated are shown above and to the right.

Boys Girls

Review n mean sd n mean sd

week 1 10 3.8 0.2 10 3.6 0.3

week 2 20 3.7 0.3 22 3.5 0.2

week 3 28 3.8 0.3 30 3.6 0.3


23/33

We decided to weigh all the babies (in Kgs) deliveredeach week, and review the data weekly.

The results at the end of 3 weeks (third reviews) are

presented in the table to the left, and the Triangular plot

shown to the right.

As the test is 2 sided, two triangles, one for boys>girls,and the other for girls>boys.

The straight line borders are for use if the data isreviewed after every case, and the inward christmastree like narrowings are border adjustments when thereare more than 1 case added between reviews, theextent of the narrowing depends on the sample sizebetween reviews.

The relationship between V and Z are shown in the line

joining the dots.

At the first review after 1 week, there were 10boys with mean=3.8 and sd=0.2, and 10 girlsmean 3.6 and sd=0.3. The resulting plot hasreached the inner border of the lower triangle,and at this point the conclusion that girls are notsignificantly heavier than boys could be drawn.However the plot was still within the borders ofthe upper triangle, so no decision could be madeas to whether boys are heavier than girls, so the

correct decision was to continue to collect data. At the second review after 2 weeks, the number

of babies delivered has increased. There werenow 20 boys with mean=3.7 and sd=0.3, and 22girls with mean=3.5 and sd=0.2. The plot hasnow reached the adjusted border, and at thispoint the study could and should have stopped,and the conclusion that boys are heavier than


24/33

girls could be made. The extension of the study to the third week was

unnecessary, and it merely confirmed the findingof the second review.

The terminal analysis at the end of the second weekshowed effect size(Z/sqrt(V))=2.5255 p= 0.0058. Thesample size at that time was 20, less than the 110maximum required and well short of the 71 casesrequired for a fixed sample size study.

Back to Top

Unpaired ordinal arrays

Non parametric data are difficult to handle using

sequential analysis, and often the data is transformed insome manner so that they can be handled as normallydistributed measurements or as proportions.

However, in many instances, the ordinal data must behandled the way it is presented. This is so formeasurements such as a 3 point pain scale (0=none,1=some pain, 2=severe pain) or the Likert scale of 5levels of agreement with something. For these theTriangular Test for ordinal arrayscan be used, theparameters to be set are as follows

Probability of Type I error, usually set to 0.05 Power (1 - ), usually set to 0.8 The number of divisions in the ordinal scale. For

example, 5 in a Likert scale. The anticipated ratio between the two groups

(n1/n2). This is usually 1 for equal size samples. The expected difference between the means of

the two groups is set in detail, in an array of twocolumns, the number of rows being the same asthe number of divisions. Each cell of this matrix

contains the probability of the group (column)and level(row). The sum of each column must be1 (100%).

The setting of parameters and interpretation of theoutput is best explained by the following example

Example of sequential comparison of 2
http://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpairedOrd_Pgm.phphttp://www.stattools.net/SeqUnpairedOrd_Pgm.phphttp://www.stattools.net/SeqUnpairedOrd_Pgm.phphttp://www.stattools.net/SeqUnpaired_Exp.php#top


25/33

ordinal arrays

Tmt Control

none 0.3 0.05

Moderate 0.65 0.8

Severe 0.05 0.15

We wish to compare the effect of an analgesics with noanalgesics on pain experienced during a minor surgicalprocedure. We will measure pain experienced in a 3point scale (0=none, 1=mild pain, 2=severe pain).

Effect size = 2.1

Maximum number of subjects = 55Border (significant difference) Z = 2.0419 + 0.7336VBorder (no significant difference) Z = -2.0419 + 2.2007V

We will consider the analgesic effective if the responseis as good or better than that set out to test is as shownabove and to the left. Without analgesics (column 2),we anticipate 5% to have no pain, 80% moderate pain,and 15% with sever pain. With analgesics, we hope30% will experience no pain, 65% some pain, and 5%severe pain.

The borders as calculated are shown above and to theright.

Review Pain Tmt Control

week 1 none 2 0

moderate 3 5

severe 0 2

week 2 none 3 0

moderate 9 10

severe 1 3


26/33

week 3 none 8 1

moderate 10 12

severe 1 5

We decided to recruit suitable subjects from the surgicallist, randomise participants to receiving or not receivingthe analgesics, and obtain their response at the end ofthe operation. We planned to review the data on aweekly basis.

The results at the end of 3 weeks (third reviews) arepresented in the table to the left, and the Triangular plot

shown to the right.

As the test is 2 sided, two triangles, one fortreated>controls, and the other for control>treated.



At the first review after 1 week, there were 5treated subjects, 2 (40%) had no pain and 3(60%) had moderate pain. There were also 7control subjects, none (0%) were pain free, 5(71%) had moderate pain and 2 (29%) hadsevere pain. The resulting plot has crossed theinner border of the upper triangle, and at thispoint the conclusion those treated had no greater


27/33

pain than the control cases could be drawn.However the plot was still within the borders ofthe lower triangle, so no decision could be madeas to those treated had less pain.

At the second review after 2 weeks, there were

now 13 treated cases, 3 (23%) were pain free, 9(69%) moderate pain, and 1 (8%) severe pain.There were also 13 control cases, none (0%)were pain free, 10 (77%) had moderate pain, and3 (23%) severe pain. The Trianglular Test plothowever remained within the lower triangle, somore data needed to be collected.

At the third review after 3 weeks, there were now19 treated cases, 8 (42%) were pain free, 10(53%) with moderate pain, and 1 (5%) withsevere pain. There 18 control cases, 1 (6%) was

pain free, 12 (67%) had moderate pain, and 5(17%) severe pain. The Trianglular Test plot nowcrosses the outter border of the lower triangle, sothe study can terminate, and the conclusiondrawn that those receiving the analgesics hadsignificantly less pain.

The terminal analysis at the end of the second weekshowed effect size(Z/sqrt(V))=2.8407 p= 0.002. Thesample size at that time was 37, less than the 55maximum required but 2 more than the 35 required for

a fixed sample size study. this shows that, although theaveraged sample size required for the Triangular Test isless than that in a fixed sample size study, there arecircumstances when the same or greater sample sizewill be used.

Also, when the study is reviewed, the Triangular Testplot at the end of the second week was so close tostatistical significance that an argument can be made tobreak protocol, and make the third review sooner. Hadthis happened, a significant result using a smaller

sample size might have been achieved.

Back to Top

Unpaired proportions

For theTriangular Test for comparing two proportions,the parameters to be set are as follows
http://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpairedProp_Pgm.phphttp://www.stattools.net/SeqUnpairedProp_Pgm.phphttp://www.stattools.net/SeqUnpairedProp_Pgm.phphttp://www.stattools.net/SeqUnpairedProp_Pgm.phphttp://www.stattools.net/SeqUnpaired_Exp.php#top


28/33

Probability of Type I error, usually set to 0.05 Power (1 - ), usually set to 0.8 The expected proportion in the two groups. The anticipated ratio between the two groups

(n1/n2). This is usually 1 for equal size samples.

The interpretation of the output is best explained by thefollowing example

Example of sequential comparison of 2unpaired proportions

Effect size log(odds ratio) = 1.3863Maximum number of subjects = 112Border (significant difference) Z = 3.0889 + 0.484VBorder (no significant difference) Z = -3.0889 + 1.4548V

The transport department noticed that only 20% ofteenagers pass their driving test in the first attempt. Itwas decided to introduce a training course, but this isexpensive, so the department wishes to go ahead onlyif this can increase the pass rate to at least 50%.

We designate =0.05 and power=0.8, the twoproportions 0.5 and 0.2, and set the ratio of the twogroups to 1. The borders as calculated are shownabove and to the right.

Passed Failed

Review Training Control Training Control

Month 1 4 2 4 7

Month 2 7 4 8 11

Month 4 15 6 15 24

Teenagers were recruited to the trial as they applied fortesting, and randomly allocated to training before thetest. A decision to review the result monthly was made.The results at the end of 3 months (third reviews) arepresented in the table to the left, and the Triangular plot


29/33

shown to the right.

As the test is 2 sided, two triangles, one fortrained>controls, and the other for controls>trained.

The straight line borders are for use if the data isreviewed after every case, and the inward christmastree like narrowings are border adjustments when there

are more than 1 case added between reviews, theextent of the narrowing depends on the sample sizebetween reviews.


At the first review after 1 month, there were 8teenagers trained, and 4(50%) passed thedriving test. There were 9 controls and 2 (22%)passed. The Triangular Test plot at this time

impinged on the inner border of te lower triangle,so a conclusion that those trained did not doworse than the control could be made. Howeverthe plot was within the upper triangle, so noconclusion could be made whether those traineddid better, so data collection had to continue.

At the second review after 2 months, the numbertrained was 15 and 7 (47%) passed. There were15 controls 4(27%) of whom passed. The plotremained within the upper triangle so datacollection continues. As the plot was nowhere

near the border, it was decided to improve thepower by not reviewing the data at 3 months,rather to do so in 2 months time at month 4.

At the third review 4 months after the studybegan, the number trained was 30 and 15 (50%)of whom passed the test. There were also 30controls, 6 (20%) of whom passed. The plot nowcrosses the adjusted outter border of the upper


30/33

triangle, so the study can be terminated, and theconclusion made that the training coursesignificantly improved the pass rate of the test.

The terminal analysis at the end of the second week

showed effect size(Z/sqrt(V))=2.436 p= 0.007. Thesample size at that time was 60, less than the 112maximum required and short of the 72 cases requiredfor a fixed sample size study.

Back to Top

Unpaired survivals

In theTriangular Test for survival, the parameters to beset are as follows

Probability of Type I error, usually set to 0.05 Power (1 - ), usually set to 0.8 The two survival or hazard rates anticipated. It

does not matter whether it is hazard or survivalrate, as hazard rate = 1 - survival rate. Also, ascalculations is based on survival in a two sidedmodel, the results will not be much effected.However, when the ratio of the sample size(r=n1/n2) is not 1, then assigning the rateswrongly to the groups will effect the calculations

for the theoretical maximum sample sizes forboth the sequential and the fixed sample sizemodels.

The number of intervals in the survival analysis.For example, 5 for 5 year survival rate.

The anticipated ratio between the two groups(n1/n2). This is usually 1 for equal size samples.

The setting of parameters and interpretation of theoutput is best explained by the following example

Example of sequential comparison of 2survival rates

Effect size = 0.66Maximum number of subjects = 277Border (significant difference) Z = 6.4449 + 0.2324VBorder (no significant difference) Z = -6.4449 + 0.6972V
http://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpairedSurv_Pgm.phphttp://www.stattools.net/SeqUnpairedSurv_Pgm.phphttp://www.stattools.net/SeqUnpairedSurv_Pgm.phphttp://www.stattools.net/SeqUnpairedSurv_Pgm.phphttp://www.stattools.net/SeqUnpaired_Exp.php#top


31/33

The current 3 year survival rate for a particular cancer is30% (0.3). We wish to compare the survival ratebetween the current treatment (control) and the use of anew drug (Tmt). We will accept the new drug to bebetter if it can improve the 3 year survival rate to 50%

(0.5). We designate =0.05, power=0.8, the two survivalrates to compare as 0.3 and 0.5, and the number ofintervals = 3 for 3 year survival rate. We will allocateequal numbers to the two groups so that the ratio = 1.The borders as calculated are shown above and to theright.

Control Treated

died survived died survived

Review 1 year 1 3 30 3 20

year 2 6 20 4 10

year 3 5 11 2 6

Review 2 year 1 8 40 5 45

year 2 12 25 10 32

year 3 8 15 9 20

Review 3 year 1 20 60 8 62

year 2 16 40 11 41

year 3 15 20 11 25

We decided to recruit suitable subjects for the study.We will perform the first review 3 years after the start ofthe study, then repeat the review on a yearly basis.

Survival rate year 3

Control Tmt

Review1 0.4808 0.4658

Review2 0.3672 0.4729


32/33

Review3 0.3061 0.4849

The results at the end of 5 years (third reviews) arepresented in the table above and to the left, and thesurvival rates in the two groups at the end of the thirdyear at the 3 reviews are shown above and to the right.Triangular plot shown to the right.

As the test is 2 sided, two triangles, one fortreated>controls, and the other for control>treated.



At the first review after 3 years, the 3 yearsurvival rates were 48% and 47% for the controland treatment groups. The Triangular plot hasnot crossed any of the borders, and the onlydecision possible is to continue with the study.

At the second review 1 year later, the 3 yearsurvival rates were 37% and 47% for the control

and treatment groups. The Triangular plot hasnot crossed any of the borders, and the onlydecision possible is to continue with the study.

At the third review 1 year later (5 years after thestart of the study), the 3 year survival rates were31% and 49% for the control and treatmentgroups. The Triangular plot has now reached theoutter border of the lower triangle, so the study


33/33

can be termionated, and the conclusion that thenew treatment offers a better 3 year survival ratecan be made.

The terminal analysis at the end of the second week

showed effect size(Z/sqrt(V))=2.375 p= 0.0088. Thesample size is difficult to state, as subjects entered andremained in the study for different periods of time.However there were 80 control and 70 treated in thestudy, a sample size of 150, in excess of the 71calculated for a fixed sample size study.

Back to Top

References

Whitehead John (1992). The Design and Analysis ofSequential Clinical Trials (Revised 2nd. Edition) . JohnWiley & Sons Ltd., Chichester, ISBN 0 47197550 8. p.48-50

Brunier H and Whitehead J (1998) Planning andEvaluation of Sequential Trials (PEST) Ver 3. Dept.

Applied Statistics, University of Reading, Earley Gate 3Whiteknights Rd., PO Box 238, Reading RG6 2AL, UK.(this is where the complete software package can bepurchased).

Back to Top
http://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpaired_Exp.php#tophttp://www.stattools.net/SeqUnpaired_Exp.php#top

Date post:	14-Apr-2018
Category:	Documents
Upload:	gaurav-pareek
View:	222 times
Download:	0 times

Sequential Analysis Explained

Documents