Bayesian Hypothesis Testing for Proportions

Antonio Nieto / Sonia Extremera / Javier Gómez

PhUSE Annual Conference, 9th-12th Oct 2011, Brighton UK

Introduction

•Tests on proportions –Frequentist approach

If pvalue < significance level → Null hypothesis will be rejected

–Bayesian approach

Probability under any hypotheses → Comparison to see what is the most plausible alternative

Both approaches can coexist and theyshould be used in the statistical interest

Bernouilli distribution

•The variable that records the patient’s response follows a Bernouilli distribution

–Discrete probability distribution, which takes value 1 “success” with probability “p” and 0 “failure” with probability “1-p”

qpxVarpxE

otherwise

xifqp

xifp

xf

][ ][

0

0 1

1

)(

60% to be responder

40% to be non-responder

Bernouilli

•Considering the probability to respond is 0.60

After treatment

FAILURE

SUCESS

Binomial distribution

•Sum of “n” Bernouilli experiments

–Discrete probability distribution, which counts the sum of successes/failures out of ‘n’ independent samples

qpnxVarpnxE

nqpx

nxf xnx

][ ][

...1,0x )(

Binomial

Considering the probability to respond (p=0.60) in 10 patients then

E(x)=10 x 0.6=6 Var(x)=10 x 0.6 x 0.4=2.4

Exact confidence intervals, hypothesis tests can be calculated, binomial could be also approximated by the Normal distribution

Frequentist approach

A possible solution: Binomial distribution will be approximated with the Normal distribution and then taking a decision based on the pvalue associated to the Gauss curve

)1,0(p-ˆ

,ˆ

valuefixed-pre a is p where

pp: H1 pp :H0

00

0000

0

0

0

NZ

nqp

p

n

qppN

n

xp

Bayes’ theorem (1763)

• It expresses the conditional probability of a random event A given B in terms of the conditional probability distribution of event B given A and the marginal probability of only A

• Let {A1,A2,...,An} a set of mutually exclusive events, where the probability of each event is different from zero. Let B any event with known conditional probability p(B|Ai). Then, the probability of p(Ai|B) is given by the expression:

(B) p

)(A p)A|(B pB)(A p

yprobabilit posteriori B)|(A p-

Ain B ofy probabilit )A|(B p-

yprobabilit priori a )(A p-

:where

i

ii

i

iii

|

Bayes’ in medicine

• Sensitivity: Probability of positive test when we know that the person suffers the disease

• Specificity: Probability of negative test when we know that the person does not suffer the disease

Probability of hypertension=0.2, sensitivity=91% specificity=98%

Probability to have hypertension if positive test is obtained

p=0.91 x 0.2/ (0.91 x 0.2+(1-0.98) x 0.8)=0.9192

Bayesian approach

•A priori distribution

•Sample distribution

•Posterior conjugate distribution

Beta distribution

•Continuous distribution in the interval (0,1)

•Posterior Beta (a,b) where a=∑xi+α, b=n-∑xi+ ß

2

11

b)(a 1)b(a

ab][

ba

a ][

)!1((n) ;0b 0,a ;)-(1 (b)(a)

b)(a )(

xVarxE

nxxxf ba

No ‘a priori’ information

•As initial assumption probability any value between zero and one Uniform (0,1)=Beta (1,1)

•Sample distribution Binomial (n,p)

•Posterior Beta (a,b) where a=∑xi+1, b=n-∑xi+1

Example 1

N=40, no prior information:– H0: Proportion of responders is ≤40%– H1: Proportion of responders is >60%

If 24 successes then posterior probability Beta (25,17)

H0 H1 X N TestProb. under

H0Prob. under

H1

p<=0.4 p>0.6 24 40H1 is more probable than H0

0.005347226 0.48303

Prior distribution: Uniform (0,1)

Prior Knowledge

•Bayesian tests is enhanced when some information is available

– Example the probability will fall [0.3-0.7]– In values relatively high of α and ß, Beta~Normal then >95% of the probability [m±2s]; where m=mean and s=standard deviation (s) – By means of a moment‘s method type

• m=α / (α + ß); s2=m(1-m) / (α + ß + 1) • α = [m2 (1-m) /s2] –m; ß = (α-mα)/m=[m (1-m)2 /s2] + m -1

•Sample distribution Binomial (n,p)

•Posterior Beta (a,b) where a=∑xi+α, b=n-∑xi+ ß

Example 2

N=40, probability will fall [0.3-0.7] with a 95% probability:– H0: Proportion of responders is ≤40%– H1: Proportion of responders is >60%

If 24 successes then posterior probability Beta (36,28)

H0 H1 X N TestProb. under

H0Prob. under

H1

p<=0.4 p>0.6 24 40H1 is more probable than H0

0.004406341 0.27539

Prior distribution: Beta (12,12)

SAS® macro

Beta distribution plots

Example 1 Example 2

Example 2 (other prior)

Prior (6,2)

Posterior (30,18)

Prob

abili

ty d

ensi

ty

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

5.5

6.0

6.5

X

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Prior (6,6)

Posterior (30,22)

Prob

abili

ty d

ensi

ty

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

5.5

6.0

6.5

X

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Prior (2,2)

Posterior (26,18)

Prob

abili

ty d

ensi

ty

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

5.5

6.0

6.5

X

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Prior (2,6)

Posterior (26,22)

Prob

abili

ty d

ensi

ty

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

5.5

6.0

6.5

X

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Conclusion

• Bayesian tests are nowadays being increasingly used, especially in the context of adaptive designs

• Very important aspects are:– Good selection of the distributions

– Clear definition of the ”a priori” information collected

• A Bayesian approach has been presented to be included in the statistical armamentarium to test proportion hypotheses – It can be also extended to other endpoints and distributions

Questions