IMST 2008 / FIM XVI
Decision theoretic Bayesian hypothesis testing with focus on skewed alternatives
By
Naveen K. BansalRu Sheng
Marquette UniversityMilwaukee, WI 53051
Email: [email protected]
http://www.mscs.mu.edu/~naveen/
0:1,0:1)0,(0:0
),P( :Modely Probabilit
HHvssayH
IMST 2008 / FIM XVI
Directional Error (Type III error):
Sarkar and Zhou (2008, JSPI)Shaffer (2002, Psychological Methods)Finner ( 1999, AS)Lehmann (1952, AMS; 1957, AMS)
Main points of these work is that if the objective is to find the true Direction of the alternative after rejecting the null, then a Type III error must be also controlled.
Type III error is defined as P( false directional error if the null is rejected). The traditional method does not control the directional error. For example,
0. if occurserror an , and ,|| 2/2/ tttt
IMST 2008 / FIM XVI
.action gfor takin loss),(},1,0,1{
)(0,in containedsupport ith density w)(1
,0)(-in containedsupport ith density w)(1
)0()(1),0()(0),0()(1)(1-
where
)(11)(00)(11)(
0:1,0:1,)0,(0:0
),P( :Modely Probabilit
AaaLA
g
g
IgIIg
ppp
HHsayH
IMST 2008 / FIM XVI
same. theare error) I (Type ),0( for which rules all of class theconsider
}:))(),0(),({()S(
space thecomparingby compared becan rulesdecision ,prior fixed aFor
),0()0(
)(),()(
)(),()( where
)()0()()(
bygiven is ruledecision afor risk Bayes The
*11011
0
0 111
0 111
11100111
R
Drrr
Rr
dRr
dRr
rprprpr
BayesRule
)(1 r
)(1 r11 and on
depends slopepp
11 pp
Bayes Rule
11 pp
IMST 2008 / FIM XVI
direction. positive in therisk averagehan smaller t be willdirection negative in the
rule Bayes theofrisk average e then th),11( 1an likely th more is 1t known tha isit apriori if that implies theoremThis:
)'('
1)(1
and
)'('
1)(1
thenly,respective ' and under rules Bayes theare 'B and B if and , '
11 If :1
)' tocomparisonin under likelely more is 1 (i.e. '11
)(2'1)(00)(1
'1-)('
),(11)(00)(11-)(
priors twofollowing heConsider t
ppHH
BrBr
BrBr
pp
Hpp
ppp
ppp
Remark
Theorem
IMST 2008 / FIM XVI
Relationship with the False Discovery Rates
s.discoverie false theseof average weighted theis )( rate.dicovery -non false expected as )(
and rate,discovery positive-non false expected as dinterprete becan )(Similarly framwork.Bayesian in the ratediscovery negative-non false expected iswhich
)]1|0([1
)]0|0([)()0()(
0)0(0)0(0)0(
),(
0100
)1,(0100
)0,(01 0 0
,-1)(
loss 1"-"0 heConsider t
0
1
0 11
BB
B
B
rrr
PE
PEdPr
PPP
R
LLL
IMST 2008 / FIM XVI
rule Bayes randomized-non}1,0,1{
tindependen ,...,2,1),(11)(00)(11~
0:1,0:1,0:0
,...,2,1),,;(~i
withdatat independen },...,2,1,{
iB
mipppi
iiHi
iHiiH
miiif
mIi
xX
X
ProblemHupotheses Multiple
An Application:
An experiment was conducted to see the effect of a sequence knockdown from non-coding genes. Hypothesis was that this knockdown will cause the overexpression of the mRNAs of the coding genes. The implication of this is that this will shows how the non-coding genes interact with coding genes. In other words, non-coding genes also play a part in protein synthesis.
Here it can be assumed that that the effect of knockdown on the coding genes (if there is any) would be mostly overexpression of mRNAs than underexpression of mRNAs. The objective would be to detect as many of overexpressed genes as possible.
IMST 2008 / FIM XVI
IMST 2008 / FIM XVI
Accept Accept Accept Total
is true
is true
is true
Total
U 1V 2V 0m
1H1T 1S 2W 1m
1H2T 1W 2S 2m
0R 1R 2R
0H 1H 1H
0H
Table 1Possible outcomes from hypothesis tests
m
m
)2003,(]0|[]0|[
][][
22
2211
1
111
2
221
1
111
ASStoreyRRWVEpFDRR
RWVEpFDR
RWVEFDR
RWVEFDR
IMST 2008 / FIM XVI
)1|1()1|0()(1
)1|1()1|0()(1
then trials,l trinomiaidenticalandt independen are ,...,2,1),1,0,1( If ly.respective 1
and 0,1 wrt of onsdistributi marginal theare 1,0,1
where,1)1(0)0(1)1(~| that Assume
.,...,2,1each for ly,respective ,1 and 1 accepting of
regions thebe 1 and 1Let .,...,2,1 statisticson test based are testshypothesis that Suppose :
THPTHPpFDR
and
THPTHPpFDR
miiHiHiHiTFFF
FiHIFiHIFiHIiidiHiT
miiHiHmTTTm
2 Theorem
IMST 2008 / FIM XVI
This theorem and our previous discussion on the Bayes decision rule implies that under the independent hypotheses testing setting, the Bayes rule obtained from a single test problem
which minimizes the average expected false discoveries, can be used under multiple hypotheses problem. The false discovery rates of this procedure can be obtained by computing the posterior probabilities.
,0:1,0:1 vs.0:0 HHH
2 and 1 pFDRpFDR
)1|1()1|0()(1
)1|1()1|0()(1
BHPBHPBpFDR
BHPBHPBpFDR
The prior probabilities will determine whether the positive hypothesis or the negative hypothesiswill be more frequently correctly detected.
1 and 1 pp1H 1H
IMST 2008 / FIM XVI
Computation of Bayes Rule:
].,1|)1,([ and ],1|)1,([ computing and)|1( and ),|0(),|1( computing invole partsn Computatio
ly.respective )(1 and )(1under of densities posterior theare ),1|( and ),1|( and ly;respective 1 and ,0,1 ofiesprobabilitposterior theare )|1( and ),|0(),|1( Here
),1|()|1()0()|0(),1|()|1()|(
]}|),([1,0,1
min]|),([:)1,0,1({)(
XXXXX
XXXXX
XXXXXX
XX
HLEHLEHHH
HHHHHHHH
HHIHHH
jLEj
iLEiB
IMST 2008 / FIM XVI
))],(/)0,([log()(
is )( of choice good One null. thefrom departure of magnitude hereflect t )( Here,
.constants negative-non some are 1 and 1,0 where
))(1(1)(1)),(1(1)(1 )),((10)(0
be wouldchoice goodA
.1)(1)(1)(0 loss, 1"-"0 For the
0)(1
00)1,(,0)(1
00)1,(,0)(0
00)0,(
XfXfEq
lll
qlqqlqqlq
qqq
qLqLqL
Choice of the loss function:
IMST 2008 / FIM XVI
)1|(1)()1|(1
if )1(1select Otherwise,
.0
1)1|(
)0|( and
0
1)1|(
)0|(
if 0reject ds,other worIn
.under density marginal theis )|( Here
)}1,0,1),|(max()|(:{
bygiven is rule Bayes theloss, 1"-"0 he Under t:
loss. 1"-"0 a have then we,1)(1)(1)( If
HfpHfp
HH
p
p
Hf
Hf
p
p
Hf
Hf
H
jHjHf
jjHfjpiHfipiB
qqq
XX
X
X
X
X
XX
XX
3 Theorem
IMST 2008 / FIM XVI
ondistributi )1,0( half-right:1
ondistributi )1,0( half-left:1
)(11)0(0)(11)(:Prior
0:1,0:1.0:0
)1,( :Modely Probabilit
:Example
N
N
pIpp
HHvsH
N
IMST 2008 / FIM XVI
)1/0(11 and )1/0(1
1
toequivalent isregion rejection theThus ly.respective functions,increasing and decreasinglly monotonica are 1 and 1 that Note
)1
())1(2
22exp(
12)(1
)1
())1(2
22exp(
12)(1
where),(11)()(11 if )1(1select otherwise
,1/0)(1 and 1/0)(1
if 0reject rule Bayes theloss, 1"-"0 he Under t:
ppTxppTx
TT
nnx
nxn
nxT
andnnx
nxn
nxT
xTpxTpHH
ppxTppxT
H
4 Theorem
IMST 2008 / FIM XVI
Power Comparison:
line dotted red : testunbiased UMPline) blue :22.01,39.00,39.01 test withBayes
unbiased. UMPisregion rejection the,11When
ppp
pp
-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
mean value
prob
of r
ejec
tion
IMST 2008 / FIM XVI
-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
mean value
prob
of r
ejec
tion
line dotted red : testunbiased UMPline) blue :39.01,39.00,22.01 test withBayes ppp
IMST 2008 / FIM XVI
where),(1)|1()()(1)|1E( if )1(1select otherwise
,)|1(
)|0()(1 and
)|1(
)|0()(1
if 0reject rule Bayes theloss, 1"-"0 he Under t:5
result. following get the weproblem, normal For the
ly.respective )|1( and ),|0(),|1(by 1,0,1 replace tobe woulddifference
only e that thsee easy to isIt prior.Dirichlet aconsider can One
xTpExTpHH
pE
pExT
pE
pExT
H
XpEXpEXpEpandpp
XX
X
X
X
X
Theorem
unknown? are 1pand,0p,1p ifWhat
IMST 2008 / FIM XVI
Another Approach:
values.in dispersion the and values,in skewness thereflects Here
).0()()(2)('
normal-skew example,for prior; skewed a is )(' where
),(')01()0(0p)(
prior heConsider t
I
pI
IMST 2008 / FIM XVI
-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
mean value
prob
of r
ejec
tion
Power Comparison of a skew-normal Bayes with the UMP test
IMST 2008 / FIM XVI