Date post: | 07-Aug-2018 |
Category: |
Documents |
Upload: | cengiz-kaya |
View: | 212 times |
Download: | 0 times |
of 17
8/21/2019 LECTURE_02.pptx
1/17
ELEC6111: Detection and Estimation Theory
Minimax Hypothesis TestingIn deriving Bayes decision rule, we assumed that we know both the a priori
probabilities 0 and 1 as well as the likelihoods )|( 0Hyp and )|( 1Hyp . Thismeans that we have both the knowledge of the mechanism generating the state of the
nature and the mechanism affecting our observations (our measurements about the
state of nature). It is, however, possible that we may not have access to all the
information. For example, we may not know the a prioriprobabilities. In such a case,
the Bayes decision rule is not a good rule since it can only be derived for a given a
prioriprobability.An alternative to the Byes hypothesis testing, in this case, is the Minimax
Hypothesis testing. The minimax decision rule minimizes the maximum possible risk,
i.e., it minimizes,
)(),(max 10 RR over all .
Lets look at),( 0 r , i.e., the overall risk for a decision rule when the a priori
probability is]1,0[0 . It can be written as:
)()1()(),( 10000 RRr
8/21/2019 LECTURE_02.pptx
2/17
ELEC6111: Detection and Estimation Theory
Minimax Hypothesis Testing
Note that, for a given decision rule , as 0 varies from 0 to 1, ),( 0 r goes
linearly from ),0()(1 rR to ),1()(0 rR . Therefore, for a given decision rule
the maximum value of ),( 0 r as 0 varies over the interval ]1,0[ occurs either at
00 or 10 and it is )}(),(max{ 10 RR .
So minimizing )}(),(max{ 10 RR is equivalent to minimizing ),(max 010 0
r
.
Thus, the minimax decision rule is:),(maxmin 0
10 0
r
Let0
denote the optimum (Bayes) decision rule for the a priori probability 0 .
Denote the corresponding minimum Bayes risk by )( 0V , i.e., ),()( 00 rV .
It is easy to show that )( 0V is a continuous concave function of 0 for
]1,[0 o and has the end points 11)0( CV and 00)1( CV .
8/21/2019 LECTURE_02.pptx
3/17
ELEC6111: Detection and Estimation Theory
Minimax Hypothesis Testing
The following figure shows a typical graph of )( 0V and),( 0 r . Lets draw a tangent to )( 0V parallel to ),( 0 r .
Denote this line by ),(00
r .
8/21/2019 LECTURE_02.pptx
4/17
ELEC6111: Detection and Estimation Theory
Minimax Hypothesis TestingSince ),(
00 r lies entirely below the line ),( 0 r , it has a lower maximum
compared to ),( 0 r . Also note that since it touches )( 0V at 00 then 0 is the
minimum risk (Bayes) rule for a prioriprobability 0 . Since for any ]1,0[0 we can
draw a tangent to )( 0V and find the minimum risk rule as a Bayes rule, it is clear that
the minimax decision rule is the Bayes rule for the value of 0 that maximizes )( 0V .
Denoting this point by L , we note that point,
)()()}(),(max{ 1010 LLLL RRRR
8/21/2019 LECTURE_02.pptx
5/17
ELEC6111: Detection and Estimation Theory
Minimax Hypothesis Testing
Proposition: The Minimax Test
Let L be the a prioriprobability that maximizes )( 0V and such that either 0L , or 1L , or
),()( 10 LL RR
thenL
is a minimax rule.
8/21/2019 LECTURE_02.pptx
6/17
ELEC6111: Detection and Estimation Theory
Minimax Hypothesis Testing
Proof
Let ),()( 10 LL RR then for any 0 we have,
),,(),(),(minmax 0010 0
LL rrr L
So, we have
),,(maxmin),(max),(minmax 010
010
010 000
rrrL
Also, for each we have).,(minmax),(max 0
100
10 00
rr
This implies that,
).,(minmax),(maxmin 010
010 00
rr
Combining the two inequalities, we get,
.),(minmax),(maxmin 010
010 00
rr
Therefore,
.),(maxmin),( 010 0
rr LL
That is,L
is the minimax rule.
8/21/2019 LECTURE_02.pptx
7/17
ELEC6111: Detection and Estimation Theory
Minimax Hypothesis TestingDiscussion
By definition: ),()(0
00 rV . So, for every ]1,0[0 , we have )(),( 00
0 Vr and
).(),( 00
0 Vr Since ),(
00 r , as a function of 0 is a straight line, it has to be tangent to
)( 0V at .00 If )( 0V is differentiable at 0 , we have,
).()(/),()(000 10000
RRddrV
Now consider the case that )( 0V has an interior maximum but is not differentiable at that point. In
this case we define two decision rules 00lim LL
and .lim00 LL
The critical regions for these two decision rules are,
),|()()|())(1(|{01000101111
HypCCHypCCyLL
and
),|()()|())(1(|{01000101111
HypCCHypCCyLL
Take a number ]1,0[q and devise a decision rule~
L that uses the decision rule
L with probability
q and uses L
with probability q1 . It means that it decides1
H if 1
y , decides0
H if cy )(1
and
decides 1H with probability q ify is on the boundary of 1 .
8/21/2019 LECTURE_02.pptx
8/17
ELEC6111: Detection and Estimation Theory
Minimax Hypothesis TestingDiscussion
Note that the Bayes risk is not a function ofq , so )(),(
~
LL Vr L but the conditional risksdepend on q ,
).()1()()(~
LLL jjj
RqqRR
To achieve )()(~
1
~
0 LLRR , we need to choose,
.
)()()()(
)()(
1010
10
LLLL
LL
RRRR
RRq
Note that )()()( 00 LL
RRV L , so we have:
.)()(
)(
LL
L
VV
Vq
This is called a randomized decision rule.
8/21/2019 LECTURE_02.pptx
9/17
ELEC6111: Detection and Estimation Theory
Example: Measurement with Gaussian Error
Consider the measurement with Gaussian error with unifom costs
The function )( 0V can be written as,
)),(1)(1()()( 100
00
QQV
With
.2
)1
log( 10
0
0
01
2
We can find the rule making conditional risks )(0 R and )(1 R equal by letting,
))(1()( 10
and solving for .
8/21/2019 LECTURE_02.pptx
10/17
ELEC6111: Detection and Estimation Theory
Example: Measurement with Gaussian Error
We can solve this by inspection and get:
.2
10
L
So, the minimax decision rule is:
.2/)(0
2/)(1)(
10
10
yif
yify
L
Conditional risks for measurement with Gaussian error
8/21/2019 LECTURE_02.pptx
11/17
ELEC6111: Detection and Estimation TheoryNeyman-Pearson Hypothesis Testing
In Bayes hypothesis testing as well as minimax, we are concerned with the average risk, i.e., the
conditional risk averaged over the two hypotheses. Neyman-Pearson test, on the other hand, recognizes theasymmetry between the two hypotheses. It tries to minimize one of the two conditional risks with the other
conditional risk fixed (or bounded).
In testing the two hypotheses 0H and 1H , the following situations may arise:
0H is true but 1H is decided. This is called a type I error or afalse alarm. This comes from radarapplication where 0H represents no target and 1H is the case of target present. The probability
of this event is calledfalse alarm probability orfalse alarm rate and is denoted as )(F
P
1H is true but 0H is decided. This is called a type II error or a miss. The probability of this event iscalled miss probability and is denoted as )(MP
0H is true and 0H is decided. Probability of this event is )(1 FP . 1H is true and 1H is decided. This case represents a detection. The detection probabilityis
)(1)( MD PP .
In testing 0H versus 1H , one has to tradeoff between the probabilities of two types of errors. Neyman-
Pearson criterion makes this tradeoff by bounding the probability of false alarm and minimizing miss
probability subject to this constraint, i.e., the Neyman-Pearson test is,
)(max
DP subject to ,)( FP
where is the bound on false alarm rate. It is called the levelof the test.
8/21/2019 LECTURE_02.pptx
12/17
ELEC6111: Detection and Estimation TheoryNeyman-Pearson Hypothesis Testing
For obtaining a general solution to the Neyman-Pearson test, we need to define a randomized decisionrule. We define the randomized test,
L
L
L
yLif
yLifq
yLif
yL
)(0
)(
)(1
)(~
where L is the threshold corresponding to L .
While in a non-randomized rule, )(y gives the decision, in a randomized rule, )(
~
yL gives the probability ofdecision.
Then we have,
,)|()()}({)( 0~~
0
~
dyHypyYEPF
where {.}0E is expectation under hypothesis 0H . Also,
.)|()()}({)(1
~~
1
~
dyHypyYEPD
8/21/2019 LECTURE_02.pptx
13/17
ELEC6111: Detection and Estimation TheoryNeyman-Pearson Lemma
Consider a hypothesis pair 0H and 1H :00 ~: PYH
Versus
11 ~: PYH
where jP has density )|()( jj Hypyp for 1,0j . For 0 , the following statements are true:
1. Optimality: Let ~be any decision rule satisfying .)( ~ FP Let ~be any decision rule of the form)(
),|()|(0
)|()|()()|()|(1
01
01
01~
A
HypHypif
HypHypifyHypHypif
where 0 and 1)(0 y are such that .)(~
FP Then ).()(~~
DD PP
This means that any size- decision rule of form (A) is Neyman-Pearson rule.
2. Existence: For any )1,0( there is a decision rule, NP~ , of form (A) with 0)( y for which.)(
~
NPFP
3. Uniqueness: Suppose that is any Neyman-Pearson rule of size- for 0H versus 1H . Then must be ofthe form (A).
8/21/2019 LECTURE_02.pptx
14/17
ELEC6111: Detection and Estimation TheoryNeyman-Pearson Lemma (Proof)
1.Not that, by definition, we always have 0)]|()|()][()([ 01
~~
HypHypyy (why?)So, we have,
.0)]|()|()][()([ 01
~~
dyHypHypyy Expanding the above expression, we get,
.)|()()|()()|()()|()( 0
~
0
~
1
~
1
~
dyHypydyHypydyHypydyHypy
Applying the expressions for the detection probabilityandfalse alarm rate, we have:
.0)]([)]()([)()(~~~~~ FFFDD PPPPP
2. Let 0 be the smallest number such that (look at the Figure in next slide): )]|()|([ 0010 HYpHYpP .
Then if )]|()|([ 0010 HYpHYpP , choose,
)]|()|([
)]|()|([
0010
00100
HYpHYpP
HYpHYpP
Otherwise, choose 0 arbitrarily. Consider a Neyman-Pearson decision rule, NP~
, with 0 and 0)( y . For this
decision rule, the false alarm arte is,
)]|()|([)]|()|([}{)( 001000010~
0
~
HYpHYpPHYpHYpPEP NPNPF .
8/21/2019 LECTURE_02.pptx
15/17
ELEC6111: Detection and Estimation TheoryNeyman-Pearson Lemma (Proof)
.
2. See the text.
8/21/2019 LECTURE_02.pptx
16/17
ELEC6111: Detection and Estimation TheoryNeyman-Pearson Lemma (Example): Measurement with Gaussian Error
For this problem, we have,
),(1)()())([)]|()|([ 00010
QYPyLPHYpHYpP
where .2
)log( 10
01
2
Any value of can be achieved by choosing,
.)1()( 01
0
1
0
Q
Since 0)( 0 YP , the choice of 0 is arbitrary and we can choose 10 . So, we have
.0
1)(
0
0~
yif
yifyNP
8/21/2019 LECTURE_02.pptx
17/17
ELEC6111: Detection and Estimation TheoryNeyman-Pearson Lemma (Example): Measurement with Gaussian Error
The detection probability for NP
~
is dQQQQQYPYEP NPNPD
)()()()()}({)( 10111001
~
1
~