Generalized performance parameter for single-thresholddetection systems
Paul R. Prucnal
A new generalized performance parameter for single-threshold detection systems is investigated that pro-vides a more consistent measure of system performance than the conventional signal-to-noise ratio (SNR).It is shown explicitly that this parameter decreases monotonically as the Bayes risk and probability of errorincrease, and that it reduces to the SNR under appropriate conditions. An example is presented in whichthis parameter is monotonic with system performance, but the SNR is not. The analysis, which utilizes nor-malizing transformations, is also useful in the direct calculation of the Bayes risk, probability of error, andsensitivity, as is considered in detail subsequently (P. R. Prucnal, Applied Optics 19, 3606 (1980)).
I. Introduction
The performance of digital communications systemsthat detect the presence or absence of a signal embed-ded in noise can be characterized in a number of ways,including the probabilities of error, detection, and falsealarm, the sensitivity (detectability), and the traditionalsignal-to-noise ratio (SNR). The SNR, which is a ratioof signal power to noise power, has a variety of specificdefinitions,1-8 depending upon the particular applica-tion, and is easy to measure experimentally in terms ofpeak or root-mean-square values. It is well known thatthe traditional SNR,
SNR = (s)(varn)112
(1)
provides an adequate measure of performance for thecase of a single observation r of a constant signal withprobability density function pdf 6 (s - (s)) embed-ded in additive independent Gaussian noise with pdf- N(O, varn). In this case, the SNR varies monotoni-
cally with the probability of error e and is thereforesufficient to describe system performance. In general,however, the SNR might not vary monotonically withe and is not necessarily sufficient as the sole measure ofsystem performance.6 9"10 The inadequacy of the SNRis evident where the noise is non-Gaussian or signalfluctuations are important, and has been observed in
The author is with Columbia University, Columbia RadiationLaboratory, Department of Electrical Engineering, New York, NewYork 10027.
Received 13 June 1980.0003-6935/80/213606-05$00.50/0.
optical homodyne, 6 heterodyne,10 and direct-detection 5
systems.For example, consider the slightly more general case
of a single observation r of a Gaussian signal in additiveand independent Gaussian noise, with the various pdfsspecified by N((s), vars) and N(O, varn). Using thewell-known likelihood ratio test, and performing alge-braic manipulations, optimum processing is given by
r + r2 > (varn + vars)1/22(s)varn Ho
In 1 vars + v1/21XSNRH + I arn )1/2
2 SNRH(2)
where H, corresponds to the decision "signal is present,"and Ho corresponds to the decision "signal is notpresent," and where q is a constant. The quantity
SNRH (s)/(varn + vars)1/2 , (3)
has been used9 as an alternative to the SNR in this case,to account more adequately for random signal fluctu-ations. However, it is not immediately apparentwhether the performance is monotonic with SNRH, noris it apparent whether SNRH is alone sufficient to de-scribe system performance [the form of Eq. (2) suggeststhat this is not the case].
From the preceding discussion it is evident that itwould be useful to obtain a generalized parameter thatvaries monotonically with system performance. Theneed for this generalized parameter was also empha-sized by Jakeman et al.6 They observed that this pa-rameter would depend on signal statistics, and thereforecould not be unique. Their conclusion was that a fullanalysis must be performed before making statements
Here the constant Pj is the a priori probability of sourceoutput j, and the constant Cij is the cost of hypothesis
H, Hi given source output j. It is assumed that PF and PD
Ho ' may be varied by changing t. For the special case whereCo, = Cio = 1 and Coo = C1l = 0, !1 reduces to theprobability of error
Fig. 1. Digital communication system using single-thresholdprocessing.
about performance. The following sections develop indetail such a generalized parameter for system perfor-mance.
II. Calculation of Performance Using NormalizingTransforms
In this section an expression is derived for digitalsystem performance utilizing normalizing transforms.This technique, which involves a transformation ofrandom variables into the normal distribution, is dis-tinctly different from conventional Gaussian approxi-mation techniques.1 The expression we obtain is alsouseful in direct calculation of digital system perfor-mance in situations where closed form expressions forthe signal and noise probability density functions arenot available.12 Furthermore, expressing digital systemperformance in terms of normalizing transforms willlead to a new generalized parameter that describessystem performance, which we will derive in Sec. III.
The general classical binary detection problem il-lustrated in Fig. 1 will be considered here. The sourcegenerates a one or a zero, which results in a randomprocess at the input of the receiver. Signal-indepen-dent noise is added at the receiver input. A single ob-servation r is made of the input random process, whichis then processed, and a hypothesis Hi, i E {0,11 isformed as to which source output was generated.
For the Bayes or Neyman-Pearson decision criteria,optimum processing of r involves the well-known like-lihood ratio test1 For a broad class of problems wherethe signal is nonnegative (or nonpositive), the likelihoodratio test reduces to the single-threshold processor
Hir t,
Ho
where t is a fixed threshold. According to a recentsult by Prucnal and Teich,13 ' 14 Eq. (4) is an optimtest provided the logarithm of the noise probabildensity function (pdf) does not have a point of infltion.
The probabilities of detection and false alarm for Isingle-threshold processor in Fig. 1 are
PD = f p(rI1)dr,
PF= f p(r0)dr,
where the integrals are replaced by sums if r is ccrete-valued. The receiver performance is charactized by the Bayes risk
(4)
E=POPF+P(l -PD). (8)
In the analysis that follows we utilize two knownfunctions g and h that transform the pdfs p (r I 0) andp (r I 1) into the Gaussian (normal) density functionsN((M), var) and N(Q(), vart). The functions g and hare known exactly in the normal, chi-squared, noncen-tral chi-squared, and lognormal cases, 5 1 6 and a numberof approximate forms of these transformations havebeen developed for specific probability densities, suchas the Poisson, binomial, and negative-binomial.17 -21
Furthermore, g and h can be derived easily for any pdffor which the variance can be expressed as a function ofthe mean.22 A complete discussion of the normalizingtransformations g and h is presented elsewhere.15
The functions g and h are monotonic nondecreasing.This can be shown by letting the value of the outcomeP increase, for which Pr(t < g(P)) = Pr(r < P) is nonde-creasing, implying thatg(Q') is nondecreasing. A similarargument can be made for the monotonicity of h. Usingthe monotonicity of h and g, Eqs. (5) and (6) are trans-formed into
PD = X exp[-( - ))2/2 vartI dS,Jh(t) V2ir var
PF= s exp[-( - ())2/2 varfi d g.g(t) \2irvar~
(9)
(10)
With a simple change of variables, Eqs. (9) and (10)become
PD = 1/2 erfctD,
PF = 1/2 erfctF,
where
h(t) -()(2 var)1/2'
t _ g(t) - ( )(2 var¢)1/2
Substituting Eqs. (11)-(14) intoyields
l = CooPo - 1 C [OPo erfc [g - (g(r))l2 t[2 varg(r)]1/2j
(11)
(12)
(13)
(14)
Eqs. (7) and (8)
the + - C10Po erfc (t) - g(r))12 142 varg(r)] 1 2
J
+ 1 C erf h(t) - (h(r))(5) 2 142 varh(r)I"1 2J
+ op .. ChP ef h(t - (h (r) ~(6) + colP-2 (P, erfc 1[2 varh(r)1I"'
[is- E e=P o+ erfeg(t) - (g(r))I Pi f h(t) - (h(r) 2 142 varg(r)]'1 2 J 2 1[2 varh(r)]112 J
(15)
(16)
In cases where p (r I 1) and p (r I 0) cannot be expressed
in closed form, Eqs. (15) and (16) provide a more con-venient solution of ? and E than Eqs. (7) and (8), forwhich PD and PF must be calculated directly. In suchcases, g and h can be obtained from the moment gen-erating functions of p (r l 0) and p (r l 1).12,15 Further-more, Eqs. (15) and (16) will be useful in the derivationof a generalized performance parameter in Sec. III.
Ill. New Generalized Performance ParameterTo find a parameter a that increases monotonically
with system performance, the conditionoR(afl
,3 <0, (17)ba ,5
must be satisfied. The linearly independent coordinatevectors a and are functions of PF and PD, and will bediscussed subsequently. Differentiating Eq. (15) withrespect to a, and substituting the result into Eq. (17)yields a general criterion for monotonicity of systemperformance with a
(C1 CGO)POexp(-tF) (tF
+ (Col-Cl1p1 exp(-t) tD| < 0. (18)
It is assumed that the costs Cij and a priori probabilitiesPo and P1 are constant. Note that Eq. (18) is not nec-essarily satisfied for a = SNR, indicating that in generalsystem performance is not necessarily monotonic withSNR, as discussed earlier.
The coordinates a and : may be any functions of tDand t that are linearly independent and satisfy Eq.(18). We will specifically consider the case where a and/ have the form
a = d/ (tF - ). (19)
= C3tF + C4tD, (20)
to insure that a reduces to the conventional SNR underthe appropriate conditions. Linear independence re-quires that the constants 3 5s -C 4 . Solving Eqs. (19)and (20) for t and tD, differentiating with respect toa, and substituting the result into Eq. (18), yields thecondition for monotonicity of system performance witha
c3 - Po(Cio - Coo) exp( - t). (21)C4 P(C 0 - C) D F
Therefore a increases monotonically with system per-formance and is linearly independent of /3 provided (21)is satisfied and C3 /C4 i -1. Both conditions are satis-fied if 3 and 4 are nonnegative, although this is not anecessary condition.
Finally, using Eqs. (13), (14), (19), and (20), we findthat the parameter
(22)
increases monotonically with system performance andis linearly independent of
Equation (22) represents a new generalized param-eter a that increases monotonically with system per-formance for single-threshold detection systems. Asindicated prior to Eq. (19), a could have been chosen tobe any function of tF and D that satisfies both Eq. (18)and the condition of linear independence. The casewhere a is a linear combination of t and tD has beenspecifically addressed to insure that a reduces to theconventional SNR under the appropriate conditions,as follows. Consider again the case of a single obser-vation of a fixed signal in additive independentGaussian noise, for which single-threshold detection isoptimum. In this case g and h are identity transfor-mations, and the generalized performance parameterin Eq. (22) reduces to
t - 0 o _ e S/ = t(/(varn)1/2(varn)1 ' (varn)1= v (24)
which is the conventional SNR given in Eq. (1), as de-sired.
Finally, we note that the parameter /3 given in Eq.(23) represents the constant of partial differentiationin Eq. (17). Therefore, if a is to change monotonicallywith.Y?, /3 must remain constant. Continuing the pre-vious example of a fixed signal in Gaussian noise, Eq.(23) reduces to
c3(t -0) +c4 (t - (s))(2 varn) 1
/2 (2 varn)1/ 2 '(25)
where C3 and C4 are nonnegative and not both equal tozero. Substituting Eq. (1) in Eq. (25) we find
0 = C4 [(1 + C31 t ]-SN
Since must remain constant if a is to change mono-tonically with , Eq. (26) indicates that t must be ad-justed accordingly.
IV. Comparison of a and SNR
The case of a Gaussian signal in additive and inde-pendent Gaussian noise was presented in Sec. III as anexample for which the SNR is not sufficient to describesystem performance. The generalized performanceparameter a will be discussed in detail for this case, andcompared with the SNR.
A single observation is made of a Gaussian signal withPS(s) N((s), vars) embedded in additive indepen-dent Gaussian noise with PN(n) N(0,1). In this caseg and h are identity transformations, and Eqs. (22) and(23) yield
(26)
a = t + (s) t(1 + vars)112 '
A c3t + c4 (t - (s))/2 (2 + 2 vars)1/2
(27)
(28)
The values of 3 and 4 may be chosen arbitrarily pro-vided they are nonnegative and not both equal to zero.Since system performance increases monotonically witha while the constant of partial differentiation remainsfixed, the choice of 3 = X and 4 = 0, for example,corresponds to fixing t (i.e., the Neyman-Pearson test).
The parameters e, a, and SNR in Eqs. (31), (29), and(32) will now be compared as (s) increases. Since varsis an independent parameter, it may be specified arbi-trarily. Rather than setting vars equal to a constant,we will choose a particular relationship between varsand (s) that dramatically contrasts a with the SNR.We choose the relationship between vars and (s) tobe
to demonstrate an example where a is monotonic withE, but the SNR is not. It is shown in the Appendix thatsingle-threshold processing is approximately satisfiedwhen vars is specified by Eq. (33).
Figure 2 represents a linear plot of the probability oferror in Eq. (31) vs the SNR (dashed curve) and thegeneralized performance parameter a (solid curve), forthe above described case. The parameter a is presentedin Eq. (29) and is plotted on the lower scale. The SNRis presented in Eq. (32) and is plotted on the upper scale.From the solid curve we see that the probability of errordecreases (i.e., the performance increases) monotoni-cally as the generalized performance parameter a in-creases, indicating that a is indeed sufficient to describesystem performance. In contrast, the dashed curveshows that the probability of error (and therefore theperformance) is not monotonic with SNR, indicatingthat the SNR is not sufficient to describe system per-formance in this case. In fact, E increases with SNR for(s) > 3, which is precisely the opposite of what is ex-pected. Thus, even in the relatively simple case of aGaussian signal in Gaussian noise, the generalizedperformance parameter a provides a more consistentmeasure of system performance than the conventionalSNR.
For completeness, we note that the form of a for theslightly more general case, where PN(n) - N((n), varn)and ps(s) '-N((s), vars) is
a=t (n) _ -(s) (n) (4(varn)1/2 (varn + vars)1/2
where the threshold t is specified by the false alarm rate[see Eq. (6)]. This is somewhat different from the pa-rameter SNRH, which was proposed in Eq. (3) as analternative to the SNR for this case. As noted earlier,normalizing transforms can also provide a more con-
venient solution for system performance, as is consid-ered in detail subsequently.12
It is a pleasure to thank M. C. Teich for valuablesuggestions.
This work was supported by the Joint ServicesElectronics Program (U.S. Army, U.S. Navy, and U.S.Air Force) under contract DAAG29-79-C-0079.
Appendix
For the example presented in Sec. IV, the signal is notnonnegative, so that according to the results of Prucnaland Teich1 314 single-threshold processing is not nec-essarily optimum. Using the likelihood ratio test di-rectly leads to Eq. (2) with varn = 1. Solving thisquadratic equation for the threshold t yields two realsolutions of the form
t ) (s) f(S ()\2+(S) 2
vars Lars vars
+ ' + 1) ln(l + vars)j (Al)LrsI
where the subscript H(L) corresponds to the plus(minus) sign, proving that single-threshold processingis indeed not optimum in this case. However, we have
w
0W 4t'j .46I-0I-
CD4 44co0a.T
SIGNAL-TO-NOISE RATIO4 5
I I I I
-1.6 -1.2 -0.8 -OA 0 0.4 0.8 1.2PERFORMANCE PARAMETER a
Fig. 2. Linear plot of probability of error in Eq. (31) vs the SNR(dashed curve) and the generalized performance parameter a (solidcurve) for the case of a single observation of a Gaussian signal in ad-ditive, independent, standard Gaussian noise. The relationshipbetween vars and (s) is specified by Eq. (33). Maximum likelihooddetection is assumed, with PF = 0.0228. The parameter a is givenin Eq. (29) and is plotted on the lower scale. The SNR = (s), ac-
cording to Eq. (32), and is plotted on the upper scale.
specified the relationship between vars and (s) in a way[see Eq. (33)] that forces ps to be so narrow that thelower intersection of PN and PS*PN is far out on the tailof the density function. In this way, the probability
Pr(r < tL I ) = f p(rlj) dr j E 0,11,f_
(A2)
is made arbitrarily small, so that the lower threshold tLcan be ignored. From the geometry of this case, it isapparent that j = 1 yields the higher probability in Eq.(A2). Substituting for p(rI 1) in Eq. (A2), and using anasymptotic expansion for the error function (ref. 23,formula 7.1.23), yields
Using Eq. (33) it is found that the probability in Eq.(A3) is <5 X 10-8 over the region of (s) we will dealwith, which is negligible compared to the false alarm anddetection probabilities that will be encountered in thisexample. The lower threshold tL can then be ignoredand single-threshold processing is approximately op-timum when the variance is specified by Eq. (33).
References1. H. L. Van Trees, Detection, Estimation and Modulation Theory,
Part 1 (Wiley, New York, 1968).2. C. W. Helstrom, Statistical Theory of Signal Detection (Perga-
mon, New York, 1960).3. R. S. Kennedy, Fading Dispersive Communication Channels
(Wiley, New York, 1969).4. R. M. Gagliardi, Introduction to Communication Engineering
(Wiley, New York, 1978).5. E. V. Hoversten, "Optical communication theory," in Laser
Handbook, F. T. Arecchi and E. 0. Schulz-Du Bois, Eds.(North-Holland, Amsterdam, 1972), p. 1839.
6. E. Jakeman, C. J. Oliver, and E. R. Pike, "Optical homodynedetection," Advances in Physics (Taylor and Francis, London,1975), vol. 24, pp. 349-404.
7. S. D. Personick, Proc. IEEE 65, 1670 (1977).8. A. Papoulis, Probability, Random Variables, and Stochastic
Processes (McGraw-Hill, New York, 1965).9. M. Elbaum and M. C. Teich, Opt. Commun. 27, 257 (1978).
10. M. Elbaum and P. Diament, Appl. Opt. 15, 2268 (1976).11. S. D. Personick, P. Balaban, J. H. Bobsin, and P. R. Kumar, IEEE
Trans. Commun. COM-25, 541 (1977).12. P. R. Prucnal, Appl. Opt. 19, 3611 (1980).13. P. R. Prucnal and M. C. Teich, Appl. Opt. 17, 3576 (1978).14. P. R. Prucnal and M. C. Teich, IEEE Trans. Inf. Theory IT-25,
213 (1979).15. P. R. Prucnal and M. C. Teich, J. Math. Psychol. 21, 168
(1980).16. B. Saleh, Photoelectron Statistics (Springer, New York, 1978).17. M. S. Bartlett, J. R. Stat. Soc. Suppl. 3, 68 (1936).18. W. G. Cochran, Ann. Math. Stat. 9, 335 (1940).19. J. H. Curtiss, Ann. Math. Stat. 14, 107 (1943).20. F. J. Anscombe, Biometrika 35, 246 (1948).21. M. F. Freeman and J. W. Tukey, -Ann. Math. Stat. 21, 607
(1950).22. M. G. Kendall and A. Stuart, The Advanced Theory of Statistics
(Hafner, New York, 1966), p. 88.23. M. Abramowitz and I. A. Stegun, Eds., Handbook of Mathe-
12-23 Optical Science & Eng. course, Tucson P. Slater, OpticalSystems & Eng. Short Courses, Inc., P.O. Box 18667,Tucson, Ariz. 85731
19-22 APS Mtg., N.Y. W. W. Havens, Jr., 335 E. 45 St., N. Y.,N.Y 1001 7
22-27 ASP-ACSM Ann. Mtg., Wash., D.C. ASP, A. Cokenias,ASP Tech. Program Comm., U.S. Geological Survey,524 Natl. Ctr., Reston, Va. 22092 ACSM, C. Moore,ACSM Tech. Program Comm., 9606 Duffer Way,Gaithersburg, Md. 20760
February
1-6 1981 IEEE Power Engineering Soc., Atlanta R. Josephs,Sperry Univac, P.O. Box 500, Blue Bell, Pa. 19424
9-13 SPIE Los Angeles Tech. Symp., N. Hollywood Los An-geles Tech. Symp. 1981, SPIE Tech. Programs Comm.,P.O. Box 10, Bellingham, Wash. 98225
March
2-4 Measurement of Electrical Quantities in Pulse PowerSystems, Boulder S. Kelley, 220/Rm. B 344, NBS,Wash., D.C. 20234
9-13 Pittsburgh Conference on Analytical Chemistry andApplied Spectroscopy, Atlantic City E. S. Hodge, 440Fifth Ave., Pittsburgh, Pa. 15213
16-20 APS Mtg., Phoenix W. W. Havens, Jr., 335 E. 45 St.,N.Y., N.Y. 10017
25-27 Progress in Plasma Processes for Film Preparation &Etching at Low Pressures in Ionized & Activated Gasescourse, U. Sussex, Brighton Conf. & Courses Unit,Sira Inst., Ltd., South Hill, Chislehurst, Kent BR75EH, England
27-30 APS Mtg., Phoenix W. W. Havens, Jr., 335 E. 45 St.,N.Y., N.Y. 10017
29-3 Apr. 181st ACS Natl. Mtg., Atlanta A. T. Winstead, 1155 16thSt. N. W., Wash., D.C. 20036
30-31 SPIE Semiconductor Microlithography VI Conf., SanJose Program Comm., Micro VI, SPIE, P.O. Box 10,Bellingham, Wash. 98225
April
1-2 SPIE Optical Characterization Techniques for Semi-conductor Technology Conf., San Jose ProgramComm., OCTST, SPIE, P.O. Box 10, Bellingham,Wash. 98225
6-9 ISA/81 Industry Oriented Conf. & Exhibit, St. LouisISA/81 St. Louis, Instrument Soc. Am., 67 AlexanderDr., P.O. Box 12277, Research Triangle Park, N.C.27709
6-10 Optical Production Technology course, Cranfield, En-gland Registrar, Cranfield Inst. of Tech., Cranfield,Bedford MK43 OAI, England
