3172 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, … web... · Theory of the Stochastic...

3172 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 7, JULY 2007

Theory of the Stochastic Resonance Effect inSignal Detection: Part I—Fixed Detectors

Hao Chen, Student Member, IEEE, Pramod K. Varshney, Fellow, IEEE, Steven M. Kay, Fellow, IEEE, andJames H. Michels, Fellow, IEEE

Abstract—This paper develops the mathematical framework toanalyze the stochastic resonance (SR) effect in binary hypothesistesting problems. The mechanism for SR noise enhanced signal de-tection is explored. The detection performance of a noise modifieddetector is derived in terms of the probability of detection D andthe probability of false alarm FA. Furthermore, sufficient condi-tions are established to determine the improvability of a fixed de-tector using SR. The form of the optimal noise pdf is determinedand the optimal stochastic resonance noise pdf which renders themaximum D without increasing FA is derived. Finally, an illus-trative example is presented where performance comparisons aremade between detectors where the optimal stochastic resonancenoise, as well as Gaussian, uniform, and optimal symmetric noisesare applied to enhance detection performance.

Index Terms—Hypothesis testing, non-Gaussian noise, nonlinearsystems, signal detection, stochastic resonance (SR).

I. INTRODUCTION

STOCHASTIC RESONANCE (SR) is a nonlinear physicalphenomenon in which the output signals of some nonlinear

systems can be enhanced by adding suitable noise under certainconditions. Since its discovery by Benzi et al. in 1981 [1], theSR effect has been observed and applied in numerous nonlinearsystems [2]. The classic SR signature is the signal-to-noise ratio(SNR) gain of certain nonlinear systems, i.e., the output SNR ishigher than the input SNR when an appropriate amount of noiseis added [3]–[17]. Some approaches have been proposed to tunethe SR system by maximizing SNR. It has been shown that theSNR of a summing network of excitable units is optimum at acertain level of noise [3]. Later, for some SR systems, robustnessenhancement using non-Gaussian noises was reported by Castro

Manuscript received December 14, 2005; revised July 24, 2006. The asso-ciate editor coordinating the review of this manuscript and approving it forpublication was Prof. L. Collins. This work was supported by AFOSR undercontract FA9550-05-C-0139. This paper was presented in part at the 2006 In-ternational Conference on Acoustics, Speech, and Signal Processing, Toulouse,France, May 2006.

H. Chen and P. K. Varshney are with the Department of EECS, Syracuse Uni-versity, Syracuse, NY 13244 USA (e-mail: [email protected]; [email protected]).

S. M. Kay is with the Department of Electrical and Computer Engineering,University of Rhode Island, Kingston, RI 02881 USA (e-mail: [email protected]).

J. H. Michels is with the JHM Technologies, Ithaca, NY 14852 USA (e-mail:[email protected]).

Color versions of one or more of the figures in this paper available online athttp://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TSP.2007.893757

et al. [10]. For a fixed type of noise, Mitaim and Kosko [18]proposed an adaptive stochastic learning scheme performing astochastic gradient ascent on the SNR to determine the optimalnoise level based on the samples from the process. Rather thanadjusting the input noise level, Xu et al. [19] proposed a numer-ical method for realizing SR by tuning system parameters tomaximize SNR gain. Although SNR is a very important mea-sure of system performance, SNR gain based SR approacheshave several limitations. First, the definition of SNR is not uni-form and it varies from one application to another. Second, tooptimize the performance, the complete a priori knowledge ofthe signal is required. Finally, for detection problems where thenoise is non-Gaussian, SNR is not always directly related to de-tection performance; i.e., optimizing output SNR does not guar-antee optimizing probability of detection.

SR was also found to enhance the mutual information (MI)between input and output signals [20]–[25]. Similar to the SNRscenario, for a specified type of SR noise, Mitaim and Kosko[25] showed that almost all noise probability density functionsproduce some SR effect in threshold neurons and a new statisti-cally robust learning law was proposed to find the optimal noiselevel. McDonnell et al. [26] pointed out that the capacity of aSR channel can not exceed the actual capacity at the input. Com-pared to SNR, MI is more directly correlated with the transferredinput signal information.

In signal detection theory, SR also plays a very importantrole in improving the signal detectability. In [27] and [16], im-provement of detection performance of a weak sinusoid signalis reported. To detect a dc signal in a Gaussian mixture noisebackground, Kay [28] showed that under certain conditions, per-formance of the sign detector can be enhanced by adding somewhite Gaussian noise. For another suboptimal detector, the lo-cally optimal detector (LOD), Zozor and Amblard [17] pointedout that detection performance is optimum when the noise pa-rameters and detector parameters are matched. A study of thestochastic resonance phenomenon in quantizers conducted bySaha and Anand showed that a better detection performancecan be achieved by a proper choice of the quantizer thresholds[29]. Recently, Rousseau and Blondeau [30] pointed out thatthe detection performance can be further improved by usingan optimal detector on the output signal. Despite the progressachieved by the above approaches, the study of the SR effectin signal detection systems is rather limited and does not fullyconsider the underlying theory. In this paper, we explore the un-derlying mechanism of the SR phenomenon for a more generaltwo hypotheses detection problem which can be formulated asfollows.

1053-587X/$25.00 © 2007 IEEE

CHEN et al.: STOCHASTIC RESONANCE EFFECT IN SIGNAL DETECTION 3173

Consider a two hypotheses detection problem where given andimensional data vector , we have to decide between

two hypotheses or

(1)

where and are the pdfs of under and ,respectively. In order to make a decision, a test (random or non-random) is needed to choose between the two hypotheses. Thistest can be completely characterized by a critical function (de-cision function) [31] where for all . For anyobservation , this test chooses the hypothesis with proba-bility . In many cases, can be implicitly expressed byusing a test statistic which is a function of and a threshold

such that

(2)

where its corresponding critical function is

(3)

where is a suitable number. The probability ofdetection is given by

(4)

and the probability of false alarm is given by

(5)

where the superscripts on and in (4) and (5) indicatethat the test in (2) is employed for the data vector . Althoughthe critical function and the test statistic can takeany form, we know that the optimum Neyman-Pearsondetector involves a likelihood ratio test (LRT) where

. Although a Neyman-Pearsondetector is optimum in the sense of maximizing given afixed , the associated LRT requires the complete knowledgeof the pdfs and which is not always available in apractical application. Also, the input data statistics may varywith time or may change from one application to another.To make matters worse, there are many detection problemswhere the exact form of the LRT is too complicated to beimplemented. Therefore, simpler and more robust suboptimaldetectors are used in numerous applications [32]. To improve asuboptimal detector detection performance, two approaches arewidely used. In the first approach, the detector parameters arevaried [15]–[17], [29], [33]. Alternatively, when the detectoritself cannot be altered or the optimum parameter values aredifficult to obtain, adjusting the observed data becomes a viableapproach. It is well known that the detection performance canbe improved by adding additional noise that is statistically de-pendent on the existing noise and/or with pdf that depends uponwhich hypothesis is true [28]. However, adding a dependent

noise is not always possible because pertinent prior informationis usually not available. Therefore, in this paper, we constrainthe additive noise to be independent noise. For some suboptimaldetectors, as Kay pointed out in [28], detection performancecan be improved by adding an independent noise to the dataunder certain conditions. For a given type of SR noise, theoptimal amount of noise can be determined that maximizes thedetection performance for a given suboptimal detector [34].In an effort to explain this noise enhanced phenomenon, forsome integrate-and-fire neuron models, Tougaard demonstratedthat the detection performance gain is caused by the nonlinearproperties of the spike-generation process itself [35]. However,despite the progress made in the literature, the underlyingmechanism of this Stochastic Resonance phenomenon in de-tection problems has not fully been explored. For example, aninteresting question is to determine the best ‘noise’ to be addedin order to achieve the best achievable detection performancefor the suboptimal detector and this question remains unsolved.In this case, the detection problem can be stated as: Given thatthe test is fixed; i.e., the critical function (for example,

and ) is fixed, can we improve the detection performanceby adding SR noise? If the answer is yes, what kind of noiseand how much noise (i.e., noise pdf) should we add to theobserved data to maximize without increasing ? Inthis paper, a theoretical analysis is presented to gain furtherinsight into the SR phenomenon and the detection performanceof the noise modified observations is obtained. Furthermore,the optimum noise pdf, i.e., not only the noise level but alsothe noise type is determined. As an illustrative example, theoptimum noise pdf and some suboptimal noise pdfs for the signdetector are derived. Compared to the earlier definitions of SR[1], [2], we further extend the concept of “SR” to a pure noiseenhanced phenomenon, i.e., a phenomenon of some nonlinearsystems in which the system performance is enhanced due tothe addition of an independent noise at the input. In this paper,the terminologies “SR” and “noise enhanced” are used inter-changeably. However, we point out that the latter is actually thegeneralization of the former.

The paper is organized as follows. In Section II, we formulatethe noise modified detection problem and the conditions for thebest SR noise pdf are derived. The exact form of the optimumSR noise pdf is derived in Section III. An illustrative example ispresented in Section IV. Conclusions and further comments aregiven in Section V.

II. PROBLEM FORMULATION

In order to study a possible enhancement of the detection per-formance, we add noise to the original data process and obtaina new data process given by

(6)

where is either an independent random process with pdfor a nonrandom signal. Notice that here we do not have anyconstraint on . For example, can be white noise, colorednoise, or even be a deterministic signal , corresponding to

. As will be shown later, depending on the de-tection problem, an improvement of detection performance may


not always be possible. In that case, the optimal noise is equal tozero. The pdf of is expressed by the convolutions of the pdfssuch that

(7)

The binary hypotheses testing problem for this new observeddata can be expressed as

(8)

Since the detector is fixed, i.e., the critical function of is thesame as that for , the probability of detection based on datais given by

(9)

where

(10)

Alternatively

(11)

Similarly, we have

(12)

(13)

where

(14)

corresponding to hypothesis . , are the expectedvalues based on distributions and , respectively, and

, . To simplify notation, we omit the sub-script of and and denote them as , , and , re-spectively. Further, from (14), and are actuallythe probability of detection and probability of false alarm, re-spectively, for this detection scheme with input .For example, is the of this detection scheme with

Fig. 1. An example of the relationship between P , P , P , P and p .The optimum noise pdf p (n) = �(n + A).

input . Therefore, it is very convenient for us to obtain theand values by analytical computation if , and are

known. When they are not available, and can be obtainedfrom the data itself by processing it through the detector andrecording the detection performance.1 From (11) and (13), wemay formalize the optimal SR noise definition as follows.

Consider the two hypotheses detection problem as in (1). Thepdf of optimum SR noise is given by

(15)

where1) , ;2) ;3) .Conditions 1) and 2) are fundamental properties of a pdf func-

tion. Condition 3) ensures that , i.e., the con-straint specified under the Neyman-Pearson Criterion is satis-fied. Further, if the inequality of condition 3) becomes equality,the constant false alarm rate (CFAR) property of the original de-tector is maintained.

A simple illustration of the effect of additive noise is shown inFig. 1. In this example, and

, hence which means the optimal SRnoise is a dc signal with value . In practical ap-plications, some additional restrictions on the noise may alsobe applied. For example, the type of noise may be restricted,(e.g., may be specified as Gaussian noise), or we may requirea noise with even symmetric pdf to ensurethat the mean value of is equal to the mean value of . How-ever, regardless of the additional restrictions, the conditions 1),2), and 3) are always valid and the optimum noise pdf can bedetermined for these conditions.

III. OPTIMUM SR NOISE FOR NEYMAN-PEARSON DETECTION

In general, it is difficult to find the exact form of directlybecause of condition 3). However, an alternative approach con-siders the relationship between and . From (14), fora given value of , we have , where isthe inverse function of . When is a one-to-one mappingfunction, is a unique vector. Otherwise, is a set of

1Thus, it is not necessary to have complete knowledge regarding �(�) andp (�).


for which . Therefore, we can express a value or aset of values of as

(16)

Given the noise distribution of in the original do-main, , the noise distribution in the domain can alsobe uniquely determined. Further, the conditions on the optimumnoise can be rewritten in terms of equivalently as

4) ;5) ;6) ;

and

(17)

where is the SR noise pdf in the domain.Compared to the original conditions 1), 2), and 3), this equiv-

alent form has some advantages. First, the problem complexityis dramatically reduced. Instead of searching for an optimal so-lution in , we are now looking for an optimal solution in asingle dimensional space. Second, by applying these new con-ditions, we avoid the direct use of the underlying pdfs and

and replace them with and . Note that, in some cases,it is not very easy to find the exact form of and . How-ever, recall that and are the Probability of De-tection and Probability of False Alarm, respectively, of the orig-inal system with input . In practical applications, we maylearn the relationship by Monte Carlo simulation using impor-tance sampling. In general, compared to and , and aremuch easier to estimate and once the optimum is found,the optimum is determined, as well by the inverse of thefunctions and .

Let us now consider the function such thatis the maximum value of given . Clearly,

. From (17), it follows that for anynoise , we have

(18)

Therefore, the optimum is attained whenand .

A. Determination of the Improvability of Detection via SR

Improvability of the given detector when SR noise is addedcan be determined by computing and comparing and .When , the given detector is improvable by addingSR noise. However, it requires the complete knowledge ofand and significant computation. For a large class of de-tectors, however, depending on the specific properties of , wemay determine the sufficient conditions for improvability andnonimprovability more easily. These are given in the followingtheorems.

Theorem 1 (Improvability of Detection via SR): Ifor when is second-order continu-

ously differentiable around , then there exists at least one

noise process with pdf that can improve the detectionperformance.

Proof: First, when , from the definition offunction, we know that there exist at one least one such that

and , therefore, thedetection performance can be improved by choosing a SR noisepdf . When and is continuousaround , there exists an such that on

. Therefore, from Theorem A-1, is convexon .2 Let us add a noise with pdf

where and. Due to the convexity of ,

. Thus, detectionperformance can be improved via the addition of SR noise.

We will illustrate this result with an example in the nextsection.

Theorem 2 (Nonimprovability of Detection via SR): Ifthere exists a nondecreasing concave function where

and for every ,then for any independent noise, i.e., the detectionperformance cannot be improved by adding noise.

Proof: For any noise and corresponding , we have

(19)

The third inequality of the Right Hand Side (RHS) of (19) isobtained using the concavity of the function. The detectionperformance cannot be improved via the addition of SR noise.

Again, we will illustrate this result in the next section.

B. Determination of the Form of Optimum SR Noise PDF

Before determining the exact pdf of , we first present thefollowing result for the form of optimum SR noise.

Theorem 3 (Form of Optimum SR Noise): To maximize ,under the constraint that , the optimum noise can beexpressed as3

(20)

where . In other words, to obtain the maximumachievable detection performance given the false alarm con-straints, the optimum noise is a randomization of two discretevectors added with probability and , respectively.

Proof: Letbe the set of all pairs of . Since ,

is a subset of the linear space . Furthermore, let be theconvex hull of . Since , its dimension .

2Please refer to [36] or the Appendix for the related definitions and Theorems.3This form of optimum noise pdf is not necessarily unique. There may exist

other forms of noise pdf that achieve the same detection performance.


Similarly, let the set of all possible be . Sinceany convex combination of the elements of , say

can be obtained by setting the SR noise pdfsuch that , we have, .It can also be shown that . Otherwise, there would existat least one element such that , but . In thiscase, there exists a small set and a positive number suchthat and ', where

' denotes an empty set. However, since , bythe well known property of integration, there always exists a fi-nite set with finite elements such that and , aconvex combination of the elements of , such that

. Since , which con-tradicts the definition of . Therefore, . Hence, .From Theorem A-4, can be expressed as a convexcombination of three elements. Also, since we are only inter-ested in maximizing under the constraint that ,the optimum pair can only belong to , the set of the boundaryelements of . To show this, let be an arbitrary non-boundary point inside . We know that there exists asuch that . Therefore, is inadmissibleas an optimum pair. Thus, the optimum pair can only exist onthe boundary. Therefore, each on the boundary of can beexpressed as the convex combination of only two elements in

. Hence,

(21)

where , . Therefore, wehave

(22)

Equivalently, , whereand are determined by the equations

(23)

Alternatively, the optimum SR noise can also be expressed interms of , such that

(24)

From (22), we have

(25)

and

(26)

C. Determination of the pdf of Optimum SR Noise

Depending on the location of the maxima of , we havethe following theorem.

Theorem 4: Let and

. It follows thatCase 1) If , then and ,

i.e., the maximum achievable detection performance

is obtained when the optimum noise is a dc signalwith value , i.e.

(27)

where and .Case 2) If , then , i.e., the

inequality of (26) becomes equality. Furthermore

(28)

Proof: For Case 1, notice thatand . Therefore,

the optimum detection performance is obtained when the noiseis a dc signal with value with .

We use the contradiction method here to prove Case 2. First,let us suppose that the optimum detection performance is ob-tained when with noise pdf . Let

. It is easy to verify that is a valid pdf. Let. We now have

and

But, this contradicts (15), the definition of . Therefore,, i.e., the maximum achievable detection perfor-

mance is obtained when the probability of false alarm remainsthe same for the SR noise modified observation .

For Case 2 of Theorem 4, i.e., when ,4 let us con-sider the following construction to derive the form of the op-timum noise pdf. From Theorem 4, we have the condition that

is a constant. Let us define an auxil-iary function such that

(29)

where . We have. Hence, also maximizes

and vice versa. Therefore, under the conditionthat , maximization of is equivalent to maxi-mization of . Let us divide the domain of intotwo intervals and . Let be theminimum value that maximizes in and letbe the minimum value that maximizes in . Also, let

and be the correspondingmaximum values. Since for any , is monotonicallydecreasing when is increasing, and are mono-tonically decreasing while and are monotonicallynonincreasing when is increasing. Since , there-fore, , furthermore, when is very large,we have . Hence,there exists at least one such that .For illustration purposes, the plots of for the detec-tion problem discussed in Section IV are shown in Fig. 4. Let

4This case is usually true because, for a reasonable detector, a higher Pyields a higher P .


us divide the [0,1] interval into two nonoverlapping parts ,, such that

and . Next, represent as

(30)

where for and is zero otherwise (an indi-cator function). From (5), we must have

(31)

and

(32)

Note that for all . Clearly, the upper boundcan be attained when for all , i.e., .Therefore, . From (28),we have

(33)

Notice that by letting, (33) is equivalent to (22).

Equivalently, we have the expression of as

(34)

Further, in the special case where is continuously differ-entiable, is also continuously differentiable. Since and

are at least local maxima, we have. Therefore, from the derivative of (29),

we have

(35)

(36)

In other words, the line connecting andis the bitangent line of and is its

slope. Also,

(37)

In this section, we have derived the condition under whichSR noise can improve detection performance. Also, we haveobtained the specific form of the optimum SR noise. Next weillustrate the ideas by applying the theory to a specific detectionproblem.

IV. A DETECTION EXAMPLE

Here, we consider the same detection problem as consideredby Kay [28]. The two hypotheses and are given as

(38)

for , is a known dc signal, andare i.i.d noise samples with a symmetric Gaussian mixture noisepdf

(39)

where . Here,we set , and . A suboptimal detector isconsidered with test statistic

(40)

where . From (40), this detectoris essentially a fusion of the decision results of i.i.d. signdetectors.

When , the detection problem reduces to a problemwith the test statistic , threshold (sign detector)and the probability of false alarm . The distributionof under the and hypotheses can be expressed as

(41)

and

(42)

respectively. The critical function is given by

(43)

The problem of determining the optimal SR noise is to find theoptimal where for the new observation , theprobability of detection is maximum whilethe probability of false alarm

.When , the detector is equivalent to a fusion of in-

dividual detectors and the detection performance monotonicallyincreases with . Like the case, when the decision func-tion is fixed, the optimum SR noise can be obtained by a similarprocedure. Due to space limitations, here only the suboptimalcase where the additive noise is assumed to be an i.i.d noiseis considered. Under this constraint, since the s and s ofeach and every detector are the same, it can be shown that theoptimal noise for the case is the same as becauseagain, we need to fix for each individual detectorwhile increasing its . Hence, in the following discussion, weonly consider the one sample case . However, the per-formance of the case can be derived similarly.


Fig. 2. F (x) andF (x) as a function of x as given in (44) and (45) for � = 3,A = 1, and � = 1, respectively.

A. Determination of the Optimal SR Noise pdf

From (11) and (13), it can be shown that in this case

(44)

and

(45)

where . It is also easy to showthat in this case, and both are monotonicallyincreasing with . Therefore, , and

is a single curve. Fig. 2 shows the values of andas a function of while the relationship between and isshown in Fig. 3. , the convex hull of all possible andafter is added is shown as the light and dark shadowed regions,respectively, in Fig. 3. Note that a similar nonconcave ROC oc-curs in distributed detection systems [37] and dependent ran-domization is employed to improve system performance [38],[39].

Fig. 3. Relationship between F (x) and F (x) as derived from (44) and (45),respectively. The shadowed region [including both yellow (light gray) anddashed green (dashed dark gray)] is the convex hull V of U . The green dashedregion is the region of (f ; f ) where possible SR effect may take place.

Taking the derivative of w.r.t. , we have

(46)

and,

(47)

where . Since, we have and

(48)

Next, let us discuss the improvability of this detector. First,when , setting (48) equal to zero and solving the equationfor , we have , the zero pole of (48)

When , we have and inthis example, . From Theorem 1, thisdetector is improvable by adding independent SR noise. When

, , and the improvability


cannot be determined by Theorem 1. However, for this partic-ular detector, as we discuss later, the detection performance canstill be improved.

We now determine the two discrete values as well as the prob-ability of their occurrence by solving equations (35) and (36).From (44) and (45), the relationship between , and , and(46), we have

(49)

Although it is generally very difficult to solve the above equationanalytically, fortunately, in this particular detection problem, wehave

so that ,and

,given . Thus, the roots , of (49) canbe approximately expressed as and

. Correspondingly,and .

Hence

(50)

and

(51)

B. Optimal Symmetrical Noises

In this subsection, we consider the special cases where the SRnoise is constrained to be symmetric. These include symmetricnoise with arbitrary pdf , white Gaussian noise

and white uniform noise , ,. The noise modified data processes are

denoted as , and , respectively. Here, for illustration pur-poses, we find the pdfs of these suboptimal SR noises using the

functions. The same results can be obtained by applyingthe same approach as in the previous subsection using and

functions.

Fig. 4. An illustration of the relationship between G(f ; k), f , f (k), � (k)with i = 1, 2 and different k value 0,1 and 2.

For the arbitrary symmetrical noise case, we have thecondition

(52)

Therefore, is also a symmetric function, so that. By (43) and (52), we have

(53)

Since , we also have

(54)

(55)

From (9) and (53), we have the of given by

(56)


TABLE ICOMPARISON OF DETECTION PERFORMANCE FOR DIFFERENT SR NOISE ENHANCED DETECTORS

Fig. 5. Different H(x) curves where � = 3, A = 1.

where . Fig. 5 shows a plot offor several values. Finally, from (42), we have

When , since when , we have,, . From (56), for any , i.e.,

in this case, the detection performance of this detector cannotbe improved by adding symmetric noise. When and

we also have , . Therefore, addingsymmetric noise will not improve the detection performance aswell. However, when , has only a single rootfor and , , , anddetection performance can be improved by adding symmetricSR noise. From (56), we have

(57)

and

Furthermore, sinceand given

, we have . Therefore

(58)

The pdf of for the hypothesis becomes

(59)

Hence, when is large enough,. Note that, as decreases, increases,

i.e., better detection performance can be achieved by adding theoptimal symmetric noise.

Similarly, for the uniform noise case,

(60)

Substituting (60) for in (56) and taking the derivative w.r.t, we have

(61)

Setting it equal to zero and solving, we have inthe pdf of uniform noise defined earlier. Additionally, we have

.For the Gaussian case, the optimal WGN level is readily de-

termined since

(62)

Let . Taking the derivative w.r.t in (62), settingit equal to zero and solving, we obtain

(63)

and , and, correspondingly,. Therefore, when , adding WGN with variance

can improve the detection performance to a constant level.

C. Detection Performance Results

Table I shows the values of for these different typesof SR noise. Compared to the original data process with

, the improvement of different detectors are0.1811, 0.1593, 0.0897, and 0.0693 for optimum SR noise, op-timum symmetric noise, optimum uniform noise, and optimumGaussian noise enhanced detectors, respectively.

Fig. 6 shows as well as the maximum achievable withdifferent values of . The detection performance is significantlyimproved by adding optimal SR noise. When , a certaindegree of improvement is also observed by adding suboptimalSR noise. When is small, and , the detec-tion performance of the optimum SR noise enhanced detector isclose to the optimum symmetric noise enhanced one. However,when , the difference is significant. When ,

, , so that ,i.e, the optimal symmetric noise is zero (no SR noise). How-ever, by adding optimal SR noise, is still larger than

, i.e., the detection performance can still be improved. When


Fig. 6. P as a function of signal level A in Gaussian mixture noise when� = 3 and � = 1. “LRT” is the P obtained by applying the optimum LRTon the observed data x. “opt”, “opt Sym”, “opt Unif”, “opt WGN” and “NoSR” are the P of the optimum noise, optimum symmetric noise, optimumuniform noise, optimum white Gaussian noise and original data (no SR noise),respectively.

Fig. 7. P as a function of � for different types of noise enhanced detectorswhen � = 3 and A = 1.

, the improvement is not that significant becausewhich is already a very good detector.

The maximum achievable detection performance of differentSR noise enhanced detectors with different background noise

is shown in Fig. 7. When is small, for the optimum SRnoise enhanced detectors , while for the symmetricSR noise case . When increases, increasesand the detection performance of SR noise enhanced detectorsdegrades. When , becomes a unimodal noise andthe decision function is the same as the decision function de-cided by the optimum LRT test given the false alarm .Therefore, adding any SR noise will not improve . Hence, allthe detection results converge to .

Fig. 8 compares the detection performance of different detec-tors w.r.t. when and is fixed. , and

Fig. 8. P as a function of � for different types of noise enhanced detectorswhen � = 1 and A = 1.

monotonically decrease when increases. Also, thereexist a unique value , such that when is small,

is still a unimodal pdf, so that the decision function is theoptimum one for . An interesting observation fromFig. 8 is that the of the “optimum LRT”, after the lowestvalue is reached, increases when increases. The explanationof this phenomenon is that when is sufficiently large, the sep-aration of the two peaks of the Gaussian mixtures increases asincreases so that the detectability is increased. When ,the two peaks are sufficiently separated, so that the detectionperformance of “LRT” is equal to the when .

Finally, Fig. 9 shows the ROC curves for this detectionproblem when and the different types of i.i.d SRnoise determined previously are added. Different degrees ofimprovement are observed for different SR noises pdfs. Theoptimum SR detector and the optimum symmetric SR detectorperformance levels are superior to those of the uniform andGaussian SR detectors and more closely approximate the LRTcurve.

V. CONCLUDING REMARKS

In this paper, we have established the mathematical theoryfor the SR noise modified detection problem. Several funda-mental theorems on SR in detection theory are established. Weanalyzed the detection performance of a SR noise enhanced de-tector where, for any additive noise, the detection performancein terms of and can be obtained by applying the expres-sions we have developed. Based on that, we have established theconditions of potential improvement of via the SR effect.This leads to the sufficient condition for the improvability/non-improvability of most suboptimal detectors. The exact form ofthe optimal SR noise pdf has been proposed. The optimal SRnoise is shown to be a proper randomization of no more than twodiscrete signals. Also, the upper limit of the SR enhanced detec-tion performance is obtained. Given the distributions and ,a theoretical approach is proposed to determine the optimal SRconsisting of the two discrete signals and their correspondingweights.


Fig. 9. ROC curves for different SR noise enhanced sign detectors, N = 30.For “LRT”, its performance nearly perfect (P � 1 for all P ’s).

Fig. 10. An illustration of a SR detection system and the corresponding“Super” detector.

It should be pointed out that the results obtained in this paperare very general and are applicable to a variety of SR detec-tors considered in the literature, e.g., bistable systems. The SRdetectors presented in [22], [30], [33]–[35] can be included inour framework as shown in Fig. 10. For example, the nonlinearsystem block of Fig. 10 can depict the bistable system [33]–[35].Let be the input to the nonlinear system,and be the output of the system asshown, where is the appropriate nonlinear function.The decision problem based on can be described by decisionfunction as shown. It is easy to observe that the corre-sponding decision function for the “super” detector (non-linear system plus detector) is . Thus the SRdetectors proposed in the literature can be incorporated in ourframework and the theory developed in the paper is applicableto these general situations.

Based on our mathematical framework, for a particular detec-tion problem, we have compared the detection performance ofsix different detectors, namely, the optimum LRT detector, op-timum noise enhanced sign detector, optimum symmetric noiseenhanced sign detector, optimum uniform noise enhanced signdetector, optimum Gaussian noise enhanced sign detector andthe original sign detector. Compared to the traditional SR ap-proach where the noise type is predetermined, much better de-tection performance is obtained by adding the proposed op-timum SR noise to the observed data process.

This fundamental theory well explains the observed SR phe-nomenon in signal detection problems, and greatly advances our

ability to determine the applicability of SR in signal detection.It can also be applied to many other signal processing problemssuch as distributed detection and fusion as well as pattern recog-nition applications.

APPENDIX

REVIEW OF CONVEX FUNCTIONS AND CONVEX SETS [36]

In this section, we put together some background informationof convex functions and convex sets for reader’s convenience.More details are available in [36].

A. Convex Functions

A function is called convex if

(64)

for all and in the open interval (0,1). It is calledstrictly convex provided that the inequality (64) is strict for

. Similarly, if is convex, then we say thatis concave.Theorem A-1: Suppose exists on . Then is convex

if and only if . And if on , then isstrictly convex on the interval.

B. Convex Sets

Let be a subset of a linear space . We say that is convexif implies that for all

.Theorem A-2: A set is convex if and only if every

convex combination of points of lies in .We call the intersection of all convex sets containing a given

set the convex hull of , denoted by .Theorem A-3: For any , the convex hull of consists

precisely of all convex combinations of elements of .Furthermore, for the convex hull, we have Carathéodory’s

theorem for convex sets.Theorem A-4 (Carathéodory’s Theorem): If and its

convex hull has dimension , then for each ,there exist point of such that is a convexcombination of these points.

REFERENCES

[1] R. Benzi, A. Sutera, and A. Vulpiani, “The mechanism of stochasticresonance,” J. Phys. A: Math. General, vol. 14, pp. L453–L457, 1981.

[2] L. Gammaitoni, P. Hänggi, P. Jung, and F. Marchesoni, “Stochasticresonance,” Rev. Mod. Phys., vol. 70, no. 1, pp. 223–287, Jan. 1998.

[3] J. J. Collins, C. C. Chow, and T. T. Imhoff, “Stochastic resonancewithout tuning,” Nature, vol. 376, pp. 236–238, 1995.

[4] K. Loerincz, Z. Gingl, and L. Kiss, “A stochastic resonator is ableto greatly improve signal-to-noise ratio,” Phys. Lett. A, vol. 224, pp.63–67, 1996.

[5] S. M. Bezrukov and I. Vodyanoy, “Stochastic resonance in non-dy-namical systems without response thresholds,” Nature, vol. 385, pp.319–321, Jan. 1997.

[6] F. Chapeau-Blondeau, “Input-output gains for signal in noise in sto-chastic resonance,” Phys. Lett. A, vol. 232, pp. 41–48, 1997.

[7] F. Chapeau-Blondeau and X. Godivier, “Theory of stochastic reso-nance in signal transmission by static nonlinear systems,” Phys. Rev.E, vol. 55, no. 2, pp. 1478–1495, 1997.

[8] F. Chapeau-Blondeau, “Periodic and aperiodic stochastic resonancewith output signal-to-noise ratio exceeding that at the input,” Int. J.Bifurc. Chaos, vol. 9, pp. 267–272, 1999.


[9] P. Hänggi, M. E. Inchiosa, D. Fogliatti, and A. R. Bulsara, “Nonlinearstochastic resonance: The saga of anomalous output-input gain,” Phys.Rev. E, vol. 62, no. 5, pp. 6155–6163, 2000.

[10] F. J. Castro, M. N. Kuperman, M. Fuentes, and H. S. Wio, “Exper-imental evidence of stochastic resonance without tuning due to non-Gaussian noises,” Phys. Rev. E, vol. 64, no. 051105, 2001.

[11] Z. Gingl, P. Makra, and R. Vajtai, “High signal-to-noise ratio gain bystochastic resonance in a double well,” Fluctuat. Noise Lett., vol. 1, no.3, pp. L181–L188, 2001.

[12] G. P. Harmer, B. R. Davis, and D. Abbott, “A review of stochasticresonance: Circuits and measurement,” IEEE Trans. Instrum. Meas.,vol. 51, no. 2, pp. 299–309, Apr. 2002.

[13] P. Makra and Z. Gingl, “Signal-to-noise ratio gain in non-dynamicaland dynamical bistable stochastic resonators,” Fluctuat. Noise Lett.,vol. 2, no. 3, pp. L145–L153, 2002.

[14] P. Makra, Z. Gingl, and T. Fulei, “Signal-to-noise ratio gain in sto-chastic resonators driven by coloured noises,” Phys. Lett. A, vol. 317,pp. 228–232, 2003.

[15] S. Zozor and P.-O. Amblard, “Stochastic resonance in discrete timenonlinear ar(1) models,” IEEE Trans. Signal Process., vol. 47, pp.108–122, Jan. 1999.

[16] S. Zozor and P.-O. Amblard, “On the use of stochastic resonance insine detection,” Signal Process., vol. 7, pp. 353–367, Mar. 2002.

[17] S. Zozor and P.-O. Amblard, “Stochastic resonance in locally op-timal detectors,” IEEE Trans. Signal Process., vol. 51, no. 12, pp.3177–3181, Dec. 2003.

[18] S. Mitaim and B. Kosko, “Adaptive stochastic resonance,” Proc. IEEE,vol. 86, pp. 2152–2183, 1998.

[19] B. Xu, J. Li, and J. Zheng, “How to tune the system parameters torealize stochastic resonance,” J. Phys. A: Math. General, vol. 36, pp.11 969–11 980, 2003.

[20] X. Godivier and F. Chapeau-Blondeau, “Stochastic resonance in theinformation capacity of a nonlinear dynamic system,” Int. J. Bifurc.Chaos, vol. 8, no. 3, pp. 581–589, 1998.

[21] I. Goychuk and P. Hänggi, “Stochastic resonance in ion channelscharacterized by information theory,” Phys. Rev. E, vol. 61, no. 4, pp.4272–4280, 2000.

[22] N. G. Stocks, “Suprathreshold stochastic resonance in multilevelthreshold systems,” Phys. Rev. Lett., vol. 84, no. 11, pp. 2310–2313,Mar. 2000.

[23] B. Kosko and S. Mitaim, “Stochastic resonance in noisy threshold neu-rons,” Neural Netw., vol. 16, pp. 755–761, 2003.

[24] B. Kosko and S. Mitaim, “Robust stochastic resonance for simplethreshold neurons,” Phys. Rev. E, vol. 70, no. 031911, 2004.

[25] S. Mitaim and B. Kosko, “Adaptive stochastic resonance in noisy neu-rons based on mutual information,” IEEE Trans. Neural Netw., vol. 15,no. 6, pp. 1526–1540, Nov. 2004.

[26] M. McDonnell, N. Stocks, C. Pearce, and D. Abbott, “Stochastic reso-nance and data processing inequality,” Electron. Lett., vol. 39, no. 17,pp. 1287–1288, 2003.

[27] A. Asdi and A. Tewfik, “Detection of weak signals using adaptive sto-chastic resonance,” in Proc. Int. Conf. Acoust., Speech, Signal Process.(ICASSP 95), May 1995, vol. 2, pp. 1332–1335.

[28] S. Kay, “Can detectability be improved by adding noise?,” IEEE SignalProcess. Lett., vol. 7, no. 1, pp. 8–10, Jan. 2000.

[29] A. A. Saha and G. Anand, “Design of detectors based on stochasticresonance,” Signal Process., vol. 83, pp. 1193–1212, 2003.

[30] D. Rousseau and F. Chapeau-Blondeau, “Stochastic resonance andimprovement by noise in optimal detection strategies,” Digit. SignalProcess., vol. 15, pp. 19–32, 2005.

[31] E. L. Lehmann, Testing Statistical Hypotheses, 2nd ed. New York:Springer, 1986.

[32] J. Thomas, “Nonparametric detection,” Proc. IEEE, vol. 58, no. 5, pp.623–631, May 1970.

[33] V. Galdi, V. Pierro, and I. M. Pinto, “Evaluation of stochastic-res-onance-based detectors of weak harmonic signals in additive whiteGaussian noise,” Phys. Rev. E, vol. 57, no. 6, pp. 6470–6479, Jun. 1998.

[34] M. E. Inchiosa and A. R. Bulsara, “Signal detection statistics of sto-chastic resonators,” Phys. Rev. E, vol. 53, no. 3, pp. R2021–R2024,Mar. 1996.

[35] J. Tougaard, “Signal detection, detectability and stochastic resonanceeffects,” Biolog. Cybern., vol. 87, pp. 79–90, 2002.

[36] A. W. Roberts and D. E. Varberg, Convex Functions. New York: Aca-demic, 1973.

[37] P. K. Varshney, Distributed Detection and Data Fusion. New York:Springer, 1997.

[38] P. Willet and D. Warren, “The suboptimality of randomized tests indistributed and quantized detection systems,” IEEE Trans. Inf. Theory,vol. 38, no. 2, pp. 355–361, 1992.

[39] F. Gini, F. Lombardini, and L. Verrazzani, “Decentralised detectionstrategies under communication constraints,” Inst. Elect. Eng. Proc.Radar, Sonar and Navigat., vol. 145, no. 4, pp. 199–208, Aug. 1998.

Hao Chen (S’06) received the B.S. and M.S. degreesin electrical engineering from the University ofScience and Technology of China (USTC), Hefei, in1999 and 2002, respectively.

Since 2002, he has been pursuing the Ph.D. degreewith the Department of Electrical Engineering andComputer Science, Syracuse University, Syracuse,NY. His research interests are in the areas of statis-tical signal processing and its applications, includingdetection, estimation, remote sensing, and imageprocessing.

Pramod K. Varshney (S’72–M’77–SM’82–F’97)was born in Allahabad, India, on July 1, 1952. Hereceived the B.S. degree in electrical engineering andcomputer science (with highest honors), and the M.S.and Ph.D. degrees in electrical engineering fromthe University of Illinois at Urbana-Champaign, in1972, 1974, and 1976, respectively.

During 1972–1976, he held teaching and researchassistantships with the University of Illinois. Since1976 he has been with Syracuse University, Syracuse,NY, where he is currently a Professor of electrical en-

gineering and computer science and the Research Director of the New YorkState Center for Advanced Technology in Computer Applications and Soft-ware Engineering. He served as the Associate Chair of the department during1993–1996. He is also an Adjunct Professor of Radiology at Upstate MedicalUniversity, Syracuse. His current research interests are in distributed sensornetworks and data fusion, detection and estimation theory, wireless commu-nications, image processing, radar signal processing, and remote sensing. Hehas published extensively. He is the author of Distributed Detection and DataFusion (New York: Springer-Verlag, 1997). He has served as a consultant toseveral major companies.

Dr. Varshney was a James Scholar, a Bronze Tablet Senior, and a Fellow, allwhile he was with the University of Illinois. He is a member of Tau Beta Pi andis the recipient of the 1981 ASEE Dow Outstanding Young Faculty Award. Hewas elected to the grade of Fellow of the IEEE in 1997 for his contributions inthe area of distributed detection and data fusion. He was the Guest Editor of theSpecial Issue on Data Fusion of the PROCEEDINGS OF THE IEEE, January 1997.In 2000, he received the Third Millennium Medal from the IEEE and Chan-cellor’s Citation for exceptional academic achievement at Syracuse University.He serves as a distinguished lecturer for the AES Society of the IEEE. He is onthe editorial board of Information Fusion. He was the President of InternationalSociety of Information Fusion during 2001.

Steven M. Kay (M’75–S’76–M’78–SM’83–F’89)was born in Newark, NJ, on April 5, 1951. Hereceived the B.E. degree from Stevens Institute ofTechnology, Hoboken, NJ, in 1972, the M.S. degreefrom Columbia University, New York, NY, in 1973,and the Ph.D. degree from the Georgia Instituteof Technology, Atlanta, in 1980, all in electricalengineering.

From 1972 to 1975, he was with Bell Labora-tories, Holmdel, NJ, where he was involved withtransmission planning for speech communications

and simulation and subjective testing of speech processing algorithms. From1975 to 1977, he attended the Georgia Institute of Technology to study commu-nication theory and digital signal processing. From 1977 to 1980, he was withthe Submarine Signal Division, Portsmouth, RI, where he engaged in researchon autoregressive spectral estimation and the design of sonar systems. He ispresently a Professor of Electrical Engineering with the University of Rhode Is-land, Kingston, and a consultant to numerous industrial concerns, the Air Force,


the Army, and the Navy. As a leading expert in statistical signal processing, hehas been invited to teach short courses to scientists and engineers at governmentlaboratories, including NASA and the CIA. He has written numerous journaland conference papers and is a contributor to several edited books. He is theauthor of the textbooks Modern Spectral Estimation (Englewood Cliffs, NJ:Prentice-Hall, 1988), Fundamentals of Statistical Signal Processing, Vol. I:Estimation Theory (Englewood Cliffs, NJ: Prentice-Hall, 1993), Fundamentalsof Statistical Signal Processing, Vol. II: Detection Theory (Englewood Cliffs,NJ: Prentice-Hall, 1998), and Intuitive Probability and Random Processesusing MATLAB (New York: Springer, 2005). His current interests are spectrumanalysis, detection and estimation theory, and statistical signal processing.

Dr. Kay is a member of Tau Beta Pi and Sigma Xi. He has been a distinguishedlecturer for the IEEE Signal Processing Society. He has served on the IEEEAcoustics, Speech, and Signal Processing Committee on Spectral Estimationand Modeling, on the IEEE Oceans committees, has been an Associate Editorfor the IEEE SIGNAL PROCESSING LETTERS and is currently an Associate Editorfor the IEEE TRANSACTIONS ON SIGNAL PROCESSING. He has received the IEEESignal Processing Society Education Award “for outstanding contributions ineducation and in writing scholarly books and texts. . ..” He has recently beenincluded on a list of the 250 most-cited researchers in the world in engineering.

James H. Michels (M’82–SM’90–F’05) receivedthe M.S. degrees from Syracuse University, Syra-cuse, NY, in both physics and electrical engineering,and the Ph.D. degree in electrical engineering fromthe same university in 1991. His Ph.D. thesis in mul-tichannel signal processing provided the foundationfor in-house and contractual research programs inhis former position with the Air Force ResearchLaboratory (AFRL) in Rome, NY.

Currently, he is the founder and Senior ResearchEngineer of JHM Technologies, Ithaca, NY. He also

serves as a volunteer Research Professor and Adjunct Faculty member withSyracuse University, where he assists a graduate student research group. Hiscurrent research interests include detection, estimation, multichannel adaptivesignal processing, multi- and hyperspectral image processing, time seriesanalyses, space-time adaptive processing (STAP), change detection, particlefiltering, and stochastic resonance. He has more than fifty papers and sixpatents in these technical areas.

Date post:	12-May-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

3172 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, … web... · Theory of the Stochastic...

Documents