Psychological Bulletin1974, Vol. 81, No. 12, 945-958

SIGNAL DETECTION THEORY:

CONSIDERATIONS FOR GENERAL APPLICATION

R. E. PASTORE1 AND C. J. SCHEIRER

State University of New York at- Binghamton

While there exist a number of papers describing the theory of signal detection,it appears that many psychologists are not aware of the ease with whichsignal detection theory can be applied, the range of applications possible, orthe limitations of signal detection theory. This paper briefly summarizes theassumptions of signal detection theory and describes the procedures, the limi-tations, and practical considerations relevant to its application. A workedexample of an application of signal detection theory to the study of cognitiveprocesses is included.

In recent years, researchers in many di-verse areas of psychology have begun toemploy the theory of signal detection to sepa-rate the ability of the subject to differentiatebetween classes of events from motivationaleffects or response biases. In addition to itsextensive application in sensory psychophys-ics, signal detection theory has found applica-tion in such diverse areas as speech percep-tion (Egan & Clarke, 1956), memory (Banks,1970; Bernbach, 1967; Parks, 1966), animallearning (Rilling & McDiarmid, 1965; Su-boski, 1967), audiology (Campbell & Moulin,1968), attention (Moray, 1970; Sorkin, Pas-tore, & Pohlmann, 1972), clinical psychology(Sutton, 1972), and sensory-evoked poten-tials (Hillyard, Squires, Bauer, & Lindsay,1971). The purpose of this article is to re-view and briefly summarize the more commonmodels 2 of signal detection theory, describethe procedures required to apply each model,and discuss the limitations inherent in each.While there are a number of excellent theo-retical papers describing signal detectiontheory and its various models (e.g., Egan &Clarke, 1966; Green & Swets, 1966; Lick-

1 The authors wish to thank Crawford Clark, DonRonken, John Swets, Douglas Creelman, WilliamLutz, and Charlotte MacLatchy for their helpfulcriticisms of earlier drafts of this paper.

Requests for reprints may be sent to either author,Department of Psychology, State University of NewYork, Binghamton, New York 13901.

2 The term model is used in this article to connotea special case of the theory based on certain clearlydefined assumptions and, therefore, is limited inscope.

lider, 1959; Peterson, Birdsall, & Fox, 1954;Swets, Tanner, & Birdsall, 1961; Van Meter& Middleton, 1954), there is a clear needfor a concise, unified explanation of how andwhen to use the various models of signaldetection theory. This article attempts to ful-fill that need. The first section of this articlepresents a brief summary of the models of sig-nal detection theory on a general level. Thesecond section presents practical considera-tions for the application of signal detectiontheory and the specific procedures used inthese applications. The third section outlinesthe potential use of signal detection theory inseveral experimental situations and presents aworked example of an application to thestudy of cognitive processes.

OVERVIEW

The purpose of this section is to provide abrief overview and summary of the theoreti-cal underpinnings of signal detection theory.For a more complete introduction and theo-retical presentation, the reader should refer tothe articles by Egan and Clarke (1966), bySwets et al. (1961), or others. Signal detec-tion theory is an adaptation of statisticaldecision theory (e.g., Wald, 1950). A majoraspect of both signal detection theory and sta-tistical decision theory concerns the specifi-cation of a set of ideal processes or observersas a standard against which a subject's per-formance is compared. While this comparisonis an important aspect of signal detectiontheory, the specification of an ideal observerdepends on the exact area or modality under

945

946 R. E. PASTORE AND C. J. SCHEIRER

study and the assumed capabilities of theobserver. Therefore, this aspect of signaldetection theory is not considered in thisarticle; for a general discussion of the use ofideal observers, see Tanner (1961) or Tan-ner and Sorkin (1972).

General Case

Signal detection theory is applicable tothose situations in which two classes of eventsare to be discriminated. It also can be gen-eralized to situations involving more than twoclasses of events (Tanner, 1956; but also seeLuce, 1963), although this generalization isnot discussed in this article. The basic as-sumption of signal detection theory is thateach decision made by the subject is based ona statistic that is derived from the many (i.e.,M) characteristics of the event in question.This statistic reflects the relative probabilitythat the observed characteristics of the eventarose from one of a specific class of events.The optimum statistic for such a decision isthe likelihood ratio or some monotonic trans-form of the likelihood ratio (Green & Swets,1966). The likelihood ratio, X(«), is the rela-tive likelihood that the event, u,. arose fromone as opposed to the other class of events.That is, /»(«), the likelihood that the givenM-dimensional observation u arose from classi, is the product of the probability that eachof the M observed characteristics arose froma class i event, and

X(«) =/!(«)//,(«). [1]

The theory assumes that the subject computes\(u), or some monotonic transform ofA(w)[e.g., \'(u) = log A.(w)], for each eventand makes a response decision based on thatcomputed value. It is further assumed thatthe subject adopts a fixed criterion value ofA(«), called /3 and that the decision corre-sponding to any event, u, is simply a state-ment of whether \(u) is greater than /?. Thecapability of the subject to discriminate be-tween the two classes of events is inverselyproportional to the total area common to thetwo conditional probability density functions[/{(«); i = 1, 2]. This common area is as-

sumed to be invariant during the measure-ment interval.3

Assumption of Normality

One specific set of signal detection theorymodels, the Gaussian models, assumes that thetwo conditional probability density functions[/»(«)] are Gaussian (normal). One theo-retical basis for this assumption is that thereare a large number (i.e., M) of independentcharacteristics of the event sampled on eachobservation, and that the system performs alogarithmic transform of the computed likeli-hood ratio (Egan & Clarke, 1966). It shouldbe noted that logarithmic transforms arecommon in perceptual and behavioral data(e.g., Fechner's and Stevens' laws). Becauseof this hypothetical logarithmic transforma-tion, the separate likelihood statistics are thesums of a large number of independent fac-tors. Then, according to the central limit the-orem, the distribution of the likelihood sta-tistics approximates a normal distribution.

On a practical level, the important questionis whether there exists evidence that in thegiven experimental situation, the assumptionof normality is tenable. Such evidence mightbe in the form of reliable results published inthe literature or in a test of the assumptionby the experimenter (see following sectionsentitled "Assumptions of Normality andEqual Variance" and "Rating Procedure").If the Gaussian assumption cannot be justi-fied, then alternative statistics should be em-

3 The computation of a likelihood ratio statisticassumes that the observer knows the probabilitydistribution for each sampled characteristic condi-tional on each of the two classes of events. Obvi-ously, the discriminaWlity of the events from thetwo classes of events reflects the subject's knowl-edge of the actual differences between the twoclasses. The assumption of a decision statistic basedon the likelihood ratio is simply an assumption thatthe subject's knowledge of the classes of events canbe used in terms of the conditional probability den-sity functions for each characteristic, ,and that thisprobability information is combined in an efficient,systematic manner. The theory further states thatany factor (i.e., learning) that changes the subject'sknowledge of these differences will alter the likeli-hood statistics, and therefore the discriminabilityof the two classes of events.

SIGNAL DETECTION THEORY 947

ployed. Such statistics might include simpleresponse probabilities, estimates of "thresh-olds," statistics derived from a different para-metric model of signal detection, or non-parametric indexes of signal detection theory.Some of these alternatives, including non-parametric indexes of signal detection theory,are also discussed in the section entitled"Nonparametric Model."

Since the Gaussian distribution is com-pletely defined by its first two moments, themean and variance, the area common to twoGaussian density functions, and thus the dis-criminability of the two classes of eventsgiving rise to these functions, is a monotonicfunction of the distance between the distri-bution means scaled in terms of the pooledor average variance (a z transformation). Themost common version of the Gaussian modelis based on the further assumption that thevariances of the probability density functionsconditional on the two classes of events areequal. If this and the other assumptions arevalid, then the differential weighting of thetwo distributions caused by any responsebias does not affect the estimate of the pooledvariance, and the scaled distance between thetwo means is called d'. This value is equal tothe difference of the signed distances, in z-score units, from each mean to the subject'scriterion. These distances may be estimatedby converting into z-score units the proba-bility (relative frequency) of the subject cor-rectly identifying events from the separateclasses. The value of d' also may be esti-mated from appropriate tables of d' (e.g.,Elliott's tables in Swets, 1964) by arbitrarilycalling one of the two classes of events (i.e.,Class 1) the "signal," and the other class,"noise."

If one is interested in motivational or cri-terion factors, the criterion (/?) employed bythe subject should be examined. In the equal-variance Gaussian model this criterion is inde-pendent of d' and is defined by the followingequation:

. . . . [2]

The values of f(b\i) may be estimated fromthe values of P (response i event i) with atable of ordinates of the normal curve. Itshould be noted that P (Response 2 1 Event2) and /8 are monotonically related.

Receiver Operating Characteristics (ROC)

A receiver operating characteristic is thelocus of points representing the performanceof a subject across all criteria under a fixedexperimental condition. The ordinate of thereceiver-operating-characteristic function isP( response * event i) and the abscissa is/•(response z|event ;'), i ̂ ;, for each criterion.Since these probabilities are determined bythe form of the underlying density functions,the shape of the receiver-operating-character-istic curve also is determined by the natureof these underlying distributions.*

The receiver-bperating-characteristic curveis generated by sampling the performance ofa subject under a given experimental condi-tion while the subject uses various criteria.The subject's criterion can be manipulatedthrough the use of instructions or by the useof a differential payoff schedule; in standardbinary decision tasks (yes-no and two-alternative forced choice) the criterion is ma-nipulated across, but not within (see sectionentitled "Criterion Stability") blocks of trials.A receiver operating characteristic can also begenerated within blocks of trials by the useof the rating scale procedure discussed in asubsequent section entitled "Rating Proce-dure." Since it is assumed that the distribu-tions on which the subject bases decisions areinvariant under fixed experimental conditionsindependent of the criterion employed, thereceiver-operating-characteristic curve is the

where }(b\i) is the height of the probabilitydensity function for class i at the criterion orboundary, b, between the two response classes.

4 Any factor that affects the density functions onwhich the subject bases his or her decision willaffect the shape of the receiver-operating-character-istic curve. Some factors known to be important inthese respects include the., experimental paradigmemployed by the experimenter (Markowitz & Swets,1967), the strategy" (i.e., decision rule) employed bythe, subject (Luce, 1963), the modality (auditory,visual, memory, etc.) in which the subject is operat-ing (Green & Swets, 1966), the subject's criterionstability (Healy & Jones, 1973), and the intertrialintervals employed (Green & Swets, 1966).

948 R. E. PASTORS AND C. J. SCHEIRER

map of data points for all possible criteriaat a fixed level of sensitivity. Thus, thereceiver-operating-characteristic curve is alsocalled the Isosensitivity curve. In memorytasks, it is referred to as the memory oper-ating characteristic (MOC).

APPLICATION CONSIDERATIONS

Assumptions of Normality and EqualVariance

If there is sufficient evidence in the litera-ture to warrant the assumption of equal-variance Gaussian density functions, d' and /?may be employed to describe, respectively, thesubject's ability to discriminate the twoclasses of events and the subject's responsebias, subject to the considerations describedbelow. If there is insufficient evidence tosupport the assumptions of the model (thatthe density functions are Gaussian and ofequal variance), these assumptions should betested directly in applying the model. Themost typical method for testing these assump-tions is the use of a rating procedure (cf.Egan, Schulman, & Greenberg, 1959). Thisprocedure should be used also if the densityfunctions are suspected of being Gaussian,but of unequal variance, and might beemployed by careful researchers even whenboth assumptions are supported by previousresearch.

Rating Procedure

With the rating procedure, the subject isasked to use N different responses that reflectthe subject's confidence that a Class 1 eventhas occurred. Typically, five to eight confi-dence ratings or responses are employed. Itis assumed that the subject operates in amanner similar to that employed in binarydecision tasks (yes-no or two-alternativeforced choice) and adopts N — 1 criteriaseparating each adjacent pair of the N re-sponses rather than the single criterion em-ployed in the binary task. The results gen-erated by the use of these N — 1 criteria areplotted as N — 1 points (%, ys), where ; = 1,2, . . . , N — 1, and where x} and ys are

defined as

iXj = ̂ P (response i\ Event 2)

and

ys = Y. P (response i] Event 1). [3]

Thus Xj and y} are the values of the distribu-tion functions for Events 1 and 2 at criterion;'. Obviously, if the assumptions of equal-variance Gaussian functions hold, the ex-pected values of d' calculated for each point(Xj, yf) are equal and therefore not cor-related with the criterion employed by thesubject.

The functional relationship between y^ (theprobability of a "hit") and xt (the probabil-ity of a "false alarm") describes the contourof criteria for a fixed set of density functionsunder a specific experimental condition. Thisfunction of equal sensitivity is the receiver-operating-characteristic curve described ear-lier. If the probabilities *» and yt are trans-formed to the equivalent z scores [x'j = z of(Xj — .5) and y'} = z of (y} — .5)], the nor-malized receiver-operating-characteristic curvecan be used to test the validity of the assump-tions of normality and equal variance. If theunderlying density functions are Gaussian,the normalized receiver-operating-character-istic curve will describe a linear function:

= ax' + c. [4]

The slope of this function, a, is equal to theratio of the standard deviations of the twodensity functions (a = o-2/<ri), and the inter-cept, c, is related to the distance between thedistribution means,

If the receiver-operating-characteristic curveexhibits a systematic deviation from linearity,the Gaussian assumption may be invalid. Ifthis deviation from linearity is large, but notsystematic, there exists an actual deviationfrom normality and/or a large error factorthat may be correlated with the criterion ofthe subject. Any criterion-correlated errorfactor will distort the form of the normalizedreceiver-operating-characteristic curve. How-

SIGNAL DETECTION THEORY 949

ever, an equally important concern is thatany large error factor might mask an actualdeviation from linearity. Such error factorsmay be due to a number of different prob-lems, including a high degree of criterionvariability and/or an insufficient number oftrials employed in the experiment (see sec-tion entitled "Number of Trials"). If theerror factor is large, or is suspected of beinglarge, the interpretation of the results shouldreflect this fact.

The linear function best describing thedata, expressed as standard normal deviates,should be estimated by standard curve-fittingoperations.6 If the data are adequately de-scribed by a linear function, the Gaussianassumption is supported. It should be noted,however, that this use of cumulative ratingdata to generate the receiver-operating-char-acteristic curve provides data points that arenot independent and, at a minimum, imposesa monotonic relationship between successivedata points.

If the Gaussian assumption is not rejected,the equal-variance assumption is tested withthe slope, a, of the normalized receiver-operating-characteristic curve (Equation 4).If the slope is approximately equal to 1.0, theequal-variance assumption is not rejected andthe equal-variance Gaussian model may belegitimately employed with d' and ft as pa-rameters. While the values of d' estimatedfrom the N — 1 criteria are not totally inde-pendent estimates, the mean or median valueof d' may be employed as the estimate of ob-server sensitivity. The measures derived for

5 Many researchers (e.g., Swets et al., 1961) haveused simple visual fits to determine the "best-fit-ting" straight line. This crude method is probablysufficient for most proposed uses of the function.Conventional least squares curve-fitting proceduresare theoretically inappropriate because both variablesare dependent variables and subject to error. Forrough approximations, this problem is of minorimportance since the error introduced is likely to besmall relative to the noise in the data. However, forresearchers interested in precise estimates of theparameters of receiver-operating-characteristic curvesand in a test for goodness-of-fit of the theoreticalmodel, maximum-likelihood estimators giving exactfits have been developed by Ogilvie and Creelman(1968).

the unequal-variance Gaussian model, dis-cussed in the next section, and the nonpara-metric model, discussed in the section entitled"Nonparametric Model," may be more desir-able than those derived from the equal-vari-ance model since fewer restrictive assumptionsare involved.

General Gaussian Case

If the equal-variance assumption is vio-lated, d' and /? will be correlated to a degreethat is related to the deviation from equalityof variance. The general Gaussian (or un-equal-variance Gaussian) model is applicablewhen the Gaussian assumption is justified, in-dependent of the relative magnitude of thevariances. Application of the general Gaussianmodel requires knowledge of the slope, a, ofthe normalized receiver-operating-character-istic curve (Equation 4) which may be esti-mated with the rating procedure (see sectionentitled "Rating Procedure"). The basic goalof the general Gaussian model is to developa statistic that describes sensitivity, is inde-pendent of the subject's criterion, and reflectsthe average spread or variance of the twodistributions. Several statistics use the factthat when ft = 1.0, ft is equidistant from thetwo distribution means in terms of standardnormal deviates (z scores) for each of thegiven distributions. At J3 = 1.0, the "hit rate"for the two classes of events [P(response i\event i)] are equal. Thus d' computed aty8 = 1.0, the minimum total error criterion(given equal probability of presentation forthe two classes of events), will be based onthe average standard deviation with equalweighting given to the two distributions. Thisminimum error criterion is the negative diago-nal (y' = — x') of the receiver-operating-characteristic space (see Figure 1). The co-ordinates (x'm, y'm) of the intersection ofthe estimated receiver-operating-characteristiccurve (Equation 4) and the negative diagonaldefine the value of d' for this minimum errorcriterion. The value of d' for this point isequal to the distance along the negative diag-onal from the positive diagonal (chance line)scaled in terms of the difference between thecoordinates (y'm — x'm), and is called d's

950 R. E. PASTORE AND C. J. SCHEIRER

(Clarke, Birdsall, & Tanner, 1959). This valueof d' (d's) is the distance between the dis-tribution means scaled in terms of the aver-age of the standard deviations for the twodistributions.

With knowledge of the slope, a, of the nor-malized receiver-operating-characteristic func-tion, one can compute d's from a single datapoint (x'k, y'k) by the formula

'd'. = 2(y'k -ax' * a). [5]

Other measures of sensitivity that may bederived for the general Gaussian model in-clude d'e, which equals the square of d's(Egan, 1958; Egan & Clarke, 1966) and Am,the x intercept of the normalized receiver-operating-characteristic curve (Green & Swets,1966).

Whenever d' or d's is computed from repli-cated sets of binary decisions, the variousestimates of sensitivity for a given subjectshould be examined for any relationship be-tween the sensitivity measure and the prob-ability of a false alarm, P( Response l| Event2). Any such relationship indicates a devia-tion from the assumed slope of the receiver-operating-characteristic curve. This simplecheck should be followed whenever repeatedestimates of sensitivity are obtained for eachsubject, whether or not the variances areassumed to be equal.

Nonparametric Model

The Gaussian models of signal detectiontheory are members of a class of models inwhich the parameters describing the abilityof the subject to perform the given task andthe subject's decision rule are dependent onthe assumption of certain specific underlyingdensity functions. Other parametric models ofsignal detection theory based on different as-sumed density functions (e.g., negative expo-nential, rectangular, Raleigh, and Rice) canbe derived (Green & Swets, 1966; Pollack &Hsieh, 1969), but such models have not beenfully developed for general use and, in anycase, would be of questionable utility sincethe user must be able to justify the assump-tion of the given underlying density func-

tions. Since it is unlikely that, in a particularsituation, data that are sufficiently deviant tolead to rejection of the Gaussian model wouldbe sufficiently regular to support an alterna-tive model, a nonparametric model of signaldetection theory whose measures are indepen-dent of the exact nature of the underlyingdensity function would be of general utility.

Green (1964) proposed the use of thearea under the receiver-operating-characteristiccurve, Ag, as a measure of observer sensitiv-ity. Assuming only that the subject bases hisdecision on two continuous probability den-sity functions that are identical under thevarious experimental procedures, this measurecan be shown to be identical to the expectedpercentage correct in a two-alternative forcedchoice experiment (Green & Moses, 1966).Since this relationship is independent of theform of the underlying probability densityfunctions, it may be employed with no priorassumptions concerning the shape of thesedensity functions. This area measure of sensi-tivity may be employed for both rating data(Pollack, Norman, & Galanter, 1964) andsingle data points (Pollack & Norman, 1964).

The derivation of the area measure andthe corresponding nonparametric measure ofcriterion is based on the unit square (seeGrier, 1971). This square has as its abscissaand ordinate, xt and yi, as denned in Equa-tion 3. When only a single point relating xt

and yi is available, the area under the curvejoining the points (0,0) to (x,y) to (1,1) isdetermined. This area, Ag, is then taken to bethe index of observer sensitivity; Ag is theaverage of the maximum and minimumpossible areas under the receiver-operating-characteristic curve and is given by the fol-lowing formula:

^,=.5+(y-*)(l+y-*)/4y(l-*). ,[6]

The Ag measure has also been extended tocases where more complete receiving-oper-ating-characteristic curves are available (Pol-lack, Norman, & Galanter, 1964) for exam-ple, when data have been obtained throughthe rating procedure (see previous section en-titled "Rating Procedure"). In this case thearea under the curve Ag, can be estimated by

SIGNAL DETECTION THEORY 951

the formula:

A. = (1/2) Z (*y+i - *y) (yi+1 + yy), [7]

where Xj and y^ are defined by Equation 3.Pollack and Hsieh (1969) have used Monte

Carlo methods to sample from various den-sity functions in order to investigate thesampling distributions of Ag and d'e (dis-cussed in a previous section entitled "GeneralGaussian Case"). They determined that whenthe normality assumptions of signal detectiontheory are satisfied, d'e and a Gaussian trans-form of Ag, N(Ag),w& related by the formula

N(Ag) = [.707 - .2341og2 [8]

They found that the empirically determinedvalues of N(Ag) tended to overestimate thevalues of d'e by l%-6%.

Hodos (1970) developed a nonparametricmeasure of criterion or bias. This measurewas based on the fact that the negative diag-onal of the unit square represents the locusof points where the subject would be equallylikely to respond "i" or "j" given ambiguousstimulus conditions. The measure reflects thedegree to which a data point deviates fromthe negative diagonal relative to the maxi-mum possible deviation. A computationalformula for the nonparametric measure ofcriterion, ft', based on the Hodos measure,has been developed by Grier (1971). Theformula is:

(? = 1 - *,-(! - *,)/?,-(! - *), [9]

where Xi and yt are defined in Equation 3.

Criterion Stability

If the criterion adopted by the subject isnot stable during any given session, the vari-ability of the criterion will affect the results.The presence of criterion variability cannotbe detected easily, and will have the sameeffect on the results as an increase in thevariance of both likelihood density functions.Criterion variability therefore decreases theestimate of d' by an amount that is relatedto the size of the criterion variance without

actually affecting the true discriminability ofthe two classes of events. Obviously, anyexperimental manipulation that affects cri-terion variability will alter the estimate of d'.Safeguards against criterion variability in-clude the use of trained subjects, strictinstructions to the subjects about maintaininga stable criterion, and strict definitions of thesubject's response classes. Since the subject'scriterion is partially determined by the expec-tation of the probability of presentation ofthe two classes of events, the subject shouldbe made aware of the absence of sequentialdependencies across trials.

While it may be reasonable to assume thatthe criterion employed by a single subjectduring any measurement session (block oftrials) is stable, it is less reasonable to assumethat the subject will employ the same cri-terion across sessions, or even across separateblocks of trials within a session. Therefore,only the data for a single block of trialsshould be used to estimate a value of d'. Theestimates of d' from the various blocks oftrials may then be averaged.

Malingering

The positive diagonal of the receiver-oper-ating-characteristic space (x' = y') defineschance performance. Under the equal-varianceGaussian model, the receiver-operating-char-acteristic curve that corresponds to the posi-tive diagonal is generated under the conditionof exact equality for the two density functions.Data points below this chance line can begenerated only by (a) measurement error or(b) the subject performing the discriminationand then emitting a response that is incon-sistent with the computed decision statistic[A'(y)l- If a subject consistently producesdata that fall below the chance line, there isjustification to assume that the subject canperform the discrimination, but is malingering.

Number of Trials

In applying signal detection theory, the ex-perimenter is assuming that there are twofixed internal probability density functions,and the subject has established a fixed cri-

952 R. E. PASTORE AND C. J. SCHEIRER

terion along the dimension (decision axis) onwhich these functions lie. The purpose of theexperiment is to estimate the area [P (re-sponse "i"\event j)] in the tail of each of thetwo distributions from the relative fre-quencies of the responses. The expectedstandard error in estimating the probablity[P("i"\j)] as a function of both tie samplesize and the expected value of this probabilityis p • q/s, where p is the expected value of theprobability, q = 1 —p, and s is the samplesize. The expected error of estimation for d'can be obtained by applying a z transforma-tion to p — (p • q/s). Green and Moses(1966) found that the actual error involvedin estimating this parameter is slightly largerthan predicted by this assumption of binomialvariability. Pollack and Hsieh (1969), in acomputer simulation, found that the errorvariance in Ag was slightly smaller than pre-dicted by the assumption of binomial vari-ability. Therefore, the expected binomial vari-ability would seem to reflect the magnitudeof error to be expected in a given measure.Obviously, the use of only a small number oftrials for one or both of the event classes re-sults in a large expected error in estimates ofthe parameters of the model. Furthermore,reliable estimates of sensitivity require a largenumber of trials when based upon extremevalues of P("i"\j).

APPLICATIONS OF SIGNAL DETECTION THEORY

Since signal detection theory provides theresearcher with a means of evaluating inde-pendently both the ability of an organism todiscriminate between classes of events andmotivational or other response effects, it canbe a powerful research tool having applica-tion in a variety of different experimentalsettings. It is the purpose of this section tooutline some potential applications of signaldetection theory in areas of psychology wherethis method has not been widely used. Theapplications discussed include the evaluationof (a) the state of the organism or environ-ment, (b) the relationship between stimuliand potential or actual responses, and (c) theindependence of "channels" for processingstimulus information. Finally, a more tradi-

tional example from memory work is pre-sented in some detail to provide a workedexample for persons unfamiliar with the com-putational procedures involved in a signaldetection theory analysis.

Evaluating the Condition of Subjects orEnvironment

The ability of a subject to perform detec-tion, discrimination, or recognition tasks canbe altered by a number of conditions includ-ing the psychological or physiological condi-tion of the subject (e.g., behavioral or organicdysfunction), the existence of a drug state, orthe imposition of an external stimulus. Inmany cases, however, it is unclear whetherthe performance difference is due to changesin the ability of the subject to perform thetask or changes in the response tendencies ofthe subject. Signal detection theory may beused in a between-groups design to evaluatethe cause of the observed differences betweenan altered and a control population. Simi-larly, signal detection theory could be usedin a pretest-posttest design to investigate thelocus of performance differences as a resultof pharmacological or surgical interventions.A somewhat less obvious potential applica-tion occurs in the area of motivation. If anexperimenter discovers that rats initially ex-hibit a preference for a given solution overwater, but after two months of continuousad libium intake of the solution exhibit nodifferential preference, the experimenter doesnot know whether the motivation of the tatsor their ability to discriminate between thetwo solutions ;has been altered. However, byusing the given solution and water as thediscriminative stimuli with either an appeti-tive or avoidance conditioning technique, theresearcher could apply nonparametric mea-sures of signal detection theory to the per-formance data to evaluate the nature of thischange.

Evaluating Response Factors

Performance in any given task is deter-mined by two main classes of variables:those -that affect the discriminability of

SIGNAL DETECTION THEORY 953

stimulus events or conditions and those thataffect the motivational state and responsetendencies of the given subject. These twoclasses of variables are completely analogousto the sensitivity and criterion factors con-sidered by signal detection theory. Therefore,differences between response or motivationaleffects may be evaluated by signal detectiontheory analysis techniques. For example, theresponses of infrahuman subjects may beevaluated in terms of the "hit" and "falsealarm" rates with nonparametric measures ofsignal detection theory applied to separatethe two classes of effects. In addition, if oneis willing to assume that the magnitude of theresponse (e.g., galvanic skin response orreaction time) is directly (or inversely) pro-portional to the likelihood ratio, one can gen-erate a receiver-operating-characteristic curveand employ parametric measures of signaldetection theory (cf. Pike, 1973).

An assumption with greater face validityand some empirical support (Emmerich,Gray, Watson, & Tanis, 1972) is that themagnitude of a response parameter reflects theabsolute value of the difference betweenthe given likelihood ratio and the subject'sresponse criterion. Partitioning the data onthe basis of a subject's binary (yes-no, etc.)response, the strength or speed of a "yes"(or "signal") response may be assumed to bemonotonically related to the likelihood ratio,with "no" responses reflecting likelihoodratios that are below those for "yes" re-sponses and are an inverse monotonic func-tion of the strength or speed of response.Once a receiver-operating-characteristic curveis generated, the assumptions for the para-metric measures (df, d's, etc.) can be tested,and the appropriate measure employed. Foran example of this type of analysis appliedto response latency, see Murdock (1966).

The use of modern data-acquisition andanalysis procedures have opened the area ofthe neural coding of stimuli to study. Manyanalysis techniques (e.g., average evokedpotential, poststimulus time, and interpulseinterval histograms) pool the data acrosstrials to extract statistically the average re-sponse from the data. While these techniques

have statistical validity, the "average" re-sponse they extract may be typical of noneof the actual responses. Signal detectiontheory offers another statistical means ofextracting a "characteristic response." As-sume that in a study of multicellular activityin a given nucleus, it is found that vibratorystimuli presented simultaneously with a lightflash cause a characteristic "averaged" re-sponse that differs in certain aspects fromthe response to only a light flash. An analy-sis of the individual trial data using anarbitrarily defined response can estimate acomplete outcome matrix (hit, false alarm,etc.) for the given response from which asignal detection theory measure of the ade-quacy of that "response" in distinguishing"stimulus-plus-noise" events from "noise-only" events. Systematic modification ofthe definition of a "response" can then beused to determine the response pattern thatbest discriminates the stimulus from thenoise. Thus, the averaging may have indi-cated the importance of a burst of respond-ing 10 milliseconds after the stimulus, whilethe iterative signal detection theory tech-nique may indicate the requirement of twodistinct bursts that must begin between 7 and9 milliseconds, and must be separated by a2-millisecond lull in firing.

Evaluating Channel Independence

Recently there has been considerable inter-est in the ability of subjects to process inde-pendently the information presented to differ-ent sensory or perceptual "channels." Onefactor contaminating much of the early workin this area is the change in the criterion ofthe subject with changes in the requirementsof the task. Eijkman and Vendrik (1965)studied the independence of processing ofauditory and visual signals. Signal detectiontheory measures of the detectability of theauditory and visual stimuli were estimatedfor a set of subjects in separate experiments.Then the same subjects were asked to per-form simultaneously the same independentauditory and visual detection tasks with sepa-rate sets of responses for each type of stimu-

954 R. E, PASTORS AND C. J. SCHEIRER

lus; signal detection theory measures of sensi-tivity were computed for each of these twostimulus channels. A comparison of the signaldetection theory measure under the simpleand simultaneous condition provides an indi-cation of the degree of interaction betweenthe two tasks, which may be evaluated inquantitative terms by methods developed byTaylor, Lindsay, and Forbes (1967). Becausethere is independence across stimuli andacross most stimulus-response categories inthe response matrix, it is possible to calculatea signal detection theory measure of sensitiv-ity in one channel conditional on the simul-taneous stimulus, response, or outcome eventin the other channel. With modern data-handling techniques, partitioning of the datain this manner has become a simple matter.Pastore and Sorkin (1972) examined the ef-fects on sensitivity in a single sensory chan-nel as a function of the various possiblestimulus events and outcomes in the secondchannel in a simultaneous two-channel detec-tion paradigm. This technique of analysis hasalso been successfully employed by Harveyand Treisman (1973) in a simultaneous taskand by Sorkin, Pohlmann, and Gilliom (1973)in a successive two-channel task.

Evaluating Memory Processes

Lutz and Scheirer (1974) investigateddifferences in the processes involved in theencoding of visually presented verbal andpictorial stimuli. Each subject was presentedwith a series of 190 stimulus items; eachitem was presented for either .25, .50, 1.00,or 2.00 seconds with a fixed interstimulusinterval of either .25, 1.00, or 2.00 seconds.All conditions in the 4 X 3 X 2 (PresentationInterval X Interstimulus Interval X StimulusType)' factorial arrangement were presentedto independent groups of 12 subjects.

At the beginning of the session the sub-jects were told that a series of items wouldappear on the screen in front of them. Thesubjects were instructed to "pay careful at-tention to each item . . ." since the subjectswould "later. . . be given a test based onthese items." Following the presentation ofthe 190 stimulus items, the subjects were told

that a second series of items would be pre-sented, each for a few seconds. They weretold that some of the items had been pre-sented previously and some had not, and weregiven the following instructions:

When an item appears, look at it and decidewhether the item appeared in the first series. Youshould record your decision on the answer sheet.There are six categories you can respond with:

+H-+ if you are definite that you have seen theitem before;

++ if you believe that you have seen the itembefore;

+ if you guess that you have seen the itembefore;

— if you guess that you have not seen theitem before;if you believe that you have not seen theitem before;if you are definite that you have not seenthe item before.

For each stimulus you should respond with one andonly one of the above categories. For each item thatappears, circle the appropriate symbol on the answer,sheet as soon as you have made your decision. Tryto use all six categories but only where appropri-ate ... In summary, when the first item appearslook at it, decide whether the item appeared in thefirst series . . . Then wait for the next item toappear [p. 317].

AH instructions were read aloud to the sub-jects, with the category definitions typed ona card given to the subject for referenceduring the experiment. A series of 120 teststimuli were presented to the subject, 60 ;ofthese test stimuli were randomly chosen fromthe original 190 items. These "old" itemswere randomly mixed with 60 "new" itemsthat were not in the original set....

The relative frequency of responding witheach of the six categories to each of the twoclasses of events is shown in Table 1 for twoof the subjects. These rating response datawere converted to cumulative response prob-abilities as described by Equation 3 and thenplotted as receiver-operating-characteristiccurves. Figure 1 is the normalized receiver-operating-characteristic curve for the two sub-jects reported in Table 1. The upper and rightmargins of the figure are delineated inz-score units. The lower and left margins aredelineated in terms of probabilities. Thedata points are labeled according to the limits

SIGNAL DETECTION THEORY 955

TABLE 1

RELATIVE AND CUMULATIVE FREQUENCIES WITH EACH CATEGORY TOR THE SUBJECTSDESCRIBED IN TEXT° AND IN FIGURE 1

Subject

1

Total

2

Total

response i

+ + ++ +

+

—

+ + ++ +

+—

P(j'lnew)

.033

.000

.067

.067

.200

.6331.000

.033

.250

.167

.183

.234

.1331.000

P(»|old)

.533

.050

.117

.133

.100

.0671.000

.517

.200

.050

.100

.117

.0161.000

SP(t|new)

.033

.033

.100

.167

.3671.000

.033

.283

.450

.633

.8671.000

SP(»Iold)

.533

.583

.700

.833

.9331.000

.517

.717

.767

.867

.9841.000

»See section entitled "Rating Procedure.'

of summation indicated in Equation 3 andTable 1. The linear function describingeach set of data is plotted in Figure 1.The data for the second subject appearto be curvilinear. Had the data for amajority of the other subjects been curvi-linear, the Gaussian assumption would haveto be rejected. However, a small and approxi-mately equal number of receiver-operating-characteristic curves were curvilinear in eachdirection, and most functions were linear(e.g., see the curve for Subject 1 in Figure 1).Therefore, the Gaussian assumption was heldto be supported by the data and the fewdeviations from linearity were assumed to bedue to error.

The linear functions for the two subjectsplotted in Figure 1 have estimated slopes of.91 and .85, respectively. While these slopesdo not differ substantially from the slope ofunity required by the assumption of equal-variance Gaussian distributions, larger devia-tions were found for a number of subjects.Since acceptance of the equal-variance as-sumption is therefore tenuous and since therating procedure allows the use of the generalGaussian model (see sections "Rating Proce-dure" and "General Gaussian Case"), d'g wasused as the measure of discriminability rather

than d'. The negative diagonal (minimumerror criterion) of the receiver-operating-characteristic space in Figure 1 is delineatedin units of d's. The intersection of this diag-onal with the linear regression lines for theobtained data yield d's estimates of 1.94 and1.14 for the two subjects. The correspondingvalues of the nonparametric area measure ofsensitivity, Ae, computed with the use ofEquation 7, are .892 and .785. These areestimates of the ability of the subjects to dis-criminate the two classes of events indepen-dent of the criteria employed and any dif-ferences in variability within the classes ofevents.

Using any positive (+, ++, + + + )response as a response indicating an "old"stimulus and any negative response (—, — — ,

) as a response indicating a "new"stimulus, the probability of a correct responsewas computed for each subject. A within-cellproduct-moment correlation of .91 was ob-tained between d's and the probability of beingcorrect. This high correlation appears to be atleast partially due to the use of a strict setof criterion categories. While this procedurewas intended to minimize within-subject vari-ability, it also appears to have caused mostsubjects to adopt a set of criteria whose

956 R. E. PASTORS AND C. J. SCHEIRER

-2.5 -2.0 -1.5 -1.0 -0.5 1.0 1.5 2.0 2.5

0.99-

0.98-

0.90-

0.70-

Y 0.50-

0.30-

0.10-

2.5

0.02-

0.01-

i........... Subj. 1 Y' = 0.91 X' +- 1.84

(X> Subj. 2 Y'= 0.85 X'-i-UO _L _2 0

i i i i—r -2.5

0.01 0.02 0.10 0.30 0.50 0.70 0.90 0.98 0.99

X

FIGURE 1. Normalized receiver-operating-characteristic for two subjects,which is the probability of a "hit" [y = P(response i| stimulus »)1 plotted asa function of the probability of a "false alarm" lx = P(response z|stimulus;), Z96;'] in terms of z scores. (The right and upper edges of the figure aredelineated in terms of z scores lx', y ' ] , while the left and lower edges aredelineated in terms of the equivalent probabilities lx, y\. The procedures usedto generate the receiver-operating-characteristic curve are described in thelast part of this article. Units of d's are delineated along the negative diagonaland are explained in the section entitled "General Gaussian Case.")

centroid was consistent. This improved thevalidity of the probability-of-correct responsemeasure by limiting the criterion variabilitybetween subjects.

FINAL CONSIDERATIONS

While this article is intended to provide thereader with an overview of the general theoryand requisite procedures to use signal detec-tion theory, it is recommended that thisarticle also be used as a guide to reading themore detailed statements of signal detectiontheory, most of which deal only superficiallywith many of the application considerationsdescribed in the second section. This articlehas treated only the simple, more commonmodels of signal detection theory. Models of

signal detection theory have been generalizedto multicomponent recognition tasks (Pastore& Sorkin, 1971; Tanner, 1956), and to abroad spectrum of research applications inmodern psychology (i.e., Swets, 1973). Inaddition, statistical tests for various signaldetection theory parameters have been devel-oped (i.e., Gourevitch & Galanter, 1967;Ogilvie & Creelman, 1968). Critiques of signaldetection theory may be found in Luce(1963), and Abrahamson and Levitt (1969).

REFERENCES

Abrahamson, I. G., & Levitt, H. Statistical analysisof data from experiments in human signal detec-tion. Journal of Mathematical Psychology, 1969,6, 391-417.

SIGNAL DETECTION THEORY 957

Banks, W. P. Signal detection theory and humanmemory. Psychological Bulletin, 1970, 74, 81-99.

Bernbach, H. A. Decision processes in memory.Psychological Review, 1967, 74, 462-480.

Campbell, R. A., & Moulin, L. K. Signal detectionaudiometry: An exploratory study. Journal ofSpeech and Hearing Research, 1968, 11, 402-410.

Clarke, F. R., Birdsall, T. G., & Tanner, W. P., Jr.Two types of ROC curves and definitions ofparameters. Journal of the Acoustical Society ofAmerica, 1959, 31, 629-630.

Egan, J. P. Recognition memory and the operatingcharacteristic. (Tech. Note AFCRC-TN-58-51)Indiana University: Hearing and CommunicationLaboratory, 19S8.

Egan, J. P., & Clarke, F. R. Source and receiverbehavior in the use of a criterion. Journal of theAcoustical Society of America, 1956, 28, 1267-1269.

Egan, J. P., & Clarke, F. R. Psychophysics andsignal detection. In J. B. Sidowski (Ed.), Experi-mental methods and instrumentation in psychol-ogy. New York: McGraw-Hill, 1966.

Egan, J. P., Schulman, A. I., & Greenberg, G. A.Operating characteristics determined by binarydecisions and by ratings. Journal of the AcousticalSociety of America, 1959, 31, 768-773.

Eijkman, E., & Vendrik, A. J. H. Can a sensorysystem be specified by its internal noise? Journalof the Acoustical Society of America, 1965, 37,1102-1109.

Emmerich, D. S., Gray, J. L., Watson, C. S., &Tanis, D. C. Response latency, confidence, andROCs in auditory signal detection. Perception andPsychophysics, 1972, 11, 65-72.

Green, D. M. General prediction relating yes-no andforced choice results. Journal of the AcousticalSociety of America, 1964, 36, 1042.

Green, D. M., & Moses, F. L. On the equivalenceof two recognition measures of short-term memory.Psychological Bulletin, 1966, 65, 228-234.

Green, D. M., & Swets, J. A. Signal detectiontheory and Psychophysics. New York: Wiley,1966.

Gourevitch, V., & Galanter, E. A significance testfor one parameter isosensitivity functions. Psycho-metrika, 1967, 32, 25-33.

Grier, J. B. Nonparametric indexes for sensitivityand bias: Computing formulas. PsychologicalBulletin, 1971, 75, 424-429.

Harvey, N., & Treisman, A. M. Switching attentionbetween the ears to monitor tones. Perception andPsychophysics, 1973, 14, 51-59.

Healy, A. F., & Jones, C. Criterion shifts in recall.Psychological Bulletin, 1973, 79, 335-340.

Hillyard, S. A., Squires, K. C., Bauer, J. W., &Lindsay, P. H. Evoked potential correlates ofauditory signal detection. Science, 1971, 172, 1357-1360.

Hodos, W. Nonparametric index of response bias foruse in detection and recognition experiments.Psychological Bulletin, 1970, 74, 351-354.

Licklider, J. C. R. Three auditory theories. In S.

Koch (Ed.) Psychology: A study of a science.Vol. 1. New York: McGraw-Hill, 1959.

Luce, R. D. Detection and recognition. In R. D.Luce, R. R. Bush, & E. Galanter (Eds.), Hand-book of mathematical psychology. New York:Wiley, 1963.

Lutz, W. J., & Scheirer, C. J. Encoding processesin pictures and words. Journal of Verbal Learn-ing and Verbal Behavior, 1974, 13, 316-320.

Markowitz, J., & Swets, J. A. Factors affecting theslope of empirical ROC curves: Comparison ofbinary and rating responses. Perception and Psy-chophysics, 1967, 2, 91-100.

Moray, N. Time sharing in auditory perception:Effect of stimulus duration. Journal of the Acous-tical Society of America, 1970, 47, 660-661.

Murdock, B. B., Jr. The criterion problem in short-term memory. Journal of Experimental Psychol-ogy, 1966, 72, 317-324.

Ogilvie, J. C., & Creelman, C. D. Maximum-likeli-hood estimation of receiver operating character-istic curve parameters. Journal of MathematicalPsychology, 1968, 5, 377-391.

Parks, T. E. Signal-detectability theory of recogni-tion-memory performance. Psychological Review,1966, 73, 44-58.

Pastore, R. E., & Sorkin, R. D. Simultaneous two-channel signal detection. I. Simple binaural stim-uli. Journal of the Acoustical Society of America,1972, 51, 544-551.

Peterson, W. W., Birdsall, T. G., & Fox, W. C. Thetheory of signal detectability. Transactions IREProfessional Group on Information Theory, 1954,4, 171-212.

Pike, R. Response latency models for signal detec-tion. Psychological Review, 1973, 80, 53-68.

Pollack, I., & Hsieh, R. Sampling variability of thearea under the ROC-curve and of d'',. Psycho-logical Bulletin, 1969, 71, 161-173.

Pollack, L, & Norman, D. A. A non-parametricanalysis of recognition experiments. PsychonomicScience, 1964, 1, 125-126.

Pollack, L, Norman, D. A., & Galanter, E. Anefficient non-parametric analysis of recognitionmemory. Psychonomic Science, 1964, 1, 327-328.

Rilling, M., & McDiarmid, C. Signal detection infixed-ratio schedules. Science, 1965, 148, 526-527.

Sorkin, R. D., Pastore, R. E., & Pohlmann, L. D.Simultaneous two-channel signal detection. II.Correlated and uncorrelated signals. Journal of theAcoustical Society of America, 1972, 51, 1960-1965.

Suboski, M. D. Signal detection methods in theanalysis of classical and instrumental discrimina-tion conditioning experiments. Proceedings of the75th Annual Convention of the American Psycho-logical Association, 1967, 2, 37-38.

Sutton, S. Fact and artifact in the psychology ofschizophrenia. In M. Hammer, K. Salzinger, & S.Sutton (Eds.), Psychopathology. New York:Wiley, 1972.

Swets, J. A. (Ed.). Signal detection and recogni-tion by human observers. New York: Wiley, 1964.

958 R. E. PASTORE AND C. J. SCHEIRER

Swets, J. A. The relative operating characteristic inpsychology. Science, 1973, 182, 990-1000.

Swets, J. A., Tanner, W. P., Jr., & Birdsall, T. G.Decision processes in perception. PsychologicalReview, 1961, 68, 301-340.

Tanner, W. P., Jr. Theory of recognition. Journalof the Acoustical Society of America, 1956, 28,882-888.

Tanner, W. P., Jr. Physiological implications ofpsychophysical data. Annals of the New York ̂Academy of Sciences, 1961, 89, 7S2-76S. *

Taylor, M. M., Lindsay, P. H., & Forbes, S. M.Quantification of shared capacity processing inauditory and visual discrimination. Ada, Psycho-logica, 1967, 27, 223-229.

Van Meter, D., & Middleton, D. Modern statisticalapproaches to reception in communication theory.Transactions IRE Professional Group on Informa-tion Theory, 1954, 4, 119-141.

Wald, A. Statistical decision functions. New York:Wiley, 19SO.

' (Received January 16, 1974)