After online publication, subscribers (personal/institutional) to this journal will have access to the complete article via the DOI using the URL: If you would like to know when your article has been published online, take advantage of our free alert service. For registration and further information, go to: http://www.springerlink.com. Due to the electronic nature of the procedure, the manuscript and the original figures will only be returned to you on special request. When you return your corrections, please inform us, if you would like to have these documents returned. Dear Author Here are the proofs of your article. • You can submit your corrections online, via e-mail or by fax. • For online submission please insert your corrections in the online correction form. Always indicate the line number to which the correction refers. • You can also insert your corrections in the proof PDF and email the annotated PDF. • For fax submission, please ensure that your corrections are clearly legible. Use a fine black pen and write the correction in the margin, not too close to the edge of the page. • Remember to note the journal title, article number, and your name when sending your response via e-mail or fax. • Check the metadata sheet to make sure that the header information, especially author names and the corresponding affiliations are correctly shown. • Check the questions that may have arisen during copy editing and insert your answers/corrections. • Check that the text is complete and that all figures, tables and their legends are included. Also check the accuracy of special characters, equations, and electronic supplementary material if applicable. If necessary refer to the Edited manuscript. • The publication of inaccurate data such as dosages and units can have serious consequences. Please take particular care that all such details are correct. • Please do not make changes that involve only matters of style. We have generally introduced forms that follow the journal’s style. • Substantial changes in content, e.g., new results, corrected values, title and authorship are not allowed without the approval of the responsible editor. In such a case, please contact the Editorial Office and return his/her consent together with the proof. • If we do not receive your corrections within 48 hours, we will send you a reminder. • Your article will be published Online First approximately one week after receipt of your corrected proofs. This is the official first publication citable with the DOI. Further changes are, therefore, not possible. • The printed version will follow in a forthcoming issue. Please note http://dx.doi.org/10.3758/s13423-010-0017-1
Transcript
After online publication, subscribers (personal/institutional) to this journal will haveaccess to the complete article via the DOI using the URL:
If you would like to know when your article has been published online, take advantageof our free alert service. For registration and further information, go to:http://www.springerlink.com.
Due to the electronic nature of the procedure, the manuscript and the original figureswill only be returned to you on special request. When you return your corrections,please inform us, if you would like to have these documents returned.
Dear Author
Here are the proofs of your article.
• You can submit your corrections online, via e-mail or by fax.
• For online submission please insert your corrections in the online correction form.
Always indicate the line number to which the correction refers.
• You can also insert your corrections in the proof PDF and email the annotated PDF.
• For fax submission, please ensure that your corrections are clearly legible. Use a fine
black pen and write the correction in the margin, not too close to the edge of the page.
• Remember to note the journal title, article number, and your name when sending your
response via e-mail or fax.
• Check the metadata sheet to make sure that the header information, especially author
names and the corresponding affiliations are correctly shown.
• Check the questions that may have arisen during copy editing and insert your
answers/corrections.
• Check that the text is complete and that all figures, tables and their legends are included.
Also check the accuracy of special characters, equations, and electronic supplementary
material if applicable. If necessary refer to the Edited manuscript.
• The publication of inaccurate data such as dosages and units can have serious
consequences. Please take particular care that all such details are correct.
• Please do not make changes that involve only matters of style. We have generally
introduced forms that follow the journal’s style.
• Substantial changes in content, e.g., new results, corrected values, title and authorship are
not allowed without the approval of the responsible editor. In such a case, please contact
the Editorial Office and return his/her consent together with the proof.
• If we do not receive your corrections within 48 hours, we will send you a reminder.
• Your article will be published Online First approximately one week after receipt of your
corrected proofs. This is the official first publication citable with the DOI. Further
changes are, therefore, not possible.
• The printed version will follow in a forthcoming issue.
40 Abstract The theoretical framework of General Recognition Theory (GRT; Ashby & Townsend, 1986) coupled with the empirical analysis tools of Multidimensional Signal Detection Analysis (MSDA; Kadlec & Townsend, 1992) have become one important method for assessing dimensional interactions in perceptual decision-making. In this article, we critically examine MSDA and characterize cases where it is unable to discriminate two kinds of dimensional interactions, perceptual separability, and decisional separability. We performed simulations with known instances of violations of perceptual or decisional separability, applied MSDA to the data generated by these simulations, and evaluated MSDA on its ability to accurately characterize the perceptual versus decisional source of these simulated dimensional interactions. Critical cases of violations of perceptual separability are often mischaracterized by MSDA as violations of decisional separability.
10 Abstract The theoretical framework of General Recogni-11 tion Theory (GRT; Ashby & Townsend, 1986) coupled12 with the empirical analysis tools of Multidimensional13 Signal Detection Analysis (MSDA; Kadlec & Townsend,14 1992) have become one important method for assessing15 dimensional interactions in perceptual decision-making. In16 this article, we critically examine MSDA and characterize17 cases where it is unable to discriminate two kinds of18 dimensional interactions, perceptual separability, and de-19 cisional separability. We performed simulations with20 known instances of violations of perceptual or decisional21 separability, applied MSDA to the data generated by these22 simulations, and evaluated MSDA on its ability to23 accurately characterize the perceptual versus decisional24 source of these simulated dimensional interactions. Critical25 cases of violations of perceptual separability are often26 mischaracterized by MSDA as violations of decisional27 separability.
28 Keywords GRT.MSDA . Decisional . Perceptual
29 How are dimensions of a stimulus combined and used to30 make a perceptual decision? Are dimensions processed31 independently or do they interact, and if so, how? This32 fundamental question has been asked for a broad range of33 domains, including simple perceptual stimuli (e.g.,Q1 Shepard,34 1964), faces (e.gQ2 ., Richler, Gauthier, Wenger, & Palmeri,35 2008; Thomas, 2001; Wenger & Ingvalson, 2002), multi-36 modal perception-action (e.g., Amazeen & DaSilva, 2005)
37and visual-haptic stimuli (e.g., Oberle & Amazeen, 2003),38and social perception (e.g., Farris, Viken, & Treat, 2010).39A central issue of characterizing dimensional interactions is40distinguishing between interactions at perceptual or decisional41levels. For example, faces are widely believed to be processed42holistically, such that a whole face is recognized without43explicit recognition of face parts. Holistic processing effects44suggest that the different dimensions of a face (nose, eyes,45mouth, etc.) are combined, but at what level does this46interaction occur? Are the face dimensions encoded into a47holistic perceptual representation (e.g., Hole, 1994; Young,48Hellawell, & Hay, 1987) or are the face dimensions encoded49independently at the perceptual level but interact at a later50decisional stage (e.g., Wenger & Ingvalson, 2002, 2003)?51Understanding not only that stimulus dimensions interact but52also how they interact provides insight into the processes53underlying perceptual decision-making.54The theoretical framework of General Recognition55Theory (GRT; Ashby & Townsend, 1986) coupled with56the empirical analysis tools of Multidimensional Signal57Detection Analysis (MSDA; Kadlec & Townsend, 1992)58has become one important method for assessing dimen-59sional interactions. In this article, we critically examine60MSDA and characterize cases where it is unable to61determine the nature of certain kinds of dimensional62interactions. After briefly reviewing GRT and MSDA, we63investigate the application of MSDA through a series of64simulations. Known perceptual or decisional dimensional65interactions are embedded in these simulations, MSDA is66then applied to the data generated by these simulations, and67MSDA is evaluated on its ability to accurately characterize68the perceptual versus decisional source of the simulated69dimensional interactions. We observed that perceptual70interactions are often mischaracterized by MSDA as71decisional interactions.
M. L. Mack : J. J. Richler : I. Gauthier : T. J. Palmeri (*)Psychology Department, Vanderbilt University,PMB 407817, 2301 Vanderbilt Place,Nashville, TN 37240-7817, USAe-mail: [email protected]
Psychon Bull RevDOI 10.3758/s13423-010-0017-1
JrnlID 13423_ArtID 17_Proof# 1 - 27/10/2010
AUTHOR'S PROOF
UNCORRECTEDPROOF
72 Dimensional interactions in general recognition theory
73 GRT (Ashby & Townsend, 1986) is a multidimensional74 generalization of classic signal detection theory (SDT;75 Green & Swets, 1966), offering a rigorous theoretical76 framework for investigating dimensional interactions. Like77 SDT, GRT assumes that perception is inherently noisy. In78 SDT, perceptual effects are represented by univariate79 normal distributions of percepts. GRT extends perceptual80 effects to a multidimensional perceptual space, with stimuli81 represented by multivariate probability distributions.82 Figure 1a illustrates the distributions for four stimuli83 defined by two dimensions, A and B, which can each take84 on one of two possible levels, 1 and 2; for purposes of85 notation, as an example, level 1 along dimension B will86 be denoted B1, and a stimulus that has level 2 along87 dimension A and level 1 along dimension B will be88 denoted A2B1. The vertical dimension reflects the likeli-89 hood that a physical stimulus will be perceived as some90 combination of the two perceptual dimensions. Decision91 bounds, represented by dotted lines in Fig. 1, parse the92 space into different response regions. These boundaries93 can be linear or nonlinear. They can be orthogonal or94 nonorthogonal to the axes of the representational dimen-95 sions. To simplify the visual representation of these96 multidimensional distributions, we can draw contours of97 equal likelihood, which are cross sections of the distribu-98 tions at some particular likelihood (Fig. 1b), thereby99 illustrating variance along each individual dimension and100 covariance between dimensions. In Fig. 1b, decision101 boundaries are represented by dotted lines that define four102 response regions.103 Within GRT, dimensional interactions in multidimen-104 sional stimuli can be characterized by either perceptual or
105decisional factors (Ashby & Townsend, 1986; Kadlec &106Townsend, 1992). Specifically, dimensional interactions107can have a perceptual locus as violations of Perceptual108Independence (PI) or violations of Perceptual Separability109(PS), or a decisional locus as violations of Decisional110Separability (DS). Examples of how these GRT constructs111can be violated are illustrated in Fig. 2, and we will now112discuss each in turn.113Stimulus dimensions are perceptually independent when114the perceptual effect of one dimension is statistically115independent of the perceptual effect of another dimension.116When PI is satisfied, variability in the perception of117dimension A is uncorrelated with variability in the118perception of dimension B, as illustrated by the circular119equal likelihood contours in Fig. 1b. PI is violated when the120two perceptual dimensions of a stimulus are correlated, as121reflected by the diagonal ellipses in Fig. 2a. In this case,122some intrinsic property of perceptual processing gives rise123to correlated noise across the two dimensions. Unlike124violations of PS and DS, PI is a within-stimulus effect, in125that it can be observed in a single stimulus.126Stimulus dimensions are perceptually separable when the127distribution of perceptual effects for one dimension does128not vary across levels of the other dimension. If PS holds,129the distribution of the perceived A dimension is unaffected130by the level along dimension B, as illustrated by the131perceptual distributions forming a rectangle in Fig. 1b. PS132is violated when the perception of one dimension depends133on the level of the other dimension, which could be134reflected in the mean or the variance or both, as illustrated135by a non-rectangular arrangement of perceptual distribu-136tions in Fig. 2b. In this case, the perception of A2 depends137on whether the stimulus has value B1 or B2 along138dimension B.
B2
B1
A1 A2
A) B)
Fig. 1 Panel a illustrates the distribution for four stimuli in twodimensions. The third dimension reflects the likelihood that a physicalstimulus will be perceived as some combination of the twodimensions. The two dotted lines parallel to the dimensions represent
decision boundaries. Panel b illustrates a simplified representation ofthese multidimensional distributions as contours of equal likelihoodand decision boundaries that carve the space into different responseregions
Psychon Bull Rev
JrnlID 13423_ArtID 17_Proof# 1 - 27/10/2010
AUTHOR'S PROOF
UNCORRECTEDPROOF
139 Finally, responses to each dimension of a stimulus are140 decisionally separable when the location of the boundary141 for making a decision about one dimension does not142 depend on the level of the other dimension. For example,143 if DS holds, the boundary used for decisions about144 dimension A is in the same location irrespective of the145 level of dimension B, as illustrated in Fig. 1b. When DS is146 violated, the location of the decision boundary for one147 dimension depends on the level of the other dimension, as148 illustrated in Fig. 2c. For example, if DS is violated,149 participants might be biased to respond that dimension B150 has one level versus another level depending on the level of151 dimension A of the stimulus.152 The GRT framework offers a fine-grained approach to153 considering qualitatively different kinds of dimensional154 interactions. Of particular interest is the insight from GRT155 that dimensional interactions that are observed during what156 is ostensibly a perceptual task could reflect interactions that157 are taking place at a perceptual level, decisional level, or158 both. Applying GRT to empirical data to uncover percep-159 tual and decisional loci of dimensional interactions has160 been performed using two main approaches. One approach161 involves fitting models to observed data that impose162 parameter constraints that implement particular violations163 of GRT constructs (e.g., Ashby & Lee, 1991; Macho, 2007;164 Maddox, 2001; Maddox & Bogdanov, 2000; Thomas,165 2001;Q3 Wickens, 1992). Analysis and comparison of these166 models permits inferences about which GRT constructs167 hold and which are violated for a given task and stimulus168 set.169 Here we focus our analysis on the second approach,170 called Multidimensional Signal Detection Analysis171 (MSDA; Kadlec & Townsend, 1992). MSDA is a172 statistical toolbox that implements a series of theorems173 that can be used to make inferences about violations of174 GRT constructs. While developed over a decade ago, this175 toolbox has gradually been gathering users doing
176research across a wide range of domains. What follows177is a summary of MSDA, followed by a series of178simulations to test the inferential validity of MSDA.179Our focus is on a key inferential limitation and its180impacts on distinguishing perceptual versus decisional181loci of dimensional interactions.
182Multidimensional signal detection analysis
183MSDA consists of a set of theorems about the relationship184between observed response probabilities and the latent185perceptual representations and decisional processes embod-186ied in GRT (Kadlec & Townsend, 1992). An array of187statistical tests determines whether empirical data satisfy188these theorems, thereby allowing inferences about viola-189tions of PI, PS, and DS. MSDAwas originally developed in190the context of experimental paradigms using simple feature-191present/feature-absent stimulus dimensions (Kadlec &192Townsend, 1992; Kadlec & Hicks, 1998). However, MSDA193has since been applied to a far wider range of paradigms to194understand face recognition (Richler et al., 2008; Wenger &195Ingvalson, 2002, 2003), perception-action coupling196(Amazeen & DaSilva, 2005), visual-haptic interactions197(Oberle & Amazeen, 2003), and social perception (Farris,198Viken, & Treat, 2010). Furthermore, MSDA is the method199of analysis advocated by Macmillan and Creelman (2005)200for multidimensional experimental designs.201The statistical tests in MSDA are conducted at two levels202of analysis: marginal and conditional. Here we focus on203inferences about violations of PS and DS that are assessed204with marginal analyses. These include (a) a test of marginal205response invariance and (b) tests of equivalence of marginal206d’ and marginal beta values. The test of marginal response207invariance evaluates whether the probability of correctly208reporting the level of one dimension is independent of the209level of the other dimension; for example, is the probability
Violation of PI Violation of PS Violation of DS
B2
B1
A1 A2
B2
B1
A1 A2
B2
B1
A1 A2
A) B) C)
Fig. 2 Schematic of violations of GRT constructs (PI (a), PS (b), and DS (c)). Perceptual distributions are represented by equal likelihoodcontours and decision boundaries by dashed lines
Psychon Bull Rev
JrnlID 13423_ArtID 17_Proof# 1 - 27/10/2010
AUTHOR'S PROOF
UNCORRECTEDPROOF
210 of correctly reporting that dimension A has level A1
211 independent of whether dimension B has level B1 or B2?212 The tests of marginal equivalence compare differences213 between signal detection parameters d’ or beta for each214 level of one dimension collapsed across both levels of the215 other dimension; for example, one marginal test compares216 d’ when dimension A has level A1 versus level A2
217 collapsed across both levels of dimension B.218 The statistical tests of MSDA are related to GRT219 constructs through a set of theorems and propositions220 outlined by Kadlec and Townsend (1992). We will briefly221 review the relevant propositions regarding PS and DS.222 According to Proposition 1a, PS holds for a dimension if223 marginal d’ values are equal across the levels of the other224 dimension. However, as described in Proposition 1b,225 equivalent marginal d’ values does not imply PS since d’226 is a standardized difference in distribution means. Proposi-227 tion 1c summarizes the necessary conditions for concluding228 that PS holds for both dimensions: (i) equal variances of the229 marginal densities for one dimension across the levels of230 the other dimension, (ii) equivalence of marginal d’ for both231 dimensions across levels of the other dimensions, and (iii)232 the means of the perceptual distributions satisfy a Euclidean233 diagonal relationship. Figure 2b offers a simple illustration234 of a violation of this proposition: in this case, marginal d’235 for dimension A when dimension B has level B1 is not236 equal to marginal d’ for dimension A when dimension B237 has level B2, thus condition (ii) is not satisfied, thereby238 indicating a violation of PS.239 There are three important points to highlight about this240 proposition: (1) PS holds for both dimensions only when all241 three of these conditions are satisfied, (2) PS is assessed242 independently of DS, and (3) satisfying both conditions (ii)243 and (iii) requires a rectangular configuration of the244 perceptual distributions. One test of the rectangularity of a245 perceptual space, known as a diagonal d’ test, was initially246 suggested by Kadlec and Townsend (1992) and fully247 described by Kadlec and Hicks (1998). The test involves248 assessing the distances between the diagonally separated249 distributions in separate blocks (i.e., the distance between250 A1B1 and A2B2 versus the distance between A1B2 and251 A2B1); rectangular configurations will have equal diagonal252 distances. However, the diagonal d’ test is known to be253 inappropriate when PI is violated or when perceptual254 distributions have unequal variances (Thomas, 1995,255 1999, 2003).256 The necessary conditions for DS are described in257 proposition 2a and 2b of Kadlec and Townsend (1992).258 Proposition 2a states that if DS and PS hold for a259 dimension, the marginal betas for that dimension are equal260 across the levels of the other dimension. Figure 2c261 illustrates a violation of this proposition: The criterion262 value for dimension B depends on the level of dimension
263A, resulting in a difference in marginal betas, thereby264indicating a violation of DS. Unlike the direct test of PS,265the test of DS is indirect in that it depends on the status of266PS. This relationship is further clarified in proposition 2b267(i): If DS holds but PS fails, then it is not necessarily true268that marginal beta values for one dimension across the269levels of the other dimensions will be equal. In other words,270a difference in criterion values is consistent with a violation271of DS, but it does not logically follow that DS is actually272violated.273Two implications fall out of these propositions. The274more general implication is that applying MSDA’s inferen-275tial logic to empirical data is governed by the relationship276between DS and PS: Inferences about the status of DS277depend on whether PS is supported or rejected.278The second implication is that a violation of PS may279influence estimates of the decision criteria used to make280inferences about DS. At first blush, this seems to mean only281that the inference for PS must be considered before282assessing DS. Indeed, following the inferential logic283proposed by Kadlec and Townsend (1992, their Fig. 8, p.284352), when PS and marginal response invariance are285rejected, no inferences can be drawn about DS based on286marginal tests. This speaks to the asymmetry in MSDA’s287inferential logic; if PS and DS hold, marginal estimates288will be equivalent, but equivalent marginal estimates do289not necessarily indicate that PS and DS hold. Beyond this290general limitation of MSDA’s logic, another aspect of this291implication that is not universally recognized is that the292estimation of critical measures for assessing DS may be293influenced by any deviation in marginal d’ values,294regardless of whether statistical tests suggest that PS is295supported or PS is rejected. This may lead to erroneous296inferences about DS. Here we investigate in a series of297simulations whether violations of PS have a systematic298influence on the estimation of the decision criteria,299thereby influencing how MSDA draws inferences regard-300ing DS.
301Simulations
302Our tests of MSDA follow a straightforward logic: A303simulated space of distributions and decision boundaries are304created in a way that violates one specific GRT construct in305some qualitative way and by some quantitative degree. If306MSDA successfully uncovers that violation, and does not307erroneously uncover a violation that is not present, then308MSDA has made a successful inference; otherwise, it has309not.310Each simulation included four stimulus conditions. Each311stimulus condition was associated with a multivariate312normal distribution in two-dimensional space, as illustrated
Psychon Bull Rev
JrnlID 13423_ArtID 17_Proof# 1 - 27/10/2010
AUTHOR'S PROOF
UNCORRECTEDPROOF
313 earlier. Each simulation used a total of 2,000 trials.1 On314 each simulated trial, a random sample stimulus was drawn315 from one of the four distributions. Because normal316 distributions are used, a sample stimulus from any of the317 four stimuli distributions could be located in any of the four318 response regions defined by the decision boundaries. This319 results in a 4×4 confusion matrix, with each row a stimulus320 and each column a response. The resulting confusion321 matrix was analyzed with MSDA, as described below. We322 repeated the simulation and MSDA analyses 5,000 times323 for each space of distributions and decision boundaries.324 We conducted two versions of MSDA marginal statistical325 tests on the response probabilities in the simulated confusion326 matrix. The first followed the standard methods of estimating327 and comparing signal detection parameters and variances,328 like that outlined in Macmillan and Creelman (2005).329 Marginal response invariance was assessed by an equiva-330 lence test of probabilities of responding to a dimension331 across the levels of the other dimension. Violations of PS332 were assessed by differences in marginal d’ values for the333 relevant dimension. Violations of DS were assessed by334 differences in marginal c = –0.5[Φ-1(hit rate) + Φ-1(false335 alarm rate)], where Φ is the standard normal distribution336 function. Marginal c is used instead of marginal beta as an337 estimate of decision criteria because statistical equivalence338 tests exist for marginal c, but not for marginal beta.339 The second version of MSDA tests followed the340 methods of Kadlec (1995) using a Matlab implementa-341 tion of the MSDA_2 software (Kadlec, 1995, 1999); our342 Matlab implementation produces identical results to the343 original Pascal implementation of MSDA_2. We used344 MSDA_2 because it has become a common off-the-shelf345 tool for conducting MSDA analyses (Amazeen &346 DaSilva, 2005; Copeland & Wenger, 2006; Farris et al.,347 2010; Oberle & Amazeen, 2003; Richler et al., 2008;348 Wenger & Ingvalson, 2002, 2003). Unlike the standard349 method, MSDA_2 decision bounds are estimated by350 marginal crit = -Φ-1(false alarm rate), denoted henceforth351 by z(FAR).352 Each simulation began with a configuration representing353 no violations of PS or DS: the means of the multivariate354 normal distributions were equally spaced from neighboring355 distributions by 1 unit, the distributions were organized in a
356square configuration, all distributions had equal standard357deviation of 1 unit along each dimension and no covari-358ance, and decision boundaries equally separated the359distributions at the midpoint between the means along a360dimension, as illustrated in Fig. 1. Systematic violations of361GRT constructs were then introduced by systematically362varying parameters of the multivariate normal distributions363or decision boundaries (see Fig. 3). DS was violated by364shifting the decision boundary for one dimension depend-365ing on the level of the other dimension (Δc; Fig. 3a). PS366was violated in two different ways: by shifting the location367of one distribution along one dimension to increase the368marginal d’ for one value of the second dimension (Δd’;369Fig. 3b) and by shifting the location of two distributions370along one dimension to introduce a nonrectangularity in the371configuration of the distributions (Δd’; Fig. 3c). Each372construct (PS and DS) was investigated independently; for373example, when examining a violation of PS (Δd’ > 0), we374assumed no violation of DS (Δc = 0).375Simulation results are summarized in Fig. 3. Each of376the three simulated violations, depicted in the left column,377is a row in the figure. The results of the marginal tests,378using the standard method of estimating signal detection379parameters (d’, c, MRI) as well as the criterion value from380MSDA_2 (z(FAR)), are shown in the middle column as the381proportion of simulations that resulted in a significant382difference on the test. The right column summarizes the383inferential conclusions of MSDA both with the standard384method (black bars) and MSDA_2 (white bars). The plots385in the right column show the proportion of simulations386with the various combinations of PS and DS inferences for387the highest degree of the simulated violation used in the388middle column (e.g., in Fig. 3a, the right column panel389corresponds to the MSDA inferences when Δc = 0.4).390Notationally, the x-axis of the right column panels signify391all six possible combinations of inferences, with DS or PS392denoting no violation, ~DS or~PS denoting a violation,393and ?DS or ?PS denoting cases where inferences cannot be394made.395We first present simulations of violations that serve as a396simple test of MSDA and allow us to validate our397simulation methods. For a violation of DS (Fig. 3a), the398marginal tests of MSDA correctly inferred the nature of the399violations that produced the data. The marginal c and z400(FAR) tests showed significant differences that increased in401proportion with larger violations while marginal d’ tests402were unaffected. Following the inferential logic of MSDA,403the constant marginal d’ values infer support for PS and the404significant difference in marginal c and z(FAR) values infer405a violation of DS. This is reflected in the relatively large406proportion of correct “PS, ~DS” inferences in the plot in the407right column. Since the z(FAR) measure of MSDA_2 is408dependent only on the lower marginal distribution, it is less
1 The number of trials per simulation (2000) is similar to the numberof trials used in a recent study that employed MSDA in the context offace recognition (Richler et al., 2008). We also conducted simulationswith fewer trials (200, 400, and 1,000). In general, fewer trials persimulation led to fewer significant differences detected in all of thestatistical tests, as would be expected by the lower power of thesetests. Importantly, the relative proportion of significant differencesbetween the marginal tests and the inferences with regard to PS andDS were qualitatively similar to the results reported here forsimulations with 2,000 trials.
Psychon Bull Rev
JrnlID 13423_ArtID 17_Proof# 1 - 27/10/2010
AUTHOR'S PROOF
UNCORRECTEDPROOF
409 sensitive to the shift in criteria than marginal c. This results410 in fewer inferences of a violation of DS and more411 inferences of “PS, ?DS” (the status of DS cannot be412 inferred if PS holds, marginal c values are equivalent, and413 MRI does not hold [Kadlec & Townsend, 1992]); even so,
414MSDA_2 makes the correct inference in the largest415proportion of simulations.416For the first simulated violation of PS (Fig. 3b), the417proportion of significant differences in marginal d’ in-418creased with a larger violation, suggesting a violation of PS.
B2
B1
A1 A2
{
B2
B1
A1 A2
{{
B2
B1
A1 A2
{
0.0 0.1 0.2 0.3 0.40.0
0.2
0.4
0.6
0.8
1.0
d’
z(FAR)MRI
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion
of s
imul
atio
ns
PSDS
PS~DS
~PS?DS
~PS~DS
~PSDS
PS?DS
StandardMSDA_2
0.0 0.1 0.2 0.3 0.40.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
PSDS
PS~DS
~PS?DS
~PS~DS
~PSDS
PS?DS
MSDA_2
0.0 0.05 0.1 0.15 0.20.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
PSDS
PS~DS
~PS?DS
~PS~DS
~PSDS
PS?DS
MSDA_2
A)
C)
B)
Fig. 3 Results for simulated violations of DS and PS. The left columnillustrates the type of violation in each simulation in terms of thechange in perceptual distributions or decision criteria. The middlecolumn shows the results of the MSDA marginal tests. The proportionof iterations with a significant difference in the marginal responseinvariance test (gray line), marginal d’ test (black line), marginal c test(dashed line), and marginal z(FAR) test (dotted line) are plotted as
function of the size of the simulated violation (Δc or Δd’). The rightcolumn summarizes the inferential conclusions of both the standardmethod of MSDA (black bars) and MSDA_2 (white bars; Kadlec,1995, 1999). The plots show the proportion of simulations with thevarious combinations of PS and DS inferences (“~” indicates aviolation, “?” indicates that an inference cannot be made) for thehighest degree of the simulated violation
Psychon Bull Rev
JrnlID 13423_ArtID 17_Proof# 1 - 27/10/2010
AUTHOR'S PROOF
UNCORRECTEDPROOF
419 Note that the proportion of significantly different marginal420 c values matched that of the measures for detecting PS421 violations; this is consistent with the known relationship422 between violations of PS and certain estimates of decision423 criteria (Kadlec & Townsend, 1992; proposition 2b).424 MSDA includes the necessary logic to manage this425 relationship; PS is violated and marginal response invari-426 ance does not hold, so no inferences about DS can be427 drawn. The marginal z(FAR) measure in MSDA_2 is not428 affected by the violation of PS. In this simulation, both429 versions of MSDA correctly report that PS is violated (~PS)430 and the status of DS is unknown (?DS).431 The simulation above shows that differences in marginal432 d’ values can introduce an artifact in the estimates of433 marginal c, such that when PS is violated, DS cannot be434 assessed. We next show that this same artifact in estimating435 decision criteria can occur when PS is violated but MSDA436 fails to detect that violation. This leads to an erroneous437 inference that a violation of DS is present, when it is not.438 Figure 3c illustrates the other simulated violation of PS,439 which is a version of mean-shift integrality (Maddox,440 1992).2 The relative distance between the perceptual441 distributions along a dimension at the two levels of the442 other dimension are equivalent, but there exists a (mean) shift443 in the representations depending on the level of a dimension:444 the representation of one dimension depends on the level of445 the other dimension. Note that the decision boundaries used in446 this set of simulations remain constant across the levels of the447 two dimensions. So, in these simulations, we know that PS is448 violated and DS holds. It has been well documented that449 standard application of MSDA as originally proposed by450 Kadlec and Townsend (1992) is incapable of dealing with451 mean-shift integrality. Without a test of the rectangular452 configuration of the perceptual distributions, mean-shift453 integrality goes undetected. To be clear, the propositions of454 MSDA clearly define mean-shift integrality as a violation of455 PS (Kadlec & Townsend, 1992). The limitation is in456 detecting this violation when applying MSDA to empirical457 data. As expected, when the simulated data were analyzed458 using both methods of MSDA, marginal d’ values were459 constant across the magnitude of the simulated violation and460 PS is erroneously inferred, as reflected in the right panel of461 Fig. 3c.
462What about DS? As the size of the simulated mean shift463increases, the proportion of significant differences in464marginal c and z(FAR) values also increases, as illustrated465in the figure. According to the propositions underlying466MSDA, we expect violations of PS to create significant467differences in the test of marginal c values. Similarly, a shift468in the mean of the lower marginal distribution will create a469significant differences in the test of marginal z(FAR). These470simulations emphasize that a mindset that might be adopted471using unidimensional signal detection theory should not be472applied to the multidimensional case. Here, the significant473changes in criterion, marginal c and z(FAR), do not reflect a474true decisional effect, but are artifacts caused by an475underlying violation of PS.476Moreover, with a straight application of MSDA, without477a test for mean shift integrality, finding that PS holds (in478this case erroneously) and that there is a significant479difference in marginal criterion values, implies that there480is a violation of DS (also erroneous). With the standard481method version of MSDA, the nonrectangular configuration482of perceptual distributions goes undetected and an incorrect483inference about a violation of DS occurs on approximately48490% of the simulations, as show in the right panel of485Fig. 3c. MSDA_2 makes the incorrect inference about a486violation of DS on approximately 55% of the simulations,487again because of its less sensitive criterion measure. For488both versions of MDSA, the appropriate inference (“~PS, ?489DS”) occurred in only 5% of simulations.
490General discussion
491There has been growing interest in characterizing percep-492tual versus decisional components of dimensional interac-493tions in a wide variety of domains, ranging from494multimodal interactions to face recognition to social495perception (e.g., Amazeen & DaSilva, 2005; Copeland &496Wenger, 2006; Farris et al., 2010; Oberle & Amazeen,4972003; Richler et al., 2008; Wenger & Ingvalson, 2002,4982003). This work has used a statistical technique called499Multidimensional Signal Detection Analysis (MSDA;500Kadlec & Townsend, 1992) to characterize perceptual501versus decisional loci using constructs from General502Recognition Theory (GRT; Ashby & Townsend, 1986).503We reported simulations that highlight a significant infer-504ential limitation of MSDA that has been underappreciated505in its application to distinguishing perceptual versus506decisional sources of dimensional interactions.507The key focus of our critique was a form of dimensional508interaction called mean shift integrality (e.g., see Maddox,5092001), a violation of perceptual separability in the language510of GRT. It has been long acknowledged that the standard511application of MSDA, including the widely used MSDA_2
2 We also conducted simulated violations of PS caused by differencesin variance along particular values of dimensions, keeping meansconstant. While the MSDA propositions have conditions that mandateequal variances and covariances, testing these conditions are not partof standard MSDA analyses and are rarely tested in practice. About60% of simulations correctly inferred violations of PS, despite the factthat the MSDA analyses are not designed specifically to pick upviolations that might be caused by differences in variance. Theremaining simulations inferred no violation of PS, with about 20%inferring ~DS and 10% each inferring DS or ?DS.
Psychon Bull Rev
JrnlID 13423_ArtID 17_Proof# 1 - 27/10/2010
AUTHOR'S PROOF
UNCORRECTEDPROOF
512 toolkit (Kadlec, 1999), does not include tests for mean shift513 integrality in its inferential logic. On its own, this could514 simply mean that some violations of PS might go515 undetected if only MSDA were used. However, inferences516 about DS depend entirely upon whether valid inferences517 about PS are made. According to the propositions under-518 lying MSDA, if PS is violated, then no valid inferences519 about DS can be made. Therefore, if violations of PS go520 undetected, erroneous inferences about violations of DS can521 be the result.522 This is what happens in simulated cases of mean shift523 integrality. Differences in the location of the perceptual524 distributions introduce an artifact in estimates of decision525 criteria. This mean shift goes undetected by tests of526 marginal d’ values but leads to a significant difference in527 marginal c and z(FAR) values. PS is violated but goes528 undetected; DS is not violated, but an erroneous violation529 of DS is inferred because of the significant difference in530 criterion. Failing to detect mean shift integrality that is531 present is not simply a matter of failing to characterize a532 potentially important perceptual locus of dimensional533 interactions. Failing to detect mean shift integrality that is534 present can lead to erroneous inferences that a decisional535 locus of dimensional interactions exists when it does not.536 We have concentrated on a somewhat idealized version537 of mean-shift integrality where the mean shift for both538 distributions along one value of a dimension is equivalent.539 However, the problem we are describing is not limited to540 this special case. Any shift in the means of the marginal541 distributions, equivalent across distributions or not, can542 introduce an artifact in the estimation of decision criteria.543 When the mean difference goes undetected (e.g., under-544 powered analyses, small effect, high variability), the545 inference for DS will be confounded.546 Both of the MSDA methods we tested assessed PS with547 tests of marginal d’ and MRI without any test of the548 rectangularity of the perceptual distributions, so the strength549 of the inferences that can be made about PS, and hence DS550 as well, are limited (Kadlec & Townsend, 1992). As noted551 earlier, a diagonal d’ test has been proposed as an additional552 constraint on assessing PS (Kadlec & Hicks, 1998).553 However, this test requires the assumption of a distance554 classifier and has been shown to be invalid when perceptual555 distributions exhibit unequal or correlated variances across556 stimulus dimensions (Thomas, 1995, 1999, 2003). These557 assumptions are clearly inappropriate for any experimental558 setting. New tests are needed, not only to correctly559 characterize the full spectrum of violations of PS, but to560 allow valid inferences regarding DS as well.561 It is important to place our criticism in its appropriate562 context. We are not rejecting the theoretical framework of563 GRT (Ashby & Townsend, 1986) or MSDA (Kadlec &564 Townsend, 1992). GRT and the theoretical underpinnings of
565MSDA are sound. The main issue we have highlighted is a566breakdown in applying the propositions of MSDA. From567the theoretical perspective of MSDA, any violation of PS568prevents any inferences to be drawn about DS. Often,569differences in the location (or variances) of marginal570densities are not rigorously tested. Even if tested, care must571be taken to avoid the possibility that these differences go572undetected due to a variety of factors (e.g., small effect size,573too few data points, high variability), which could lead to574an erroneous inference that PS holds.575To our knowledge, this work is the first to document576problems with MSDA related to incorrect inferences577regarding DS driven by violations of PS. The limitations578of MSDA in inferring certain violations of PS per se have579been long known and acknowledged (e.g., Kadlec &580Townsend, 1992). However, when PS is violated but581remains undetected, following the propositional logic of582MSDA can lead to erroneous conclusions about DS.583Illustrating this problem seems particularly important584considering that the vast majority of studies that apply the585MSDA framework find evidence for violations of DS,586sometimes in cases where such violations seem counterin-587tuitive (Amazeen & DaSilva, 2005; Farris et al., 2010;588Oberle & Amazeen, 2003; Valdez & Amazeen, 2008;589Wenger & Ingvalson, 2002, 2003) including some of our590own work ( Q4Richler et al., 2008). All of these studies591employed MSDA methods similar to the approaches we592used here (Kadlec & Townsend, 1992; Kadlec, 1995,5931999), one of these studies included additional tests of594diagonal d’ (Wenger & Ingvalson, 2003), and a few595included converging model-fitting methods (Copeland &596Wenger, 2006; Cornes, Donnelly, Godwin, & Wenger,5972010; Valdez & Amazeen, 2008).598It is quite possible that many of these cases reflect true599violations of DS. There is converging evidence that certain600kinds of dimensional interactions that seem perceptual may601be caused by decisional factors (e.g., Cheung, Richler,602Palmeri, & Gauthier, 2008). Our research reported in this603paper does not discount the decisional results found using604MSDA. Instead, those inferences remain equivocal. The605critical problem is distinguishing true violations of DS from606violations of DS produced by artifacts. One alternative607direction is found in the method of fitting GRT models to608empirical data (e.g., Ashby & Lee, 1991; Macho, 2007;
Q5609Wickens, 1992) alongside drawing inferences with MSDA610(e.g., Thomas, 2001) to find converging evidence for the status611of GRT constructs. Unfortunately, such techniques often612require paradigms that demand significantly more data points613than those that have been typically analyzed using MSDA.614The recent use of MSDA in new domains (e.g., Farris et615al., 2010) and its recommendation in the latest edition of616Macmillan and Creelman’s Detection Theory: A User’s617Guide (2005) for designs that are aimed at assessing SDT in
Psychon Bull Rev
JrnlID 13423_ArtID 17_Proof# 1 - 27/10/2010
AUTHOR'S PROOF
UNCORRECTEDPROOF
618 multidimensional spaces, press for even greater awareness619 of the current limitations in applying MSDA in practice. We620 hope that this article will prompt further research into621 developing new inferential tools that will allow researchers622 to feel confident about making inferences regarding623 perceptual versus decisional loci of dimensional interac-624 tions using the language of GRT.625626 Acknowledgements This work was supported by the Temporal627 Dynamics of Learning Center (SBE-0542013), an NSF Science of628 Learning Center, and a grant from the James S. McDonnell629 Foundation.
630Q6 References631
632 Amazeen, E. L., & DaSilva, F. (2005). Psychophysical test for the633 independence of perception and action. Journal of Experimental634 Psychology: Human Perception and Performance, 31, 170–182.635 Ashby, F. G., & Lee, W. W. (1991). Predicting similarity and636 categorization from identification. Journal of Experimental637 Psychology: General, 120, 150–172.638 Ashby, F. G., & Townsend, J. T. (1986). Varieties of perceptual639 independence. Psychological Review, 93, 154–179.640 Cheung, O. S., Richler, J. J., Palmeri, T. J., & Gauthier, I. (2008).641 Revisiting the role of spatial frequencies in the holistic process-642 ing of faces. Journal of Experimental Psychology: Human643 Perception and Performance, 34, 1327–1336.644 Copeland, A. M., & Wenger, M. J. (2006). An investigation of645 perceptual and decisional influences on the perception of646 hierarchical forms. Perception, 35, 511–529.647Q7 Cornes, K., Donnelly, N., Godwin, H., & Wenger, M. J. (2010).648 Perceptual and decisional factors influencing the discrimination649 of inversion in the thatcher illusion. Journal of Experimental650 Psychology: Human Perception and Performance.651 Farris, C., Viken, R. J., & Treat, T. A. (2010). Perceived association652 between diagnostic and non-diagnostic cues of women’s sexual653 interest: General Recognition Theory predictors of risk for sexual654 coercion. Journal of Mathematical Psychology, 54(1), 137–149.655 Green, D. M., & Swets, J. A. (1966). Signal detection theory and656 psychophysics. New York: Wiley.657 Hole, G. J. (1994). Configurational factors in the perception of658 unfamiliar faces. Perception, 23, 65–74.659 Kadlec, H. (1995). Multidimensional signal detection analyses660 (MSDA) for testing separability and independence: A Pascal661 program. Behavior Research Methods, Instruments, & Com-662 puters, 27, 442–458.663 Kadlec, H. (1999). MSDA2: Updated version of software for664 multidimensional signal detection analyses. Behavior Research665 Methods, Instruments, & Computers, 31, 384–385.666 Kadlec, H., & Hicks, C. L. (1998). Invariance of perceptual spaces667 and perceptual separability of stimulus dimensions. Journal of
668Experimental Psychology: Human Perception and Performance,66924, 80–104.670Kadlec, H., & Townsend, J. T. (1992). Signal detection analysis of671dimensional interactions. In F. G. Ashby (Ed.), Multidimensional672models of perception and cognition (pp. 181–228). Hillsdale:673Lawrence Erlbaum Associates.674Macmillan, N. A., & Creelman, C. D. (2005). Detection theory: A675user's guide (2nd ed.). Mahwah: Lawrence Erlbaum Associates.676Macho, S. (2007). Feature sampling in detection: Implications for the677measurement of perceptual independence. Journal of Experimen-678tal Psychology: General, 136(1), 133–153.679Maddox, W. T. (1992). Perceptual and decisional separability. In680F. G. Ashby (Ed.), Multidimensional models of perception681and cognition (pp. 147–180). Hillsdale: Lawrence Erlbaum682Associates.683Maddox, W. T. (2001). Separating perceptual processes from684decisional processes in identification and categorization. Percep-685tion & Psychophysics, 63, 1183–1200.686Maddox, W. T., & Bogdanov, S. V. (2000). On the relation between687decision rules and perceptual representation in multidimensional688perceptual categorization. Perception & Psychophysics, 62, 984–689997.690Oberle, C. D., & Amazeen, E. L. (2003). Independence and691separability of volume and mass in the size-weight illusion.692Perception & Psychophysics, 65, 831–843.693Richler, J. J., Gauthier, I., Wenger, M. J., & Palmeri, T. J. (2008).694Holistic processing of faces: Perceptual and decisional compo-695nents. Journal of Experimental Psychology. Learning, Memory,696and Cognition, 34, 328–342.697Thomas, R. D. (1995). Gaussian general recognition theory and698perceptual independence. Psychological Review, 102, 192–200.699Thomas, R. D. (1996). Separability and independence of dimensions700in the same-different judgment task. Journal of Mathematical701Psychology, 40, 318–341.702Thomas, R. D. (1999). Assessing sensitivity in a multidimensional703space: Some problems and a definition of a general d’.704Psychonomic Bulletin & Review, 6, 224–238.705Thomas, R. D. (2001). Perceptual interactions of facial dimensions in706speeded classification and identification. Perception & Psycho-707physics, 63, 625–650.708Thomas, R. D. (2003). Further considerations of a general d’ in709multidimensional space. Journal of Mathematical Psychology,71047, 220–224.711Valdez, A., & Amazeen, E. (2008). Sensory and perceptual inter-712actions in weight perception. Perception & Psychophysics, 70,713647–657.714Wenger, M. J., & Ingvalson, E. M. (2002). A decisional component of715holistic encoding. Journal of Experimental Psychology. Learn-716ing, Memory, and Cognition, 28, 872–892.717Wenger, M. J., & Ingvalson, E. M. (2003). Preserving informational718separability and violating decisional separability in facial719perception and recognition. Journal of Experimental Psychology.720Learning, Memory, and Cognition, 29, 1106–1118.721Young, A. W., Hellawell, D., & Hay, D. C. (1987). Configurational722information in face perception. Perception, 16, 747–759.
723
Psychon Bull Rev
JrnlID 13423_ArtID 17_Proof# 1 - 27/10/2010
AUTHOR'S PROOF
UNCORRECTEDPROOF
AUTHOR QUERIES
AUTHOR PLEASE ANSWER ALL QUERIES.
Q1. “Shepard, 1964” is cited in text but not given in the reference list. Please provide details in thelist or delete the citation from the text.
Q2. Please check citation ‘Richler et al., 2008’ ‘a’ was deleted here ‘Richler et al., 2008a’.Q3. “Wickens, 1992” is cited in text but not given in the reference list. Please provide details in the
list or delete the citation from the text.Q4. “Richler et al., 2008” is cited in text but not given in the reference list. Please provide details in
the list or delete the citation from the text.Q5. “Wickens, 1992” is cited in text but not given in the reference list. Please provide details in the
list or delete the citation from the text.Q6. Following reference are not cited in text: Thomas RD. 1996.Q7. Please provide journal number and page range
![Tutorial: Bayesian statistics (part 1)bachlab.org/.../uploads/2019/...BayesianTutorial01.pdf · a Bayesian perspective. Psychonomic Bulletin & Review, 25(1), 178-206. [Lin00] Lindley,](https://static.documents.pub/doc/80x56/5ec87964c0e03e5e2d506b0a/tutorial-bayesian-statistics-part-1-a-bayesian-perspective-psychonomic-bulletin.jpg)
