European Journal of Social Psychology 00 (2015) 00-00 Copy
Abstract
Statistical tests of indirect effects can hardly distinguish between genuine andspurious mediation effects. The present research demonstrates, however,that mediation analysis can be improved by combining a significance test ofthe indirect effect with assessing the fit of causal models. Testing only the in-direct effect can be misleading, because significant results may also be ob-tained when the underlying causal model is different from the mediationmodel. We use simulated data to demonstrate that additionally assessingthe fit of causal models with structural equation models can be used to ex-clude subsets of models that are incompatible with the observed data. The re-sults suggest that combining structural equation modeling with appropriateresearch design and theoretically stringent mediation analysis can improvescientific insights. Finally, we discuss limitations of the structural equationmodeling approach, and we emphasize the importance of non-statisticalmethods for scientific discovery.
Experimental designs are commonly consideredthe major method for making causal inferences(e.g., Shadish, Cook, & Campbell, 2002). If an indepen-dent variable X is manipulated between randomizedexperimental conditions, variation in a dependent vari-able Y can be attributed to variation in the independentvariable. For example, if persons are randomly assignedto one of the two conditions, either receiving social sup-port or not, resulting differences in persons’ well-beingcan be assumed to reflect the impact of social support.However, this basic experimental approach is limitedbecause it does not explain how the independent vari-able affects the dependent variable. In pursuing a hypo-thetical answer to this question, a researcher may wantto investigate whether the effect of social support onwell-being comes about through changes in thepersons’ attribution style (e.g., the tendency to attributeproblems to uncontrollable factors). Testing such anexplanatory hypothesis is a case for mediation analysis(e.g., Baron & Kenny, 1986; Hayes, 2013; MacKinnon,2008). This method allows researchers to investigatewhether the empirical evidence is consistent with amediation model X→Z→Y, which states that the impact
of an independent variable X on a dependent variableY is (at least in part) causally mediated by a proposedmediator Z.Traditionally, statistical mediation analysis tests
whether there is a significant indirect effect of the in-dependent variable X via the proposed mediator Z onthe dependent variable Y. Over several decades, a va-riety of statistical procedures have been developed fortesting an indirect effect. One common procedure in-volves a series of models that regress the dependentvariable on the independent variable, with and with-out the mediator as a predictor (e.g., Baron & Kenny,1986; Judd & Kenny, 1981). The indirect effect ofthese regression models—that is, the effect of X onY mediated by Z—can be tested using bootstrap pro-cedures (e.g., Frazier, Tix, & Barron, 2004; Hayes &Scharkow, 2013; Preacher & Hayes, 2004) or para-metric significance tests (e.g., Baron & Kenny, 1986;Hayes, 2013; Sobel, 1982).However, applying these statistical approaches can be
misleading because a significant test result for a media-tor Z does not logically imply that Z is the true mediator.Any statistical test of the model X→Z→Y presupposes as
, Ltd.
D. Danner et al.Mediation analysis with structural equation models
an a priori premise that this is indeed the causalmodel oftheoretical interest. There is no logical reason to excludethat other mediators Z′ or Z″may provide a better expla-nation or that the three focal variables (X, Z, and Y) maybe related in different ways. In other words, researchersengaging in a test ofX→Z→Y rely on the a priori premisethat Z deserves to be considered the true mediator, andthey merely test whether this selected assumption canaccount for a significant part of the variance. As longas they do not test the fit of alternativemodels, theymustnot infer from a significant mediation test that they havefound the truemediator statistically (e.g., James, Mulaik,&Brett, 2006;MacCallum,Wegener, Uchino,& Fabrigar,1993; MacKinnon, 2008). As a consequence, a signifi-cant mediation test result provides necessary but not suf-ficient evidence for a hypothetical, selectively testedmediation model. A significant test of the mediationmodel X→Z→Y may be obtained even when theunderlying covariance structure is different (e.g., Fiedler,Schott, & Meiser, 2011; MacKinnon, Krull, & Lockwood,2000). Fiedler et al. (2011) demonstrated in Monte Carlosimulations that significant Sobel tests are regularly ob-tained when the third variable Z is not a true mediatorbut, for example, merely a correlate of the dependentvariableY (i.e.,when Z is not generated to reflect an influ-ence ofX but of Y). In our example, a person’s attributionstyle may not mediate social support but actually reflecta by-product of well-being. Such demonstrations of“mediation mimicry” imply that statistical tests can leadto wrong conclusions, because they do not discriminatebetween alternative causal structures that may also giverise to the observed covariance structure of X, Y, and Z.Hence, a more comprehensive analysis is needed tochoose between alternative models.The aim of the present research is to demonstrate sys-
tematically that combining theoretical considerations,study design, and statistical testing affords a means ofovercoming this fundamental weakness of mediationanalysis. First, we show that the number of possiblethree-variate structures (involving X, Z, and Y) in anexperimental design can be limited by a few sensibleconstraints. In particular, given the controlled manipu-lation of independent variable X in an experimentaldesign, so that the independent variable cannot beaffected by the dependent variable or the proposed me-diator (e.g., MacCallum et al., 1993; MacKinnon, 2008;Stone-Romero & Rosopa, 2011), there are only 12 pos-sible causal structures. Then, we simulate empirical datafor each causal structure and illustrate that a significantindirect effect may reflect not only a mediation struc-ture but also several alternative structures (Fiedleret al., 2011; MacKinnon et al., 2000). Using structuralequation modeling, we then demonstrate that there is
European J
a class of causal model that fits with the empirical dataand that there is a class of causal models that do not fitwith the data.Our paper adds to the existing literature by showing
which of the 12 causal structures can or cannot be dis-criminated from each other. It will be seen that a speci-fiable subset of causal models can be excluded asincompatible with the given data, thus reducing thenumber of viable candidates substantially. The founda-tion for this approach has been laid by several authors(e.g., Cole & Maxwell, 2003; Lee & Hershberger, 1990;MacCallum et al., 1993; MacKinnon, 2008), who sug-gest specifying alternative models by omitting or chang-ing paths in a structural equation model.
Imposing Constraints on Viable Causal Structures
To be sure, the problem that three variables may becausally related in several ways is a thorny one. Thereare a large number of different causal models that mayall potentially explain an observed covariation patternbetween the independent and dependent variablesand the proposed mediator. As already noted, onemethod to reduce the number of possible models is tomanipulate the independent variable X experimentally,which effectively rules out the possibility thatX can be af-fected by Y or Z. However, even in this genuinely exper-imental case, there still remains a variety of structuralmodels, in particular with regard to the relation of theproposed mediator to the dependent variable. Figure 1provides an overview of all possible models with refer-ence to the example of attribution as a potentialmediatorof the impact of social support on well-being.
(i) Independence model: The variables do not affecteach other. For example, persons are randomlyassigned to receive social support or no support,but this does not affect well-being or attributionstyle.
(ii) Single effect (X→Y): Social support affects well-being, but this effect is not mediated via the per-sons’ attribution style.
(iii) Single effect (X→Z): Social support affects attribu-tion style but not well-being.
(iv) Single effect (Z→Y): The persons’ attribution styleaffects their well-being. Social support does nothave an effect even though manipulatedexperimentally.
(v) Single effect (Y→Z): The persons’well-being causestheir attribution style. The manipulation of socialsupport does not have an effect.
(vi) Complete mediation (X→Z→Y): Social support af-fects the persons’ attribution style, which in turnaffects the persons’ well-being.
Fig. 1: Illustration of possible causal structures in an experimental setting
D. Danner et al. Mediation analysis with structural equation models
(vii) Common cause (X→Z, X→Y): Social support affectsthe persons’ attribution style as well as their well-being.
(viii) Common effect on Y (X→Y, Z→Y): Social support aswell as the persons’ attribution style influencesthe persons’ well-being. However, the treatmentdoes not affect the persons’ attribution style.
(ix) Reflection model (X→Y→Z): Social support affectsthe persons’ well-being, which changes the per-sons’ attribution style.
(x) Common effect on Z (X→Z, Y→Z): The persons’ at-tribution style is affected by social support as wellas by the persons’ well-being. However, socialsupport does not affect the persons’ well-being.
(xi) Partial mediation (X→Z→Y, X→Y): Social supportaffects the persons’ attribution style, which in turnaffects their well-being. In addition, social supportaffects the persons’ well-being directly.
(xii) Inverse mediation (X→Z, X→Y→Z): Social supportaffects the persons’well-being, which affects theirattribution style. In addition, social support affectsthe persons’ attribution style directly.
Structural Equation Modeling
Several researchers recommended structural equationmodeling as the preferred method for mediation analy-sis (e.g., Baron & Kenny, 1986; Frazier et al., 2004;Hoyle & Smith, 1994). One important reason is that
the unreliability of the mediator and the dependentvariable will attenuate systematic relationships inmulti-ple regression,whereas themediator and the dependentvariable may be separated from their measurement er-rors in structural equation modeling. Another reasonis that structural equation modeling is much more flex-ible than regression (e.g., it is quite easy to include mul-tiple mediators or dependent variables). This flexibilitybecomes a crucial point when a decision between differ-ent causal models is the aim of the analysis. In particu-lar, in multiple regression analysis, a causal model istranslated into a series of regression equations, and eachcoefficient has to be estimated and tested separately (asit is carried out with the series of three regression equa-tions in the mediation analysis of Baron & Kenny,1986). In contrast, structural equation modeling allows(i) estimating and testing the entire causal model and(ii) comparing different causal models using sophisti-cated goodness-of-fit statistics (see details as follows).Given a limited number of possible causal models by
the study design, structural equation modeling can beused to investigate the fit of alternative causal modelsand thereby help to reduce the number of viablemodels. In particular, structural equation models testwhether the constraints of specific models fit with theobserved data (e.g., Bollen, 1989; Hoyle, 1995; Kline,2011). This allows researchers to reject those modelsthat are incompatible with the given data and to identifythose models that have to be examined more closely as
, Ltd.
D. Danner et al.Mediation analysis with structural equation models
rivals of a hypothesized mediation model. For instance,if the observed data (generated in accordance with asimulated model) fit not only with a complete media-tion structure (X→Z→Y) but also with a reflectionmodel (X→Y→Z), further research must be designedto distinguish between these two models.Let us briefly elaborate on this example to further ex-
plain the logic of our approach. The complete mediationmodel (X→Z→Y) states that X is an exogenous variableandZ is a functionofX and a randomvariable υ of the formZ=a*X+υ. In addition, thismodel states thatY is a functionof Z and a randomvariableω, Y= b*Z+ω. If the variablesX, υ, and ω are not correlated, this implies that the var-iance σ2 of Z is a function of the variances of X and υ,σ2Z= a
2*σ2X+σ2υ, and that the variance of Y is a function
of the variances of Z and ω, σ2Y=b2*σ2Z+σ
2ω. Accord-
ingly, the covariance ρ between X and Y is ρX,Y=a*b*σ2X,
the covariance between X and Z is ρX,Z=a*σ2X, and the
covariance between Y and Z is ρY,Z=a2*b*σ2X+b*σ
2υ.
By comparison, the reflection model (X→Y→Z) statesthat X is an exogenous variable and Y is a function of Xand a random variable ω of the form Y= c*X+ω. In addi-tion, this model states that Z is a function of Y and a ran-dom variable υ, Z=b*Y+υ. Again, if the variables X, υ,and ω are not correlated, this implies that the varianceσ2 of Y is a function of the variances of X and ω,σ2Y= c
2*σ2X+σ2ω, and that the variance of Z is a function
of the variances ofY and υ, σ2Z=b2*σ2Y+σ
2υ. Accordingly,
the covariance betweenX and Y is ρX,Y= c*σ2X, the covari-
ance betweenX and Z is ρX,Z=b*c*σ2X, and the covariance
between Y and Z is ρY,Z=b*c2*σ2X+b*σ
2ω. Hence, the two
models imply a different structure of the variances andcovariances between X, Y, and Z.In sum, structural equation modeling allows compar-
ing rival causal models for given data (e.g., Cole &Maxwell, 2003; Lee & Hershberger, 1990; MacCallumet al., 1993; MacKinnon, 2008). It can thus elucidatewhich causal models can be distinguished by the givenempirical data and which models cannot (e.g., Jameset al., 2006; MacCallum et al., 1993; Stelzl, 1986). Ofcourse, structural equation modeling is a statistical ap-proach. It therefore cannot afford a final proof of a truecausal model (e.g., MacKinnon, 2008; Shadish et al.,2002; Stone-Romero & Rosopa, 2010). It is always pos-sible that other variables that have not been observed inthe experiment can account for the covariance of X, Y,and Z. Moreover, it is always possible that replacing Zby statistically related but psychologically different vari-ables Z′, Z″, and so onmay afford better solutions. How-ever, we use simulated data to illustrate that the presentapproach allows us to reduce the theoretical uncertaintyabout the causal structure within the trivariate theoreti-cal space spanned by X, Y, and Z.
European J
Aim of the Present Simulation
The present simulation aims at an empirical settingwhere the independent variable X was experimentallymanipulated and both the dependent variable Y andthe hypothesized mediator Z have been measured. Asargued previously, there are 12 possible causal struc-tures that may have generated the observed variancesand covariances of the three variables, and only two ofthese causal structures are mediation (complete andpartial, respectively). Using simulation techniques, wewill illustrate that a significance test of the indirecteffect alone will yield false decisions. This part of oursimulation aims to replicate and extend our knowl-edge about the fallibility of this method (as alreadydemonstrated by Fiedler et al., 2011; MacCallumet al., 1993). In a second step, we will illustrate thata combination of testing the indirect effect and struc-tural equation modeling (i.e., deciding in favor formediation only if the test of the indirect effect is signif-icant and if the mediation model can be accepted) canincrease the precision of the decision. This latter anal-ysis will go beyond what is presently known, and it ishoped that this will be a substantial improvement ofthe methodology.
METHOD
Simulating Data
To illustrate the present approach, we simulated 12classes of data sets, one class for each causal structurein Figure 1. Each class included 1000 data sets, andeach data set included N=200 observations. In eachdata set, we first generated the latent constructvariables: the independent variable X, the dependentvariable Y, and the proposed mediator Z. We gener-ated X as a binary variable (1=experimental condition,0= control condition). Depending on the causal struc-ture, we generated Y and Z as randomly distributed nor-mal variables (M=0, standard deviation [SD]=1) or, ifdependent on each other, as linear combinations of eachother plus randomly distributed normal variables (M=0,SD=1). For example, in the complete mediation datasets, we generated the independent variableX as a binaryvariable.We generated themediator variable Z as a com-bination of the independent variable X and a randomlydistributed normal variable υ (M=0, SD=1), Z=X+υ.Then, we generated the dependent variable as a combi-nation of themediator variable Z and a randomly distrib-uted normal variable ω (M=0, SD=1), Y=Z+ω. Thealgebraic specifications for all causal models are shownin Table 1.
Note: υi /ωi = normally distributed random variables (M = 0.00, standard
deviation = 1.00) and i = data set. There were 1000 data sets per class.
D. Danner et al. Mediation analysis with structural equation models
Second, we included three indicator variables for thedependent variable Y and three indicator variables forthe proposed mediator variable Z. Each indicator vari-able was computed as a linear combination of the latentconstruct variable and a normally distributed randomerror variable ε (M=0, SD=1). For example, the threeindicators for the dependent variable Y were generatedas Y1=Y+ ε1, Y2=Y+ ε2, and Y3=Y+ ε3. The algebraicspecifications for all indicator variables in all causalstructures are shown in Appendix A. For the structuralequationmodel analyses, we used the independent var-iable X, the three indicators for the proposed mediatorvariables Z1, Z2, and Z3, and the three indicators for thedependent variables Y1, Y2, and Y3.
1
1All parameters in the simulationwere fixed to be either zero or one. In
doing so, we followed Cohen (1990) who suggested using a simple-is-
better principle in multiple regression analyses. In particular, he sup-
posed to use unit weights in prediction equations with +1 when the
predictors are positively related and with 0 when they are poorly re-
lated. The rationale is that empirically estimated beta weights always
depend on the particular sample that was used for their estimation.
He argues that the precision of the prediction will be more likely to be-
come worse when empirically estimated betas are used instead of unit
weights (see Cohen, 1990, p. 1306 for an analytical demonstration of
this rule). In the absence of other constraints, we decided to apply this
simple-is-better principle in the present simulations in hope of a great
generalizability of our results to a wide range of empirical settings. In
the simulated data sets, the effect sizes for the direct effects between
the manifest variables (mean of the indicators) were r = .40. According
to Cohen (1990), these associations reflect medium to large effects and
hence can be seen as realistic for many experimental settings.
We tested the indirect effect with the Sobel test,bootstrapped confidence intervals (CI), and structuralequation models in each data set. The Sobel test is aregression approach designed for analyzing threemanifest variables (e.g., Baron & Kenny, 1986; Sobel,1982). First, the regression coefficient a of the pro-posed mediator Z on the independent variable Xand its standard error sa are estimated. Second, theregression coefficient b of the dependent variable Yon the proposed mediator Z and its standard error sbare estimated. Third, the indirect effect a*b and itsstandard error sa*b are computed (see Goodman,1960; MacKinnon et al., 2000; Preacher & Hayes,2004; Sobel, 1982 for a discussion of different formu-
las). Sobel’s Z is calculated by the formula Z ¼ a*bsa*b
,
whereby |Z|>1.96 indicates a significant indirect ef-fect. The analyses were based on the independentvariable X, the mean score of the three indicators ofthe dependent variable Y (Cronbach’s α= .65–.92),and the mean score of the three indicators of theproposed mediator Z (Cronbach’s α= .65–.93). Foreach data set, we estimated the indirect effect a*band its Z-value.Because the Sobel test has statistical limitations
(e.g., Hayes & Scharkow, 2013; Preacher & Hayes,2004; Shrout & Bolger, 2002), bootstrapping can beconsidered superior. Bootstrapping estimates the upperlimit and the lower limit of the CI of an indirect effect. ACI above zero indicates a significant positive indirecteffect. We computed bootstrapped 95% CI (2000bootstrap samples). For the present analysis, we used anonparametric bootstrap procedure with the SAS (SASInstitute, Cary, NC, USA) script provided by Preacherand Hayes (2004).In addition, we tested the indirect effect with
structural equation models (see details as follows).Within the structural equation modeling framework,the magnitude of an indirect effect was estimatedwith an iterative algorithm (e.g., maximum likeli-hood), and the significance was tested by dividingthe magnitude of the indirect effect by its standarderror, producing a standard normally distributed var-iable Z (e.g., Iacobucci, Saldanha, & Deng, 2007;Kline, 2011; MacKinnon, 2008). Testing indirect ef-fects with structural equation models has been ap-plied in different contexts (e.g., Ahearne, Mathieu,& Rapp, 2005; Hilliard et al., 2013; King, King, Fairbank,Keane, & Adams, 1998; Quilty, Godfrey, Kennedy, &Bagby, 2010).
, Ltd.
Fig. 2: Example of a complete mediation model with three indicators for the
proposed mediator Z and three indicators for the dependent variable Y
D. Danner et al.Mediation analysis with structural equation models
Testing the Model Fit with Structural Equation Models
Structural equation modeling can be used to testwhether a predefined structural model fits an observeddata set. In a first step, a covariance matrix of the ob-served data is estimated without any model assump-tions. In a second step, a covariance matrix of theobserved data is estimated given the predefined modelstructure (e.g., a complete mediation model X→Z→Y)by searching for model parameters that minimize thediscrepancy between the model-free estimated covari-ance matrix and the model-dependent covariance ma-trix of the observed variables. In a third step, themodel fit is computed by comparing both covariancematrices. A goodmodel fit indicates that there is no sub-stantial difference between the matrices.We used two criteria to evaluate the fit of the models:
the root mean square error of approximation (RMSEA)and the significance test of the estimated model param-eters. The RMSEA is a parsimonious fit index that takesthe discrepancy between the observed and the model-implied covariance matrices into account as well as themodel’s complexity. A smaller RMSEA indicates a bettermodel fit. Models with an RMSEA>0.06 should berejected (Hu & Bentler, 1999). Models containing non-significant parameter estimates (including the indirecteffect from X to Y) were rejected because all modelsare nested in either the partial mediation model or theinverse mediation model. In other words, a model con-taining a zero parameter is equivalent to another moreparsimonious model. For example, a partial mediationmodel with a zero path coefficient from the latent inde-pendent variable X to the latent dependent variable Y isequivalent to the complete mediation model.In addition, we used the χ2 value and the Bayesian in-
formation criterion (BIC) for comparing models witheach other. The χ2 difference test can be used to testwhether two nested models2 differ significantly. Asmaller χ2 value indicates a better fit. Models that differin one degree of freedomdiffer significantly if they differat least by Δχ2=3.84, models that differ in two degreesof freedom differ significantly if they differ at least byΔχ2=5.99, and models that differ in three degrees offreedom differ significantly if they differ by Δχ2=7.81.If twomodels differ significantly, the better fittingmodelis accepted. If two models do not differ significantly, themore parsimonious model (with more degrees of free-dom) is accepted. The BIC value is a descriptive fit
2Twomodels are nested if onemodel can be formulated as a special case
of the other model. For example, the reflection model can be formu-
lated as a special case of the inverse mediation model where the path
between the independent variable and the proposed mediator is set
to zero.
European J
index, which takes the parsimoniousness of the modelsinto account and can also be used to compare non-nested models. A ΔBIC>10 indicates a meaningful dif-ference (Raftey, 1995).We estimated the fit of all 12 causal models to all data
sets. The structural models were specified as shown inFigure 1. We modeled each construct as a latent variablewith three indicators, thus yielding sufficient degrees offreedom for the parameter estimation. The measurementmodels for the latent dependent variable and the latentmediator variable were specified as τ-congeneric models.3
A graphical illustrationof the completemediationmodel isshown in Figure 2. Themodel parameters were estimatedusing the maximum likelihood function implemented inthe CALIS procedure in SAS 9.3 (SAS Institute).
RESULTS AND DISCUSSION
Tests of Indirect Effects (Bootstrap, Sobel, andStructural Equation Models)
We estimated bootstrapped CI of the indirect effect in alldata sets. Therewere 1000 data sets in each class and 200observations in each data set. We used the independentvariable X, the mean score of the three indicators of thedependent variableY, and themean score of the three in-dicators of the proposed mediator Z for the analyses. Theresults are summarized in Table 2.
3A τ-congenericmeasurementmodel specifies all indicators Yi of a scale
as a weighted linear combination of a latent construct variable τ and a
D. Danner et al.Mediation analysis with structural equation models
As can be seen, the bootstrapped CI correctly indi-cated a mediation effect (a CI above zero) in 100% ofthe data sets that were actually generated according tothe complete mediation structure (CI= [0.48; 1.05])and in 100% of the data of the partial mediation struc-ture (CI= [0.48; 1.05]). However, bootstrapping alsosupported amediation effect in 99%of the data sets thatwere generated according to the reflection structure(CI= [0.23; 0.64]) and the common-effect-on-Z struc-ture (CI= [0.23; 0.64]) and 100% of the data sets ofthe inverse mediation structure (CI= [0.62; 1.12]). Asevident from Table 2, the Sobel test and significance
Table 3. Percentage of how often a causal model (columns) was accepted in th
Note: There were 1000 data sets per class and N = 200 observations per da
significant.
Bold figures indicate how often the correct model fit the data.
European J
tests via structural equation models yield the same pat-tern of results (cf. Hayes, 2013). Across all models andanalyses, we would falsely accept a (partial) mediationin 31% of all cases when relying on the bootstrap testof the indirect effect alone. These findings demonstratethat observing a significant indirect effect—no matterwhether it was tested with a bootstrap procedure, aSobel test, or a structural equationmodel—does not un-equivocally prove that the apparent mediation effectwas actually used to generate the data. Even in an ex-perimental setting, a significant indirect effect can becaused by different structures.
e different data sets (rows) based on root mean square error of approxima-
Percentage model was accepted
4 5 6 7 8 9 10 11 12
5 5 0 0 0 0 0 0 0
0 0 0 4 7 6 0 0 0
0 0 2 5 0 0 5 0 0
95 95 2 0 4 5 5 0 0
94 94 2 0 5 5 5 0 0
0 0 94 0 0 42 13 4 91
0 0 5 95 0 4 0 2 5
3 3 1 0 94 43 0 2 0
1 1 43 0 14 94 1 88 5
3 3 43 0 0 1 95 0 5
0 0 15 0 1 94 0 93 5
0 0 94 0 0 16 1 5 93
set. Models were accepted if RMSEA ≤ 0.06 and model parameters were
e different data sets (rows) based on Bayesian information criterion (BIC)
Percentage model was accepted
4 5 6 7 8 9 10 11 12
0 0 0 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0
0 0 0 2 0 0 0 0 0
99 99 1 0 2 1 2 0 0
99 99 1 0 2 1 2 0 0
5 5 98 3 1 34 34 11 98
0 0 0 100 0 0 0 0 0
4 5 1 0 99 14 0 1 0
5 5 33 3 33 98 1 99 13
6 5 14 0 0 1 100 0 2
0 0 5 5 4 93 0 98 5
0 0 95 5 0 6 5 5 98
ta set. Models were accepted if ΔBIC ≤ 10 and model parameters were
D. Danner et al. Mediation analysis with structural equation models
Model Fit and Model Comparison
Considering the fit between the observed data and thevarious causal models allows us to discard distinctmodels as incompatiblewith the given covariance struc-ture. Table 3 shows how often models were accepted inthe different data sets based on their RMSEA. A modelwas accepted when RMSEA≤0.06 (Hu & Bentler,1999). In addition, Table 4 shows how often modelswere accepted based on the χ2 value. A model was ac-cepted if there was no nested model that fitted the datasignificantly better (Bollen, 1989). Table 5 shows howoften models were accepted based on their BIC. Amodel was accepted if no other model had a BIC thatwas 10 points smaller (Raftey, 1995).As can be seen, no matter which fit index is used, the
number of viable causal models can substantially be re-duced. For example, the RMSEA suggests that in thedata sets that were generated according to the completemediation model, a complete mediation model (94%),an inverse mediation model (91%), or a reflectionmodel (42%) fit the data best. Similarly, the χ2 valuessuggest that a complete mediation model (91%) or aninverse mediation model (68%) fits the data. The BICsuggests that a completemediationmodel (98%), an in-verse mediation model (98%), or a reflection orcommon-effect-on-Z model (34%) fit best.Likewise, the RMSEA suggests that in the data
sets that were generated according to the commoncause model, a common cause model (99%) fitsthe data best. Likewise, the BIC suggests that thecommon cause model explains these data sets best(100%). The χ2 difference test suggests that not
Table 5. Percentage of how often a causal model (columns) was accepted in t
only the common cause model fits the data best(94%) but also the complete mediation model(91%) or the reflection model (91%) fits the data.This is due to the fact that the χ2 difference testcan only distinguish between nested models, andthe common cause model, complete mediationmodel, and the reflection model are not nested ineach other.As can be seen in Tables 3, 4, and 5, this pattern
of results was the same for all generated data sets.For each class of data sets, a class of causal modelscan be rejected, whereas another class of viablecausal models remains. Therefore, it is not possibleto use the structural equation modeling approachto completely eliminate the ambiguity that arisesfrom testing the indirect effect only. However, thesefindings also show that structural equation model-ing can reduce the number of possible causal expla-nations to a considerable degree, typically from 12possible explanatory models down to two or threeviable models that are consistent with the data(Tables 3, 4, and 5). For example, combining a testof the indirect effect with assessing the fit of theunderlying causal models (via the RMSEA)decreases the false alarm rate (for a mediation)from 31% to 19% across all data sets (cf. Tables 2and 3). In addition, structural equation modelingreveals which alternative causal models can or can-not explain the observed data. It should be notedthat well-fitting models always imply covariancematrices that are very similar to the empirical co-variance of the manifest variables. Thus, similarity ofthe covariance matrices is, not surprisingly, the key
he different data sets (rows) based on χ2 difference test
Percentage model was accepted
4 5 6 7 8 9 10 11 12
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
95 95 0 0 0 0 0 0 0
95 95 0 0 0 0 0 0 0
5 5 91 1 2 12 11 0 68
0 0 91 94 3 91 3 0 0
4 5 34 3 90 87 0 0 0
5 4 11 0 12 89 2 67 1
6 5 89 0 0 34 90 0 0
0 0 5 5 5 94 0 86 0
0 0 96 5 0 6 5 0 85
If two nestedmodels differed significantly in their χ2 values, themodel with
ntly, the more parsimonious model was accepted.
, Ltd.
D. Danner et al.Mediation analysis with structural equation models
to understanding the clusters of non-discriminablecausal models.4
Appendix B additionally shows the average fit indicesfor the different data sets. These numbers suggest thatthe number of eligible causal models can be further re-duced by comparing the non-rejected models with eachother. For example, for the data that were generated ac-cording to the reflectionmodel, the RMSEA suggests thata reflectionmodel, a partial mediationmodel, a completemediationmodel, or an inverse mediationmodel fit withthe data. The χ2 values and the BIC further suggest thatthe complete mediation model fit the data substantiallyworse than a reflectionmodel, a partialmediationmodel,or an inversemediationmodel. Hence, by combining theRMSEA with the χ2 and the BIC, the number of eligiblemodels may further be reduced.
GENERAL DISCUSSION
A significant indirect effect does not prove that the datahave been generated by a causal mediation mechanism.Even in experimental settings, other causal structuresthan a mediation structure can also give rise to a signifi-cant test result of the indirect effect. However, as thepresent article demonstrates, combining a significancetest of the indirect effectwith evaluating thefit of alterna-tive causal model can reduce the uncertainty within agiven trivariate theory space. Our simulation results illus-trate that structural equation modeling allows re-searchers to reject specific classes of alternative causalmodels and to concentrate on a clearly reduced set of vi-able models. Therefore, combining theoretical consider-ations, a significance test of the indirect effect, andstructural equation modeling can be very useful to reacha better understanding of the mechanisms that may ex-plain a given array of empirical evidence. We recom-mend a six-step approach to investigating mediation:(i) Before data acquisition, limit the number of possi-
ble causal models by design: An experimental de-sign allows excluding all models in which the
4For the present analysis, we used a sample size of N = 200 observation
per simulated data set because this is the lower bound recommendation
for structural equation models (e.g., Hoyle, 1995) and, especially in ex-
perimental designs, it may be expensive to assess larger samples. As in
some experimental settings, it may not be possible to assess 200 partic-
ipants, while in other settings, it may be possible to assess even larger
samples; we additionally run the analysis with N = 100 observations
per data set and N = 500 observations per data set, which revealed sim-
ilar results. Likewise,we changed the cutoff value toRMSEA ≤ 0.10 andalternatively used the cutoff value comparative fit index> 0.95, which
also reveal similar results.
European J
independent variable is affected by the proposedmediator or the dependent variable.
(ii) Identify the causal models for your experimentaldesign. In a standard experimental design withthree constructs, there are 12 different causalmodels (Figure 1).
(iii) Specify all remaining causal models as structuralequation models: Specify the proposed mediatorand the dependent variable as latent variables. Spec-ify the relation between the latent variables accord-ing to the remaining causal models that you haveidentified (an example of a complete mediationmodel is shown in Figure 2). Additional modelscan be constructed by changing the path betweenthe latent construct variables. Structural equationmodels also allow testing whether an indirect ef-fect is significant.
(iv) Investigate the fit of each causal model and com-pare the fit between models. By this means, thenumber of eligible causal models can substantiallybe reduced. The RMSEA allows deciding whethera model sufficiently fits observed data or whethera model should be rejected. Models with anRMSEA>0.06 can be rejected because of insuffi-cient fit (Hu & Bentler, 1999). Models containingnonsignificant parameters can be rejected becausea model containing a zero parameter is equivalentto another more parsimonious model. The χ2 dif-ference test further allows deciding whether twonested models fit the data equally well or whetheronemodel fits the data significantly better. The BICallows deciding whether non-nested models fit thedata equally well or whether one model fits thedata substantially better, where a ΔBIC>10 sug-gests a meaningful difference. Using the χ2
difference test and the BIC allows affirmingthat the models that were rejected based ontheir RMSEA do fit worse than the non-rejectedmodels and also further reducing the numberof eligible models by showing that some of thenon-rejected model fits the data better than othernon-rejected models.
(v) Identify the remaining causal models that cannotbe rejected: The present study suggests that foreach causal structure, there are typically two orthree models between which cannot be discrimi-nated by structural equation modeling.
(vi) Engage in deliberate attempts to reduce the num-ber of remaining models: This can be carried outby either a follow-up experiment where theproposed mediator variable is manipulated(e.g., Shrout & Bolger, 2002; Spencer, Zanna, &Fong, 2005; Stone-Romero & Rosopa, 2011), a
D. Danner et al. Mediation analysis with structural equation models
longitudinal study (e.g., Cole &Maxwell, 2003), ora theory-driven discourse.
Limitations
The Approach is Designed for an Experimental Setting
The basis for the present analysis approachwas rejectingseveral causal models by design. The experimentalmanipulation of the independent variable allowed usto reject all causal models in which the dependent vari-able or the proposed mediator affected the independentvariable. In a non-experimental design, one would notbe able to reject these models on a priori grounds. Theset of possible causal models would then grow from 12classes of models up to at least 27 different classes ofmodels if the assumption is given up that one indepen-dent variable cannot be affected by the other variables.This would increase the number of models that fits withan observed data set, leaving various causal explana-tions for observed data. Likewise, there would beseveral models that show an identical fit with the ob-served data (James et al., 2006; MacCallum et al.,1993; Stelzl, 1986).To be sure, an experimental manipulation is not the
only reason for excluding specific models. Given suchvariables as biological sex or age, it is possible to excludeall models that assign them the role of a dependent var-iable or mediator (e.g., MacCallum et al., 1993). For ex-ample, if somebody wants to investigate whethergender role self-concept mediates the relation betweenbiological sex and dominant behavior, all causal modelscould be excluded in which sex is affected by behavioror gender role self-concept. Longitudinal design mayalso sometimes allow one to discard certain models ona priori ground. This highlights the usefulness of com-bining theory, design, and statistical methods and notblindly applying multivariate statistics.
5However, Ledgerwood and Shrout (2011) note that the standard er-
rors of indirect effects based on latent variables can be greater than
the standard errors based on manifest variables, especially with hetero-
geneous items.
The Measurement of the Proposed Mediator and De-pendent Variable Must be Valid
The results of the present analysis are only valid if theindicators of the dependent variable and the indicatorsof the proposed mediator are valid. For one thing, theresults of structural equation modeling will be biased ifthe discriminant validity of the indicators of the pro-posed mediator and the dependent variable cannot beestablished. Let us illustrate this complication withreference to the well-being example. Suppose that wewant to investigate whether the relation between socialsupport and well-being is mediated by attribution style.One of the items that we use to measure well-being is
“I am a great person.” One person may feel well andtherefore agree to this item. However, another personwho does not feel very well may still be convinced tobe a great person because he or she often achieves suc-cess. Such an item would be a blend of well-being andattribution style, containing both well-being varianceand attribution style variance. Hence, this item will co-vary with the attribution style items and with the otherwell-being items regardless of whether the attributionstyle and well-being are related. This must bias thestructural equation modeling results because it artifi-cially increases the association between the latentwell-being variable (the dependent variable) and the la-tent attribution style variable (the proposed mediator).Therefore, it is necessary to ensure discriminant validityof the manifest indicators.Another problem arises if all indicators of the pro-
posed mediator are correlated with another non-observed construct. This problem was also discussed asthe spurious mediation problem (e.g., Fiedler et al.,2011; MacKinnon et al., 2000). In particular, the valid-ity of statistical testing of indirect effects has been criti-cized because the measurement of the proposedmediator Z could alternatively be interpreted as a corre-late of another potential mediator Z′. For one more il-lustration of this fundamental problem, suppose thatthe persons’ attribution style ismeasuredwith items like“Success depends on good relations,” “You have to belucky to be successful,” and “I cannot influence muchof what is happening to me.” These items may indeedmeasure attribution style, but theymay aswell measureother latent variables, such as optimism. Wheneveritem overlap reduces the discriminant validity of latentconstructs, no statistical test can decide whether the ef-fect of social support is mediated by one or the otherconstruct. This argument applies to regression analysiswith manifest variables as well as to structural equationmodeling.Still, regarding the validity of single indicators, the
structural equation modeling approach is more robustthan the traditional regression approach.5 This is be-cause the latent mediator variable is specified as thecommon variance of all indicator variables. A seriousvalidity problem only arises if all or most indicators arecorrelated with the same non-observed construct. Novalidity problem exists if only one indicator is correlatedwith another construct or if each indicator overlaps witha different non-observed construct, because the
, Ltd.
D. Danner et al.Mediation analysis with structural equation models
“unwanted” variance portions of these indicators wouldbe treated as specific variance components in the struc-tural equationmodel and thus as part of the residual er-ror variance. For example, assuming that we measuredattribution style with three items, one being Sartre’squote “I am the architect of my own self, my own char-acter and destiny.” One could criticize that this particu-lar item does not only measure attribution style butalso measure the person’s liking of Jean-Paul Sartre.Using a traditional regression approach, the Sartre itemmay affect the mean score of the indicators and hence,bias the results of the mediation analysis. However,using the structural equation modeling approach, theSartre-specific variance proportion would be treated asmeasurement error in the model and hence, not biasthe results of the mediation analysis (see also Stone-Romero & Rosopa, 2010; Shadish et al., 2002).
The present results corroborate the contention that inmost cases, more than one causal model fits a data set.For example, the common-effect-on-Z data set couldbe fitted not only by the common-effect-on-Z modelbut also by the inverse mediation model. Likewise,there aremodels that are statistically equivalent. For ex-ample, model 4 (Z→Y) implies the same covariancestructure than model 5 (Y→Z) and thus cannot be dis-tinguished by statistical procedures. Structural equationmodeling only allows reducing the number of possibleexplanations for the indirect effect within the trivariateframework of given variables. However, as a matter offact, researchers have to admit that no statisticalmethodcan rule out all alternative models involving other vari-ables not included in the trivariate framework. There-fore, as a matter of principle, researchers must alwaysgo beyond statistics to complete the picture.
The Present Simulations Illustrate a Method But Do NotDemonstrate General Rules
When we use our method, there remains a class ofmodels that cannot be further distinguished and thatall may explain the data equally well. Because the re-sults of a simulation depend on the choice of the param-eters, it is entirely possible that the class of remainingmodels might have different members if we had chosendifferent parameters. However, we are confident thatthe main findings of our simulation may replicate forother parameters, that is, the structural equationmodel-ing approach helps to eliminate some although not all ofthe alternative models. We therefore consider our
European J
simulation study not as a demonstration of general rulesbut as an illustration that our approach helps to decidebetween concurrent models for which we could not de-cide if we rely only on the Sobel test or on the bootstrapevaluation of the alleged mediation effect.
Beyond Statistical Testing
The approach we have described so far is a purely statis-tical one. Of course, there are ancillary and in severalsettings preferable approaches for an analysis of media-tion. For example, Spencer, Zanna, and Fong (2005)suggested investigating mediation by manipulating themediator variable experimentally. This approach hasthe great advantage that the direction of the relation be-tween the proposed mediator and the dependent vari-able can be controlled. For example, if we couldexperimentally control how persons attribute their suc-cesses and failures, we could exclude all causal modelsthat state that well-being affects attribution style. How-ever, it may not always be possible to control constructssuch as attribution style (especially over a longer periodof time). In addition, it is not certain that the effect of theexperimental manipulation of the proposed mediator isthe same as the indirect effect of the mediator triggeredby the independent variable (Kenny, 2008). Stone-Romero and Rosopa (2010) and Mathieu and Taylor(2006) discuss further approaches to investigatingmedi-ation analysis with different designs, as the authors con-clude the following: “ideally, the results of studies usingall such alternatives should converge (p. 700).”It has also been suggested to measure the mediator
and the dependent variable at several measurement oc-casions or in a prescribed timely order (e.g., MacCallumet al., 1993; MacKinnon, 2008; Cole & Maxwell, 2003).The rationale behind this approach is that the variablethat is measured second cannot affect the variable thatwas measured first. However, this may not be true inall settings because the variablesmay have affected eachother before the first measurement took place (see alsoMacKinnon, 2008). For example, social support may af-fect a person’s well-being after a first treatment. Fromthat moment on, persons may start to attribute theirsuccesses to their capability. At the beginning of the sec-ond treatment session, the researcher assesses the per-sons’ attribution style and registers that the style haschanged. At the beginning of the third session, the re-searcher assesses the person’s well-being and registersthat their well-being has improved. Therefore, theselected time interval is critical, and even if the variableswere measured in a prescribed timely order, the direc-tion of the relation may not be clear (e.g., Cole &Maxwell, 2003; MacKinnon, 2008).
D. Danner et al. Mediation analysis with structural equation models
CONCLUSION
Combining a significance test of the indirect effectwith evaluating the fit of alternative causal modelswith structural equation models can improve media-tion analyses. However, structural equation modelingis not a silver bullet and has its limitations and weak-nesses as does every method. The strength of thepresent approach is that it can be combined with com-plementary approaches. For example, bootstrappingor a Sobel test may indicate that an indirect effect issignificant, granted the mediation occurred. Structuralequation modeling can demonstrate that several alter-native causal explanations for the significant indirecteffect can be rejected. Experimentally, manipulatingthe mediator variable or measuring the proposed me-diator before measuring the dependent variable mayfurther support this hypothesis. However, the mostimportant methodological tool for a scientific explana-tion consists of cleverly designed follow-up experi-ments informed by the insights gained fromstructural equation modeling. We hope that such acombined approach can improve the use of mediationanalysis in future research in social psychology andthereby help scientists to gain a deeper understandingof empirical reality.
ACKNOWLEDGEMENTS
This research was supported by grants awarded by theDeutsche Forschungsgemeinschaft to the second(HA3044/7-1) and third authors (Fi 294/23-1). Wegratefully thank Matthias Blümke, Jemaine Clement,Sebastian Nagengast, Oliver Schilling, and three anony-mous reviewers for helpful comments on an earlierdraft of this paper.
REFERENCES
Ahearne, M., Mathieu, J., & Rapp, A. (2005). To empoweror not to empower your sales force? An empirical exam-ination of the influence of leadership empowermentbehavior on customer satisfaction and performance.Journal of Applied Psychology, 90(5), 945–955. http://dx.doi.org/10.1037/0021-9010.90.5.945
Arbuckle, J. L. (2013). Amos (version 22.0) [computer pro-gram]. Chicago: SPSS.
Baron, R.M., &Kenny, D. A. (1986). Themoderator–mediatorvariable distinction in social psychological research: Concep-tual, strategic, and statistical considerations. Journal of Per-sonality and Social Psychology, 51(6), 1173–1182. http://dx.doi.org/10.1037/0022-3514.51.6.1173
Bollen, K. A. (1989). Structural equations with latent variables.Oxford England: John Wiley & Sons.
Frazier, P. A., Tix, A. P., & Barron, K. E. (2004). Testing mod-
erator and mediator effects in counseling psychology
research. Journal of Counseling Psychology, 51(1), 115–134.
http://dx.doi.org/10.1037/0022-0167.51.1.115Goodman, L. A. (1960). On the exact variance of products.
Journal of the American Statistical Association, 55,708–713.http://dx.doi.org/10.1080/01621459.1960.10483369
Hayes, A. F. (2013). Introduction to mediation, moderation, andconditional process analysis: A regression-based approach.New York: Guilford Press.
Hayes, A. F., & Scharkow, M. (2013). The relative trustwor-thiness of inferential tests of the indirect effect in statisticalmediation analysis: Does method really matter? Psychologi-cal Science, 24(10), 1918–1927. http://dx.doi.org/10.1177/0956797613480187
Hilliard, M. E., Holmes, C. S., Chen, R., Maher, K., Robinson,
E., & Streisand, R. (2013). Disentangling the roles of
parental monitoring and family conflict in adolescents’
management of type 1 diabetes. Health Psychology, 32(4),
388–396. http://dx.doi.org/10.1037/a0027811Hoyle, R. H. (1995). Structural equation modeling: Concepts, is-
sues, and applications. Thousand Oaks, CAUS: Sage Publica-tions, Inc.
Hoyle, R. H., & Smith, G. T. (1994). Formulating clinical re-
search hypotheses as structural equationmodels: A concep-
tual overview. Journal of Consulting and Clinical Psychology,
62, 429–440.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in
covariance structure analysis: Conventional criteria versus
new alternatives. Structural Equation Modeling, 6, 1–55. doi:
10.1080/10705519909540118
Iacobucci, D., Saldanha, N., & Deng, X. (2007). A meditation
on mediation: Evidence that structural equations models
perform better than regressions. Journal of Consumer Psy-
7408(07)70020-7James, L. R.,Mulaik, S. A., & Brett, J.M. (2006). A tale of two
methods. Organizational Research Methods, 9(2), 233–244.http://dx.doi.org/10.1177/1094428105285144
Judd,C.M.,&Kenny,D.A. (1981). Process analysis: Estimatingmediation in treatment evaluations. Evaluation Review, 5(5),602–619. http://dx.doi.org/10.1177/0193841X8100500502
, Ltd.
APPENDIX A: TABLE A1
Equations used for generating the variables in the simulated data sets
Model class Variable Equation
Independence model Xij
[0;1]
Y1ij
1 * υij+ 1 * ε
1ij
Y2ij
1 * υij+ 1 * ε
2ij
Y3ij
1 * υij+ 1 * ε
3ij
Z1ij
1 * ωij+ 1 * ε
4ij
Z2ij
1 * ωij+ 1 * ε
5ij
Z3ij
1 * ωij+ 1 * ε
6ij
Simple effect (X→Y) Xij
[0;1]
Y1ij
1 * Xij+ 1 * υ
ij+ 1 * ε
1ij
Y2ij
1 * Xij+ 1 * υ
ij+ 1 * ε
2ij
Y3ij
1 * Xij+ 1 * υ
ij+ 1 * ε
3ij
Z1ij
1 * ωij+ 1 * ε
4ij
Z2ij
1 * ωij+ 1 * ε
5ij
Z3ij
1 * ωij+ 1 * ε
6ij
Simple effect (X→Z) Xij
[0;1]
Y1ij
1 * υij+ 1 * ε
1ij
Y2ij
1 * υij+ 1 * ε
2ij
Y3ij
1 * υij+ 1 * ε
3ij
Z1ij
1 * Xij+ 1 * ω
ij+ 1 * ε
4ij
Z2ij
1 * Xij+ 1 * ω
ij+ 1 * ε
5ij
Z3ij
1 * Xij+ 1 * ω
ij+ 1 * ε
6ij
Simple effect (Z→Y) Xij
[0;1]
Y1ij
1 * ωij+ 1 * υ
ij+ 1 * ε
1ij
(Continues)
D. Danner et al.Mediation analysis with structural equation models
Kenny, D. A. (2008). Reflections onmediation.OrganizationalResearch Methods, 11(2), 353–358. http://dx.doi.org/10.1177/1094428107308978
King, L. A., King, D. W., Fairbank, J. A., Keane, T. M., &Adams, G. A. (1998). Resilience–recovery factors inpost-traumatic stress disorder among female and maleVietnam veterans: Hardiness, postwar social support,and additional stressful life events. Journal of Personalityand Social Psychology, 74(2), 420–434. http://dx.doi.org/10.1037/0022-3514.74.2.420
Kline, R. B. (2011). Principles and practice of structural equationmodeling (3rd ed.). New York, NY US: Guilford Press.
Ledgerwood, A., & Shrout, P. E. (2011). The trade-offbetween accuracy and precision in latent variable modelsof mediation processes. Journal of Personality and SocialPsychology, 101(6), 1174–1188. http://dx.doi.org/10.1037/a0024776.
Lee, S., &Hershberger, S. (1990). A simple rule for generatingequivalentmodels in covariance structuremodeling.Multi-variate Behavioral Research, 25(3), 313–334. http://dx.doi.org/10.1207/s15327906mbr2503_4
MacCallum, R. C., Wegener, D. T., Uchino, B. N., &Fabrigar, L. R. (1993). The problem of equivalent modelsin applications of covariance structure analysis. Psycholog-ical Bulletin, 114(1), 185–199. http://dx.doi.org/10.1037/0033-2909.114.1.185
MacKinnon, D. P. (2008). Introduction to statistical mediationanalysis. New York, NY: Taylor & Francis Group/Lawrence Erlbaum Associates.
MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000).Equivalence of the mediation, confounding and suppres-sion effect. Prevention Science, 1(4), 173–181. http://dx.doi.org/10.1023/a:1026595011371
Mathieu, J. E. & Taylor, S. R. (2006). Clarifying condi-tions and decision points for mediational typeinferences in organizational behavior. Journal of Orga-nizational Behavior, 27, 1031–1056. http://dx.doi.org/10.1002/job.406
Preacher, K. J., & Hayes, A. F. (2004). SPSS and SASprocedures for estimating indirect effects in simple medi-ation models. Behavior Research Methods, Instruments, &Computers, 36(4), 717–731. http://dx.doi.org/10.3758/bf03206553
Quilty, L. C., Godfrey, K. M., Kennedy, S. H., & Bagby,
R. M. (2010). Harm avoidance as a mediator of treat-
ment response to antidepressant treatment of patients
with major depression. Psychotherapy and Psychoso-
000276372Raftey, A. E. (1995). Bayesian model selection in social re-
search. Sociological Methodology, 25, 111–163.Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Ex-
perimental and quasi-experimental designs for generalizedcausal inference. Boston, MA US: Houghton, Mifflin andCompany.
European J
Shrout, P. E., & Bolger, N. (2002). Mediation in experimentaland nonexperimental studies: Newprocedures and recom-mendations. Psychological Methods, 7(4), 422–445. http://dx.doi.org/10.1037/1082-989X.7.4.422
Sobel, M. E. (1982). Asymptotic confidence intervals for indi-rect effects in structural equations models. In S. Leinhart(Ed.), Sociological methodology (pp. 290–312). San Francisco:Jossey-Bass.
Spencer, S. J., Zanna, M. P., & Fong, G. T. (2005). Estab-lishing a causal chain: Why experiments are often moreeffective than mediational analyses in examining psy-chological processes. Journal of Personality and SocialPsychology, 89(6), 845–851. http://dx.doi.org/10.1037/0022-3514.89.6.845
Stelzl, I. (1986). Changing a causal hypothesis withoutchanging thefit: Some rules for generating equivalent pathmodels. Multivariate Behavioral Research, 21(3), 309–331.http://dx.doi.org/10.1207/s15327906mbr2103_3
Stone-Romero, E. F., & Rosopa, P. J. (2010). Researchdesign options for testing mediation models and theirimplications for facets of validity. Journal of ManagerialPsychology, 25(7), 697–712. http://dx.doi.org/10.1108/02683941011075256
Stone-Romero, E. F., & Rosopa, P. J. (2011). Experimentaltests of mediation models: Prospects, problems, and somesolutions. Organizational Research Methods, 14(4), 631–646.http://dx.doi.org/10.1177/1094428110372673
Average root mean square error of approximation (RMSEA) for the different causal models (columns) in the different data sets (rows) and standard deviation
in brackets
Model used to
generate data
RMSEA for model
1 2 3 4 5 6 7 8 9 10 11 12
1 Independence model 0.02
(0.02)
0.02
(0.02)
0.01
(0.02)
0.02
(0.02)
0.02
(0.02)
. . . . . . .
2 Simple effect (X→Y) 0.11
(0.02)
0.02
(0.02)
0.11
(0.02)
0.11
(0.01)
0.11
(0.02)
. 0.02
(0.02)
0.02
(0.02)
0.01
(0.02)
. . .
3 Simple effect (X→Z) 0.11
(0.02)
. 0.02
(0.02)
. . 0.02
(0.02)
0.02
(0.02)
. . 0.02
(0.02)
. .
4 Simple effect (Z→Y) 0.16
(0.02)
0.17
(0.02)
0.17
(0.02)
0.02
(0.02)
0.02
(0.02)
0.02
(0.02)
. . 0.02
(0.02)
0.02
(0.02)
. .
5 Simple effect (Y→Z) 0.16
(0.02)
0.17
(0.02)
0.17
(0.02)
0.02
(0.02)
0.02
(0.02)
0.02
(0.02)
0.18
(0.02)
0.02
(0.02)
0.02
(0.02)
0.02
(0.02)
. .
6 Complete mediation
(X→Z, Z→Y)0.21
(0.02)
0.20
(0.02)
0.19
(0.02)
0.11
(0.02)
0.11
(0.02)
0.02
(0.02)
0.17
(0.02)
0.11
(0.02)
0.06
(0.03)
0.09
(0.02)
. 0.02
(0.02)
7 Common cause
(X→Z, X→Y)0.15
(0.02)
0.11
(0.02)
0.11
(0.02)
0.15
(0.02)
0.15
(0.02)
0.10
(0.02)
0.02
(0.02)
0.12
(0.02)
0.10
(0.02)
0.11
(0.02)
. 0.02
(0.02)
8 Common effect on Y(X→Y, Z→Y)
0.18
(0.02)
0.17
(0.02)
. 0.10
(0.02)
0.10
(0.02)
0.17
(0.02)
. 0.02
(0.02)
0.06
(0.03)
. . .
9 Reflection model
(X→Y, Y→Z)0.21
(0.02)
0.19
(0.02)
0.20
(0.02)
0.11
(0.02)
0.11
(0.02)
0.06
(0.03)
0.17
(0.02)
0.09
(0.02)
0.02
(0.02)
0.11
(0.02)
0.02
(0.02)
0.02
(0.03)
10 Common effect on Z
(X→Z, Y→Z)
0.18
(0.02)
0.20
(0.02)
0.17
(0.02)
0.10
(0.02)
0.10
(0.02)
0.06
(0.03)
0.18
(0.02)
. 0.10
(0.02)
0.02
(0.02)
. 0.02
(0.02)
11 Partial mediation
(X→Z, Z→Y, X→Y)0.25
(0.02)
0.20
(0.02)
0.23
(0.02)
0.16
(0.02)
0.16
(0.02)
0.08
(0.02)
0.17
(0.02)
0.11
(0.02)
0.02
(0.02)
0.16
(0.02)
0.02
(0.02)
0.01
(0.02)
12 Inverse mediation
(X→Z, X→Y, Y→Z)0.25
(0.02)
0.23
(0.02)
0.20
(0.02)
0.16
(0.02)
0.16
(0.02)
0.02
(0.02)
0.17
(0.02)
0.16
(0.02)
0.08
(0.03)
0.11
(0.02)
0.02
(0.02)
0.02
(0.02)
Note: There were 1000 data sets per class and N = 200 observations per data set. The average RMSEA is reported if the model parameters are significant in
at least 5% of the simulated data sets.
TABLE B2
Average χ2 value for the different causal models (columns) in the different data sets (rows) and standard deviation in brackets
Model used to
generate data
χ2 value for model
1 2 3 4 5 6 7 8 9 10 11 12
1 Independence
model
15.09
(5.43)
14.00
(4.73)
13.59
(4.48)
14.17
(4.71)
14.17
(4.71)
. . . . . .
2 Simple effect
(X→Y)49.45
(12.50)
14.20
(5.56)
48.35
(11.23)
46.81
(11.49)
46.81
(11.49)
47.47
(9.93)
13.93
(5.77)
12.66
(5.23)
12.38
(5.03)
. 14.64
(13.39)
.
3 Simple effect
(X→Z)49.32
(12.02)
. 14.05
(5.27)
. . 13.30
(5.72)
12.92
(4.69)
. . 13.06
(4.82)
. .
4 Simple effect
(Z→Y)92.25
(16.74)
90.99
(14.76)
94.65
(15.80)
14.29
(5.38)
14.29
(5.38)
12.40
(3.73)
. . 13.16
(6.49)
13.41
(4.62)
. .
5 Simple effect
(Y→Z)
93.10
(17.02)
96.37
(16.36)
93.95
(16.03)
14.13
(5.45)
14.13
(5.45)
14.38
(5.33)
93.22
(17.17)
14.72
(4.99)
13.72
(5.19)
13.74
(5.97)
. .
6 Complete
mediation
(X→Z, Z→Y)
146.54
(22.86)
125.52
(19.11)
117.27
(18.08)
50.12
(12.20)
50.12
(12.20)
13.26
(5.27)
90.18
(16.50)
47.51
(11.76)
24.87
(8.45)
33.32
(10.15)
. 12.25
(5.09)
7 Common cause
(X→Z, X→Y)
83.08
(16.17)
48.35
(12.00)
47.80
(11.95)
78.51
(14.42)
38.54
(14.42)
38.54
(11.26)
13.07
(5.15)
50.43
(13.36)
38.95
(10.92)
47.85
(10.58)
. 12.08
(4.70)
8 Common effect
on Y (X→Y, Z→Y)112.31
(18.25)
91.22
(16.63)
. 43.17
(11.39)
43.17
(11.39)
40.42
(11.38)
. 13.31
(5.26)
24.67
(8.10)
. . .
(Continues)
D. Danner et al.Mediation analysis with structural equation models
Note: There were 1000 data sets per class and N = 200 observations per data set. The average χ2 value is reported if the model parameters are significant in
at least 5% of the simulated data sets.
TABLE B3
Average Bayesian information criterion (BIC) value for the different causal models (columns) in the different data sets (rows) and standard deviation in
brackets
Model used to
generate data
BIC value for model
1 2 3 4 5 6 7 8 9 10 11 12
1 Independence
model
83.97
(5.43)
88.17
(4.73)
87.76
(4.48)
88.34
(4.71)
88.34
(4.71)
. . . . . . .
2 Simple effect
(X→Y)118.32
(12.50)
88.37
(5.56)
122.53
(11.23)
120.99
(11.49)
120.99
(11.49)
. 93.40
(5.77)
92.14
(5.23)
91.85
(5.03)
. . .
3 Simple effect
(X→Z)118.20
(12.02)
. 88.22
(5.27)
. . 92.77
(5.27)
92.39
(4.69)
. . 92.53
(4.82)
. .
4 Simple effect
(Z→Y)
161.13
(16.74)
165.17
(14.76)
168.83
(15.80)
88.46
(5.38)
88.46
(5.38)
91.87
(3.73)
. . 92.63
(6.49)
92.89
(4.62)
. .
5 Simple effect
(Y→Z)161.98
(17.02)
170.55
(16.36)
168.13
(16.03)
88.31
(5.45)
88.31
(5.45)
93.85
(5.33)
172.70
(17.17)
94.19
(4.99)
93.19
(5.19)
93.22
(5.97)
. .
6 Complete
mediation
(X→Z, Z→Y)
215.42
(22.86)
199.69
(19.11)
185.44
(18.08)
124.29
(12.20)
124.29
(12.20)
92.74
(5.27)
169.65
(16.50)
126.99
(11.76)
104.35
(8.45)
112.79
(10.15)
. 97.02
(5.09)
7 Common
cause (X→Z,X→Y)
151.96
(16.17)
122.53
(12.00)
121.98
(11.95)
152.69
(14.42)
152.69
(14.42)
118.02
(11.26)
92.55
(5.15)
129.90
(13.36)
118.43
(10.92)
127.32
(10.58)
. 96.85
(4.70)
8 Common
effect on Y(X→Y, Z→Y)
181.19
(18.25)
165.39
(16.63)
. 117.34
(11.39)
117.34
(11.39)
119.89
(11.38)
. 92.79
(5.26)
104.15
(8.10)
. . .
9 Reflection
model (X→Y,
Y→Z)
216.35
(22.87)
186.40
(18.13)
200.55
(19.45)
124.14
(12.48)
124.14
(12.48)
104.09
(8.76)
170.53
(16.84)
112.72
(10.03)
92.64
(5.37)
126.06
(11.97)
96.81
(5.02)
98.07
(6.35)
10 Common
effect on Z(X→Z, Y→Z)
182.07
(18.22)
199.46
(17.88)
166.20
(16.97)
117.45
(11.30)
117.45
(11.30)
104.17
(8.43)
173.70
(16.21)
. 119.11
(10.68)
92.54
(5.22)
. 96.98
(4.24)
11 Partial
mediation
(X→Z, Z→Y,X→Y)
266.13
(25.82)
199.73
(19.09)
236.15
(20.18)
159.97
(15.41)
159.97
(15.41)
112.06
(10.43)
169.75
(16.47)
127.06
(11.92)
92.72
(5.25)
163.69
(15.02)
97.10
(5.11)
96.12
(4.30)
(Continues)
TABLE B2 (Continued)
D. Danner et al. Mediation analysis with structural equation models
Note: There were 1000 data sets per class and N = 200 observations per data set. The average BIC is reported if the model parameters are significant in at
least 5% of the simulated data sets.
APPENDIX C: GUIDELINE FOR APPLYING THE PRESENT APPROACH WITH AMOS
AMOS (Arbuckle, 2013) offers a graphical user interface where the user can specify a model by drawing a path diagram as shown in Figure 2:
(i) Specify a manifest variable for the independent variable X.
(ii) Specify a latent variable for the proposed mediator Z with three manifest variables.
(iii) Specify a latent variable for the dependent variable Y with three manifest variables.
(iv) Specify the relation between the latent variables according to the causal model shown in Figure 1. There will be one structural equation model
for each causalmodel. Thesemodels differ in the number and directing of the path between the latent variables. Add a residual variable for each
endogenous variable in the model. Endogenous variables are variables that are explained by other variables in the model (e.g., the dependent
variable in the simple effect on Y model or the proposed mediator Z and the dependent variable Y in the complete mediation model)
(v) Estimate the model parameters and the model fit for each causal model.
(vi) Investigate the fit of each causal model and compare the fit between models. Models with an RMSEA> 0.06 can be rejected because of insuf-
ficient fit (Hu & Bentler, 1999). Models containing nonsignificant parameters can be rejected because a model containing a zero parameter is
equivalent to another more parsimonious model. Nested models can be compared with the χ2 difference test, where a significant difference
suggests rejecting the worse fitting model. Non-nested models can be compared based on their BIC valuewhere a ΔBIC> 10 suggests a mean-
ingful difference.
(vii) Engage in further attempts to reject the remaining models by theoretical considerations or a follow-up study.
APPENDIX D: GUIDELINE FOR APPLYING THE PRESENT APPROACH WITH SAS
The syntax requires SAS 9.3 and a SAS data file containing an independent variable, three indicators for the dependent variable, and three indicators for the
dependent variable.
First, the location and the name of the SAS data file and the name of the variables must be specified. This can easily be carried out by modifying the last line
of the syntax. In the present example, “C:\” is the location of the SAS file, “data” is the name of the SAS file, “v1” is the name of the independent variable,
“v2”–“v4” are the names of the indicators for the dependent variables, and “v5”–“v7” are the names of the indicators of the proposed mediator variable:
The syntax produces a table containing the RMSEA, χ2 value, the df, and the BIC for each model. In addition, the script provides estimates of the indirect
effect based on the complete mediation model and the partial mediation model. After evaluating the results, engage in further attempts to reject the re-
maining models by theoretical considerations, specific model comparisons, or a follow-up study. The SAS syntax and an exemplary output are shown as
follows. The syntax can also be downloaded at http://www.gesis.org/fileadmin/upload/dienstleistung/methoden/spezielle_dienste/zis_ehes/semmed.sas
%macro semmed(location,file,x,y1,y2,y3,z1,z2,z3);
/*Creating the data file*/libname library "&location."
/*Significance of the structural paths*/data parameter;set parameter;if _name_ = "a" or _name_ = "b" or _name_ = "c";keep _name_ p;p = 1-probnorm(_estim_/_stderr_);
data RMSEA;format model $7.;merge RMSEA parameter indirect_&model.;if a<.05 and b<.05 and c<.05 and pindirect<.05 then RMSEA=fitvalue;model = "&model.";keep model RMSEA;
if a<.05 and b<.05 and c<.05 and pindirect<.05 then BIC=fitvalue;model = "&model.";keep model BIC;
run;
data fit_&model.;format model $37. RMSEA 3.2 Chi2 6.2 df 3.0 BIC 6.2;merge RMSEA Chi2 df BIC;by model;
run;
%mend;
/*Specifying the different causal models*/%sem(model1, e7, e8);%sem(model2, e7, c f_x + e8);%sem(model3, a f_x + e7, e8);%sem(model4, e7, b f_z + e8);%sem(model5, b f_y + e7, e8);%sem(model6, a f_x + e7, b f_z + e8);%sem(model7, a f_x + e7, c f_x + e8);%sem(model8, e7, c f_x + b f_z + e8);%sem(model9, b f_y + e7, c f_x + e8);%sem(model10, a f_x + b f_y + e7, e8);%sem(model11, a f_x + e7, b f_z + c f_x + e8);
D. Danner et al.Mediation analysis with structural equation models
data fit;set fit_model1-fit_model12;label model = "Model";if model="model1" then model="Independence model";if model="model2" then model="Simple effect (X->Y)";if model="model3" then model="Simple effect (X->Z)";if model="model4" then model="Simple effect (Z->Y)";if model="model5" then model="Simple effect (Y->Z)";if model="model6" then model="Complete mediation (X->Z, Z->Y)";if model="model7" then model="Common cause (X->Z, X->Y)";if model="model8" then model="Common effect on Y (X->Y, Z->Y)";if model="model9" then model="Reflection model (X->Y, Y->Z)";if model="model10" then model="Common effect on Z (X->Z, Y->Z)";if model="model11" then model="Partial mediation (X->Z, Z->Y, X->Y)";if model="model12" then model="Inverse mediation (X->Z, X->Y, Y->Z)";
/*Last line of code where location, name of file, and name of variables can be specified*/%semmed(location=C:\, file=data, x=v1, y1=v2, y2=v3, y3=v4, z1=v5, z2=v6, z3=v7);
