Date post: | 18-Jan-2016 |
Category: |
Documents |
Upload: | sabina-hampton |
View: | 213 times |
Download: | 0 times |
Assessing Hypothesis TestingFit Indices
Kline Chapter 8 (Stop at 210)Byrne page 68-84
Fit Indices
• It’s complicated yo.– There are a lot of them.– They do not imply that you are perfectly right.– They have guidelines but they are not perfect
either.– People misuse them.– Etc.
Fit Indices
• Limitations:– Fit statistics indicate an overall/average model fit. • That means there can be bad sections, but the overall
fit is good.
– No one magical number/summary.– They do not tell you where a misspecification
occurs.
Fit Indices
• Limitations– Do not tell you the predictive value of a model.– Do not tell you if it’s theoretically meaningful.
Fit Indices
• Model test statistic – examines if the reproduced correlation matrix matches the sample correlation matrix– Sometimes called “badness of fit”– Want these to be small
Fit Indices
• Traditional NHST = reject-support context– You reject the null to show your research
hypothesis is correct.• SEM Hyp Testing = accept-support context– You do not reject the null showing that your model
is consistent with the normal
Fit Indices
• Both types of statistical inferences have their problems … and especially in SEM because it is easy to find statistics that you would normally reject, even with good model fit.
• Tends to be too black and white (reject or not to reject!)
Fit Indices
• Approximate Fit Indices – Not traditionally a dichotomous yes-no decision – Do not distinguish between sampling error and
evidence against the model
Fit Indices
• Approximate Fit Indices– Absolute Fit Indices– Incremental Fit Indices– A parsimony-adjusted Index– Predictive Fit Indices
Fit Indices
• Absolute fit indices – Proportion of the covariance matrix explained by
the model– You can think about these as sort of R2
– Want these values be high
Fit Indices
• Incremental Fit Indices– Also known as comparative fit indices– Compared to the improvement over the
independence model (remember that’s the one with no relationships between the variables)
– Not necessarily the best indices
Fit Indices
• Parsimony-adjusted index– These include penalties for model complexity
(which normally gives you better fix by adding more paths)
– These will have smaller values for simpler models
Fit Indices
• Predictive fix indices– Estimate model fit in a hypothetical replication of
the study with the same sample size randomly drawn from the population
– Not always used
Fit Indices
• What size? I need a rule?!– Everyone cites Hu and Bentler (1999) for the
golden standards.– Same problem that Cohen had (we love rules).
• So when the fit is messy, cite Kline (page 197) as reasons that’s not a bad thing– This section is an interesting read, especially if you
have trouble publishing, but not crucial to your understanding of fit indices
Model Test Statistic
• Chi-square (listed as CMIN in your output)– Formula = (N-1)FML
– FML = is the minimum fit function in ML estimation– P values are based on df for your model and a chi-
square distribution– You want this to be nonsignificant.• But this is a catch 22!
Model Test Statistic
• Chi-square is biased by– Multivariate non-normality – Correlation size – bigger correlations can be bad
for you (harder to estimate all that variance)– Unique variance – Sample size
Model Test Statistic
• Everyone reports chi-square, but people tend to ignore significant values– (I’m sort of eh on his YOU MUST PAY ATTN OR DIE
talk in this section)
Model Test Statistic
• Normed chi-square or (X2/df) – this used to be widely reported and used– The criterion was < 3.00 were good models– Now most people have moved away from this
procedure
Approximate Fit Indices
• Some examples:– RMSEA (root mean square error of approximation)– SRMR (standardized root mean square residual)
– A/GFI (adjusted/goodness of fit index) – CFI (comparative fix index)– TLI (Tucker-Lewis Index)– NFI (Normed Fit Index)
RMSEA
• Parsimony-adjusted index• Want small values– Excellent < .06 (not a typo different than book)– Good < .08– Acceptable < .10– Eeek > .10
• Report CI!
Pclose
• Tests if the RMSEA is in the excellent range• You want p > .50 to show that there is a high
probability that RMSEA is effectively zero
SRMR
• Parsimony-adjusted index• Want small values– Excellent < .06 (not a typo different than book)– Good < .08– Acceptable < .10– Eeek > .10
GFI
• Do not use this sucker unless you want to get a nasty review.– GFI, AGFI, PGFI
• Lots of research showing it’s positively biased• Want large values
CFI
• Incremental Fit Index– Values are 0 to 1 (sometimes you’ll get slightly
over 1, usually indicates something is wrong)– Want high values• Excellent >.95• Good > .90• Blah < .90
The Other FIs
• NFI – a variation of the CFI, as it was said to underestimate for small samples
• RFI (relative fit index)• IFI (incremental fit index)• TLI (Tucker Lewis Index)– All have the same basic rules and formulas as CFI
See Tabachnick 720-725 for how these and the next slides are calculated
Some other statistics
• Pratio – parsimony index• PNFI, PCFI are parsimony adjustments for NFI,
CFI• NCP – noncentrality parameter (tells you how
much it leans from the normal for that distribution)
Some other statistics
• FMIN – minimum discrepancy function used to calculate chi-square and other statistics – Include confidence interval in Amos
Model Comparisons
• Let’s say you want to adjust your model– You can compare the adjusted model to the
original model to determine if the adjustment is better
• Let’s say you want to compare two different models– You can compare their fits to see which is better
Model Comparisons
• Nested models– If you can create one model from another by the
addition or subtraction of parameters, then it is nested• Model A is said to be nested within Model B, if
Model B is a more complicated version of Model A. – For example, a one-factor model is nested within a
two-factor as a one-factor model can be viewed as a two-factor model in which the correlation between factors is perfect).
Nested Models
• Chi-square difference test– | Subtract Model 1 CMIN – Model 2 CMIN |– Subtract Model 1 df – Model 2 df– Use a chi-square table to look up p < .05 for
difference in df– See if the first step is greater than that value• If yes, you say the model with the lower chi-square is
better• If no, you say they are the same and go with the
simpler model
Nested Models
• So how can I tell what to change?• NOTE: JUST CHANGE ONE THING AT A TIME!• Use modification indices!– They tell you what the chi-square change would be
if you add the path suggested.– Based on X2(1) – called a Lagrange Multiplier
• Remember that p < .05 = 3.84, so Amos automatically gives you everything > 4.
• You can change this to see fewer options if you have a lot.
Non-Nested Models
• AIC – Akaike Information Criterion– Related CAIC (consistent AIC)
• BIC – Bayesian Information Criterion• BCC – Browne-Cudeck Criterion
– All of these are how much the sample will cross validate in the future
– You want them to be small, so you pick the smallest one of the two models (how different?)
Non-Nested Models
• ECVI – expected cross validation index• MECVI – modified ECVI
– Again, you want small values, so you pick the model with the smallest ECVI
OMG!
• So what to do?– Mainly people report: X2, RMSEA, SRMR, CFI– Determine the type of model change to use the
right model comparison statistic