http://apm.sagepub.com
Applied Psychological Measurement
DOI: 10.1177/0146621609338592 2010; 34; 122 originally published online Aug 17, 2009; Applied Psychological Measurement
Chongming Yang, Sandra Nay and Rick H. Hoyle Parceling, Latent Scoring, and Shortening Scales
Three Approaches to Using Lengthy Ordinal Scales in Structural Equation Models:
http://apm.sagepub.com/cgi/content/abstract/34/2/122 The online version of this article can be found at:
Published by:
http://www.sagepublications.com
can be found at:Applied Psychological Measurement Additional services and information for
http://apm.sagepub.com/cgi/alerts Email Alerts:
http://apm.sagepub.com/subscriptions Subscriptions:
http://www.sagepub.com/journalsReprints.navReprints:
http://www.sagepub.com/journalsPermissions.navPermissions:
http://apm.sagepub.com/cgi/content/refs/34/2/122 Citations
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
Three Approaches toUsing Lengthy Ordinal Scalesin Structural Equation ModelsParceling, Latent Scoring, and Shortening ScalesChongming YangSandra NayRick H. HoyleDuke University
Lengthy scales or testlets pose certain challenges for structural equation modeling (SEM) if all
the items are included as indicators of a latent construct. Three general approaches to modeling
lengthy scales in SEM (parceling, latent scoring, and shortening) have been reviewed and eval-
uated. A hypothetical population model is simulated containing two exogenous constructs with
14 indicators each and an endogenous construct with four indicators. The simulation generates
data sets with varying numbers of response options, two types of distributions, factor loadings
ranging from low to high, and sample sizes ranging from small to moderate. The population
model is varied to incorporate one of the following: (a) single parcels, (b) various parcels as
indicators of two exogenous constructs, (c) latent scores as observed exogenous variables, and
(d) four and six individual items as indicators of two exogenous constructs. The dependent
variables evaluated are biases in the covariance and partial covariance population parameters.
Biases in these parameters are found to be minimal under the following conditions: (a) when
parcels of indicators of five response options are used as indicators of two latent exogenous
constructs, (b) when latent scores are used as observed variables at sample sizes above 100 and
with indicators that are relatively less skewed in the case of dichotomous indicators, and (c)
when four or six individual items with high or diverse factor loadings are used as indicators of
two exogenous constructs. These findings provide guidelines for resolving the inconsistency of
findings from applying various approaches to empirical data.
Keywords: testlets; latent scores; scale length; growth modeling; structural equation
modeling
Lengthy ordinal scales or testlets pose challenges for structural equation modeling
(SEM) if all the items are used as indicators of a latent construct. For instance, a
model could have too many parameters to estimate relative to the available sample size,
resulting in reduced power to detect important parameters. In addition, it might not fit the
data sufficiently well because individual items may have less than ideal measurement
Applied Psychological
Measurement
Volume 34 Number 2
March 2010 122-142
� 2010 The Author(s)
10.1177/0146621609338592
http://apm.sagepub.com
Authors’ Note: This research was supported by National Institute on Drug Abuse (NIDA) Grant P30
DA023026. Its contents are solely the responsibility of the authors and do not necessarily represent the official
views of NIDA. Please address correspondence to Chongming Yang, Social Science Research Institute, Duke
University, Durham, NC 27708; e-mail: [email protected].
122
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
properties, leading to the rejection of a plausible model. Three general approaches that
can be used to address these challenges are parceling, shortening, and transforming a
lengthy scale into latent score variables through preliminary analyses. Each approach has
advantages and disadvantages. The research reported here was designed to evaluate these
approaches under certain data conditions to determine which ones best reflect the true
model parameters, particularly the covariance of the exogenous constructs and partial cov-
ariances between the exogenous and endogenous constructs.
Parceling
The prevailing approach to incorporating a lengthy scale into SEM has been using the
mean or sum of the scale as an indicator of a latent construct (Cattell, 1956). Empirical
justifications for parceling include increasing reliability, achieving normality, adapting to
small sample sizes, reducing idiosyncratic influences of individual items, simplifying
interpretation, and obtaining better model fit (Bandalos & Finney, 2001). Methods for par-
celing include parceling all items into a single parcel, splitting all odd and even items into
two parcels, balancing item discrimination and difficulty across three or four parcels (e.g.,
item–construct balance; Little, Cunningham, Shahar & Widaman, 2002), randomly select-
ing a certain number of items to create three or four parcels (e.g., Kishton & Widaman,
1994; Nasser & Wisenbaker, 2003), and parceling items that have similar factor loadings
(i.e., contiguity; Cattell & Burdsal, 1975). Desirable conditions for parceling that have
been identified so far include having more than 12 items (Marsh, Hau, Balla, & Grayson,
1998) and having items that reflect a unidimensional construct (Hall, Snell, & Foust,
1999).
Psychometrically, parceling has been controversial despite the above advantages and its
prevalence in practice (Bandalos & Finney, 2001; Little et al., 2002). One concern was
that parceling results in a loss of information about the relative importance of individual
items (Marsh & O’Neill, 1984), because items are implicitly weighted equally in parcels
(Bollen & Lennox, 1991). Another concern was that parceling of ordinal scales results in
indicators with undefined values, potentially changing the original relations between the
indicators and latent variables, for instance, from nonlinear to linear relations (Coanders,
Satorra, & Saris, 1997). Parceling binary or trichotomous items could result in limited
range as opposed to the latent trait scale, thereby biasing variance and covariance para-
meters in SEM (Wright, 1999). Compared with using individual items, parceling could
underestimate the relations of the latent variables if the reliability of the scale is low
(Shevlin, Miles, & Bunting, 1997).
Empirically, the effects of parceling ordinal indicators on the covariance and partial cov-
ariance of latent constructs have been unknown under some conditions. Previous studies on
the effects of parceling on estimates of covariance parameters have typically considered
only parceling of continuous indicators with equal discrimination functions or factor load-
ings (e.g., Alhija & Wisenbaker, 2006; Bandolas, 2002; Hau & Marsh, 2004). Under these
conditions, certain parceling strategies yielded little difference in the covariance of two
latent constructs. However, most lengthy ordinal scales do not have equal item discrimina-
tion functions. In the case of continuous indicators, equal loadings imply that the latent
Yang et al. / Modeling Lengthy Measures 123
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
construct can explain equal variances and covariances within each parceling condition.
Thus, it is not surprising that parceling has performed well in reflecting the covariance or
correlation of latent constructs. In the case of ordinal indicators, a recent study parceled 12
ordinal items with six categories, normal distribution, and loadings varying from 0.65 to
0.85, and found no difference in predictive utility in a one-exogenous and one-endogenous
construct model (Sass & Smith, 2006). Although this finding was supportive of parceling,
such results could have arisen from the fact that ordinal indicators in the study approached
the properties of normal continuous indicators (Coanders et al., 1997) and both exogenous
and endogenous indicators were parceled identically. Given the limited conditions con-
trolled for in previous studies, it remains unclear to what extent various parceling strategies
have reflected the true covariances and partial covariances of a multiple exogenous con-
structs model when ordinal indicators have a broader range of categories (e.g., two, three,
five, and seven) and varied discrimination functions. It could be assumed that the partial
covariance parameters would not be unduly biased by parceling normally distributed multi-
ple-category indicators, because such indicators have approximately linear relations with
their latent construct; moreover, parceling as a linear transformation typically does not
change the indicators’ linearity and have performed well (Coanders et al., 1997; Ferrando,
2009). However, no assumptions could be made about the magnitude and directions of
biases of parceling indicators of two or three categories and varied measurement qualities.
Latent Scoring
Lengthy ordinal scales can be transformed through item response theory (IRT) model-
ing into latent scores for further modeling (Hambleton, Swaminathan, & Rogers, 1991).
IRT describes the probabilities that individuals will respond to a set of test items given a
particular level of ability or personality trait. Samejima (1969) proposed the following
two-parameter model (ai and bk) for ordinal scales:
P= 1
1þ expð�aiðyj � bk�1ÞÞ� 1
1þ expð�aiðyj � bkÞÞ,
where P is the probability that person j responds to a particular category k of an item at a
given trait level yj, k = 1, 2, . . . , m categories, ai = a discrimination parameter of an item,
bk is a threshold at which person j has a .50 probability for the chosen response category,
and exp stands for exponentiation. After estimating unknown ai, bk, and yj, the latent score
(yj) for each individual can be saved for subsequent modeling. Latent scores obtained
through this process are theoretically intervals with a normal distribution that best reflects a
population. Such a transformation is more likely to eliminate artifactual effects (Embretson,
1996) and detect legitimate effects (Fletcher, 2005) than using raw scores.
The two-parameter IRT model can be equivalent to confirmatory factor analysis (CFA)
with categorical indicators (Takane & de Leeuw, 1987) within the advanced latent vari-
able modeling framework of Mplus (L. K. Muthen & Muthen, 1998-2006). CFA in Mplus
typically models the probability (P) of choosing a response category (m) given the indivi-
dual’s latent trait level (Z) with a probit link (�) and a residual (d):
124 Applied Psychological Measurement
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
Pðm= 1|ZÞ=�½ð−tþ lZÞd�1=2�,
where thresholds (t), factor loadings (l), and factor scores (Z) are conceptually equivalent
to the threshold (b), the discrimination parameter (a), and the latent trait score (y) in the
two-parameter IRT model, respectively (Reise, Widaman, & Pugh, 1993). Although cer-
tain transformations are needed to obtain exactly the same parameter estimates of a(=ld�1=2) and b (=t/l) from a typical two-parameter IRT modeling (L. K. Muthen &
Muthen, 2006), factor scores obtained from CFA with categorical indicators are equiva-
lent to the latent scores from an IRT modeling. Because CFA has been recommended as
the first step of SEM to ensure unidimensionality and measurement quality (Anderson &
Gerbing, 1988), and factor scores are by-products of this process, it would be of practical
value to ascertain whether using factor scores to reduce test length for SEM would result
in accurate estimates of covariance and partial covariance of latent constructs. (Factor
scores and latent scores are used interchangeably hereafter.)
Sample size could be an important factor to consider in using factor scores. Item
response modeling requires large samples (N > 350) to yield accurate parameter estimates
(Reise & Yu, 1990). Similarly, small samples that are typical of social science research
might yield less accurate parameter estimates from CFA with categorical indicators.
Although the minimum sample size for CFA with categorical indicators depends on model
size, distribution of variables, strength of relationships, and proportions of missing values
(B. O. Muthen & Muthen, 2002), factor scores could be assumed to perform well in repre-
senting the true relations among latent variables with relatively large samples.
Shortening a Scale
A few indicators of a latent construct with certain content and predictive validity may be
selected from a lengthy scale to yield a shortened scale (Moore, Halle, Vandivere, & Marina,
2002), which can be easily incorporated into SEM. According to behavior domain theory
(Guttman, 1955; McDonald, 1996), an underlying construct could have an infinite number
of indicators. Therefore, a shortened scale is merely a smaller sample of all possible indica-
tors. One need not be overly concerned that the shortened scale may not be commensurate
with the large scale in its content validity, because neither is a perfect measure. Empirically,
a 6-item scale selected from 27 items with empirical data could be equivalent to the full
scale (Moore et al., 2002). It has been shown that a smaller number of continuous indicators
with moderate diversity of loadings performed as well as six indicators with high diversity
of loadings in recovering the true correlation of the two constructs (Little, Lindenberger, &
Nesselroade, 1999), although it was unclear to what extent such finding could be general-
ized to ordinal scales. Short scales can be used in large-scale surveys, in which many con-
structs are assessed, if they have desirable measurement properties and similar predictive
utility to their large source scales (e.g., Stephenson, Hoyle, Palmgreen, & Slater, 2003).
Shortening a widely used scale has raised other issues besides concerns about content
validity. One issue was how to determine the minimum number of indicators. From the
perspective of model identification, Kenny (1979) advocated that four indicators are the
best for a latent construct and ‘‘anything more is gravy.’’ Shortening a large scale to fewer
Yang et al. / Modeling Lengthy Measures 125
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
than four items might undermine its content validity, although its predictive validity could
be retained (Little et al., 1999). Another issue was which strategies to apply to reduce the
scale length. One could rely on the magnitude of correlations between the endogenous
and exogenous construct indicators, as in the study by Moore et al. (2002), or on the width
of the behavioral domain the indicators reflect (Little et al., 1999), or on discrimination
functions (factor loadings) because measurement errors of the exogenous constructs
attenuate associations with other constructs (Bollen & Lennox, 1991). Based on these
findings and considerations, shortening an ordinal scale to four or six indicators appeared
reasonable. Therefore, in this study, four or six items with different measurement proper-
ties were selected from a hypothetical lengthy ordinal scale to examine their efficacy in
recovering population parameters.
In sum, it is not known how well various approaches to incorporating lengthy ordinal scales
into SEM would reflect the true covariance and partial covariance of latent constructs under
conditions of varied discrimination functions or factor loadings, different numbers of response
category, distributions, sample sizes, and full SEM models.. An efficient method to gain such
knowledge was to simulate all these conditions with artificial data generated from a known
population model, so that the various approaches could be applied to these artificial data and
evaluated against the population model (Paxton, Curran, Bollen, Kirby, & Chen, 2001). Find-
ings from these simulations can also be applied to empirical data to solve practical issues.
Design and Analysis
Population Model, Parameters, Scales, and Data
The hypothetical population model was set to have two exogenous constructs (F1 and
F2) and one endogenous construct (F3), as shown in Figure 1. This model allows estima-
tion of the covariance between the exogenous constructs and the effects of the exogenous
constructs on the endogenous construct (partial covariances), as has been used in previous
simulation studies (e.g., Bandalos, 2002; B. O. Muthen, 1984). Each exogenous construct
was measured by 14 items (X1-X14 and X15-X28), and the endogenous construct was
measured by four items (Y1-Y4). The factor loadings and structural coefficients are also
shown in Figure 1. The population parameters estimated included the partial covariance
from F1 to F3 (g1 = 0.50), the partial covariance from F2 to F3 (g2 = 0.40), and the covar-
iance between F1 and F2 (f= 0.20). The variances of the exogenous constructs and dis-
turbance (d) of the endogenous construct were assigned at 0.50 and 0.60, respectively.
Multivariate continuous data were first generated from the population model and then
categorized into two-, three-, five-, and seven-category ordinal variables. The cut points
(thresholds) of the underlying dimension, which are equivalent to z values of a normally
distributed random variable, were selected to specify the proportion of each response cate-
gory and thereby vary the distributions of the categorized variables. The threshold and dis-
tribution of each response category are listed in Table 1. Sample sizes considered were
100, 350, and 600, which reflected the size of typical clinical studies and medium to large
intervention studies. Using the Mplus program (Version 4), 100 data sets were generated
for each of the 24 conditions (2 levels of distribution× 4 levels of response category× 3
126 Applied Psychological Measurement
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
levels of sample size). For the sake of comparisons, various approaches to modeling the
lengthy scales (parceling, latent scoring, and shortening) were viewed as repeated treat-
ments of the same data sets that had been generated and categorized for different condi-
tions. The acceptable range of average bias in this study was set at ±0.14, rounding to 0.1
in absolute value.
Figure 1
Population Model
F2
x28
x27
x26
x25
x24
x23
x22
x21
x20
x19
x18
x17
x16
x15
.40
.43
.46
.50
.53
.56
.60
.63
.66
.70
.73
.76
.80
.83
F1
x14
x13
x12
x11
x10
x9
x8
x7
x6
x5
x4
x3
x2
x1e1
e2
e3
e4
e5
e6
e7
e8
e9
e10
e11
e12
e13
e14
e15
e16
e17
e18
e19
e20
e21
e22
e23
e24
e25
e26
e27
e28
.43
.46
.53
.56
.60
.63
.66
.70
.73
.76
.80
.83
.86
.90
F3
y1 e29
y2 e30
y3 e31
y4 e32
.70
.70
.70
.70
.50
.40
d
.20
Yang et al. / Modeling Lengthy Measures 127
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
Tab
le1
Pop
ula
tion
Ind
icato
rL
oad
ings,
Item
sC
hose
nfo
rE
ach
Parc
el,
an
dIt
emL
oad
ings
of
Sh
ort
ened
Sca
les
Po
pu
lati
on
Par
cels
Sh
ort
ened
Sca
leo
fF
ou
rIn
dic
ato
rsS
ho
rten
edS
cale
of
Six
Ind
icat
ors
Ind
icat
or
Lo
adin
g
Od
d-E
ven
Sp
lit
Bal
ance
Ran
do
m
Fo
ur
Ran
do
m
Six
Co
nti
gu
ou
s
Fo
ur
Co
nti
gu
ou
s
Six
Div
ersi
tyH
igh
Med
ium
Lo
wD
iver
sity
Hig
hM
ediu
mL
ow
X1
.90
AA
AA
AA
.90
.90
.90
.90
X2
.86
BB
AB
AA
.86
.86
X3
.83
AC
CC
AA
.83
.83
X4
.80
BC
DD
AB
.80
.80
.80
X5
.76
AB
BE
BB
.76
.76
.76
.76
X6
.73
BA
BE
BB
.73
.73
.73
X7
.70
AA
DF
BC
.70
.70
.70
X8
.66
BB
BF
BC
.66
.66
X9
.63
AC
AB
CD
.63
.63
.63
.63
X1
0.6
0B
CB
CC
D.6
0.6
0.6
0.6
0
X1
1.5
6A
BA
AC
E.5
6.5
6.5
6
X1
2.5
3B
AC
DD
E.5
3.5
3
X1
3.4
6A
CA
FD
F.4
6
X1
4.4
3B
BD
ED
F.4
3.4
3.4
3
X1
5.8
3C
DD
GE
G.8
3.8
3.8
3.8
3
X1
6.8
0D
EB
HE
G.8
0.8
0
X1
7.7
6C
FC
HE
G.7
6.7
6
X1
8.7
3D
FA
GE
H.7
3.7
3.7
3
X1
9.7
0C
ED
IF
H.7
0.7
0.7
0
X2
0.6
6D
DA
JF
H.6
6.6
6.6
6.6
6
X2
1.6
3C
DB
KF
I.6
3.6
3.6
3
X2
2.6
0D
EC
KF
I.6
0.6
0
X2
3.5
6C
FA
JG
J.5
6.5
6.5
6.5
6
X2
4.5
3D
FA
LG
J.5
3.5
3.5
3
X2
5.5
0C
ED
IG
K.5
0.5
0.5
0.5
0
X2
6.4
6D
DB
KH
K.4
6.4
6
X2
7.4
3C
FC
HH
L.4
3
X2
8.4
0D
EB
LH
L.4
04
0.4
0
No
te:
Th
esa
me
lett
ers
ina
colu
mn
des
ign
ate
ind
icat
ors
for
the
sam
ep
arce
l.
128
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
Parcel and Latent Score Creation and ItemSelection for Shortened Scales
Parcels were created from indicators of only the exogenous constructs in this study. A
single parcel was created for each of F1 and F2 by taking the means of all 14 indicators of
F1 and F2, respectively. Two odd–even parcels were created for each of F1 and F2 by tak-
ing the means of the odd- and even-numbered indicators. Three item-to-construct balance
parcels were created by selecting an item for each parcel in the order of discrimination func-
tions and repeating the selection process in a reversed order, as indicated by letters in the
Balance column in Table 1. Four random parcels were created by randomly selecting indi-
cators without replacement. Four and six adjacent-loading parcels were created by taking
the means of indicators that had contiguous loadings. The final indicators for each parcel are
listed in Table 1 and denoted by capital letters. Latent scores for the two exogenous con-
structs (F1 and F2) were obtained by fitting a measurement model of the three constructs of
the population model to the generated categorical data. The indicators were specified as
categorical and the latent (factor) scores produced from this process were saved in the data
files. Four shortened scales were created by selecting four or six individual indicators of
each of F1 and F2. The factor loadings of these exogenous construct indicators were chosen
to be diverse, high, moderate, and low, as listed in Table 1.
Model Estimation
In the case of single parcels or latent scores, the population model was altered to two
single parcels or two latent scores as the observed exogenous variables (in place of F1 and
F2) predicting the endogenous construct (F3). With two or more parcels, the population
model was altered to use parcels as indicators of F1 and F2. Means and standard devia-
tions of the parameter estimates and the number of converged solutions were recorded in
this process. Mean biases (parameter estimate – population parameter) were reported or
calculated for subsequent analyses (Paxton et al., 2001).
The performances of various estimation methods in SEM have been compared for mod-
els with categorical indicators. The number of categories typically selected has been two,
three, five, or seven. Under various distributions, the weighted least squares estimator with
degrees of freedom adjusted for means and variances (WLSMV) in Mplus has proven to
be fairly robust to various categorical indicators with large sample sizes (B. O. Muthen &
Kaplan, 1985, 1992). To eliminate any effects from the estimation method, indicators of
the endogenous latent variables were specified as categorical and all the models were esti-
mated with the WLSMV estimator.
Empirical Data
The empirical data were adopted from the Child Development Project, an ongoing long-
itudinal study of children’s social and emotional development (Lansford et al., 2006). A
total of 585 families were recruited from two cohorts in consecutive years, 1987 and
1988, from Nashville and Knoxville, Tennessee, and Bloomington, Indiana. Data collec-
tion began the year before the children entered kindergarten and data have been collected
Yang et al. / Modeling Lengthy Measures 129
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
annually ever since. A linear growth model of aggression was chosen for the empirical
example, because it could provide an opportunity to examine the effects of various
approaches on the mean levels of latent constructs. Aggression was measured from kin-
dergarten through adolescence using the Child Behavior Checklist (Achenbach, 1978),
with less than 30% attrition at the final measurement of aggression. Our main interest
involved examining whether findings from the simulation study could be applied to resol-
ving any practical issues.
Results
Based on converged solutions, the average biases of each approach in the three para-
meters (g1, g2, and f) for each data condition were recorded and are depicted in Table 2.
The standard deviations and numbers of converged solutions for each condition are listed
in Table 3.
Parceling
Data in Table 2 reveal a clear pattern and three key findings: First, any parceling of
ordinal indicators of two or three response categories overestimated the partial covar-
iances but underestimated the covariance of the exogenous constructs. Second, parceling
of indicators with seven response categories underestimated the partial covariances (g1
and g2) by 0.10 to 0.24 but more accurately reflected the covariance of the two exogenous
constructs (f). Third, odd–even split, random parceling (4 or 6 parcels), and item–
construct-balance parceling of indicators of five response categories biased the estimates
of the effects (g1 and g2) maximally by 0.1 in absolute value, and biased the correlation of
the two constructs maximally by 0.13 in absolute value. Parceling of indicators with con-
tiguous factor loadings attenuated the partial covariances (g1 and g2) by more than 0.1,
which appeared to be slightly less advantageous than other parceling approaches. Thus,
parceling (odd–even split, random, and item-construct balance) was most effective when
categorical indicators had five response categories and parcels were further used as indica-
tors of the latent constructs.
Latent Scoring
An analysis of variance confirmed that the number of response categories produced
some difference in g1, F(3, 14)= 40.96, p< .01, and g2, F(3, 14)= 18.99, p< .01; type of
distribution produced some difference in g1, F(1, 14)= 16.34, p< .01; and the two effects
were dependent on each other in g1, evidenced by a significant interaction effect between
the number of response categories and type of distribution, F(3, 14)= 10.27, p< .01. As
shown in Table 2, latent scoring biased the two direct effect estimates maximally by 0.08
in absolute value in all the data conditions, except in situations with binary indicators and
small samples or extremely skewed binary indicators. The average bias in the covariance
of F1 and F2 (f) was consistently between 0.06 and 0.11 across all the conditions. Thus,
130 Applied Psychological Measurement
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
Tab
le2
Ab
solu
teB
ias
inP
op
ula
tion
Mod
elP
ara
met
ers
of
Each
Ap
pro
ach
toL
ength
yS
cale
sU
nd
erD
iffe
ren
tD
ata
Con
dit
ion
s
Dat
aS
etC
on
dit
ion
s
Ap
pro
ach
es
Lat
ent
Sco
res
Sin
gle
Par
cel
Ev
en-S
pli
tP
arce
lsIt
em−C
on
stru
ctB
alan
ce
Cat
.D
istr
ibu
tio
n(T
hre
sho
lds)
Ng 1
g 2f
g 1g 2
fg 1
g 2f
g 1g 2
f
Tw
o7
0,3
0(0
.52
44
)1
00
.06
.15
−.0
9.4
7.4
2−.
19
0.9
70
.76
−.1
90
.91
0.9
2−.
19
35
0.0
4.0
8−.
10
.42
.39
−.1
90
.82
0.7
3−.
19
0.7
50
.73
−.1
9
60
0.0
5.0
6−.
11
.41
.37
−.1
90
.81
0.6
8−.
19
0.7
60
.68
−.1
9
85
,1
5(1
.03
6)
10
0.1
7.1
9−.
10
.63
.55
−.2
01
.26
1.2
5−.
19
1.2
61
.21
−.1
9
35
0.1
3.1
4−.
11
.66
.56
−.2
01
.25
1.1
1−.
20
1.2
11
.09
−.1
9
60
0.1
6.1
1−.
11
.68
.58
−.2
01
.28
1.1
3−.
20
1.2
81
.09
−.1
9
Th
ree
60
,3
0,1
0(0
.25
53
,1
.28
2)
10
0.0
1.0
2−.
08
.10
.10
−.1
70
.29
0.2
6−.
17
0.2
90
.28
−.1
7
35
0.0
3.0
7−.
11
.09
.11
−.1
80
.29
0.3
0−.
18
0.2
60
.28
−.1
7
60
0.0
3.0
5−.
10
.09
.11
−.1
80
.28
0.2
8−.
17
0.2
70
.28
−.1
7
70
,2
5,5
(0.5
24
4,1
.64
5)
10
0.0
2.0
3−.
06
.10
.16
−.1
70
.36
0.3
8−.
17
0.3
10
.41
−.1
7
35
0.0
5.0
8−.
11
.19
.19
−.1
80
.45
0.4
2−.
18
0.4
10
.43
−.1
8
60
0.0
3.0
6−.
10
.18
.19
−.1
80
.42
0.4
2−.
18
0.4
00
.40
−.1
8
Fiv
e1
02
54
02
51
0
(−1
.28
2,−0
.38
53
,
0.6
74
5,
1.2
82
)
10
0.0
5.0
3−.
10
−.1
4−.
12
−.1
2−0
.03
−0.0
4−.
12
−0.0
3−0
.04
−.1
1
35
0−.
02
.02
−.0
9−.
15
−.1
1−.
11
−0.0
7−0
.03
−.1
1−0
.08
−0.0
3−.
11
60
0−.
01
.02
−.1
0−.
15
−.1
1−.
11
−0.0
7−0
.03
−.1
1−0
.08
−0.0
4−.
11
25
,4
0,2
0,1
0,
5
(−0
.67
45
,0
.38
53
,
1.0
36
,1
.64
5)
10
0−.
02
.04
−.0
9−.
16
−.1
1−.
12
−0.0
6−0
.02
−.1
1−0
.08
−0.0
1−.
11
35
0−.
01
.02
−.1
0−.
16
−.1
1−.
12
−0.0
7−0
.01
−.1
2−0
.08
−0.0
3−.
11
60
0.0
0.0
1−.
10
−.1
5−.
11
−.1
2−0
.06
−0.0
2−.
11
−0.0
7−0
.04
−.1
1
Sev
en5
,1
0,2
0,3
0,
20
,1
0,5
(−1
.64
5,−1
.03
6,−0
.38
53
,
0.3
85
3,
1.0
36
,1
.64
5)
10
0−.
02
.04
−.0
9−.
26
−.1
8−.
04
−0.1
9−0
.12
−.0
3−0
.14
−0.1
2−.
03
35
0−.
02
.02
−.0
9−.
24
−.1
8−.
04
−0.1
8−0
.13
−.0
3−0
.20
−0.1
3−.
02
60
0−.
01
.01
−.1
0−.
24
−.1
8−.
04
−0.1
8−0
.13
−.0
3−0
.19
−0.1
3−.
02
14
,3
4,2
2,1
4,
9,5
,2
(−1
.08
0,−0
.05
15
,0
.52
44
,
0.9
94
5,
1.4
76
,2
.05
4)
10
0.0
3−.
02
−.0
9−.
25
−.2
1−.
05
−0.1
6−0
.16
−.0
4−0
.17
−0.1
5−.
04
35
0.0
0.0
0−.
10
−.2
4−.
19
−.0
5−0
.18
−0.1
3−.
04
−0.1
9−0
.14
−.0
3
60
0.0
0.0
2−.
10
−.2
4−.
19
−.0
5−0
.18
−0.1
3−.
04
−0.1
8−0
.13
−.0
3
No
te:
Cat
.=n
um
ber
of
cate
go
ries
.
131
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
Tab
le2
(con
tin
ued
)
Ap
pro
ach
es
Ran
do
miz
edto
Fo
ur
Par
cels
Ran
do
miz
edto
Six
Par
cels
Fo
ur
Par
cels
of
Co
nti
gu
ou
sL
oad
ing
s
Six
Par
cels
of
Co
nti
gu
ou
sL
oad
ing
s
Fo
ur
Item
so
f
Div
erse
Lo
adin
gs
Fo
ur
Item
so
f
Hig
hL
oad
ing
s
g 1g 2
fg 1
g 2f
g 1g 2
fg 1
g 2f
g 1g 2
fg 1
g 21
f
1.2
60
.97
−.1
91
.04
0.8
7−.
19
0.7
1.6
7−.
19
.76
0.6
5−.
19
.20
.27
−.0
8.0
2−.
01
−.0
9
0.9
70
.76
−.1
90
.62
1.1
3−.
19
0.5
9.5
6−.
19
.58
0.5
6−.
19
−.1
2.0
3−.
08
−.0
6−.
08
−.0
9
0.9
10
.74
−.1
90
.95
0.6
6−.
19
0.6
0.5
2−.
19
.57
−0.3
0−.
19
−.0
9−.
06
−.0
8−.
11
−.0
7−.
09
1.4
41
.61
−.2
01
.69
1.2
5−.
20
1.0
8.9
5−.
19
.95
1.0
7−.
19
.08
.19
−.0
9.0
2−.
01
−.0
8
1.4
61
.16
−.2
01
.49
1.0
8−.
20
0.9
9.8
2−.
19
.95
0.7
9−.
19
−.0
7.0
3−.
09
−.0
6−.
01
−.0
9
1.5
11
.17
−.2
01
.57
1.0
5−.
20
0.9
7.8
1−.
19
.95
0.7
9−.
19
−.0
5−.
04
−.1
0−.
08
−.0
5−.
09
0.4
10
.35
−.1
80
.45
0.2
7−.
17
0.1
9.1
6−.
16
.15
0.1
6−.
16
.05
−.1
0−.
07
−.0
7.0
0−.
06
0.3
80
.30
−.1
80
.40
0.2
6−.
18
0.1
6.1
8−.
16
.15
0.1
7−.
16
−.1
1.0
0−.
10
−.1
0−.
03
−.1
0
0.3
90
.30
−.1
80
.39
0.2
5−.
18
0.1
5.1
7−.
16
.14
0.1
5−.
16
−.1
0−.
04
−.0
9−.
09
−.0
7−.
09
0.4
90
.36
−.1
80
.49
0.4
1−.
17
0.2
1.2
4−.
15
.21
0.2
4−.
15
.31
−.5
6−.
04
−.2
2.2
0−.
04
0.5
90
.45
−.1
90
.59
0.4
1−.
18
0.2
8.2
8−.
17
.26
0.2
7−.
17
−.0
7.0
0−.
10
−.0
9−.
02
−.0
9
0.5
40
.45
−.1
80
.55
0.4
1−.
18
0.2
5.2
8−.
17
.24
0.2
6−.
17
−.1
1−.
05
−.0
9−.
11
−.0
8−.
09
0.0
2−0
.01
−.1
20
.06
−0.0
4−.
12
−0.1
0−.
08
−.0
8−.
10
−0.0
9−.
07
.01
.21
−.1
1−.
04
−.0
3−.
09
−0.0
2−0
.02
−.1
20
.00
−0.0
5−.
12
−0.1
4−.
09
−.0
7−.
15
−0.1
0−.
06
−.1
0−.
04
−.0
9−.
10
−.0
5−.
09
−0.0
2−0
.02
−.1
2−0
.01
−0.0
5−.
12
−0.1
4−.
10
−.0
7−.
15
−0.1
1−.
06
−.0
9−.
07
−.0
9−.
11
−.0
5−.
09
−0.0
20
.01
−.1
20
.02
0.0
0−.
12
−0.1
3−.
06
−.0
7−.
13
−0.0
7−.
07
−.0
5−.
09
−.0
8−.
09
−.0
3−.
08
−0.0
2−0
.02
−.1
30
.00
−0.0
4−.
12
−0.1
5−.
09
−.0
8−.
15
−0.1
0−.
07
−.0
8−.
07
−.0
9−.
10
−.0
6−.
09
−0.0
1−0
.02
−.1
20
.00
−0.0
4−.
12
−0.1
3−.
09
−.0
7−.
14
−0.1
0−.
07
−.1
0−.
05
−.0
9−.
08
−.0
7−.
09
−0.1
6−0
.10
−.0
5−0
.14
−0.1
1−.
05
−0.2
4−.
16
.04
−.2
4−0
.16
.04
−.0
5.0
5−.
09
−.1
3.0
3−.
08
−0.1
5−0
.11
−.0
5−0
.14
−0.1
3−.
05
−0.2
3−.
17
.04
−.2
4−0
.18
.06
−.0
8−.
07
−.0
8−.
10
−.0
6−.
09
−0.1
5−0
.13
−.0
5−0
.14
−0.1
4−.
04
−0.2
4−.
18
.05
−.2
4−0
.18
.06
−.1
0−.
06
−.0
9−.
10
−.0
7−.
09
−0.1
3−0
.14
−.0
6−0
.10
−0.1
5−.
05
−0.2
2−.
19
.04
−.2
2−0
.19
.04
−.0
2−.
05
−.0
8−.
02
−.1
1−.
08
−0.1
4−0
.13
−.0
6−0
.14
−0.1
4−.
05
−0.2
3−.
18
.03
−.2
4−0
.18
.05
−.0
9−.
03
−.0
9−.
08
−.0
8−.
09
−0.1
4−0
.13
−.0
6−0
.13
−0.1
4−.
05
−0.2
3−.
18
.04
−.2
4−0
.18
.05
−.0
8−.
06
−.0
9−.
09
−.0
6−.
09
132
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
Tab
le2
(con
tin
ued
)
Ap
pro
ach
es
Fo
ur
Item
so
f
Med
ium
Lo
adin
gs
Fo
ur
Item
so
f
Lo
wL
oad
ing
s
Six
Item
so
f
Div
erse
Lo
adin
gs
Six
Item
so
f
Hig
hL
oad
ing
s
Six
Item
so
f
Med
ium
Lo
adin
gs
Six
Item
so
f
Lo
wL
oad
ing
s
g 1g 2
fg 1
g 2f
g 1g 2
fg 1
g 2f
g 1g 2
fg 1
g 2f
.11
.13
−.1
1−.
05
.62
−.1
2−.
11
.08
−.0
8−.
02
.02
−.0
7.2
6.2
1−.
10
0.3
80
.49
−.1
2
−.0
8.0
8−.
11
.27
.26
−.1
5−.
09
−.0
5−.
08
−.1
0−.
07
−.0
9−.
06
.05
−.1
20
.08
0.2
0−.
14
−.0
2.0
0−.
12
.12
.12
−.1
5−.
10
−.0
7−.
09
−.1
0−.
08
−.0
9−.
04
−.0
2−.
12
0.0
30
.07
−.1
3
.32
−.1
8−.
08
.05
.45
−.1
1.1
2.2
2−.
08
−.0
4.1
3−.
07
.16
.17
−.0
8−0
.03
0.1
3−.
12
.04
.06
−.1
2.3
2.1
9−.
14
−.0
7−.
01
−.0
9−.
08
−.0
4−.
09
.01
.06
−.1
20
.11
0.4
5−.
14
−.0
6.0
8−.
11
.33
.15
−.1
5−.
06
−.0
5−.
09
−.0
9−.
06
−.0
9−.
03
.03
−.1
10
.18
0.0
9−.
14
.05
.26
−.0
9.4
8−.
04
−.1
3−.
03
−.0
6−.
06
−.0
8−.
07
−.0
6.0
2.1
2−.
09
0.1
50
.16
−.1
2
−.0
2.0
7−.
12
.25
.05
−.1
4−.
10
−.0
5−.
10
−.1
0−.
05
−.1
0−.
02
.02
−.1
20
.10
0.0
6−.
14
−.0
4−.
02
−.1
2.1
5.1
5−.
15
−.0
9−.
06
−.0
9−.
10
−.0
7−.
09
−.0
4−.
02
−.1
20
.09
0.0
9−.
14
.02
.09
−.0
7.1
3.2
3−.
11
−.1
2.0
3−.
03
−.1
5.1
3−.
03
.05
.01
−.0
70
.77
2.0
8−.
11
−.0
2.0
4−.
12
.25
.13
−.1
5−.
09
−.0
4−.
09
−.0
9−.
04
−.0
9−.
03
.01
−.1
20
.05
0.1
8−.
14
−.0
4.0
2−.
12
.10
.16
−.1
5−.
12
−.0
5−.
09
−.1
0−.
08
−.0
9−.
05
.01
−.1
20
.02
0.1
1−.
14
.13
.00
−.1
2.4
2.2
5−.
14
.09
−.0
8−.
10
−.0
4−.
05
−.0
9.0
8.0
1−.
12
0.2
60
.15
−.1
3
−.0
3−.
01
−.1
1.0
8.1
2−.
14
−.0
9−.
07
−.0
9−.
10
−.0
5−.
09
−.0
4−.
01
−.1
10
.03
0.1
0−.
14
−.0
3−.
04
−.1
1.1
5.0
6−.
15
−.1
0−.
06
−.0
9−.
11
−.0
6−.
09
−.0
3−.
04
−.1
10
.04
0.0
5−.
14
.06
.08
−.1
1.1
5.4
5−.
13
−.0
9−.
06
−.0
7−.
09
−.0
4−.
08
.01
.08
−.1
00
.15
−0.0
1−.
12
−.0
7.0
3−.
12
.12
.14
−.1
5−.
10
−.0
6−.
09
−.1
0−.
07
−.0
9−.
08
.01
−.1
10
.07
0.1
0−.
14
−.0
5−.
01
−.1
2.1
2.0
8−.
14
−.1
0−.
05
−.0
9−.
09
−.0
8−.
09
−.0
4.0
0−.
12
0.0
50
.10
−.1
4
−.0
3.0
4−.
11
.10
.30
−.1
4−.
06
−.0
4−.
08
−.1
2.0
4−.
09
−.0
5.2
1−.
11
−1.0
70
.64
−.1
3
−.0
4.0
2−.
11
.14
.10
−.1
4−.
10
−.0
6−.
08
−.1
0−.
06
−.0
9−.
03
.00
−.1
10
.04
0.0
8−.
13
−.0
5−.
01
−.1
1.1
0.0
8−.
14
−.1
0−.
06
−.0
9−.
10
−.0
7−.
09
−.0
6.0
1−.
12
0.0
40
.07
−.1
4
.04
.04
−.1
2.3
1.2
4−.
13
−.0
4−.
13
−.0
8−.
04
−.0
9−.
08
.07
−.0
4−.
12
0.1
30
.07
−.1
3
−.0
4.0
1−.
11
.13
.10
−.1
5−.
10
−.0
7−.
09
−.1
0−.
07
−.0
9−.
03
−.0
1−.
11
0.0
90
.09
−.1
4
−.0
3−.
02
−.1
1.1
2.0
6−.
14
−.0
9−.
06
−.0
9−.
10
−.0
6−.
09
−.0
4−.
02
−.1
10
.03
0.0
6−.
14
133
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
Tab
le3
Sta
nd
ard
Dev
iati
on
of
Para
met
erE
stim
ate
sU
nd
erV
ari
ou
sD
ata
Con
dit
ion
san
dA
pp
roach
esan
dC
orr
esp
on
din
gS
am
ple
Siz
es(n
)
Ap
pro
ach
es
Dat
aS
etC
on
dit
ion
sL
aten
tS
core
sS
ing
leP
arce
lsE
ven
-Sp
lit
Par
cels
Item
-Co
nst
ruct
Bal
ance
Cat
.D
istr
ibu
tio
nN
g 1g 2
fn
g 1g 2
fn
g 1g 2
fn
g 1g 2
fn
Tw
o7
0,3
01
00
0.6
50
.69
.06
10
0.4
0.3
8.0
01
00
0.8
50
.82
.00
99
0.8
90
.92
.00
10
0
35
00
.20
0.2
6.0
31
00
.19
.20
.00
10
00
.35
0.3
9.0
01
00
0.3
30
.39
.00
10
0
60
00
.16
0.1
6.0
21
00
.15
.14
.00
10
00
.28
0.2
8.0
01
00
0.2
80
.27
.00
10
0
85
,1
51
00
1.0
20
.10
.07
10
0.4
3.5
4.0
01
00
1.1
91
.62
.00
10
01
.19
1.6
1.0
01
00
35
00
.31
0.3
2.0
31
00
.28
.26
.00
10
00
.65
0.6
1.0
01
00
0.5
60
.62
.00
10
0
60
00
.25
0.2
4.0
21
00
.21
.22
.00
10
00
.43
0.4
9.0
01
00
0.5
10
.50
.00
10
0
Th
ree
20
,6
0,2
01
00
0.4
90
.62
.05
10
0.2
3.2
6.0
11
00
0.4
60
.48
.01
10
00
.46
0.5
7.0
11
00
35
00
.18
0.1
9.0
21
00
.13
.12
.00
10
00
.22
0.2
1.0
11
00
0.2
20
.21
.01
10
0
60
00
.14
0.1
6.0
21
00
.09
.10
.00
10
00
.15
0.1
9.0
01
00
0.1
60
.18
.01
10
0
70
,2
0,1
01
00
0.7
81
.37
.06
10
0.2
6.3
0.0
11
00
0.6
40
.80
.01
98
0.5
80
.77
.01
10
0
35
00
.20
0.2
2.0
21
00
.15
.16
.00
10
00
.29
0.3
0.0
01
00
0.2
80
.31
.00
10
0
60
00
.16
0.1
5.0
21
00
.12
.11
.00
10
00
.20
0.2
0.0
01
00
0.2
20
.20
.00
10
0
Fiv
e5
,1
5,6
0,1
5,
51
00
0.3
30
.32
.04
10
0.1
4.1
2.0
31
00
0.2
10
.20
.03
10
00
.23
0.2
0.0
41
00
35
00
.15
0.1
5.0
21
00
.07
.07
.02
10
00
.11
0.1
0.0
21
00
0.1
00
.10
.02
10
0
60
00
.11
0.1
3.0
11
00
.06
.06
.01
10
00
.08
0.0
9.0
11
00
0.0
80
.09
.01
10
0
5,1
0,2
0,4
0,
25
10
00
.29
0.4
1.0
51
00
.12
.15
.03
10
00
.22
0.2
2.0
49
90
.19
0.2
5.0
41
00
35
00
.15
0.1
6.0
31
00
.07
.07
.02
10
00
.11
0.1
2.0
21
00
0.1
10
.12
.02
10
0
60
00
.11
0.1
3.0
21
00
.05
.06
.01
10
00
.08
0.1
0.0
21
00
0.0
80
.09
.02
10
0
Sev
en5
,1
0,2
0,3
0,
20
,1
0,5
10
00
.37
0.4
1.0
41
00
.11
.12
.05
10
00
.17
0.2
0.0
61
00
0.1
80
.18
.07
10
0
35
00
.15
0.1
7.0
31
00
.06
.06
.03
10
00
.09
0.1
0.0
31
00
0.0
80
.09
.04
10
0
60
00
.11
0.1
0.0
21
00
.04
.04
.02
10
00
.06
0.0
6.0
31
00
0.0
60
.06
.03
10
0
3,7
,1
2,1
8,
35
,2
0,5
10
00
.42
0.3
5.0
41
00
.10
.11
.05
10
00
.18
0.1
9.0
71
00
0.1
70
.18
.07
10
0
35
00
.16
0.1
8.0
21
00
.06
.06
.03
10
00
.09
0.1
0.0
41
00
0.0
80
.10
.04
10
0
60
00
.12
0.1
5.0
21
00
.04
.05
.02
10
00
.07
0.0
8.0
31
00
0.0
60
.08
.03
10
0
No
te:
Cat
.=n
um
ber
of
cate
go
ries
.
134
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
Tab
le3
(con
tin
ued
)
Ap
pro
ach
es
Ran
do
miz
edto
Fo
ur
Par
cels
Ran
do
miz
edto
Six
Par
cels
Fo
ur
Par
cels
of
Ad
jace
nt
Lo
adin
gs
Six
Par
cels
of
Ad
jace
nt
Lo
adin
gs
Fo
ur
Item
so
f
Div
erse
Lo
adin
gs
Fo
ur
Item
so
f
Hig
hL
oad
ing
s
g 1g 2
fn
g 1g 2
fn
g 1g 2
fn
g 1g 2
fn
g 1g 2
fn
g 1g 2
fn
1.3
91
.21
.00
10
00
.90
1.0
3.0
01
00
0.7
60
.74
.01
10
00
.08
0.8
1.0
11
00
1.0
22
.19
.11
82
.66
0.9
7.0
99
6
0.4
10
.40
.00
10
00
.46
0.3
9.0
01
00
0.3
10
.34
.00
10
00
.30
0.3
4.0
01
00
0.1
80
.60
.05
98
.18
0.1
8.0
51
00
0.3
20
.30
.00
10
00
.31
0.2
6.0
01
00
0.2
50
.23
.00
10
00
.25
0.2
2.0
01
00
0.1
90
.18
.05
10
0.1
40
.13
.04
10
0
1.2
92
.47
.00
10
01
.56
1.5
8.0
09
81
.57
1.6
8.0
09
91
.20
1.6
5.0
01
00
0.6
51
.87
.14
84
.65
1.3
2.1
48
3
0.6
40
.59
.00
10
00
.71
0.6
4.0
01
00
0.4
80
.50
.00
10
00
.49
0.5
0.0
01
00
0.4
90
.56
.06
98
.26
0.3
5.0
51
00
0.4
90
.51
.00
10
00
.56
0.4
9.0
01
00
0.3
50
.39
.00
10
00
.37
0.3
8.0
01
00
0.2
40
.24
.11
10
0.1
80
.17
.04
10
0
0.5
80
.65
.01
10
00
.59
0.5
1.0
11
00
0.4
10
.44
.02
10
00
.39
0.4
4.0
21
00
1.0
20
.76
.10
86
.43
0.9
6.0
89
8
0.2
40
.21
.01
10
00
.24
0.2
2.0
11
00
0.1
80
.19
.01
10
00
.18
0.1
9.0
11
00
0.2
00
.25
.04
10
0.1
70
.18
.03
10
0
0.1
80
.18
.00
10
00
.18
0.1
7.0
11
00
0.1
20
.15
.01
10
00
.13
0.1
5.0
11
00
0.1
20
.18
.03
10
0.1
30
.13
.03
10
0
0.7
40
.91
.01
10
00
.78
0.7
7.0
11
00
0.5
00
.70
.02
10
00
.52
0.7
3.0
21
00
3.0
53
.48
.13
81
.82
1.0
0.0
99
6
0.3
40
.31
.00
10
00
.33
0.3
3.0
01
00
0.2
20
.25
.01
10
00
.22
0.2
5.0
11
00
0.3
40
.33
.05
99
.17
0.2
5.0
41
00
0.2
50
.22
.00
10
00
.26
0.2
2.0
01
00
0.1
80
.17
.01
10
00
.18
0.1
7.0
11
00
0.1
50
.17
.04
10
0.1
10
.13
.32
10
0
0.2
40
.26
.04
10
00
.28
0.2
1.0
41
00
0.1
90
.18
.05
10
00
.19
0.1
7.0
51
00
0.6
41
.48
.07
96
.04
0.3
6.0
61
00
0.1
20
.11
.02
10
00
.13
0.1
0.0
21
00
0.0
90
.09
.03
10
00
.09
0.0
9.0
31
00
0.1
70
.20
.04
10
0.1
30
.13
.03
10
0
0.0
90
.09
.01
10
00
.09
0.0
9.0
11
00
0.0
70
.07
.02
10
00
.07
0.0
7.0
21
00
0.1
40
.13
.03
10
0.0
90
.11
.02
10
0
0.2
10
.27
.03
10
00
.27
0.2
8.0
41
00
0.1
60
.22
.05
10
00
.16
0.2
2.0
51
00
0.4
80
.86
.08
96
.29
0.4
0.0
79
9
0.1
20
.12
.02
10
00
.13
0.1
2.0
21
00
0.0
90
.10
.03
10
00
.09
0.1
0.0
31
00
0.1
80
.17
.04
10
0.1
50
.13
.03
10
0
0.0
90
.10
.01
10
00
.10
0.1
0.0
11
00
0.0
70
.08
.02
10
00
.06
0.0
8.0
21
00
0.1
20
.15
.03
10
0.1
00
.10
.03
10
0
0.1
90
.20
.06
10
00
.21
0.2
2.0
61
00
0.1
40
.18
.09
10
00
.15
0.1
9.0
91
00
0.5
30
.59
.07
97
.30
0.3
9.0
69
8
0.1
00
.10
.03
10
00
.10
0.1
0.0
31
00
0.0
80
.08
.05
10
00
.08
0.0
8.0
51
00
0.1
50
.21
.04
10
0.1
20
.12
.04
10
0
0.0
70
.06
.03
10
00
.07
0.0
6.0
31
00
0.0
50
.05
.04
10
00
.05
0.0
5.0
51
00
0.1
10
.13
.03
10
0.1
00
.09
.02
10
0
0.1
80
.20
.06
10
00
.06
0.1
3.1
51
00
0.1
40
.17
.09
10
00
.15
0.1
7.0
91
00
0.5
30
.76
.08
96
.37
0.3
0.0
69
9
0.1
00
.11
.03
10
00
.10
0.1
0.0
41
00
0.0
70
.08
.05
10
00
.07
0.0
8.0
51
00
0.1
70
.21
.04
10
0.1
50
.13
.03
10
0
0.0
80
.07
.02
10
00
.08
0.0
8.0
31
00
0.0
60
.07
.04
10
00
.06
0.0
6.0
41
00
0.1
40
.16
.03
10
0.1
00
.11
.02
10
0
135
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
Tab
le3
(con
tin
ued
)
Ap
pro
ach
es
Fo
ur
Item
so
f
Med
ium
Lo
adin
gs
Fo
ur
Item
so
f
Lo
wL
oad
ing
s
Six
Item
so
f
Div
erse
Lo
adin
gs
Six
Item
so
f
Hig
hL
oad
ing
s
Six
Item
so
f
Med
ium
Lo
adin
gs
Six
Item
so
f
Lo
wL
oad
ing
s
g 1g 2
fn
g 1g 2
fn
g 1g 2
fn
g 1g 2
fn
g 1g 2
fn
g 1g 2
fn
1.4
40
.93
.09
78
1.9
12
.46
.09
61
0.8
20
.89
.09
95
.47
0.6
5.0
81
00
1.3
82
.47
.09
88
1.7
92
.01
.08
78
0.2
60
.33
.05
99
1.2
50
.97
.04
94
0.1
70
.25
.05
10
0.1
30
.17
.04
10
00
.19
0.2
4.0
41
00
0.6
00
.60
.04
10
0
0.1
80
.20
.04
10
00
.34
0.3
5.0
39
90
.14
0.1
4.0
41
00
.12
0.1
1.0
31
00
0.1
60
.18
.03
10
00
.23
0.3
2.0
31
00
1.4
71
.41
.15
74
1.6
21
.3.1
25
31
.31
1.1
1.1
29
1.4
41
.19
.12
91
1.1
71
.56
.10
79
1.1
11
.42
.11
67
0.4
10
.66
.05
99
0.8
91
.56
.05
87
0.2
20
.28
.06
10
0.1
90
.20
.04
10
00
.26
0.4
5.0
51
00
0.7
91
.35
.05
95
0.2
40
.41
.04
10
00
.74
0.5
8.0
49
90
.21
0.1
9.0
41
00
.14
0.1
4.0
31
00
0.1
80
.22
.03
10
00
.44
0.3
4.0
31
00
0.9
61
.37
.09
92
1.9
1.2
.07
69
0.6
10
.92
.08
97
.31
0.3
9.0
79
80
.97
0.9
9.0
89
71
.48
1.8
9.0
78
4
0.2
20
.36
.03
10
00
.93
0.4
8.0
39
90
.16
0.1
6.0
41
00
.14
0.1
3.0
31
00
0.1
90
.27
.03
10
00
.39
0.3
4.0
31
00
0.1
30
.18
.03
10
00
.38
0.3
9.0
31
00
0.1
20
.14
.03
10
0.1
20
.12
.03
10
00
.12
0.1
5.0
21
00
0.2
50
.28
.02
10
0
0.9
91
.16
.07
86
2.5
61
.78
.09
76
0.6
60
.80
.10
97
.43
0.7
3.0
99
90
.91
0.6
9.0
89
54
.42
17
.5.0
88
6
0.2
90
.33
.04
10
00
.68
0.6
9.0
49
60
.15
0.2
0.0
41
00
.14
0.1
5.0
41
00
0.2
00
.21
.04
10
00
.71
0.9
2.0
49
8
0.1
60
.22
.03
10
00
.32
0.3
6.0
31
00
0.1
30
.13
.03
10
0.1
00
.10
.03
10
00
.14
0.1
8.0
31
00
0.2
10
.23
.03
10
0
0.8
41
.06
.06
96
1.1
71
.64
.06
86
0.6
00
.50
.06
10
0.2
70
.27
.06
10
00
.37
0.4
7.0
59
91
.41
.64
.05
95
0.2
10
.20
.03
10
00
.34
0.4
4.0
39
90
.16
0.1
3.0
31
00
.12
0.1
3.0
31
00
0.1
70
.17
.03
10
00
.23
0.3
5.0
31
00
0.1
20
.13
.02
10
00
.25
0.2
3.0
21
00
0.1
10
.10
.02
10
0.0
80
.10
.02
10
00
.12
0.1
2.0
21
00
0.1
70
.21
.02
10
0
0.5
40
.43
.07
10
01
.25
1.3
3.0
78
70
.46
0.6
1.0
79
9.2
30
.30
.06
10
00
.45
0.3
3.0
61
00
1.4
71
.66
.06
95
0.1
60
.19
.04
10
00
.39
0.6
3.0
31
00
0.1
40
.18
.03
10
0.1
20
.11
.03
10
00
.15
0.1
8.0
31
00
0.2
40
.39
.03
10
0
0.1
10
.16
.02
10
00
.31
0.3
8.0
21
00
0.1
00
.12
.02
10
0.1
00
.09
.02
10
00
.11
0.1
5.0
21
00
0.1
70
.27
.02
10
0
0.3
90
.57
.06
96
1.1
21
.48
.06
93
0.3
50
.38
.05
10
0.3
00
.43
.05
10
00
.58
1.1
9.0
61
00
12
.94
.92
.05
96
0.1
90
.23
.03
10
00
.38
0.4
0.0
31
00
0.1
50
.16
.03
10
0.1
10
.11
.04
10
00
.17
0.1
7.0
31
00
0.2
80
.37
.02
10
0
0.1
30
.15
.02
10
00
.25
0.2
4.0
21
00
0.1
00
.10
.03
10
0.0
90
.08
.02
10
00
.11
0.1
2.0
21
00
0.1
70
.18
.02
10
0
0.5
90
.70
.07
96
1.7
92
.19
.07
84
0.5
00
.74
.06
10
0.3
30
.26
.05
10
00
.46
0.5
2.0
51
00
0.6
11
.16
.05
90
0.1
90
.22
.03
10
00
.29
0.5
0.0
31
00
0.1
30
.16
.03
10
0.1
30
.13
.03
10
00
.18
0.1
8.0
31
00
0.2
60
.43
.03
10
0
0.1
40
.13
.02
10
00
.29
0.2
5.0
21
00
0.1
10
.13
.02
10
0.0
90
.10
.02
10
00
.12
0.1
2.0
21
00
0.1
90
.24
.02
10
0
136
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
the latent scoring approach performed well with large samples and when binary indicators
were not extremely skewed.
Shortened Scales
To compare and simplify the information concerning the effects of shortening scales, a
repeated measures analysis of variance was used to compare the biases of shortened-scale
approaches, including four ways of using four indicators and four ways of selecting six
indicators. This analysis yielded a significant within-subject effect (shortening strategies)
on g1, F(7, 147)= 13.98, p< .01; g2, F(7, 147)= 10.10, p< .01; and f, F(7, 147)=476.89, p< .01; and a significant between-subject effect of sample size on g1, F(1,
21)= 3.95, p< .05; g2, F(2, 21)= 8.99, p< .05; and f, F(1, 21)= 7.76, p< .01. These
effects suggest that various shortening strategies and sample sizes yielded different biases
in the three parameters. The estimated marginal means showed that selecting four or six
items with diverse or high factor loadings for SEM biased all three parameters within the
maximum acceptable range of 0.10. The raw means in Table 2 show that biases could
exceed 0.20 with small sample sizes. The estimated marginal means also show that select-
ing four or six items with medium factor loadings for the model biased the partial covar-
iances maximally by 0.08 but underestimated the covariance of F1 and F2 by more than
0.10 on average. Low factor loadings in the four or six items biased g1 minimally by 0.07
on average and the other two parameters (g1 and f) minimally by 0.13. In sum, the opti-
mal conditions for using four or six indicators selected from large scales were when indi-
vidual items had diverse or high factor loadings and sample sizes were relatively large.
Findings From Empirical Data
The selected approaches produced mixed results when applied to the empirical data. A
measurement model of aggression as a latent construct with 20 categorical indicators was
estimated and found to fit the data modestly from each wave. Standardized factor loadings
for each wave, their reliabilities, and model fit indices are listed in Table 4. Latent scores
were obtained from estimating each measurement model. The goodness of fit indices and
the factor loadings suggest that these items measured a unidimensional construct. Only
three kinds of parceling (single parcel, two odd–even split parcels, and four random par-
cels) were applied for the measurement of each wave, because the factor loadings of this
longitudinal data did not display the same pattern specified for the population model in
the simulation. A linear growth model was estimated using the latent scores or single par-
cels as the observed variables, two odd–even split parcels or four random parcels as indi-
cators of the aggression construct, or six individual items as indicators of the aggression
construct. The six individual items were selected to have the highest mean loadings over
time, and are marked with P in Table 4. All the models fit the data acceptably, with
CFI= .92 to .95, TLI= .94 to .98, and RMSEA= .06 to .07. The means of the intercept
(ai) and slope (as) and the covariance (fis) of the intercept and slope factors differed in
their magnitude and statistical significance across different approaches to using the scale.
The models of single parcels and odd–even split parcels as indicators of the latent
Yang et al. / Modeling Lengthy Measures 137
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
Tab
le4
Sta
nd
ard
ized
Fact
or
Load
ings
of
the
Late
nt
Con
stru
ctof
Aggre
ssio
nM
easu
red
Fro
mK
ind
ergart
ento
Gra
de
8
Item
and
Co
nte
nts
Kin
der
gar
ten
Gra
de
1G
rad
e2
Gra
de
3G
rad
e4
Gra
de
5G
rad
e6
Gra
de
7G
rad
e8
3.A
rgu
es.8
6.8
8.9
3.9
1.9
2.8
7.8
8.8
7.9
1
7.B
rag
s.7
5.7
3.7
0.7
5.7
0.8
0.7
5.7
3.8
1
16
.C
ruel
too
ther
sP
.88
.94
.92
.94
.93
.92
.89
.87
.94
19
.D
eman
ds
atte
nti
on
s.7
6.8
1.8
4.8
0.7
9.8
1.8
2.7
4.8
9
20
.D
estr
oy
so
wn
thin
gs
.80
.76
.78
.80
.79
.82
.76
.63
.87
21
.D
iffi
cult
yfo
llo
win
g
dir
ecti
on
s
.84
.84
.85
.90
.84
.86
.90
.84
.92
23
.D
iso
bed
ien
tat
sch
oo
lP
.87
.85
.91
.91
.90
.91
.88
.87
.90
27
.E
asil
yje
alo
us
.69
.82
.81
.69
.73
.71
.74
.81
.82
37
.F
igh
tsP
.90
.89
.91
.92
.89
.83
.85
.87
.88
43
.L
ies
.79
.75
.87
.79
.84
.74
.82
.72
.83
57
.P
hy
sica
lly
atta
cksP
.87
.81
.89
.91
.88
.91
.87
.92
.88
68
.S
crea
ms
.82
.69
.89
.90
.83
.82
.88
.81
.93
74
.S
ho
ws
off
.82
.82
.80
.77
.75
.79
.81
.84
.87
86
.S
tub
bo
rn.8
2.8
5.8
7.8
4.8
6.8
5.8
6.8
4.7
6
87
.M
oo
dy
.68
.78
.82
.79
.81
.81
.87
.82
.76
93
.T
alk
sto
om
uch
.78
.77
.80
.75
.82
.80
.81
.83
.88
94
.T
ease
s.7
8.8
8.8
2.8
3.8
5.8
1.8
5.8
6.8
6
95
.H
ot
tem
per
P.8
5.8
8.8
6.8
9.8
9.8
9.8
9.9
1.8
8
97
.T
hre
aten
sP
.90
.85
.89
.91
.91
.92
.88
.93
.94
10
4.
Lo
ud
.87
.86
.83
.84
.87
.78
.88
.89
.93
Mea
sure
men
tm
od
elfi
tw2
=2
58
.11
,
df=
50
,
p<
.01
,
CF
I=
.94
,
TL
I=
.98
,
RM
SE
A=
.08
w2=
24
5.3
1,
df=
48
,
p<
.01
,
CF
I=
.94
,
TL
I=
.98
,
RM
SE
A=
.08
w2=
19
1.0
4,
df=
50
,
p<
.01
,
CF
I=
.96
,
TL
I=
.99
,
RM
SE
A=
.07
w2=
18
8.4
0,
df=
53
,
p<
.01
,
CF
I=
.97
,
TL
I=
.99
,
RM
SE
A=
.07
w2=
18
0.8
1,
df=
42
,
p<
.01
,
CF
I=
.96
,
TL
I=
.98
,
RM
SE
A=
.08
w2=
20
4.3
5,
df=
52
,
p<
.01
,
CF
I=
.94
,
TL
I=
.98
,
RM
SE
A=
.08
w2=
17
9.0
2,
df=
43
,
p<
.01
,
CF
I=
.95
,
TL
I=
.98
,
RM
SE
A=
.08
w2=
13
1.7
9,
df=
.32
,
p<
.01
,
CF
I=
.96
,
TL
I=
.98
,
RM
SE
A=
.08
w2=
15
0.4
5,
df=
37
,
p<
.01
,
CF
I=
.97
,
TL
I=
.99
,
RM
SE
A=
.08
No
te:
CF
I=
com
par
ativ
efi
tin
dex
;T
LI=
Tu
cker
–L
ewis
ind
ex;
RM
SE
A=
roo
tm
ean
squ
are
erro
rap
pro
xim
atio
n.
138
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
construct found significant upward linear growth, as = .06, z= 3.13, p< .01, and as = .05,
z= 2.96, p< .01, whereas no growth was found by modeling latent scores, as = .05,
z= 1.33, p> .05, an aggression construct with four random parcels as indicators, as = :01,
z= 0.76, p> .05, or six individual items as indicators of the aggression construct,
as = .06, z= 0.72, p> .05.
Discussion
This study was designed to evaluate the effectiveness of three approaches to incorporat-
ing lengthy ordinal scales of varied measurement qualities into SEM, with a focus on
biases in estimates of partial variances and the covariance between exogenous constructs.
Three approaches were similarly effective under their own appropriate conditions in this
study. First, parceling approximately reflected the true population parameters when the
indicators had five response categories and parcels were further used as indicators of latent
constructs. Desirable parceling procedures included odd–even splitting into two parcels,
balancing item discrimination functions (factor loadings) to form three parcels, and ran-
dom parceling into four or six parcels. An ineffective procedure under the conditions of
this study was forming four or six parcels of items with contiguous factor loadings. Sec-
ond, latent scoring best reflected the true parameters in most data conditions of this study
except in the case of binary indicators with small samples or extremely skewed binary
indicators even with large samples. Third, scale length could be shortened to four or six
items with diverse or high factor loadings to adequately recover the population parameters
with medium to large samples. The best approach depended on the available data
conditions.
In contrast to previous findings that used continuous data, identical measurement of
both the exogenous and endogenous constructs, or a measurement model rather than a full
SEM, biases of parceling in the two partial covariance parameters were found to be
severely upward when the indicators had two or three response categories but slightly
downward when the indicators had seven categories. Besides differences in data, model,
and identical parceling of indicators of both exogenous and endogenous constructs, these
inconsistencies could also be attributed to the fact that parceling changed the original non-
linear functions between categorical indicators and latent constructs into linear functions,
resulting in transformation errors. These errors became large with binary indicators,
whose variances and means are dependent on each other and are typically bound into a
range from 0 to 1 (Ferrando, 2009). Parceling of trichotomous indicators could result in
distorted variance and covariance, because the range of the parcel was limited to the origi-
nal scale, as opposed to that of the latent construct it reflected, which is theoretically
infinite. Parceling under these conditions would lose information on variances and covar-
iances of the items and thus bias the estimates of partial covariances more severely.
Consistent with previous findings based on continuous data, optimal conditions for parcel-
ing in this study were found to be when parcels were created from items having five categories
and used as indicators of the latent constructs, regardless of parceling strategies and degree of
nonnormality (e.g., Sass & Smith, 2006). Parceling of five-category indicators could increase
the range of the parcel to approximately 4 (± 2) standard deviations of a normal distribution.
Yang et al. / Modeling Lengthy Measures 139
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
As a linear transformation of the categorical variables into continuous ones, parceling did not
lose much information on variances and covariances and thus yielded estimates that approxi-
mated the population partial covariance parameters (Ferrando, 2009).
Small sample size appeared to be disadvantageous to both latent scoring and shortened-
scale approaches to SEM. The latent scoring approach produced trivial biases in the esti-
mates of effects, except when the measurement model for obtaining the latent scores had
binary indicators and was estimated with small samples, or when the binary indicators
were extremely skewed even with large samples. As expected, a small sample typically
could not offer sufficient information for any model to describe its population with accu-
rate parameters. This problem could be complicated with binary indicators, whose means
and variances are dependent on each other. Simply increasing the sample size did not
improve the information that severely skewed binary indicators could provide about the
population, partly because certain information might have been lost when multivariate
continuous data were categorized into these severely skewed indicators. In spite of these
limitations, biases of latent scores in the covariance (f) and the two partial covariances
(g1 and g2) were not as far from the population parameters as those caused by using par-
cels. With large samples, shortened scales with either good or diverse measurement quali-
ties also became robust in recovering the population parameters.
Findings from the simulation study could shed some light on results of linear growth
modeling of the empirical data that adopted several approaches to the lengthy aggression
scale. Although discrimination functions of the aggression scale did not exactly match
those of the simulated data either in magnitude or pattern, no increase in aggression was
consistently detected by modeling the latent scores or factors with six individual indica-
tors of good measurement qualities. These two approaches had performed equivalently
well in recovering the population parameters with the simulated data under favorable sam-
ple sizes. In addition, the linear growth model showed no increase of factors with four
random parcels as indicators. Four random parcels as indicators of the aggression con-
struct should have provided more reliable measurement than a single parcel or two parcels
as indicators of the construct (Kenny, 1979). These consistent findings suggested that
aggression might not have increased over time.
In contrast, a significant increase in aggression over time was detected with linear
growth modeling of single parcels or a latent construct with two odd–even split parcels as
indicators. This finding corresponded to the overestimation of the partial covariances
when parcels of trichotomous indicators were used as indicators in the simulation study.
Therefore, using findings from the simulation study as guidelines, the efficient approaches
to the empirical data problem were (a) latent growth modeling of factors reflected by a
shortened scale with the best six indicators and (b) modeling of latent scores of the full
scale, which led to the inference that aggression did not increase over time in this sample.
Conclusions
If all items in a lengthy ordinal scale or testlets with various item discrimination func-
tions are to be included for SEM, parceling is one desirable option given the following
conditions: Items have five response categories and are parceled by odd–even splitting of
140 Applied Psychological Measurement
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
the scale; item discrimination functions are balanced or randomized; and two to six par-
cels obtained through parceling are used as indicators of a construct. Another option is to
obtain latent scores through preliminary item response modeling or CFA with categorical
indicators based on medium or large samples. However, binary items should not be extre-
mely skewed. If some items of a lengthy scale are not desirable, four or six items having
good, diverse measurement properties may be selected to provide robust indicators of a
construct, preferably based on a medium or large sample.
References
Achenbach, T. M. (1978). Child-Behavior Profile: I. Boys aged 6-11. Journal of Consulting and Clinical Psy-
chology, 46, 478-488.
Alhija, F. N., & Wisenbaker, J. (2006). A Monte Carlo study investigating the impact of item parceling strate-
gies on parameter estimates and their standard errors in CFA. Structural Equation Modeling, 13, 204-228.
Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recom-
mended two-step approach. Psychological Bulletin, 103(3), 411-423.
Bandalos, D. L. (2002). The effects of item parceling on goodness-of-fit and parameter estimate bias in struc-
tural equation modeling. Structural Equation Modeling, 9, 78-102.
Bandalos, D. L., & Finney, S. J. (2001). Item parceling issues in structural equation modeling. In G. A. Mar-
coulides & R. E. Schumacker (Eds.), New developments and techniques in structural equation modeling
(pp. 269-275). Mahwah, NJ: Lawrence Erlbaum.
Bollen, K., & Lennox, R. (1991). Conventional wisdom on measurement: A structural equation perspective.
Psychological Bulletin, 110, 305-314.
Cattell, R. J. (1956). Validation and intensification of the sixteen personality factor questionnaire. Journal of
Clinical Psychology, 12, 205-214.
Cattell, R. J., & Burdsal, C. A., Jr. (1975). The radial parceling double factoring design: A solution to the
item-vs.-parcel controversy. Multivariate Behavioral Research, 10, 165-179.
Coanders, G., Satorra, A., & Saris, W. E. (1997). Alternative approaches to structural equation modeling of
ordinal data: A Monte Carlo study. Structural Equation Modeling, 4, 261-282.
Embretson, S. E. (1996). Item response theory and inferential bias in multiple group comparisons. Applied
Psychological Measurement, 20, 201-212.
Fletcher, T. D. (2005). The effects of parcels and latent variable scores on the detection of interactions in
structural equation modeling. Dissertation Abstracts International: 66-5B, 2872.
Ferrando, P. (2009). Difficulty, discrimination, and information indices in the linear factor analysis model for
continuous item responses. Applied Psychological Measurement, 33(1), 9-24.
Guttman, L. (1955). The determinacy of factor score matrices with implications for five other basic problems
of common-factor theory. British Journal of Statistical Psychology, 8, 65-81.
Hall, R. J., Snell, A. F., & Foust, M. S. (1999). Item parceling strategies in SEM: Investigating the subtle
effects of unmodeled secondary constructs. Organizational Research Methods, 2, 757-765.
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Thousand
Oaks, CA: Sage.
Hau, K. T., & Marsh, H. W. (2004). The use of item parcels in structural equation modeling: Non-normal data
and small sample sizes. British Journal of Mathematical Statistical Psychology, 57, 327-351.
Kenny, D. A. (1979). Correlation and causality. New York: John Wiley.
Kishton, J. M., & Widaman, K. F. (1994). Unidimensional versus domain representative parceling of question-
naire items: An empirical example. Educational and Psychological Measurement, 54, 757-765.
Lansford, J. E., Malone, P. S., Dodge, K. A., Crozier, J. C., Pettit, G. S., & Bates, J. E. (2006). A 12-year pro-
spective study of patterns of social information processing problems and externalizing behaviors. Journal
of Abnormal Child Psychology, 34, 715-724.
Yang et al. / Modeling Lengthy Measures 141
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from
Little, T. D., Cunningham, W. A., Shahar, G., & Widaman, K. F. (2002). To parcel or not to parcel: Exploring
the questions, weighing the merits. Structural Equation Modeling, 9, 151-173.
Little, T. D., Lindenberger, U., & Nesselroade, J. R. (1999). On selecting indicators for multivariate measure-
ment and modeling with latent variables: When ‘‘good’’ indicators are bad and ‘‘bad’’ indicators are good.
Psychological Methods, 4, 192-211.
Marsh, H. W., Hau, K., Balla, J. R., & Grayson, D. (1998). Is more ever too much? The number of indicators
per factor in confirmatory factory analysis. Multivariate Behavioral Research, 33, 181-220.
Marsh, H. W., & O’Neill, R. (1984). Self Description Questionnaire III: The construct validity of multidimen-
sional self-concept ratings by late adolescents. Journal of Educational Measurement, 21, 153-174.
McDonald, R. P. (1996). Latent traits and the possibility of motion. Multivariate Behavioral Research, 31,
593-601.
Moore, K. A., Halle, T. G., Vandivere, S., & Marina, C. L. (2002). Scaling back survey scales: How short is
short? Sociological Research Methods, 30, 530-567.
Muthen, B. (1984). A general structural equation model with dichotomous, ordered categorical and continuous
latent variable indicators. Psychometrika, 49, 115-132.
Muthen, B., & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non-normal
Likert variables. British Journal of Mathematical and Statistical Psychology, 38, 171-189.
Muthen, B., & Kaplan, D. (1992). A comparison of some methodologies for the factor analysis of non-normal
Likert variables: A note on the size of the model. British Journal of Mathematical and Statistical Psychol-
ogy, 45, 19-30.
Muthen, B. O., & Muthen, L. K. (2002). How to use a Monte Carlo study to decide on sample size and deter-
mine power. Structural Equation Modeling, 9, 599-620.
Muthen, L. K., & Muthen, B. O. (1998-2006). Mplus user’s guide (4th ed.). Los Angeles: Muthen & Muthen.
Muthen, L. K., & Muthen, B. O. (2006). IRT in Mplus. Retrieved December 06, 2008, from http://www.stat-
model.com/download/MplusIRT1.pdf
Nasser, F., & Wisenbaker, J. (2003). A Monte Carlo study investigating the impact of item parceling on mea-
sures of fit in confirmatory factor analysis. Educational and Psychological Measurement, 63, 729-757.
Paxton, P., Curran, P. J., Bollen, K. A., Kirby, J., & Chen, F. (2001). Monte Carlo experiments: Design and
implementation. Structural Equation Modeling, 8, 287-312.
Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory:
Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552-566.
Reise, S. P., & Yu, J. (1990). Parameter recovery in the graded response model using MULTILOG. Journal of
Educational Measurement, 65, 143-151.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika
Monograph, No. 17.
Sass, D. A., & Smith, P. L. (2006). The effects of parceling unidimensional scales on structural parameter esti-
mates in structural equation modeling. Structural Equation Modeling, 13, 566-586.
Shevlin, M., Miles, J. N. V., & Bunting, B. P. (1997). Summated rating scales: A Monte Carlo investigation of the
effects of reliability and collinearity in regression models. Personality and Individual Differences, 23, 665-676.
Stephenson, M. T., Hoyle, R. H., Palmgreen, P., & Slater, M. D. (2003). Brief measures of sensation seeking
for screening and large-scale surveys. Drug and Alcohol Dependence, 72, 279-286.
Takane, Y., & de Leeuw, J. (1987). On the relationship between item response theory and factor analysis of
discretized variables. Psychometrika, 52, 393-408.
Wright, B. D. (1999). Fundamental measurement for psychology. In S. E. Embretson & S. L. Hershberger
(Eds.), The new rules of measurement: What every psychologist and educator should know (pp. 65-104).
Mahwah, NJ: Lawrence Erlbaum.
For reprints and permission queries, please visit SAGE’s Web site at http://www.sagepub.
com/journalsPermissions.nav
142 Applied Psychological Measurement
at DUKE UNIV on March 1, 2010 http://apm.sagepub.comDownloaded from