The use of item parcels in structural equation.pdf

8/16/2019 The use of item parcels in structural equation.pdf

1/25


2/25

‘indicator’ interchangeably to represent a variable that varies along an underlying

continuous distribution). Furthermore, for researchers who are concerned with largenumbers of measured variables or estimated parameters, particularly when N is small,

item parcelling will substantially improve the ratio of N to these characteristics (Hall,Snell, & Foust, 1999; but see also Marsh et al., 1998). Despite these noted potential

advantages with item parcels, there are only a few empirical studies that evaluate thesesuggestions.

The use of item parcels instead of items is common. In their review of articles inmajor education, psychology and marketing journals, Bandalos and Finney (2001) found

that 20% of the applied structural equation model studies used some form of parcellingprocedure. The proposed advantages of parcels included: increased reliability of item-

parcel responses, more definitive rotational results, less violation of normality assumptions, closer approximations to normal theory-based estimation, fewer parameters to be estimated (optimizing the ratio of N to the number of variables, asdiscussed above), more stable parameter estimates, reduction in idiosyncratic

characteristics of items, and simplification of model interpretation. Bandalos andFinney further emphasized that even though Marsh (e.g. Marsh & O’Niell, 1984) has

been cited most often (70% of studies using parcelling in their review) in support of thepractice of parcelling, researchers have typically failed to heed his advice that items

being parcelled should be reasonably unidimensional. Of particular relevance to thepresent investigation, we examine potential advantages of parcelling items that areknown to be unidimensional (on the basis of simulated data based on a knownpopulation-generation model).

2. Effects of sample size, number of indicators per factor, and itemparcels

In a classic Monte Carlo study, Boomsma (1982) evaluated the robustness of CFA solutionsfor small N (25–400). He found that the percentage of proper solutions, the accuracy of

parameter estimates, the sampling variability in parameter estimates, and theappropriateness of the ML x2 test statistic were all favourably influenced by larger values

of N (see also Boomsma & Hoogland, 2001; Gerbing & Anderson, 1993). Herecommended that N should be at least 100, but that 200 or more was desirable. Marsh

et al. (1998) and others (e.g. MacCallum, Widaman, Zhang, & Hong, 1999; Velicer & Fava,

1998) argued that concerns about the minimum N for factor analysis have produced many

guidelines but limited empirical research. They noted that whereas it might be good tohave N as large as possible, there was no systematic support for rules positing minimum

N as a function of the number of items, factors, or estimated parameters. Velicer and Fava(see also Marsh et al., 1998) also found that convergence to proper solutions and accuracy

of estimates were favourably influenced by increasing N , the number of measured variables per factor ( p / f ratios), and saturation (factor loadings).

Several studies evaluating the effects of p / f ratios (e.g. Anderson & Gerbing, 1984;

Boomsma, 1982, 1985; Ding, Velicer, & Harlow, 1995; Gerbing & Anderson, 1987;MacCallum et al., 1999; Marsh et al., 1998; Velicer & Fava, 1998) all reported that thelikelihood of fully proper solutions increased with increasing p / f , N , and saturation.

Gerbing and Anderson (1987) and Marsh et al.

also demonstrated that the standarderrors (SEs) of parameter estimates were smaller when N and p / f were higher. Many rules of thumb and ‘conventional wisdom’ imply that researchers should limit the

Kit-Tai Hau and Herbert W. Marsh328


3/25

number of items to be considered when N is small (e.g. rules about the minimum value

of N / p or of N divided by the number of parameter estimates) and they also seem tosupport the use of item parcels, particularly when N is small. Hence, the use of item

parcels is a potential compromise between having a large number of items per factor and having a small number of indicators (parcels) in the actual analysis. However,

Marsh et al. found that there were no particular advantages or disadvantages in the useof parcels in terms of convergence to proper solutions, parameter estimates (factor

correlations, loadings), and SEs of parameter estimates. Thus, for example, solutionsbased on 12 items per factor performed systematically better than solutions based on

four items per factor, but solutions based on 12 items per factor performed similarly tosolutions based on four parcels (of three items each) that were built from the same

12 items. Hence, the critical determining factor for better-behaved solutions was thenumber of items rather than whether these items were represented as items or itemparcels in the actual analyses. Nevertheless, Marsh et al. did find that solutions based ononly two item-parcels per factor (i.e. two parcels of six items each) performed poorly.

However, Marsh et al. also emphasized that there may be advantages in using parcels,noting that responses to item parcels tended to violate assumptions of normality to a

lesser degree than corresponding responses to items that were not normally distributed.

3. Non-normality and CFA

ADF estimation procedures do not impose a multivariate normality assumption(see Browne, 1984; Hu, Bentler, & Kano, 1992). However, because the ADF weight

matrix is so large ð pð p þ 1Þ=2 £ pð p þ 1Þ=2Þ; where p is the number of indicators in themodel), ADF procedures require extremely large N and small models in order to

generate reasonably precise estimates—for example, West et al. (1995) recommendedthat N should be at least 1000–5000; see also Hu et al. (1992). The method wouldbecome problematic—with a lot of non-convergence improper solutions, and excessiverejection of the null hypothesis—when used for large models with small or moderate

size samples. If, however, researchers used item parcels instead of items, the size of this weight matrix would be substantially reduced. Hence, one purpose of this study is to

examine whether ADF estimation can be used effectively with item parcels even whenthey cannot be used with items.

West et al. (1995) reviewed empirical studies on the performance of various

estimators on CFA solutions. Their major conclusions relevant to the present studies are

(i) the ML, generalized least squares (GLS) and ADF T statistics ( T ML , T GLS and T ADF ) aretoo large for non-normal item responses (see also Muthén & Kaplan, 1985; Sharma,

Durvasula, & Dillon, 1989); (ii) T GLS, T ADF and T ML are too large when N is small, even when the data are normal; and (iii) non-normality leads to underestimation of SEs of

parameter estimates (see also Sharma et al., 1989).In order to improve the performance of ADF procedures for sample sizes typically

encountered in practice, Yuan and Bentler (1997) proposed a finite-sample correction

( T YB ) to T ADF which employed nonlinear regression methodology and residual-basedGLS techniques. Although this correction can be applied to data of any knowndistributional form for which there are appropriate GLS weight matrices, in our study it

is used on the ADF statistic.Other modifications to the T statistics obtained with non-normal data have beenproposed so that the test statistic corresponds more closely to large-sample normal

The use of item parcels in structural equation modelling 329


4/25

theory T distribution (e.g. the Satorra– Bentler T ML statistic, T SB ) and related adjustments

are made to produce robust SEs of parameter estimates (see Hu et al., 1992; Satorra &

Bentler, 1994). In particular, Satorra and Bentler (1994) developed corrected normal-

theory test statistics (among them T SB ) that they argued were better than the previous

robust test statistics (Anderson, 1996; Chou, Bentler, & Satorra, 1991; Hu et al., 1992;

Yuan & Bentler, 1997). This was achieved by multiplying the ML test statistics by acorrection factor based on the data and model. The Satorra – Bentler corrections can also

be extended to augmented moment structures, multisamples and categorical data

(Muthén, 1993; Satorra, 1992).

In general, ADF procedures tend to produce biased parameter estimates and T ADF is

too large when N is small to moderate. Due to this biased estimation and the sample size

requirements, ADF estimation and its modifications (e.g. T YB ) have not been very

popular among users despite their concern for the non-normality of their data. As ML

estimation is apparently robust to many violations of underlying assumptions (e.g. Hu

et al., 1992; Muthén & Kaplan, 1985), and particularly with the incorporation of the

Satorra– Bentler corrections ML remains the most popular estimation procedure. Yuanand Bentler (1997, 1998a, 1998b; Bentler & Yuan, 1999) examined various approaches

and test statistics for non-normal data and small sample sizes. For medium to large

sample sizes, they recommended T ML for normal data and T YB (or an F -test based on

Hotelling’s T 2 statistics) for non-normal data, but the latter may still over-reject true

models at very small sample sizes. In the present investigation, we compared the

effectiveness of ML and ADF estimation procedures for varying degrees of non-normality

in combination with the use of item parcels. As our main interest was in the

effectiveness of parcelling (versus items) rather than in the comparison of the test

statistics, we limited consideration to the most common statistics ( T ML and T ADF ) as well

as their respective adjustments ( T SB and T YB ).

4. Present investigation

The aim of this paper is to evaluate empirically the parcelling strategy for dealing with non-normality, particularly with small N . In our two simulation studies, we evaluated

the convergence behaviour, goodness of fit and parameter estimates of the solutionsby systematically varying: (a) the non-normality of the item distribution (in terms of

size and direction of kurtosis and skew); (b) N (from 50 to 1000); (c) the way of

indicator formation (8 items per factor are used to construct 4 and 2 parcels per factor); (d) the parcelling strategy (parcels from items with uniformly positive skewsand kurtoses versus counterbalancing skews and kurtoses); and (e) the estimation

procedure (ML or ADF).

5. STUDY 1

Here, we studied the possible benefits of item parcelling by comparing item and item-parcel solutions with varying degrees of non-normality. In one specific model, we also

deliberately counterbalanced both skew and kurtosis. Specifically, item parcels wereformed such that positively skewed items were parcelled with negatively skewed itemsand positive-kurtosis items were parcelled with negative-kurtosis items.



5/25

5.1. Methods

Study 1 was a Monte Carlo simulation study. Two-factor models were constructed in which each indicator loaded on one and only one factor (see Marsh et al., 1998). The

initial model consisted of eight items per factor, all with unit variance and .70 factor loadings (uniqueness .51). The study was based on a 3 (parcelling strategies) £ 4 (levels

of N —number of cases per replicate) £

5 (types of non-normality) design. Item parcels were formed by averaging 2 or 4 items to form 4 or 2 parcels per factor, respectively. The

factors had unit variance and a correlation of .30 with each other. The four levels of N were 50, 100, 250, and 1000, with the simulated cases divided into 2500 replicates of

N ¼ 50 cases, 1000 replicates of N ¼ 100; 400 replicates of N ¼ 250; and 100 replicatesof N ¼ 1000: We specifically generated more replicates for smaller values of N becausethe behaviour of the solutions for small N was a primary focus of this investigation andthese solutions tended to be less stable. As our interest is in solutions based on small N ,

we have used N as small as 50. However, because N ¼ 50 is lower than the generalacceptable standard for structural equation model analyses and for the asymptotic ML

and ADF methods, appropriate caution is needed in the interpretation of the followinganalyses with these extremely small N conditions.

We systematically manipulated the degree of non-normality in terms of item skew and kurtosis: normal (skew ¼ 0; kurtosis ¼ 0); slightly non-normal (skew ¼ 0:5;kurtosis ¼ 0:5); moderately non-normal (skew ¼ 1:0; kurtosis ¼ 1:5); and very non-normal (skew ¼ 1:5; kurtosis ¼ 3:25). In addition, we considered a ‘balanced, conditionin which skews were þ0.5 or 20.5 and kurtoses were þ0.5 or 20.5. Here we created

counterbalanced item parcels that contained items with positive and negative skewsand with positive and negative kurtoses.

The data were fitted with a two-factor model using EQS 5.3 (Bentler & Wu, 1995).

The factor variance was fixed at unity, and factor correlations, factor loadings, and

uniquenesses were all freely estimated; the maximum number of iterations was set to300. We examined several strategies for the analysis of non-normal data. In addition to

ML estimation, we used the ADF fit functions even though this typically required large N (and could not be estimated for item-level data and the smallest values of N ). EQS also

offers T SB, T YB, and robust SEs for parameter estimates. However, the main purpose of the study was to evaluate the proposal by West et al. (1995) and Marsh et al. (1998) that

the use of item parcels would reduce the degree of non-normality and subsequently improve the behaviour of the solutions.

All models tested in this study were ‘true’ in that the pattern of fixed and freeparameters was the same in the population-generating model used to generate the

simulated data and the approximating model used to fit the data. Hence, to the extentthat the T statistic is behaving appropriately, the mean T /df ratio should not differ

systematically from 1.0. (As the df associated with different models varied substantially,the T statistic should differ substantially from model to model and so it was less useful for

our present purposes). Similarly, because the variance of a central chi-square is 2 df, wedivided the variance of T estimates by df so that these variances across different cells

were in a more comparable metric. As the purpose of the present investigation was notto evaluate different fit indices per se, we do not present results for a variety of goodness-

of-fit indices (e.g. Marsh et al., 1988), but we briefly summarize these findings in relationto the T /df results.

Analyses were conducted in the covariance metric. This was particularly importantbecause the expected variance of each parcel score, depending on the number of items in the parcel, was substantially less than the variability of the item scores



6/25

(1.0 in the population) used to construct the item parcel, by virtue of the central limit

theorem. This feature of the parcels also complicated comparisons between item and

parcel solutions.In the present Monte Carlo simulation study, all non-normal data were generated

with PRELIS and LISREL 8 (Jöreskog & Sörbom, 1993) using a methodology described by

Fleishman (1978) and Vale and Maurelli (1983). Specifically, the multivariate non-normaldata were generated using a combination of the matrix decomposition procedure (Vale

& Maurelli, 1983) and Fleishman’s (1978) method. In the population, the correlation

between any pair of items from the same factor was .490, and .147 ð:7 £ :7 £ :3Þ whenthey were from different factors. These correlations were identical for the non-normal

items. The respective correlations were .658 and .198 for two-item parcels (item pairs)

and were .792 and .238 for four-item parcels (item quadruples).The creation of item parcels, of course, altered the summary statistics of the

measured variables. Thus, for example, in the very non-normal condition, the

descriptive statistics changed from standard deviation ðSDÞ ¼ 1; skew ¼ 1:5; and

kurtosis ¼ 3:25 for items, to SD ¼ 0:86; skew ¼ 1:3; and kurtosis ¼ 2:4 for item-pair parcels, and SD ¼ 0:78; skew ¼ 1:2; and kurtosis ¼ 2:0 for item-quadruple parcels. Inthe balanced condition where individual items had skews and kurtoses of þ0.5 or 20.5,

the item-pair parcels had skews close to zero and kurtoses of 20.22, whereas the item-

quadruple parcels had skews of 0 and kurtoses of 20.25. In this respect, the item

parcelling strategy was successful in substantially reducing the violation of normality

assumption of the measured variables.

5.2. Results

5.2.1. ConvergenceNon-convergence (Table 1) was frequent with two-indicator (item-quadruple) parcels,

particularly when N was small ð N ¼ 50– 250Þ; and slightly more frequent for ADFsolutions at N ¼ 50: In contrast, there was close to 100% convergence to fully proper solutions for all ML solutions based on items and item pairs. Furthermore, non-normality had little effect on convergence for the ML solutions. The poor convergence behaviour of the two-indicator solutions based on small N was consistent with previous research and, for this reason, we did not give results based on these solutions much attention in

the subsequent discussion of results. All subsequent analyses of parameters and modelfit in Studies 1 and 2 were also based on proper solutions.

5.2.2. Factor correlation estimates We focus particularly on the evaluation of factor correlation estimates because the

metric of these is not altered by the changing from solutions based on individual items,item pairs, and item quadruples. The estimated factor correlations (Table 1) werereasonably accurate for all ML solutions based on eight items or four item pairs (i.e. the

mean of correlation estimates is close to the population value of .3). The extent of non-normality and value of N had little or no systematic effect on these factor correlationestimates. In contrast, for all ADF solutions based on N , 1000; the factor correlation

estimates tended to be somewhat positively biased and the extent of this bias variedinversely with N . Although the poor convergence behaviour of the two-indicator

solutions made interpretation of these solutions problematic (since fewer replicates



7/25

T a b l e

1 .

S t u d y 1 : P e r c e n t a g e c o n v e r g e n c

e t o p r o p e r s o l u t i o n a n d f a c t o r c o r r e l a t i o n e s t i m a t e s ( m e a n ,

e m p i r

i c a l S E a n d p r e d i c t e d S E ) b y s a m p l e s i z e ,

n u m b e r o f

i n d i c a t

o r s ,

d e g r e e o f n o n - n o r m a l i t y a n d

fi t t i n g f u n c t i o n

N ¼

5 0

N ¼

1 0 0

N ¼

2 5 0

N ¼

1 0 0 0

M L

A D F

M L

A D F

M L

A D F

M L

A D F

D e g r e e o

f

N o n -

n o r m a l i t y

N o .

o f

i n d i c .

%

M

S E e

S E p

S E r

%

M

S E e

S e p

%

M

S E e

S E p

S E r

%

M

S E e

S E p

%

M

S E e

S E p

S E r

%

M

S E e

S E p

%

M

S E e

S E p

S E r

%

M

S E e

S E p

N o r m a l

8

1 0 0

. 2 9

. 1 5

. 1 5

. 1 4

–

–

–

–

1 0 0

. 3 0

. 1 1

. 1 0

. 1 0

–

–

–

–

1 0 0

. 3 0

. 0 7

. 0 7

. 0 7

1 0 0

. 3 1

. 1 0

. 0 5

1 0 0

. 3 0

. 0 4

. 0 3

. 0 3 1 0 0

. 3 0

. 0 4

. 0 3

( s ¼

0 ;

k ¼

0 )

4

1 0 0

. 2 9

. 1 5

. 1 5

. 1 4

9 2

. 3 2

. 2 6

. 1 0

1 0 0

. 3 0

. 1 1

. 1 0

. 1 0

1 0 0

. 3 1

. 1 4

. 0 9

1 0 0

. 3 0

. 0 7

. 0 7

. 0 7

1 0 0

. 3 0

. 0 7

. 0 6

1 0 0

. 3 0

. 0 4

. 0 3

. 0 3 1 0 0

. 3 0

. 0 4

. 0 3

2

2 1

. 4 1

. 1 2

. 1 4

. 1 3

2 1

. 4 0

. 1 1

. 1 3

3 8

. 3 5

. 0 9

. 1 0

. 1 0

3 6

. 3 6

. 0 9

. 1 0

6 9

. 3 1

. 0 7

. 0 7

. 0 6

6 9

. 3 1

. 0 7

. 0 6

9 9

. 3 0

. 0 4

. 0 3

. 0 3

9 9

. 3 0

. 0 4

. 0 3

S l i g h t l y

8

1 0 0

. 2 9

. 1 5

. 1 4

. 1 4

–

–

–

–

1 0 0

. 2 9

. 1 1

. 1 0

. 1 0

–

–

–

–

1 0 0

. 3 0

. 0 7

. 0 7

. 0 7

1 0 0

. 3 1

. 1 0

. 0 5

1 0 0

. 3 0

. 0 4

. 0 3

. 0 3 1 0 0

. 3 0

. 0 4

. 0 3

n o n -

n o r m a l

4

1 0 0

. 2 9

. 1 5

. 1 4

. 1 4

9 0

. 3 1

. 2 6

. 1 0

1 0 0

. 3 0

. 1 1

. 1 0

. 1 0

1 0 0

. 3 1

. 1 4

. 0 9

1 0 0

. 3 0

. 0 7

. 0 7

. 0 7

1 0 0

. 3 0

. 0 7

. 0 6

1 0 0

. 3 0

. 0 4

. 0 3

. 0 3 1 0 0

. 3 0

. 0 4

. 0 3

( s ¼

:

5 ;

k ¼

:

5 )

2

2 1

. 4 1

. 1 2

. 1 4

. 1 3

2 0

. 4 1

. 1 2

. 1 3

3 8

. 3 5

. 0 9

. 1 0

. 1 0

3 6

. 3 6

. 0 9

. 1 0

6 9

. 3 1

. 0 6

. 0 7

. 0 7

7 0

. 3 1

. 0 7

. 0 7

9 9

. 3 0

. 0 4

. 0 3

. 0 3

9 9

. 3 0

. 0 4

. 0 3

M o d e r a t e l y

8

1 0 0

. 2 9

. 1 5

. 1 5

. 1 4

–

–

–

–

1 0 0

. 2 9

. 1 1

. 1 0

. 1 0

–

–

–

–

1 0 0

. 3 0

. 0 7

. 0 7

. 0 7

1 0 0

. 3 4

. 1 0

. 0 5

1 0 0

. 3 0

. 0 4

. 0 3

. 0 3 1 0 0

. 2 9

. 0 4

. 0 3

n o n -

n o r m a l

4

1 0 0

. 2 9

. 1 5

. 1 4

. 1 4

8 9

. 3 2

. 2 6

. 1 0

1 0 0

. 2 9

. 1 1

. 1 0

. 1 0

1 0 0

. 3 0

. 1 4

. 0 9

1 0 0

. 3 0

. 0 7

. 0 7

. 0 7

1 0 0

. 2 9

. 0 8

. 0 6

1 0 0

. 3 0

. 0 4

. 0 3

. 0 3 1 0 0

. 3 0

. 0 4

. 0 3

( s ¼

1 : 0

;

k ¼

1 :

5 )

2

1 9

. 4 2

. 1 1

. 1 4

. 1 3

1 9

. 4 1

. 1 2

. 1 3

3 6

. 3 5

. 0 9

. 1 0

. 1 0

3 3

. 3 6

. 0 9

. 1 0

6 7

. 3 2

. 0 7

. 0 7

. 0 7

6 4

. 3 2

. 0 7

. 0 7

9 9

. 3 0

. 0 4

. 0 3

. 0 3

9 8

. 3 0

. 0 4

. 0 3

V e r y

8

1 0 0

. 2 9

. 1 6

. 1 4

. 1 4

–

–

–

–

1 0 0

. 2 9

. 1 1

. 1 0

. 1 0

–

–

–

–

1 0 0

. 2 9

. 0 7

. 0 7

. 0 7

1 0 0

. 3 6

. 1 1

. 0 5

1 0 0

. 2 9

. 0 4

. 0 3

. 0 3 1 0 0

. 2 8

. 0 5

. 0 3

n o n -

n o r m a l

4

1 0 0

. 2 9

. 1 6

. 1 4

. 1 4

8 4

. 3 4

. 2 6

. 1 0

1 0 0

. 2 9

. 1 1

. 1 0

. 1 0

9 9

. 3 0

. 1 5

. 0 9

1 0 0

. 2 9

. 0 7

. 0 7

. 0 7

1 0 0

. 2 8

. 0 8

. 0 7

1 0 0

. 2 9

. 0 4

. 0 3

. 0 3 1 0 0

. 2 9

. 0 4

. 0 3

( s ¼

1 : 5

;

k ¼

3 :

2 5

)

2

1 9

. 4 1

. 1 3

. 1 4

. 1 3

1 7

. 4 2

. 1 2

. 1 3

3 4

. 3 5

. 1 0

. 1 0

. 1 0

3 1

. 3 5

. 1 0

. 1 0

6 1

. 3 2

. 0 7

. 0 7

. 0 7

6 0

. 3 2

. 0 7

. 0 7

9 7

. 3 0

. 0 4

. 0 3

. 0 3

9 7

. 2 9

. 0 4

. 0 4

S l i g h t l y

n o n n o r m

a l

b a l a n c e d

( s ¼

^ 0

:

5 ;

k ¼

^ 0

:

5 )

8

1 0 0

. 3 0

. 1 5

. 1 5

. 1 4

–

–

–

–

1 0 0

. 3 0

. 1 1

. 1 0

. 1 0

–

–

–

–

1 0 0

. 3 0

. 0 7

. 0 7

. 0 7

1 0 0

. 3 1

. 1 0

. 0 5

1 0 0

. 3 0

. 0 4

. 0 3

. 0 3 1 0 0

. 3 0

. 0 4

. 0 3

4

1 0 0

. 3 0

. 1 5

. 1 5

. 1 4

8 1

. 3 2

. 2 6

. 1 0

1 0 0

. 3 0

. 1 1

. 1 0

. 1 0

1 0 0

. 3 2

. 1 4

. 0 9

1 0 0

. 3 0

. 0 7

. 0 7

. 0 7

1 0 0

. 3 1

. 0 7

. 0 6

1 0 0

. 3 0

. 0 4

. 0 3

. 0 3 1 0 0

. 3 0

. 0 4

. 0 3

2

2 1

. 4 1

. 1 2

. 1 4

. 1 3

2 1

. 4 1

. 1 2

. 1 3

3 9

. 3 5

. 0 9

. 1 0

. 1 0

3 7

. 3 5

. 0 9

. 1 0

7 2

. 3 1

. 0 7

. 0 7

. 0 7

7 1

. 3 1

. 0 7

. 0 7

9 9

. 3 0

. 0 4

. 0 3

. 0 3

9 9

. 3 0

. 0 4

. 0 3

N o t e .

%

¼

p e r c e n t a g e o f s o l u t i o n s t h a t w

e r e f u l l y p r o p e r . N o .

o f i n d i c . ¼ n

u m b e r o f i n d i c a t o r s : 8 ( i t e m s ) , 4 ( i t e m

p a i r s ) , 2 ( i t e m

q u a d r u p l e s ) . ‘ –

’ i n d i c a t e s s a m p l e

s i z e t o o

s m a l l f o r

A D F . s ¼

s k e w , k

¼

k u r t o s i s .

M

¼

m e a n ,

S E e ¼ e

m p i r i c a l S E ,

S E p ¼

p r e d i c t e d S E ,

S E r ¼

p r e d i c t e d

r o b u s t

S E .

P o p u l a t i o n

f a c t o r

c o r r e l a t i o n ¼

. 3 0 .

C o r r e l a t i o n e s t i m a t e s

a r e b a s e d o n l y o n r e s u l t s f r o m f u

l l y p r o p e r s o l u t i o n s .



8/25

T a b l e

2 .

S t u d y 1 : M e a n a n d v a r i a n c e o f

g o o d n e s s o f fi t ( T / d f ) b y s a m p l e s i z e ,

n u m b e r o f i n d i c a t o r s ,

d e g r e e o f n o n - n o r m a l i t y ,

a n d fi t t i n g f u n c t i o n

N ¼

5 0

N ¼

1 0 0

N ¼

2 5 0

N ¼

1 0 0 0

M L

A

D F

M L

A D F

M L

A D F

M

L

A D F

D e g r e e o f n o n - n o r m a l i t y

I n

d i c a t o r s / f a c t o r

T

T S B

T

T Y B

T

T S B

T

T Y B

T

T S B

T

T Y B

T

T S B

T

T Y B

M E A N

o f ( T / d f )

N o r m a l ( s ¼

0 ;

k ¼

0 )

8

1 . 1

7

1 . 2

4

–

–

1 . 0

8

1 . 1

1

–

–

1 . 0

4

1 . 0

5

1 . 8

8

1 . 0

5

1 . 0

2

1 . 0

2

1 . 1

6

1 . 0

3

4

1 . 0

8

1 . 1

5

1 . 7 3

1 . 0

1

1 . 0

4

1 . 0

7

1 . 3

4

1 . 0

5

1 . 0

2

1 . 0

3

1 . 1

2

1 . 0

2

1 . 0

1

1 . 0

1

1 . 0

3

1 . 0

1

2

0 . 6

7

0 . 7

3

0 . 7 5

0 . 7

2

0 . 8

1

0 . 8

7

0 . 8

0

0 . 7

8

0 . 8

9

0 . 9

2

0 . 8

9

0 . 8

8

0 . 8

7

0 . 8

7

0 . 8

7

0 . 8

7

S l i g h t l y

n o n - n o r m a l

8

1 . 2

0

1 . 2

5

–

–

1 . 1

2

1 . 1

2

–

–

1 . 0

7

1 . 0

6

1 . 8

7

1 . 0

5

1 . 0

5

1 . 0

2

1 . 1

6

1 . 0

3

( s ¼

0 :

5 ;

k ¼

0 :

5 )

4

1 . 1

0

1 . 1

6

1 . 6 8

1 . 0

0

1 . 0

6

1 . 0

8

1 . 3

3

1 . 0

4

1 . 0

4

1 . 0

3

1 . 1

1

1 . 0

2

1 . 0

5

1 . 0

3

1 . 0

5

1 . 0

2

2

0 . 7

1

0 . 7

6

0 . 7 3

0 . 7

0

0 . 8

4

0 . 8

7

0 . 8

1

0 . 8

0

0 . 8

1

0 . 8

2

0 . 8

4

0 . 8

3

0 . 8

5

0 . 8

5

0 . 8

5

0 . 8

5

M o d e r

a t e l y n o n - n o r m a l

8

1 . 2

8

1 . 2

6

–

–

1 . 2

0

1 . 1

2

–

–

1 . 1

7

1 . 0

6

1 . 8

5

1 . 0

4

1 . 1

4

1 . 0

2

1 . 1

7

1 . 0

4

( s ¼

1 :

0 ;

k ¼

1 :

5 )

4

1 . 1

7

1 . 1

7

1 . 6 6

0 . 9

9

1 . 1

3

1 . 0

8

1 . 3

1

1 . 0

3

1 . 1

1

1 . 0

3

1 . 1

0

1 . 0

1

1 . 1

2

1 . 0

3

1 . 0

6

1 . 0

4

2

0 . 7

5

0 . 8

0

0 . 7 1

0 . 6

8

. 8 7

. 8 5

0 . 8

0

0 . 7

9

0 . 8

8

0 . 8

4

0 . 8

2

0 . 8

2

0 . 8

9

0 . 8

5

0 . 8

5

0 . 8

5

V e r y n

o n - n o r m a l ( s ¼

1 :

5 ;

k ¼

3 :

2 5 )

8

1 . 4

2

1 . 2

8

–

–

1 . 3

4

1 . 1

3

–

–

1 . 3

1

1 . 0

6

1 . 8

1

1 . 0

3

1 . 2

9

1 . 0

3

1 . 1

7

1 . 0

4

4

1 . 2

7

1 . 1

8

1 . 6 2

0 . 9

8

1 . 2

4

1 . 0

9

1 . 2

8

1 . 0

1

1 . 2

1

1 . 0

3

1 . 1

0

1 . 0

1

1 . 2

4

1 . 0

3

1 . 0

6

1 . 0

4

2

0 . 8

3

0 . 8

5

0 . 7 5

0 . 7

2

0 . 9

1

0 . 8

7

0 . 7

5

0 . 7

3

0 . 9

9

0 . 8

8

0 . 7

8

0 . 7

8

0 . 9

7

0 . 8

8

0 . 8

7

0 . 8

7

B a l a n c e d ( s ¼

^ 0

:

5 ;

k ¼

^ 0

:

5 )

8

1 . 1

7

1 . 2

4

–

–

1 . 0

8

1 . 1

1

–

–

1 . 0

4

1 . 0

6

1 . 8

9

1 . 0

5

1 . 0

1

1 . 0

2

1 . 1

5

1 . 0

3

4

1 . 0

8

1 . 1

5

1 . 7 3

1 . 0

1

1 . 0

4

1 . 0

8

1 . 3

4

1 . 0

5

1 . 0

2

1 . 0

3

1 . 1

2

1 . 0

2

1 . 0

1

1 . 0

1

1 . 0

2

1 . 0

0

2

0 . 6

7

0 . 7

0

0 . 7 0

0 . 6

7

0 . 7

8

0 . 8

3

0 . 7

5

0 . 7

4

0 . 8

6

0 . 8

9

0 . 8

7

0 . 8

6

0 . 8

6

0 . 8

6

0 . 8

6

0 . 8

6

V a r i a n c

e o f ( T / d f )



9/25

T a b l e

2 .

C o n t i n u e d

N ¼

5 0

N ¼

1 0 0

N ¼

2 5 0

N ¼

1 0 0 0

M L

A

D F

M L

A D F

M L

A D F

M L

A D F

D e g r e e o f n o n - n o r m a l i t y

I n

d i c a t o r s / f a c t o r

T

T S B

T

T Y B

T

T S B

T

T Y B

T

T S B

T

T Y B

T

T S B

T

T Y B

N o r m a l ( s ¼

0 ;

k ¼

0 )

8

2 . 6

7

3 . 0

8

–

–

2 . 2

9

2 . 4

7

–

–

2 . 2

6

2 . 3

2

1 3 . 5

7

1 . 2

9

2 . 8

7

2 . 8

7

3 . 9

6

2 . 4

7

4

2 . 2

5

2 . 5

5

7 . 5 4

0 . 9

4

2 . 0

6

2 . 2

0

4 . 3

0

1 . 6

2

2 . 1

2

2 . 1

3

2 . 6

2

1 . 8

5

2 . 2

1

2 . 2

2

2 . 4

4

2 . 2

1

2

0 . 8

0

0 . 9

6

1 . 0 2

0 . 8

8

1 . 0

4

1 . 2

5

1 . 0

9

1 . 0

1

1 . 5

1

1 . 7

1

1 . 5

2

1 . 4

6

1 . 2

3

1 . 2

0

1 . 2

0

1 . 1

9

S l i g h t l y

n o n - n o r m a l

8

2 . 8

4

3 . 0

5

–

–

2 . 3

8

2 . 3

8

–

–

2 . 4

1

2 . 3

2

1 3 . 2

0

1 . 2

7

3 . 0

1

2 . 8

4

4 . 0

8

2 . 5

1

( s ¼

0 :

5 ;

k ¼

0 :

5 )

4

2 . 3

1

2 . 5

2

6 . 6 6

0 . 8

7

2 . 0

9

2 . 1

2

4 . 0

2

1 . 5

3

2 . 1

6

2 . 0

7

2 . 4

8

1 . 7

6

2 . 4

2

2 . 2

9

2 . 4

4

2 . 2

1

2

0 . 9

0

1 . 1

6

0 . 8 7

0 . 7

6

1 . 2

0

1 . 2

8

1 . 0

2

0 . 9

6

1 . 2

3

1 . 2

7

1 . 3

5

1 . 3

0

1 . 1

5

1 . 1

0

1 . 1

0

1 . 0

9

M o d e r

a t e l y n o n - n o r m a l

8

3 . 3

7

2 . 9

4

–

–

2 . 8

0

2 . 2

9

–

–

2 . 7

7

2 . 2

6

1 1 . 7

7

1 . 1

4

3 . 5

3

2 . 7

7

3 . 9

6

2 . 4

4

( s ¼

1 :

0 ;

k ¼

1 :

5 )

4

2 . 6

2

2 . 4

8

6 . 1 7

0 . 8

3

2 . 3

1

2 . 0

4

3 . 6

6

1 . 4

2

2 . 3

7

1 . 9

9

2 . 2

5

1 . 6

0

2 . 7

7

2 . 2

7

2 . 3

7

2 . 1

6

2

0 . 9

5

1 . 0

4

0 . 7 2

0 . 6

4

1 . 2

3

1 . 0

8

0 . 9

7

0 . 9

1

1 . 4

4

1 . 2

7

1 . 2

7

1 . 2

3

1 . 1

4

1 . 0

3

1 . 0

3

1 . 0

2

V e r y n

o n - n o r m a l

8

4 . 6

3

2 . 8

4

–

–

3 . 8

8

2 . 2

6

–

–

3 . 5

3

2 . 1

4

9 . 4

6

0 . 9

5

4 . 2

9

2 . 6

4

3 . 6

0

2 . 2

6

( s ¼

1 :

5 ;

k ¼

3 :

2 5 )

4

3 . 2

7

2 . 4

5

5 . 2 4

0 . 7

4

2 . 7

7

1 . 9

8

3 . 1

2

1 . 2

5

2 . 8

3

1 . 9

3

2 . 0

6

1 . 4

7

3 . 2

6

2 . 2

1

2 . 2

7

2 . 0

7

2

1 . 3

1

1 . 3

3

0 . 9 3

0 . 8

1

1 . 2

9

1 . 1

7

0 . 8

0

0 . 7

5

2 . 1

2

1 . 5

2

0 . 8

9

0 . 8

7

1 . 3

7

1 . 0

9

1 . 0

9

1 . 0

8

B a l a n c e d ( s ¼

^ 0

:

5 ;

k ¼

^ 0

:

5 )

8

2 . 7

0

3 . 1

2

–

–

2 . 2

9

2 . 4

4

–

–

2 . 2

0

2 . 2

3

1 3 . 1

3

1 . 2

7

2 . 8

7

2 . 8

7

3 . 9

6

2 . 4

7

4

2 . 2

5

2 . 5

3

7 . 2 1

0 . 9

1

2 . 0

7

2 . 2

0

4 . 4

1

1 . 6

3

2 . 1

6

2 . 2

0

2 . 6

9

1 . 8

9

2 . 3

7

2 . 3

8

2 . 5

0

2 . 2

9

2

0 . 9

8

1 . 1

0

0 . 9 7

0 . 8

4

1 . 0

2

1 . 2

2

1 . 0

4

0 . 9

6

1 . 4

3

1 . 6

5

1 . 5

4

1 . 4

9

1 . 1

4

1 . 1

2

1 . 1

2

1 . 1

1

N o t e . ‘

– ’ i n d i c a t e s s a m p l e s i z e t o o s m a l l

f o r A D F . s ¼

s k e w ,

k ¼

k u r t o s i s .



10/25

were left in each cell), these correlation estimates were also systematically biased,

particularly for small N .The SDs of the distribution of the correlation estimates (empirical SEs) provided an

empirical description of how variable the factor correlation estimates actually were, whereas the SEs and robust SEs (predicted SEs) were estimates of how variable the

factor correlations should be (Table 1). In this respect, the empirical SEs provided anempirical test of the predicted SEs. Whereas the variability of the parameter estimates

decreased as N increased for both the empirical and predicted SEs, there was not much effect on non-normality in these variability estimates. For all distributions considered

here—including the normal condition—the predicted ML SEs and ML robust SEs wereslightly negatively biased in that they underestimated the empirical SEs. In most cases,

for small values of N , the ML robust SEs were very slightly more negatively biased thanthe normal ML SE. ADF empirical SEs were very large (i.e. actual estimates were most

variable) and ADF predicted SEs were much smaller than the ML estimates or the ADFempirical SEs. Hence, the ADF estimates were substantially biased, and this bias was still

evident even for N ¼ 1000: In summary, all predicted SEs were negatively biased(i.e. smaller than the corresponding empirical SEs), but the ADF predicted SEs were

substantially more biased than the ML and ML robust SEs.The pattern of results based on factor loading estimates (not shown) was similar to

that for factor correlations. The ML estimates were reasonably accurate for all ML solutions based on eight items or four item pairs, and these estimates were little affectedby N or non-normality. ADF estimates were negatively biased, but the sizes of this biasdecreased as N increased.

5.2.3. Goodness of fit

Even for the normal data, there were systematic biases in the T ML /df and T ADF /df ratiosin that the values were too large (compared to the expected value of 1.0 for ‘true’models) for eight-indicator (item) solutions and too small for two-indicator (item-quadruple parcel) solutions. For the ML solutions, the T ML /df ratios increased with

increasing non-normality, but the T SB /df ratios were better behaved. For the ADFsolutions, the T ADF /df ratios were substantially larger than 1.0 for the four- and eight-

indicator solutions, but the T YB /df were better behaved. Interestingly, the T ML /df ratiosfor the balanced non-normal condition (with skews and kurtoses of þ0.5 and 20.5)

were more like the normal condition than the slightly non-normal condition (with

skews and kurtoses of þ0.5). Although we expected this for item-parcel solutions

(because the positive and negative values were counterbalanced in constructing theparcels), it is interesting to note that this pattern was evident in the solution based on

eight items as well.The T ML /df ratios were systematically less positively biased (closer to 1.0) for the

four-indicator solutions than for the eight-indicator solutions. This pattern, however, was evident even for the normal condition when N was small. Hence, at least part of thisapparent advantage of item parcels may be due to the number of indicators rather than

the use of item parcels per se. Whereas the T SB /df ratios were also somewhat better for the four-indicator solution than for the eight-indicator (item) solutions, this differencedid not vary systematically with the degree of non-normality.

The variances of the T ML

/df ratios across various models were also compared(Table 2). In all ML solutions based on items and item-pair parcels, the variances of the

T ML /df ratios were larger than 2.0 (see Section 5.1 such comparisons) and increased



11/25

substantially with increasing non-normality. Although this trend was evident in both the

eight-indicator and four-indicator solutions, the increase was substantially less for four-indicator solutions. In contrast to the variance of the T ML /df ratio, the variance of the

T SB /df ratio tended to decrease slightly with increasing non-normality. For the normalcondition, the variance of the T SB /df ratio was somewhat larger than the T ML /df ratio, but

with increasing non-normality the variance of the T ML /df ratio became substantially larger. The variability of the T ADF /df ratio was substantially greater than 2.0 in all

conditions, whereas the variance of the T YB /df ratio was systematically less than 2.0 for all but the N ¼ 1000 condition. For both the T ML /df and T SB /df ratios, the variability wassomewhat smaller for the item-pair parcel solutions than for item solutions. Whereasthis pattern was also evident in the T ADF /df ratio, the T YB /df ratio was smaller for the item

solution than for the item-pair parcel solution. At N ¼ 250 when N was sufficiently largeto use the ADF estimator for item-level solutions, the variances of the T ADF /df ratio weremuch larger than 2.0, and this trend was still evident for N ¼ 1000: This again suggestedthe inappropriateness of ADF for values of N considered here.

5.3. Summary and discussion

Several important findings came from Study 1. Convergence behaviour was substantially poorer for two-indicator solutions and ADF ð N ¼ 50Þ solutions, but there wassurprisingly little effect of non-normality or the use of item-pair parcels. Estimates of factor correlations and factor loadings for ML solutions demonstrated little effect of non-

normality, although ADF estimates were systematically biased and more variable for

N , 1000: There was some effect of non-normality evident in T ML /df estimates—interms of both mean levels and variability—that was reduced by the use of item parcelsand by the Satorra– Bentler correction. For ADF solutions, T ADF /df ratios were

consistently greater than 1.0, although those based on the Yuan–Bentler correctionseemed to be better behaved. Overall, the ML solutions behaved well despite thepresence of non-normality and, in contrast to the four-indicator solutions, there wasclear evidence that two-indicator solutions performed poorly. Compared to the ML

solutions, the ADF estimation procedure performed poorly, particularly for small N .The effects of parcelling were examined in greater detail. Ignoring the two-indicator

solutions, there was not much difference between the eight-indicator and four-indicator solutions in the means and SEs of their parameter estimates. However, the four-indicator solution resulted in systematically smaller estimates of T ML /df and its variability than the

corresponding eight-indicator solution. As non-normality was associated with system-

atically larger estimates of T ML /df and its variability (compared to the correspondingnormal conditions), the results suggested the use of item-pair parcels resulted in less bias

associated with non-normality—a major focus of the present investigation. A morecareful evaluation of the pattern of results, however, suggested that this conclusion

might be premature and too simplistic. Even with the normal data, there was an obviousreduction in the positive bias (e.g. the mean of T ML /df dropped from 1.17 to 1.08 at

N ¼ 50; see Table 2) and a decrease in T ML /df ratio variability in going from eight-indicator to four-indicator solutions, in a way similar to those with the non-normal datasets. Thus, our results showed that the T ML /df estimates and their variability weresimilarly less positive in both the normal and non-normal conditions. The pattern of

results was similar for normal and non-normal data, which led us to conclude that theeffect of using item pairs instead of items was not specific to non-normal data. Hence, it was unclear whether the apparent advantages of using item pairs with non-normal data



12/25

observed here were due to the number of indicators rather than the use of item parcels

per se.Interestingly, there was some evidence that the use of item pairs in the

counterbalanced non-normal condition resulted in less positively biased estimates of the T ML /df than the use of item pairs based on items with uniformly positive skews and

kurtoses. Whereas it is tempting to suggest that this is due to our strategy of constructing item pairs based on items with counterbalanced skews and kurtoses,

careful evaluation of the results again suggested a counter-interpretation. In particular,the solutions based on counterbalanced items resulted in smaller T ML /df estimates than

those based on items with uniformly positive skews and kurtoses, demonstrating thatthis pattern of results was not specific to the parcelled solutions. Thus, the smaller

T ML /df estimates of parcels with counterbalanced (as opposed to uniformly positive)skews and kurtoses were apparently due to the items used to construct these parcelsrather than the use of parcels per se.

Despite some important findings with practical implications, Study 1 was not

completely effective in evaluating strategies to overcome problems associated with non-normality. One potential problem was that the degree of non-normality in the simulated

data constructed in Study 1 was not sufficiently extreme to produce substantialproblems. The main exception to this pattern, perhaps, was the increase in the T ML /df

ratios with increasing non-normality that was reduced by the use of item-pair parcellingand the Satorra–Bentler correction.

6. STUDY 2

In constructing data varying in degree of non-normality, not all combinations of kurtosis

and skew are possible. Kurtosis is limited by skew in the sense that it is impossible toconstruct data sets with zero kurtosis but extreme skew. As we counterbalanced itemsin terms of both skew and kurtosis (positive skew matched with negative skew, positive

kurtosis matched with negative kurtosis), the non-normality considered in Study 1 wasnot extreme. In order to test the generalizability of our findings in Study 1, we generatedextremely non-normal items in Study 2 (skew ¼ 3; kurtosis ¼ 25—large kurtosis wasused because of the importance of its influence on CFA solutions; see Hu et al., 1992).

We also examined conditions when these extremely positively skewed items werecounterbalanced with negatively skewed items within the same factor or parcel.

6.1. Methods

In Study 1 we could not find particular advantages of counterbalancing non-normalconditions (with positive and negative skew) through parcelling. Given the smaller

skews and kurtoses (see Section 5.1) of these counterbalanced parcels (than those of theitems they were based on), the findings were unexpected and we used conditions

violating even more extremely the normality assumption in Study 2. Otherwise, the

methodology of Study 2 was like that already presented in Study 1. Study 2 was a3 £ 4 £ 3 design. In Study 2 we constructed items that violated more substantially thenormality assumption (skew ¼ þ3 or 23, kurtosis ¼ 25) than in Study 1. There were

three types of distribution in which items were all normally distributed, all positively skewed, or a mixture of positive and negative skews. For all three distributions, wecompared results based on analyses of eight items, fo

Date post:	06-Jul-2018
Category:	Documents
Upload:	ketut-widya
View:	217 times
Download:	0 times

The use of item parcels in structural equation.pdf

Documents