Structural examination of RIASEC scales in high school students: Variation across ethnicity and method q Saurabh Gupta a , Terence J.G. Tracey a, * , Paul A. Gore Jr. b a Arizona State University, 302 Payne Hall, MC-0611, Tempe, AZ 85287-0611, USA b University of Utah, 1705 Campus Center Dr., Rm. 327, Salt Lake City, UT 84112, USA Received 8 October 2007 Available online 3 December 2007 Abstract The structural validity of Holland’s model of vocational interests across racial/ethnic groups was examined in the pop- ulation of high school juniors in two states. The fit of the circumplex model to Holland’s RIASEC types as assessed by the UNIACT-R was evaluated for the general sample and five subgroups: Caucasian/Euro-Americans, African Americans, Asian Americans, Latinos, and Native Americans. Four different methods were used to test the proposed circumplex struc- ture, each with various circumplex definitions. Results indicate that nonparametric methods generally showed good model- data fit, whereas SEM-based results indicated less support. No differences in fit were found across ethnicity supporting the usage with U.S. ethnic groups. Ó 2007 Elsevier Inc. All rights reserved. Keywords: RIASEC structure; Circumplex; Interest structure; Ethnicity 1. Introduction Structural validity, a key facet of construct validity, is an oft neglected and poorly understood characteristic of psychological tests and measures. However, it is critical to establish the structural validity of the models upon which these tests and measures are based by confirming that the conceptually distinct, defining elements of the models relate to one another in a manner consistent with theory. In other words, since psychological constructs cannot be measured directly, the validity of their measurement can only be inferred by finding an expected pattern of covariation among variables in the observed data. This is a particularly important con- sideration when applying models and measures developed on one group to a racially/ethnically distinct group (Rounds & Tracey, 1996). Although Holland’s (1973, 1985) model of vocational interests has received consid- 0001-8791/$ - see front matter Ó 2007 Elsevier Inc. All rights reserved. doi:10.1016/j.jvb.2007.10.013 q A version of this manuscript was presented at the annual meeting of the American Educational Research Association, San Francisco, March 2007. This research was conducted by the first author while an intern with ACT, Inc. and appreciation is expressed to ACT for use of these data. * Corresponding author. Fax: +1 480 775 2735. E-mail address: [email protected](T.J.G. Tracey). Available online at www.sciencedirect.com Journal of Vocational Behavior 72 (2008) 1–13 www.elsevier.com/locate/jvb
Transcript
Available online at www.sciencedirect.com
Journal of Vocational Behavior 72 (2008) 1–13
www.elsevier.com/locate/jvb
Structural examination of RIASEC scales in highschool students: Variation across ethnicity and method q
Saurabh Gupta a, Terence J.G. Tracey a,*, Paul A. Gore Jr. b
a Arizona State University, 302 Payne Hall, MC-0611, Tempe, AZ 85287-0611, USAb University of Utah, 1705 Campus Center Dr., Rm. 327, Salt Lake City, UT 84112, USA
Received 8 October 2007Available online 3 December 2007
Abstract
The structural validity of Holland’s model of vocational interests across racial/ethnic groups was examined in the pop-ulation of high school juniors in two states. The fit of the circumplex model to Holland’s RIASEC types as assessed by theUNIACT-R was evaluated for the general sample and five subgroups: Caucasian/Euro-Americans, African Americans,Asian Americans, Latinos, and Native Americans. Four different methods were used to test the proposed circumplex struc-ture, each with various circumplex definitions. Results indicate that nonparametric methods generally showed good model-data fit, whereas SEM-based results indicated less support. No differences in fit were found across ethnicity supporting theusage with U.S. ethnic groups.� 2007 Elsevier Inc. All rights reserved.
Structural validity, a key facet of construct validity, is an oft neglected and poorly understood characteristicof psychological tests and measures. However, it is critical to establish the structural validity of the modelsupon which these tests and measures are based by confirming that the conceptually distinct, defining elementsof the models relate to one another in a manner consistent with theory. In other words, since psychologicalconstructs cannot be measured directly, the validity of their measurement can only be inferred by findingan expected pattern of covariation among variables in the observed data. This is a particularly important con-sideration when applying models and measures developed on one group to a racially/ethnically distinct group(Rounds & Tracey, 1996). Although Holland’s (1973, 1985) model of vocational interests has received consid-
0001-8791/$ - see front matter � 2007 Elsevier Inc. All rights reserved.doi:10.1016/j.jvb.2007.10.013
q A version of this manuscript was presented at the annual meeting of the American Educational Research Association, San Francisco,March 2007. This research was conducted by the first author while an intern with ACT, Inc. and appreciation is expressed to ACT for useof these data.
2 S. Gupta et al. / Journal of Vocational Behavior 72 (2008) 1–13
erable attention vis-a-vis structural validity (e.g., Day, Rounds, & Swaney, 1998; Rounds & Tracey, 1993,1996; Tracey & Rounds, 1993), previous studies have suffered from a number of methodological limitationsand truncated samples, which naturally raise doubts about the accuracy and generalizability of their conclu-sions. The purpose of this study was to re-examine the question of structural validity across U.S. racial/cul-tural groups using five of the most contemporary methods for examining Holland’s structure of vocationalinterests on a sample that is much more representative than has heretofore been studied, specifically the pop-ulation of high school juniors from two different states, not just a typical sample of college students or highschool students with college aspirations. We will begin by articulating a set of issues which drive cross-culturalinvestigations such as these. Then a brief overview of the nature of the structure of vocational interests asoperationalized by Holland will be followed by a review of the limitations of prior studies.
The use of any measure normed on one cultural group may yield specious comparisons when used on adifferent cultural group without the establishment of construct equivalence. In the establishment of constructequivalence, emic and etic issues are critical to consider. Emic and etic examinations can be thought of asbeing located on ends of a continuum in which the etic refers to cross-cultural applicability and universalisms,where as emic refers to culturally specific adaptation (Berry, 1979). Etic examinations involve evaluating thegeneralizability of unmodified models and measures across cultures with respect to intra-measure relations(comparable structural model-data fit across cultures) and/or extra-measure relations (corresponding conver-gent and divergent validity with constructs external to the model in other cultures). Emic examinations focuson the adaptation or translation of measures which are linguistically, culturally or institutionally tailored toanother culture (Tracey & Gupta, 2007). Like most others seen in the literature, the present study is an exam-ination of cross-cultural generalizability of Holland’s (1973, 1985) model of interests with more of an eticfocus.
Holland (1973, 1985, 1997) articulated a model of career interests that is represented structurally by a hexa-gon of six personality types. The six types, in order around the hexagon, are: Realistic (R), Investigative (I),Artistic (A), Social (S), Enterprising (E), and Conventional (C), and that can be used to classify people basedon their vocational and avocational interests. These same types can be used to classify occupational environ-ments based on the skills required of the worker and work demands of these environments. The ordered typesare arranged with equal spacing around the hexagon. In addition, relations among these types are propor-tional to their proximity. Several implications stem from this model. Chief among them for vocational guid-ance practitioners is the notion that selecting a career corresponding to one’s interests promotes satisfactionand success. Although the hexagon has been used to visually represent the relations among interest variables, acircle may be superimposed on the vertices of the hexagon (Tracey & Rounds, 1993). Also, since statisticallythese relations represent what has been referred to as a circumplex (Guttman, 1954), the terms hexagon andcircle will be used interchangeably in this paper.
Empirical investigations examining the structural validity across cultures, ethnic groups, and nationalitieshave been conducted for over three decades (Armstrong, Hubert, & Rounds, 2003; Day & Rounds, 1998; Dayet al., 1998; du Toit & de Bruin, 2002; Farh, Leong, & Law, 1998; Fouad & Dancer, 1992; Fouad, Harmon, &Borgen, 1997; Fouad & Mohler, 2004; Hansen, Scullard, & Haviland, 2000; Leong, Austin, Sekaran, &Komarraju, 1998; Rounds, Tracey, & Hubert, 1992; Ryan, Tracey, & Rounds, 1996; Sverko & Babarovic,2006; Swanson, 1992; Tak, 2004; Tang, 2001; Tracey, Watanabe, & Schneider, 1997; Yom, Doughtie, Chang,Alston, & Wakefield, 1975), including a special issue devoted to this topic (Tinsley, 1992), two structural meta-analyses (Rounds & Tracey, 1996; Tracey & Rounds, 1993) and two longitudinal studies (Darcy & Tracey,2007; Tracey & Rounds, 2005). In sum, the results of these studies are equivocal, with some indicating poorfit, others suggesting comparable fit and yet others demonstrating superior fit of U.S. minority groups relativeto the dominant group. We contend that sample artifacts, varying methods and analyses deployed in thesestudies may well account for inconsistent results.
The earliest of these structural investigations (Yom et al., 1975) is better understood as a study of structuralinvariance rather than structural validity per se. The vocational interests of a minority group were mapped outon two dimensions and geometric distance analyses were performed to evaluate the extent to which they weresimilar to a Caucasian group’s interests with respect to proximity. Geometric distance analysis was also usedto test the hexagon model, which specifies the order relations among the variables in Holland’s model (Fouad& Dancer, 1992; Swanson, 1992). Apart from the incomplete specification of the order relations in these stud-
S. Gupta et al. / Journal of Vocational Behavior 72 (2008) 1–13 3
ies (Tracey & Rounds, 1993), the use of the geometric distance analysis has been demonstrated to be an inap-propriate and flawed test of structure (Hubert & Arabie, 1987; Rounds et al., 1992).
A number of other researchers conducted studies seeking to examine the structure of vocational interests ofvarious samples by employing exploratory spatial mapping techniques through the use of principal compo-nents analysis (PCA), exploratory factor analysis (EFA), and multidimensional scaling (MDS). Then, byexamining the plots some judgment was made about accordance with the model structure. However, Tracey(2000) has argued that this method amounts to little more than eyeballing and is far too subjective without atangible fit statistic. In addition, any two plots may deviate in different ways from the expected model, render-ing an objective comparison of any two samples’ model-data fit untenable. Fabrigar, Visser, and Browne(1997) are also critical of these methods because of the lack of clear criteria for the number of factors/com-ponents to extract and the common misuse of the VAF statistic as a global fit statistic.
Various confirmatory approaches then began to supplant exploratory methods, each with its own uniquespecification of Holland’s RIASEC circumplex (see Tracey, 2000 for a more detailed explanation of the vary-ing ways in which circumplex models are operationalized). Among the most commonly used confirmatoryapproaches is the randomization test of hypothesized order relations (Hubert & Arabie, 1987). Otherapproaches include constrained MDS, circular unidimensional scaling or CUS (Armstrong et al., 2003),and CIRCUM (Browne, 1992).
Several studies have seen the application of these methods (e.g., Armstrong et al., 2003; Day & Rounds,1998; Day et al., 1998; du Toit & de Bruin, 2002 Farh et al. 1998; Fouad et al., 1997; Fouad & Mohler,2004; Hansen et al., 2000; Leong et al., 1998; Rounds et al., 1992; Ryan et al., 1996; Sverko & Babarovic,2006; Swanson, 1992; Tak, 2004; Tang, 2001; Tracey, et al., 1997). However, none have drawn together eachof these methods in one examination of the same data sets. In addition, conclusions drawn from these studiesabout generalizability are necessarily limited because sample artifacts.
Notwithstanding, a particular investigation is notable in the diversity of methods used and the range andsize of its samples. Darcy and Tracey (2007) recently completed a study in which they sought to address manyof the shortcomings represented in the prior literature by drawing together several methods of analysis andevaluating the fit of Holland’s model using large representative samples of college aspirants. In their study,they examined the fit of the model longitudinally, and across gender and found few notable differences inthe fit of the model across gender. This generally supports much of the prior research. However, they did finddifferences in model-data fit across methods. Although the scope of cultural groups from which Darcy andTracey sampled and the variety of methods they utilized represents considerable methodological progressand improvements in generalizability, ultimately they were limited to the data of college aspirants. What couldbe concluded about cross-cultural structural validity from a study in which samples are not truncated byfuture educational goals? A more inclusive sample could potentially indicate a different picture on this score.This is the question examined in this study. Specifically, we focused on examining the circumplex structure ofHolland’s RIASEC types across the major U.S. ethnic groups using the prominent methods on the entire pop-ulations of high school juniors in two states. As such, this sample is unique in the literature. Additionally, webelieve that the array of analytical methods compensates for any potential blind-spots of any one method,allowing us to be more certain that the results are not idiosyncratic of the analytical method.
2. Methods
2.1. Sample
The sample, which represents the uniqueness and strength of this study, is a census-tested sample that satfor the ACT assessment in 2004 and completed the UNIACT (Unisex Edition of the ACT Interest Inventory).This sample is really the entire population of high school juniors in Illinois and Colorado, where the ACTassessment is required of all students in the junior year as a high-stakes test. The sample is invaluable in thatit does not preclude individuals who are not college bound from inclusion. The total sample size of this census-tested group was 115,567 participants. Among these were 55,634 males (48.14%) and 59,933 (52.86%) females.The racial/ethnic composition of this group by self-identification was: 11,865 African Americans (10.27%);5147 Asian Americans (4.45%); 83,489 Euro-Americans (72.24%); 14,084 Latinos (12.19%); and 982 Native
4 S. Gupta et al. / Journal of Vocational Behavior 72 (2008) 1–13
Americans (0.85%). Latinos represent all those who self-identify as Mexican, Puerto Rican, Cuban or ‘‘otherHispanic’’ Americans.
2.2. Measures
2.2.1. Unisex Edition of the ACT Interest Inventory (UNIACT-R, Level I, Swaney, 1995)
The UNIACT-R is a 90-item interest inventory packaged with the ACT national assessment. Each itemconsists of activities that correspond to one of six RIASEC interest areas. Respondents indicate preferencefor the activity on a three-point scale (dislike, indifferent, like). The responses, based on 15 items for each inter-est area, yield six scale scores. The UNIACT-R was developed to provide scores based on a unisex set of itemsand combined sex norms. Extensive reliability and validity support for the UNIACT-R have been provided bySwaney (1995).
2.3. Analyses
The units of analysis were the correlation matrices among the RIASEC scale scores. The exception wasMDS and CUS, which used dissimilarity matrices which was a linear transformation of correlation matrices(1 � r). For each of the samples, an observed correlation matrix for the general sample was produced. In addi-tion, the general sample was disaggregated by self-reported race/ethnicity resulting in an additional five matri-ces, specifically African Americans, Asian Americans, Latino Americans, Native Americans, and EuropeanAmericans. The total number of matrices was thus 6.
Besides each method varying in assumptions made and means used in evaluating model-data fit, they alsovary on the specific model being fit to the data. There are several alternative means of specifying a circumplexand the differences in these model representations need to be understood as well as the difference in the specificanalytic approaches.
2.3.1. Randomization test of hypothesized order relations
The randomization test of hypothesized order relations (Hubert & Arabie, 1987; Rounds et al., 1992) wasapplied to each of the matrices. Specifically this analysis tests the ‘‘tight circular order’’ model where a perfectcircumplex of equally distant types (called a circulant, Wiggins, Steiger, & Gaelick, 1981) is assumed and therelative ordering of the types examined. Specifically the correlations of types adjacent on the circle (e.g., R–I)should be greater than the correlations between types one step apart on the circle (e.g., R–A), which in turnshould be greater than correlations between types opposite on the circle (e.g., R–S). This results in a total of 72unique order predictions for a tight circular order model. The extent to which each correlation matrix matchedthe tight circular ordering was assessed and then this distribution was compared with the distribution thatresulted from examining all the permutations of the rows and columns of the correlation matrix, resultingin an exact p value. In addition, the randomization test yields a Correspondence Index (CI) which indicatesthe extent to which the model predictions were met in the data. The CI is a correlation and ranges from�1 (where every order prediction was not met) to +1 (where every prediction was met). In a meta-analysisof RIASEC studies, Rounds and Tracey (1996) found a benchmark CI value of .70 across a broad sampleof U.S. samples and measures. This value serves as a comparison. This analysis was conducted on each matrixusing the RANDALL program (Tracey, 1997). The program was also used to examine the difference in modelfit across pairwise samples. In this randomization comparison, the difference in model fit to the data is exam-ined relative to the random distribution of the rows and columns of the data rather than just the fit of themodel to the data. Specifically, the number of times the model predictions were confirmed in one sampleand not the other was compared to the random distribution of times these occurred when using the permu-tation of the rows and columns of the data matrices (Anderson, Tracey, & Rounds, 1997).
2.3.2. Multidimensional scaling (MDS)
Constrained MDS was applied to the general sample and each of the five ethnic groups; however, the cor-relations matrices were transformed into dissimilarity matrices by multiplying each element by (1 � r). TheSINDSCAL program (Pruzansky, 1975) was used to conduct the analyses. Specifically, we constrained the
S. Gupta et al. / Journal of Vocational Behavior 72 (2008) 1–13 5
MDS to a perfect circumplex model of equal spacing among the types (i.e., a circulant). This was accom-plished by using the following coordinates for the RIASEC scales: 1.0, 0.5, �0.5, �1.0, �0.5, 0.5 on dimension1 and 0.0, 0.86, 0.86, 0.0, �0.86, �.86 on dimension 2. These values are the geometric coordinates of a hexagonwith a radius of 1.0. The fit of the circulant model to each data set was indicated using the variance accountedfor (VAF). MDS provides only this information index; there is no inferential test of fit.
2.3.3. CIRCUM
Browne’s (1992, 1995) Circular Stochastic Process Model with a Fourier series correlation function(CSPMF) was also applied to each of the 6 correlation matrices, as operationalized by the CIRCUM program.This program requires that the data be normally distributed and uses a structural equation modeling to exam-ine four successively more constrained circular models. The loose circular ordering model is one where thescales are loosely arranged around a circle of unequal distances between scales and unequal radii. The nextmodel is one where the scales are constrained to have equal communality (i.e., have equal radii). The thirdmodel is one of equal spacing of the scales around the circle. The final model is one of equal spacing and equalcommunality (i.e., the circulant model). Each of these models is fit to each data set and the model with the bestfit is selected.
Similar to the estimation procedures used in most SEM applications, CIRCUM produces estimates basedon maximum likelihood estimation and also generalized least squares. For the purposes of this paper the for-mer was used. The program allows the user to change several input parameters. Of concern is the specificationof the ‘‘m’’ parameter which is used in initialization. Fabrigar et al. (1997) recommended that several differentvalues of ‘‘m’’ be used and the one with the best fit retained. As such, we examined each of the four differentmodels (loose circle, equal communality, equal spacing, and circulant) with m equal to 1, 2, and 3 in successiveanalyses. This results in 12 unique combinations of free parameters and model constraints for each of the 6data matrices tested. A cumbersome 72 total analyses were conducted and their associated fit statisticsproduced.
The CIRCUM program provides the chi-square goodness of fit statistic and the Root Mean Squared Errorof Approximation (RMSEA) as fit statistics. Because the chi-square goodness of fit statistic will almost alwaysbe significant (i.e., indicating bad fit) in cases with large sample sizes as is true here (e.g., Marsh, Balla, &McDonald, 1988), we focus on the RMSEA as the indicator of fit. Browne and Cudeck (1993) suggest anRMSEA cut off of .05 or less as indicative of very close model-data fit. RMSEA values between .05 and.08 are considered good, while values between .08 and .10 are considered mediocre (MacCallum, Browne,& Sugawara, 1996).
2.3.4. Circular unidimensional scaling
Finally, circular unidimensional scaling (CUS, Armstrong et al., 2003), using MATLAB software wasapplied to the same 6 dissimilarity matrices [i.e., correlations transformed by multiplying by (1 � r)] as usedin the MDS analyses. The MATLAB files needed to conduct the CUS analyses are found at the following ftpsite: ftp://www.psych.uiuc.edu/pub/cda. CUS is a nonparametric approach to the scaling of data similar toMDS except it focuses on a circular structure. It enables examination of two models: a loose ordering of scalesaround the circle (i.e., unequal spacing and unequal radii), and an equal spaced circulant model. Like MDS, fitinformation is provided via variance accounted for (VAF). There is no inferential statistic of model-data fit.To help in this area, Armstrong et al. (2003) conducted a Monte Carlo study examining the fit of the looseordering and the circulant models to random data. They found a mean VAF of .35 for the loose orderingmodel and .07 for the circulant. Using Cohen’s (1977) large effect size of .25 as a guide, they suggested thata good fit would be demonstrated by VAF values of greater than .60 (.35 + .25) for the loose ordering modeland .33 (.07 + .25) for the circulant model. Further differences between the loose ordering and circulant mod-els of less than .27 (.60 � .33) were proposed as indicative of viability of the circulant model relative to theloose ordering model.
The CUS approach first requires the determination of the optimal ordering of the scales around the circle.As this process is subject to local minima problems, it needs to be repeated multiple times to ensure that theoptimal order is selected. Following this, the fit of the loose ordering (unconstrained) and the circulant (con-strained) models were estimated.
6 S. Gupta et al. / Journal of Vocational Behavior 72 (2008) 1–13
So overall, there were evaluations of five different representations of the circumplex (i.e., loose circularordering, equal communality, equal spacing, tight circular ordering, and circulant), examined using four dif-ferent methods (Randomization test of hypothesized order relations, Constrained MDS, CUS, and CIRCUM)as they fit six different RIASEC matrices (general sample, African Americans, Asian Americans, Latino Amer-icans, Native Americans, and European Americans).
3. Results
The results of the randomization test of hypothesized order relations are also summarized in Table 1. Theresults indicated that the tight circular order model significantly fit the data of the general sample and each ofthe subsamples at the p = .0167 level. The correspondence indexes of each of the samples are above theRounds and Tracey (1996) U.S. benchmark of .70, with the exception of the Asian American sample, whichproduced a CI value of .68. The fit of the circular order model for Asian Americans is still quite good and notsignificantly different from that of any of the other samples. In fact, pair-wise comparisons of every samplewith every other yielded no significant differences. These results indicate support for the circular order modelof Holland’s hexagon vis-a-vis the six samples. Additionally, when compared to the CI values obtained byDay et al. (1998), who concluded good model-data fit across all ethnic groups, the CI values obtained in thisstudy were much higher. Day et al. obtained an average CI across groups of .68, whereas in this study weobserved an average of .80 across the six matrices tested.
The results of the circulant constrained MDS are presented in the right column of Table 1. All of the var-iance accounted for estimates (VAF’s) were quite high and rather uniform, ranging from .79 for the AsianAmerican and African American samples, to .85 for the General, Native American, and Euro-American sam-ples. VAF for the Latino data set was .83. These results indicate that the constrained circulant model accountsfor a large and relatively equal proportion of the variance in each of the samples. These values were similar inmagnitude to those found in previous studies (e.g., Day et al., 1998; Rounds & Tracey, 1993, 1996).
The results of the many different models and parameters examined in the CIRCUM analysis are summa-rized in Table 2 and the model with the best fit to the data in each sample is underlined. For every sample thisbest model was the loose order model (unconstrained) where there was an acceptable fit to the data in all butone sample with RMSEA values of .08 or lower. The sole exception was the Asian American sample where theRMSEA value for the loose ordering model was .113, indicating a poor fit to the data. The fit of each of themore constrained models to the data was poor. The plot of the loose ordering model for each sample is pre-sented in Fig. 1. The lengths of the radii indicate the extent to which the variance in variables is explained bythe dimensions underlying the structure. The angular distance between any two variables is a graphical rep-resentation of their relatedness. For example, across samples, it appears that responses to Investigative andArtistic scales were more similar than responses to Conventional and Realistic. The proximity of the Investi-gative and Artistic scales may raise doubt about whether two distinct variables were really being measured. Itis also notable that although Asian Americans demonstrated the poorest relative fit, as compared to the otherracial/ethnic groups, Fig. 1 shows a representation of parameter estimates that seems more circular than anyother group’s. However, the Investigative and Artistic items seem less distinct than for other groups.
Table 1Results of the randomization test of hypothesized order relations and the constrained multi dimensional scaling
8 S. Gupta et al. / Journal of Vocational Behavior 72 (2008) 1–13
Finally, the results of the circular unidimensional scaling analysis are summarized in Table 3. Both the looseordering and circulant models were analyzed. As indicated by the VAF values in Table 3, all of the samples’unconstrained (loose ordering) analyses were found to be above the acceptable threshold of .60, recommendedby Armstrong et al. (2003). In addition, all of the constrained (circulant) analyses were well above the recom-mended VAF of .33. Also, because the change in VAF in each case was considerably less than .27, the circ-ulant model was more appropriate structure for representing the relations among the RIASEC types becausethe changes in VAF for the unconstrained model are due to the greater number of parameters being estimated.
4. Discussion
The aim of this study was to examine the structure of interests across various American racial/ethnic groupsof high school students to determine the relative fit of the RIASEC hexagon, and therefore validity, of Hol-land’s (1973, 1985, 1997) structural model as operationalized by the UNIACT-R. This study sets a precedent
General African American Asian American
Caucasian Latino Native American
R
I
A
S
E
C
R
IA
S
E
C
R
I
A
S
E
C
R
I
A
S
E
C
R
I
A
S
E
C
R
IA
S
E
C
Fig. 1. Graphic representation of RIASEC types for each ethnic group of the loose circular model from the CIRCUM analyses.
Table 3Variance accounted for (VAF) in circular unidimensional scaling analysis
S. Gupta et al. / Journal of Vocational Behavior 72 (2008) 1–13 9
in that it represents the first attempt to draw together various contemporary methods for assessing variousstructural models with samples of five different self-identified racial/ethnic groups. In addition, the samplein this study is almost completely free of selection artifact, owing to the fact that the data were drawn fromall high school juniors, not just those who are college-bound. This makes a much stronger case for the repre-sentativeness of the sample than could be made under the typical circumstances of empirical studies. Thisstudy also presents an occasion to evaluate the different methods of circumplex structural analysis. Thesemethods are the randomization test of hypothesized order relations, constrained multidimensional scaling, cir-cular unidimensional scaling, and the SEM based CIRCUM. The results indicate that there were largely neg-ligible differences in the structure of the RIASEC scores across racial/ethnic group, as operationalized by theUNIACT-R. However, there were clear differences in the extent to which the structure was supported acrossthe different methods used.
In general, where racial/ethnic group differences are concerned, each of the nonparametric methods (i.e.,MDS, randomization test of hypothesized order relations, and the UCS) failed to detect notable differencesper the specific structural model tested. Each of these approaches indicated support for the models examined.
10 S. Gupta et al. / Journal of Vocational Behavior 72 (2008) 1–13
Specifically, the randomization test examines the fit of the tight circular order model, while MDS and CUS testthe fit of the circulant model. Where difference tests were available, as in the randomization test, none of thepairwise comparisons of fit among the different racial/ethnic groups reached significance. This parallels thefindings of Day et al. (1998), who found no differences across Caucasians, Hispanics and African Americansfrom their sample (N = 4133) of 10th graders, which they drew from the UNIACT normative sample. Thus, itis not surprising that the results of this study, which extend to two additional racial/ethnic groups, also failedto find group differences.
The results of the CIRCUM analyses are less supportive of the fit of the circumplex model to the data. Ineach sample, the best fit to the data was the least constrained, loose ordering model. The loose ordering modelwas an adequate fit in each sample with the exception of the Asian American sample where none of the modelsfit the data adequately.
It is also notable that almost without exception, Asian Americans demonstrated the lowest relative fit to thevarious models tested, as compared to the general sample—of which it is a part—and the other four racial/ethnic groups. However, it may be untenable to make any strong conclusions regarding this result since itis an assessment based on relative degree of fit rather than an inferential test of differences in fit. Only the ran-domization test allows for such an evaluation and, as stated earlier, revealed no significant differences in fitacross any of the 6 matrices tested. This suggests that although the ways in which Asian Americans construevocational interest items may be least consistent with the theoretical model, the scores of this group are none-theless valid and interpretable.
The different conclusions yielded by the nonparametric analyses and the SEM based analysis (i.e., CIR-CUM) are similar to those found by Darcy and Tracey (2007) in their examination of gender and age differ-ences in a sample of college aspirants. They found support for the circumplex nature of RIASEC types usingthe nonparametric tests while the SEM based test indicated a general lack of support. They attribute theseresults to the different question addressed in each analysis. The nonparametric tests each compare the fit ofthe model to the fit attainable by chance. Is the fit better than that that would be obtained if the data wererandom? The SEM based analyses focus on assessing the fit relative to perfect fit. Does the model perfectlyfit the data? So one approach uses random as the comparative standard and the other uses the ideal. Relativeto random, the circumplex fits all racial/ethnic groups well and equally. Relative to ideal, RIASEC data isloosely ordered in a circular manner with varying radii and spacing. We view these results as supportingthe general adequacy and equality of the RIASEC circumplex structure in general and across ethnic groupsbut also that there are more finely tuned deviation from a circulant model.
The circulant model would yield spacing that has 60 degrees separating each RIASEC scale. However, asshown in the modeling of estimates of polar angles from the CIRCUM analysis (Fig. 1), the Realistic, Inves-tigative, and Artistic scales seem less distinct that one may expect based on the model. In contrast, it appearsthat the items that make up the Conventional scale clearly look as if they are being construed very differentlyfrom the items of any other scale. It has been demonstrated that interest items are distributed uniformlyaround the circle (Tracey & Rounds, 1995), so perhaps carving up vocational interests into only six slicesmay be too crude. Perhaps eight or ten slices may prove to be a better fit.
The central question addressed is whether the RIASEC structure, as operationalized by the UNIACT-R isreified by the data. In other words, do the groups in general, and the various racial/ethnic groups in specificconstrue the items in the way they are intended; and is it safe to conclude that the constructs that the assess-ment aims to measure, are actually being measured? According to the result, there seems to be a preponder-ance of evidence in the affirmative. The evidence is quite strong that the circular order model and models ofloose circular arrangement fit the data of the groups evaluated. In other cases, the more restrictive circulantmodel demonstrated less unequivocal support. This indicates that in their work with people from most racial/ethnic groups in the United states, career counselors can use the results of vocational interest assessments withconfidence that that they posses some measure of validity.
Ultimately, Holland’s and Gottfredson’s (1992) cautionary editorial about stalking the perfect hexagonmay need to be heeded, but in a critical light. Although some analytic methods, which assume perfect hexag-onal fit, will indeed fail to find it, nonparametric methods, which require a fit to model that is better thanchance can reasonably be expected to show adequate model-data fit for this instrument with diverse Americanracial/ethnic groups.
S. Gupta et al. / Journal of Vocational Behavior 72 (2008) 1–13 11
Vocational guidance professionals can use the UNIACT-R to identify broad interest areas and make roughassessments of vocational personalities of high school students that take the ACT assessment to help them intheir career exploration and decision-making process. In addition, the results suggest that students who indi-cate strong interests in Investigative careers may also likely find Realistic- or Artistic-type careers similarlyworthwhile. By contrast, they are least likely to also find Conventional-type careers of interest. However,the results of the CIRCUM analysis indicate that vocational guidance professionals may need to exercise addi-tional caution when working with Asian Americans, who may construe the organization of careers alongdimensions that are more dissimilar to the intended model. That is, they may be less apt to think of the dis-tinctions among various careers as characterized by data-ideas and people-things dimensions. Researchersshould posit and test other models for Asian Americans that, based on theoretical grounds, would better rep-resent how this group parses out careers. Additionally this speaks to the need for emic investigations based onhow Asian Americans evaluate vocational choices. Research which has been conducted which articulate dif-ferent theories about the vocational behavior of various Asian American groups (e.g., Gupta & Tracey, 2005;Hardin, Leong, & Osipow, 2001) may suggest starting points for such future investigations.
Some limitations of this study should be noted in order to provide a context for the results of the study.First, although the range of the sample was broader than seen in previous studies in terms of type of student(high school juniors as well as college-bound high school seniors), the sample was restricted to two states: Illi-nois and Colorado. The generalizability may be limited, insofar as regional differences may affect how mem-bers of various ethnic groups construe vocational interests. Also, the results indicate support for Holland’smodel as operationalized by the UNIACT-R. However, several other currently used vocational interest mea-sures have not undergone the same testing and scrutiny. Another limitation is the restriction in the age rangeof this sample. The results may be generalized to 15–17 year-olds, however, it may be erroneous to apply theresults to older adults.
It is critical to provide evidence for the construct validity of the model upon which a measure is based. Sub-sequent studies should explore the model-data fit for other popular interest inventories that are commonlyused in practice today using similar methods seen here. In addition, as the ACT assessment is increasinglyapplied in a growing number of states as a graduation requirement, tests of model-data fit on the UNIACTshould be re-conducted to include a broader sample of students from a wider geographical range. Greatereffort should be placed on obtaining larger samples of Native Americans and Asian Americans. In future stud-ies, researchers may wish to examine this issue with subsets of equal samples. For example, a researcher couldrandomly select the data of 500 students from each racial/ethnic group and re-run the analyses several times.Having equal sample sizes would enable more comparisons particularly comparison across groups using thechi-square fit statistic, which is contingent upon sample size.
The real strengths of this study are two fold. First, a comprehensive set of disparate methods to test circu-larity in data have been drawn together in one study and tested on the same six samples. Further, the methodsused are the most methodologically sophisticated and contemporary procedures drawn from the literature andmost of them should be familiar to researchers. This enables consumers of this research to evaluate the relativeconsistency of results across methods and samples. In addition, because each method tests different versions ofthe circumplex with different assumptions, less uncertainty remains about the potential blind-spots of any onemethod. Using the various methods also enables some comparisons across methods as well as within methods,as discussed earlier.
The second primary strength of the study is its sample characteristics. The use of this broad-based sample isunprecedented in the literature. The data is drawn from a very large representative sample, which includes allseniors in high school. There is perhaps no other institution from which one can be in a position to sampleinterest data and attain comparable generalizability as high school. That is, a statewide sample of high schoolseniors represents the most diverse sampling of people who would otherwise be quite unlikely to be capture bythe same net cast by a researcher. Post high school represents a considerable dispersion of people into differentsocial classes, occupations, educational strata and racial/ethnic group clusters. Obtaining data from theseniors in this study is a unique opportunity to test Holland’s model with considerably less concern for sampleartifact. As a result, the data also enable meaningful comparison across groups. In terms of the literature, thisstudy is placed such that it addresses the gaps left by Darcy and Tracey (2007), who used the same methods,but only across gender and time; and Tracey and Robbins (2005), who compared similar racial/ethnic subs-
12 S. Gupta et al. / Journal of Vocational Behavior 72 (2008) 1–13
amples, but used a more limited methodology. Ultimately, the results of this study suggest that the UNIACT-R has good construct validity for both European Americans—on whom its underlying theory was devel-oped—and minority groups alike. Using the measure with members of all groups will potentially yield impor-tant information which can guide the vocational guidance and career development process.
References
Anderson, M. Z., Tracey, T. J. G., & Rounds, J. (1997). Examining the invariance of Holland’s vocational interest model across sex.Journal of Vocational Behavior, 50, 349–364.
Armstrong, P. I., Hubert, L., & Rounds, J. (2003). Circular unidimensional scaling: A new look at group differences in interest structure.Journal of Counseling Psychology, 50, 297–308.
Berry, J. W. (1979). Research in multicultural societies: Implications of cross-cultural methods. Journal of Cross-Cultural Psychology, 10,415–434.
Browne, M. W. (1992). Circumplex models for correlation matrices. Psychometrika, 57, 469–497.Browne, M. W. (1995). CIRCUM: Satellite program to AUFIT for circumplex models. Columbus, OH: Psychology Department, Ohio
State University.Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural
equation models (pp. 136–162). Newbury Park, CA: Sage.Cohen, J. (1977). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.Darcy, M. U. A., & Tracey, T. J. G. (2007). Circumplex structure of Holland’s RIASEC interests: An examination of models by time,
gender and method. Journal of Counseling Psychology, 54, 17–31.Day, S. X., & Rounds, J. (1998). Universality of vocational interest structure among racial and ethnic minorities. American Psychologist,
53, 728–736.Day, S. X., Rounds, J., & Swaney, K. (1998). The structure of vocational interests for diverse racial–ethnic groups. Psychological Science,
9, 40–44.du Toit, R., & de Bruin, G. P. (2002). The structural validity of Holland’s RIASEC model of vocational personality types for young Black
South African men and women. Journal of Career Assessment, 10, 62–77.Fabrigar, L. R., Visser, P. S., & Browne, M. B. (1997). Conceptual and methodological issues in testing the circumplex structure of data in
personality and social psychology. Personality and Social Psychology Review, 1, 184–203.Farh, J., Leong, F. T. L., & Law, K. S. (1998). Cross-cultural validity of Holland’s model in Hong Kong. Journal of Vocational Behavior,
52, 425–440.Fouad, N. A., & Dancer, L. S. (1992). Cross-cultural structure of interests: Mexico and the United States [Special issue]. Journal of
Vocational Behavior, 40, 129–143.Fouad, N. A., Harmon, L. W., & Borgen, F. H. (1997). Structure of interests in employed male and female members of U.S. racial–ethnic
minority and nonminority groups. Journal of Counseling Psychology, 44, 339–345.Fouad, N. A., & Mohler, C. J. (2004). Cultural validity of Holland’s theory and the Strong Interest Inventory for five racial/ethnic groups.
Journal of Career Assessment, 12, 423–439.Gupta, S., & Tracey, T. J. G. (2005). Dharma and interest-occupation congruence in Asian Indian college students. Journal of Career
Assessment, 13, 320–336.Guttman, L. (1954). A new approach to factor analysis: The radix. In P. F. Lazarsfeld (Ed.), Mathematical thinking in the social sciences
(pp. 258–348). Glencoe, IL: Free Press.Hansen, J. C., Scullard, M. G., & Haviland, M. G. (2000). The interest structures of Native American college students. Journal of Career
Assessment, 8, 159–172.Hardin, E. E., Leong, F T. L., & Osipow, S. H. (2001). Cultural relativity in the conceptualization of career maturity. Journal of Vocational
Behavior, 58, 36–52.Holland, J. L. (1973). Making vocational choices: A theory of careers. Englewood Cliffs, NJ: Prentice Hall.Holland, J. L. (1985). Making vocational choices: A theory of vocational personalities and work environments (2nd ed.). Englewood Cliffs,
NJ: Prentice-Hall.Holland, J. L. (1997). Making vocational choices (3rd ed.). Odessa, FL: Psychological Assessment Resources.Holland, J. L., & Gottfredson, G. D. (1992). Studies of the hexagonal model: An evaluation: Or The perils of stalking the perfect hexagon
[Special issue]. Journal of Vocational Behavior, 40, 158–170.Hubert, L., & Arabie, P. (1987). Evaluating order hypotheses within proximity matrices. Psychological Bulletin, 102, 172–178.Leong, F. T. L., Austin, J. T., Sekaran, U., & Komarraju, M. (1998). An evaluation of the cross-cultural validity of Holland’s theory:
Career choices by workers in India. Journal of Vocational Behavior, 52, 441–455.MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure
modeling. Psychological Methods, 1, 130–149.Marsh, H. W., Balla, J. R., & McDonald, R. P. (1988). Goodness of fit indexes in confirmatory factor analysis: The effects of sample size.
Psychological Bulletin, 103, 391–410.Pruzansky, S. (1975). How to use SINDSCAL: A computer program for individual differences in multidimensional scaling. Murray Hill NJ:
AT & T Bell Laboratories.
S. Gupta et al. / Journal of Vocational Behavior 72 (2008) 1–13 13
Rounds, J., & Tracey, T. J. (1993). Prediger’s dimensional representation of Holland’s RIASEC circumplex. Journal of Applied
Psychology, 78, 875–890.Rounds, J., & Tracey, T. J. (1996). Cross-cultural structural equivalence of RIASEC models and measures. Journal of Counseling
Psychology, 43, 310–329.Rounds, J., Tracey, T. J., & Hubert, L. (1992). Methods for evaluating vocational interest structural hypotheses. Journal of Vocational
Behavior, 40, 239–259.Ryan, J. M., Tracey, T. J. G., & Rounds, J. (1996). The generalizability of Holland’s structure of vocational interests across ethnicity,
gender, and socioeconomic status. Journal of Counseling Psychology, 43, 330–337.Sverko, I., & Babarovic, T. (2006). The validity of Holland’s theory in Croatia. Journal of Career Assessment, 14, 490–507.Swaney, K. B. (1995). Technical manual: Revised Unisex Edition of the ACT Interest Inventory (UNIACT). Iowa City, IA: American
College Testing.Swanson, J. L. (1992). The structure of vocational interests for African American college students [Special issue]. Journal of Vocational
Behavior, 40, 129–143.Tak, J. (2004). Structure of vocational interests for Korean college students. Journal of Career Assessment, 12, 298–311.Tang, M. (2001). Investigation of the structure of vocational interests of Chinese college students. Journal of Career Assessment, 9,
365–379.Tinsley, H. E. (1992). Career decision making and career indecision [Special issue]. Journal of Vocational Behavior, 41, 209–211.Tracey, T. J. G. (1997). RANDALL: A Microsoft FORTRAN program for the randomization test of hypothesized order relations.
Educational and Psychological Measurement, 57, 164–168.Tracey, T. J. G. (2000). Analysis of circumplex models. In H. E. A. Tinsley & S. D. Brown (Eds.), Handbook of applied multivariate
statistics and mathematical modeling (pp. 641–664). San Diego: Academic Press.Tracey, T. J. G., & Gupta, S. (2007). Interest assessment in an international context. In J. Athanasou & R. van Esbroeck (Eds.),
International handbook of career guidance. London: Springer.Tracey, T. J. G., & Robbins, S. B. (2005). Ethnic and gender differences in interests and skills: A longitudinal examination of grades 8
through 12. Journal of Vocational Behavior, 67, 335–364.Tracey, T. J. G., & Rounds, J. B. (1993). Evaluating Holland’s and Gati’s vocational interest models: A structural meta-analysis.
Psychological Bulletin, 113, 229–246.Tracey, T. J. G., & Rounds, J. (1995). The arbitrary nature of Holland’s RIASEC types: Concentric circles as a structure. Journal of
Counseling Psychology, 42, 431–439.Tracey, T. J. G., Watanabe, N., & Schneider, P. L. (1997). Structural invariance of vocational interests across Japanese and American
cultures. Journal of Counseling Psychology, 44, 346–354.Wiggins, J. S., Steiger, J. H., & Gaelick, L. (1981). Evaluating circumplexity in personality data. Multivariate Behavioral Research, 16,
263–289.Yom, B. L., Doughtie, E. B., Chang, W. C., Alston, H. L., & Wakefield, J. A. Jr., (1975). The factor structure of the Vocational Preference
Inventory for black and white college students. Journal of Vocational Behavior, 6, 15–18.
