Date post: | 03-Jun-2018 |
Category: |
Documents |
Upload: | natalia-platon |
View: | 223 times |
Download: | 0 times |
of 26
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
1/26
SMALLGROUPRESEARCH/October2001Devine,Philips/COGNITIVEABILITYINTEAMS
DO SMARTER TEAMS DO BETTER
A Meta-Analysis of Cognitive Ability andTeam Performance
DENNIS J. DEVINE
Indiana UniversityPurdue University, Indianapolis
JENNIFER L. PHILIPS
University of Akron
This study reports the results of several meta-analyses examining the relationship between
four operational definitions of cognitive ability within teams (highest member score, lowest
member score, mean score, standard deviation of scores) and team performance. The three
indices associated with level yielded moderate and positive sample-weighted estimates of
the populationrelationship(.21 to .29), butsamplingerrorfailed to account forenough vari-
ation to rule outmoderator variables.In contrast, theindexassociated with dispersion (i.e.,
standarddeviationof member scores) wasessentiallyunrelatedto teamperformance (.03),
andsampling error provideda plausible explanation for the observed variation across stud-
ies.A subgroupanalysisrevealed that mean cognitive abilitywas a much better predictorof
team performance in laboratory settings (.37) than in field settings (.14). Study limitations,
practical implications, and future research directions are discussed.
Cognitive ability is the capacity to understand complex ideas,
learn from experience, reason, problem solve, and adapt (Neisser
et al., 1996; Sternberg, 1997). After hundreds of empirical studies,
it is now clear that cognitive ability is one of the best predictors of
individual job performance (Hunter & Hunter, 1984; Schmidt &
Hunter, 1998; Wagner, 1997). Given that many tasks performed by
small groups or teams involve learning, reasoning, and problem
solving, it seemslikelythat thecognitiveability of team members is
related to team performance. However, the issue is not as straight-
507
AUTHORSNOTE: We thankBill Rogers,EricSundstrom,and Jane Williamsfor theirhelp-
ful comments on earlier versions of this article and Marc Fogel forhis assistance in collect-
ing and coding data.
SMALL GROUP RESEARCH, Vol. 32 No. 5, October 2001 507-532
2001 Sage Publications
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
2/26
forward as intuition suggests. It has been known for some time that
relationships at one level of analysis do not necessarily hold at
another (Kozlowski & Klein, 2000; Robinson, 1950; Thorndike,
1939).Klein,Dansereau, andHall (1994)offered a classic example
of this phenomenon in discussing the relationshipbetween popular
votesand outcomes inU.S. presidentialelections. At thestate level,
thecandidatereceiving themost popular voteswins allthe electoral
votes (i.e., winner takes all); however, as highlighted by the2000
U.S. presidential election, the winner at the national level does not
necessarily receive the most popular votes in the country as a
whole. Essentially, the relationship between popular votes and
election outcome differs across levels. Assuming that the state-levelrelationshipholds at thenational level would be akin to making
a cross-level fallacy (Rousseau, 1985).
Ina similar fashion,as far as workgroups are concerned, it is not
appropriate to assume a relationship between cognitive ability and
performance at the team level based on studies conducted at the
individual level. In other words, the strong positive relationship
between cognitive ability and performance at the individual level
does notcause (or imply) a relationshipat the team level. Consider,
for example, a decision-making team composed of members from
several functional areas of an organization. Individually, the mem-
bers may be intelligent and perform their specific roles well. How-
ever, this does not ensure that the team as a whole will do well. Forinstance, differences in perspective associated with the various
functional areas may prevent the effective integration of relevant
information (Hinsz, Tindale, & Vollrath, 1997; Larson & Christensen,
1993). In essence, the existence of a team-level relationship
between cognitive ability and performance is an empirical issue: It
could be positive, negative, nonexistent, or variable from situation
to situation. With the growing use of work groups and teams in
organizations, it would be beneficial for practitioners to identify
valid, low-cost, practical predictors of team performance. Cogni-
tiveability tests mayproveuseful in this regard, but their valuecan-
not be assumed.
508 SMALL GROUP RESEARCH / October 2001
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
3/26
Complicating assessment of the relationship between cognitive
ability and performance in work groups is the variety of ways in
which cognitive ability can be operationally defined at the team
level. In theempirical literature onsmall groups, three indices asso-
ciatedwith thefunctional amount(i.e., level)ofcognitive resources
available to the team have appeared most often: (a) the value of the
teamshighest scoring individual, (b)thevalueof theteamslowest
scoring individual, and (c) themean of team member scores. These
three operational definitions correspond to three task types identi-
fied by Steiner (1972), involving different functional relationships
between individual and group performance: (a) tasks where the
best individual performance determines the groups performance,(b) tasks where the worst individual performance determines the
groups performance, and (c) tasks where all individual perfor-
mances contribute to group performance in a summative fashion.
Steiner labeled the first type of taskdisjunctive, the second type
conjunctive, and the third typeadditive.
In addition to these three operational definitions associated with
the functional level of cognitive ability present in a work group, the
rise of diversity issues in the workplace has called attention to the
potential influence of team-level variation on team dynamics and
effectiveness (Jackson, 1996). This line of work suggests a fourth
way of treating cognitive ability at the team level: in terms of the
dispersion of member scores. Underlying much of the work ondiversity are thenotions that team memberdiversity is a good thing
and that heterogeneous groups should outperform homogeneous
groups(especially ontasksrequiringcreativityor innovation)because
they possess a largerpool of task-relatedresources(K.Y. Williams&
OReilly, 1998). Although there has been little systematic discus-
sion concerning theimpact of team memberdiversitywith regard to
cognitive ability, a number of recent studies have nonetheless
examined the association between the standard deviation of mem-
ber cognitive ability scores and team performance.
From a practical perspective, it would be advantageous to learn
if thefour operational definitions varywith regard to their criterion-
Devine, Philips / COGNITIVE ABILITY IN TEAMS 509
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
4/26
related validity. To the extent they do, this wouldsuggest strategies
for composing work groups. For example, if the best predictor of
team performance across a variety of tasks turned out to be the
score of the most intelligent member, this would suggest ensuring
that allwork groupshaveat least onevery intelligent team member.
Conversely, if mean cognitive ability score was found to be thebest
predictor of team performance, efforts could be made to select as
many intelligent members as possible without excessive concern
over finding a single genius.
Fortunately, althoughresearch oncognitive ability inworkgroups
is a relatively recent phenomenon, there is now enough data to
make a preliminary assessment of theteam-level relationshipusingmeta-analytic methods. In the next section, we report the results of
several meta-analyses concerning the relationship between team-
level cognitive ability and teamperformance using fouroperational
definitions of team-level cognitive ability (i.e., mean member
score, highest member score, lowest member score, and standard
deviation of scores). Given the strength and robustness of the indi-
vidual-level relationship, our primary research hypothesis was as
follows: Team-level indices reflecting the highest, lowest, and
mean cognitive ability scoreswithinwork groupswill each be posi-
tively related to team performance. Furthermore, given the critical
role of member interdependence in most work groups, it seems
likely that the performance of few (if any) work groups would bedetermined solely by the actions of a single member as implied by
Steiners (1972) disjunctive and conjunctive task types. Even in
work groups where one member is clearly most important, it is dif-
ficult to imagine that performance is notaffected to some degree by
other members. Therefore, we expected that the observed relation-
ship between team-level cognitive ability and performance would
be stronger when indexed by the mean member score as opposed to
the highest or lowest score on the team. Given the lack of theoreti-
caldevelopment in theliterature,we didnot haveany formalexpec-
tations with regard to the relationship between the dispersion of
cognitive ability scores and team performance.
510 SMALL GROUP RESEARCH / October 2001
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
5/26
METHOD
LITERATURE SEARCH
Before any meta-analysis can be conducted on antecedents of
team performance, it is necessary to address the issue of whether
work groups and teams will be treated as the same thing. Although
persuasive arguments have been made for treating them separately,
thedistinction is relativelyrecent, andthetwo termshavebeen used
synonymously throughout much of the literature on small groups
(Guzzo, 1996; Ilgen,1999).Furthermore, in any event, preliminary
inspection revealed that few empirical reports described the studycontext well enough to make fine distinctions. As a result, we use
the termsgroup,work group, andteaminterchangeably in refer-
ence to small collectivesof individuals that interact for thepurpose
of accomplishing one or more shared goals while operating with
some degree of interdependence.
Severalconvergent methods were used to searchforstudies with
usable data. First, we conducted a computerized search of the
PSYCHINFO database for the years 1967 to 1999. The following
terms (and relevant combinations) were used in the computerized
database search: team, group, workgroup, cognitive ability, mental
ability, generalability, ability, intelligence,performance, effective-
ness, efficiency,productivity, and outcome. The authors also manu-ally scanned the titles in each issue of the following journals for the
past 10 years:Journal of Applied Psychology,Personnel Psychol-
ogy,Academy of Management Journal,Journal of Management,
Journal of Organizational Behavior, Organizational Behavior and
Human Decision Processes, Small Group Research, Journal of
Personality and Social Psychology, andIntelligence. Abstracts and
method sections were consulted for any article possessing a title
that suggested team-level research. Reference lists for recent theo-
retical andempiricalpapersandbooks onwork groupor team com-
position were also examined (e.g., Barrick, Stewart, Neubert, &
Mount, 1998; Cohen & Bailey, 1997; Guzzo & Dickson, 1996;
Devine, Philips / COGNITIVE ABILITY IN TEAMS 511
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
6/26
Ilgen,1999; Milliken & Martins, 1996).Given thesmall numberof
published studies on this topic, special emphasis was placed on
acquiringunpublished data. In particular, after reviewingabstracts,
we obtained several unpublished theses and dissertations through
interlibrary loan; two of these eventually proved useful (Blades,
1976; OConnell, 1994). We also generated a list of researchers
known to conduct research on groups and teams and contacted
these individuals via e-mail. Data from four studies (i.e., Gully,
1997; Hollenbeck et al., 1999; Sundstrom & Futrell, 1999; Zukin,
1999) were obtained in this fashion.
INCLUSION CRITERIA
A study was included in the meta-analysis if it did each of the
following: (a) measured individual-level cognitive ability and
formed one or more team-level indices using some explicit aggre-
gation process, (b) measured team performance, and (c) empiri-
cally assessed the degree of association between team-level cogni-
tive ability and team performance using a Pearsonsrcorrelation.
The following were considered acceptable measures of cogni-
tive ability: (a) scores from an established measure of cognitive
ability (e.g., Wonderlic Personnel Test, 1992) or (b) general apti-
tude (e.g., American College Test Verbal, Scholastic Assessment
Test Quantitative) or multiaptitude test scores (e.g., compositescores on the General Aptitude Test Battery or Differential Apti-
tude Test). Although scores from an established measure of cogni-
tive ability are obviously preferable, general or composite aptitude
scores tend to be highlycorrelated with measures of cognitive abil-
ity because they measure one or more primary components (Wag-
ner, 1997). Team performance was defined as the degree to which
the team accomplished its goal or mission. We initially hoped to
conduct separate analyses for accuracy and quality criteria and
speed and quantity criteria, but only three studies were found that
reported associations between measures of team-level cognitive
ability andproductivity (i.e., Neuman& Wright, 1999; OConnell,
1994; Sundstrom & Futrell, 1999), so this was not possible. When
multiple criteria were reported (e.g., production countsandsubjec-
512 SMALL GROUP RESEARCH / October 2001
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
7/26
tive ratings), we chose the one that best appeared to capture overall
accomplishment of the teams mission. When subjective perfor-
mance measures were available for both supervisors and team
members, we used supervisor ratings because of (a) their common
use in administrative decision making in organizations, (b) their
availability in each study, and (c) their tendency to exhibit more
variabilitythan team memberratings. Thus, although both supervi-
sors and team members can provide useful information on team
performance related to their unique perspectives, we chose to use
supervisor ratings because of their greater salience, availability,
and variability relative to aggregated team member self-ratings.
All studies appearing to be relevant were read in entirety by oneof the authors,and a final judgmentwas thenmadeas towhether the
study provided usabledata. Themajority of empirical studies iden-
tified in the computerized search were discarded because they did
not analyze (or, in a few cases, report) the relationship between
cognitive ability andperformance at the team level. Because of our
focus on general cognitive ability, we didnot include studies solely
involving team-level indices of task-specific ability. These studies
used individual performance scores on some task as the basis for
constructing a group-level index of task-specific ability, often as a
benchmark for determining process gain or loss associated with
group performance on the same task. We also did not include stud-
ies that used a domain-specific aptitude test score instead of a mea-sure of general cognitive ability (e.g., Gurnee, 1937; Kabanoff &
OBrien, 1979). Finally, we wish to note that we were unable to
obtain usable data from three well-known studies of group compo-
sition (i.e., Spector & Suttell, 1957; Terborg, Castore, & DeNinno,
1976; Tziner & Eden, 1985) because cognitive ability scores were
dichotomized to allow formation of teams with varying degrees of
heterogeneity but no quantitative values at the team level were
calculated and/or reported.
In total,ourreviewof theliteratureyielded 19studies that metall
three inclusion criteria; Table 1 provides descriptive summaries of
each. Several studies reported data from several related experi-
ments or settings (i.e., Blades, 1976; OBrien & Owens, 1969;
Sundstrom & Futrell, 1999), resulting in a total of 25 independent
Devine, Philips / COGNITIVE ABILITY IN TEAMS 513
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
8/26
TABLE 1: Descriptive Information for Studies Included in One or More Meta-Analyses
Study Task Cognitive Ability Measure Performanc
Barrick, Stewart, Neubert, Small appliance and electronic Wonderlic Personnel Test Supervisor rating of
and Mount (1998) assembly; fabrication and maintenance (sum of 8 dimensio
Blades (1976) (1 and 2) Operating an army mess hall Henman-Nelson Mental Average of superviso
Ability Test (two raters)
Bottoms (1998) SouthEast Airlines top management Wonderlic Personnel Test Profit (from algorithm
simulation
Brandt (1998) Energy International top management Wonderlic Personnel Test Time taken to identif
simulation for position
Clayton (1998) SouthEast Airlines top management Wonderlic Personnel Test Profit (from algorithm
simulationColarelli and Boos (1992) Evaluation of a personnel program Composite of global ACT Average score on cou
and GPA (two raters)
Devine (1999) SouthEast Airlines top management Wonderlic Personnel Test Profit (from algorithm
simulation
Fiedler and Meuwese (1963) Operating antiaircraft artillery guns AGCT scores Commander ranking
Gully (1997) TEAM TANDEM military command Wonderlic Personnel Test Quality and quantity
and control simulation classification
Hollenbeck et al. (1999) Dynamic distributed decision-making Wonderlic Personnel Test Computer-generated
military command and control
simulation
LePine, Hollenbeck, Ilgen, TIDE2
military command and control ACT/SAT Accuracy of target ai
and Hedlund (1997) simulation
Neuman and Wright (1999) Human resource problem solving Thurstone Test of Mental Supervisor ratings (f
Alertness
514
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
9/26
OBrien and Owens (1969) (1) Collective letter writing for recruitment AGCT Composite rating of
(based on 5 dimens
OBrien and Owens (1969) (2) Constructing a chart AGCT Number of errors ma
OBrien and Owens (1969) (3) Generating a short fictional story ACT-English Composite ratings of
5 dimensions)
OBrien and Owens (1969) (4) Generating a short fictional story ACT-English Composite ratings of
5 dimensions)
OBrien and Owens (1969) (5) Generating a short fictional story ACT-English Composite ratings of
5 dimensions)
OConnell (1994) Automotive construction, paint and GATB composite Supervisor rating of
assembly; maintenance (based on 6 dimens
Stevens, Jones, Fischer, and Manual/technical jobs SRA verbal subtest Supervisor ratings of
Kane (1999)
Sundstrom and Futrell (1999) Automotive headlamp assembly IRT/DAT composite Number of units asse
Tziner and Vardi (1983) Operating military tanks IDF composite Commander ranking
W. M. Williams and Brainstorming Henmon-Nelson Test of Overall rating of solu
Sternberg (1988) Mental Ability
Zukin (1999) Solving murder mystery Wonderlic Personnel Test Correct answer (Yes
NOTE: ACT= American CollegeTest; GPA = gradepoint average;AGCT= ArmyGeneral Classification Test; SAT = ScholasticAssesGeneral AptitudeTestBattery;SRA = Science Research AssociatesSurveyof Basic Skills;IRT = Industrial ReadingTest; DAT = DiffeIDF = Israeli Defense Force battery based on Otis-Lenon, command of Hebrew, and interview.
515
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
10/26
samples. The majority of studies occurred in a controlled setting
(11 of 19) within the past 10 years (14 of 19).
CODING
Fivequantitativevariableswere codedfrom usable studies: (a)num-
ber of teams involved, (b) correlation between the mean cognitive
ability score within the team and team performance, (c) correlation
between the highest cognitive ability score within the team and
team performance, (d)correlationbetween thelowest cognitive abil-
ity score within the team andteam performance,and(e) correlation
between the standard deviation of member cognitive ability scoresand team performance. We did not code reliability estimates for
cognitive ability or teamperformance for several reasons. First, many
studies did not report reliability estimates for either individual-
level cognitive ability or teamperformance. Second, cognitive abil-
ity measures tend to be very reliable, and many studies used objec-
tive performance indices (i.e., counts or standardized algorithms)
that can be assumed to have near perfect interrater reliability
(Mento, Steel,& Karen,1987).Third, in many cases,correction for
measurement error has a minor impact on parameters estimated via
meta-analysis (Koslowsky & Sagie, 1994).
Table 2 contains effect-size information for each independent
sample for the four operational definitions of team-level cognitiveability examined in this study. Effect sizes ranged from strong and
positive toweakand negative. Samplesizes varied froma low of 16
(OBrien & Owens, 1969) to a high of 514 (Sundstrom & Futrell,
1999), with the average study involving 71 teams. With regard to
operational definitions, 24 samples reported a correlation between
themean cognitive ability score of team members and team perfor-
mance (i.e., all but Neuman & Wright, 1999), 16 samples provided
a correlation between the highest member score and team perfor-
mance, 17 samples provided a correlation between the lowest
member score and team performance, and 9 provided a correlation
between the standard deviation of scores and team performance.
516 SMALL GROUP RESEARCH / October 2001
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
11/26
PROCEDURE
We conducted four main analyses corresponding to the follow-
ing operational definitions of team-level cognitive ability: (a) arith-
metic mean of member scores (M), (b) score of the lowest scoring
member (LOW), (c) score of the highest scoring member (HIGH),
Devine, Philips / COGNITIVE ABILITY IN TEAMS 517
TABLE 2: Coding Information for Studies Included in Meta-Analyses
rxy
Study N M HIGH LOW SD Setting
Barrick, Stewart, 51 .23 .03 .02 .22 Field
Neubert, and Mount (1998)
Blades (1976) (1) 49 .02 . . . Field
Blades (1976) (2) 51 .22 . . . Field
Bottoms (1998) 50 .22 .24 .08 . Lab
Brandt (1998) 54.(52)a
.32 .14 .17 .05 Lab
Clayton (1998) 55 .21 .07 .21 .15 Lab
Colarelli and Boos (1992) 86 .24 . . . Field
Devine (1999) 52 .40 .19 .38 .09 Lab
Fiedler and Meuwese (1963) 24 .19 . . . Field
Gully (1997) 81 .38 . . . Lab
Hollenbeck et al. (1999) 76 .26 . . . Lab
LePine, Hollenbeck, Ilgen, 26 .34 .37 .30 . Lab
and Hedlund (1997)
Neuman & Wright (1999) 79 . . .33 . Field
OBrien & Owens (1969) (1) 20 .58 .48 .56 . Lab
OBrien & Owens (1969) (2) 20 .13 .12 .12 . Lab
OBrien & Owens (1969) (3) 16 .52 .32 .49 . Lab
OBrien & Owens (1969) (4) 16 .03 .15 .04 . Lab
OBrien & Owens (1969) (5) 16 .52 .19 .56 . Lab
OConnell (1994) 118 .13 .07 .16 .16 Field
Stevens (1999) 56 .29 . . .20 Field
Sundstrom and Futrell (1999) (1) 117 .43 .30 .28 .10 Lab
Sundstrom and Futrell (1999) (2) 514 .40 .26 .34 .05 Lab
Tziner and Vardi (1983) 115 .30 . . . Field
Williams and Sternberg (1988) 24 .65 .65 .43 . LabZukin (1999) 62 .26 .26 .15 .09 Lab
NOTE:N= number of teams included in the study.M= arithmetic mean of member scores;HIGH = score of the highest scoring member; LOW = score of the lowest scoring member;SD= standard deviation of member scores.a.N= 54 forMandSD, andN= 52 for the HIGH and LOW.
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
12/26
and (d) standard deviation of member scores (SD). For each main
analysis,a sample-weighted meancorrelation was calculatedusing
the corresponding distribution of effects along with the following
statistics: (a) chi-square test for the homogeneity of observed effect-
sizes (Rosenthal, 1991), (b) percentage of observed variance
accounted for by sampling error (Hunter & Schmidt, 1990), (c)95%
credibility intervalformed around the estimatedpopulation param-
eter using the corrected standard deviation (Hunter, Schmidt, &
Jackson, 1982),and(d)95%confidence interval formedaroundthe
estimated population parameter based on the standard deviation of
the sampling distribution (Whitener, 1990).
The first three statistics were used to evaluate the likelihood thatthe cognitive abilityperformance relationship is moderated at the
team level. The existence of one or more moderators is suggested
when (a) the chi-square test for coefficient homogeneity is statisti-
callysignificant,(b) thepercentageof observedvarianceaccounted
for by sampling error is less than 75%, and (c) the credibility inter-
val is large and/or includes zero. Conversely, the 95% confi-
dence interval assesses the amount of sampling error influencing
the sample-weighted mean correlation. When all of the observed
variation in a distribution of coefficients is accounted for by sam-
pling error, the credibility interval will be zero, and the confidence
interval will reflect the sampling error affecting the estimate of the
one true population relationship. When sampling error accountsfor a relatively small portion of the observed variation in a set of
coefficients, the credibility interval will be nonzero, and the confi-
dence intervalwill provide a measure of thesamplingerror inherent
in theestimate of themean relationship across multiple subgroups.
HunterandSchmidt (1990)noted that support fora particular mod-
erator exists when there is (a) a meaningful difference in the esti-
mated effect size across moderator subgroups and (b) reduced
within-subgroupvariation relative to theoverall distribution (i.e., a
high percentage of variation accounted for by sampling error, a
nonsignificant chi-square, anda small credibility interval that does
not include zero).
518 SMALL GROUP RESEARCH / October 2001
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
13/26
RESULTS
Results of the four main analyses are presented in Table 3. As is
evident from inspection of the table, theMoperational definition
yielded thestrongestestimatedrelationshipwithteamperformance
(r= .294) followed by the LOW (r= .246) and HIGH (r= .208)
indices, whereas theSDindex produced a very weak and negative
estimate (r= .026). Furthermore, 95% confidence intervals cre-
ated around the sample-weighted means for all three operational
definitions involving leveldidnot include zero, suggesting observed
criterion-related validity associated with these indices will gener-
ally be positive. However, the confidence intervals for all threeoperational definitions did overlap, so our expectation that the
mean score wouldproduce a stronger observed relationshipreceived
only limited support (i.e., from the point estimates). Overall, these
data are consistent with thenotion that the three operational defini-
tions involving level have positive relationships with team perfor-
mance,but none is clearly superior. Incontrast, the95%confidence
interval for theSDindex included zero, consistent with the notion
that there may well be no relationship between the dispersion of
member cognitive ability scores and team effectiveness.
Table 3 also contains evidence that other variables moderate the
strength of these team-level relationships. Specifically, the chi-
square tests for homogeneity in the observed coefficients associ-ated with theM, HIGH,andLOWanalyses were statistically signif-
icant, indicating that variability in the distribution of coefficients
was significantly greater than what would be expected by chance.
In addition, sampling error accounted for less than 59% of the
observed variance in sample-weighted effect sizes for these three
analyses, in all cases below the 75% value suggested by Hunter
et al. (1982) as sufficient to conclude that one true effect character-
izes the relationship in the population as a whole. Finally, 95%
credibility intervals created around each sample-weighted mean
correlationwere relatively large forall three estimatesandincluded
zero for the LOW index. In light of this consistency, there is good
reason to suspect that the magnitude of the team-level relationship
Devine, Philips / COGNITIVE ABILITY IN TEAMS 519
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
14/26
TABLE 3: Summaries for Main and Moderator Meta-Analyses
95%
K N rxy 2
%2,SE Credibility Interval C
Operational definition
HIGH 16 1,209 .208 27.35* 59.16 .027, .389
M 24 1,749 .294* 55.87*** 41.36 .042, .545
SD 9 1,079 .026 11.45 77.33 .124, .071
LOW 17 1,288 .246 39.54*** 40.73 .010, .505 Study-setting moderator
Lab 16 1,199 .368* 15.65 103.36
Field 8 550 .137 17.63* 44.31 .123, .339
NOTE:K = numberof samples inanalysis; N = numberof teamsin analysis;rxy= meansample-weighted correlation;2
= chi-squarevaof coefficients;%
2, SE= percentage of observed variance accounted forby sampling error;HIGH = score of thehighest scoring mem
mean of member scores;SD= standard deviation of member scores; LOW = score of the lowest scoring member.*p< .05. **p< .01. ***p< .001.
520
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
15/26
between cognitive ability and performance varies across situations
forall threeoperational definitions involvinglevel.Considering the
weak estimated relationship, the narrow confidence interval that
includes zero, and the relatively large amount of observed variance
accounted for by sampling error, it appears from these data that
work group diversity in terms of cognitive ability is essentially
unrelated to team performance in most situations.
Given these data, we proceeded to look for potential moderators
of the team-level relationship. We initially hoped to code the stud-
ies in our database for several task characteristics, but this proved
impossible because (a) most studies provided only a cursory
description of the task and (b) study characteristics that could bereliably coded from available information tended to be highly cor-
related, making it impossible to isolate their effects. In particular,
studies conducted in fieldsettings tendedto involve standing teams
engaged in familiar behavioral tasks with long time frames; con-
versely, labstudies tendedto usenovel intellectual tasks,short time
frames, and ad hoc groups of students who had little familiarity
with one another. As a result, we opted to code only one surface
characteristic: study setting.Lab studies were defined as those that
took place under controlled conditions wherein the focal task was
created solely for the purpose of scientific investigationandhad no
intrinsic importance or meaningfulness. Fieldstudies were defined
as any study that did not meet the definition of a lab study. Further-more, given theextremely smallnumberof studies conductedin the
field that reported correlation for the HIGH and LOW indices, we
onlyexaminedthestudy-settingmoderatorusing thedistribution of
coefficients associated with theMindex.
As seen inTable 3, thestudy-setting moderatorprovides a useful
starting point for understanding the nature of the team-level rela-
tionshipbetween cognitive ability and performance. With regard to
the magnitude of the team-level relationship, the two subgroups
exhibited a large difference: The estimated relationship was con-
siderablystronger in labstudies (r= .37) than actual organizational
settings (r= .14). Furthermore, all the observed variation in study
coefficients associated with lab studies was explained by sampling
error, and the chi-square test for heterogeneity was nonsignificant
Devine, Philips / COGNITIVE ABILITY IN TEAMS 521
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
16/26
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
17/26
sentativeness of the studies included in these meta-analyses. Thus,
these results should be viewed with caution, and parameter esti-
mates should be treated as preliminary.
IMPLICATIONS
The positive relationshipobserved for all three operational defi-
nitions associated with cognitive ability level within a team sug-
gests that the functional amount of cognitive ability in teams does
indeed predict team performance across a broad variety of team
contexts. All other things being equal, the cognitive ability of the
most intelligent member accounts for roughly 4.3% of thevariancein team performance, the cognitive ability of the least intelligent
member explains about 6.1%, and the mean cognitive ability of
team members captures approximately 8.6% of the variance in
team performance (i.e., twice as much as thecognitive ability of the
most intelligent member). Furthermore, in situations where it is
desirable to predict how well a team will perform, it appears more
valuable to know the mean level of cognitive ability of members
than the score of the highest or lowest scoring individual.
Perhaps the most important finding of this study, however, was
thestrong evidenceof moderation affecting the team-level relation-
ship forall three operational definitions associated with level. With
regard to composing work groups, the main analyses indicate therelationship between mean cognitive ability and team performance
varies across situations. Furthermore, the moderator analysis find-
ings suggest that in organizational settings, the predictive efficacy
of team-level cognitive ability will be weak in some team contexts
and nonexistent in others. Overall, there is little evidence here to
suggest that cognitive ability tests representa panacea forpractitio-
ners charged with assembling effective work groups.
This of course begs the question of which characteristics are
responsible for the variation in the team-level relationships. Few
empirical studies in the literature have examinedpotential modera-
tors, but theory and empirical research suggest a potential role for
the following: (a) task complexity, (b) degree of physical activity,
and(c) task familiarity. Taskcomplexityisa functionof thenumber
Devine, Philips / COGNITIVE ABILITY IN TEAMS 523
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
18/26
of information cues and behavioral acts required of a task, as well
as the coordination and adaptation required of members (Wood,
1986).Giventhat task complexity moderates therelationshipbetween
cognitive ability andperformance for individuals (Hunter & Hunter,
1984), team-level indices of cognitive ability maybe more strongly
related to team performance on complex tasks than simple tasks.
With regard to physical activity, intellectual tasks (e.g., planning,
decision making, problem solving) differ from behavioral tasks
(production, assembly, maintenance) in that the latter involve sub-
stantial movement and coordination of team members as well as
their tools or equipment (McGrath, 1984). Although information-
processing demands arecertainlypresent in both types of tasks, theteam-level cognitiveability shouldbe morestronglyrelated to team
performance in intellectual tasks than behavioral tasks due to the
strong influence of psychomotor and physical abilities in the latter.
Finally, regardingtaskfamiliarity, research byKanferandAckerman
(1989) suggests the relationshipbetween cognitive ability and task
performance decreases over time for individuals as they acquire
more experience with a task. Essentially, they arguedthat cognitive
ability is important in the early stages of learning a new task but
becomes progressivelylessimportantas knowledge is acquiredand
skills becomeproceduralized. Extending this argument to the team
level, the correlation between team-level cognitive abilityand team
performance shouldbe highest for novel tasks and shoulddecreaseover time (i.e., repetitions or cycles) as team members become
more familiar with their roles and the roles of other members.
Overall, to the extent several task moderators are operating,
there will likely be situations where team-level cognitive ability
indices are strongly correlated with team performance and other
situations where the two are unrelated or perhaps even negatively
related. Combining the empirical evidence of moderation and the
theoretical discussion above, team-level cognitive ability might be
expected to yield its lowest predictive validity for performance
when team tasks aresimple, familiar, andbehavioral.Applying this
logic to organizational settings, team-levelcognitive abilitymaybe
only marginally useful for predicting the effectiveness of standing
work teams engaged in repetitive, standardizedactivities (e.g., pro-
524 SMALL GROUP RESEARCH / October 2001
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
19/26
duction or assembly teams or maintenance crews). On the other
hand, given the strong relationship found with intellectual tasks in
the lab, team-level cognitive ability may be a better predictor of
performance forad hocteams facinga relatively complex task with
a finite life span (e.g., selection committees, research and develop-
ment teams, or quality circles). Furthermore, there may be higher
order interactions among the task characteristics such that the
effect of one task characteristic depends on the level of another. For
instance, theeffect of task familiarity could be more pronounced in
complex tasks as opposed to simple tasks.
Finally, the results of the main analysis for the standard devia-
tion indexsuggest that thedispersion ofcognitive ability in teamsisnot an important concern for researchers or practitioners. It should
be noted that the samplesize for the SD analysis wasthesmallestof
thefour main analyses,but these studies didshowreasonable heter-
ogeneity with regard to study setting, and there was relatively little
variation in the observed coefficients across studies. Combined
with the lack of a compelling theoretical rationale for why the dis-
persion of cognitive ability shouldbe important, it is unlikely that a
substantial relationship exists between cognitive ability dispersion
in teams and team performance in general. At the same time,
depending on the nature of the task, it is conceivable that variation
in member cognitive ability (or lack thereof) might be related to
team performance in some situations.
FUTURE RESEARCH DIRECTIONS
Given thepreviousdiscussion,a good firststep wouldbe to iden-
tify the team contexts where the cognitive ability of team members
is most strongly related to team performance. The hypotheses
implied above could be tested in controlled settings by having
teams engage in tasks that vary along key task dimensions, allow-
ing examination of potential interactions among task characteris-
tics and changes over time. Alternatively, given the complex con-
texts and multiple task responsibilities associated with real work
groups, it might be possible to agree on a taxonomy of work group
types found in organizational settings (e.g., Sundstrom, 1999) and
Devine, Philips / COGNITIVE ABILITY IN TEAMS 525
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
20/26
then examine the different types in field settings. This would sacri-
fice precision related to identifying operative characteristics in
favor of greater realism and enhanced generalizability. Takentogether,
the two strategies could further ourunderstanding of themicro task
characteristics that affect the team-level relationship as well as the
macro team contexts in which team-level cognitive ability is most
important.
In addition to identifying theconditions when it is most andleast
important to maximize the cognitive ability of work groups, it
would also be helpful to identify particular roles where cognitive
ability is most important once major team types have been distin-
guished. Borrowing Steiners (1972) emphasis on the criterion, itmight be useful to usecognitive ability measures topredict individ-
ualrole performancesandthen focus onbuildingcompilationmod-
els relating individual role performances to team performance in
specific contexts (Kozlowski & Klein, 2000). In particular, mem-
ber role performances could be represented by main effects and
dyadic and triadic performance-related exchanges represented by
higher order interaction terms (i.e., AB, ABC). Given the strong
link between individual-level cognitive ability and job (i.e., role)
performance, specific roles or role exchanges could be identified
where cognitive ability is most important in particular team types
with fairly standard roles (e.g., sports teams or top management
groups). For instance, if a strong effect were obtained for the ABinteraction term, this would suggest that the quality of the
task-related exchange between Members A and B was strongly
related to overall team performance. Given that cognitive ability is
a good predictorof individual performance,a manager or executive
could then try to ensure that two of the more intelligent members
occupied these roles on the team.
In addition to this focus on settings and specific member roles,
practitioners could also benefit from more research on how
team-level cognitive ability relates to team dynamics. In particular,
it would also be useful to have a better understanding of how the
cognitive ability of members affects team outcomes via process
variables such as task-related communication, behavioral coordi-
nation, interpersonal conflict, encouragement, and performance
526 SMALL GROUP RESEARCH / October 2001
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
21/26
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
22/26
CONCLUSION
Team-level cognitive ability appears to be positively related to
team performance for three operational definitions that focus on
amount, but themagnitudeof all three relationships appears to vary
as a function of one or more contextual variables. Although we
have highlighted several moderator candidates pertaining to thetask, there is currentlynotenoughdata available to identify the fac-
tors responsible for the observed variation in these relationships.
Most important, the findings from this study suggest that team-
level cognitive ability indices may not be good predictors of team
performance in some organizational settings. More research is
neededto identify the team contexts where cognitive ability is most
and least useful as a predictor of team performance.
REFERENCES
Barrick, M. R., Stewart, G. L., Neubert, M. J., & Mount, M. K. (1998). Relating member
ability and personality to work-team processes and team effectiveness. Journal of
Applied Psychology,83(3), 377-391.
528 SMALL GROUP RESEARCH / October 2001
Individual
Cognitive Ability
Team-Level
Cognitive Ability
LOW
HIGH
MEAN
SD
Team Outcomes
Effectiveness
Viability
Task Characteristics
Complexity
Physical Activity
Familiarity
Team Process
Information
Sharing
Information
Integration
Conflict
Coordination
Individual
Role Performance
Figure 1: A Conceptual Model of the Cognitive AbilityTeam Performance
Relationship
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
23/26
Blades, J. W. (1976).The influenceof intelligence,task abilityand motivationon group per-
formance (Unpublished doctoral dissertation, University of Washington).Dissertation
Abstracts International,37, 1463.
Bottoms, N. (1998).Group decision-making in a simulated business environment. Unpub-
lished bachelors honors thesis, Indiana UniversityPurdue University, Indianapolis.
Brandt, K. M. (1998).The effects of personality compatibility on team processes and team
performance. Unpublished masters thesis, Indiana UniversityPurdue University,
Indianapolis.
Clayton, L. D. (1998).The effects of gender composition and task complexity on group pro-
cesses and performance. Unpublished masters thesis, Indiana UniversityPurdue Uni-
versity, Indianapolis.
Cohen,S. G.,& Bailey, D. E. (1997). What makes teams work:Groupeffectivenessresearch
from the shop floor to the executive suite.Journal of Management,23, 239-290.
Colarelli, S. M., & Boos, A. L. (1992). Sociometric and ability-based assignment to work
groups: Someimplications for personnelselection.Journal of OrganizationalBehavior,
13, 187-196.
Devine, D. J. (1999). Effects of cognitive ability, task knowledge, information sharing, and
conflict on group decision-making effectiveness.Small Group Research,30, 608-634.
Fiedler, F. E., & Meuwese, W.A.T. (1963). Leaders contribution to task performance in
cohesiveand uncohesivegroups.Journalof AbnormalandSocial Psychology, 67, 83-87.
Gully, S. M. (1997).The influences of self-regulatory processes on learning and perfor-
mance in a team training context. Unpublished doctoral dissertation, Michigan State
University, East Lansing.
Gurnee, H. (1937). Maze learning in the collective situation. Journal of Psychology,3,
437-443.
Guzzo, R. A. (1996). Fundamental considerations about work groups. In M. A. West (Ed.),
Handbook of work group psychology(pp. 3-21). San Francisco: Jossey-Bass.
Guzzo,R. A.,& Dickson, M. W. (1996). Teams in organizations:Recent research on perfor-
mance and effectiveness.Annual Review Psychology,47, 307-338.
Hinsz, V. B., Tindale, R. S., & Vollrath, D. A. (1997). The emerging conceptualization of
groups as information processors.Psychological Bulletin,121, 43-64.Hollenbeck, J. R., Ilgen, D. R., Sheppard, L., Ellis, A., Moon, H., & West, B. (1999, May).
Person-team fit: A structural approach. Paper presented at the 14th Annual Meeting of
the Society for Industrial Organizational Psychology, Atlanta, GA.
Hunter,J. E.,& Hunter,R. F. (1984).Validity andutilityof alternatepredictors of jobperfor-
mance.Psychological Bulletin,96, 72-98.
Hunter, J.E., & Schmidt,F.L. (1990).Method of meta-analysis:Correctingerrorandbias in
research findings. Newbury Park, CA: Sage.
Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982).Meta-analysis: Cumulating research
findings across studies. Beverly Hills, CA: Sage.
Ilgen, D. R. (1999). Teams embedded in organizations: Some implications.American Psy-
chologist,54, 129-139.
Jackson, S. E. (1996). The consequences of diversity in multidisciplinary work teams. In
M. A. West (Ed.),Handbook of work group psychology(pp. 53-76). Chichester, UK:
Wiley.
Kabanoff, B., & OBrien, G. E. (1979). Cooperation structure and the relationship of leaderandmember abilityto group performance.Journalof Applied Psychology, 64, 526-532.
Devine, Philips / COGNITIVE ABILITY IN TEAMS 529
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
24/26
Kanfer, R., & Ackerman, P. L. (1989). Motivation and cognitive abilities: An integrative/
aptitude-treatment interaction approach to skill acquisition.Journal of Applied Psychol-
ogy,74, 657-690.
Klein,K. J.,Dansereau, R.G., & Hall, R.J. (1994). Level issuesin theorydevelopment,data
collection, and analysis.Academy of Management Review,19, 195-229.
Koslowsky, M., & Sagie, A. (1994). Components of artifactual variance in meta-analytic
research.Personnel Psychology,47, 561-574.
Kozlowski, S.W.J., & Klein, K. J. (2000). A multilevel approach to theory and research in
organizations: Contextual, temporal, and emergent processes. In S.W.J. Kozlowski &
K. J. Klein (Eds.),Multilevel theory, research, and methods in organizations: Founda-
tions, extensions, and new directions(pp. 3-90). San Francisco: Jossey-Bass.
Larson, J. R., & Christensen, C. (1993). Groups as problem-solving units: Toward a new
meaning of social cognition.British Journal of Social Psychology,32, 5-30.
LePine, J. A.,Hollenbeck, J. R.,Ilgen,D. R.,& Hedlund, J. (1997). Effects of individualdif-
ferences on the performance of hierarchical decision-making teams: Much more thang.
Journal of Applied Psychology,82(5), 803-811.
McGrath, J. E. (1984). Groups: Interaction and performance. Englewood Cliffs, NJ:
Prentice Hall.
Mento,A. J.,Steel, R. P., & Karren, R. J. (1987). A meta-analytic study of theeffectsof goal
setting on task performance.Organizational Behavior and Human Decision Processes,
39, 52-83.
Milliken, F. J., & Martins, L. L. (1996). Searching for common threads: Understanding the
multiple effects of diversity in organizational groups.Academy of Management Review,
21, 402-433.
Neisser,U., Boodoo,G., Bouchard, T. J., Jr.,Boykin,A. W.,Brody, N.,Ceci, S.J.,Halpern,D. F.,
Lohelin, J. C., Perloff, R., Sternberg, R. J., & Urbina, S. (1996). Intelligence: Knowns
and unknowns.American Psychologist,51, 77-101.
Neuman, G.A.,& Wright,J. (1999). Team effectiveness:Beyondskillsand cognitive ability.
Journal of Applied Psychology,84, 376-389.
OBrien, G. E., & Owens, A. G. (1969). Effects of organizational structure on correlations
between member abilities and group productivity.Journal of Applied Psychology,53,525-530.
OConnell, M. S. (1994). The impact of team leadership on self-directed work team perfor-
mance: A field study (Unpublished doctoral dissertation, University of Akron).Disser-
tation Abstracts International,55, 1699.
Robinson, W. S. (1950). Ecological correlations and the behavior of individuals.American
Sociological Review,15, 351-357.
Rosenthal, R. (1991).Meta-analytic procedures for social research. Newbury Park, CA:
Sage.
Rousseau, D. M. (1985). Issues of levelin organizational research. In L. L. Cummings &
B. Staw (Eds.),Research in organizational behavior(Vol. 7, pp. 1-37). Greenwich, CT:
JAI.
Schmidt,F.L.,& Hunter,J. E. (1998).The validityand utilityof selectionmethodsin person-
nel psychology: Practical and theoretical implications of 85 years of research findings.
Psychological Bulletin,124, 262-274.
Spector, P., & Suttell, B. J. (1957).An experimental comparison of the effectiveness of threepatterns of leadership behavior(Unpublished technical report, AIR-196-57-FR-164).
Washington, DC: American Institute for Research.
530 SMALL GROUP RESEARCH / October 2001
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
25/26
Steiner, I. D. (1972).Group process and productivity. New York: Academic Press.
Sternberg, R. J. (1997). The concept of intelligence and its role in lifelong learning and suc-
cess.American Psychologist,52, 1030-1037.
Stevens, M. J., Jones, R. G., Fischer, D. L., & Kane, T. D. (1999).Team performance and
individual effectiveness: Personality and team context. Paper presented at the 14th
Annual Meeting of the Society for Industrial and Organizational Psychology, Atlanta,
GA.
Sundstrom, E. (1999). The challenges of supporting work team effectiveness. In E. Sundstrom
(Ed.),Supporting work team effectiveness: Best management practices for fostering
high performance(pp. 3-23). San Francisco: Jossey-Bass.
Sundstrom, E.,& Futrell, D. A. (1999). Work group productivity andgeneral mental ability:
Two field studies of group composition. Unpublished manuscript.
Terborg, J. R.,Castore, C.,& DeNinno, J. A. (1976).A longitudinalfield investigationof the
impact of group composition on group performance and cohesion.Journal of Personal-
ity and Social Psychology,34(5), 782-790.
Thorndike, E. L. (1939). On the fallacy of imputing the correlations found for groups to the
individuals or smaller groups composing them.American Journal of Psychology,52,
122-124.
Tziner, A., & Eden, D. (1985). Effects of crew composition on crew performance: Does the
whole equal the sum of its parts?Journal of Applied Psychology,70(1), 85-93.
Tziner, A., & Vardi, Y. (1983). Ability as a moderator between cohesiveness and task crew
performance.Journal of Occupational Behavior,4, 137-143.
Wagner, R. K. (1997). Intelligence, training, and employment.American Psychologist,52,
1059-1069.
Weingart, L. R. (1997). How did they do that? The ways and means of studying group pro-
cess.Research in Organizational Behavior,19, 189-239.
Whitener, E. M. (1990). Confusion of confidence intervals and credibility intervals in
meta-analysis.Journal of Applied Psychology,75, 315-321.
Williams, K. Y., & OReilly, C. A., III. (1998). Demography and diversity in organizations:
A review of 40 years of research.Research in Organizational Behavior,20, 77-140.
Williams, W. M.,& Sternberg, R. J. (1988). Groupintelligence: Whysomegroups arebetterthan others.Intelligence,12, 351-377.
Wonderlic Personnel Test. (1992).Wonderlic Personnel Test & scholastic level exam users
manual. Milwaukee, WI: Author.
Wood, R. E. (1986). Task complexity: Definition of the construct. Organizational Behavior
and Human Decision Processes,37, 60-82.
Zukin, L. (1999).Individual differences and individual-in-team performance: An investiga-
tion of the impact of personality and ability in a decision making team with distributed
information. Unpublisheddoctoral dissertation,George MasonUniversity,Fairfax, VA.
Dennis J. Devine is an assistant professor of industrial and organizational psychol-
ogyat Indiana UniversityPurdue University, Indianapolis.He receivedhis Ph.D.in
industrial and organizational psychology from Michigan State University in 1996.
His research interests include team dynamics and effectiveness, jury decision mak-ing, expert-novice differences in job performance, and organizational recruitment.
Devine, Philips / COGNITIVE ABILITY IN TEAMS 531
8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance
26/26
Jennifer L. Philips is a 1st-year graduatestudent in the industrial and organizational
psychologyPh.D. program at the University of Akron.Her research interests include
the effects of team member composition on team effectiveness, group decision mak-
ing, legal influences on human resources decision making, organization culture or
climate, and organizational citizenship behaviors.
532 SMALL GROUP RESEARCH / October 2001