The Role of Nonverbal Ability Tests in Identifying Academically Gifted Students: An Aptitude Perspective

David Lohman

The University of Iowa

http://faculty.education.uiowa.edu/dlohman

Overview• Background• Nonverbal tests

– Advantages– Disadvantages

• Understanding abilities • Aptitude perspective

– For minority students

• Recommendations

Background

Why use nonverbal tests?

1. Measure abilities in ways that are fair to all students

2. Increase the diversity in programs for academically gifted and talented

3. Actively assist children who have not had the advantages of wealth or who have not from birth been immersed in English

Other factors to consider

• Get the right kids, not just the right number

• Especially critical for minority students– Next generation of writers, scientists,

mathematicians

• Crafting policy for the identification and development of a diversity of academic talents

Nonverbal Tests• Present visual stimuli (objects, line

drawings) and • Require a nonverbal response (assemble a

puzzle, point, fill in a circle)• “Nonverbal” describes the test, not the

cognitive processes used to solve items• Involvement of verbal processes

– Explicit (UNIT Analogic reasoning subtest)– Implicit (Figural Reasoning tests)

UNIT Analogy

Advantages

• Reduced oral/written language load

• Verbal knowledge, verbally mediated strategies can be in any language

• Reduced mean differences between monolingual and bilingual students

Disadvantages

• Pictorial tests1. Deciphering line drawings

2. Shorter directions are not necessarily better directions

3. Unforeseen linguistic confusions

UNIT Analogy

:

J K L M N

CogAT Figure Analogy

Disadvantages

• Figural reasoning tests– Task specificity greater than for V or Q

Task Specificity 1

Test 1

Task Specificity

Test 1 Test 2+ =

Task Specificity

Figural Verbal

Disadvantages

• Figural reasoning tests– Task specificity greater than for V or Q– Large practice effects– Largest Flynn effect

Example of Flynn Effect

70

75

80

85

90

95

100

105

1910 1920 1930 1940 1950 1960 1970 1980 1990 2000

Year

IQ S

core

Gains in Wechsler-Binet IQ for the U.S. White population. Sources J. Horgan (1995) and D. Schildlovsky.

Example of Flynn Effect

Disadvantages

• Figural reasoning tests– Task specificity greater than for V or Q– Large practice effects– Largest Flynn effect– Appearance of measuring something innate

Fluid-Crystallized Continuum (1)

Fluid Crystallized

Cognitiveabilities

Physicalskills

General fluidability (Gf)

Scienceachievement

Mathachievement

Social studiesachievement

Knowledge ofliterature

Specificfactual

knowledge

Generalphysicalfitness

Basketball

Swimming

Football

Field hockey

Volleyball

WrestlingCycling

Fluid-Crystallized Continuum (2)

Fluid Crystallized

Cognitiveabilities

Physicalskills

General fluidability (Gf)

Scienceachievement

Mathachievement

Social studiesachievement

Knowledge ofliterature

Specificfactual

knowledge

Generalphysicalfitness

Basketball

Swimming

Football

Field hockey

Volleyball

WrestlingCycling

Disadvantages

• Figural reasoning tests– Task specificity greater than for V or Q– Large practice effects– Largest Flynn effect– Appearance of measuring something innate – Appearance of being culture fair

Culture fair?

• Intuitively plausible but long discredited idea

• Anastasi & Urbina (1997) Psychological Testing (7th ed.)– “no test can be equally fair to all cultures”– “nonlanguage tests may be more culturally

loaded than language tests”

Cronbach quote

• Cronbach (1990) Essentials of Psychological Testing (5th ed).

– “no behavioral evidence is culture free.”– “the term ‘culture fair’ makes a dubious claim”

Scarr quote

• Scarr (1994) In Sternberg’s Encyclopedia of Intelligence

– “intelligence and ability tests sample human cultural knowledge, acquired (through) development.”

– “Although tests such as the Raven Matrices may seem fair… puzzle-like tasks turn out to have their own limitations.”

:

J K L M N

CogAT Figure Analogy

Disadvantages

• Figural reasoning tests– Task specificity greater than for V or Q– Large practice effects– Largest Flynn effect– Appearance of measuring something innate – Appearance of being culture fair– Distal predictors of academic success

Example r = .6 using PR

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 100

Nonverbal Reasoning

Mat

hem

atic

s A

chie

vem

ent

Example of r = .6M

ath

em

ati

cs A

ch.

Example r = .6 using PR

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 100

Nonverbal Reasoning

Mat

hem

atic

s A

chie

vem

ent

Example r = .6 M

ath

em

ati

cs A

ch.

Example r = .6 using PR

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 100

Nonverbal Reasoning

Mat

hem

atic

s A

chie

vem

ent

Example r = .6M

ath

em

ati

cs A

ch.

Example r = .6 using PR

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 100

Nonverbal Reasoning

Mat

hem

atic

s A

chie

vem

ent

29%71%

Example r = .6M

ath

em

ati

cs A

ch.

Construct Representation

Verbal

Quantitative

Nonverbalg

What predicts academic achievement?

VerbalReasoning

Quantitative Reasoning

NonverbalReasoning

AcademicAchievement

Selecting students on the basis of a test of nonverbal reasoning ability would:

• admit many students who are unprepared for advanced instruction in mathematics or science or other content-rich domains.

• exclude many students who either have already demonstrated high levels of accomplishment in these domains OR whose high verbal or quantitative reasoning abilities make them much more likely to succeed in such programs.

Figural reasoning as an inaptitude?

1 40 50 60 99

VerbalQuantitative

Nonverbal

1 40 50 60 99

VerbalQuantitative

Nonverbal

N +

N -

Figural reasoning as an inaptitude?

• Students with an N+ profile do less well in school than students with an N- profile

• Gohm, Humphreys & Yao (1998) find high spatial students do poorly on a wide range of academic outcomes.

• High N, high Q = engineer profile

• Cannot look at Nonverbal test alone

Extravagant Claims, Unlikely Promises

• NNAT claims– culture fair – a “very good” predictor of school achievement– small and inconsequential difference in mean

scores for White, Black, & Hispanic students– identify equal proportions of high-scoring

White, Black, & Hispanic students

Predict achievement?

• r (NNAT, Reading) = .56

• r (NNAT, Reading in Spanish) = .32

• r (NNAT, Math) = .6

• r (CogAT, Reading) = .80

• r (CogAT, Math) = .81

Small mean differences between ethnic groups? equal proportions of high scorers?

• Exceedingly implausible– e.g. NAEP differences 1 SD– Matrix format much studied

• Sample is small and unrepresentative– 5.6 % Urban school children– More high SES Hispanics & Blacks

• Numbers do not add up– W-B and W-H differences inconsistent– means < 100 for all three groups– SD’s all greater than 15

Black

Urbanicity NNAT U.S.

Urban 11.0 54.9 -43.9

Suburban 56.1 31.7 24.4

Rural 32.8 13.3 19.5

Note. Urbanicity is defined as in the U.S. Census.

Demographics: Urbanicity

NNAT

SES White Black Hispanic

Low 19.2 20.8 42.0

Low middle 20.1 26.2 29.3

Middle 20.4 8.4 3.0

High middle 23.7 19.5 6.2

High 16.6 25.2 19.5

Note. SES was a composite of median family income in the community and the percent of adults with high school diplomas.

Demographics: SES

Mean W-B, W-H differences

W-B W-H

• Naglieri & Ronning (2000) 4.2 2.8

• Naglieri & Ford (2003) 3.2 2.0

Aptitude Perspective

• Aptitude is – the degree of readiness to perform well in a

particular situation or fixed domain.

• Examples– Ability to comprehend instructions– To use previously acquired knowledge and skill

appropriately– To make good inferences and generalizations– To manage one’s emotions

Academic accomplishment 1

LearningContext

1

LearningContext

2

Person characteristics

AcademicAccomplishment/

Expertise

On-grade and above-grade

achievement tests

Performance Assessments

Teacher grades/

evaluations

Academic accomplishment 2

CogATVerbal

CogATQuantitative

MathematicsAchievement

CogATNonverbal

Predicting Math Achievement in Grades 1-12 from CogAT 6

CogATVerbal

CogATQuantitative

MathematicsAchievement

CogATNonverbal

Multiple R = .80

.23

.50

.15

Predicting Math Achievement in Grades 1-12 from CogAT 6

CogATVerbal

CogATQuantitative

ReadingComprehension/

Vocabulary

CogATNonverbal

Predicting Reading Comprehension/ Vocabulary in Grades 1-12 from

CogAT 6

CogATVerbal

CogATQuantitative

ReadingComprehension/

Vocabulary

CogATNonverbal

Multiple R = .80

.63

.15

.07

Predicting Reading Comprehension/ Vocabulary in Grades 1-12 from

CogAT 6

CogATVerbal

CogATQuantitative

ReadingComprehension/

Vocabulary

CogATNonverbal

Multiple R = .81 (.80)

.66 (.72)

.14 (.12)

.06 (.04)

Predicting Reading Comp/ Vocab for All Students (Hispanics) grades 1-6

Predictors of Achievement

• The regression equations that best predict achievement in Reading, Mathematics, Social Studies, & Science from CogAT Verbal, Quantitative, and Nonverbal reasoning are the same for White, Black, Hispanic, and Asian-American students

• Other investigators find the same (Keith)

Predicting Future Achievement

Achievement

Grade 4 Grade 9

Predicting Future Achievement

Reading Achievement

ReadingAchievement

VerbalReasoning

Grade 4 Grade 9

.35

.30

Predicting Future Achievement

Math Achievement

Math Achievement

Quant.Reasoning

Grade 4 Grade 9

.27

.19

NonverbalReasoning

.19

Recap

1. Structure of abilities the same within ethnic groups

2. Predictors of concurrent achievement are the same in White, Black, Hispanic, & Asian-Americans

3. Best predictors of future achievement in a domain are current achievement in that domain and the ability to reason in the symbol system(s) used to communicate new knowledge in the domain

4. Therefore….

Bilingual students

Verbal achievement in both L1 and L2 depend on a common set of verbal processes

– Phonemic awareness in Spanish predicts reading in English (Lindsey et al. 2003)

– Grades in English are more strongly related to (Swedish) verbal abilities than are grades in Swedish (Gustafsson & Balke, 1993).

– Best predictors of learning French are verbal abilities and achievements in English -- not mathematical or figural reasoning abilities (Carroll, 1981)

Aptitude versus Achievement

• Estimates of academic aptitude must always be judged relative to circumstances.

• Estimates of academic attainment, on the other hand, must be made on a scale that is similar for all.

Common Cut Scores?

• Current Accomplishment

Common standards more reasonable

• Potential for future accomplishment

Common standards not defensible

Lead to the search for aptitude tests that predict achievement but not group differences in achievement

Get more kids, but more of the wrong kids

Distinguishing Present Accomplishment from Predicted Accomplishment

99

95

90

Accelerate

?

Accomplishment

One year later

99

95

90

Same

Improve

Decline

Who is most likely to improve?

• Strongest reasoning abilities in the symbol systems used to communicate knowledge in the domain

• Best, most appropriately challenging instruction

• Motivation and persistence

Guidelines1. Except for very young children, academic

giftedness should be defined primarily by measures of academic accomplishment.

2. The primary cognitive aptitudes for future academic accomplishment are domain-specific achievement and the ability to reason in the symbol systems used to communicate new knowledge.

3. The predictors of achievement are the same in different ethnic groups.

4. Use the nonverbal test as a helpful adjunct, but as a measure of last resort.

5. Provide different levels of challenge to those who have already exhibited academic excellence and those who are working to attain it

6. Use common aptitude measures but uncommon cut scores (e.g., rank within group) when identifying minority students most likely to profit from intensive instruction.

The Role of Nonverbal Ability Tests in Identifying Academically Gifted Students: An Aptitude Perspective

David Lohman

The University of Iowa

http://faculty.education.uiowa.edu/dlohman