Implications of 20 Years of CHC Cognitive-Achievement Research: Back-to-the-Future and Beyond CHC

Implications of 20 Years of CHC Cognitive-Achievement Research: Back-to-the-Future and Beyond CHC

Kevin S. McGrew PhD

Woodcock-Muñoz Foundation

http://www.woodcock-munoz-foundation.org/index.html

Staying current with “IQ McGrew”

(@iqmobile)

ICDP Blog

http://www.woodcock-munoz-foundation.org/index.html

http://www.iqscorner.com/

http://www.iapsych.com/IAPWEB/iapweb.html

Dr. Woodcock’s legacy & impact on my career and this paper

Introduction and Context

My WJ data sandbox

Beyond CHCBack-to-the-future

The Journey (2002now)

General Intelligence (g)

Mathematical knowledge (KM)

Mathematical achievement

(A3)

Reading decoding (RD)

Reading comprehension

(RC)

Reading speed (RS)

Spelling ability (SG)

English usage (EU)

Writing ability (WA)

Writing speed (WS)

General verbal information (K0)

Language development

(LD)

Lexical knowledge (VL)

Listening ability (LS)

Communication ability (CM)

Grammatical sensitivity (MY)

Induction (I)

General sequential

reasoning (RG)

Quantitative reasoning (RQ)

Memory span (MS)

Working memory capacity (MW)

Associative memory (MA) *

Meaningful memory (MM) *

Free-recall memory (M6) *

Ideational fluency (FI) **

Associational fluency (FA) **

Expressional fluency (FE) **

Sens. to probs. /altern. Sol.

fluency (SP) **

Originality /creativity (FO)

**

Naming facility (NA)**

Figural Fluency (FF) **

Figural flexibility (FX) **

Visualization (Vz)

Speeded rotation (SR)

Closure speed (CS)

Flexibility of closure (CF)

Visual memory (MV)

Spatial scanning (SS)

Serial perceptual integration (PI)

Length estimation (LE)

Perceptual illusions (IL)

Perceptual alternations (PN)

Imagery (IM)

Phonetic coding (PC)

Speech sound discrimination

(US)

Resistance to auditory stimulus

distortion (UR)

Memory for sound patterns

(UM)

Maintaining & judging rhythm

(U8)

Musical discrim. & judgment (U1

U9)

Absolute pitch (UP)

Sound localization (UL)

Quantitative Knowledge

(Gq)

Reading & Writing (Grw)

Comp -Knowledge

(Gc)

Fluid Reasoning (Gf)

Short-Term Memory (Gsm)

Long-Term Storage &

Retrieval (Glr)

Visual Processing (Gv)

Auditory Processing (Ga)

Processing Speed (Gs)

Perceptual speed (P)

Rate of test-taking (R9)

Number facility (N)

Reading speed/fluency

(RS)

Writing speed/fluency

(WS)

Word Fluency (FW) **

Acquired Knowledge + Memory * Learning Efficiency** Retrieval Fluency

GeneralSpeed +

Sensory-Motor DomainSpecific Abilities (Sensory) +

Domain-Independent General Capacities +

Functional groupings

Conceptual groupings

+ = additional CHC abilities in groupings in Part 2 of model

General

Broad

Narrow

Figure 1. CHC v2.0 model based on Schneider and McGrew (2012)


? Simple reaction time (R1)

Choice reaction time (R2)

Semantic processing speed

(R4)

Speed of limb movement (R3)

Writing speed (fluency) WS

Olfactory memory (OM) ? ?

Domain Specific Know.

(Gkn)

Reaction & Decision Speed

(Gt)

Psychomotor Speed (Gps)

Olfactory Abilities (Go)

Tactile Abilities(Gh)

Kinesthetic Abilities (Gk)

Psychomotor Abilities (Gp)

Static strength (P3)

Multilimb coordination (P6)

Finger dexterity (P2)

Manual dexterity (P1)

Arm-hand steadiness (P7)

Acquired Knowledge +

Control precision (P8)

Aiming (A1)

Gross body equilibrium (P4)

Speed of articulation (PT)

Movement time (MT)

Mental comparison speed (R7)

Inspection time (IT)

General Speed +

Sensory-Motor Domain Specific Abilities +

MotorFunctional groupings

Conceptual groupings

+ = additional CHC abilities in groupings in Part I of model

General

Broad

Narrow

Figure 1 (continued). CHC v2.0 model based on Schneider and McGrew (2012)

CHC COGACH Relations: What We Know Today

•Almost all available CHC-designed COGACH research is limited to the WJ Battery •The primary action in CHC COGACH relations is at the narrow ability level

• There is a future for “intelligent” intelligence testing, even in the current response-to-intervention (RTI) environment

Established narrow CHCrdg./math ach. relations abridged summary

Induction (I)

General sequential

reasoning (RG)


Visualization (Vz)


Visual memory (MV)

Number facility (N)

Rdg AchievementMath Achievement


Comp -Knowledge

(Gc)



Long-Term Storage &

Retrieval (Glr)






(LD)


Working memory capacity (MW

Associative memory (MA)

Naming facility (NA)


Ach. Domain General Cognitive Abilities


Memory span (MS)

Meaningful memory (MM)



(US)


distortion (UR)

Rdg. Domain Specific Cognitive Abilities

Math. Domain Specific Cognitive Abilities

[Developmental (age-based) differences are not captured by this abridged summary. See McGrew & Wendling (2010) for this information]

Clarification of Ability Construct Terminology

Ability

“as used to describe an attribute of individuals, ability refers to the possible variations over individuals in the liminal levels of task difficulty (or in derived measurements based on such liminal levels) at which, on any given occasion in which all conditions appear favorable, individuals perform successfully on a defined class of tasks” (p. 8, italics in original).

“every ability is defined in terms of some kind of performance, or potential for performance (p. 4).”

Cognitive Abilities

Abilities on tasks “in which correct or appropriate processing of mental information is critical to successful performance” (p. 10; italics in original).

Achievement abilities

“refers to the degree of learning in some procedure intended to produce learning, such as an informal or informal course of instruction, or a period of self study of a topic, or practice of a skill” (p. 17). As noted by Carroll (1993)

Aptitude (Defined in this paper—narrow sense, not

broader Richard Snow definition)

Aptitude is defined as the combination, amalgam or

complex of specific cognitive abilities, that when combined,

best predict a specific achievement domain

What is “aptitude”

MathApt

RdgApt

Conceptual distinction between Abilities: Cognitive abilities, achievement abilities, and aptitudes


Long-Term Storage &

Retrieval (Glr)






Comp -Knowledge

(Gc)

Reading & Writing (Grw)

Quantitative Knowledge

(Gq)

Abilities

Cognitive AbilitiesAchievement Abilities

Etc. Etc.Etc. Etc. Etc. Etc.Etc.Etc.Etc.

Ach. domain-general apt.

Ach. domain-specific apt.

Vertical columns represent abilities, factors or latent traits (primarily factor-analysis derived internal structural validity constructs)

Horizontal arrow rows represent aptitudes (primarily multiple regression derived external [predictive] validity constructs)

CHC Theory

CHC-based batteries

CHC Cog-Ach

Research Synthesis

Intelligent “RFSA”

Selective Referral-Focused Assessment (RFSA)

Kaufman’s “Intelligent” Intelligence testing

Established narrow CHCrdg./math ach. relations abridged summary



(LD)


Working memory capacity (MW

Associative memory (MA)

Naming facility (NA)


Memory span (MS)

Meaningful memory (MM)



(US)


distortion (UR)


Comp -Knowledge

(Gc)



Long-Term Storage &

Retrieval (Glr)





Induction (I)

General sequential

reasoning (RG)


Visualization (Vz)


Visual memory (MV)

Number facility (N)

Rdg AchievementMath Achievement

Ach. Domain General Cognitive Abilities

Rdg. Domain Specific Cognitive Abilities

Math. Domain Specific Cognitive Abilities

[Developmental (age-based) differences are not captured by this abridged summary. See McGrew & Wendling (2010) for this information]

Two illustrative CHC general selective referral-focused assessment (SRFA) scenarios: BRS problems for ages 6 to 8 yrs

WJ SAPTs

WJ-R SAPTs

WJ III Pred. Ach. GIA option

Developmentally sensitive CHC-designed SAPTs

The evolution of differential Scholastic Aptitude Clusters (SAPTs)

Developmentally-Sensitive CHC-Consistent Scholastic Aptitude Clusters

(McGrew, 1986, 1994)

CHC COG>ACH

res. synthesis

Select WJ III tests based on first step

for initial predictor

pool

Run MR models

across entire school-age WJ III norm

sample

Backward deletion of

tests from MR model. Inspect

each step results noting “bridesmaid”

predictors

Run final MR model at each

age and smooth regression

coefficients by age

ITD - Developmentally-Sensitive CHC-Consistent Scholastic Aptitude Clusters

0 5 10 15 20

TECHAGE

-0.10

0.05

0.20

0.35

0.50

Val

ue

Age group (in years)

Stan

dard

ized

regr

essi

on c

oeffi

cien

t

Verbal Comp. (Gc-LD/VL)

Vis-Aud Learning (Glr-MA)


Smoothed standardized regression coefficients of best set of WJ III cognitive test predictors of WJ III Math Reasoning (MR) cluster from ages 6 thru 18. Table is % of MR variance accounted for by GIA-Std and MR Aptitude as constructed

and weighted per the figure.

0 5 10 15 20


-0.1

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Reg

ress

ion

coef

ficie

nt Verbal Comp. (Gc-LD/VL)

Analysis-Synthesis (Gf-RG)

Number Matrices (Gf-RQ)

Visual Matching (Gs-P)


Analysis-Synthesis (Gf-RG)


Number Matrices (Gf-RQ)Numbers Reversed(Gsm-WM)

Numbers Reversed(Gsm-WM)


Stan

dard

ized

regr

essi

on c

oeffi

cien

t

Age 5 6 7 8 9 10 11 12 13 14 15 16 17 18GIA-Std. 32 39 44 46 53 56 50 60 64 56 53 65 53 47MR-Apt. 46 42 47 53 56 61 62 63 71 72 64 77 64 66Difference 6 3 3 7 3 5 12 3 13 12 11 12 11 19

0 5 10 15 20

TECHAGE

-0.1

0.0

0.1

0.2

0.3

0.4

0.5V

alue


Stan

dard

ized

regr

essi

on c

oeffi

cien

t

Verbal Comp. (Gf-LD/VL)


Sound Awareness (Ga-PC/Gsm-WM)

Sound Awareness (Ga-PC/Gsm-WM)

Numbers Reversed (Gsm-Wm)

Numbers Reversed (Gsm-Wm)

Sound Blending (Ga-PC)

Sound Blending (Ga-PC)





Smoothed standardized regression coefficients of best set of WJ III cognitive test predictors of WJ III Basic Reading Skills (BRS) cluster from ages 6 thru 18. Table is percent of BRS variance accounted for by GIA-Std and BRS Aptitude as

constructed and weighted per the figure.

Age 5 6 7 8 9 10 11 12 13 14 15 16 17 18GIA-Std. 33 40 42 39 41 50 43 35 43 48 48 48 59 45BRS-Apt. 50 49 50 48 45 56 48 43 50 54 52 52 63 52Difference 17 9 8 9 4 6 5 6 7 6 4 4 4 7

ITD: SAPTs require a mixture of domain-general and domain-specific CHC cognitive abilities


ITD: “Intelligent” Test Design Principles

• Test developers should utilize the extant CHC COGACH relations literature when selecting the initial pool of tests to include in the prediction models

ITD: SRFA requires 3-way thinking. 3-way interaction of CHC abilities X achievement domains X age (developmental status).

ITD: SAPTs are better predictors of achievement than g-based composites


ITD: “Intelligent” Test Design Principles

ITD: Developmental trends are critically important in aptitude-achievement comparisons

• Test developers should provide age-based developmental weighting of the tests in the different CHC-consistent SAPTs

•Those who implement an aptitude-achievement consistency/concordance SLD model must be cautious and not use a "one size fits all" approach when determining which CHC COG abilities should be examined for the aptitude portion of the consistency model


Group vs individual centered focus (McGrew & Flanagan, 1998)

• Group-based statistical results may not translate perfectly to all individuals

• “Intelligent” testing is required

• “We are the instrument”

CHC-Consistent Scholastic Aptitude Clusters SRFA Strategy

WJ III example in basic reading skills (BRS) and math reasoning (MR)

Optimal developmentally weighted linear combination of WJ III tests


Comp -Knowledge

(Gc)



Long-Term Storage &

Retrieval (Glr)




Analysis-Synthesis

(RG)

Number Matrices(RQ)

Visual Matching(P)

Numbers Reversed

(MW)

Verbal Comprehension

(LD)WJ III Math Reason. Aptitude

Examine PSW within aptitude clusters (and as suggested by other tests administered and other non-test information) to determine additional selective

follow-up assessment in narrow ability domains

Verbal Comprehension

(LD)

Numbers Reversed

(MW)

Vis.-Aud. Learning

(MA)

Snd Blending(PC) Visual Matching

(P)WJ III Basic Rdg. Skills Aptitude

Snd Awareness(PC;Gsm-WM)

CHC COGACH relations research & SRFA provides opportunity to engage in “intelligent” testing (ala,

A. Kaufman)

“ Tests do not think for themselves, nor do they directly communicate with patients. Like a stethoscope, a blood pressure gauge, or an MRI scan, a psychological test is a dumb tool, and the worth of the tool cannot be separated from the sophistication of the clinician who draws inferences from it and then communicates with patients and professionals”

Meyer et al. (2001). Psychological testing and psychological assessment. American Psychologist,

Beyond CHC

Beyond CHC # 2: WJ III Productive Exploratory Rabbit Hole (circa 2009-2010) Experience

Data Sets

•WJ III norm data•WJ III+ other batteries

(WISC-R; WAIS-III/WMS-III/KAIT)•WAIS-IV subtest correlations

Methods•Cluster analysis•Multidimensional scaling analysis (MDS) – 2D and 3D•Standard and Carroll EFA+CFA exploratory factor analysis•Model-generation CFA (SEM)•CHC cognitive causal SEM models

Beyond CHC: Linear minds living in a non-linear world

“A fundamental limitation of any theory built on a rectilinear system of factors it that it is not of a form that well describes natural phenomena. It is thus unlikely to be fully adequate. It is a system that can accurately describe rectangular structures built by humans…but not the rounded and irregular structures of mother nature. The phenomena of nature are not usually well described by the linear equations of a Catesian coordinate system….The equations that describe the out structure and convolutions of brains must be parabolas, cycloids, cissoids, spirals, foliums, exponentials, hyperboles, and the like (p. 84). (Horn & Noll, 1997)

Gf-Gc/ CHC

theory

WJ-R WJ III

CHC COC-ACH reg studies

g+ specific abilities

COG>ACHSEM res.

+IP/CPM models

CHC COG<>ACH

SEM res.(person

fit?)

Beyond CHC #1: CHC + Information Processing Causal SEM Models


(Not

e: R

esid

uals

and

sig

nifi

cant

cor

rela

tion

s be

twee

n re

sidu

als

are

omit

ted

from

the

diag

ram

for

rea

dabi

lity

pur

pose

s

TeCog. Test 1

TeCog. Test 2

TeCog. Test 4

TeCog. Test 3

TeCog. Test 5

TeCog. Test 6

TeCog. Test 7

TeCog. Test 8

TeCog. Test nth

Independent Variables (IV)) – Cog.

Dependent Variable (DV) – Ach.

CogLV2

CogLV nth

g

CogLV1

CogLV3

TeAch. Test 2

TeAch. Test 1AchLV1

TeAch. Test 4

TeAch. Test 3AchLV2

TeAch. Test 6

TeAch. Test 5AchLV3


Pic. Recognition

Sound Blending

Gf

Gv

Glr

Ga

Oral Comp.

WordAttack

Gc

Gen. Info.

Inc. Words

Sound Patterns

Vis.-Aud. Lrg.

Block Rotation

Spat. Relations

DR Vis-Aud.Lrg.

Ret. Fluency

Word Attack

Verbal Comp..44

.35

.40

.48

.69

.78

.64

.49

.45

.96

g

.85

.94

.87

.84

.93

.78

.89

Mem.for Names .52

.79

.36

Anal.-Synth.

Conc. Form.

Numerical Reas.

.63

.74

.63

Ages 6-8

Visual Matching

Decision Speed Gs Mem. Span(MS)

Cross Out

.82

.64

.73

.44Mem.for Sent.

Mem.for Words

.78

.69

Wrk. Mem.(WM)

Num. Reversed

Aud. Wrk. Mem..62

.67

.62.46

Cognitive efficiency

Plausible CHC/IP COGWord Attack causal model in WJ III norm data (ages 6-8)

Indirect effect

Direct effect

EffectsDirect Indirect Total

Gs 0.19 0.40 0.59MS 0.00 0.34 0.34WM 0.00 0.54 0.54g 0.36 0.23 0.59Ga 0.27 0.00 0.29

Chi-square =1016.5. df=239 GFI=.93; AGFI=.91; PGFI=.74RMSEA=.055 (.051-.058)

.93.19

.27

“while we acknowledge the principle of parsimony and endorse it whenever applicable, the evidence points to relative complexity rather than simplicity. Insistence on parsimony at all costs can lead to bad science” (p. 16).

Stankov, Boyle and Cattell (1995) who stated, within the context of research on human intelligence“

Pic. Recognition

Visual Matching

Decision Speed

Sound Blending

Gf

Gv

Gs

Glr

Ga

Mem. Span(MS)

Oral Comp.

WordAttack

Gc

Gen. Info.

Inc. Words

Sound Patterns

Vis.-Aud. Lrg.

Block Rotation

Spat. Relations

DR Vis-Aud.Lrg.

Ret. Fluency

Word Attack

Verbal Comp.

Cross Out

.44

.35

.40

.82

.64

.73

.48

.69

.78

.64

.49

.45

.96

g

.85

.94

.87

.84

.93

.44

.78

.89

Mem.for Names .52

.79.36

Anal.-Synth.

Conc. Form.

Numerical Reas.

.63

.74

.63

Mem.for Sent.

Mem.for Words

.78

.69

Wrk. Mem.(WM)

Num. Reversed

Aud. Wrk. Mem..62

.67.62

.93

.46

.27

.19

Ages 6-8

Cognitive efficiencyIndirect effect

Direct effect

Chi-square =1016.5. df=239 GFI=.93; AGFI=.91; PGFI=.74RMSEA=.055 (.051-.058)

Beyond CHC #1: Develop SEM “person fit” indices ?

A challenge to the LISRELites, AMOSites, MPLUSites in the room

Build it an they shall come.


Example:


Example:

WJ/WJ-R SAPTs

Third method SLD

models (apt-ach

consistency)

Psych trait complex theory & research

Beyond Jöreskog

syndrome

Cog-Apt-Ach Trait

Complexes (CAATC)New SLD

model ideas

Beyond CHC #2: Cognitive-Aptitude-Achievement Trait Complexes (CAATC’s)


AcademicDomain

Cognitive-Aptitude-Achievement Trait Complex

Degree of cohesion

Aptitude

for Acd. Domain

CognitiveAbilities

Beyond CHC: Jöreskog syndrome

American psychology, and mainstream quantitative school psychology, have expressed little interest in non-confirmatory statistical methodological lens (e.g., exploratory cluster analysis; MDS) in favor of what I call Jöreskog syndrome—an almost blind allegiance and belief in structural equation modeling confirmatory factor analysis (SEM-CFA) methods as the only way to see the “true light” of the structure of intelligence and intelligence tests

The law of the instrument

“Give a small boy a hammer, and he will find that everything he encounters needs pounding”

Beyond CHC: Jöreskog syndrome

Important Reminder: All statistical methods, suchas factor analysis (EFA or CFA) have limitations and constraints.

It only provides evidence of structural/internal validity and typically nothing about external, developmental, heritability, neurocognitive validity evidence

Need to examine other sources of evidence and use other methods – looking/thinking outside the factor analysis box

WJ/WJ-R SAPTs

Third method SLD

models (apt-ach

consistency)

Psych trait complex theory & research

Beyond Jöreskog

syndrome

Cog-Apt-Ach Trait

Complexes (CAATC)New SLD

model ideas


-2 -1 0 1 2

-2

-1

0

1

2

GLR (MA)

GSM (MS)

GS (P)

GA (PC)

GC (LD/VL)

GF (I/RG)

BCAEXT

MAPTBRDG

GRWAPT

BWLANG

BMATH

GV (MV/CS)

WJ-R CHC factor clusters

WJ-R broad achievement lcusters

WJ-R Broad Cognitive Ability &Scholastic Aptitude Clusters

A B = Visual-figural/numeric/quantitative Auditory-linguistic/language dimensionC D = Cognitive operations/processes Acquired knowledge /product dimension

C

D

BA

Note: Measures closer to the center are more cognitively complex. The distance between points represents the inter-relations between variables. Highly-related variables are spatially closer-have less distance between their circles.

Notes on WJ-R Derived Scholastic Aptitude Clusters (SAPTs)

GRWAPT = Gc(LD/VL) + Gs(P) + Ga(PC) + Glr(VAL) or Gsm-MS

(RAPT and WLAPT nearly overlapped in figure. Given their high degree of overlap, they were combined into a single GRWAPT in the figure)

MAPT = Gc(LD/VL) + Gs(P) + Gf(I) + Gf(RG)

-WJ-R SAPTs each comprised of 4 tests with equal weightings (.25)

-Bold font designates shared test CHC ability content in GRWAPT and MAPT

Figure 9. Guttman radex MDS analysis summary of WJ-R cognitive, aptitude, and achievement measures across all ages in WJ-R norm sample

-2 -1 0 1 2

-2

-1

0

1

2

GLR (MA)

GSM (MS)

GS (P)

GA (PC)

GC (LD/VL)

GF (I/RG)

BCAEXT

MAPTBRDG

GRWAPT

BWLANG

BMATH

GV (MV/CS)

Notes on WJ-R Derived Scholastic Aptitude Clusters (SAPTs)

GRWAPT = Gc(LD/VL) + Gs(P) + Ga(PC) + Glr(VAL) or Gsm-MS

(RAPT and WLAPT nearly overlapped in figure. Given their high degree of overlap, they were combined into a single GRWAPT in the figure)

MAPT = Gc(LD/VL) + Gs(P) + Gf(I) + Gf(RG)

-WJ-R SAPTs each comprised of 4 tests with equal weightings (.25)

-Bold font designates shared test CHC ability content in GRWAPT and MAPT

WJ-R CHC factor clusters

WJ-R broad achievement lcusters

WJ-R Broad Cognitive Ability &Scholastic Aptitude Clusters

C

D

BA

AB = Visual-figural/numeric/quantitative Auditory-linguistic/language dimensionCD = Cognitive operations/processes Acquired knowledge /product dimension

Figure 10. WJ III based reading and math cognitive-aptitude-achievement trait complexes (CAATC)

GF (I/RG)MAPT

BMATH

GV (MV/CS)

GA (PC)

GC (LD/VL)

BRDG

GRWAPT

BWLANG

Angle = approximately 57o

r = approximately .55

Math (Gq) cognitive-aptitude-achievement trait complex

Reading/Writing (Grw) cognitive-aptitude-

achievement trait complex

r =.55

Cognitive-aptitude-achievement trait complexes

Cognitive-aptitude-achievement trait complex (CAATC)

A constellation or combination of related cognitive, aptitude, and achievement traits that, when combined together in a functional fashion, facilitate or impede the acquisition of academic learning

Cognitive-aptitude-achievement trait complexes

CAATCs emphasize the constellation or combination of elements that are related and are combined together in a functional fashion Imply a form of a centrally inward directed force that pulls elements together much like magnetism

Cohesion appears the most appropriate term for this form of multiple element bonding. Cohesion is defined, as per the Shorter English Oxford Dictionary (Brown, 2002), as “the action or condition of sticking together or cohering; a tendency to remain united” (Brown, 2002, p. 444).

Element bonding and stickiness are also conveyed in the APA Dictionary of Psychology (VandenBos, 2007) definition of cohesion as “the unity or solidarity of a group, as indicated by the strength of the bonds that link group members to the group as a whole” (p. 192).

Cohesion defined

Cognitive Strength

Academic weakness

Cognitive weakness

Discrepant/Discordant


Consistent/Concordant

Common Components of Third-MethodApproaches to SLD Identification

(adapted from Flanagan & Alfonso, 2011)Suggested re-conceptualization of academic and cognitive weaknesses

(and possible SLD identification model) based on cognitive-aptitude-achievement trait complexes (CAATC)

Dashed shapes designate academic domain related cognitive abilities.

Cognitive / Academic Strengths


AcademicDomain


Degree of cohesion

Aptitude

for Acd. Domain

CognitiveAbilities

Beyond CHC: Comparison of current PSW and CAATC SLD models

Math (Gq) cognitive-aptitude-achievement trait complex

Reading/Writing (Grw) cognitive-aptitude-

achievement trait complex

-2 -1 0 1

-2

-1

0

1

2

GLR (MA)

GSM (MS)

GS (P)

GA (PC)

GC (LD/VL)

GF (I/RG)

BCAEXT

MAPTBRDG

GRWAPT

BWLANG

BMATH

GV (MV/CS)

Angle = approximately 57o

r = approximately .55

C

D

BA

r =.55

AcademicDomain


Degree of cohesion

Aptitude

for Acd. Domain

CognitiveAbilities

The identification of CAATC taxon’s that better approximate “nature carved at the joints” (Meehl, 1973, as quoted and explained by Greenspan, 2006, in the context of MR/ID diagnosis).

Such a development would be consistent with Reynolds and Lakin’s (1987) plea, 25 years ago, for disability identification methods that better represent dispositional taxon’s rather than classes or categories based on specific cutting scores which are grounded in “administrative conveniences with boundaries created out of political and economic considerations” (p. 342).

Beyond CHC: Potential benefit of CAATC based SLD models

Beyond CHC: Proposed CAATC based SLD model (early ideas)

Dashed shapes designate academic domain related cognitive abilities.

Cognitive / Academic Strengths


AcademicDomain


Degree of cohesion

Aptitude

for Acd. Domain

CognitiveAbilities

• Evaluating the degree of cohesion within a CAATC is integral and critical first step

• The stronger the within-CAATC cohesion, the more confidence one could place in the identification of a CAATC as possibly indicative of a SLD

• If the CAATC demonstrates very weak cohesion, the hypothesis of a possible SLD should receive less consideration

• PSW-based SLD identification would be based first on the identification of a weakness in a cohesive specific CAATC which is then determined to be significantly discrepant from relative strengths in other cognitive and achievement domains

Beyond CHC: Proposed CAATC based SLD model (early ideas)

Quantifying degree of cohesion is likely possible via use of Euclidean Geometry metrics

For example, Mahalanobis distance measure which can

quantify the cohesion between CAATC measures as well as

distance from the centroid of a CAATC exist (see Schneider, 2012)

First CHC IQ

batteries focused on

broadstratum

CHC COG>ACH

rels. “Narrow is better”

MDS and “cognitive

complexity” findings

Optimizing cognitive

complexity of CHC

measures

Beyond CHC #3: Optimizing Cognitive Complexity of CHC measures


CHC factor breadth Cognitive complexity

-2 -1 0 1 2

-2

-1

0

1

2

GCGLR

GV

GV3

GA

GF

GF3

GS

GSM

PHNAWR

PHNAW3

WRKMEM

ASMEM

VISUAL

SNDISC

AUDMS

PERSPD

MTHBR

MTHCAL

MTHREA

RDGBR

RDGBS

GIA-EXTRDGCMP

NUMREA

GIA-EXT and three-test broad clusters

Two-test broad clusters

Two-test narrow clusters

MDS radex model based cognitive complexity analysis of primary WJ III clusters

-2 -1 0 1 2

-2

-1

0

1

2

GCGLR

GV

GV3

GA

GF

GF3

GS

GSM

PHNAWR

PHNAW3

WRKMEM

ASMEM

VISUAL

SNDISC

AUDMS

PERSPD

MTHBR

MTHCAL

MTHREA

RDGBR

RDGBS

GIA-EXTRDGCMP

NUMREA





-2 -1 0 1 2

-2

-1

0

1

2

GCGLR

GV

GV3

GA

GF

GF3

GS

GSM

PHNAWR

PHNAW3

WRKMEM

ASMEM

VISUAL

SNDISC

AUDMS

PERSPD

MTHBR

MTHCAL

MTHREA

RDGBR

RDGBS

GIA-EXTRDGCMP

NUMREA






According to Lohman (2011), those tests closer to the center of an MDS radex model are more cognitively complex, and this is due to five possible factor:

• Larger number of cognitive component processes

• Accumulation of speed component differences

• More important component processes (e.g., inference)

• Increased demands of attentional control and working memory

• More demands on adaptive functions (assembly, control, and monitoring).

Beyond CHC #3: Optimizing Cognitive Complexity—Implications for Test Battery Design and Assessment Strategies

•The push to feature broad CHC clusters in contemporary IQ batteries (or in XBA assessments) fails to recognize the importance of cognitive complexity

•Developing factorially complex measures is one way to achieve cognitive complexity (e.g., KABC-II, DAS-II, Wechslers)

•ITD: It is proposed that within-CHC domain cognitive complexity should be an important ITD


As per Brunswick Symmetry and BIS Model: Need to pay more attention to matching the predictor-criteria space on the dimension of cognitive complexity (e.g., levels of aggregation)

Beyond CHC #3: Cognitive Complexity and CHC COGACH relations

McGrew & Wendling’s (2010) “narrow is better” may need revision to…

“Within CHC-domain cognitively complexity is better”


Possible implication for use of the WJ III Battery:ITD: Broad+narrow hybrid example to optimize ach. prediction

“Front end” featured clusters

• Fluid Reasoning (Gf)• Comprehension-Knowledge (Gc)• Long-term Retrieval (Glr)• Working Memory (Gsm-MW)• Phonemic Awareness 3 (Ga-PC)• Perceptual Speed (Gs-P)• Visualization (not clear winner)

Then, if broad Gsm, Ga, Gs, Gv measures are desired..supplemental testing as per administration of

• Gs (Decision Speed)• Gsm (Memory for Words)• Gv (Picture Recognition)


ITD: IQ test batteries of the future might better be based on a hybrid (broad+narrow) partially inverted CHC model that

deliberately incorporates within-CHC domain cognitive complexity into the test/cluster design process and battery configuration or

suggested testing sequence

Concluding CommentsProximal Implications

“Intelligent” selective-referral focused assessments (SRFA)

• Types of Strategies

• General SRFA• Scholastic Aptitude Cluster-based SRFA

• Important considerations

• Recognize domain-general and domain-specific CHC COG-ACH relations• Recognize 3-way COC x ACH x Age interaction• Recognize importance of cognitive complexity in SRFA

• Narrow may not necessarily be better as a general rule• Use broad+narrow inverted CHC hybrid approach to assessment

• Cautious use of CHC COG-ACH relations findings with non-WJ III batteries

Concluding Comments

Proximal Implications

Develop Developmentally-Sensitive CHC-based Scholastic Aptitude Clusters (ITD)

• The research knowledge and statistical and computer software technology exists

• e.g., WJ III GIA; WJ III Predicted Achievement

Investigate and validate more “dynamic/interacting” CHC COGACH SEM models

Use more “Intelligent Test Design” (ITD) principles when revising old test batteries or developing new test batteries

Concluding Comments

More Distal ImplicationsDevelop SEM “person fit” statistics for possible diagnostic and instructional purposes

Pursue research into the validity and utility of identifying cognitive-aptitude-achievement trait complexes (CAATCs)

• Identify and validate CAATCs

• Develop metrics for operationalizing CAATCs

• Ability domain cohesion metrics

• Investigate validity and utility of CAATC based SLD models for understanding learning and identifying learning problems

Concluding Comments

More Distal ImplicationsUse more “Intelligent Test Design” (ITD) principles when revising old test batteries or developing new test batteries

Incorporate suggested “Intelligent Test Design” (ITD) principles into current “best practice” test development principles when developing new test batteries

• Broad+narrow inverted CHC hybrid approach (ITD)

Concluding Comments

Enduring ImplicationsIntelligence researchers and test developers need to embrace a wider diversity of validated theories, models, and data analytic methodological lenses to counter Jöreskog syndrome.

”If I have seen farther, it is by standing on the

shoulders of giants”

As stated by Isaac Newton in a letter to Robert

Hooke in 1676:

Concluding Comments

Enduring Implications

Exploratory research methods need to be used more frequently by intelligence researchers

Many a scientific adventurer sails the uncharted seas and sets his course for a certain objective only to find unknown land and unsuspected ports in strange parts. To reach such harbors, he must ship and sail, do and dare; he must quest and question. These chance discoveries are called “accidental” but there is nothing fortuitous about them, for laggards drift by a haven that may be a heaven. They pass by ports of opportunity. Only the determined sailor, who is not afraid to seek, to work, to try, who is inquisitive and alert to find, will come back to his home port with discovery in his cargo (p. 177)

Date post:	03-Dec-2014
Category:	Documents
Upload:	kevin-mcgrew
View:	1,699 times
Download:	3 times

Implications of 20 Years of CHC Cognitive-Achievement Research: Back-to-the-Future and Beyond CHC

Documents