Post on 16-Feb-2018
transcript
The New WJ IV Battery: Introduction and Overview
Dr. Kevin McGrew
Institute for Applied Psychometrics (IAP)
WJ IV Presentation Menu: Main
Introduction and Overview
New Tests (COG/OL)
Cognitive Battery (GIA & CHC clusters)
Oral Language Battery
Achievement Battery
Variations and Comparisons
General Technical Information
Tech info.
Scholastic Aptitude clusters New Ga
Tech info.
Tech info.
WJ IV Presentation Menu: Sub
Design principle: Cognitive complexity
Ga: No longer the Rodney Dangerfield of CHC abilities
Tech: Norm sample
Tech: Norms and score construction
Beyond CHC: CHC + Info. Proc. + AC
Hot off the press: Analyses in process
Contemporary CHC broad and narrow ability content coverage by WJ-IV Cognitive, Oral Language, and
Achievement batteries. General
Intelligence (g)
Mathematical achievement
(A3)
Reading decoding (RD)
Reading comprehension
(RC)
Reading speed (RS)
Spelling ability (SG)
English usage (EU)
Writing ability (WA)
Writing speed (WS)
General verbal information (K0)
Lexical knowledge (VL)
Listening ability (LS)
Induction (I)
General sequential
reasoning (RG)
Quantitative reasoning (RQ)
Memory span (MS)
Working memory capacity (WM)
Associative memory (MA)
Meaningful memory (MM)
Speed of Lexical access (LA)
Naming facility (NA)
Visualization (Vz)
Visual Memory (MV)
Spatial Scanning (SS)
Phonetic coding (PC)
Quantitative Knowledge
(Gq)
Reading & Writing (Grw)
Comp -Knowledge
(Gc)
Fluid Reasoning (Gf)
Short-Term Wk Mem.
(Gsm)
Long-Term Retrieval (Glr)
Visual Processing (Gv)
Auditory Processing
(Ga)
Processing Speed (Gs)
Perceptual speed (P)
Number Facility (N)
General science info. (K1)
Knowledge of culture (K2)
Domain-Specific Knw.
(Gkn)
Geography ach. (A5)
Shading designates proposed changes in CHC model based on analysis of WJ IV COG, OL. ACH norm data
(see Chapter 1 and Appendix A in WJ IV Technical Manual)
Language development
(LD)
Attentional Control (AC)
Memory for Sound Patterns
(UM)
Word Fluency (FW)
Verbal (print) lang. comp. (V)
Gsm has been renamed Gwm at the round table of cognitive CHC abilities
During the past two decades, and the last decade in particular, cognitive neuroscience has indicated that the more narrow Gsm definition was outdated and incorrect (Dehn, 2008). Working memory refers to a dynamic, temporary storage system that allows information to be held immediate awareness and be manipulated. Working memory refers to individual differences in both the capacity (size) of primary memory and to the efficiency of attentional control mechanisms that manipulate information within primary memory. Short-term memory refers to tasks that involve significant storage but only minimal processing or manipulation.
Attentional Control (Gwm-AC). The ability to focus on task-relevant stimuli and ignore task-irrelevant stimuli. The ability to regulate intentionality and direct cognitive processing. Sometimes referred to as spotlight or focal attention, focus, control of attention, executive controlled attention or executive attention.
Memory for Sound Patterns (Ga-UM). Ability to retain (on a short-term basis) auditory codes such as tones, tonal patterns, or speech sounds.
Word Fluency (Glr-FW). Ability to rapidly produce words that share a phonological (e.g., fluency of retrieval of words via a phonological cue) or semantic feature (e.g., fluency of retrieval of words via a meaning-based representation). Also includes the ability to rapidly produce words that share non-semantic features (e.g., fluency of retrieval of words starting with the letter “T”). Speed of Lexical Access (Glr-LA). Ability to rapidly and fluently retrieve words from an individual's lexicon; verbal efficiency or automaticity of lexical access.
Proposed changes/additions to CHC narrow ability taxonomy
OM
PC US UM U8 UR U1 U9 UP UL
Vz SR MV CS SS CF IM PI LE IL PN
Sens
ory-
Mot
or D
omai
n-
Spec
ific
Abili
ties
Sensory
The CHC Periodic Table of Human
Abilities
Adapted from Schneider & McGrew (2012) and McGrew, LaForte and Schrank (2014)
I RG RQ
WM MS AC
R3 PT MT
P N R9
R1 R2 R4 R7 IT
MA MM M6 FI FA FE SP F0 NA FW LA FF FX
Ideas Words Figures
Dom
ain-
Inde
pend
ent
Capa
citie
s
Glr-Learning efficiency
Glr-Retrieval fluency
Broad ability
Narrow ability
© Institute for Applied Psychometrics (IAP) Dr. Kevin McGrew 4-17-14
KM A3
LD VL K0 LS CM MY
KL K1 A5 MK KF LP BC
V RD RC RS WA SG EU WS
Acqu
ired
Know
ledg
e Sy
stem
s K2
PI P2 P3 P4 P6 P7 P8
U1 U9 UP UL
Motor
A1
KM A3
LD VL K0 LS
K1 A5
V RD RC RS WA SG EU WS
PC UM
Vz MV SS
Acqu
ired
Know
ledg
e Sy
stem
s Se
nsor
y-M
otor
Dom
ain-
Sp
ecifi
c Ab
ilitie
s I RG RQ
WM MS AC
P N
MA MM NA FW LA
Ideas Words Figures
Dom
ain-
Inde
pend
ent
Capa
citie
s
K2
Broad and Narrow Abilities Measured by the WJ IV COG, OL and ACH
FI
© Institute for Applied Psychometrics (IAP) Dr. Kevin McGrew 4-11-14
Gc Gwm Gs Ga Glr Gv
Gf
Vocab. (2)
(These two clusters require one test each from Cognitive and Oral Language easels)
Aud. Mm. Sp. (2)
Organization of WJ IV Cognitive (COG) and Oral Language (OL) batteries
General Intellectual Ability – g (7)
Brief Intellectual Ability (3)
Gf+Gc Composite (4)
Clusters available from Standard Cognitive
easel (10 tests)
(#) = # tests
Clusters available from Oral Language
easel(12 tests)
(#) = # tests
ST Wk. Mem. (2)
Cmp. Know. (2)
Fld. Reas. (2)
Cognitive Efficiency (2)
Other clusters available by
combining Standard Cognitive tests with
select tests from Extended Cognitive
easel (8 tests)
(#) = # tests
Cog. Pr. Spd. (2)
LT Retrieval (2)
Visual. Proc. (2)
Aud. Proc. (2)
ST Wk. Mem. (3)
Cmp. Know. (3)
Fld. Reas. (3)
Qnt. Reas. (2)
Perc. Spd. (2)
Number Facility (2)
Cognitive Efficiency (4)
Reading, Math and Writing Scholastic Aptitudes (each comprised of a mix of 4 different CHC abilities)
Broad Oral Lang (3)*
Oral Expression (2)
Oral Lang. (2)*
Listening Comp. (2)* (* English or Spanish)
Phon. Cod. (2)
Spd. Lx. Acc. (2)
WJ IV COG and OL tests by CHC factor domains
Oral Vocabulary
(VL)
Number Series (RQ)
General Information
(K0)
Object-Num. Sequencing
(WM)
Pair Cancellation (P/WM-AC)
Memory for Words (MS)
Analysis-Synthesis
(RG)
Concept Formation
(I)
Numbers Reversed
(WM)
Picture Recognition
(MV)
Visual-Aud. Learning
(MA)
Story Recall (MM)
Number-Pat. Matching
(P)
COG
OL
New or sign. change
Verbal Attention (WM/AC)
Letter-Pat. Matching
(P)
Nonword Repetition
(PC/UM-MS)
Phonological Processing (PC/Glr-LA)
Visualization (Vz)
Picture Vocabulary
(VL/LD)
Oral Comp. (LS)
Under. Directions
(WM/Gc-LS)
Sentence Repetition
(MS)
Retrieval Fluency (FI/LA)
Rapid Picture Naming (NA/LA)
Sound Awareness
(PC)
Sound Blending
(PC)
Segmentation (PC)
CHC Factors
Comp -Knowledge
(Gc)
Fluid Reasoning (Gf)
Long-Term Retrieval (Glr)
Visual Processing (Gv)
Auditory Processing
(Ga)
Processing Speed (Gs)
Short-Term Wrk Mem
(Gwm)
WJ IV ACH tests by CHC factor domains
Grw -Reading
CHC Factors
New or sign. change
Grw -Writing Gq Gc/Gkm
Letter-Word Identification
(RD)
Passage Comp.
(RC)
Editing (EU)
Social Studies (K0/K2)
Sentence Wr. Fluency
(WA/WS)
Reading Vocabulary (RC/Gc-VL)
Writing Samples
(WA)
Calculation (A3)
Humanities (K0/K2)
Spelling (SG)
Science (K0/K1)
Applied Problems (A3/RQ)
Word Attack (RD/Ga-PC)
Sentence Rdg. Fluency (RC/Gc-RS)
Spelling of Sounds
(SG/Ga-PC)
Math Facts Fluency
(A3/Gs-N)
Reading Recall
(RC/Glr-MM)
Word Rdg. Fluency
(RC/Gs-RS)
Oral Reading (RD/V)
Number Matrices (Gf-RQ)
Grw-W
Grw-R Gq Gc/Gkn
Phon.-Graph. Know. (2)
Rdg. Cmp. (2)
Other clusters available by combining Standard ACH
tests with select tests from Extended ACH easel (9
tests)
(#) = # tests
Rdg. Cmp. (3)
Rdg. Rate (2)
Bas. Wr. Sk. (2)
Math Pr.Slv. (2)
Acd. Knw. (3)
Clusters available from Standard Ach
easel (11 tests)
(#) = # tests
See Table 1-3 for COG test-cluster
interpretive cluster
composition
Organization of WJ IV
Achievement (ACH) battery
Bas. Rdg. Sk. (2)
Brd. Rdg. (3)
Rdg. Flu. (2)
Rdg. (2)
Wr. Exp. (2)
Brd. Wr. Lg. (3)
Wr. Lg. (2)
Math Cal.Sk. (2)
Brd. Math. (3)
Math. (2)
Academic Skills (3)
Academic Applications (3)
Academic Fluency (3)
Broad Achievement (9)
Brief Achievement (3)
Organizational overview of the WJ IV variations and comparisons
Dr. Kevin McGrew Institute for Applied Psychometrics (IAP)
Comparison Options
• GIA/Achievement • Scholastic Aptitude/Achievement • Gf-Gc/Achievement/other abilities • Broad Oral Language/Achievement • Academic Knowledge/Achievement
Five ability/achievement difference score procedures to help compare ability to current
levels of achievement.
[Procedures account for regression-to-the mean (and how it varies by age)]
Variation Options
• Intra-cognitive based on COG Tests 1—7 • Intra-achievement
– Based on ACH Tests 1—6 – Based on Academic Skills, Academic Fluency,
and Academic Applications clusters • Intra-oral language based on OL Tests 1—4
Four variation procedures to help document an individual’s pattern
of strengths and weaknesses.
Note: All score distributions represent real scores for all 9-13 year old norming subjects from
WJ IV co-normed sample. Actual
prediction models vary by age or grade (developmentally shifting prediction models).
Note: The SD of predicted and difference score distributions
are not 15. They would only be 15 if GIA/Brd. Rdg. correlation
was perfect (1.0).
Illustration of procedures used to develop ALL WJ IV variation/ comparison difference (standard score) norms (GIA-Broad Reading ACH example)
- E
(minus)
B Actual Broad Reading scores
in WJ IV norm data
40 80 120 160 100 140 60
-1 to +1 SD
SD = 15
A General Intellectual
Ability (GIA) scores in WJ IV norm data (predictor score)
40
80
120
160
100
140
60
-1 to
+1
SD
SD =
15
= (equals)
F Distribution of Actual-Predicted GIA/Broad Reading Difference
Scores in WJ IV norm data
(SD of difference
score distribution allows for
specification & evaluation of significant S/W with SD
and PR metrics)
-40 -30 -20 -10 0 10 20 30 40
-1 to +1 SD
SD = 10.8
D Predicted (criterion)
Broad Reading scores in WJ IV norm data
40 80 120 160 100 140 60
-1 to +1 SD
SD = 11.6
General Intellectual Ability (GIA) cluster (Predictor score)
General Intellectual Ability (GIA) /Achievement comparison procedure
Gwm Gs Ga Glr Gv Gc
Gf
CHC Cognitive Abilities and WJ IV COG and OL Clusters
Oral Language
Grw Gq
Brief Achievement
Academic Skills
Academic Fluency
Academic Applications
Broad Achievement
Reading
Brd. Rdg.
Rdg. Flu.
Bas. Rdg. Sk.
Rdg. Cmp. (& Ext)
Rdg. Rate
Wr. Lng.
Brd. Wr. Lg.
Bas. Wr. Sk.
Wr. Exp.
Phn.-Grp.Kn.
Math.
Brd. Math
Math Cal.Sk.
Math Pr. Slv.
(Possible criterion or predicted target scores) Broad Oral Language
Oral Expression
Listening Comprehension
CHC Achievement Abilities and WJ IV
clusters
Regression-based prediction models that account for regression-to-the-mean (and how it varies by age)
Predicted Target
Cluster Score
Actual Target Cluster Score
(minus) (equals)
Difference Score
SD and PR for specified SD cut-off score
(Compare to distribution of difference scores in WJ IV
norm sample to determine significant strength or
weakness)
CHC Cognitive Abilities and WJ IV COG and OL Clusters
Gwm Gs Ga Glr Gv Gc
Gf
Oral Language/Achievement comparison procedure
SD and PR for specified
SD cut-off score
(Compare to distribution of difference scores in WJ IV
norm sample to determine significant strength or
weakness)
Broad Oral Language
(Predictor score)
Sp. Lx. Acc. Phon. Cod.
(Possible criterion or predicted target scores)
Regression-based prediction models
that account for regression-to-the-mean (and how it
varies by age) Predicted
Target Cluster Score
Actual Target Cluster Score
(minus) (equals)
Difference Score
Academic Skills
Academic Fluency
Academic Applications
Brd. Rdg.
Rdg. Flu.
Bas. Rdg. Sk.
Rdg. Cmp. (& Ext)
Rdg. Rate
Wr. Lng.
Brd. Wr. Lg.
Bas. Wr. Sk.
Wr. Exp.
Phn.-Grp.Kn.
Brd. Math
Math Cal.Sk.
Math Pr. Slv.
Reading Math.
Grw Gq
CHC Achievement Abilities and WJ IV ACH clusters
(Possible criterion or predicted
target scores)
© Institute for Applied Psychometrics; Kevin McGrew 01-23-14
Phon. Cod. Sp. Lx. Acc.
ST Wk. Mem. (&Ext)
Cog .Pr.Spd. LT Retrieval Visual Proc. Aud. Proc.
Perc. Spd.
Cognitive Efficiency (& Ext)
Aud. Mm. Sp.
Other CHC broad/narrow cognitive and oral language abilities and WJ IV clusters
Gwm Gs Ga Glr Gv Grw Gq
Brief Achievement
Academic Skills
Academic Fluency
Academic Applications
Broad Achievement
Reading
Brd. Rdg.
Rdg. Flu.
Bas. Rdg. Sk.
Rdg. Cmp. (& Ext)
Rdg. Rate
Wr. Lng.
Brd. Wr. Lg.
Bas. Wr. Sk.
Wr. Exp.
Phn.-Grp.Kn.
Math.
Brd. Math
Math Cal.Sk.
Math Pr. Slv.
CHC achievement abilities and WJ IV clusters
Gf-Gc Composite (Predictor score)
Gc
Gf
WJ IV Gf+Gc cognitive Ability cluster
Gf-Gc “hybrid” variation/comparison procedure
Num. Fac.
Predicted Target Cluster
Score
Actual Target Cluster Score
(minus) (equals)
SD and PR for specified SD cut-off score
Difference Score
(Possible criterion or predicted target scores)
(Compare to distribution of difference scores in WJ IV norm sample to determine significant
strength or weakness)
Regression-based prediction models that account for
regression-to-the-mean
(See Table 1-12for the specific tests and cluster information)
Phon Cod Sp Lex Acc
ST Wrk Mem (&Ext)
Cog Prc Spd LT Retrieval Visual Proc Aud Proc
Perc Spd
Cognitive Efficiency (& Ext)
Aud Mem Sp
Other broad/narrow cognitive abilities and processing abilities and WJ IV clusters
Possible S/W predicted targets available for
evaluation
Gwm Gs Ga Glr Gv
Gf-Gc Composite (Predictor)
Gc
Gf
Gf+Gc cognitive ability and WJ IV cluster
WJ IV Gf+Gc strength and weakness comparison procedure and options
Grw Gq
Brief Achievement
Academic Skills
Academic Fluency
Academic Applications
Broad Achievement
Reading
Brd Rdg
Rdg Flu
Bas Rdg Sk
Rdg Cmp (& Ext)
Rdg Rate
Wr Lng
Brd Wr Lng
Bas Wrt Sk
Wr Exp
Phn-Grp Kn
Math
Brd Math
Math Calc Sk
Math Pr Solv
Achievement domains and WJ IV clusters
Possible S/W
predicted targets
available for evaluation
Regression-based predictions that account for regression-to-the-
mean and produce “real discrepancy norms” to identify possible patterns of significant
strengths and weaknesses
Grw Gq
Brief Achievement
Academic Skills
Academic Fluency
Academic Applications
Broad Achievement
Reading
Brd. Rdg.
Rdg. Flu.
Bas. Rdg. Sk.
Rdg. Cmp. (& Ext)
Rdg. Rate
Wr. Lng.
Brd. Wr. Lg.
Bas. Wr. Sk.
Wr. Exp.
Math.
Brd. Math
Math Cal.Sk.
Math Pr. Slv.
CHC achievement abilities and WJ IV clusters
Academic Knowledge (Predictor
score)
Academic Knowledge/Achievement comparison procedure
Predicted Target Cluster
Score
Actual Target Cluster Score
(minus) (equals)
SD and PR for specified SD cut-off score
Difference Score
(Possible criterion or predicted target scores)
(Compare to distribution of difference scores in WJ IV norm sample to determine significant
strength or weakness)
Gwm Gs Ga Glr Gv Gc
Gf
CHC cognitive abilities and WJ IV cognitive and oral language clusters
Sp. Lx. Acc. Phon. Cod.
(See Table 1-8 for the specific tests and cluster information)
Regression-based prediction models that account for regression-to-the-mean
Gc Gwm Gs Ga Glr Gv
Gf
WJ IV Scholastic Aptitude Cluster Organization
Visualization (Vz)
Numbers Reversed
(WM)
Oral Vocabulary
(LD/VL)
Analysis- Synthesis (RG/RQ)
Pair Cancellation
(P/EF)
Visualization (Vz)
Oral Vocabulary
(LD/VL)
Number Series (RQ)
Num Pattern Matching
(P)
Phonological Processing
(PC)
Oral Vocabulary
(LV/VL)
Verbal Attention
(WM)
Num Pattern Matching
(P)
Phonological Processing
(PC)
Oral Vocabulary
(LD/VL)
Verbal Attention
(WM)
Num Pattern Matching
(P)
Phonological Processing
(PC)
Oral Vocabulary
(LV/VL)
Story Recall (MM)
Num Pattern Matching
(P)
Phonological Processing
(PC)
Oral Vocabulary
(LD/VL)
Concept Formation
(I)
Reading Broad Reading Reading Comp Reading Comp-Ext Reading Fluency Reading Rate
Basic Rdg Skills
Writing Broad Writing Written Expression
Basic Writing Skills
Math Broad Math Math Calc Skills
Math Prob Solving
WJ IV Ach Clusters
Grw/Gq domain general Grw domain specific Gq domain specific
Reading Brd. Rdg.
Rdg. Flu.
Bas. Rdg. Sk.
Rdg. Rate
Wr. Lng. Brd. Wr. Lg.
Bas. Wr. Sk.
Wr. Exp.
Math. Brd. Math
Math Cal.Sk.
Math Pr. Slv.
Grw Gwm Gs Ga Glr Gv Gc
Gf
Gq
Rdg. Cmp.
Rdg Cmp. Ex
Rdg. Apt. A (Predictor Score)
Rdg. Apt. B (Predictor Score)
WL. Apt. A (Predictor Score)
WL. Apt. B (Predictor Score)
Math Apt. A (Predictor Score)
Math Apt. B (Predictor Score)
Regression-based prediction models that account for regression-to-the-mean
CHC cognitive abilities and WJ IV Scholastic Aptitude cognitive clusters CHC achievement abilities and WJ IV clusters
Predicted Target Cluster
Score
Actual Target Cluster Score
(minus) (equals)
SD and PR for specified SD cut-off score
Difference Score
(Compare to distribution of difference scores in WJ IV norm sample to
determine significant strength or weakness) Scholastic Aptitude/Achievement comparison procedures
(See Table 1-10 for the specific tests and cluster information)
Copyright, Institute for Applied Psychometrics, K.McGrew; 01-23-14
Scholastic Aptitude Clusters: Potential Uses
• Designed to predict near term academic performance • Time efficient referral-focused selective testing • Time efficient academic domain-specific screening • Time efficient annual review evaluations • Gifted and talented screening – domain-specific talents? • Potentially useful in estimating quickness of response to
intervention • Provide information regarding the concept of “expected
underachievement” • Formulation of differential academic domain expectations • ?
One of the WJ IV guiding design objectives was for the WJ IV to retain a focus on psychometric quality that has been associated with the previous editions of Woodcock-Johnson batteries:
•By providing a new, large, and nationally-representative standardization sample of the US population;
•By updating items and simplifying test administration and interpretation procedures;
•By augmenting the underlying scaling of speeded tests; and
•By utilizing state-of-the art data collection, test development, and data analytic methods as models to facilitate progress in the field of applied test development.
WJ IV Gs tests based on new Rasch “rate-based” item calibration method: WJ III Visual Matching / WJ IV
Letter-Pattern Matching example
WJ III Visual Matching (3 minutes) • 60 items • 22 Possible “artificial” bonus points • 82 Possible raw score points (27 % of total scale was “artificial” bonus points)
WJ IV Letter-Pattern Matching (3 minutes) • 84 items (24 more “real” items to eliminated need for bonus points) • Test scores based on count of number of correct responses in each of the three minute intervals. Thus, three different rate-based “item” scores • Rasch rating scale model (Andrich, 1978) use to obtained difficulty estimate for each minute of testing
• Need for artificial bonus points eliminated • Method rewards examinees who work quickly and accurately • Method penalizes examinees who respond quickly and not cautiously
(Note: IRT scaling of speed tests has historically been problematic due to violations of IRT assumptions. IRT methods still typically used for practical purposes, recognizing the assumption violations. See McGrew, Werder & Woodcock, 1991)
National Norm Sample
• 7,416 participants • Preschool (664) • K-12 (3,891) • College/University (775) • Adult (2,086)
• Ages 2-90+ years, Grades K.0-18.0 • 100 geographically diverse communities from 46 states and the
District of Columbia • The Standards for Educational and Psychological Testing (aka., the
Joint Standards) (American Psychological Association, American Educational Research Association, & National Council on Measurement in Education, 1999; in press) guided the norming and technical analyses
Median reliabilities
(across all technical age groups)
for WJ IV COG clusters
General Intellectual Ability 0.97 Brief Intellectual Ability 0.94 Gf-Gc Composite 0.95 Comprehension-Knowledge 0.93 Comprehension-Knowledge- Extended 0.94 Fluid Reasoning 0.94 Fluid Reasoning- Extended 0.96 Short-Term Working Memory 0.91 Short-Term Working Memory- Extended 0.93 Cognitive Processing Speed 0.94 Auditory Processing 0.92 Long-Term Retrieval 0.97 Visual Processing 0.86 Quantitative Reasoning 0.94 Auditory Memory Span 0.90 Number Facility 0.90 Perceptual Speed 0.93 Cognitive Efficiency 0.95 Cognitive Efficiency-Extended 0.95 Reading Aptitude A 0.93 Reading Aptitude B 0.93 Math Aptitude A 0.94 Math Aptitude B 0.95 Writing Aptitude A 0.92 Writing Aptitude B 0.93
Median reliabilities
(across all technical age groups) for WJ IV OL
clusters
Oral Language 0.90 Broad Oral Language 0.92 Oral Expression 0.89 Listening Comprehension 0.90 Phonetic Coding 0.95 Speed of Lexical Access 0.89 Vocabulary 0.93
Copyright; Institute for Applied Psychometrics; K.McGrew 02-5-14
Reading 0.95 Broad Reading 0.97 Basic Reading Skills 0.95 Reading Comprehension 0.93 Reading Comprehension-Extended
0.96
Reading Fluency 0.96 Reading Rate 0.96 Mathematics 0.96 Broad Mathematics 0.97 Math Calculation Skills 0.97 Math Problem Solving 0.95 Written Language 0.94 Broad Written Language 0.95 Basic Writing Skills 0.95 Written Expression 0.92 Academic Skills 0.97 Academic Applications 0.96 Academic Fluency 0.97 Academic Knowledge 0.95 Phoneme-Grapheme Knowledge 0.94 Brief Achievement 0.97 Broad Achievement 0.99
Median reliabilities
(across all technical age groups)
for WJ IV ACH clusters
Sources of validity evidence presented
•Representativeness of the WJ IV Test Content, Process, and Construct Coverage •Developmental Patterns of WJ IV Ability Clusters •Internal Structure and Relations within the WJ IV •Relationship of WJ IV Scores to Other Measures of Cognitive Abilities, Oral Language, and Achievement •Performance of Clinical Samples on WJ IV Measures
Stage 3: Confirmatory
structural model cross-
validation phase
Stage 2: Exploratory structural model
generation (MG) and evaluation
Ages 3-5 A (n = 209)
Ages 3-5 B (n = 208)
Ages 6-8 A (n =412)
Ages 6-8 B (n = 411)
Ages 9-13 A (n = 785)
Ages 9-13 B (n = 787)
Ages 14-19 A (n = 842)
Ages 14-19 B (n = 843)
Ages 20-39 A (n = 625)
Ages 20-39 B (n = 626)
Ages 40-90+ A (n = 571)
Ages 40-90+ B (n = 575 )
Model Development
(MD) samples (A)
Model Cross-Validation
(MCV) samples (B)
Stage 1: Split-sample
random sample generation
Three-stage internal/ structural validity procedures for WJ IV battery
+
+
Cluster analysis (CA)
Multidimensional scaling analysis
(MDS)
Principal component
analysis (PCA)
Stage 2A Exploratory structural analysis in Sample A
Review of prior WJ,
WJ-R, WJ III structural
validity research
Specify initial confirmatory
factor analysis model (CFA)
Review of contemporary
CHC and cognitive
neuroscience research
CFA of MG model result from MD (A) sample in MCV (B)
sample(s) Yes
Most plausible and best
fitting odel?
Model adjustment(s) No
Stage 2B Specify initial CFA
models
Stage 2C Evaluate and adjust
CFA model
Wards cluster analysis of ages 9-13 year sample (model development sample A; n=785)
NUMREV OBJNUM VRBATN
MEMWRD SENREP
NWDREP UNDDIR
MEMNAM WRDATK SPLSND
SNDAWR SEGMNT PHNPRO SNDBLN
VAL VISUAL
ANLSYN CONFRM
VISCLO PICREC
RPCNAM RETFLU
ORLCMP VRBANL
HUM PICVOC
ORLVOC GENINF
SOC SCI
STYREC WRTSMP RDGREC NUMSEN APPROB
CALC NUMSER NUMMAT
SPELL LWIDNT
PSGCMP RDGVOC
EDIT ORLRDG MTHFLU WRDFLU SNRDFL SNWRFL NUMPAT LETPAT PAIRCN
Gc
Glr-MM/Grw
Gq/Gf-RQ
Grw
Gs Gs-ACH
Gs-COG
Acquired knowledge systems
Gv
Gf Gf+Gv
Glr-LA
Gwm- WM
Gwm- MS Gwm
Ga-PC
Ga-PC/Grw Ga
Ga+Gwm (sequential/serial processing)
Cognitive Operations: Controlled/deliberate/system 2
General intelligence (g)
Bold font = Broad or general CHC abilities Underlined font= Possible intermediate level stratum dimensions Groupings to the left of the vertical dashed lines represent the groupings interpreted as representing valid CHC abilities. Higher-order groupings to the left of the dashed line are potentially interesting hypothesized intermediate ability dimensions.
Glr-NA
Glr (Retrieval Fluency)
Cognitive Operations: Automatic/system 1
-2 -1 0 1 2 -2
-1
0
1
2
NUMSER
VRBATN LETPAT
PHNPRO
VAL VISUAL
GENINF
NUMREV
NUMPAT
NWDREP
STYREC
PICREC
ANLSYN
OBJNUM PAIRCN
MEMWRD
SEGMNT
RPCNAM
SENREP
UNDDIR
SNDBLN
RETFLU
SNDAWR
LWIDNT
SPELL
PSGCMP
CALC
WRTSMP ORLRDG
MTHFLU
SNWRFL
RDGREC
NUMMAT
EDIT
WRDFLU
SPLSND
RDGVOC
SCI
MEMNAM
VRBANL
VISCLO
Gs-Cog
Gs-Ach
Glr-LA
Ga
Grw
Gv
Glr-MM
Gc
Gq/Gf-RQ
Gwm- WM
Gwm- MS
WRDATK
MDS (Guttman Radex) of ages 9-13
year sample (model
development sample A; n=785 )
Gf CONFRM
Test
EFA
Other exp. methods
CFA
Cross-validation
WJ III No No Yes No SB5 No No Yes No DAS-II No No Yes No KABC-II No No Yes (in “exploratory
manner”) No
WAIS-IV No No Yes No WISC-IV Yes No Yes Yes: Factor score (7)
congruence in random CV sample of 440
WJ IV Yes Yes: MDS and cluster
analysis
Yes: Exploratory model generating CFA (MD samples)
and CV CFA (CV samples)
Yes: Randomly split samples at six age groups. CV of final MD models in
CV sample
Structural validity method comparisons across 7 intelligence batteries
15 independent WJ IV concurrent validity studies differentiated by age/grade
WJIV COG compared to:
• WISC-IV, WAIS-IV, WPPSI-III, KABC-II, DAS-II
WJIV OL compared to:
• CELF-F, PPVT-4, CASL, OWLS
WJIV ACH compared to:
• KTEA-II, WAIT-III, OWLS-WE
Clinical Validity Group Inclusion Criteria
Learning Disability (LD): 6–17 years
Documented learning disability in reading Reading LD must be the primary diagnosis or eligibility category Currently receiving special education services under SLD category
Learning Disability (LD): Math 6–17 years
Documented learning disability in math Math LD must be the primary diagnosis or eligibility category Currently receiving special education services under SLD category
Learning Disability (LD): Writing 6–17 years
Documented learning disability in writing Writing LD must be the primary diagnosis or eligibility category Currently receiving special education services under SLD category
Head Injury 7–17 years Documented traumatic brain injury TBI must be the primary diagnosis Currently receiving special education services
Language Delay 7–13 years Documented language delay Language delay must be the primary diagnosis Currently receiving special education services
Autism Spectrum Disorder (ASD) 6–17 years
Documented Autism Spectrum Disorder1 ASD is the primary diagnosis Currently receiving special education services
Attention Deficit Hyperactivity
Disorder (ADHD) 7–12 years
Documented ADHD2 ADHD is the primary diagnosis Currently receiving special education services and/or 504 plan
accommodations
Gifted3 4–9 years Currently participating in high ability/gifted and talented school curriculum Currently receiving gifted services
Mental Retardation 6–12 years Documented mental retardation Mental retardation is the primary diagnosis Currently receiving special education services
WJ IV clinical validity evidence reported for 9 clinical groups
What is cognitive complexity?
Cognitive complexity
CHC factor breadth
Degree of g-loading
Complicated (Does not
necessarily equal)
Factorial complexity
Content/stimulus dimension (Extension of BIS and Brunswick Symmetry Model)
Cognitive knowledge domains/systems
Cognitive operations
Cognitive control
Cognitive efficiency
Sensory functions
Motor functions
Abilt
y do
mai
n di
men
sion
Type
I Pr
oces
sing
Ty
pe II
Pr
oces
sing
Note: CHC taxonomy is embedded in the ability domain dimension
McGrew (2010) – First formal introduction of cognitive complexity in “Beyond CHC” test development /evaluation
framework(Neuropsych. Conference in Fremantle, Australia)
Cognitve complexity dimension
High
Medium
Low
© Institute for Applied Psychometrics; Kevin McGrew 05-21-14
This is a heuristic framework and is not intended to represent a theoretical model of intelligence
• Larger number of cognitive component processes (factorial complexity) • Accumulation of speed component differences • Increased demands of attentional control (AC) & working memory •More demands on adaptive functions (assembly, control, and monitoring – executive functions) • More important component processes (e.g., inference; eduction of relations)
Parameters of cognitive efficiency
in info. proc.
models
Cognitively complex design characteristics (Lohman & Lakin, 2011)
Approach 1. Increasing the cognitive complexity of a test is often accomplished by making the test a mixed measure of more than one narrow CHC ability (factorially complex mixed CHC measures) Approach 2. A second approach is to increase the complexity of information processing demands of the tests within a specific narrow CHC domain (Lohman & Larkin, 2011; McGrew, 2012). This second form of cognitive complexity, not to be confused with factorial complexity, places greater demands on cognitive information processing (cognitive load), requires greater allocation of key cognitive resources (working memory or attentional control), and invokes the involvement of more cognitive control or executive functions (Arend, Colom, Botella, Contreras, Rubio, & Santacreu, 2003; Jensen, 2011; Lohman & Larkin, 2011; Marshalek, Lohman, & Snow; 1983). This second approach to increasing test cognitive complexity was a primary design principle for the WJ IV.
One design objective in the WJ IV was to increase the cognitive complexity requirements for selected tests and clusters to provide greater ecological validity and
interpretive relevance of the measures.
Most contemporary CHC designed individual tests have focused on developing relatively pure measures of each cognitive ability
(mental pulley)
Gf Gwm Ga Gv
Gc Gs Glr Gs
Analogy: Think of general intelligence (g) as a system of relatively independent cognitive abilities (relatively construct
“pure” pulleys) working together to deal with a specific cognitive task load
In this approach a test is designed to be a mixed measure of two (or more) cognitive abilities (mental pulleys; Gf + Gv)
Gf + Gv
Gf + Gv Gwm Gs Ga Glr Gv Gs
Approach 1 to developing cognitively complex tests: Construct factorially complex measures (a system of pulleys from 2 or more domain functions working in
combination).
Ga Gv Gwm Gs Gf Glr Gv Gs
Approach 2 is to increase the complexity of information processing demands of the tests within a specific CHC cognitive functional domain. Tasks are still as relatively pure a measure of the CHC domain as possible but there is a deliberate increase in the number of “mini-pulleys” (cognitive information component complexity) that work together within the CHC domain. This was the primary approach used for certain WJ IV tests.
Working Memory
Long-Term Memory
Storage Retrieval
Central Executive (Executive
functions or control?
• Inhibit •Shift
•Update
Perception Sensation
Decisions Output
????
Conceptual explanation of parameters of cognitive efficiency
Focus of Attention
Recently activated concepts (Blue-new; Red-old)
0.4 0.5 0.6 0.7 0.8 g-loading -0.10
0.45
1.00
1.55
2.10 M
DS
rela
tive
CC
(rC
C)
ORLVOC-Gc
SNDAWR-Ga
ORLCMP-Gc
VRBATN-Gwm
PHNPRO-Ga
OBJNUM-Gwm
NUMSER-Gf CONFRM-Gf UNDDIR-Gwm
NUMREV-Gwm
PICVOC-Gc
STYREC-Glr SENREP-Gwm
SEGMNT-Ga
ANLSYN-GF MEMWRD-Gwm
SNDBLN-Ga
VISUAL-Gf
RETFLU-Glr LETPAT-Gs
NWDREP-Ga NUMPAT-Gs
PAIRCN-Gs
RPCNAM-Glr
PICREC-Gv
Relationship between g-loadings and MDS-based relative cognitive complexity (rCC) for WJ IV COG and OL tests
Correlation = .93; but correspondence diverges as test become higher in g and
cognitive complexity
• COG: Verbal Attention • COG Number Series • COG Letter-Pattern Matching • COG Visualization • COG Phonological Processing • COG Nonword Repetition • OL Segmentation
New COG OL Tests
COG Test 2: Number Series
• Was in WJ III Diagnostic Supplement • Gf-RQ (Quantitative Reasoning) • Not a “controlled learning” test as are Concept Formation (Gf-I) and Analysis-Synthesis (Gf-RG)
• More Gf “in the wild” – without examiner provided scaffolding
• Extensive history as a premier Gf test in the psychometric measurement of intelligence • High in cognitive complexity and g. Best single test predictor of achievement. Best indicator of Gf factor. • In GIA, BIA, Gf-Gc Composite, Gf, Gf-Ext, Quantitative Reasoning (RQ), and one Math Aptitude clusters.
COG Test 7: Visualization
• Measure of Gv-Visualization (Vz) •Visualization consists of two subtests that each measure Gv-Vz (visualization) via tasks that vary on task complexity and degree of “minds eye” (mental rotation) manipulations
• Within Gv, highest on cognitive complexity, g, Gv factor, and prediction of achievement • In GIA, Gv and both Math Aptitude clusters
COG Test 3: Verbal Attention
• Measure of Gwm (working memory-WM; attentional control-AC)
• More ecological “real world” valid measure of working memory
• High in cognitive complexity and g. Within Gwm, the most cognitively complex, one of best indicators of Gwm factor, and best predictor of achievement
• In GIA, BIA, Gwm, Gwm3, Cognitive Efficiency, and one Reading and 1 Written Language Aptitude clusters.
COG Test 4: Letter-Pattern Matching
• Measure of Gs (perceptual speed) and orthographic processing • This speeded test (all WJ IV speeded tests) is based on a new rate-based method of scaling the scores that eliminates the need for bonus points • Within Gs, it matches Number Pattern Matching in g, Gs factor loading, and prediction of achievement. Is more cognitively complex than Number Pattern Matching • In GIA, Gs, Perceptual Speed (P), Cog. Eff. and Cog. clusters
COG Test 5: Phonological Processing
• Ga (PC) / Glr (LA/FW) •3 subtests (Word Access; Word Fluency; Substitution • Measures three aspects of speech sound processing that requires the efficiency construction of sound- based lexical representations • High in cognitive complexity and g. Best single Ga test predictor of achievement. High loading on Ga and secondary low loading on Gc (accessing the lexicon). Also loaded on narrow LA factor in broard+narrow bottom-up CFA models. • In GIA, Ga, and all reading and writing scholastic aptitude clusters
Nonword repetition tasks
Examinee listens to a nonsense word and then must repeat the word exactly. Requires temporary storage of phonological segments in immediate awareness. Significant body of research has found such tasks to be significantly related to (and be possible “markers” of) reading disabilities, dyslexia and SLI (specific language impairment)
OL Test 3: Segmentation
• Ga (PC) • Examinee listens to words and identifies word parts • In OL Phonetic Coding (PC) cluster • Highest loading test on Ga factor across all ages • A moderate measure of g and predictor of ach. across
all ages; much more so (and more cognitively complex) than Sound Blending.
• Such tasks have been reported to be strong predictors of early reading (Bouwmeester et al, 2011; Geuden & Sandra, 2003)
Contemporary CHC broad and narrow ability content coverage by WJ IV Cognitive (as per WJ IV authors)
General Intelligence (g)
Mathematical achievement
(A3)
Reading decoding (RD)
Reading comprehension
(RC)
Reading speed (RS)
Spelling ability (SG)
English usage (EU)
Writing ability (WA)
Writing speed (WS)
General verbal information (K0)
Lexical knowledge (VL)
Listening ability (LS)
Induction (I)
General sequential
reasoning (RG)
Quantitative reasoning (RQ)
Memory span (MS)
Working memory capacity (WM)
Associative memory (MA)
Meaningful memory (MM)
Speed of Lexical access (LA)
Naming facility (NA)
Visualization (Vz)
Visual Memory (MV)
Spatial Scanning (SS)
Phonetic coding (PC)
Quantitative Knowledge
(Gq)
Reading & Writing (Grw)
Comp -Knowledge
(Gc)
Fluid Reasoning (Gf)
Short-Term Wk Mem.
(Gsm)
Long-Term Retrieval (Glr)
Visual Processing (Gv)
Auditory Processing
(Ga)
Processing Speed (Gs)
Perceptual speed (P)
Number Facility (N)
General science info. (K1)
Knowledge of culture (K2)
Domain-Specific Knw.
(Gkn)
Geography ach. (A5)
Shading designates abilities measured by WJ IV COG battery
Language development
(LD)
Attentional Control (AC)
Memory for Sound Patterns
(UM)
Word Fluency (FW)
Verbal (print) lang. comp. (V)
© Institute for Applied Psychometrics; Kevin McGrew 01-23-14
The WJ IV COG is not your fathers intelligence test!
The WJ IV COG GIA is a much more cognitively complex (and high g) measure of intelligence
How did we do this?
What evidence do we have to support this conclusion?
This image cannot currently be displayed.
© Institute for Applied Psychometrics; Kevin McGrew 01-23-14
WISC-IV WAIS-IV WPPSI-III KABC-II SB-5 DAS-II FS IQ FS IQ FS IQ FCI FS IQ GCA
(n=174) (n=177) (n=99) (n=50) (n = 50) (n = 49) WJ IV g-measures
General Intellectual Ability (GIA)
0.86 0.84 0.72 0.77 0.80 0.83
Brief Intellectual Ability (BIA) 0.83 0.74 0.76 0.79
Gf-Gc Composite 0.83 0.78 0.71 0.82
Note. Correlations in italic represent correlations with a pseudo-WJ IV GIA score computed from 7
WJ IV tests (WJ IV COG does not provide an GIA-Edv cluster)
Select concurrent validity evidence: Correlations of WJ IV primary COG g-scores with external measures
Conclusion: The WJ IV GIA, BIA and Gf-Gc composite clusters demonstrate strong validity evidence as measures of general intelligence when the criterion are the global composite/total
scores from other major IQ batteries in the field
WJ III GIA other IQ score correlations
were from .67 to .76
GIA - Standard
GIA Oral Vocabulary
General Information
Number Series
Concept Formation
Verbal Attention
Numbers Reversed
Story Recall
Visual-Aud Learning
Visualization
Picture Recognition
Letter-Pat. Match.
Pair Cancellation
Phonological Processing
Nonword Repetition
Primary WJ III and WJ IV COG tests and Clusters
Tests in WJ IV COG clusters
Tests in WJ III COG Clusters
WJ IV GIA WJ III GIA-Standard
* Visual Matching is renamed Number-Pattern Matching in WJ IV
Comparison of composition of primary WJ III and WJ IV COG CHC and GIA clusters
(WJ III Spatial Relations is half
of WJ IV Visualization)
(WJ IV Oral Vocab. was part
of WJ III Verbal Comp.)
Auditory Attention
Decision Speed
Analysis-Synthesis
Retrieval Fluency
Spatial Relations
Verbal Comp. Memory for Words
Sound Blending
Visual Matching *
General Intelligence (g)
Comp -Knowledge
(Gc) Fluid
Reasoning (Gf) Long-Term
Retrieval (Glr) Visual
Processing (Gv)
Auditory Processing
(Ga)
Processing Speed (Gs)
Short-Term Wrk Mem
(Gwm)
© Institute for Applied Psychometrics; Kevin McGrew 01-23-14
Four primary design principles have resulted in the WJ IV COG GIA being significantly different in
composition than the WJ III COG GIA-Std.
The WJ IV COG GIA cluster tests should: 1. Be the best factor indicators of each CHC broad domain
2. Be the best predictors of achievement from each CHC broad domain
3. Be the most cognitively complex indicators from each CHC broad domain
4. Be the best measures of g (general intelligence) from each CHC broad
domain
5. Collectively should have a relatively equal balance of type of stimulus characteristics (verbal, numeric, figural)
(Note: Trade-offs between the four design objectives were sometimes necessary)
© Institute for Applied Psychometrics; Kevin McGrew 02-05-14
This image cannot currently be displayed.
The Cognitive tests were evaluated on the basis of four (of five total) quantifiable COG design criteria
Data augmented by Siskel and Ebert informal rating system
Average CHC factor loadings Average achievement correlation across domains Average degree of g-loadings Average degree of relative cognitive complexity
(Average across ages 6-90+)
Comprehension – Knowledge (Gc)
Lexical Knowledge (VL)
General Information (K0)
Language Development (LD)
ORLVOC* GENINF PICVOC
* Test in GIA
Author/expert CHC narrow
factor classifications
(Gc cluster) (Voc. Cluster- LD/VL)
Induction (I)
General Sequential Reasoning (RG)
Quantitative Reasoning (RQ)
ANLSYN CONFRM NUMSER* (Gf cluster) (Gf3
cluster)
(Quantitative Reasoning Cluster-RQ)
* Test in GIA
Fluid Reasoning (Gf)
Author/expert CHC narrow
factor classifications
Short-Term Working Memory (Gwm)
Working Memory (WM)
Memory Span (MS)
Attentional Control (AC)
MEMWRD
SENREP
(Auditory Memory Span Cluster-MS)
OBJNUM VRBATN* NUMREV
* Test in GIA
(Gwm cluster)
(Gwm3 cluster)
Author/expert CHC narrow
factor classifications
© Institute for Applied Psychometrics; Kevin McGrew 2-04-14
Cognitive Processing Speed (Gs)
Attentional Control
Perceptual Speed (P)
PAIRCN NUMPAT LETPAT*
* Test in GIA
Short-Term Working Memory (Gwm)
(Gs cluster) (Perceptual Speed
cluster-P)
Author/expert CHC narrow
factor classifications
Long-Term Retrieval (Glr)
Associative Memory (MA)
Meaningful Memory (MM)
* Test in GIA
VAL STREC* (Glr cluster)
Speed of Lexical Access (LA)
RETFLU RPCNAM (Speed of Lexical
Access cluster-LA)
Author/expert CHC narrow
factor classifications
Auditory Processing (Gs)
Phonetic Coding (PC)
NWDREP PHNPRO*
* Test in GIA
Short-Term Working Memory (Gwm)
Memory for Sound Patterns (UM)
SEGMNT SNDBLN
(Ga cluster)
(Phonetic Coding cluster-PC)
Author/expert CHC narrow
factor classifications
© Institute for Applied Psychometrics; Kevin McGrew 2-04-14
This image cannot currently be displayed.
The WJ IV Auditory Processing (Ga) cluster is not your fathers Ga measure.
WJ IV still has the Oldsmobile Ga (Phonetic
Coding) in OL: COG now has more cognitively complex Ga measures
The WJ IV has taken a broader contemporary view
of the domain of Ga © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
Auditory Processing (Ga) abilities, when properly measured, should have a
prominent chair at the round table of cognitive CHC abilities
© Institute for Applied Psychometrics; Kevin McGrew 02-01-14
Visual Processing (Gv)
Visual Memory (MV)
Visualization (Vz)
* Test in GIA
PICREC VISUAL* (Gv cluster)
Author/expert CHC narrow
factor classifications
COG/OL Mdn. Clusters r r2 General Intellectual Ability 0.76 0.58 Brief Intellectual Ability 0.73 0.53 Gf-Gc Composite 0.68 0.46
Comprehension-Knowledge (Gc) 0.58 0.34 Comprehension-Knowledge-Ext (Gc3) 0.58 0.34 Fluid Reasoning (Gf) 0.64 0.41 Fluid Reasoning-Ext (Gf3) 0.59 0.35 Short-Term Working Memory (Gwm) 0.53 0.28 Short-Term Working Memory-Ext (Gwm3) 0.52 0.27 Cognitive Processing Speed (Gs) 0.49 0.24 Auditory Processing (Ga) 0.51 0.26 Long-Term Retrieval (Glr) 0.43 0.18 Visual Processing (Gv) 0.38 0.14 Quantitative Reasoning (Gf-RQ) 0.59 0.35 Auditory Memory Span (Gsm-MS) 0.47 0.22 Number Facility (Gs-N) 0.62 0.38 Perceptual Speed (Gs-P) 0.58 0.34 Cognitive Efficiency (Gsm+Gs) 0.55 0.30 Cognitive Efficiency-Ext (Gsm+Gs) 0.66 0.44
Oral Language 0.55 0.30 Broad Oral Language 0.57 0.32 Oral Expression 0.54 0.29 Listening Comprehension 0.55 0.30 Phonetic Coding (Ga-PC) 0.44 0.19 Speed of Lexical Access (Glr-LA) 0.37 0.14 Vocabulary (Gc-LD/VL) 0.60 0.36
WJ IV COG and OL cluster
correlations with WJ IV ACH
clusters: Correlations
across 15 ACH clusters (ages
6-90+)
WISC-IV WAIS-IV WPPSI-III KABC-II SB-5 DAS-II FS IQ FS IQ FS IQ FCI FS IQ GCA
(n=174) (n=177) (n=99) (n=50) (n = 50) (n = 49) WJ IV g-measures
General Intellectual Ability (GIA)
0.86 0.84 0.72 0.77 0.80 0.83
Brief Intellectual Ability (BIA) 0.83 0.74 0.76 0.79
Gf-Gc Composite 0.83 0.78 0.71 0.82
Note. Correlations in italic represent correlations with a pseudo-WJ IV GIA score computed from 7
WJ IV tests (WJ IV COG does not provide an GIA-Edv cluster)
Select concurrent validity evidence: Correlations of WJ IV primary COG g-scores with external measures
Conclusion: The WJ IV GIA, BIA and Gf-Gc composite clusters demonstrate strong validity evidence as measures of general
intelligence when the criterion are the global composite/total scores from other major IQ batteries in the field
WJ III GIA other IQ score correlations
were from .67 to .76
WISC-IV WISC-IV WISC-IV WISC-IV VCI PRI WMI PSI
WJ IV Measures (Gc) (Gf/Gv) (Gwm) (Gs) CHC Factor Clusters
Comprehension-Knowledge (Gc) 0.79 Fluid Reasoning (Gf) 0.70 Short-Term Working Memory (Gwm) 0.72 Processing Speed (Gs) 0.55 Auditory Processing (Ga) Long-Term Retrieval (Glr) Visual Processing (Gv) 0.55
Narrow Ability & Clinical Clusters Quantitative Reasoning (RQ) 0.65 Auditory Memory Span (MS) 0.52 Number Facility (N) 0.57 Perceptual Speed (P) 0.56
Note: Bold font values represent correlations between best comparable CHC broad composites. Red bold font are interesting correlations given the Wechsler composites composition. WISC-IV/WAIS-IV PRI and WMI composites are not comparable--different mixtures of CHC abilities. (see next slide)
WJ IV COG /WISC-IV CHC composite select score correlations ( n = 174)
Block Design-Gv-Vz Pic Cocnepts – Gf-I Matrix Reasoning – Gf-I
Digit Span – Gwm-MS/MW Letter-Num Seq – Gwm-MW
WAIS-IV WAIS-IV WAIS-IV WAIS-IV VCI PRI WMI PSI
WJ IV Measures (Gc) (Gv/Gf) (Gwm/Gq) (Gs) CHC Factor Clusters
Comprehension-Knowledge (Gc) 0.74 Fluid Reasoning (Gf) 0.57 Short-Term Working Memory (Gwm) 0.67 Processing Speed (Gs) 0.44 Auditory Processing (Ga) Long-Term Retrieval (Glr) Visual Processing (Gv) 0.57
Narrow Ability & Clinical Clusters Quantitative Reasoning (RQ) 0.54 0.53 Auditory Memory Span (MS) Number Facility (N) 0.65 0.52 Perceptual Speed (P) 0.61
Note: Bold font values represent correlations between best comparable CHC broad composites. Red bold font are interesting correlations given the Wechsler composites composition. WISC-IV/WAIS-IV PRI and WMI composites are not comparable--different mixtures of CHC abilities. Noted Wechsler correlations with tests classifications based on Flanagan et al., 2013.
WJ IV COG / WAIS-IV CHC composite select score correlations ( n = 177)
Block Design-Gv-Vz Visual Puzzles – Gv-Vz Matrix Reasoning – Gf-I
Digit Span – Gwm-MS/MW Arithmetic – Gwm-WM/Gf-RQ (Gq?)
KABC-II KABC-II KABC-II KABC-II KABC-II Know./ Plan./ Sim./ Seq./ Lrng./
WJ IV Measures Gc Index
Gf Index
Gv Index
Gsm Index
Glr Index
CHC Factor Clusters Comprehension-Knowledge (Gc) 0.82 Fluid Reasoning (Gf) 0.46 Short-Term Working Memory (Gwm)
0.42
Processing Speed (Gs) Auditory Processing (Ga) Long-Term Retrieval (Glr) 0.64 Visual Processing (Gv) 0.37
Note: Bold font values represent correlations between best comparable CHC broad composites. Low to moderate corresponding Gf, Gv and Gwm correlations most likely reflect narrow ability content differences in composites (classifications based on Reynolds et al., 2007 and Flanagan et al., 2013).
WJ IV COG / KABC-II composite select score correlations (n = 50)
Pattern Reasoning - Gf-I/Gv-Vz Story Completion – Gf-RG
Number Recall – Gsm-MS Word Recall – Gsm-MS/WM?
Rover – Gv-SS Triangles – Gv-Vz
SB-5 SB-5 SB-5 SB-5 SB-5 Fluid Quant. Vis.-Spatial Working
Know. Reas. Reas. Proc. Memory WJ IV Measures (Gc) (Gf) (Gf-RQ) (Gv) (Gwm)
CHC Factor Clusters Comprehension-Knowledge (Gc) 0.68 0.75 0.72 0.72 Fluid Reasoning (Gf) 0.67 0.56 0.66 0.66 Short-Term Working Memory (Gwm) 0.62 0.69 Cognitive Processing Speed (Gs) Auditory Processing (Ga) 0.68 0.73 0.72 Long-Term Retrieval (Glr) Visual Processing (Gv) 0.40
Note: Bold font values represent correlations between best comparable CHC broad composites. Red bold font are interesting correlations given the SB-5 composites composition. The convergent/divergent validity of the SB-5 CHC composite scores has been seriously questioned (Canivez, 2008; DiStefano & Dombrowski, 2006; Keith & Reynolds, 2010) as well as very high composite score intercorrelations in the SB-5 technical manual (.65 to .75)
WJ IV COG / SB-5 CHC composite select score correlations (n = 50)
• 7 new tests • Oral Reading, Reading Recall, • Word Reading Fluency • Number Matrices • Science, Social Studies, Humanities
• 8 new clusters • Reading, Reading Comprehension-Extended,
Reading Fluency, Reading Rate • Written Language • Mathematics • Brief Achievement, Broad Achievement
Achievement: What’s New?
• Conormed with WJ IV Tests of Cognitive Abilities and WJ IV Tests of Oral Language
• Qualitative Observation Checklists for Tests 1-11
– Located in Test Record – Helps document important information about how
examinee performed on the task – Includes data on percentage of age mates at each rating
Achievement: What’s New?
Test 1: Letter-Word Identification
Test 2: Applied Problems
Test 3: Spelling
Test 4: Passage Comprehension
Test 5: Calculation
Test 6: Writing Samples
Test 7: Word Attack
Test 8: Oral Reading--NEW
Test 9: Sentence Reading Fluency (Renamed)
Test 10: Math Facts Fluency (Renamed)
Test 11: Sentence Writing Fluency (Renamed)
Tests of Achievement
Standard Battery
WJ IV Tests of Achievement---Extended Battery
Test 12: Reading Recall--NEW
Test 13: Number Matrices--EXPANDED
Test 14: Editing
Test 15: Word Reading Fluency--NEW
Test 16: Spelling of Sounds
Test 17: Reading Vocabulary--REVISED
Test 18: Science--EXPANDED
Test 19: Social Studies--EXPANDED
Test 20: Humanities--EXPANDED
Provides clusters for: •Reading Comprehension •Reading Comprehension -Extended •Reading Rate •Math Problem Solving •Basic Writing Skills •Academic Knowledge
• Includes 8 tests for evaluating different aspects of reading – Letter-Word Identification, Passage Comprehension, Word
Attack, Sentence Reading Fluency, Reading Vocabulary – Oral Reading, Reading Recall, Word Reading Fluency
• Provides 7 clusters for a comprehensive evaluation of
reading performance • Reading, Broad Reading, Basic Reading Skills, Reading
Comprehension • Reading Fluency, Reading Rate, Reading Comprehension-
Extended
Summary: Reading
3 new tests
3 new clusters
• Includes 4 tests measuring various aspects of math achievement: – Applied Problems, Calculation, Math Facts
Fluency, and Number Matrices
• Provides 4 clusters for a comprehensive evaluation of math performance: – Mathematics, Broad Mathematics, Basic Math
Skills, Math Problem Solving
Mathematics: Summary
• Includes 5 tests measuring various aspects of written language: – Spelling, Writing Samples, Sentence Writing
Fluency, Editing, and Spelling of Sounds • Provides 4 clusters for a comprehensive
evaluation of writing performance: – Written Language, Broad Written Language, Basic
Writing Skills, Written Expression • Contributes to Phoneme-Grapheme Cluster
(Spelling of Sounds + Word Attack)
Written Language: Summary
• Includes 3 tests measuring various aspects of academic knowledge (comprehension-knowledge [Gc]) • Test 18: Science • Test 19: Social Studies • Test 20: Humanities
• Provides 1 cluster: Academic Knowledge
– Serves as the ability score in an ability/achievement comparison procedure
Academic Knowledge
• A cluster that includes tests from different academic domains (reading, writing, math)
• 7 cross-domain clusters are available
– Brief Achievement and Broad Achievement – Academic Knowledge – Academic Skills – Academic Fluency – Academic Applications – Phoneme-Grapheme Knowledge
Cross-Domain Clusters
Two options
Intra-Achievement using Tests 1-6 •Test 1. Letter Word Identification •Test 2. Applied Problems •Test 3. Spelling •Test 4. Passage Comprehension •Test 5. Calculation •Test 6. Writing Samples
Intra-Achievement using 3 clusters: •Academic Skills •Academic Fluency •Academic Applications
Intra-Achievement Variation Procedures
• Additional tests can be added to procedures • Compares performance on one test or cluster to
average performance on other tests or clusters • Provides information about examinee’s
strengths and weaknesses within academic areas
• Helps pinpoint a pattern of strengths and weaknesses
Intra-Achievement Variation Procedures
Academic Skills/Academic Fluency/ Academic Applications
When ACH Tests 1 through 6 and 9 through 11 are administered, a cross-domain comparison is made. • Academic Skills cluster is compared to a predictor score
from the Academic Applications and Academic Fluency clusters;
• Academic Applications cluster is compared to Academic Skills and Academic Fluency clusters;
• Academic Fluency cluster is compared to a predictor score from the Academic Skills and Academic Applications clusters.
Median reliabilities
(across all technical age groups)
for WJ IV ACH clusters
Reading 0.95 Broad Reading 0.97 Basic Reading Skills 0.95 Reading Comprehension 0.93 Reading Comprehension-Extended 0.96 Reading Fluency 0.96 Reading Rate 0.96 Mathematics 0.96 Broad Mathematics 0.97 Math Calculation Skills 0.97 Math Problem Solving 0.95 Written Language 0.94 Broad Written Language 0.95 Basic Writing Skills 0.95 Written Expression 0.92 Academic Skills 0.97 Academic Applications 0.96 Academic Fluency 0.97 Academic Knowledge 0.95 Phoneme-Grapheme Knowledge 0.94 Brief Achievement 0.97 Broad Achievement 0.99
KTEA-II KTEA-II KTEA-II KTEA-II KTEA-II WJ IV Measures Oral. Lang. Rdg. Math Wr. Lang Comp. Ach.
Oral Language Clusters Oral Language 0.64 Oral Comprehension 0.74 Oral Expression 0.64 Listening Comprehension 0.74
Reading Achievement Clusters Reading 0.94 Broad Reading 0.92 Basic Reading Skills 0.93 Reading Comprehension 0.85 Reading Fluency 0.86 Reading Speed 0.78
Math Achievement Clusters Mathematics 0.94 Broad Mathematics 0.91 Math Calculation 0.83 Math Problem Solving 0.82
Written Language ACH Clusters Written Language 0.81 Broad Writing 0.84 Basic Writing Skills 0.91 Written Expression 0.65
Cross Domain ACH Clusters Brief Achievement 0.88 Broad Achievement 0.91
WJ IV ACH OL / KTEA-II composite select score correlations Ages 8-12 (n = 49)
KTEA-II KTEA-II KTEA-II KTEA-II KTEA-II Oral Wr. Comp.
WJ IV Measures Lang. Rdg. Math Lang Ach. Oral Language Clusters Oral Language 0.74 Oral Expression 0.64 Listening Comprehension 0.58
Reading Achievement Clusters Reading 0.83 Broad Reading 0.78 Basic Reading Skills 0.81 Reading Comprehension 0.68 Reading Fluency 0.79 Reading Speed 0.65
Math Achievement Clusters Math 0.87 Broad Math 0.86 Math Calculation 0.82 Math Problem Solving 0.69
Written Language ACH Clusters Written Language 0.87 Broad Writing 0.85 Basic Writing Skills 0.89 Written Expression 0.66
Cross Domain ACH Clusters Brief Achievement 0.91 Broad Achievement 0.88
WJ IV ACH OL / KTEA-II composite select score correlations Ages 13-18 (n = 50)
WJ IV ACH OL /WIAT-III composite select score correlations– Grades 1-8 (n=51)
WIAT-III WIAT-III WIAT-III WIAT-III WIAT-III WIAT-III WIAT-III WIAT-III Oral Total Basic Rdg. Cmp. Math Wr. Total
WJ IV Measures Lang. Rdg. Rdg. & Flu. Math. Flu. Expr. Ach. Oral Language Clusters Oral Language 0.82 Oral Expression 0.57 Listening Comprehension 0.60
Reading Achievement Clusters Reading 0.93 0.92 0.82 Broad Reading 0.89 0.87 0.82 Basic Reading Skills 0.93 0.94 0.78 Reading Comprehension 0.88 0.85 0.81 Reading Fluency 0.87 0.84 0.80 Reading Speed 0.67 0.62 0.64
Math Achievement Clusters Math 0.92 0.57 Broad Math 0.90 0.79 Math Calculation 0.82 0.85 Math Problem Solving 0.83 0.53
Written Language ACH Clusters Written Language 0.78 Broad Writing 0.77 Basic Writing Skills 0.79 Written Expression 0.62
Cross Domain ACH Clusters Brief Achievement 0.93 Broad Achievement 0.90
Bold font designates correlations >= .70
WJ IV ACH OL /WIAT-III composite select score correlations—Grades 9-12 (n=49)
WIAT-III WIAT-III WIAT-III WIAT-III WIAT-III WIAT-III WIAT-III WIAT-III Oral Total Basic Rdg. Cmp. Math Wr. Total
WJ IV Measures Lang. Rdg. Rdg. & Flu. Math. Flu. Expr. Ach. Oral Language Clusters Oral Language 0.79 Oral Expression 0.68 Listening Comprehension 0.75
Reading Achievement Clusters Reading 0.78 0.75 0.74 Broad Reading 0.79 0.72 0.80 Basic Reading Skills 0.86 0.86 0.74 Reading Comprehension 0.54 0.50 0.60 Reading Fluency 0.78 0.72 0.77 Reading Speed 0.66 0.61 0.72
Math Achievement Clusters Math 0.84 0.42 Broad Math 0.76 0.68 Math Calculation 0.58 0.72 Math Problem Solving 0.79 0.45
Written Language ACH Clusters Written Language 0.73 Broad Writing 0.76 Basic Writing Skills 0.77 Written Expression 0.52
Cross Domain ACH Clusters Brief Achievement 0.85 Broad Achievement 0.85
Note. font designates correlations >= .70
Contemporary CHC (Schneider & McGrew, 2012) broad and narrow ability content coverage by WJ-IV Oral Language
General Intelligence (g)
Mathematical achievement
(A3)
Reading decoding (RD)
Reading comprehension
(RC)
Reading speed (RS)
Spelling ability (SG)
English usage (EU)
Writing ability (WA)
Writing speed (WS)
General verbal information (K0)
Lexical knowledge (VL)
Listening ability (LS)
Induction (I)
General sequential
reasoning (RG)
Quantitative reasoning (RQ)
Memory span (MS)
Working memory capacity (WM)
Associative memory (MA)
Meaningful memory (MM)
Speed of Lexical access (LA)
Naming facility (NA)
Visualization (Vz)
Visual Memory (MV)
Spatial Scanning (SS)
Phonetic coding (PC)
Quantitative Knowledge
(Gq)
Reading & Writing (Grw)
Comp -Knowledge
(Gc)
Fluid Reasoning (Gf)
Long-Term Retrieval (Glr)
Visual Processing (Gv)
Auditory Processing
(Ga)
Processing Speed (Gs)
Perceptual speed (P)
Number Facility (N)
General science info. (K1)
Knowledge of culture (K2)
Domain-Specific Knw.
(Gkn)
Geography ach. (A5)
Shading designates abilities measured by WJ IV OL battery
Language development
(LD)
Attentional Control (AC)
Memory for Sound Patterns
(UM)
Word Fluency (FW)
Verbal (print) lang. comp. (V)
ST Working Mem (Gwm)
© Institute for Applied Psychometrics; Kevin McGrew 01-23-14
Median reliabilities (across all technical age groups) for
WJ IV OL clusters
Oral Language 0.90 Broad Oral Language 0.92 Oral Expression 0.89 Listening Comprehension 0.90 Phonetic Coding 0.95 Speed of Lexical Access 0.89 Vocabulary 0.93
CELF-4 CELF-4 CELF-4 CELF-4 CELF-4 CELF-4 PPVT-4
WJ IV Measures Core Rec. Expr. Lang. Lang. Work. Lang. Lang Lang. Con. Struc. Mem.
Oral Language Clusters Oral Language 0.63 0.59 0.64 0.67 0.67 0.50 0.74 Oral Expression 0.74 0.64 0.72 0.67 0.73 0.58 0.70 Listening Comprehension 0.64 0.62 0.63 0.63 0.68 0.59 0.69 Speed of Lexical Access 0.31 0.37 0.32 0.35 0.38 0.57 0.43
Bold font correlations > = .60
WJ IV OL / CELF-4 and PPVT-4 composite select score
correlations—Ages 5-8 (n = 50)
CELF-4 CELF-4 CELF-4 CELF-4 CELF-4 CELF-4 PPVT-4
WJ IV Measures Core Rec. Expr. Lang. Lang. Work. Lang. Lang Lang. Con. Struc. Mem.
Oral Language Clusters Oral Language 0.75 0.46 0.69 0.49 0.65 0.17 0.76 Oral Expression 0.83 0.56 0.79 0.49 0.74 0.33 0.62 Listening Comprehension 0.76 0.50 0.68 0.42 0.69 0.30 0.55 Speed of Lexical Access 0.42 0.19 0.36 0.18 0.39 0.23 0.14
. Bold font correlations > = .60
WJ IV OL / CELF-4 and PPVT-4 composite select score correlations—Ages 10-18 (n = 56)
CASL
OWLS
Core Basic Sent. Syntax Parag. Prag. Oral List. Oral
WJ IV Measures Comp. Conc. Comp Constr Comp Judge Comp Comp Exp Oral Language Clusters
Oral Language 0.60 0.56 0.57 0.47 0.41 0.49 0.60 0.42 0.61 Oral Expression 0.48 0.43 0.51 0.42 0.41 0.31 0.46 0.31 0.49 Listening Comprehension
0.58 0.60 0.53 0.49 0.44 0.46 0.57 0.44 0.55
Speed of Lexical Access
0.48 0.41 0.36 0.43 0.36 0.40 0.24 0.07 0.33
Note. n = 50 for all measures except WJ IV Phonological Processing (n = 48). All correlations with WJ IV Phonological Processing have n = 48 sample size.
WJ IV OL / CASL & OWLS composite score correlations in ages 3-6 sample
WJ IV Oral Language Clusters Oral Oral Oral Listening Speed of
CASL/OWLS Measures Language Comp. Expression Comp. Lex. Acc. CASL Measures
Core Composite 0.85 0.82 0.72 0.76 0.57 Antonyms 0.82 0.78 0.64 0.75 0.39 Synonyms 0.71 0.73 0.55 0.70 0.32 Sentence Completion 0.76 0.73 0.66 0.68 0.51 Idiomatic Language 0.82 0.80 0.75 0.72 0.42 Syntax Construction 0.71 0.68 0.57 0.64 0.47 Paragraph Comprehension 0.66 0.70 0.54 0.75 0.56 Grammitcal Morphemes 0.63 0.60 0.53 0.56 0.47 Sentence Comprehension 0.59 0.60 0.58 0.57 0.47 Grammaticality Judgement 0.67 0.65 0.70 0.58 0.57 Nonliteral Language 0.67 0.63 0.64 0.54 0.40 Meaning from Context 0.77 0.74 0.63 0.67 0.56 Inference 0.70 0.70 0.62 0.67 0.42 Amibgious Sentences 0.72 0.73 0.58 0.70 0.45 Pragmatic Judgement 0.52 0.47 0.34 0.42 0.48
OWLS Measures Oral Composite 0.68 0.68 0.62 0.64 0.41 Listening Comprehension 0.53 0.57 0.50 0.56 0.25 Oral Expression 0.67 0.64 0.60 0.59 0.45
Bold font are correlations >= .70
WJ IV OL / CASL & OWLS score correlations in ages 7-17 sample (n = 50)