Opportunities and Challenges for Diagnosing Teachers’ Multiplicative
Reasoning
NSF DR-K12 PI Meeting
December 3, 2010
Andrew Izsák
University of Georgia
Diagnosing Teachers’ Multiplicative Reasoning
Andrew Izsák, Jonathan Templin, Allan Cohen
University of Georgia
Joanne Lobato
San Diego State University
Chandra Orrill
University of Massachusetts Dartmouth
Supported by the National Science Foundation under Grant No. DRL-0903411. The opinions expressed are those of the authors and do not necessarily reflect the views of NSF.
Existing Approaches to Assessing Teacher Knowledge
• Composite measures– Count college mathematics courses teachers
completed– Use item response theory to measure
Mathematical Knowledge for Teaching• Case studies
– Investigate topics such as subtraction with regrouping, arithmetic with fractions, and functions
Essential Features of DTMR• Assessing mathematical knowledge of middle grades
teachers• Emphasize knowledge needed for teaching:
– multiplication and division of fractions and decimals– proportional reasoning – using problem situations and drawn models to develop general
methods
• Combine mathematics education research with psychometric research on an emerging class of models called Diagnostic Classification Models (DCMs)
• One of the first projects to develop a test for DCMs using STEM education research
Opportunities• Curricular trends
– Reform-oriented curricula– Common Core Standards
• Need measures suitable for tracking growth and change in PD• Large literature
– Multiple components of reasoning – Glean attributes
• DCMs are multi-dimensional models that (compared to MIRT) can be reliably estimated with smaller samples and shorter tests
• Expand range of psychometric models applied to STEM education research
Fraction Attributes
A
B
1
B
• Referent Units– Understand units to which numbers refer
• Partitioning– Using whole-number multiplication to guide
partitioning
• Iterating– Interpret to mean A copies of
• Appropriateness– Identifying multiplication and division situations
Example: Referent Unit & Iterating
I. The diagram can show .
II. The diagram can show 1 .
III. The diagram can show .
3
5
2
3
5
2
Which of the following interpretations are sensible?
The Mastery Profile(Fractions)
• DCMs estimate an attribute mastery profile for each teacher:
Estimated Probability of Mastery Referent Units .3 Partitioning .5 Iterating .7 Appropriateness .8
0 0.5 1 Not Mastered Unsure Mastered
Anticipated Uses
• Use as formative assessment to inform PD• Detect growth and change in PD characterized
as increased proficiency with the attributes• Determine distribution of attribute patterns in
large samples of teachers• Examine relationships between attribute
patterns and enactment of curricular materials• Examine relationships between attribute
patterns and student achievement
Challenge: What are Workable Attributes?
• Existing examples from psychometrics – Steps in numeric algorithms– Branches of mathematics
• Our criteria for attributes– Written responses provide reliable information– Separable from one another – Separate teachers
– Cut across topics • Cannot translate research findings directly
– Multiple cycles of using attributes to write items and interviewing teachers
Example: Composed Unit Reasoning
One week Mr. Compton drove to a training course that required him to drive 8/3 the distance he usually drives to work. He noticed that 24 min. had passed when he had drive half way to the course. How long does it take Mr. Compton to drive to work?
Mr. Vargas gave the following task to his students:
One student tried to model the problem using two number lines as shown but is stuck. How could you help the student?
0 min
0 km
24 min
8
3
Challenge: Item Design
• Machine scoreable– Multiple-choice– Constructed response items
• Case studies often rely on observed strategies to make inferences– Composed unit reasoning– Fractions as multiplicative relationships
• Numeric computation should not help find correct response– Teachers’ use of computation obscures access to their
reasoning with quantities
Challenge: Item Design (Cont.)• Attention to pedagogy can drive responses
– Teachers are not always comfortable evaluating students
– How teachers would teach a topic and what they know about a topic are not the same
– Teachers can deflect mathematical issues by appealing to what their students can understand
• Drawings– Teachers do not always interpret diagrams in
ways that we intend
Challenge: Balancing Constraints
• Psychometric modeling and interviewing teachers to investigate constraints that include– Grain-size of attributes
– Item design
– Number of attributes
– Number of items per attribute
– Number of items (test length)
– Sample size
Proportional Reasoning Attributes
• Covariation and Invariance– Multiplicative relationship invariant as quantities co-
vary
• Connections between Ratios and Fractions– Conceptual links between ratios and fractions
• Appropriateness – Direct proportion vs inverse proportion – Direct proportion vs linear relationship
• Multiplicative Object
Summary
• Develop one of the first tests for use with Diagnostic Classification Models (DCMs) based on STEM education research
• Opportunities– Assess select aspects of teachers’ multiplicative reasoning
– Detect growth and change during professional development
• Challenges– Interpret the term “attribute” for mathematics education
research on multiplicative reasoning– Design items that get at fine-grained aspects of
multiplicative reasoning using drawn models
Expanded Fractions Attributes Attribute Sub Attributes
Norming Referent Units for Multiplication
Referent Units
Referent Units for Division Simple Partitioning Partitioning in Stages Partitioning Using Common Denominator
Partitioning
Partitioning Using Common Numerator
Iterating Unit Fractions Identifying Multiplication Situations Identifying Quotitive Division Situations
Appropriateness
Identifying Partitive Division Situations
Expanded Proportional Reasoning Attributes
Attribute Sub-Attributes
Iterating and Partitioning a Composed Unit Consolidating operations on composed units Making multiplicative comparisons within measure spaces
Covariation & Invariance
Making multiplicative comparisons across measure spaces Using composed unit reasoning to reinterpret a ratio as a fraction Using a multiplicative comparison to reinterpret a ratio as a fraction
Connections between Ratios & Fractions
Differentiating fraction and ratio operations
Appropriateness
Rate as an equivalence class of rat ios Meaning for equality in a proportion
Formation of a Multiplicative Object
Ratio-as-measure