The Role of Item Response AttractorsIn Modified Items for Students with Disabilities

Michael C. RodriguezUniversity of Minnesota

Peter Beddow, Elizabeth Compton, Stephen Elliott, Ryan Kettler, Andrew Roach

CAAVES Team

So what about those distractors?

• Distractors are designed to distract students, hopefully those with less ability.

• Distractors tend to be fillers, to occupy the other three or four options.

• Distractors are sometimes absurd or even humorous.

• “It’s hard to write 3 or 4 good distractors!”• Distractors are usually not attractors.

Item Response Attractors

• Attractors attract students with specific misconceptions or reasoning errors.

• Attractors focus attention on attracting the right students – those with lower ability.

• Attractors require the incorrect options to be plausible, yet not the best answer.

• Attractors are not filler options.

Item Response Attractors

• Attractors contribute to the overall quality of the item and test.

• Attractors play a central role in determining the difficulty of an item.

• Attractors are explicitly designed to inform us about prevailing misconceptions and errors.

• Attractors are explicitly intentional.• Attractors are explicit.

To be Consistent

We are compiling evidence from our own work and the work of others regarding the utility of being direct and explicit in our item writing, relying on good item writing guidelines, elements of Universal Design, concepts from Cognitive Load Theory, and research based on language complexity and accessibility for diverse students.

TAMI provides a systematic guide.

1. Who was elected President of the United States in 1932 during the Great Depression?

A.Daniel BooneB.Dwight D. EisenhowerC.Ronald W. ReaganD.Franklin D. Roosevelt

2. Who was elected President of the United States in 1932 during the Great Depression?

A.Calvin CoolidgeB.Herbert HooverC.Franklin RooseveltD.Theodore Roosevelt

1. What is the most effective method to reduce the internal air temperature of a house in a humid subtropical climate?A. FanB. Evaporative coolerC. Air conditionerD. Dehumidifier

2. What is the most effective way to cool a home in a humid climate?A. Air conditionerB. Evaporative coolerC. Fan

Effects of Item Modification

• Item Difficulty

• Item Discrimination

• Attractor Functioning

Change in Attractor Discrimination

Minimum Maximum Mean SD

Mathematics -.27 .20 .04 .08

Reading -.23 .23 .06 .09

Change in Attractor Response Rates

Minimum Maximum Mean SD

Mathematics -12% 11% -0.3% 4.9%

Reading -35% 22% -1.2% 7.3%

http://education.umn.edu/

Option N % PtbsA 10 4 -0.17B 161 69 0.38C 36 16 -0.23D 25 11 -0.18

Option N % PtbsA [B] 323 68 0.40B [C] 99 21 -0.25C [D] 53 11 -0.27

Option N % PtbsA 10 4 -0.17B 161 69 0.38C 36 16 -0.23D 25 11 -0.18

Option N % PtbsA 139 60 0.37B 24 10 -0.25C 39 17 -0.21D 31 13 -0.08

Option N % PtbsA 323 68 0.35B [C] 88 19 -0.34C [D] 67 14 -0.10

Option N % PtbsA 86 35 0.40B 51 21 -0.28C 53 22 -0.19D 53 22 0.01

Option N % PtbsA 246 53 0.47B 107 23 -0.41C 112 24 -0.14

Option N % PtbsA 58 24 -0.27B 24 10 -0.37C 21 9 -0.23D 136 57 0.59

Option N % PtbsA [A] 46 10 -0.46B [C] 35 8 -0.36C [D] 377 82 0.61

Option N % PtbsA 66 30 -0.22B 111 50 0.49C 22 10 -0.25D 24 11 -0.24

Option N % PtbsA [B] 349 74 0.61B [C] 67 14 -0.42C [D] 57 12 -0.38

Option N % PtbsA 131 58 -0.31B 30 13 -0.10C 53 24 0.48D 10 4 -0.07

Option N % PtbsA [A] 123 26 -0.38B [B] 85 18 -0.23C [C] 265 56 0.51

Option N % PtbsA 85 38 0.23B 69 31 -0.26C 40 18 0.21D 30 13 -0.21

Option N % PtbsA [A] 111 24 -0.03B [B] 272 58 -0.03C [C] 89 19 0.07

Attention to Attractors

• What makes this option an attractor?• Is this an effective attractor?• How many attractors are needed?• How can we improve the effectiveness of the

item attractors?• Does a focus on attractors rather than

distractors matter? Does it help us make items and tests more widely accessible?