Post on 09-Apr-2018
transcript
Using R to Assess Mathematical Sense-Making in Introductory Physics Courses
2010 UseR! Conference - July 21, 2010
by Brian Danielak, Eric Kuo, Michael M. Hull,and Andrew Elby
1
I work behind here
2
3
Students in conventional courses often treat formulas as algorithms.
Sherin, B. L. (2001). How Students Understand Physics Equations. Cognition and Instruction, 19(4), p. 479
4
v = v0 + at
if I know the initial velocity...
The uniform acceleration...
and the time...
Then I can calculate the final velocity!
5
Sense-making is about seeking coherence
v = v0 + at
v = v0 + at
8
v = v0 + at
if I know the initial velocity...
The uniform acceleration...
and the time...
Then I can calculate the final velocity!
v = v0 + at
v = v0 + at+=t1
9
v = v0 + at
if I know the initial velocity...
The uniform acceleration...
and the time...
Then I can calculate the final velocity!
v = v0 + at
v = v0 + at+
+
=
=
t1
t2
10
v = v0 + at
if I know the initial velocity...
The uniform acceleration...
and the time...
Then I can calculate the final velocity!
v = v0 + at
v = v0 + at+
+
+
=
=
=
t1
t2
t3
11
The same mathematical structuremodels lots of other situations
12
+
+
+
=
=
=
t1
t2
t3
money = P + rt
13
+
+
+
=
=
=
x1
x2
x3
Y = β0 + β1X1
So here’s the bigQuestion
14
15
15
v = v0 + at
if I know the initial velocity...
The uniform acceleration...
and the time...
Then I can calculate the final velocity!
v = v0 + at
v = v0 + at+
+
+
=
=
=
t1
t2
t3
10
v = v0 + at
if I know the initial velocity...
The uniform acceleration...
and the time...
Then I can calculate the final velocity!
How can we test for thinking like this
When we usually test for thinking like this?
16
A Tale of 3 Instructors
1 32New Veteran Veteran
Sense-making
Traditional Approach
And one Final Exam
17
Multiple Choice Free Response
And one Final Exam
18
Multiple Choice Free Response
426 Students
3 Classes
2
1
6
Sense-makingitems
19
5
3
4
chosen a priori7
8
Class 1 Class 2 Class 3
3 4 2
e-score
20
21
Looking graphicallyat the data
22
Raw Score Distributions by Instructor
rawscore
coun
t
0
5
10
15
20
25
30
35
0
5
10
15
20
25
30
35
0
5
10
15
20
25
30
35
0 2 4 6 8
12
3
Sense-making(new)
Sense-making(vet)
Traditional(vet)
Across all 8 MC items
23
Escore Distributions by Instructor
escore
coun
t
0
10
20
30
40
0
10
20
30
40
0
10
20
30
40
−1 0 1 2 3 4 5 6
12
3
Sense-making(new)
Sense-making(vet)
Traditional(vet)
Just the 5 Sense-making Items
We ran a one-way ANOVA, then MCPs
24
Result 1
25
Across all 8 Multiple Choice items,
Sense-making > traditional approach
26
−0.15 −0.10 −0.05 0.00 0.05 0.10
3−2
3−1
2−1
95% family−wise confidence level
Differences in mean levels of instructorRawratio
27
- - -
TukeyHSD
p-adj
Cohen’s d
0.061 -0.019 -0.080
0.053 0.751 0.007
0.277 -0.0839 -0.365
Across all 8 MC items95% Family-wise Confidence Level
1 32 1 3 2
Result 2
28
Across the 5 specific sense-making items,
Sense-making >> traditional approach
29
−0.20 −0.15 −0.10 −0.05 0.00 0.05 0.10
3−2
3−1
2−1
95% family−wise confidence level
Differences in mean levels of instructorEratio
30
- - -
TukeyHSD
p-adj
Cohen’s d
0.0450 -0.0871 -0.132
0.296 0.0130 < 0.001
0.182 -0.331 -0.510
On Sense-making MCs95% Family-wise Confidence Level
1 32 1 3 2
31
Exploring Free-Response Data
?
32
What happens to mechanical energyafter the explosion?
What Happens to the Mechanical Energy of the Two Blocks?
instructor
coun
t
0
20
40
60
80
100
120
140
1 2 3
q09dstays the sameincreasesblankdecreasesmissing valueuncodeable
33
What Happens to the Mechanical Energy of the Two Blocks?
instructor
coun
t
0
20
40
60
80
100
120
140
1 2 3
q09dstays the sameincreasesblankdecreasesmissing valueuncodeable
34
What Happens to the Mechanical Energy of the Two Blocks?
instructor
coun
t
0
20
40
60
80
100
120
140
1 2 3
q09dstays the sameincreasesblankdecreasesmissing valueuncodeable
43 4372
35
What Happens to the Mechanical Energy of the Two Blocks?
instructor
coun
t
0
20
40
60
80
100
120
140
1 2 3
q09dstays the sameincreasesblankdecreasesmissing valueuncodeable
43 4372
Mechanical EnergyStays the Same
36
43 4372
Mechanical EnergyStays the Same
=
37
Sense-making(new)
Sense-making(vet)
Traditional(vet)
Mechanical EnergyStays the Same
=
38
43 4372
Sense-making(new)
Sense-making(vet)
Traditional(vet)
Result 3
39
When considering energy, more traditional students asserted something counter-intuitive
43 4372
Sense-making(new)
Sense-making(vet)
Traditional(vet)
What Happens to the Mechanical Energy of the Two Blocks?
instructor
coun
t
0
20
40
60
80
100
120
140
1 2 3
q09dstays the sameincreasesblankdecreasesmissing valueuncodeable
40
5975
33
Mechanical EnergyIncreases
41
5975
33
Mechanical EnergyIncreases
42
5975
33
Mechanical EnergyIncreases
43
>
5975
33
Mechanical EnergyIncreases
44
Sense-making(new)
Sense-making(vet)
Traditional(vet)
>
5975
33
Mechanical EnergyIncreases
45
Sense-making(new)
Sense-making(vet)
Traditional(vet)
Result 4
46
When considering energy, more sense-making students were consistent with simple intuition.
5975
33
Sense-making(new)
Sense-making(vet)
Traditional(vet)
Ahead,
DRAGONS!
http://www.flickr.com/photos/aztlek/2421800480/in/set-72157605762980035/
!ar b"...
47
http://www.flickr.com/photos/aztlek/2421800480/in/set-72157605762980035/
This stuff ’sin beta.
48
Students Who Quoted ‘‘Conservation of Energy'' as their #§*Explanation for q09d
instructor
coun
t
0
5
10
15
20
25
30
1 2 3
variable55x
49
811
32
Mechanical EnergyStays the Same...
50
811
32
Mechanical EnergyStays the Same...
Because Energyis Conserved
51
Sense-making(new)
Sense-making(vet)
Traditional(vet)
Result 5
52
When justifying their answer, more traditional students just quoted a rule, ignoring that it was tautologicaland inaccurate.
811
32
Sense-making(new)
Sense-making(vet)
Traditional(vet)
Summary
53
54
Result 3
39
When considering energy, more traditional students asserted something counter-intuitive
43 4372
Sense-making(new)
Sense-making(vet)
Traditional(vet)
Result 4
46
When considering energy, more sense-making students were consistent with simple intuition.
5975
33
Sense-making(new)
Sense-making(vet)
Traditional(vet)
Result 5
52
When justifying their answer, more traditional students just quoted a rule, ignoring that it was tautologicaland inaccurate.
811
32
Sense-making(new)
Sense-making(vet)
Traditional(vet)
29
−0.20 −0.15 −0.10 −0.05 0.00 0.05 0.10
3−2
3−1
2−1
95% family−wise confidence level
Differences in mean levels of instructorEratio
55
http://github.com/briandk/user2010public
EricKuo
MikeHull
AyushGupta
AndyElby
Project Directors
Graduate Assistants
Thanks!
56
Many Thanks
NSF-DRL grant # 0733613NSF-EEC grant # 0835880
57
58
Result 3
39
When considering energy, more traditional students asserted something counter-intuitive
43 4372
Sense-making(new)
Sense-making(vet)
Traditional(vet)
Result 4
46
When considering energy, more sense-making students were consistent with simple intuition.
5975
33
Sense-making(new)
Sense-making(vet)
Traditional(vet)
Result 5
52
When justifying their answer, more traditional students just quoted a rule, ignoring that it was tautologicaland inaccurate.
811
32
Sense-making(new)
Sense-making(vet)
Traditional(vet)
29
−0.20 −0.15 −0.10 −0.05 0.00 0.05 0.10
3−2
3−1
2−1
95% family−wise confidence level
Differences in mean levels of instructorEratio