Improving on the Recitation Section:
Tutorials in Introductory Physics
Wed Brown BagSJP Fa '05
What are Tutorials?
• Research-based• Student-centered• “Elicit-confront-resolve”• Process (discussion, consensus) + Sense-making
The Tutorial space (basement, across from Jerry Leigh)
Tutorials at CU• Used in 1110 & 1120, twice each (+ 1120, now)
• Part of successful courses => motivation to continue
• Today: some data Continue support?
Resources
• Space, material, training, personnel Ongoing support necessary.
• “Learning Assistants” (LA’s) - undergrad STEM majors (interested in teaching)
Courses studied so far Recitation Homework Text:
• Phys 1110 – Fa 03 Tutorials CAPA HRW– Sp 04 Tutorials CAPA HRW– Fa 04 Knight workbook MP Knight– Sp 05 Trad recitations MP Knight
• Phys 1120–Fa 04 Tutorials CAPA HRW–Sp 05 Tutorials CAPA Knight
pre/post measures
• Phys 1110: FMCE• Phys 1120: BEMA• common exam questions• formal + informal survey questions
Phys 1110: Distinguishing features
1: “Tut” (Sp04) Tutorials + CAPA
2: “Workbook” (Fa04) Small groups/Knight (+ Mastering Physics)
3: “(More) Trad” (Sp05) Mostly traditional recit’s (Otherwise much like "2" above)
(different instructors, semesters …)
1110 summary - up front!• Tutorial courses : strongest results on
learning gains and attitude surveys
• Middle course (“2”) (research-based text and hw, clickers + small-group recitations, not Tutorials): good results.
• Last course (like “2” except recitations): gains solid, lowest of three.
Phys 1110 PretestFMCE (Force and Motion concept evaluation)
A nationally validated conceptual exam, like the FCI (but harder)Comparing FMCE pretest
0
510
1520
2530
35
40
0 9 18 27 36 45 55 64 73 82 91 100
Score (%)
% of students
1PreS04 (28% +/-1) 2PreF04 (34%) 3PreS05 (28% )
Matched, valid data only. (Eliminate pre>93%) Ave
Pre ave = 28%(1) / 34%(2) / 28%(3) Spring (2) higher (fall term)
Phys 1110 Posttest
Comparing FMCE posttest
0
510
15
20
2530
35
40
0 9 18 27 36 45 55 64 73 82 91 100Score (%)
% of students
1PreS04 (74% +/-1) 2PreF04 (69%) 3PreS05 (58% +/- 2%)
Ave
Post ave = 74%(1) / 69%(2) / 58%(3)
Phys 1110 normalized gains
gain(1)= .66 +/-.02
distribution of g: compared
05
1015202530
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1normalized gain
% of students
SP04
Phys 1110 normalized gains
gain(1)= .66 +/-.02 g(2)= .585 +/-.02 7.5 points lower => (more than half a letter grade)
distribution of g: compared
05
1015202530
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1normalized gain
% of students
SP04FA04
Phys 1110 normalized gains
gain(1)= .66 +/-.02 g(2)= .585 +/-.02 g(3)= .45 +/-.02(trad recit.) => significantly lower gains. (still, double nat’l standards!)
distribution of g: compared
05
1015202530
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1normalized gain
% of students
SP04SP05
Impact on different pretest populations:"low starters" pretest <=12.5%
normalized gain for low pretest
0
0.1
0.2
0.3
0.4
0.5
0.6
S04 (23%) F04 (20%) S05 (22%)Semester (% of class in this
g
pretest <= 12.5%
(% of class in this pool)
Course (1) (2) (3)
Impact on different pretest populations:"high starters" 50<pre<93%
normalized gain for high pretest
0
0.1
0.20.3
0.4
0.5
0.60.7
0.8
0.9
S04 (13%) F04 (22%) S05 (14%)Semester (% of class in this
g
pretest >50%
(% of class in this pool)
Course (1) (2) (3)
Beyond the FMCE: Exam comparisons
• #2 (Knight workbooks/small groups): 34 common exam q’s
• #3 (Trad recitations): 30 common q’s
(17 are “tutor. materials”, 9 are “quant/trad”)
(12 are “tutor. materials” 6 are “quant/trad”)
Beyond the FMCE: Exam comparisons( Tutorial score - other course score)
All q’ Tut mater. quant/trad(2) Workbook: +6 +7 +6(3) Trad: +10 +14 +10
(All ± 2) N.B. 12 points is roughly 1 letter grade.
Tutorial courses: significantly better exam scores:both conceptual and quantitative/traditional.
Other data• Replication study (compare with UW)• CLASS (attitudes/beliefs)• Surveys (did you like, did it help?)
All favorable (or neutral) for tutorials, ask for details!
1110 Summary• Compare Tut-based with “workbook/small group” measurable diffs (FMCE, exams, CLASS)
• Compare Tut-based with “more trad rec” significant diffs.
• Tutorials only one effect. (Instructors, course structure …!?) • But in 1120, changing instructors + text => no impact
Phys 1120:
Tut1 (Fa04) and Tut2 (Sp05)Attempt at replication.Main differences: Tut2 has… • different instructor• different semester• different textbook• follows up 1110 without Tutorials• no long answer on exams
Summary (up front!)
• Despite changes in course elements, we replicated Tutorials + basic course structure.
• Result: identical (high) learning gains. (Final score matches our juniors.
Hard exam!)
1120 BEMA pre/post
BEMA = “Brief E&M Assessment”, F04 (N=319) Pretest ave 26%
BEMA (matched) (CU scoring) Fa04
02468
101214161820
0 6 12 18 24 30 36 42 48 55 61 67 73 79 85 91 97Score (%) (CU scoring)
% of students
PreF04
1120 BEMA pre/post
F04 (N=319) Pretest: 26% S05 (N=232): 27%
BEMA (matched) (CU scoring) Compare Fa04 and Sp05
02468
101214161820
0 6 12 18 24 30 36 42 48 55 61 67 73 79 85 91 97Score (%) (CU scoring)
% of students
PreF04 PreS05
1120 BEMA pre/post
F04 (N=319) 26% -> 59%, S05 (N=232) 27% -> 59%g(ave, F04) = .44+/- .01 g(ave, S05)=.43+/- .01
BEMA (matched) (CU scoring) Compare Fa04 and Sp05
02468
101214161820
0 6 12 18 24 30 36 42 48 55 61 67 73 79 85 91 97Score (%) (CU scoring)
% of students
PostF04 PostS05
1120 E&M pre/postBEMA (matched) (CU scoring) Fa04
g(ave)=.44, N=331
05
101520253035404550
0 6 12 18 24 30 36 42 48 54 60 66 72 78 84 90 96Score (%) (CU scoring)
# of students
Pre (ave=26)Post (ave=59)
LA pre
LA post
TA post
TA pre
CU upp div post
1120 post
NCSU tradpost
NCSU honor& CMU tradpost
CMU M&Ipost
CMU uppdiv post
Other data
• Common exam questions (no change)• Replication study (compare with UW)• CLASS (attitudes/beliefs)• Surveys (did you like, did it help?)
All results similar for both semesters!
Affect: survey results
1120 (Tut2) followed 1110 using Knight “workbook” small-group recitations.
Asked at end:
Which is better, Tut or trad rec? 39% vs 40%Which would you enjoy more? 39% vs 39%
1120 mini-summaryDifferent instructors, text, exam structures,
semesters… same Tutorials and Conceptests: no sig diff on BEMA, CLASS, exams.
Validated survey scores high.
Slight differences in surveys: ~neutral student satisfaction. No disasters, room to improve
Bottom line
Tutorials successful, productive course elements.
• Cost:$1500/LA/semester * (6-8 LA’s per course)• Need 1.5 hr/week training session (TA’s too!)
• Benefits: Some LA’s => K-12 cert. (+ their learning gains very high)• TA’s exposed to research-based learning.
Recommendation
• We should continue implementing Tutorials (and collecting data)
• Need support for LA’s, and training infrastructure
(=> faculty or experienced grad student assigned to teach the TA’s/LA’s)
Questions, Discussion!
THE END
Replication
UW (no Tut)
UW (with Tut)
CU (with Tut)
Atwood: tension
25 50 55
Identify Newton III partners
15 70 70
McDermott et al., AJP 62, 1994
Replication
UW (no Tut)
UW (with Tut)
CU (with Tut)
Atwood: tension
25 50 55
Identify Newton III partners
15 70 70
1110 Overall CLASS
55
75
12 22% unfavorable
% favorable
1-Tut
3-Trad
2-Workbook
“Beliefs” survey: CLASS pre/post
1110 Overall CLASS
0
20
40
60
80
100
0 20 40 60 80 100% unfavorable
% favorable
CLASS pre/post (full scale)
Tutorial-based course: no shiftTwo others: small but statistically significant declines
CLASS pre/post1120 Overall CLASS
60
70
10 15 20% unfavorable
% favorable
Sp05Tut1
Tut2
pre- to post shifts (attitudes and beliefs survey). Slightly negative (!), though it’s usually worse.
CLASS pre/post1120 Overall CLASS
0
20
40
60
80
100
0 20 40 60 80 100% unfavorable
% favorable
(Shown on full 0-100 scales)
Affect: survey results 1110(Likert scale 1= “no” to 5=“yes”)
• Did the Tutorial help you learn Physics in 1110? 3.53 +/- .05 (20% neg, 60% pos)
• Did you like the Tutorials?2.41 +/- .05 (62% neg, 21% pos)
Affect: survey results 1120Likert scale 1-5: “very bad” to “very good”)
• How do you feel about the use of Tutorials in Phys 1120?
Tut1: 3.32 +/- .07Tut2: 2.96 +/- .09
(33% neg, 53% pos)(46% neg, 43% pos)
• How much help was the Tutorial?Tut1: 3.33 +/- .06Tut2: 2.90 +/- .08
Force Concept Inventory (FCI)
R. Hake, ”…A six-thousand-student survey…” AJP 66, 64-74 (‘98).
<g> = post-pre 100-preFa03/Sp04Fa98
red = trad, blue = interactive engagement
FCI at CU
Beyond the BEMA - exam q’s• 31 common exam q’s (13 explicitly “tutorial material”, rest => other topics)
All Tut material only(Fa04-Sp05) -1.9% ± 2% -1.8%
~40%
~75%
* Mazur, E. Peer Instruction, Prentice Hall