Getting Students To Do the WorkUsing Learner Analytics Data to Inform Instructional StrategiesNew Directions in Online Learning San Diego, 2014 David Lindrum
WEBTEXT
Study Questions vs. Assignments
• 33 Audio / Video Selections
• 34 Articles and Studies
• 8 Infographics, Websites, etc.
Data Collected From Student Activities
Log-In
- first enrollment - each log-in time - most recent date
Active Time
Time spent on each page typing, clicking, or scrolling.
Participation
- pages viewed - questions answered - number of resets
Scoring
- on multiple choice - first attempt - most recent attempt
Higher Order
- assignments - short answer - response boards
About Our DataIn two studies, assignment scores (non-resettable, application-type questions) are significant predictors of midterm and final scores. Other webtext activities are also correlated with course success, including minutes per question and question completion.
In multiple online courses (over 8,000 students), webtext success pass closely matches course pass rate.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Webtext Completion*PSY Pilot Instructor 1
*N=35 students; excludes 5 students with 0 time.
Percentage Complete
median✦ 21% webtext
completion ✦ 102 questions
answered ✦ 8.8 hours
active time ✦ 6.2 minutes
per correct question
PSYSummer 2012
median
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Webtext Completion*
*N=46 students; excludes 1 student with 0 time.
median✦ 99% webtext
completion ✦ 475 questions
answered ✦ 16.4 hours
active time ✦ 2.9 minutes
per correct question
PSYFall 2012
PSY Pilot Instructor 2
median
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Webtext Completion*
*N=46 students; excludes 1 student with 0 time.
median✦ 99% webtext
completion ✦ 475 questions
answered ✦ 16.4 hours
active time ✦ 2.9 minutes
per correct question
PSYFall 2012
PSY Pilot Instructor 2
median
30% Course Credit for Webtext Assignments and Study Questions
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Webtext Completion*PSY Pilot Instructor 1
*N=35 students; excludes 5 students with 0 time.
Percentage Complete
median✦ 21% webtext
completion ✦ 102 questions
answered ✦ 8.8 hours
active time ✦ 6.2 minutes
per correct question
PSYSummer 2012
median
0% Course Credit for Webtext Assignments and Study Questions
Students don’t do optional
Assign credit for formative activities that will prepare students to do well on final.
Credit should generally be between 20-30% of course credit.
What We Learned
Students don’t do optional
Assign credit for formative activities that will prepare students to do well on final.
Credit should generally be between 20-30% of course credit.
Students respond to personal outreach (nagging, parenting)
Instructor 2 emailed students at the end of every week
Go get your learning resource. You can’t pass this class without it.
I see that you’re falling behind. If you don’t catch up, you won’t pass this class.
What We Learned
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Webtext Completion*PSY Pilot Instructor 1
*N=35 students; excludes 5 students with 0 time.
Percentage Complete
median✦ 21% webtext
completion ✦ 102 questions
answered ✦ 8.8 hours
active time ✦ 6.2 minutes
per correct question
PSYSummer 2012
median
What exactly are successful-student behaviors?
Assignments
Study Questions
Polls
Your Turns
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
A’s B’s C’s D’s F’s
Median Webtext Completion PercentagesBy Assignment Scores
Fall 2012POL
No credit
n=48 at R1, f2f
“[L]earning techniques such as taking practice tests and spreading study sessions out over time — known as distributed practice — were found to be of high utility because they benefited students of many different ages and ability levels and enhanced performance in many different areas.” Improving Students’ Learning with Effective Learning Techniques (Dunlosky, et al. 2013)
Assignments
Study Questions
0 10 20 30 40 50 60 70 80 90 100
A’s B’s C’s D’s F’s
Average Webtext CompletionBy Final Grade POL
n=57 at R1, f2f
0
10
20
30
40
50
60
70
80
90
100
Ch 1 Ch 2 Ch 3 Ch 4 Ch 5 Ch 6
Median Webtext Scores (Lines)
Fs
Bars represent completion. !Lines represent percentage of points earned per module. !
PSYAs of October 28, 2013
Ds
Cs
Bs
As
n=539
0
10
20
30
40
50
60
70
80
90
100
Ch 1 Ch 2 Ch 3 Ch 4 Ch 5 Ch 6
Median Webtext Scores (Lines) and Completion (Bars)
Fs
Bars represent completion. !Lines represent percentage of points earned per module. !
PSYAs of October 28, 2013
Ds
Cs
Bs
As
n=539
Webtext Hours
0 4.5 9 13.5 18
A’s B’s C’s D’s F’s
Median Active Time in Webtext - HoursBy Webtext ScoresAs of October 28, 2013
ORG
n=1,040 at online
0
0.8
1.6
2.4
3.2
4
0%
20%
40%
60%
80%
100%
As Bs Cs Ds Fs
Assignment Score Assignment Completion Hours
Median Assignment TIme and Scores
As of October 28, 2013HISBy Webtext Scores
n=560 at online
0
50
100
150
200
250
300
350
400
450
500
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
Median Time
Median Correct
Active Time in Minutes
Co
rrec
t R
esp
on
ses
Active Time and Correct Responses All Students, All Questions (large, 2-year)
0
1.25
2.5
3.75
5
0%
20%
40%
60%
80%
100%
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6
Pre-Term Score Pre-Term Hours Week 1 Score Week 1 Hours Week 2+ Score Week 2+ Hours
Median Webtext Scores By Week
As of October 28, 2013PSYWith Median Hours Per Week
n=539 at online