What is a successful learning process in a MOOC?Wil van der Aalst, Joos Buijs, Hanneke Duisterwinkel, Maikel Leemans
TU/e @ MOOC’s
• Partner of Coursera since 2013• Showing the excellence of TU/e • Emphasize niche• International brand awareness• Partnership with Eurotech• Experimenting with new way’s of teaching to
improve our on-campus education
PAGE 215-04-2023
15-04-2023
Projectteam MOOCs
Professor
Information Expertise Center
Communication Expertise Center Video production Educational
support ICT Services Data expertise
PHD Projectleader Support
PAGE 3
4
Statistics (first run)• 42.146 enrollments• 821 signature tracks• 178 different countries• 671.927 videos watched• 36.751 quizzes submitted• 4,415 forum post made
Statistics• 16.091 enrollments• 164 signature tracks• 153 different countries• 204.843 videos watched• 12.083 quizzes
submitted• 1.031 forum post made
New MOOC projects:
• Eurotech MOOC • One MOOC on Energy Efficient Building and
Communities developed by: DTU, TUM, EPFL and TU/e
• EIT ICT First Year online master Embedded Systems
• Six specializations in total • Two specialization from TU/e
PAGE 515-04-2023
15-04-2023
Our ‘process model’: structure!
Nov 12 Nov 19 Nov 26 Dec 3 Dec 10 Dec 17 Dec 24 Dec 31 Jan 7
Week 1
Week 2
Week 3
Week 4
Week 5
Week 6
Final Quiz
Tool Quiz
Peer Ass.
Peer Review
PAGE 8
A week: ~8 videos of ~15 min. each
+ a week quiz
For ‘distinction’
15-04-2023
General Statistics
PAGE 9
Registered Visited Watched a lecture Browsed forums Submitted exercise Obtained certificate
43218
29209
20868
5845 5798
1688
68% 71% 28%
29%
15-04-2023
Annotate students
• Achievement:• Failed• Successful
− Normal− Distinction
• Signature track:• Yes• No
• Lectures watched• Week 1 only• Week 1&2 only• Week 1,2 and 3• All weeks• Any (e.g. at least 1)
PAGE 11
15-04-2023
Basic Group Statistics
Signature Track?
Certificate #Students #Watched Videos
Avg. per student
Yes Failed 226 8,069 36
Normal 223 15,367 69
Distinction 285 21,392 75
No Failed 13,465 143,721 11
Normal 702 41,467 59
Distinction 293 19,203 66
PAGE 12
Heterogeneous group:- 10,996 watched lecture from week 1
(3,451 week 1+2, 1,407 1+2+3)- Only 269 failed students watched a video
of each week
15-04-2023
Signature Track vs Non-Sign.
PAGE 13
Certificate Weeks Watched
#Students Avg. videos per student
Sign. Non-S Sign. Non-S
failed all weeks 39 269 79 69
failed any 226 13,465 36 11
successful all weeks 354 628 92 83
successful any 508 995 72 61
normal all weeks 158 427 87 82
normal any 223 702 69 59
distinction all weeks 196 201 97 86
distinction any 285 293 75 66
Signature track always watches
more videos than non-signature.
Failed students watch many, but less videos than
successful students
15-04-2023 PAGE 15
DistinctionNormal
Signature
Non-Signature
Failed
~PreviousSlide
Seem togive up
More intensewatching of weeks 5 & 6
Followschedule
15-04-2023
Do Successful Students Study Sequentially?
PAGE 17
Which group studies most ‘sequentially’?
Signature yes/no?
Grading failed/normal/distinction?
Watched weeks 1-2-3/all/any?
signature grading weeks % matchno failed all weeks 0.660 no normal all weeks 0.634 no distinction all weeks 0.624 yes normal all weeks 0.593 no distinction any 0.592 no normal any 0.591 yes failed all weeks 0.573 yes normal any 0.571 no failed week 1,2,3 0.548 yes distinction all weeks 0.547 yes distinction any 0.534 no failed week 1,2 0.466
15-04-2023
Behavior of Non-Sign. Failed Students that Watched all Weeks
PAGE 18
Week 1
Week 6
Each lecture watched 10-20 times by 269 students
overall
15-04-2023
But how do Successful Students Watch?
PAGE 19(285 Students on signature track that got distinction)
Intro & Week 1
Week 2 & 3 Week 4, 5 & 6
15-04-2023 PAGE 21
Towards deadlines of final quiz and peer assignment watch behavior (far) less structured
Weeks 5 & 6 very unstructured
15-04-2023
Behavior Compared
PAGE 22
Normal Cert. Cert. with Distinction
Signature Track Sequence but every lecture can be skipped.Week 6 is unordered.(223 students)
Week 2-4 structured,Week 1 unorderedWeek 6 very unstructured and possible repetitive(285 students)
Non-Signature Track Sequence, even week 6.Every lecture can be skipped.(702 students)
Sequence, but every lecture can be skipped.Week 6 is very unstructured and repetitive(293 students)
15-04-2023
Behavior Compared
PAGE 23
Normal Cert. Cert. with Distinction
Signature Track Rather structured. Some lectures in different orders.Difficult lecture watched multiple times.Week 6 unordered.(158 students)
Sequence for some weeks without skips.Week 1 unordered.Difficult lectures repeated.Weeks 4 and 5 unstructured.(196 students)
Non-Signature Track Very sequential(427 students)
Very sequential(201 students)
For students that watched at least 1 Lecture of each week
15-04-2023
Overall Conclusion
1. Successful students do not necessarily study structurally
• Distinction and/or signature students have significant preliminary knowledge?!?!
2. Structured studying does not guarantee success
3. Signature track students aim for a certificate while non-signature track students aim for knowledge?
PAGE 24
15-04-2023
Future Analysis
• Include more information• Quizzes (and their results)• Analyse timing of watching and quiz making• Forum interaction
• Compare runs of the same MOOC• Who re-joined?• Is the 2nd group of students different? (gut feeling: yes!)
PAGE 25
http://www.edsa-project.eu/