24
Combining the formative with the summative - an effective use of online tests Susanne Voelkel School of Life Sciences
Combining the formative with the summative - an effective use of online tests
Susanne Voelkel
School of Life Sciences
Learning …
• Is assessment-driven1
• Improves with “Time on task”2
• Is improved by frequent formative assessments3
1 Gibbs, 2010, 2Chickering & Gamson, 1987, 3Black & Williams, 1998
BUT: Low staff to student ratios make it difficult to introduce regular formative assessment exercises
Advantages of online tests
• Easy to disseminate
• Easy, any time access for students
• Easy to manage (VLE)
• Automatic response
– Independent of class size
– Automatic marking
– Instant “feedback”
The starting point (2008 and before)
• Year 2 theory module
• 70 – 100 students
• 18 lectures in 6 weeks
• Final exam
• No formative assessment
Introduction of voluntary online tests
• A series of online tests
• Self assessment
• No marks
• Feedback:
– Scores
– Answer right or wrong
Module marks
Voluntary online tests
Summative online tests
2-staged online tests
0
10
20
30
40
50
60
70
80
2007 2008 2009 2010 2011
Ave
rage
mar
k (%
)
No online tests
*
Better results through self assessment?
0
10
20
30
40
50
60
70
80
90
yes no
Pe
rce
nt
of
stu
de
nts
re
ceiv
ing
a ce
rtai
n m
ark
Use of self assessment tests
Fail or III
II.1 or I
BUT: only a third of the class completed voluntary exercises!
Making online tests compulsory
• Weekly summative tests (6)
• Worth 20% of module mark
What about collusion?
• Late submission possible
• Students could share out correct results
Making online tests compulsory
• Weekly summative tests (6)
• Worth 20% of module mark
• Feedback
– Initially only scores
– Right/wrong answers released after a week
Module marks
Voluntary online tests
Summative online tests
0
10
20
30
40
50
60
70
80
2007 2008 2009 2010 2011
Ave
rage
mar
k (%
)
No online tests
*
Student views
“It (the online test) makes you concentrate on your work, like actually go over your notes”
“I think the problem was, you got your mark back and you didn’t know what you got wrong”
“Yeah, because then, if the system told you what was wrong you’d actually learn from what you got wrong, whereas now we’ve done the assessment and then it’s like, okay, don’t think about it”
Formative assessment
• Low stake
• Main purpose: feedback
• High quality feedback4
– Prompt
– Specific
– How to “close the gap”
4Nicol and MacFarlane-Dick (2006)
A two-stage approach to allow feedback, enforce completion
and discourage collusion
Formative
Summative
Two-stage online tests
• Weekly two-stage online tests (6)
• Each test has two parts, A and B
– A purely formative
– B summative
How the two-stage online tests work
• Part A
– Does not count towards mark
– Multiple attempts
– Immediate, specific feedback
X
Feedback:
Two-staged online test
• Part A
– Does not count towards mark
– Multiple attempts
– Immediate, specific feedback
– Has to achieve 80% to move to B
• Part B
– Does count towards mark
– Only one attempt
– Has to be completed by deadline
Does it work?
• Student learning
– Improved exam results
– Effect size 0.58
Module marks
Voluntary online tests
Summative online tests
2-staged online tests
0
10
20
30
40
50
60
70
80
2007 2008 2009 2010 2011
Ave
rage
mar
k (%
)
No online tests
*
Does it work?
• Student learning
– Improved exam results
– Effect size 0.58
• Student views
– Students liked test AND feedback
– Students felt they learnt more
82
88
89
14
8
8
4
2
2
Tests help my understanding
Found feedback helpful
Tests make me work more
Agree Neither Disagree
Opinions about the online tests (%)
N = 64 – 74
“The continuous assessment throughout the module meant that I was forced to go over lectures from that week and make sure I understood them”
“I really appreciate the tests. My first attempt is usually quite bad and after repeating the tests several times to get above 80% I find I slowly start to learn the content”
Student comments
Thank you
Contact: Susanne Voelkel ([email protected])
This study is published in: Voelkel S (2013) Combining the formative with the summative: the development of a two-staged online test to encourage engagement and provide personal feedback in large classes. Research in Learning Technology 21: 19153 http://dx.doi.org/10.3402/rlt.v21i0.19153
