Combining the formative with the summative - an effective use of online tests

Susanne Voelkel

School of Life Sciences

Learning …

• Is assessment-driven1

• Improves with “Time on task”2

• Is improved by frequent formative assessments3

1 Gibbs, 2010, 2Chickering & Gamson, 1987, 3Black & Williams, 1998

BUT: Low staff to student ratios make it difficult to introduce regular formative assessment exercises

Advantages of online tests

• Easy to disseminate

• Easy, any time access for students

• Easy to manage (VLE)

• Automatic response

– Independent of class size

– Automatic marking

– Instant “feedback”

The starting point (2008 and before)

• Year 2 theory module

• 70 – 100 students

• 18 lectures in 6 weeks

• Final exam

• No formative assessment

Introduction of voluntary online tests

• A series of online tests

• Self assessment

• No marks

• Feedback:

– Scores

– Answer right or wrong

Module marks

Voluntary online tests

Summative online tests

2-staged online tests

0

10

20

30

40

50

60

70

80

2007 2008 2009 2010 2011

Ave

rage

mar

k (%

)

No online tests

*

Better results through self assessment?

0

10

20

30

40

50

60

70

80

90

yes no

Pe

rce

nt

of

stu

de

nts

re

ceiv

ing

a ce

rtai

n m

ark

Use of self assessment tests

Fail or III

II.1 or I

BUT: only a third of the class completed voluntary exercises!

Making online tests compulsory

• Weekly summative tests (6)

• Worth 20% of module mark

What about collusion?

• Late submission possible

• Students could share out correct results

Making online tests compulsory

• Weekly summative tests (6)

• Worth 20% of module mark

• Feedback

– Initially only scores

– Right/wrong answers released after a week

Module marks

Voluntary online tests

Summative online tests

0

10

20

30

40

50

60

70

80

2007 2008 2009 2010 2011

Ave

rage

mar

k (%

)

No online tests

*

Student views

“It (the online test) makes you concentrate on your work, like actually go over your notes”

“I think the problem was, you got your mark back and you didn’t know what you got wrong”

“Yeah, because then, if the system told you what was wrong you’d actually learn from what you got wrong, whereas now we’ve done the assessment and then it’s like, okay, don’t think about it”

Formative assessment

• Low stake

• Main purpose: feedback

• High quality feedback4

– Prompt

– Specific

– How to “close the gap”

4Nicol and MacFarlane-Dick (2006)

A two-stage approach to allow feedback, enforce completion

and discourage collusion

Formative

Summative

Two-stage online tests

• Weekly two-stage online tests (6)

• Each test has two parts, A and B

– A purely formative

– B summative

How the two-stage online tests work

• Part A

– Does not count towards mark

– Multiple attempts

– Immediate, specific feedback

X

Feedback:

Two-staged online test

• Part A

– Does not count towards mark

– Multiple attempts

– Immediate, specific feedback

– Has to achieve 80% to move to B

• Part B

– Does count towards mark

– Only one attempt

– Has to be completed by deadline

Does it work?

• Student learning

– Improved exam results

– Effect size 0.58

Module marks

Voluntary online tests

Summative online tests

2-staged online tests

0

10

20

30

40

50

60

70

80

2007 2008 2009 2010 2011

Ave

rage

mar

k (%

)

No online tests

*

Does it work?

• Student learning

– Improved exam results

– Effect size 0.58

• Student views

– Students liked test AND feedback

– Students felt they learnt more

82

88

89

14

8

8

4

2

2

Tests help my understanding

Found feedback helpful

Tests make me work more

Agree Neither Disagree

Opinions about the online tests (%)

N = 64 – 74

“The continuous assessment throughout the module meant that I was forced to go over lectures from that week and make sure I understood them”

“I really appreciate the tests. My first attempt is usually quite bad and after repeating the tests several times to get above 80% I find I slowly start to learn the content”

Student comments

Thank you

Contact: Susanne Voelkel (svoelkel@liv.ac.uk)

This study is published in: Voelkel S (2013) Combining the formative with the summative: the development of a two-staged online test to encourage engagement and provide personal feedback in large classes. Research in Learning Technology 21: 19153 http://dx.doi.org/10.3402/rlt.v21i0.19153