Efficiency vs effectivenessof assessment – the tensionsChris Rust

Lots of evidence current practices are not very effective nor very efficient

QAA subject reviews National Student survey “the Achilles’ heel of quality” /Summative assessment

practices “in disarray” (Knight 2002a, p.107; Knight 2002b, p.275)/“Broken” (Race 2003, p. 5)

“There is considerable scope for professional development in the area of assessment” (Yorke 2000, p7)

“No longer fit for purpose” (Burgess 2007, p5) Media accusations of dumbing down & grade inflation: 60%

gaining 1sts & 2.1s in 2010 cf 30% in 1970 (Sunday Telegraph, 2/1/11, p6)

“serious grounds for concern” (IUSS 2009, p116)

Danger our assessment practices encourage a Surface approach to learning

“The types of assessment we currently use do not promote conceptual understanding and do not encourage a deep approach to learning………Our means of assessing them seems to do little to encourage them to adopt anything other than a strategic or mechanical approach to their studies.” “…students become more interested in the mark and less interested in the subject over the course of their studies.” (Newstead 2002, p2)

Many research findings indicate a declining use of deep and contextual approaches to study as students’ progress through their degree programmes (Watkins & Hattie, 1985; Gow & Kember, 1990; McKay & Kember,1997; Richardson, 2000; Zhang & Watkins, 2001; Arum & Roksa, 2011)

Resources and Validity vs Reliability[Dependability: one-handed clock (Stobart, 2008)]

Manageability(resources)

Reliability

Construct validity(authenticity)

Resources and Validity vs Reliability

Manageability(resources)

Reliability

Construct validity(authenticity)

‘This quest for reliability tends to skew assessment towards the

assessment of simple and unambiguous achievements,

and considerations of cost add to the skew away from

judgements of complex learning’ (Knight 2002, p278)

Resources and Validity vs Reliability

Manageability(resources)

Reliability

Construct validity(authenticity)

assessing airline pilots

Resources and Validity vs Reliability

Manageability(resources)

Reliability

Construct validity(authenticity)

assessing airline pilots

trade-off, dependant on context

Resources and Validity vs Reliability

Manageability(resources)

Reliability

Construct validity(authenticity)

if more purely formative

assessment,

could increase

authenticity with less cost?

Resources and Validity vs Reliability

Manageability(resources)

Reliability

Construct validity(authenticity)

if more purely formative

assessment, could increase

authenticity

if less summative

assessment,

could afford/should aspire

for it to be here?

Module assessment vs Programme assessment

Equivalence of demand: choice vs standardisation

Equivalence of demand should consider total expected student learning hours, not try to single out assessment

Any standardisation should avoid confusing consistency with conformity, and a reliance on often ‘numbers-based’ rules (e.g. 1st year essays 3,000 words max, 3rd year essays 5,000 words max; or, no more than two pieces of assessment per module) and crude word-equivalence

Improving feedback: a potential win/win