Centre for Educational DevelopmentORHEP Project
1
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
Centre for Educational DevelopmentORHEP Project
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
Assessment and Feedback
John DermoCentre for Educational Development
Centre for Educational DevelopmentORHEP Project
2
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
In pairs/groups, discuss (5 minutes)
Why do we have assessment in Higher Education?
Can you list at least three reasons?
Centre for Educational DevelopmentORHEP Project
3
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
Purposes of AssessmentGibbs (1999):• capturing student time and attention (e.g. through
motivation);• generating appropriate learning activity in students;• providing feedback which students pay attention to;• helping students to internalise a discipline’s standards
and notions of quality;• marking to enable pass/fail decisions to be made;• quality assurance through providing evidence to
outsiders enabling judgements about appropriateness of standards to be made.
Centre for Educational DevelopmentORHEP Project
4
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
Some words used to discuss assessment
4
test quiz
exam
formative
summative
high-stakeslow-stakesnorm-referenced
criterion-referenced
diagnostic constructed response
selected response
objectivesubjective
assignment
peer assessment
peer review
self-assessmentgroup assessment
evaluation
assessed coursework
moderation
item analysis
marking criteria
assessment of learning
assessment for learning
Centre for Educational DevelopmentORHEP Project
5
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
What makes a good assessment?
What do you think are the main features of a “good” assessment?
Centre for Educational DevelopmentORHEP Project
6
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
Fundamental Principles of assessment
• Validity• Reliability• Practicality
• Accessibility, Inclusivity, Authenticity
Centre for Educational DevelopmentORHEP Project
7
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
Validity - Bloom’s Taxonomy
Synthesis
Analysis
Application
Comprehension
Knowledge
7
Making decisions and supporting views: required understanding and values.
Identifying components: determining arrangement, logic and semantics.
Restating in your own words: paraphrasing, summarising, translating.
Combining information to form a unique product: requires creativity and originality.
Using information to solve problems: transferring abstract or theoretical ideas to practical situations. Identifying connections and relationships and how they apply.
Memorizing verbatim information. Being able to remember but not necessarily fully understanding the material.
Evaluation
Centre for Educational DevelopmentORHEP Project
8
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
Test and item
Reliability
Rater (inter and intra)
PracticalityResources
“Real world” issues
Accessibility, Inclusivity, Authenticitycf University’s Core Values
Reflective / Adaptable / Inclusive Supportive / Ethical / Sustainable
Centre for Educational DevelopmentORHEP Project
9
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
“It is now thirty years since serious doubts were raised about examinations, yet despite the fact that there has been no serious shortage of critics since then, very little has changed.” Cox (1967: 352)
“For many years I taught in universities. . . . I marked thousands of scripts without examining what the scripts could teach me about my capacity as a teacher and examiner.” Ashby (1985: v)
“Something like 90% of a typical university degree depends on unseen time-constrained written examinations, and tutor-marked essays and/or reports.” Race (2001: 5)
Some quotations on assessment in HE:
Are they true today?
“Students can avoid bad teaching but they can’t avoid bad assessment” (Boud 1994)
Centre for Educational DevelopmentORHEP Project
10
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
“...institutional assessment efforts should not be concerned about valuing what can be measured but, instead, about measuring that which is valued.” Banta et al (1996: 5)
“Description of a grade: An inadequate report of an inaccurate judgment by a biased and variable judge of the extent to which a student has attained an undefined level of mastery of an unknown proportion of an indefinite material.” Dressel (1983:12)
“assessment plays a critical role in determining the quality of student learning” and “a conception of assessment for learning first and grading second implies the use of a spectrum of methods” Ramsden (1992:177 and 185)
Some quotations on assessment in
HEAre they true
today?
Centre for Educational DevelopmentORHEP Project
11
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
National Union of Students’ Principles of Effective Assessment
(2009)
1. Should be for learning, not simply of learning.
2. Should be reliable, valid, fair and consistent.
3. Should consist of effective and constructive feedback.
4. Should be innovative and have the capacity to inspire and motivate.
5. Should measure understanding and application, rather than technique and memory.
6. Should be conducted throughout the course, rather than being positioned as a final event.
7. Should develop key skills such as peer and reflective assessment.
8. Should be central to staff development and teaching strategies, and frequently reviewed.
9. Should be of a manageable amount for both tutors and students.
10. Should encourage dialogue between students and their tutors and students and their peers.
11
Centre for Educational DevelopmentORHEP Project
12
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
NSS: Assessment and feedback
5. The criteria used in marking have been clear in advance.
6. Assessment arrangements and marking have been fair.
7. Feedback on my work has been prompt.
8. I have received detailed comments on my work.
9. Feedback on my work has helped me clarify things I did not understand.
Since 2005 NSS evidence suggests that students nationally find assessment and feedback among the least satisfactory elements of their experience of higher education.
We also know that assessment takes up ever more of our time and energy.
Centre for Educational DevelopmentORHEP Project
13
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
Grading, Marking Criteria, Moderation
What are marking criteria?
How can they help tutors?
How can they help students?
What about moderation/standardisation?
Can grading ever be “objective”?
Look at the example criteria
from this module
Consider this:
Centre for Educational DevelopmentORHEP Project
14
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
Plagiarism
14
• “...work which is not undertaken in an examination room under supervision but which is submitted by a student for formal assessment must be written by the student and in the student’s own words...” (UoB 2003)
Cheating or weak study skills?
• Misunderstanding of or not understanding – Academic Integrity
• Greater heterogeneity of student body requires new consistent approach to detailing what required and what acceptable
Why does it happen?
• Pandora’s box (Sutherland-Smith 2005)
• Reluctance to discuss plagiarism openly
• Application and understanding of University policy on plagiarism
• Integrity, honesty and trustworthiness
How to prevent it and deal with it?
Centre for Educational DevelopmentORHEP Project
15
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
Feedback• National Student Survey• NUS Principles• What is feedback?
Student views on feedback (10 minute video)http://vimeo.com/channels/154640
What would your students say on
the subject?
Centre for Educational DevelopmentORHEP Project
16
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
References 1 Ashby, E. (1985), preface to Brewer, I. Learning more and Teaching less.
Guildford: Society for Research into Higher Education & NFER-Nelson.
Atkins, M.J., Beattie, J. and Dockerell, W.B. (1993) Assessment Issues in Higher Education, Department of Employment.
Banta, T. W., Lund, J. P., Black, K. E., & Oblander, F. W. (1996) Assessment in practice: Putting principles to work on college campuses. San Francisco: Jossey-Bass.
Boud, D (1995) Enhancing learning through self-assessment London, Routledge.
Cox, R. (1967) “Resistance to Change in Examining”, Universities Quarterly, 21, pp. 352–358.
Dichtl J. (2003) Teaching Integrity The History Teacher 36: 3, 367-373 Society for History Education
Dressel, P. (1983) "Grades: One more tilt at the windmill." in A.W. Chickering (Ed.), Bulletin. Memphis: Memphis State U. Center for the Study of Higher Education.
Centre for Educational DevelopmentORHEP Project
17
www.orhep.brad.ac.uk
www.orhep.brad.ac.ukThis work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License - http://creativecommons.org/licenses/by-sa/3.0/
References 2 Gibbs, G. (1999) Using assessment strategically to change the way
students learn. In: Assessment matters in Higher Education (eds Brown, S. & Glasner, A.), pp. 41-53, Society for Research in Higher Education and Open University Press, Buckingham.
Race, P. (2001) The Lecturer's Toolkit. (2nd ed) London: Kogan Page
Ramsden, P. (1992) Learning to Teach in Higher Education. London: Routledge.
Sutherland-Smith W. (2005) Pandora’s box: academic perceptions of student plagiarism in writing. Journal of English for Academic Purposes 4 (2005) 83–95
UoB (2003) Statement on Academic Integrity. Academic Standards and Support Unit University of Bradford http://www.brad.ac.uk/admin/acsec/assu/statement_on_academic_integrity.htm accessed 20/1/10