Date post: | 29-Mar-2015 |
Category: |
Documents |
Upload: | linda-philley |
View: | 213 times |
Download: | 0 times |
Peer feedback in formative assessment to aid learning:
Annie CushingAngela Hall
Olwyn Westwood
Centre for Medical Educations
CETL 2008
Drivers
• Professional Skill – Multisource Feedback as part of professional practice
• NSS – students want feedback
• Formative feedback as a support for learning
• Large student numbers
• Proximity of practice/feedback/practice
s
CETL 2008
Study 1 in Nov 2008 Graduate Entry Students: Year 1: n=93
Medical & Nursing [Adult, Child, Mental Health]
• 3 Stations• Based on PEBL cases• 5-minute station with 3-min feedback• Actors play patient/carer
Study 1: Nov 2008 ( 78 attended = 84%)
Students:• Generate the marking criteria*• Undertake all OSCE stations as ‘candidates’• Undertake ‘examiner’ role in 1 station.• Actors give verbal feedback
Evaluation by:
Questionnaire*: 20 statement items Likert response scale Focus Group: nominal group technique.
* Adapted from FAST project Open Univ/Sheffield Hallam (Brown et al 2003)
Q1. Being the examiner was helpful in providing clear
instructions on what was expected of me(N=77)
All students (n = 77)
-100.0 -80.0 -60.0 -40.0 -20.0 0.0 20.0 40.0 60.0 80.0 100.0
Strongly disagree
Disagree
Agree
Strongly agree
Q5 The feedback was rapid and helpfulN=78
All students (n = 78)
-100.0 -80.0 -60.0 -40.0 -20.0 0.0 20.0 40.0 60.0 80.0 100.0
Strongly disagree
Disagree
Agree
Strongly agree
Q13. I found it difficult giving feedback to my colleagues. (p <0.05)
Female students (n = 52)
-100.0 -80.0 -60.0 -40.0 -20.0 0.0 20.0 40.0 60.0 80.0 100.0
Strongly disagree
Disagree
Agree
Strongly agree
Male students (n = 26)
-100.0 -80.0 -60.0 -40.0 -20.0 0.0 20.0 40.0 60.0 80.0 100.0
Strongly disagree
Disagree
Agree
Strongly agree
Q20. I felt that I was able to be honest in my feedback
Female students (n = 52)
-100.0 -80.0 -60.0 -40.0 -20.0 0.0 20.0 40.0 60.0 80.0 100.0
Strongly disagree
Disagree
Agree
Strongly agree
Male students (n = 26)
-100.0 -80.0 -60.0 -40.0 -20.0 0.0 20.0 40.0 60.0 80.0 100.0
Strongly disagree
Disagree
Agree
Strongly agree
Focus Group (7) Helpful to Learning
Seeing examples of:• good and bad communications (content &
delivery) (5 votes)
• differences between people in the same situation (5 votes)
• others’ behaviour helps you understand what you should do yourself (3 votes)
Unhelpful to Learning
• Checklist is too long (7 votes)
• Peers are ‘amateurs’, both at the skills being role-played and at feedback (5 votes)
• Some people are not brave enough in giving critical feedback (4 votes)
Anxiety about giving Negative Feedback
“I thought, ‘You really should never say that in front of somebody’, and I didn’t, to be honest, have the guts to tell them.”
“You want to say something positive but also you want to get across what they need to improve. ………. But it can be quite difficult, I think, because if you’re peers, it's harder than if you're a tutor.”
Feedback Sargeant J et al (2008) J Contin Educ Health Prof
Three inter-related factors for acceptance of
negative feedback:
1. Credibility – specificity, personal observation, explicit standards of performance
2. Emotion- when inconsistent with self perceptions of being a ‘good doctor’
3. Reflection – a good mediator of assimilation, acceptance and application
Incorporating Feedback
• “I thought I'd done something quite well, and someone pointed out that actually I could have done it this way and it might have been better. And then seeing someone else do it, I thought, ‘Oh well, yeah, that’s a much better way of doing it’. But it hadn't even occurred to me that I could improve on the way I was doing it. And I don't think it's until someone points it out to you in quite a specific way that you think, ‘Well actually, yeah, I’ll try that’. And I've done it since, and it is better.”
Learning Transfer
“With a patient who wanted a long conversation, I had to draw on the OSCE session. ‘OK, how do I make sure that I'm actually quite interested in what you're saying now?’ Because I've got three other patients that I really need to deal with.’”
(Nurse)
Study 2: May 2009Changes in response to Study 1
Teaching on ‘Giving Feedback’ (video CETL).
What would you give feedback on? - generate a marking scheme
How does the facilitator do with her feedback?- generate principles of feedback
Behavioural, specific, balanced +ve with areas to improve.
Changes in Study 2In response to Study 1
• More feedback time:
4-mins interview and 4-mins feedback• Actor to give written rather than verbal feedback
Empathy
Clarity• Cases to be circulated in advance.
Feedback
Responses to Questionnaire Statements [1]
-80.0
-60.0
-40.0
-20.0
0.0
20.0
40.0
60.0
80.0
100.0
Being theexaminer
w as helpfulin providing
clearinstructions
on w hatw as
expected ofme
With thisexercise
there w aslots of
feedback
I w ouldlearn better
if thefeedbackhad beenmore in-depth
Thefeedbackw as rapidand helpful
Feedbackmainly tells
me howw ell I did
compared toothers
Feedbackhelps me tounderstand
issuesbetter
Feedbackhelps me to
improve
Receivingfeedbackhelped meunderstandw hy I gotthe mark I
did
I found thefeedbackfrom thesimulated
patienthelpful
I listencarefully tofeedback
I w as ableto act upon
thefeedback I
got insubsequent
stations
Feedbackstimulatesme to go
back overmaterial
covered onother areas
of thecourse
I learnt newthings
w hilst beingan examiner
I felt that Iw as able tobe honest in
myfeedback
Strongly disagree Disagree Agree Strongly agree
(n = 48)
(n = 48)
(n = 48)
(n =48)
(n = 48)
(n = 48)
(n = 48)
(n = 48) (n = 48)
(n = 48)
(n = 48)
(n = 48)
(n = 48)
(n = 48)
Responses to Questionnaire Statements [2]
-80.0
-60.0
-40.0
-20.0
0.0
20.0
40.0
60.0
When I got things w rong ormisunderstood, I did notreceive much guidance
The feedback I received w asnot useful
I did not understand some ofthe feedback
I w ould prefer feedback fromtutors rather than my
colleagues
I found it diff icult givingfeedback to my colleagues
Feedback is not likely to bemuch help for the future
performances
Strongly disagree Disagree Agree Strongly agree
(n = 48)(n = 48)
(n = 48)
(n = 48) (n = 48)
(n = 48)
Free Text Comments
“I understand it was part of the research but it definitely would have been valuable to have been given verbal feedback from the actor as well as written”
“Really good – nice to see that we have improved since the last one, both in interviewing and giving feedback to colleagues”
“Peer feedback: I think it is very helpful, although we are relatively at the same level so limited in depth of feedback”
“Love the OSCE sessions with the actors. They are invaluable and a great learning tool”.
I
Introduced into Curriculum’08 MBBS 2009-10
Core Curriculum in Year 3• Both Communication (3) and Clinical (3) stations:
• Faculty supported, supervised and guided sessions on peer feedback engage learners actively
• Learners need training on giving constructive feedback
• Learners value these sessions
• Resourcing – cost effectiveness
Spin-offs
• www.bbc.co.uk/health
• BSc Med EdEffect of intervention on student performance in end of year
summative OSCE marker station.
Sample:
GEP yr 1 cohort (formative OSCE)
Graduates in Year 2
Thank you
Any Questions?
ReferencesSargeant J et al. 2007 Challenges in multi-source feedback: Intended
and unintended outcomes. Med Educ. 41:583-591Nicol & MacFarlane (2004) Rethinking Formative Assessment in HE: a
theoretical model and seven principles of good feedback practice
Boud, D. (2000). Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education. 22 (2), 151-167.
Kurtz S, Silverman J. 2005. Teaching & Learning Communication Skills in Medicine. Radcliffe Pub Oxford. (Calgary-Cambridge Guide)
Weaver M. (2006) Do students value feedback? Student perceptions of tutors’ written responses, Assessment and Evaluation in Higher Education 31(3) pp. 379-394
Bing-You R.G., Paterson, P., Levine, M.A. (1997) Feedback falling on deaf ears: residents’ receptivity to feedback tempered by sender credibility, Medical Teacher, 19, pp. 40-44
Cost effectiveness
120 students All got chance to:• Examine 6 peers• Practice 6 stations• Be a patient in 2 stations
Tutors = 8‘Bodies’ = 12Actor patient = 4