12/9/2013
1
The Use of an Integrated OSCE to Assess Pharmacy Students’ Clinical SkillsNancy Letassy, Pharm.D., CDE, ProfessorOUHSC College of Pharmacy
Cell phones and pagers should be turned to silent or off. Thank you!
Thank you for joining us! Please remember to complete your pre- and post-evaluations and turn them in at the end of the presentation.
Nancy Letassy, Pharm.D., CDE Professor, OU College of Pharmacy
12/9/2013
2
OSCE Description Limitations of multi-station OSCE Development of an integrated OSCE Test Procedures Advantages and Disadvantages
Objective Structured Clinical Examination
Introduced by Ronald M. Harden, M.D. 1975
University of Dundee, College of Medicine Developed to address
disadvantages of traditional clinical examinations
12/9/2013
3
“An approach to the assessment of clinical competence in which the components of competence are assessed in a planned or structured way with attention being paid to the objectivity of the examination.” RM Harden, MD
Medical Teacher 1988
Medical Teacher 1988;10;19-22
Objective
Assesses competence in a wide range of skills
Uses a rubric to define what is tested and grade assignment
Establishes criteria for passing prior to exam
Employs standard patients
Trains multiple examiners
12/9/2013
4
Structure
Reflects the objectives of the course
Determines in advance exam content and competencies assessed
Assesses a range of competencies through multiple stations
Structures movement through the exam
Organizes stations by time, information, standard patient, and tasks to be performed
Clinical Examination
Assesses clinic performance of skills and application of clinical knowledge
Depends on interaction with a live or simulated patient
Presents “real world” problems, interventions, or tasks to assess or perform
12/9/2013
5
Traditional OSCE Consists of multiple,
timed, standard stations Assesses 1 to 2 specific
clinical tasks per station Conducted in an
interactive environment Uses patient actors or
standardized patients Observed and graded by
trained examiners
History taking Physical
examination Interpretation of
findings Communication,
verbal and written Patient education
12/9/2013
6
Multi-station OSCE limitations in evaluating integrated patient care skills: Assesses a narrow range of skills and knowledge in
isolation
Assesses clinical skills in a number of patients in isolation instead of comprehensively examining a single patient
Assessing isolated parts not equivalent to the whole integrated performance
Limited ability to replicate real-life patient care situations
Some clinical skills cannot be assessed properly
Impact of an absent patient or examiner on exam process
Consider these factors in planning an OSCE Planning and administrative time Number of students to be assessed Space Cost Patient availability Training and cost of standardized patients Set-up time Time to administer exam Personnel required to administer exam
12/9/2013
7
Ambulatory Care Experiential OSCE
Alternate pathway for the Doctor of Pharmacy program
Drug Information Course Ambulatory Care APPE
12/9/2013
8
Fourth Professional Year APPE
9—one month rotations in the P4 year
▪ 2 months of Ambulatory Care
▪ 1 month in acute care
▪ 1 month in community pharmacy practice
▪ 1 month in hospital pharmacy practice
▪ 4 selectives
Family Medicine Center Pharmacist-directed
service Diabetes and
Anticoagulation Service 8 week rotation 4th year Professional
Students see
~100-120 patients/month
12/9/2013
9
Faculty dissatisfaction with traditional assessment methods to evaluate clinical competence
Multi-station OSCE impractical due to space, time, manpower
Integrated OSCE format viewed as a better format to assess clinical skills
OSCE adopted as end-of-the month assessment for APPE in June 2006
Patient Interviews Medication History Taking Interpretation of patient data Identify problems Patient assessment Communication Recommend management Patient education
12/9/2013
10
Patient Encounters (2 videotaped and 1 live) Patient interview Oral communication Professional behavior Medication taking Patient education
SOAP Notes –2 graded Written communication Patient assessment Drug therapy assessment
OSCE
Assess clinical knowledge and skills
Conducted at week 4 and week 8
Five students per month on rotation
Written examination
Assesses knowledge
12/9/2013
11
OSCE status in assessment of students on an ambulatory care APPE
Summative – performance contributes to the student’s course grade
High stakes – defined as an assessment in which poor performance on the assessment could prevent progression in the P4 year.
Grading
Performed by the faculty using a grading rubric
Steps to create an OSCE: Used exam blueprint to develop patient cases Determined skills and knowledge to be
assessed Created realistic patient cases through group
case writing and review Script written for standardized patient
12/9/2013
12
Created grading rubric to assign grades
Determined standard for pass/fail Assigned time for each task Outlined test procedure Trained faculty to serve as patients,
examiners, and providers
Student package includes: Directions Time allotment Physician note Reason for referral Problem list Lab and vitals Past response – INR and dosing history or blood
glucose log Medication refill history Note template
12/9/2013
13
Same as the student package Directions to the standardized patient
History Medications Diet Reasons for adherence problems Disease related Psychosocial factors Directions for what to ask student if they do not ask Directions on what to withhold to make the student
dig for the information
Revisions and maintaining exam security Developed multiple cases Cycle cases 2-3 times throughout the
year Review cases and rubrics prior to each
use Revise cases and rubrics on a periodic
basis
12/9/2013
14
Conducted in the clinic where the students practice throughout the month
OSCE station is a clinic room familiar to the student
All students are examined using the same case on the same day
Assess all clinical competencies at one station
Ambulatory care practice faculty serve as a patient and a provider
12/9/2013
15
Student enters assigned room Given 15 minutes to review patient
information packet and prepare for interview Patient enters room Student is in charge of the visit Given 15 minutes to interview patient, obtain
medication history, determine problem, and educate patient if necessary
Time called and patient exits room
Student has 25 minutes to construct assessment and plan and write SOAP note
Upon exiting room, faculty complete rubric for patient interview
Time is called and faculty enter room as a provider
Student has 5 minutes to present A/P to provider
Provider interaction with student is not scripted
12/9/2013
16
When time is called the faculty collects all material
Faculty exits the station Student’s assessment and plan are graded
according to the rubric by the faculty who played the patient
Student presentation to the provider is graded by the faculty who played the provider
Grading rubric is completed Faculty who played the patient reviews
performance with student Feedback is given to student Student’s are able to provide feedback as well Feedback can be used to revise OSCE cases
as needed
12/9/2013
17
Points
Full
Credit
Partial
Credit
No
Credit
Patient Interview
1 Student introduces him/herself
1 Student addresses patient concerns about treatment regimen
7 Student performs a medication history. The following points are
awarded for all medications with exception of Insulin:
Student performs a medication history using open ended
questions. (1 pt)
The history is complete including all medications and
dosage regimen. (4 pt)
Assess adherence with all medications. (1 pt)
Includes questions about OTC and herbal products. (1 pt)
(specific questions about Insulin are in next line)
4
Student asks patient how she is taking her Lantus and Novolog;
assesses compliance
Students establishes when patient takes insulins and doses
(2 pts)
Assesses adherence to prescribed regimen (2 pt)
Written Assessment/Plan
Full
Credit
Partial
Credit
No
Credit
4
Assessment:
hypoglycemia due to
o decreased food intake ( 1 pt)
o too much Novolog before breakfast (1 pt)
o increased activity during the morning (1 pt)
o atenolol decreasing recognition of
hypoglycemic symptoms (1 pt)
12/9/2013
18
13
Recommendations:
Changes in insulin regimen should include:
o Overall decrease in insulin daily dose –somewhere in
the range of 5 to 10 units.—(1 pt)
o Increase percent of insulin given as basal and reduce
percent given as bolus – towards 50-50%.—(1 pt)
o Specifically decrease Novolog at breakfast when she
eats less or plans to exercise (2 pt)
o F/u with patient in 3 to 7 days to evaluate changes in
blood glucose and hypoglycemia occurrence (2 pt)
Skip Novolog if she does not eat breakfast (2 pt)
Monitoring (4 pt):
o Recheck blood pressure in 2 -4 weeks
o Recheck potassium in 2-4 weeks
o Recheck ALT/AST in 1 -3 months
o Recheck lipids in 1-3 months
Recommends a foot exam to assess sensate, pulses and
vascular status (1 pt)
Full
Credit
Partial
Credit
No
Credit
Presentation to Physician Full
Credit
Partial
Credit
No
Credit
1 Student introduces him/herself and purpose of
presentation
1 Student displayed professional maturity with
presentation
3 Student provides appropriate information from SOAP
note in a complete, concise manner
1 Student convincingly justified his/her plan
3 Student appeared to have utilized the 5 steps of clinical
reasoning (RT modules)
1 Student gives presentation in a professional
manner(appropriate medical terminology, eye contact, no
distracting mannerisms)
10 pt possible
12/9/2013
19
Cost minimal Manpower
One faculty per student tested
One clinic staff to keep time
Minimal test set-up
One hour to prepare exam packets
10-15 minutes for clinic set-up
One hour to administer exam Grading and feedback --30 minutes
Modifications over time Time allowance increased for review of
patient information, patient interview, and time to construct the SOAP note
Grading rubric has become more detailed Instructions for the standardized patient have
become more detailed
12/9/2013
20
Patient case replicates the usual patient-student interaction
All clinical skills are assessed with one patient case
Allows assessment of an integrated clinical performance
Allows an integrated evaluation
Limitations Relator bias of the examiner ▪ Effect of the faculty playing the patient
and provider
▪ Faculty-student relationship
▪ Faculty –patient is also the major evaluator
Only one station
12/9/2013
21
Using an integrated OSCE in APPE assessments Cost, time, and space effective
Minimal personnel requirements
All students see the same case on the same day
Using faculty as standardized patients decreases cost and avoids the problems of no show patients
Allows assessment of clinical skills in an integrated fashion that reflects realistic pharmacist-patient interactions