How to Achieve Practice Standards for Licensure and Continual Competency Assessment Kyle Wilby BSP ACPR PharmD Assistant Professor - Clinical Pharmacy and Practice, College of Pharmacy, Qatar University Clinical Pharmacy Specialist – Infectious Diseases, National Center for Cancer Care & Research and Nadir Kheir PhD FNZCP Associate Professor of Pharmacy Practice, Coordinator CPPD College of Pharmacy, Qatar University Disclaimer: PRESENTING AUTHORS HAVE NO RELATIONSHIPS TO DISCLOSE
Transcript
How to Achieve Practice Standards for Licensure and Continual Competency Assessment
Kyle Wilby BSP ACPR PharmD
Assistant Professor - Clinical Pharmacy and Practice, College of Pharmacy, Qatar University Clinical Pharmacy Specialist – Infectious Diseases, National Center for Cancer Care & Research
and Nadir Kheir PhD FNZCP
Associate Professor of Pharmacy Practice, Coordinator CPPD College of Pharmacy, Qatar University
Disclaimer: PRESENTING AUTHORS HAVE NO RELATIONSHIPS TO DISCLOSE
Objectives
1. To explain the main features of competency based and performance-based assessments and the differences between these assessment and traditional assessments like examinations and MCQs.
2. To discuss the main elements of an Objective Structured Clinical Examination (OSCE) and a Structured Multi - skill Assessment (SMSA).
3. To list the main components of a typical grading sheet used with an OSCE station.
4. To demonstrate application of knowledge and skills by completing a mock-structured practical assessment.
5. To assess effective analytical and communication skills through peer evaluation and discussion.
8 December 2014 OSCE-SMSA-CPPD-QU-2014-Wilby-Kheir
Performance-Based Assessments • The process of collecting evidence and making judgments on the nature and extent of progress towards the performance requirements set out in a standard (VEETAC cited in ACTRAC 1994, p.1).
• Requires a student to create an answer or a problem that demonstrates his or her knowledge or skills
• Based on the actual skills and knowledge a person can demonstrate in the workplace
OSCE-SMSA-CPPD-QU-2014-Wilby-Kheir 8 December 2014
Performance-Based Assessments (Authentic) vs. Traditional Assessment
8 December 2014 COP ANK
Authentic Assessment Traditional Objective Assessment (Test)
Direct measures of student performance “Indirect" : Test items are designed to "represent competence" by extracting knowledge and skills from their real-life contexts
Tasks by design are "ill-structured challenges" since their goal is to help students prepare for the complex ambiguities of the “real” world.
Test only one domain of knowledge or skill so as to avoid ambiguity for the test taker
Focus on processes and rationales There is no single correct
One-time measures that rely on a single correct response to each item; they offer no opportunity for demonstration of thought processes, revision, or interaction with the teacher
Students are aware of how and on what knowledge and skills they are to be assessed. Assessment is both an evaluative device and a learning activity
Require brief responses, often machine-scored, and students construct their responses in only the most minimal way, and often by only plugging in a piece of knowledge
Poor reliability and high validity: Reflects real-life tasks, as well as the multi-faceted character of curriculum in ways that a one-shot evaluation cannot
Limited potential tests to measure higher-order thinking skills since, by definition, those skills involve analysis, interpretation, and multiple perspectives
An assessment that aims to measure not only the correctness of a
response, but also the thought processes involved in arriving at the
response
8 December 2014 COP ANK OSCE-SMSA-CPPD-QU-2014-Wilby-Kheir
The SMSA • The College of Pharmacy’s version of the Objective
Structured Clinical Examination (OSCE) • Both are performance-based assessment methods and
consist of standardized tasks that cover a number of competencies using structured marking sheets
8-Dec-14 COP ANK SMSA-QU-2013-NKheir
8-Dec-14 COP ANK
What is an OSCE or an SMSA? • Composed of a series of stations that student moves through, solving pharmacy-related problems or performing pharmacy-related tasks
• Evaluates professional competencies • A measure of the student’s skills and response when confronted with a clinical situation
• Also a method for assessing processes as well as critical thinking
• A good OSCE-SMSA contains an appropriate sample of the competences that are meant to be assessed
• SMSA is1st introduced in P-1 (Spring Semester) • OSCE is a summative & high-stake assessment
SMSA-QU-2013-NKheir
8-Dec-14 COP ANK
Features Interactive Flexible structure and content Mimics real world Tests competencies best tested as scenarios Structured instructions, test, and marking Encourages critical thinking
SMSA-QU-2013-NKheir
The SMSA and the OSCE are like integrated, multidimensional and dynamic video-clip approaches to learning, while a traditional test is more like a single static snapshot
8-Dec-14 COP ANK SMSA-QU-2013-NKheir
8-Dec-14 COP ANK
Developing • Constructing a Blueprint:
• - competencies to be assessed • - clinical situations to demonstrate the competencies Example competencies: • consulting/interviewing a patient • finding drug-related problems • developing a short care plan • finding drug information • performing patient assessment
Can include cases of health conditions studied (ACS, AF, HT)
SMSA-QU-2013-NKheir
8-Dec-14 COP ANK
Highlights • Possible use of standardized or real patients • Possible use of scenarios • Can have additional oral exam element • Can be open book • Written instructions to the student • Written instructions to the patient/examiner • Written instructions to the marker
SMSA-QU-2013-NKheir
8-Dec-14 COP ANK SMSA-QU-2013-NKheir
8-Dec-14 COP ANK SMSA-QU-2013-NKheir
8-Dec-14 COP ANK SMSA-QU-2013-NKheir
8-Dec-14 COP ANK SMSA-QU-2013-NKheir
8-Dec-14 COP ANK SMSA-QU-2013-NKheir
8-Dec-14 COP ANK SMSA-QU-2013-NKheir
OSCE: LOGISTICS AND TIPS FOR SUCCESS
Dr. Kyle Wilby, BPS, ACPR, PharmD Assistant Professor, College of Pharmacy, Qatar University Clinical Pharmacy Specialist, NCCCR November 17th, 2014
What is an OSCE? • Series of independent stations that each contain a clinical
scenario for candidates to solve • No set number of stations per exam but typically high
stakes exams will have at least 6 for validity
• Goal is to simulate clinical scenarios encountered in daily practice (relevance to all practice settings)
OSCE-SMSA-CPPD-QU-2014-Wilby-Kheir 8 December 2014
What can I expect? • Candidates are usually guided by exam staff to their
assigned starting station
• A “stem” is provided on the door of the station that will describe the scenario:
OSCE-SMSA-CPPD-QU-2014-Wilby-Kheir
“You are about to meet Mr. Ahmed in a Community Pharmacy. Provide care as you would in practice.”
8 December 2014
What can I expect? • After 30s a whistle is blown and candidates are expected
to enter the room
• Once inside, candidates will find the Stem, any other relevant patient information, references, and props (drug products, devices, etc.)
• An assessor (evaluator) will be sitting inside the room away from the interaction setting
OSCE-SMSA-CPPD-QU-2014-Wilby-Kheir 8 December 2014
What can I expect? • After 30s from the candidate entering the room, a
standardized patient or health professional will enter and address the candidate:
OSCE-SMSA-CPPD-QU-2014-Wilby-Kheir
“Can you recommend something for my cough?”
8 December 2014
Let the fun begin! • Candidates will have between 5 and 10 minutes to
complete the interaction with the actor
• Interactions are based on practice and may include: • OTC Recommendations • Patient Counseling • Referral to Other Healthcare Professional • Answering Drug Information Requests • Dealing with an Ethical or Legal Issue • Management Related Tasks • Many more…
OSCE-SMSA-CPPD-QU-2014-Wilby-Kheir 8 December 2014
The Final Moments • Once the candidate is finished interacting with the patient
or healthcare professional, they will close and the actor will leave the room
• If the candidate has not finished at the time of the last whistle, the actor will leave the room without closing
• If the candidate finished early, they will be instructed to sit quietly without interacting with the assessor until the final whistle is blown and they can leave the room
OSCE-SMSA-CPPD-QU-2014-Wilby-Kheir 8 December 2014
Tips for Success • Be systematic!
OSCE-SMSA-CPPD-QU-2014-Wilby-Kheir
Patient Identification Chief Complaint Signs and Symptoms Past Medical History Current Medications OTCs and Herbals Allergies Social History (smoking, alcohol, etc.) Family History Others?
8 December 2014
Tips for Success • Ask the patient if they have any questions
• Always provide opportunity for follow up
• Practice good communication skills (verbal and
nonverbal)
• Use references, when needed
OSCE-SMSA-CPPD-QU-2014-Wilby-Kheir 8 December 2014
Tips for Preparation • Think of common practice scenarios and write them down
• Work with a colleague to develop role-play cases that are
timed based on your documented scenarios
• Be familiar with references available during exam
• Practice good communication skills by focusing on speaking slowly and clearly with eye contact and positive body language
OSCE-SMSA-CPPD-QU-2014-Wilby-Kheir 8 December 2014
Time to Practice! • Please arrange yourselves in groups with a facilitating
‘expert assessor’
• Volunteer to complete a role play
• Observe others and provide feedback on both content and communication skills
• A debriefing session will occur at the end of the activity for the large group
OSCE-SMSA-CPPD-QU-2014-Wilby-Kheir 8 December 2014
