11/15/2010
1
TRANSITIONING FROM
NEEDS ASSESSMENT TO
PRACTICE GAPS
November 16, 2010
2:00 – 3:00 PM ET
PRESENTERS
Chris PrestaSenior Manager, Enduring MaterialsAmerican Academy of Dermatology
Rebecca DeVivo, MPH, MSWSenior Director, Professional EducationAmerican Society for Gastrointestinal Endoscopy
OBJECTIVES
Define the difference between needs
assessment and practice gap.
Identify evidence-based sources to
gather practice gap data.
Develop tools to collect practice gap
data from learners and experts.
11/15/2010
2
DISCLAIMER
• This is only our experience – We have not had any
direct feedback from the ACCME.
• All experiences are different depending on your
organization.
WHERE I STARTED:
NEEDS ASSESSMENT
Question: Why is this activity needed?
Typical answers:- Data says this is a topic of importance and
education is needed:- Published, peer-reviewed research- Practice Guidelines
- Members say they want education in this area - Evaluation data- Member surveys
WHERE I STARTED:
NEEDS ASSESSMENT
EXAMPLE 1: Capsule Endoscopy
Evidence Base: ASGE Guideline
States education requirements before competency can be assessed:
“Completion of a hands-on course with a minimum of 8 hours CME Credit, endorsed by a national or international GI or surgical society and review of first 10 capsule studies by a credentialed capsule endoscopist.”
11/15/2010
3
BUT, ALONG CAME NEW ACCME
CRITERIA and LABELS…
Activity Labels for Criterion 2 (for the November 2011 cohort):
#1: Verification that the activity is developed from a professional practice gap connected to your own learners.
(What do physicians need to be able to do?)
(What is the practice-based problem?)
BUT, ALONG CAME NEW ACCME
CRITERIA and LABELS…
Activity Labels for Criterion 2 (for the November 2011 cohort):
#2: Verification that educational needs(knowledge, competence or performance) underlying the professional practice gaps were identified.
(Why does the problem exist?)
(How can we close the practice gap?)
UPSHOT:
“NEEDS ASSESSMENT” NOT ENOUGH!
We had identified an educational need, but based on what professional practice gap?
However, our Practice Guideline also states the Endoscopists wishing to perform capsule endoscopy:
- Must be able to integrate capsule endoscopy into the overall clinical evaluation of appropriately selected patients.
- Must be familiar with the hardware and the softwarenecessary to perform and interpret the capsule endoscopy images
- Must be able to accurately identify and interpret capsule endoscopy findings
These are practice gaps!
11/15/2010
4
UPSHOT:
“NEEDS ASSESSMENT” NOT ENOUGH!
To be more specific:
Practice Gap (what do they need to be able to?): - Must be able to integrate capsule endoscopy into the
overall clinical evaluation of appropriately selected patients.
Educational Need (Why does the gap exist? How can we help them close the gap?):
- Knowledge: Evidence-based information on when it is appropriate to use capsule endoscopy in the treatment of a patient.
- Competence: Skills training with the equipment- Performance: Additional experience/confidence in
reading cases
= Hands-on CME Course
So, what are some examples of resources
we can use?
Practice Guidelines
Went through and picked out specific
recommendations related to practice.
Collated all “practice gap” information into one document that can be used for multiple activity files.
ACCME has indicated they are looking for an
overall analysis of practice gaps.
11/15/2010
5
EVALUATION DATA
Ask attendees to describe a clinical question in
practice encountered in the last year.
Dermatologists told us that they need strategies to:
Safely administer biologic treatments in pediatric
patients
Skills to use a dermoscope
Appropriate treatment for alopecia in women
11/15/2010
6
EXPERTS
The Epidemiology ERG serves as resource of “experts” to identify practice gaps. The practice gap evaluation was based on the ERG review of literature, informal surveys and interviews with course directors and speakers.
Specific gaps in practice and knowledge were proposed for the following: Reporting adverse events for drugs and/or other treatments
Patient compliance with recommended treatments and behaviors
Use of total body skin exam to screen for skin cancers
Identification of ocular symptoms in rosacea that might lead to corneal disease
Identification of psoriatic arthritis in psoriasis patients that might lead to morbidity and disability
JOURNALS
The sited article reports 8 pts (2 from a previous
study) responding well to UVA-1 therapy. Irradiation with UVA-1 increases the collagenase
production by cultured fibroblasts, which could be
the pathophysiologic mechanism that explains the
good effect of UVA-1. This form of phototherapy
may be a treatment option for patients with this disabling and often therapy-resistant disease.
Kroft EB, Jong EM. Scleredema diabeticorum case
series: successful treatment with UVA-1. Arch
Dermatol. Vol 144:(No7):947-948.
11/15/2010
7
VERIFICATION THAT THE NEEDS UNDERLYING
THE GAP WERE IDENTIFIED?
Planning Documentation:
- Planning tools – Grids, templates, applications – may need updating.
Previously: Why is this activity needed?
Now:
GAP: What do physicians need to be able to do?
NEED: Why aren’t they able to do it? How can we help them do it? (i.e., knowledge, skills, etc.)
- Planning committee minutes/emails
- Board reports
OKAY, SO BACK TO WHERE I STARTED:
NEEDS ASSESSMENT
Question: Why is this activity needed?
Typical answers:- Data says this is a topic of importance and
education is needed:- Published, peer-reviewed research- Practice Guidelines
- Members say they want education in this area - Evaluation data
- Member surveys
AND NOW?
GAP ANALYSIS
New Question: What do our physicians need to be able to
do?
(that they’re not doing now…)
Typical answers (for ASGE):
- Integrate new technologies or procedures into practice
- Evaluate the latest research in the context of their own practice
- Efficiently and accurately process reimbursement
- Train their nursing staff/endoscopy team
THEN, what do we need to do to help them
do it?
11/15/2010
8
KEY POINTS
Gap Analysis identifies and documents the
objectives – what do physicians need to be able to do?
Needs Assessment defines the content – how can
we help them do it?
Sources can include:
Clinical Guidelines
Evaluations
Experts
Journals
Results of Performance Improvement Activities
SOME THINGS YOU CAN DO RELATIVELY
EASILY…
1. Look at your planning process:
Odds are, the answers are already there. You just need to tap into them.
A. How do you decide to do educational activities?
B. Who decides to do educational activities?
C. What are they basing their decision on?
11/15/2010
9
SOME THINGS YOU CAN DO RELATIVELY
EASILY…
2. Look at your data – does the data address practice gaps specifically?
A. Practice guidelines often specify practice gaps.
B. Evaluation/Survey Data: Should the questions be updated to assess practice gaps more specifically?
“What topics would interest you for future courses”
vs.
“What practice areas could you use additional training in to practice more effectively?”
SOME THINGS YOU CAN DO RELATIVELY
EASILY…
C. If you’re using published research, how does it
relate to physician practice?
D. Other sources?
• Public Adverse Events
• New technologies approved by FDA
SOME THINGS YOU CAN DO RELATIVELY
EASILY…3. Look at your planning and evaluation tools:
Do they document the process of planning the activity based on a practice gap?
Do your learning objectives focus on what participants will be able to do?
Does your evaluation assess the practice gaps you identified?
11/15/2010
10
SOME EASY EVALUATION
QUESTIONS THAT MAY HELP…
My ability to [insert Learning Objective #1] is:
Example:
My ability to define the difference between needs assessment and practice gaps is:
- Significantly increased
- Somewhat increased
- About the same
My ability to identify evidence-based sources to gather practice gap data is:- Significantly increased
- Somewhat increased
- About the same
What will you change in your practice as a result of attending this session?
AUDIENCE RESPONSE
My ability to define the difference between needs assessment and practice gaps is:
- Significantly increased
- Somewhat increased
- About the same
AUDIENCE RESPONSE
My ability to identify evidence-based sources to gather practice gap data is:
- Significantly increased
- Somewhat increased
- About the same
11/15/2010
11
SOME THINGS YOU CAN DO RELATIVELY
EASILY…
4. Everything should line up:
Practice Gaps: What physicians need to be able to do…
Learning Objectives: What physicians will be able to do as a result of the activity…
Evaluation: What physicians indicate they are able to do as a result of the activity
Follow-up Evaluation: What physicians did as a result of the activity.
TAKE HOME MESSAGES
The most important thing is to shift how you are thinking about “needs assessment” to focus on “practice gaps.”
In most cases, the data and information you need is already there or from the same resource – it’s all in how you identify and document it.
You can use the same tools you have been, but you may want to update the language to ease documentation needs. Better to start now!
ADDITIONAL RESOURCES
National Quality Forum:
http://www.qualityforum.org/
National Guideline Clearinghouse: http://www.guideline.gov/
ACCME tutorials: www.accme.org
11/15/2010
12
FINAL THOUGHTS…
Thank you
Questions?
QUESTIONS
We welcome your questions and comments.
Please use the chat box on the left side of your screen to submit a question.
THANK YOU!
Please take a moment to complete the brief
evaluation.
We appreciate your feedback!
11/15/2010
13
THANK YOU!
Join us for the Alliance’s next Takeout Tuesday webinar!
Engaging Physicians in Quality Improvement by
Teaching a Fair Culture: Partnering of CME, Medical Staff, and QI Leaders
Tuesday, December 14, 2010
2:00 – 3:00 PM ET
www.acme-assn.org