Ongoing Evaluation of Physician Performance:
Developing a Performance Portfolio
Cary Sennett, MD, PhDMedBiquitous Annual Conference
May 15, 2008
Overview
“Performance Portfolio”—what are we talking about?
Design Criteria—what would a portfolio look like?
One—I hope promising—effort (in a bit more detail)
Summary…
Defining terms
Portfolio: A (comprehensive) information set that a physician can use • To establish his/her qualifications• To improve care
A “performance portfolio” to improve care• Must begin with information about practice
performance• Must contain elements that speak to the
capability of those factors that determine practice performance
Elements that “speak to capability”
How my practice is performing depends on• My competence (my knowledge and skill)• How effectively that competence is deployed
“Deployment”—that is, the conversion of capability to results—depends upon environmental factors• “Micro-environment”• “Macro-environment”• And (of course) the patient…
For purposes of our discussion, it may be less important to consider the “macro-environment”
Practice performance• Are we achieving the results that we could?• What limits our ability to do better?
His or her own competence• Do I have the knowledge and skill necessary to deliver excellent
care?• And how can I close critical gaps?
(Micro)-system • Is the (micro-) system in which I function optimally configured to
support my efforts to deploy that competence?• Am I using it effectively?• And, if not, how can I improve it?
My patients• Have I maximized their ability to achieve desired health results?• And, if not, how can I do so more effectively?
So a portfolio should help a physician evaluate
Designing a portfolio (to support practice improvement)
Practice results
Physician knowledge and skill
Systems infrastructure
Patient self-management
Need for broad set of inputs
What “Practice Results?”
Must address the range of results that are relevant to patients (and other customers…)
It may be helpful to think of that range as the IOM does: care that is• Safe• Timely• Effective• Efficient (delivered at appropriate cost)• Equitable• Patient-Centered
But, in any event, it is essential to recognize that performance is multidimensional
What “Knowledge and Skill?”
ACGME (ABMS) competencies Professionalism Patient Care Medical Knowledge Communication and Interpersonal Skill Systems-based Practice Practice-based Learning and Improvement
What “System Infrastructure?”
Information management Patient activation Access and communication with patients Safety and efficiency Consultation and referral Team function Improvement process
How do we get there?
A lot of activity right now—but widely distributed
ABMS Certification/Maintenance of Certification may be a helpful “seed” around which this work can organize
Step on the path—but not the end of the road
Board Certification Professional effort to evaluate competency (capability) of
individual physician
Structured so as to offer a window into other key portfolio elements• Practice results• Practice systems infrastructure• Patient self-management
Designed—and objective is—to support improvement
But essential function is assessment (both formative and summative)
The Structure of Board Certification
Specific reference to ACGME competencies Two elements
• Initial certification• Periodic recertification/Maintenance of Certification
Structure common across 24 ABMS Boards• Assessment of actions against license• Self-assessment of medical knowledge• High stakes secure examination of cognitive
knowledge and judgment• Self-assessment and improvement of practice
performance
Boards vary with respect to implementation—strategies and timelines
Self-Assessment of Practice Performance
Designed to force “a new way of thinking” about quality and QI• Quality is about what the system produces• Quality improvement is not about working harder (or
knowing more)—but about “diagnosing and treating” system problems
Designed to promote adult (experiential) learning
Physicians can receive up to 20 units category I CME credit, as well as credit toward renewing their certificate
The ABIM Practice Improvement Module (PIM™)
Performance Report
Performance Report
Improvement
Patient survey
Impact
PlanDo
Study
Act
Practice survey
Chart reviewApply quality measures to practice
Apply quality measures to practice
Compare performance to guidelines
Compare performance to guidelines
Test a process change aimed at improving
care
Test a process change aimed at improving
care
Examine practice infrastructure and
process
Examine practice infrastructure and
process
Report what was learned
Report what was learned
The PIM as Portfolio
Provides window on practice results
Can link to other information about physician competency (knowledge and skill)
Provides window on systems infrastructure
Potential window on “patient competence”
Increasing robustness
Need broader window on practice performance—more, and more diverse—data
Need tighter and more intentional link to information about individual physician competency
Need mechanism to track (all elements over time)
Need expanded window on “patient competence”
Getting from here to there: “PIM Future” Links to extant datasets—power practice and self-
assessment off of available data
Performance (and personal) “dashboards”: turning data into information that speaks (meaningfully and reliably) to practice performance and to core drivers
Links to—truly integration with—support needed to go from• “I see an opportunity to improve” to• “I’m able to capitalize on that opportunity to improve”
What will it take?
More (more standardized and more available) data from many, many sources
More (much more) research
Collaboration/partnerships that will link assessment capability to improvement capability
Collaboration/partnerships to organize and focus energy and resources
Summary It is possible to conceive of a portfolio that will
support practice improvement
Such a portfolio will need to address a broad ranges of issues
Board certification (Maintenance of Certification) may be an important part of early efforts to build a portfolio
To get where we need to go will require collaboration—which will require much, much more standardization