Quality Improvement: Unlock Your Hidden Superpowers · 2018-05-11 · Quality Improvement: Unlock...

MSHP Annual Meeting 2018 #MASHP18

Quality Improvement Projects 1

Quality Improvement: Unlock Your Hidden SuperpowersLindsay Arnold, Pharm.D., BCPS

Kevin J. Horbowicz, Pharm.D., BCPS

William Vincent, Pharm.D., BCPS, BCCCP

Department of Pharmacy

Boston Medical Center

mmPharmD

Disclosures

• Lindsay Arnold, Kevin Horbowicz, and William Vincent have nothing to disclose



Learning Objectives

1. Explain the rationale for utilizing formal quality improvement and improvement science to drive improvements in patient care and deliver value to your organization

2. Describe the elements of the Model for Improvement to solve problems at your organization

3. Explain the importance of representing data over time for your improvement project

4. Discuss a plan to build the improvement capability of your residency training program

An Improvement Zealot’s Story



Our traditional tools aren’t good enough

It’s time to use the right tools for the job



ASHP has shifted the focus towards QI2005 StandardsGoal R4.1: Conduct a practice-related project using effective project management skills.

2014 StandardsGoal R2.2: Demonstrate ability to evaluate and investigate practice, review data, and assimilate scientific evidence to improve patient care and/or the medication-use system.

OBJ R4.1.1: (Synthesis) Identify a topic for a practice-related project of significance.

Objective R2.2.1: (Analyzing) Identify changes needed to improve patient care and/or the medication-use system.

OBJ R4.1.2: (Synthesis) Formulate a feasible design for a practice-related project.

Objective R2.2.2: (Creating) Develop a plan to improve patient care and/or the medication-use system.

OBJ R4.1.3: (Synthesis) Secure any necessary approvals, including IRB and funding, for one’s design of a practice-related project.

OBJ R4.1.4: (Synthesis) Implement a practice-related project as specified in its design.

Objective R2.2.3: (Applying) Implement changes to improve patient care and/or the medication-use system.

OBJ R4.1.5: (Synthesis) Effectively present the results of a practice-related project. Objective R2.2.5: (Creating) Effectively develop and present, orally and in writing,

a final project report.OBJ R4.1.6: (Synthesis) Successfully employ accepted manuscript style to prepare a final report of a practice-related project.

OBJ R4.1.7: (Evaluation) Accurately assess the impact, including sustainability if applicable, of the residency project.

Objective R2.2.4: (Evaluating) Assess changes made to improve patient care or the medication-use system.

https://www.ashp.org/professional-development/residency-information/residency-program-directors/residency-accreditation/accreditation-standards-for-pgy1-pharmacy-residencies

When to use a QI framework

• Quality improvement projects are typically designed, or intended to:

• Improve patient care

• Compare a program/process to an established set of standards such as standard of care, recommended practice guidelines, or other benchmarks

• Improve the performance of institutional practice or local systems

• Bring about improvements in health care delivery

Adapted from Boston Medical Center’s Quality Improvement (QI)/Quality Assurance (QA) Activities versus Regulated Human Subjects Research checklist. Updated July 21, 2014.



When to use a traditional research framework

• Human Subjects Research is defined be the United States Secretary of Health and Human Services as a…

• “systematic investigation, including research development, testing and evaluation designed to develop or contribute to generalizable knowledge.”

US Department of Health and Human Services. Protection of Human Subjects: 45CFR 46. 2005; http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html. Accessed March 22, 2018.

Differences between QI and clinical researchCharacteristic Improvement Science Clinical Research

IntentImprovement of care system, process (i.e. health care delivery)

New generalizable knowledge

Test observability Test observable Test blinded

BiasAccept consistent bias, obsess about bias

Design to eliminate bias

Sample size“Just enough” data; small sequential samples

“Just in case” data

Flexibility of hypothesisFlexible; changes as learning occurs

Fixed

Testing strategy Sequential tests One large test

Determining if change is improvement

Run & Shewhart (control) chartsHypothesis tests (t-tests, chi-square, p-values)

Provost LP, Murray SK. The Health Care Data Guide. First edition;2011:p27.



Model for Improvement

• To be used in conjunction with various change models to accelerate improvement work

• Refers to the selecting, testing and implementation of change

• Primary components• Three fundamental questions

• Plan-Do-Study-Act cycles for test of change

• Right team members integral to successful project

Taylor et al. BMJ Qual Safety 2013;0:1-9., IHI, Science of Improvement: How to Improve, Accessed March 17, 2018.

IHI, Science of Improvement: How to Improve, Accessed March 17, 2018

• Setting The Aims• Define the population or system of interest

• Measurable & time specific

• Establishing Measures• Quantitative methods

• Selecting Changes• Ideas from those working with the system

& those with QI knowledge

• Testing Changes• Testing change in the real world



Anatomy of Aim Statement

1. State the aim clearly.

2. Include specific numerical goals that require fundamental change to the system.

3. Set stretch goal.

4. Avoid aim drift.

5. Be prepared to refocus.

“Decrease medication errors.”

“Improve STAT medication turn around time.”

“Increase sedation vacations in the cardiac ICU by 15% by June 1st, 2018.”

“Decrease median time to antibiotic administration to <1 hour in pediatric ICU patients in 3 months.”

IHI How to Improve. Tips for Setting Aims. Accessed April 15, 2018.

Taylor et al. BMJ Qual Safety 2013;0:1-9.

Plan

DoStudy

Act

DoCarry out the change. Small can be meaningful. Document problems. Begin reviewing data.

ActAdopt the change, abandon it, or repeat the cycle again. What changes do you need to make? What will your next PDSA be?

StudyComplete analysis & examine the results. What did we learn? What went wrong?

PlanPlan a change. What do you expect to find? What will you do? Who will do it?



Cycle of Change

Wyoming Department of Public Health, Accessed March 17, 2018.

Plan-Do-Study-Act: Not always what they seem

Ogrinc et al. BMJ Qual Saf 2014;23:265-267.



The Big DifferencesCharacteristic Improvement Science Clinical Research

IntentImprovement of care system, process (i.e. health care delivery)

New generalizable knowledge

Test observability Test observable Test blinded

BiasAccept consistent bias,obsess about bias

Design to eliminate bias

Sample size“Just enough” data; small sequential samples

“Just in case” data

Flexibility of hypothesis

Flexible; changes as learning occurs

Fixed

Testing strategy Sequential tests One large test

Determining if change is

improvement

Run & Shewhart (control) charts

Hypothesis tests (t-tests, chi-square, p-values)

Provost LP, Murray SK. The Health Care Data Guide. First edition;2011:p27.

0

10

20

30

40

50

60

70

Pre-Intervention Post-Intervention

Time to Antibiotic Administration

Traditional Research: Pre vs Post Analysis

• The ED recently completed an intervention with the goal to decrease time to administration of antibiotics

• Outcome metric: Time to antibiotic administration

• Baseline data were collected for 7 weeks prior to initiation of the intervention

Pre-intervention Post-intervention

63 minutes 41 minutes

p <0.01



Displaying data over time is fundamental to QI

• Makes performance visible

• Enables team to determine if a change resulted in improvement, in real-time

• Reveals to team if it is sustaining the improvement made by tests of change in PDSA cycles

• Allows for a temporal or dynamic view of data versus a static view

Sample Comparisons

Pre vs Post Analysis



Sample Comparisons


Sample Comparisons




Sample Comparisons


Sample Comparisons




Examples of Data Over Time – IV Acetaminopen Use and Turnaround Time for STAT Meds

25Am J Health-Syst Pharm 2018; 75:e125-32

Pediatric Antibiotics – Time to Administration



Code blue pharmacist response process development and implementation QI project

▪ Average time spent at a code was 27 minutes

▪ RPH prepared medication at ~40% of codes

▪ RPh made medication recommendations ~34% of codes

▪ Recommendations were related to ACLS more than 12.5% of the time

▪ RPh facilitated medication deliveries ~44% of the time (most commonly amiodarone)

Pharmacist participation has dramatically increased since rollout…… and interventions appear to be

meaningful

15.4%

50.0%

44.4%42.9%

6.3%

46.2%

40.0%

100.0%

80.0%

88.9%

July

Aug

ust

Sep

tem

be

r

Octo

be

r

No

vem

ber

De

cem

ber

Jan

ua

ry

Fe

bru

ary

Ma

rch

Apri

l

Ma

y

Jun

e

Median

Pharmacist

Training Begins

Go-live day

and evening on

one campus

Go-live day

and evening

on second

campus

Why QI for BMC Residency Programs

• Need to connect resident-led projects to department, institution for services and program expansion

• Doing good research is difficult• 1 year is not enough• Often single center, observational research

has limited validity = difficult to publish

• Create a new identity for our residency programs → new set of super powers for graduates• Formal training needed to bridge the

“chasm” between residency and practice

Am J Health-Syst Pharm 2013;70:1610-5Am J Health-Syst Pharm 2017;74:152-61



QI Interventions for Our Residency Programs

• Cultural and practice model shift

• Institute for Healthcare Improvement open school, basic certificate in quality & safety, and national forum attendance

• Institutional Review Board support for quality improvement

• Timeline for our residents

• QI Macro or other data over time software

• Knowledge and attitudes assessment

• Scholarship committee seminar series

• Resources library

Do the Work and Improve the Work

“It is every employee’s responsibility to do the work and improve the work.”

The IHI Open School has increased our quality improvement literacy



Institute for Healthcare Improvement Basic Certificate in Quality and Safety

Quality Improvement Activities vs. Regulated Human Subject Research Checklistsa,b – Sample

Checklist 1 Y N

1.Are all persons who get the intervention expected to benefit?

2.

Is the purpose of measurement related to your project to determine the effect of process change, measure performance, or for submission to a national or state registry and/or database that is authorized to improve the delivery of clinical care?

3. Is the purpose of this project to improve the process or delivery of care consistent with established quality standards?

4. Will all persons involved in the initiative receive standard of care at a minimum?

5.

Does the initiative involve implementation of evidence or consensus based quality standards or best practice to all persons involved in the project?

6. Does the project involve systemic data collection to monitor and compare performance to defined standards?

7.

Will the project be described as “quality improvement” in public presentations, academic curriculum vitae, publications, and/or other representations to any third party audience?

Checklist 2 Y N

1. Is the intent of the project either to test a novel hypothesis, answer a research question or replicate another researcher’s original study?

2. Does the project seek to test interventions, treatments or practices that are not currently considered standard of care (neither consensus-based, nor evidence-based)?

3. Does the project involve suppressing any aspect of standard of care?

4. Is the intent of the project to design or develop a new standard of care or benchmark?

5. Will the physician and/or staff be blinded to any aspect of the patient’s care?

6. Will persons (including patients and investigators) be exposed to risks beyond standard of care?

7. Will the project involve a practice that over-rides clinical decision-making?

8. Does the project involve a medication or instrument not used outside of usual medical practice, or for evaluation of any off-label uses of Food and Drug Administration (FDA) approved medications, devices, or non-FDA approved agents?

9. Does the project involve external funding, participation by entities outside of the clinical setting or organization, or backing by a body that requires IRB approval?

10. Will the project be described as researchc in representations such as publications, presentations, or academic dossier?

aBorrowed with permission from the Boston University Medical Campus Institutional Review BoardbInstructions: Complete both checklists. If the answer to all questions in Checklist 1 is “yes,” Institutional Review Board submission is generally not required. If the answer to any questions in Checklist 2 is “yes,” Institutional Review Board submission is required.cQI/QA findings, may be published but should not be characterized as research.



Residency QI Project Gantt Chart – Sample

Timeline

• Q2 heavy – baseline data analysis, testing interventions, timeline development

• Limited waiting on others, except for IT• No IRB

• Mostly small scale, informal tests of change initially

• Timeline is not linear – harder to translate into manuscript format vs. research

QI Macro Gave Us the Power to Represent Data Over Time



Transition from Research to QI

Year 1: 2015-16• Provided limited workshops to pharmacists that

focused the Model for Improvement, data over time, using the QI Macro, and sampling

• Revised project proposal process to require explicit connection to organization’s QuEST goals

• Selected projects that advanced departmental or organizational priorities and matched them with each residents’ interests

• Required the completion of the IRB QI vs. research checklist

• Provided limited formal guidance to pharmacists supervising the projects or residents leading the projects

• Sent two pharmacists to IHI National Forum in December 2015

Year 2: 2016-17• Required residents to complete IHI Open School

courses

• Began to require some pharmacists to complete IHI Open School courses or the IHI-Harvard edX course on quality improvement

• Developed a more comprehensive series of workshops on the Model for Improvement, fishbone and driver diagrams, data over time, sampling, and control charts

• More experienced pharmacists in quality improvement provided direct support and guidance for some of the projects

• Sent two pharmacists to IHI National Forum in December 2016

• Outgoing resident knowledge and attitudes assessment

Year 2: 2016-2017 Residency Class

Item (1=strongly disagree, 2=disagree, 4=agree, 5=strongly agree)

Mean SD

Apply the IHI FMI 3.9 0.4

Write a clear and effective aim statement 4.6 0.5

Identify outcome, process, and balancing metrics 4.0 0.6Describe change theory (e.g. fishbone, driver diagrams, maps)

3.7 0.5

Conduct PDSA cycles 4.0 0.6Prioritize interventions based on degree of belief and feasibility

3.9 0.7

Explain the importance of measuring data over time 4.6 0.8Explain the benefit of data over time compared with summary statistics for data for improvement

4.1 0.9

Create and interpret run charts 3.7 1.0

Create and interpret statistical process control charts 3.3 1.0

Apply the SQUIRE 2.0 guidelines to write a manuscript 3.7 1.0

Resident knowledge of performing quality improvement project activities → lowest scores in analyzing data over time, publishing

CategoryNo. of

Questions% Correct

Residency learning experience

4 68%

Introduction 5 63%

Methods 9 67%

Results 4 14%

Discussion 3 67%

Total 25 58%

Table 1 – Attitudes Assessment (n=7) Table 2 – Knowledge Assessment (n=7)




• Aims (by June 2018)• 100% of our residents and preceptors will complete the IHI open school basic certificate in

quality & safety• Average score > 70% for residents and preceptors on a pharmacy improvement science

exam• Submit 50% of our project manuscripts for publication, publish one!

• Change concepts• Expand scholarship committee membership to support a larger role in curriculum

development, implementation, and assessment (in addition to project assistance)• Install MS Excel QI macro on all department computers• Promote SQUIRE 2.0 guidelines through an Asana project template• Develop a QI project toolkit• Increase preceptor participation in resident projects


BMC Scholarship Committee QI Seminar Series• Introduction to quality improvement – August 11,

2017

• Change theory – September 1, 2017

• Metrics – September 8, 2017

• Data for improvement – September 21, 2017

• IHI charter – September 29, 2017

• Biostatistics – October 12, 2017

• Sampling – October 20, 2017

• Control charts 1 of 1 – November 9, 2017

• Control charts 2 of 2 – December 14, 2017

• Presenting QI reports – March 18, 2018

• Writing a manuscript – May 18, 2018

Scholarship Committee Requirements for Project Preceptors• All preceptors

• Complete IHI open school basic certificate in quality & safety

• Secondary preceptor• QI project committee

experience/member and/or led short-cycle QI project

• Primary preceptor• Served as secondary preceptor on

previous resident longitudinal project• Led short-cycle QI project




Increased number of resident-led QI projects over time

0

1

2

3

4

5

6

7

8

9

10

2012

2013

2014

2015

2016

2017

QI Research

Pharmacy Resident Projects, By Type,

2012-2017

No

. o

f P

roje

cts

Nearly all of our preceptors have completed the IHI certificate

0

5

10

15

20

25

30

35

40

45

50

0

100

200

300

400

500

600

Ma

r-1

7

Apr-

17

Ma

y-1

7

Jun

-17

Jul-

17

Aug

-17

Sep

-17

Oct-

17

No

v-1

7

De

c-1

7

Jan

-18

Fe

b-1

8

Cumulative Courses

Cumulative Certificates

IHI Improvement Open School Coursework and Basic

Certificates in Quality & Safety Completed, Department of

Pharmacy, Cumulative Count, March 2017-March 2018

Op

en

Sch

oo

l C

ou

rse

s C

om

ple

ted B

asic

Certific

ate

s C

om

ple

ted

Month-Year

Pharmacists’ Attitudes About Completing the Institute for Healthcare Improvement Basic Certificate in Quality & Safety

Pharmacist Responses0 4 7 11 14 18

Not too much time

Learned a lot

Useful in my practice

Glad I did it

Would like to attend IHI

Will add to my CV

n=14 responses, 36% of preceptorsDisagree

Neutral

Agree



Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0) Guidelines• Background

• Available knowledge – Literature review and local problem (1 slide)• Rationale – IHI MFI, aim statement (1 slide)

• Methods• Interventions – outline/overview of interventions and timeline (1 slide)• Interventions – highlight specific key intervention(s) in more detail (1 slide)• Measures & Analysis – list outcome, process, and balancing metrics (1 slide)• Ethical considerations (1 slide) – NHSR per IRB and research committee. Include relevant conflicts and funding

disclosures

• Results• Outcome metric in SPC/run chart (1 slide)• Other metric(s) in SPC/run chart(s) (1 slide)

• Discussion• Summary & Interpretation (1 slide)• Limitations (1 slide)• Conclusions (1 slide)• Acknowledgements (1 slide)

BMJ Qual Saf 2016;25(12):986-992

But can we publish our resident QI projects?

• Only 5-10% of resident research projects are published, 40% are observational studies → time to try a different approach?

• ASHP PGY1 residency standards have evolved• 2005 – OBJ R4.1.6: (Synthesis) Successfully employ accepted manuscript style to prepare a

final report of a practice-related project.• 2014 – Objective R2.2.5: (Creating) Effectively develop and present, orally and in writing, a

final project report.

• Challenges• IRB approval may be needed – ask for formal post-hoc determination• Is it more difficult to demonstrate improvement (statistical significance) with data over

time?• Translate iterative process and describe interventions for an external audience• Medical, surgical, and nursing journals have adopted SQUIRE 2.0 standards; pharmacy has

not• Newly available Journal of American College of Clinical Pharmacy may be a forum

Am J Health-Syst Pharm 2012; 69:59-62Am J Health-Syst Pharm 2010; 67:830-6



QI Resources• Langley GL, Nolan KM, Nolan TW, Norman CL, Provost LP, eds. The improvement

guide: a practical approach to enhancing organizational performance. 2nd ed. San Francisco: Jossey-Bass; 2009:89-108.

• Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf. 2016;25(12):986-992.

• Lloyd RC. Navigating the turbulent sea of data: the quality measurement journey. Clin Perninatol. 2010;37:101-22.

• Perla RJ, Provost LP, Murray SK. Sampling considerations for health care improvement. Qual Manag Health Care. 2013;22:36-47.

• Perla RJ, Provost LP, Murray SK. The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual Saf. 2011;20:46-51.

• Provost L, Bennett B. What’s your theory? Driver diagram serves as tool for building and testing theories for improvement. Quality Progress. 2015 Jul:36-43.

• Davidoff F, Dixon-Woods M, Leviton L, Michie S. Demystifying theory and its use in improvement. BMJ Qual Saf. 2015;24:228-38.

• Ogrinc G, Nelson WA, Adams SM, O'Hara AE. An Instrument to differentiate between clinical research and quality improvement. IRB: Ethics & Human Research. 2013;35:1-8.

Summary

• Background• Early adoption of new project-related residency standards• Alignment of resident projects with care-focused priorities• Do the work and improve the work!

• Institute for Healthcare Improvement Model for Improvement offers a useful framework for improving the medication use system• Data over time preferred over summary statistics

• Quality improvement can be successfully incorporated into pharmacy residency training programs• Institute for Healthcare Improvement offers great resources• Additional tools needed (e.g. improvement science curriculum, data over time

software, timeline



Quality Improvement: Unlock Your Hidden SuperpowersLindsay Arnold, Pharm.D., BCPS

Kevin J. Horbowicz, Pharm.D., BCPS

William Vincent, Pharm.D., BCPS, BCCCP

Department of Pharmacy

Boston Medical Center

mmPharmD

Date post:	12-Jun-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Quality Improvement: Unlock Your Hidden Superpowers · 2018-05-11 · Quality Improvement: Unlock...

Documents