Welcome Back

Quality Leadership AcademySession 3

“How Do You Know it Works?”Anna Roth, RN, MS, MPH

Report on Projects to Date

• Refine and refresh aim statements

• Share results of our small tests of change

• Will or did you revise your test of change?

• If so, what would/did you revise and why?

Theory

Today’s ObjectivesReview results of your project

Review small tests of change

Review techniques for organizing and displaying data for maximum impact

Your toolkit- Driver Diagram

Share examples of reports designed to get the attention of those who need the information

Action Planning

How will we know?

Why Else Should We Measure?

• You can’t manage what you don’t measure

• How else would you know that your steps are making things better or worse?

• It’s often cause for reward, recognition and celebration

Choosing appropriate statistics

Median v Mean

• 10 people are on the bus• The mean income of the

riders is $50,000/yr• The median income of

the riders is $50,000/yr

• What does this tell us?????

Median v Mean

• The median income of the riders remains $50,000/year

• The mean income is now approx $50 million

• So is the average income of bus riders now $50 million because Bill Gates got on the bus?

Mean

Mean (average)Measures the center, or

middle, of a numerical data set

The sum of all the numbers divided by the total number of numbers

May not be a fair representation of the data

Easily influenced by outliers

Median

Median Also measures the center of a

numerical data set Much like the median of an

interstate highway The point at which there are an

equal number of data points whose values lie above and below the median value

Is truly the middle of the data set

Better measure of CT than the mean when there are outlying values in the data set

Percentage or Percentile?

• Suppose your score on the GRE was reported to be the 80th percentile

• Does this mean you scored 80% of the questions correctly?

Honest Errors

• Arithmetic errors or omissions– Check to see if

everything adds up– Double check even the

basic calculations– Verify the total to put

results in proper perspective; if sample size really small you may not want to use

Excercise

Report Out

Back in 15 minutes

Finding your way/Telling your story

Data Display and Analysis

• How do you want to tell your story??

• Who are you going to tell your story to?

Common types of data display

• Pie charts• Bar graphs• Tables• Time charts• Run charts• Control charts

Charts and Graphs and Spiders Oh My

• Watch for pitfalls• Size matters! • Be aware of tick marks

on the y-axis• 10s, 20s, 100s, 1000s?• Check the scale to put

results in perspective

Sizing up a pie chart

• Do the percentages add up to 100

• Beware of slices that are called ‘other’ if they are larger than many other slices of the pie

• Look for a reported total number of units so you can see how big the pie was before it was divided up

UCL

LCL

X

Indi

cato

r

Time

An indication of a special cause

Elements of a Control Chart

Non-Random Rules for Run Charts

VariationCommon Cause vs. Special Cause

Common causeAlways presentInherent in processIs due to regular, natural,

ordinary causesResults in a stable process

that is predictable

Special causeAbnormal, unexpectedDue to causes not inherent

in processAlso known as non-random

or assignable process

Special cause• Identify and study special

cause• If negative, minimize or

prevent• If positive, build into

process

Appropriate Actions to TakeCommon causeIf undesirable need to

change the process.If only common cause

variation and treat as special cause (tampering), leads to greater variation, mistakes, defects

First 24 Observations from Red Bead Data

(without outlier employee)

12 Runsexpect to find between 8 and 18

runs

On Death, Dying & Data

DENIAL

ANGER

BARGAINING

DEPRESSION

ACCEPTANCE

On Death, Dying & Data

DENIAL“The data are wrong”

ANGER“The data are right, but it’s not a problem”

BARGAINING“The data are right; it is a problem; but it is

not my problem.”

DEPRESSION“This feels too hard to do”

ACCEPTANCE“I accept the burden

of improvement”

Stages of Facing Reality: “To live divided no more”

• “The data are wrong”• “The data are right, but it’s not a problem”• “The data are right; it is a problem; but it is

not my problem.”

“I accept the burden of improvement”

39

Crimson Bead Company

“Every system is perfectly designed to achieve the results that it achieves”

Berwick: central law of improvement BMJ1996 312:619-622

Discussion

Oversight

Oversight

Project-level e.g.• % AMI patients getting

evidence-based care• % Pneumonia patients

getting evidence-based care• Time to answer call light on

5 West

System-level e.g.• Hospital mortality rate• Cost per admission• Adverse drug events per

1000 doses• Patient satisfaction scores

Lesson #3 Execution

46

Projects Connected to Big Dots

* Mortality Rate* Cost per Admission

* Adverse Events* Functional Outcomes* Patient Satisfaction

* 3rd Available Appointment* Voluntary Turnover

* Condition-specific, clinical process indicators

* Preventive care measures* Office visit cycle time

* ER to bed placement time* PACU to bed placement time* ICU to bed placement time* Bed to LTC placement time

* ICU mortality* Catheter related BSI

* Average ventilator days per patient * Adverse events/ICU day

47

* Surgical Site Infection Rate* Percent of un-reconciled medications* Staff reporting positive safety climate

* Percent of turnover in first year

* Employee loyalty

A Senior Leader Perspective on Projects

The Project: e.g., Ventilator-Acquired Pneumonia

Spreading and Sustaining This Improvement

Spreading and Sustaining These Design Concepts: “A Place Where…”

Changing the Organization:•HR•IT•Finance•Leadership Processes•Business Strategy•Environmental Strategy

Issues at Each Tier (Examples)

Tier 1: Big Dot

Tier 2:Portfolio

Tier 3:Projects

Aims of strategic importance to the system as a whole “Big Dot” measure of progress Executive, Board and Senior Leader engagement Vision and the associated structural changes Strong linkage to finance Learning and mitigation of risks Managing the learning, the politics, and the risks

Understanding “drivers” and causal linkages Outcomes of consequence tracked over time Middle Management key“Connecting the Dots” – putting the learning together Continual readjustment of portfolio Strong linkage to finance Some structural changes (e.g., job roles) Team organization and capacity matter Process and outcome tracked over time Leaders remove obstacles Change concepts help Ability to run PDSA cycles Temporary infrastructures facilitate progress

Project Level Measure (Tier 3)• Family assistance• May 05 to Oct 06: 17 months of NO VAP’s• IHI Mentor Hospital

• Bundled orders with opt out• 30 degree head of bed elevation

marked on walls with tape• Now spreading to floor beds post

extubation

“One Patient, One List”

Project Level Measure (Tier 3)

Project Level Measure (Tier 3)

• % meds unreconciled:admission 25% 3%• % meds unreconciled:transfer 12% 4%• % pre-admit meds unreconciled 19%1%• % of patients with ANY unreconciled

meds decreased from 36% 3%

• Discharge….still testing

Driver Diagrams

PrimaryDriversOutcome

SecondaryDrivers

ProcessChanges

AIM:A New

ME!

Calories In

Limit dailyintake

TrackCalories

CaloriesOut

Substitutelow calorie

foods

Avoidalcohol

Work out 5days

Walk toerrands

PlanMeals

Drink H2ONot Soda

drives

drives

drives

drives

drives

drives

drives

drives

What Changes Can We Make?Understanding the System for Weight Loss

“Every system is perfectly designed to achieve the results that it gets”

How Will We Know We Are Improving?Understanding the System for Weight Loss with Measures

PrimaryDriversOutcome

SecondaryDrivers

ProcessChanges

AIM:A New

ME!

Calories In

Limit dailyintake

TrackCalories

CaloriesOut

Substitutelow calorie

foods

Avoidalcohol

Work out 5days

Walk toerrands

PlanMeals

Drink H2ONot Soda

drives

drives

drives

drives

drives

drives

drives

drives

• Weight• BMI• Body Fat• Waist size

• Daily caloriecount

• Exercisecalorie count • Days between

workouts

• Avg drinks/week

• Runningcalorie total

• % ofopportunitiesused

• Sodas/week

• Meals off-plan/week

• Avg cal/day

Etc...

Measures let us• Monitor progress in improving the

system• Identify effective changes

AIM Primary Driver Secondary Driver

• At your tables write down 4-6 primary drivers for your project

• For each primary driver, come up with 2-3 secondary drivers

• If you have time, write a few small tests of change for each secondary driver

Report Out

Tying it together

Transforming Care at the Bedside (TCAB)

Med-Psych Workgroup

Clinical Informatics

ED Safety

Central Line Infection Team

Multidisciplinary Rounds

Rapid Response TeamOffice Practice Team

Perinatal Impact Team

Total Joint Team

VAP Prevention Team

Perioperative Care

Medication Reconciliation Team

Care that is;

safe, effective, patient-

centered, timely, efficient and equitable

Staff satisfaction

Involve Patients in all improvement teams

Involve ethics in all improvement and operations

Culture of continuous quality improvement

Build Innovation engine

Mortality-RRT, Sepsis Medication safety Falls Pressure Ulcers Re-admissions– Transitions Harm/Adverse events Infection-SSI,UTI,VAP,MRSA

Ownership of agreed upon set of outcomes Review of outcomes at each meeting Quality and safety comprises 25% of agenda Involve patients in safety Visible on all senior leader agenda Culture of Safety/Fair and Just

Shared meaningful vision from Board to the patient

Expert at communication and marketing methods coaching

Program design and structure

Infrastructure supports improvement measurement

Clear, shared measurement set

Inventory national programs and measurements

Recovery plans for unmet outcomes

Strengthen IT infrastructure

Secondary Drivers Primary Drivers

OPERATIONS/QUALITY DRIVERS

Leadership and Culture

Deliver the Program

Measurement

Communication

Capacity and Infrastructure

System Level Aims

System Level Aims

Primary System Aims

Additional System Level AimsZero Hospital acquired infections

Patient overall satisfaction to be >90%

Readmission rate to decrease by 30%

Planned System Level Aims to begin by 2010

Eliminate inequality in at least ten improvement /operational areas by 25%

Reduce Ambulatory Care Sensitive Admissions (ACS) to CCRMC by 15%

Patient engagement on every innovation and improvement team by January 1, 2010

Develop a formal process for engagement of ethics expertise in operations and quality improvement.

Prophylactic Antibiotics One Hour Prior to Incision

Hours of Behavioral Restraint Use

Inpatient Psychiatry: Discharge Care Planning

VAP per 1000 Ventilator Days

11.610.8

1.5 1.3

3.1

0

2

4

6

8

10

12

14

2003 2004 2005 2006 2007

Ventilator Days were 777 in 2006 and 645 in 2007

VAP per 1000 Ventilator Days

Number of VAPs and Ventilator Days

CCRMC 30 Day Readmission Rates

Heart Failure Discharge Instructions Given

Heart Failure Discharge Instructions Given

Aiming for Perfect Care

•Discharge Instructions

•Evaluation of LVS Function

•ACEI or ARB for LVSD

•Adult Smoking Cessation Advice/Counseling

Percent of Patients Who Received All Heart Failure Interventions at CCRMC

Percent of Patients Who Received All Heart Failure Interventions at CCRMC

All-or-Nothing Measurement

Why the time is now

Who will if not you?

What can you do by next Tuesday?

Thank you

Anna Roth, CEOContra Costa Regional Medical Center

aroth@hsd.cccounty.ussafetynethospital.blogspot.com