Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable...

Reliability Theory Concepts: (How to be a Coach)

Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System

Roger Resar MDSenior Fellow Institute for Healthcare

Improvement

Sept 2012Sept 2012

© 2000 Institute for Healthcare Improvement

Session Objective

Learn how to evaluate improvement work for reliable design concepts

Highlights of Reliable Process Design

(See Word Document Handout for Details)

Key Questions To Analyze Testing and Implementation

Key Question Your EvaluationIs the connection between goals and process clear?

Is the design strategy primarily vigilance and hard work?

Has some degree of segmentation been used to test the design?

Is standard work with testing been part of the design?

Is a design methodology being used?

Are small tests of change being used in a rapid cycle?

Is data collection rapid enough?


Testing :Connection Between Goals and

Process

Key Question: Is the connection between goals

and process clear?


Prime Importance

Designing and testing a process or a sequenced set of processes which fails to connect to the outcome goal makes no sense


Outcome Goals Key Drivers Processes

Decrease Falls Reliable risk assessment of patients for falls

Red socks for at risk patients


Outcome Goals

Key Drivers Processes

Decrease Falls

Toileting in at risk patients

Every 2 hour toileting rounds on at risk patients










Implementation and SpreadVigilance and Hard Work as a

Design Strategy

Key Question: Am I guilty of hard work and vigilance

as my only design strategy?

Improvement Concepts Associated with less than 95% Performance (Primarily can be described as intent, vigilance, and hard work)

Common equipment, standard order sheets, multiple choice protocols, and written policies/procedures

Personal check lists

Feedback of information on compliance

Suggestions of working harder next time

Awareness and training

Improvement Concepts Associated with 95% Performance (Uses human factors and reliability science to design sophisticated failure prevention, failure identification, and mitigation)

Decision aids and reminders built into the system

Desired action the default (based on scientific evidence)

Redundant processes utilized

Scheduling used in design development

Habits and patterns know and taken advantage of in the design

Standardization of process based on clear specification and articulation is the norm


Hard Work and Vigilance

Is a commendable trait, but not a good design strategy

Is necessary but not sufficient to attain and sustain 95% performance

Feels comfortable because we have been trained to expect hard work and vigilance in our colleagues

I would be So Bold as to Say:

If human factor concepts do not make up at least 25% of the improvement effort on a given project, chances are the project is

stalled or will never achieve sustained 10-2 performance

How to measure the human factor component

• Determine the number of tests of change involving human factors testing

• Study the finished design

• Look at the defects generated and the approach to solving them










Testing and SpreadSegmentation and testing designs

Key Question: Have we taken advantage of

segmentation?


Why Segmentation is Helpful

Allows for the control of some variables

Defines the boundaries around which sequential expectations for success can be found

More likely to test the validity of the design rather than deal with barriers

Fosters a deeper understanding of the design complexity required for the project

Forces understanding of the differences between segments as design strategies

Allows the formation of more predictable timelines

Finding your first segment• The segment must represent a reasonable volume

• The segment should have clear cut defined boundaries

• The segment should have willing participants so the barrier of agreeing is not a problem

• The segment should allow for key articulated variables or barriers to be neutralized

• The first segment should establish a design theme

20

Identification of Other Segments

• The total number of segments for a topic should not exceed 4-5

• Segments should follow some theme in design (route of admission, type of physician, etc)

• Segments should differ by a distinct design feature

• The initial division of segments can be adjusted as the design is developed

• The segments should cover the population involved in the topic

Team Results IHI Reliability Collaborative









Testing and ImplementationStandardization versus Standard

Work

Key Question: Is the team committed to really setting up

standard work?

Standard Work

• Standard work allows for the training of new employees and the testing of current employees

• If 5 front line process users can describe the process you have a good chance to achieve 95% performance and to sustain the performance over time

The Pneumovax Example

• Commonly “Give Pneumovax if indicated”

• Poorly defines a process

• Default is too commonly not to give the Pneumovax

• No testing of competency or training of new employees can occur

New Standardization Concepts

• Standardize to provide the appropriate infrastructure (the how, what, where, who and when)

• The “what” we are standardizing is based on medical evidence

• The “how” does not need medical evidence but rather on systems knowledge

• Initial standardized protocols with small time investment by experts tested at a very small scale

• Changes to the protocol in the initial stages should be required and encouraged

• Defects are studied and used to redesign the process

Key Learning Point

A single standardized process within the acceptable science is superior to allowing multiple processes while we decide which is the best because it allows testing for competency and training new employees









The Three Step Design for Reliability

Design Techniques Steps1-Identify the process to standardize

2-Segment the population to test the

design for anomalies

Prevent initial failure by standardizing the

process to achieve 80% (step 1)

1-Utilize a robust HF concept to make visible failures from step 1 after step 1 has achieved 80% reliability

2-Once the failure is identified, apply an action to mitigate the failure

Identify failures in step 1 and apply an action to achieve 80% for these failures (step 2)

1-Identify common failures

2-Develop a method to measure and study failures

3-Utilize knowledge of common failures to redesign either step 1 or step 2

In either step 1 and/or step 2 detect the failures, and use the knowledge from analysis of the failures to redesign (step 3)

Methodology High Points

• Perfection is the enemy of reliable design (step one 80%; step two 80%)

• People who use the design do the designing

• Constant testing based on what defects were observed

• Segmentation tests the design










TestingRapid Cycle Change Methodology

Key Question: How frequent are the cycles of

testing?


How to Evaluate

Are testing cycles being used on an acceptable basis?

Are huddles occurring after the tests for quick redesign?

Are records being kept of the tests?

Do all team members have test responsibilities?

Critical Failure Mode Essentials

• A measurement of critical failure modes needs to be part of the initial design strategy

• Assesses the defects that occur from the current design

• Should be prioritized in terms of overall affect on the reliability of the process change

• Should be used to redesign the process









Testing and ImplementationData Collection and

Measurement

Key Question: Are small samples being collected by the team as the project is ramped up and placed

on a run chart?

Measurement

• Small samples over time should be use to determine if the process is improving

• Data should be collected by the team with strict attention to the agreed upon tempo

• Data can initially be collected for segments

• Process measurements should be the primary team measures

• Outcome measures are needed but do not need to be collected by the team

• Outcome aims can be set at 0 or 100%, but your process aims should be 95% (10-2)

5 charts/day run chart

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

day 1 day 2 day 3 day 4 day 5 day 6

5 charts/day run chart


Testing Implementation and SpreadHitting the Wall

Key Question: What should we do when the team just seems not to be making

headway?


1-Use the 7 Question Analysis for testing and implementation 2-Check the rules of engagement

3-Use the spread analysis

What Teams Should Expect From Leadership

• Clearly describe the organizational outcome goals (VAP, CLI, Mortality etc)

• Understand the relationship between the processes the teams are working on and the outcome goals of the organization

• Set process expectations for the teams all elements of the ventilator bundle will be done 95% of the time on eligible patients)

• Demand data to show how reliable the process has become

• Setting reasonable timelines

• If outcomes have not improved and process reliability is high provide resources to determine the “correctness of performance” of the processes

What Leaders Should Expect of Teams to Reliably Achieve the Organizations Safety Goals

• Expect the Initial focus of work should be on “getting the process right” with a known connection to an outcome

• Expect the team to take a set of processes to an agreed upon level of reliability within a specified timeline

• Expect the teams to use good design principles in improvement work not just hard work and vigilance

• Expect teams to develop good designs by using rapid cycle small tests of change

• Expect adequate process structure to sustain the work

Key Questions To Analyze Spread

Key Question Your EvaluationHave you repeated the small test cycles as you spread from the initial site

Is the process of spread dependent on one person

Has some degree of segmentation been used to spread

Has customization been allowed or encouraged

Is the same team who developed the pilot now responsible for spread

Have you shifted your focus from process reliability to outcomes too early

Date post:	03-Jan-2016
Category:	Documents
Upload:	brent-jacobs
View:	215 times
Download:	2 times

Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable...

Documents