Utah Association of Local Health Departments Quality Improvement Workshop September 21 st and 22 nd,...

transcript

Utah Association of Local Health Departments

Quality Improvement Workshop

September 21st and 22nd, 2011

Workshop Overview

Welcome Remarks

Introductions

Why are we here?

Expectations

Pre Test

…PHF Mission:

We improve the public’s health by strengthening the quality and performance of public health practice

Innovative Solutions. Measurable Results.

http://www.phf.org

Contact InformationJack Moran

jmoran@phf.org

T: 207- 439 – 0560

Grace Duffy

grace683@embarqmail.com

T: 352-406-8262 cell

Introduction to QI

"If you tell me, I will listen. If you show me, I will see. But if you let me experience, I will learn.“

Lao-Tse, 5th-century BC philosopher

Why Quality Improvement?

Foundation of new accreditation program

Results of investment in public health

Getting better all the time

Definition of Quality Improvement In Public Health

“Quality improvement in public health is the use of a deliberate and defined improvement process, such as Plan-Do-Check-Act, which is focused on activities that are responsive to community needs and improving population health.

It refers to a continuous and ongoing effort to achieve measurable improvements in the efficiency, effectiveness, performance, accountability, outcomes, and other indicators of quality in services or processes which achieve equity and improve the health of the community.”

Defining Quality Improvement in Public Health; Journal of Public Health Management & Practice: January/February 2010 - Volume 16 - Issue 1 - p 5–7, Riley, William J. PhD; Moran, John W. PhD, MBA, CQIA, CQM, CMC; Corso, Liza C. MPA; Beitsch, Leslie M. MD, JD; Bialek, Ronald MPP; Cofsky, Abbey -

Continuous Improvement

The continuous improvement phase of a process is how youmake a change in direction.The change usually is because the process output is deterioratingor customer needs have changed

INDIVIDUAL

Turning Point

Baldrige

Daily Management

Big ‘QI’Little ‘qi’

Individual ‘qi’

QI Teams

Rapid Cycle Advance Tools of

Basic Tools of QI

Continuous Quality Improvement System in Public Health

General Approach On How To Use The Basic Tools Of Quality Improvement

Issue ToConsider

Flow ChartExisting Process

Brainstorm& ConsolidateData

Cause & EffectDiagram – Greatest Concern

Use 5 Whys ToDrill Down ToRoot Causes

Gather DataOn Pain Points

Translate DataInto Information

• Pie Charts• Pareto Charts• Histograms• Scatter Plots, etc.

Flow ChartNew Process

Monitor New Process & Hold

The Gains

• Run Charts• Control Charts

Data ManagementStrategy

“As Is” State to “Should Be” State

“As Is” StateBrainstormingForce and Effect

Analyze Information andDevelop Solutions

Solution andEffect Diagram

“AIM”

Quick Check Of Your

Enthusiasm Level

Mathematical Skills

elHigh

Low High

Mathematical Skills

Low/Low

High/Low High/High

Low/High

elHigh

Low High

Mathematical Skills

Low/Low

High/Low High/High

Low/High

Plan•Strategic•Preventive Assure

•Tactical•Preventive

Control•Operational•Real time

Inspect•Operational•After the fact

QA, QC, QI

They Are Not the Same

Quality Assurance

Reactive

Works on problems after they occur

Regulatory usually by State or Federal Law

Led by management

Periodic look-back

Responds to a mandate or crisis or fixed schedule

Meets a standard (Pass/Fail)

Quality Improvement

Proactive

Works on processes

Seeks to improve (culture shift)

Led by staff

Continuous

Proactively selects a process to improve

Exceeds expectations

Topic Big ‘QI’ – organization-wide Little ‘qi’ – program/unit

Improvement

Quality Improvement Planning

Evaluation of Quality

Processes

Quality Improvement Goals

Individual ‘qi’

Contrasting Big “QI”, Little “qi”, and Individual “qi

System focus

Tied to the Strategic Plan

Responsiveness to a community need

Cut across all programsand activities

Strategic Plan

Specific project focus

Program/unit level

Performance of a processover time

Delivery of a service

Individual program/unit level plans

Daily work level focus

Tied to yearly individualperformance

Performance of daily work

Daily work

Individual performance plans

General Approach On How To Use The Basic Tools Of Quality Improvement

Issue ToConsider

Flow ChartExisting Process

Brainstorm& ConsolidateData

Cause & EffectDiagram – Greatest Concern

Use 5 Whys ToDrill Down ToRoot Causes

Gather DataOn Pain Points

Translate DataInto Information

• Pie Charts• Pareto Charts• Histograms• Scatter Plots, etc.

Flow ChartNew Process

Monitor New Process & Hold

The Gains

• Run Charts• Control Charts

Data ManagementStrategy – Ch. 14

“As Is” State to “Should Be” State

“As Is” StateBrainstormingForce and Effect

Analyze Information andDevelop Solutions

Solution andEffect Diagram

“AIM”

Large Issue, Cross

Functional Problem, or Sensitive Situation

Explore

Brainstorming Affinity Diagram

Sort &Prioritize

Interrelationship DiGraphPrioritization Matrix

Understand &Baseline

Radar ChartSWOT Analysis

Develop Actions &

TasksTreeDiagram

PrioritizeActions &

Control & Influence PlotsPrioritization MatrixKnow & Don’t Know Matrix

DevelopProject Plans

Monitor

PERTGantt Chart

SMART Chart

Figure 4

ProblemPrevention

General Approach On How To Use The Advance Tools OfQuality Improvement

What Is Quality?

Today the most progressive view of quality is that it is defined entirely by the customer or end user and is based upon that person's evaluation of his or her entire customer experience.

The customer experience is the aggregate of all the Touch Points that customers have with the organization’s product and services, and is by definition a combination of these.

Deming Cycle – PDCA or PDSA

PDCA was made popular by Dr. Deming who is considered by many to be the father of modern quality control; however it was always referred to by him as the "Shewhart cycle."

Continuous Improvement

The continuous improvement phase of a process is how youmake a change in direction.The change usually is because the process output is deterioratingor customer needs have changed

1. Identify and Prioritize Opportunities

2. Develop AIMStatement

3. Describe the CurrentProcess

4. Collect Data on Current Process

5. Identify All PossibleCauses

6. Identify PotentialImprovements

7. Develop Improvement Theory

8. Develop Action Plan

1. Implement theImprovement

2. Collect and DocumentThe data

3. Document Problems,Observations, and Lessons

Learned

Check/Study

1. Reflect on the Analysis

2. Document Problems,Observation, and Lessons learned

Abandon

Standardize

The ABC’s of PDCA, G. Gorenflo and J. Moran

Maintenance and Standardization

The Maintenance and Standardization phase of a process is how we hold the gains. If our process is producingthe desired results we standardizewhat we are doing.

Integrated CycleThe SDCA and PDCA cycles areseparate but rather integrated.Once we have made a successfulchange we standardize and holdthe gain.When the process is not performingcorrectly we go from SDCA to PDCAand once we have the process performing correctly we standardize again.This switching back and forth betweenSDCA and PDCA provides us with the opportunity to keep our processcustomer focused.

Knowledge & Experience

PDCA should be repeatedly implemented in spirals of increasing knowledge of the system that converge on the ultimate goal, each cycle closer than the previous.

Hold the Gains

Rapid Cycle*

Rapid Cycle PDCA

The Basic Tools of QI

Flow Chart

Cause and Effect Diagrams

Pareto Chart

Check Sheet

Histogram

Scatter Diagram

Control Chart

Enter Building

Greeter

Patient Flow

Possibly not

Clerical

Screener

Okay to

vaccinate?

Triage

Okay to

vaccinate?

Need medic

al attenti

NoExit Leave Building

Is patient able

to leave

on own?

EMT transports patient to medical facility

Cause and Effect Diagram

PoorHIV Testing

ClientTest Location

Don’t see benefit

Counseling

Not Client Centered

Inconvenient

Not Respectful

Fearful

Not Offered Poor Experience

Too Public

Don’t Want Test

NC Accreditation Collaborative

MeasuredIn Inches

Tally GroupedAbsoluteFrequency

AbsoluteCumulative Frequency

Relative Frequency

CumulativeRelative Frequency

.506 I 3 100 0.01 1.00

.505 II 99 0.02 0.99

.504 IIII 97 0.04 0.97

.503 IIIII IIIII 29 93 0.10 0.93

.502 IIIII IIIII IIIII 83 0.15 0.83

.501 IIIII IIIII IIIII III 68 0.18 0.68

.500 IIIII IIIII IIIII IIIII I 53 50 0.21 0.50

.499 IIIII IIIII IIII 29 0.14 0.29

.498 IIIII IIII 15 0.09 0.15

.497 IIII 14 6 0.04 0.06

.496 I 2 0.01 0.02

.495 I 1 1 0.01 0.01

CellMid-Point

Cell Boundary

Grouped Frequency Distribution Table

Frequency Polygon & Histogram – Grouped Data

0.41 0.494 0.497 0.500 0.503 0.506 0.509

Obese Children

Age in Years

2 6 10 14 18

Scatter Plot

Run Chart

Median Line

5 10 15 20

Obse rvation

MR Chart

LCL = 0.0

CL = 4.0

UCL = 13.2

X Chart

LCL = 1.6

CL = 12.4

UCL = 23.2

Jones County WIC Lobby Wait TimeXMR Chart

Control Chart

AIM Problem Statements

DiscreteCurrent State

Time Bound

Measureable Baseline

Improvement Target

Measures of the Target – know if we succeed

AIM• Discrete

• Measureable• Time Bound

Control

Influence

External

Internal

Operational

Strategic

Outcome

Process

Measurem

Capacity

Some Tools To Help Create AIM Statements

1. Current and Future State Model

2. AIM Work Sheet

3. Force Field Diagram

4. Force and Effect Diagram

Current State

• What is the current state?

• Why is this important?

• What is it costing us time/dollars/staff/etc?

• What is the impact on our customer/clients?

• What is the impact on our division/agency?

Future State:

• What are the important aspects of the future state?

• What is driving us to this future state?

• What might be the consequences of not moving to the future state?

• What might change?

• What is the proposed timeline?

Pathway

Consequences

Driving Forces:

Benefits

AIM or Opportunity StatementAn opportunity exists to improve the

________________________________________________________________________(name process, or area to work on)

beginning with ___________________________________________________________(beginning boundary, starting point)

and ending with __________________________________________________________.(ending boundary, finish point)

This effort should improve_______________________________________________________________________

_______________________________________________________________________

(key characteristics of area the team is working on)

for the _______________________________________________________________________________

_________________________________________________________.(customers, staff or those affected by the process under improvement)

This process is important to work on now because_______________________________________________________________________

_______________________________________________________________________

(what will it improve and for whom?)

Positive Forces Negative Forces

CurrentState

Force Field Diagram - Basic

Force and Effect

The Force & Effect (F&E) Diagram is designed to identify barriers to agreement among team members concerning an AIM Statement.

The F&E Diagram combines a Force Field and Cause & Effect Diagram.

Instead of having one box on the cause and effect diagram we use a double headed effect.

The first effect (far left) is the current state and the second effect (far right) is the desired future state. In between are branches of main causes that maintain the status quo. Too often we focus only on the causes of the current state without looking at what pushes us to change

AIM Statement Example

Determine if our client education and teaching is effective for positive pregnancy testers in identifying the importance of seeking early prenatal care. Goal is to have 95-100% of our patients will receive prenatal care appointments in 2-3 weeks of positive pregnancy tests from the current level of 65% and 100% of clients will be using Prenatal vitamins which is currently at 70%.

This project will take three months to analyze and develop solutions to trial. The trial period will last three months and then we will analyze the results, determine where we are in achieving our goal , and either standardize the solution or make adjustments or refinements if required.

FOCUSING STATEMENT

Currently we are using 5 different methods to collect mosquitoes within the county. Each of the 5 inspectors/collectors is using a method that best fits their schedule training, and motivation.

This difference may be causing an extreme disparity among the collectors in regard to number of mosquitoes collected. These differences can translate into a lack of data in the respective area.

The lack of data could lead to a use or misuse of control measures for mosquito control when there is an unidentified need.

Lack of control may lead to mosquito borne disease outbreak and an unusually high number of nuisance complaints.

The use or misuse of pesticides or other control measures costs the health department dollars. Also it could have a negative effect in regard to the public’s perception of use of pesticides. The chemicals we use are expensive and if a true measure of mosquito populations could be gained, overuse would be minimized.

AIM STATEMENT

In the coming mosquito seasons, our division needs to focus on ensuring that all the traps are set up in a manner to collect the maximum number of mosquitoes per trap site.

The motivation behind this need is that some states and the EPA are moving to reduce the amounts of pesticide applied. These new laws require applicators to show with set evidence and numbers the need to apply pesticides.

When these guidelines or codes go into effect, there will be harsh penalties for violations.

Furthermore failure to move forward will possibly place the XYZ Health Department in violation. This is in addition to wasted money for over use of pesticides. The implementation of the program can be completed in the next (20XX) mosquito season.

COMPONENTS OF ISSUE STATEMENT

The environmental division has the ability to complete the project with employees already in place. We have complete control over the element.

The project can be completed if the players allow the changes to be made if they put them into practice.

Influence must be used by management to ensure that all collectors are properly executing the project. There should not be any out side influences in effect.

The only element out of our control is the laziness of the collector in ensuring that the trap is set and operating properly.

We need to focus on 1. Knowledge of collector 2. Use of the same water in trap 3. Ensuring batteries last duration of trap cycle 4. Moving to a 1 night trap event 5. Keep the collector motivated 6. Who is your collection substitute?

Components of the Issue Statement

Element Control Implement Involve &Influence

Outside OurControl &Influence

For each element check which column(s) applyFrom this select the area(s) of focus, develop a ranking of the elements to focus on, and write the problem statement for the Quality Improvement project to be started.

Reduce Inconsistency Mosquito Trapping

Element Control Implement Involve &Influence

Outside OurControl &Influence

For each element check which column(s) applyFrom this select the area(s) of focus, develop a ranking of the elements to focus on, and write the problem statement for the Quality Improvement project to be started.

Education Within Within Within In

Same water Within Within Need influence In

Batteries Out Within Need influence In

1 night trapping Within Out Need influence In

Motivation Out Out Need influence Outside

LHD’s Work On Their Draft AIM Statements

Customer Centric Organizations

Who are recognized as top customer centric organizations?

Marriott – extra mile for the customer

Southwest Airlines – letter of apology

Publix Market – 10 by 10 rule

Nordstrom

Others?

Who are recognized as bottom of the barrel customer centric organizations?

Dell Software Support

Cable Companies

Credit Card Companies

Cell phone Companies

Others?

Who are recognized as bottom of the barrel customer centric organizations?

Why?Misplaced paper workUnhelpful clerksShort tempered clerksMisleading adsSurprise feesUnderstaffed call centersHang Up on CustomerOther reasons?

Shopping at our store should be enjoyable, exciting and designed for you.

L.L. Bean – Customer Delight

Unexpected service and attentionMore than the customer expected

More than satisfying the customer

Deliver the unexpected

Deliver it with enthusiasm and sincerity

Surprise the customer

Create a memory

Next Door To L.L. Bean is J. L. Coombs - The Oldest Shoe Company in the United States - 1830

“If You Do Not Like MyShoes the Hell with you!”

Customer Touch Points

When your customer (internal/external) comes in touch with your process what do they:

See? (Initial reaction?)

Sense?

Experience?

Understanding Your Customer

Need to obtain the Voice of Your Customer:

Satisfiers

Dis-satisfiers

Future needs and wants

Levels of Customer Satisfaction

Satisfied

Need is metNeed not met

Dissatisfied

Expected

The Kano Model

Voices

VOP – Voice of the Process (VOP)

VOC – Voice of the Customer (VOC)

VOG – Voice of the Organization (VOO)

Voice of the Future (VOF)

Need to balance them

Who is Your Customer for Your Issue?

What are their/your wants and needs?

What will satisfy them?

What will satisfy You??

How can we align our needs??

Communication is the key

How To Obtain Customer Data

Survey and ask them

Where to do the survey

What to ask?

Best time to do it

How often to do it?

Flow Charting

“If you can't describe what you are doing as a process, you don't know what you're doing.”

W. Edwards Deming

W Edwards Deming (1900-1993)"Draw a flowchart for whatever you are doing. Until you do, you do not fully understand what you are doing. You just have a job.“

"The first step in any organization is to draw a flow diagram to show how each component depends on others. Then everyone may understand what their job is. If people do not see the process, they cannot improve it."

SIPOC+CMWhat it is:

SIPOC is a data collection form that is used before we start to construct a flow chart since it helps us to gather relevant information about the process.

Assists in gathering information about Suppliers, Inputs, Process, Outputs, and Customer of the process.

SIPOC is high level view of the “As Is” state of a process under investigation.

The C stands for constraints (barriers) facing the system and the M for the measures to be used.

SIPOC+CM

When to use it:

When first starting to investigate a process and a team needs to understand the basics that make up the process.

When a team needs a way to get the collective knowledge of the team members about a process recorded in an easy to view format.

When we need to make a concise communication to others about a process and the parameters that it encompasses.

SIPOC+CMHow to use it:

On a piece of flip chart paper draw the SIPOC+CM diagram with seven blocks indicating the components of SIPOC+ CM.

Clearly identify the process under study and define the process boundaries (start and end points) so that everyone involved understands the limits of the analysis.

On the SIPOC+CM form identify the data available for each of the following major categories:

Suppliers – who or what (internal or external) provides the raw materials, information, or technology to the process

Inputs – what are the material or information specifications that are needed by the process

Process – a highly level flow chart of the key 5 to 7 core activities that comprise the process. This is a 30,000 foot view of the process. The detail steps will be developed in the flow chart.

Outputs – what the process produces as products, services, or technology

Customers – who are the main users of the process’s output

+ C –constraints facing the system or process

+ M – measures being used or to be used

Review the form for completeness with relevant stakeholders, sponsors, and other interested parties.

Process/Activities:

Begins With:

Ends With:

Inputs:

Suppliers:

Outputs:

Customers:

Constraints:

High Level S I P O C+CM Collection Form

Measures

High Level S I P O C Collection Form

Process/Activities:

Begins With: Ends With:

Inputs:

Supplier(s):

Outputs:

Customers:

Constraints:

Identifying required service standards of performance

Trained and effective employees committed to using the standards in all areas.

Limited funds for benchmarking other public health organizations. Human Resources is short two staff and overworked already.

LPH System values and vision. Current job descriptions and job performance expectations. Benchmarking from other PHDs and service organizations. Training on interviewing and employee selection criteria. General idea of client, inspector, nurse, and other staff expectations

Approved standards of service performance excellence for PHD Training modules developed for all levels of management and employees Announcement campaign to provide awareness and support of standards Rollout of training to all staff Training to supervisors on how to use the standards in performance planning.

PHD senior management, Human Resources, Benchmarking organizations, consultant on hiring and interviewing, clients, health officer, nurses, supervisors, and employees.

Employees, Human Resources, supervisors, PHD clients, senior management, partners, nurses, community.

Benchmark other PHDs for standards of service performance. Gather and review all current job descriptions for existing standards and expectations. Research current journals for trends on behaviorally-based characteristics. Work with H/R and Senior management to establish LPH standards. Update all job descriptions and performance planning models to new standards. Develop training materials to roll out new standards for current employees and new hiring. Announce rollout, timelines and measurements. Work with supervisors and employees to put standards into each performance plan. Gather feedback, adjust, report and maintain.

LPH standards of performance completed % of standards signed by employees % of job descriptions updated Training materials for standards complete on time % employees completing training…

Measures

SIPOC+CM

Flow Charting

Flow charting is the first step we take in understanding a process

Organized combination of shapes, lines, and text

Flow charts provide a visual illustration, a picture of the steps the process undergoes to complete it's assigned task

From this graphic picture we can see a process and the elements comprising it

Shows how interactions occur

Makes the invisible visible

Flow Chart BenefitsCreates a common vision

Establishes the “AS IS” baseline – Current State

Baseline to measure improvements

Identifies wasteful steps – activities/waits

Uncovers variations

Shows where improvements could be made and potential impacts

Training tool

Flow Chart People Benefits

People involved in constructing a flow chart begin to:

Better understand the process

Understand the process in the same terms

Realize how the process and all the people involved, including them, fit into the overall process or business

Identify areas for improving the process

Become enthusiastic supporters to quality and process improvement

Olmsted County , MN – Performance Appraisal Process

Flow Charting ConstructionClearly define the process boundaries to be studied

Define the first and last steps – start and end points

Get the right people in the room

Decide on the level of detailComplete the big picture first – macro viewFill in the details – micro view

Gather information of how the process flows: ExperienceObservationConversationInterviewsResearch

Clearly define each step in the process Be accurate and honest

Flow Charting StepsUse the simplest symbols possible – Post-Its

Make sure every loop has an escape

There is usually only one output arrow out of a process box. Otherwise, it may require a decision diamond.

Trial process flow – walk through people involved in the process to get their comments

Make changes if necessary

Identify time lags and non-value-adding steps.

Flow Charting Moments

Surprise

Bafflement

Embarrassment

Disappointment

Flow Chart Construction

Use a form of Post-It Notes – easier to rearrange

Realize everyone is not doing it the same way – there will be disagreements

It will take multiple passes to get to the “As Is” State

Flow Chart Symbols

Activity:Operation/Inspection

Decision

Start/EndBookends

Document

Wait/Delay

Storage

Data Base

Transport

Output

Flow Lines

A Connector

CommentCollector

Input/OutputData

ManualOperation

Preparation

ManualInput

Display

Unfamiliar/Research

Constructing a Flow Chart

Asking questions is the key to flow charting a process.

For this process:Who is the customer(s)?Who is the supplier(s) ?What is the first thing that happens? What is the next thing that happens?Where does the input(s) to the process come from? How does the input(s) get to the process? Where does the output(s) of this operation go? Is their anything else that must be done at this point? What is the baseline measurement of this process?

Adding Time LinesAs Is Flow Chart Could Be Flow Chart Should Be Flow Chart

Time Time

Analyzing A Flow Chart

Examine each: Activity symbol – value/cost?

Decision point – necessary/redundant?

Choke Points – bottlenecks?

Rework loop – time/cost?

Handoff – is it seamless?

Document or data point – useful?

Wait or delay symbol – why?/reduce/eliminate

Transport Symbol – time/cost/location?

Data Input Symbol – right format/timely?

Document/Form Symbol – needed/cost/value?

Flow Charting Basic

DataVoice of the Process - VOP

Customer - VOC• Internal• External

Flow Chart Summary MatrixPHF E-News, March 2, 2010, http://www.phf.org/pmqi/Flow-Chart-Summary-Matrix.pdf

∑Flow Chart Step Number

Type of Step

Type of Step: P – process, D – decision, T – transport, W – wait, S – storage

Delta = Proposed – Actual – the more negative the subtraction the better – more savings

1. Touch Point (√)

2. Cost

3. FTEs/Person Hrs.

4. Supplies Required

5. Equipment Required

6. Space Required

7. Time

8. Cost of Quality

8. Partnerships Needed

9. Etc.

10. Value added

ActualDelta

+/-∑Proposed

P D P T W P D S

1 2 3 4 5 6 7 8

Flow Charting Exercise

Lean Check List

5S is a visual method of setting the workplace in order. It is a system for workplace organization and standardization.

The five steps that go into this technique: Seiri – sort – essential items

Seiton – set in order – promote work flow

Seison – shine – clean workplace

Seiketsu – standardize - consistency

shitsuke)- sustain – hold the gains

Definition of 8 Types of Waste:

Waste Description Public Health Example

Overproduction Items being produced in excess quantity and products being made before the customer needs them

Insurance filing or immunization record opened before all required information is received

Waiting Periods of inactivity in a downstream process that occurs because an upstream activity does not produce or deliver on time.

Paperwork waiting for management signature or review

Unnecessary Motion

Extra steps taken by employees and equipment to accommodate inefficient process layouts.

Immunology testing equipment stored in cabinets far from specialist work area.

TransportationHandling

Unnecessary movement of materials or double handling

Department vehicles stored in central facility, requiring constant movement of vehicles to and from other high traffic locations

Over-processing Spending more time than necessary to produce the product or service

Combining client survey instruments into one form rather than develop specific instruments for each program

Unnecessary Inventory

Any excess inventory that is not directly required for the current client’s order

Over estimating vaccination support materials requiring additional locked storage cages, inventory counting and reconciliation

Defects Errors produced during a service transaction or while developing a product.

Ineffective scripts for initial intake applications. Unclear directions for filling out required forms

Duplication Having to re-enter data or repeat details on forms. Poorly designed client intake computer screens or services checklists

Measurement

Part 1

“Not everything that can be counted counts, and not everything that counts can be counted.”

Albert Einstein

"In God we trust, all others bring data.“

-W. Edwards Deming

Why Measure?

You can't manage what you don't measure. It is an old management adage that is accurate today.

Unless you measure something you don't know if it is getting better or worse.

You can't manage for improvement if you don't measure to see what is getting better and what isn't.

Definitions

Measure: The verb means "to ascertain the

measurements of“

Measurement: The figure, extent, or amount obtained by measuring“

Metric: "A standard of measurement“

Benchmark: "A standard by which others may be measured or compared"

Attributes of Measures

Aligned to the strategy of the organizationAccuracy of Input - answers critical questionsTimeliness of inputQuality/use ability of outputAccuracy of outputReadily availableEnergizes user into actionManage what you measureGraphically displayed - show it in a simple usable format

Measurement

Measurement is critical to performance improvement and is the most difficult part of the process

Start thinking of Measurement at the very beginning of the process.

Ask how can we measure that when developing the AIM Statement?

Develop an improvement theory.

An improvement theory is a statement that articulates the effect that you expect the improvement to have on the problem.

Writing an improvement theory crystallizes what you expect to achieve as a result of your intervention, and documents the connection between the improvement you plan to test and the measurable improvement objective.

Test Methodology

Test method is a definitive procedure that produces a test result.

A test can be considered as technical operation that consists of determination of one or more characteristics of a given product, process or service according to a specified procedure.

The test result can be: qualitative (yes/no)

quantitative (a measured value)

personal observation

output of a precision measuring instrument.

Sampling and Surveying

Tips and Techniques

The Public Health Quality Improvement Handbook, R. Bialek, G. Duffy, J. Moran, Quality Press, 2009, p. 147

What is the purpose of sampling?

To draw conclusions about populations from samples.

To help us determine a population's characteristics by directly observing only a portion or sample of the population using statistics

We obtain a sample rather than a complete enumeration (a census ) of the population for many reasons:

Economy Timeliness Inaccessibility of some of the population Destructiveness of the observation

Target Population

The researcher must clearly define the target population.

There are no strict rules to follow, and the researcher must rely on logic and judgment.

The population is defined in keeping with the objectives of the study

Problem Definition

In sampling, this includes defining the population from which our sample is drawn.

A population can be defined as including all people or items with the characteristic one wishes to understand.

Because there is very rarely enough time or money to gather information from everyone or everything in a population, the goal becomes finding a representative sample (or subset) of that population.

Sampling Frame

It has the property that we can identify every single element and include any in our sample

The sampling frame must be representative of the population

Sampling Frame Problems

• 1. Missing elements: Some members of the population are not included in the frame.

• 2. Foreign elements: The non-members of the population are included in the frame.

• 3. Duplicate entries: A member of the population is surveyed more than once.

• 4. Groups or clusters: The frame lists clusters instead of individuals.

Sampling MethodsClassified as either Probability or Non-Probability.

Probability samples, each member of the population has a known non-zero probability of being selected.

The advantage of probability sampling is that sampling error can be calculated. Sampling error is the degree to which a sample might differ from the population.

Non-probability sampling, members are selected from the population in some nonrandom manner.

In non-probability sampling, the degree to which the sample differs from the population remains unknown.

Types of Sampling• Systematic Sampling

• Stratified Sampling

• Convenience Sampling

• Judgment Sampling

• Snowball Sampling

Non-Sampling Error

Non-sampling errors are caused by the mistakes in data processing:

Overcoverage: Inclusion of data from outside of the population.

Undercoverage: Sampling frame does not include elements in the population.

Measurement error: The respondent misunderstand the question.

Processing error: Mistakes in data coding.

Non-response

Type I risk, or alpha risk is the "reasonable doubt." It is the chance of wrongly rejecting the null hypothesis when it is true. In acceptance sampling, it is the producer's risk, or risk of wrongly rejecting a lot that meets requirements.

The Type II risk or beta risk is the chance of accepting the null hypothesis when it is false. The "consumer's risk" is the Type II risk for an acceptance sampling plan. It is the chance of passing a lot that does not meet the requirements.

Type I risk – convict an innocent defendantType II risk – acquit a guilty defendant.

Errors

Null Hypothesis isTrue

Null Hypothesis isFalse

Statistical Decision

Reject the Null Hypothesis

Type 1 Error

Accept the Null Hypothesis Type 2 Error

Correct

Null Hypothesis – person is innocent

Convict an InnocentPerson

Acquit a Guilty Person

Sample Size Determinants

Accuracy/Confidence – alpha/beta risk

Precise – understand variation

Difference trying to measure

Sample Size – Rules of Thumb

1. Trial and Error – n > 3 – 80% confidence

2. CLT – n > 30

3. Reliability – n = 60 - 95% confident

4. Shewhart - n >100 – 4 sets of 25 to determine process stability

How much data to collect?

n = 30 (Good or Bad?)

A complete cycle of the process?

Time based – when it is done?

Representative?

Sample Size

http://www.zoomerang.com/Sample-Size/

Cause and Effect Diagrams

Moving from Treating Symptoms

Treating Causes

Problem Solving – What we usually see is the tip of iceberg – “The Symptom”

The Symptom

The Root Causes

Invisible

Hidden

Problem Solving

When confronted with a problem most people like to tackle the obvious symptom and fix it

This often results in more problems

Using a systematic approach to analysis the problem and find the root cause is more efficient and effective

Symptom – sign or indication

Cause – whatever makes something happen

Cause and Effect Diagrams - Construction

Write the issue as a problem statement on the right hand side of the page and draw a box around it with an arrow running to it. This issue is now the effect

Effect

Generate ideas as to what are the main causes of the effect

Label these as the main branch headers

Organizes group knowledge about causes of a problem and display the information graphically

Effect

HeaderHeader

Typical Main Header are:

4 M’s – Manpower, Materials, Methods, Machinery

People

Policies

Materials

Equipment

Life style

Environment

For each main cause category brainstorm ideas as to what are the related sub-causes that might effect our issue

Use the 5 Why techniques when a cause is identified

Keep repeating the question until no other causes can be identified

List the sub-cause using arrows

Effect

HeaderHeader

Selecting Items to Investigate

When the Cause and Effect Diagram is finished it is time to decide what few areas should be focused on to develop solutions to solve the effect.

Some are obvious – low hanging fruit

Some require some research using the other QI tools such as:

Pareto DiagramsRun ChartsSurveysHistogramsEtc.

Obese Children

Life Style

PoliciesEnvironment

TV ViewingNo Time For Food Prep

No Outdoor PlayUnsafe

Juices

Bottle Pacifier

Less Fruits and Veg.

Less Income Maternal

Choices

Less Vigorous Exercise

Curriculum

No Sidewalks

Unhealthy Food Choices

Few Community Recreational Areas or Programs

Built Environment For Strollers Not Toddling

Less Indoor Mobility

TV Pacifier

UnsafeHousing

Sodas/Snacks

Decreased Breast Feeding

Early Feeding Practices

Genetics

Syndromes

Pre NatalPractices

Excess Maternal Weight Gain

Over Weight Newborn

Over WeightPre School At School

At Home

Problem (Effect)

5 Why’s Technique

Root Cause Analysis Rating Form

PotentialRootCause

ImprovedQuality

ReducedCosts

ImprovedCustomerSatisfaction

Others Total Score

Ranking

Impact Scoring Scale: Low = 1, Medium = 3, High = 5

Impact on the Problem

Cause and Effect Exercise

Why EmployeesAre Late For Work?

Cause and Effect Diagram

Solution and Effect Diagram

Similar to the Cause and Effect Diagram

Identifies changes and recommendations

Effect is now made into a positive statement:“What are the causes of Childhood Obesity”

How to prevent Childhood Obesity”

Solution and Effect Diagrams - Construction

Place the Solution and Effect Diagram opposite the Cause and Effect Diagram

Write the issue as a positive statement on the left hand side of the page and draw a box around it with an arrow running to it.

This issue is now the effect

Positive Effect

Generate ideas as to what are the main Solutions of the effect

Label these as the main branch headers

Effect

SolutionSolution

For each main Solution category brainstorm ideas as to what are the related sub-solutions that might effect our issue

Use the 5 How techniques when a solution is identified

Keep repeating the question until no other solutions can be identified

List the sub-solutions using arrows

Effect

SolutionSolution

EffectEffect

Why?Why?

Cause CauseSolutionSolution

Solution

How?How?

C = Cause CategoryS = Solution Category

LessObese Children

Life Style

PolicesEnvironment

More Mobility

Early Feeding Practices

Genetics

Pre NatalPractices

Solution (Effect)

5 How’s Technique

5 How’s of More Vigorous Exercise

Less TV and Video Games

More Community Sponsored Recreation Programs

Safe Play Areas

Additional Resources

More Family Recreational Activities

Selecting Items to Investigate

When the Solution and Effect Diagram is finished it is time to decide what few areas should be focused on to develop solutions to solve the effect.

Root Cause Analysis Solution Impact Analysis

Potential Solutions

CorrectiveAction Type

VerificationMethod

CostToFix

BenefitOf Fix

CostBenefitRatio

Selected?Y/N

Corrective Action Type: Immediate but Interim – II, Short-term Temporary - ST,Permanent – Short Term – PST, and Permanent – Long Term - PLT

Solution and Effect Diagrams

Cautions:

Do not jump to quickly to Solution and Effect

Use after a detailed analysis of “Cause”

If you are still listing Causes – not enough detail on the Cause and Effect Diagram

Ask if the proposed solution(s) will improve the effect or cause more problems

Solution and Effect Diagram Exercise

How To Make Employees OnTime For Work?

What Can Go Wrong? - PDPC

Ask the killer question

Do not be surprised by the amount of problems a group can generate about a solution they have devised

People thrive on failure

Everyone knows something will go wrong – it is not “If” but “When”

Remember the National Lampoon’s Griswold vacation to Walley World?

What Can Go Wrong?

If we wanted this to fail, how could we accomplish that?

What assumptions are we making that could turn out to be wrong?

What has been our experience in similar situations in the past?

Does this depend on actions, conditions or events?

Are these controllable or uncontrollable?

Decide how practical each countermeasure is by using criteria such as:

time required

ease of implementation

effectiveness

How will we know if we are successful?

What are the indicators of success?

1. Identify and Prioritize Opportunities

2. Develop AIMStatement

3. Describe the CurrentProcess

4. Collect Data on Current Process

5. Identify All PossibleCauses

6. Identify PotentialImprovements

7. Develop Improvement Theory

8. Develop Action Plan

1. Implement theImprovement

2. Collect and DocumentThe data

3. Document Problems,Observations, and Lessons

Learned

Check/Study

1. Reflect on the Analysis

2. Document Problems,Observation, and Lessons learned

Abandon

Standardize

The ABC’s of PDCA, G. Gorenflo and J. Moran

Check/Study/Observe

This phase involves analyzing the effect of an intervention.

Compare the new data to the baseline data to determine whether an improvement was achieved.

Whether the measures in the aim statement were met.

Check/Study/Observe

1. Reflect on the analysis, and consider any additional information that emerged as well. Compare the results of your test against the measurable objective

2. Document lessons learned, knowledge gained, and any surprising results that emerged.

Source: The ABCs of PDCA, Grace Gorenflo and John W. Moran,

Check/Study/Observe

Tools that can assist :

Pareto charts

Histograms

Run charts

Scatter plots

Control charts

Radar charts.

Measurement

Part 2

Data by itself has no value – it just shows information.

Information requires interpretation for it to have value.

Decision makers add value through interpretation

40% Data 60% Information

Decision Maker

Types of Data

Quantitative

Can be discrete or continuous

Discrete Variables - counted or enumerated - # pills in a bottle

Continuous – measured – length, width, weight, pressure, etc.

Types of Data

QualitativeAlways discrete Attribute dataPlaced into two or more attribute categories

Yes/NoInfected/Not InfectedPass/FailPositive/NegativeGood/BadMale/FemaleRed/Yellow/GreenDead/Alive

Data Management

“Whenever there is fear, you will get wrong figures.”

W. Edwards Deming

Data Management Strategy

CollectRaw Data

Interpret

Communicate

Consolidate

Collecting

Translating – use data tables

Summarizing – use descriptive statistics

Framing - use charts/graphs

Data Collection Questions

Before collecting data we must answer the following questions:

What is the purpose for collecting this data? What type of data is going to be collected?Where will the data be collected?Who will collect the data?When will they collect the data?How will they be trained to collect the data?What will we do with it after we collect the data?How will we summarize and present the data?

Getting Good Data

Understand the process being studied – walk it

Make the collection simple

Define where the data will be collected – collection points

Use checksheets and checklists to help

Minimize the “other” category – by good classifications - too often the largest bar on a chart

Establish collection rules – sampling

Getting Poor Data

Lack of training on what to doUnclear directionsAmbiguous terminology – need yearly data – fiscal or calendar year?Different units of measures – 9/23/99 – Mars spacecraft was a $125m lost because it missed entry by 100km – NASA used the metric system and Lockheed used the English units – inches versus metersMathematical errors – rounding, calculation, order of calculations, etc.

There are three kinds of lies:

Damned Lies,

and Statistics

Descriptive Statistics

The majority of our data collection will be done through sampling

Populations versus Samples

Population parameters: μ and σ

Sample statistics: Х and s

Measures of Central Tendency:

Mean – arithmetic average of the items sampled

Median – middle value in the sample

Mode – the one that most frequently occurs

These statistical measures help us to understand how the data is distributed:

Symmetrical – normal – bell shaped

Skewed – left or right

Rectangular

Always plot the data and confirm the shape of the data

“A central theme of the statistical approach to data analysis is this:

Variability always exists. No experiment can be repeated exactly.

Variability can never be totally eliminated.”

“Statistics An Introduction”, A Rickmers and H. Todd, Mc Graw-Hill, 1967, page v

Descriptive Statistics – Variation

Variation is everywhere

It is found in the output of any process of manufacturing, service, or administrative

But variation is not all bad since it always displays a pattern or a distribution of itself

These patterns or distributions can tell us a great deal about the process itself and the causes of problems found in the process

Histograms help us identify and interpret these patterns

Measures of Variability: Dispersion

Range – Highest Value – Lowest Value

Variance – how much spread in our sample data - scatter

Standard Deviation – dispersion of a random variable about its mean

The measures of central tendency and the measures of variability when compared to historical data can help us determine if:

The center of the process has shifted

The variability of the process has increased

Combination of both of the above

When constructing data tables and graphs:

Do not throw any data awayKnow what type of data you are dealing with – qualitative or quantitativeKnow if the data is discrete or continuous Label everything appropriatelyKeep it clear and simple

Target

Performance

ProcessGap

TAPP/PDCA Integration*

Gap = Performance versus Target

Act only when Performance is below Target

Small gaps – just fix it

Larger gaps need a PDCA Cycle

Monitor

* Reference Article

How will we know if we are successful?

What are the indicators of success?

Develop a few Indicators of project success

Project Planning

OngoingActivities

New Activities

StartTime

Low High

Resources committed to change

Transitional Time Line

OngoingActivities

New Activities

StartTime

Low High

Resources committed to change

Transitional Time Line

Gantt Chart

History

The first Gantt Chart was developed by Karol Adamiecki, who called it a Harmonogram

Because Adamiecki did not publish his chart until 1931, this famous chart bears Henry Gantt's name (1861–1919) designed his chart in 1910

Wikipedia, the free encyclopedia

Gantt ChartHenry L. Gantt – WWI

Franklin Arsenal 1910

Progress Chart

Work planned and accomplished are shown in the same space

Emphasizes work movement through time

Deals with plans and progress

Helps identify and eliminate obstacles

Gantt Charts

A Gantt chart is a matrix diagram

The vertical axis lists all the tasks to be performed for a project

Each row contains a single task identification

The horizontal axis is headed by columns indicating estimated task duration in hours, days, weeks, months, etc.

Use Of Gantt Charts

Establish order of tasks:Sequential and Parallel

Identify resources requirements

Timing of resource needs

Identify the critical path

Monitor the project “On-Time” Schedule

Alerts where remedial action is required

Traffic Light Gantt Chart

Task: City of XYZ HD 29-Feb 7-Mar 14-Mar 21-Mar 28-Mar

Finalize assessment analysis X

Gain consensus on priorities X

Identify comm. with elected off. X

Plan PHF consultant visit X

Set agenda and travel schedule X

City HD/PHF PI meeting X

On Schedule

Late or at Risk

Solution works:Standardize

Measure

Continue to improve

Solution does not work:Regroup

New team

New AIM

Communication Plan

Decide who/what/where/when of the Communication Plan

Describe the end “game” – begin with the end in mind

Make it flow – easy to understand and digest

Front load – important points first – get their interest

Stress the benefits to the listener – WIIFM

Ask for commitment – will you support the effort?If not – why?

Documenting The Impact Of QI

How many times have you reached the end of a quality improvement project only to be unsure of what has actually been accomplished?

It is not unusual to get so caught up in the solving of the problem that we forget to accurately document what we did, when we did it, and what it accomplished.

A quality improvement team moves quickly on its quest to gather data and solve problems but can easily delay recording what the interventions actually accomplished.

It is difficult to recall or recreate history since people involved in the project usually have sketchy and conflicting memories of what was done, when it was done and the associated impact.

Start as soon as you have developed the AIM statement.

Develop a process by which you can document what quality improvements take place”

What they were?

When they where implemented?

What change resulted?

How much was the change?

Utah Association of Local Health Departments Quality Improvement Workshop September 21 st and 22 nd,...

Documents