2B - Control Charts - IHIapp.ihi.org/.../Event-2590/Document-4161/Control_Charts.pdfAdvanced...

Advanced Measurement for Improvement

Cambridge, MA • March 26-27, 2015

1

2B – Control Charts

Advanced Measurement for Improvement Seminar

March 26-27, 2015

Two Types of Variation

Common Cause

Is inherent in the design of the

process

Reflects the ‘business as

usual’ state of the process

Is due to regular, natural or

ordinary causes

Affects all the outcomes of a

process

Results in a “stable”

distribution that is predictable

Also known as random or

unassignable variation

Special Cause

Due to irregular or unnatural

causes that are not inherent in

the design of the process

Reflects a ”different mode” of

the process

Affects some, but not

necessarily all aspects of the

process

Results in an “unstable”

process that is not predictable

Also known as non-random or

assignable variation

21-Mar-15 • 2



2

A Stable Process

A predictable (stable) process has only common causes in play.

!

A Stable Process

© R. Scoville & IHI • 5

Successive samples from a stable process differ only by chance

0

2

4

6

8

10

12

07.

5 1522

.5 3037

.5 4552

.5 6067

.5 7582

.5 9097

.5

0

2

4

6

8

10

12

14

16

07.

5 1522

.5 3037

.5 4552

.5 6067

.5 7582

.5 9097

.5

0

2

4

6

8

10

12

14

07.

5 1522

.5 3037

.5 4552

.5 6067

.5 7582

.5 9097

.5

0

2

4

6

8

10

12

14

16

18

07.

5 1522

.5 3037

.5 4552

.5 6067

.5 7582

.5 9097

.5

0

2

4

6

8

10

12

14

07.

5 1522

.5 3037

.5 4552

.5 6067

.5 7582

.5 9097

.5

0

2

4

6

8

10

12

14

16

07.

5 1522

.5 3037

.5 4552

.5 6067

.5 7582

.5 9097

.5

Time



3

What Common Cause Variation Looks Like

Points equally likely above or below center line

No trends or shifts or other patterns

0

10

20

30

40

50

60

70

80

90

100

1/1/200

8

1/3/200

8

1/5/200

8

1/7/200

8

1/9/200

8

1/11

/200

8

1/13

/200

8

1/15

/200

8

1/17

/200

8

1/19

/200

8

1/21

/200

8

1/23

/200

8

1/25

/200

8

1/27

/200

8

1/29

/200

8

1/31

/200

8

2/2/200

8

2/4/200

8

2/6/200

8

2/8/200

8

A Stable Process is Predictable

Thus you can confidently

Counsel patients about what to expect

Plan for the future

Inform management

Use PDSA testing to improve it!

!



4

Stable ≠ “OK”

A process may be operating in a stable, predictable fashion but still produce unacceptable results!

“All the shops in Soviet Union were limited to 2-3

types of merchandise, all over country, in every city

or a small village same things were sold, produced

on a few Russian state owned plants.”

Source: www.englishrussia.com

Special Causes

Unintentional

When the system is out of control

February April

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

16 Patients in February and 16 Patients in April

Min

ute

s

2.5

5.0

7.5

10.0

12.5

15.0

17.5

20.0

22.5

25.0

27.5

30.0

A

B

C

C

B

A

UCL=15.3

CL=10.7

LCL=6.1

XmR ChartWaiting times

Intentional

When we’re trying to change the

system

An unstable system!



5

Where Do Special Causes Come From?

Inherent instability in the process

� Lack of standardization – a chaotic process

� Changes in personnel, equipment, management, etc.

Unusual extrinsic events

� Catastrophes, breakdowns, accidents, personnel

issues

Entropy

� Equipment wear, desensitization, habit, emerging culture

Intentional changes – part of an improvement initiative

Unintended Special Causes

An unstable process is subject to special causes. These represent fluctuations in underlying processes.

Process A

(the one we think

we’re measuring)

Process B

Process C

Time



6

Removing Special Causes

Standardize the process by imposing a design & selectively eliminating special causes.

Now your process changes can have testable, repeatable impact.

Special causes

present:

CHAOS!

Special causes

removed

Stabilize, Then Improve

Once the process is stable, your changes can have a predictable, repeatable impact.

HERDING

CATS MOVIE HERE

If you can’t predict the

future behavior of the process, you’re improvements won’t stick!



7

Managing with Data

Changing the Process

Intervention

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

A successful process change is an intentional special cause!



8

Improvement

Once special causes are eliminated and the process is

stable, we can test changes to improve process capability

Reduce variation

Move the mean

Tools for Detecting Change

Line Charts

� No decision rules

� ‘Ocular’ tests only

Run Charts

� Decision rules based on ‘50/50’ principle

� Minimum of 6 points to detect an improvement shift

Control Charts

� Most sensitive tools for detecting special causes

including successful change



9

Shewhart Control Chart

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

Week

Percent of Patients with Pressure Ulcers

3-sigma control limits

Center line

Subgroup

Normal Distribution (Xbar-Sigma Charts)

95.46%

99.73%

68.26%

-3σ +3σ

-2σ +2σ

-1σ +1σ

Mean

If we assume that the data are distributed

normally, then +/- 3 sigma limits include 99.7%

of the common-cause values.



10

Proceed with Caution

There are many types of control charts, which are appropriate for different types of data.

Calculation methods are specific to the type of chart, but interpretation is the same for most chart types.

You cannot create a valid control chart using a simple standard deviation calculation.

Shewhart Control Chart

Upper control limit

Lower control limit

Center: “best guess” value

±3σ limits include 99.7% of common

cause values!

A Shewhart chart is a ‘special cause detector’ – a statistical display that helps you infer the

presence of special causes in a process, beyond a reasonable doubt.



11

Special Cause

Upper control limit

Lower control limit

Center: “best guess” value

±3σ limits include 99.7% of common cause values!

A single point outside the control limits is likely

NOT generated by a stable process, but by some

others set of causes.

Tests for Special Cause

UCL

LCL

Center

2σ

A cluster of points far from the center line is

relatively unlikely from a stable, normally-

distributed process: Special Cause!



12


UCL

LCL

Center

2σ

So are other non-random patterns. These too

are evidence of special causes.

A single point outside the control limits

Six consecutive points increasing (trend up) or

decreasing (trend down)

Two our of three consecutive points near a control

limit (outer one-third)

Eight or more consecutive points above or below

the centerline

Fifteen consecutive points close to the centerline

(inner one-third)

API Rules for

Detecting Special

Cause



13


Outside of limits: A data point that falls outside the limits on the chart, either above the upper limit or below the lower limitShift: Eight or more consecutive POINTS either all above or all below the mean. Skip values on the mean and continue counting points. Values on the mean DO NOT make or break a shiftTrend: Six points all going up or all going down. If the value of two or more successive points is the same, ignore one of the points when counting; like values Do Not make or break a trendTwo Out of Three: Two out of three consecutive points in the outer third of the chart. The two out of three consecutive points can be on the same side, or on either side of the center line. 15 points Hugging the Centerline: 15 consecutive points close to (within inner third of limits) centerline

Testing a Change with a Shewhart Chart

1.Plot the baseline

data & calculate

limits

2.Extend the limits

(centerline only for

some charts)



14

Testing a Change with a Shewhart Chart

3. Plot new data using

baseline limits

(centerline)

Apply decision rules

for special cause

4. If change is

confirmed, plot

limits for new phase

of process

If You Don’t Have Baseline Data

1. Plot all of your data

2. Apply the decision rules

3. Do all of these points appear to be part of the same stable process?

4. Is the pattern consistent with changes?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1/1

/08

2/1

/08

3/1

/08

4/1

/08

5/1

/08

6/1

/08

7/1

/08

8/1

/08

9/1

/08

10

/1/0

8

11

/1/0

8

12

/1/0

8

1/1

/09

2/1

/09

3/1

/09

4/1

/09

5/1

/09

6/1

/09

7/1

/09

8/1

/09

9/1

/09

10

/1/0

9

11

/1/0

9

12

/1/0

9

1/1

/10

2/1

/10

3/1

/10

4/1

/10

5/1

/10

6/1

/10



15

P Charts (for Proportions)

Underlying observation measures a binary attribute, e.g..� Dead or alive � Completed within 1 hour

� Infected or not � Risk assessed at current visit

Observations are randomly sampled from the process in each subgroup

Each plotted data point is the percent (between 0 and 100%) of all observations in the subgroup with the attribute

Common cause variation in the underlying process is modeled by the binomial distribution

P Chart Calculations

ni

xi

si

p

Number of items in measurement window 1

Number of items that “have it” in measurement window 1

∑∑

nx

i

iCenter lineobserveditemsTotal

ithavethatitemsTotal

__

"__"__

Standard deviation

ni

pp )1( −



16


Control Limits

isp 3+Upper Control Limit

isp 3−Lower Control Limit

DVT Data

Let’s build a control chart to illustrate how it works.

Obs

Number

of

Patients

Number with

appropriate

DVT

1 17 5

2 19 12

3 21 10

4 20 10

5 21 8

6 21 7

7 18 7

8 22 8

9 21 9

10 20 11

11 18 8

12 22 7

SUM 240 102



17


For the first DVT Observation

171

=n

51 =x

102=∑ ix

240=∑ in785.)12(.3425. =+=UCL

065.)12(.3425. =−=LCL425.240

102==pMean

12.17

)425.1(425.=

−=isStandard deviation

Is the Process Stable Before the Change?

Pts With DVT Prophylaxis

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10 11 12



18

Obs

Number

of

Patients

Number with

appropriate

DVT

% Approp Mean SD UCL LCL

1 17 5 29.4% 0.425 0.120 0.785 0.065

2 19 12 63.2% 0.425 0.113 0.765 0.085

3 21 10 47.6% 0.425 0.108 0.749 0.101

4 20 10 50.0% 0.425 0.111 0.757 0.093

5 21 8 38.1% 0.425 0.108 0.749 0.101

6 21 7 33.3% 0.425 0.108 0.749 0.101

7 18 7 38.9% 0.425 0.117 0.775 0.075

8 22 8 36.4% 0.425 0.105 0.741 0.109

9 21 9 42.9% 0.425 0.108 0.749 0.101

10 20 11 55.0% 0.425 0.111 0.757 0.093

11 18 8 44.4% 0.425 0.117 0.775 0.075

12 22 7 31.8% 0.425 0.105 0.741 0.109

13 20 16 80.0% 0.425 0.111 0.757 0.093

14 17 16 94.1% 0.425 0.120 0.785 0.065

15 24 22 91.7% 0.425 0.101 0.728 0.122

SUM 301 156

A P-Chart Confirms A Change


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15



19

Two Mistakes to Avoid

‘Jumping the Gun’ (Type I Error or False Positive)

� Responding to a data point as if it were a special

cause when, in fact, the system is stable

Failure to Detect (Type II Error or False Negative)

� Ignoring a data point that indicates a special cause when, in fact, the system of causes has changed

What Do Type I and II Errors Mean for DVT?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15




20

Mammography Screening

Measure: Percent of women over 50 in a sample of 50 who obtain documented mammograms within 3 months of receiving reminder

Data for subgroups collected by month reminder is sent out

References

Benneyan, J. C. (2001). "Number-between g-type statistical quality control charts for monitoring adverse events." Health Care Manag Sci 4(4): 305-18.

Benneyan, J. (2008). "Design, use and performance of statistical control charts for clinical process improvement." International Journal of Six Sigma 4(3): 219-239.

Langley, G. J., K. M. Nolan, et al. (2009). The improvement guide : a practical approach to enhancing organizational performance. San Francisco, Jossey-Bass.

Moen, R. D., T. W. Nolan, et al. (1999). Quality improvement through planned experimentation. New York, McGraw Hill.

Mohammed, M. A., P. Worthington, et al. (2008). "Plotting basic control charts: tutorial notes for healthcare practitioners." Qual Saf Health Care 17(2): 137-145

Perla, R. J., L. P. Provost, et al. (2011). "The run chart: a simple analytical tool for learning from variation in healthcare processes." BMJ Qual Saf 20(1): 46-51.

Provost, L. P. and S. K. Murray (2010). The Data Guide - Learning from data to improve health care. Austin TX, Associates in Process Improvement - www.pipproducts.com.



21

Choosing and Using Control Charts

Which chart to use?

Data types; subgroups; rates

Practice with chart interpretation

Type of Data

Discrete

Count Fraction

No

Denominator

With

Denominator

(Rate)

Percent or

proportion

Continuous

1

observation

per

subgroup

C-Chart U-Chart P-Chart Individuals X-bar & S

>1

observation

per

subgroup



22

Discrete and Continuous Data

Discrete - assign an observation to a category

Continuous - take on any fractional value on a continuous scale


Discrete - assign an observation to a category

Continuous - take on any fractional value on a continuous scale



23


In making this determination, consider the nature of the

observation; i.e. disregard aggregations such as counts,

percentages, averages, etc.

Temperature Continuous

Height or weight Continuous

LDL Continuous

Cycle time for patient visits Continuous

Complete in less than 1 hr Discrete

# Patient falls in Unit 12 last week Discrete

Average A1C for practice’s DM patients Continuous

% of women receiving m’grams in 90 days Discrete

From Continuous to Discrete

You can always convert a continuous measure into a discrete measure by ‘slicing’ it into categories.

You cannot go the other way: Information is lost!

BMI

RULE OF THUMB: Store “unprocessed” data values in your database –you can always assign to categories later:• BMI• Blood pressure• Birth date



24

Rate or Percent?

Percents need a P-Chart

Rates go on a U-Chart

Subgroup

A point on a Shewhart control chart

A set of observations pulled from the stream of process data (within a frame)

Process should be stable within the subgroup (maximize variation between subgroups; minimize variation within subgroups)

A subgroup is a usually a ‘snapshot’ of the process at a particular place and time



25

Subgroups

Observations are sampled from the process within a frame (aka ‘area of opportunity’)

Subgroup values combine 1 or more individual observations:• Count• Percent• Rate• Average• Single continuous value

A Subgroup is a Point on A Chart

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Week

Medication Errors Per 10,000 Orders

A subgroup(with 1 or more observations inside)



26

Subgroup Example

Measure: time to process a new hospital admission

We suspect that admission staff on different shifts use different procedures for processing patients.

We should choose a subgroup to minimize variation within the subgroups

Subgroup = average time for 5 admissions randomly selected within a shift on Unit XWhat are possible subgroups for admission process time?

Type of Data

Discrete

Count Fraction

No

Denominator

With

Denominator

(Rate)

Percent or

proportion

Continuous

1

observation

per

subgroup

C-Chart U-Chart P-Chart Individuals X-bar & S

>1

observation

per

subgroup



27

Inpatient Medication Process

Exercise: Medication Measures



28

Chart Calculations

Each chart type has its own construction formulas & procedures.

See Mohammed & Worthington (2008)* for overview of calculation of basic chart types. See The Data Guide for details.

Once constructed, all charts are interpreted in the same way.

*Mohammed, M. A., P. Worthington, et al. (2008). "Plotting basic control charts: tutorial notes for

healthcare practitioners." Qual Saf Health Care 17(2): 137-145.

When Do We Recalculate Limits?

You’re still gathering data to find a stable baseline: you have “trial” limits with <20 subgroups

You have identified special causes and want to assess stability with those subgroups removed

When improvements have be made to the process and the improvements result in special causes on the Shewhart chart

When you have reason to believe that the process is now operating in a new mode, and you want to assess its stability



29

Measuring Infrequent Events

Holy Family HospitalRate of occurrence of hospital-acquired MRSA infections

Is this team improving its infection rate?

What is the problem? How could you fix it?


Rate of occurrentce of MRSA BSI and HAP per 1000 patient days

0.00

0.50

1.00

1.50

2.00

2.50

Sept Oct Nov Dec Jan

When events are infrequent, it’s hard to see whether our work is having the intended result.



30


Plotting the number of days since the prior event shows that infections are becoming steadily less frequent.

15 days since last event (today 1/31/2008)

Note – frequency of cases should be relatively constant!


Time between: Number of days between events

� Use T-Chart to model

� Assumes volume is relatively constant

� Data are just dates of occurrence

Cases between: Number of processed items (e.g. cases, patients) between events

� Use G-Chart

� Standardizes volume

� Requires more complex data extraction



31

G-Chart for Events-Between

Rare Events: Joint Revisions*

Have we improved?

*Simulated Data

T-Chart plots days

between events.



32

Discussion: Interpreting Control Charts

Carey, R. G. and Lloyd, R.C. (1995). Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control Applications. Milwaukee, WI, ASQ Quality Press.

Subgroup Size - Effect on Limits

Equally sized subgroups yield straight limits

5

10

15

20

25

30

1/7

/07

2/7

/07

3/7

/07

4/7

/07

5/7

/07

6/7

/07

7/7

/07

8/7

/07

9/7

/07

10

/7/0

7

11

/7/0

7

12

/7/0

7

1/8

/08

2/8

/08

3/8

/08

4/8

/08

5/8

/08

6/8

/08

7/8

/08

8/8

/08

9/8

/08

10

/8/0

8

11

/8/0

8

12

/8/0

8

1/9

/09

2/9

/09

3/9

/09

4/9

/09

5/9

/09

6/9

/09

7/9

/09

8/9

/09

9/9

/09

10

/9/0

9

11

/9/0

9

12

/9/0

9

Hospital Code Rate (codes / 1000 discharges)

Month Codes Discharges

J 36 2500

F 32 2500

M 43 2500

A 52 2500

M 59 2500

J 34 2500

J 52 2500

A 36 2500

S 32 2500

O 49 2500

N 48 2500

D 68 2500

J 54 2500

F 62 2500

M 35 2500

A 24 2500

M 47 2500

J 41 2500U-Chart with constant sample size



33

Subgroup Size - Effect on Limits

Unequal number of observations per subgroup results in ‘squiggly’ limits (smaller n means wider limits, means less sensitivity)

0

5

10

15

20

25

30

35

40

3/6

/06

4/6

/06

5/6

/06

6/6

/06

7/6

/06

8/6

/06

9/6

/06

10

/6/0

6

11

/6/0

6

12

/6/0

6

1/7

/07

2/7

/07

3/7

/07

4/7

/07

5/7

/07

6/7

/07

7/7

/07

8/7

/07

9/7

/07

10

/7/0

7

11

/7/0

7

12

/7/0

7

1/8

/08

2/9

/08

3/1

2/0

8

4/1

3/0

8

ADEs by Month

Month # ADEs Doses

M-06 50 17110

A-06 44 12140

M-06 47 17990

J-06 32 14980

J-06 51 21980

A-06 57 15320

S-06 43 12990

O-06 61 19760

N-06 30 8670

D-06 32 12680

J-07 41 20330

F-07 47 18550

M-07 31 14310

A-07 11 9730

M-07 3 11470

J-07 11 21390 P-Chart with varying sample sizes

P-Chart Subgroup Sizes

Date post:	25-May-2018
Category:	Documents
Upload:	dohanh
View:	215 times
Download:	0 times

2B - Control Charts - IHIapp.ihi.org/.../Event-2590/Document-4161/Control_Charts.pdfAdvanced...

Documents