Post on 15-Dec-2015
transcript
A Comparison ofShewhart and CUSUM Methodsfor Diagnosis in a Vendor Certification Study
Erwin M. SanigaDept. of Bus. Admin.
University of DelawareNewark, DE 19716
302-831-2555sanigae@be.udel.edu
James M. LucasJ.M. Lucas and Associates
302-368-12145120 New Kent RoadWilmington, DE 19808
Jamesm.lucas@worldnet.att.net
Darwin J. DavisDept. of Bus. Admin.
University of DelawareNewark, DE 19716
302-831-2555davisd@be.udel.ed
Presenter: Erwin Saniga
Purpose: To examine the performance of alternative methods to study processes where quality is measured by counts and counts are low.
Example Large Wilmington (DE) area credit card bank Processes credit card applications Four vendors process these applications Wish to implement a vendor certification and
quality improvement program
BankVendorA
VendorD
VendorB
VendorC
We will show in this presentation:
When exploring this type of data for performance evaluation or process capability analysis, four different types of plots can reveal different things about the process A traditional sequence plot Adding a Shewhart UCL and a
method to detect improvements to the traditional sequence plot
A CUSUM plot Adding a “V-mask” to the
CUSUM plot
Actual results for the four vendors
Each point represents the number of defectives resulting from an inspection of 50 random credit card applications that were processed during the day. These were taken at the end of the day.
Each credit card application can be processed correctly or incorrectly.
Vendor Analysis
Average vendor performance Vendor A average p = 0.0246 Vendor B average p = 0.0510 Vendor C average p = 0.0247 Vendor D average p = 0.0294
Analyst questions: What caused the spikes at various points in time for
each vendor? What caused the sequence of “good” (zero
defective) samples for various vendors? Why is Vendor B doing poorly when compared to
Vendors A, C, and D? Are these substantive differences?
If the data were available in real time and we could plan the data collection we might:
Investigate the cause of a spike or run of good points immediately
Keep a diary or log of variables identified during a focus group meeting of employees, managers, etc.
CUSUM SEQUENCE (DIAGNOSTIC) PLOTS
Let Xj = the actual number of defectives observed in the jth sample
The ith CUSUM is then:
where k is the reference value. We use a reference value of
k=1.25 which is the average count of defectives in a sample of 50 for the three “good” vendors.
i
1jji kXC
Process Averages for CUSUM Sequence Plot
The CUSUM sequence plot can identify “good”, “average” or “bad” regimes.
Regime average is determined by the slope (in this case, slope = 0 implies 1.25 defects).
The average count from periods L to M is given by:
Vendor A ExampleRegime 1 – Days 46 to 122
Ave. count = 1.25 + [-27.5 – (-0.25)]/[122 – 46 + 1]
= 0.896
Regime 2 – Days 176 to 199
Ave. count = 1.25 + (6.25 – (-21.75)/(199 – 176 + 1)
= 2.417
1LM
CC25.1CountAverage 1LM
CUSUM Sequence Plot Summary
There are various “regimes’ noted by the CUSUM sequence plots that are not immediately recognizable from the traditional sequence plot of the counts.
Some of these regimes indicate notable “good” or “bad” performance.
CUSUM sequence plot questions: What are the reasons for the good
performance in certain regimes? What are the reasons for the bad
performance in certain regimes? What happened on the particular days a
change point was observed?
If the data were available in real time we might keep a diary or log of variables possibly associated with performance and investigate these.
Vendor A Comparison
A comparison of the CUSUM sequence plot with the traditional sequence plot
Traditional sequence plot: spikes at 35, 128, 157, 169, 193, and 197
zero counts from 200 to 207 consecutively
CUSUM sequence plot:“Average” process on days 1-14
“Good” process on days 15 to 32
“Bad” process on days 33 to 45
“Good” process on days 46 to 122
“Average” process on days 123 to 175
“Bad” process on days 176 to 199
“Good” process on days 200 to 207
Summary of plots
Traditional sequence plots are good for detecting shocks to the system and rare events
CUSUM sequence plots are good for detecting regimes (periods of good, bad, and average behavior)
Other Questions
For traditional sequence plots: Are the spikes unusual when
compared to what might happen under pure chance?
Are the zero count sequences unusual in the sense that they indicate the process has improved?
For CUSUM sequence plots: Are the regimes we observed unusual
when compared to what might happen under pure chance?
Traditional Sequence Plot
Shewhart chart provides a simple and effective way to “signal” spikes as being significantly unusual.
Control Limits for the Shewhart chart:
where p is the average proportion defective.
)p1(np3npLCL,UCL
The Upper Control Limit
For our data with an average count of 1.25 (p=0.025, n=50) we have
UCL =4.56 (the Shewhart 3 sigma limit)
In our examples we use UCL = 5 (signal at 6) to ensure the ARL in control is sufficiently large.
For UCL= 4 (signal at 5) ARL in control = 123
For UCL= 5 (signal at 6) ARL in control = 662
(Generally, for low count data adding 1 to the UCL yields a more desirable in control ARL)
The Lower Control Limit
LCL = -2.06 (no lower control limit)Solutions for the lower side (detecting
improvement) Ryan(1989) and Schwertman and Ryan(1997)
suggest equal tail probability methods. These do not work well when P(0) is high:
In control ARL is low or, equivalently, false alarm probabilities are high.
Acosta-Mejia (1999) suggests counting successive results below a modified centerline Dominated by CUSUM methods in terms of
ARL. Does not work well for P(0) large. CUSUM methods (e.g. Reynolds and
Stoumbos (1999, 2000) Optimal for a particular shift in terms of
ARL yields Harder to design and use than special
CUSUMs we will provide.
Count the Zeros: Special CUSUMs
The simplest low side CUSUMs
Method 1: Signal if k in a row samples have zero defectives
Method 2: Signal if 2 in t samples have zero defectives
Properties: Optimal for large shifts Easy to use Easy to design Easy to understand
Low Side Schemes
Comparison of our special CUSUM (k in a row) and the CUSUM.
Recommended Policies and ARL Formulas for Low Side Attribute Schemes with
Low Side In-Control ARL 500.
P(0) Recommended Policy
for Low Side Attribute Scheme
Low Side ARL Formula
P(0) 0.046 k in a row
(k 2)
1k
ojk
j
)0(P
)0(PARL
0.002 < P(0) 0.046 2 in t (t 2) ])]0(P1[1[*)0(P
)]0(P1[2ARL
1t
1t
P(0) 0.002 Shewhart
Lower Control Limit
1
1ARL
The Computational CUSUM
yi = the observed count
For the high side
Si = Max {0, Si-1 - kH + yi }
If Si > hH in a signal is given
For the low side
Si = Max {0, Si-1 + kL - yi }
If Si > hL in a signal is given
For our example we use:
kH = 2 hH = 4
kL = 0.75 hL = 5.25
Comparison of Signals
Vendor A
CUSUM SIGNALS SHEWHART SIGNALS High Side Low Side High Side Low Side
35 S 35 S 205 R 45 R 157 S 157 S 197 S
180-197R(2 signals) 207 R
Vendor B
CUSUM SIGNALS SHEWHART SIGNALS High Side Low Side High Side Low Side
37-72 R (9 signals) 37 S 91-101 R (3 signal) 46 S
52 S 55 S 58 S 63 S 72 S
106-123 R (13 signals)
106-123 R (8 signals)
Comparison of Signals
VENDOR C
CUSUM SIGNALS SHEWHART SIGNALS High Side Low Side High Side Low Side
72 S 12 R 55 S 12-19 R 89 R 103 R 72 S 101-104 R
106-107 R 89 S 105-107 R
(3 signals)
VENDOR D
CUSUM SIGNALS SHEWHART SIGNALS High Side Low Side High Side Low Side
30-44 R (5 signals) 30 S 55, 56 R 116-120 (2 signals) 38-423 R 147 R 170-173 (2 signals) 58 S
116 S 120 S 170 S