DMAIC ^^^ CHAPTER 10 Measurement Systems Analysis R&R STUDIES FOR CONTINUOUS DATA Discrimination, stability, bias, repeatability, reproducibility, and linearity Modern measurement system analysis goes well beyond calibration. A gage can be perfectly accurate when checking a standard and still be entirely unac- ceptable for measuring a product or controlling a process. This section illus- trates techniques for quantifying discrimination, stability, bias, repeatability, reproducibility and variation for a measurement system. We also show how to express measurement error relative to the product tolerance or the process var- iation. For the most part, the methods shown here use control charts. Control charts provide graphical portrayals of the measurement processes that enable the analyst to detect special causes that numerical methods alone would not detect. MEASUREMENT SYSTEM DISCRIMINATION Discrimination, sometimes called resolution, refers to the ability of the measurement system to divide measurements into ‘‘data categories.’’ All parts within a particular data category will measure the same. For example,
Transcript
DMAIC^ ^ ^
CHAPTER
10
Measurement SystemsAnalysis
R&R STUDIES FOR CONTINUOUS DATADiscrimination, stability, bias, repeatability,reproducibility, and linearity
Modern measurement system analysis goes well beyond calibration. A gagecan be perfectly accurate when checking a standard and still be entirely unac-ceptable for measuring a product or controlling a process. This section illus-trates techniques for quantifying discrimination, stability, bias, repeatability,reproducibility and variation for a measurement system. We also show how toexpress measurement error relative to the product tolerance or the process var-iation. For the most part, the methods shown here use control charts. Controlcharts provide graphical portrayals of the measurement processes that enablethe analyst to detect special causes that numerical methods alone would notdetect.
MEASUREMENT SYSTEMDISCRIMINATIONDiscrimination, sometimes called resolution, refers to the ability of the
measurement system to divide measurements into ‘‘data categories.’’ Allparts within a particular data category will measure the same. For example,
if a measurement system has a resolution of 0.001 inches, then items measur-ing 1.0002, 1.0003, 0.9997 would all be placed in the data category 1.000, i.e.,they would all measure 1.000 inches with this particular measurement system.A measurement system’s discrimination should enable it to divide the regionof interest into many data categories. In Six Sigma, the region of interest isthe smaller of the tolerance (the high specification minus the low specifica-tion) or six standard deviations. A measurement system should be able todivide the region of interest into at least five data categories. For example, ifa process was capable (i.e., Six Sigma is less than the tolerance) ands ¼ 0:0005, then a gage with a discrimination of 0.0005 would be acceptable(six data categories), but one with a discrimination of 0.001 would not(three data categories). When unacceptable discrimination exists, the rangechart shows discrete ‘‘jumps’’ or ‘‘steps.’’ This situation is illustrated inFigure 10.1.
326 MEASUREMENT SYSTEMS ANALYSIS
Figure 10.1. Inadequate gage discrimination on a control chart.
Note that on the control charts shown in Figure 10.1, the data plotted are thesame, except that the data on the bottom two charts were rounded to the nearest25. The effect is most easily seen on the R chart, which appears highly stratified.As sometimes happens (but not always), the result is to make the X-bar chartgo out of control, even though the process is in control, as shown by the controlcharts with unrounded data. The remedy is to use a measurement system cap-able of additional discrimination, i.e., add more significant digits. If this cannotbe done, it is possible to adjust the control limits for the round-off error byusing a more involved method of computing the control limits, see Pyzdek(1992a, pp. 37^42) for details.
STABILITYMeasurement system stability is the change in bias over time when using a
measurement system to measure a given master part or standard. Statistical sta-bility is a broader term that refers to the overall consistency of measurementsover time, including variation from all causes, including bias, repeatability,reproducibility, etc. A system’s statistical stability is determined through theuse of control charts. Averages and range charts are typically plotted on mea-surements of a standard or a master part. The standard is measured repeatedlyover a short time, say an hour; then the measurements are repeated at predeter-mined intervals, say weekly. Subject matter expertise is needed to determinethe subgroup size, sampling intervals and measurement procedures to be fol-lowed. Control charts are then constructed and evaluated. A (statistically) stablesystemwill showno out-of-control signals on anX-control chart of the averages’readings. No ‘‘stability number’’ is calculated for statistical stability; the systemeither is or is not statistically stable.
Once statistical stability has been achieved, but not before, measurement sys-tem stability can be determined. One measure is the process standard deviationbased on the R or s chart.
R chart method:
�̂� ¼�RR
d2
s chart method:
�̂� ¼�ss
c4
The values d2 and c4 are constants from Table 11 in the Appendix.
R&R studies for continuous data 327
BIASBias is the difference between an observed average measurement result and a
reference value. Estimating bias involves identifying a standard to representthe reference value, then obtaining multiple measurements on the standard.The standardmight be amaster part whose value has been determined by amea-surement systemwithmuch less error than the system under study, or by a stan-dard traceable to NIST. Since parts and processes vary over a range, bias ismeasured at a point within the range. If the gage is non-linear, bias will not bethe same at each point in the range (see the definition of linearity above).
Bias can be determined by selecting a single appraiser and a single referencepart or standard. The appraiser then obtains a number of repeated measure-ments on the reference part. Bias is then estimated as the difference betweenthe average of the repeated measurement and the known value of the referencepart or standard.
Example of computing biasA standard with a known value of 25.4 mm is checked 10 times by one
mechanical inspector using a dial caliper with a resolution of 0.025 mm. Thereadings obtained are:
The average is found by adding the 10measurements together and dividing by10,
�XX ¼254:051
10¼ 25:4051 mm
The bias is the average minus the reference value, i.e.,
bias ¼ average� reference value
¼ 25:4051 mm� 25:400 mm ¼ 0:0051 mm
The bias of themeasurement system can be stated as a percentage of the toler-ance or as a percentage of the process variation. For example, if this mea-surement system were to be used on a process with a tolerance of � 0.25 mmthen
This is interpreted as follows: this measurement systemwill, on average, pro-duce results that are 0.0051 mm larger than the actual value. This differencerepresents 1% of the allowable product variation. The situation is illustrated inFigure 10.2.
REPEATABILITYAmeasurement system is repeatable if its variability is consistent. Consistent
variability is operationalized by constructing a range or sigma chart based onrepeated measurements of parts that cover a significant portion of the processvariation or the tolerance, whichever is greater. If the range or sigma chart isout of control, then special causes aremaking themeasurement system inconsis-tent. If the range or sigma chart is in control then repeatability can be estimatedby finding the standard deviation based on either the average range or the aver-age standard deviation. The equations used to estimate sigma are shown inChapter 9.
Example of estimating repeatabilityThe data in Table 10.1 are from a measurement study involving two inspec-
tors. Each inspector checked the surface finish of five parts, each part waschecked twice by each inspector. The gage records the surface roughness in m-inches (micro-inches). The gage has a resolution of 0.1 m-inches.
Table 10.1.Measurement system repeatability study data.
PART READING #1 READING #2 AVERAGE RANGE
INSPECTOR#1
1 111.9 112.3 112.10 0.4
2 108.1 108.1 108.10 0.0
3 124.9 124.6 124.75 0.3
4 118.6 118.7 118.65 0.1
5 130.0 130.7 130.35 0.7
INSPECTOR #2
1 111.4 112.9 112.15 1.5
2 107.7 108.4 108.05 0.7
3 124.6 124.2 124.40 0.4
4 120.0 119.3 119.65 0.7
5 130.4 130.1 130.25 0.3
The data and control limits are displayed in Figure 10.3. The R chart analysisshows that all of the R values are less than the upper control limit. This indicatesthat the measurement system’s variability is consistent, i.e., there are no specialcauses of variation.
Note that many of the averages are outside of the control limits. This is theway it should be! Consider that the spread of the X-bar chart’s control limits isbased on the average range, which is based on the repeatability error. If theaverageswerewithin the control limits it wouldmean that the part-to-part varia-tion was less than the variation due to gage repeatability error, an undesirablesituation. Because the R chart is in control we can now estimate the standarddeviation for repeatability or gage variation:
�e ¼�RR
d�2ð10:1Þ
where d�2 is obtained from Table 13 in the Appendix. Note that we are using d�2and not d2. The d
�2 values are adjusted for the small number of subgroups typi-
cally involved in gage R&R studies. Table 13 is indexed by two values: m is thenumber of repeat readings taken (m ¼ 2 for the example), and g is the numberof parts times the number of inspectors (g ¼ 5� 2 ¼ 10 for the example).This gives, for our example
�e ¼�RR
d�2¼
0:51
1:16¼ 0:44
R&R studies for continuous data 331
Figure 10.3. Repeatability control charts.
The repeatability from this study is calculated by 5:15�e ¼ 5:15�0:44 ¼ 2:26. The value 5.15 is the Z ordinate which includes 99% of a standardnormal distribution.
REPRODUCIBILITYA measurement system is reproducible when different appraisers produce
consistent results. Appraiser-to-appraiser variation represents a bias due toappraisers. The appraiser bias, or reproducibility, can be estimated by com-paring each appraiser’s average with that of the other appraisers. The standarddeviation of reproducibility (�o) is estimated by finding the range betweenappraisers (Ro) and dividing by d
�2 . Reproducibility is then computed as 5.15�o.
Reproducibility example (AIAGmethod)Using the data shown in the previous example, each inspector’s average is
computed and we find:
Inspector #1 average ¼ 118:79�-inches
Inspector #2 average ¼ 118:90�-inches
Range ¼ Ro ¼ 0:11�-inches
Looking in Table 13 in the Appendix for one subgroup of two appraisers wefind d�2 ¼ 1:41 ðm ¼ 2, g ¼ 1), since there is only one range calculation g ¼ 1.Using these results we find Ro=d
�2 ¼ 0:11=1:41 ¼ 0:078.
This estimate involves averaging the results for each inspector over all of thereadings for that inspector. However, since each inspector checked each partrepeatedly, this reproducibility estimate includes variation due to repeatabilityerror. The reproducibility estimate can be adjusted using the following equa-tion:
As sometimes happens, the estimated variance from reproducibility exceedsthe estimated variance of repeatability + reproducibility. When this occurs theestimated reproducibility is set equal to zero, since negative variances are the-oretically impossible. Thus, we estimate that the reproducibility is zero.
and the measurement system variation, or gage R&R, is 5.15�m. For our datagage R&R¼ 5:15� 0:44 ¼ 2:27.
Reproducibility example (alternative method)One problem with the above method of evaluating reproducibility error is
that it does not produce a control chart to assist the analyst with the evaluation.The method presented here does this. This method begins by rearranging thedata in Table 10.1 so that all readings for any given part become a single row.This is shown in Table 10.2.
Observe that when the data are arranged in this way, theR valuemeasures thecombined range of repeat readings plus appraisers. For example, the smallestreading for part #3 was from inspector #2 (124.2) and the largest was frominspector #1 (124.9). Thus, R represents two sources of measurement error:repeatability and reproducibility.
R&R studies for continuous data 333
Table 10.2.Measurement error data for reproducibility evaluation.
INSPECTOR#1 INSPECTOR#2
Part Reading 1 Reading 2 Reading 1 Reading 2 X bar R
1 111.9 112.3 111.4 112.9 112.125 1.5
2 108.1 108.1 107.7 108.4 108.075 0.7
3 124.9 124.6 124.6 124.2 124.575 0.7
4 118.6 118.7 120 119.3 119.15 1.4
5 130 130.7 130.4 130.1 130.3 0.7
Averages! 118.845 1
The control limits are calculated as follows:Ranges chart
�RR ¼ 1:00
UCL ¼ D4�RR ¼ 2:282� 1:00 ¼ 2:282
Note that the subgroup size is 4.
Averages chart
��XX�XX ¼ 118:85
LCL ¼ ��XX�XX � A2�RR ¼ 118:85� 0:729� 1 ¼ 118:12
UCL ¼ ��XX�XX þ A2�RR ¼ 118:85þ 0:729� 1 ¼ 119:58
The data and control limits are displayed in Figure 10.4. The R chart analysisshows that all of the R values are less than the upper control limit. This indicatesthat the measurement system’s variability due to the combination of repeatabil-ity and reproducibility is consistent, i.e., there are no special causes of variation.
Using this method, we can also estimate the standard deviation of repro-ducibility plus repeatability, as we can find �o ¼ Ro=d
Using this we estimate reproducibility as 5:15� 0:19 ¼ 1:00.
PART-TO-PARTVARIATIONThe X-bar charts show the part-to-part variation. To repeat, if the measure-
ment system is adequate,most of the parts will fall outside of the X -bar chart con-trol limits. If fewer than half of the parts are beyond the control limits, thenthe measurement system is not capable of detecting normal part-to-part vari-ation for this process.
Part-to-part variation can be estimated once the measurement process isshown to have adequate discrimination and to be stable, accurate, linear (seebelow), and consistent with respect to repeatability and reproducibility. If thepart-to-part standard deviation is to be estimated from themeasurement systemstudy data, the following procedures are followed:
1. Plot the average for each part (across all appraisers) on an averages con-trol chart, as shown in the reproducibility error alternate method.
2. Con¢rm that at least 50% of the averages fall outside the control limits. Ifnot, ¢nd a better measurement system for this process.
3. Find the range of the part averages, Rp.4. Compute �p ¼ Rp=d
�2 , the part-to-part standard deviation. The value of
d�2 is found in Table 13 in the Appendix usingm ¼ the number of partsand g ¼ 1, since there is only one R calculation.
5. The 99% spread due to part-to-part variation (PV) is found as 5.15�p.
Once the above calculations have been made, the overall measurement sys-tem can be evaluated.
1. The total process standard deviation is found as �t ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi�2m þ �2
p
q. Where
�m ¼ the standard deviation due to measurement error.2. Total variability (TV) is 5.15�t.3. The percent repeatability and reproducibility (R&R) is 100� ð�m=�tÞ%.
R&R studies for continuous data 335
4. The number of distinct data categories that can be created with this mea-surement system is 1.41� (PV/R&R).
EXAMPLEOFMEASUREMENT SYSTEMANALYSISSUMMARY
1. Plot the average for each part (across all appraisers) on an averages con-trol chart, as shown in the reproducibility error alternate method.Done above, see Figure 10.3.
2. Con¢rm that at least 50% of the averages fall outside the control limits. Ifnot, ¢nd a better measurement system for this process.4 of the 5 part averages, or 80%, are outside of the control limits. Thus,the measurement system error is acceptable.
3. Find the range of the part averages, Rp.Rp ¼ 130:3� 108:075 ¼ 22:23.
4. Compute �p ¼ Rp=d�2 , the part-to-part standard deviation. The value of
d�2 is found in Table 13 in the Appendix usingm ¼ the number of partsand g ¼ 1, since there is only one R calculation.
m ¼ 5, g ¼ 1, d�2 ¼ 2:48, �p ¼ 22:23=2:48 ¼ 8:96.5. The 99% spread due to part-to-part variation (PV) is found as 5.15�p.
5:15� 8:96 ¼ PV ¼ 46:15.
Once the above calculations have been made, the overall measurement sys-tem can be evaluated.
1. The total process standard deviation is found as �t ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi�2m þ �2
4. The number of distinct data categories that can be created with this mea-surement system is 1:41� ðPV=R&RÞ.
1:41�46:15
2:27¼ 28:67 ¼ 28
336 MEASUREMENT SYSTEMS ANALYSIS
Since the minimum number of categories is five, the analysis indicates thatthis measurement system is more than adequate for process analysis or processcontrol.
Gage R&R analysis using MinitabMinitab has a built-in capability to perform gage repeatability and reproduci-
bility studies. To illustrate these capabilities, the previous analysis will berepeated using Minitab. To begin, the data must be rearranged into the formatexpected by Minitab (Figure 10.5). For reference purposes, columns C1^C4contain the data in our original format and columns C5^C8 contain the samedata in Minitab’s preferred format.
Minitab offers two different methods for performing gage R&R studies:crossed and nested. Use gage R&R nested when each part can be measured byonly one operator, as with destructive testing. Otherwise, choose gage R&Rcrossed. To do this, select Stat>Quality Tools>Gage R&R Study (Crossed)to reach the Minitab dialog box for our analysis (Figure 10.6). In addition tochoosing whether the study is crossed or nested, Minitab also offers both the
R&R studies for continuous data 337
Figure 10.5.Data formatted for Minitab input.
ANOVA and the X-bar and R methods. You must choose the ANOVA optionto obtain a breakdown of reproducibility by operator and operator by part. Ifthe ANOVA method is selected, Minitab still displays the X-bar and R chartsso you won’t lose the information contained in the graphics. We will useANOVA in this example. Note that the results of the calculations will differslightly from those we obtained using the X-bar and R methods.
There is an option in gage R&R to include the process tolerance. This willprovide comparisons of gage variation with respect to the specifications in addi-tion to the variability with respect to process variation. This is useful informa-tion if the gage is to be used to make product acceptance decisions. If theprocess is ‘‘capable’’ in the sense that the total variability is less than the toler-ance, then any gage that meets the criteria for checking the process can also beused for product acceptance.However, if the process is not capable, then its out-put will need to be sorted and the gage used for sorting may need more discrimi-natory power than the gage used for process control. For example, a gagecapable of 5 distinct data categories for the process may have 4 or fewer for theproduct. For the purposes of illustration, we entered a value of 40 in the processtolerance box in the Minitab options dialog box (Figure 10.7).
OutputMinitab produces copious output, including six separate graphs, multiple
tables, etc. Much of the output is identical to what has been discussed earlier inthis chapter and won’t be shown here.
Table 10.3 shows the analysis of variance for the R&R study. In the ANOVAtheMS for repeatability (0.212) is used as the denominator or error term for cal-culating the F-ratio of the Operator*PartNum interaction; 0.269/0.212 = 1.27.The F-ratio for the Operator effect is found by using the Operator*PartNuminteraction MS term as the denominator, 0.061/0.269 = 0.22. The F-ratios areused to compute the P values, which show the probability that the observed var-iation for the source row might be due to chance. By convention, a P value lessthan 0.05 is the critical value for deciding that a source of variation is ‘‘signifi-
cant,’’ i.e., greater than zero. For example, the P value for the PartNum row is 0,indicating that the part-to-part variation is almost certainly not zero. The Pvalues for Operator (0.66) and the Operator*PartNum interaction (0.34) aregreater than 0.05 so we conclude that the differences accounted for by thesesources might be zero. If the Operator term was significant (P < 0.05) wewould conclude that there were statistically significant differences betweenoperators, prompting an investigation into underlying causes. If the interactionterm was significant, we would conclude that one operator has obtained differ-ent results with some, but not all, parts.
Minitab’s next output is shown in Table 10.4. This analysis has removed theinteraction term from the model, thereby gaining 4 degrees of freedom for theerror term and making the test more sensitive. In some cases this might identifya significant effect that was missed by the larger model, but for this examplethe conclusions are unchanged.
Minitab also decomposes the total variance into components, as shown inTable 10.5. The VarComp column shows the variance attributed to each source,while the % of VarComp shows the percentage of the total variance accountedfor by each source. The analysis indicates that nearly all of the variation isbetween parts.
The variance analysis shown in Table 10.5, while accurate, is not in originalunits. (Variances are the squares of measurements.) Technically, this is the cor-rect way to analyze information on dispersion because variances are additive,while dispersion measurements expressed in original units are not. However,there is a natural interest in seeing an analysis of dispersion in the originalunits so Minitab provides this. Table 10.6 shows the spread attributable to the
340 MEASUREMENT SYSTEMS ANALYSIS
Table 10.4. Two-way ANOVA table without interaction.
Source DF SS MS F P
PartNum 4 1301.18 325.294 1426.73 0
Operator 1 0.06 0.061 0.27 0.6145
Repeatability 14 3.19 0.228
Total 19 1304.43
different sources. The StdDev column is the standard deviation, or the squareroot of the VarComp column in Table 10.5. The Study Var column shows the99% confidence interval using the StdDev. The % Study Var column is theStudy Var column divided by the total variation due to all sources. And the %Tolerance is the Study Var column divided by the tolerance. It is interestingthat the % Tolerance column total is greater than 100%. This indicates that themeasured process spread exceeds the tolerance. Although this isn’t a processcapability analysis, the data do indicate a possible problem meeting tolerances.The information in Table 10.6 is presented graphically in Figure 10.8.
LinearityLinearity can be determined by choosing parts or standards that cover all or
most of the operating range of the measurement instrument. Bias is determinedat each point in the range and a linear regression analysis is performed.
Linearity is defined as the slope times the process variance or the slope timesthe tolerance, whichever is greater. A scatter diagram should also be plottedfrom the data.
LINEARITY EXAMPLEThe following example is taken from Measurement Systems Analysis, pub-
lished by the Automotive Industry Action Group.
R&R studies for continuous data 341
Table 10.5. Components of variance analysis.
Source VarComp % of VarComp
Total gage R&R 0.228 0.28
Repeatability 0.228 0.28
Reproducibility 0 0
Operator 0 0
Part-to-Part 81.267 99.72
Total Variation 81.495 100
Administrator
Line
Administrator
Line
342 MEASUREMENT SYSTEMS ANALYSIS
Table 10.6. Analysis of spreads.
Source StdDev
StudyVar
(5.15*SD)% Study Var
(%SV)
%Tolerance(SV/Toler)
Total gage R&R 0.47749 2.4591 5.29 6.15
Repeatability 0.47749 2.4591 5.29 6.15
Reproducibility 0 0 0 0
Operator 0 0 0 0
Part-to-Part 9.0148 46.4262 99.86 116.07
Total Variation 9.02743 46.4913 100 116.23
Figure 10.8.Graphical analysis of components of variation.
