3RD Edition February 2003 OMNEX, Inc 3025 Boardwalk Suite 120

8/14/2019 3RD Edition February 2003 OMNEX, Inc 3025 Boardwalk Suite 120

1/38

MSA(Measurement System Analysis)

1

3RD EditionFebruary 2003

OMNEX, Inc3025 Boardwalk Suite 120

Ann Arbor, MI 48108USA

Tel: 734.761.4940Fax: 734.761.4966

e-mail: [email protected]

www.omnex.com

Omnex provides training and software to the international market with offices inthe USA, Brazil, Canada, India and T hailand. Omnex offers over 25 training

courses in business a nd quality management systems worldwide.

1998-2003

2

Intent of MSA Training

To provide participants with:

An understanding of the importance of MSA incontrolling and improving the process

An understanding that Measurement systems areprocesses for decision making

A practical knowledge of the use of statisticalmethods in analyzing measurement systems

Information needed to establish quantifiers,measurables and limits for measurement uncertaintyto make professional, informed estimates.

3

Chapter 1 Introduction to MSA

Chapter 2 MSA: ISO/9001:2000 & Sector-specific Requirements

Chapter 3 Statistical Properties of Measurement Systems3b Discrimination, Ca libration, Reference Standards3c Bias, Linearity, Stability3d GRR Studies

Chapter 4 Other Topics4a Advanced analysis (ANOVA)4b Attribute Systems,4c Automatic Systems4d Non-Replicable Systems

Chapter 5 Measurement Planning

Chapter 6 Application of MSA to Continual Improvement

Chapter 7 Analysis of Results

Chapter 8 Summary and definitions

Chapter 9 Appendix

MSA Course Map

Chapter 1

Introduction to MSAIntroduction to MSA

5

Measurement system is the collection of instrumentsor gages, standards, operations, methods, fixtures,software, personnel, environment and assumptionsused to quantify a unit of measure or fix assessmentto the feature characteristic being measured; the

complete process used to obtain measurement.

What is a Measurement System?
http://www.pdfcomplete.com/cms/hppl/tabid/108/Default.aspx?r=q8b3uige22


2/38

6

A measurement system is a process

What is a Measurement System?

OperationInput Output

Key Process Output VariableKPOV (deliverable) = a decision on a product,

process, or service

via a number assigned to a characteristic of aproduct, process, or service

The first step in assessing a system is to understandthis process and determine if it will satisfy ourrequirements

7

From this definition, it follows that a measurement process maybe viewed as a manufacturing process that produces decisions

via numbers (data) for its output.

Viewing a measurement system this way is useful because itallows us to bring to bear all the concepts, philosophy and toolsthat have already demonstrated their usefulness in the area ofstatistical process control.

What is a Measurement System?(contd)

8

Foundation of Analysis

Data is derived from objects, situations, or

phenomenon in the form of measurements.

Data is used to classify, describe, improve

or control objects, situations or phenomenon.

9

Measurement System Example

As a result of theactivity, we make adecision based on avalue that representsthe diameter

To measure the insidediameter of a tube, weuse a system thatincludes:

item of interest

personnel

method to use theequipment

environment wherethe measurement isperformed

10

1. Use experience, not data.

2. Collect data, but just look at the numbers.

3. Group the data so as to form charts and graphs.

4. Use census data with descriptive statistics.

5. Use sample data with descriptive statistics.

6. Use sample data with inferential statistics.

Levels of Analysis

11

Measurement Issues

Does the measurement system have adequatediscrimination?

Is the measurement system statistically stable overtime?

Is the measurement error (variation) small?


3/38

12

Measurement knowledge to be obtained

How big is the measurement error?

What are the sources of measurement error?

Is the measurement system stable over time?

Is the measurement system capable for this study?

How do we improve the measurement system?

13

What is MSA?

Measurement System Analysis (MSA)

MSA primarily deals with analyzing the effect of

the measurement system on the measured value

The objective of MSA is to assess the quality ofthe measurement system.

We test the system to determine its statisticalproperties, and use them in comparison with acceptedstandards, our needs and customer requirements.

14

Overall Objective of MSA

Understanding the Measurement System as a process

Uncertainty of Measurement

The range within which the true value of acharacteristic is estimated to lie.

System properties can be expressed as

statistical distribution of a series of measurements,

standard deviations,

probability,

percentages,

error as the difference between actual value and thereference value,

as points on a control chart or diagram,

etc.

15

Best-in-Class Approach

Determinations on these fundamental issues aremost meaningful if made relative toprocessvariation

Reporting measurement error as only a per cent oftolerance is generally inadequate for the worldwidemarket where emphasis is on continual processimprovement

Chapter 2

MSA & ISO 9001:2000& Sector-Specific Requirements

MSA & ISO 9001:2000

& Sector-Specific Requirements

17

QS 9000

Control of Inspection, Measuring and Test Equipment - 4.11

General - 4.11.1

all inspection, measuring and test equipment

including software or hardware shall have themeasurement uncertainty known and be withinacceptable guidelines


4/38

18

Control Procedure - 4.11.2

determine accuracy and precision (required)

product accepted previously by inspection,measurements, and test devices discovered tobe out of calibration must be re-inspected

environmental conditions during calibrationand testing must be evaluated to assess theireffect on the measurement system

ensure handling, preservation and storage

safeguard inspection of test hardware andsoftware from adjustments

QS 9000

19

Inspection, Measuring and Test Equipment - 4.11.3

Records shall extend to employee gages

gage conditions and actual readings must be recordedwhen gages are checked

notify customer if suspect material is shipped

conform to measurement system analysis manual ofcustomer approval methods

Caution:Most people interpret MSA as nothing more than GR&R.

This is a misconception far from the truth

QS 9000

20

Measurement System Analysis - 4.11.4

conduct statistical studies on each type ofinspection, measurement and test system as definein the customer approved control plan

suppliers should extend it from studying gage typeto at least each family of products

QS 9000

21

ISO/TS 16949:2002

7.6 Control of Monitoring and Measuring Devices

The organization shall establish processes to ensurethat monitoring and measurement can be carried outand are carried out in a manner that is consistent withthe monitoring and measurement requirements.

22

ISO/TS 16949:2002

7.6.1. MEASUREMENT SYSTEMS ANALYSIS

Statistical studies shall be conducted to analyze thevariation present in the results of each type of measuringand test equipment system. This requirements shallapply to measurement systems referenced in thecontrol plan. The analytical methods and acceptancecriteria used shall conform to those in customerreference manuals on measurement systems analysis.Other analytical methods and acceptance criteria maybe used if approved by the customer.

23

PPAP MSA Requirements

I.2.2.10 MEASUREMENT SYSTEM ANALYSIS

shall include bias, linearity, stability and GageR&R studies forALL new or modified gages, testand measuring equipment.


5/38

24

Customer/Industry MSA RequirementsTypically satisfied with GRR

Proposed Semi-conductor supplement to TS7.6.1S Measurement system analysis

In order to provide adequate measurement system discrimination, formeasurement equipment used to measure special characteristics,the apparent resolution of the equipment shall be at most one-tenthof the total process six sigma standard deviation. (ReferenceMeasurement System Analysis, Section 2.)

NOTE: Suppliers are expected to obtain value a dded state of the artmeasurement equipment. There are rare instances when a s pecialcharacteristic distribution is very narrow and the state of the artequipment cannot discriminate at the stated level The supplier shouldrepeat gage R&R studies when warranted by measurement systemchange (including operator) and have a systematic method toimprove gaging.

The supplier gage R&R studies shall be per Measurement SystemAnalysis reference manual, consistent with process Cp.

25

Common Sense MSA Requirements

What is in the Control Plan?

Does it cover Process and Product Parameters?Does it cover IMT from incoming to shipping?

Does it cover end of line and requirements fordownstream activities?

Does it cover lab equipment used for testing andanalysis?

Does it cover IMT for characterization and validationtesting?

26

Best-in-class: Control Of IMT Equipment

Minimize types and number of IMTs at each site

Buy IMT for part and process families

Conduct statistical studies per MSA guideline by part family

Only accept IMT per the 10-to-1 Rule and MSA guidelines

Do not allow personal IMT

Use six sigma process spread vs.specification/tolerancelimits to determine acceptance for product control

27

ImplementingGood Measurement Systems

Identify all inspection, measuring and test equipment (IMT)

Ascertain bias, linearity, and stability of IMT

show calibration status

records of calibration shall include employee gages

Conduct variation studies of IMT

Provide validity of previous results when IMT is found outof calibration

Ensure handling, preservation, cleaning, maintenance andstorage of all IMT

Use all criteria of MSA

Chapter 3

Statistical Properties ofMeasurement Systems

Statistical Properties of

Measurement Systems

29

APPLICATION OF MSA


6/38

30

Measurement is Not Always Exact

Measurement system variation affects individualmeasurements and decisions based on data

Measurement system errors are classified into fivecategories:

Bias

Stability

Linearity

Repeatability

Reproducibility

31


Bias

Difference between the observed average ofmeasurements and the reference value

A systematic error component of themeasurement system

32


Stability

The change in bias over time

A stable measurement

process is in statistical control

with respect to location

Alias: Drift

33


Linearity

The change in bias over the normal operating range

The correlation of multiple and independent bias errorsover the operating range

A systematic error component of the measurementsystem

34


Repeatability

Variation in measurements obtained with one measuringinstrument when used several times by an appraiser whilemeasuring the identical characteristic on the same part

The variation in successive (short term) trials under fixedand defined conditions of measurement

Commonly referred to asE.V. Equipment Variation

Instrument (gage) capabilityor potential

Within-system variation

35


ReproducibilityVariation in the average of the measurements made bydifferent appraisers using the same gage when measuringa characteristic on one part

For product and process qualification, error may beappraiser, environment (time), or method

Commonly referred to as A.V. Appraiser Variation

Between-system (conditions) variation

ASTM E456-96 includes repeatability, laboratory, andenvironmental effects as well as appraiser effects


7/38

36

Ideal Measurement Systems

Produce correct measurements each time used andeach measurement agrees with a master standard

Have statistical properties of:

zero variance

zero bias

And consequently

zero probability of mis-classifying any productmeasured

37

Measurement Systems Variation

Measurement process components and their interactionscontribute variation in outcome of data

MeasuredValues

Variation

EnvironmentEnvironment MethodMethod Equipment(Machine)

Equipment

(Machine)

MaterialMaterial PeoplePeopleStandardsStandards

38

How Materials & People Affect Data

Material:

Composition

Variation

People:

Training

Alertness

39

Measurement System

Temp Fluxctuation

Line Voltage Variati

Vibration

Cleanliness

Humidity

Algorithm Instabilit

Electrical Instabili

Wear

Mechanical Integrety

Operator Technique

Standard Procedure

Sufficient Work time

Maintenance Standard

Calibration Frequenc

Operator Training

Ease of use

Density

Conductivity

Hardness

Corrosion

Weight

Dimension

Temperature

Cleanliness

Wear

Stability

Resolution

Calibration

Precision

Design

Temperature

Cleanliness

Vision

DexterityKnowledge

Coordination

Speed

Interpretation

Calibration Error

AttentionFatigue

Procedure

Men

Machines

Materials

Methods

Measurements

Environment

Measurement System C&E Matrix

Sources of Measurement Variation

40

How Environment Affects Data

Temperature variation could cause expansion and/orcontraction, giving different reading for the samecharacteristic on the same unit

Poor lighting could hinder correct reading

Glare could cause false reading of dataTime affects material - ie. aluminum, plastic, glass

Humidity

Contamination - ie. electricity, dust

Vibration

41

How Measurement Equipment

Affects Data

The type of measurement equipment must beappropriate

The incremental division of the equipment must besmaller than that of the specification

The accuracy and precision of the equipment

The Bias and Linearity

The Repeatability and Reproducibility

Stability


8/38

42

How Measurement Equipment

Affects Data

Methods, procedures or work instructions must be

precisely followed by everyone performingmeasurements, testing or calibration or any part ofit, such as sample preparation or set up of thedevices.

If National, International, industry or professionally(e.g. SAE, ASTM, JEDEC) accepted methods arenot used, the organization must validate anydocumentation created or revised.

(continued)

43

Equipment Examples

Industry-specific types of inspection, measuring and testequipment

ball shear testwire pull test

profilometer

dial indicators

high power microscope

x ray thickness gage

What types of IMTequipment do you have?

44

Mathematical Perspective

Data collected for controlling a processcontains variations from two different andindependent sources:

Manufacturing Process Variation (MPV)

Measurement System Variation (MSV)

Total Variation (TV) = MPV + MSV

45

Sources of Variation

Product Variability

(Actual variability)

Product Variability

(Actual variability)

Measurement

Variability

Measurement

Variability

Total Variability

(Observed variability)

Total Variability

(Observed variability)

46

Measurement system variation must be smaller than themanufacturing process variation

MSV < MPV

Total Variation of

Manufacturing Process

+MSV MPV

Total Variation (TV)

Specification Tolerance

Note: Measurement system variation must be as small as possible

47

Effects of Measurement Error

AveragesAverages

VariabilityVariability

m m mtotal product measurement = +

s s stotal product measurement 2 2 2= +

Measurement

System Bias Determined through

Bias Study

Measurement System

Variability

Determined through

R&R Study


9/38

48

Measurement ErrorEffect on Capability Index

CUSL LSL

p

Act

Act =-

6s

s s s Act Obs MS = -2 2

CU SL L SL

pAct

Obs MS

=-

-6 2 2s s

We know that

Therefore:

where

pC

49

R&R Effect on Capability

0.0

1.0

2.0

3.0

4.0

5.0

6.0

0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0

Observed Cp

ActualC

p 0%

10%

20%

30%

40%

50%

60%

70%

%R&R

70% 60% 50%

40%

30%

10%

* As % of tolerance

Chapter 3b

Discrimination and UncertaintyDiscrimination and Uncertainty

51

APPLICATION OF MSA

52

Measurement Uncertainty

Measurement Uncertainty is the sum of all thevariations assigned to the variables that make upthe measurement system.

The total of those probabilities should be weighed

and carry importance in proportion to theseriousness and criticality of the measurementsbeing made.

53

Measurement Uncertainty

Decisions resulting from measurement system analysisinclude:

Use the system as is, taking into account its

uncertainty.Improve the system to control the variation in thecontributing factors.

Consider other measurement systems of higher levelsof discrimination and capability. (These will usually costmore but your MSA data will help to identify and justify adequateresources.)


10/38

54

Measurement Uncertainty and Calibration

Uncertainty-traceability chain

The first measurement uncertainty associatedwith a measurement system is generated by theprocess of calibration.

Calibration permits the estimate of errors ofindication of the measurement instrument,measuring system, or the assignment of values tomarks on arbitrary scales

55

Measurement Uncertainty and Calibration

The reference material itself, the calibrationprocess and the environment and personnel

performing the activity actually contribute to themeasurement uncertainty.

Thus the reason for accredited and/or qualifiedlabs and the usefulness of the data you shouldreceive regarding calibrations you do or pay tohave done for your measurement, inspection andtest equipment.

56

Discrimination

Understand measurement system capability toprovide information on process variability

Measurement system is unacceptable for analysis ifit cannot detect process variation

Measurement system is unacceptable for controllingprocess if it cannot detect special cause variation

57

Understanding Discrimination

Measuring a Coins Thickness

Which measurement systemprovides better information onthe variation of the three coins?

Discrimination: The ability of

the system to detect andindicate even small changes of

the measured characteristic;

also known as resolution.

58

Discrimination & the Control Chart

Example

Measurement of same samples by two system

Create X-bar and R charts (shown on following

page)

Observe contrast between measurement systemwith resolution of 0.001 and one with 0.01

59

Process Control Charts


11/38

60

Inadequate Discrimination

Inadequate discrimination of a measurement systemis shown on Range Charts when:

only one, two or three values for ranges canbe read

more than 1/4 of ranges are zero

To correct inadequate discrimination choose gages ofresolution proportionally smaller than the specificationor process variation

61

Measurement system variation must be smallcompared to the specification and process tolerance

(+ 3s)Measurement system s incremental markings(discrimination) must be smaller than those of thespecification

Specifications or Standard Deviations

Specification: 2.530 +/- 0.005

Measurement system increment: 0.0001

This is our rule of 10-to-1

62

Discrimination Decision Rules

Resolution of 1/10th of tolerance or process spread

Study gage discrimination during APQP and test foradequacy before PPAP

Evaluate range chart of manufacturing process orsimilar process, per previous page and examples

For continual improvement, 1/10th tolerance may beinadequate.MSA recommends 1/10th of six sigma (total)manufacturing standard deviation.

Chapter 3c

Bias, Linearity and StabilityBias, Linearity and Stability

64

APPLICATION OF MSA

65

Accuracy & Precision

AccuracyHow close is the measured value to the reference value

ASTM includes the effect of location and width errorsMSA evaluates bias instead of accuracy

Precision

Closeness of repeated readings to each otherMSA evaluates repeatability and reproducibility

Accuracy and Precision are not calculated as part of aMSA Analysis


12/38

66

Accuracy & Precision Example

A has best AccuracyB has best PrecisionC has better Accuracy than B

Compare the performance of A & CCompare the performance of A & C

Contestant A

Contestant B

Contestant C

Average for Contestant A

Average for Contestant B

Average for Contestant C

67

The Nature of Process Variation

Precise but not AccuratePrecise but not Accurate

Accurate but not PreciseAccurate but not Precise

. . . . . .Test equipment MUST be a least 10 times

more accurate & precise then whats being tested

Rule of thumb:

68

Work Around Gage Error

If you want to decrease your measurement error takeadvantage of the standard error square root of the sample:

Example: A measurement error of 50% can be cut in

halfif your point estimate is a sample of 4 data points.

Example: A measurement error of 50% can be cut in

halfif your point estimate is a sample of 4 data points.

THIS CAN BE USED AS A SHORT TERM APPROACH TO PERFORM

A STUDY, BUT YOU MUST FIX THE GAGE.

n = sample sizess

xx

n=

69

Terminology

True value:

Theoretically correct value unknown andunknowable

Reference standards

Traceable to NIST standards

Bias

Distance between average value of allmeasurements and reference value

Amount gage is consistently off target

Systematic error or offset

Reference Value

70

Reference Value

A value that serves as an agreed upon referencefor comparisonA reference value can be determined by averaging severalmeasurements with a higher level of measuring equipment

Sometimes known as:master valueaccepted valueconventional valueassigned valuebest estimate of the valuemaster measurementmeasurement standard

71

Reference Material

A material or substance with one or moreproperties which are sufficiently wellestablished to be used for the calibrationof an apparatus, assessment of a

measurement method, or for assigningvalues to materials


13/38

72

National / International

Measurement StandardsA material measure, measuring instrument, reference material or system

intended to define, realize, conserve or reproduce a unit, or one or more

values of a quantity, in order to transmit them to other measuring

instruments by comparison.These standards are recognized by an official national decision or by an

international agreement to serve i nternationally, as the basis for fixing

the value of all other standards of the quantity concerned.

Some Examples:

solution of cortisol in human serum as a standard of concentration

Iodi ne-Sta bi lize d H eli um 100 W standard resistor

Neon laser length Standard 1 Kg mass

Weston Standard cel l caesium atomic f requency standard

Standard gage b lock Josephson Ar ray Vo ltage Standard

73

National / International

Measurement Standards

Using a traceable standard provides:

common point for comparisonmeasurement system validity

measurement system accuracy estimates

conflict management between parties

most direct line of verification

74

Traceable Standard Limitations

Difficult to use in destructive testing

Some product characteristics and process resultshave no defined industry or national standards

Some tests have no defined industry or nationalstandards

Discuss limitations during design and development,contract review and APQP; managementresponsibility issue

75

Options

For calibration you may need to use GoldenUnits, other in house verification units and/ormutual consent standards. These are the bestproduct evaluated by the highest level ofmeasurement equipment.

Comparative analysis resulting in precise datathat will define amount of adjustment neededfor calibrating

Inter-laboratory Comparisons: organization,performance and evaluation of equipment onthe same or similar items by 2 or more labs inaccordance with predetermined conditions.

76

BIAS Is the difference between theobserved average of the measurement and thereference value.

The reference value can be determined byaveraging several measurements with a higherlevel (e.g., metrology lab) of measuring

equipment.

Warning: Dont assume your metrologyreference is gospel.

Observed

Average Value

Reference

Value

BIAS is often referred to as ACCURACY

BIAS Definition

77

Why Do a Gage Bias Study?

Because it is required

To determine if the bias is acceptablei.e. statistically zero


14/38

78

Bias

Test of hypothesis approach.

Expanded the analysis to include control chartmethod.

79

Analysis of Bias

Average

Confidence Band

ReferenceValue

80

Calculating Bias

1

n

i

i

x

Xn

==

where is taken from Appendix C

withg= 1 and m = n

( ) ( )*2

max mini ix x

drepeatabilitys

-=

*

2d

bias = observed average measurement reference value

rb

n= ss

b

biast=

s

where are found in MSA Appendix C

withg= 1 and m = n andis found using the standard ttables.

( ) ( )b b, 1 , 12 2

Bias t zero Bias t a an n- - - +

*

2 2, andd d n

,12

t an -

81

TheThe aa level which is used depends on the level oflevel which is used depends on the level ofsensitivity which is needed to evaluate/controlsensitivity which is needed to evaluate/control

the process and is associated with the lossthe process and is associated with the loss

function (sensitivity curve) of thefunction (sensitivity curve) of the

product/process. Customer agreement should beproduct/process. Customer agreement should be

obtained if anobtained if an aa level other than the default valuelevel other than the default valueof .05 (95% confidence) is used.of .05 (95% confidence) is used.

82

Gage Bias Work Instruction

1. Obtain accepted reference value using a master ormeasuring equipment of higher level, such as layoutequipment to create a golden unit or master partto use as a reference value.

2. Measure same part by same appraiser minimum of10 times using gage under evaluation

83

Gage Bias Work Instruction

3. Analysis:plot the histogram of all the readings

Determine if any special causes are present

calculateaverage of all the readingsbias = average of all the reading - reference valuesd(bias) = standard deviation of all the readings dividedby the square root of the sample sizeThe t-statistic and/or confidence bounds.

determine if the bias is statistically zero; i.e.The t-statistic is less than the critical valueZero is contained within the confidence bounds


15/38

84

Bias Calculation- Example

Reference

Value = 6.00Bias

1 5.8 -0.2

2 5.7 -0.33 5.9 -0.1

T 4 5.9 -0.1

R 5 6.0 0.0

I 6 6.1 0.1

A 7 6.0 0.0

L 8 6.1 0.1

S 9 6.4 0.4

10 6.3 0.3

11 6.0 0.0

12 6.1 0.1

13 6.2 0.2

14 5.6 -0.4

15 6.0 0.0

85

Bias Calculation- Example

5.6 5.7 5.8 5.9 6.0 6.1 6.2 6.3 6.4

0

1

2

3

4

F r e q u e n c y

Measured value

86

Bias Calculation- Example Range Method

Reference

Value = 6.00Bias

1 5.8 -0.2

2 5.7 -0.3

3 5.9 -0.1

T 4 5.9 -0.1

R 5 6.0 0.0

I 6 6.1 0.1

A 7 6.0 0.0

L 8 6.1 0.1

S 9 6.4 0.4

10 6.3 0.3

11 6.0 0.0

12 6.1 0.1

13 6.2 0.2

14 5.6 -0.4

15 6.0 0.0

1 90.16.0067

15

n

i

i

x

Xn

== = =

n = 15

( ) ( )*2

max min

6.4 5 .6.2254

3.55

i ix x

drepeatabilitys

-=

-= =

.2254.0582

15r

bn

ss = = =

.0067.1151

.0582b

biast

s= = =

bias = observed average measurement

reference value = 6.0067 6.00 = .006787

Bias Calculation- Example Range Method

Given: m = n = 15 g=1

d2* = 3.55 d2 = 3.47 df = 10.8

t.975, 10.8 = 2.2060

.1151t=.0582bs =

( ) ( )b b, 1 , 12 2


( ) ( )3.47*.05813 3.47*.05813

.0067 2.2060 0 .0067 2.20603.55 3.55

- +

.1186 0 .1320-

88

Bias - Control Chart Method

bias = reference valueX

*2

repeatability

R

ds =

where is based on the subgroup size

(m) and the number of subgroups in thechart (g).

*

2d

rb

gm

ss =

b

bias

t = s

( ) ( )b b, 1 , 12 2


Where are found in

MSA Appendix C and is

found using the standard ttables.

*2 2, andd d n

,12

t an -

89

Bias- Control Chart Example

0Subgroup 10 20

5.7

5.8

5.9

6.0

6.1

6.2

6.3

SampleMean

6.021

UCL=6.297

LCL=5.746

0.0

0.5

1.0

SampleRange

0.4779

UCL=1.010

LCL=0

Xbar/R Chart for Stability


16/38

90

BIAS EXAMPLE

OD VALUES

TRIALS (inches)

1 0.726602 0.72440

3 0.72535

4 0.726305 0.72710

6 0.72745

7 0.72630

8 0.725159 0.72525

10 0.72570

Average (X-bar) = 0.72596

Bias = Observed Average - Reference Value= 0.72596 - 0.72650 = - 0.00054

The observed measurements on the averagewill be 0.00054 smaller than the reference

value

The outside diameter of a shaft is measured ten times by the sameoperator. The data is given below.

The process variation is estimated as 0.00310.

The reference value is 0.72650.

91

BIAS EXAMPLE

Average = 0.72596

Bias = - 0.00054

Overall StdDev = 0.000954

StdDev Bias = 0.000954/sqrt(10) = 0.000302

Critical tvalue = 2.26216 for df = 9 (from ttable)

Calculated t = abs(-0.00054)/0.000302 = 1.79

Confidence Bounds = (-0.001223, 0.000143)

Analysis

Since Calculated tis less than theCritica l tvalue

And zero is contained within the Confidence Bounds

Accept the hypothesis that the bias is zero

The bias is statistically zero

92

BREAKOUT 1

Bias

93

Cause of Unacceptable Bias

Error in measuring instrument

Instrument:

worn, dirty, broken

improper design

made to wrong dimension

worn, dirty, broken

improper discrimination

improper calibration

improper use by operator

94

2 3 4 5 6 7 98 10

LINEARITY EXAMPLELINEARITY EXAMPLE Y = 0.736667 - 0.131667X R-Sq = 71.4

reference values

Linearity StudyLinearity Study -- Graphical AnalysisGraphical Analysis

bias

bias

1

0

-1

bias = 0bias = 0

Regression95% ClBias Average

Linearity

Difference in the bias values of a measurementsystem through its expected operating range

95

Linearity

Test of hypothesis approach.

Added examples and more explanation oninterpretation of results.


17/38

96

LinearityDifference in the bias values of a measurementsystem through its expected operating range

2 3 4 5 6 7 8 9 10

0.0

0.5

1.0

Master

-bias-

-bias-= -0.0000433+0.0000083 Master

S =0.232676 R-Sq= 0.0% R-Sq(adj) = 0.0%

Regression

95% CI

Good LinearityUnacceptable LinearityAcceptable Linearity

97

Gage Linearity

Gage linearity can be determined by conducting biasstudies through expected operating range

Minimum of two bias studies should be conducted,one at each end of operating range

Middle of range should also be considered

98

Gage Linearity Study

Gage Linearity Work Instruction

1. Select five to eight parts that can be measured atdifferent operating ranges of measurement system

2. Determine reference value for each part usinglayout inspection or use a set of standards

3. Use one appraiser and the same instrument tomeasure parts

4. Take 5 or more repeated measurements on eachpart

5. Calculate each part s bias

bias = observed average - reference value

99

Gage Linearity Study(continued)

6. Create a scatter plot by putting the reference valuesfrom smallest to largest on the x-axis, and thecorresponding bias values on the y-axis.

7. Calculate the 95% confidence intervals for the controllimits.

Note: you will find it helpful to graph all of the actual

groups of readings stacked vertically at the related

reference value points

100


8. To analyze the graph:

a. The bias = 0 line must fall entirely within the CI foracceptable linearity

b. Watch for individual readings that do not follow

the pattern of the groups (e.g. outliers). Also watchfor other patterns indicating unusual variation orabnormal behavior.

9. Remember that a measurement system with large(i.e. unacceptable) repeatability can indicate anstatistically acceptable bias even if it practicallyunacceptable.

101


10. If measurement system has a linearity problem,use problem-solving methods to determinemodifications necessary to achieve zero linearity


18/38

102

102 3 4 5 6 7 98

PARTREFERENCE

VALUE

1

2.00

2

4.00

3

6.00

4

8.00

5

10.00

1 2 .70 5.10 5.8 0 7.60 9.10

2 2 .50 3.90 5.7 0 7.70 9.30

3 2 .40 4.20 5.9 0 7.80 9.504 2 .50 5.00 5.9 0 7.70 9.30

5 2 .70 3.80 6.0 0 7.80 9.40

6 2 .30 3.90 6.1 0 7.80 9.50

7 2 .50 3.90 6.0 0 7.80 9.50

8 2 .50 3.90 6.1 0 7.70 9.50

9 2 .40 3.90 6.4 0 7.80 9.60

10 2 .40 4.00 6.3 0 7.50 9.20

11 2 .60 4.10 6.0 0 7.60 9.30

12 2 .40 3.80 6.1 0 7.70 9.40

LINEARITY STUDY DATALINEARITY STUDY DATA

TT

RR

IIAA

LL

SS

PARTREFERENCE

VALUE

1

2.00

24.00

36.00

48.00

510.00

1 0.7 1.1 -0.2 -0.4 -0.9

2 0.5 -0.1 -0.3 -0.3 -0.7

3 0.4 0.2 -0.1 -0.2 -0.5

4 0.5 1 -0.1 -0.3 -0.7

5 0.7 -0.2 0.0 -0.2 -0.6

6 0.3 -0.1 0.1 -0.2 -0.5

7 0.5 -0.1 0.0 -0.2 -0.5

8 0.5 -0.1 0.1 -0.3 -0.5

9 0.4 -0.1 0.4 -0.2 -0.4

10 0.4 0.0 0.3 -0.5 -0.8

11 0.6 0.1 0.0 -0.4 -0.7

12 0.4 -0.2 0.1 -0.3 -0.6

BIAS AVG 0 .4 91 66 7 0 .1 25 0 .0 25 - 0. 29 16 7 - 0.6 16 67

BB

II

AA

SS

Charting Linearity

LINEARITY EXAMPLELINEARITY EXAMPLE

1

-1

0

Y = 0.736667 -0.131667X R-Sq = 71.4

Linearity StudyLinearity Study -- Graphical AnalysisGraphical Analysis

bias

=0

bias

=0

reference values

95%Cl

Regression

Bias Average

103

Breakout 2Linearity

104

Causes of Non-linearity

Improper calibration of the gage at the lowerand upper end of operating range

Error in the master used at the minimum andmaximum range

Instrument wear

Design characteristics of the instrument

105

Stability

A Measurement System isStable if there are nospecial cause variationaffecting the measurementsystem s bias over time

106

Stability

Stability (or drift) in ameasurement systemevaluated bymeasuring the samemaster(s) or part(s) on

a single characteristicover an extended timeperiod(a time periodis days, not hours)

107

Measurement System Stability

Typically not as large a problem as GRR

Useful to help determine calibration intervals

Should track from test to test and chart (or at leastrecord actual readings and other pertinent data inthe gage record)


19/38

108

Affects On Gage Stability

Time long idle periods or intermittent use

Very large or very small number of measurements

taken between stability testsEnvironment or system changes, such as: humidity;air pressure

potential confusion with statistical stabilityfactors, such as warm up effects, rate of wear,lack of maintenance, untrained operators orlab technicians

109

Cause of Gage Stability Error

Infrequent or too frequent calibration

Lack of air pressure regulator or filter

Warm-up period for electronic or other gages

Lack of maintenance

Wear or damage not readily observable

Oxidization (corrosion)

110

Gage Stability Study

Stability Analysis Instructions

1. Use a standard set of parts or reference/mastermaterials as sample

Retain these as appropriate (life of product) in aprotected environment

Label them with name and number for trackingand further studies include samples at the low,mid and high range.

111

Gage Stability Study (continued)

2. (Recommended) Perform a Bias or Linearity study onthe sample. Use this information to establish the controlchart parameters

3. Measure part(s) three to five times (based onknowledge of measurement system) at different timesof the day.Plot data on X-bar and R or X-bar and S chart

3. Evaluate per normal SPC requirements

Note: Maintain a chart for each part/master

4. Evaluate the within sample standard deviation(repeatability) of measurements to determine suitabilityfor the application

112

Analyzing Stability Charts

0Subgroup 10 205.7

5.8

5.9

6.0

6.1

6.2

6.3

SampleMean

6.021

UCL=6.297

LCL=5.746

0.0

0.5

1.0

SampleRange

0.4779

UCL=1.010

LCL=0

Xbar/R Chart for Stability

113

Analyzing Stability Charts

If stability is not acceptable, the X-bar and possibly S

or R charts will show a shift or out-of-control condition

out-of-control chart conditions indicatemeasurement system not measuring consistently

Check for:bias changed - determine cause of change andcorrect

if cause is wear - may be fixed by re-calibration


20/38

Chapter 3d

GRR StudiesGRR Studies

115

APPLICATION OF MSA

116

GRR

Intent

provide an understanding of GRR using theMSA 3rd edition

Note that:

repeatability and reproducibility deal with thewidth or spread of the measurement systemvariation

bias, stability and linearity deal w ith the locationof the measurement system variation

117

Repeatability

Variation in measurements obtained with onemeasurementinstrumentwhen used several times byone appraiserwhile measuring the same characteristicon the same part

118

Repeatability

The inherent variability of the measurementsystem

Variation in measurements obtained with agage when used several times by one

appraiser while measuring a characteristic onone part.

Estimated by the pooled standard deviation ofthe distribution of repeated measurements

Repeatability is less than the total variation ofthe measurement system

sG

s GR

d=

2

*

119

Repeatability Example

Contestant A

Contestant B

Contestant CAverage for Contestant A




21/38

120

Reproducibility

Variation in average of measurements made bydifferentappraisers using same measuringinstrumentwhen measuring the same characteristic onthe same parts

121

Reproducibility

Appraiser variability of the measurement

systemVariation in the average of the measurements

made by different appraisers using the same

measurement system when measuring the

same characteristic on one part

Must be adjusted for repeatability

Reproducibility is less than the total variation

of the measurement system

As

122

Reproducibility Example

Contestant A

Contestant B

Contestant C

Average for Contestant A



to Reproducibility between A & B

to Reproducibility between A & C

to Reproducibi lity between B & C

123

Possible Sources of Process Variation

Long-term

Process Variation

Short-term

Process Variation

Variation

w/i sample

Actual Process Variation

Stabi lity LinearityRepeatability Bias

Variation due

to instrument

Variation due

to appraisers

Measurement Variation

Observed Process Variation

We look at repeatability and reproducibility as these

are the usually primary contributors to measurement error.

We look at repeatability and reproducibility as these

are the usually primary contributors to measurement error.

Reproducibility

124

Preparation for a Measurement Study

Determine if reproducibility is an unknown or a concern. If it is,select the number of appraisers to participate.

Appraisers selected should normally use the measurement system.

Select samples that represent the entire operating range.

Gage must have graduations that allow at least one-tenth of theexpected process variation.

Insure defined gaging procedures are followed.

Measurements should be made in random order.

Study must be observed by someone who recognizes theimportance of conducting a reliable study.

Determine if reproducibility is an unknown or a concern. If it is,select the number of appraisers to participate.

Appraisers selected should normally use the measurement system.

Select samples that represent the entire operating range.

Gage must have graduations that allow at least one-tenth of theexpected process variation.

Insure defined gaging procedures are followed.

Measurements should be made in random order.

Study must be observed by someone who recognizes theimportance of conducting a reliable study.

125

GRR Study

GRR Study Instructions

1. Select two or three appraisers who are users of themeasurement system

2. Obtain a sample of n (10 or more) parts that

represent actual or expected range of processvariation

3. Number parts 1 through n so that numbers are notvisible to appraisers

4. Calibrate gage if this is part of the normal gagingprocedures


22/38

126

GRR Study (continued)

5. Measure the n parts in random order by appraiserA, with an observer recording results

6. Repeat step 5 with other appraisers conceal otherappraisers readings

7. Repeat step 5 and 6 using a different random orderof measurement

8. Using the Average and Range approach, enter thedata onto the form and follow the form instructions

calculate the average and ranges for all readingsfor each appraiser

127

Plot the average and Range charts and analyze forstability


128

GRR

Repeatability

variation in measurementsobtained with onemeasurement instrument

when used several timesby one appraiserwhilemeasuring the samecharacteristic on samepart

graph on Rchart

Reproducibility

variation in average ofmeasurements made bydifferentappraisers usingsame measuringinstrumentwhenmeasuring the samecharacteristic on samepart

graph on average chart

129

Range Chart Example

130

Range Chart Conclusion

There appears to be differences among assessors

A reading of one appraiser is outside the controllimits; the conclusion is that the appraiser s methoddiffers from other appraisers

In general if all appraisers have some points outsidecontrol limit, then conclusion is measurement systemsensitive to appraiser technique and needsimprovement to obtain useful data

131

Average Chart Example


23/38

132

Average Chart Conclusion

Used to determine

Consistency between appraisers

Adequacy to detect part variation

Adequacy of resolution

Adequacy of sample

In this study, 22 out of 30 points are outside thecontrol limit

Since this is more than half of the points, theconclusion is that the measurement system isadequate to detect part-to-part variations

133

Misc:

Tolerance:

Reported by:

Dateof study:

Gage name:

1.11.00.90.8

0.70.6

0.50.40.3

321

Xbar Chart by Operator

SampleM

ean

X=0.80753.0SL=0.8796

-3.0SL=0.7354

0.15

0.10

0.05

0.00

321

R Chart by Operator

SampleRange

R=0.03833

3.0SL=0.1252

-3.0SL=0.000

10987654321

1.1

1.0

0.9

0.8

0.7

0.6

0.5

0.4

Gasket

OperatorOperator*Gasket Interaction

Avera

ge

1

2

3

321

1.1

1.0

0.9

0.8

0.7

0.6

0.5

0.4

Operator

Response by Operator

10987654321

1.1

1.0

0.9

0.8

0.7

0.6

0.5

0.4

Gasket

Response by Gasket

%TotalVar

%Study Var

Part-to-PartReprodRepeatGageR&R

100

50

0

Components of Variation

Percent

Gage R&R (Xbar/R) for Thickness

Example of MSA Graphics

134

Breakout 3

Graphing GRR

135

Calculate the overall average and range, part range,and maximum difference between appraiser averages

Calculate repeatability for equipment variation:

Calculate reproducibility for appraiser variation

Calculate GRR

Calculate part variation

Calculate total variation

Calculate the percent indices for above

Calculate the ndc

Analyze the numerical results


136

Breakout 4

Calculating GRR

137

Gasket Thickness Study

PT1 PT2 PT3 PT4 PT5 PT6 PT7 PT8 PT9 PT10 AP/TRIAL

0.65 1.00 0.85 0.85 0.55 1.00 0.95 0.85 1.00 0.60 A1

0.60 1.00 0.80 0.95 0.45 1.00 0.95 0.80 1.00 0.70 A2

0.55 1.05 0.80 0.80 0.40 1.00 0.95 0.75 1.00 0.55 B1

0.55 0.95 0.75 0.75 0.40 1.05 0.90 0.70 0.95 0.50 B2

0.50 1.05 0.80 0.80 0.45 1.00 0.95 0.80 1.05 0.85 C1

0.55 1.00 0.80 0.80 0.50 1.05 0.95 0.80 1.05 0.80 C2

Xbar & R Example

Specification: 0.6 - 1.0 mm

Process Variation: 1.6 mm

Reference: Measurement System Analysis Manual, 2nd Edition


24/38

138

Gage R&R Study for Thickness XBar/R Method

Source Variance StdDev 5.15*Sigma

Total Gage R&R 2.08E-03 0.045650 0.235099

Repeatability 1.15E-03 0.033983 0.175015

Reproducibility 9.29E-04 0.030481 0.156975Part-to-Part 3.08E-02 0.175577 0.904219

Total Variation 3.29E-02 0.181414 0.934282

Source %Contributio n %Study Var

Total Gage R&R 6.332 25.164

Repeatability 3.509 18.733Reproducibility 2.823 16.802

Part-to-Part 93.668 96.782

Total Variation 100.000 100.000

Number of distinct categories = 5

139

Calculation Explanation

5.15 Sigma = 5.15 the factor standard deviation.5.15 was developed empirically to approximate the gage populationdistribution variation. (99% area for Normal forms)

% Contribution = Percent contribution of each factor based uponthe variance.Repeatability = 100 repeatability variance/ total variationvariance.

% Study Variation = the factor standard deviation divided by thetotal variation s tandard deviation.Repeatability = 100 repeatability standard deviation/ totalvariation standard deviation.

5.15 Sigma = 5.15 the factor standard deviation.5.15 was developed empirically to approximate the gage populationdistribution variation. (99% area for Normal forms)

% Contribution = Percent contribution of each factor based uponthe variance.Repeatability = 100 repeatability variance/ total variationvariance.

% Study Variation = the factor standard deviation divided by thetotal variation s tandard deviation.Repeatability = 100 repeatability standard deviation/ totalvariation standard deviation.

140

Calculation Explanation

% Tolerance = the factor standard deviation divided by thetolerance/6.Repeatability = 100 repeatability standard deviation/(tolerance/6)

% Process Variation = the factor standard deviation divided bythe process variation.

Repeatability = 100 x repeatability standard deviation/ processvariation.

Number of Distinct Categories = part standard deviation dividedby the total measurement standard deviation times 1.41.

% Tolerance = the factor standard deviation divided by thetolerance/6.Repeatability = 100 repeatability standard deviation/(tolerance/6)

% Process Variation = the factor standard deviation divided bythe process variation.

Repeatability = 100 x repeatability standard deviation/ processvariation.

Number of Distinct Categories = part standard deviation dividedby the total measurement standard deviation times 1.41.

141

GRR Acceptance Guidelines

Acceptance criteria based on estimated value of R&R(%R&R) of system

if % GRR < 10%, system is acceptable

if 10% < % GRR < 30%, system may beacceptable based on application, cost of gage, costof repair, etc.

if % GRR > 30%, system needs improvement orcorrective action

Note: THE SUM OF THE PERCENT CONSUMED BYEACH FACTOR WILL NOT EQUAL 100%.

142

Analysis with MSA Pro

Show percentages

Show charts

Draw conclusions

143

Application of GRR

When repeatability is large compared to reproducibility:

instrument needs maintenance

redesign gage for more rigidity

improve clamping or location of gauging

excessive within-part variation


25/38

144

Application of GRR (continued)

When reproducibility is large compared to repeatability:

Appraisers need better gage use training

Need better operational definition

Incremental divisions on instrument are not readable

Need fixture to provide consistency in gage use

Chapter 4a

Advanced Analysis - ANOVAAdvanced Analysis - ANOVA

146

Advanced Analysis

Intent

provide an understanding of how to evaluatemeasurement systems using Analysis of variance (ANOVA)

Scope

ANOVA is a standard statistical technique and can be usedto analyze the measurement error and other sources ofvariability of data in a measurement systems study.

the variance can be decomposed into four categories:

parts,

appraisers,

interaction between parts and appraisers, and

replication error due to the gage.

147

When To Use ANOVA

Product cannot be usedagain, such as destructivetests

pull test

tensile strength test

chemical compositiontest

Product changed byevaluation equipment

Requires knowledge ofvariation contributed by:

equipment

appraisers

units measured

their interaction

other sources of variation

148

ANOVA table

The ANOVA table here is composed of fivecolumns

Source column is the cause of variation.

DFcolumn is the degree of freedom

associated with the source.SSor sum of squares column is the deviationaround the mean of the source.

MSor mean square column is the sum ofsquares divided by degrees of freedom.

F-ratio column, calculated to determine thestatistical significance of the source value.

149

ANOVA table

Source DF SS MS F

Appraiser 2 3.1673 1.58363 34.44*

Parts 9 88.3619 9.81799 213.52*

Appraiser

by Part

18 0.3590 0.01994 0.434

Equipment 60 2.7589 0.04598

Total 89 94.6471

* Significant at = 0.05 level


26/38

150

Estimate of

Variance

Std.

Dev. (s)% Total

Variation

%

Contribution

2t = 0.039973(Repeatability)

EV= 0.199933 18.4 3.4

2w = 0.051455

(Appraiser)

AV= 0.226838 20.9 4.4

2g = 0

(Interaction)

INT= 0 0 0

System = 0.09143

( 2 2 2t g w+ + )GRR = 0.302373 27.9 7.8

2s = 1.086446

(Part)

PV= 1.042327 96.0 92.2

Total Variation TV=1.085 100.0

Note: % Total Variation for Repeatability = (EV/TV) x 100

% Contribution for Repeatability = (EV/TV)2 x 100

Chapter 4b

Attribute MeasurementsAttribute Measurements

152

lVariable Gage R&R

Numbers

Units of measure

lAttribute Gage R&R

Subjective (visual defects)

Scatter of defects

feel/visual

Types of R&R Studies

153

Attribute Measurement

An attribute gage:

compares each part to a specific set of limits andaccepts the part if the limits are satisfied

is designed to accept/reject a set of master parts

cannot indicate how good or how bad a part is, onlywhether the part is accepted or rejected (pass/fail)

154

Analyzing Attribute Data

Always Try To Convert Attribute To VariablesAlways Try To Convert Attribute To Variables

Examples:l End Disk Height

l Likert Scale

l Leak Rate (go/no go)

l Mass Spec

Then use Variable Data Analysis TechniquesThen use Variable Data Analysis Techniques

155

The Inspection Exercise

The Necessity of Training Farm Hands for First

Class Farms in the Fatherly Handling of Farm Live

Stock is Foremost in the Eyes of Farm Owners.

Since the Forefathers of the Farm Owners Trainedthe Farm Hands for First Class Farms in the

Fatherly Handling of Farm Live Stock, the Farm

Owners Feel they should carry on with the Family

Tradition of Training Farm Hands of First Class

Farmers in the Fatherly Handling of Farm Live

Stock Because they Believe it is the Basis of Good

Fundamental Farm Management.

Task: Count the number of times the 6th letter of the alphabet

appears in the following text. One minute time limit


27/38

156

The Inspection Exercise

The Necessity ofTrainingFarm HandsforFirst

Class Farms in the Fatherly Handling of Farm Live

Stock is Foremost in the Eyes of Farm Owners.

Since the Forefathers of the Farm Owners Trained

the Farm HandsforFirst Class Farms in the

Fatherly Handling ofFarm Live Stock, the Farm

Owners Feel they should carry on with the Family

Tradition ofTrainingFarm Hands of First Class

Farmers in the Fatherly Handling of Farm Live

Stock Because they Believe it is the Basis of Good

FundamentalFarm Management.

Task: Count the number of times the 6th letter of the alphabet

appears in the following text. One minute time limit

157

Operational Definitions

An operational definition is one that people can dobusiness with. An operational definition of safe, round,reliable, or any other quality [characteristic] must be

communicable, with the same meaning to vendor as to thepurchaser, same meaning yesterday and today to theproduction worker. Example:

1. A specific test of a piece of material or an assembly

2. A criterion (or criteria) for judgment

3. Decision: yes or no, the object or the material did ordid not meet the criterion (or criteria)

W. E. Deming, Out of the Crisis (1982, 1986), p. 277.

158

Attribute MSA Study 3rd Edition

2nd Edition Short and Long Term Methods

Short Term Method: no longer referenced

Long Term Method: Name changed (to Analytic)

Approaches:

Risk Analysis Method

Contingency table analysis/Multiple Tests of Hypothesis

Signal detection theory

Analytic Method

159

RISK ANALYSIS METHODS

In some attribute situations it is not feasible to get sufficientparts with variable reference values. In such cases, the risksof making wrong or inconsistent decisions can be evaluatedby using

Hypothesis Test Analyses

Signal Detection Theory

These methods do notquantify the measurement systemvariability, they should be used only with the consent of thecustomer.

Selection and use of such techniques should be based ongood statistical practices, an understanding of the potentialsources of variation which can affect the product andmeasurement processes, and the effect of an incorrectdecision on the remaining processes and the final customer

The sources of variation of attribute systems should be

minimized by using the results of human factors andergonomic research.

160

Attribute MSA Study

Used when measurements are subjective classifications or ratings by people.

Attribute data can be ORDINAL orNOMINAL.

Ordinal Data are categorical variables that have three or more possible levelswith a natural ordering, such as strongly disagree, disagree, neutral, agree, and

strongly agree. Or, you can use a numeric scale such as 1-5.

Kendalls Coefficients are used.

----- to measure nonparametric correlations.

Nominal Data are categorical variables that have two or more possible levels

with no natural ordering.

Pass, Fail

Crunchy, Mushy, and Crispy(Food tasting study).

161

Nominal Data

Kappa Coefficients are used.

Kappa is an indicator of inter-rater agreement; i.e., the ratioof the proportion of agreement (corrected for chance)divided by the maximum number of times they could agree(corrected for chance).

Where Pr (agreement ) = Proportion(Agreement)Pr (error) = Proportion (Error)

A Kappa coefficient of 90% may be interpreted as 90%agreement.

Pr( ) Pr( )1 Pr( )

agreement errorKerror

-=-


28/38

162

Kappa CoefficientsKappa CoefficientsBased onBased on an article by Altman, 1991 recommends the followingan article by Altman, 1991 recommends the following

kappa interpretation scale. The definition of the interpretationkappa interpretation scale. The definition of the interpretations,s,

such assuch as PoorPoor, must be agreed upon by the customer and the, must be agreed upon by the customer and the

supplier.supplier.

Kappa ValueKappa Value InterpretationInterpretation

< 0.20< 0.20 PoorPoor

0.210.21--0.400.40 FairFair

0.410.41--0.600.60 ModerateModerate

0.610.61--0.800.80 GoodGood

0.810.81--1.001.00 Very GoodVery Good

In addition, Gardner (1995) recommends that kappa exceed .70In addition, Gardner (1995) recommends that kappa exceed .70

before you proceed with additional data analyses.before you proceed with additional data analyses.

163

Hypothesis Test Analyses

A * B Crosstabulation

44 6 50

15.7 34.3 50.0

3 97 100

31.3 68.7 100.0

47 103 150

47.0 103.0 150.0

Count

Expected Count

Count

Expected Count

Count

Expected Count

.00

1.00

A

Total

.00 1.00

B

Total

164

Count & Expected Count CalculationsCount & Expected Count Calculations


44 6 50

15.7 34.3 50.0

3 97 100

31.3 68.7 100.0

47 103 150

47.0 103.0 150.0

Count

Expected Count

Count

Expected Count

Count

Expected Count

.00

1.00

A

Total

.00 1.00

B

Total

165

There are 34 times where

A-1 = 1 and B-1 = 1(that is, of the 50 parts checked

there were 34 matches by A and Bon their FIRST check)

Table 12: Attribute Study Data SetTable 12: Attribute Study Data Set

MSA 3MSA 3rdrd, pg 127, pg 127

Part A- 1 A - 2 A -3 B- 1 B - 2 B - 3 C - 1 C -2 C- 3 Reference

1 1 1 1 1 1 1 1 1 1 1

2 1 1 1 1 1 1 1 1 1 1

3 0 0 0 0 0 0 0 0 0 0

4 0 0 0 0 0 0 0 0 0 0

5 0 0 0 0 0 0 0 0 0 0

6 1 1 0 1 1 0 1 0 0 1

7 1 1 1 1 1 1 1 0 1 1

8 1 1 1 1 1 1 1 1 1 1

9 0 0 0 0 0 0 0 0 0 0

10 1 1 1 1 1 1 1 1 1 1

11 1 1 1 1 1 1 1 1 1 1

12 0 0 0 0 0 0 0 1 0 0

13 1 1 1 1 1 1 1 1 1 1

14 1 1 0 1 1 1 1 0 0 1

15 1 1 1 1 1 1 1 1 1 1

16 1 1 1 1 1 1 1 1 1 1

17 1 1 1 1 1 1 1 1 1 1

18 1 1 1 1 1 1 1 1 1 1

19 1 1 1 1 1 1 1 1 1 1

20 1 1 1 1 1 1 1 1 1 1

21 1 1 0 1 0 1 0 1 0 1

22 0 0 1 0 1 0 1 1 0 0

23 1 1 1 1 1 1 1 1 1 1

24 1 1 1 1 1 1 1 1 1 1

25 0 0 0 0 0 0 0 0 0 0

26 0 1 0 0 0 0 0 0 1 0

27 1 1 1 1 1 1 1 1 1 1

28 1 1 1 1 1 1 1 1 1 1

29 1 1 1 1 1 1 1 1 1 1

30 0 0 0 0 0 1 0 0 0 0

31 1 1 1 1 1 1 1 1 1 1

32 1 1 1 1 1 1 1 1 1 1

33 1 1 1 1 1 1 1 1 1 1

34 0 0 1 0 0 1 0 1 1 0

35 1 1 1 1 1 1 1 1 1 1

36 1 1 0 1 1 1 1 0 1 1

37 0 0 0 0 0 0 0 0 0 0

38 1 1 1 1 1 1 1 1 1 1

39 0 0 0 0 0 0 0 0 0 0

40 1 1 1 1 1 1 1 1 1 1

41 1 1 1 1 1 1 1 1 1 1

42 0 0 0 0 0 0 0 0 0 0

43 1 0 1 1 1 1 1 1 0 1

44 1 1 1 1 1 1 1 1 1 1

45 0 0 0 0 0 0 0 0 0 0

46 1 1 1 1 1 1 1 1 1 1

47 1 1 1 1 1 1 1 1 1 1

48 0 0 0 0 0 0 0 0 0 0

49 1 1 1 1 1 1 1 1 1 1

50 0 0 0 0 0 0 0 0 0 0

ATTRIBUTE GAGECALCULATIONS FOR

COUNTS

166


COUNTS

There are 32 times whereA-2 = 1 and B-2 = 1

(that is of the 50 parts checked

there were 32 matches by A and Bon their SECOND check)

Part A -1 A -2 A - 3 B - 1 B- 2 B-3 C - 1 C -2 C- 3 Reference

1 1 1 1 1 1 1 1 1 1 1

2 1 1 1 1 1 1 1 1 1 1

3 0 0 0 0 0 0 0 0 0 0

4 0 0 0 0 0 0 0 0 0 0

5 0 0 0 0 0 0 0 0 0 0

6 1 1 0 1 1 0 1 0 0 1

7 1 1 1 1 1 1 1 0 1 1

8 1 1 1 1 1 1 1 1 1 1

9 0 0 0 0 0 0 0 0 0 0

10 1 1 1 1 1 1 1 1 1 1

11 1 1 1 1 1 1 1 1 1 1

12 0 0 0 0 0 0 0 1 0 0

13 1 1 1 1 1 1 1 1 1 1

14 1 1 0 1 1 1 1 0 0 1

15 1 1 1 1 1 1 1 1 1 1

16 1 1 1 1 1 1 1 1 1 1

17 1 1 1 1 1 1 1 1 1 1

18 1 1 1 1 1 1 1 1 1 1

19 1 1 1 1 1 1 1 1 1 1

20 1 1 1 1 1 1 1 1 1 1

21 1 1 0 1 0 1 0 1 0 1

22 0 0 1 0 1 0 1 1 0 0

23 1 1 1 1 1 1 1 1 1 1

24 1 1 1 1 1 1 1 1 1 1

25 0 0 0 0 0 0 0 0 0 0

26 0 1 0 0 0 0 0 0 1 0

27 1 1 1 1 1 1 1 1 1 1

28 1 1 1 1 1 1 1 1 1 1

29 1 1 1 1 1 1 1 1 1 1

30 0 0 0 0 0 1 0 0 0 0

31 1 1 1 1 1 1 1 1 1 1

32 1 1 1 1 1 1 1 1 1 1

33 1 1 1 1 1 1 1 1 1 1

34 0 0 1 0 0 1 0 1 1 0

35 1 1 1 1 1 1 1 1 1 1

36 1 1 0 1 1 1 1 0 1 1

37 0 0 0 0 0 0 0 0 0 0

38 1 1 1 1 1 1 1 1 1 1

39 0 0 0 0 0 0 0 0 0 0

40 1 1 1 1 1 1 1 1 1 1

41 1 1 1 1 1 1 1 1 1 1

42 0 0 0 0 0 0 0 0 0 0

43 1 0 1 1 1 1 1 1 0 1

44 1 1 1 1 1 1 1 1 1 1

45 0 0 0 0 0 0 0 0 0 0

46 1 1 1 1 1 1 1 1 1 1

47 1 1 1 1 1 1 1 1 1 1

48 0 0 0 0 0 0 0 0 0 0

49 1 1 1 1 1 1 1 1 1 1

50 0 0 0 0 0 0 0 0 0 0167


COUNTS

There are 31 times where

A-3 = 1 and B-3 = 1(that is of the 50 parts checkedthere were 31 matches by A and B

on their THIRD check)

Total : where A-x =1 and B-x =1= 34+32+31= 97

Part A - 1 A - 2 A- 3 B - 1 B - 2 B - 3 C - 1 C - 2 C - 3 Reference

1 1 1 1 1 1 1 1 1 1 1

2 1 1 1 1 1 1 1 1 1 1

3 0 0 0 0 0 0 0 0 0 0

4 0 0 0 0 0 0 0 0 0 0

5 0 0 0 0 0 0 0 0 0 0

6 1 1 0 1 1 0 1 0 0 1

7 1 1 1 1 1 1 1 0 1 1

8 1 1 1 1 1 1 1 1 1 1

9 0 0 0 0 0 0 0 0 0 0

10 1 1 1 1 1 1 1 1 1 1

11 1 1 1 1 1 1 1 1 1 1

12 0 0 0 0 0 0 0 1 0 0

13 1 1 1 1 1 1 1 1 1 1

14 1 1 0 1 1 1 1 0 0 1

15 1 1 1 1 1 1 1 1 1 1

16 1 1 1 1 1 1 1 1 1 1

17 1 1 1 1 1 1 1 1 1 1

18 1 1 1 1 1 1 1 1 1 1

19 1 1 1 1 1 1 1 1 1 1

20 1 1 1 1 1 1 1 1 1 1

21 1 1 0 1 0 1 0 1 0 1

22 0 0 1 0 1 0 1 1 0 0

23 1 1 1 1 1 1 1 1 1 1

24 1 1 1 1 1 1 1 1 1 1

25 0 0 0 0 0 0 0 0 0 0

26 0 1 0 0 0 0 0 0 1 0

27 1 1 1 1 1 1 1 1 1 1

28 1 1 1 1 1 1 1 1 1 1

29 1 1 1 1 1 1 1 1 1 1

30 0 0 0 0 0 1 0 0 0 0

31 1 1 1 1 1 1 1 1 1 1

32 1 1 1 1 1 1 1 1 1 1

33 1 1 1 1 1 1 1 1 1 1

34 0 0 1 0 0 1 0 1 1 0

35 1 1 1 1 1 1 1 1 1 1

36 1 1 0 1 1 1 1 0 1 1

37 0 0 0 0 0 0 0 0 0 0

38 1 1 1 1 1 1 1 1 1 1

39 0 0 0 0 0 0 0 0 0 0

40 1 1 1 1 1 1 1 1 1 1

41 1 1 1 1 1 1 1 1 1 1

42 0 0 0 0 0 0 0 0 0 0

43 1 0 1 1 1 1 1 1 0 1

44 1 1 1 1 1 1 1 1 1 1

45 0 0 0 0 0 0 0 0 0 0

46 1 1 1 1 1 1 1 1 1 1

47 1 1 1 1 1 1 1 1 1 1

48 0 0 0 0 0 0 0 0 0 0

49 1 1 1 1 1 1 1 1 1 1

50 0 0 0 0 0 0 0 0 0 0


29/38

168

A* BCrosstabulation

44 6 50

15.7 34.3 50.0

3 97 100

31.3 68.7 100.0

47 103 150

47.0 103.0 150.0

Count

Expected Count

Count

Expected Count

Count

Expected Count

.00

1.00

A

Total

.00 1.00

B

Total

"EXPECTED COUNTS""EXPECTED COUNTS"

The expected counts is calculated using the the followingThe expected counts is calculated using the the following

formula (based on Chiformula (based on Chi--Square)Square)

Expected Count = Column Total x [Row Total/Grand Total]Expected Count = Column Total x [Row Total/Grand Total]

From the A*BFrom the A*B CrosstabulationCrosstabulation Table (pg 128)Table (pg 128)

Column Total = 103Column Total = 103

Row Total = 100Row Total = 100

The Grand Total = 150The Grand Total = 150

HenceHenceFor A=1 and B=1For A=1 and B=1

the Expected Count = 103 x [100/150] = 68.7the Expected Count = 103 x [100/150] = 68.7

97 from previous

slide

169

o

e

1

where p = Sum of the observed proportions in the diagonal cells (left to right direction)

p = Sum of the expected proportions in the diagonal cells (left to right direction)

o e

e

p pkappa

p-=

-

44 97 15.7 68.70.94 0.56150 150o e

p p+ += = = =

0.94 0.560.86

1 1 0.56o e

e

p pkappa

p- -= = =

- -


44 6 50

15.7 34.3 50.0

3 97 100

31.3 68.7 100.0

47 103 150

47.0 103.0 150.0

Count

Expected Count

Count

Expected Count

Count

Expected Count

.00

1.00

A

Total

.00 1.00

B

Total

170

Miss Rate & False Alarm Rate For Appraiser A

Miss Rate1: Calling aBAD part GOOD.

= 3/48 = 6.3%

1 Type II error, Consumer s risk2 Type I error, Producer s risk

False Alarm Rate2:

Calling a GOOD

part BAD.

= 5/102 = 4.9%

171

Signal Detection Theory

Not Covered in TI class

172

Attribute Risk Analysis Workshop

lDetermine the operational definition for visual defects for

M&M's

e.g. clear markings (M's), no chips, roundness, etc.

lPerform attribute risk analysis study using M&M's.

Use 3 operators/inspectorsUse a random sample of 25 M&Ms

lComplete attribute risk analysis and report results

lDetermine the Defects Per Unit of the sample

lSwitch with another team and determine the DPU of the

new sample using your operational definition

lDetermine the operational definition for visual defects for

M&M's

e.g. clear markings (M's), no chips, roundness, etc.

lPerform attribute risk analysis study using M&M's.

Use 3 operators/inspectors

Use a random sample of 25 M&Ms

lComplete attribute risk analysis and report results

lDetermine the Defects Per Unit of the sample

lSwitch with another team and determine the DPU of the

new sample using your operational definition

Chapter 4c

Automated SystemsAutomated Systems


30/38

174

Automatic Test Systems

Address during APQP - highest importance

Program measurement system identification

Determination of limits, ranges, targets, thresholdsCompensate for settling times, rise and fall times

of equipment, mechanical and environmentalphenomena

Determine discrimination level needed and Plan forProcess Control

Choose an appropriate set of nominal values for theIC process

175

Identify product parametric yield loss such as 2.9-3.9or 3.8-4.0 for the Process Controls

Do Bias, Linearity and GR&R studies on themeasurement/test system for that product with theselimits

Use capability of the equipment to read out variabledata or at least to store it for retrieval when needed

Analyze for improvement in each variable and/oridentified range


176

Test Setup

Optimize measurement window delay andmeasurement window length

Plan testing sequences carefully

MSA cannot be done on test fixtures or loadboards separately. It has to be done as part of thesystem

Do studies consideringAllnormal variables,Where the tests are done, Bythe personnelassigned to set up and run them.


Chapter 4d

Non-Replicable SystemsNon-Replicable Systems

178

Non-Replicable Analysis-Guidelines

Tightly control all variables

Use a large sample from a stable process

Take samples from a homogeneous lot

Use multiple lots to represent the parts

The part is not expected to age during the studyEach trail is conducted with a new sample

Use standard GRR techniques

ANOVA can be used to highlight variation fromappraisers, materials, day-to-day, etc

179

Non-Replicable GRR Case Study


31/38

180

Non-Replicable

Replication: the action or process of reproducing;performance of an experiment or procedure morethan once

Not always Destructive

Another R - sorry

181

Standardize Conditions

Qualified appraisers

Adequate lighting, environment

Proper work instructions

Equipment properly maintained

Failure modes understood

etc., etc., etc.

182

Assumptions/Prerequisites

Only looking at Repeatability

ANOVA (nested) software

Samples

homogeneous within

heterogeneous between

Pre-research on process

stable

nature of variation understood

183

Aside

If process is Stable

And if process is Capable

May be no need to do study

process variation includes measurement

this requires some process history/research

184

Process Information

A

B C

D

E

F G

H

I

J K

L

M

N O

P

Q

R S

T

U

V W

X

Welders

Press

1

2

3

4

5

6

7

8

9

10

Rods

RawMaterial

185

Standard GRR Layout

Appraiser 1 Appraiser 2

Part # Trial 1 Trial 2 Trial 3 Trial 1 Trial 2 Trial 3

1 1 1 1 1 1 1

2 2 2 2 2 2 2

3 3 3 3 3 3 3

4 4 4 4 4 4 4

5 5 5 5 5 5 5

6 6 6 6 6 6 6

7 7 7 7 7 7 7

8 8 8 8 8 8 8

9 9 9 9 9 9 9

10 10 10 10 10 10 10


32/38

186

Non-Replicable GRR Layout



1 1-1 1-2 1-3 1-4 1-5 1-62 2-1 2-2 2-3 2-4 2-5 2-6

3 3-1 3-2 3-3 3-4 3-5 3-6

4 4-1 4-2 4-3 4-4 4-5 4-6

5 5-1 5-2 5-3 5-4 5-5 5-6

6 6-1 6-2 6-3 6-4 6-5 6-6

7 7-1 7-2 7-3 7-4 7-5 7-6

8 8-1 8-2 8-3 8-4 8-5 8-6

9 9-1 9-2 9-3 9-4 9-5 9-6

10 10-1 10-2 10-3 10-4 10-5 10-6

187

Randomize Presentation

Each part has equal chance of being selected

Not haphazard

Not convenience

188

Randomize Presentation



1-1R 1-6R 1-6R53 1-5R25 1-3R7 1-4R40 1-1R34 1-2R32

2-1P 2-6P 2-2P27 2-6P35 2-5P57 2-4P12 2-3P43 2-1P17

3-1H 3-6H 3-1H21 3-2H36 3-6H1 3-5H56 3-4H10 3-3H264-1G 4-6G 4-4G46 4-6G42 4-3G8 4-2G28 4-1G55 4-5G305-1E 5-6E 5-4E5 5-3E20 5-1E13 5-6E54 5-2E39 5-5E50

6-1F 6-6F 6-1F52 6-3F3 6-4F37 6-5F29 6-2F51 6-6F457-1M 7-6M 7-6M16 7-4M11 7-1M23 7-2M6 7-3M15 7-5M14

8-1O 8-6O 8-6O49 8-3O60 8-1O33 8-5O41 8-2O44 8-4O199-1Q 9-6Q 9-5Q31 9-6Q59 9-3Q24 9-2Q4 9-4Q9 9-1Q2

10-1T 10-6T 10-2T22 10-5T18 10-3T47 10-4T58 10-1T48 10-6T38

189

Process Information

A

B C

D

E

F G

H

I

J K

L

M

N O

P

Q

R S

T

U

V W

X

Welders

Press

1

2

3

4

5

6

7

8

9

10

Rods

RawMaterial

190

Results

8500

9000

9500

10000Appr1 Appr2

Xbar Chart by Appr No.

SampleMean

Mean=9156

UCL=9525

LCL=8787

0

500

1000 Appr1 Appr2

R Chart by Appr No.

SampleRange

R=360.7

UCL=928.6

LCL=0

Appr1 Appr2

8000

9000

10000

Appr No

By Appr No.

1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10Appr1 Appr2

8000

9000

10000

Part2Appr No

By "Part" (Appr No.)

%Contribution

%StudyVar

Ga ge R &R R ep ea t R ep ro d P ar t- to -P ar t

0

50

100

Components of Variation

Percent

1

2

3

4

5

40% GR&R

FlatIn Control

50% Out of Control

191

Case Study Summary

Doing SOMETHING is Better than doingNOTHING (with customer approval)

Develop approach for each situation

Focus on learning

Be careful conducting study and interpretingresults

Consult statistical resources


33/38

Chapter 5

Measurement PlanningMeasurement Planning

193

Where To Start?

Evaluate the components of the measuring systemand control variation as much as possible to ensurethat an item of measuring equipment is in a state ofcompliance with requirements for its intended use

Expand our consideration of Measurement ProcessVariation to Measurement System StatisticalProperties and Measurement Uncertainty

Follow the basics of SPC

194

Measurement Planning

MSA Plan is the output of APQP (NPD) Phase 3, the ProcessDevelopment

At TI, this plan should originate in process development and becompleted by the production verification team.

It starts with the customer requirements and ends withconfidence of determination that these requirements are metand result in customer satisfaction

Is repeated as often as appropriate to control new equipment,new processes, new methods, new personnel, changes or anyvariables that may effect our ability to make accurate decisions

195

Measurement System Data

Define data needed, how to use the measurementsystem at APQP

Determine if measurement system statistical propertiesmeet needed limits and requirements and are worth timeand cost of using the system

The quality of a Measurement System is determined bythe statist ical properties of data produced:

Bias and Linearity, < 10%

R&R < 10%; 10-30% marginal and

total measurement uncertainty < 30%

196

Common Properties

Measurement System

Must be in a state of statistical control

Variation must be small compared to the

manufacturing process variation and specificationlimits

Increments no greater than 1/10th or smaller ofeither process variability or specification limits

Worst variation must be small relative to smallerof either process variation or specification limits

197

Measurement Systems must be in a state of statisticalcontrol i.e., free of special causes, as is true of allprocesses

1. In general, the absence of points in

the special cause region indicatesthe process is in a state of statistical

control.

2. If there is no trend or shift in the dataas explained in SPC, then along with(1) we could say that the process is

in a state of statistical control.

Statistical Control

COMM

ONCA

USER

EGION

SPECIAL CAUSE REGION

SPECIAL CAUSE REGION


34/38

198


The Data

What is to be measured: physical, contact, electrical,condition, dynamic

How important is this data

What will be done with the results

Who cares about the results

Process capability

Data range

What is the acceptance criteria

199


The Appraisers

Operators, engineers, inspectors

Training or certificationInstructions: Selection, set up, execution,

Automated vs manual

Can they influence results

Will attitude, stress affect results

Will the environment allow them to read the same results

Special handling, storage

200


Defining the physical system

Dedicated or flexible

Contact vs non-contact

Destructive

Measurement points

Fixturing

Part orientation

Part preparation

Transducer location

Resolution, sensitivity

Environment

201


Supporting Activities

What is the objective of the study

Has an equipment and MS FMEA been done

Do the equipment/fixtures have to be built

How have we defined the specifications

Is the source selected and proven

How will acceptance be conducted

202


Supporting Activities (continued)

Is the cost in the budget

Is there a preventative maintenance plan

Is there a user/troubleshooting guide

What are the utility requirements

What is the calibration plan

Can results be correlated with others: same

gage, same method, same measurements

What inputs are required

Chapter 6

Analysis of ResultsAnalysis of Results


35/38

204

Analysis of Results

Methods to determine problems are discussed in Chap. 3. Thistext is to be specific to TI per the intended procedures.

Chapter 7

Conducting a GR&R StudyConducting a GR&R Study

206

Measurement System Study

Typical preparation includes:

Work Instruction for the study

Number of appraisers andsample parts

Number of repeated readingsor trials

Criticality of dimension

Part configuration

Operators who use measurementEquipment as part of their work

Sample parts that represententire operating range

Is the equipment ready

Measurement equipment musthave discrimination thatallows at least 1/10th ofprocess variation ofcharacteristic to be read

Chapter 8

MSA Application to Continual

Improvement

MSA Application to Continual

Improvement

208

Continual Improvement Applications

Reduce Calibration or Maintenance Events

Based on stability studies and traditional percentageof error, calibration or maintenance intervals may

be extended.

Of course the studies could point to the need for moremaintenance or calibration events to keep thesystem capable.

209


Measure Appraiser Competence

TS requires the demonstration of competence forall personnel performing work affecting product

quality. An unacceptable MSA on a repeated basiscan point out the need for more capability ortraining or automation.


36/38

210


Increase process tolerance

Show how gray area reduction expands good/good

areaReduce false accepts or rejects

211


Improve selection of Measurement Systems

Retire incapable systems

212


Benchmark and compare to other fabs

213

Do You Understand?

l The language of Measurement ?

l The importance of Measurement?

l How to perform a Gage R&R Study and

how to interpret results ?

l Use MSAPro to analyze GRR results?

l The language of Measurement ?

l The importance of

Date post:	30-May-2018
Category:	Documents
Upload:	amiya691
View:	219 times
Download:	0 times

3RD Edition February 2003 OMNEX, Inc 3025 Boardwalk Suite 120

Documents