Transcat Webinar: :Suitability Of Instruments: Presented By: Howard Zion

Suitability of Instruments

Presented by

Better by Your Every Measure • 800.828.1470 • www.transcat.com

December 1, 2014

. . . six Takata airbag fatalities and multiple injuries!!

What is the risk of bad product?

Takata airbag malfunctions are due to:

• Age of the airbags

• Exposure to humidity

• “. . . potential production issues which we have worked to identify and address.”

- Hiroshi Shimizu, Senior Vice President of Global Quality Assurance

What are the root causes of bad product?

Age of the airbags:

• Product has a shelf life and must be replaced after the “safe usage period” has expired

and/or

• Design change of the product materials in order to achieve an acceptable “safe usage period”.

Translation

Exposure to humidity:

• Higher humidity causes the explosive used to inflate the airbag to deteriorate.

• This causes the inflator housing to scythe out of its holder and into the passenger compartment.

• Pieces of the broken housing act as shrapnel, which is propelled into passengers.

• The investigation is focusing on the explosive inflators and how well (or poorly) they were sealed.

• Was it bad materials that could not maintain the seal?

• Was it false acceptance of bad product due to measurement risk?

Translation

“. . . potential production issues which we have worked to identify and address.”

• This vague answer indicates a lack of process controls which could be the result of:

• inferior product or component design by Takataand/or its suppliers

• selection of faulty materials

• false test acceptance of product/component

• All of these end in higher Consumer Risk

• All are traced to suitability errors

Translation

• ISO/TS 16949: Quality Management Systems –

Automotive Suppliers

What is Suitability?

• “Each manufacturer shall ensure that all

inspection, measuring, and test

equipment, including mechanical,

automated, or electronic inspection and

test equipment, is suitable for its intended

purposes and is capable of producing valid

results.”(21 CFR 820.72).

. . . while this is the right context of suitability, it still remains undefined.

What is Suitability?

Suitable for intended purpose

Let’s start out with a simple example . . .

a) What is the most suitable tool for the job?

b) Which have you never used to drive a nail?


Who said the Hammer is the right tool for this job?


Really – you’d use a hammer on a screw??


Suitability means using the right tool for the job …


… and using it the right way!

1. Understand components of Suitability for instruments

2. Formulate your own definition of Suitability

3. Compare to your organization’s current definition• Create definition for your organization if one doesn’t exist

• Enhance definition for your organization if one already exists

Learning Objectives

Reasons People Use the Wrong Tool

1. Inconvenient; don’t want to go get the right tool

2. Don’t know how to use the right tool/instrument

3. Don’t own the right tool/instrument

4. Can’t afford the right tool/instrument

Simple vs. Complex Process

• These are simple hand tool examples

• If it’s this easy for people to use the wrong tool (or the right tool the wrong way) in a simple process . . .

Simple vs. Complex Process

• . . . what are the chances of introducing errors when using more complex tools/instruments in more complex processes?

Measurement Quality Assurance

• Metrological Traceability is important, but . . .

• It means nothing if the industrial processes don’t maintain a good measurement quality assurance program

• MQA is critical to industrial manufacturing processes:– To keep product costs at a minimum

– To maintain safety for employees and product consumers

– To keep product lead times from being delayed

– To make the traceability chain a worthwhile effort (and its costs)

Where does Risk Creep into Your Process?

Measurement Quality Assurance encompasses a number of assumptions within a manufacturing process:

1. Product Acceptance You made the right decision about the acceptance or rejection of your product (or a component of your product) during the manufacturing and quality acceptance process.

2. Basis of Decision The decision was based (partially or completely) on quantitative information provided by one or more measuring instruments.

3. Suitability for Intended Purpose

You made the right choice in selecting instrument(s) that are appropriate for the measurements in your process.

4. Instrument Application The instrument(s) were used correctly in the manufacturing process when making decisions about product quality.

Where does Risk Creep into Your Process?

Measurement Assurance encompasses a number of assumptions (con’t):

5. Beginning of Period Reliability

The calibration of the instrument prior to your decision about the product indicated it met its performance expectations.

6. End of Period Reliability

The subsequent calibration of the instrument following your decision about the product indicates that it continues to meet its performance expectations.

7. Calibration Process Calibrations are executed correctly and support the application(s).

8. Non-Conformance Review (NCR)

If a calibration indicates that the instrument failed to meet its performance expectations, your quality system reviewed any impact to the decision(s) made about your product(s) that were based on the quantitative values presented by the instrument.

9. NCR Accuracy This impact study was thorough, followed an unbiased, forensic approach, and did not miss the mark on determining any detrimental affect on product.

10. Risk mitigation Any negative result from the impact study was properly mitigated to remove or reduce product risk.

Example: Module transfer process between a space vehicle and the International Space Station (ISS). Heaters prevent ice buildup in the transfer module and spacesuits (Space Suit UP!), specifically in the gloves and helmet camera. The main battery pack provides 12.5 ±1.5 VDC

– Helmet camera heater requires 12 ± 1 VDC

– Glove heaters provide a resistance of 57.7 ± 2.9 Ω; require 9 ± 0.5 VDC

Note: No matter how simple the process, following a consistent procedure will keep you from making ‘honest mistakes’ that end up wasting time/money or that cause safety issues.

Step 1: Determine all measurements that will be made in this process

• VDC: 12.5, 12, 9

• Resistance (Ω): 57.7


Example: Module transfer process between a space vehicle and the International Space Station (ISS). Measuring heater voltage and resistance.

Step 2: Determine initial list of instruments for consideration

• Centech 37772 Handheld DMM

• Fluke 87V Handheld DMM

• Agilent 34401A Benchtop DMM

• Agilent 3458A Opt 002 8.5 Digit Multimeter



Step 3: Collect the OEM specifications for all instruments

• Centech 37772




• Fluke 87V




• Agilent 34401A




• Agilent 3458A Opt 002



Step 4: Evaluate instruments for Suitability

a) Parameter Verification: Determine if the selected instruments cover the parameters of interest

• In this case, all four instruments can measure both parameters: DC Voltage and DC Resistance




b) Range Verification: Determine if the selected instruments have ranges that cover the target measurements

– DC Voltage: 9V, 12V, 12.5 V

• Centech 37772: The 20V range covers all of the target measurements

• Fluke 87V: The 60V range covers all of the target measurements

• Agilent 34401A: The 10V range covers the 9V measurement; 100V range covers other voltages

• Agilent 3458A: The 10V range covers the 9V measurement; 100V range covers other voltages

– Resistance: 57.7 Ω

• Agilent 3458A: The second lowest range covers the resistance measurement

• All other models: The lowest range covers the resistance measurement




c) Accuracy/Resolution Verification: Using the accuracy statement of each instrument, convert to a tolerance in the engineering unit of the measurand.

Centech 37772: Measure 12.5 VDC on the 20V range

Accuracy (20V range): ±(0.5% of rdg + 1 digit); resolution is 10mV on this range

Tolerance = ±(0.5% x 12.5V + 10 mV)

= ±0.0725 V

= ±0.07 V (considering resolution)




c) Accuracy/Resolution Verification: Repeat for all other measurements and instruments.




d) Process Accuracy Ratio (PAR) Calculation: Use the following formula to calculate PAR for each measurement and each instrument.

where:

UPL = Upper Process Limit

LPL = Lower Process Limit

UIL = Upper Instrument Limit

LIL = Lower Instrument Limit






d) Process Accuracy Ratio (PAR) Calculation: Use the following formula to calculate PAR for each measurement and each instrument.

Centech 37772 for the 57.7 Ω process measurement:




d) PAR Calculation: Repeat for all measurements and instruments.



Step 5: Intended use: other applications?

If you believe the selected instrument may be used in other applications, you must perform this same procedure to determine suitability of the instrument for those applications.



Step 5: Cost Factor

Centech 37772: $20PAR: One at 3.6:1

Others > 8:1

Fluke 87V: $400PAR: All > 9:1

Agilent 34401A: $1kPAR: All > 300:1

Agilent 3458A: $9kPAR: All > 2000:1


One more thought . . .

After you become comfortable with this Suitability process and calculating PAR, you really should consider taking it a step further . . .

Process Uncertainty Ratio (PUR): Takes into consideration other errors being introduced while making process measurements which include:

• Operator training

• Gage Repeatability & Reproducibility (Gage R&R) variations

• Environment in which the instrument is being used

• Storage conditions for the instrument when not in use

• Handling conditions for the instrument over its calibration interval


• An active Measurement Assurance program consists of:– Right tool for the job (i.e., suitability)– Regular calibration

• Calibrations that support the process where the instrument is used• Calibrations that supply the correct tolerances• Calibrations that are valid (i.e., calibration measurement uncertainties

support the measurements made)

– Using the instrument/tool correctly– Accounting for irregularities in the production measurement

process (i.e., Process measurement uncertainties)– OOT investigations performed properly– Corrective action taken for OOT impacting product


• Example: Temperature measurement

– Mfg Engineer needs to measure solution used to treat product

– Process measurement is 350° ± 2° C

– Which instrument is suitable for this measurement?LL:

Process

348°C

UL:

Process

352°C350°C


• Process measurement

– An instrument with an accuracy equal to the process limits?

– Not good because the instrument can drift the full amount over its

cal interval, directly impacting previous process measurements

348°C 352°C350°C

LL:

Process

UL:

Process



– Instrument has been calibrated today and adjusted to nominal

– Instrument is used to measure solution temperature the same day

– Solution temperature is right at noinal: 350° C

348°C 352°C350°C

LL:

Process

UL:

Process



– Instrument has been calibrated today and adjusted to nominal

– Instrument is used to measure solution temperature the same day

– Solution temperature is right at nominal: 350° C

348°C 352°C350°C

LL:

Process

UL:

Process



– Instrument is on its last day of use before it will be recalibrated

– Instrument is used to measure solution temperature


348°C 352°C350°C

LL:

Process

UL:

Process



– Instrument is on its last day of use before it will be recalibrated

– Instrument is used to measure solution temperature


348°C 352°C350°C

LL:

Process

UL:

Process


• Temperature Instrument Calibration

– Instrument is recalibrated (and subsequently was adjusted)

– As Found reading shows the instrument drifted to upper limit:352°C

– This drift occurred over its calibration interval

348°C 352°C350°C

LL:

UUT

UL:

UUT



– Instrument is recalibrated (and subsequently was adjusted)

– As Found reading shows the instrument drifted to upper limit:352°C

– This drift occurred over its calibration interval

348°C 352°C350°C

LL:

UUT

UL:

UUT



– Did it move all at once or a little over time?

– How does this impact the measurements of the process since

the last time the temperature instrument was calibrated?

348°C 352°C350°C

LL:

UUT

UL:

UUT


• OOT-NCR Evaluation

– It must be assumed this shift occurred immediately following the

last calibration because that is the only supporting evidence to

show the last known good condition of its temperature values

348°C 352°C350°C

LL:

Process

UL:

Process

x



– So the process measurements taken on the day the instrument was

previously calibrated were not actually 350° C, they were 348° C

– The same applies to the process readings at the end of the cal cycle

348°C 352°C350°C

LL:

Process

UL:

Process

x



– So the process measurements taken on the day the instrument was

previously calibrated were not actually 350° C, they were 348° C

– The same applies to the process readings at the end of the cal cycle

348°C 352°C350°C

LL:

Process

UL:

Process

x



– What if some process readings indicated the solution was at its lower

limit: 348° C? This would be acceptable for the process test, but . . .

– The In Tolerance cal would not have flagged this situation as ‘at risk’!!

348°C 352°C350°C

LL:

Process

UL:

Process

x



– What if some process readings indicated the solution was at its lower

limit: 348° C? This would be acceptable for the process test, but . . .

– The In Tolerance cal would not have flagged this situation as ‘at risk’!!

348°C 352°C350°C

LL:

Process

UL:

Process

x


• Instrument with better accuracy:

– What if an instrument was selected with an accuracy twice as good?

– If it drifted to its limit, process readings would only be off half of the

process tolerance

348°C 352°C350°C

LL:

Process

UL:

Process

349°C

LL:

UUT

351°C

UL:

UUT

x



– What if an instrument was selected with an accuracy 4X as good?

– If it drifted to its limit, process readings would only be off one quarter

of the process tolerance

348°C 352°C350°C

LL:

Process

UL:

Process

349.5°C

LL:

UUT

350.5°C

UL:

UUT

x



– How far can we take this? 10X better? 100X better?

– There are limits of technology and/or it eventually becomes cost

prohibitive to go too far with this concept

348°C 352°C350°C

LL:

Process

UL:

Process

349.5°C 350.5°C

LL:

UUT

UL:

UUT

x



– Traditionally, a 4:1 ratio is sufficient to reduce the probability that an

OOT will have an impact on the product/process

– For some measurements, this cannot be achieved, so risk is higher

348°C 352°C350°C

LL:

Process

UL:

Process

349.5°C 350.5°C

LL:

UUT

UL:

UUT

x



– However, if one or more of the process measurements was at its

lower limit . . . risk still exists for this “In Tolerance” calibration

– Using the “As Found” cal value, all readings were higher by 0.5° C

348°C 352°C350°C

LL:

Process

UL:

Process

349.5°C 350.5°C

LL:

UUT

UL:

UUT

x347.5°C



– However, if one or more of the process measurements was at its

lower limit . . . risk still exists for this “In Tolerance” calibration

– Using the “As Found” cal value, all readings were higher by 0.5° C

348°C 352°C350°C

LL:

Process

UL:

Process

349.5°C 350.5°C

LL:

UUT

UL:

UUT

x347.5°C


• Lesson Learned:– Even “In Tolerance” results can impact your process

measurements!– All cal data must be reviewed against process measurements to

understand the impact to the product (IT-NCR evaluation)– Impact studies are expensive; consumes valuable resources– Can cost thousands of dollars per evaluation event– Parts or product likely have already been released/shipped– Could require product recall– If not released, could require rework– This risk can be reduced– Guard Band the process limits!


• What is Guard Band?

– Determine realistic tolerance limits for the process measurement

– Reduce these limits by the tolerance of the instrument used in the

process to arrive at Lower/Upper Acceptance Limits

350°C348°C

LL:

Process

352°C

UL:

Process

349.5°C 350.5°C

LL:

UUT

UL:

UUT

x347.5°C

348.5°C

LAL:

Process

351.5°C

UAL:

Process

x


• Guard Band result:

– Now if the instrument cal reveals a drift up to the maximum tolerance

there is no need to perform IT-NCR evaluation

– If instrument error is greater than this (i.e. it is OOT), then OOT-NCR

348°C 352°C350°C

LL:

Process

UL:

Process

349.5°C 350.5°C

LL:

UUT

UL:

UUT

x

348.5°C

LAL:

Process

351.5°C

UAL:

Process


• Guard Band result:

– Now if the instrument cal reveals a drift up to the maximum tolerance

there is no need to perform IT-NCR evaluation

– If instrument error is greater than this (i.e. it is OOT), then OOT-NCR

348°C 352°C350°C

LL:

Process

UL:

Process

349.5°C 350.5°C

LL:

UUT

UL:

UUT

x

348.5°C

LAL:

Process

351.5°C

UAL:

Process


• Summary:

– Implement a good Measurement Assurance Program

– Understand and exercise good Suitability selection

– Guard Band the process tolerance limits to reduce costly

NCR evals

– Understand both the process and the calibration to ensure

the intent of preserving good measurements on the product is

not lost

– Be thorough in your NCR evaluations

– Get help if you’re in over your head!


• In the end, running your business is not only about international, federal, customer, or internal requirements, policies, or procedures – it’s about making a safe, reliable, superior product that fills a need/desire in the market place and is profitable for your company.

• Measurement Quality Assurance should be designed to help you minimize risks in your decision making process about your product’s safety and quality.

• Suitability of Instruments is a key part of guaranteeing good measurements!

• Too often MQA is not fully implemented (all 10 categories), causing the reliability of your Measurement Quality Assurance Program to lose value and become ineffective.

• If you’re going to put the effort and money into only some parts of this program, or you simply do not recognize all of the factors, it will likely cost you elsewhere through rework, recall, or consumer perceptions.

• Make your Measurement Quality Assurance Program robust so that it works for you to keep cost and safety issues to a minimum and profits up!

Measurement Quality Assurance/Risk

Thank you for attending!

800.828.1470www.transcat.com

Questions?

Date post:	15-Jul-2015
Category:	Engineering
Upload:	transcat
View:	246 times
Download:	4 times

Transcat Webinar: :Suitability Of Instruments: Presented By: Howard Zion

Engineering