0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing

7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing

1/33

Danish Fundamental Metrology DFM- 2004-R78 Phone: +45 4593 1144

B 307 Matematiktorvet 6406 HDJ Fax: +45 4593 1137DK-2800 Kgs. Lyngby, DENMARK 2004-11-30 Web: www.dfm.dtu.dk

DRAFT(please send comments to [email protected])

Practical guide for the estimation of uncertainty in testing

Hans Dalsgaard Jensen

Danish Fundamental Metrology

B 307, Matematiktorvet,

DK-2800 Kgs. Lyngby

and

George Teunisse

Nederlands Meetinstitut

Schoemakerstraat 97

NL-2628 VK Delft

Abstract:

Measurement uncertainty plays a fundamental role in the documentation of trace-

ability and the reliability of measurement results. Physical testing often relies on

quantitative test results, and such are to be regarded as any other quantitative

measurement, and thus requires a reliable uncertainty statement for conclusions to

be drawn. However, a tradition and routine in calculating measurement uncertain-

ties for test results is only in the beginning of its development.

This guide seeks to develop and present some tools for the physical testing com-

munity, showing how to approach the problem of assigning uncertainty using the

philosophy and methods of the ISO Guide to the Expression of Uncertainty inMeasurement.


2/33

2

6406 HDJ 2004 11 30

1 Table of contents

1 Table of contents ................................................................................................................ 2

2 Introduction ........................................................................................................................ 2

3 Connection with other Guides............................................................................................ 2

4 Considerations for evaluation of uncertainty in testing ..................................................... 34.1 Why uncertainty in the first place? ............................................................................ 4

5 Methods for uncertainty estimation.................................................................................... 4

5.1 General - GUM........................................................................................................... 5

5.1.1 Hints on establishing model functions ............................................................... 8

5.2 Generic models........................................................................................................... 9

5.3 Repeatability - reproducibility ................................................................................. 14

5.3.1 Basic assumptions and use of nested designs................................................... 15

5.3.2 Intercomparison data ........................................................................................16

5.4 Other basic tools....................................................................................................... 17

5.4.1 Translation of instrument specifications....................................................... 17

5.4.2 Calculation, use and uncertainty of calibration curves .................................... 18

5.4.3 Drift ..................................................................................................................21

5.4.4 Handling ill-defined measurement conditions .................................................21

6 Examples from electrical testing ...................................................................................... 22

6.1 Measurement of temperature using thermocouples .................................................22

6.2 Heating in Black Test Corner................................................................................... 24

6.3 Glow wire test .......................................................................................................... 25

6.4 EMC - Radiated Immunity Test (IEC 61000-4-3) ................................................... 26

7 Appendix .......................................................................................................................... 28

7.1 Table of distribution functions often used for type-B evaluation: ........................... 32

8 References ........................................................................................................................ 33

2 Introduction

The widespread use of accreditation in testing as a means to ensure the proper level of process

and quality control of test results has lead to an increased focus on the estimation of uncer-

tainty of quantitative test results. The introduction of ISO 17025 as the laboratory standard

has further increased the focus on estimation of uncertainty in quantitative testing.

A main problem is the distinction between measurand and characteristic, where the latter

is a property determined by a test. However, for quantitative testing, where a quantitative

characteristic is determined, there is no practical distinction to that of a measurement quantity.

Another problem is the fact that due to ill defined test set up or test environment in quite a

number of standards, the result of a test can be strongly influenced by these conditions.

The evaluation of all the influences on the measurement quantity could be a very complex and

time-consuming operation. This guide aims to supply solutions to some of these problems.

3 Connection with other Guides

There exist several other guides for the evaluation of uncertainty in testing. Accreditation

bodies (e.g. A2LA, UKAS), regional cooperation organisations (e.g. EA) and laboratory asso-

ciations (Eurolab) have all published documents concerning this issue, see section 8References. Some specific information on classical concepts in testing, such as repeatability


3/33

3

6406 HDJ 2004 11 30

and reproducibility, are described in detail in ISO 5725 and brought into the GUM framework

in ISO 21748. Some of the documents fall in the category of a more or less policy documents,

and do not provide specific guidance for the combination and calculation of uncertainty.

Other documents provide examples of calculation of uncertainty, but with very specific ex-

amples without overall guidance on how to handle specific types of uncertainty contributions.

In the present document we will try to bridge this gap and focus on the method of assigning

uncertainty, generalize some of the specific uncertainty contributions, open a toolbox of ge-

neric measurement methods and discuss how to assign uncertainties for the results from use of

these methods.

4 Considerations for evaluation of uncertainty in testing

Paper standards (norms) from the beginning of their existence have served as a means to be

able to produce or reproduce a property by the extensive description of methods for manufac-

turing or testing of an object.

This method of describing in itself causes the first sources of uncertainty. Differences in in-terpretation could already lead to different outcome. Therefore it is necessary that there al-

ways is some platform where people can exchange their information in order to diminish the

possible influence of this source.

It even appears (which is often the case in e.g. EMC test methods) that the testing conditions

are ill defined.

The increasing accuracy and measurement techniques have lead to a growing awareness of the

imperfections in many of the described methods in standards, especially when the measurand

is presented as an absolute measurement value.

However due to the fact that most of the standards (norms) for testing often have been applied

for many years it will lead to a heavy oppositions to modify the described methods and espe-cially in the case that these standards do not contain any margins to the measurand in terms of

uncertainty.

Would an inventory been made on the uncertainty contributions and their values in these

standards then there could be made a comparison between such a method and any new

method to be introduced.

Like mentioned this is not often the case so there will be no strong arguments to change a

method to a more sophisticated method even when this appears to be a far more reproducible

method.

The only elegant way to alter this defensive attitude to a more constructive attitude is toclearly show the benefits of the knowledge of the uncertainties in the measurements by intro-

ducing in the standards a dependency of the values of the testing limits from the magnitude of

these uncertainties.

Of course when starting such an approach it will need an inventory on the measurement un-

certainties in all existing (applicable) standards.

Though this inventory could be assumed a quite time consuming and extensive job it does not

at the same time mean that it is neither impossible nor unprofitable.

The following chapters will show that when applying a systematic method in which a distinc-

tion is made between the contributions coming from the instrumentation on the one hand and

from the test set up on the other hand.


4/33

4

6406 HDJ 2004 11 30

It will also be shown that in a number of cases it is not always necessary that the testing lab

produce the values of all the contributors to the uncertainty as long as the normalization

committees use the same methods of evaluation when integrating test methods in the stan-

dards they produce.

In the next chapter first of all the use of a uniform method for the estimation of uncertainty

will be described, which is already common practice for accredited calibration laboratories.

Extending this method for testing set ups is possible when introducing some additional tools.

4.1 Why uncertainty in the first place?

The argument has been heard, that uncertainty in testing is not worth the effort, because the

test is performed within the limits of the influence factors as specified in the standard, and

the uncertainty of the measurement results themselves are small in comparison with the

overall test result, hence there is no reason to evaluate or report measurement uncertainty.

This of course does not make sense; compliance with as well the conditions of the test as

the result of the test itself, does require evaluation of uncertainty. But there are more im-portant arguments for performing a thorough investigation of the uncertainty of measure-

ments, although the task can be time-consuming. These arguments are also stressed by the

accreditation organisations:

Reporting a qualified measurement uncertainty statement assists in issues such as riskcontrol and the credibility of test results, and can represent a direct competitive advan-

tage by adding value and meaning to the result.

The knowledge of quantitative effects of single quantities on the test result improves thereliability of the test procedure. Corrective measures may be implemented more effi-

ciently and hence become more cost-effective.

The evaluation of measurement uncertainty enables one to optimise the test proceduresand calibration costs may be reduced if it is found that particular influence quantities do

not contribute substantially to the uncertainty.

When evaluating the result of a test and stating compliance with a specification, it isnecessary with information on the uncertainty associated with results.

5 Methods for uncertainty estimation

Quantitative measurement results without a stated uncertainty are basically meaningless.

However, we must realize that in many, maybe even most, cases in our daily lives this is how

results are reported. Often there is an implicit assumption on the accuracy of the numbergiven: When taking of the age of a person, we implicitly assume an uncertainty of less than

one year, when reading a reference value in a data book we assume the accuracy is of the or-

der of the last significant digit. When we buy a measurement instrument, we assume that most

of the digits shown have meaning; otherwise the manufacturer would probably have chosen a

cheaper display. However, for scientifically based measurements or for measurements in

which we wish to put just the slightest amount of trust, these implicit assumptions simply are

not enough. We need some quantitative indication of the quality of the result; otherwise we

have no means of comparing results among themselves, with other measurement results, with

reference values or limits. And most importantly, we must be very explicit on the assumptions

that are made in the process, because assumptions will still be necessary.


5/33

5

6406 HDJ 2004 11 30

According to the newest version of the Vocabulary of Basic and General Terms in Metrology

(VIM, 3rd ed), measurement uncertainty is defined as

a parameter that characterizes the dispersion of the quantity values that are

being attributed to a measurand, based on the information used.

Hence, the purpose of estimation of uncertainty is to quantify this dispersion, and the re-mainder of this paper is devoted to the discussion of the methods used for this.

It is the expressed policy of the accreditation bodies as well as of the national metrology insti-

tutes, that measurement uncertainty shall be estimated as a realistic, scientifically based quan-

tity, and the method by which to estimate uncertainty should be universal (applicable to all

kinds of measurements), consistent (independent of the grouping and decomposition into sub-

components) and transferable (the results of one uncertainty calculation can be directly ap-

plied in a second calculation). The method universally agreed upon is described in The ISO

Guide on the Expression of Uncertainty in Measurement, or short, GUM.

5.1 General - GUMThe GUM, is the Master document used on the calcula-

tion and reporting of uncertainty for accredited laborato-

ries. The GUM employs the BIPM Method developed

by BIPM in the late 1980s, which takes the basic start-

ing point of handling all measurement quantities, includ-

ing their factors of influence, as stochastic variables.

A stochastic variable describes a quantity as having

many possible different outcomes, with a given prob-

ability distribution.

The value of an input quantity is the estimate of the ex-pectation value of the distribution function, and the associated standard uncertainty of the

value of the quantity is the square-root of the estimate of the variance of the distribution

function. The methods to ascribe the distribution function of the input quantities may be

performed either through statistical means (called type-A evaluation) or by other means

(called type-B evaluation), e.g. by assumption of a probability distribution. For a primer on

how to evaluate and assign probability distribution functions, see section 7.

The measured quantity or the output quantity is described in terms of a model function in

terms of the input quantities. Finally, the distribution function of the output quantity/-ies is

found using the calculation rules for the propagation of variances.

The GUM method is described in various details with examples in many other documents,

so a thorough guide will not be given here. For completion, the method may be summa-

rised to comprise of the following steps:

1. Specification of the measurand. Consider the nature of the quantity intended as the

result of the measurement. Is it a direct value, is it a deviation from some reference,

e.g. the nominal value, is it a dispersion from an average (or is it the average itself)

or is it some ensemble property, e.g. the average temperature of a selection of

measurement points. The specification of the measurand is naturally critical in any

kind of assignment of uncertainty. Often the focus is only on uncertainty

components, and are collected without regard to their comparability and how the

value of the component combines with the result, thus, the measurand becomesimplicitly defined; one of the most dangerous starting points for uncertainty evalua-

tion.

Probability

Value

Fig. 1 A stochastic variable as a quan-

tity with a probability distribution


6/33

6

6406 HDJ 2004 11 30

2. Input quantities. Identify all sources that have a direct relation to the end result, e.g.

instrument readings, applied correction values, reference data, and any source that

may affect the result of the measurement indirectly, i.e. influence quantities such as

environmental conditions, operator variability,

3. Model function. Set up a model function, which relates the final result (the output

quantity), the value of the measurand, to the sources in (2) (the input quantities). The

model function may reflect the physical measurement process or may be a purely

empirical relation. The starting point for any model function should naturally reflect

the way in which the measurement result is derived, even if it consists of a direct

reading of an instrument. The model function must then be expanded to take into ac-

count all the input quantities identified in (2). For tips on taking the indirect quanti-

ties into account, see below.

4. Uncertainty distributions. For each of the input quantities, an expectation value and

the variance must be retrieved, either from a statistical treatment or from an assumed

distribution function of the input quantity. This part of the process sometimes require

just as much thought as setting up the model function itself. Again the documentsmentioned above and listed in the appendix provides many examples of derivations

for different types of influence quantities. This subject is treated more extensively

below.

Consider possible correlations between the input quantities, e.g. when using differ-

ences between readings. Consider whether two or more of the input quantities are re-

lated to each other, even via a third parameter, so that a change in one would be ob-

served at the same time as a change in the other. For example, two quantities may

depend on temperature. If this underlying temperature fluctuates, the values of both

parameters fluctuate, in the same direction or in the opposite; there is a relation (a

correlation) between them. When this interdependence is found or expected to besignificant, the degree of correlation must be taken into account when calculating

measurement uncertainty. To the extent possible and practical, such a third parameter

should be put into the model function as an independent input quantity, and the influ-

ence it has on the two parameters in question be described by the model function.

The important aspect is the conversion of any uncertainty statement that may be

the initial point of discussion, to that of a standard uncertainty in terms of the

square root of the estimate of the variance of the distribution function assigned to the

quantity. For different types of uncertainty statements, e.g. the very common rec-

tangular distribution a, calculate the standard uncertainties by applying the factor

to the parameter appropriate for the distribution function assigned, e.g. for the rec-

tangular distribution 31a , and in the case of statistical treatment based on aver-ages, be sure to consider whether the quantity refers to one observation or an average

value of several observations. See section 7.1 for a list of distribution functions, their

parameters and conversion to standard uncertainties.

5. Propagation of uncertainty. Apply the rule of propagation of uncertainty, i.e. calcu-

late the sensitivity factors for the model function for each input quantity and multiply

by the standard uncertainty of that quantity, and include the terms due to correlation.

It is recommended to use some software calculation tool, to avoid the tedious, error

prone calculations when the model function is more than a simple sum/difference or

product/quotient.


7/33

7

6406 HDJ 2004 11 30

6. Coverage probability. Due to the nature of the calculation of uncertainty based on

standard uncertainties, the interval covered by the result plus and minus the standard

uncertainty, will only cover 68% of the expected outcomes, if all the assumptions

hold. It is therefore common practice to quote an expanded uncertainty, U, in that the

standard uncertainty is multiplied by a coverage factor to achieve either 95 % or

99 % coverage probability. For an output quantity that follows a Normal distribution,a coverage factor ofk= 2 is used to achieve approximately 95% coverage probabil-

ity. If the output quantity is dominated by, say a rectangular distributed input quan-

tity, the appropriate coverage factor is k= 1,65.

In cases where the confidence in the evaluation of the individual uncertainties is

small, it is necessary to take into account an appropriate number ofdegrees of free-

dom. When the combined uncertainty has a small number of degrees of freedom (less

than 10), the factork= 2 above is not sufficient to achieve a 95 % coverage probabil-

ity. The relevant coverage factor may be found from a table of the two-tailed Stu-

dents t-distribution with the given number of degrees of freedom.

7. Reporting the result. A measurement result should be quoted:x Uand provide in-formation on the coverage probability and the coverage factor used.

Two simple cases are mentioned here, which may be applied in maybe 90 % of the cases

relevant for the intended audience: A sum/difference of contributions and a prod-

uct/quotient of contributions.

Sum and difference

Given a measurement of a quantityy found as a sum or differences of other quantities, say

a direct reading of a value plus a series of corrections, all stated in the same unit as the

quantity being measured (an important point: you cannot add apples and pears!) The model

function then becomes a simple sum:

=+++=i

ixxxxy ...321

The corrections applied to the direct reading could be a known deviation, an estimated drift

or the dispersion determined by experience of the type of instrument used. Again, it is im-

portant to stress, that both a value and an uncertainty must be assigned to each of the input

quantitiesxi (even if the appropriate value is 0, thus indicating that no numerical correction

is performed).

If the input quantities are uncorrelated, the combined standard uncertainty for the output

quantity becomes

=+++=i

ixuxuxuxuyu22

3

2

2

2

1 )(...)()()()(

Product and quotient

A model function in the form of products and quotients may be handled in a similar way. If

the measurand may be modelled in the form

=+=i

ixxxxy ...321

where the factors are e.g. corrections factors to the main input quantity, say x1. Such cor-

rection factors will then typically have a value of 1 (hence not changing the numericalvalue of the result) and their uncertainty could be in units of % or or similar. For such a


8/33

8

6406 HDJ 2004 11 30

product (or quotient) model, it is convenient to work in relative uncertainties w(x) = u(x) /

x.

If the input quantities are uncorrelated, the combined standard uncertainty for the output

quantity becomes

=+++= i ixwxwxwxwyw22

32

22

1 )(...)()()()(

and u(y) = w(y) y.

In an very large number of cases, and especially in the cases where more empirically based

model functions are used, the sum/difference or the product/quotient, or a combination of

the two, is all that is needed. This is due to the consistency of the method; it does not de-

pend on any specific way of grouping the uncertainties, and may be evaluated in an arbi-

trary number of steps. For example, given the expression

( )

65

4321

xx

xxxxy

+

++=

This may be reduced in the following way:

89

4719

658

327xxy

xxxx

xxx

xxx=

++=

+=

=

with the calculation of uncertainty done at each step as well, i.e. for the auxiliary quantities

x7,x8, and x9. If the quantities are correlated or the same quantity appears in several places

in the expression, this approach cannot be used without taking the correlations into account

in the steps.

SoftwareIt is strongly recommended to use a software package for the tedious calculation of uncer-

tainty to avoid the necessity for reductions as shown above. Choose a program that allows

you to input quantities, distribution functions, correlations and most importantly general

model functions, and that calculates the sensitivity coefficients, uncertainty contributions

and handles limited degrees of freedom.

5.1.1 Hints on establishing model functions

The establishment of a model function is most often an iterative process. Always start

out by the relation that is used to calculate the measurement result, even if it as simple

as y = x, e.g. the reading of an instrument. In that case always consider a reading assimply a number thrown at you with no direct meaning; not until a calibration (estab-

lishing the relation between the indication of an instrument and the physical quantity)

is performed. Even a fixed standard with a stamped on nominal value, say a 100 re-sistor, should not be considered as a resistance of 100 without taking into account thatthere is a difference between the physical quantity presented and the collection of sym-

bols 1-0-0- stamped on the box. There will always be a term describing the differ-ence between the nominal value and the true physical value; in simple cases it may have

the value zero, but its standard uncertainty will enter the equation for the standard un-

certainty of the quantity being measured.

Given a quantity that is known to affect the total uncertainty of a measurement process,but for which it may be difficult to establish the functional relationship with the output


9/33

9

6406 HDJ 2004 11 30

quantity, one hint is to exaggerate its influence, assume perfect knowledge of its value

and determine how a correction would be applied. First give the quantity a name and a

symbol so that it may be handled algebraically.

E.g. in a test, the influence of the experience of the operator is known to affect, maybe

not the result, but at least the expected variation in results. Using the hint, an assump-

tion to make in the process of setting up the model function could be: I known that the

operator always measures a value for quantity A which is 25% lower than the true

value. If that was so, one would correct quantityA and instead useA/(100 % 25 %) =

A/0,75. Hence, the quantity I look forf is the influence quantity, the relative magni-

tude by which the operator measures too low, and it enters as A/(1 f). Now, given an

observation that for a specific test, operator variability is found as a standard deviation

in the results equal to 2%, and the fact that no specific bias (too high values or too low)

may be assigned, and hence no correction can be applied, the influence quantity f may

now be specified asf= 0, u(f) = 0,02.

It is important to list all possible sources of influence and quantify them in a form that

may be expressed in a model function. Whether that needs to be in absolute or relativequantities, with one or another unit (remember also scaling factors in the model func-

tion; e.g. not mixing mV and V and similar!) depends of course of the situation, but a

decision on some specific quantity is necessary. The method above may help in that re-

spect, when formulating a specific exaggeration, and that will then aid in the proper in-

clusion into the model function.

Second, it is not necessary to start by formulating the model function directly in terms

of the quantity to be determined = an expression with all the input quantities. Write

up the physical relations, add the appropriate influence components and then perform

the algebra to achieve the direct functional form. E.g. say we wish to determine the

mass of water in a bottle and we need to take into account the thermal expansion of wa-ter and bottle given that the temperature is different from a reference temperature. The

physical volume in the flask equals the physical volume of the water, hence we have the

physical relations, and the result of the derivation:

( )

( ) ( ) ( )( )TVTTVmVmTVTV

waterglassalnoflaskwaterwaterglassalnoflaskwaterwater

waterwaterwaterwateralnowaterglassalnoflask

+++=

=+=+

11/1

,11

min,min,

min,min,

5.2 Generic models

Many measurement processes are covered by three generic measurement methods: 1) di-rect measurement, 2) substitution, and 3) balance or difference measurement.

For each of these three types, it is possible to set up a generic model function

Direct measurement

Direct measurements come in two flavours: generation of a known signal onto a measure-

ment instrument (DUT = device under test), or determination of an unknown signal on a

calibrated indicator. Calibration of either device is typically performed using the comple-

mentary method.


10/33

10

6406 HDJ 2004 11 30

When calibrating a measurement instru-

ment, the measurand is the error of indica-

tion on the measurement instrument,

x_err.

The direct input quantities may be speci-

fied as:

The setting of calibrator/source,x_source

The indicated value on the DUT,x_indicate.

The first approximation of the model function can then be formulated as,

the error of indication equals the indication minus the setting of the source,

x_err=x_indicate x_source.

Now we may build up other influence factors to the model. If the indication (reading of themeter) is limited by instrument resolution or the need to interpolate an observation on a

analog scale, it is appropriate to add a quantity for resolution loss, x_res, containing the

part of the reading not accessible, e.g. the digits truncated off the display, or the inability to

determine the subdivision of a scale.

The value of the physical quantity generated by the source is not equal to the setting (dial

setting or an entered value), but there is some deviation, x_transfer, which has been deter-

mined in a prior calibration. Hence we must add this correction (or subtract a deviation,

depending on the how this information is obtained) to the value of the setting of the source.

The manufacturer of the source may furthermore have specified the performance of the in-

strument under different conditions, i.e. the change in the value of the generated physical

quantity with respect to the setting,x_stab, e.g. drift over time, influence of environmental

conditions and other influence factors.

Combining these we get the model function

x_err= (x_indicate +x_res) (x_source + x_transfer + x_stab)

One can then collect information on each of these input quantities:

x_indicate: From repeated observations we may find an average value and a standard de-

viation, which may then serve as our estimate for the mean and the standard uncertainty,

respectively. If the indication is stable, i.e. a constant value, we may take the uncertainty of

indication as zero (the resolution is taken care of by thex_res).

x_res: In the first case an appropriate assumption is that digit is lost due to rounding,

i.e. digit


11/33

11

6406 HDJ 2004 11 30

x_transfer: This quantity is the difference between the value of actual physical quantity

transferred to the DUT from the setting or nominal value of the source. This quantity

would be determined by a calibration of the source, and the information needed should be

available from a calibration certificate. To find the standard uncertainty we must examine

the calibration certificate to find the confidence interval or the coverage probability or the

coverage factor. For most accredited calibration certificates a coverage factor ofk= 2 isused, hence we find the standard uncertainty as the quoted uncertainty divided by 2:

u = U/ k.

x_stab: The influence of the environment or usage conditions to the output of a source is

often described by the specifications of the manufacturer. For a more detailed discussion

on the use of manufacturers specifications, see section 5.4.1. The most used strategy, given

no other information, is to assign a rectangular distribution to the manufacturers specifica-

tion, i.e. we obtainx_stab = 0, and u(x_stab) = specification / 3.

If determining the indication requires skill and hence depend on the operator, it may be ap-

propriate to include such a component, i.e. add x_operatorto x_indicate, with a value of

zero and standard uncertainty equal the experimental standard deviation of readings by dif-ferent operators under repeatability conditions.

The model function is that of a simple sum/difference, hence we obtain the resulting stan-

dard uncertainty as the root-sum-square of the standard uncertainties of the components.

The use of a calibrated instrument for di-

rect measurement follows closely the dis-

cussion above, with a slight rearrange-

ment of the components. The basic rela-

tion for a direct measurement, is to con-

sider the indication on the instrument

equal the value of the physical quantitypresented by the object being measured

(DUT). The basic model is thus:

x_dut=x_indicate

But again we have the influence quantities as described above: The limited resolution of

the indication, , the instrument error, as determined above, , and the influence of environ-

ment, usage and time on the instrument, . All of these quantities would simply add to the .

However, not that the definition of error is error = indication true value, hence the

error determined above would need to be subtractedthe indication. We arrive at the model

function

x_dut=x_indicate +x_res x_err + x_stab

The source and handling of the components is equivalent to that discussed above.

Substitution and scaling

Substitution and scaling: A transfer instrument is used to generate an indication equal to

(or with a small difference from) the measurement object and a known standard using an

assumption of a stability and linearity of the indicating instrument.

The quantity to determine is the value of the DUT,x_dut.

The basic quantities that determine the value ofx_dutare: the value of the reference,x_ref,

and the indications on the transfer instrument,x_i, refandx_i, dut.

INSTRx_errx_indicate x dut

x_stabx_res


12/33

12

6406 HDJ 2004 11 30

The basic assumption used in the substitu-

tion method is, that the relationship be-

tween the indication on the transfer in-

strument, when measuring the reference

and the DUT is equal to the relationship

between the value of the reference and theDUT.

The basic model function, i.e. the way we

calculate the value is derived from the re-

lation

refix

dutixrefxdutx

refix

dutix

refx

dutx

,_

,___

,_

,_

_

_==

Again, we may now expand the basic model function to take into account relevant influ-

ence factors. The value of the reference, which we assume to know from its calibration,

may still be subject to a change over time or operating conditions. This influence we assignas an additive valuex_stab.

When reading out the indication we may again be limited by the resolution of the instru-

ment or the readability of the scale. We assign the quantityx_res for the missing part of the

reading due to rounding. In this type of measurement setup we have two contributors to the

loss in resolution, one from the indication of the reference and one from the DUT. Note

that we may assume that the two quantities are independent; there is no immediate connec-

tion between the rounding in the two cases.

The final contribution we consider is the capability of the transfer instrument to perform

the transfer,x_transfer. One essential part of the quantity is the linearity of the scale, how-

ever more important in the present case is the variation of the deviation from linearity. Toillustrate this point, let us assume the relation between the reading on the instrument and

the physical quantity is x_true = x_indicate (1 + lin_err), where lin_err is the deviation

from the scale factor 1. For such an instrument we find that the ratio of two readings is un-

affected by the linearity error. Hence, we must quantify the deviations from linearity and

the stability over the measurement time. Thus, we assignx_transferas the change in linear-

ity over the part of the scale used, and will include it in our model function as a factor on

the ratio of the indications.

Adding the components to the basic model function we obtain:

( ) ( )transferxresxrefixresxdutix

stabxrefxdutx _1_,_

_,_

___ 2

1

++

+

+=

We collect the information for each of the input quantities:

x_i,ref and x_i,dut: From repeated observations we use the average and the standard devia-

tion. For a constant reading, u(x_i,..) = 0 and the uncertainty is handled byx_res.

x_res1,2: Again the appropriate contribution isx_res = 0 and u(x_res) = 1 digit / 2 3.

x_ref: The value and the uncertainty should be obtained from a calibration certificate.

x_stab: The expected change in the reference value over time or over usage conditions

may be found either from own analysis based on the results of previous calibrations, ex-

perience or from specifications from the manufacturer of the standard. Depending on thesource of the information different methods of arriving at the value (which would be 0 if

INSTRx_i, ref

x_i, dutx_dut

x_ref

x_transfer

x_stab

x_resx_res


13/33

13

6406 HDJ 2004 11 30

no correction is applied) and the standard uncertainty may be employed. E.g. a manufac-

turers specification may be assigned a rectangular distribution and treated accordingly, or

if a historic drift curve is used, the calculated standard uncertainty from the linear regres-

sion analysis is appropriate.

x_transfer: The transfer component will at best be characterised by the short term repro-

ducibility of the indicating instrument. Thus it may be characterised by performing re-

peated measurements. If more information is available, such as a specific relation for the

short term stability of the indicating instrument, this should be taken into account. In case

even slightly different values are read, the non-linearity of the scale should be considered,

either from a calibration or from information such as the manufacturers specification.

The model function is unfortunately not a simple sum/difference form hence other calcula-

tion tools must be taken into use, or calculated in steps by evaluating the various sums

separately and combine the factors using relative uncertainties.

Balance or difference method

The unknown quantity is balanced againsta known reference, with a small differ-

ence nulled out or with an absolute meas-

urement of a small difference.

An unknown quantity is balanced against

a known reference and a small difference

is measured or nulled out using an ad-

justment. The measurement principle is

similar to that of substitution, except the

DUT and reference will typically be physically connected, and the indicating instrument

will be used on its lowest range.

The quantity to determine is the value of the DUT,x_dut.

The basic quantities that determine the value ofx_dutare: the value of the reference,x_ref,

the indication on the null instrument,x_null and the value of the adjustmentx_adjust.

The basic model function when balanced, i.e. the way we calculate the value, is found as

adjustxrefxdutx ___ +=

Again, we may now expand the basic model function to take into account relevant influ-

ence factors. The value of the reference, which we assume we know from its calibration,

may still be subject to change over time or operating conditions. This influence we assign

as an additive valuex_stab.

When the null indication is achieved, we may again be limited by the resolution of the in-

strument or the readability of the scale. We assign the quantityx_res for missing part of the

reading due to rounding.

The measurement of the null is also subject to uncertainty. If the instrument is used as a

true null detector by applying an adjustment signal x_adjustto achieve a null reading, our

x_null should reflect the ability to achieve a true null, e.g. if the null instrument is a volt-

meter, a comparison with a short circuit is appropriate. If polarity reversal is possible, the

null value may be derived from the forward and reverse polarity readings.

Finally, thex_adjustmay be either the direct difference reading on the null instrument if noadjustment is performed, or the value of the additional signal we apply to achieve a null.

INSTRx_nullx_dut

x_ref

x_adjust

x_stab

x_res


14/33

14

6406 HDJ 2004 11 30

The demands on the relative uncertainty ofx_adjustare much less stringent than that of

x_ref; if, say,x_adjustis 1 % ofx_refand its relative uncertainty is 1 %, the overall rela-

tive uncertainty contribution becomes 10-4.

Adding the components to the basic model function we obtain:

( ) )__(____ resxnullxadjustxstabxrefxdutx ++++= We collect the information for each of the input quantities:

x_ref: The value and the uncertainty should be obtained from a calibration certificate.

x_stab: The expected change in the reference value over time or over usage conditions may

be found either from ones own analysis based on the results of previous calibrations, ex-

perience or from specifications from the manufacturer of the standard. Depending on the

source of the information different methods of arriving at the value (which would be 0 if

no correction is applied) and the standard uncertainty may be employed. E.g. a manufac-

turers specification may be assigned a rectangular distribution and treated accordingly, or

if a historic drift curve is used, the calculated standard uncertainty from the linear regres-

sion analysis is appropriate.

x_adjust: The additional signal applied may come from a source, and hence handled simi-

lar tox_ref. Ifx_adjustis a direct measurement of a small difference, we may thus need the

manufacturers specification for the range used and the value read.

x_null: From repeated observations we use the average and the standard deviation. For a

constant reading, u(x_i,..) = 0 and the uncertainty is handled byx_res.

x_res: Again the appropriate contribution isx_res = 0 and u(x_res) = 1 digit / 2 3.

The model function is again a simple sum/difference and the root-sum-square of the uncer-

tainty contributions gives the standard uncertainty of the result.

5.3 Repeatability - reproducibility

Often in testing, series of trails are used to validate the measurement set-up, the method

employed as well as in the measurement process itself. Employing statistical methods on

the final test results is a powerful method of obtaining an overall estimate of the total un-

certainty. However, the requirement for this to be true is that the influence quantities are

varied suitably over their full range during such a series of trails.

The ISO 5725 series of standards cover much of the detail on the design of experiments

and the analysis. The basic idea is to quantify from each level of repetition, the variance of

one or more influence quantities in terms of a standard deviation. From the results of twoor more levels, it is then possible to quantify the consistency of the assumption on the

number of independent influence quantities.

This process of determining repeatability and reproducibility may be summarized with the

termprecision. Secondly when it becomes important to consider comparability outside the

laboratory, the concept of trueness, or the determination of bias, must be taken into ac-

count.

At this point it is important to stress:

Precision, i.e. repeatability and reproducibility, is a measure for the ability of the lab togenerate consistentresults, and can be estimated from internal sources.


15/33

15

6406 HDJ 2004 11 30

Trueness or bias, is a measure for the ability of the lab to generate correctresults orcomparable results, and therefore, must be estimated from external sources: calibra-

tion, reference materials, intercomparisons, etc.

5.3.1 Basic assumptions and use of nested designs

It is important to stress, that the methods described in ISO 5725 and ISO 21748 are

based on a pragmatic or empirical model function and not model functions based on the

physics of the measurement procedure. There is no contradiction with the philosophy of

the GUM and the methods are fully applicable for this type of treatment.

The empirical model can be stated as (ISO 21748)

( ) excBmy ii ++++= where y is the observed value, m is the ideal or true (unknown) value, is the in-trinsic method bias, B is the laboratory bias and e is an error term. The sum of cixi

represents other effects than those giving rise to andB.Hence, m in this treatment could be the reading of the (complex) measurement equip-

ment in the form of a black box, which we attribute to the quantity we wish to assign

a value to. The parameter is the bias or error the particular method of measurementhas been assigned or that one must assume exists when comparing results based on

(very) different measurement principles or similar. The parameterB is also a bias or er-

ror, but related to the laboratory implementation of the method, i.e. if several laborato-

ries implement the same measurement using the same method, the parameterB would

signify the dispersion expected; i.e. the method reproducibility as implemented in the

laboratory. In some cases it may not be possible or relevant to distinguish between and

B. The terms cixi are physically modelled influence factors. The formulation used al-lows one to, e.g. take environmental influences into account, say it is assumed the tem-

perature in the laboratory affects the result with 1 %/C (i.e. the sensitivity factor), one

would then be able to take into account the uncertainty in temperature combined with

sensitivity factor. Finally, e signifies the pure random errors, or the repeatability term

(an average value or other statistically derived value would be part ofm).

The assumptions in the treatment are, that the expectation ofB and e is zero (hence no

correction is applied for the laboratory bias!), and have variances L and r respec-tively. The quantity e basically describes the repeatability of the measurement, whileB

signifies the reproducibility under ideal conditions.

The reproducibility observed would be the combination 22rLR += .

The whole idea of the exercise is to find from the observed reproducibility and the ob-

served repeatability to find whether the ideal reproducibility is significant or not, and

hence if there are contributors to the total uncertainty that must be described separately.

Given two sources of variability, the purpose of the nested design experiments and

ANOVA is to determine the significance of one over the other. Letx_A be a component

that signifies the within-day variability of some measurement results, and letx_B be a

component that signifies the between-day variability. How, and to what extent can we

determine the significance ofx_B and does it make sense to try and split the observed

dispersion into these two components?


16/33

16

6406 HDJ 2004 11 30

Lets assume a balanced nested design. Each day we perform Nmeasurements. The ex-

perimental variance 2As of one of the within-day results would be an estimate of the

variance of the individual measurements. The average of the within-day results would

then have an estimated variance of NsA2 , where N is the number of measurement

within a day. Hence, when we calculate the observed variance of the between-day re-

sults, 2as , each result being the average of the results of that day, we should expect a

value compatible with NsA2 . If the observed 2as is significantly larger than this, we

must assume there is a contributorx_B with variance2

Bs to the variability and which we

have not taken into account, so that 222BAa sNss += .

To test for significance, an f-test may be performed, by comparing the critical fvalue

with the ratio of two estimates of 2As . One we obtain by pooling the2

As values, which

for the balanced design is the average of these, Mss Ab = 22 , whereMis the numberof days we conduct our experiment; this estimate has (N 1)Mdegrees of freedom the

other as 2asN , which hasM 1 degrees of freedom. We calculate the ratio

22

ba ss and

compare to the critical value F0,95(M1, (N1)M)1. E.g. if we do 5 measurements each

day for 10 days, the critical value is F0,95(9, 40) = 2,12.

If the ratio is smaller than the critical value, we cannot conclude that 2as is significantly

larger than NsA2 , hence there is nox_B component.

If the ratio is larger than the critical value, we can conclude that 2as is significantly lar-

ger than NsA2 , hence there is ax_B component. We may even estimate the magnitude

of 2Bs as Nsss AaB

222 = (but be careful if these numbers are close; remember the uncer-tainty of the variance estimates!)

Thus, appropriate nested designs allow us to extract information of separate uncertainty

components based on an empirical model and statistical treatment of the measurement

data. However, the method has its limitations. A simple systematic effect such as drift in

the measurement value, e.g. caused by a change in a reference standard or the measure-

ment instrument, cannot be distinguished from purely random sources. Hence, it is im-

portant, before relying on empirical models, to examine the measurement problem from

a physical-technical standpoint, and incorporate the influences identified into the model

function. The general expression above allows partly taking such physically modelled

effects into account through the terms cixi.

5.3.2 Intercomparison data

From the discussion above, it is found that the characterisation of repeatability and in-

house reproducibility, together with the technical-physical characterisation of the meas-

urement problem may give an overall estimate of the total uncertainty. However, it is

not until a comparison is made with an outside source that a systematic laboratory bias

may be uncovered.

In order for the laboratory bias, i.e. the systematic difference between the laboratory in

question and the outside world, may be stated to be in control, the laboratory must

1 In Excel, the F-test critical value can be generated as FINV(1-p, 1, 2), wherep is the confidence level (e.g.95%), and 1 and 2 are the degrees of freedom of the variance estimates being compared.


17/33

17

6406 HDJ 2004 11 30

establish that its reproducibility with variation of several of the influence factors cov-

ered by the statistical treatment is within the standard deviation of the between-

laboratory results of a comparison. Unfortunately, intercomparisons in the field of LVD

testing are so infrequent and with a small number of participants, that a good statistical

basis for this evaluation is very difficult to obtain.

Another method of establishing the magnitude of any bias, is the use of certified refer-

ence materials. Again, in the field of LVD testing, such items or materials are usually

not available.

5.4 Other basic tools

To generate a realistic uncertainty budget many sources must be taken into consideration.

However, many of these a common to measurement situations across the fields of metrol-

ogy. Below are given some examples of such effects that may be treated in the same way

independent of the field of measurement.

5.4.1 Translation of instrument specifications

Measurement instruments are typically accompanied by a set of specifications, indicat-

ing the performance to be expected by the instrument under specific conditions of use,

as determined by the manufacturer. These are often given as direct or relative limits of

the error of indication or generation of the measurement quantity in question. However,

there are almost as many ways of indicating the performance of instruments, as there are

manufacturers.

First, it is necessary to determine to which level of confidence the specifications are

given. In few cases statements such as 99% confidence or 95% confidence are

given. If it assumed, that the specification may be interpreted as a quantity with a Nor-mal distribution, a 99 % confidence interval corresponds to a coverage factor ofk= 2,58

and a 95 % interval corresponds to k= 2,58. To use the specification one would divide

the specification limit with the appropriate k-factor to get the standard uncertainty.

However, often no such indication is given and it is most appropriate to assume a rec-

tangular distribution for the limit, and hence divide by a factor3.

Second, it is necessary to determine which limitations apply to the specifications given.

Do they cover all secondary settings (e.g. integration time, filter settings), or are there

additional terms which must be added, e.g. for measurement of resistance, specifications

are often quoted for four-wire measurements, and an additional term is quoted for two-

wire measurement.

Third, in many cases, specifications are often quoted in not-so-rigorous language, and

caution must be applied in translating these into more strict mathematical form, includ-

ing separating absolute value terms and relative value terms. In some cases, absolute

value terms are even given as a relative term of a range or maximum value, which may

not be the possible maximum reading for a range, and even visa-versa. E.g. for some in-

struments a range may be called 1, but has a maximum reading of 2 and the con-

stant contribution for a measurement on that range is to be calculated from 2 times the

range uncertainty.

An example is a common statement such as a table heading (% reading + % range)

and table entries for specific measurement ranges, e.g. 100 mV 0,005+0,004. Thisshorthand would then be interpreted for a reading of V, as the interval (510-5 V +


18/33

18

6406 HDJ 2004 11 30

4 V). With the assumption of a rectangular distribution, we achieve the standard uncer-

tainty u(V) = 2,910-5V+ 2,3 V.

5.4.2 Calculation, use and uncertainty of calibration curves

Under many measurement conditions, some transducer is used to convert from onephysico-chemical property into a property, say voltage, resistance or similar, that is

more easily handled and measured. To characterise the transducer and allow the transfer

back and forth the two properties, a calibration curve or transfer function is needed.

The calibration curve or transfer function is in some cases given by a parameterised, but

standardised expression, e.g. the R(T) for resistance thermometers and thermistors, or

the V(T) for thermocouples.

A calibration curve will typically be

mapped by performing a calibration: ap-

plying a series of known signals and re-

cording the responses. From the valuesand standard uncertainties of the inputs

and outputs, and appropriate parameterised

calibration curve may be determined, by

determining the value of the parameters

and their covariance (because the parame-

ters are all determined from the same set

of inputs, they will be correlated, thus we

must derive and use the full covariance

matrix of the parameters). With the pa-

rameters and covariances in hand, we may

for a given observed output derive the ap-propriate input and its standard uncertainty.

Hence the starting point is a set of applied stimuli {xi , u(xi)}, the measured responses

{yi , u(yi)} and an empiric or physical bondf(x,y) = 0.

The simplest transfer function is a linear relation,f(x,y;a,b) = a+ bx y, with parameters

a and b.

Linear regression,y =a +b x

If the uncertainties in x are so small that we may ignore their contribution, we obtain

the standard formulas (following Ref. 7):

( )2xxx

yxxy

xyxyxx

SSS

SSSSb

SSSSa

=

=

=

where

==

===

==

===

N

i i

ii

xy

N

i i

i

xx

N

i i

iy

N

i i

ix

N

i i

yu

yxS

yu

xS

yu

yS

yu

xS

yuS

1 21 2

2

12

12

12

)()(

)()()(

1

x

y

Fig. 1 Calibration curve

The red points signify the calibration points,

known x with observed y, giving the green

calibration curve Given an observation (blue)

we derive the stimuli oran e .


19/33


20/33

20

6406 HDJ 2004 11 30

they-values in the vectorY, then the solution a to the least square estimation (the esti-

mate ofa) becomes

CaaXXCYXCa === ),(.)(where, 111 TTT u

The fitted values are aXY = and a predicted value for an input becomes ax =y withT

yu xCx=2)(

Linear regression in matrix notation

How does this look in an implementation? For the linear regression above we have a =

(a, b) and x = (1,x). With a data set of {xi,yi, u(yi)} without correlations between they-

values we have

=

=

=2

2

111

)(00

00

00)(

,,

1

1

1

NNN yu

yu

y

y

x

x

OMM YX

We then get (using the notation above)

( ) ( )

=

==

=

xy

yT

x

xxxT

xxx

xT

S

S

SS

SSC

SS

SSYXXXXX

1111 ,1

and finally

( )

=

=

==

SS

SSu

SSSS

SSSS

S

S

SS

SS

x

xxxT

yxxy

xyxyxx

xy

y

x

xxxT 1),(,11

1 aaYXCa

which is the same result as quoted above. If the y-values are correlated, the matrix

above is replaced by the general covariance matrix fory.

The inclusion of uncertainties in the x-values can be included by replacing the covari-

ance matrix by

=+=2

21

21

2

1

2

)(

),(

),()(

,

Nxu

xxu

xxuxu

b

LM

MO

L

xxy

For a polynomial expression we would have something like x = (1,x, x2, x3, ) and the

procedure to derive the formulas and expressions similar to that shown above.

Using the general linear least squares

We may obtain the combined uncertainty when using the calibration curve to look up

the value from a measured output y including its uncertainty u(y). One requirement is

that they is independent from they-values used to determine the calibration curve and

hence not correlated with the calibration curve parameters.

Let us for simplicity assume that x = x(t), e.g. x(t) = (1, t, t2); e.g. ifa = (R0,A,B) this

could be the calibration curve for a resistance thermometer, with tas temperature andy

as resistance:y = a x =R0 +At+Bt2.

The general formula for the least squares solution above gives


21/33

21

6406 HDJ 2004 11 30

( )( ))(,

)()()()(

2

22

tyt

yutttu

T

T

xaxa

Cxx=

+=

For the temperature curve above, the denominator term becomes

( ) tBAtBARt TT 2)2,1,0(),,( 0 +== xa

The calculation of the covariance matrix C is in this case a bit more complicated, but

easy to handle numerically. Such least squares algorithms can easily be implemented in

a spreadsheet program such as Microsoft Excel, which supports matrix algebra.

The inclusion of uncertainties in x may be included in the determination of the model

parameters, as was shown for the linear regression above. It requires the scaling ofu(xi)

ontoy using the first approximation ofa. In the case where x = x(t) and we assume no

covariances, one would replace u(yi)2 by ( )[ ] 22 )()()( iii tuttyu + xa .

Non-linear general least squares

If it is necessary to be able to take both uncertainties in x andy into account, e.g. if theuncertainties in the independent variable are significant, numerical methods must be

used. An algorithm for such full calculation is given in Ref. 8. The method described

there allows also for implicitly given model functions, i.e. general f(x, y) = 0, and de-

scribes also how to test for the consistency of the fit obtained.

5.4.3 Drift

Most material standards are affected by internal and external sources, which changes

their characteristics over time. Hence the value represented by the standard changes

over time. In many cases the change may be sufficiently small and the change is insig-

nificant. Regular calibration may be used to quantify this change and one may be able topredicted the drift between calibrations. Thus, the change may be compensated and/or

the knowledge of the time-dependence can be used to fix calibration intervals; if the

standard proves stable enough, longer intervals are possible and visa-versa.

The most common method is prediction from linear regression. Hence the methods and

formula described in the section on calibration curves may be applied to drift as well. A

drift component would then be handled as an additive contribution to the value of the

standard, with a standard uncertainty as calculated from the linear regression.

5.4.4 Handling ill-defined measurement conditions

If the variability of the test results is caused by a variation in the measurement condi-

tions, this variability should be quantified by including the influence quantities into the

modelling of the measurement situation. This may be done by direct physical modelling

or by including a empirical model quantity of the form cixi.

However, it could be argued, that the variation due to ill-defined measurement condi-

tions is not the responsibility of the testing laboratory, but should be part of the method

validation studies. That is, such studies should be performed in the process of develop-

ing the test method, and should be information provided in the standard. It is the respon-

sibility of the laboratory to evaluate measurement uncertainties of the measurements

performed. It would then be appropriate to compare the measurement uncertainty

against such method uncertainty in the validation of a specific test setup, and in theprocess of intercomparisons.


22/33

22

6406 HDJ 2004 11 30

Alternatively, in order to facilitate comparison of test results, the actual measurement

conditions should be specified when reporting the test result. If differences are found,

e.g. in a comparison, and it is observed that measurement conditions vary, it would be

appropriate for an investigation as to the magnitude of the influence factor. If, however,

the dispersion is much smaller than the test limits, this will only be an academic exer-

cise and not influence the conclusions or decisions drawn from test results.

6 Examples from electrical testing

6.1 Measurement of temperature using thermocouples

In the next two examples of basic LVD testing, the measurement of temperature using

thermocouples is the basic physical measurement used. We therefore present here an ex-

ample of an uncertainty budget for such basic measurement.

Note, however: This example is constructed to present some of the considerations, inter-

pretations and arguments in setting up the model function and assigning uncertainties, andshould not be taken as a direct example on what uncertainty to assign a thermocouple tem-

perature measurement!

Thermocouples are made from two wires of dissimilar metals brought together in a joint.

Due to the thermoelectric effect and different coefficient for different metals, different

voltage drops will occur along the two wires from the common joint, and a voltage differ-

ence can be measured at the open end of the thermocouple wires. Given a thermocouple of

a specific type, there are standard expressions for the voltage-temperature relationship, e.g.

for type K thermocouples a 9th degree polynomial expression plus an exponential term is

the reference expression. An inverse expression, giving the temperature as a function of the

voltage difference, is also available stated in the same form as a 9 th degree polynomial. The

conversion error is typically 0,05 C.

To avoid the additional thermal voltages when connecting the thermocouple to the voltme-

ter, an additional thermocouple is inserted held at a standard temperature, 0 C, allowing

for the same junction type at the voltmeter. Alternatively, a correction based on the tem-

perature of the junctions at the voltmeter is applied. For the purpose of the measurements

for heating testing, we do not seek the best uncertainties, and will assume an internal refer-

ence junction compensation is used.

The basic for the measured temperature thus becomes

T= t(Vt) with Vt= Vm V0,

where t(Vt) is the reference conversion curve, Vm is the thermocouple voltage differenceand V0 is the compensation voltage.

Assume we wish to measure 950 C using a type K thermocouple. The reference voltage is

Vref = 39,3139 mV and the sensitivity c950 = dT/dV= 25,3077 C/mV. We perform a first

order expansion of the reference curve and obtain

T= 950 C + c950(Vm V0 Vref).

We may then begin to expand the model function based on other contributions. First, the

deviation of the thermocouple in question to that of an ideal thermocouple. We will state

that as the additional voltage generated by the thermocouple at the temperature T, Vtc.

Second, for the voltmeter we have its measurement errorVm from calibration, the resolu-tion VI, the drift and operating conditions from the specification VD, and the error in the


23/33

23

6406 HDJ 2004 11 30

compensation voltage (internally based on the reference curve and the measured tempera-

ture on the connection block) V0. Finally, we have the conversion error itselfTc.

To expand the model function we may argue: If the meter reading is perfect Vm and the

temperature is T, the thermocouple generates too high a voltage Vtc. Hence we must sub-tract this from the reading. Also the reading presented is corrected for the cold junction

voltage, so V0 is in reality included in Vm. Next, the reading is not perfect, but is in error by

Vm as determined by calibration, hence we must also subtract this from the reading. Alsothe drift and operating conditions give too high a reading, so we must subtract VD. Themeter compensates the cold junction by calculating the cold junction voltage and subtract-

ing it to the measured voltage, however, this voltage is in error by V0, hence we must sub-tract that from V0 i.e. add to Vm. Finally, the imperfectness of the temperature-voltage con-

version gives an errorTc in temperature, which we must subtract the reference tempera-tur. With these arguments, we arrive at the expanded model function

T= 950 C + Tc + c950(Vm + V0 Vtc Vm VDVref).

Now we examine each of the components and derive the value to assign and the standarduncertainty.

Tc: The reference curve specification gives the conversion error as 0,05 C. We will as-sume there is equal probability in this range, and assign a rectangular distribution function.

Hence we obtain the value Tc = 0 C, with standard uncertainty u(Tc) = 0,05 C/3 =0,029 C.

Vm: During measurement we acquire 10 readings. We calculate an average 39,151 mV and

a standard deviation 0,011 mV. Hence, Vm = 39,151 mV, u(Vm) = 0,011 mV/10 =0,0034 mV.

V0: The instrument specifies that the cold junction compensation is done with an accuracyof 0,5 C. Converted to voltage, this corresponds to 20 V. We choose to understand this,as the compensation voltage is accurate in the interval 20 V to +20 V. Hence, we as-

sign a symmetric rectangular distribution and get V0 = 0 V and u(V0:) = 20 V / 3 =11,5 V.

Vtc: The thermocouples are made in the laboratory using good quality thermocouple wire.A batch of thermocouples are made and calibrated in an oven and measure their voltage

output in a specialised setup. It is found at 950 C that the average voltage generated is

39,250 mV with a standard deviation of 21 V. The uncertainty of the generated tempera-

ture is estimated as 0,2 C measured by a standard thermometer, and the uncertainty of the

voltage measurement is 5 V. The average voltage error is (39,250 mV 39,314 mV) =

0,064 mV. The 0,2 C temperature uncertainty corresponds to 8 V uncertainty in the gen-

erated voltage, hence the combined uncertainty of the voltage error is (212 +52 + 82) V =23 V. Hence, for a thermocouple from this batch we use Vtc = 0,064 mV, u(Vtc) =0,023 mV.

Vm: The meter has been calibrated at an accredited laboratory. At 40 mV on the 100 mVrange, the error was found to be 6 V with an uncertainty of 5 V. The uncertainty is

stated as a 95% confidence value, i.e. overage factor k= 2. Hence we use Vm = 6 V,u(Vm) = 5 V / 2 = 2,5 V.

VD: The specification of the meter is stated in the form 0,02 % reading + 0,03 % range.

The meter is used on the 100 mV range, thus 0,03 % range is 30 V. The reading is 40 mVand 0,02 % reading is 8 V. Added we get 38 V. This is the manufacturers estimate of


24/33

24

6406 HDJ 2004 11 30

the absolute limits of variation of the error for 1 year and under the prescribed measure-

ment conditions. Hence, we interpret this as a symmetric rectangular distribution, (0 38)

V, i.e. VD = 0 V, u(VD) = 38 V / 3 = 22 V. The resolution is much lower than thisterm, hence we regard this as included in the specification.

We will evaluate the uncertainty in steps:

The voltage terms form a sum and difference expression; hence we may combine the un-

certainties directly by root-sum-square. We thus obtain V= 0,105 mV, and u(V) = 34 V,

dominated by the meter specification and the thermocouple error.

We may note, that the uncertainty found for the reading is a factor of 10 lower, hence does

not contribute significantly. We may therefore without loss of accuracy reduce the number

of readings to, say, four; we have an estimate of the standard deviation, 11 V, and the av-

erage of four readings would give a standard uncertainty of 5,5 V. With this value the

combined voltage uncertainty remains 34 V within the rounding.

The combination of the voltage and the scale factor is a product, but we have not assigned

an uncertainty to the scale factor because the conversion uncertainty is covered by Tc.Hence the uncertainty in voltage scales directly to an uncertainty in temperature, and we

obtain Tc = c950V = 25,3077 C/mV (0,105) mV = 2,65 C and u(Tc) = c950u(V) =

25,3077 C/mV 0,034 mV = 0,86 C.

The final result is thus T= 950 C + 0 C 2,65 C = 947,35 C. The standard uncertainty

is the combination of the converted voltage uncertainty and the reference curve uncer-

tainty. The latter is more than a factor of 10 lower, thus we may safely ignore it and obtain

u(T) = 0,86 C.

6.2 Heating in Black Test Corner

The tests specified in EN 60335-1 relates to heating of household appliances. The test re-

quires the measurement of temperature on and close to the surface of the appliance under

normal operating conditions.

A black test corner is implemented as an array of thermocouples connected to Cu disc

mounted on wood plates. By using a number of plates, a corner with dimensions of the or-

der 1 m x 1 m x 1 m can be constructed. This arrangement is typically used to measure side

and bottom temperatures of large appliances.

If we look at the measurement situation, we again must measure the voltage of the thermo-

couples. The voltage of the thermocouples is related to their temperature, but we are inter-

ested in the temperature at the surface of the appliance. Thus we must take into account the

difference in temperature from the thermocouple and the position of the surface.

Alternatively, we may apply a temperature to the thermocouple array and thus indirectly

take into account the transfer, however, we will then be subjecting the thermocouple wire

to a different temperature distribution under calibration than under use.

Hence we must carefully specify our measurand: Do we report the temperature of the

thermocouples in the arrangement specified by the standard, or do we try to evaluate the

temperature of the surface of the appliance?

In the first case, our measurement model is clear: We have the voltage measurement, trans-

lated into temperature using the standard transfer curve, combined into the variable TV, the

calibration measurement of the thermocouple voltages when subjected to temperaturesgenerated in an airbath, combined into the variable Tc, and, if we treat all thermocouple


25/33

25

6406 HDJ 2004 11 30

alike, and do not perform individual corrections, we have the deviation of the thermocou-

ple voltage from the reference voltage, again scaled to temperature, Tc.

Our model function:

T= TV+ Tc + Tt , with standard uncertainty222 )()()()(

tcV TsTuTuTu ++=

Given, for example, the voltage measurement uncertainty and the reference curve conver-

sion has specification 0,4 C, we use u(TV) = 0,4 C / 3 = 0,23 C, the total calibrationuncertainty of the temperature bath u(Tc) = 0,04 C, and the observed standard deviationof the individual thermocouple temperatures, u(Tt) = s(Tt) = 0,69 C. Combined weachieve u(T) = 0,73 C, and we see the dominating contribution comes from the spread of

the thermocouples. Because this component is near a Normal distribution, we may use

coverage factor k 0 2 and achieve U(T) = 1,5 C.

If we were to specify our measurand as the surface temperature, we would need to evaluate

the possible temperature difference between our thermocouple and the surface, across the

airgap present. One way of evaluating this would be to subject the thermocouple plate to acalibration plate. The difference may then be measured and an appropriate component

added, with the uncertainty evaluated from, e.g. the maximum deviation combined with the

temperature measurement uncertainty.

6.3 Glow wire test

The objective of a glow wire test (EN 60695-2), is to investigate the non-flammability of

insulation materials. A piece of material, typically plastic, is subjected to a hot wire, typi-

cally NiCr, heated by an electric current. It is observed that the material does not ignite af-

ter contact with the wire for a specific time at the specified temperature. The temperature is

measured by a thermocouple in the wire near the point of contact.There are some physical dimensions (diameter of wire and penetration) and mechanical

specifications (pressure applied) that must kept for the test procedure to be fulfilled. Also

the construction of the thermocouple and its integration into the glow wire is specified in

the standard.

The measurement itself is specified as the temperature of the glow wire. The specification

for the thermocouple to use is 0,4 % T(in C), and a verification measurement specified is

the generation of the melting of silver with a tolerance of 15 C; a somewhat relaxed re-

quirement in comparison.

Let the thermocouple voltage be read by a voltmeter with an internal calculation of a type

K thermocouple temperature. The 1-year specification of the instrument is given in theform:

The specification for a Type K thermocouple is thus 1,0 C. We interpret this as a rectan-

gular distribution of the instrument error of (0,0 1,0) C, hence the instrument error is

TV = 0 C, u(TV) = 1 C / 3 = 0,58 C.


26/33

26

6406 HDJ 2004 11 30

A certified class 1 type K thermocouple, the tolerance (above 375 C) is given as 0,4 % of

the temperature. Hence, when the voltage generated equals the reference value at tempera-

ture T, the true temperature is in the interval T (1 0,4 %). The temperature indicated on

the display of the meterTI will be that of the reference equation, the error due to the probe

id thus Tc = 0 C, with u(Tc) = 0,4 % / 3 TI . The value ofTIwill be given by the read-

ing, and let us assume a constant reading is obtained, i.e. s(TI) = 0. There will then be acontribution from the resolution of the display, a rounding of 0 0,05 C; hence TI = 0,u(TI) = 0,05 C / 3 = 0,029 C.

Our basic model function becomes:

T= TI+ TI+ Tc + TV

The uncertainty is derived from the simple sum/difference formula:

u(T)2 = (0 C)2 + (0,029 C)2 + (0,23 % TI)2 + (0,58 C)2,

varying from 2,1 C at 900 C to 2,4 C at 1000 C.

Because our dominating term is the tolerance of the thermocouple, which we assigned arectangular distribution, the correct coverage factor is in fact k= 1,65. The expanded un-

certainty at 950 C thus becomes U(950 C) = 3,7 C.

Is it necessary to evaluate uncertainty contributions from the measurement conditions to

the result?

No, it should not!

The laboratory must document that the conditions specified (mechanical properties and

dimensions) are fulfilled, but it would be the task of the standardization committee to

translate the measurement conditions into an overall method uncertainty. This method

uncertainty would then be the appropriate uncertainty to compare against in interlaboratory

comparisons.

The uncertainty reported by a laboratory for a quantitative test result should be the meas-

urement uncertainty obtained for the measurement.

6.4 EMC - Radiated Immunity Test (IEC 61000-4-3)

For EMC compliance an instrument must be immune to an applied electromagnetic field.

A test is performed where an instrument is placed in a electric field of known field strength

and the ability of the instrument to function is monitored. To fulfil the test conditions, the

field distribution must be uniform over the test volume within 6 dB, and the field strength

be known. Thus, the test conditions rely on a calibrated field probe and measurementequipment, and compliance with the test specification of a uniform field.

The measurement uncertainty budget below is based on the assumption that it has been

demonstrated during calibration that the 6 dB field uniformity has been achieved. Once the

expanded uncertainty has been calculated then the value is related to the standard severity

level (e.g. 3 V/m) to provide an increased test level, which accounts for the uncertainty in

the test configuration.

The mathematical model for the measurement process is assumed to be:

F = FSM+ FSAW+ PD + PAH+ FD + RS+ REUT

It should be noted that this model does not account for the unknown influence of the un-specified semi-anechoic chamber and the significant variations caused by reflections from


27/33

27

6406 HDJ 2004 11 30

the EUT. The contribution form the test set up will need to be evaluated and given a value.

In the mean time the 6 dB field uniformity restriction is a means to at least have some con-

trol over this contribution for an empty chamber. Thus, to include the lack of informa-

tion/characterisation of the measurement setup, the 6 dB requirement should be added to

the uncertainty forFD.

Sym-

bolQuantity Units Comment

FSM Field Strength monitor dBV As reported on the Field Probe calibration certificate

FSAWField Strength accept-

ability windowdB

Usually a set parameter to allow the software to ac-

cept a value for field strength that is within an ac-

ceptable window of the calibration level.

PDForward power Meas-

urement DriftdB

A function of the power meter or other device con-

nected to monitor forward power.

PAHPower Amplifier

Harmonics

dBContribution due to inclusion of amplifier distortion

in power meter reading.

FDEffect of field distur-

bancedB

Possible effects due to table and/ supporting struc-

tures.

Mismatch Not applicable.

RSMeasurement System

RepeatabilitydB

The repeatability of the measurement system is de-

termined by obtaining the standard deviation of a

series of repeated readings on a stable EUT. It

should include typical variations that will occur in

normal testing.

REUT Repeatability of EUT dB

This will only need to be considered if the measured

result is close to the specification limit to the extent

that variations due to the EUT may affect compli-

ance.


28/33

28

6406 HDJ 2004 11 30

7 Appendix

Statistics and uncertainty primer

A stochastic variable or a random variable is a variable that may assume different values, atdifferent trials or outcomes. A stochastic variable, as it may appear in measurements, may be

fully characterized by a probability distribution function.

A distribution function is a function,f(x) describing the probability of occurrence of the out-

come of a random variable. For a continuous distribution, the probability that Xfalls in the

interval x < X< x + dx equals f(x)dx. For a discrete distribution, such as the outcome of a

throw with a dice,f(x) is a set of values. It is so that 1)( =xall

dxxf .

The expectation value (or mean value) of a distribution function is given by

= xall dxxfxXE )()(The variance of a distribution function is given by

( ) =xall

dxxfXExXV )()()(2

The GUM method relies on the basic concept:

The resultis an estimate of the expectation value and the standard uncertainty is the square-

root of an estimate of variance.

TheNormal distribution or Gaussian distribution has a special significance. It can be shown

(the Central Limit Theorem) that a combination of random variables with any well-behaved

distribution functions will approach the Normal distribution. Hence, in many physical proc-

esses, where sources from the scale of molecular motion and up affect the outcome, quantities

follow a Normal distribution.

The Normal distribution is described by two parameters, and , and the probability distribu-

tion function is given by

( )2

2

2

2

1)(),;(

==

x

exfxN .

For the Normal distribution, the expectation value isE(X) = and the variance is V(X) = 2

.

Combining uncertainties

Given a model function connecting input quantities nXXX ,,, 21 K and an output quantity Y

by a mathematical function ),,( 1 NXXfY K= , the propagation of uncertainties follows froma first order Taylor expansion off:

The result is given by ),,(

Date post:	02-Apr-2018
Category:	Documents
Upload:	lthyagu
View:	217 times
Download:	0 times

0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing

Documents