Greater Validity in Uncertainty Calculation 2004/artigo8-MC.pdf · Greater Validity in Uncertainty...

Greater Validity in Uncertainty Calculation

Maurice G Cox and Peter M Harris

National Physical Laboratory

Teddington TW11 0LW, UK

Abstract

The concern of this paper is with obtaining greater validity in uncertainty

calculation. The attitude is taken that for the formulated problem, i.e., for

a given model of measurement and prescribed probability distributions for

the values of the input quantities to the model, the uncertainty calculation

should be carried out in a way that fully respects this information. The

provision of the probability distribution for the value of the model output

quantity in terms of this information is known as the propagation of distri-

butions. From this distribution, a best estimate of the value of the output

quantity, and the associated standard uncertainty, and a coverage inter-

val for the value of the output quantity that corresponds to a stipulated

coverage probability can be determined. Although the framework for un-

certainty calculation provided in the Guide to the Expression of Uncertainty

in Measurement (GUM) is widely used, the conditions for it to be applied

in a valid manner are not always verified. The GUM uncertainty frame-

work and the use of analytical methods and a numerical method (Monte

Carlo simulation) are reviewed and compared in the context of regarding

these approaches as implementations of the propagation of distributions.

An example of co-ordinate transformation is used to illustrate the points

made.

Greater Validity in Uncertainty Evaluation

1 Introduction

The title ‘Greater validity in uncertainty calculation’ relates to obtaining

reliable evaluations of uncertainty given (a) an acceptable (physical) model

of a measurement and (b) uncertainty information relating to the values of

the input quantities to the model. In this regard, the problem of uncertainty

evaluation is considered to constitute two phases: problem formulation and

uncertainty calculation (section 2). Problem formulation involves establish-

ing the model and assigning probability distributions to the values of the

model input quantities. Uncertainty calculation involves (a) the use of this

information to derive a probability distribution for the value of the model

output quantity, a process known as the propagation of distributions, and

(b) from this distribution to infer the results commonly required. These

results typically consist of a best estimate of the value of the output quan-

tity, and the associated standard uncertainty, and a coverage interval for

the value of the output quantity that corresponds to a stipulated coverage

probability. Uncertainty calculation is the main concern here, but some

remarks are made concerning problem formulation.

The uncertainty framework in the Guide to the Expression of Uncertainty in

Measurement (GUM) [4] is the approach most commonly used for the eval-

uation of measurement uncertainty. Candidate approaches for uncertainty

calculation are considered (section 3). Three such approaches are covered

here, viz., analytical methods (section 4), the GUM uncertainty framework

itself (section 5), and a numerical method based on Monte Carlo simula-

tion (section 6). All three constitute implementations (to some degree of

approximation) of the propagation of distributions. The three approaches

are contrasted theoretically (section 7). In particular, the conditions for the

validity of the GUM uncertainty framework are highlighted.

Qualidade e Metrologia

Co-ordinate transformation (section 8), used in electrical, optical and di-

mensional metrology, provides an example of the application of the ap-

proaches. In particular, because analytical methods can be applied, at least

partly, in this instance, Monte Carlo simulation and the GUM uncertainty

framework can be compared in this case with the analytical results as a

reference.

2 The two phases of uncertainty evaluation

In the problem formulation phase of uncertainty evaluation a model of the

measurement is developed. It is necessary that a measurement be modelled

mathematically to the degree imposed by the required accuracy of measure-

ment [4, Clause 3.4.1].1 Probability density functions (PDFs)—rectangular

(uniform), Gaussian (normal), etc.—are assigned to the values of the model

input quantities, defined in terms of the parameters of these functions (e.g.,

central value and semi-width for a rectangular PDF, or expectation and

standard deviation for a Gaussian PDF), including correlation parameters

for joint PDFs. These PDFs are obtained from an analysis of series of ob-

servations [GUM 2.3.2, 3.3.5] or based on scientific judgement using all the

relevant information available [GUM 2.3.3, 3.3.5], [23].

Probability distributions in the form of PDFs obtained from prior knowledge

provide the most fundamental characterizations of the values of the quan-

tities of concern. For instance, if the only information available concerning

the value of a quantity is a lower and upper bound, it is appropriate to assign

a rectangular PDF with limits given by those bounds to the value [26]. As a

second instance, if the only information available is an estimate of the value

of the quantity and the standard uncertainty associated with that estimate,

1Subsequently, citations to clauses of the GUM are given as [GUM 3.4.1], for example.


it is appropriate to assign a Gaussian (normal) PDF with expectation equal

to the estimate and standard deviation equal to the standard uncertainty

associated with that estimate [26].

The model provides a functional relationship between the quantities about

which information is available (the input quantities) and the quantity about

which information is required (the output quantity). The input quantities

are denoted by X = (X1, . . . , XN )T and the output quantity by Y . The

model

Y = f(X) = f(X1, . . . , XN ) (1)

can be a mathematical formula, a calculation procedure, computer software

or other prescription.

The Xi may be mutually dependent, i.e., their values correlated, in which

case additional information would be needed to quantify the correlations.

There may be more than one output quantity, viz., Y = (Y1, . . . , Ym)T. In

this case the model is

Y = f(X) = f(X1, . . . , XN ),

where f(X) = (f1(X), . . . , fm(X))T, a vector of model functions. In full,

this ‘vector model’ is Y1 = f1(X1, . . . , XN ), Y2 = f2(X1, . . . , XN ), . . . , Ym =

fm(X1, . . . , XN ). The output quantities Yj would almost invariably be mu-

tually dependent, since in general each Yj would depend on several or all of

the input quantities.

A model with a single output quantity Y is known as a univariate model

and a model with a general number of output quantities Y as a multivariate

model.

For mutually independent input quantities and a single output quantity,

the uncertainty calculation phase can be summarized as follows. Given


the model Y = f(X) and the PDFs gi(ξi) (or the indefinite integrals,

Gi(ξi) =∫ ξi

−∞ gi(z)dz, the distribution functions) for the values of the input

quantities Xi, i = 1, . . . , N , determine the PDF g(η) (or the distribution

function G(η)) for the value of Y . The determination of g(η) is basic to

uncertainty evaluation: the best estimate y of the value of Y is taken as the

expectation (mean) of g(η), and the associated standard uncertainty u(y) as

the standard deviation of g(η), and a coverage interval Ip(Y ) for the value

of Y , corresponding to a stipulated coverage probability p, as the shortest

interval containing 100p % of g(η). The formation of g from f and the gi is

known as the propagation of distributions.

It is generally not possible to express g in simple or even closed mathemat-

ical form. Formally, if δ(·) denotes the Dirac delta function, for mutually

independent input quantities,

g(η) =∫ ∞

−∞

∫ ∞

−∞· · ·

∫ ∞

−∞g1(ξ1) . . . gN (ξN )δ(y − f(ξ))dξN . . . dξ1, (2)

where ξ = (ξ1, . . . , ξN )T [6]. Approaches for determining g(η) or G(η) are

addressed in section 3. Such determination ranges from being analytically

very simple to difficult or impossible, depending on the complexity of the

model and the PDFs for the values of the input quantities. Hence, ap-

proximations to g or G provided by approximate analytical approaches or

numerical methods are relevant.

Figure 1 illustrates the propagation of distributions, in which the PDFs (or

the corresponding distribution functions) for the values of the input quanti-

ties are propagated through the model to provide the PDF (or distribution

function) for the value of the output quantity.

When the Xi are mutually dependent, in place of the N individual PDFs gi(ξi),

i = 1, . . . , N , there is a joint PDF. The commonest joint PDF is the multi-

variate Gaussian PDF [22].


-

g3(ξ3)

-

g2(ξ2)

-

g1(ξ1)

Y = f(X) -

g(η)

Figure 1: Illustration of the propagation of distributions. The value of input

quantity X1 is assigned a Gaussian PDF g1(ξ1), the value of X2 a triangu-

lar PDF g2(ξ2) and the value of X3 a (different) Gaussian PDF g3(ξ3). The PDF

for the value of the output quantity Y is illustrated as being asymmetric, as can

arise for non-linear models when one or more of the PDFs for the Xi has a large

standard deviation.

The values of a general number of output quantities are described by a joint

PDF. The counterpart of coverage interval Ip(Y ) for the value of Y is a

coverage region Rp(Y ) of smallest volume for the value of Y . When the

joint PDF is multivariate Gaussian, Rp(Y ) is ellipsoidal.

3 Candidate approaches

All candidate approaches to uncertainty calculation can be regarded as hav-

ing a common starting point:

1. A model of the form (1) [GUM 4.1.1].

2. The value of each Xi characterized by a PDF [GUM 4.1.6].2

2If the Xi are mutually dependent, their values are characterized by a joint PDF.


The common endpoint is a PDF for the value of Y , from which the required

results can be obtained.3

Three approaches are considered: (1) analytical methods, (2) the GUM

uncertainty framework and (3) a numerical method in the form of a Monte

Carlo simulation. All three approaches are consistent with the GUM. The GUM

highlights PDFs as providing a basis for uncertainty evaluation [GUM 3.3.5].

Even greater emphasis is given today to this consideration, primarily be-

cause such a PDF describes the state of knowledge of the value of a quantity

in terms of the information available of that value. In particular, the JCGM

has provided a version for review of Supplement 1 to the GUM concerned

with numerical methods for the propagation of distributions.4 The GUM

uncertainty framework is the procedure that is widely used and summarized

in GUM Clause 8. Analytical methods and numerical methods fall in the

category of ‘other analytical and numerical methods’ [GUM G.1.5].

4 Analytical methods

Analytical methods to obtain g(η) or g(η) are ‘ideal’ in that they do not

introduce any approximation, but can be applied in relatively simple cases

only. A treatment of such methods, based essentially on the use of Formula

(2) is available [12]. Instances that can be so handled include additive

models, Y = c1X1 + . . . + cNXN , where the values of all Xi are Gaussian

or all are rectangular. In the latter case, unless N is small, the multipliers

ci must be equal and the semi-widths of the rectangular PDFs identical5 to

3The endpoint would be a joint PDF when there is a general number of output quan-

tities.4The JCGM signifies the Joint Committee for Guides in Metrology, the

body responsible for maintaining the GUM. The Web address of the JCGM is

http://www.bipm.fr/enus/2 Committees/JCGM.shtml.5In this case the value of Y has a PDF that is a B-spline with uniform knots [5].


avoid formidable algebra.

The case of a single input quantity (N = 1) is especially amenable to ana-

lytic treatment [20, pp57-61].

Example 1 A logarithmic transformation

Consider the model Y = loge(X) with the value of X having a rectan-

gular PDF with limits [a, b]. The application of the abovementioned

treatment gives

G(η) =

0, η ≤ ln(a),

(exp(η)− a)/(b− a), ln(a) ≤ η ≤ ln(b),

1, ln(b) ≤ η.

Figure 2 depicts such a rectangular PDF for the value of X and (right)

the corresponding PDF for the value of Y . This case is important in,

say, electromagnetic compatibility measurement, where conversions are

often carried out between quantities expressed in linear and decibel

units using exponential or logarithmic transformations [24].

Example 2 A linear combination of Gaussian distributions

Consider the model

Y = c1X1 + · · ·+ cNXN ,

where c1, . . . , cN are specified constants, and, for i = 1, . . . , N , the

value of Xi has a Gaussian distribution with expectation xi and stan-

dard deviation u(xi). The value of Y has a Gaussian distribution

with mean y = c1x1 + · · · + cNxN and standard deviation u(y) =

(c21u

2(x1) + · · ·+ c2Nu2(xN ))1/2.


1 2 3−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0 0.5 1

−0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Figure 2: A rectangular PDF for the value of the input quantity X and the

corresponding PDF for the value of the output quantity Y , where Y is related

to X by the model Y = ln(X).

Example 3 The sum of N rectangular distributions of the same semi-width

Consider the model

Y = X1 + · · ·+ XN

and, for i = 1, . . . , N , the value of Xi has a rectangular PDF with

expectation xi and semi-width a (and hence standard deviation a/√

3).

The value of Y has a PDF that is a B-spline of order N (degree N −1)

with expectation y = x1 + · · ·+ xN and standard deviation a√

N/3.

Example 4 The sum of two rectangular distributions of arbitrary semi-

widths

Consider the model

Y = c1X1 + c2X2,

where, for i = 1, 2, the value of Xi has a rectangular PDF with

mean xi and semi-width ai (and hence standard deviation ai/√

3).

The value of Y has a symmetric trapezoidal PDF g(η) with expecta-

tion y = c1x1 + c2x2, semi-width c1a1 + c2a2 and standard deviation


- Y�� A

AAA

Figure 3: The PDF for the value of a general linear combination Y =

c1X1 + c2X2 of two rectangular PDFs with arbitrary semi-widths.

{(c21a

21 + c2

2a22)/3}1/2. Geometrically, this PDF takes the form indi-

cated in Figure 3. It is trapezoidal and symmetric about its midpoint.

It would reduce to a triangle if a1 = a2.

Analytical solutions in some other simple cases are available [3, 12, 13].

5 The GUM uncertainty framework

The GUM provides a framework for calculating the uncertainty associated

with a measurement result, which uses a model of the measurement, the

law of propagation of uncertainty [GUM 3.4.1, 3.4.8], and the assignment

of a particular PDF to the value of the output quantity. Component parts

of the framework are:6

1. Estimates xi of the values of the Xi, taken as the expectations of

the PDFs for the values of Xi [GUM 4.1.1, 4.1.6].7

2. An estimate y of the value of Y , given by evaluating the model at the

estimates xi. Thus, y = f(x1, . . . , xN ) [GUM 4.1.4].8

6The terminology used here and subsequently differs a little from that in the GUM to

reflect that adopted by the JCGM.7When the values of the Xi have a joint PDF, the xi are the expectations of that PDF.8When a set of independent determinations of the value of Y is available, it may be

preferable to take y as their arithmetic mean [GUM 4.1.4, Note]. Doing so is a discrete

counterpart of taking y as the expectation of the PDF for the value of Y .


3. The standard uncertainties u(xi) associated with the xi, taken as the

standard deviations of the gi(ξi) [GUM 3.3.4].9

4. The standard uncertainty u(y) associated with y, given by the law

of propagation of uncertainty to combine the standard uncertain-

ties u(xi) [GUM 3.3.6].

5. A Gaussian PDF, with expectation y and standard deviation u(y),

assigned to the value of Y [GUM G].10

6. A coverage interval, corresponding to a prescribed coverage probabil-

ity p, for the value of Y , determined as y±U , where U is the product

of a coverage factor kp, obtained from the Gaussian or t–distribution,

and u(y) [GUM 3.3.7].

The GUM uncertainty framework can be regarded as constituting an ap-

proximate analytical method.

6 Numerical method: Monte Carlo simulation

It would rarely be a practical proposition to use the integral expression

(2) in section 2 as the basis for the numerical determination of g(η). A

multivariate quadrature rule11 would need to be devised that was capable

of delivering g(η) to a prescribed numerical accuracy for each choice of η.

Further, the quadrature rule would have to be applied at a sufficiently fine

set of η-values to provide g(η) adequately.9When the Xi are mutually dependent and hence have a joint PDF, there will also be

mutual uncertainties, or covariances, associated with the xi. These mutual uncertainties

are taken as the covariances of the joint PDF.10A t–distribution is used instead if the degrees of freedom associated with u(y) is

finite.11A quadrature rule is a numerical integration procedure, an example being Simpson’s

rule.


Rather than attempting to evaluate expression (2), Monte Carlo simulation

[2, 6, 7, 9, 21, 25] encompasses a different numerical approach, based on

the following considerations. The expected value of Y is conventionally

obtained by evaluating the model at the estimates xi of the input quantities

to give y. However, since the value of each input quantity is described by

a PDF rather than a single number, a value as legitimate as its expectation

can be obtained by drawing a value at random from this function.

Monte Carlo simulation operates in the following manner, based on this

consideration. Generate a particular value at random from the PDF for

the value of each Xi and form the corresponding value of Y , by evaluating

the model for these particular values of the input quantities. Repeat this

process many times, to obtain in all M , say, values of Y . These M values are

used to approximate the distribution function G(η) for the value of Y [3].

According to the Central Limit Theorem [20, p169], the arithmetic mean y

of such values converges as 1/M1/2, if the standard deviation u(y) asso-

ciated with y exists. Irrespective of the number N of input quantities, it

is (only) necessary to quadruple M in order to halve the expected uncer-

tainty associated with the estimate of u(y). In contrast, standard numerical

quadrature would require a factor of 2M/2 for this purpose. Thus, the basic

concept of Monte Carlo simulation has reasonable convergence properties.

It is straightforward to implement for many problems. A broad introduction

to Monte Carlo simulation is available [17].

The approach is as follows.

1. Select the number M of Monte Carlo trials to be made.

2. Generate M samples xr of the values of the (set of N) input quanti-

ties X.


3. For each sample xr, evaluate the model to give the corresponding

values yr = f(xr), r = 1, . . . ,M , of the output quantity Y .

4. Sort the yr into non-decreasing order, and use them to obtain an

approximation G(η) to the distribution function G(η) for the value of

the output quantity.

5. The expectation y of G(η) approximates the expectation of the PDF g(η)

for the value of Y and is taken as the estimate y of Y .

6. The standard deviation u(y) of G(η) approximates the standard devi-

ation of the PDF g(η) for the value of Y and is taken as the standard

uncertainty u(y) associated with y.

7. Form (an approximation to) the shortest 95 % coverage interval for

the value of Y from the distribution function G(η).

8. Regard the values yr, when assembled into a histogram (with suitable

cell widths) as a (scaled) approximation to g(η).12

Detailed guidance on these steps is available [10, 11], as is a variant of the

procedure, in which M in step 1 is chosen adaptively given a required degree

of approximation in the results.

The approach extends, in principle, to problems with more than one output

quantity.13 For each sample taken at random from the PDFs for the value of

the input quantities, the corresponding values of the multivariate model are

determined. The resulting (joint) distribution approximates the distribution

for the values of the set of output quantities.12Most calculations will not be carried out in terms of this histogram, the ‘shape’

of which depends sensitively on the choice of cell size [15], but in terms of G(η). The

histogram is, however, a useful visual aid to understanding the nature of g(η).13However, no counterpart of step 4 can yet be recommended, since work on this aspect

remains at the research level.


7 Discussion of approaches

Comments on some of the attributes of the three approaches for the formu-

lated problem are made.

7.1 Analytical methods

Analytical methods are in a sense ideal when applicable: they provide an

exact solution to the formulated problem. However, a closed-form solution

cannot be obtained in many cases. In any case, the services of an expert may

be required. However, analytical methods, when applicable, can be used to

provide a ‘reference solution’, valuable when comparing approaches.

7.2 GUM uncertainty framework

The GUM uncertainty framework is the ‘method of choice’ for many orga-

nizations. However, there are conditions and qualifications concerning the

use of the GUM uncertainty framework, which are sometimes ignored (and

even by some uncertainty evaluation software):

1. The law of propagation of uncertainty is based on an expansion of the

model f about the xi as a first-order Taylor series [GUM G.6.1]. This

approximation is usually adopted [GUM 5.1.5].14

2. The coverage factor can only be found if there is extensive knowledge

of the PDFs for the values of the Xi and if these PDFs are com-14The GUM states that, when the non-linearity of f is significant, higher-order terms

in the Taylor series expansion must be included in the expression for u2(y) [GUM 5.1.2].

However, only the most important terms of next highest order are provided in the GUM.

Moreover, although it is not stated in the GUM, these terms apply specifically to cases

where all the values of the Xi follow mutually independent Gaussian distributions.


bined to obtain the PDF for the value of Y . The xi and the u(xi)

by themselves are inadequate for this purpose [GUM G.6.1]. More-

over, it is difficult to state the coverage probability associated with

a coverage interval, because doing so requires knowledge of the PDF

characterized by y and u(y). Although these parameters are of critical

importance, they are by themselves insufficient to establish intervals

having exactly known coverage probabilities [GUM 6.3.2].15 Because

the extensive computations required to combine PDFs are seldom jus-

tified by the extent and reliability of the available information, an

approximation to the PDF for the value of the output quantity is (of-

ten) acceptable. Because of the Central Limit Theorem, it is usually

sufficient to assume that the PDF for the value of (y − Y )/u(y) is

the t-distribution and take the coverage factor as tp(νeff), with the t-

factor based on an effective degrees of freedom νeff of u(y) obtained

from the Welch-Satterthwaite formula [GUM G.6.2].16

It follows that, in some circumstances, the results provided by the use of

the framework may be invalid.17 In other words, a consequence of working

with a linearization of the model and not employing the actual PDFs is that

the GUM uncertainty framework is generally approximate, with a degree of

approximation that is beyond the control of the user.

The situation concerning the computational aspect in point 2 above is differ-

ent today from when the GUM was originally published in 1993. First, the

15When the Central Limit Theorem [GUM G.2.1, G.6.6] applies, the PDF for the value

of Y can be represented by a Gaussian distribution or in terms of a t–distribution.16This formula [GUM G.4.2] is approximate [16], and assumes that the estimates of

the values of the Xi are mutually independent and the associated standard deviations

likewise.17The GUM anticipates such a possibility by stating that where appropriate other

analytical or numerical methods may be required.


‘extensive computations’ required to combine probability distributions can

often readily be carried out on personal computers. Second, the JCGM has

stated [3] that the uncertainty calculation should be carried out in a man-

ner that respects the problem formulation, with the objective of providing

a solution that is consistent with that formulation.

Many ‘large-scale’ uncertainty calculations [14] can be carried out using

the ‘GUM framework’. The solutions so obtained can be expected to be re-

liable because the conditions for that framework to apply hold substantially

well.

In contrast, small-scale instances can frequently be such that the GUM

uncertainty framework is inapplicable or at best provides a poor approxi-

mation. A case that causes difficulties with interpretation in practice arises

when a coverage interval provided using the GUM uncertainty framework

contains infeasible values, e.g., chemical concentrations less than zero or

greater than 100 %. A case of an inadequate approximation occurs when a

standard uncertainty or a coverage interval is substantially less (or greater)

than the value that should be obtained with a more valid calculation. In

a mass application, the use of the GUM uncertainty framework gave a

standard uncertainty associated with a mass estimate that was 40 % too

small [3].

7.3 Monte Carlo simulation

Monte Carlo simulation (MCS) can be implemented relatively straightfor-

wardly, the core requirements being random-number generation and model

evaluation [11].18 Because control can be exercised over the number of digits

18Implementations made by the authors have been applied to explicit and implicit

models (where Y can and cannot be expressed directly in terms of X), and complex-valued

models (for electrical metrology), with univariate and multivariate output quantities.


delivered by selecting the number of trials a priori or adaptively [3], MCS

can also be used to validate (i) the results provided by GUM uncertainty

framework, and (ii) software implementations of this framework. Although

many evaluations based on the GUM uncertainty framework may be sound,

it is important to demonstrate in any one particular case that this is so. If (a

legitimate implementation of) MCS indicated that certain results obtained

using the GUM uncertainty framework were invalid, it is recommended that

consideration be given to using MCS instead [3].

MCS is increasingly used by national measurement institutes and industrial

organizations. Much greater use is anticipated following the publication of

Supplement 1 to the GUM [3].

Given an implementation of MCS, a particular uncertainty calculation can

generally be carried out more straightforwardly than with the GUM frame-

work. One reason is that derivatives of first order are not required, let

alone making a decision as to whether higher-order derivatives are needed.

MCS applies to any model, regardless of the extent of its non-linearity and

whether the PDFs concerned are Gaussian or not. In particular, an approx-

imation to the PDF for the value of the output quantity is provided that

permits the user to assess its departure from Gaussian. Finally, a coverage

interval corresponding to any required coverage probability can readily be

obtained from this PDF that will always be feasible (concentrations between

zero and 100 %, for example).

8 Co-ordinate system transformation

This section contains an example concerned with the transformation from

the Cartesian representation (real and imaginary parts) of complex-valued

quantities to the polar representation (magnitude and phase) of those quan-


tities. Such transformations arise in electrical, acoustical and dimensional

metrology. The Cartesian (X, Y ) representation is often used for numerical

processing, whereas the polar (R, Θ) representation is more natural for the

metrology community concerned, corresponding to magnitude and phase.19

It is frequently necessary to transform from one representation to the other.

Here, only the transformation from Cartesian to polar is considered. The

model describing the transformation is simple, but multivariate (two output

quantities) and non-linear. The uncertainties associated with such a trans-

formation are evaluated using (a) an analytical treatment, (b) the GUM

uncertainty framework and (c) MCS.

This example examines the case, common in practice, when the standard

uncertainties associated with the estimates of the values of the two input

quantities are equal and the covariance associated with the estimates is zero.

8.1 Problem formulation

The model of measurement relating input quantities (X, Y )T to output

quantities (R,Θ)T is20

R2 = X2 + Y 2, tanΘ = Y/X. (3)

19X and Y are used in this section to denote the input quantities to accord with

convention in representing Cartesian co-ordinates, and are not to be confused with the

generic notation elsewhere in this paper, where X is an input and Y an output quantity.20This is an implicit model for R and Θ [11]. However, since R is a magnitude,

and hence non-negative, it is determined uniquely by taking the positive square root

of R2. Moreover, by using a ‘four-quadrant inverse tangent function’ (often denoted

by ‘atan2’), Θ is determined uniquely from the model to satisfy −π < Θ ≤ π (or when Θ

is expressed in degrees, as in figure 4, −180◦ < Θ ≤ 180◦).


Estimates (x, y)T of the values of Cartesian co-ordinates (X, Y )T are avail-

able. The uncertainty matrix (covariance matrix)

Vx,y =

u2(x) u(x, y)

u(x, y) u2(y)

, (4)

where u(x), u(y) and u(x, y) denote the standard uncertainties and co-

variance associated with x and y, is also available. Here, u(x) = u(y)

and u(x, y) is taken as zero. A bivariate Gaussian PDF with expecta-

tion (x, y)T and uncertainty matrix Vx,y is assigned to the values of the

Cartesian co-ordinates (X, Y )T, e.g., on the basis of maximum entropy con-

siderations [11, 26, 27]. In this case it can be expressed as the product of

two univariate Gaussian PDFs.

Required is the bivariate PDF for the values of the polar co-ordinates (R,Θ)T.

From this PDF the expectation (r, θ)T of the values of (R,Θ)T, the uncer-

tainty matrix Vr,θ associated with (r, θ)T, and a coverage region for the

values of (R,Θ)T, corresponding to a stipulated coverage probability, can

then be determined (section 2). The particular result sought here is a cov-

erage interval for the value of R.

8.2 Uncertainty calculation

The purpose of this section is to determine the PDFs for the values of the

output quantities (R,Θ)T obtained from the application of the GUM un-

certainty framework and MCS, and given by an analytical treatment of the

problem. From the application of the GUM uncertainty framework, values

are obtained for the parameters of a bivariate Gaussian PDF assigned to

the values of (R,Θ)T. From MCS, the distribution function is approximated

using a sample of values of (R,Θ)T.


8.2.1 Analytical treatment

In accordance with the above, assign independent Gaussian PDFs N(x, u2(x))

and N(y, u2(y)), with u(x) = u(y), to, respectively, the values of X and Y .21

Expressions for the joint PDF for the values of R and Θ and their respec-

tive marginal PDFs22 (expressions (5) and (6)) are available [8]. The PDF

corresponding to the marginal distribution for R is

gR(R) =R

u2(x)exp

(−R2 − r2

2u2(x)

)I0

(Rr

u2(x)

), R ≥ 0, (5)

where r2 = x2 + y2 and I0 is a modified Bessel function of the first kind [1].

This is the PDF for a Rice distribution [19] with parameters r and u2(x).

Similarly, the PDF corresponding to the marginal distribution for Θ is

gΘ(Θ) =12π

exp(

−r2

2u2(x)

) (1 +

√πPeP 2

erfc (−P ))

, −π < Θ ≤ π,

(6)

where

P =x cos Θ + y sinΘ√

2u(x), erfc(x) = 1− 2√

π

∫ x

0

exp(−t2

)dt,

erfc denoting the complementary error function [1].

When x = y = 0, analysis of the joint distribution [8] shows that the

output quantities are mutually independent, with the value of R described

by a χ-distribution with two degrees of freedom,23 or equivalently a Rayleigh

21The notation N(µ, σ2) is used to denote a Gaussian distribution with expectation µ

and variance σ2.22A marginal PDF is given by integrating the joint PDF over all values of the other

quantities. It corresponds to treating separately the output quantity of concern.23The square root of the sum of the squares of two standard Gaussian variates.


distribution [8], [20, p96] with PDF

g(R) =R

u2(x)exp

(−R2

2u2(x)

), R ≥ 0,

and the value of Θ by a rectangular distribution with PDF

g(Θ) =12π

, −π < Θ ≤ π.

Figure 4 shows the PDFs (5) and (6) for particular values of x, y and u(x).

Both PDFs differ substantially from a Gaussian form.

Figure 4: The marginal PDF for the value of magnitude R corresponding to

x = −1, y = 0 and u(x) = u(y) = 1, and (right) that for the value of phase

angle Θ, corresponding to x = −10, y = 0 and u(x) = u(y) = 1.

8.2.2 GUM uncertainty framework

In order to implement the counterpart of the GUM uncertainty framework

for two output quantities, the Jacobian of the model, viz., the matrix of

partial derivatives of first order of the output quantities with respect to the

input quantities, is required [10, 11]:

J =

∂R/∂X ∂R/∂Y

∂Θ/∂X ∂Θ/∂Y

.


Forming these partial derivatives and evaluating them at (x, y)T gives

J =

x/r y/r

−y/r2 x/r2

=

cos θ sin θ

− sin θ/r cos θ/r

, (7)

where tan θ = y/x. The uncertainty matrix associated with r and θ is then

evaluated [7, 11] from

Vr,θ = JVx,yJT. (8)

Finally, the result of applying the GUM uncertainty framework is to as-

sign a bivariate Gaussian PDF, with expectation (r, θ)T and uncertainty

matrix Vr,θ, to the value of (R,Θ)T.

For mutually independent input quantities,

Vr,θ =

cos2 θu2(x) + sin2 θu2(y) (cos θ sin θ/r)(u2(y)− u2(x))

(cos θ sin θ/r)(u2(y)− u2(x)) (sin2 θ/r2)u2(x) + (cos2 θ/r2)u2(y)

.

Furthermore, when u(x) = u(y), the off-diagonal elements of Vr,θ are zero

and hence R and Θ are uncorrelated. In this special circumstance, of un-

correlated quantities assigned a bivariate Gaussian distribution, it follows

that R and Θ are mutually independent [18, Theorem 3.1.3].24 Conse-

quently, the result of applying the GUM uncertainty framework is to assign

Gaussian distributions N(r, u2(r)) and N(θ, u2(θ)) to the values of R and Θ,

where

u2(r) = cos2 θ u2(x) + sin2 θ u2(y) = u2(x),

which is independent of x and y or of r and θ, and

u2(θ) = (sin2 θ/r2)u2(x) + (cos2 θ/r2)u2(y) =u2(x)

r2,

which depends only on r.24In contrast, in the analytical treatment (section 8.2.1), where a bivariate Gaussian

distribution is not assigned to the value of (R, Θ)T, the conditions u(x, y) = 0 and

u(x) = u(y) are not sufficient to ensure that R and Θ are mutually independent. An

additional condition, viz., x = y = 0, is required.


8.2.3 Monte Carlo simulation

Monte Carlo simulation was applied as indicated in section 6. For k =

1, . . . ,M = 10 000, random samples (xk, yk)T were drawn from a bivariate

Gaussian distribution with expectation (x, y)T and uncertainty Vx,y given

by expression (4). An existing procedure for the sampling was used [10].

For each random sample (xk, yk)T so obtained, the model (3) was used to

determine corresponding values (rk, θk)T of the output quantities, viz.,

r2k = x2

k + y2k, tan θk = yk/xk, k = 1, . . . ,M.

Some surfaces, especially approximations to PDFs generated using MCS, are

represented by point clouds (unless further processing is used to provide,

usually, continuous approximations). Figure 5 shows 10 000 points sampled

randomly from a PDF given by the product of two Gaussian PDFs, for the

values of X and Y , each having expectation zero and unit standard devi-

ation, and (right) the corresponding points in the magnitude-phase plane.

The joint PDF for the value of R and Θ is evidently far from Gaussian.

Figure 5: 10 000 points sampled randomly from a PDF given by the product of

two Gaussian PDFs, for the values of X and Y , each having mean zero and unit

standard deviation, and (right) the corresponding points in the magnitude-phase

plane.


Because R and Θ are mutually independent in the above instance, the

joint PDF for the value of R and Θ can be expressed as the product of

the PDF for the value of R and that for Θ. Thus, an approximation to

the PDF for the value of R can be provided by presenting the (10 000) R-

values obtained as a ‘scaled histogram’, with a similar statement for Θ.

Figure 6 shows such an approximation to the PDF for the value of R. The

range of R-values is divided into 20 sub-intervals of equal width. The area

of a rectangle of the histogram is proportional to the number of R-values

within the corresponding sub-interval, with the resulting histogram scaled

to have an area of unity. The figure also shows the exact PDF, a Rayleigh

distribution (section 8.2.1). Figure 6 also shows the counterpart for Θ, for

which the exact PDF is a rectangular distribution. MCS provides realistic

approximations to the analytical PDFs.

Figure 6: An approximation in the form of a scaled histogram to the PDF for

the value of R and the analytic PDF for the value of R, a Rayleigh distribution,

and (right) the corresponding approximation to the PDF for the value of Θ, with

the analytic PDF for the value of Θ, a rectangular distribution.

It is clear from the figures that the behaviour is far from Gaussian. Thus, re-

sults obtained under a Gaussian assumption might not be reliable. The MCS

results resemble closely those obtained analytically.


8.3 Discussion

In this instance, an analytical treatment is possible, and the results obtained

using MCS agree well with those from that treatment. In contrast, the

results from the GUM uncertainty framework disagree.

To indicate the differences, consider the numerical values used in figure 4

(left), viz., x = −1, y = 0 and u(x) = u(y) = 1. The results obtained are

given in table 1.

Approach r u(r) I0.95(R)

Analytical 1.55 0.78 [0.16, 3.01]

GUM uncertainty framework 1.00 1.00 [−0.96, 2.96]

MCS (50 000 trials) 1.55 0.78 [0.15, 3.02]

Table 1: The application of an analytical approach, the GUM uncertainty frame-

work and MCS to determining magnitude from Cartesian co-ordinates.

There are some anomalies that arise from the application of the GUM uncer-

tainty framework. First, mutually independent Gaussian distributions are

assigned to the values of R and Θ, with parameters given above, irrespective

of the values of x, y and u(x) = u(y). For an estimate (x, y)T of the value

of the quantity close to the origin this can be expected to give a PDF for the

value of the magnitude R of the quantity that includes negative (infeasible)

values.

Second, it is not clear how to apply the GUM uncertainty framework in the

case that x = y = 0, i.e., the estimate of the value of the quantity is at the

origin (and in practice also when x and y are small). In this case, θ, the

corresponding estimate of the value of phase angle Θ, and J , the Jacobian

of the model, are not well-defined.

MCS and an analytical treatment, when applicable, generally provide robust


approaches.

9 Conclusions

The concern of this paper has been with obtaining greater validity in un-

certainty calculation. The attitude has been taken that for the formulated

problem, i.e., for a given model of measurement and prescribed probabil-

ity distributions for the values of the input quantities to the model, the

uncertainty calculation should be carried out in a way that fully respects

this information. The concept of the propagation of distributions, viz., the

provision of the probability distribution for the value of the model output

quantity in terms of this information, was reviewed. The conditions and

qualifications associated with the use of the GUM uncertainty framework

were also reviewed, and an acknowledgment made of the fact that the con-

ditions for this framework to be applied in a valid manner are not always

verified in practice. This framework and the use of analytical methods

and a numerical method (Monte Carlo simulation), all of which essentially

constitute implementations (with various degrees of approximation) of the

propagation of distributions were considered and compared. An example of

co-ordinate transformation was used to illustrate the points made.

References

[1] M. Abramowitz and I. A. Stegun. Handbook of Mathematical Functions.

Dover, New York, 1972.

[2] M. Basil and A. W. Jamieson. Uncertainty of complex systems by

Monte Carlo simulation. In North Sea Flow Measurement Workshop,

Gleneagles, 26-29 October 1998, 1998.


[3] BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP, and OIML. Guide to

the Expression of Uncertainty in Measurement. Supplement 1: Numeri-

cal Methods for the Propagation of Probability Distributions. Technical

report, Joint Committee for Guides in Metrology, 2004. Draft.

[4] BIPM, IEC, IFCC, ISO, IUPAC, IUPAP, and OIML. Guide to the

Expression of Uncertainty in Measurement, 1995. ISBN 92-67-10188-9,

Second Edition.

[5] M. G. Cox. The numerical evaluation of B-splines. J. Inst. Math. Appl.,

10:134–149, 1972.

[6] M. G. Cox, M. P. Dainton, A. B. Forbes, P. M. Harris, P. M. Schwenke,

B. R. L Siebert, and W. Woger. Use of Monte Carlo simulation for un-

certainty evaluation in metrology. In P. Ciarlini, M. G. Cox, E. Filipe,

F. Pavese, and D. Richter, editors, Advanced Mathematical Tools in

Metrology V. Series on Advances in Mathematics for Applied Sciences

Vol. 57, pages 93–104, Singapore, 2001. World Scientific.

[7] M. G. Cox, M. P. Dainton, P. M. Harris, and N. M. Ridler. The evalu-

ation of uncertainties in the analysis of calibration data. In Proceedings

of the 16th IEEE Instrumentation and Measurement Technology Con-

ference, pages 1093–1098, Venice, 24-26 May 1999, 1999.

[8] M. G. Cox, T. J. Esward, P. M. Harris, N. McCormick, N. M. Ridler,

M. J. Salter, and J. P. R. B. Walton. Visualization of uncertainty

associated with classes of electrical measurement. Technical Report

CMSC 44/04, National Physical Laboratory, Teddington, UK, 2004.

[9] M. G. Cox and P. M. Harris. Design and use of reference data sets for

testing scientific software. Analytica Chimica Acta, 380:339–351, 1999.


[10] M. G. Cox and P. M. Harris. Software specifications for uncertainty

evaluation. Technical Report CMSC 40/04, National Physical Labora-

tory, Teddington, UK, 2004. CMSC 10/01 revised.

[11] M. G. Cox and P. M. Harris. SSfM Best Practice Guide No. 6. Un-

certainty evaluation. Technical report, National Physical Laboratory,

Teddington, UK, 2004.

[12] C. F. Dietrich. Uncertainty, Calibration and Probability. Adam Hilger,

Bristol, UK, 1991.

[13] EA. Expression of the uncertainty of measurement in calibration. Tech-

nical Report EA-4/02, European Co-operation for Accreditation, 1999.

[14] A. B. Forbes. Efficient algorithms for structured self-calibration prob-

lems. In J. Levesley, I. Anderson, and J. C. Mason, editors, Algorithms

for Approximation IV, pages 146–153, Huddersfield, UK, 2002. Univer-

sity of Huddersfield.

[15] M. J. Fryer. A review of some non-parametric methods of density

estimation. J. Inst. Maths. Applics.., 20:335–354, 1977.

[16] B. D. Hall and R. Willink. Does “Welch-Satterthwaite” make a good

uncertainty estimate? Metrologia, 38:9–15, 2001.

[17] J. M. Hammersley and D. C. Handscomb. Monte Carlo Methods. Wiley,

New York, 1964.

[18] K. V. Mardia, J. T. Kent, and J. M. Bibby. Multivariate Analysis.

Academic Press, London, 1979.

[19] A. Papoulis. Probability, Random Variables and Stochastic Processes.

McGraw-Hill, New York, 1984.


[20] J. R. Rice. Mathematical Statistics and Data Analysis. Duxbury Press,

Belmont, Ca., USA, second edition, 1995.

[21] H. Schwenke, B. R. L. Siebert, F. Waldele, and H. Kunzmann. As-

sessment of uncertainties in dimensional metrology by Monte Carlo

simulation: proposal of a modular and visual software. Annals CIRP,

49:395–398, 2000.

[22] G. Strang and K. Borre. Linear Algebra, Geodesy and GPS. Wiley,

Wellesley-Cambridge Press, 1997.

[23] B. N. Taylor and C. E. Kuyatt. Guidelines for evaluating and express-

ing the uncertainty of NIST measurement results. Technical Report

TN1297, National Institute of Standards and Technology, 1994.

[24] UKAS. The treatment of uncertainty in EMC measurements. Technical

Report NIS 81, United Kingdom Accreditation Service, 1994.

[25] P. A. G. van der Geest. An algorithm to generate samples of multi-

variate distributions with correlated marginals. Comput. Stat. Data

Anal., 27:271–289, 1998.

[26] K. Weise and W. Woger. A Bayesian theory of measurement uncer-

tainty. Measurement Sci. Technol., 3:1–11, 1992.

[27] W. Woger. Probability assignment to systematic deviations by the

Principle of Maximum Entropy. IEEE Trans. Instr. Measurement, IM-

36:655–658, 1987.

Date post:	21-Aug-2020
Category:	Documents
Upload:	others
View:	11 times
Download:	0 times

Greater Validity in Uncertainty Calculation 2004/artigo8-MC.pdf · Greater Validity in Uncertainty...

Documents