Slide 1

How do we classify uncertainties? What are their sources?Lack of knowledge vs. variability.

What type of measures do we take to reduce uncertainty?Design, manufacturing, operations & post-mortemsLiving with uncertainties vs. changing them

How do we represent random variables?Probability distributions and moments Uncertainty and Uncertainty reduction Measures

In this lecture we will deal with three topics relating to uncertainty. First we will distinguish between two different kinds of uncertainty. For this part and a general Introduction, a good source is Chapter 1 of.S-K Choi, RV Grandhi, and RA Canfield, Reliability-based structural design, Springer 2007. Available on-line from UF library http://www.springerlink.com/content/w62672/#section=320007&page=1 Then we will have a short discussion on measures that we take to reduce uncertainty. Finally we will introduce our main tool for modeling uncertainties, which are probability distributions. For that part, Chapter 2 of Choi, Grandhi, and Canfield is helpful.

1Classification of uncertaintiesAleatory uncertainty: Inherent variabilityExample: What does regular unleaded cost in Gainesville today?

Epistemic uncertainty Lack of knowledgeExample: What will be the average cost of regular unleaded January 1, 2014?

Distinction is not absoluteKnowledge often reduces variabilityExample: Gas station A averages 5 cents more than city average while Gas station B 2 cents less. Scatter reduced when measured from station average!

Source: http://www.ucan.org/News/UnionTrib/

The main classification of uncertainties is into aleatory and epistemic uncertainties. Aleatory uncertainties are due to inherent variability. The example here is of gas prices on a given day in the city of Gainesville. Somebody doing a survey of gas stations may summarize it by noting that the average is $2.25 a gallon for regular, with a standard deviation of 8 cents. When dealing with engineering systems, aleatory uncertainty is due to variability in manufacturing and material properties of nominally identical parts, or variability in operating conditions. It is sometimes called irreducible uncertainty because additional knowledge does not reduce it, but manufacturing variability, for example, can be reduced by investment in more stringent tolerances. Even for the gas price example, we may reduce the variability by learning which stations are the cheapest.Epistemic uncertainty reflects lack of knowledge. An example is predicting the average price of gas a year ahead. In engineering applications we use analytical and numerical models to estimate response and performance, and the epistemic uncertainty mostly reflects the limitations of the models.2A slightly differentuncertainty classification.

British Airways 737-400Distinction between Acknowledged and Unacknowledged errorsType of uncertaintyDefinitionCausesReduction measuresErrorDeparture of average from modelSimulation errors, construction errorsTesting and model refinementVariabilityDeparture of individual sample from averageVariability in material properties, construction tolerancesTighter tolerances, quality control

3For this course, we assume that our key epistemic uncertainty is due to errors, and so the main distinction is between error and variability. Furthermore, since we calculate uncertainty for engineering systems, we will usually deal with a population of nominally identical parts or vehicles. In such setting it is useful to make the distinction between error and variability based on which apply to the entire population and which apply to the individuals.Specifically we consider errors the departure of the average from the model. For example, the population may include 500 737-400 airliners. Our model predicts that they have a range of 3,000 miles, but they average only 2,900. The 100 mile departure is due to errors in our model that predicts range, or errors in the input to the model (for example, we have the wrong aircraft weight).Variability is the departure of the population from the average. In the case of the airliners it may reflect variability in the efficiency of their engines due to manufacturing or operating conditions.Errors are often divided into acknowledged and unacknowledged errors. Acknowledged errors are often reduced by refining a model. Unacknowledged ones may be uncovered an corrected by tests. Variability may be reduced by tighter manufacturing tolerance or quality control.Modeling and Simulation.

The epistemic uncertainties that we are mostly interested in are due to our attempts to simulate reality by a computerized model. This figure, originally from the Society of Computer simulation) but borrowed directly from Oberkampf et al., illustrates the processes responsible for simulation errors.The first step is modeling reality with a conceptual model, such as Newton laws or Einstein laws of motion to describe the trajectory of a rocket. The trajectory of the rocket will then require analysis leading to a set of differential equations that describe the motion. The process of model qualification requires checks to ensure that the model is appropriate. For example, if we model the rocket as a point mass, we will need some checks that its finite size is not likely to have a large effects on the solution. The next step is to write or select software to solve the conceptual model, e.g., the differential equations. The process of checking the fidelity of the software for providing the solution is called verification. It is typically done by comparing with analytical solutions or solutions obtained by other software.Verification only ensures that the software can provide an accurate solution to the conceptual model. Comparison with reality, via physical tests, checks the entire process and is called model validation.

Oberkmapf et al. Error and uncertainty in modeling and simulation, Reliability Engineering and System Safety, 75, 333-357, 2002

4Error modeling

Having errors in simulations is acceptable when the magnitude of the errors can be estimated, so that decisions based on the simulations take them into account. For example, if our errors in predicting the trajectory of a rocket are excessive, we may compensate by installing a control system that will measure errors and correct the trajectory accordingly.The processes of qualification, verification and validation often provide error estimates. For example, the validation test may have revealed 3% difference, and that may have been judged acceptable. Personal or group experience may provide additional information on error magnitude. For example, an analyst with experience in trajectory calculation, may have an idea how well a given model may approximate reality. However, both sources of error estimation most often are expressed as bounds on the error rather than more detailed assessment.We often settle on numerical models that are coarse or simplified because we need to analyze them many times, for example for optimization. 5Uncertainty reduction measures Design: Refined simulation models, building block tests. Aleatory or epistemic?

Manufacture: Quality control. A or E?

Operation: Licensing of operators, maintenance and inspections. A or E?

Post-mortem: Accident investigations. A or E?Living with uncertainties by using safety factors

For most products we engage in vigorous uncertainty reduction measures intended to reduce both aleatory and epistemic uncertainties. At the design stage we often start with coarse simulation models and gradually refine them. We test components and then entire prototypes in a process that is sometime referred to as building block tests.

Manufacturing errors are compensated in part by quality control processes, and once a product goes into service manufacturing glitches and ageing effects are caught by inspections and maintenance. Operators and maintenance workers are often licensed by governmental agencies in order to reduce errors.

. Since some uncertainties remain, we compensate for them by extra safety margins in the design and operations of the system. The slide asks you to answer whether each uncertainty measure targets aleatory or epistemic uncertainty. That is, does it limit variability or does it improves our knowledge of the average product.

Finally, failures do happen, and when they are fatal they are often followed by accident investigations. These help limit the effect of systematic design or operation errors to a small number of cases, by leading to recalls or other preventive measures.

6Representation of uncertaintyRandom variables: Variables that can takemultiple values with probability assigned toeach value

Representation of random variablesProbability distribution function (PDF)Cumulative distribution function (CDF)Moments: Mean, variance, standard deviation, coefficient of variance (COV)

7Probability density function (PDF)If the variable is discrete, the probabilities of each value is the probability mass function. For example, with a single die, toss, the probability of getting 6 is 1/6.If you toss a pair of dice the probability of getting twelve (two sixes) is 1/36, while the probability of getting 3 is 1/18.The PDF is for continuous variables. Its integral over a range is the probability of being in that range.

One common way of describing a random variable is by giving the probability for each value it can take. For example, with a single die toss, the random variable may be the number on the top face of the die; the probability of taking 1,2,3,4,5. or 6 is 1/6. If we toss two dice, and the random variable is the sum of the two, then the probabilities of different outcomes are different. For example the probability of 12 is 1/36 (only one combination out of 36) while the probability of getting 11 is 1/18 (two ways 5,6 and 6,5). The function that gives the probability for each possible value is called probability mass function.For continuous variables we have instead a probability density function (PDF), whose integral over any range gives the probability of falling in that range. For example, the figure shows the PDF of a normal (Gaussian) distribution. The integral of the PDF over the center region (darker blue) is 0.5 for the top figure and 0.6827 for the bottom one (Figure from Wikipedia). Since the integral over a single point is zero, the probability of taking any given value is zero. For example, If the random variable is the top sustained wind speed of a hurricane hitting Gainesville Florida, the probability of that speed being exactly 85 MPH is zero, while the probability of its being between 84.5 and 85.5 is the integral of the PDF from 84.5 to 85.5.8HistogramsProbability density functions have to be inferred from finite samples. First step is a histogram.Histograms divide samples to finite number of ranges and show how many samples in each range (box)Histograms below generated from normal distribution with 50 and 500,000 samples.

If the PDF is not known in advance we can get an idea of its shape from a sample by plotting a histogram. The histogram is obtained by dividing the overall range of the sample into a finite number of intervals (usually of equal size) and showing the number of samples in each interval.For example, the Matlab sequence z=randn(1,50)+10; hist(z,8); generates 50 samples from the normal distribution (see lecture on random variable distributions) and then divides the range to eight intervals. The resulting plot on the left, only vaguely resembles a normal distribution. With z=randn(1,500000)+10 (500,00 samples) we get a better resemblance, but we may have benefitted from more intervals (boxes).It is also worth noting that random variables can infinite range, and many common distributions, like the normal distribution do. For those distributions the probability of getting very large or very small values is usually small, so the larger the sample the larger the range of the sample. This is shown in the two figures in that the range of values for 500,000 samples is almost twice as large as the range for 50 samples.

9Number of boxes

10Histograms and PDFHow do you estimate the PDF from a histogram?Only need to scale.

11

Cumulative distribution functionIntegral of PDF

Experimental CDF from 500 samples shown in blue, compares well to exact CDF for normal distribution.

12Probability plotA more powerful way to compare data to a possible CDF is via a probability plot (500 points here)

13

MomentsMean

Variance

Standard deviation

Coefficient of variationSkewness

A compact way to give information about samples and distributions is to provide some of their moments. The first moment is the mean, and it is commonly denoted by the letter . The operation of integrating a function of a random variable times its PDF is also called calculating the expectation of the function. So the mean is also the expected value of the random variable.The square deviation of the random variable from its mean, or its second moment is called the variance. We normally use the square root of the variance, known as the standard deviation. Alternatively, we use the coefficient of variation, which is the ratio of the standard deviation to the mean.Higher order moments are also used, especially normalized central moments. They are centralized by taking them around the mean, and normalized by the standard deviation. The third normalized central moment is called skewness and it measures the asymmetry in the distribution. 14QuestionsOur random variable is the number seen when we roll one die. What is the CDF of 2?Our random variable is the sum on a pair of dice. What is the CDF of 2? Of 13?