Week06&07_Curve Fitting.pdf

transcript

8/20/2019 Week06&07_Curve Fitting.pdf

http://slidepdf.com/reader/full/week0607curve-fittingpdf 1/38

WEEK 6&7

Curve Fitting Linear regression

Polynomial regression

Multiple regression

General linear least squares

Nonlinear regression 2

At the end of this topic, the students will be able:

To fits the data using linear and polynomial

regression

To fits the data using multiple linear and

non-linear regression

To assess and choose the preferred method

for any particular problems

LESSON OUTCOMES

• Describes techniques to fit curves (curve fitting ) to

discrete data to obtain intermediate estimates.

• There are two general approaches for curve fitting: – Data exhibit a significant degree of error . The strategy is to

derive a single curve that represents the general trend of the data. – Data is very precise. The strategy is to pass a curve or a series of

curves through each of the points.

• In engineering two types of applications are

encountered normally when fitting experimental data: – Trend analysis. Predicting values of dependent variable, may

include extrapolation beyond data points or interpolation betweendata points.

– Hypothesis testing. Comparing existing mathematical model with

measured data.

Curve Fitting

Three attempts to fit a “best”

curve through uncertain data

points.

a) Least-squares regression Linear regression

Polynomial regression

Multiple regression General linear least squares

Nonlinear regression

b) Linear interpolation

c) Curvilinear interpolation

Mathematical background in Simple Statistics

• Arithmetic mean . The sum of the individual data

points (yi) divided by the number of points (n).

• Standard deviation . The most common measure of a

spread for a sample.

mean)theand pointsdata between

residualstheof squarestheof sumtotalt(S

n y y s

• Variance . Representation of spread by the square of

the standard deviation.

• Coefficient of variation . Has the utility to quantify the

spread of data and provides a normalized measure of

the spread.

%100.. y

Degrees of freedom

Least-Squares Regression

• Regression analysis is a study of the relationships among

variables.

• Get the ‘best’ straight line to fit through a set of uncertain

data points.

• Calculate the slope and the intercept of the line.

• Also fit the ‘best’ polynomial to data.

• Consider multiple linear regression for a case when one

variable depends on two or more variables in linear function.

Linear Regression

• Fitting a straight line to a set of pairedobservations: (x 1 , y 1 ), (x 2 , y 2 ),… ,(x n , y n ) .

• The mathematical expression for straight line:

y=a 0 +a 1 x+e

where a 0 = intercept

a 1 = slopee = error, or residual, between the model and theobservations (e = y – a 0 – a 1 x ; discrepancy between

true value of y and approximate value, a 0 + a 1 x )

Criteria for a ‘Best’ Fit

• Minimize the sum of the residual errors for all available data:

n = total number of points

• However, this is an inadequate criterion

because the errors cancel.

• So, another logical criteria might be to minimize the sum of

the absolute values of discrepancies:

• However, this criterion is also

inadequate. It will minimize the sum of

the absolute values. This criterion also

does not yield a unique best fit.

i xaa ye1

ii xaa ye 1

• Best strategy is to minimize the sum of the squares of the

residuals between the measured y and the y calculated withthe linear model:

• Yields a unique line for a given set of data. The line chosen thatminimizes the maximum distance that individual point fall

from the line.

ir xaa y y yeS 1 1

2 )()model,measured,( Eq.(17.3)

Least-Squares Fit of a Straight Line

• To determine values for a0 and a1, Eq.(17.3) is differentiated

with respect to each coefficient:

xa xa x y

x xaa ya

xaa yaS

xaa yS

Setting derivatives equal to

zero will result in a

minimum Sr

y x y xna

y xa xa x

ya xna

rule,sCramer'usingBy

Mean values

Normal equations,

can be solved

simultaneously

Example 1

Fit a straight line to the x and y values in the first twocolumns of Table 1

1.0 0.5

2.0 2.5

3.0 2.04.0 4.0

5.0 3.5

6.0 6.0

7.0 5.5

4286.37

24y 24y

28x 28x

140x 5.119x 7

0714.0)4)(8393.0(4286.3 8393.0)28()140(7

)24)(28()5.119(7

xa ya x xn

y x y xn

0714.08393.0

isfitsquares-leastThe

y = 0.8393x + 0.0714R² = 0.8683

0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0

Exercise 1

Fit the best straight line to the following set of x and yvalues using the method of least-squares.

y = 4.1071x + 1.5357R² = 0.9822

0 1 2 3 4 5 6 7

Quantification of Error of Linear Regression

• Recall that the sum of the residuals squared

which measures the deviations of the regression

line from the actual data values is

iir xaa yeS 1

The square of the residual

represents the square of the

vertical distance between thedata and another measure of

central tendency-the straight line.

• The analogy can be extended further for cases:

The spread of points around the line is of similar

magnitude along the entire range of data. The distribution of these points about the line is

normal.

• The following statistical formulas may be used to

quantify the error associated with linear regression

S s xaa yS

S s y yS

r x yiir

• Standard error of the estimate

quantifies the spread the data.• sy/x quantifies the spread

around the regression line

• Original standard deviation,

sy quantified the spreadaround the mean.

“Goodness” of our fit

Determine

1) Total sum of the squares around the mean for the

dependent variable, y, is S t

2) Sum of the squares of residuals around the

regression line is S r3) Difference between two quantities, S t -S r quantifies

the improvement or error reduction due to

describing data in terms of a straight line rather than

as an average value.

4) Correlation of determination , r 2 and correlation

coefficient , r .

coefficient of determination

• For a perfect fit, Sr= 0 and r=r2=1, signifying that the

line explains 100 percent of the variability of thedata.

• For r =r2=0, Sr=St, the fit represents no improvement.

correlation coefficient

y yn x xn

y x y xnr

In a more convenient form for computation of r is

Example 3Determine the total standard deviation, standard

error of the estimate and coefficient of correlation forthe linear regression line obtained in the Example 1.

xi yi (yi-ymean)2 (yi-a0-a1xi)2

1.0 0.5 8.5765 0.1687

2.0 2.5 0.8622 0.5625

3.0 2.0 2.0408 0.3473

4.0 4.0 0.3265 0.3265

5.0 3.5 0.0051 0.5897

6.0 6.0 6.6122 0.7970

7.0 5.5 4.2908 0.1994

24.0000 22.7143 2.9911

932.0868.0

868.07143.22

9911.27143.22

0.77342-7

2.9911

9911.2)(

1.94571-7

22.7143

7143.22)(

xaa yS

Solution

Exercise 4Determine the total standard deviation, standard

error of the estimate and coefficient of correlation forthe linear regression line obtained in the Example 2.

0 21 5

4 175 24

Linearization of Nonlinear Relationships

• Most of the engineering functions has nonlinearrelationships.

• These functions cannot adequately be described by

a linear line.

• Transformations can be used to express the data ina form that is compatible with linear regression.

• Examples of functions that can be linearized are

Exponential equation

Power equation

Saturation-growth rate equation

11 lnln x y22 logloglog x y

Polynomial Regression

• Some engineering data is poorly

represented by a straight line wherethe error is high.

• Example (a) and (b): A curve would

be suited to fit the data.

• Instead of trying to linearize some

of the nonlinear functions and use

linear regression,we may alternatively

fit polynomials to the data using

polynomial regression.

• The least-squares procedure can be

readily extended to fit the data to a

higher order polynomial.

• For a second-order polynomial or quadratic:

y=a 0 +a 1 x+a 2 x 2 + e

• For this case, the sum of the squares of the residual is

• Derivative with respect to each of the unknown coefficients of the

polynomial

• These eq. set to be zero and develop set of normal equations:

iiir xa xaa yS 1

210 )(

xa xaa y xa

xa xaa y

y xa xa xa x

ya xa xan

)()()(

• The coefficients of the unknowns can be calculated directly

from the observed data.

• Then, solving a system of three simultaneous linear

equations.

• The standard error is formulated as

• Coefficient of determination for polynomial regression can

also be as

Example 6Fit a second order polynomial to the data

2 13.6

3 27.2

4 40.9

5 61.1

Solution: Refer to page 472 text book

Multiple Linear Regression

Case where y is a linear function of two or more independent

variable:

y=a 0 +a 1 x 1 +a 2 x 2 + e

n x1i x2i a0 yi

x1i (x1i)2 x1ix2i a1 = x1iyi

x2i x1ix2i (x2i)2 a2 x2iyi

iiir xa xaa yS 1

222110 )(

x y Example:

Refer to page 475 text book

• Multiple linear regression can also be extended to the power

equation of the general form,

• This equation is extremely useful when fitting experimental

• To use multiple regression analysis, we transform the above

equation by taking logarithm of base 10 which yield

General Linear Least Squares

residualsE

tscoefficienunknownA

variabledependenttheof valuedobservedY

t variableindependentheof valuesmeasuredat the

functions basistheof valuescalculatedtheof matrix

functions basis1are

221100

E A Z Y

m , z , , z z

e z a z a z a z a y

ji jir z a yS

Minimized by taking its partial

derivative with respect to each of

the coefficients and setting the

resulting equation equal to zero

YTZAZTZ

• The normal equations can be expressed concisely in matrix

form as

1Y T Z Z T Z A

• To solve it, matrix inverse can be employed:

Nonlinear Regression

• The nonlinear models as those which have a nonlinear

dependence on their parameters. There are times when thenonlinear model must fit the data.

• As with linear least squares, nonlinear regression is based on

determining values of the parameters that minimize the sum of the

squares of the residual.

• The Gauss-Newton method is one of the procedures to achieve

the least-squares criterion. It uses a Taylor series expansion to

approximate the nonlinear equation in linear form.

• The least-squares theory is applied to obtain new parameter

values that move in the direction of minimizing the residuals.

criterion.stoppingaceptablean belowfallsuntil

converges,solutiontheuntilrepeatedis procedureThis

1,11,1

0,01,0

For example nonlinear models,

Example:

Refer to page

483 text book

Week06&07_Curve Fitting.pdf

Documents