Linear regression

Non-linear regression

5. Curve fitting

1

Polynomial Model

2

),( , ... ),,(),,( 2211 nn yxyx yxGiven best fit m

mxa...xaay

10

)2( nm to a given data set.

Figure. Polynomial model for nonlinear regression of y vs. x data

m

mxaxaay

10

),(nn

yx

),(11

yx

),(22

yx

),(ii

yx

)(ii

xfy

3

The residual at each data point is given by

m

imiii xa...xaaye 10

The sum of the square of the residuals then is

n

i

m

imii

n

i

ir

xa...xaay

eS

1

2

10

1

2

4

To find the constants of the polynomial model, we set the derivatives with respect to ia where

0)(....2

0)(....2

0)1(....2

1

10

1

10

1

1

10

0

m

i

n

i

m

imii

m

r

i

n

i

m

imiir

n

i

m

imiir

xxaxaaya

S

xxaxaaya

S

xaxaaya

S

,,1 mi

equal to zero.

5

These equations in matrix form are given by

n

i

i

m

i

n

i

ii

n

i

i

m

n

i

m

i

n

i

m

i

n

i

m

i

n

i

m

i

n

i

i

n

i

i

n

i

m

i

n

i

i

yx

yx

y

a

a

a

xxx

xxx

xxn

1

1

1

1

0

1

2

1

1

1

1

1

1

2

1

11

......

...

...........

...

...

The above equations are then solved for maaa ,,, 10

6

Example 2-Polynomial Model

Temperature, T (oF)

Coefficient of thermal

expansion, (in/in/oF)

80 6.47106

40 6.24106

40 5.72106

120 5.09106

200 4.30106

280 3.33106

340 2.45106

Regress the thermal expansion coefficient vs.temperature data to a second order polynomial.

1.00E-06

2.00E-06

3.00E-06

4.00E-06

5.00E-06

6.00E-06

7.00E-06

-400 -300 -200 -100 0 100 200

Temperature, oF

Th

erm

al

exp

an

sio

n c

oeff

icie

nt,

(in

/in

/oF

)

Table. Data points for temperature vs

Figure. Data points for thermal expansion coefficient vs temperature.

7

2210 TaTaa

We are to fit the data to the polynomial regression model

n

i

ii

n

i

ii

n

i

i

n

i

i

n

i

i

n

i

i

n

i

i

n

i

i

n

i

i

n

i

i

n

i

i

T

T

a

a

a

TTT

TTT

TTn

1

2

1

1

2

1

0

1

4

1

3

1

2

1

3

1

2

1

1

2

1

The coefficients 210 , a,aa are found by differentiating

square of the residuals with respect

values equal to zero

8

Solution:

the sum of the

to each variable and setting the

to obtain

The necessary summations are as follows

Temperature, T

(oF)

Coefficient of thermal expansion,

(in/in/oF)

80 6.47106

40 6.24106

40 5.72106

120 5.09106

200 4.30106

280 3.33106

340 2.45106

Table. Data points for temperature vs.

57

1

2 105580.2 i

iT

77

1

3 100472.7 i

iT

107

1

4 101363.2

i

iT

57

1

103600.3

i

i

37

1

106978.2

i

iiT

17

1

2 105013.8

i

iiT 9

1

3

5

2

1

0

1075

752

52

105013.8

106978.2

103600.3

101363.2100472.7105800.2

100472.7105800.210600.8

105800.2106000.80000.7

a

a

a

Using these summations, we can now calculate 210 , a,aa

Solving the above system of simultaneous linear equations we have

11

9

6

2

1

0

102218.1

102782.6

100217.6

a

a

a

The polynomial regression model is then

21196

2

210

T101.2218T106.2782106.0217

TaTaa

10

Exponential Model

11

),( , ... ),,(),,( 2211 nn yxyx yxGiven best fitbxaey to the data.

Figure. Exponential model of nonlinear regression for y vs. x data

bxaey

),(nn

yx

),(11

yx

),(22

yx

),(ii

yxibx

i aey

12

Finding Constants of Exponential Model

n

i

bx

ir

iaeyS1

2

The sum of the square of the residuals is defined as

Differentiate with respect to a and b

021

ii bxn

i

bx

ir eaey

a

S

021

ii bx

i

n

i

bx

ir eaxaey

b

S

13

Rewriting the equations, we obtain

01

2

1

n

i

bxn

i

bxi

ii eaey

01

2

1

n

i

bxi

n

i

bxii

ii exaexy

14

Finding constants of Exponential Model

Substituting a back into the previous equation

01

2

1

2

1

1

n

i

bx

in

i

bx

bxn

i

ibx

i

n

i

ii

i

i

i ex

e

ey

exy

The constant b can be found through numerical methods such as bisection method.

n

i

bx

n

i

bxi

i

i

e

ey

a

1

2

1

Solving the first equation for a yields

15

Example 3-Exponential Model

Many patients get concerned when a test involvesinjection of a radioactive material. For example forscanning a gallbladder, a few drops of Technetium-99misotope is used. Half of the techritium-99m would begone in about 6 hours. It, however, takes about 24 hoursfor the radiation levels to reach what we are exposed toin day-to-day activities. Below is given the relativeintensity of radiation as a function of time.

16

Find: a) The value of the regression constants

A andb) The half-life of Technium-99m

c) Radiation intensity after 24 hours

The relative intensity is related to time by the equation

tAe

17

t(hrs) 0 1 3 5 7 9

1.000 0.891 0.708 0.562 0.447 0.355

Table. Relative intensity of radiation as a function of time.

Plot of data

18

Solution:

a) Constants of the Model

The value of is found by solving the nonlinear equation

01

2

1

2

1

1

n

i

tin

i

t

n

i

ti

ti

n

ii

i

i

i

i et

e

e

etf

n

i

t

n

i

t

i

i

i

e

e

A

1

2

1

tAe

19

Setting up the Equation in MATLAB

01

2

1

2

1

1

n

i

tin

i

t

n

i

ti

ti

n

ii

i

i

i

i et

e

e

etf

t (hrs) 0 1 3 5 7 9

1.000 0.891 0.708 0.562 0.447 0.35520

Setting up the Equation in MATLAB

01

2

1

2

1

1

n

i

tin

i

t

n

i

ti

ti

n

ii

i

i

i

i et

e

e

etf

t=[0 1 3 5 7 9]gamma=[1 0.891 0.708 0.562 0.447 0.355]syms lamdasum1=sum(gamma.*t.*exp(lamda*t));sum2=sum(gamma.*exp(lamda*t));sum3=sum(exp(2*lamda*t));sum4=sum(t.*exp(2*lamda*t));f=sum1-sum2/sum3*sum4;

115080.

21

Calculating the Other Constant

The value of A can now be calculated

6

1

2

6

1

i

t

i

t

i

i

i

e

e

A

9998.0

The exponential regression model then is

t.e . 11508099980

22

23

b) Half life of Technetium 99 is when02

1

t

hours.t

.lnt.

.e

e.e.

t.

.t.

02326

50115080

50

999802

199980

115080

0115080115080

Plot of data and regression curve

te 1151.0 9998.0

24

Relative Intensity After 24 hrs

The relative intensity of radiation after 24 hours

2411508099980 .e. 2103160.6

This result implies that only

%317.61009998.0

10316.6 2

radioactive intensity is left after 24 hours.25

Transformation of Data

To find the constants of many nonlinear models, itresults in solving simultaneous nonlinear equations. Formathematical convenience, some of the data for suchmodels can be transformed. For example, the data foran exponential model can be transformed.

As shown in the previous example, many chemical and physical processes are governed by the equation,

bxaey

Taking the natural log of both sides yields, bxay lnln

Let yz ln and aa ln0

(implying) oaea with ba 1

We now have a linear regression model where xaaz 10

26

Linearization of data cont.

Using linear model regression methods,

_

1

_

0

1

2

1

2

11 11

xaza

xxn

zxzxn

an

i

n

i

ii

n

i

i

n

i

n

i

iii

Once 1,aao are found, the original constants of the

0

1

aea

ab

27

model are found as

Example 4-Linearization of data

t(hrs) 0 1 3 5 7 9

1.000 0.891 0.708 0.562 0.447 0.355

Many patients get concerned when a test involves injection of a radioactive material. For example for scanning a gallbladder, a few drops of Technetium-99m isotope is used. Half of the technetium-99m would be gone in about 6 hours. It, however, takes about 24hours for the radiation levels to reach what we are exposed to in day-to-day activities. Below is given the relative intensity of radiation as a function of time.

Table. Relative intensity of radiation as a function of time

0

0.5

1

0 5 10

Rela

tive in

ten

sit

y o

f ra

dia

tio

n,

Time t, (hours)

Figure. Data points of relative radiation intensity

vs. time28

Find:

a) The value of the regression constants A and

b) The half-life of Technium-99m

c) Radiation intensity after 24 hours

The relative intensity is related to time by the equation

tAe

29

tAe

Exponential model given as,

tA lnln

Assuming lnz , Aao ln and 1a we obtain

taaz10

This is a linear relationship between z and t

30

Using this linear relationship, we can calculate

10,aa

n

i

n

ii

n

ii

n

i

n

iiii

ttn

ztztn

a

1

2

1

2

1

11 1

1

and

taza 10

where

1a

0a

eA31

12

3

4

5

6

0

1

3

5

7

9

1

0.891

0.708

0.562

0.447

0.355

0.00000

0.11541

0.34531

0.57625

0.80520

1.0356

0.0000

0.11541

1.0359

2.8813

5.6364

9.3207

0.0000

1.0000

9.0000

25.000

49.000

81.000

25.000 2.8778 18.990 165.00

Summations for data linearization are as follows

Table. Summation data for linearization of data model

i it i iiz ln ii zt2

it

With 6n

000.256

1

i

it

6

1

8778.2i

iz

6

1

990.18i

iizt

00.1656

1

2 i

it

32

Calculating 10,aa

21 2500.1656

8778.225990.186

a 11505.0

6

2511505.0

6

8778.20

a

4106150.2

Since Aa ln0 0aeA

4106150.2 e 99974.0

11505.01 aalso

33

Resulting model is te 11505.099974.0

0

0.5

1

0 5 10

Time, t (hrs)

Relative

Intensity

of

Radiation,

te 11505.099974.0

Figure. Relative intensity of radiation as a function of temperature using linearization of data model.

34

The regression formula is then

te 11505.099974.0

b) Half life of Technetium 99 is when02

1

t

hours.t

.t.

.e

e.e.

t.

.t.

02486

50ln115050

50

9997402

1999740

115080

0115050115050

35

c) The relative intensity of radiation after 24 hours is then

2411505.099974.0 e

063200.0

This implies that only %3216.6100

99983.0

103200.6 2

of the radioactive

36

material is left after 24 hours.