Non Linear Regression Y i = f( x i ) + i Marco Lattuada Swiss Federal Institute of Technology -...

Non Linear RegressionNon Linear RegressionYYii = f( = f(xxii) + ) + ii

Non Linear RegressionNon Linear RegressionYYii = f( = f(xxii) + ) + ii

Marco Lattuada

Swiss Federal Institute of Technology - ETHInstitut für Chemie und BioingenieurwissenschaftenETH Hönggerberg/ HCI F135 – Zürich (Switzerland)

E-mail: [email protected]://www.morbidelli-group.ethz.ch/education/index

Marco Lattuada

Swiss Federal Institute of Technology - ETHInstitut für Chemie und BioingenieurwissenschaftenETH Hönggerberg/ HCI F135 – Zürich (Switzerland)

E-mail: [email protected]://www.morbidelli-group.ethz.ch/education/index

Marco Lattuada – Statistical and Numerical Methods for Chemical EngineersNonlinear Regressions – Page # 2

PuromycinPuromycin

Description:Puromycin is an antibiotic used by scientists in bio-research to select cells modified by genetic engineering.

Mechanism of action:This is described by the Michaelis-Menten model for enzyme kinetics, which relates the initial velocity on an enzymatic reaction to the substrate concentration x trough the equation:

1

2

,x

f xx

θ


Puromycin KineticsPuromycin Kinetics

0 0.2 0.4 0.6 0.8 1 1.20

20

40

60

80

100

120

140

160

180

200

220

Substrate Concentration [ppm]

Re

ac

tio

n V

elo

cit

y [

co

un

ts/m

in2 ]

Raw Data

1

2

,x

f xx

θ

The model:


Model LinearizationModel Linearization

Puromycin Kinetics:

Model Rearrangement:

Linearized Model:

Puromycin Kinetics:

Model Rearrangement:

Linearized Model:

1

2

,x

f xx

θ

2 2

1 1 1

1 1 1

,

x

f x x x

θ

1 2,f y y β



0 5 10 15 20 25 30 35 40 45 50 550.004

0.006

0.008

0.01

0.012

0.014

0.016

0.018

0.02

0.022

1/x

1/f

Linearized Raw Data

Regression Line1 = 0.00510722 = 0.00024722



0 0.2 0.4 0.6 0.8 1 1.20

20

40

60

80

100

120

140

160

180

200

220


Re

ac

tio

n V

elo

cit

y [

co

un

ts/m

in2]

Regression from linearized model1 = 195.82 = 0.048407


Puromycin KineticsPuromycin Kinetics

0 0.2 0.4 0.6 0.8 1 1.20

20

40

60

80

100

120

140

160

180

200

220


Re

ac

tio

n V

elo

cit

y [

co

un

ts/m

in2 ]

Raw Data

1

2

,x

f xx

θ

The model:

Linearized model is needed to estimate 2


Nonlinear RegressionNonlinear Regression

Object

To minimize the objective function

where n is the number of observations, yi the responses, xi is the vector of the observations, the vector of the parameters and f(xi,) the nonlinear model function.

It is possible to plot the objective function S() as a function of the parameter values, in order to reveal the presence of a minimum.

Object

To minimize the objective function

where n is the number of observations, yi the responses, xi is the vector of the observations, the vector of the parameters and f(xi,) the nonlinear model function.

It is possible to plot the objective function S() as a function of the parameter values, in order to reveal the presence of a minimum.

2

1

( ) ,n

i ii

S y f

θ x θ


Objective Function S(Objective Function S())

Contour plot of S(q)Contour plot of S(q)

1200

1250

1250

1300

1300

1350

1350

1350

1400

1400

1400

1600

1600

1600

1600

1800

1800

1800

1800

1800

2000

2000

2000

2000

2000

2000

2500

2500

2500

2500

2500

2500

2500

3000

3000

3000

3000

3000

3000

3000

4000

4000

4000

4000

4000

4000

4000

5000

5000

5000

5000

5000

5000

7500

7500

7500

7500

1000

0

10000

10000

1250

0

12500

15000

17500

20000

1

2

180 190 200 210 220 230 2400.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

Minimum

Estimated value of from linearization


Minimization of S(Minimization of S())

Model linearization:

where:

so the residuals are:

Search for minimum with Gauss-Newton method:

Model linearization:

where:

so the residuals are:

Search for minimum with Gauss-Newton method:

0

0 0,1

,, ,

pi i

i i i i j jj j

ff f

θ

x θx θ x θ

1 2

i i

i

f x

x

1

22 2

i i

i

f x

x

0 0 0 0 0 ε θ f θ y f θ y J θ θ ε θ J θ

( ) TS θ ε ε 0 0 0 0T T J J θ J ε θ

J0 = Jacobian


Gauss-Newton Method Applied to S(Gauss-Newton Method Applied to S())

1200

1250

1250

1300

1300

1350

1350

1350

1400

1400

1400

1600

1600

1600

1600

1800

1800

1800

1800

1800

2000

2000

2000

2000

2000

2000

2500

2500

2500

2500

2500

2500

2500

3000

3000

3000

3000

3000

3000

3000

4000

4000

4000

4000

4000

4000

4000

5000

5000

5000

5000

5000

5000

7500

7500

7500

7500

1000

0

10000

10000

1250

0

12500

15000

17500

20000

1

2

180 190 200 210 220 230 2400.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

Convergence path of Gauss-Newton Method(1)opt = 212.66(2)opt = 0.064091


Nonlinear RegressionNonlinear Regression

0 0.2 0.4 0.6 0.8 1 1.20

20

40

60

80

100

120

140

160

180

200

220


Re

ac

tio

n V

elo

cit

y [

co

un

ts/m

in2]

Nonlinear Regression

Regression from linearized model


Ellipsoidal Confidence RegionEllipsoidal Confidence Region

The ellipsoidal confidence region can be evaluate from the linearized model around the point , which is the vector of the parameters for which the objective function has a minimum.

In practice, every vector of the parameters which satisfies the following condition:

is within the confidence interval, where n is the number of observations, p the number of parameters and s the standard deviation:

The ellipsoidal confidence region can be evaluate from the linearized model around the point , which is the vector of the parameters for which the objective function has a minimum.

In practice, every vector of the parameters which satisfies the following condition:

is within the confidence interval, where n is the number of observations, p the number of parameters and s the standard deviation:

2ˆ ˆˆ ˆ ( , , )T

T ps F p n p θ θ J J θ θ

θ̂

2 ˆ /s S n p θ


Ellipsoidal Confidence RegionEllipsoidal Confidence Region

180 190 200 210 220 230 2400.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

1

2

cap

99%95%90%


True Confidence Region for ParametersTrue Confidence Region for Parameters

The real confidence region can be estimated by plotting the region of space for which:

The real confidence region can be estimated by plotting the region of space for which:

ˆ 1 , ,p

S S F p n pn p

θ θ

180 190 200 210 220 230 240 2500.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0.11

0.12

1

2

cap

95%90%99%Contours


Matlab Nonlinear Regression RoutineMatlab Nonlinear Regression Routine

First, create a function providing the residuals for the n observation as a function of the parameter values:

Then, use the routine 'nlinfit';

First, create a function providing the residuals for the n observation as a function of the parameter values:

Then, use the routine 'nlinfit';


Tukey-Ancombe PlotTukey-Ancombe Plot

40 60 80 100 120 140 160 180 200 220-15

-10

-5

0

5

10

15

20

25

30

Fitted y

Re

sid

ua

ls


Normal PlotNormal Plot

-10 -5 0 5 10 15 20 25

0.05

0.10

0.25

0.50

0.75

0.90

0.95

Data

Pro

babi

lity

Normal Probability Plot

>> normplot(r)


Matlab Estimation of Parameter CIMatlab Estimation of Parameter CI

Parameter confidence interval can be estimated by Matlab as follows:

The confidence interval can be estimated using the following Matlab GUI:

Parameter confidence interval can be estimated by Matlab as follows:

The confidence interval can be estimated using the following Matlab GUI:

Date post:	15-Jan-2016
Category:	Documents
View:	214 times
Download:	0 times

Non Linear Regression Y i = f( x i ) + i Marco Lattuada Swiss Federal Institute of Technology -...

Documents