9/14/2015

1

OPTIMIZATIONCH403ProcessEconomicsandPlantDesign

Whyweneedtooptimize? Economicoptimization:maximizenetpresentworth or net returns of an investmentworthornetreturnsofaninvestment

Processoptimization: Higherprocessingrates Higheryieldofdesiredproduct Reducedenergyconsumption Longproductivetimebetweenshutdowns

Plantoptimization: Lessersupervisoryandmanagementissues

9/14/2015

2

BasicprobleminOptimizationProblem:Findtheclosestpointonthecurvef

= 2x2 + 3x + 1 from the origin 5

6

2x 3x 1 fromtheorigin Formulateaobjectivefunction: Distanceofanypoint(xp,f(xp))squaredis

Ifwecanminimize squareofdistance,wegettheclosestpoint

2222 132 ppp xxxdD 222 132DMi

2

3

4

5

Takederivative andputtozero

Solve

22 132 ppp xxxDMin 06283616 23' ppp xxxD

3414.0px Threethingstonote:1. Nonlinear2. Unconstrained3. Realvalued

x-1 0 1

-1

0

1

OptimizationinChemicalProcess ProcessVariables:

t t t ti fl t t temperature,pressure,concentration,flowrate,etc. reactionrate,heattransfercoefficient,etc.and equipmentspecificationssuchassize,surfacearea,no.oftrays,valveposition,etc.

Dependent VariablesDecision Variables DependentVariablesThesearerelatedtodecisionvariablethroughconstraints

DecisionVariablesTheseareeithersetor

determined

9/14/2015

3

AnotherExample Objectivefunction: 124: 2 xxxfMinimize Constraints:

124: 21 xxxfMinimize

0

013

0341010

025

22

21

222

211

22

21

xxxx

xxxxxx

Threethingstonote:1. Nonlinear2. Constrained

Notethatoneofx1 orx2 canbeeliminatedusingtheequalityconstraint.

Onlyoneindependent/decision variableandotherwillbedependent variable

0, 21 xx 3. Realvalued

OptimizationinChemicalProcess ProcessVariables:

temperature,pressure,concentration,flowrate, reactionrate,heattransfercoefficient,etc.and equipmentspecificationssuchassize,surfacearea, Specificationssuchasno.oftrays,valveposition,etc.

ContinuousVariablesThesecantakenoninteger

values.

DiscreteVariablesThesetakeintegervalues

suchason/off,closed/open,/

ObjectiveFunction:Expressthequantity thatistobeoptimized(eitherminimizedormaximized)intermsofdecisionvariablessuchascostofconstructionandoperation.

values. yes/on,etc.

9/14/2015

4

TypesofOptimizationConstrainedOptimization

UnconstrainedOptimizationOptimization Optimization

IntegerLinearProgramming

(ILP)

MixedIntegerLinear

Programming(MILP)

LinearProgrammingProblem(LPP)

NonLinearProgrammingProblem(NLPP)

TypesofOptimizationLinear NonLinearLinear

ProgrammingProblem(LPP)

Non LinearProgrammingProblem(NLPP)

GraphicalMethodSuccessiveLinearProgramming(SLP)SuccessiveQuadraticProgramming (SQP)

SimplexMethod

Programming(SQP)Augmented

Lagrangian Method(ALM)

GeneralizedReducedGradientMethod

(GRG)

9/14/2015

5

ClassicalOptimization Classicalsinglevariableoptimization

f f( ) d f d h

However,x=x*couldalsobeamaximumorpointofinflection

NECESSARYCONDITION:Forafunctionf(x),definedintheintervalaxb,hasaminimumatx=x*,whereax*b,ifthederivatived(f(x))/dx =f(x)existsasafinitenumberatx=x*,thenf(x)=0.

SUFFICIENTCONDITION:Ifhigherorderderivativesexists,suchthatf(x*)=f(x*)==f(n1)(x*)=0,butf(n)(x*)0,then(i) f(x*)isaminimum,iff(n)(x*)>0andniseven.(ii) f(x*)isamaximum,iff(n)(x*)

9/14/2015

6

MultivariableOptimization Consider,f(x,y)=x2y2

f0,0

00

22

yxfy

x

yfxf

f

20

02, 22

2

2

2

ffyxf

xf

yxH

Tofindoutwhether(x,y)=(0,0) isminimum,maximumorpointofinflection,weneedtotestthesufficientcondition.

202yf

xyf

MultivariableOptimization ChecksignoffordifferentXo

T k X (0 1 0) 0 02 0 ooTo XXHX T XXHX

1

0.5

TakeXo =(0.1,0),=0.02>0 TakeXo =(0,0.1),=0.02

9/14/2015

7

MultivariableOptimization Checkforyourselfthat

)( 2233fhasaminimumat(0,0).

120

40

60

80

100

642),( 2221

32

3121 xxxxxxf

42

0-2

-4-4-2

02

0

-20

20

40

4

Threethingstonote:1. Nonlinear2. Unconstrained3. Realvalued

UnconstrainedOptimization MethodofSteepestDescent(Minimization)orAscent (Maximization)Ascent(Maximization)

21222121 coscos, xxxxxxfMin GlobalMaximum

LocalMaximum

LocalMinimum

9/14/2015

8

UnconstrainedOptimization

22exp xxxxxfMin MethodofSteepestDescent

21121 exp, xxxxxfMin

30

35

40

5 10 15 20 25 30 35 40

5

10

15

20

25

),( *2*1

* xxX

UnconstrainedOptimization Startwithsomeinitialguess Find a direction to proceed (search direction)

ooo xxX 21 ,oS

MethodofSteepestDescent

Findadirectiontoproceed(searchdirection) Determinehowfartoproceed Reachanewlocation Repeatuntilconvergence

So

oX

ooo SXxxX

12111 ,

kk XfXf 1

9/14/2015

9

UnconstrainedOptimization Searchdirection,then Determine how far to proceed

kk XfS kkkX

kkkk XfXSXXk

1MethodofSteepestDescent

Determinehowfartoproceed suchthatisminimumi.e.k kkk SXf 0 kkkk SXfdd

00

0

0

1

1

kTkkkkTk

k

ki

kki

i

kkkki

k

ki

i

kkkki

XfXfSXfS

dsxdSXf

x

ddxSXf

x

Innerproductiszeromeansorthogonality

UnconstrainedOptimization Minimize Initial guess:

MethodofSteepestDescent )exp(, 222121 xxxxf Initialguess:

Determine:

SearchDirection:

2,2, 21 ooo xxX 2221

2

1 exp2 xxxx

f

1

Distancetomove: 8exp

11

40

fS o

2222111 exp0 oooo sxsxf

9/14/2015

10

UnconstrainedOptimization Solve,

MethodofSteepestDescent

1 oooo Nextstep,

21

-0.2

-0.4

-0.6

-0.8

-12

8exp212221 2211

oo

oooo

sssxsx

112

1 oo SXX 2

10

-1-2-2

-10

1

00

8exp11

48exp21

22

UnconstrainedOptimization Minimize

i i l

MethodofSteepestDescent )4exp(, 222121 xxxxf 22oooX Initialguess:

Tryyourself!

-0.2

2,2, 21 ooo xxX

0 5

1

1.5

2

21

0-1

-2-2

-1

0

1

-1

-0.8

-0.6

-0.4

2

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

9/14/2015

11

UnconstrainedOptimization Methodofconjugategradient:Insteadoforthogonality we use conjugacy betweenorthogonality,weuseconjugacy betweensuccessivesearchdirection Twodirectionssi andsj aresaidtobeconjugatewithrespecttoapositivedefinitematrixQ if(si)TQ(sj) =0

IfQ=I (identitymatrix),(si)T(sj)=0.Searchesarealongvectors in orthogonal coordinate spacevectorsinorthogonalco ordinatespace.

IfQ=H(Hessianmatrix),searchesarealongvectorsintransformedspacedefinedbyeigenvectorsofH.

UnconstrainedOptimization FirstsearchdirectionisS b l

MethodofConjugateGradient oo XfS

kkTkkk XfXf 1111 Subsequently, Determinehowfartomove, suchthatisminimumi.e. Supposeapproximationtosecondorderisused,i.e.

kkTkkk XfXf XfXfSXfS 11k kkk SXf 0 kkkk SXfdd kkTkkTkk1k SXHS21fSXfXf then, 2 kkTk

kTk

optSXHS

Sf

9/14/2015

12

UnconstrainedOptimization Minimize

i i l

MethodofConjugateGradient 32 2221 xxXf 22oooX Initialguess: 2,2, 21 ooo xxX

4

6

8

1

1.5

2

21

0-1

-2-2

-1

0

1

-4

-2

0

2

4

2

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

ConstraintOptimization Consideroptimizationproblemwithonlyequalityconstraintsconstraints,

),...,,()(,...,2,10)(

:

21 n

i

xxxXnmmiXg

XfMinimize

Di t S b tit ti L M lti liDirectSubstitutionEliminatethemvariablesbysubstitutionandsolvethe

unconstraintoptimizationprobleminnmvariables

LagrangeMultipliersUseLagrangemultiplierstoaugment

theobjectivefunctionwithconstraintsandthensolvethe

unconstrainedoptimizationproblem

9/14/2015

13

ConstrainedOptimization Minimize nxxxxXXf ,,,,; 321

mjbXh 21 0 jjj bXhXgMethodofLagrangeMultipliers

AugmenttheobjectivefunctionusingLagrangemultipliers(i)

Foroptimalpoint:

mjbXh jj ,,2,1

mmj

jjj bXhXfXL ,,,, 211

hfL m 0,XL

0jjj bXhXg

mjbXhLni

xh

xf

xL

jjj

i

jm

jj

ii

,,2,10

,2,101

Thereare(n+m)variablesand

(n+m)unknowns

ConstrainedOptimization Minimize: 2221 xxXf

MethodofLagrangeMultipliers

AugmentedLagrangeFunction

Foroptimalpoint(necessarycondition)is

022 21 xxXh 22),( 212221 xxxxXL

0LL

52,5

4

54022

202

022

*2

*1

21

222

111

xx

xxL

xxxL

xxxL

9/14/2015

14

ConstrainedOptimization Minimize nxxxxXXf ,,,,; 321 kX 21

MethodofLagrangeMultipliers

AugmenttheobjectivefunctionusingLagrangemultipliers

OptimalpointbyKarushKuhnTucker(KKT)Condition:

rkcXg kk ,,2,1 rkkkr

kk cXgXfXL ,,,, 21

1

XXf r ** rkcXgcXg

cXgnixXg

xXf

kkkkkk

kki

kr

kk

i

,,2,1;00

0;,,2,10

****

**

1

**

Thereare(n+r)variablesand(n+r)unknowns

Foractiveconstraint,gk(X*)=ckForinactiveconstraint,k*=0

ConstrainedOptimization Minimize 221 1ln1ln xxXf 02211 xxXg

MethodofLagrangeMultipliers

Augmenttheobjectivefunction

Optimal condition

0

002

23

12

211

xXgxXgxxXg

0,,21ln1ln, 2121221 xxxxxxXL

AtX*=(1/3,5/3)g1 isabindingoractive

constraint,whileg2 andg3 arenonbindingorinactiveconstraints

Optimalcondition

43,

35,

31

02

01

2;01

1

**2

*1

21

2211

xx

xxLxx

Lxx

L

9/14/2015

15

ConstraintOptimization Minimize

MethodofLagrangeMultipliers 22 12 yxXf

02or2;02 yxyxXgxyXg Augmenttheobjectivefunctionwithconstraints

Findoptimalpoint

02or 2;0 21 yxyxXgxyXg

0,,,212,

21

22

122

yxyxxyyxMXL

LL

3/2 point, optimal theis 1,1

1,4;1,202;0

012;0222

*2

*1

**

2

2

1

2121

yx

yxyxLyxL

yyLxx

xL

ConstrainedOptimization Minimize

and nxxxxXXf ,,,,; 321

jbXh 21 rkcxg 21 MethodofLagrangeMultipliers

and AugmentedLagrangefunctionis

Optimal point is

mjbXh jj ,,2,1 rkcxg kk ,,2,1

rmkk

r

kk

m

jjjj

M

cXgbXhXfMXL

,,,,,,,

,,

2121

11

Thereare(n+m+r)variablesand Optimalpointis

rkcXgcXgrkcXgmjbXh

nixXg

xXh

xXf

kkkkkk

kkjj

i

kr

kk

m

j i

jj

i

,,2,1;00

,,2,10;,,2,10

,,2,10

****

**

*

1

*

1

**

*

(n+m+r)unknowns

9/14/2015

16

ConstrainedOptimization Secondordersufficientcondition

MethodofLagrangeMultipliers

SUFFICIENTCONDITION:If(X*,*,*)isanoptimalpoint,thenif

y is vector orthogonal to gradients active at the optimal point

0

vector nonzero 0,,*

***2

yXJyyMXLy x

T

yisvectororthogonaltogradientsactive attheoptimalpointJismatrixwhoserowsaregradientsoftheconstraintsthatareactiveatX*.

ConstrainedOptimization Minimize:

MethodofLagrangeMultipliers 2221 12 xxXf

2x

AugmentedLagrangeFunctionis

Optimal point

012;014 21

22

1 xxXhxxXg

1

41212,, 22

21

212

22

1 xxxxxxXL

Optimalpoint

471,

271

014

;012

2212;2122

*2

*1

22

21

21

222

111

xxxxLxxL

xxxLxx

xL

9/14/2015

17

ConstrainedOptimization Bothconstraintsareactive;

MethodofLagrangeMultipliers

37523

Forsecondordercondition23

287;

25

7223 **

37

230

037

2341

)1(20

02

2,, ***2

XLx

7

212

714

71

2122 *2

*1* xxXJ

0212

714

710

2

1*

yy

yXJ

ConstrainedOptimization Therefore,

1221a

yyayy*

2323

)1(22

2

1)1(20

02

21

2

**

222

2*

*

22

ay

ayayyLy x

T

minima

037

2337

234

222

ay

9/14/2015

18

ConstrainedOptimization Minimize:

MethodofLagrangeMultipliers 2221 1 xxXf

AugmentedLagrangeFunctionis:

ByKKTcondition:

0221 xx

2212221 1, xxxxXL 012 1 xL

;2

1,21,1

;0,0,2

0,0

0

022

012

*2

*1

*

*2

*1

*

221

221

222

11

xx

xx

xx

xxL

xxxL

xx

Wehavethreesolutionsbutonlytwoarevalid:(0,0)and(1/2,1/2)

ConstrainedOptimization CheckSecondOrderSufficientCondition:

Fi d H i t i

MethodofLagrangeMultipliers

FindHessianmatrix

Forthepoint(0,0), =2,

12002

22

2

21

221

2

21

2

2

xL

xxL

xxL

xL

Lx

20022Lx

Theconstraintgivenisactive

Check

20 21

2

1*2

* any ,001)(0121)( yyyy

yXJxXJ

02020

020 22

22

2

yyyyLy x

T

Foranyy2,itisnegativedefiniteimpliesmaxima

9/14/2015

19

ConstrainedOptimization CheckSecondOrderSufficientCondition:

Fi d H i t i

MethodofLagrangeMultipliers

FindHessianmatrix

Forthepoint(1/2,1/2), =1,

12002

22

2

21

221

2

21

2

2

xL

xxL

xxL

xL

Lx

00022Lx

Theconstraintgivenisactive

Check

00 0221)(2121)( 21

2

1*2

*

yyyy

yXJxXJ

0412

0002

12 22222

yyyyLy xTForanyy2,itispositivedefiniteimpliesminima

LinearProgrammingProblem Minimize 2121 54, xxxxf

GraphicalMethod

M i @ (2 33 1 33)

2

3

4

024;1;52;62

21

21

21

21

xxxx

xxxxxx

52 21 xx

1 xx

Maxima@(2.33,1.33)=16.00

-3 -2 -1 0 1 2 3 4-1

0

10, 21 xx 62 21 xx121 xx

24 21 xxMinima@(0.67,0.33)

=4.33

9/14/2015

20

LinearProgrammingProblem Simplexmethodwithslack,surplus andartificial variables

SimplexMethod

;6262 xxxxx

Slack variables x x

2424;11

;5252;6262

862121

752121

42121

32121

xxxxxxxxxxxx

xxxxxxxxxx

Slack variables x3,x4 Surplusvariables x5,x6 Artificialvariables x7,x8

LinearProgrammingProblem Formulatetheartificialproblem

SimplexMethod

:Minimize xxxxg

Substitute the artificial variables in terms of independent

24;1

;52;62

,:Minimize

8621

7521

421

321

8787

xxxxxxxx

xxxxxx

xxxxg

Substitutetheartificialvariablesintermsofindependent,slackandsurplusvariables

3)0()0()0()0(520421 :Minimize

87654321

216215

xxxxxxxxxxxxxxXg

9/14/2015

21

Formthesimplextable:choosebasisBasis x1 x2 x3 x4 x5 x6 x7 x8 b

LinearProgrammingProblemSimplexMethod

Basis x1 x2 x3 x4 x5 x6 x7 x8 bx3 2 1 1 0 0 0 0 0 6x4 1 2 0 1 0 0 0 0 5x7 1 1 0 0 -1 0 1 0 1x8 1 4 0 0 0 -1 0 1 2g -2 -5 0 0 1 1 0 0 -3f 4 5 0 0 0 0 0 0 0

6/1=6

5/2=2.5

1/1=1

2/4=0.5

Pickthevariablewithmostnegativecoefficientintheobjectivefunctionthisisthevariabletoenterbasis

Modifyrighthandsidebbydividingbycoefficientsofthisvariable. Theequationwithleastr.h.s.aftermodificationistobereformulated Inthiscase,variablex2 intoenterthebasisandx8 istoleave.

f

Formthesimplextable:changebasisBasis x1 x2 x3 x4 x5 x6 x7 x8 b

LinearProgrammingProblemSimplexMethod

Basis x1 x2 x3 x4 x5 x6 x7 x8 bx3 1 0 0x4 0 1 0x7 0 0 1x2 1/4 1 0 0 0 -1/4 0 1/4 2/4g 0 0 1f 0 0 0

Leavingotherbasisvectorsassuchchangex2 tobasisvector. Coefficientofx2 ismade1bydividingtheequationentireequationby4.

f

9/14/2015

22

Formthesimplextable:withnewbasisBasis x1 x2 x3 x4 x5 x6 x7 x8 b

LinearProgrammingProblemSimplexMethod

Basis x1 x2 x3 x4 x5 x6 x7 x8 bx3 7/4 0 1 0 0 0 -1/4 11/2x4 0 0 1 0 0 -1/2 4x7 0 0 0 -1 1 -1/4 x2 1 0 0 0 -1/4 0 g -3/4 0 0 0 1 -1/4 1 5/4 -1/2f 11/4 0 0 0 0 5/4 0 -5/4 -5/2

Leavingotherbasisvectorsassuchchangex2 tobasisvector. Coefficientofx2 ismade1bydividingtheequationentireequationby4. Coefficientofx2 ismade0inotherequationsbyrowoperations.

Nowwerepeattheprocesstogetthenewbasis

f

Formthesimplextable:changebasisBasis x1 x2 x3 x4 x5 x6 x7 x8 b

LinearProgrammingProblemSimplexMethod

1 2 3 4 5 6 7 8x3 7/4 0 1 0 0 0 -1/4 11/2x4 0 0 1 0 0 -1/2 4x7 0 0 0 -1 1 -1/4 x2 1 0 0 0 -1/4 0 g -3/4 0 0 0 1 -1/4 1 5/4 -1/2f 11/4 0 0 0 0 5/4 0 -5/4 -5/2

22/7

8

2/3

2

Pickthevariablewithmostnegativecoefficientintheobjectivefunction thisisthevariabletoenterbasis

Modifyrighthandsidebbydividingbycoefficientsofthisvariable. Theequationwithleastr.h.s.aftermodificationistobereformulated Inthiscase,variablex1 intoenterthebasisandx7 istoleave. Continueuntilnomorenegativecoefficientsarelefting(X)

9/14/2015

23

Formthesimplextable:withnewbasisBasis x1 x2 x3 x4 x5 x6 x7 x8 B

LinearProgrammingProblemSimplexMethod

Basis x1 x2 x3 x4 x5 x6 x7 x8 Bx3 0 0 1 0 7/3 -1/3 -7/3 1/3 13/3x4 0 0 0 1 2/3 1/3 -2/3 -1/3 11/3x1 1 0 0 0 -4/3 1/3 4/3 -1/3 2/3x2 0 1 0 0 1/3 -1/3 -1/3 1/3 1/3g 0 0 0 0 0 0 1 1 0f 0 0 0 0 11/3 1/3 -11/3 -1/3 -13/3

Thevalueofthedecisionatminimumis(2/3,1/3) Theminimumvalueis13/3or4.33

f

References

Edgar,T.F.,Himmelblau,D.M.andLasdon,L.S.,Optimization of Chemical Processes 2nd EditionOptimizationofChemicalProcesses,2nd Edition,McGrawHill,2001.

Peters,M.S.,Timmerhaus,K.andWest,R.E.,PlantDesignandEconomicsforChemicalEngineers,McGrawHillEducation,5th Edition,2002.

h d h Mohan,C.andDeep,K.,OptimizationTechniques,NewAgeInternationalPublishers,1st Edition,2009.