01 Zhang Slides KKT Good

8/6/2019 01 Zhang Slides KKT Good

1/32

The KuhnThe Kuhn--Tucker conditionsTucker conditions

HaoHao ZhangZhang

Computational mechanicsComputational mechanics


2/32

OverviewOverview

General caseGeneral case

ExampleExample

Convex problemConvex problem


3/32

General caseGeneral case

General form of optimization problemsGeneral form of optimization problems

j

minimize ( );such that ( ) 0; 1,...,

h ( ) 0; 1,...,

n

j

f x x Rg x j p

x j q

=

= =


4/32

This problem may have several localThis problem may have several localminima.minima.

The necessary conditions for a minimumThe necessary conditions for a minimumof the constrained problem are obtainedof the constrained problem are obtained

by using theby using the

Lagrange multiplier methodLagrange multiplier method

.

.


5/32

LagrangianLagrangian functionfunction

are unknown Lagrange multipliers.are unknown Lagrange multipliers.

Only consider the special case of equality constraints.Only consider the special case of equality constraints.

( )1

, ( ) ( )

en

j j

j x f x h x

== L

j


6/32

The necessary conditions for a stationaryThe necessary conditions for a stationarypoint arepoint are

Apply only at theApply only at the regular pointregular point..

1

0en

j

j

ji i i

hf

x x x

=

= =

L

( ) 0j

j

h x

= =

L

1,...,i n=

1,..., ej n=


7/32

Regular pointRegular point

The gradients of the constraints areThe gradients of the constraints arelinearly independent at that point.linearly independent at that point.


8/32

Now we take the inequality constraints intoNow we take the inequality constraints into

account.account. First, we transform the inequality constraints toFirst, we transform the inequality constraints to

equality constraints by adding slack variables.equality constraints by adding slack variables.

is a slack variable which measures how faris a slack variable which measures how farthethejthjth constraint is from being critical.constraint is from being critical.

2

( ) 0,j jg x t = 1,...,gj n=

jt


9/32

Form theForm the LagrangianLagrangian functionfunction

Differentiating theDifferentiating the LagrangianLagrangian functionfunction

with respect to x,with respect to x,

,t,t

2

1

( , , ) ( )

gn

j j j

j

x t f g t =

= L


10/32

This three equations are the necessaryThis three equations are the necessary

conditions for a stationary regular point.conditions for a stationary regular point.

1

0,gn

j

g

ji i i

gf

x x x

=

= =

L

1,...,i n=

1,...,g

j n=

1,..., gj n=

2 0,j j

j

g t

= + =

L

2 0,j j

j

tt

= =

L


11/32

KuhnKuhn--Tucker conditionsTucker conditions A point x is a local minimum of an inequalityA point x is a local minimum of an inequality

constrained problem only if a set of nonnegativeconstrained problem only if a set of nonnegativejjss may be found such that:may be found such that:

1.1.

is satisfiedis satisfied

2.The corresponding2.The corresponding jj is zero if a constraint isis zero if a constraint isnot active.not active.

1

0,g

n

j

g

ji i i

gf x x x

=

= =

L 1,...,i n=


12/32

Geometrical interpretationGeometrical interpretation


13/32

A geometrical interpretation of the KuhnA geometrical interpretation of the Kuhn--Tucker conditions isTucker conditions is

illustrated in Fig 1 for the case of two constraints. anillustrated in Fig 1 for the case of two constraints. andddenote the gradients of the two constraints which are orthogonaldenote the gradients of the two constraints which are orthogonal toto

the respective constraint surfaces. The vector s shows a typicalthe respective constraint surfaces. The vector s shows a typical

feasible direction which does not lead immediately to any constrfeasible direction which does not lead immediately to any constraintaint

violation. For the twoviolation. For the two--constraints case. We getconstraints case. We get

Assume that we want to determine whether point A is aAssume that we want to determine whether point A is aminimum or not.minimum or not.

To improve the design we proceed from A in a directionTo improve the design we proceed from A in a direction

s that iss that is usableusable andand feasiblefeasible..

1 1 2 2( ) f g g = +

1g 2g


14/32

UsableUsable

A small move along theA small move along thedirection should decreasedirection should decrease

the f, so it must form anthe f, so it must form an

acute angle withacute angle with

FeasibleFeasible

This direction should formThis direction should forman obtuse angle withan obtuse angle with

andand

f

1g

2g

The direction sThe direction s


15/32

Any vector which forms an acute angle withAny vector which forms an acute angle with

will also form an acute angle withwill also form an acute angle witheither oreither or

f

1g

2g


16/32

MathematicallyMathematically

The condition for a feasible directionThe condition for a feasible direction

The condition for a usable directionThe condition for a usable direction

So we obtainSo we obtain

It is impossible if theIt is impossible if the

jjss are positive.are positive.

0,Tj As g j I

0Ts f

0,Tj

s h = 1,..., ej n=


21/32

When inequality constraints are present,When inequality constraints are present,the vector s also needs to be orthogonalthe vector s also needs to be orthogonal

to the gradients of the active constraintsto the gradients of the active constraints

with positive Lagrange multipliers. Forwith positive Lagrange multipliers. Foractive constraints with zero Lagrangeactive constraints with zero Lagrange

multipliers, s must satisfymultipliers, s must satisfy

when andwhen and0,T

js g 0jg

=0j

=


22/32

ExampleExample

Find the minimum ofFind the minimum of

Subject toSubject to

The KuhnThe Kuhn--Tucker conditions areTucker conditions are

3 2 3

1 2 1 22 10 6 2 f x x x x= +

1 1 2

2 1

3 2

10 0,

0,

10 0

g x x

g x

g x

=

=

=

2

1 1 2 2

2

2 2 1 1 3

3 10 0

4 6 0

x x

x x x

+ + =

+ + =


23/32

We have to check for all possibilities of activeWe have to check for all possibilities of activeconstraints.constraints.

1. No constraints are active,1. No constraints are active,we get xwe get x11=1.826,x=1.826,x22=0,f=6.17=0,f=6.17

The Hessian matrix of theThe Hessian matrix of the LagrangianLagrangian,,

is clearly negative definite, so that this point isis clearly negative definite, so that this point isa maximum.a maximum.

1 2 3 0 = = =

1 12

1 2

6

4 12

xL

x

=


24/32

2.2. The first constraint is active,The first constraint is active,

so that and gso that and g22 is inactive andis inactive and

The third constraintThe third constrainta. Activea. Active

we get xwe get x11=1,x=1,x

22=10,=10,

11==--0.70.7,,

33=639.3=639.3

so this point is neither a minimum nor aso this point is neither a minimum nor a

maximum.maximum.

1 2 10x x =

1 0x 2 0 =

b I ib I ti


25/32

b. Inactiveb. Inactive

33=0 we obtain=0 we obtain

xx11=3.847, x=3.847, x22=2.599,=2.599, 11=13.24, f==13.24, f=--73.0873.08

This point satisfies the KT condition for a minimum.This point satisfies the KT condition for a minimum.The Hessian of theThe Hessian of the LagrangianLagrangian at that pointat that point

is negative definite, so that it cannot satisfy the sufficiencyis negative definite, so that it cannot satisfy the sufficiency

condition.condition.

2

1 1 2

2

2 2 1 1

1 2

3 10 0

4 6 0

10

x x

x x x

x x

+ + =

+ =

=

223.08 13.24

13.24 35.19L

=


26/32

3. g3. g11 is not active, so thatis not active, so that 11=0, and=0, and

a. then ga. then g22=0 and we get=0 and we get

xx11=0, x=0, x

22=0,=0,

22=10, f==10, f=--66

oror

xx11=0, x=0, x22==--2/3,2/3, 22=10, f==10, f=--6.996.99

Both points satisfy the KT conditions for aBoth points satisfy the KT conditions for a

minimum, but not the sufficiency conditions.minimum, but not the sufficiency conditions.

2

1 2

2

2 2 3

3 10 0

4 6 0

x

x x

+ =

+ =2 30, 0 =


27/32

b. so that gb. so that g33=0.=0.

We get xWe get x11=1.826, x=1.826, x22=10,=10, 33=640, f==640, f=--21942194

This point satisfies the KT conditions, but it isThis point satisfies the KT conditions, but it is

not a minimum either. It is easy to check thatnot a minimum either. It is easy to check thatis negative definite in this case, so that theis negative definite in this case, so that the

sufficiency condition could not be satisfied.sufficiency condition could not be satisfied.

Finally, we consider the case xFinally, we consider the case x11=0, x=0, x22=10,=10,

22=10,=10, 33=640, f==640, f=--2206.2206.now the KT conditions are satisfied, and thenow the KT conditions are satisfied, and the

number of active constraints is equal to thenumber of active constraints is equal to the

number of design variables, it is the minimum.number of design variables, it is the minimum.

2 30, 0 =

2 L


28/32

Convex ProblemsConvex Problems There is a class of problems namelyThere is a class of problems namely

convex problems, for which the Kuhnconvex problems, for which the Kuhn--

Tucker conditions are not only necessaryTucker conditions are not only necessary

but also sufficient for a global minimum.but also sufficient for a global minimum.


29/32

A set of points S is convex whenever theA set of points S is convex whenever the

entire line segment connecting two pointsentire line segment connecting two pointsthat are in S is also in S. That isthat are in S is also in S. That is

if thenif then

A function is convex ifA function is convex if

It can be shown that a function of nIt can be shown that a function of nvariables is convex if its matrix of secondvariables is convex if its matrix of secondderivatives is positive semiderivatives is positive semi--definite.definite.

1 2, , x x S 1 2(1 ) , 0 x x S 1 +

Date post:	07-Apr-2018
Category:	Documents
Upload:	iqbal1987
View:	226 times
Download:	0 times

01 Zhang Slides KKT Good

Documents