Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 1 of 22

Constrained Optimization

Objective of Presentation: To introduce Lagrangean as a basic conceptual method used to optimize design in real situations

Essential Reality: In practical situations, the designers are constrained or limited by

physical realities design standards laws and regulations, etc.

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 2 of 22

Outline Unconstrained Optimization (Review)

Constrained Optimization – Lagrangeans Approach Lagrangeans as Equality constraints

Interpretation of Lagrangeans as “Shadow prices”

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 3 of 22

Unconstrained Optimization: Definitions

Optimization => Maximum of desired quantity, or => Minimum of undesired quantity

Objective Function = Formula to be optimized= Z(X)

Decision Variables = Variables about whichwe can make decisions= X = (X1….Xn)

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 4 of 22

Unconstrained Optimization: Graph

B and D are maxima A, C and E are minima

A

B

C

D

E

F(X)

X

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 5 of 22

Unconstrained Optimization: Conditions By calculus: if F(X) continuous, analytic

Primary conditions for maxima and minima: F(X) / Xi = 0 i

( symbol means: “for all i”)

Secondary conditions:2F(X) / Xi

2 < 0 = > Max (B,D)2F(X) / Xi

2 > 0 = > Min (A,C,E)These define whether point of no change in Z is a

maximum or a minimum

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 6 of 22

Unconstrained Optimization: Example Example: Housing insulation

Total Cost = Fuel cost + Insulation costx = Thickness of insulation

F(x) = K1 / x + K2x

Primary condition: F(x) / x = 0 = -K1 / x2 + K2

=> x* = {K1 / K2} 1/2

(starred quantities are optimal)

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 7 of 22

Unconstrained Optimization: Graph of Solution to Example

Optimizing Cost Example

0100

200300400

500600

1 3 5 7 9 11

Inches of Insulation

Cost

FuelInsulationTotal

If: K1 = 500 ; K2 = 24 Then: X* = 4.56

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 8 of 22

Constrained Optimization: General “Constrained Optimization” involves the

optimization of a process subject to constraints

Constraints have two basic types Equality Constraints -- some factors have to

equal constraints Inequality Constraints -- some factors have to

be less less or greater than the constraints (these are “upper” and “lower” bounds)

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 9 of 22

Constrained Optimization: General Approach

To solve situations of increasing complexity, (for example, those withequality, inequality constraints) …

Transform more difficult situation into one we know how to deal with

Note: this process introduces new variables!

Thus, transform “constrained” optimization to “unconstrained”

optimization

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 10 of 22

Equality Constraints: Example Example: Best use of budget Maximize: Output = Z(X) = a0x1

a1x2a2

Subject to (s.t.):Total costs = Budget = p1x1 + p2x2

Z(x) Z*

Budget X

Note: Z(X) / X 0 at optimum

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 11 of 22

Lagrangean Method: Approach Transforms equality constraints into

unconstrained problem Start with:

Opt: F(x)s.t.: gj(x) = bj => gj(x) - bj = 0

Get to: L = F(x) - j j[gj(x) - bj]j = Lagrangean multipliers (lambdas sub j) -- are unknown quantities for which we must solve

Note: [gj(x) - bj] = 0 by definition, thus optimum for F(x) = optimum for L

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 12 of 22

Lagrangean: Optimality Conditions

Since the new formulation is a single equation, we can use formulas for unconstrained optimization.

We set partial derivatives equal to zero for all unknowns, the X and the

Thus, to optimize L:L / xi = 0 I

L / j = 0 J

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 13 of 22

Lagrangean: Example Formulation Problem:

Opt: F(x) = 6x1x2s.t.: g(x) = 3x1 + 4x2 = 18

Lagrangean: L = 6x1x2 - (3x1 + 4x2 - 18)

Optimality Conditions: L / x1 = 6x2 - 3 = 0

L / x2 = 6x1 - 4 = 0L / j = 3x1 + 4x2 -18 = 0

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 14 of 22

Lagrangean: Graph for Example

Isoquants for F(X) = 20 and 40 and Constraint

-2.00-1.000.001.002.003.004.005.006.007.008.00

1 2 3 4 5

X (sub 1)

X (s

ub 2

)

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 15 of 22

Lagrangean: Example Solution Solving as unconstrained problem: L / x1 = 6x2 - 3 = 0

L / x2 = 6x1 - 4 = 0L / i = 3x1 + 4x2 -18 = 0

so that: = 2x2 = 1.5x1 (first 2 equations) => x2 = 0.75x1

=> 3x1 + 3x1 - 18 = 0 (3rd equation) x1* = 18/6 = 3 x2* = 18/8 = 2.25 * = 4.5 F(x)* = 40.5

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 16 of 22

Lagrangean: Graph for Solution

Isoquants for F(X) = 20 and 40 and Constraint

-2.00-1.000.001.002.003.004.005.006.007.008.00

1 2 3 4 5

X (sub 1)

X (s

ub 2

)

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 17 of 22

Shadow Price Shadow Price = Rate of change of objective

function per unit change of constraint= F(x) / bj = j

It is extremely important for system design

It defines value of changing constraints, and indicates if worthwhile to change them

Should we buy more resources? Should we change environmental constraints?

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 18 of 22

Lagrangean Multiplier is a Shadow Price

The Lagrangean multiplier is interpreted as the shadow price on constraint

SPj = F(x)*/ bj = L*/ bj = {F(x) - j j[gj(x) - bj] } / bj = j

Naturally, this is an instantaneous rate

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 19 of 22

Lagrangean = Shadow Prices Example

Let’s see how this works in example, by changing constraint by 0.1 units:

Opt: F(x) = 6x1x2s.t.: g(x) = 3x1 + 4x2 = 18.1

The optimum values of the variables are x1* = (18.1)/6 x2* = (18.1)/8 * = 4.5

Thus F(x)* = 6(18.1/6)(18.1/8) = 40.95

F(x) = 40.95 - 40.5 = 0.45 = * (0.1)

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 20 of 22

Generalization In general constraints are “inequalities”:

Upper bounds : gj(x) < bj Lower bounds: gj(x) > bj

At optimum, some constraints will limit solution (they are “binding”) others not

Example: airline bags: weight < 40kg ; sum of dimensions < 2.5m. Your bag might be limited by weight, not by size.

Shadow prices > 0 for all “binding” constraints= 0 for all others

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 21 of 22

Design Implications Expanding range of design variables (x),

increases freedom to improve design, thus adds value

This is called “relaxing” the constraints Increasing upper bounds Decreasing lower bounds

As any constraint relaxed, it may no longer be “binding” , and others can become so

SHADOW PRICES DEPEND ON OTHER CONSTRAINTS, “PROBLEM DEPENDENT”

Engineering Systems Analysis for Design Richard de Neufville © Massachusetts Institute of Technology Constrained Optimization Slide 22 of 22

Take-aways

Relaxing design constraints adds value (in terms of better performance, F(x) )

This value is the “shadow price” of that constraint

Knowing this can be very important for designers, shows way to improve quickly

NOTE: Value to design has no direct connection to cost of constraint, not a “price in ordinary terms