8/11/2019 L7 (2013)

1/41

8/11/2019 L7 (2013)

2/41

8/11/2019 L7 (2013)

3/41

3

Optimization GeneralTypical problems

In what sequence should parts be produced on amachine in order to minimize the change-over time?

How can a dress manufacturer lay out its patterns on

rolls of cloth to minimize wasted material?

How many elevators should be installed in a new office

building to achieve an acceptable expected waitingtime?

What is the lowest-cost formula for chicken food whichwill provide required quantities of necessary minerals

and other nutrients?

What is the generation plan for the power plants duringthe next six hours?

8/11/2019 L7 (2013)

4/41

4

Optimization - GeneralThe general optimization problem:

min ( )

s.t. ( )( )

f x

g x bh x c

x x x

x x

f(x): objective function x: optimization variables (vector)

g(x),h(x): constraints (vectors)

and : variabel limits (vector)

Defines the feasible set

8/11/2019 L7 (2013)

5/41

5

Optimization Formulation

Formulation of optimization problems:

1. Describe the problem in words: Think through the problem

2. Define symbols Review the parameters and variables

involved

3. Write the problem mathematically Translate the problem defined in words

to mathematics

8/11/2019 L7 (2013)

6/41

6

Optimization FormulationThe problem:

A manufacturer owns a number of factories. How

should the manufacturer ship his goods to thecustomers?

?

8/11/2019 L7 (2013)

7/417

Optimization Formulation

1. The problem in words:

How should the merchandise be shipped inorder to minimize the transportation costs? Thefollowing must be fulfilled:

The factories cannot produce more thantheir production capacities.

The customers must at least receive theirdemand

8/11/2019 L7 (2013)

8/41

8

2. Define symbols:

Index and parameters:

mfactories, factory ihas capacity aincustomers, customer jdemand bjunits of thecommodity

The transportation cost from factory ito customerjis cijper unit

VariablesLet xij, i = 1...m, j = 1...ndenote the number of unitsof the commodity shipped from factory itocustomer j

Optimization Formulation

8/11/2019 L7 (2013)

9/41

9

3. Mathematical formulation:

Objective function

Transportation cost:

Constraints

Production capacity:

Demand:

Variable limits:

m

i

n

j

ijijxc1 1

miax i

n

j

ij ...11

njbx j

m

i

ij ...11

njmixij ...1,...10

Optimization Formulation

8/11/2019 L7 (2013)

10/41

10

The optimization problem:

njmix

njbx

miax

xc

ij

j

m

iij

i

n

j

ij

m

i

n

j

ijij

...1,...1,0

...1,

...1,s.t.

min

1

1

1 1

Optimization Formulation

8/11/2019 L7 (2013)

11/41

11

LP GeneralLinear Programming problems (LP problems)

Class of optimization problems with linear objectivefunction and constraints

All LP problems can be written as (standard form):

0

s.t.

min

x

bAx

xcT

8/11/2019 L7 (2013)

12/41

12

LP General LP problems will here be explained through

examples (similar as in Appendix A)

Will have to look at the following:

Extreme points

Slack variables

Non existing solution

Not active constraints

No finite solution or unbounded problems

Degenerated solution

Flat optimum Duality

Solution methods

8/11/2019 L7 (2013)

13/41

13

LP Example The problem:

0,0

12124

2s.t.

105min

21

21

21

21

xx

xx

xx

xxz

8/11/2019 L7 (2013)

14/41

14

LP Example

Optimum in:

x1+x224x1+12x212

x10

x20

z = 20

z = 30

z = 40

Objective:

5.0

5.1

2

1

x

x

5.12z

x2

1

2

3

4

x1

1 2 3 4

8/11/2019 L7 (2013)

15/41

15

LP Extreme points

The corners of thefeasible set are called

extreme points Optimum is always

reached in one or moreextreme points!

x1+x224x1+12x212

x10

x20

z = 20

z = 30

z = 40

x2

1

2

3

4

x1

1 2 3 4

8/11/2019 L7 (2013)

16/41

16

LP Slack variables LP problem on standard form:

Introduce slack variablesin order to write

inequality constraints using equality

0

s.t.

min

x

bAx

xcT

8/11/2019 L7 (2013)

17/41

17

LP Slack variables Without slack variables:

(Ax b)

0,0

12124

2

21

21

21

xx

xx

xx

0,0,0,0

12124

2

4321

421

321

xxxx

xxx

xxx With slack variables:

(Ax = b)

8/11/2019 L7 (2013)

18/41

18

Can happen that the constraints makes nosolution feasible! (Bad formulation of problem)

0,01

12124

2s.t.

105min

21

21

21

21

21

xxxx

xx

xx

xxz

New constraint

LP Non existing solution

8/11/2019 L7 (2013)

19/41

19

B: Feasible setdefined by (c)

A: Feasible set definedby (a)& (b)

x1+x224x1+12x212

x1 0

x20

x2

1

2

3

4

x1

1 2 3 4

(a) (b)

x1+x21

(c)

Feasibleset isempty!

LP Non existing solution

A B

8/11/2019 L7 (2013)

20/41

20

LP Not active constraints Some constraints might not be activein

optimum.

0,07

12124

2s.t.105min

21

21

21

21

21

xxxx

xx

xxxxz

New constraint

8/11/2019 L7 (2013)

21/41

21

x2

x1

1

2

3

4

1 2 3 4

x1

+x2

2

x20

4x1+12x212

x10

z = 20

z = 30

z = 40

x1+x27

Not active constraints

Activeconstraints

LP Not active constraints

8/11/2019 L7 (2013)

22/41

22

LP No finite solution If the constraints dont limit the objective: |z|

0,0

12124

2s.t.min

21

21

21

21

xx

xx

xxxxz

New objective function

8/11/2019 L7 (2013)

23/41

23

x1+x22

x20

4x1+12x212

x10

x2

x1

1

2

3

4

1 2 3 4

z = -5

z = -4

z = -3

minz -

LP No finite solution

8/11/2019 L7 (2013)

24/41

24

LP Degenerated solution More than one extreme point can be optimal. All

linear combinations of these points are then alsooptimal

0,0

12124

2s.t.

1010min

21

21

21

21

xx

xx

xx

xxzNew objective

8/11/2019 L7 (2013)

25/41

25

x1+x22

x20

4x1+12x212

x10

x2

1

2

3

4

1 2 3 4

z = 30z = 40

z = 50The line betweenthe extremepoints are alsooptimal solutions

x1

LP Degenerated solution

8/11/2019 L7 (2013)

26/41

26

LP Flat optimum A number of extreme points can have almost the

same objective function value.

0,0

12124

2s.t.

99.910min

21

21

21

21

xx

xx

xx

xxz

New objective

8/11/2019 L7 (2013)

27/41

27

LP Flat optimum

x1+x22

x20

4x1+12x212

x10

x2

1

2

3

4

1 2 3 4

z = 30z = 40

z = 50

x1

z = 19.980

z = 19.995

8/11/2019 L7 (2013)

28/41

28

LP Duality All LP problems (primalproblem) have a corresponding dual

problem

are called dual variables

0

s.t.

min

x

bAx

xcT

0

s.t.

max

cA

b

T

T

Primal problem: Dual problem:

8/11/2019 L7 (2013)

29/41

29

LP DualityTheorem (strong duality):

If the primal problem has an optimal

solution, then also the dual problemhas an optimal solution and theobjective values of these solutions arethe same.

8/11/2019 L7 (2013)

30/41

30

LP Duality Q: What does the dual variables describe?

A: Dual variables = Marginal value of theconstraint that the dual variable represents, i.e.how the objective function value changes when

the right-hand side of the constraint changes

0

s.t.

min

x

bAx

xcT

One dual variablefor each constraint!

8/11/2019 L7 (2013)

31/41

31

LP Dualityx2

x1

1

2

3

4

1 2 3 4

x1+x22

x20

4x1+12x212

x10

z = 20

z = 30

z = 40

1

2

x1+x27

3In optimum:

1 > 0 (active)2 > 0 (active)

3 = 0 (not active)

8/11/2019 L7 (2013)

32/41

32

Dual variables

Smallchanges in right-hand side b

Change in objective function value z:

LP Duality

z = Tb

8/11/2019 L7 (2013)

33/41

33

MILP GeneralMixed Integer Linear Programming problems (MILP

problems)

Class of optimization problems with linear objective

function and constraints Some variables can only take integer values

1

2

1

2

1

2

min

s.t.

0

0,1,2,...

T xcx

xA bx

x

x

8/11/2019 L7 (2013)

34/41

34

MILP Example

Optimum in:

x1+x224x1+12x212

x10

x20

Objective:

)5.0(1

)5.1(1

2

1

x

x

)5.12(15z

x2

1

2

3

4

x1

1 2 3 4

z = 20

z = 30

z = 40

8/11/2019 L7 (2013)

35/41

35

MILP Solving Easy to implement integer variables.

Hard to solve the problems.

Execution times can increase exponentiallywith the number of integer variables

Avoid integer variables if possible!

Special case of integer variables: Binary variables

x {0,1}

8/11/2019 L7 (2013)

36/41

36

MILP Example Minimize the cost for buying something

Variable, x: Quantity of something, x 0.

Not constant cost per unit - cost function.

For example: Discount when buying more than aspecified quantity:

Cost [SEK]

Numberof units

xb

x1x2 Splitxinto two

different variables,

x1andx2. Observethat

x1 xb. Also note that

bothx1andx2are 0.

8/11/2019 L7 (2013)

37/41

37

MILP Example

Cheaper to buy in segment 2 Must force theproblem to fill the first segment before enteringsegment 2.

Can be performed by introducing a binaryvariable.

Cost [SEK]

Numberof units

xb

x1x2

8/11/2019 L7 (2013)

38/41

38

MILP Example

Introduce the followingconstraints:

0

0

2

1

Msx

sxx bs=0 s=1

Cost [SEK]

Numberof units

xb

x1x2

Check ifs=0: 000,0 2221 xxxx

Check ifs=1:

MxMx

xxxxxx bbb

22

111

0

,0

whereMis an arbitrarely large number.

Letsbe an integer variableindicating whether being insegment 1 or 2.

8/11/2019 L7 (2013)

39/41

39

LP Solution methods Simplex:

Returns optimum in extreme point (also when

degenerated solution)

Interior point methods:

If degenerated solution: Returns solution between two

extreme points

8/11/2019 L7 (2013)

40/41

40

LP Solution methods

x1+x

22

x20

4x1+12x212

x10

x2

1

2

3

4

1 2 3 4

z = 30

z = 40

z = 50

Simplex

Interior point method

8/11/2019 L7 (2013)

41/41

PROBLEM 11 - Solutions toLP problems

Assume that readymade software is used to

solve an LP problem. In which of the following

cases do you get an optimal solution to the LP

problem and in which cases do you have toreformulate the problem (or correct an error in

the code).

a) The problem has no feasible solution.

b) The problem is degenerated.c) The problem does not have a finite solution.