Math Prog

8/11/2019 Math Prog

http://slidepdf.com/reader/full/math-prog 1/44

1

IS703:

Decision Support and Optimization

Week 5: Mathematical Programming

Lau Hoong Chuin

School of Information Systems

8/11/2019 Math Prog


2

Mathematical Programming - Scope

• Linear Programming

• Integer Programming

• Network flows (an introduction in previous class)

• Not Covered: – Convex Programming

– Non-Linear Programming

– etc

8/11/2019 Math Prog


3

Linear Programming

( Adapted from http://www.cs.princeton.edu/~wayne/cs423/lectures/lp.ppt )

Powerful model generalizes many classic problems:

– shortest path, max flow, multicommodity flow, MST, matching, 2-person zero sum games

Essential tool for optimal allocation of scarce resources, among a

number of competing activities.

Ranked among most important scientific advances of 20th century. – accounts for a major proport ion of all scientific computation

Helps find "good" solutions to NP-hard optimization problems.

– optimal solutions (branch-and-cut)

– provably good solutions (randomized rounding)

8/11/2019 Math Prog


4

Linear Programming

LP "standard" form.

Input data: rational numbers c j, b i, aij .

Maximize linear objective function.

Subject to linear inequalit ies.

n jx

mibxa

xc

j

in

j jij

n

j j j

≤

≤

∑

10

1t.s.

max(P)

1

1

0t.s.

max(P)

≥

≤

•

x b Ax

x c

8/11/2019 Math Prog


5

LP: Geometry

Geometry.

Forms an n-dimensional

polyhedron.

Convex: if y and z are feasible solutions, then so is αy + (1-α)z, for all 0<α<1.

n jx

mibxa

xc

j

i

n

j jij

n

j j j

≤

≤

∑

10

1t.s.

max(P)

1

1

z

y

z

y

Convex Not convex

Extremepoints

8/11/2019 Math Prog


6

LP: Geometry

Extreme Point Theorem. If there exists an optimal solution tostandard form LP (P), then there exists one that is an extremepoint.

Only need to consider finitely many possible solutions.

Greed. Local optima areglobal optima.

8/11/2019 Math Prog


7

LP: Algorithms

Simplex. (Dantzig 1947) Developed shortly after WWII in response to logistical

problems:

used for 1948 Berlin airli ft.

Practical solution method that moves from one extreme pointto a neighboring extreme point.

Finite (exponential) complexity, but no polynomial

implementation known.

8/11/2019 Math Prog


8

LP: Polynomial Algorithms

Ellipsoid. (Khachian 1979, 1980)

Solvable in polynomial time: O(n4 L) bit operations.

– n = # variables

– L = # bits in input

Theoretical tour de force.

Not remotely practical.

Karmarkar's algorithm. (Karmarkar 1984)

O(n3.5 L).

Polynomial and reasonably efficient

implementations possible.

Interior point algorithms. O(n3 L).

Competitive with simplex!

– will l ikely dominate on large problems soon

Extends to even more general problems.

8/11/2019 Math Prog


9

LP Application: Weighted Bipartite Matching

Assignment problem. Given a complete bipartite network Kn,n and edgeweights c ij, find a perfect matching of minimum weight.

Birkhoff-von Neumann theorem (1946, 1953). All extreme points of theabove polyhedron are {0-1}-valued.

Corollary. Can solve assignment problem using LP techniques since LP

algorithms return optimal solution that is an extreme point.

Remarks. (1) The above theorem is directly proven by applying theconcept of totally unimodular matrix .

(2) Polynomial combinatorial algor ithms also exist.

n ji x

n j x

ni x

x c

ij

niij

n jij

n jijij

ni

≤

≤

≤

∑

≤

≤

≤

,10

11

11t.s.

min

1

1

11

8/11/2019 Math Prog


10

LP Application: Multicommodity Flow

Multicommodity flow. Given a network G = (V, E) with edge capacitiesu(e)

≥

0, edge costs c(e)≥

0, set of commodities K, and supply /demand dk(v) for commodity k at node v, find a minimum cost flowthat satisfies all of the demand.

Applications.

Transportation networks. Communication networks.

VLSI design.

K k E ee x

E eeue x

K k V vvd e xe x

e xec

k

K k

k

k

ve

k

ve

k

E e

k k

K k

∈∈≥

∈≤

∈∈=−

∑

∑∑

∑∑

∈

∈∈

,0)(

)()(

,)()()(t.s.

)()(min

of outin to

Remark. The problem is NP-hard i f flows are integral!

8/11/2019 Math Prog


Integer Programming

An Integer Programming model is a linear programmingproblem where some or all of the variables are required tobe non-negative integers.

These models are in general substantially harder thansolving linear programming models.

Network models are special cases of integer programmingmodels and are very efficiently solvable.

Several applications of integer programming models.

Branch and bound technique, one of the most popular

algorithm to solve integer programming models.

8/11/2019 Math Prog


12

Integer Programming

Problem: Given rational numbers aij, b i, c j, find integers x j that solve:

Claim. INTEGER-PROGRAMMING is NP-hard.

Proof. VERTEX-COVER ≤ P INTEGER-PROGRAMMING.

n j x

n j x

mi b x a

x c

j

j

i n

j jij

n

j j j

≤

≤

≤

∑

1integral

10

1t.s.

min

1

1

8/11/2019 Math Prog


13

Weighted Vertex Cover

Weighted vertex cover. Given an undirected graph G = (V, E) withvertex weights wv ≥ 0, find a minimum weight subset of nodes S suchthat every edge is incident to at least one vertex in S.

NP-hard even if all weights are 1.

Integer programming formulation.

If x* is optimal solut ion to (ILP),

then S = {v ∈ V : x*v = 1} is a min

weight vertex cover.

3

6

10

7

A

E

H

B

D I

C

F

J

G

6

16

10

7

23

9

10

9

33

Total weight = 55.

32

V v x

Ewv x x

xw ILP

v

wv

V vvv

∈

∈

∑

∈

}1,0{

),(1t.s.

min)(

8/11/2019 Math Prog


Classifications of IP Models

Pure IP Model: Where all variables must take integer values.

Maximize z = 3x1 + 2x2

subject to x1 + x2 ≤ 6x1, x2 ≥ 0, x1 and x2 integer

Mixed IP Model: Where some variables must be integer whileothers can take real values.


subject to x1 + x2 ≤ 6x1, x2 ≥ 0, x1 integer

0-1 IP Model: Where all variables must take values 0 or 1.

Maximize z = x1 - x2

subject to x1 + 2x2 ≤ 22x1 - x2 ≤ 1, x1, x2 = 0 or 1

8/11/2019 Math Prog


Classifications of IP Models (contd.)

LP Relaxation: The LP obtained by omitting all integer or 0-1constraints on variables is called the LP relaxation of IP.

IP:

Maximize z = 21x1 + 11x2subject to

7x1 + 4x2 ≤ 13x1, x2 ≥ 0, x1 and x2 integer

LP Relaxation:


subject to

7x1 + 4x2 ≤ 13x1, x2 ≥ 0

Fact:Optimal objective value of IP ≤

Optimal objective value of LP relaxation

8/11/2019 Math Prog


Solving IP via LP relaxation

Solve the LP relaxation Round-off the solution to the nearest feasible integer

solution

Any potential difficulty with this approach?

x x xx

x

x

x

x

x1

x2

1 2 3

1

3

2

7x1 + 4x2= 13

8/11/2019 Math Prog


Capital Budgeting Problem

Stockco Co. is considering four investments

It has $14,000 available for investment

Formulate an IP model to maximize the NPV obtained fromthe investments

IP:

Maximize z = 16x1 + 22x2 + 12x3 + 8x4subject to

5x1 + 7x2 + 4x3 + 3x4 ≤ 14x1, x2,,x3, x4 0, 1

Investmentchoice

1 2 3 4

Cashoutflow

$5000 $7000 $4000 $3000

NPV $16000 $22000 $12000 $8000

8/11/2019 Math Prog


Fixed Charge Problem

Gandhi cloth company manufactures three types ofclothing: shirts, shorts, and pants

Machinery must be rented on a weekly basis to make each

type of clothing. Rental Cost:

$200 per week for shirt machinery$150 per week for shorts machinery$100 per week for pants machinery

There are 150 hours of labor available per week and 160square yards of cloth

Find a solution to maximize the weekly profitLabor hr Cloth yd Var. cost Price

Shirts 3 4 $12 $6Shorts 2 3 $8 $4Pants 6 4 $15 $8

8/11/2019 Math Prog


Fixed Charge Problem (contd.)

Decision Variables:

x1 = number of shirts produced each weekx2 = number of shorts produced each week

x3 = number of pants produced each week

y1 = 1 if shirts are produced and 0 otherwisey2 = 1 if shorts are produced and 0 otherwisey3 = 1 if pants are produced and 0 otherwise

Formulation:

Max. z = 6x1 + 4x2 + 7x3 - 200y1 - 150 y2 - 100y3

subject to3x1 + 2x2 + 6x3 ≤

1504x1 + 3x2 + 4x3 ≤ 160

x1 ≤ M y1, x2 ≤ M y2, x3 ≤ M y3

x1, x2,,x3 ≥ 0, and integer; y1, y2,,y3 ≥ 0, and integer

8/11/2019 Math Prog


Either-Or Constraints

Dorian Auto is considering manufacturing three types ofauto: compact, midsize, large.

Resources required and profits obtained from these cars

are given below.

We have 6,000 tons of steel and 60,000 hours of laboravailable.

If any car is produced, we must produce at least 1,000 unitsof that car.

Find a production plan to maximize the profit.

Compact Midsize LargeSteel Req. 1.5 tons 3 tons 5 tons

Labor Req. 30 hours 25 hours 40 hours

Profit $2000 $3000 $4000

8/11/2019 Math Prog


Either-Or Constraints (contd.)

Decision Variables:

x1, x2, x3 = number of compact, midsize and large cars producedy1, y2, y3 = 1 if compact , midsize and large cars are produced or

not

Formulation:

Maximize z = 2x1 + 3x2 + 4x3

subject tox1 ≤ My1; x2 ≤ My2; x3 ≤ My3

1000 - x1 ≤ M(1-y1)1000 - x2 ≤ M(1-y2)

1000 - x3 ≤ M(1-y3)1.5 x1 + 3x2 + 5x3 ≤ 6000

30 x1 + 25x2 + 40 x3 ≤ 60000

x1, x2, x3 ≥ 0 and integer; y1, y2, y3 = 0 or 1

8/11/2019 Math Prog


Set Covering Problems

Western Airlines has decided to have hubs in USA.

Western runs flights between the following cit ies: Atlanta,Boston, Chicago, Denver, Houston, Los Angeles, New

Orleans, New York, Pittsburgh, Salt Lake City, SanFrancisco, and Seattle.

Western needs to have a hub within 1000 miles of each ofthese cities.

Determine the minimum number of hubs

Cities within 1000 miles At lanta (AT) AT, CH, HO, NO, NY, PIBoston (BO) BO, NY, PIChicago (CH) AT, CH, NY, NO, PIDenver (DE) DE, SL

Houston (HO) AT, HO, NOLos Angeles (LA) LA, SL, SFNew Or lean s (NO) AT, CH, HO, NONew York (NY) AT, BO, CH, NY, PIPittsburgh (PI) AT, BO, CH, NY, PISal t Lake Ci ty (SL) DE, LA, SL , SF, SESan Francisco (SF) LA, SL , SF, SESeattle (SE) SL, SF, SE

8/11/2019 Math Prog


Formulation of Set Covering Problems

Decision Variables:

x i = 1 if a hub is located in city ix i = 0 if a hub is not located in city i

Minimize x AT + xBO + xCH + xDE + xHO + xLA + xNO + xNY + xPI + xSL

+ xSF + xSE

subject to AT BO CH DE HO LA NO NY PI SL SF SE Requ ir ed

AT 1 0 1 0 1 0 1 1 1 0 0 0 xAT >= 1

BO 0 1 0 0 0 0 0 1 1 0 0 0 xBO >= 1

CH 1 0 1 0 0 0 1 1 1 0 0 0 xCH >= 1

DE 0 0 0 1 0 0 0 0 0 1 0 0 xDE >= 1

HO 1 0 0 0 1 0 1 0 0 0 0 0 xHO >= 1

LA 0 0 0 0 0 1 0 0 0 1 1 0 xLA >= 1

NO 1 0 1 0 1 0 1 0 0 0 0 0 xNO

>= 1

NY 1 1 1 0 0 0 0 1 1 0 0 0 xNY >= 1

PI 1 1 1 0 0 0 0 1 1 0 0 0 xPI >= 1

SL 0 0 0 1 0 1 0 0 0 1 1 1 xSL >= 1

SF 0 0 0 0 0 1 0 0 0 1 1 1 xSF >= 1

SE 0 0 0 0 0 0 0 0 0 1 1 1 xSE >= 1

8/11/2019 Math Prog


IP: Logistics/Supply Chain Applications

Set covering and partitioningScheduling airline crews

Manpower rostering

Facility locationLocating warehouses or franchises

Multicommodity distribution

Distributing auto parts

Traveling salesman / Vehicle routingRouting pickups and deliveries

Exercise: Model TSP as an IP

8/11/2019 Math Prog


Branch and Bound

The most popular approach for solving integerprogramming problems

Enumerate the entire solution space but only implicity;

hence they are called implicit enumeration algorithms.

A general-purpose solution technique which must bespecialized for individual IP's.

Running time grows exponentially with the problemsize, but small to moderate size problems can besolved in reasonable time.

A divide and conquer algorithm, where a problem isdivided into smaller and smaller subproblems. Eachsubproblem is solved separately, and the best solution

is taken.

8/11/2019 Math Prog


26

Branch-and-Bound Search

• Represent search space as trees

• Search tree

– Each leaf node represents a feasible solution – Non-leaf node represents a subset of feasible solutions

– Parent-child relationship represents partition of search

space into sub-spaces

• Idea: if we can drastically reduce solution space

from one level to next, total number of nodes

explored will be reduced

8/11/2019 Math Prog


27

Branch-and-Bound Search

S

S1

S2

S3

S4

S1 S2 S3 S4

S24

(a)

(b)

(c)

S1

S4

S21

S22

S31

S32

*

*

* = does not contain

optimal solution

S1 S2 S3 S4

S21 S22 S23

Figure 1: Illustration of the search space of B&B.

8/11/2019 Math Prog


28

Elements of Branch-and-Bound

1. Branching rule• Rule for dividing the solution space into sub-spaces (sub-problem)

2. Node-selection (search/exploration) strategy

• Select next node to be explored next• DFS, BFS, Best-First Search

3. Bounding function• Function that computes the lower bound for the best solution

obtainable in that search space4. Upper bounding

• Process of obtaining an upper bound for each subproblem

5. Pruning strategy

• If for a subproblem, UB ≤ LB, then the subproblem need not beexplored further

8/11/2019 Math Prog


29

Bounding Function

g

S

P

f

Figure 2: The relation between the bounding function g and the objective func-

tion f on the sets S and P of feasible and potential solutions of a problem.

8/11/2019 Math Prog


30

Branch-and-Bound Algorithm (sketch)

• Start with complete feasible solution space and a naïveinitial solution

• upper bound = objective value of best solution found so far

• Among unexplored nodes do – Select a non-leaf node based on selection strategy

– Partition the feasible space into sub-spaces based onbranching rule

– For each sub-space do

• If leaf node reached and beats best solution, updatebest solution, else discard and exit

• Compute lower-bound• If lower bound >= upper bound, prune the sub-space

• Else create a sub-space node and add to list ofunexplored nodes

8/11/2019 Math Prog


31

Example: TSP

• Find optimal tour by systematically considering alltours

• Strategy 1 (exhaustive search):

– Consider all permutations of the cities (vertices)

– Evaluate each tour (given by particular vertex order)

– Keep track of shortest tour

– (n-1)! permutations, each takes O(n) time to evaluate• Don’t look at all n permutations, since we don’t care about starting

point of tour: A,B,C,(A) is same tour as C,A,B,(C)

– Unacceptable for large n• Strategy 2 (dynamic programming):

– O(n22n)

8/11/2019 Math Prog


32

Example: TSP

• Strategy 3: Branch-and-Bound

• Construct a tree with a root that represents all tours

(tours are feasible solutions)

• Each node has two children

– One child represents tours with a particular edge

– Other child represents tours without that edge

– Choose edges in lexicographic order, i.e. ab, ac, ad, etc

8/11/2019 Math Prog


33

TSP Search Tree

all tours

tours with(a,b)

tours without

(a,b)

tours with(a,b)

and (a,c)

tours with(a,b)

not (a,c)

tours with(a,c)

not (a,b)

tours without(a,b)

or (a,c)

tours with

(a,b)and (a,c)not (a,d)

tours with(a,b) and(a,d) not

(a,c)

tours with

(a,b) not(a,c) or(a,d)

tours with

(a,d) not(a,b) or(a,c)

tours without

(a,d),(a,b) or

(a,c)

not shown: pruned

(infeasible)

8/11/2019 Math Prog


34

Lower bound

• Compute a lower bound on the cost of feasiblesolutions “below” node n

– If current solution is better (lower) than lower bound for

node n, need not explore any nodes below n

• For TSP, lower bound on cost of tour

– What’s an obvious loose lower bound?

– Tighter bound: half the sum over all nodes n of the 2least cost edges incident upon n

– More precisely, let e1(n) and e2(n) be the two least cost

edges incident to n, then LB = (∑

n∈V e1(n)+e2(n)) / 2 – Proof. Left as an exercise. (Hint: consider situation at

each node, and sum up over all nodes.)

8/11/2019 Math Prog


35

Example

a

c

b

e

d

3

3

44

8

7

2

6

6

5

2 least cost edges adjacent to a are: (a,d) and (a,b) for total of 2+3=5

2 least cost edges adjacent to b are: (a,b) and (b,e) for total of 3+3=6c, d, e: total of 8, 7, 9 respectively

⇒ lower bound on cost of tour is (5+6+8+7+9)/2 = 17.5

8/11/2019 Math Prog


36

Lower Bound for Subset of Tours

• Lower bound on the cost of node in search tree

– Each node represents tours defined by a set of edges

that must be in tour and set of edges that may not be

included

• Assume we must include (a,e) and exclude (b,c)a: (a,d) and (a,e), total 9 // must include (a,e) instead of (a,b)

b: (a,b) and (b,e), total 6c: (a,c) and (c,d), total 9 // cannot use (b,c)

d: (a,d) and (c,d), total 7

e: (a,e) and (b,e), total 10 // must include (a,e) instead of (e,d)

⇒ lower bound = (9+6+9+7+10)/2 = 20.5

8/11/2019 Math Prog


37

Reconstruct Search Tree

• Each time we branch, decide on which edge toinclude/exclude based on: – Include (x,y) if it becomes impossible for x or y to have two adjacent

edges in tour otherwise

– Exclude (x,y) if it raises degree of x or y above 2

– Exclude (x,y) if it would complete a non-tour cycle with selectededges

• After branch, compute lower bounds for bothchildren

• Prune child whose lower bound is as high or higherthan best tour found so far

• If neither child can be pruned, explore child withsmaller lower bound first

8/11/2019 Math Prog


38

New Search Tree

B 17.5ab

C 18.5ab

D 20.5ac ad ae

E 18ac

G 23ad ae

F 18ad ae

H

bc bd bece de cd

tour=abcedacost=23

I

bc cd cebd de be

tour=abecdacost=21

pruned after

discovery of I

pruned after

discovery

of I

J 18.5ac

K 21ac ad ae

pruned

M 23.5ad ae

L 18.6ad ae

pruned

N

bc cd cebd de be

tour=acbedacost=19

P

bc bd be

ce de cd

tour=

acebdacost=23

A 17.5no constraints

LB

8/11/2019 Math Prog


39

Analysis

• Search space: still (n-1)! in worst-case

• But much better on average

– Depends on the 3 elements of B&B

8/11/2019 Math Prog


40

A simple way to get the initial solution:

Set xi = 1 for i = 1,2,… until the constraint is violated.

e.g. x1=1, x2=1, x3=0, x4=0.

max z = 16x1+22x2+12x3+8x4

subject to the following constraints:

5x1+7x2+4x3+3x4 <= 14x j= 0 or 1, for all j

Lower bound = 16x1 + 22x2 = 38

Any sub-space that has an upper bound <=38 can be

pruned. What is a good upper bound?

0-1 Knapsack Problem

Integer programming (IP)

formulation

Upper Bounding Function:

8/11/2019 Math Prog


41

Upper Bounding Function:Fractional Knapsack Problem

0

,5.0

,1

4

3

21

=

=

==

x

x

x x

:solutionLP j p j w j ratio ranking

1 16 5 3.2 1

2 22 7 3.1 2

3 12 4 3.0 3

4 8 3 2.7 4

valueingnonincreastoaccordingitemsorder :itsolveTo j

j

w

p

Greedy!i.e.possible,aslargeasorder thisins'themakeand j x

Linear programming

(LP) Relaxation

max z = 16x1+22x2+12x3+8x4

subject to the following constraints:

5x1+7x2+4x3+3x4 <= 14

0<= x j<=1, for all j

B h d B d T

8/11/2019 Math Prog


42

Branch and Bound Tree

Subproblem 1z = 44, LB=38x1 = x2 = 1

x3 =.51

x3 = 1

2

Subproblem 3z = 43.4, LB=38

x1 =x3= 1,x2 = .7, x4=0

Subproblem 6z = 42, LB=38x1 =0, x2=x3=1

x4 = 15

Subproblem 7LB = 42

Infeasible6

x1 = 0 x1 = 1

x3 = 0

7

Subproblem 2z = 43.3, LB=38

x1 = x2=1x3 = 0, x4 =.67

x4 = 1

8 9

Subproblem 8z = 38, LB=42

x1 = x2=1x3 = x4 = 0

Subproblem 9z= 42.85, LB=42

x1 = x4 =1x3 = 0, x2 = .85

x4 = 0

pruned

x2 = 0

Subproblem 5z = 43.6, LB=38x1 =.6, x2=x3=1

x4 = 0

Subproblem 4z = 36, LB=38

x1 = x3=1x2 = 0, x4 =1

x2 = 1

3 4

pruned

LB=42

pruned

8/11/2019 Math Prog


43

Practical Developments

• Good formulations, heuristics and theoryGoal: to get LP solution as close as possible to IPsolution

Disaggregation, adding constraints (cuts)• Preprocessing Automatic methods for reformulationSome interesting graph theory is involved

• Cut generation (branch-and-cut) Add cuts during the branch-and-bound

• Column generation

Improve formulation by introducing an exponentialnumber of variables.

E i G li d A i t P bl

8/11/2019 Math Prog


44

Exercise: Generalized Assignment Problem

In this problem, we assign m jobs to n machines

– Machine: The total available time of machine j is T j

– Jobs: If job i is performed on machine j, it costs aij, and

requires tij time units. A job must be assigned to one

machine in its entirety

The problem is to find a minimum cost assignment of all

jobs.

Design a B&B approach to find the optimal solution.

Date post:	02-Jun-2018
Category:	Documents
Upload:	abhijeet-jha
View:	214 times
Download:	0 times

Math Prog

Documents