Linear Programming

Algorithms and Data Structures 2014

Simplex

Last lecture!

What is Linear Programming?

Say you own a 200 hectares farm. On this farm you can grow wheat, barley, corn or some combination of the 3. You have a limited supply of fertilizer and pesticide, both of which are needed (in different quantities) for each crop grown. Let’s say wheat sells at $7 a bushel, barley is $3, and corn is $3.50.

So, how many of each crop should

you grow to maximize your profit?

What is Linear Programming?

A mathematical tool for

maximizing or minimizing a

quantity (usually profit or cost of

production), subject to certain

constraints.

Of all computations and decisions

made by management in

business, 50-90% of those

involve linear programming.

Mixture Problems

Problem where a limited number of resources are used to

produce a combination of products in a fashion that

maximizes profit from the sale of these products

Production of … pretty much anything

■ Wheat, barley and corn

…what will our solution look like?

Mixture Problems consist of…

■ Resources

■ Products

■ Recipes

■ Profit

■ Objective

● A Linear Programming model seeks to maximize or

minimize a linear function, subject to a set of linear

constraints.

● The linear model consists of the following components:■ Decision variables

○ representing levels of activity of an operation

■ Objective function

○ a relationship reflecting the objective of an operation

○ most frequent objective of business firms is to maximize profit

○ most frequent objective of individual operational units (such as a production or packaging department) is to minimize cost

■ Constraint

○ a linear relationship representing a restriction on decision making

Linear Programming

LP Model: Example

Labor Clay Revenue

PRODUCT (hr/unit) (lb/unit) ($/unit)

Bowl 1 4 40

Mug 2 3 50

There are 40 hours of labor and 120 pounds (55

kilograms) of clay available each day

Decision variables

x1 = number of bowls to produce

x2 = number of mugs to produce

RESOURCE REQUIREMENTS

LP Formulation: Example

Maximize Z = 40 x1 + 50 x2

Subject to

x1 + 2x2 40 hr (labor constraint)

4x1 + 3x2 120 lb (clay constraint)

x1 , x2 0

Solution is x1 = 24 bowls x2 = 8 mugs

Revenue = 1,360

Graphical Solution Method

1. Plot model constraint on a set of coordinates in a plane

2. Identify the feasible solution space on the graph where

all constraints are satisfied simultaneously

3. Plot objective function to find the point on boundary of

this space that maximizes (or minimizes) value of

objective function

Graphical Solution: Example

4 x1 + 3 x2 120 lb

x1 + 2 x2 40 hr

Feasible solutions

50 –

40 –

30 –

20 –

10 –

0 – |

10

|

60

|

50

|

20

|

30

|

40 x1

x2

Z = 40 x1 + 50 x2

Z = 800

Z = 1600

Computing Optimal Values

x1 + 2x2 = 40

4x1 + 3x2 = 120

4x1 + 8x2 = 160

-4x1 - 3x2 = -120

5x2 = 40

x2 = 8

x1 + 2(8) = 40

x1 = 24

4 x1 + 3 x2 120 lb

x1 + 2 x2 40 hr

40 –

30 –

20 –

10 –

0 – |

10

|

20

|

30

|

40

x1

x2

Z = $50(24) + $50(8) = $1,360

24

8

Extreme Corner Points

x1 = 224 bowls

x2 = 8 mugs

Z = $1,360 x1 = 30 bowls

x2 = 0 mugs

Z = $1,200

x1 = 0 bowls

x2 = 20 mugs

Z = $1,000

A

B

C|

20

|

30

|

40

|

10 x1

x2

40 –

30 –

20 –

10 –

0 –

4x1 + 3x2 120 lb

x1 + 2x2 40 hr

40 –

30 –

20 –

10 –

0 –

B

|

10

|

20

|

30

|

40 x1

x2

C

A

Z = 70x1 + 20x2 Optimal point:

x1 = 30 bowls

x2 = 0 mugs

Z = $2,100

Objective Function

Minimization Problem

CHEMICAL CONTRIBUTION

Brand Nitrogen (lb/bag) Phosphate (lb/bag)

Gro-plus 2 4

Crop-fast 4 3

Minimize Z = 6x1 + 3x2

subject to

2x1 + 4x2 16 lb of nitrogen

4x1 + 3x2 24 lb of phosphate

x1, x2 0

14 –

12 –

10 –

8 –

6 –

4 –

2 –

0 – |

2

|

4

|

6

|

8

|

10

|

12

|

14

x2

A

B

C

Graphical Solution

x1 = 0 bags of Gro-plus

x2 = 8 bags of Crop-fast

Z = $24

Z = 6x1 + 3x2

Solving linear programs

1. Start at some vertex (often (0,0))

2. Repeat: look for an adjacent vertex of better objective

value.

• Hill-climbing

More products, more variables

LP Formats: standard form

Converting into standard form

● LP not in standard form:

1. Objective function: minimize i.s.o. maximize

2. Variables without nonnegativity constraints

3. Equality constraints i.s.o. inequality constraints (= i.s.o. ≤)

4. Constraints with opposite sign (≥ i.s.o. ≤)

● Conversion:

1. Negate coefficients in the objective function

2. If 𝑥𝑗 has no nonnegativity constraint:○ replace 𝑥𝑗by 𝑥′𝑗 − 𝑥"𝑗○ add 𝑥′𝑗 ≥ 0, 𝑥"𝑗 ≥ 0

3. Suppose LP contains 𝑗=1𝑛 𝑎𝑖𝑗𝑥𝑗 = 𝑏𝑗

○ Replace this inequality by 𝑗=1𝑛 𝑎𝑖𝑗𝑥𝑗 ≤ 𝑏𝑗 and 𝑗=1

𝑛 𝑎𝑖𝑗 ≥ 𝑏𝑗

4. If LP contains 𝑗=1𝑛 𝑎𝑖𝑗 ≥ 𝑏𝑗

○ Replace this inequality by 𝑗=1𝑛 −𝑎𝑖𝑗𝑥𝑗 ≤ −𝑏𝑗

Conversion example

● Step 1

Conversion example (cont)

● Step 2

Conversion example (cont)

● Step 3

Conversion example (cont)

● Step 4

LP Formats: slack form

● Only nonnegativity constraints are inequalities

● Replace

● by

New variable: slack variable

● Convention: i.s.o. s we use 𝑥𝑛+𝑖 to denote the slack variable associated with equality i.

Slack form example

● Slack form

Denotes the value of the objective function

Slack form example

Basic variables Nonbasic variables

Simplex Method

● A mathematical procedure for solving linear programming problems according to a set of steps

● Idea: convert iteratively a slack from into a new slack form until a situation is reached that can be solved directly.■ Each iteration corresponds to a basic solution obtained from the

slack form by setting each nonbasic variable (rhs variable) to 0.

Simplex Example

● Basic solution:

● Next iteration: greater z obtained by increasing a nonbasicvariable as much as possible.

● Always choose a variable with a positive coefficient■ Let’s choose 𝑥1

● How much can we increase 𝑥1?

■ Third constraint is the most restrictive: max value of 𝑥1 is 9

■ Then 𝑥6 will be 0: switch roles of 𝑥1and 𝑥6

Simplex Example (cont)

● Rewriting third equation:

● Replace all occurrences of 𝑥1:

Simplex Example (cont)

● Repeat these steps

Simplex Example (cont)

● And again

Simplex Example (cont)

● All coefficients negative: basic solution is optimal

Single-Source Shortest Paths

● Weighted graph G = (V,E) , weight function 𝑤 ∶ 𝐸 →ℝ, 𝑠, 𝑡 ∈ 𝑉, compute d[t] the shortest path from s to t.

● Bellman-Ford algorithm: for each (u,v) ∈ 𝐸:𝑑 𝑣 ≤ 𝑑 𝑢 + 𝑤 𝑢, 𝑣

● SSSP as an LP

Maximum flow

● Given a network of pipelines along which oil can be sent.

● Goal: ship as much oil as possible from s (source) to t

(sink)

■ Each pipeline has a capacity (maximum flow it can handle)

■ Oil cannot be stored en route.

Maximum flow

● Weighted graph G = (V,E) ; two special nodes 𝑠, 𝑡 ∈ 𝑉 (source and

sink), and capacities 𝑐𝑒 > 0, 𝑒 ∈ 𝐸 on the edges

● A flow consists of variables 𝑓𝑒 , 𝑒 ∈ 𝐸 satisfying

1. 0 ≤ 𝑓𝑒 ≤ 𝑐𝑒 , ∀ 𝑒 ∈ 𝐸

2. ∀ 𝑢 ∈ 𝑉, 𝑢 ≠ 𝑠, 𝑡: (𝑤,𝑢)∈𝐸 𝑓𝑤𝑢 = (𝑢,𝑧)∈𝐸 𝑓𝑢𝑧

● The size of a flow 𝑓: size(𝑓) = (𝑠,𝑢)∈𝐸 𝑓𝑠𝑢

● The maximum flow problem as an LP:

Maximum flow: Simplex

● Behaviour of Simplex:

■ Start with zero flow.

■ Repeat: choose an appropriate path from s to t, and increase flow along the

edges of this path as much as possible.

● A.K.A. Ford-Fulkerson algorithm.

Maximum flow: Simplex

● Solution: allow paths that cancel existing flows:

● Complication: what if we had chosen:

Maximum flow: Residual graph

● Residual graph (roughly): initial graph – current flow

Simplex: loose ends

● Starting vertex: what if the basic solution is not feasible?

● Answer: ITA p. 886-890

■ Also detects whether a LP has a solution

● Complexity: worst-case exponential (in number of variables), in practice

quite fast.

Next week

Christmas break