Linear Programming - cs.iastate.educs511/handout10/LP Basics.pdf · Linear Programming Computer...

Linear Programming

Computer Science 511

Iowa State University

September 24, 2010

Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 1 / 27

Example: Profit Maximization

Joe sells two kinds of beer: Regular Duff and Duff Special Brew.

Profit per pint for Regular Duff is $1.

Profit per pint for Duff Special is $2.

Demand for Regular Duff is at most 200 pints per day.

Demand for Duff Special is at most 400 pints per day.

Duff will sell Joe a combined total of at most 500 pints of RegularDuff and Duff Special per day.

How many pints of each beer should Joe buy per day so as tomaximize his profit?


Example: LP Formulation

Let x1 and x2 denote the number of pints of Regular Duff and Duff Specialthat Joe will buy per day.

maximize x1 + 2x2

subject tox1 ≤ 200

x2 ≤ 400x1 + x2 ≤ 500x1 , x2 ≥ 0


Linear Programming

Maximize (or minimize) a linear function

n∑j=1

cjxj

subject to linear equalities/inequalities of the form

n∑j=1

ajxj = b,

n∑j=1

ajxj ≤ b, orn∑

j=1

ajxj ≥ b.


Terminology

Feasible solution: Any x = (x1, . . . , xn) that satisfies all the constraints.

Feasible region: Set of all feasible solutions.

The intersection of a collection of halfspaces andhyperplanes.⇒ A convex polyhedral region.

Objective function: The function of x1, . . . , xn that we want to maximize.

Objective value: The value of the objective function for some specificx1, . . . , xn.


Example

maximize x1 + 2x2

subject tox1 ≤ 200

x2 ≤ 400x1 + x2 ≤ 500x1 , x2 ≥ 0

400

300

100

0100 200 300 400

400

500 !!

!"

Obj. val. = 900

Obj. val. = 400

200

Sunday, September 12, 2010


Infeasibility and unboundedness

Definition

A linear program is infeasible if it has no feasible solution.

Example

Maximize x subject to x ≤ −1 and x ≥ 0.

Definition

A linear program is unbounded if its objective value can be madearbitrarily large (small).

Example

Maximize x subject to x ≥ 0.


The Simplex Algorithm (Dantzig 1947)

Moves from vertex to neighboring vertex of feasible region.

Objective value improves at each step.

Stop when no neighbor gives a higher value.

By convexity, this vertex must be optimum.


Standard Form

Fact

Every linear program can be put into the following standard form.

maximizen∑

j=1

cjxj

subject ton∑

j=1

aijxj ≤ bi i = 1, . . . ,m

xj ≥ 0 j = 1, . . . n


Converting to Standard Form

1 To turn a minimization problem into maximization problem, multiplythe coefficients of the objective function by −1.

2 To reverse the direction of an inequality, multiply both sides by −1.

3 To change an equality constraint into inequalities rewrite∑nj=1 ajxj = b as

n∑j=1

ajxj ≤ b andn∑

j=1

ajxj ≥ b.

4 If x is a variable that is unrestricted in sign, introduce two nonnegativevariables, x+, x− ≥ 0 and replace every occurrence of x by x+ − x−.


Slack Form

For the ith constraint, define a nonnegative slack variable xn+i :

xn+i = 0 ⇐⇒ constraint i is tight.

At all times, express constraints as a set of equalities where the nvariables in RHS are 0.

I RHS variables correspond to tight inequalities.

Tight inequalities define a vertex of the feasible region.I A point in Rn is defined by n hyperplanes.


Original

maximizen∑

j=1

cjxj

subject ton∑

j=1

aijxj ≤ bi i = 1, . . . ,m

xj ≥ 0 j = 1, . . . n

Slack Form

z =n∑

j=1

cjxj

xn+i = bi −n∑

j=1

aijxj i = 1, . . . ,m

(x1, . . . , xn+m ≥ 0)


Slack Form: Example

Original

maximize 2x1 + 5x2

subject to2x1 − x2 ≤ 4x1 + 2x2 ≤ 9−x1 + x2 ≤ 3

x1 ≥ 0x2 ≥ 0

Slack Form

z = 0 + 2x1 + 5x2

x3 = 4 − 2x1 + x2

x4 = 9 − x1 − 2x2

x5 = 3 + x1 − x2

(x1, x2, x3, x4, x5 ≥ 0)


Dictionaries

At all times, simplex maintains a dictionary:

z = v ′ +∑j∈N

c ′jxj

xi = b′i −∑j∈N

a′ijxj for i ∈ B

Variables in RHS are called non-basic.I For each j ∈ N, xj = 0.

Variables in LHS are called basic.I For each i ∈ B, xi = b′

i .

The values of the variables implied by a dictionary give a basicfeasible solution (b.f.s.).

I z = v ′ is the objective value of the b.f.s.


Pivoting

z = v ′ +∑j∈N

c ′jxj

xi = b′i −∑j∈N

a′ijxj for i ∈ B

Greedy Approach

Find a j ∈ N such that c ′j > 0.I z can be increased by increasing xj .I If no such j exists, current b.f.s. is

optimal.

Increase xj as much as possible,keeping xi ≥ 0 for all i ∈ B.

I xj enters the basis.

For some i ∈ B, xi drops to 0.I xi leaves the basis.


Dictionary 1

z = 0 + 2x1 + 5x2

x3 = 4 − 2x1 + x2

x4 = 9 − x1 − 2x2

x5 = 3 + x1 − x2

Basic feasible solution:

(x1, x2, x3, x4, x5) = (0, 0, 4, 9, 3)

Objective value: 0

Entering variable: x2

Leaving variable x5

Dictionary 2

z = 15 + 7x1 − 5x5

x3 = 7 − x1 − x5

x4 = 3 − 3x1 − 2x5

x2 = 3 + x1 − x5


(x1, x2, x3, x4, x5) = (0, 3, 7, 3, 0)

Objective value: 15


Leaving variable x4


Dictionary 2

z = 15 + 7x1 − 5x5

x3 = 7 − x1 − x5

x4 = 3 − 3x1 − 2x5

x2 = 3 + x1 − x5


(x1, x2, x3, x4, x5) = (0, 3, 7, 3, 0)

Objective value: 15


Leaving variable x4

Dictionary 3

z = 22 − 73x4 − 1

3x5

x3 = 6 + 13x4 − 5

3x5

x1 = 1 − 13x4 + 2

3x5

x2 = 4 − 13x4 − 1

3x5


(x1, x2, x3, x4, x5) = (1, 4, 6, 0, 0)

Objective value: 22

Optimal


Unboundedness

Fact

Simplex can be modified to discover unboundedness.

Method

If a linear program is unbounded, at some point, Simplex will find anon-basic xi such that

xi has a positive coefficient in the objective function and

none of the constraints binds xi .

At this point, stop and return unbounded.


Cycling

Successive pivots may encounter solutions with same objective value.

Reason: the same vertex may be defined by different sets ofconstraints.

Cycling — i.e., getting stuck at a vertex — is a possibility.

Bland’s rule

When choosing entering or leaving variable, always pick the one withsmallest index.

Fact

Using Bland’s rule, Simplex will never cycle.


Finding an Initial Basic Feasible Solution

The origin, (x1, . . . , xn) = (0, . . . , 0), is not always feasible.

Happens when bi < 0 for some i .

Example

maximize 2x1 + 5x2

subject to2x1 − x2 ≤ 4x1 + 2x2 ≤ 9−x1 + x2 ≤ 3−x1 − x2 ≤ −1

x1 , x2 ≥ 0


Finding an Initial Basic Feasible Solution

Set up and solve an auxiliary LP.

Auxiliary LP always has a solution.I An initial b.f.s. for auxiliary problem can be obtained easily.

Optimal solution to auxiliary LP gives an initial b.f.s. for original LP.

Optimal solution to auxiliary problem also tells us if original problemis feasible.


The Auxiliary Problem: Example

Original Problem

max 2x1 + 5x2

subj. to2x1 − x2 ≤ 4x1 + 2x2 ≤ 9−x1 + x2 ≤ 3−x1 − x2 ≤ −1

x1 , x2 ≥ 0

Auxiliary Problem

max −x0

subj. to−x0 + 2x1 − x2 ≤ 4−x0 + x1 + 2x2 ≤ 9−x0 − x1 + x2 ≤ 3−x0 − x1 − x2 ≤ −1

x0 , x1 , x2 ≥ 0

The auxiliary problem has an obvious solution with x1, x2 = 0.I Not a b.f.s., though.

The original problem also has a solution, but it is less obvious.


A B.F.S. for the Auxiliary Problem: One Pivot is Enough

Auxiliary Problem in Slack Form

z = 0 − x0

x3 = 4 + x0 − 2x1 + x2

x4 = 9 + x0 − x1 − 2x2

x5 = 3 + x0 + x1 − x2

x6 = −1 + x0 + x1 + x2

Not a legal dictionary:

(x0 = x1 = x2 = 0)⇒ (x6 = −1)

Make x0 enter the basisand x6 leave.

Not a “normal” pivot,since objective valuedecreases.

Legal Starting Dictionary

z = −1 + x1 + x2 − x6

x3 = 5 − 3x1 + x6

x4 = 10 − 2x1 − 3x2 + x6

x5 = 4 − 2x2 + x6

x0 = 1 − x1 − x2 + x6


The Auxiliary Problem: General Case

Original

maximizen∑

j=1

cjxj

subject ton∑

j=1

aijxj ≤ bi i = 1, . . . ,m

xj ≥ 0 j = 1, . . . n

Auxiliary

maximize −x0

subject ton∑

j=1

aijxj − x0 ≤ bi i = 1, . . . ,m

xj ≥ 0 j = 0, . . . n


The Auxiliary Problem

Fact

The original LP is feasible if and only if optimum objective value for theauxiliary LP is 0.

Proof.

Suppose (x1, . . . , xn) is a feasible solution to the original problem.I ⇒ x = (0, x1, . . . , xn) is a feasible solution to the auxiliary problem.I The objective value of x is 0.I Since we require x0 ≥ 0, x must be an optimal solution.

Suppose the optimum solution to the auxiliary problem has value 0.I ⇒ x0 = 0.I ⇒ (x1, . . . , xn) is a feasible solution for the original problem.


The Fundamental Theorem of Linear Programming

Theorem

For any linear program in standard form:

1 if there is no optimal solution, then the problem is either infeasible orunbounded,

2 if a feasible solution exists, then a basic feasible solution exists,

3 if an optimum solution exists, then there exists a basic feasiblesolution that is also an optimum solution.


Running Time of the Simplex Algorithm

Time per pivot is polynomial.

Number of basic feasible solutions is ≤(m+n

n

).

In the worst case, Simplex may have to run through most of them.I Run time of Simplex is exponential in worst case.

However, this rarely happens.I In practice Simplex is fast.

There are polynomial time algorithms for LP (Khachiyan 1979,Karmarkar 1984)


Date post:	30-Aug-2018
Category:	Documents
Upload:	hoangnhi
View:	224 times
Download:	2 times

Linear Programming - cs.iastate.educs511/handout10/LP Basics.pdf · Linear Programming Computer...

Documents