Lesson 31 (KH, Section 11.2)The Simplex Method

Math 20

December 5, 2007

Announcements

I Pset 11 due December 10. Pset 12 due December 17.

I ML OH: Today 1–3 (SC 323)

I SS OH: Tonight 8:30–9:30 in Quincy dining hall

I HW coming

I Midterm II review slides online

I Midterm II: tomorrow 7–8:30pm in Hall A

Outline

Setup

Illustrative ProblemSlack Variables

The Simplex Method, By ExampleThe Initial Basic Feasible SolutionCreating a New Tableau

Recap of Steps

Example

Setup

A standard linear programming problem is to maximize thequantity

c1x1 + c2x2 + . . . cnxn = c′x

subject to constraints

a11x1+ a12x2+. . .+ a1nxn ≤ b1

a21x1+ a22x2+. . .+ a2nxn ≤ b2

. . .

am1x1+am2x2+. . .+amnxn ≤ bm

orAx ≤ b.

We usually include the nonnegativity constraint x ≥ 0. Today wewill also assume b ≥ 0.

Any vector x which satifies all the inequalities is called a feasiblesolution to the given problem, and a feasible solution maximizingthe objective function is called an optimal solution.

Outline

Setup

Illustrative ProblemSlack Variables

The Simplex Method, By ExampleThe Initial Basic Feasible SolutionCreating a New Tableau

Recap of Steps

Example

Illustrative Problem

We will use the baker of before. He is trying to maximize

z = 8x + 10y

subject to the constraints

2x+ y ≤50

x+ 2y ≤70

x ≥ 0

y ≥ 0.

Slack VariablesWe can turn the inequalities into equalities by inserting newvariables, which are called slack variables. Thus the first equationof constraint becomes

2x + y ≤ 50 =⇒ 2x + y + u = 50,

and the second

x + 2y ≤ 70 =⇒ x + 2y + v = 70.

But u and v are nonnegative. So the new problem is to maximize8x + 10y subject to constraints

2x+ y+u =50

x+2y +v =70

x ≥ 0 y ≥ 0 u ≥ 0 v ≥ 0

In general, we insert slack variables u1, u2, . . . , um and theequations of constraint become

Ax + u = b,

along with x ≥ 0, u ≥ 0.

DefinitionThe vector x in Rn+m is called a basic solution if its obtained bysetting n of the variables in this equation equal to zero and solvingfor the remaining n variables. The m variables are we solve for arecalled the basic variables, and the n variables set equal to zero arecalled the nonbasic variables. The vector x is called a basicfeasible solution if it is a basic solution that also satisfies theinequalities x ≥ 0.

Why are basic feasible solutions necessary?

TheoremIf a LP problem has an optimal solution, then it has a basicoptimal solution.

This is just a restatement of the corner principle. So we only needto find the basic feasible solutions!

Back to the Baker

2x+

y=

50

x + 2y = 70

x10 20 30 40 50 60 70

y

10

20

30

40

50

(0, 0) (25, 0) (70, 0)

(0, 35)

(0, 50)

(10, 30)

Back to the Baker

u=

0

v = 0

x10 20 30 40 50 60 70

y

10

20

30

40

50

(0, 0) (25, 0) not feasible

(0, 35)

not feasible

(10, 30)

How many basic feasible solutions are there? Out of the m + nvariables, we choose n to set equal to zero, and solve for the rest.This can be done (

n + m

m

)=

(n + m)!

m! n!

ways. That’s a lot!The simplex method is a way to arrive at an optimal solution bytraversing the vertices of the feasible set, in each step increasingthe objective function by as much as possible.

Outline

Setup

Illustrative ProblemSlack Variables

The Simplex Method, By ExampleThe Initial Basic Feasible SolutionCreating a New Tableau

Recap of Steps

Example

We’ll work with the illustrative problem. We can start with thebasic feasible solution x = 0, y = 0. Thus u = 50 and v = 70.This is our initial basic solution.We’ll start writing everything in a table (or tableau), so let’s alsowrite the objective function with a right-hand side of zero. Thus

−8x − 10y + z = 0.

We put this all together, forming what is called the initial tableau:

x y u v z valueu 2 1 1 0 0 50v 1 2 0 1 0 70z −8 −10 0 0 1 0

x y u v z valueu 2 1 1 0 0 50v 1 2 0 1 0 70z −8 −10 0 0 1 0

Is the solution u = 50, v = 70 (i.e., x = 0, y = 0 optimal?)

No,increasing x or y would increase z .

Optimality Criterion

If the objective row of a tableau has no negative entries in thecolumns labeled with variables, then the indicated solution isoptimal and we can stop our computation.

x y u v z valueu 2 1 1 0 0 50v 1 2 0 1 0 70z −8 −10 0 0 1 0

Is the solution u = 50, v = 70 (i.e., x = 0, y = 0 optimal?) No,increasing x or y would increase z .

Optimality Criterion

If the objective row of a tableau has no negative entries in thecolumns labeled with variables, then the indicated solution isoptimal and we can stop our computation.

x y u v z valueu 2 1 1 0 0 50v 1 2 0 1 0 70z −8 −10 0 0 1 0

Is the solution u = 50, v = 70 (i.e., x = 0, y = 0 optimal?) No,increasing x or y would increase z .

Optimality Criterion

If the objective row of a tableau has no negative entries in thecolumns labeled with variables, then the indicated solution isoptimal and we can stop our computation.

x y u v z valueu 2 1 1 0 0 50v 1 2 0 1 0 70z −8 −10 0 0 1 0

I Move from one basic solution to another

I One of the zero (nonbasic) variables becomes nonzero andone of nonzero (basic) variables becomes zero

I Do this as efficiently as possible

Which of x or y would you most like to increase? An increase of 1in y gives an increase of 10 in z . Let’s make y > 0. y enters theset of basic variables, so it’s called the entering variable for thisstep.

x y u v z valueu 2 1 1 0 0 50v 1 2 0 1 0 70z −8 −10 0 0 1 0

I Move from one basic solution to another

I One of the zero (nonbasic) variables becomes nonzero andone of nonzero (basic) variables becomes zero

I Do this as efficiently as possible

Which of x or y would you most like to increase?

An increase of 1in y gives an increase of 10 in z . Let’s make y > 0. y enters theset of basic variables, so it’s called the entering variable for thisstep.

x y u v z valueu 2 1 1 0 0 50v 1 2 0 1 0 70z −8 −10 0 0 1 0

I Move from one basic solution to another

I One of the zero (nonbasic) variables becomes nonzero andone of nonzero (basic) variables becomes zero

I Do this as efficiently as possible

Which of x or y would you most like to increase? An increase of 1in y gives an increase of 10 in z . Let’s make y > 0. y enters theset of basic variables, so it’s called the entering variable for thisstep.

How much can we increase y? Well, since x is still zero, theequations of constraint can be written

u = 50− y

v = 70− 2y

We still need u ≥ 0 and v ≥ 0, so the most we can increase y is to35. This is the smallest of the ratios 50

1 = 50 and 702 = 35. So

we’re going to increase y to 35. This will make v = 0. We call vthe departing variable.The new basic solution therefore has y = 35, v = 0, u = 15, andx = 0. The new value of the objective function is z = 10y = 350.

How much can we increase y? Well, since x is still zero, theequations of constraint can be written

u = 50− y

v = 70− 2y

We still need u ≥ 0 and v ≥ 0, so the most we can increase y is to35. This is the smallest of the ratios 50

1 = 50 and 702 = 35. So

we’re going to increase y to 35. This will make v = 0. We call vthe departing variable.

The new basic solution therefore has y = 35, v = 0, u = 15, andx = 0. The new value of the objective function is z = 10y = 350.

How much can we increase y? Well, since x is still zero, theequations of constraint can be written

u = 50− y

v = 70− 2y

We still need u ≥ 0 and v ≥ 0, so the most we can increase y is to35. This is the smallest of the ratios 50

1 = 50 and 702 = 35. So

we’re going to increase y to 35. This will make v = 0. We call vthe departing variable.The new basic solution therefore has y = 35, v = 0, u = 15, andx = 0. The new value of the objective function is z = 10y = 350.

Creating a New TableauWe are exchanging the basic variable v for y . This means the ob-jective row has to be replaced with one that has a zero in the ycolumns. We can do this by adding multiples of row 2. Since y isan entering variable, we might as well normalize row 2 to have aone.So we scale the second row to have a one in the y column.

x y u v z valueu 2 1 1 0 0 50v 1 2 0 1 0 70z −8 −10 0 0 1 0

Creating a New TableauNow we zero out the rest of this column by adding 10 times row 2to row 3, and subtracting row 2 from row 1.

x y u v z value2 1 1 0 0 50

1/2 1 0 1/2 0 35z −8 −10 0 0 1 0

Creating a New TableauThe new tableau. By looking at the columns, we see y and u arethe basic variables. The value of the objective function has alsochanged.

x y u v z valueu 3/2 0 1 −1/2 0 15y 1/2 1 0 1/2 0 35z −3 0 0 5 1 350

Rinse, Lather, RepeatThe x column in the objective row has a negative entry, so increasingx will increase z .

x y u v z valueu 3/2 0 1 −1/2 0 15y 1/2 1 0 1/2 0 35z −3 0 0 5 1 350

Rinse, Lather, RepeatThe x column in the objective row has a negative entry, so increasingx will increase z . How much can we increase it?

x y u v z valueu 3/2 0 1 −1/2 0 15y 1/2 1 0 1/2 0 35z −3 0 0 5 1 350

Rinse, Lather, RepeatThe x column in the objective row has a negative entry, so increasingx will increase z . How much can we increase it? The minimum ofthe two ratios 15

3/2 = 10 and 351/2 = 70.

x y u v z valueu 3/2 0 1 −1/2 0 15y 1/2 1 0 1/2 0 35z −3 0 0 5 1 350

Rinse, Lather, RepeatThe x column in the objective row has a negative entry, so increasingx will increase z . How much can we increase it? The minimumof the two ratios 15

3/2 = 10 and 351/2 = 70. So x in the entering

variable and u is the departing variable.

↓ x y u v z value← u 3/2 0 1 −1/2 0 15

y 1/2 1 0 1/2 0 35z −3 0 0 5 1 350

Rinse, Lather, RepeatThe x column in the objective row has a negative entry, so increasingx will increase z . How much can we increase it? The minimumof the two ratios 15

3/2 = 10 and 351/2 = 70. So x in the entering

variable and u is the departing variable.We scale row one by 2/3 to make it one in the basic column.

x y u v z valueu 3/2 0 1 −1/2 0 15y 1/2 1 0 1/2 0 35z −3 0 0 5 1 350

Rinse, Lather, RepeatThe x column in the objective row has a negative entry, so increasingx will increase z . How much can we increase it? The minimumof the two ratios 15

3/2 = 10 and 351/2 = 70. So x in the entering

variable and u is the departing variable.We scale row one by 2/3 to make it one in the basic column.And we zero out the rest of the column by subtracting half of row1 from row 2, and adding 3 times row 1 to row 3.

x y u v z valueu 1 0 2/3 −1/3 0 10y 1/2 1 0 1/2 0 35− 3 0 0 5 1 350

Rinse, Lather, RepeatThe x column in the objective row has a negative entry, so increasingx will increase z . How much can we increase it? The minimumof the two ratios 15

3/2 = 10 and 351/2 = 70. So x in the entering

variable and u is the departing variable.We scale row one by 2/3 to make it one in the basic column.And we zero out the rest of the column by subtracting half of row1 from row 2, and adding 3 times row 1 to row 3.

x y u v z valueu 1 0 2/3 −1/3 0 10y 1/2 1 0 1/2 0 35− 3 0 0 5 1 350

Rinse, Lather, RepeatThe x column in the objective row has a negative entry, so increasingx will increase z . How much can we increase it? The minimumof the two ratios 15

3/2 = 10 and 351/2 = 70. So x in the entering

variable and u is the departing variable.We scale row one by 2/3 to make it one in the basic column.And we zero out the rest of the column by subtracting half of row1 from row 2, and adding 3 times row 1 to row 3.

x y u v z valuex 1 0 2/3 −1/3 0 10y 0 1 −1/3 2/3 0 30

0 0 2 4 1 380

x y u v z valuex 1 0 2/3 −1/3 0 10y 0 1 −1/3 2/3 0 30

0 0 2 4 1 380

Now any increase in the decision variables or slack variables wouldresult in a decrease of z . We are done!

Outline

Setup

Illustrative ProblemSlack Variables

The Simplex Method, By ExampleThe Initial Basic Feasible SolutionCreating a New Tableau

Recap of Steps

Example

Recap of Steps

1. Set up the initial tableau.

2. Apply the optimality test. If the objective row has no negativeentries in the columns labeled with variables, then theindicated solution is optimal; we can stop.

3. Choose a pivotal column by determining the column with themost negative entry in the objective row. If there are severalcandidates for a pivotal column, choose any one.

4. Choose a pivotal row. Form the ratios of the entries above theobjective row in the rightmost column by the correspondingentries of the pivotal column for those entries in the pivotalcolumn which are positive. The pivotal row is the row forwhich the smallest of these ratios occurs. If there is a tie,choose any one of the qualifying rows. If none of the entriesin the pivotal column above the objective row is positive, theproblem has no finite optimum. We stop.

5. Perform pivotal elimination to construct a new tableau andreturn to Step 2.

Outline

Setup

Illustrative ProblemSlack Variables

The Simplex Method, By ExampleThe Initial Basic Feasible SolutionCreating a New Tableau

Recap of Steps

Example

Another Example

Example

Maximize z = 3x1 − x2 + 6x3 subject to the constraints

2x1+4x2+ x3 ≤ 4

−2x1+2x2−3x3 ≥−4

2x1+ x2− x3 ≤ 8

x1 ≥ 0

x2 ≥ 0

x3 ≥ 0.

Negating row two puts this problem into standard form.

Answer.x1 = 0, x2 = 4/7, x3 = 12/7, z = 687.

Another Example

Example

Maximize z = 3x1 − x2 + 6x3 subject to the constraints

2x1+4x2+ x3 ≤ 4

−2x1+2x2−3x3 ≥−4

2x1+ x2− x3 ≤ 8

x1 ≥ 0

x2 ≥ 0

x3 ≥ 0.

Negating row two puts this problem into standard form.

Answer.x1 = 0, x2 = 4/7, x3 = 12/7, z = 687.

We insert slack variables u1, u2, and u3.The equations of constraint become

2x1+4x2+ x3+u1 ≤4

2x1−2x2+3x3 +u2 ≤4

2x1+ x2− x3 +u3 ≤8

with all variables nonnegative.