1

Introduction to optimization

Dr. Md. Golam HossainProfessorDepartment of StatisticsUniversity of RajshahiRajshahi-6205, BangladeshE-mail:hossain95@yahoo.comHP: +8801914254013

2

Module Outline Mathematical Concepts for Optimization Introduction to Linear Programming Graphical Method Simplex Method for Maximization (only type of

constraints “≤”) Problem Simplex Method for Minimization or

Maximization (Type of constraints ≥ or mix) Problem

Duality in Linear Programming

3

Mathematical Concepts for Optimization

• In Mathematics, Statistics, Empirical Sciences, Computer Science or Management Science, Mathematical optimization (Alternatively, Optimization or Mathematical Programming) is the selection of a best element (with regard to some criteria) from some set of available alternatives. • More generally, optimization includes finding "best

available" values of some objective function given a defined domain, including a variety of different types of objective functions and different types of domains.

4

Optimization Problem

An optimization problem can be represented in the following way:A function f : A   R from some set A to the real numbers. An element x0 in A such that f(x0) ≤ f(x) for all x in A (minimization) or such that f(x0) ≥ f(x) for all x in A (maximization). Such a formulation is called an optimization problem or a mathematical programming problem.  

5

We say that the function f (x) has a relative maximum value at x = a, if f (a) is greater than any value in its immediate neighborhood. We also say that a function f (x) has a relative minimum value at x = b, if f (b) is less than any value in its immediate neighborhood.

The value of the function, the value of y, at either a maximum or a minimum is called an extreme value.

6

Sufficient conditionsWe can now state these sufficient conditions for extreme values of a function at a critical value a:The function has a minimum value at x = a  if f '(a) = 0 and f ''(a) = a positive number.The function has a maximum value at x = a  if f '(a) = 0 and f ''(a) = a negative number.

Example.  Let  f(x) = x² − 6x + 5.  Are there any critical values -- any turning points?  If so, do they determine a maximum or a minimum?  And what are the coordinates on the graph of that maximum or minimum?

7

History of Optimization Greek mathematicians solved some optimization problems that were related to their geometrical studies

• Euclid [Father of Geometry] Residence: Alexandria, Egypt, Field: Mathematics, Known for: Euclidean geometry, Euclid's Elements.

Contribution: 300 BC Euclid proved that a square has the greatest area among the rectangles with given total length of edges. • Heron (100 BC) Residence: Alexandria, Roman Egypt, Field: Mathematics.

Contribution: 100 BC Heron proved that light travels between two points through the path with shortest length when reflecting from a mirror.

8

17th and 18th centuries Before the invention of calculus of variations (calculus) only some separate

optimization problems were being investigated.

Johannes Kepler (Germany) [1571-1630]Field: Astronomy, Astrology, Mathematics and Natural Philosophy

Pierre de Fermat (France): [1601-1665]Field: Mathematics and Law.

Contribution: In 1646 P. de Fermat showed that light travels between two points in minimal time.

Contribution: In 1615 he Calculated the optimal dimension of wine barrel.

9

I. Newton(1660s) and G. W. von Leibniz(1670s) create mathematical analysis that forms the basis of calculus. Some separate finite optimization problems are also considered.

Isaac Newton (England): [1642-1727]Field: Physics, Mathematics, Astronomy, Natural Philosophy,  Alchemy, Christian Theology.

Contribution: In 1687 Newton studied the body of minimal resistance.

Gottfried Wilhelm Leibniz (German): [1646-1716]Field: Mathematics, Metaphysics, Theodicy.

Contribution: In 1670 he studied some separate finite optimization problems in calculus.

10

Leonhard Euler (Switzerland): [1707-1983]Field: Mathematics and Physics.

Contribution: In 1740 he was first researcher who published research on general theory of calculus.

Joseph Louis Lagrange (France): [1736-1813].Field: Mathematics and Mathematical Physics.

Contribution: In 1760 he formulated the Plateau’s problem, the problem of minimal surfaces.

Gaspard Monge (France): [1746-1818]. Field: Mathematics and Engineering Education.

Contribution: In 1784 G. Monge investigated a combinational optimization problem known as the transportation problem.

11

19th centuryThe first optimization algorithms are presented. K.T.W. Weierstrass, J. Steiner, W.R. Hamilton and C.G.J. Jacobi further develop of calculus of variations.

Karl Theodor Wilhelm Weierstrass (German): [1815-1897].Field: Mathematics.

William Rowan Hamilton (Irish): [1805–1865]. Field: Physics, Astronomy and Mathematics.

Carl Gustav Jacob Jacobi (German):

[ 1804- 1851]

Field: Mathematics

12

Jean Baptiste Joseph Fourier (France): [1768-1830].Fields: Mathematics, Physics and History.

Contribution: In 1826 J.B.J. Fourier formulated LP-problem for solving problems arising in mechanics and probability theory.

Léon Walras (France) : [1834-1910]. Field: Economics, Marginalism.

Contribution: In 1870 Walras and Cournot shifted the focus of economists to utility maximizing individuals optimization becomes an integral part of economic theory.

13

20th Century

Johan Ludwig William Valdemar Jensen (Denmark): [1859-1925] Field: Mathematics.

Contribution: He introduced Convex functions in 1905.

Harris Hancock (America): [1867-1944]Field: Mathematics.

Contribution: H Hancock published the first text book on optimization, Theory of Maxima and Minima in 1917.

Leonid Kantorovich (Soviet Russia): [1912-1986]Field: Mathematics.

Contribution: 1939 L.V. Kantorovich presented LP-model and an algorithm for solving it. In 1975 Kantorovich and T.C. Koopmans got the Nobel memorial price of economics for their contributions on LP-problem.

After the world War II optimization develops simultaneously with operations research. J. Von Neumann is an important person behind the development of operations research. The field of algorithmic research expands as electronic calculation develops.

John von Neumann (United States): [1903-1957]Field: Mathematics and Computer Science

Contribution: Additionally he established the theory of duality for LP-problems

George Bernard Dantzig (America): [1914 – 2005] Field: Mathematics, Operation Research, Computer Science, Economics and Statistics

Contribution: In 1947 G.B.Dantzig designed the “simplex method” for solving linear programming formulations of U.S. Air Force planning problems.

1949 the first international congress, International Symposium on Mathematical Programming, on optimization is held in Chicago. The number of papers presented in the congress was 34.

15

Linear ProgrammingLinear programming has nothing to do with computer

programming.

The use of the word “programming” here means “choosing a

course of action.”

Linear programming involves choosing a course of action when

the mathematical model of the problem contains only linear

functions.

Historical Perspective for LP

Historical Perspective for LP

1928 – John von Neumann published related central theorem of game theory

1944 – Von Neumann and Morgenstern published Theory of Games and Economic Behavior

1936 – W.W. Leontief published "Quantitative Input and Output Relations in the Economic Systems of the US" which was a linear model without objective function.

1939 – Kantoravich (Russia) actually formulated and solved a LP problem

1941 – Hitchcock poses transportation problem (special LP) WWII – Allied forces formulate and solve several LP problems

related to military

A breakthrough occurred in 1947...

SCOOP

SCOOP

US Air Force wanted to investigate the feasibility of applying mathematical techniques to military budgeting and planning.

George Dantzig had proposed that interrelations between activities of a large organization can be viewed as a LP model and that the optimal program (solution) can be obtained by minimizing a (single) linear objective function.

Air Force initiated project SCOOP (Scientific Computing of Optimum Programs)

NOTE: SCOOP began in June 1947 and at the end of the same summer,

Dantzig and associates had developed: 1)An initial mathematical model of the general linear

programming problem. 2)A general method of solution called the simplex method.

Requirements of linear Programming In general, linear programming can be used for optimization problems if the following conditions are satisfied: • There must be a well-defined objective function (profit, cost or quantities produced) which

is to be either maximized or minimized and which can be expressed as a linear function of decision variables.

• There must be restrictions on the amount or extent of attainment of the objective and these restrictions must be capable of being expressed as linear equalities or inequalities in terms of variables.

• There must be alternative course of action. For example, a given product may be processed by two different machines and the problem may be as to how much of the product to allocate to which machine.

• Another necessary requirement is that decision variables should be interrelated and non-negative. The non-negativity condition shows that linear programming deals with real-life situations for which negative quantities are generally illogical.

• The recourses must be in limited supply. For example, if a firm starts producing greater number of a particular product, it must make smaller number of other products as the total production capacity is limited.

19

Linear Programming (LP) Problem

If both the objective function and the constraints are linear, the problem is referred to as a linear programming problem.

Linear functions are functions in which each variable appears in a separate term raised to the first power and is multiplied by a constant (which could be 0).

Linear constraints are linear functions that are restricted to be "less than or equal to", "equal to", or "greater than or equal to" a constant.

20

Guidelines for Model Formulation Understand the problem thoroughly. Describe the objective. Describe each constraint. Define the decision variables. Write the objective in terms of the decision

variables. Write the constraints in terms of the decision

variables.

21

Example1:Iron Works, Inc. seeks to maximize profit by making two products from steel. It just received this month's allocation of 19 pounds of steel. It takes 2 pounds of steel to make a unit of product 1, and 3 pounds of steel to make a unit of product 2. The physical plant has the capacity to make at most 6 units of product 1, and at most 8 units of total product (product 1 plus product 2). Product 1 has unit profit 5, and product 2 has unit profit 7. Formulate the linear program to maximize profit.

Answer: Here is a mathematical formulation of the objective. Let x1 and x2 denote this month's production level of

product 1 and product 2. The total monthly profit = (profit per unit of product 1) x (monthly production of product 1) + (profit per unit of product 2) x (monthly production of product 2)

= 5x1 + 7x2

Maximize total monthly profit: Max z = 5x1 + 7x2

22

Here is a mathematical formulation of constraints. The total amount of steel used during monthly production = (steel used per unit of product 1) x (monthly production of product 1) + (steel used per unit of product 2) x (monthly production of product 2) = 2x1 + 3x2

That quantity must be less than or equal to the allocated 19 pounds of steel (the inequality < in the constraint below assumes excess steel can be freely disposed; if disposal is impossible, then use equality =) 2x1 + 3x2 < 19The constraint that the physical plant has the capacity to make at most 6 units of product 1 is formulated x1 < 6The constraint that the physical plant has the capacity to make at most 8 units of total product (product 1 plus product 2) is x1 + x2 < 8

23

Adding the non-negativity of production completes the formulation

Max Z= 5x1 + 7x2

s.t. x1 < 6

2x1 + 3x2 < 19

x1 + x2 < 8

x1 > 0 and x2 > 0

Objective

Function

“Regular”Constraint

sNon-negativity Constraints

“Max” means maximize, and “s.t.” means subject to.

24

Linear Programming Problem can be solved by

• Graphical Method

Corner point solution method

Isoprofit line solution method

• Simplex Method

Simplex Method I (Maximization, for constrains, ≤)

Simplex Method II (Minimization or maximization for constraints, ≥ or mix)

Some important mathematical items related to method solving LPP • Hyperplane in Rn: The set of points x =[ x1, x2,…,xn]

satisfying the equation, z = c1x1+c2x2+...+cnxn, where ci’s are constants not all zero, and z is a constant, is called a hyperplane, determined by the vector c = (c1, c2, …, cn) and scalar z.

• Half Spaces: In Rn, let c.x =z be a hyperplane, then the two sets of points

K+= {x|c.x≥z} and K- = {x|c.x≤z} are called the closed half spaces, determined by c.x=z.

• Convex Set: A subset S C Rn is said to be convex, if for each pair of points x, y in S, the line segment [x:y] is contained in S.

Some Basic Definitions:• Objective Function: The linear function, z = c1x1+c2x2+…+cnxn, which is

to be minimized or maximized is called the objective function of the GLPP.

• Subject to the Constraints ( Linear constraints): are linear functions that are restricted to be "less than or equal to", "equal to", or "greater than or equal to" a constant.

– Thus 2x1 + 3x2 < 19 is a linear constraint, but

2x1 + 3x2 < 19 and 2x1 + 3x1x2 < 19 are not.

• Solution: Values of decision variables xj (j=1, 2, 3,…,n) which satisfy subject to the constraints of a GLLP is called the solution to that LPP.

• Feasible solution: Any solution to a GLPP which satisfies the non-negative restrictions of the problem, is called a feasible solution to that problem.

• Optimum solution: Any feasible solution which optimizes (minimizes or maximizes) the objective function of a GLPP is called an optimum solution to that problem.

27

Example 1: Graphical Solution

First Constraint Graphed x2

x1

x1 = 6

(6, 0)

8

7

6

5

4

3

2

1

1 2 3 4 5 6 7 8 9 10

Shaded regioncontains all

feasible pointsfor this constraint

28

Example 1: Graphical Solution

Second Constraint Graphed

2x1 + 3x2 = 19

x2

x1

(0, 6 1/3)

(9 1/2, 0)

8

7

6

5

4

3

2

1

1 2 3 4 5 6 7 8 9 10

Shadedregion containsall feasible pointsfor this constraint

29

Example 1: Graphical Solution

Third Constraint Graphed x2

x1

x1 + x2 = 8

(0, 8)

(8, 0)

8

7

6

5

4

3

2

1

1 2 3 4 5 6 7 8 9 10

Shadedregion containsall feasible pointsfor this constraint

30

Example 1: Graphical Solution

x1

x2

8

7

6

5

4

3

2

1

1 2 3 4 5 6 7 8 9 10

2x1 + 3x2 = 19

x1 + x2 = 8

x1 = 6

Combined-Constraint Graph Showing Feasible Region

Feasible Region

31

Example 1: Graphical Solution Objective Function Line

x1

x2

(7, 0)

(0, 5)Objective Function5x1 + 7x2 = 35Objective Function5x1 + 7x2 = 35

8

7

6

5

4

3

2

1

1 2 3 4 5 6 7 8 9 10

32

Example 1: Graphical Solution

Selected Objective Function Lines

x1

x2

5x1 + 7x2 = 355x1 + 7x2 = 35

8

7

6

5

4

3

2

1

1 2 3 4 5 6 7 8 9 10

5x1 + 7x2 = 425x1 + 7x2 = 42

5x1 + 7x2 = 395x1 + 7x2 = 39

33

Example 1: Graphical Solution

Optimal Solution

x1

x2 MaximumObjective Function Line5x1 + 7x2 = 46

MaximumObjective Function Line5x1 + 7x2 = 46

Optimal Solution(x1 = 5, x2 = 3)Optimal Solution(x1 = 5, x2 = 3)

8

7

6

5

4

3

2

1

1 2 3 4 5 6 7 8 9 10Feasible /Convex Region

Problem. A firm manufactures pain relieving pills in two sizes A and B, size A contains 4 grains of element a, 7 grains of element b and 2 grains of element c, size B contains 2 grains of element a, 10 grains of element b and 8 grains of c. It is found by users that it requires at least 12 grains of element a, 74 grains of element b and 24 grains of element c to provide immediate relief. It is required to determine that least no. of pills a patient should take to get immediate relief. Formulate the problem as standard LPP.

36

Graphical Solution Algorithm Formulate the given linear programming problem in

mathematical format. Construct the graph for the problem so formulated. Identify the convex region (solution space) and the

vertices of the convex region. Evaluate the values of the objective function at these

vertices of the convex region. Determine the vertex at which the objective function

attains its maximum or minimum (optimum). Interpret the optimum solution so obtained.

37

Slack and Surplus Variables

A linear program in which all the variables are non-negative and all the constraints are equalities is said to be in standard form.

Standard form is attained by adding slack variables to "less than or equal to" constraints, and by subtracting surplus variables from "greater than or equal to" constraints.

Slack and surplus variables represent the difference between the left and right sides of the constraints.

Slack and surplus variables have objective function coefficients equal to 0.

38

Max Z= 5x1 + 7x2 + 0s1 + 0s2 + 0s3

s.t. x1 + s1 = 6

2x1 + 3x2 + s2 = 19

x1 + x2+ s3 = 8

x1, x2 , s1 , s2 , s3 > 0

Example 1 in Standard Form

Slack Variables (for < constraints)

s1 , s2 , and s3 are slack variables

39

Surplus Variables (for ≥ constraints)

Write down the following LPP in standard form:Minimize Z = 3x1+ 2x2+5x3

Subject to the constraints: 2x1+3x2 ≥3x1+2x2+3x3 ≥102x1+3x3≥8x1, x2, x3 ≥0

The Standard Form LPP is,Minimize Z = 3x1+ 2x2+5x3+0.s1 +0.s2+0.s3

Subject to the constraints: 2x1+3x2 –s1 =3x1+2x2+3x3 –s2 =102x1+3x3 –s3 =8x1, x2, x3, s1, s2, s3 ≥0

Where, s1 , s2 , and s3 are surplus variables

40

Some important definitions

Standard form of LPP: Let x, c ЄRn and z= cTx be a linear function on Rn. Let A be an m x n real matrix of rank m. Then, the problem of determining x, so as to Maximize z = cTx, subject to the constraints: Ax = b, x≥0, where b is an m x 1 real matrix, is said to be a linear programming problem written in its standard form.

Basic Solution: Given a system of m simultaneous linear equations in n unknowns

Ax = b; x ЄRn, where A is an m x n real matrix of rank m (m<n). Let B be m x m sub matrix, formed by m linearly independent columns of A. Then a solution obtained by setting n-m variables not associated with the columns of B equal to zero, and solving the resulting system, is called a Basic solution to the given system of equations.

The m variables, which may all be different from zero, are called Basic variables.

The m x m non-singular submatrix B is called Basis matrix and the columns of B as basic vectors.

Degenerate Solution: A basic solution to the system Ax = b is called Degenerate if one or more of the basic variables vanish.

Basic feasible solution: A feasible solution to the LPP which is also basic solution to the problem, is called a basic feasible solution to the LPP.

Associated cost vector: Let xB be a basic feasible solution to the LPP:

Maximize z= cTx, Subject to constraints: Ax =b, x ≥0. Then the vector

cB = [ cB1, cB2, …, cBn], where cBi are components of c associated with the basic variables, is called cost vector associated with the basic feasible solution xB.

Optimum Basic Feasible Solution: A basic feasible solution xB to the LPP.

Maximize z = cTx, subject to the constraints: Ax =b, x ≥0, is called an optimum basic feasible solution if z0= cB

T xB ≥z*, where z* is the value of the objective function for any feasible solution.

Example: Find the basic feasible solution to the system of equations:

2x1 +x2 -x3 = 23x1 + 2x2 +x3 = 3

Solution. The given system of equations can be written as Ax =b

where, A= x = and b= .

Since rank of A is 2, the maximum number of linearly independent columns of A is 2. 2 X 2 sub matrices of A are;

The variables not associated with the columns of these sub matrices are, respectively, x3, x1 and x2.

123

112

3

2

1

x

x

x

3

2

23

12

12

11

13

12and

Considering B= , a basic solution to the given system is obtained by setting x3= 0 and solving the systemThus a basic solution to the given problem is, (Basic) x1 = 1, x2 = 0; (Non-basic) x3= 0. Similarly the other two solutions are, (Basic) x2= 5/3, x3 = -1/3; (Non-basic) x1= 0, and (Basic) x1= 1, x3=0; (Non-basic) x2=0. In each of the two basic solutions, at least one of the basic variables is zero. Hence two of the basic solutions are degenerate solutions. The solution [o, 5/3, -1/3] is not a feasible solution. The two basic solutions [1,0,0] and [1,0,0] are basic feasible solutions.

23

12

23

12

2

1

x

x

=

3

2

Some Importance Theorems for Simplex Methods (without proof)

Theorem1: The set of feasible solutions to an LPP is a convex set.

Theorem2: [Fundamental theorem of LP] If the feasible region of an LPP is a convex polyhedron, then there exists an optimal solution to the LPP and at least one basic feasible solution must be optimal.

Theorem3: [Reduction of feasible solution to a basic feasible solution] If an LPP has a feasible solution, then it also has a basic feasible solution.

Theorem4: [Replacement of a Basic vector] Let an LPP have a basic feasible solution. If we drop one of the basic vectors and introduce a non-basic vector in the basic set, then the new solution obtained is also a basic feasible solution, provided these vectors are suitably selected.

x1 x2 xn

Cost VectorcBi

Basic VariablexBi

Value of xBi

c1 c2 cn

cB1

cB2

cBm

Index Row zj-cj

A (Coefficient

Matrix)

Sample of Simplex Table

• zj = Σi=1cBi yij is called the

evaluation and the

number zj-cj is called

the net evaluation

corresponding to each column.

Theorem 5: [Conditions of optimality] A sufficient condition for a basic feasible solution to an LPP to be an optimum (maximum) is that zj-cj ≥0 for all j for which the column vector aj ЄA is not, in the basis B.

Theorem 6: [ Unbounded solution] Let there exist a basic feasible solution to a given LPP. If for at least one j, for which yij ≤0 (I = 1,2,…, m), zj-cj is negative, then there does not exist any optimum solution to this LPP.

Example: Solve the following LPP by Simplex Method:

Max, z= x1+2x2, S.t. x1-x2 ≤4, x1-5x2 ≤8, x1, x2 ≥0

Solution: Standard form of LPP is

Objective function

Max, z = x1+2x2+0.x3+0.x4

Subject to the constrains:

x1-x2 +x3+0.x4=4

x1-5x2 + 0.x3 + x4 =8

x1, x2, x3, x4 ≥0

cj 1 2 0 0

cBi xBi Value of xBi

x1 x2 x3 x4

0 x3 4 1 -1 1 0

0 x4 8 1 -5 0 1

zj 0 0=z1 0=z2 0=z3 0=z4

Index row zj-cj

z1-c1=-1 Z2-c2=-2 Z3-c3=0 Z4-c4=0

Initial Simplex Table

FLOW CHART

Simplex Algorithm for Maximization

Reformulate the given LPP as standard form

Obtain an initial basic feasible solution of the problem

Compute the net Evaluation, set up in initial Table

Examine the index row of net evaluation

Is there any negative net evaluation?

Optimum solution has been attained and STOP

Are all elements of a column non-positives and corresponding net evaluation also negative?

There exist an unbound solution to the problem,STOP

Choose the most negative net evaluation. Corresponding the most negative evaluation column is KEY column.

Select the positive element from key column and divide the corresponding values of the current basic variables by them

Choose the smallest ratio. Row contains the smallest ratio is called Key row

Identify the Pivot/Key element. Key column and Key row meet at Pivot/key element.

Update the simplex table by appropriate operation (pivoting)

NO

YES

Example2: A firm manufactures 3 products A, B and C. The profits are $3, $2, and $4 respectively. The firm has 2 machines and below is the required processing time in minutes for each machine on each product. Machine G and H have 2000 and 2500 machine-minutes respectively. How many units of each product should be produced in order to gain maximum profit?

Machine Time per unit (Minutes) Machine Capacity (Minutes)

A B C

G 4 3 5 2000

H 2 2 4 2500

Solution: Define the objective Maximize profit.Define the decision variables: x 1 = number of products of type A, x 2 = number of products of type B and x 3 = number of products of type C .

Objective Function,Max Z = 3x 1 + 2x 2 + 4x 3 Subject to the constraints,4x 1 + 3x 2 + 5x 3 ≤ 2000 2x 1 + 2x 2 + 4x 3 ≤ 2500 x 1 , x 2 , x 3 ≥ 0

The standard form of LPP is, Objective Function,Max Z = 3x1 + 2x2 + 4x3 +0.x4+0.x5

Subject to the constraints,4x1 + 3x2 + 5x3 +x4 +0.x5 = 2000 2x1 + 2x2 + 4x3 +0.x4 +x5 = 2500 x1 , x2 , x3, x4, x5 ≥ 0

An initial basic feasible solution is:

x1 = x2= x3 =0, then x4 = 2000 and x5 = 2500

cj 3 2 4 0 0 Minimumratio

cBi xBi Value of xBi

x1 x2 x3 x4 x5 xBi/yij; yij>0

0 x4 2000 4 3 1 0 2000/5=400

0 x5 2500 2 2 4 0 1 2500/4=625

zj 0 0 0 0 0 0

Index Row (zj-cj) -3 -2 -4 0 0

Initial Simplex Table

Since all elements in index row are non-positive, thus we have not attained optimum solution. The current solution can be further improved.

Key Column

Key Row5

5 is Pivot/Key element

Algorithm for Iteration

Step1: Identify the KEY Column, KEY row and the leading element (pivot/key element) from previous table.

Step2: Determine the departing variable: The key row and corresponding basic variable will leave the basis.

Step3: Determine the introducing variable: The corresponding non-basic variable of Key column will be replaced in basic for the next table.

Step 4: Compute values for the key row (previous table) for new table: To do this, simply divide every number in the key row (previous table) by the element. [ y^

kj=ykj/ykr]

Step 5: Compute values for each remaining row (rows) for new table: All remaining row (s) for the next simplex table can be calculated by the formula:

New row number = (Number in old row) – [Corresponding number in key row] X [Corresponding fixed ratio], [y^ij = yij-(ykj/ykr)yir]

where fixed ratio = old number in the key column/key element (number).

Step6: Put the new values in new simplex table (First Iteration):

Step7: Test for optimality and take decision after looking Index row.

Simplex Table1 (First Iteration)

3 2 4 0 0

cBi xBi Value of xBi x1 x2 x3 x4 x5

4 x3 2000/5 =400 4/5 3/5 5/5=1 1/5 0

0 x5 2500-2000X4/5 =900

2-4X4/5=-6/5

2-3X4/5=-2/5

4-5X4/5=0

0-1X4/5=-4/5

1-0X4/5=1

zj 1600 16/5 12/5 4 4/5 0

Index Row (zj-cj) 1/5 2/5 0 4/5 0

We observe that all values in index row of above table are non-negative thus we have reached optimum solution and the optimum solution is x1=0, x2 =0, x3 = 400. The maximum profit, z = $1600.

Fixed Ratio = 4/5

52

Surplus Variables (for ≥ constraints)

Write down the following LPP in standard form:Minimize Z = 3x1+ 2x2+5x3

Subject to the constraints: 2x1+3x2 ≥3x1+2x2+3x3 ≥102x1+3x3≥8x1, x2, x3 ≥0

The Standard Form LPP is,Minimize Z = 3x1+ 2x2+5x3+0.s1 +0.s2+0.s3

Subject to the constraints: 2x1+3x2 –s1 =3x1+2x2+3x3 –s2 =102x1+3x3 –s3 =8x1, x2, x3, s1, s2, s3 ≥0

where, s1 , s2 , and s3 are surplus variables

Simplex Method II (≥ constraints, or ≤ and ≥ constraints, or ≥ and = constraints or ≥, ≤ and = constraints)

There are two procedures for solving the problem:

• Method of Penalties or Big M Method-Developed by A. Charnes• Two-Phase Method-Developed by Dantzig, Orden and Wolfe

Method of Penalties or Big M Method-Developed:

The Standard Form LPP is,Minimize Z = 3x1+ 2x2+5x3+0.s1 +0.s2+0.s3 +MA1+MA2+MA3

Subject to the constraints: 2x1+3x2 –s1+0. s2 +0. s3 +A1+0.A2+0.A3 =3x1+2x2+3x3 +0.s1 –s2 +0.s3+0.A1+A2+0.A3=102x1+3x3 +0.s1+0.s2 –s3 +0.A1+0.A2+A3=8x1, x2, x3, s1, s2, s3, A1, A2 , A3≥0where, s1 , s2 and s3 are surplus variables, and A1, A2 and A3 are artificial variables.

For getting initial basic solution, we put, x1=x2=x3=s1=s2=s3=0, we get A1=3, A2=10 and A3= 8 [Basic]

54

Example3: A small jewelry manufacturing company employs a person who is a highly skilled gem cutter, and it wishes to use this person at least 6 hours per day for this purpose. On the other hand, the polishing facilities can be used in any amount up to 8 hours per day. The company specializes in three kinds of semiprecious stones P, Q and R. Relevant cutting, polishing and cost requirements are listed in the following table: P Q R

Cutting 2 hr 1 hr 1 hr

Polishing 1 hr 1 hr 2 hr

Cost per stone $ 30 $ 30 $ 10

How many gemstones of each type should be processed each day to minimize the cost of the finished stones? What is the minimum cost? Solve this problem by Simplex Method.

55

Solution: Le x1, x2 and x3 represent the number of type p, q and r stones finished per day, respectively. The following mathematical problem to solve:Minimize, z = 30 x1+30 x2 + 10x3 Subject to constrains: 2x1+ x2 + x3 ≥6 x1+ x2 + 2x3 ≤8 x1, x2 , x3 ≥0

Introducing slack, surplus and artificial variables, the modified problem is: Minimize, z = 30 x1+30 x2 + 10x3 +0.s1 +MA + 0. s2

Subject to constrains: 2x1+ x2 + x3 -s1 +A =6 x1+ x2 + 2x3 + s2 =8 x1, x2 , x3 , s1 , s2 and A≥0

From the above system of equations, an initial basic feasible solution is:A = 6 and s2 = 8

Which can be displayed in the following table:

56

Simplex Tableau I, IIcj 30 30 10 0 M 0

CBi Basic variables

Solution values

x1 x2 x3 s1 A s2Minimum

ratio

M A 6 2 1 1 -1 1 0 3

0 s2 8 1 1 2 0 0 1 8

Cj-Zj 30-2M 30-M 10-M M 0 0

30 x1 3 1 1/2 1/2 -1/2 0 6

0 s2 5 1 10/3

Cj-Zj 0 15 -5 15 0

30 x1 4/3 1 1/3 0 -2/3 -1/3

10 x3 10/3 0 1/3 1 1/3 2/3

Cj-Zj 220/3 0 50/3 0 50/3 10/3

Thus, the optimal solution for the problem is: x1 = 4/3, x2 = 0, x3 = 10/3 with min. z = 220/3. The minimum cost of $ 220/3 for gemstones will be produced if 4/3 unit of P, 0 unit of Q and 10/3 unit of R.

Artificial Variable—LPP in which one or more constraints are of “ ≥” or “=“ type (after ensuring that all all bs’ are non negative) in such cases we introduce another type of variable known as artificial variables in order to get the initial basic feasible solution.

What is the physical meaning of artificial variable? Ans: These variables are fictitious and have no physical meaning or economic significance. They are merely a device to get the starting basic feasible solution so that simplex algorithm may be applied. These variables are required because in such problems the basic matrix is not obtained as identity matrix in the starting simplex table.

Remark: By assigning a very large per unit penalty to each of the artificial variables in the objective function, these variables can be removed from the simplex table as soon as they become non-basic. Such a penalty is designated as –M for maximization problems and +M for minimization problems, where M≥0.

58

Procedure for Big M methodStep1: If negative value of right hand side of any constraint, multiply -1 to obtain positive.

Step2: Introduce slack variable (s) if “≤” constraint (s), surplus and artificial variables if “≥” constraints and artificial variable if “=“ constraint.

Step3: For each artificial variable Ai add -M Ai to the objective function cease of maximization and + M Ai in case of minimization problem. Use the same constant M for artificial variable.

Step4: form the simplex tableau for the modified problem.

Step5: Solve the modified problem by simplex method, as we described earlier.

Step6: Relate the solution of the modified problem to the original problem:(i) If the modified problem has no solution then the original problem has no solution. (ii) If any artificial variables are non-zero in the solution to the modified problem though the all elements in index row are non-negative, then the original problem has no solution.

59

Two-Phase Simplex Method

Two-phase simplex method is an alternative procedure to solve LPP’s involving artificial variables.

Phase I: Test the feasibility (consistency) of the given problem.

If the coefficient matrix A contains a unit sub-matrix Im, where m is the rank of A, then it will serve as an initial basis. Proceed on to Phase 2. We can consider the LPP determining (x, xart) so as to maximize

Z* = -(xa1+xa2+…+xap),

subject to constraints:

Ax+ Ipxart =b; x≥0, xart≥0.

This will be called an auxiliary LPP in which artificial variables are assigned a cost -1 each and all others a cost 0.

Clearly max. z*≤0 (since all xart ≥0), therefore this LPP cannot have an unbounded solution.

Two cases now arise:

Case1: If Max Z* <0, in this case the original LPP has no feasible solution.

Case 2: If Max Z* = 0, in this case the original LPP has feasible solution.

Phase 2

The final simplex table of Phase 1 is converted into the initial simplex table of Phase2, just by deleting the non-basic artificial columns and re-computing z* and the net evaluations, in accordance with the original objective function.

Assign the actual costs to the variables in the objective function and a zero cost to every artificial variable that appears in the basis at the zero level. This new objective function is now maximized subject to the given constraints by applying simplex method.

Example 4: Use two-Phase simplex method to maximize z= 2x1+x2-x3

Subject to the constraints:

4x1+6x2+3x3≤8

3x1-6x2-4x3 ≤ 1

2x1+3x2-5x3≥4

x1, x2, x3 ≥0

Solution:

The standard form of LPP is below: 4x1+6x2+3x3+x4=8 3x1-6x2-4x3+x5 =1

2x1+3x2-5x3-x6+A=4where, x4, x5 , x6 and A are slack, surplus and artificial variables, respectively, and x1, x2, x3, x4, x5, x6, A ≥0.

The initial basic feasible solution is, xB = [8, 1, 4]. Phase 1: The objective function of the auxiliary LPP is Z* = Σ0.xj-A.

cj 0 0 0 0 0 0 -1

cBi xBi Value of xBi x1 x2 x3 x4 x5 x6 A

0 x4 8 4 6* 3 1 0 0 0

0 x5 1 3 -6 -4 0 1 0 0

-1 A 4 2 3 -5 0 0 -1 1

Index row (zj-cj)

-4 -2 -3 5 0 0 1 0

Initial Simplex Table

cj 0 0 0 0 0 0 -1

CBi xBi Value of xBi

x1 x2 x3 x4 x5 x6 A

0 X2 4/3 2/3 1 ½ 1/6 0 0 0

0 X5 9 7 0 -1 1 1 0 0

-1 A 0 0 0 -13/2 -1/2 0 -1 1

Index row (zj-cj) 0 0 0 13/2 1/2 0 1 0

First iteration. Introduce x2 and drop x4

Since all numbers in index row are non-negative, a optimum solution has been attained in auxiliary LPP, and max Z* =0 and the artificial vector A appears in the basis at the zero level. Thus Phase I has found out a starting BFS to the given LPP.

Phase II will start from this BFS and move towards optimality.

Phase II: We consider the actual costs associated with the original variables and assign a cost zero to the artificial variable A, which appeared at zero level in Phase I. The new objective function is therefore,

Z= 2x1+x2-x3+0.x4+0.x5+0.x6+0.A.

We wish to maximize the new objective function subject to the given constraints by applying the simplex method. The optimum basic feasible solution if any, thus attained will be an optimum basic feasible solution to the original LPP.

cj 2 1 -1 0 0 0 0

cB XB Value of XB

x1 x2 x3 x4 x5 x6 A

1 X2 4/3 2/3 1 ½ 1/6 0 0 0

0 X5 9 7* 0 -1 1 1 0 0

0 A 0 0 0 -13/2 -1/2 0 -1 1

Index row 4/3 -4/3 0 3/2 1/6 0 0 0

Staring Table

Cj 2 1 -1 0 0 0 0

CB XB Value of XB

x1 x2 x3 x4 x5 x6 A

1 X2 10/12 0 1 25/42 1/14 -2/21 0 0

0 X1 9/7 1 0 -1/7 1/7 1/7 0 0

0 A 0 0 0 -13/2 -1/2 0 -1 1

Index row 4/3 0 0 55/42 5/14 4/21 0 0

First Iteration. Introduce x1 and drop x5

We observe from the above table, since all n umber in the index row are non-negative, therefore an optimum basic feasible solution has been attained. Hence an optimum basic feasible solution to the given LPP is

x1= 9/7, x2 = 10/12 and x3=0 and Max. Z= 64/21.

DualityThe term duality implies that every linear programming problem whether of maximization or minimization has associated with it another linear programming problem based on the same data. The original (given) problem is called the primal problem while the other is called its dual problem.

Given the primal problem (in standard form)

nn xcxcxcz ...2211

11 1 12 2 1 1

21 1 22 2 2 2

1 1 2 2

1 2 1 2

...

...

.

.

...

, ,..., 0, , ,..., 0

n n

n n

m m mn n m

n m

a x a x a x b

a x a x a x b

a x a x a x b

x x x b b b

Maximize

Subject to

The dual problem is the LPP

mm ybybybw ...2211

11 1 21 2 1 1

12 1 22 2 2 2

1 1 2 2

1 2

...

...

.

.

...

, ,..., unrestricted in sign

m m

m m

n n mn m n

n

a y a y a y c

a y a y a y c

a y a y a y c

y y y

Minimize

Subject to

66

We thus note the following:1. In the dual, there are as many (decision) variables as there are constraints in the

primal.

We usually say yi is the dual variable associated with the ith constraint of the primal.

2. There are as many constraints in the dual as there are variables in the primal.

3. If the primal is maximization then the dual is minimization and all constraints are

If the primal is minimization then the dual is maximization and all constraints are .

4. In the primal, all variables are 0 while in the dual all the variables are unrestricted in sign.

5. The objective function coefficients cj of the primal are the RHS constants of the dual constraints.

6. The RHS constants bi of the primal constraints are the objective function coefficients of the dual.

7. The coefficient matrix of the constraints of the dual is the transpose of the coefficient matrix of the constraints of the primal.

67

Example 5: Obtain the dual of

The primal problem

Minimize z= 16x1+9x2+21x3

Subject to the constraints:

x1+x2+3x3≥16

2x1+x2 +x3≥12

x1, x2, x3 ≥0

Solution:

Step1. Form the matrix A:

A = 1 1 3 16

2 1 1 12

16 9 21

Step2: Form the Matrix B (the transpose of A)

1 2 16

1 1 9

3 1 21

16 12

B = AT=

Step3: State the dual problem:

Maximize z* = 16y1+12 y2

Subject to the constraints: y1+2y2≤16, y1+y2≤9, 3y1+y2≤21

y1, y2≥0

68

Some Important Theorems for Duality (without proof)

Theorem: The dual of the dual is the primal

Theorem: [Basic Duality Theorem] Let a primal problem be Maximize f(x) = cT x, subject to Ax≤b, x≥0 and x, c ЄRn and the associated dual be Minimize g(w) = bTw subject to AT w≥c, w ≥0; w, b ЄRm. If x0 (w0) is an optimum solution to the primal (dual), then there exists a feasible solution w0(x0) to the dual (primal), such that cT x0= bTw0

Theorem [Fundamental theorem of Duality] Let a primal problem be Maximize f(x) = cTx x subject to Ax≤b, x≥0 and x, c ЄRn, and its dual be Minimize g(w) = bTw subject to AT w≥c, w ≥0; w, b ЄRm. A neceassary and sufficient condition for a feasible solution x0(w0) to the primal (dual) problem, to be an optimum solution is that there exists a feasible solution w0 (x0) to the dual (primal) problem such that cTx0=bTw0. The feasible solution w0 (x0) then itself is an optimum solution to the dual (primal) problem, that is Max. f(x) = min. g(w) holds for all such pairs of x0 and w0.

Example 6: A firm makes two products A and B. Each product requires production on each of the two machines:

Product Available

(in hours)Machine A B

M1 6 4 60

M2 1 2 22

Total time available is 60 hours and 22 hour on machines M1 and M2 respectively. Product A and B contribute $3 and $ 4 per unit respectively. Determine the optimum product in order to gain the maximum profit. Write the dual of this problem and give its economic interpretation.

Solution: Mathematical Formulation. The primal and its

dual linear programming problems are:

Primal problem

Maximize z = 3x1 +4x2

Subject to constraints:

6x1 +4x2≤60

x1 +2x2≤22

x1, x2 ≥0where x1 =umber of units of product A

and x2 = number of units of product B

Dual problem

Minimize z* = 60 w1 + 22 w2

Subject to the constraints:

6w1 + w2 ≥3

4w1 + 2w2 ≥4

w1, w2 ≥0

where w1 = cost of one hour on machine M1 and w2 = cost of one hour on machine M2.

70

Solution to the Primal problem The optimum solution of the primal problem can easily be obtained by simplex method. The optimum simplex table is given below:

3 4 0 0

cBi xBi value of xBi

x1 x2 x3 x4

3 x1 4 1 0 1/4 -1/2

4 x2 9 0 1 -1/8 -3/4

zj -cj 48 0 0 0.25 1.50

Final Iteration:

The optimum solution is: x1 = 4, x2 = 9 and the maximum profit = $ 48

Using duality, optimum solution to the dual problem can be obtained directly from the above table (optimum simplex table); w1 = $0.25 per hour, w2 = $ 1.50 per hour, and minimum cost = $48.

71

Economic Interpretation of Dual: As mentioned above shadow prices are the opportunity costs that indicate the potential profit that is lost by not having an additional unit of the respective right-hand side (resource), assuming that all right-hand side values are used optimally. Thus w1 =0.25 and w2= 1.50 means that an additional processing hour on machine M1 and M2 will increase the profit by $0.25 and $1.50 respectively.

If we increases the total available hours on machine M1 from 60 hours to 61 hours; the new

set of constraints will be: 6x1 +4x2≤60 and x1 +2x2≤22.

On solving the primal problem with new set of constraints, the optimum solution becomes: x1 = 0.25 and x2 = 8.875 with maximum z = $48.25.

Shadow Price: A shadow price is the maximum price that management is willing to pay for an extra unit of a given limited resource.

Dual Simplex Algorithm

Since any LPP can be solved by using simplex method,

the method is applicable to both the primal as well as its dual.

References

MacTutor history of mathematicsHistory of mathematical symbols and terms (J. Miller)History of economics (The New School University)History of game theory (P. Walker)IEEE history pagesHistoria Mathematica journalEuclid's elementsWolfram's MathworldMathematical programming glossary (INFORMS)Von Neumann prize winners (INFORMS)Brief hostory of feedback control (F.L. Lewis)

Dantzig, G. B. (1963) Linear Programming and Extensions. Princeton University Press.

Dewdney, A.K. (1989) Linear Programming. The New Turing Omnibus. Computer Science Press.

Kapoor V.K. (1990) Operations Research. Sultan Chand & Sons press, New Delhi, India.

Gupta P.K. and Mohan M. (2001) Linear programming and theory of game. Sultan Chand & Sons press, New Delhi, India.

Links for Mathematical concepts for optimization:

Questions/Queries?

Thank you for your kind attention and listening