Introduction to Linear and Integer Programming

Introduction to Linear and Integer ProgrammingSaba Neyshabouri

Operations ResearchOR was developed and used during world war IIWhile the technology was advancing fast, the problems that analysts were facing were getting bigger and more complex.Decision making for complex systems are very complicated and is out of humans mind capability to solve problems with many variables.Operations Research and optimization methods try to find The Best solution for a problem.

The best example of first application of OR was how to deploy radars to get the maximum coverage.2Operation ResearchThe founder of the field is George B. Dantzig who invented Simplex method for solving Linear Programming (LP) problems.With Simplex it was shown that the optimal solution of LPs can be found.

His book, linear programming and extensions , is where he has gathered all of his ideas and notable research.3LP StructureThere are 3 main parts that forms an optimization problem:Decision Variables: Variables that represent the decision that can be made.Objective function: Each optimization problem is trying to optimize (maximize/minimize) some goal such as costs, profits, revenue.Constraints: Set of real restricting parameters that are imposed in real life or by the structure of the problem. Example for constraints can be: Limited budget for a projectLimited manpower or resourcesBeing limited to choose only one option out of many options (Assignment)

In real world problems, we are always constrained in our decision making. We are bounded to choose from our options and not exceed limits.4General Form of LPThe general form for LP is:

(1) is the objective function(2),(3) are the set of constraintsX: vector of decision variables (n*1)C: vector of objective function coefficients (n*1)A: Technology matrix (m*n)b: vector of resource availability (m*1)

This is the matrix form of the LP. There are algebraic representations which are exactly the equivalent of this representation.5LP exampleProduction planning: a producer of furniture has to make the decision about the production planning for 2 of its products: work desks and lunch tables. There are 2 resources available for production which these products are using: lumber and carpentry. 20 units of each resource is available. Here is the table containing the data for the problem:

DeskTableSelling Price$15$20DeskTableAvailabilityLumber1220Carpentry2120

6LP exampleFirst table shows the selling prices of each product, and second table show the amount of each resource that is used to produce one unit of product as well as resource availability. The question is how many of each product should be produced in order to maximize the revenue?

To model this problem as a linear programming formulation we should define our decision variables:

Here explain what the goal is and how do we go about identifying the decision variables.7Example FormulationDefining the decision variables of the problem, the formulation will be:

(1) is the total revenue generated by producing x1 desks and x2 tables.(2) is the constraint for the total number of lumbers that exists.(3) is the constraint for the total carpentry hours available.(4) is the non-negativity constraints, saying that production can not be negative.

8Feasible RegionFeasible region is defined by the set of constraints of the problem, which is all the possible points that satisfy the all the constraints.In production planning problem, the feasible set defined by the constraints looks like this plot:

Red line is for the first constraintAnd the green line is the second constraint9Feasible RegionThe red line represents the line for (2)The green line represents the line for (3)The region with blue line represents entire feasible region of the problem.

Geometric SolutionTo find the solution of LP problems, the line for objective function should be plotted and the point in feasible region which maximize (minimize) the function is the solution.Dashed parallel lines are representing the objective function line for 2 different values.

Note that to draw the objective function line, we have to fix the value of Z (revenue) so that we can draw the line, and the goal is to find the biggest Z such that the line still intersects the feasible region.11Simplex and LPSolution of LP problems are always at the extreme points of the feasible region:

Thanks to the properties of linear constraints and convex sets, the LP solution is always on the edge of the feasible region and if we know the edges of the feasible region we can try them all and fine the best solution.

Pay attention to the extreme points {(0,0),(10,0),(0,10),(6.66,6.66)} 12Simplex and LPKnowing the structure of LP problems and where the solutions are, there are finite number of extreme points that need to be checked in order to find the optimal solution.However the number of extreme points are finite, but for a problem with n variables and m constraints the upper bound for the number of extreme points is (n>m) :

Simplex is an algorithm that systematically explores the extreme points of the feasible region and moves from an extreme point to its improving neighbor.

For C(50, 10) the number is 10,272,278,170 which is huge. And the problems of real size are much larger (problems with million variables and constraints are common)Knowing how big the number of edges can be, checking all of them is not the smartest way and it is computationally vey expensive (or impossible)Simplex algorithm, takes advantage of the properties of convex optimization problems and moves on the edges of the feasible region trying to improve the objective function by moving to a better neighbor solution.13LP Solution (Production Planning)For the production planning example the solutions are:

Since the number of desks and tables can not be fractional, we might be able to round the solution down to get our integer solution, with a revenue of 210.Does rounding the solution yield the optimal solution to integer problem?

X1X2Revenue0001001500102006.666.66233.33

14Optimization SoftwareThere are various optimization tools developed based on Simplex and other optimization algorithms that are developed for LP problems.The MPL code for our example is :TITLEProduction_PlanningINDEXProduct=(Desk,LunchTable);Resource=(Lumber, Carpentry);DATAPrice[Product] = (15 , 20);Availability[Resource] = (20 , 20);Usage [Product, Resource]= (1 , 2, 2 , 1);VARIABLESX[Product];MODELMax Revenue= SUM (Product: Price * X);SUBJECT TOResourceLevel[Resource]: SUM (Product: Usage * X)