8/10/2019 Mixed Integer Linear Programming
1/42
6. Mixed Integer Linear Programming
Javier Larrosa
Albert Oliveras
Enric Rodrguez-Carbonell
Problem Solving and Constraint Programming (RPAR)
Session 6 p.1/40
8/10/2019 Mixed Integer Linear Programming
2/42
Mixed Integer Linear Programming
Amixed integer linear program(MILP,MIP) is of the form
min cTx
Ax=b
x0
xiZ
i IIf all variables need to be integer,it is called a(pure) integer linear program(ILP, IP)
If all variables need to be0or1(binary, boolean),it is called a0 1linear program
Session 6 p.2/40
8/10/2019 Mixed Integer Linear Programming
3/42
Applications of MIP
Used in contexts where, e.g.:
it only makes sense to take integral quantities
of certain goods or resources, e.g.:men(human resources planning)
power stations(facility location)
binary decisions need to be taken
producing a product(production planning)
assigning a task to a worker(assignment problems)assigning a slot to a course(timetabling)
And many many more...
Session 6 p.3/40
8/10/2019 Mixed Integer Linear Programming
4/42
Computational Complexity: LP vs. IP
Including integer variables increases enourmously themodeling power,at the expense of more complexity
LPs can be solved inpolynomial timewith interior-pointmethods (ellipsoid method, Karmarkars algorithm)
Integer Programming is anNP-completeproblem. So:
There isno known polynomial-time algorithm
There arelittle chancesthat one will ever be found
Even small problems may be hard to solveWhat follows is one of the many approaches(and one of the most successful) for attacking IPs
Session 6 p.4/40
8/10/2019 Mixed Integer Linear Programming
5/42
LP Relaxation of a MIP
Given a MIP
(IP)
min cTx
Ax=b
x0
xi Z i I
itslinear relaxationconsists in the LP obtained bydropping the integrality constraints:
(LP)
min cTx
Ax=b
x0
Can we solve IPby solvingLP? By rounding?Session 6 p.5/40
8/10/2019 Mixed Integer Linear Programming
6/42
Branch & Bound (1)
The optimal solution of
max x+y
2x+ 2y 1
8x+ 10y 13
x, y
Z
is(x, y) = (1, 2)with objective3
The optimal solution of its LP relaxation
is(x, y) = (4, 4.5)with objective9.5
No direct way of getting from(x, y) = (4, 4.5)to(x, y) = (1, 2)by rounding!
Something more elaborate is needed: branch & boundSession 6 p.6/40
8/10/2019 Mixed Integer Linear Programming
7/42
Branch & Bound (2)
(1, 2)
y
x
x 0
y0
max x+y
(4, 4.5)
2x + 2y1
8x + 10y13
Session 6 p.7/40
8/10/2019 Mixed Integer Linear Programming
8/42
Branch & Bound (3)
Assume integer variables have lower and upper boundsLetP0 initial problem,LP(P0)LP relaxation of P0
If in optimal solution of LP(P0)all integer variables takeinteger values then it is also an optimal solution to P0
Else
Roundingthe solution ofLP(P0)mayyieldtonon-optimalornon-feasiblesolutions for P0!
Letxj be integer variable whose value j at optimal
solution ofLP(P0)satisfies j Z. Define
P1:=P0 xj j
P2:=P0 xj j
Feasible solutions to P0 =feasible solutions to P1 orP2Session 6 p.8/40
8/10/2019 Mixed Integer Linear Programming
9/42
Branch & Bound (4)
Letxj be integer variable whose value j at optimalsolution ofLP(P0)satisfies j Z.
P1:=P0 xj j P2 :=P0 xj j
Pi can besolved recursively
We can build abinary tree of subproblemswhoseleaves correspond topendingproblems still to be solved
Terminates as integer vars have finite bounds and, ateach split, range of one var becomes strictly smaller
IfLP(Pi)has optimal solution where integer variablestake integer values then solution is stored
IfLP(Pi)is infeasible then Pi can be discarded
(pruned, fathomed)
Session 6 p.9/40
8/10/2019 Mixed Integer Linear Programming
10/42
Example (1)
Max obj: x + y
Subject To
c1: -2 x + 2 y >= 1
c 2 : - 8 x + 1 0 y < = 1 3
End
====================================================================
Status: OPTIMAL
Objective: obj = 8.5 (MAXimum)
No. Column name St Activity Lower bound Upper bound
------ ------------ -- ------------- ------------- -------------
1 x B 4 0
2 y B 4.5 0
Session 6 p.10/40
8/10/2019 Mixed Integer Linear Programming
11/42
8/10/2019 Mixed Integer Linear Programming
12/42
Example (3)
Max obj: x + y
Subject To
c1: -2 x + 2 y >= 1
c 2 : - 8 x + 1 0 y < = 1 3
Bounds
y < = 4
End
====================================================================
Status: OPTIMAL
Objective: obj = 7.5 (MAXimum)
No. Column name St Activity Lower bound Upper bound
------ ------------ -- ------------- ------------- -------------1 x B 3.5 0
2 y NU 4 0 4
Session 6 p.12/40
8/10/2019 Mixed Integer Linear Programming
13/42
8/10/2019 Mixed Integer Linear Programming
14/42
Example (5)
Max obj: x + y
Subject To
c1: -2 x + 2 y >= 1
c 2 : - 8 x + 1 0 y < = 1 3
Bounds
x < = 3
y < = 4
End
====================================================================
Status: OPTIMAL
Objective: obj = 6.7 (MAXimum)
No. Column name St Activity Lower bound Upper bound------ ------------ -- ------------- ------------- -------------
1 x NU 3 0 3
2 y B 3.7 0 4
Session 6 p.14/40
8/10/2019 Mixed Integer Linear Programming
15/42
Example (6)
Max obj: x + y
Subject To
c1: -2 x + 2 y >= 1
c 2 : - 8 x + 1 0 y < = 1 3
Bounds
x < = 3
y = 4
End
====================================================================
GLPSOL: GLPK LP/MIP Solver 4.38
...
PROBLEM HAS NO PRIMAL FEASIBLE SOLUTION
Session 6 p.15/40
8/10/2019 Mixed Integer Linear Programming
16/42
Example (7)
Max obj: x + y
Subject To
c1: -2 x + 2 y >= 1
c 2 : - 8 x + 1 0 y < = 1 3
Bounds
x < = 3
y < = 3
End
====================================================================
Status: OPTIMAL
Objective: obj = 5.5 (MAXimum)
No. Column name St Activity Lower bound Upper bound------ ------------ -- ------------- ------------- -------------
1 x B 2.5 0 3
2 y NU 3 0 3
Session 6 p.16/40
8/10/2019 Mixed Integer Linear Programming
17/42
Example (8)
Max obj: x + y
Subject To
c1: -2 x + 2 y >= 1
c 2 : - 8 x + 1 0 y < = 1 3
Bounds
x = 3
y < = 3
End
====================================================================
GLPSOL: GLPK LP/MIP Solver 4.38
...
PROBLEM HAS NO PRIMAL FEASIBLE SOLUTION
Session 6 p.17/40
8/10/2019 Mixed Integer Linear Programming
18/42
Example (9)
Max obj: x + y
Subject To
c1: -2 x + 2 y >= 1
c 2 : - 8 x + 1 0 y < = 1 3
Bounds
x < = 2
y < = 3
End
====================================================================
Status: OPTIMAL
Objective: obj = 4.9 (MAXimum)
No. Column name St Activity Lower bound Upper bound------ ------------ -- ------------- ------------- -------------
1 x NU 2 0 2
2 y B 2.9 0 3
Session 6 p.18/40
8/10/2019 Mixed Integer Linear Programming
19/42
Example (10)
Max obj: x + y
Subject To
c1: -2 x + 2 y >= 1
c 2 : - 8 x + 1 0 y < = 1 3
Bounds
x < = 2
y = 3
End
====================================================================
GLPSOL: GLPK LP/MIP Solver 4.38
...
PROBLEM HAS NO PRIMAL FEASIBLE SOLUTION
Session 6 p.19/40
8/10/2019 Mixed Integer Linear Programming
20/42
Example (11)
Max obj: x + y
Subject To
c1: -2 x + 2 y >= 1
c 2 : - 8 x + 1 0 y < = 1 3
Bounds
x < = 2
y < = 2
End
====================================================================
Status: OPTIMAL
Objective: obj = 3.5 (MAXimum)
No. Column name St Activity Lower bound Upper bound------ ------------ -- ------------- ------------- -------------
1 x B 1.5 0 2
2 y NU 2 0 2
Session 6 p.20/40
8/10/2019 Mixed Integer Linear Programming
21/42
Example (12)
Max obj: x + y
Subject To
c1: -2 x + 2 y >= 1
c 2 : - 8 x + 1 0 y < = 1 3
Bounds
x = 2
y < = 2
End
====================================================================
GLPSOL: GLPK LP/MIP Solver 4.38
...
PROBLEM HAS NO PRIMAL FEASIBLE SOLUTION
Session 6 p.21/40
8/10/2019 Mixed Integer Linear Programming
22/42
Example (13)
Max obj: x + y
Subject To
c1: -2 x + 2 y >= 1
c 2 : - 8 x + 1 0 y < = 1 3
Bounds
x < = 1
y < = 2
End
====================================================================
Status: OPTIMAL
Objective: obj = 3 (MAXimum)
No. Column name St Activity Lower bound Upper bound------ ------------ -- ------------- ------------- -------------
1 x NU 1 0 1
2 y NU 2 0 2
Session 6 p.22/40
8/10/2019 Mixed Integer Linear Programming
23/42
Pruning in Branch & Bound
We have already seen that if relaxation is infeasible,the problem can be pruned
Now assume an (integral) solution has been foundIf solution has costZthen any pending problem Pjwhose relaxation has optimal value> Zcan be ignored
cost(Pj)cost(LP(Pj))> Z
The optimum will not be in any descendant of Pj!
Thispruningof the search tree has a huge impacton the efficiency of Branch & Bound
Session 6 p.23/40
8/10/2019 Mixed Integer Linear Programming
24/42
Unboundedness in Branch & Bound
We assumed integer variables are bounded
In mixed problems,
we allow non-integer variables to be unboundedAssumeLP(Pi)isunbounded. Then:
If in basic solution integer variables take integervalues then the problem is unbounded(assuming that problem data are rational numbers)
Else we proceed recursivelyas if an optimal solution toLP(Pi)had been found.
Whats different wrtLP(Pi)having optimal solution?
Session 6 p.24/40
8/10/2019 Mixed Integer Linear Programming
25/42
Unboundedness in Branch & Bound
We assumed integer variables are bounded
In mixed problems,
we allow non-integer variables to be unboundedAssumeLP(Pi)isunbounded. Then:
If in basic solution integer variables take integervalues then the problem is unbounded(assuming that problem data are rational numbers)
Else we proceed recursivelyas if an optimal solution toLP(Pi)had been found.
Whats different wrtLP(Pi)having optimal solution?IfLP(P
i)is unbounded then P
icannot be pruned:
no need to check this!Session 6 p.24/40
8/10/2019 Mixed Integer Linear Programming
26/42
Branch & Bound: Algorithmic Description
S:={P0
} /* set of pending problems */Z := + /* best cost found so far */
whileS=do
removeP fromS; solveLP(P)
ifLP(P)is feasiblethen
Letbe basic solution obtained after solving LP(P)
ifsatisfies integrality constraintsthen
ifis optimal forLP(P)then
ifcost()< Z thenstore; updateZ
else returnUNBOUNDED
else
ifis optimal forLP(P)Pcan be prunedthen continue
Letxj be integer variable such thatj Z
S:=S {P xj j, P xj j}returnZ
Session 6 p.25/40
8/10/2019 Mixed Integer Linear Programming
27/42
Heuristics in Branch & Bound
Possible choices in Branch & Bound
Choosing apending problem
Depth-firstsearchBreadth-firstsearchBest-firstsearch (select node with best cost value)
Choosing abranching variable
Thatclosest to halfway two integervaluesThat withleast costcoefficientThat which isimportant inthemodel(0-1 variable)That which is biggest in avariable ordering
No known strategy is best for all problems!Session 6 p.26/40
8/10/2019 Mixed Integer Linear Programming
28/42
Remarks on Branch & Bound
If integer variables not bounded, B&B may not terminate
min 0
13x 3y2
x, y Z
is infeasible but B&B loops forever looking for solutions!
New problems neednotbe solvedfrom scratchbutstarting from optimal solutionof parent problem
Dual Simplex Methodcan be used:dual feasibility preserved if change bounds of basic vars
Often Dual Simplex needs few iterations to obtain an
optimal solution to new problem(reoptimization)
Session 6 p.27/40
8/10/2019 Mixed Integer Linear Programming
29/42
Lower Bounding Procedures
Pruning at a node is achieved here bysolving the LP relaxation with dual simplex
But there exist other procedures for giving lower bounds
on best objective value:Lagrangian relaxation
(M IP)
min cTx
Axb
xu
xi Z i I
(LG)
min cTx+(Ax b)
xuxi Z i I
with0
LGis a relaxation: gives lower bound on M IPLGcan be trivially solved
For good , LGis as good as LPrelax. but cheaper
Concrete problems have ad-hoc lower bounding procs
Session 6 p.28/40
8/10/2019 Mixed Integer Linear Programming
30/42
C i Pl (2)
8/10/2019 Mixed Integer Linear Programming
31/42
Cutting Planes (2)
(1, 2)
y
xy0
max x+y
(4, 4.5)
2x + 2y1
x+ y6
8x + 10y13
x0
(0, 1)
Session 6 p.30/40
U i C f S l i MIP
8/10/2019 Mixed Integer Linear Programming
32/42
Using Cuts for Solving MIP
LetaTx bbe a cut. Then the MIP
min cTx
xS where S =
x Rn
Ax=b
aTx b
xu
xi Z i I
has thesame set of feasible solutions Sbut its LPrelaxation is strictly more constrained
Rather than splitting into subproblems as in Branch &Bound, one can add the cut and solve the relaxation
Used together with Branch & Bound: Branch & Cut
If after adding cuts no solution is found, then branch
Session 6 p.31/40
G C t (1)
8/10/2019 Mixed Integer Linear Programming
33/42
Gomory Cuts (1)
There are several techniques for deriving cuts
Some are problem-specific (e.g, travelling salesman)
Here we will see a generic technique: Gomory cuts
Let us consider a tableau with a row of the form
xi=i+
jR aijxj (i B)
Letbe an associated basic solution such that
1. i I
2. (xi) Z
3. For allj Rwe have (xj) =j or(xj) =uj
Can think that it is the optimal tableau of the relaxationSession 6 p.32/40
G C t (2)
8/10/2019 Mixed Integer Linear Programming
34/42
Gomory Cuts (2)
Let=(xi) (xi). Then0<
8/10/2019 Mixed Integer Linear Programming
35/42
Gomory Cuts (3)
xi=i+
jR aijxj
(xi) =i+jR aij(xj)Subtracting
xi (xi) = jR aij(xj (xj))=
jL aij(xj j)
jUaij(uj xj)
Finally
xi (xi)=+
jL aij(xj j)
jUaij(uj xj)
Let us define
L
+
={j L |aij 0} L
={j L |aij
8/10/2019 Mixed Integer Linear Programming
36/42
Gomory Cuts (4)
xi (xi)=+
jL aij(xj j)
jUaij(uj xj)
Assume
jLaij(xj j) jU
aij(uj xj)0. Then
+jL
aij(xj j) jU
aij(uj xj)1
jL+
aij(xj j) jU
aij(uj xj)1
jL+
aij
1 (xj j) +jU
aij1
(uj xj)1
Moreover
jLaij
(xj j) +
jU+
aij
(uj xj)0
Session 6 p.35/40
G C t (5)
8/10/2019 Mixed Integer Linear Programming
37/42
Gomory Cuts (5)
xi (xi)=+
jL aij(xj j)
jUaij(uj xj)
Assume
jLaij(xj j) jU
aij(uj xj)
8/10/2019 Mixed Integer Linear Programming
38/42
Gomory Cuts (6)
In any case
jL
aij
(xj j)+jU+
aij
(uj xj)+
jL+
aij
1 (xj j)+
jU
aij1 (uj xj)1
for any xS.However,does not satisfy this inequality
(setxj =j for j L, andxj =uj j U)
Session 6 p.37/40
E i All V ti A I t (1)
8/10/2019 Mixed Integer Linear Programming
39/42
Ensuring All Vertices Are Integer (1)
Let us assume A, bhave coefficients in ZSometimes it is possible to ensure for an IP thatall vertices of the relaxation are integer
For instance, when the matrix Aistotally unimodular:the determinant of every square submatrix is 0or1
Sufficient condition:property K
Each element of Ais0or1
No more than two non-zeros appear in each columm
Rows can be partitioned in two subsets R1 andR2 s.t.If a column contains two non-zeros of the samesign, one element is in each of the subsets
If a column contains two non-zeros of differentsigns, both elements belong to the same subset
Session 6 p.38/40
A i t P bl
8/10/2019 Mixed Integer Linear Programming
40/42
Assignment Problem
m= #of workers= #of tasksEach worker must be assigned to exactly one task
Each task is to be performed by exactly one worker
cij =cost when worker iperforms task j
Session 6 p.39/40
8/10/2019 Mixed Integer Linear Programming
41/42
Ensuring All Vertices Are Integer (2)
8/10/2019 Mixed Integer Linear Programming
42/42
Ensuring All Vertices Are Integer (2)
Several kinds of IPs satisfy property K:
Assignment
TransportationMaximum flow
Shortest path
...
Usually specialized network algorithms are moreefficient for these problems than simplex techniques
But simplex techniques are general and can be used ifno implementation of network algorithms is available