Polyhedral Approachto

Integer Linear Programming

Gerard Cornuejols

Tepper School of BusinessCarnegie Mellon University, Pittsburgh

1 / 30

Brief history

First Algorithms Polynomial Algorithms

Solving systems of linear equations

• Babylonians 1700BC• Gauss 1801

• Edmonds 1967

Solving systems of linear inequalities

• Fourier 1822• Dantzig 1951

• Khachyan 1979• Karmarkar 1984

Solving systems of linear inequalities in integers

• Gomory 1958 • Lenstra 1983

2 / 30

Mixed Integer Linear Programming

min cxx ∈ S

where S := {x ∈ Zp+ × Rn−p

+ : Ax ≥ b}

0 1 2x1

x2

Ax ≥ b

objective Linear Relaxation

min cxx ∈ P

where P := {x ∈ Rn+ : Ax ≥ b}

0

Ax ≥ b

1 2x1

x2objective

Branch-and-boundLand and Doig 1960

cut

0 1 2

Ax ≥ b

x1

x2objective

Cutting PlanesDantzig, Fulkerson and Johnson 1954

Gomory 1958

3 / 30

Polyhedral TheoryP := {x ∈ Rn

+ : Ax ≥ b} Polyhedron

S := P ∩ (Zp+ × Rn−p

+ ) Mixed Integer Linear Set

Conv S := {x ∈ Rn : ∃x1, . . . , xk ∈ S , λ ≥ 0,∑

λi = 1such that x = λ1x

1 + . . . + λkxk}THEOREM Meyer 1974If A, b have rational entries, then Conv S is a polyhedron.

Proof Using a theorem of Minkowski 1896 and Weyl 1935 :P is a polyhedron if and only if P = Q + C where Q is a polytopeand C is a polyhedral cone.

PConvS

S

P S ConvS

4 / 30

Thusmin cx

x ∈ S

0

Ax ≥ b

1 2x1

x2objective

can be rewritten as the LP

min cxx ∈ Conv S

0

Ax ≥ b

1 2x1

x2objective

We are interested in the constructive aspects of Conv S .

REMARK The number of constraints of Conv S can beexponential in the size of Ax ≥ b, BUT1) sometimes a partial representation of Conv S suffices(Example : Dantzig, Fulkerson, Johnson 1954) ;2) Conv S can sometimes be obtained as the projection of apolyhedron with a polynomial number of variables and constraints.

5 / 30

ProjectionsLet P := {(x , y) ∈ Rn × Rk : Ax + Gy ≥ b}DEFINITIONProjx(P) := {x ∈ Rn : ∃y ∈ Rk such that Ax + Gy ≥ b}

y

x

Projx(P)

P

THEOREMProjx(P) = {x ∈ Rn : vAx ≥ vb for all v ∈ Q}where Q := {v ∈ Rm : vG = 0, v ≥ 0}.

PROOFLet x ∈ Rn. Farkas’s lemma (Farkas 1894) implies thatGy ≥ b − Ax has a solution y if and only ifv(b − Ax) ≤ 0 for all v ≥ 0 such that vG = 0. ¥

6 / 30

Fractional Cuts Gomory 1958

Consider a single constraint : S := {x ∈ Zn+ :

∑nj=1 ajxj = b}.

Let b = bbc+ f0 where 0 < f0 < 1,and aj = bajc+ fj where 0 ≤ fj < 1.

THEOREM∑

j fjxj ≥ f0 is a valid inequality for S .

EQUIVALENT FORM∑

jbajcxj ≤ bbc.

APPLICATION

objective

x10

1

2

3 x2

max z = x1 + 2x2

−x1 + x2 ≤ 2x1 + x2 ≤ 5

x1 ∈ Z+

x2 ∈ R+

z + 0.5x3 + 1.5x4 = 8.5x1 − 0.5x3 + 0.5x4 = 1.5x2 + 0.5x3 + 0.5x4 = 3.5

Cutor

0.5x3 +0.5x4 ≥ 0.5

x2 ≤ 3.

7 / 30

Mixed Integer Cuts Gomory 1963Consider a single constraint : S := {x ∈ Zp

+ × Rn−p+ :

∑nj=1 ajxj = b}.

Let b = bbc+ f0 where 0 < f0 < 1,and aj = bajc+ fj where 0 ≤ fj < 1.

THEOREM

j≤p:∑

fj≤f0

fjf0

xj +

j≤p:∑

fj>f0

1− fj1− f0

xj +

j≥p+1:∑

aj>0

aj

f0xj −

j≥p+1:∑

aj<0

aj

1− f0xj ≥ 1

is a valid inequality for S .

NOTE The mixed integer cuts dominate the fractional cuts.

Experiments of Bonami and Minoux 2005 on MIPLIB 3 instancesgive the amount of duality gap = minx∈Scx −minx∈Pcx closed bystrengthening P with mixed integer cuts from the optimal basis :

gap closed : 24 %8 / 30

Yet, for thirty years, fractional cuts and mixed integer cuts werenot used in MILP solvers.

In 1991, Gomory remembered his experience with fractional cuts asfollows : In the summer of 1959, I joined IBM research and wasable to compute in earnest... We started to experience theunpredictability of the computational results rather steadily.

In 1991, Padberg and Rinaldi made the following comments :These cutting planes have poor convergence properties... classicalcutting planes furnish weak cuts... A marriage of classical cuttingplanes and tree search is out of the question as far as the solutionof large-scale combinatorial optimization problems is concerned.

9 / 30

In 1989, Nemhauser and Wolsey had this to say : They do notwork well in practice. They fail because an extremely large numberof these cuts frequently are required for convergence.

In 1985, Williams says : Although cutting plane methods mayappear mathematically elegant, they have not proved verysuccessful on large problems.

In 1988, Parker and Rardin give the following explanation for thislack of success : The main difficulty has come, not from thenumber of iterations, but from numerical errors in computerarithmetic.

10 / 30

GOMORY CLOSURE

Ax ≥ b x ∈ Zp+ × Rn−p

+

I Every valid inequality for P := {x ≥ 0 : Ax ≥ b} (6= ∅) is ofthe form uAx + vx ≥ ub − t, where u, v , t ≥ 0.

I Subtract a nonnegative surplus variable αx − s = β.

I Generate a Gomory inequality.

I Eliminate s = αx − β to get the inequality in the x-space.

I The convex set obtained by intersecting all these inequalitieswith P is called the Gomory closure.

THEOREM Cook, Kannan, Schrijver 1990

The Gomory closure is a polyhedron.

THEOREM Caprara, Letchford 2002 et Cornuejols, Li 2002

It is NP-hard to optimize a linear function over the Gomory closure.

11 / 30

Nevertheless,Balas and Saxena 2006 and Dash, Gunluck and Lodi 2007were able to optimize over the Gomory closure by solving asequence of parametric MILPs.

DUALITY GAP CLOSED BY DIFFERENT CUTSMIPLIB 3

Gomory cuts(optimal basis)

24 %

Gomory closure

80 %

12 / 30

Split Inequalities Cook-Kannan-Schrijver 1990

P := {x ∈ Rn : Ax ≥ b}S := P ∩ (Zp × Rn−p).

P

πx ≤ π0 πx ≥ π0 + 1

Π1 Π2

split inequality

For π ∈ Zn such that πp+1 = . . . = πn = 0 andπ0 ∈ Z, define

Π1:= P ∩ {x : πx ≤ π0}

Π2:= P ∩ {x : πx ≥ π0 + 1}

We call cx ≤ c0 a split inequality if there exists(π, π0) ∈ Zp × Z such that cx ≤ c0 is valid forΠ1 ∪ Π2.

The split closure is the intersection of all split inequalities.

THEOREM Nemhauser-Wolsey 1990, Cornuejols-Li 2002

The split closure is identical to the Gomory closure.

13 / 30

Chvatal Inequalities Chvatal 1973

A Chvatal inequality is a split inequality where Π2 = ∅.

P

πx ≤ π0 πx ≥ π0 + 1

Π1

The Chvatal closure is the intersection of all these inequalities.

REMARK Chvatal defined this concept in 1973 in the context ofpure integer programs.

The Chvatal closure reduces the duality gap by around 63 % onthe pure integer MIPLIB 03 instances (Fischetti-Lodi 2006) andaround 28 % on the mixed instances (Bonami-Cornuejols-Dash-Fischetti-Lodi 2007).

14 / 30

Lift-and-ProjectSherali-Adams 1990Lovasz-Schrijver 1991Balas-Ceria-Cornuejols 1993

LetS := {x ∈ {0, 1}p × Rn−p

+ : Ax ≥ b}P := {x ∈ Rn

+ : Ax ≥ b}LIFT-AND-PROJECT PROCEDURE

STEP 0 Choose an index j ∈ {1, . . . , p} .

STEP 1 Generate the nonlinear systemxj(Ax − b) ≥ 0

(1− xj)(Ax − b) ≥ 0

STEP 2 Linearize the system by substituting xixj by yi for i 6= j ,and x2

j by xj . Denote this polyhedron by Mj .

STEP 3 Project Mj on the x-space. Denote this polyhedron by Pj .

PROPOSITION Conv(S) ⊆ Pj ⊆ P.

15 / 30

THEOREM Pj = Conv{(

Ax ≥ bxj = 0

)∪

(Ax ≥ bxj = 1

)}.

0 1

Ax ≥ b

Pj

Ax ≥ bxj = 0

Ax ≥ bxj = 1 xj

THEOREM Balas 1979 Conv(S) = Pp(. . . P2(P1) . . .).

LIFT-AND-PROJECT CUTGiven a fractional solution x of the linear relaxation Ax ≥ b, find acutting plane αx ≥ β (namely αx < β) that is valid for Pj (andtherefore for S ).

DEEPEST CUTmax β − αx

αx ≥ β valid for Pj

16 / 30

CUT GENERATION LINEAR PROGRAM

Mj := {x ∈ Rn+, y ∈ Rn

+ :Ay − bxj ≥ 0,

Ax + bxj − Ay ≥ b,yj = xj}

The first two constraints come fromthe linearization in STEP 1.

In fact, one doesnot use thevariable yj

Mj := {x ∈ Rn+, y ∈ Rn−1

+ :Bjx + Ajy ≥ 0,Djx − Ajy ≥ b}

To project onto the x-space,we use the coneQ := {u, v ≥ 0 : uAj − vAj = 0}

Pj = {x ∈ Rn+ : (uBj + vDj)x ≥ vb

for all (u, v) ∈ Q}

DEEPEST CUT

max vb − (uBj + vDj)xuAj − vAj = 0u ≥ 0, v ≥ 0∑ui +

∑vi = 1

17 / 30

SIZE OF THE CUT GENERATION LP

max vb − (uBj + vDj)xuAj − vAj = 0∑ui +

∑vi = 1

u ≥ 0, v ≥ 0

Number of variables : 2mNumber of constraints : n + nonnegativity

Balas and Perregaard 2003 give aprecise correspondance between thebasic feasible solutions of the cutgeneration LP and the basic

solutions of the LPmin cxAx ≥ b

0 1

xj

Ax ≥ b

18 / 30

LIFT-AND-PROJECT CLOSURE OFP

F :=⋂p

j=1 Pj

0 1

1

F

P

x2

x1

P2

P1

REMARK Balas and Jeroslow 1980 show how to strengthencutting planes by using the integrality of the other integer variables(lift-and-project only considers the integrality of one xj at a time).

Experiments of Bonami and Minoux 2005 on MIPLIB03 instances :

Gap closed

Lift-and-project closure

37 %

Lift-and-project + strengthening

45 %19 / 30

Duality gap closed by different types of cutting planesMIPLIB 3 instances

Lift−and−project

Lift−and−project

45 %

24%37 %

~30 %

Reduce−and−split

MIRChvatal

80 %28%−63%

23%MIR heuristic

+ strengthening

Gomory from

the optimal basis

Gomory≡ Split

All these cuts are generated from integrality arguments applied toone linear equation. Can we generate deeper cuts by consideringseveral equations ?

20 / 30

Corner Polyhedron [Gomory 1969]Relax nonnegativity on basic variables xj .

In our current work [Basu, Bonami, Borozan, Conforti, Cornuejols,Margot, Zambelli 2009], we make a further relaxation :

Relax integrality on nonbasic variables.

x = f +∑k

j=1 r jsjx ∈ Zq

s ≥ 0

Example

f

r 1

r 2

Feasible set {(

x1

x2

)∈ Z2 :

(x1

x2

)= f + r1s1 + r2s2

where s1 ≥ 0, s2 ≥ 0}21 / 30

Intersection Cuts [Balas 1971]

Assume f 6∈ Zq. Want to cut off the basic solution s = 0, x = f .

f

r 1

r 2

S

Any convex set S with f ∈ int(S) with no integer point in int(S).

22 / 30

Intersection Cuts [Balas 1971]

Assume f 6∈ Zq. Want to cut off the basic solution s = 0, x = f .

f

r 1

r 2

S

intersection cut

Any convex set S with f ∈ int(S) with no integer point in int(S).Compute intersection of the rays with the boundary of S .Cut defined by these points is valid : ψ(r1)s1 + ψ(r2)s2 ≥ 1.

23 / 30

A Better Intersection Cut [Balas 1971]

f

r 1

r 2

S

Bigger convex set :

Octahedron f ∈ int(S) with no integral point in int(S).

24 / 30

A Better Intersection Cut [Balas 1971]

f

r 1

r 2

S

intersection cut

Bigger convex set :

Octahedron f ∈ int(S) with no integral point in int(S).

Better cut : ψ(r1)s1 + ψ(r2)s2 ≥ 1.

25 / 30

Maximal Lattice-Free Convex Sets in the Plane

Split, triangles and quadrilaterals

f

ff

generate split, triangle and quadrilateral inequalities∑

ψ(r)sr ≥ 1.

26 / 30

Split closure := the intersection of all split inequalities.Triangle closure := the intersection of all inequalities arising frommaximal lattice-free triangles.Quadrilateral closure := the intersection of all inequalities arisingfrom maximal lattice-free quadrilaterals.

Since all the facets of Integer Hull are induced by these threefamilies of maximal lattice-free convex sets (Andersen, Louveaux,Wolsey, Weismantel 2007), we have

Integer Hull = Split closure ∩ Triangle closure ∩ Quad closure

The split closure is a polyhedron (Cook, Kannan and Schrijver) butsuch a result is not known for the triangle closure and thequadrilateral closure.

THEOREM Basu, Bonami, Cornuejols, Margot 2008Triangle closure ⊆ Split closure andQuad closure ⊆ Split closure.

27 / 30

A theorem of Goemans 1995Let Q := {x : aix ≥ bi for i = 1, . . . , m} ⊆ Rn

+ \ {0}where ai ≥ 0 and bi ≥ 0 for all i .For α > 0 let αQ := {x : αaix ≥ bi for i = 1, . . . ,m}.

THEOREM If convex set P ⊆ Rn+ contains Q, then the smallest

value of α ≥ 1 such that P ⊆ αQ is

maxi=1,...,m

{bi

inf{aix : x ∈ P} : bi > 0

}.

In other words, the onlydirections that need to beconsidered to compute α arethose defined by the nontrivialfacets of Q.

P

Q

αQ

28 / 30

Relative strength of closuresBasu, Bonami, Cornuejols, Margot 2008

Both the triangle closure and the quadrilateral closure are goodapproximations of the integer hull :

THEOREMInteger Hull ⊆ Triangle closure ⊆ 2 (Integer Hull) andInteger Hull ⊆ Quad closure ⊆ 2 (Integer Hull)

We also show that the split closure may not be a goodapproximation of the integer hull :

THEOREMFor any α > 1, there is a choice of f , r1, . . . , rk such thatSplit closure 6⊆ α (Integer hull).

29 / 30

Papers available on http ://integer.tepper.cmu.edu/

30 / 30