Tutoriel du GDR RO - Laboratoire d'informatique de...

Tutoriel du GDR RO

Optimisation Combinatoire:resolution par approches polyedrales

et algorithmes de coupes

Pierre Fouilhoux

Laboratoire LIP6 - Universite Pierre et Marie Curie

ROADEF 2017, Metz

Tutoriel du GDR RO Optimisation Combinatoire: resolution par approches polyedrales et algorithmes de coupes

Combinatorial Optimization Problem

To find a greatest (smallest) element within a valuated finite set.

Given :- a finite subset of elements E = {e1, . . . , en}- a solution set F of subsets of E- a weight c = (c(e1), . . . , c(en))

a Combinatorial Optimization Problem is to find a solution

F ∈ F whose weight c(F ) =∑e∈F

c(e) is maximum (or min.),

i.e. max {c(F ) | F ∈ F}.

2/73


Combinatorial Optimization Problem

To find a greatest (smallest) element within a valuated finite set.

Given :- a finite subset of elements E = {e1, . . . , en}- a solution set F of subsets of E- a weight c = (c(e1), . . . , c(en))

a Combinatorial Optimization Problem is to find a solution

F ∈ F whose weight c(F ) =∑e∈F

c(e) is maximum (or min.),

i.e. max {c(F ) | F ∈ F}.

2/73


“Natural” MIP formulation

Binary variable xe =

{1 if e is chosen0 otherwise

for every e ∈ E .

max∑e∈E

c(e)xe

Ax ≤ b

xe ∈ {0, 1} ∀e ∈ E .

We will suppose here that :- Ax ≤ b is known- the linear relaxation of this MIP can be obtained

Important remark : Ax ≤ b can be non-compact,i.e. can contain an exponential number of inequalities !

3/73





for every e ∈ E .

max∑e∈E

c(e)xe

Ax ≤ b

xe ∈ {0, 1} ∀e ∈ E .



3/73





for every e ∈ E .

max∑e∈E

c(e)xe

Ax ≤ b

xe ∈ {0, 1} ∀e ∈ E .



3/73


1. Compact / non-compact formulations

2. Cutting Plane based algorithm

3. Branch&Cut algorithm

4. Polyhedral approach

5. Polyedral approach for the TSP

4/73


Compact / non-compact formulations






5/73



The Acyclic Induced subgraph problem

Let G = (V ,A) be a directed graphwith n = |V | nodes and m = |A| arcs.

A circuit of G : a sequence of arcs

C = (i1i2, i2i3, ..., ik−1i1)

Notation :V (C ) is the set of nodes of C .

1 5

4

3

2

A graph is acyclic if it contains no circuit.

6/73




Let G = (V ,A) be a directed graphwith n = |V | nodes and m = |A| arcs.

A circuit of G : a sequence of arcs

C = (i1i2, i2i3, ..., ik−1i1)

Notation :V (C ) is the set of nodes of C .

1 5

4

3

2

A graph is acyclic if it contains no circuit.

6/73




Given a node subset W ⊂ V ,

A(W ) : arcs with both endnodes in W

(W ,A(W )) : subgraph induced by W

7/73







7/73







7/73







The Acyclic Induced Subgraph Problem (AISP) is to finda node subset W inducing an acyclic subgraph with |W | maximum

7/73







The Acyclic Induced Subgraph Problem (AISP) is to finda node subset W inducing an acyclic subgraph with |W | maximumora node subset W ′ “breaking” every circuit of G with |W ′| min.

7/73




• The AISP is NP-hard.

Indeed :given a non-directed graph Gconstruct a directed graph G ′ by replacing one edge by two arcs

then finding an acyclic subgraph in G ′ is as hard as finding a maximum stable set in G .

• The AISP is polynomial for graphs of maximum degree 3 [Baiou,Barahona]

8/73



A compact formulation

Two types of variables :

- Binary variables xi =

{1 if node i is chosen0 otherwise

∀i ∈ V .

- Continuous variables ui ∀i ∈ V .

A Miller-Tucker-Zemlin (MTZ) formulation :

(FMTZ )

max∑i∈V

xi

ui − uj + 1 ≤ n(2− xi − xj ) ∀ij ∈ A1 ≤ ui ≤ n ∀i ∈ Vui ∈ IR ∀i ∈ V

xi ∈ {0, 1} ∀i ∈ V

9/73



A compact formulation

• The MTZ formulation is equivalent to the AISP.

Indeed :Given a solution (x , u) of (FMTZ ), let W = {i ∈ V | xi = 1}.If W induces a circuit C : ui + 1 ≤ uj ∀ij ∈ C , a contradiction.

• (FMTZ ) is compact- n binary variables and n continuous variables- m inequalities

Let’s go with Cplex !

10/73



A non-compact formulation

Same binary variables xi =

{1 if node i is chosen0 otherwise

∀i ∈ V .

(FC )

max

∑i∈V

xi∑i∈V (C)

xi ≤ |C | − 1 ∀C circuit of G

xi ∈ {0, 1} ∀i ∈ V

These inequalities are called the circuit inequalities.

Formulation (FC ) is clearly equivalent to the AISP.

11/73



A non-compact formulation

Circuit inequalities are in exponential number with respect to thenumber of nodes.

Formulation (FC ) cannot be directly used as compact formulationin Cplex :

• Is this formulation really better than a compact one ?

• How to use such a non-compact formulation ?

12/73


Cutting Plane based algorithm






13/73



For an integer linear formulation

(F )

max cT x

Ax ≤ bx ∈ ZZ

with n variableswith a exponential number of inequalities with respect to n.

How can we solve the linear relaxation (F ) of (F ) ?

14/73



For an integer linear formulation

(F )

max cT x

Ax ≤ bx ∈ ZZ

(F )

max cT x

Ax ≤ b

��x ∈ ZZ

with n variableswith a exponential number of inequalities with respect to n.

How can we solve the linear relaxation (F ) of (F ) ?

14/73



Polyhedron

A hyperplane of IRn is the set of points x ∈ IRn satisfying a linear equality ax = α.

A halfspace of IRn is the set of points x ∈ IRn satisfying a linear equality ax ≤ α.

A polyhedron P is the intersection of a finitenumber of halfspacesi.e.

P = {x ∈ IRn | Ax ≤ b}

with Ax ≤ b system of linear inequalities.

Such a system Ax ≤ b characterizes apolyhedron.

A polytope is a bounded polyhedron.

15/73



Extreme point

• An extreme point (or vertex) of a polytope P is a point x ∈ Ps.t. there is no solutions x1 6= x2 in P with x = 1

2x1 + 1

2x2.

• Solving a (bounded) linear formulation

(F )

{max cT x

Ax ≤ b

reduces to find an optimal extreme point of polytope

P = {x ∈ IRn | Ax ≤ b}

16/73



InitialisationLet A0x ≤ b0 be a subset of inequalities of Ax ≤ band (F 0) the linear program restricted to A0x ≤ b0.Let x0 the solution of (F 0).

(F 0)

{max cT xA0x ≤ b0

x0

Solution x0 is an extreme point of the polytope characterized by A0x ≤ b0.

Note that :if x0 satisfies every inequality of Ax ≤ bx0 is be an extreme point of the polytope characterized by Ax ≤ band x0 will an optimal solution !

17/73



InitialisationLet A0x ≤ b0 be a subset of inequalities of Ax ≤ band (F 0) the linear program restricted to A0x ≤ b0.Let x0 the solution of (F 0).

(F 0)

{max cT xA0x ≤ b0

x0

Solution x0 is an extreme point of the polytope characterized by A0x ≤ b0.

Note that :if x0 satisfies every inequality of Ax ≤ bx0 is be an extreme point of the polytope characterized by Ax ≤ band x0 will an optimal solution !

17/73



Separation problem

Definition (Separation problem)

Given a point x ∈ IRn,the separation problem associated to Ax ≤ b and x is- to determine whether x satisfies every inequality of Ax ≤ b- or to produce an inequality ax ≤ α of Ax ≤ b violated by x .

An inequality ax ≤ α of Ax ≤ b is violated by x0 if ax0 > α.

18/73



Iteration

If the separation problem for x0 produces an inequality a′x ≤ βviolated by x0

(F1)

max cT xA0x ≤ b0

a′x ≤ β

x1

a′x ≤ β

And so on !

19/73



Iteration

If the separation problem for x0 produces an inequality a′x ≤ βviolated by x0

(F1)

max cT xA0x ≤ b0

a′x ≤ β

x1

a′x ≤ β

And so on !

19/73



Cutting plane based algorithm

Definition (Cutting-plane based “method”)

While there exists an inequality ax ≤ α of Ax ≤ b violated by x i

(Fi+1)← (Fi ) + ax ≤ αSolve the linear program Fi+1

xi+1 ← solution of Fi+1

i ← i + 1

• An algorithm which solves the separation problem is called aseparation algorithm (the whole method is to reiterate theseparation algorithm).

• An inequality which is violated a solution x i is called a cutbecause the inequality separates an useless part of IRn from thepolytope characterized by Ax ≤ b.

20/73



Validity of the method

• EndingIn the worst case, the algorithm enumerates every inequality ofAx ≤ b.

• ValidityAt the end of the loop, the final solution x∗ is an extreme point ofthe whole system Ax ≤ b and maximizes cT x :then x∗ is an optimal solution of (F ).

21/73



Complexity

Theorem (Grotschel, Lovasz, Schrijver, 1981)

A cutting plane based method of rational system Ax ≤ b is polynomialif and only ifthe separation algorithm associated to Ax ≤ b is polynomial.

This fundamental results is :

Optimize ⇔ Separate

22/73



Separation algorithm for the circuit inequalities

Theorem

The circuit inequalities∑i∈V (C)

xi ≤ |C | − 1 ∀ circuit C

can be separated in polynomial time.

Note that, by setting x ′ = 1− x ,a circuit inequality can be rewritten∑

i∈V (C)

x ′i ≥ 1

23/73



Separation algorithm for the circuit inequalities

Given a point x ∈ [0, 1]n

The separation problem for the circuit inequalities is to determinewhether or not there exists a circuit inequality violated by x(and in the last case, to produce one violated inequality).

Set x ′ = 1− x .Find a circuit C of minimal weight with respect to x ′.(This can be done in polynomial time since x ′ ≥ 0)

• If∑

i∈V (C)

x ′i < 1 :∑

i∈V (C)

xi ≤ |C | − 1 is violated by x .

• If∑

i∈V (C)

x ′i ≥ 1 : there is no circuit ineq. violated by x . �

24/73



Branch-and-Cut algorithm

The cutting plane based method gives in polynomial time therelaxation value of an integer formulation.

At the end, the solution is not integer (unless if P = NP).

The only known general framewok to solve integer problemis... Branch&Bound !

A Branch&Bound that uses a cutting plane based algorithm inevery node of the Branch&Bound treeis a Branch&Cut algorithm.

25/73



Branch-and-Cut algorithm

The cutting plane based method gives in polynomial time therelaxation value of an integer formulation.

At the end, the solution is not integer (unless if P = NP).

The only known general framewok to solve integer problemis... Branch&Bound !

A Branch&Bound that uses a cutting plane based algorithm inevery node of the Branch&Bound treeis a Branch&Cut algorithm.

25/73


Branch&Cut algorithm



3. Branch&Cut algorithm3.1 Branch&Bound3.2 Relaxation value3.3 Rounding heuristic



26/73



Branch&Bound






27/73



Branch&Bound

A Branch&Bound algorithm is defined by :- an integer linear program- a branching strategy (potentially on inequalities)→ B&B node = initial integer program + branching inequalities

23,1

19,7 20,2

18,6

18,0 18,7 18,2 18,1 18.4

18,7 19,6 19

x3 = 1x3 = 0

x6 = 1x6 = 1x7 = 1x7 = 0 x3 = 0

x6 = 1 x2 = 0 x2 = 1x6 = 0

v = linear relaxation of a B&B nodeFor a “max” problem, v is is an upper bound for the sub-tree

(each node only contains solutions of values at most v).

28/73



Branch&Bound


23,1

19,7 20,2

18,6

18,0 18,7 18,2 18,1 18.4

18,7 19,6 19

x3 = 1x3 = 0

x6 = 1x6 = 1x7 = 1x7 = 0 x3 = 0

x6 = 1 x2 = 0 x2 = 1x6 = 0


(each node only contains solutions of values at most v).

28/73



Branch&Bound


23,1

19,7 20,2

18,6

18,0 18,7 18,2 18,1 18.4

18,7 19,6 19

x3 = 1x3 = 0

x6 = 1x6 = 1x7 = 1x7 = 0 x3 = 0

x6 = 1 x2 = 0 x2 = 1x6 = 0


(each node only contains solutions of values at most v).28/73



Branch&Bound

During the B&B algorithm, a best known solution can beobtained using heuristic, metaheuristic, rounding heurisitic,...

A sub-tree can be pruned if either

• the relaxation value of its root node

is ≤ than the value of the best known solution.

• its root node is integer (its relaxation gives an integer solution)

23,1

19,7 20,2

18,6

18,0 18,7 18,2 18,1 18.4

18,7 19,6 19

x3 = 1x3 = 0

x6 = 1x6 = 1x7 = 1x7 = 0 x3 = 0

x6 = 1 x2 = 0 x2 = 1x6 = 0

29/73



Branch&Bound

During the B&B algorithm, a best known solution can beobtained using heuristic, metaheuristic, rounding heurisitic,...

A sub-tree can be pruned if either

• the relaxation value of its root node

is ≤ than the value of the best known solution.

• its root node is integer (its relaxation gives an integer solution)

23,1

19,7 20,2

18,6

18,0 18,7 18,2 18,1 18.4

18,7 19,6 19

x3 = 1x3 = 0

x6 = 1x6 = 1x7 = 1x7 = 0 x3 = 0

x6 = 1 x2 = 0 x2 = 1x6 = 0

29/73



Branch&Bound

Efficiency of a B&B algorithm

Several aspects drive to an efficient B&B algorithm :

• Initial preprocessing step

• Breaking symmetries within the B&B tree

• Having the lowest possible upper bound

• Having the highest possible lower bound

• Having a lot of integer nodes

30/73



Branch&Bound






→ Having the best possible relaxation value



30/73



Branch&Bound








→ Having a good rounding heuristic


30/73



Branch&Bound








→ Having a good rounding heuristic


→ If the relaxed linear programs have “often” integer solutions

30/73



Relaxation value






31/73



Relaxation value

Formulation comparison

There is no generic method to compare the relaxation values of two formulations.

TheoremThe relaxation value of the MTZ formulation (FMTZ )≤ the relaxation value of the circuit formulation (FC ).

Sketch of the proof.First note that by summing MTZ inequalities over a circuit C , we obtain

∑i∈V (C)

xi ≤(

1−1

2n

)|C |

Let x be a solution of the relaxation (FC ), then x satisfies

∑i∈V (C)

xi ≤ |C | − 1 ≤(

1−1

2n

)|C |

By setting appropriate values ui , we get a solution (x , u) of (FMTZ ). �

32/73



Relaxation value

Formulation comparison

There is no generic method to compare the relaxation values of two formulations.

TheoremThe relaxation value of the MTZ formulation (FMTZ )≤ the relaxation value of the circuit formulation (FC ).

Sketch of the proof.First note that by summing MTZ inequalities over a circuit C , we obtain

∑i∈V (C)

xi ≤(

1−1

2n

)|C |

Let x be a solution of the relaxation (FC ), then x satisfies

∑i∈V (C)

xi ≤ |C | − 1 ≤(

1−1

2n

)|C |

By setting appropriate values ui , we get a solution (x , u) of (FMTZ ). �32/73



Relaxation value

Reinforcement

Given an integer formulation (F )

max cT x

Ax ≤ bx ∈ ZZ

Definition

An inequality ax ≤ α is valid for an integer formulation (F )if every integer solution of (F ) satisfies ax ≤ α.

Adding a valid inequality to formulation (F )does not change the solution space of (F ).

But valid inequalities can improve the relaxation value !

33/73



Relaxation value

Obtaining valid inequalities

• Summing inequalities : Chvatal-Gomory rounding method :

• Multiplying inequalities : lift-and-project Lovasz-Shrivjer meth.

• Finding particular sub-structures (stable set, knapsack,...)

• Lifting coefficients of inequalities (to obtain stronger ones)

• Disjunctive cuts, local branching inequalities,...

• ... and many others techniques ...

34/73



Relaxation value

Clique inequalities

In a directed graph, a clique is a subset K of nodes inducing acomplete subgraph.

The clique inequalities∑i∈K

xi ≤ 1 pour every clique K of G

are valid for the formulation.3

4

2

1

Indeed, at most one node can be taken among a clique.

35/73



Relaxation value

Clique inequalities

Lemma

The separation problem for the clique inequalities is NP-complete.

Indeed, proving that there is no violated clique inequality is equivalent to find a

maximal weighted clique in G, which is a famous NP-hard problem �

• However, a heuristic separation algorithm can be used !

For instance, a simple but efficient greedy heuristic :

Given a linear relaxation value x∗ :

- sort the nodes with respect to decreasing values x∗i- K ← ∅- iteratively try to add a node in K such that K stays a clique

36/73



Relaxation value

Clique inequalities

Lemma

The separation problem for the clique inequalities is NP-complete.

Indeed, proving that there is no violated clique inequality is equivalent to find a

maximal weighted clique in G, which is a famous NP-hard problem �

• However, a heuristic separation algorithm can be used !

For instance, a simple but efficient greedy heuristic :


- sort the nodes with respect to decreasing values x∗i- K ← ∅- iteratively try to add a node in K such that K stays a clique

36/73



Relaxation value

Chvatal sum techniques

For a “directed cycle” D like this one.

For each consecutive nodes i and i + 1there is a circuit inequality

xi + xi+1 ≤ 11 5

4

3

2

By summing these inequalities :

x1 + x2 ≤ 1x2 + x3 ≤ 1

....x1 + x|D| ≤ 1

∑i∈V (C)

xi ≤|D|2

37/73



Relaxation value

Chvatal sum techniques

For a “directed cycle” D like this one.

For each consecutive nodes i and i + 1there is a circuit inequality

xi + xi+1 ≤ 11 5

4

3

2

By summing these inequalities :

x1 + x2 ≤ 1x2 + x3 ≤ 1

....x1 + x|D| ≤ 1

∑i∈V (C)

xi ≤|D|2

37/73



Relaxation value

odd cycle inequalities

The left part is integer, let’s round it down

∑i∈V (C)

xi ≤|D|2

38/73



Relaxation value



∑i∈V (C)

xi ≤⌊|D|2

⌋

38/73



Relaxation value



∑i∈V (C)

xi ≤⌊|D|2

⌋

I If |D| is even, nothing new is obtained.

I If |D| is odd, we obtain a new valid inequality

∑i∈V (C)

xi ≤|D| − 1

2

Such odd cycle inequalities can be found in the stable set polytope.

They can be separated in polynomial time.

38/73



Rounding heuristic






39/73



Rounding heuristic

Primal heuristic

• A primal heuristic (often called rounding heuristic)is to use a fractional relaxation value to produce a solution of the integer formulation.

For instance for the AISP, a very simple greedy algorithm :


- sort the nodes with respect to decreasing values x∗i- S ← ∅- iteratively try to add a node in S while S induces no circuit

• In practice :Use a very quick primal heuristic at every iteration of every cutting plane basedalgorithm of every node of the branching tree !

→ Among all the solutions produced by this algorithm,there is often an optimal solution !

40/73



Rounding heuristic

Primal heuristic

• A primal heuristic (often called rounding heuristic)is to use a fractional relaxation value to produce a solution of the integer formulation.

For instance for the AISP, a very simple greedy algorithm :


- sort the nodes with respect to decreasing values x∗i- S ← ∅- iteratively try to add a node in S while S induces no circuit

• In practice :Use a very quick primal heuristic at every iteration of every cutting plane basedalgorithm of every node of the branching tree !

→ Among all the solutions produced by this algorithm,there is often an optimal solution !

40/73



Rounding heuristic

Experimental guarantee

zopt Optimal solution of the integer formulation

zH0meta-heuristic launched before B&C algorithm

zbs0upper bound from linear relaxation at the root node

zbestsol best known solution (by primal heuristic)

zbsg best known upper bound (when the B&C is stopped)

zopt zbs0zH0 zbestsol zbsgz

Max

gap

where gap =zbsg − zbestsol

zbestsol.

In the worst case, zbestsol will be at gap% from the optimum.

41/73



Rounding heuristic

What is inside a MIP solver ?

A MIP solver like CPLEX, GUROBI, XPRESS, SCIP,... are “AutomaticBranch-and-Cut process.

• Strong preprocessing phase

• Automatic use of generic valid inequalities through efficient cutting plane basedmethods

• Automatic lifting operations to reinforce known inequalities and produce nexones

• Automatic logical inference to break the symmetry of the branching tree

• Generic rounding heuristics

And it is more and more easy to add your own valid inequalities to these frameworks !

42/73


Polyhedral approach




4. Polyhedral approach4.1 Dimension and facet4.2 Characterization


43/73


Polyhedral approach

What are the “best” valid inequalities ?

Given the variable set of a formulation,

• What are the “best” valid inequalities ?

• How to have “often” integer Branch&Bound nodes ?

...

• Can we know when a linear formulation produces integersolutions ?

44/73


Polyhedral approach

What are the “best” valid inequalities ?

Given the variable set of a formulation,

• What are the “best” valid inequalities ?

• How to have “often” integer Branch&Bound nodes ?

...

• Can we know when a linear formulation produces integersolutions ?

44/73


Polyhedral approach

Integer polytope


(F )

{max cT x

Ax ≤ b

reduces to find an optimal extreme pointof polytope P = {x ∈ IRn | Ax ≤ b}

• An integer polytope is a polytope with integerextreme points.

• A rational polytope P is integer ⇔ ∀c ∈ ZZ n, max{cT x | x ∈ P} is integer.

45/73


Polyhedral approach

Integer polytope


(F )

{max cT x

Ax ≤ b




45/73


Polyhedral approach

Integer polytope


(F )

{max cT x

Ax ≤ b




45/73


Polyhedral approach

A 2-dimensional example

(F )

max z = 2x1 + x2

x1 − 4x2 ≤ 03x1 + 4x2 ≤ 15x1 ≥ 0x2 ≥ 0x1, x2 ∈ IN

4

3

2

1

0 1 2 3 4 5

x2

x1

x1 − 4x2 = 0

3x1 + 4x2 = 15

xopt = (3, 1)

xopt = ( 154, 15

16)

xopt optimal integer solution of (F )zopt optimal integer value : 7xopt optimal fractional solution of linear relaxation (F )z∗opt optimal fractional value : 8 + 7

16

Note that xopt is the (only) optimal extreme point of F .

46/73


Polyhedral approach


(F )

max z = 2x1 + x2

x1 − 4x2 ≤ 03x1 + 4x2 ≤ 15x1 ≥ 0x2 ≥ 0x1, x2 ∈ IN

4

3

2

1

0 1 2 3 4 5

x2

x1

x1 − 4x2 = 0

3x1 + 4x2 = 15

xopt = (3, 1)

xopt = ( 154, 15

16)

xopt optimal integer solution of (F )zopt optimal integer value : 7xopt optimal fractional solution of linear relaxation (F )z∗opt optimal fractional value : 8 + 7

16

Note that xopt is the (only) optimal extreme point of F .46/73


Polyhedral approach

A 2-dimensional exampleRemark : the integer solutions of (F ) are the solutions of (F ).

(F )

max z = 2x1 + x2

x1 − 4x2 ≤ 03x1 + 4x2 ≤ 15x1 ≥ 0x2 ≥ 0x1, x2 ∈ IN

4

3

2

1

0 1 2 3 4 5

x2

x1

xopt = (3, 1)

Let’s take an elastic and wrap it around these integer points...

We obtain a new polytope ! (the convex hull of the integer points)And this polytope is integer by construction.

47/73


Polyhedral approach


(F )

max z = 2x1 + x2

x1 − 4x2 ≤ 03x1 + 4x2 ≤ 15x1 ≥ 0x2 ≥ 0x1, x2 ∈ IN

4

3

2

1

0 1 2 3 4 5

x2

x1

xopt = (3, 1)



47/73


Polyhedral approach


(F )

max z = 2x1 + x2

x1 − 4x2 ≤ 03x1 + 4x2 ≤ 15x1 ≥ 0x2 ≥ 0x1, x2 ∈ IN

4

3

2

1

0 1 2 3 4 5

x2

x1

xopt = (3, 1)



47/73


Polyhedral approach


(F )

max z = 2x1 + x2

x1 − 4x2 ≤ 03x1 + 4x2 ≤ 15x1 ≥ 0x2 ≥ 0x1, x2 ∈ IN

4

3

2

1

0 1 2 3 4 5

x2

x1

xopt = (3, 1)


We obtain a new polytope ! (the convex hull of the integer points)

And this polytope is integer by construction.

47/73


Polyhedral approach


(F )

max z = 2x1 + x2

x1 − 4x2 ≤ 03x1 + 4x2 ≤ 15x1 ≥ 0x2 ≥ 0x1, x2 ∈ IN

4

3

2

1

0 1 2 3 4 5

x2

x1

xopt = (3, 1)



47/73


Polyhedral approach

Convex hullGiven a set S of points of IRn.the convex hull of S , denoted by conv(S)is the smallest convex set containing S .

Theorem (of Minkowski)

A set P ⊆ IRn is a polytopeif and only if there exists a set S of points such that P = conv(S).

Consequently :I conv(S) is a polytopeI there exists a finite subset of inequalities Dx ≤ β such that

conv(S) = {x ∈ IRn | Dx ≤ β}I max{cχ | χ ∈ conv(S)} is a linear program

48/73


Polyhedral approach

Convex hullGiven a set S of points of IRn.the convex hull of S , denoted by conv(S)is the smallest convex set containing S .

Theorem (of Minkowski)

A set P ⊆ IRn is a polytopeif and only if there exists a set S of points such that P = conv(S).

Consequently :I conv(S) is a polytopeI there exists a finite subset of inequalities Dx ≤ β such that

conv(S) = {x ∈ IRn | Dx ≤ β}I max{cχ | χ ∈ conv(S)} is a linear program

48/73


Polyhedral approach

Combinatorial polytope

Let P be a combinatorial optimization problem :- over n decisions corresponding to n integer variables.- with a function cost c.

Let S the set of the incidence vectors of the solutions of P.

Problem P ismax {cχ | χ ∈ S}

Let us consider the linear program

max {cχ | χ ∈ conv(S)}

49/73


Polyhedral approach


Let us consider the AISP on a triangle

1

2

3

The solutions are

∅ {1} {2} {3} {1, 2} {1, 3} {2, 3}

And their incidence vectors are the following points

χ∅ = [ 0 0 0 ]

χ{1} = [ 1 0 0 ]

χ{2} = [ 0 1 0 ]

χ{3} = [ 0 0 1 ]

χ{1,2} = [ 1 1 0 ]

χ{1,3} = [ 1 0 1 ]

χ{2,3} = [ 0 1 1 ]

However point

χ{1,2,3} = [ 1 1 1 ]

does not correspond to a solution

50/73


Polyhedral approach



1

2

3

The solutions are

∅ {1} {2} {3} {1, 2} {1, 3} {2, 3}


χ∅ = [ 0 0 0 ]

χ{1} = [ 1 0 0 ]

χ{2} = [ 0 1 0 ]

χ{3} = [ 0 0 1 ]

χ{1,2} = [ 1 1 0 ]

χ{1,3} = [ 1 0 1 ]

χ{2,3} = [ 0 1 1 ]

However point

χ{1,2,3} = [ 1 1 1 ]


50/73


Polyhedral approach



1

2

3

The solutions are

∅ {1} {2} {3} {1, 2} {1, 3} {2, 3}


χ∅ = [ 0 0 0 ]

χ{1} = [ 1 0 0 ]

χ{2} = [ 0 1 0 ]

χ{3} = [ 0 0 1 ]

χ{1,2} = [ 1 1 0 ]

χ{1,3} = [ 1 0 1 ]

χ{2,3} = [ 0 1 1 ]

However point

χ{1,2,3} = [ 1 1 1 ]


50/73


Polyhedral approach


What is the convex hull of these 7 points ?

This convex hull is inclu-ded into the hypercube(of dimension 3)

The hypercube is charac-terized by

x1 ≤ 1

x2 ≤ 1

x3 ≤ 1

x1 ≥ 0

x2 ≥ 0

x3 ≥ 0

51/73


Polyhedral approach



This convex hull is inclu-ded into the hypercube(of dimension 3)

The hypercube is charac-terized by

x1 ≤ 1

x2 ≤ 1

x3 ≤ 1

x1 ≥ 0

x2 ≥ 0

x3 ≥ 0

51/73


Polyhedral approach



x1 + x2 + x3 ≤ 2

52/73


Polyhedral approach



The convex hull of these 7points is characterized by

x1 + x2 + x3 ≤ 2

x1 ≤ 1

x2 ≤ 1

x3 ≤ 1

x1 ≥ 0

x2 ≥ 0

x3 ≥ 0

53/73


Polyhedral approach


Let P be a combinatorial optimization problem :- over n decisions corresponding to n integer variables.- with a function cost c.Let S the set of the incidence vectors of the solutions of P.


TheoremThe linear program


is equivalent to problem P

Indeed

- Every extreme points of the convex hull conv(S) are integer by construction.

- The optimal points of S are among the extreme points of polytope conv(S).

54/73


Polyhedral approach


Let P be a combinatorial optimization problem :- over n decisions corresponding to n integer variables.- with a function cost c.Let S the set of the incidence vectors of the solutions of P.


TheoremThe linear program


is equivalent to problem P

Indeed

- Every extreme points of the convex hull conv(S) are integer by construction.

- The optimal points of S are among the extreme points of polytope conv(S).

54/73


Polyhedral approach

Get around the combinatorial explosionOptimizing (P) reduces to optimizing a linear program on conv(S).

The convex hull is then the “unknown value” to found.

(F )

max cxAx ≤ bαx ≤ βx integer

x1

αx ≥ β

55/73


Polyhedral approach

Get around the combinatorial explosion

Optimizing (P) reduces to optimizing a linear program on conv(S).


(F )

max cxx ∈ conv(S)x continuous

which is a linear program !

x1

55/73


Polyhedral approach

Get around the combinatorial explosionOptimizing (P) reduces to optimizing a linear program on conv(S).


(F )

max cxx ∈ conv(S)x continuous

which is a linear program !

x1

Unfortunately we cannot use this process in polynomial time...Unless P=NP, finding the convex hull of a combinatorial polytope is NP-hard !

But even a “partial” knowledge of polytope conv(S) is very useful

55/73


Polyhedral approach

Dimension and facet






56/73


Polyhedral approach

Dimension and facet

Example : The acyclic induced subgraph polytopeGiven a directed graph G = (V ,A),acycl(G) : family of all node subsets inducing an acyclic subgraph of G .

Then acycl(G) is the solutions set of the AISP on G (whatever will be the costs)

Given a solution W ∈ acycl(G), the incidence vector χW is

χW [i ] =

{1 if i ∈W0 otherwise

.Some solutions :• Pour ∅ ∈ acycl(G)

χ∅ = [ 0 0 0 0 0 ]

• {i} ∈ acycl(G) ∀i ∈ V

χ{i} = [ 0 0 1 0 0 ]i

P(G) : the acyclic induced subgraph polytope of Gi.e. is the convex hull of the incidence vectors of the solutions

i.e. P(G) = conv{χW | W ∈ acycl(G)}.

57/73


Polyhedral approach

Dimension and facet




χW [i ] =



χ∅ = [ 0 0 0 0 0 ]

• {i} ∈ acycl(G) ∀i ∈ V

χ{i} = [ 0 0 1 0 0 ]i


i.e. P(G) = conv{χW | W ∈ acycl(G)}.

57/73


Polyhedral approach

Dimension and facet




χW [i ] =



χ∅ = [ 0 0 0 0 0 ]

• {i} ∈ acycl(G) ∀i ∈ V

χ{i} = [ 0 0 1 0 0 ]i


i.e. P(G) = conv{χW | W ∈ acycl(G)}.57/73


Polyhedral approach

Dimension and facet

Dimension

A combinatorial polytope P of n variables is in IRn...Can P be included into a smaller space ?

Definition• A set of points x1, ..., xk ∈ IRn are affinely independent if vectorsx2 − x1, ..., xk − x1 are linearly independent.

• A polytope P in IRn is of dimension d (denoted dim(P) = d)if P contains at least d + 1 affinely independent points.

A polytope P is said to be full dimensional if dim(P) = n.

Examples :- a plane in 3D-space is not full dimensional.

- an hypercube [0, 1]n is full dimensional in IRn but not in IRn′ if n′ < n

58/73


Polyhedral approach

Dimension and facet

Dimension







58/73


Polyhedral approach

Dimension and facet

Dimension







58/73


Polyhedral approach

Dimension and facet

Example : The acyclic induced subgraph polytope

LemmaThe AIS polytope P(G) is full-dimensional for every graph G.

Proof.It is sufficient to produce n + 1 affinely independent points of P(G).For instance, the incidence vectors of- the empty set ∅- the singletons {i} ∀i ∈ V .

Moreover, the vectors χ{u} − χ∅ are linearly independentsince they form the identity matrix.

1 0 0 0 00 1 0 0 00 0 1 0 00 0 0 1 00 0 0 0 1

�

59/73


Polyhedral approach

Dimension and facet

“Degree of freedom”

Let us assume that the characterization of a polytope P ⊆ IRn is given by

P =

{x ∈ IRn | Ai x ≤ bi , i = 1, ...,m1

Bj x = dj , j = 1, ...,m2

}.

where every inequality Ai x ≤ bi is a “true” inequality,i.e. there exists x ∈ P such that Ai x < bi .

TheoremIf P 6= ∅, then dim(P) = n − rang(B).

dim(P) gives the “degree of freedom” of a problem :n − dim(P) variables can be obtained by fixing the dim(P) others

60/73


Polyhedral approach

Dimension and facet

“Degree of freedom”

Let us assume that the characterization of a polytope P ⊆ IRn is given by

P =

{x ∈ IRn | Ai x ≤ bi , i = 1, ...,m1

Bj x = dj , j = 1, ...,m2

}.

where every inequality Ai x ≤ bi is a “true” inequality,i.e. there exists x ∈ P such that Ai x < bi .

TheoremIf P 6= ∅, then dim(P) = n − rang(B).

dim(P) gives the “degree of freedom” of a problem :n − dim(P) variables can be obtained by fixing the dim(P) others

60/73


Polyhedral approach

Dimension and facet

Redundant inequality

DefinitionLet P a polytope characterized by a system Ax ≤ b.

An inequality ax ≤ alpha of Ax ≤ b is redundantif the system “Ax ≤ b minus ax ≤ α” still characterizes P.

A non-redundant inequality is then essential.

What are the essential inequalities ?

61/73


Polyhedral approach

Dimension and facet

Facet of a polytope

Let ax ≤ α is a valid inequality for the problem corresponding to a polytope P.

Definition• The face of ax ≤ α is the set of points of P satisfying ax ≤ α to equality,

i.e. F = {x ∈ P | ax = α}

• A face F is a facet of Pif ∅ 6= F 6= P and dim(F ) = dim(P)− 1.

Theorem• If P 6= ∅, then a non-facet inequality of P is redundant.

• Every facet of P corresponds to one inequality of a characterization of P.

62/73


Polyhedral approach

Dimension and facet

Trivial facet of the AIS polytope P(G )

Given a node i0 ∈ V , the trivial inequality

xi0 ≥ 0

defines a facet of P(G).

The corresponding face is Fi0 = {χW ∈ IRn | W ∈ acycl(G) and χW [i0] = 0}.

• χ∅ ∈ Fi0 then Fi0 6= ∅

• χ{i0} /∈ Fi0 then Fi0 6= P(G)

• The vectors χ∅ and χ{i}, i 6= i0, are n affinely independent points of Fi0then dim(Fi0 ) = n − 1Hence xi0 ≥ 0 is a facet of P(G). �

63/73


Polyhedral approach

Dimension and facet

Clique inequality of the AIS polytope

Given a clique K of G , the clique inequality is∑i∈K

xi ≤ 1

If there exists K ′ a clique of G such that K ⊂ K ′

Then ∑i∈K ′ xi ≤ 1

−xi ≤ 0 ∀i ∈ K ′ \ K

∑i∈K xi ≤ 1

Thus the clique inequality associated to K is redundant and not-facet defining.

64/73


Polyhedral approach

Dimension and facet

Clique inequality of the AIS polytope

LemmaA clique inequality on K defines a facetif and only if K is inclusion-wise maximal.

The consequence of this theorem is that maximal cliques are “better inequalities” toadd to strenghen a cutting plane algorithm.

Indeed facet defining inequalities are called “the deepest cuts” !

In practice, the heuristic method we present for clique inequalities always producemaximal cliques.

65/73


Polyhedral approach

Characterization






66/73


Polyhedral approach

Characterization

Characterization

For some polynomial combinatorial problem P and its associated combinatorialpolytope P.

Some methods to show that a system Ax ≤ b characterizes P

i.e. P = {x ∈Rn | Ax ≤ b}

I to show that there is no fractional extreme point

I to show that A is a totally unimodular matrix

I to show that the primal/dual system Ax ≤ b is total dual integral

I to show that every facet of P corresponds to at least one inequality amongAx ≤ b

I ... and many others (polyhedral decomposition, extendedformulation+projection, critical extreme point study,...)

67/73


Polyhedral approach

Characterization

Characterization



i.e. P = {x ∈Rn | Ax ≤ b}






67/73


Polyhedral approach

Characterization

Characterization



i.e. P = {x ∈Rn | Ax ≤ b}






67/73


Polyhedral approach

Characterization

Characterization



i.e. P = {x ∈Rn | Ax ≤ b}






67/73


Polyhedral approach

Characterization

Characterization



i.e. P = {x ∈Rn | Ax ≤ b}






67/73


Polyhedral approach

Characterization

Characterization



i.e. P = {x ∈Rn | Ax ≤ b}






67/73


Polyhedral approach

Characterization

Bipartite matching problemLet G = (V1 ∪ V2,E) be a bipartite (undirected) graphLet c ∈ IRm a cost associated to the edges ofE .

A matching of G is a set of pairwise disjoint edges.The matching problem on bipartite graph G is to find a matching of maximal cost.

TheoremThe following linear program is integer and is equiv. to the bipartite matching problem.

max∑e∈E

c(e)x(e)

∑e∈δ(u)

x(e) ≤ 1 ∀u ∈ V1

∑e∈δ(u)

x(e) ≤ 1 ∀u ∈ V2

x(e) ≥ 0 ∀e ∈ E .

(The matrix is totally unimodular)68/73


Polyhedral approach

Characterization

The matching problemLet G = (V ,E) be an undirected graph.

The previous formulation on G is not integer !

The matching polytope is the convex hull of the incidence vectors of the matchingsi.e.

PM (G) = conv{χM ∈ IRn | M matching of G}.

Theorem (Jack Edmonds (1965))

The matching polytope is characterized by∑e∈δ(u)

x(e) ≤ 1 ∀v ∈ V

∑e∈E(S)

x(e) ≤|S| − 1

2∀S ⊆ V with |S| odd

x(e) ≥ 0 ∀e ∈ E .

69/73


Polyhedral approach

Characterization

The matching problemLet G = (V ,E) be an undirected graph.

The previous formulation on G is not integer !

The matching polytope is the convex hull of the incidence vectors of the matchingsi.e.

PM (G) = conv{χM ∈ IRn | M matching of G}.

Theorem (Jack Edmonds (1965))

The matching polytope is characterized by∑e∈δ(u)

x(e) ≤ 1 ∀v ∈ V

∑e∈E(S)

x(e) ≤|S| − 1

2∀S ⊆ V with |S| odd

x(e) ≥ 0 ∀e ∈ E .

69/73


Polyedral approach for the TSP






70/73



The travelling salesman problem (TSP)

n citiescij the transportation cost between i and jFind a Hamiltonian “tour” that visits each cities exactly once

Let xij = 1 if edge ij is chosen and 0 otherwise.

The following linear program is integer

min∑i,j

cij xij

∑j∈V

xij = 2 ∀i ∈ V ,

xij ≥ 0 ∀(i , j) ∈ V × V .

Unfortunately, the integer solutions contain “subtours”.

71/73







min∑i,j

cij xij

∑j∈V

xij = 2 ∀i ∈ V ,

xij ≥ 0 ∀(i , j) ∈ V × V .


71/73







min∑i,j

cij xij

∑j∈V

xij = 2 ∀i ∈ V ,

xij ≥ 0 ∀(i , j) ∈ V × V .


71/73



Eliminating subtours

Menger’s theorem :a graph is connected if and only if every cut contains at least one edge.

Then “cut” inequalities ∑e∈δ(W )

x(e) ≥ 1 ∀W ( V and W 6= ∅

72/73



Eliminating subtoursMenger’s theorem :a graph is connected if and only if every cut contains at least one edge.


x(e) ≥ 1 ∀W ( V and W 6= ∅

W

72/73



Eliminating subtoursMenger’s theorem :a graph is connected if and only if every cut contains at least one edge.


x(e) ≥ 1 ∀W ( V and W 6= ∅

W

72/73



The TSP formulation

min∑e∈E

c(e)x(e)∑e∈δ(v)

x(e) = 2 ∀u ∈ V ,∑e∈δ(W )

x(e) ≥ 2 ∀W ( V and W 6= ∅,

x(e) ∈ {0, 1} ∀e ∈ E .

With (a lot of) additional facet defining inequalities,this formulation succeed to solve instances with more than 200 000cities.

73/73



The TSP formulation

min∑e∈E

c(e)x(e)∑e∈δ(v)

x(e) = 2 ∀u ∈ V ,∑e∈δ(W )

x(e) ≥ 2 ∀W ( V and W 6= ∅,

x(e) ∈ {0, 1} ∀e ∈ E .

With (a lot of) additional facet defining inequalities,this formulation succeed to solve instances with more than 200 000cities.

73/73

Date post:	11-Sep-2018
Category:	Documents
Upload:	lymien
View:	217 times
Download:	0 times

Tutoriel du GDR RO - Laboratoire d'informatique de...

Documents