SOLVING INTEGER LINEAR PROGRAMSsrirams/courses/csci5654...(integer feasible) q2.ampl bestObjective...

SOLVING INTEGER LINEAR PROGRAMS 1. Solving the LP relaxation. 2. How to deal with fractional solutions?

Integer Linear Program: Example max �x1 � 2x2 � 0.5x3 � 0.2x4 � x5 + 0.6x6s.t. x1 + 2x2 � 1

x1 + x2 + 3x6 � 1x1 + x2 + x6 � 1x3 � 3x4 � 1x3 � 2x4 � 5x5 � 1x4 + 3x5 � 4x6 � 1x2 + x5 + x6 � 1

0 x1, x2, · · · , x6 10x1, . . . , x6 2 Z

Solving ILPs var x1 integer >= 0, = 0, = 0, = 0, = 0, = 0, = 1; c2: x1 + x2 + 3* x6 >= 1;c3: x1 + x2 + x6 >= 1;c4: x3 - 3* x4 >= 1;c5: x3 - 2* x4 -5* x5 >= 1;c6: x4 + 3* x5 -4 *x6 >= 1;c7: x2 + x5 + x6 >= 1;solve;display x1, x2,x3, x4, x5, x6;end;

Solution Original ILP LP Relaxation

x1.val = 0 x2.val = 1 x3.val = 4 x4.val = 1 x5.val = 0 x6.val = 0 Optimal Value: -4.2

x1.val = 0.333333333333333 x2.val = 0.666666666666667 x3.val = 2.66666666666667 x4.val = 0 x5.val = 0.333333333333333 x6.val = 0 Optimal Value: -3.333333

Case-1: Both LP and ILP are feasible.

Opt. Solution of ILP

Opt. Solution of LP relaxation

Solving ILPs Solve LP relaxation of the ILP. Case-1: LP relaxation solution satisfies integrality constraint. Case-2: LP relaxation solution does not satisfy the integrality constraint.

Dealing with Case-2 Branch and Bound Cutting Plane Method.

1

2

BRANCH-AND-BOUND: BASIC IDEA

Integer Linear Program: Example max �x1 � 2x2 � 0.5x3 � 0.2x4 � x5 + 0.6x6s.t. x1 + 2x2 � 1


0 x1, x2, · · · , x6 10x1, . . . , x6 2 Z

Step #1: Solve the LP relaxation Solving the LP relaxation.

x1.val = 0.333333333333333 x2.val = 0.666666666666667 x3.val = 2.66666666666667 x4.val = 0 x5.val = 0.333333333333333 x6.val = 0 Optimal Value (LP relaxation): -3.333333

Step #2: Choose a branch variable

x1.val = 0.333333333333333 x2.val = 0.666666666666667 x3.val = 2.66666666666667 x4.val = 0 x5.val = 0.333333333333333 x6.val = 0 Optimal Value (LP relaxation): -3.333333

Choose a variable that

should be integer

Step #3: Branching Constraints max �x1 � 2x2 � 0.5x3 � 0.2x4 � x5 + 0.6x6s.t. x1 + 2x2 � 1


0 x1, x2, · · · , x6 10

Original Problem

Original Problem Constr. +

( x3 = 3 )

b1 b2

Visualization

x3 >= 3 x3

Branch-and-Bound

Original Problem (LP Relaxation)

Orig. Problem +

xj bsjc

Orig. Problem +

xj � dsje

P1

P2

P0

Non Integral Solution

xj : sj

Orig. Problem

xj bsjc

xk btkc

Orig. Problem

xj bsjc

xk � dtke

P3

P4

xk : sk

Branch-And-Bound: Operations Branch-And-Bound Tree: Nodes are ILP instances.

P0

P1 P2

P5 P6 P3 P4

P7 P8

P “Regular” Node

Leaves

Branch-And-Bound Tree 7 Regular Node (to be explored) Leaf node Leaf node m Leaf node

P0

P1 P2

P5 P6 P3 P4

P7 P8 Leaves

P0

P3

P4

P5

Expand Node

Expanding a Node 1.  Solve the LP relaxation for the node ILP.

2.  Three cases: 1.  Infeasible. 2.  Integral Solution. 3.  Fractional Solution.

Regular Node (unexplored) LP relaxation solution

Case-1: Infeasible LP relaxation. •  LP relaxation is infeasible.

Regular Node (unexplored)

Infeasible Leaf Node

Case-2: LP relaxation yields integral solution •  LP relaxation solution is integral: ILP solution = LP solution.


Integral Leaf Node

bestObjective := max (lpOptimum, bestObjective)

Case-3: LP relaxation yields a fractional solution •  LP relaxation yields a fractional solution.


LP relaxation solution:x1: …x2: …x3: ……Opt. Solution: optSolution



Regular Node (explored)

xj bsjc xj � dsje

Optimal Pruning

Regular Node (explored)

LP relaxation solution:x1: …x2: …x3: ……Opt. Solution: optSolutionoptSolution

Optimal Pruning

P0

P1 P2

P5 P6 P3 P4

P7 P8

bestObjective

optSolution

Branch-And-Bound Initial Node

Original ILP (unexplored

node)

bestObjective := -Infinity

BRANCH-AND-BOUND (EXAMPLE)

Original Problem var x1 integer >= 0, = 0, = 0, = 0, = 0, = 0, = 1; c2: x1 + x2 + 3* x6 >= 1; c3: x1 + x2 + x6 >= 1; c4: x3 - 3* x4 >= 1; c5: x3 - 2* x4 -5* x5 >= 1; c6: x4 + 3* x5 -4 *x6 >= 1; c7: x2 + x5 + x6 >= 1; solve; display x1, x2,x3, x4, x5, x6; end;

Original ILP integer solution


LP relaxation


Initial Node

Original Problem (p0.ampl)

bestObjective := - Infinity x1.val = 0.333333333333333 x2.val = 0.666666666666667 x3.val = 2.66666666666667 x4.val = 0 x5.val = 0.333333333333333 x6.val = 0 Optimal Value: -3.333333

p1.ampl p2.ampl

x3 = 3

Tree #1


bestObjective := - Infinity

p1.ampl p2.ampl

x3 = 3

p1.ampl Infeasible!

p2.ampl

x1.val = 0.4x2.val = 0.55x3.val = 3x4.val = 0x5.val = 0.4x6.val = 0.05

BnB Tree p0 p1 p2

p3 p4

p3.ampl x1.val = 1 x2.val = 0 x3.val = 4.57142857142857 x4.val = 0 x5.val = 0.714285714285714 x6.val = 0.285714285714286 Optimal Value: -3.857143

p5 p6 p6.ampl x1.val = 1 x2.val = 0 x3.val = 5 x4.val = 0.333333333333333 x5.val = 0.666666666666667 x6.val = 0.333333333333333 Optimal Value: -4.066667

p7 p8

p8.ampl x1.val = 1 x2.val = 0 x3.val = 6 x4.val = 0 x5.val = 1 x6.val = 0.5 Optimal Value: -4.750000

BnB Tree p0 p1 p2

p3 p4

p5 p6

p7 p8

p4.ampl x1.val = 0 x2.val = 1 x3.val = 3 x4.val = 0.666666666666667 x5.val = 0.111111111111111 x6.val = 0 Optimal Value: -3.744444

p9 p10

p9.ampl x1.val = 0 x2.val = 1 x3.val = 4 x4.val = 1 x5.val = 0 x6.val = 0 Optimal Value: -4.2

p9

bestObjective = -4.2


p8

BnB Tree p0 p1 p2

p3 p4

p5 p6

p7 p8

p9 p10 p9


p8

p10.ampl x1.val = 0 x2.val = 1 x3.val = 3 x4.val = 0 x5.val = 0.333333333333333 x6.val = 0 Optimal Value: -3.833333 p11 p12

p12.ampl x1.val = 0 x2.val = 1 x3.val = 6 x4.val = 0 x5.val = 1 x6.val = 0.5 Optimal Value: -5.750000 p12

BnB Final Tree p0 p1 p2

p3 p4

p5 p6

p7 p8

p9 p10 p9


p8 p11 p12 p12


BRANCH-AND-BOUND METHOD Heuristics

BnB choices to be made • Which unexplored regular leaf node should I expand?

• How to choose the branching variables?

• Other tricks: •  Randomized rounding to find an integer solution?

•  Carrying out BnB at the dictionary level.

BRANCH-AND-BOUND Choice of unexplored node to expand

General Situation

Unexplored “frontier”

Which node should I examine first?

Branch-and-bound p0

p1 p2

p3 p4

p5 p6

p7 p8

Unexplored Nodes Q: Which one should

we examine first?

Goal • Minimize the number of nodes explored in the tree. • Which node should I expand first: •  Yield integral solutions. •  Improve the value of bestObjective.

Unexplored Node Selection Heuristic • Deepest node first. •  Select the node that has largest depth in the tree to explore first.

•  Best LP relaxation solution. •  Select the node whose LP relaxation has the best answer.

•  Breadth First •  Search breadth-first.

In Practice… •  ILPs come from some problem domain: •  eg., We are converting a minimum cost vertex cover computation to ILP.

•  Selection heuristics are best found by experimenting with a large set of problems from the domain.

•  There is no general rule, unfortunately.

BRANCH-AND-BOUND Choosing the branch variable

BnB Tree p0 p1 p2

p3 p4

p3.ampl x1.val = 1 x2.val = 0 x3.val = 4.57142857142857 x4.val = 0 x5.val = 0.714285714285714 x6.val = 0.285714285714286 Optimal Value: -3.857143

p5 p6 p6.ampl x1.val = 1 x2.val = 0 x3.val = 5 x4.val = 0.333333333333333 x5.val = 0.666666666666667 x6.val = 0.333333333333333 Optimal Value: -4.066667

p7 p8


Original Problem var x1 integer >= 0, = 0, = 0, = 0, = 0, = 0, = 1; c2: x1 + x2 + 3* x6 >= 1; c3: x1 + x2 + x6 >= 1; c4: x3 - 3* x4 >= 1; c5: x3 - 2* x4 -5* x5 >= 1; c6: x4 + 3* x5 -4 *x6 >= 1; c7: x2 + x5 + x6 >= 1; solve; display x1, x2,x3, x4, x5, x6; end;

BnB Final Tree p0 p1 p2

p3 p4

p5 p6

p7 p8

p9 p10 p9


p8 p11 p12 p12


Initial Node


bestObjective := - Infinity x1.val = 0.333333333333333 x2.val = 0.666666666666667 x3.val = 2.66666666666667 x4.val = 0 x5.val = 0.333333333333333 x6.val = 0 Optimal Value: -3.333333

p1.ampl p2.ampl

x3 = 3

What if we chose a different branch variable..?


bestObjective := - Infinity

x5 = 1


q1.ampl (integer feasible)

q2.ampl

bestObjective := -4.2

x1.val = 0x2.val = 0.5x3.val = 6x4.val = 0x5.val = 1x6.val = 0.5Optimal Value: -4.750000q2.ampl

How to select a branch variable? •  There are heuristics… •  But it is a hard problem, in general. •  User specifies a variable ordering or priority.

•  Solver heuristics: •  Choose fractional solution closest to an integer value? •  Choose binary (0-1) variables preferentially? •  Random choice?

•  The choice of a branch variable is very important.

•  There is no single “best heuristic”.

•  Often, some expertise with the problem domain informs the heuristic.

BRANCH-AND-BOUND At the dictionary level

Thus far • Use LP solver as a black box. •  This lecture: •  Consider how to fold branch-and-bound in a dictionary setup.

BnB Node (Solve LP relaxation) Final dictionary

Child # 1 Child # 2

Can I reuse the final dictionary?

View from the dictionary

BnB Node (Solve LP relaxation) Final dictionary

Child # 1

xj bsjc

xB b · · ·

z c0 +cN| xN

xB b · · ·w · · ·z c0 +cN| xN

Claim: Dictionary D’ is always infeasible.

Proof (Pictorial)

x3 >= 3 x3

Example (ILP) var x1 >= 0, = 0, = 0,

Final Dictionary x4 4.33333333333 +0.333333x8 +0.666667x9 �0.333333x3x5 8.66666666667 �0.333333x8 +0.333333x9 �2.666667x3x6 10 �x3x7 3 �3x8 +x9 �18x3x1 5.66666666667 �0.333333x8 �0.666667x9 +0.333333x3x2 1.33333333333 +0.333333x8 �0.333333x9 +2.666667x3z 7.0 +0x8 �x9 �2x3

x1 � 6 x10 : �6 + x1

Dictionary After Branch x10 : �6 + x1

x4 4.3333333333 +0.333333x8 +0.666667x9 �0.333333x3x5 8.66666666667 �0.333333x8 +0.333333x9 �2.666667x3x6 10 �x3x7 3 �3x8 +x9 �18x3x1 5.66666666667 �0.333333x8 �0.666667x9 +0.333333x3x2 1.33333333333 +0.333333x8 �0.333333x9 +2.666667x3x10 �0.33333 �0.333333x8 �0.666667x9 +0.333333x3z 7.0 +0x8 �x9 �2x3

Dictionary becomes primal infeasible. Back to initialization phase Simplex?

Dictionary After Branch

x4 4.3333333333 +0.333333x8 +0.666667x9 �0.333333x3x5 8.66666666667 �0.333333x8 +0.333333x9 �2.666667x3x6 10 �x3x7 3 �3x8 +x9 �18x3x1 5.66666666667 �0.333333x8 �0.666667x9 +0.333333x3x2 1.33333333333 +0.333333x8 �0.333333x9 +2.666667x3x10 �0.33333 �0.333333x8 �0.666667x9 +0.333333x3z 7.0 +0x8 �x9 �2x3

Dictionary becomes primal infeasible. Dual Feasible Dictionary!! But not dual final.

How to update after branch? Parent Node

(Final Dictionary)

Child Node: 1.  Add new row. 2.  Primal Infeasible but Dual Feasible

Consider dual complement dictionary. (Feasible + but non-final)

Final dual dictionary (also final primal)

add branch constraint

Opt. Phase on dual dictionaries

LP relaxation solved for child node!

General Form Simplex Parent Node

(Final Dictionary)


Consider dual complement dictionary. (Feasible + but non-final)

Final dual dictionary (also final primal)

add branch constraint

Opt. Phase on dual dictionaries

LP relaxation solved for child node!

General Form Dictionary • Give special treatment to bounds on variables.

• Modify the Simplex algorithm in two ways: •  Dictionaries now have special ways to track bounds. •  Pivoting is modified.

l x u

Branch-and-Bound with General Form Parent Node

(Final Dictionary)


Simply update variable bound.

CUTTING PLANE METHOD

LP Relaxation for ILP

Opt. Solution of ILP

Opt. Solution of LP relaxation

Removing a Fractional Solution Branch and Bound Cutting Plane Method.

1

2

Cutting Plane

How do we find a cutting plane?

GOMORY-CHVATAL CUTTING PLANE Part 1: Setting up the problem.

ILP in Standard Form

max c

|x

s.t. Ax bx � 0x 2 Z

A, b, c are all assumed to be integers.

Conversion to standard from •  Recap from LP formulations lectures: •  Equality constraints into two inequalities. •  Rewrite in case is missing:

•  Convert ≥ to ≤ by negating both sides.

• How do we make sure A, b, c are integers?

xi 7! x+i � x�i

xi � 0

Reasonable assumption: original problem coefficients are rationals.

Integer Coefficients. max 2x1 +0.3x2 �0.1x3s.t. 0.1x1 �2x2 �x3 0.25

0.5x1 �2.6x2 +1.3x3 0.15x1, x2, x3 � 0x1, x2, x4 2 Z

max 2x1 +0.3x2 �0.1x3 ⇥10s.t. 0.1x1 �2x2 �x3 0.25 ⇥20

0.5x1 �2.6x2 +1.3x3 0.15 ⇥100x1, x2, x3 � 0x1, x2, x4 2 Z

Scale constraints

+ Objective

x3

Conversion to Integer Coefficients.

max 20x1 +3x2 �x3s.t. 2x1 �40x2 �20x3 5

50x1 �260x2 +130x3 15x1, x2, x3 � 0x1, x2, x4 2 Z

max 2x1 +0.3x2 �0.1x3s.t. 0.1x1 �2x2 �x3 0.25

0.5x1 �2.6x2 +1.3x3 0.15x1, x2, x3 � 0x1, x2, x4 2 Z

Divide result by 10

Adding Slack Variables

max c

|x

s.t. Ax bx � 0x 2 Z

max c

|x

s.t. Ax+ xs = bx,xs � 0x,xs 2 Z

Summary • We assume problem is in LP standard form. •  Assume coefficients are rational.

•  ILP standard form: coefficients (A,b,c) are integers. •  Scale the original rational problem. •  Make sure that result is divided by scale factor for objective.

• Advantage of ILP standard form: •  Slack variables are naturally integers. •  No need for separate treatment of decision and slack variables.

CUTTING PLANE METHOD Part 2: Gomory-Chvatal Cuts

Overall method

Solve LP relaxation of the problem.

Is the solution Integral?

Done.

Yes

No Add a new cutting plane constraint.

Guarantee: No integer point is lost.

Cutting Plane Visualization

First optimum

0

0

1

1

2

2

3

3

0

0

1

1

2

2

3

3

00

1

1

2

2

3

3

Second Optimum

Third Optimum Cutting

Plane

Idea: cutting plane attempts to successively “expose” an integer optimal vertex.

Cutting Plane Method • Deriving the Cutting Plane •  Gomory-Chvatal Cuts

•  Integrating the method inside Simplex. •  Using the dual dictionary to avoid initialization phase Simplex.

GOMORY-CHVATAL CUTS

Overall Idea

max c

|x

s.t. Ax+ xs = bx,xs � 0x,xs 2 Z

LP Relax

Initialization Phase

Optimization Phase

Final Dictionary

1. Solve the LP relaxation using Simplex algorithm.

Final Dictionary

xB˜

b +ÃxNz z0 +c̃N xN

Optimal solution to ILP found!!

fractional entries Derive a cutting

plane

Example: Final Dictionary x1 1.2 �3.1x2 +4.3x3 �0.5x5x4 1 �x2 +x3 �x5x6 2.5 +1.3x2 �2.1x3 +x5z 1.7 �1.2x2 �2.3x3 �2.1x5

x1 = 1.2, x2 = 0, x3 = 0, x4 = 1, x5 = 0, x6 = 2.5

Cutting Plane Derivation: Step 1

x1 1.2 �3.1x2 +4.3x3 �0.5x5x4 1 �x2 +x3 �x5x6 2.5 +1.3x2 �2.1x3 +x5z 1.7 �1.2x2 �2.3x3 �2.1x5

Identify row with fractional constant. Step 1

x1 = 1.2� 3.1x2 + 4.3x3 � 0.5x5x1 + 3.1x2 � 4.3x3 + 0.5x5 = 1.2

Cutting Plane Derivation: Step 2

x1 + 3.1x2 � 4.3x3 + 0.5x5 = 1.2

(x1 + 3x2 � 5x3 + 0x5)| {z }A

+ (0.1x2 + 0.7x3 + 0.5x5)| {z }B

= 1 + 0.2

A: integer B: integer + fraction B ≥ 0

A + B = 1.2

Claim

A is integer B has a fractional part. B ≥ 0 A + B = 1.2

(x1 + 3x2 � 5x3 + 0x5)| {z }A

+ (0.1x2 + 0.7x3 + 0.5x5)| {z }B

= 1 + 0.2

A B A+B Possible

1 0.2 1.2 YES

2 -0.8 1.2 No

1.1 0.1 1.2 No

4 -2.8 1.2 No

-3 4.2 1.2 Yes

-100 101.2 1.2 Yes

Conclusion: 1.  Fractional part of B is 0.2 2.  B ≥ 0.2

In other words, 1.  B – 0.2 is an integer. 2.  B - 0.2 ≥ 0

Cutting Plane x1 1.2 �3.1x2 +4.3x3 �0.5x5x4 1 �x2 +x3 �x5x6 2.5 +1.3x2 �2.1x3 +x5z 1.7 �1.2x2 �2.3x3 �2.1x5

0.1x2 + 0.7x3 + 0.5x5 � 0.2Cutting Plane:

Cutting Plane: Definition.

Final Dictionary (LP Relax)

Cutting Plane

frac(x) = x� bxc

xB1 = b1 +a11xI1 + · · · +a1jxIj + · · · +a1nxIn...

xBk = bk +ak1xI1 + · · · +akjxIj + · · · +aknxIn...

xBm = bm +am1xI1 + · · · +amjxIj + · · · +amnxInz = c0 +c1xI1 + · · · +cjxIj + · · · +cnxIn

bk 62 Z

frac(�ak1)xI1 + · · ·+ frac(�akn)xIn � frac(bk)

Cutting Plane

Claim: Every (integer) feasible point of the ILP satisfies the cutting plane constraint.




bk 62 Z

frac(�ak1)xI1 + · · ·+ frac(�akn)xIn � frac(bk)




bk 62 ZWe have

xBk +nX

j=1

(�akj)xIj = bk (1)

Since �akj � b�akjc and xIj � 0, we obtain

nX

j=1

(�akj)xIj �nX

j=1

b�akjcxIj

Applying this to Eq.(1) we obtain:

xBk +nX

j=1

(�akj)xIj � xBk +nX

j=1

b�akjcxIj (2)

Therefore, we get

bk � xBk +nX

j=1

b�akjcxIj (3)

The RHS of (3) is an integer, therefore, we conclude:

bbkc � xBk +nX

j=1

b�akjcxIj (4)

Combining (4) with (1), we have,

bk � bbkc Pn

j=1(�akj � b�akjc)xIj

In other words,Pn

j=1 frac(�akj)xIj � frac(bk)

Example x1 1.2 �3.1x2 +4.3x3 �0.5x5x4 1 �x2 +x3 �x5x6 2.5 +1.3x2 �2.1x3 +x5z 1.7 �1.2x2 �2.3x3 �2.1x5

0.7x2 + 0.1x3 + 0x5 � 0.50.2x2 + 0.3x3 + 0.1x5 � 0.7

CUTTING PLANE METHOD Updating the dictionary Setup for subsequent iterations.

Overall method

Solve LP relaxation of the problem.

Is the solution Integral?

Done.

Yes

No Add a new cutting plane constraint.

Guarantee: No integer point is lost.

Cutting Plane Example x1 1.2 �3.1x2 +4.3x3 �0.5x5x4 1 �x2 +x3 �x5x6 2.5 +1.3x2 �2.1x3 +x5z 1.7 �1.2x2 �2.3x3 �2.1x5

0.1x2 + 0.7x3 + 0.5x5 � 0.2Cutting Plane:

Adding cutting plane to dictionary x1 1.2 �3.1x2 +4.3x3 �0.5x5x4 1 �x2 +x3 �x5x6 2.5 +1.3x2 �2.1x3 +x5z 1.7 �1.2x2 �2.3x3 �2.1x5

0.1x2 + 0.7x3 + 0.5x5 � 0.2

x1 1.2 �3.1x2 +4.3x3 �0.5x5x4 1 �x2 +x3 �x5x6 2.5 +1.3x2 �2.1x3 +x5w1 �0.2 +0.1x2 +0.7x3 +0.5x5z 1.7 �1.2x2 �2.3x3 �2.1x5

Cutting Plane Method • Claim: Resulting dictionary after adding cutting plane is primal

infeasible.

Dictionary Solution

Cutting Plane

Cutting Plane: Example

x1 1.2 �3.1x2 +4.3x3 �0.5x5x4 1 �x2 +x3 �x5x6 2.5 +1.3x2 �2.1x3 +x5w1 �0.2 +0.1x2 +0.7x3 +0.5x5z 1.7 �1.2x2 �2.3x3 �2.1x5

Naïve Approach: Re-solve initialization phase Simplex.

Dual dictionary is feasible but non-final.

Cutting Plane: Solving again after cut.

Final Dictionary LP relaxation.

Primal Infeasible Dual Feasible

Optimize the dual problem dictionary.

Final dictionary for Dual

Also final for primal

Cut

Date post:	29-Jan-2021
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

SOLVING INTEGER LINEAR PROGRAMSsrirams/courses/csci5654...(integer feasible) q2.ampl bestObjective...

Documents