of 35
8/10/2019 Branch&Bound MIP
1/35
1
Contents college 3 en 4
Book: Appendix A.1, A.3, A.4, 3.4, 3.5,4.1, 4.2, 4.4, 4.6 (not: 3.6 - 3.8, 4.2
- 4.3)
Extra literature on resource constrained
project scheduling (will be handed out)
8/10/2019 Branch&Bound MIP
2/35
2
Planning and schedulingoptimization techniques
Dispatching Rules
Composite Dispatching Rules
Adaptive search Dynamic Programming
(Integer) Linear Programming
Cutting plane methods
Branch and Bound
Beam Search
8/10/2019 Branch&Bound MIP
3/35
3
Linear programming (LP) model
LP:
Matrix form:
)n,,1j(0x
bxa...xaxa
bxa...xaxa
bxa...xaxa
:tosubject
xc...xcxcmin
j
mnmn22m11m
2nn2222121
1nn1212111
nn2211
0x
bAx
xcmin T
objective function
constraints
variable
restrictions
where:x, c: n-vectorA: m,n-matrixb: m-vector
8/10/2019 Branch&Bound MIP
4/35
4
Linear programming example
)2,1j(0x
12x4x3
4xx2:tosubject
xxmax
j
21
21
21
0
0
x
x
12
4
x
x
43
12
:tosubjectx
x
1
1max
2
1
2
1
2
1
T
or:
8/10/2019 Branch&Bound MIP
5/35
5
Linear programming example:graphical solution (2D)
1 2 3 4 5 6
x1
1
2
3
4
5
6x2
)objective(
6xx 21
4xx2 21
12x4x3 21
)2,1j(0x
12x4x3
4xx2
:tosubject
xxmax
j
21
21
21
0
solution
space
8/10/2019 Branch&Bound MIP
6/35
6
Linear programming (cont.)
Solution techniques: (dual) simplex method
interior point methods (e.g. Karmarkar algorithm)
Commercial solvers, for example: CPLEX (ILOG)
XPRESS-MP (Dash optimization)
OSL (IBM) Modeling software, for example:
AIMMS
AMPL
8/10/2019 Branch&Bound MIP
7/35
7
Integer programming (IP) models
Integer variable restriction
IP: integer variables only
MIP: part integer, part non-integer variables
BIP: binary (0-1) variables
General IP-formulation:
Complex solution space
)Zx(integerx
bAx
xcmin T
8/10/2019 Branch&Bound MIP
8/35
8
Integer programming example:graphical solution (2D)
1 2 3 4 5 6
x1
1
2
3
4
5
6x2
)objective(
6xx 21
)2,1j(Zx
12x4x3
4xx2:tosubject
xxmax
j
21
21
21
02 optimal solutions!
8/10/2019 Branch&Bound MIP
9/35
9
Total unimodularity property forinteger programming models
Suppose that all coefficients are integer in the model:
i.e.
Example: transportation problem
if A has the total unimodularity property(i.e. every square submatrix has determinant 0,1,-1)
there is an optimal integer solution x*
& the simplex method will find such a solution
0x
bAx
xcmin T
j,i,b,a iij
8/10/2019 Branch&Bound MIP
10/35
10
Integer programming tricks
PROBLEM: x = 0 or x k
use binary indicator variable y=
restrictions:
kfor x,
0for x,
1
0
0,1yykx
x)onupperboundanis(MyMx
8/10/2019 Branch&Bound MIP
11/35
11
PROBLEM: fixed costs: if xi>0 then costs C(x
i)
use indicator variable yi=
restrictions :
Integer programming tricks (2)
0x
bAx
)x(Cminimize
0.for x
0,for x
xck
0)x(C
:where
i
i
iii
i
0for x,
0for x,
1
0
i
i
0,1y
xcyk)x(C
yMx
i
iiiii
ii
)i(
8/10/2019 Branch&Bound MIP
12/35
12
Hard vs. soft restrictions
hard restriction: must hold, otherwise unfeasibility
for example:
soft restriction: may be violated, with a penalty
for example:
(Integer) programming tricks (3)
0Y,0x
Y5xx
100Yxcminimize
21
T
5xx 21
8/10/2019 Branch&Bound MIP
13/35
8/10/2019 Branch&Bound MIP
14/35
14
Conjunctive/disjunctive programming
- conjunctive set of constraints: must all be satisfied
- disjunctive set of constraints: at least one must be satisfied
example (Appendix A.4):
Integer programming tricks (5)
jkj
kjk
jjj
pxx
or
pxx
xwmin
}1,0{y
)y1(Mpxx
yMpxx
xwmin
2jkj
1kjk
jjj
8/10/2019 Branch&Bound MIP
15/35
15
IP examplenonpreemptive single machine, total weighted
completion time (App. A.3)
function)(objectivextwimizemin
jjoboftimecompletionxt
jjoboftimecompletionxt
n
1j
1-Cmax
0t
jtj
1-Cmax
0tjt
jt
1xif jt
otherwise0andt,timeatjjobif,1xjt completes
objective function: minimize weighted completion time:
model definition:
8/10/2019 Branch&Bound MIP
16/35
16
IP example (cont.)
Restriction: all jobs must be completed once:1x
1-Cmax
0tjt
Restriction: only one job per time t:
t)timeperjoboneexactly:on(restricti1x
t)duringprocessinisjjob(if1x
n
1j
pt
tsjs
pt
tsjs
j
j
if job j is in process during t, it must be completedsomewhere during [t,t+pj]
8/10/2019 Branch&Bound MIP
17/35
17
IP example (cont.)
Complete IP-model:
1)-Cmax,0,tn,,1,j(for1,0x
1)-Cmax,0,t(for1x
n),1,j(for1x
:tosubject
xtwinimizem
jt
n
1j
pt
tsjs
1-Cmax
0tjt
n
1j
1-Cmax
0tjtj
j
nCmax
integer
variables
8/10/2019 Branch&Bound MIP
18/35
18
IP example (cont.)Additional restriction: precedence constraints
Model definition: SUCC(j) = successors of job j
job j must be completed before all jobs in SUCC(j):
n),1,jSUCC(j),k(fortxtx
kjoboftimestartptx
jjoboftimecompletiontx
1maxC
0t
kt
1maxC
0t
jt
k
1maxC
0tkt
1maxC
0tjt
8/10/2019 Branch&Bound MIP
19/35
19
Integer programmingsolution techniques
Heuristic vs. explicit approach: trade-off between solution quality and computation time
trade-off between implementation effort/costs and yield
(i.e. profits gained from solution quality improvement)
Heuristic methods; for example: local search (e.g. simulated annealing, tabu search, k-opt)
(composite) dispatching rules (e.g. EDD, SPT, MS)
adaptive search
rounding fractional solutions
beam search
8/10/2019 Branch&Bound MIP
20/35
20
Explicit methods; 3 categories:
1. dynamic programming
2. cutting plane (polyhedral) methods3. branch and bound
or: hybrid methods (combination of the above)
Commercial IP solvers usually use acombination of heuristics and 2, 3
Integer programmingsolution techniques (cont.)
8/10/2019 Branch&Bound MIP
21/35
21
Dynamic programming
Problem divided into stages
Each stage can have various states
A recursive objective function is used toiterate through all states and all stages(forwards or backwards)
))}x,i(i(F)x,i(c{min)i(F tt1t1ttttx
ttt
T),0,(tx t
ti
(constant)c)i(F 000
8/10/2019 Branch&Bound MIP
22/35
22
Cutting plane methods
STEP 0: Create a relaxation of the problem byomitting restrictions
(e.g. the integrality restrictions)
STEP 1: Solve the current problem
STEP 2: If solution is infeasible then generate
a restriction that cuts of the solution,
and add it to the problem STEP 1
Otherwise: DONE
8/10/2019 Branch&Bound MIP
23/35
23
Branch and bound
Enumeration in a search tree
each node is a partial solution, i.e. a part of
the solution space
...
...
root node
child nodes
child nodes
Level 0
Level 1
Level 2
8/10/2019 Branch&Bound MIP
24/35
24
Branch and bound example 1
Disjunctive programming (appendix A.4):disjunctive set of constraints: at least one must be satisfied
xj = completion time of job j
restriction: )Ik,j(pxxorpxxeither jkjkjk
solve LP withoutdisjunctive restrictions
(= LP relaxation)
if disjunct. restr.violated for j & k
Level 0
Level 1
...
kjk pxx jkj pxx
8/10/2019 Branch&Bound MIP
25/35
25
Branch and bound (cont.)
Upper bound: e.g. a feasible solution
Lower bound:
e.g. a solution to an easier problem
Node elimination (fathom/discard nodes):
when lower bound >= upper bound
8/10/2019 Branch&Bound MIP
26/35
26
Branch and bound (cont.)
Branching strategy:
how to partition solution space
Node selection strategy: sequence of exploring nodes:
depth first (tries to obtain a solution fast)
breadth/best bound first (tries to find the best solution)
which nodes to explore (filter and beam width)
filter width: #nodes selected for thorough evaluation
beam width: #nodes that are branched on ( filter width)
Beam search
8/10/2019 Branch&Bound MIP
27/35
27
Branch and bound example 2 Single machine, maximum lateness, release
and due dates
lower bound: EDD + preemption
Jobs 1 2 3 4
p(j) 4 2 6 5
r(j) 0 1 3 5
d(j) 8 12 11 10
(?,?,?,?)
(1,?,?,?) (4,?,?,?)(2,?,?,?)
Level 0
Level 1(3,?,?,?)
8/10/2019 Branch&Bound MIP
28/35
28
Branch and bound example 2 Lower bound for: (1,?,?,?)
Lower bound: Lmax = max(0,17-12,15-11,0)=5
Jobs 1 2 3 4
p(j) 4 2 6 5r(j) 0 1 3 5
d(j) 8 12 11 10
t
r(2) r(3) r(4)
d(4)
8/10/2019 Branch&Bound MIP
29/35
29
(?,?,?,?)
(1,?,?,?) (4,?,?,?)(2,?,?,?)
Level 0
Level 1(3,?,?,?)
Branch and bound example 2 (cont.)
LB=5 LB=7*=UB
(1,2,?,?) (1,3,?,?) Jobs 1 2 3 4
p(j) 4 2 6 5r(j) 0 1 3 5
d(j) 8 12 11 10
infeasible:
(1,3,4,3,2)
LB=6*=UB
(1,2,4,3)
LB=5*=UB(1,3,4,2)
DONE
(1,2,4,3) (1,3,4,2)
8/10/2019 Branch&Bound MIP
30/35
30
Branch and bound example 3
1 2 3 4 5 6
x1
1
2
3
4
5
6x2
)objective(
6xx 21
0
4.2x
,8.0x
2
1
0x1 1x1
3x
,0x
2
1
2x
,1x
2
1
2x2 3x2
2x
,1x
2
1
3x
,0x
2
1
obj: 3 obj: 3
obj: 3 obj: 3
LP solution:
B h l 1
8/10/2019 Branch&Bound MIP
31/35
31
Beam search example 1single-machine, total weighted tardiness
Upper bound: ATC rule (apparent tardiness cost):schedule 1 job at a timeevery time a machine comes available, determineranking of jobs:
pK)0,tpdmax(
j
j
j
jj
ep
w)t(I
MS rule
WSPTrule
look-ahead parameter:
K = 4.5 + R (R 0.5)
K = 6 - 2R (R 0.5)
= due date range factor maxminmax C/ddR
8/10/2019 Branch&Bound MIP
32/35
32
Beam search example 1 (cont.)single-machine, total weighted tardiness
(?,?,?,?)
(1,?,?,?) (4,?,?,?)(2,?,?,?) (3,?,?,?)
1,2,3)(j
p
we
p
w)t(I
j
jpK
)0,tpdmax(
j
j
j
jj
Upper bound by ATC rule: )3,2,1j(0)0,tpdmax( jj
Jobs 1 2 3 4
p(j) 10 10 13 4
d(j) 4 2 1 12w(j) 14 12 1 12
w(j)/p(j) 1.4 1.2 0.1 3
8/10/2019 Branch&Bound MIP
33/35
33
Beam search example 1 (cont.)single-machine, total weighted tardiness
(?,?,?,?)
(1,?,?,?) (4,?,?,?)(2,?,?,?) (3,?,?,?)
Jobs 1 2 3 4
p(j) 10 10 13 4
d(j) 4 2 1 12
w(j) 14 12 1 12
w(j)/p(j) 1.4 1.2 0.1 3
Jobs C(j) d(j) T(j) w(j)*T(j)
1 10 4 6 84
2 24 2 22 264
3 37 1 36 36
4 14 12 2 24
Upper boundby ATC rule:
Total = 408
8/10/2019 Branch&Bound MIP
34/35
34
Beam search example 1 (cont.)single-machine, total weighted tardiness
(?,?,?,?)
(1,?,?,?) (4,?,?,?)(2,?,?,?) (3,?,?,?)
UB=408 UB=436 UB=814 UB=440
discardedexploredfurther
(beam width = 2)
4 nodesanalyzed(filter width=4)
B h l 1 ( t )
8/10/2019 Branch&Bound MIP
35/35
35
Beam search example 1 (cont.)
(?,?,?,?)
(1,?,?,?) (4,?,?,?)(2,?,?,?) (3,?,?,?)
UB=408 436 814 440
(1,2,?,?) (1,3,?,?) (1,4,?,?)
UB=480 706 408
(1,4,2,3) (1,4,3,2)
UB=408 554
b t l ti
(2,1,?,?) (2,3,?,?) (2,4,?,?)
436
(2,4,1,3) (2,4,3,1)
436 608