Date post: | 14-Jan-2016 |
Category: |
Documents |
Upload: | hubert-houston |
View: | 217 times |
Download: | 0 times |
Column Generation
By
Soumitra Pal
Under the guidance of
Prof. A. G. Ranade
Agenda
• Introduction
• Basics of Simplex algorithm
• Formulations for the CSP
• (Delayed) Column Generation
• Branch-and-price
• Flow formulation of CSP & solution
• Conclusions
Cutting Stock Problem
• Given larger raw paper rolls
• Get final rolls of smaller widths
10
5 3
Cutting Stock Problem (2)
• Raw width (W) = 10• No of finals given below
i Width (wi) Quantity (bi)
1 3 9
2 5 79
3 6 90
4 9 27
• Minimize total no of raws reqd. to be cut
Agenda
• Introduction
• Basics of Simplex algorithm
• Formulations for the CSP
• (Delayed) Column Generation
• Branch-and-price
• Flow formulation of CSP & solution
• Conclusions
5x1 - 8x2 ≥ -80 -5x1 - 4x2 ≥ -100
-5x1 - 2x2 ≥ -80-5x1 + 2x2 ≥ -50
Min -x1 -x2
Objective/
Cost fn
Constraints
(10,0)
(13,5)
(12,10)
(8,15)
(0,10)
5x1 - 8x2 ≥ -80 -5x1 - 4x2 ≥ -100-5x1 - 2x2 ≥ -80-5x1 + 2x2 ≥ -50
-x1-x2 = -10 -x1-x2 = -20 -x1-x2 = -30
Cost decreases in this direction
(10,0)
(13,5)
(12,10)
(8,15)
(0,10)
Simplex method
contraints of no.
variablesof no.
0
:s.t.
Min
1211
22222121
11212111
2211
m
l
x
bxaxaxa
bxaxaxa
bxaxaxa
xcxcxc
i
mlmlmm
ln
ln
ln
0
:s.t.
000Min
1211
222222121
111212111
212211
i
mnlmlmm
lln
lln
nllln
x
bxxaxaxa
bxxaxaxa
bxxaxaxa
xxxxcxcxc
0
:s.t.
Min
x
bAx
cx
(10,0,130,50,30,0)
(13,5,110,10,0,0)
(12,10,60,0,0,10)
(8,15,0,0,10,40)
(0,10,0, 60,60,70)
(0,0,80, 100,80,50)
Basic & non-basic 5x1 - 8x2 ≥ -80 -5x1 - 4x2 ≥ -100-5x1 - 2x2 ≥ -80-5x1 + 2x2 ≥ -50
'1'
'11'
1'1'
'
''
11
1
1
equation, previousin it Putting
variablebasic-non one increase corner,next out find To0
0 ,0
0min
NBB
NB
NB
N
B
B
BBB
B
B
BB
BNB
NxBxxNxBbBxbBNxBx
x
xx
bBx
NBI
bBcxccxbBxbBx
xbx
NBAx
xcccx
NBccc
xNBcccxcx
xNBccxcxcxccxxcNxBxcxcxccx
BN
NBN
NBNBBNNBB
NNNBBNNBB
1
'1'
'1'''
''1'''
cost reduced called isbracket in termThe
)(costin Change
)()(
cost New
• If all elements of it are non-negative, it is already optimal• Otherwise choose the non-basic variable corresponding to
most negative value to enter the basic
• How to find out outgoing basic variable?
ofcomponent kth
)( ofcomponent kth of Min value
0. becomes ofcomponent )k(say one until increased becan It .
variablebasic new with mutilpies that ofcolumn thebe Let
1
'th
1
1'1'
d
xbB
xx
NBd
bBNxBx
B
B
i
NB
Few steps of simplex algorithm
• Compute reduced cost• If all >= 0, optimal stop.• Otherwise choose the most negative
component• Corresponding variable enters basis• Compute the ratios for finding out leaving basis• Update basic variables and B for next step
NBcc BN1
'1'NBB NxBxx
Smart way of doing
variableleavingabout info gives
gives, ofcolumn theis wherecost reduced and gives
gives
d
xdNaaBd
yNcycyBxbBx
B
NB
BB
yd
Start
yd
Reduced cost vector
yd
Selection of non-basic variable
yd
y vector
yd
Selection of leaving basic variable
Preparation for next step
yd
yd
Entering variable in 2nd iteration
yd
Leaving variable in 2nd iteration
yd
Preparation for 3rd iteration
yd
Final iteration
Done so far
• Basics of (revised) Simplex algorithm– Formulation– Corners, basic variables, non-basic variables– How to move from corner to corners– Optimal value
Agenda
• Introduction
• Basics of Simplex algorithm
• Formulations for the CSP
• (Delayed) Column Generation
• Branch-and-price
• Flow formulation of CSP & solution
• Conclusions
raw k fromcut widthi of finals of no.
otherwise 0used, is raw k if 1
widthafor index iraw afor index k
widthsdifferent of no.mraws available of no.K
,,1,,1integer and 0,,11or 0
,,1
,,1 :s.t.
Min
th
th
th
1
1
1
ik
k
ik
k
k
m
iiki
i
K
kik
K
kk
x
y
miKkxKky
KkWyxw
mibx
y
First step to solution - Formulation
Kantorovich formulation
Kantorovich formulation example
1 2 3 4
1 ,4
3 ,1 ,10 ,1 ,4
21
132111
4231
k
kk
k xx
xxxyyyyK
LP relaxation• Integer programming is hard• Convert it to a linear program
raw k fromcut widthi of finals of no.
otherwise 0used, is raw k if 1
widthafor index iraw afor index k
widthsdifferent of no.mraws available of no.K
,,1,,1integer and 0,,11or 0
,,1
,,1 :s.t.
Min
th
th
th
1
1
1
ik
k
ik
k
k
m
iiki
i
K
kik
K
kk
x
y
miKkxKky
KkWyxw
mibx
y
0 ≤ yk ≤ 1
LP relaxation (2)
• LP relaxation is poor• For our example, optimal LP objective is 120.5• The optimal integer objective solution value is
157• Gap is 36.5• It can be as bad as ½ of the integer solution
• Can we get a better LP relaxation?
jpattern fromcut widthi of finals of no.
jpattern in cut raws of no.
patterns possible all ofset J
pattern cutting aj
widthsdifferent of no.m
widthafor index i
integer and 0
,,1 :s.t.
Min
th
ij
j
j
iJj
jij
Jjj
a
x
Jjx
mibxa
x
Another Formulation
Gilmore-Gomory
Formulation
27
90
79
9
000000001
000000110
000121000
321100100
:9
:6
:5
:3
9
8
7
6
5
4
3
2
1
4
3
2
1
x
x
x
x
x
x
x
x
x
w
w
w
w
Example formulation
3*1 + 5*0 + 6*1 + 9*0 = 9 ≤ 10
LP relaxation of Gilmore-Gomory
• LP relaxation is better
• For our example, 156.7
• Integer objective solution, 157
• Gap is 0.3
• Conjecture: Gap is less than 2 for practical cutting stock problems
Issues
• The number of columns are exponential
• Optimal value is still not integer
• Gilmore-Gomory proposed an ingenuous way of working with less columns
• Branch-and-price to solve the other problem
Done so far
• Basics of (revised) Simplex algorithm– Formulation– Corners, basic variables, non-basic variables– How to move from corner to corners– Optimal value
• Formulation of the CSP– LP relaxation– Bounds
Agenda
• Introduction
• Basics of Simplex algorithm
• Formulations for the CSP
• (Delayed) Column Generation
• Branch-and-price
• Flow formulation of CSP & solution
• Conclusions
Column Generation
• In pricing step of simplex algorithm one basic variable leaves and one non-basic variable enters the basis
• This decision is made from the reduced cost vector
• The component with most negative value determines the entering variable
yNcNBcc NBN or 1
Column Generation (2)
• In terms of columns, we need to find
• That is equivalent to finding a such that
• For cutting stock problem, cj=1, hence it is equivalent to
• With implicit constraint
• Pricing sub-problem
} ofcolumn a is |{min argj
Nayac jjj
} ofcolumn a is |)(min{ Aayaac
}1|max{ yaya
Wwa
Column Generation (3)
AnyNew Columns?
STOP(LP Optimal)
SolveRestricted Master Problem
(RMP)
SolvePricing Problem
Update RMP withNew Columns
No
Yes
27
90
79
9
000000001
000000110
000121000
321100100
:9
:6
:5
:3
9
8
7
6
5
4
3
2
1
4
3
2
1
x
x
x
x
x
x
x
x
x
w
w
w
w
Example formulation
T
BB
a
aaaa
aaaa
yyB
xBC
0101findcan weg,programmin dynamicor n enumeratio Using
109653constraintknapsack Given
1.1.12
1
3
1 maximize toneed we
112/13/11111
27
90
5.39
3
1/27
1/90
2/79
3/9
,
1000
0100
0020
0003
,
27
90
79
9
4321
4321
27
81
5.39
9
27
1.990
5.39
9
27
.90
5.39 ,
1000
0101
0020
0001
replaced is Bin column first ,9
*90*9*1/90*3
1/3 /
0103
1
3dt
t
xB
t
dx
daBd
B
TT
B
T
5.15627815.399 Solution
optimalalready is Hence1 subproblem ofsolution get not can We
109653costraintknapsack Given
1.1.12
1.0 maximize toneed we
112/101111
27
81
5.39
9
,
1000
0101
0020
0001
4321
4321
B
B
x
aaaa
aaaa
yyB
xB
Done so far
• Basics of (revised) Simplex algorithm– Formulation– Corners, basic variables, non-basic variables– How to move from corner to corners– Optimal value
• Formulation of the CSP– LP relaxation– Bounds
• Delayed column generation– Restricted master problem– Sub-problem to generate column– Repeated till optimal
Agenda
• Introduction
• Basics of Simplex algorithm
• Formulations for the CSP
• (Delayed) Column Generation
• Branch-and-price
• Flow formulation of CSP & solution
• Conclusions
Integer solution
• A formulation of the problem which gives ‘good’ LP relaxation
• Solved the LP relaxation using column generation
• But, the LP relaxation is not the ultimate goal, we need integer solution
• In the example x2 is 39.5• One way is to rounding
• What to do if there are multiple fractional variables?
• The method commonly followed is branch -and-bound technique of solving integer programs
• Basic fundae– create multiple sub-problems with extra
constraints
– solve the sub-problems using column generation
– take the best of solutions to the sub problems
• Key is the rules of dividing into sub problems (this is called branching strategy)
• Things to look into– Branching rule does not destroy column
generation sub problem property– Efficiency of the process
• Ingenuity of the formulation and branching strategy
• Use of running bounds to prune (fathomed)
• This technique is called branch-and-price
• We see another formulation next
Done so far• Basics of (revised) Simplex algorithm
– Formulation– Corners, basic variables, non-basic variables– How to move from corner to corners– Optimal value
• Formulation of the CSP– LP relaxation– Bounds
• Delayed column generation– Restricted master problem– Sub-problem to generate column– Repeated till optimal
• Branch-and-price– Basic fundae
Agenda
• Introduction
• Basics of Simplex algorithm
• Formulations for the CSP
• (Delayed) Column Generation
• Branch-and-price
• Flow formulation of CSP & solution
• Conclusions
Flow formulation
• Bin packing problem
• Similar to the cutting stock problem
bin items
w2 w2 w2 w2
w1w1 w1
loss loss loss loss loss
01 2 3 4 5
w2 w2
01 2 3 4 5
loss
Example• Bin capacity W = 5
• Item sizes – w1=3 and w2=2
Formulation equations
• Problem is equivalent to finding minimum flow between nodes 0 & W
0integer
0integer
,,2,1
,
1,,2,1,0
0,
:s.t.
Min
)()(
z
x
mdbxWjz
Wi
jz
xx
z
ij
Awkkdwkk
Ajkjk
Aijij
d
d
• Issue of symmetry in the graph
• 3 rules to reduce symmetry1. Arcs are specified according to decreasing
width
2. A bin can never start with a loss
3. Number of consecutive arcs corresponding to a single item size must not exceed number of orders
w2 w2 w2 w2
w1w1 w1
loss loss loss loss loss
01 2 3 4
5
Invalid as per rule 2
Invalid as per rule 1
w2 w2 w2
w1
loss loss loss
0 1 2 3 4 5
Branch-and-price
• The sub-problem finds an longest unit flow path where cost of each arc is given by y vector
• Branching heuristics– Xij >= ceil(xij)– Xij <= floor(xij)
Few other points
• Loss constraints from the LP relaxation value is equated with the sum of loss variables
• With the above constraint it can be shown that the solution will be always integral
• The algorithm uses this fact by incrementally adding the lower bound
• This is done finite no. of times• No of new constraints are added is finite• Algorithm runs in pseudo-polynomial time
Conclusions & Future work
• Column generation is a success story in large scale integer programming
• The flexibility to work with a few columns make real life problems tractable
• Ingenuous formulations and branching heuristics are used to improve efficiency
• Need to explore more applications with different heuristics
Acknowledgements
• Valerio de Carvalho for borrowing some of his explanations exactly
• http://www-fp.mcs.anl.gov/otc/Guide/CaseStudies/simplex/applet/SimplexTool.html for the java applet
• AMPL & CPLEX tools
• My guide
Thank you!