Post on 16-Aug-2020
transcript
IBM Analytics
Pierre BonamiIBM ILOG CPLEXOptimization Direct workshop - INFORMS 2015 - October 312015
Recent improvement to MISOCP in CPLEX
1 ©2015 IBM Corporation
IBM Analytics
CPLEX Optimization Studio news (12.6.2/12.6.3)
CPLEX Optimization Studio 12.6.2 (June2015)
DoCloud services: OPL on the cloud (→ IBMWorkshop and TA20)Performance improvements
CPLEX Optimization Studio 12.6.3 (upcoming)
Distributed MIP enhancementsMILP performance improvements
DOcplex (→ IBM Workshop):A Python modeling layer for CPLEX and CPOPrepared to connect to local or DOcloudFreeNotebook-ready
CPLEX Optimization Studio CommunityEdition
2 ©2015 IBM Corporation
IBM Analytics
CPLEX keeps getting better
→ A. Tramontani TA19, → R. Wunderling WD38
3 ©2015 IBM Corporation
IBM Analytics
Convex MIQCP and MISOCP
min cT x
xTQkx + aTk x ≤ a0
k k = 1, . . . ,m,Ax = b,xj ∈ Z j = 1, . . . , p.
(MIQCP)
Where all quadratic constraints can be represented as second order cones:
Ld := {(x , x0) ∈ Rd+1 :
d∑
i=1
x2
i ≤ x2
0, x0 ≥ 0}.
(Ld defines the (d + 1)-dimensional second order cone.)
4 ©2015 IBM Corporation
IBM Analytics
A Lorentz cone
It is convex
5 ©2015 IBM Corporation
IBM Analytics
Second order cone representability
CPLEX recognizes or automatically reformulates to SOC via basictransformations
Second order cones:
d∑
i=1
x2
i ≤ x2
0, with x0 ≥ 0
Rotated second order cones:
d∑
i=2
x2
i ≤ x0x1, with x0, x1 ≥ 0
Convex quadratic constraints:
xTQx + aT x ≤ a0, with Q � 0
6 ©2015 IBM Corporation
IBM Analytics
More second order cone representability
More convex nonlinear set can be represented
Euclidian norms:||x ||2 ≤ y
||Ax + b||2 ≤ cT x + d
Rational functions
y ≥1
x
y ≥1
x2
Those will not be reformulated automatically by CPLEX.
7 ©2015 IBM Corporation
IBM Analytics
MISOCP
min cT x
(xJi , xhi ) ∈ Ldi i = 1, . . . ,mAx = b,xj ∈ Z j = 1, . . . , p.
(MISOCP)
Algorithms are based on SOCP relaxation
SOCP relaxation solved efficiently by interior point methods
Supported by CPLEX since version 9.0
8 ©2015 IBM Corporation
IBM Analytics
MISOCP Applications
Application SOC Integer
Portfolio optimization Risk, utility, robustness number of assets, min in-vestment
[Bienstock, 1996, Bonami and Lejeune, 2009, Vielma et al., 2008]Truss topology optimiza-tion
Physical forces Cross section of bars
[Achtziger and Stolpe, 2006]Networks with delays Delay as function of traf-
ficPath, flows
[Boorstyn and Frank, 1977, Ameur and Ouorou, 2006]Location with stochasticservices
Demands location model
[Elhedhli, 2006]TSP with neighborhoods(Robotics)
Definition of ngbh. TSP
[Gentilini et al., 2013]Many more... see for eg. http://cblib.zib.de.
9 ©2015 IBM Corporation
IBM Analytics
Agenda
Two basic algorithms in CPLEXSOCP-B&B (since CPLEX 9.0)OA-B&B (since CPLEX 11.0)
Novelties of CPLEXCone disaggregationLift-and-project cuts for MISOCP
Comparison with CPLEX 12.6.1
10 ©2015 IBM Corporation
IBM Analytics
SOCP based branch-and-bound
Straightforward generalization of main MILPalgorithm:
Solve SOCP relaxation at each node of the tree
11 ©2015 IBM Corporation
IBM Analytics
SOCP based branch-and-bound
Straightforward generalization of main MILPalgorithm:
Solve SOCP relaxation at each node of the tree
Branch on variables with fractional value
integerfeasible
fathomedbybound
infeasible
11 ©2015 IBM Corporation
IBM Analytics
SOCP based branch-and-bound
Straightforward generalization of main MILPalgorithm:
Solve SOCP relaxation at each node of the tree
Branch on variables with fractional value
Prune by infeasibility, bounds and integerfeasibility
integerfeasible
fathomedbybound
infeasible
11 ©2015 IBM Corporation
IBM Analytics
Outer Approximation [Duran and Grossmann, 1986]
min cT x
s.t.
gi (x) ≤ 0 i = 1, . . . ,m,
Ax = b
xj ∈ Z, j = 1, . . . , p.
Idea: Take first-order approximations of constraints at different points xk andbuild an approximating MILP
12 ©2015 IBM Corporation
IBM Analytics
Outer Approximation [Duran and Grossmann, 1986]
min cT x
s.t.
gi (x) ≤ 0 i = 1, . . . ,m,
Ax = b
xj ∈ Z, j = 1, . . . , p.
Idea: Take first-order approximations of constraints at different points xk andbuild an approximating MILP
min cT x
s.t.
gi (xk) +∇gi (x
k)T (x − xk) ≤ 0 i = 1, . . . ,m, k = 1, . . . ,K
xj ∈ Z, j = 1, . . . , p.12 ©2015 IBM Corporation
IBM Analytics
OA Branch-and-cut [Quesada and Grossmann, 1992]
Initialize by solving SOCP relaxation and takingOA’s at the optimum
13 ©2015 IBM Corporation
IBM Analytics
OA Branch-and-cut [Quesada and Grossmann, 1992]
Initialize by solving SOCP relaxation and takingOA’s at the optimum
Carry out B&B on OA
13 ©2015 IBM Corporation
IBM Analytics
OA Branch-and-cut [Quesada and Grossmann, 1992]
Initialize by solving SOCP relaxation and takingOA’s at the optimum
Carry out B&B on OA
At each integer feasible node:1 Solve SOCP with integers fixed, and enrich the
set of linearizations2 Resolve the LP relaxation of the node with the
new cuts3 Repeat as long as node is integer feasible
integerfeasible
13 ©2015 IBM Corporation
IBM Analytics
OA Branch-and-cut [Quesada and Grossmann, 1992]
Initialize by solving SOCP relaxation and takingOA’s at the optimum
Carry out B&B on OA
At each integer feasible node:1 Solve SOCP with integers fixed, and enrich the
set of linearizations2 Resolve the LP relaxation of the node with the
new cuts3 Repeat as long as node is integer feasible
Never prune by integer feasibility
13 ©2015 IBM Corporation
IBM Analytics
The MISOCP solver in CPLEX ≥ 11.0
Both main algorithms available
Selection controled via parameterCPXPARAM_MIP_Strategy_MIQCPStrat
miqcpstrat 1 - SOCP based Branch-and-boundmiqcpstrat 2 - OA based branch-and-cutmiqcpstrat 0 - heuristic decision which of the two to use
14 ©2015 IBM Corporation
IBM Analytics
How bad can outer approximation be?
The following convex MIQCP
min cT x
s.t.∑n
i=1
(
xi −1
2
)2≤ n−1
4
x ∈ Zn
(1)
is infeasible:
The ball is too small to containinteger points
It is large enough to touch everyedge of the hypercube
x y
z
15 ©2015 IBM Corporation
IBM Analytics
Solving (1) with OA cuts
No OA constraint can cut 2vertices of the hypercube
If an inequality cuts twovertices, it cuts the segmentjoining themAny such inequality would cutinto the interior of (1)
x y
z
16 ©2015 IBM Corporation
IBM Analytics
Solving (1) with OA cuts
No OA constraint can cut 2vertices of the hypercube
If an inequality cuts twovertices, it cuts the segmentjoining themAny such inequality would cutinto the interior of (1)
An OA needs at least 2n OA cutsto converge
x y
z
16 ©2015 IBM Corporation
IBM Analytics
Solving (1) with OA cuts
No OA constraint can cut 2vertices of the hypercube
If an inequality cuts twovertices, it cuts the segmentjoining themAny such inequality would cutinto the interior of (1)
An OA needs at least 2n OA cutsto converge
Remark
A basic SOCP branch-and-boundalso enumerates at least 2n
integer solutions
x y
z
16 ©2015 IBM Corporation
IBM Analytics
Experimental illustration
CPLEX 12.4 1 SCIP 2.0.1 Bonmin B-Hybn 2n nodes nodes nodes10 1,024 2,047 720 11,15615 32,768 65,535 31,993 947,01420 1,048,576 2,097,151 1,216,354 . . .
1CPLEX ran in single threaded mode17 ©2015 IBM Corporation
IBM Analytics
Experimental illustration
CPLEX 12.4 1 SCIP 2.0.1 Bonmin B-Hybn 2n nodes nodes nodes10 1,024 2,047 720 11,15615 32,768 65,535 31,993 947,01420 1,048,576 2,097,151 1,216,354 . . .
Remark
Problem is simple for CPLEX/SCIP if variables are 0 − 1: replace x2
i byxi , the contradiction n
4≤ n−1
4follows.
SCIP ≥ 2.1 and CPLEX ≥ 12.6.1 solve it in a blink
1CPLEX ran in single threaded mode17 ©2015 IBM Corporation
IBM Analytics
New in CPLEX 12.6.2/12.6.3
Cone disaggregation for MISOCP
Lift-and-project cuts for MISOCP
Redesigned heuristic choice of most promising algorithm
18 ©2015 IBM Corporation
IBM Analytics
Separable mixed integer convex programs
min cT x
s.t. gi (x) ≤ 0 i = 1, . . . ,mAx = b
xj ∈ Z j = 1, . . . , pl ≤ x ≤ u
(sMINLP)
For i = 1, . . . ,m, gi are convex separable:
gi (x) =
n∑
j=1
gij(xj)
with gij : [lj , uj ] → R convex.
19 ©2015 IBM Corporation
IBM Analytics
Disaggregated formulation
Introduce one variable yij for each elementary function:
min cT x
s.t.n∑
j=1
yij ≤ 0 i = 1, . . . ,m,
gij(xj) ≤ yiji = 1, . . . ,m,j = 1, . . . , n,
Ax = b,xi ∈ Z i = 1, . . . , p,l ≤ x ≤ u.
(sMINLP∗)
20 ©2015 IBM Corporation
IBM Analytics
Cone disaggregation for MISOCP
In standard form the nonlinear constraint describing the second order cone isnot convex separable:
n∑
i=1
x2
i ≤ x2
0
[Vielma et al., 2015] divide the constraint by x0 ≥ 0 to get a convex separableconstraint:
n∑
i=1
x2
i
x0
≤ x0
Introduce y1, . . . , yn and rewrite as:
n∑
i=1
yi ≤ x0
x2
i ≤ x0yi
21 ©2015 IBM Corporation
IBM Analytics
Application to (1) [Hijazi et al., 14]
Extended formulation of (1)
min cT x
s.t.n
∑
i=1
yi ≤ (n − 1)/4
(xi − 0.5)2 ≤ yi i = 1, . . . , n
x ∈ Zn.
Its outer approximation on K points x̄k
min cT x
s.t.n
∑
i=1
yi ≤ (n − 1)/4
2(
xki − 0.5)
(xi − xki ) +(
xki − 0.5)2
≤ yii = 1, . . . , nk = 1, . . . ,K
x ∈ Zn
22 ©2015 IBM Corporation
IBM Analytics
Application to (1) [Hijazi et al., 14]
Extended formulation of (1)
min cT x
s.t.n
∑
i=1
yi ≤ (n − 1)/4
(xi − 0.5)2 ≤ yi i = 1, . . . , n
x ∈ Zn.
Its outer approximation on K points x̄k
min cT x
s.t.n
∑
i=1
yi ≤ (n − 1)/4
2(
xki − 0.5)
(xi − xki ) +(
xki − 0.5)2
≤ yii = 1, . . . , nk = 1, . . . ,K
x ∈ Zn
22 ©2015 IBM Corporation
IBM Analytics
How good can this be?
[Cornuéjols and Li, 2001] showed thatthe empty ball in dimension n has splitrank n (also holds for any outerapproximation in this space)⇒ Practically unsolvable using anyform of split cuts (even conic ones).
Instead the OA of the disaggregatedformulation has (simple) split rank 1.
23 ©2015 IBM Corporation
IBM Analytics
How good can this be?
[Cornuéjols and Li, 2001] showed thatthe empty ball in dimension n has splitrank n (also holds for any outerapproximation in this space)⇒ Practically unsolvable using anyform of split cuts (even conic ones).
Instead the OA of the disaggregatedformulation has (simple) split rank 1.
xi ≤ 0 xi ≥ 1
yi ≥14
2 points suffice to make the mixed-integer set infeasiblex1 = 0 and x2 = 1:
− xi + 0.25 ≤ yi i = 1, . . . , n
xi − 0.75 ≤ yi i = 1, . . . , n
xi ∈ {0, 1}
⇒ yi ≥ 0.25
23 ©2015 IBM Corporation
IBM Analytics
Lift-and-project cuts for MISOCP
Cuts are an essential component of MILP solvers
Can always apply MILP cuts to a linear OA of MISOCP (and we do it)
Can we generate better cuts by looking directly at nonlinear functions?partial answer: as long as the cut generated is linear it could also havebeen obtained from a linear outer approximation
In the past three years, tremendous activity towards conic cuts for conicprogramming [Andersen and Jensen, 2013, Belotti et al., 2013,Kılınç-Karzan and Yıldız, 2015, Modaresi et al., 2015] (among others)
Our goals
linear cutting planes
find suitable OA from which to derive a cut
fast
24 ©2015 IBM Corporation
IBM Analytics
A more complicated empty ellipse
n∑
i=1
(100x2
i − 98xi )−
n2
∑
i=1
4x2ix2i−1 ≤ −1, x ∈ Zn
rotated ellipse that not longer fully disaggregates
results on 12 threads with 12.6.1, 12.6.2, 12.6.2-- (no lift-and-projectcuts) and 12.6.2++ (aggressive lift-and-project cuts), 3 hours time limit
12.6.1 12.6.2-- 12.6.2 12.6.2++n nodes nodes nodes nodes5 2,261 2,045 2,045 1,82510 2,097,151 1,914,797 29 115 >23,125,426 >146,604,478 7,769 1
(Largest model solved in 2.2 sec by 12.6.2, in 5.5 sec by 12.6.2++.)
25 ©2015 IBM Corporation
IBM Analytics
Remarks
Cone disaggregation
automatically applied by default during presolve
only interesting (and done):1 if using the OA-B&B2 for cones that are long enough
Lift-and-project
only done in OA based algorithm
expensive algorithm (may not speed up every “easy” model)
26 ©2015 IBM Corporation
IBM Analytics
Heuristic algorithm selection
Redesigned heuristic to choose algorithm to apply in view of these changes
CPLEX 12.6.2: 245 models solved by at least one method and failed by none
0
20
40
60
80
100
1 10 100 1000 10000 100000
% m
odel
sol
ved
time factor
1262_miqcpstrat_01262_miqcpstrat_11262_miqcpstrat_2
Default strategy picked
OA-B&B 186 times
SOCP-B&B 4 times
55 models identical withboth
"perfect" Heuristic:
2 more models withOA-B&B
9 more models withSOCP-B&B
27 ©2015 IBM Corporation
IBM Analytics
Heuristic algorithm selection
Redesigned heuristic to choose algorithm to apply in view of these changes
CPLEX 12.6.1: 225 models solved by at least one method and failed by none
0
20
40
60
80
100
1 10 100 1000 10000 100000
% m
odel
sol
ved
time factor
1261_miqcpstrat_01261_miqcpstrat_11261_miqcpstrat_2
Default strategy picked
OA-B&B 113 times
SOCP-B&B 46 times
56 models identical withboth
"perfect" Heuristic:
14 more models withOA-B&B
36 more models withSOCP-B&B
27 ©2015 IBM Corporation
IBM Analytics
Troubleshooting
Choosing the right algorithm
Our testing shows OA is better in most cases, but can be dramatically worst:
when SOCP relaxation solved very fast,
when SOCP relaxation very strong,
when linear relaxations are weak due to structure (around the head ofcones).
Simplifying models
Ideas presented by Ed carry over:
Can linearize products by binaries (not done automatically).
Assiociated MILP can provide good solutions.
28 ©2015 IBM Corporation
IBM Analytics
Computational experiments MISOCP 12.6.3
Test bed
CPLEX internal test set: 233 models
CBLIB test set (http://cblib.zib.de): 80 models
Compare
CPLEX 12.6.3 against CPLEX 12.6.1
geometric mean of solve times 2
2All tests are carried on Linux machines: Intel X5650 @ 2.67 GHz, 24 GB RAM, 12
threads, deterministic.29 ©2015 IBM Corporation
IBM Analytics
CPLEX 12.6.1 vs 12.6.3
00.20.40.60.81.0
> 0 sec233 Models
> 1 sec154 Models
×2.78 ×4.76
CPLEX test bed
CPLEX 12.6.1: 29 time limits
CPLEX 12.6.3: 4 time limits
00.20.40.60.81.0
> 0 sec66 Models
> 1 sec46 Models
×3.57 ×6.25
CBLIB
CPLEX 12.6.1: 17 time limits
CPLEX 12.6.3: 8 time limits
30 ©2015 IBM Corporation
IBM Analytics
Legal Disclaimer© IBM Corporation 2015. All Rights Reserved.
The information contained in this publication is provided for informational purposes only. While efforts weremade to verify the completeness and accuracy of the information contained in this publication, it is provided ASIS without warranty of any kind, express or implied. In addition, this information is based on IBM’s currentproduct plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible forany damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothingcontained in this publication is intended to, nor shall have the effect of, creating any warranties orrepresentations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicablelicense agreement governing the use of IBM software.
References in this presentation to IBM products, programs, or services do not imply that they will be available inall countries in which IBM operates. Product release dates and/or capabilities referenced in this presentationmay change at any time at IBM’s sole discretion based on market opportunities or other factors, and are notintended to be a commitment to future product or feature availability in any way. Nothing contained in thesematerials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by youwill result in any specific sales, revenue growth or other results.
Performance is based on measurements and projections using standard IBM benchmarks in a controlledenvironment. The actual throughput or performance that any user will experience will vary depending uponmany factors, including considerations such as the amount of multiprogramming in the user’s job stream, theI/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be giventhat an individual user will achieve results similar to those stated here.
IBM, the IBM logo, CPLEX and SPSS are trademarks of International Business Machines Corporation in theUnited States, other countries, or both.
Intel, Intel Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the UnitedStates and other countries.
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company,product, or service names may be trademarks or service marks of others.
32 ©2015 IBM Corporation
IBM Analytics
References I
W. Achtziger and M. Stolpe. Truss topology optimization with discrete design variables – Guaranteed global optimalityand benchmark examples. Structural and Multidisciplinary Optimization, 34(1):1–20, 2006.
W. B. Ameur and A. Ouorou. Mathematical models of the delay constrained routing problem. Algorithmic OperationsResearch, 1(2):94–103, 2006.
K. Andersen and A. Jensen. Intersection cuts for mixed integer conic quadratic sets. In M. Goemans and J. Correa,editors, Integer Programming and Combinatorial Optimization, volume 7801 of Lecture Notes in Computer Science,pages 37–48. Springer Berlin Heidelberg, 2013. ISBN 978-3-642-36693-2. doi: 10.1007/978-3-642-36694-9_4.URL http://dx.doi.org/10.1007/978-3-642-36694-9_4.
P. Belotti, J. C. Góez, I. Pólik, T. K. Ralphs, and T. Terlaky. On families of quadratic surfaces having fixed intersectionswith two hyperplanes. Discrete Applied Mathematics, 161(16–17):2778 – 2793, 2013. ISSN 0166-218X. doi:http://dx.doi.org/10.1016/j.dam.2013.05.017. URLhttp://www.sciencedirect.com/science/article/pii/S0166218X13002461.
D. Bienstock. Computational study of a family of mixed-integer quadratic programming problems. MathematicalProgramming, 74:121–140, 1996.
P. Bonami and M. Lejeune. An Exact Solution Approach for Integer Constrained Portfolio Optimization Problems UnderStochastic Constraints. Operations Research, 57:650–670, 2009.
R. Boorstyn and H. Frank. Large-scale network topological optimization. IEEE Transactions on Communications, 25:29–47, 1977.
G. Cornuéjols and Y. Li. On the rank of mixed 0,1 polyhedra. In K. Aardal and B. Gerards, editors, Integer Programmingand Combinatorial Optimization, volume 2081 of Lecture Notes in Computer Science, pages 71–77. Springer BerlinHeidelberg, 2001. ISBN 978-3-540-42225-9. doi: 10.1007/3-540-45535-3_6. URLhttp://dx.doi.org/10.1007/3-540-45535-3_6.
M. A. Duran and I. Grossmann. An outer-approximation algorithm for a class of mixed-integer nonlinear programs.Mathematical Programming, 36:307–339, 1986.
S. Elhedhli. Service System Design with Immobile Servers, Stochastic Demand, and Congestion. Manufacturing & ServiceOperations Management, 8(1):92–97, 2006. doi: 10.1287/msom.1050.0094. URLhttp://msom.journal.informs.org/cgi/content/abstract/8/1/92.
33 ©2015 IBM Corporation
IBM Analytics
References II
I. Gentilini, F. Margot, and K. Shimada. The travelling salesman problem with neighborhoods: Minlp solution.Optimization Methods and Software, 28(2):364–378, 2013.
H. Hijazi, P. Bonami, and A. Ouorou. An outer-inner approximation for separable mixed-integer nonlinear programs.INFORMS Journal on Computing, 26(1):null, 14. doi: 10.1287/ijoc.1120.0545.
F. Kılınç-Karzan and S. Yıldız. Two term disjunctions on the second-order cone. Mathematical Programming, April 2015.http://link.springer.com/article/10.1007/s10107-015-0903-4.
S. Modaresi, M. Kılınç, and J. Vielma. Intersection cuts for nonlinear integer programming: convexification techniques forstructured sets. Mathematical Programming, pages 1–37, 2015. ISSN 0025-5610. doi: 10.1007/s10107-015-0866-5.URL http://dx.doi.org/10.1007/s10107-015-0866-5.
I. Quesada and I. E. Grossmann. An LP/NLP based branch–and–bound algorithm for convex MINLP optimizationproblems. Computers and Chemical Engineering, 16:937–947, 1992.
J. P. Vielma, S. Ahmed, and G. Nemhauser. A lifted linear programming branch-and-bound algorithm for mixed integerconic quadratic programs. INFORMS Journal on Computing, 20:438–450, 2008.
J. P. Vielma, I. Dunning, J. Huchette, and M. Lubin. Extended Formulations in Mixed Integer Conic QuadraticProgramming. Research Report, 2015.
34 ©2015 IBM Corporation