figures/minotaurnoline.jpg A new MINLP Solver · Mathematics and Computer Science Division ......

A new MINLP Solver

Ashutosh Mahajan and Team Minotaur

Mathematics and Computer Science DivisionArgonne National Laboratory

2011 DOE Applied Mathematics Program MeetingWashington, DC

October 18, 2011.

1 / 58

2 / 58

Introduction

Mixed-Integer Nonlinear Optimization

minx

f (x)

s.t. gi(x) ≤ 0, i = 1, 2, . . . ,m, (MINLP)

x ∈ Zp × Rn−p,

where f , gi : Rn → R, i = 1, 2, . . . ,m.

When f is a convex quadratic function and gi are linear, – Convex MIQP.

If f , gi are convex and twice differentiable, – Convex MINLP.

If p = 0, – Global Optimization.

In general, f , gi may be black-box, non-differentiable functions.For this talk, we assume f , gi are

twice differentiable at points of interest, and,“factorable”.

3 / 58

Introduction

Mixed-Integer Nonlinear Optimization

minx

f (x)

s.t. gi(x) ≤ 0, i = 1, 2, . . . ,m, (MINLP)

x ∈ Zp × Rn−p,

where f , gi : Rn → R, i = 1, 2, . . . ,m.

When f is a convex quadratic function and gi are linear, – Convex MIQP.

If f , gi are convex and twice differentiable, – Convex MINLP.

If p = 0, – Global Optimization.

In general, f , gi may be black-box, non-differentiable functions.For this talk, we assume f , gi are

twice differentiable at points of interest, and,“factorable”.

4 / 58

MINLP ApplicationsPower Transmission

Optimal Power FlowNetwork ExpansionContingency Analysis

5 / 58



Power GenerationNuclear Core ReloadingDesign of Cogeneration Plants

source: Terlaky et al., 1994

6 / 58




Chemical ReactorsDesign and RedesignBlending and Pooling


7 / 58




Chemical ReactorsDesign and RedesignBlending and Pooling

InfrastructureWater DistributionConcrete BuildingsOpen-Pit MiningWireless Networks

Biology, Logistics, Finance, . . .


8 / 58

MINOTAUR Toolkit

MixedI ntegerNonlinearOptimizationT oolkit:A lgorithms,Underestimators,R elaxations.

Goals:

Fast, usable MINLP solvers.

Callable library for use in applications.

Ease of developing new algorithms.The Story So Far

1 Implemented 4 solvers.2 > 49k lines of code excluding unit tests and examples.3 > 26k cpu-hours of testing.

9 / 58

FeaturesBonmin FilMINT BARON Couenne Minotaur

Algorithms:NLP B&B X × × × XQuesada-Grossmann X X × × XBranch-and-Reduce × × X X X

Support for Nonlinear Functions:Parse Comp. Graph × × X X XNonlinear Reform. × × × × XNative Derivat. × × × × X

Interfaces:AIMMS × × X × ×AMPL X X × X XGAMS X × X X ×

Open Source X × × X X

10 / 58






11 / 58






12 / 58






13 / 58

Basic Algorithm: Branch-and-BoundRELAX

BOUND

BRANCH

REPEAT

14 / 58


Relax integrality restrictions.Relax nonconvex constraints.

Linear: Secant approx., McCormick approx., . . . , BARON, Couenne.Quadratic: α−BB.. . .

BOUND

BRANCH

REPEAT

15 / 58




BOUND

Solve using an LP, QP, NLP or SDP solver. Obtain a lower bound.

BRANCH

REPEAT

16 / 58




BOUND


BRANCH

If solution of relaxation satisfies (MINLP), update ub,

Otherwise, partition the feasible region into two (or more) parts.

REPEAT

17 / 58




BOUND


BRANCH



REPEAT

Repeat RELAX-BOUND-BRANCH on each part recursively,

until ub−lb|ub| ≤ ε for a given ε ∈ R.

18 / 58

Four Main Components of ToolkitInterfaces for reading input

AMPL

Engines to solve LP/NLP/QP

Bqpd

Filter-SQP

Ipopt

OSI-CLP

Algorithms to solve MINLP

Branch-and-Bound

Outer-Approximation

Quesada-Grossmann

Branch-and-Reduce

BaseData Structures:

ProblemConstraintsObjectiveFunctionsModificationsGradient, Hessian, Jacobian

Tools for Search:Node ProcessorsNode RelaxersBranchersTree Manager

Utilities:LoggersOptionsTimers

19 / 58

Four Main Components of ToolkitInterfaces for reading input

AMPL

Your Interface HereEngines to solve LP/NLP/QP

Bqpd

Filter-SQP

Ipopt

OSI-CLP

Your engine here

Algorithms to solve MINLP

Branch-and-Bound

Outer-Approximation

Quesada-Grossmann

Branch-and-Reduce

Your algorithm here

BaseYour Data Structures:

ProblemConstraintsObjectiveFunctionsModificationsGradient, Hessian, Jacobian

Your Tools for Search:Node ProcessorsNode RelaxersBranchersTree Manager

Utilities:LoggersOptionsTimers

Highly Customizable20 / 58

Handlers

Above constructs are independent of type of problem.

Handlers implement (type specific) above methods.

Popular in constraint-programming, SCIP.Examples:

LinearHandlerBilinearHandlerQuadraticHandlerMultilinearHandler. . .

Handlers contain type specific methods to:Presolve, reformulate and relax a problem.Check feasibility of a given point.Separate or cut a given point.Find a branching candidate.

21 / 58

Handlers

Above constructs are independent of type of problem.

Handlers implement (type specific) above methods.

Popular in constraint-programming, SCIP.Examples:

LinearHandlerBilinearHandlerQuadraticHandlerMultilinearHandler. . .Your application specific Handler

Handlers contain type specific methods to:Presolve, reformulate and relax a problem.Check feasibility of a given point.Separate or cut a given point.Find a branching candidate.

22 / 58

Example of a Handler

How to write a convex NLP Branch-and-Bound solver with Minotaur.

RELAX

BOUND

BRANCH

REPEAT

23 / 58



RELAXmin

xf (x)

s.t. gi(x) ≤ 0, i = 1, 2, . . . ,m, →x ∈ Zp × Rn−p,

minx

f (x)

s.t. gi(x) ≤ 0, i = 1, 2, . . . ,m,

x∈ Rn.

BOUND

BRANCH

REPEAT

24 / 58



RELAXmin

xf (x)

s.t. gi(x) ≤ 0, i = 1, 2, . . . ,m, →x ∈ Zp × Rn−p,

minx

f (x)

s.t. gi(x) ≤ 0, i = 1, 2, . . . ,m,

x∈ Rn.

BOUND

Solve using an NLP solver. Obtain a lower bound, say f (xNLP).

BRANCH

REPEAT

25 / 58



RELAXmin

xf (x)

s.t. gi(x) ≤ 0, i = 1, 2, . . . ,m, →x ∈ Zp × Rn−p,

minx

f (x)

s.t. gi(x) ≤ 0, i = 1, 2, . . . ,m,

x∈ Rn.

BOUND


BRANCH

If xNLP integer feasible, update ub.

Otherwise partition: xi ≤ bxNLPi c ∨ xi ≥ dxNLP

i e.REPEAT

26 / 58



RELAXmin

xf (x)

s.t. gi(x) ≤ 0, i = 1, 2, . . . ,m, →x ∈ Zp × Rn−p,

minx

f (x)

s.t. gi(x) ≤ 0, i = 1, 2, . . . ,m,

x∈ Rn.

BOUND


BRANCH

If xNLP integer feasible, update ub.

Otherwise partition: xi ≤ bxNLPi c ∨ xi ≥ dxNLP

i e.REPEAT



27 / 58

Example of a HandlerHow to write a convex NLP Branch-and-Bound solver with Minotaur.

Node Relaxer

Do nothing!

Node ProcessorSolve Relaxn.

lb ≥ ub?

Is Feasible?

Return

Branch

Update ubReturn

yes

noyes

no

Brancher

Pick afractional variable.

Use Minotaur::IntVarHandler for all three

28 / 58


Node Relaxer

Do nothing!


lb ≥ ub?

Is Feasible?

Return

Branch

Update ubReturn

yes

noyes

no

Brancher


Use Minotaur::IntVarHandler for all threerelax() {// empty}

bool isFeasible() {// test integrality}

cand* findBrCandidates() {// return fractional variables}branch(cand) {// return modifications}

29 / 58


Node Relaxer

Do nothing!


lb ≥ ub?

Is Feasible?

Return

Branch

Update ubReturn

yes

noyes

no

Brancher


Use Minotaur::IntVarHandler for all threeSolver in < 200 lines:

Read instance. Load Engine.

Create IntVarHandler.

Load it to NodeProcessor, Brancher, NodeRelaxer.

Solve.30 / 58

Performance

0

0.2

0.4

0.6

0.8

1

1 4 16 64 256 1024 4096

Fra

ctio

n o

f In

sta

nce

s

Normalized Time

Minotaur

Bonmin

MINLP-BB

Time taken for 463 MINLP Instances from GAMS, MacMINLP, CMU test-sets.

31 / 58

PART - IIReformulating MINLPs

32 / 58




BOUND


BRANCH



REPEAT



33 / 58

What Could Go Wrong

Syn20M04M : a synthesis design problem inchemical engineering

Problem size : 160 Integer Variables,56 Nonlinear constraints

34 / 58

What Could Go Wrong



250+ nodes after solving for 45s

35 / 58




36 / 58

What Could Go Wrong




37 / 58

What Could Go Wrong





38 / 58

What Could Go Wrong






39 / 58

What Could Go Wrong






Solver Time NodesBonmin >2h >149kMINLP-BB >2h >150kMinotaur >2h >264k

40 / 58

Improving Coefficients: An Example(1) x1 + 21x2 ≤ 30

0 ≤ x1 ≤ 14

x2 ∈ {0, 1}

41 / 58


0 ≤ x1 ≤ 14

x2 ∈ {0, 1}

If x2 = 0

x1 + 0 ≤ 30

(1) is loose.

If x2 = 1

x1 ≤ 9

(1) is tight.

(0,0)

(0,1)(9,1)

(14,0)

x1 + 21x2 ≤ 30

42 / 58


0 ≤ x1 ≤ 14

x2 ∈ {0, 1}

If x2 = 0

x1 + 0 ≤ 30

(1) is loose.

If x2 = 1

x1 ≤ 9

(1) is tight.

(0,0)

(0,1)(9,1)

(14,0)

x1 + 21x2 ≤ 30

(0,0)

(0,1)(9,1)

(14,0)

x1 + 5x2 ≤ 14

43 / 58


0 ≤ x1 ≤ 14

x2 ∈ {0, 1}

If x2 = 0

x1 + 0 ≤ 30

(1) is loose.

If x2 = 1

x1 ≤ 9

(1) is tight.

(0,0)

(0,1)(9,1)

(14,0)

x1 + 21x2 ≤ 30

(0,0)

(0,1)(9,1)

(14,0)

x1 + 5x2 ≤ 14

Reformulation:(2) x1 + 5x2 ≤ 14

0 ≤ x1 ≤ 14

x2 ∈ {0, 1}

If x2 = 0

x1 + 0 ≤ 14

(2) is tight.

If x2 = 1

x1 ≤ 9

(2) is tight.

(1) and (2) are equivalent. But relaxation of (2) is tighter.44 / 58

Improving Coefficients: Linear to Nonlinear

g(x1, x2, . . . , xk) + My ≤ b,

li ≤ xi ≤ ui, i = 1, . . . , k,

y ∈ {0, 1}.

If g(x1, x2, . . . , xk) + M.0 ≤ b, is loose, we can tighten the formulation.

Let gu =maxx

g(x1, . . . , xk) (MAX-g)

s.t. li ≤ xi ≤ ui, i = 1, . . . , k.

If gu < b, then reformulate it as g(x1, . . . , xk) + (M − b + gu)y ≤ gu.In general (MAX-g) is as difficult a problem as the original MINLP.An upper bound on (MAX-g) will also tighten it somewhat.Trade-off between time and quality of bound. Fast or Tight.

45 / 58


g(x1, x2, . . . , xk) + My ≤ b,

li ≤ xi ≤ ui, i = 1, . . . , k,

y ∈ {0, 1}.


Let gu =maxx

g(x1, . . . , xk) (MAX-g)

s.t. li ≤ xi ≤ ui, i = 1, . . . , k.

If gu < b, then reformulate it as g(x1, . . . , xk) + (M − b + gu)y ≤ gu.

In general (MAX-g) is as difficult a problem as the original MINLP.An upper bound on (MAX-g) will also tighten it somewhat.Trade-off between time and quality of bound. Fast or Tight.

46 / 58


g(x1, x2, . . . , xk) + My ≤ b,

li ≤ xi ≤ ui, i = 1, . . . , k,

y ∈ {0, 1}.


Let gu =maxx

g(x1, . . . , xk) (MAX-g)

s.t. li ≤ xi ≤ ui, i = 1, . . . , k.

If gu < b, then reformulate it as g(x1, . . . , xk) + (M − b + gu)y ≤ gu.In general (MAX-g) is as difficult a problem as the original MINLP.An upper bound on (MAX-g) will also tighten it somewhat.Trade-off between time and quality of bound. Fast or Tight.

47 / 58

Improving Coefficients: Using Implications

g(x1, x2, . . . , xk) + My ≤ b,

li ≤ xi ≤ ui, i = 1, . . . , k,

y ∈ {0, 1}.Often, y, xi also occur in other constraints of MINLP. e.g.

g(x1, x2, . . . , xk)−My ≤ b

0 ≤ x1 ≤ M1y

0 ≤ x2 ≤ M2y

. . .

y ∈ {0, 1}

y = 0⇒ x1 = x2, . . . = xk = 0. (Implications)If g(0, . . . , 0) < b, then we can tighten.No need to solve (MAX-g). Fast and Tight.

48 / 58


g(x1, x2, . . . , xk) + My ≤ b,

li ≤ xi ≤ ui, i = 1, . . . , k,


g(x1, x2, . . . , xk)−My ≤ b

0 ≤ x1 ≤ M1y

0 ≤ x2 ≤ M2y

. . .

y ∈ {0, 1}

y = 0⇒ x1 = x2, . . . = xk = 0. (Implications)If g(0, . . . , 0) < b, then we can tighten.

No need to solve (MAX-g). Fast and Tight.

49 / 58


g(x1, x2, . . . , xk) + My ≤ b,

li ≤ xi ≤ ui, i = 1, . . . , k,


g(x1, x2, . . . , xk)−My ≤ b

0 ≤ x1 ≤ M1y

0 ≤ x2 ≤ M2y

. . .

y ∈ {0, 1}

y = 0⇒ x1 = x2, . . . = xk = 0. (Implications)If g(0, . . . , 0) < b, then we can tighten.No need to solve (MAX-g). Fast and Tight.

50 / 58

Just one of Presolve Methods

Advanced functions of presolve (Reformulating):

Improve coefficients.

Disaggregate constraints.

Derive implications and conflicts.

Basic functions of presolve (Housekeeping):

Tighten bounds on variables and constraints.

Fix/remove variables.

Identify and remove redundant constraints.

Check duplicacy.

Popular in Mixed-Integer Linear Optimization (Savelsbergh, 1994).

51 / 58

Presolve: Computational ResultsSyn20M04M from egon.cheme.cmu.edu

No Presolve

Basic Presolve Full Presolve

Variables: 420

328 292

Binary Vars: 160

144 144

Constraints: 1052

718 610

Nonlin. Constr: 56

56 56

Bonmin(sec): >7200

NA NA

Minotaur(sec): >7200

>7200 2.3

52 / 58


No Presolve Basic Presolve

Full Presolve

Variables: 420 328

292

Binary Vars: 160 144

144

Constraints: 1052 718

610

Nonlin. Constr: 56 56

56

Bonmin(sec): >7200 NA

NA

Minotaur(sec): >7200 >7200

2.3

53 / 58


No Presolve Basic Presolve Full PresolveVariables: 420 328 292Binary Vars: 160 144 144Constraints: 1052 718 610Nonlin. Constr: 56 56 56Bonmin(sec): >7200 NA NAMinotaur(sec): >7200 >7200 2.3

Minotaur, no presolve: 10000+ nodes after solving for 360sFull Presolve

54 / 58

Presolve

0

0.2

0.4

0.6

0.8

1

1 4 16 64 256 1024

Fra

ctio

n o

f In

sta

nce

s

Normalized Time

with presolvewithout presolve

Time taken in Branch-and-Bound on all 463 instances.

55 / 58

Presolve

0

0.2

0.4

0.6

0.8

1

1 4 16 64 256 1024

Fra

ctio

n o

f In

sta

nce

s

Normalized Time

Minotaur with presolveMinotaur without presolve

Bonmin

Time taken in Branch-and-Bound on 96 RSyn-X and Syn-X instances.

56 / 58

Closing Remarks and Future Directions

There are several important applications of MINLP.

This class of problems offer variety of challenges. Exciting times ahead!

Both theory and computation.Many, many improvements required in several aspects:

Tackling non-convexityReformulatingRelaxingSolving and re-solving relaxationsBranchingHeuristicsRepresentation, evaluation and derivatives of nonlinear functions

Minotaur will be available soon athttp://www.mcs.anl.gov/minotaur

57 / 58

http://www.mcs.anl.gov/minotaur

A new MINLP Solver

Ashutosh Mahajan and Team Minotaur

Mathematics and Computer Science DivisionArgonne National Laboratory

2011 DOE Applied Mathematics Program MeetingWashington, DC

October 18, 2011.

58 / 58

Date post:	05-Jun-2018
Category:	Documents
Upload:	hoangthuy
View:	213 times
Download:	0 times

figures/minotaurnoline.jpg A new MINLP Solver · Mathematics and Computer Science Division ......

Documents