+ All Categories
Home > Documents > A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new...

A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new...

Date post: 18-Jul-2020
Category:
Upload: others
View: 3 times
Download: 0 times
Share this document with a friend
113
A fresh CP look at MINLP – new formulations and relaxations Immanuel Bomze, Universit¨ at Wien COST MINLP Workshop@IMUS Seville, 31 March 2015
Transcript
Page 1: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

A fresh CP look at MINLP –

new formulations and relaxations

Immanuel Bomze, Universitat Wien

COST MINLP Workshop@IMUS Seville, 31 March 2015

Page 2: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Overview

1. Ternary fractional QPs

Page 3: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Overview

1. Ternary fractional QPs

2. Everything is quadratic !

Page 4: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Overview

1. Ternary fractional QPs

2. Everything is quadratic !

3. ... and therefore linearly copositive

Page 5: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Overview

1. Ternary fractional QPs

2. Everything is quadratic !

3. ... and therefore linearly copositive

4. New bounds and ...

Page 6: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Overview

1. Ternary fractional QPs

2. Everything is quadratic !

3. ... and therefore linearly copositive

4. New bounds and ...

5. ... new approximation hierarchies

Page 7: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ternary decisions

Do you like Copositive Optimization ?

{ ∈ }

{−

Page 8: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ternary decisions

Do you like Copositive Optimization ?

Yes !

{ ∈ }

{−

Page 9: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ternary decisions

Do you like Copositive Optimization ?

Yes ! +1,

{ ∈ }

{−

Page 10: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ternary decisions

Do you like Copositive Optimization ?

Yes ! +1,

No ! −1,

{ ∈ }

{−

Page 11: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ternary decisions

Do you like Copositive Optimization ?

Yes ! +1,

No ! −1,

don’t know/tell 0.

{ ∈ }

{− }

Page 12: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ternary decisions

Do you like Copositive Optimization ?

Yes ! +1,

No ! −1,

don’t know/tell 0.

Ternary optimization:

min {f(x) : x ∈ Tn}

with T = {−1,0,1}.

Page 13: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ternary decisions

Do you like Copositive Optimization ?

Yes ! +1,

No ! −1,

don’t know/tell 0.

Ternary optimization:

min {f(x) : x ∈ Tn}

with T = {−1,0,1}.

What for ? Eg., graph tri-partitioning problems as in PageRank

for Folksonomy in social media/Semantic Web [Hotho et al.’06].

Page 14: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Predicting ternary decisions – TFQP

Given dissimilarities Dij > 0 (D adjacency of aversion graph):coherence within three groups/separation between them:

small x>Dx =∑i,jDijxixj

Page 15: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Predicting ternary decisions – TFQP

Given dissimilarities Dij > 0 (D adjacency of aversion graph):coherence within three groups/separation between them:

small x>Dx =∑i,jDijxixj ? ... too simple, ignores outside option.

Page 16: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Predicting ternary decisions – TFQP

Given dissimilarities Dij > 0 (D adjacency of aversion graph):coherence within three groups/separation between them:

small x>Dx =∑i,jDijxixj ? ... too simple, ignores outside option.

Better: relative to density, so

min

(x>Dx)/∑i

x2i : x ∈ Tn \ {o}

which is APX-hard [Bhaskara et al.’12].

Page 17: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Predicting ternary decisions – TFQP

Given dissimilarities Dij > 0 (D adjacency of aversion graph):coherence within three groups/separation between them:

small x>Dx =∑i,jDijxixj ? ... too simple, ignores outside option.

Better: relative to density, so

min

(x>Dx)/∑i

x2i : x ∈ Tn \ {o}

which is APX-hard [Bhaskara et al.’12]. More general

z∗T� := min

{f(x)

g(x): x ∈ Tn \ {o}

}for quadratic f , g.

Page 18: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Predicting ternary decisions – TFQP

Given dissimilarities Dij > 0 (D adjacency of aversion graph):coherence within three groups/separation between them:

small x>Dx =∑i,jDijxixj ? ... too simple, ignores outside option.

Better: relative to density, so

min

(x>Dx)/∑i

x2i : x ∈ Tn \ {o}

which is APX-hard [Bhaskara et al.’12]. More general

z∗T� := min

{f(x)

g(x): x ∈ Tn \ {o}

}for quadratic f , g. Easier variant if g(x) 6= 0 for all x ∈ Tn:

z∗T := min

{f(x)

g(x): x ∈ Tn

}.

Page 19: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ternary fractional quadratic problems (TFQPs) ...

... are special cases of (Mixed-)Binary constrained fractional QPs

Theorem [Amaral/B.’15]: Both z∗T� and z∗T are in fact MBFQP

min

{f(v)

g(v): v ∈ Rd+ , Cv = c , vi ∈ {0,1} for all i ∈ B

},

with d = 3n and Burer’s key condition satisfied.

∈ ⇐⇒ − { }

Page 20: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ternary fractional quadratic problems (TFQPs) ...

... are special cases of (Mixed-)Binary constrained fractional QPs

Theorem [Amaral/B.’15]: Both z∗T� and z∗T are in fact MBFQP

min

{f(v)

g(v): v ∈ Rd+ , Cv = c , vi ∈ {0,1} for all i ∈ B

},

with d = 3n and Burer’s key condition satisfied.

Proof. x ∈ T ⇐⇒ x = y − z with {y, z} ⊆ {0,1} and y + z ≤ 1.

Page 21: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ternary fractional quadratic problems (TFQPs) ...

... are special cases of (Mixed-)Binary constrained fractional QPs

Theorem [Amaral/B.’15]: Both z∗T� and z∗T are in fact MBFQP

min

{f(v)

g(v): v ∈ Rd+ , Cv = c , vi ∈ {0,1} for all i ∈ B

},

with d = 3n and Burer’s key condition satisfied.

Proof. x ∈ T ⇐⇒ x = y − z with {y, z} ⊆ {0,1} and y + z ≤ 1.

... latter reminds on Densest Subgraph problem (poly-time),

but also includes APX-hard Max-Cut, k-Densest Subgraph ...

Page 22: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ternary fractional quadratic problems (TFQPs) ...

... are special cases of (Mixed-)Binary constrained fractional QPs

Theorem [Amaral/B.’15]: Both z∗T� and z∗T are in fact MBFQP

min

{f(v)

g(v): v ∈ Rd+ , Cv = c , vi ∈ {0,1} for all i ∈ B

},

with d = 3n and Burer’s key condition satisfied.

Proof. x ∈ T ⇐⇒ x = y − z with {y, z} ⊆ {0,1} and y + z ≤ 1.

... latter reminds on Densest Subgraph problem (poly-time),

but also includes APX-hard Max-Cut, k-Densest Subgraph ...

Application: e.g., gene annotation graphs [Saha et al.’10] !

Page 23: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Burer’s key condition ? Copositive formulation !

... key condition: linear constraints force binary var.s into [0,1].

∈C{ • • • − ∈ B}

Page 24: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Burer’s key condition ? Copositive formulation !

... key condition: linear constraints force binary var.s into [0,1].

Used in [Burer’09];

∈C{ • • • − ∈ B}

Page 25: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Burer’s key condition ? Copositive formulation !

... key condition: linear constraints force binary var.s into [0,1].

Used in [Burer’09]; also here, for

Theorem [Amaral/B.’15]: Any Mixed-integer FQP with linear

constraints is, under key condition, equivalent to the linear COP

z∗cop = minY∈Cd+1

{A • Y : B • Y = 1 , Cc • Y = 0, Y0i − Yii = 0 , all i ∈ B} ,

Page 26: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Burer’s key condition ? Copositive formulation !

... key condition: linear constraints force binary var.s into [0,1].

Used in [Burer’09]; also here, for

Theorem [Amaral/B.’15]: Any Mixed-integer FQP with linear

constraints is, under key condition, equivalent to the linear COP

z∗cop = minY∈Cd+1

{A • Y : B • Y = 1 , Cc • Y = 0, Y0i − Yii = 0 , all i ∈ B} ,

with A coming from f , B from g, Cc from constraints Cv = c,

duality A • Y =∑i,j AijYij and

Y ∈ Cd+1 = conv{yy> : y ∈ Rd+1

+

}

> >

Page 27: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Burer’s key condition ? Copositive formulation !

... key condition: linear constraints force binary var.s into [0,1].

Used in [Burer’09]; also here, for

Theorem [Amaral/B.’15]: Any Mixed-integer FQP with linear

constraints is, under key condition, equivalent to the linear COP

z∗cop = minY∈Cd+1

{A • Y : B • Y = 1 , Cc • Y = 0, Y0i − Yii = 0 , all i ∈ B} ,

with A coming from f , B from g, Cc from constraints Cv = c,

duality A • Y =∑i,j AijYij and

Y ∈ Cd+1 = conv{yy> : y ∈ Rd+1

+

}the completely positive matrix cone.

Reason: lifting A • (yy>) = y>Ay, homogenization [Shor ’87].

Page 28: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

General form of conic linear optimization

Let K be a convex cone of symmetric d× d matrices X.

min {C • X : Ai • X = bi , i∈ [1:m] , X ∈ K}

Page 29: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

General form of conic linear optimization

Let K be a convex cone of symmetric d× d matrices X.

min {C • X : Ai • X = bi , i∈ [1:m] , X ∈ K} , barrier ??

Page 30: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

General form of conic linear optimization

Let K be a convex cone of symmetric d× d matrices X.

min {C • X : Ai • X = bi , i∈ [1:m] , X ∈ K} , barrier ??

Familiar cases:

K = N ={X = X> : X≥O

}= N ∗ . . . LP, barrier:−

∑i,jlogXij ,

K

Page 31: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

General form of conic linear optimization

Let K be a convex cone of symmetric d× d matrices X.

min {C • X : Ai • X = bi , i∈ [1:m] , X ∈ K} , barrier ??

Familiar cases:

K = N ={X = X> : X≥O

}= N ∗ . . . LP, barrier:−

∑i,jlogXij ,

and

K = P ={X = X> : X�O

}= P∗ . . .SDP, barrier:−

∑ilogλi(X) .

Page 32: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

General form of conic linear optimization

Let K be a convex cone of symmetric d× d matrices X.

min {C • X : Ai • X = bi , i∈ [1:m] , X ∈ K} , barrier ??

Familiar cases:

K = N ={X = X> : X≥O

}= N ∗ . . . LP, barrier:−

∑i,jlogXij ,

and

K = P ={X = X> : X�O

}= P∗ . . .SDP, barrier:−

∑ilogλi(X) .

In above cases, the dual cone of K,

K∗ ={S = S> : S • X ≥ 0 for all X ∈ K

}coincides with K (self-duality)

Page 33: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

General form of conic linear optimization

Let K be a convex cone of symmetric d× d matrices X.

min {C • X : Ai • X = bi , i∈ [1:m] , X ∈ K} , barrier ??

Familiar cases:

K = N ={X = X> : X≥O

}= N ∗ . . . LP, barrier:−

∑i,jlogXij ,

and

K = P ={X = X> : X�O

}= P∗ . . .SDP, barrier:−

∑ilogλi(X) .

In above cases, the dual cone of K,

K∗ ={S = S> : S • X ≥ 0 for all X ∈ K

}coincides with K (self-duality), but in general K∗ differs from K.

Page 34: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Copositive optimization (COP), duality

A very special matrix cone:

K = C = conv{xx> : x ∈ Rn, x ≥ o

},

the cone of completely positive matrices,

K∗ C∗ ≥

}

Page 35: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Copositive optimization (COP), duality

A very special matrix cone:

K = C = conv{xx> : x ∈ Rn, x ≥ o

},

the cone of completely positive matrices, with its dual cone

K∗ = C∗ ={S = S> is copositive; means: x>Sx ≥ 0 if x ≥ o

}6= C .

C ⊂ P ∩N ⊂ P N ⊂ C∗ ≥

∗ { • • ∈ ∈ C}

∗∈

∗ ∗

Page 36: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Copositive optimization (COP), duality

A very special matrix cone:

K = C = conv{xx> : x ∈ Rn, x ≥ o

},

the cone of completely positive matrices, with its dual cone

K∗ = C∗ ={S = S> is copositive; means: x>Sx ≥ 0 if x ≥ o

}6= C .

Well known relations:

C ⊂ P ∩N P N ⊂ C∗

∗ { • • ∈ ∈ C}

∗∈

∗ ∗

Page 37: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Copositive optimization (COP), duality

A very special matrix cone:

K = C = conv{xx> : x ∈ Rn, x ≥ o

},

the cone of completely positive matrices, with its dual cone

K∗ = C∗ ={S = S> is copositive; means: x>Sx ≥ 0 if x ≥ o

}6= C .

Well known relations:

C ⊂ P ∩N ⊂ P +N

∗ { • • ∈ ∈ C}

∗ ∗

Page 38: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Copositive optimization (COP), duality

A very special matrix cone:

K = C = conv{xx> : x ∈ Rn, x ≥ o

},

the cone of completely positive matrices, with its dual cone

K∗ = C∗ ={S = S> is copositive; means: x>Sx ≥ 0 if x ≥ o

}6= C .

Well known relations:

C ⊂ P ∩N ⊂ P +N ⊂ C∗

∗ { • • ∈ ∈ C}

∗ ∗

Page 39: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Copositive optimization (COP), duality

A very special matrix cone:

K = C = conv{xx> : x ∈ Rn, x ≥ o

},

the cone of completely positive matrices, with its dual cone

K∗ = C∗ ={S = S> is copositive; means: x>Sx ≥ 0 if x ≥ o

}6= C .

Well known relations:

C ⊂ P ∩N ⊂ P +N ⊂ C∗ . . . strict for n ≥ 5 .

∗ { • • ∈ ∈ C}

∗ ∗

Page 40: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Copositive optimization (COP), duality

A very special matrix cone:

K = C = conv{xx> : x ∈ Rn, x ≥ o

},

the cone of completely positive matrices, with its dual cone

K∗ = C∗ ={S = S> is copositive; means: x>Sx ≥ 0 if x ≥ o

}6= C .

Well known relations:

C ⊂ P ∩N ⊂ P +N ⊂ C∗ . . . strict for n ≥ 5 .

Primal-dual pair in (COP):

p∗ = inf {C • X : Ai • X = bi , i∈ [1:m] , X ∈ C}and

d∗ = supy∈Rm{b>y : C−

∑i yiAi ∈ C∗

}.

Page 41: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Copositive optimization (COP), duality

A very special matrix cone:

K = C = conv{xx> : x ∈ Rn, x ≥ o

},

the cone of completely positive matrices, with its dual cone

K∗ = C∗ ={S = S> is copositive; means: x>Sx ≥ 0 if x ≥ o

}6= C .

Well known relations:

C ⊂ P ∩N ⊂ P +N ⊂ C∗ . . . strict for n ≥ 5 .

Primal-dual pair in (COP):

p∗ = inf {C • X : Ai • X = bi , i∈ [1:m] , X ∈ C}and

d∗ = supy∈Rm{b>y : C−

∑i yiAi ∈ C∗

}.

Usual weak (d∗ ≤ p∗) and strong (d∗ = p∗) duality results hold.

Page 42: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Semidefinite cone P

Page 43: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,
Page 44: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,
Page 45: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,
Immanuel
Schreibmaschinentext
Page 46: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,
Page 47: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Why duality in (COP) ? And how ?

General rule: any dually feasible solution y ∈ Rm gives lower bound

b>y ≤ d∗ ≤ p∗

if S(y) = C−∑i yiAi ∈ C∗.

≤ ∗ ≤ ∗

∈ C∗

Page 48: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Why duality in (COP) ? And how ?

General rule: any dually feasible solution y ∈ Rm gives lower bound

b>y ≤ d∗ ≤ p∗

if S(y) = C−∑i yiAi ∈ C∗. For MBFQP

yB ≤ d∗ ≤ z∗cop = z∗

if

S(y) = A− yBB + yCCc +∑i∈B

yi

0 1 01 −1 00 0 0

∈ C∗ .

Page 49: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Why duality in (COP) ? And how ?

General rule: any dually feasible solution y ∈ Rm gives lower bound

b>y ≤ d∗ ≤ p∗

if S(y) = C−∑i yiAi ∈ C∗. For MBFQP

yB ≤ d∗ ≤ z∗cop = z∗

if

S(y) = A− yBB + yCCc +∑i∈B

yi

0 1 01 −1 00 0 0

∈ C∗ .Bad news: checking S ∈ C∗ is NP-hard [Dickinson/Gijben ’14];

Page 50: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Why duality in (COP) ? And how ?

General rule: any dually feasible solution y ∈ Rm gives lower bound

b>y ≤ d∗ ≤ p∗

if S(y) = C−∑i yiAi ∈ C∗. For MBFQP

yB ≤ d∗ ≤ z∗cop = z∗

if

S(y) = A− yBB + yCCc +∑i∈B

yi

0 1 01 −1 00 0 0

∈ C∗ .Bad news: checking S ∈ C∗ is NP-hard [Dickinson/Gijben ’14];

good news: checking S ∈ P +N ⊂ C∗ is poly-time.

Page 51: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Why duality in (COP) ? And how ?

General rule: any dually feasible solution y ∈ Rm gives lower bound

b>y ≤ d∗ ≤ p∗

if S(y) = C−∑i yiAi ∈ C∗. For MBFQP

yB ≤ d∗ ≤ z∗cop = z∗

if

S(y) = A− yBB + yCCc +∑i∈B

yi

0 1 01 −1 00 0 0

∈ C∗ .Bad news: checking S ∈ C∗ is NP-hard [Dickinson/Gijben ’14];

good news: checking S ∈ P +N ⊂ C∗ is poly-time. Even better:

resulting bound beats Lagrangian dual bounds (S ∈ P) ...

Page 52: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

From linear to quadratic constraints – some thoughts

Everything is linear

Page 53: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

From linear to quadratic constraints – some thoughts

Everything is linear – really ?

Nono – everything is nonlinear –

Page 54: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

From linear to quadratic constraints – some thoughts

Everything is linear – really ?

Nono – everything is nonlinear – c’mon

Ahh ... everything is quadratic !

Page 55: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

From linear to quadratic constraints – some thoughts

Everything is linear – really ?

Nono – everything is nonlinear – c’mon

Ahh ... everything is quadratic ! – wait ...

Binarity: x(1− x) = 0, quadratic, ok; also x ≥ 0 ⇔ x = t2

Page 56: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

From linear to quadratic constraints – some thoughts

Everything is linear – really ?

Nono – everything is nonlinear – c’mon

Ahh ... everything is quadratic ! – wait ...

Binarity: x(1− x) = 0, quadratic, ok; also x ≥ 0 ⇔ x = t2

Monomial: x3 = x1x2, build∏ixmii recursively, ok.

Page 57: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

From linear to quadratic constraints – some thoughts

Everything is linear – really ?

Nono – everything is nonlinear – c’mon

Ahh ... everything is quadratic ! – wait ...

Binarity: x(1− x) = 0, quadratic, ok; also x ≥ 0 ⇔ x = t2

Monomial: x3 = x1x2, build∏ixmii recursively, ok.

So polynomial optimization is quadratic

Page 58: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

From linear to quadratic constraints – some thoughts

Everything is linear – really ?

Nono – everything is nonlinear – c’mon

Ahh ... everything is quadratic ! – wait ...

Binarity: x(1− x) = 0, quadratic, ok; also x ≥ 0 ⇔ x = t2

Monomial: x3 = x1x2, build∏ixmii recursively, ok.

So polynomial optimization is quadratic, and hence everything.

Then lift quadratics to linear objective, so everything is linearcopositive ;-)

Page 59: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

From linear to quadratic constraints – some thoughts

Everything is linear – really ?

Nono – everything is nonlinear – c’mon

Ahh ... everything is quadratic ! – wait ...

Binarity: x(1− x) = 0, quadratic, ok; also x ≥ 0 ⇔ x = t2

Monomial: x3 = x1x2, build∏ixmii recursively, ok.

So polynomial optimization is quadratic, and hence everything.

Then lift quadratics to linear objective, so everything is linearcopositive ;-)

Time to get serious: [Pena/Vera/Zuluaga ’15] show indeed that

any polynomial optimization problem is linear copositive.

Page 60: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Quadratically/linearly constrained QPs

Consider m+ 1 (possibly nonconvex) quadratic functions

qi(x) = x>Qix− 2b>i x + ci , i∈ [0:m] ,

and the problem to minimize q0 over two different sets:

F := {x ∈ Rn : qi(x) ≤ 0 , all i∈ [1:m]} ,

∗∈ ∈

Page 61: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Quadratically/linearly constrained QPs

Consider m+ 1 (possibly nonconvex) quadratic functions

qi(x) = x>Qix− 2b>i x + ci , i∈ [0:m] ,

and the problem to minimize q0 over two different sets:

F := {x ∈ Rn : qi(x) ≤ 0 , all i∈ [1:m]}

Page 62: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Quadratically/linearly constrained QPs

Consider m+ 1 (possibly nonconvex) quadratic functions

qi(x) = x>Qix− 2b>i x + ci , i∈ [0:m] ,

and the problem to minimize q0 over two different sets:

F := {x ∈ Rn : qi(x) ≤ 0 , all i∈ [1:m]} , or

F+ :={x ∈ Rn+ : qi(x) ≤ 0 , all i∈ [1:m]

}.

∗∈

∈‖

Page 63: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Quadratically/linearly constrained QPs

Consider m+ 1 (possibly nonconvex) quadratic functions

qi(x) = x>Qix− 2b>i x + ci , i∈ [0:m] ,

and the problem to minimize q0 over two different sets:

F := {x ∈ Rn : qi(x) ≤ 0 , all i∈ [1:m]} , or

F+ :={x ∈ Rn+ : qi(x) ≤ 0 , all i∈ [1:m]

}.

So consider two QCQPs

z∗ := infx∈F

q0(x) and z∗+ := infx∈F+

q0(x) .

Page 64: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Quadratically/linearly constrained QPs

Consider m+ 1 (possibly nonconvex) quadratic functions

qi(x) = x>Qix− 2b>i x + ci , i∈ [0:m] ,

and the problem to minimize q0 over two different sets:

F := {x ∈ Rn : qi(x) ≤ 0 , all i∈ [1:m]} , or

F+ :={x ∈ Rn+ : qi(x) ≤ 0 , all i∈ [1:m]

}.

So consider two QCQPs

z∗ := infx∈F

q0(x) and z∗+ := infx∈F+

q0(x) .

Other linear constraints: slack var.s, Ax = a and x ∈ Rn+ can be

treated similarly by additional qm+1(x) = ‖Ax− a‖2 ≤ 0 .

Page 65: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Quadratically/linearly constrained QPs

Consider m+ 1 (possibly nonconvex) quadratic functions

qi(x) = x>Qix− 2b>i x + ci , i∈ [0:m] ,

and the problem to minimize q0 over two different sets:

F := {x ∈ Rn : qi(x) ≤ 0 , all i∈ [1:m]} , or

F+ :={x ∈ Rn+ : qi(x) ≤ 0 , all i∈ [1:m]

}.

So consider two QCQPs

z∗ := infx∈F

q0(x) and z∗+ := infx∈F+

q0(x) .

Other linear constraints: slack var.s, Ax = a and x ∈ Rn+ can be

treated similarly by additional qm+1(x) = ‖Ax− a‖2 ≤ 0 .

Important: explicit sign constraints xj ≥ 0 in F+.

Page 66: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

(Full) Lagrangian relaxation ...

... for z∗+:

L(x; u, v) = x>Hux− 2d>u x− 2v>x + c>u ,

Page 67: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

(Full) Lagrangian relaxation ...

... for z∗+:

L(x; u, v) = x>Hux− 2d>u x− 2v>x + c>u ,

∇xL(x; u, v) = 2[Hux− du − v]

Page 68: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

(Full) Lagrangian relaxation ...

... for z∗+:

L(x; u, v) = x>Hux− 2d>u x− 2v>x + c>u ,

∇xL(x; u, v) = 2[Hux− du − v] and

D2xL(x; u, v) = 2Hu for all (x; u, v) ∈ Rn × Rm+ × Rn+ ,

with Hessian Hu = Q0 +m∑i=1

uiQi and du = b0 +m∑i=1

uibi.

Page 69: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

(Full) Lagrangian relaxation ...

... for z∗+:

L(x; u, v) = x>Hux− 2d>u x− 2v>x + c>u ,

∇xL(x; u, v) = 2[Hux− du − v] and

D2xL(x; u, v) = 2Hu for all (x; u, v) ∈ Rn × Rm+ × Rn+ ,

with Hessian Hu = Q0 +m∑i=1

uiQi and du = b0 +m∑i=1

uibi.

... for z∗ similar, just drop v:

L0(x; u) = x>Hux− 2d>u x + c>u = L(x; u, o) ,

Page 70: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

(Full) Lagrangian relaxation ...

... for z∗+:

L(x; u, v) = x>Hux− 2d>u x− 2v>x + c>u ,

∇xL(x; u, v) = 2[Hux− du − v] and

D2xL(x; u, v) = 2Hu for all (x; u, v) ∈ Rn × Rm+ × Rn+ ,

with Hessian Hu = Q0 +m∑i=1

uiQi and du = b0 +m∑i=1

uibi.

... for z∗ similar, just drop v:

L0(x; u) = x>Hux− 2d>u x + c>u = L(x; u, o) ,

∇xL0(x; u) = 2[Hux− du] and

D2xL0(x; u) = 2Hu for all (x; u) ∈ Rn × Rm+ .

Page 71: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

(Semi-)Lagrangian dual functions and problems

... for z∗, no explicit sign constraints:

Θ0(u) := inf {L0(x; u) : x ∈ Rn} and z∗dual := sup{

Θ0(u) : u ∈ Rm+}

Page 72: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

(Semi-)Lagrangian dual functions and problems

... for z∗, no explicit sign constraints:

Θ0(u) := inf {L0(x; u) : x ∈ Rn} and z∗dual := sup{

Θ0(u) : u ∈ Rm+}

... for z∗+ several options:

Page 73: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

(Semi-)Lagrangian dual functions and problems

... for z∗, no explicit sign constraints:

Θ0(u) := inf {L0(x; u) : x ∈ Rn} and z∗dual := sup{

Θ0(u) : u ∈ Rm+}

... for z∗+ several options: full Lagrangian

Θ(u, v) := inf {L(x; u, v) : x ∈ Rn}with dual problem

z∗dual,+ := sup{

Θ(u, v) : (u, v) ∈ Rm+ × Rn+}

Page 74: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

(Semi-)Lagrangian dual functions and problems

... for z∗, no explicit sign constraints:

Θ0(u) := inf {L0(x; u) : x ∈ Rn} and z∗dual := sup{

Θ0(u) : u ∈ Rm+}

... for z∗+ several options: full Lagrangian

Θ(u, v) := inf {L(x; u, v) : x ∈ Rn}with dual problem

z∗dual,+ := sup{

Θ(u, v) : (u, v) ∈ Rm+ × Rn+}

or Semi-Lagrangian dual

Θsemi(u) := inf{L0(x; u) : x ∈ Rn+

}

Page 75: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

(Semi-)Lagrangian dual functions and problems

... for z∗, no explicit sign constraints:

Θ0(u) := inf {L0(x; u) : x ∈ Rn} and z∗dual := sup{

Θ0(u) : u ∈ Rm+}

... for z∗+ several options: full Lagrangian

Θ(u, v) := inf {L(x; u, v) : x ∈ Rn}with dual problem

z∗dual,+ := sup{

Θ(u, v) : (u, v) ∈ Rm+ × Rn+}

or Semi-Lagrangian dual

Θsemi(u) := inf{L0(x; u) : x ∈ Rn+

}with

z∗semi := sup{

Θsemi(u) : u ∈ Rn+}.

Page 76: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Semi-Lagrangian bounds are always tighter

General principle: for any (u, v) ∈ Rm+ × Rn+

Θ(u, v) = inf {L(x; u, v) : x ∈ Rn}

Page 77: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Semi-Lagrangian bounds are always tighter

General principle: for any (u, v) ∈ Rm+ × Rn+

Θ(u, v) = inf {L(x; u, v) : x ∈ Rn}

≤ inf{L(x; u, v) : x ∈ Rn+

}

Page 78: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Semi-Lagrangian bounds are always tighter

General principle: for any (u, v) ∈ Rm+ × Rn+

Θ(u, v) = inf {L(x; u, v) : x ∈ Rn}

≤ inf{L(x; u, v) : x ∈ Rn+

}= inf

{L0(x, u)− 2v>x : x ∈ Rn+

}

Page 79: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Semi-Lagrangian bounds are always tighter

General principle: for any (u, v) ∈ Rm+ × Rn+

Θ(u, v) = inf {L(x; u, v) : x ∈ Rn}

≤ inf{L(x; u, v) : x ∈ Rn+

}= inf

{L0(x, u)− 2v>x : x ∈ Rn+

}≤ inf

{L0(x, u) : x ∈ Rn+

}

Page 80: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Semi-Lagrangian bounds are always tighter

General principle: for any (u, v) ∈ Rm+ × Rn+

Θ(u, v) = inf {L(x; u, v) : x ∈ Rn}

≤ inf{L(x; u, v) : x ∈ Rn+

}= inf

{L0(x, u)− 2v>x : x ∈ Rn+

}≤ inf

{L0(x, u) : x ∈ Rn+

}= Θsemi(u) .

Page 81: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Semi-Lagrangian bounds are always tighter

General principle: for any (u, v) ∈ Rm+ × Rn+

Θ(u, v) = inf {L(x; u, v) : x ∈ Rn}

≤ inf{L(x; u, v) : x ∈ Rn+

}= inf

{L0(x, u)− 2v>x : x ∈ Rn+

}≤ inf

{L0(x, u) : x ∈ Rn+

}= Θsemi(u) .

Therefore in fact

z∗dual,+ ≤ z∗semi ≤ z

∗+ .

Page 82: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Semi-Lagrangian bounds are always tighter

General principle: for any (u, v) ∈ Rm+ × Rn+

Θ(u, v) = inf {L(x; u, v) : x ∈ Rn}

≤ inf{L(x; u, v) : x ∈ Rn+

}= inf

{L0(x, u)− 2v>x : x ∈ Rn+

}≤ inf

{L0(x, u) : x ∈ Rn+

}= Θsemi(u) .

Therefore in fact

z∗dual,+ ≤ z∗semi ≤ z

∗+ .

Both bounds have conic representations !

Page 83: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

SDP formulation of Lagrangian bounds

Select (d+1)×(d+1) matrices Mi such that qi(x) = y>Miy = Mi•Ywhere Y = [1, x>]>[1, x>].

• > > >

∗ { • • ≤ ∈ • }

{ • • ≤ ∈ • }

Page 84: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

SDP formulation of Lagrangian bounds

Select (d+1)×(d+1) matrices Mi such that qi(x) = y>Miy = Mi•Ywhere Y = [1, x>]>[1, x>].

Y is psd. and satisfies J0 • Y = 1 where J0 = [1, o>][1, o>]>.

Page 85: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

SDP formulation of Lagrangian bounds

Select (d+1)×(d+1) matrices Mi such that qi(x) = y>Miy = Mi•Ywhere Y = [1, x>]>[1, x>].

Y is psd. and satisfies J0 • Y = 1 where J0 = [1, o>][1, o>]>. Thus

z∗ = infY psd.

{M0 • Y : Mi • Y ≤ 0, i∈ [1:m] , J0 • Y = 1, rk Y = 1} .

Page 86: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

SDP formulation of Lagrangian bounds

Select (d+1)×(d+1) matrices Mi such that qi(x) = y>Miy = Mi•Ywhere Y = [1, x>]>[1, x>].

Y is psd. and satisfies J0 • Y = 1 where J0 = [1, o>][1, o>]>. Thus

z∗ = infY psd.

{M0 • Y : Mi • Y ≤ 0, i∈ [1:m] , J0 • Y = 1, rk Y = 1} .

SDP relaxation: drop rank constraint, keep psd. condition,

z∗SDP,primal = infY psd.

{M0 • Y : Mi • Y ≤ 0, i∈ [1:m], J0 • Y = 1} .

Page 87: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

SDP formulation of Lagrangian bounds

Select (d+1)×(d+1) matrices Mi such that qi(x) = y>Miy = Mi•Ywhere Y = [1, x>]>[1, x>].

Y is psd. and satisfies J0 • Y = 1 where J0 = [1, o>][1, o>]>. Thus

z∗ = infY psd.

{M0 • Y : Mi • Y ≤ 0, i∈ [1:m] , J0 • Y = 1, rk Y = 1} .

SDP relaxation: drop rank constraint, keep psd. condition,

z∗SDP,primal = infY psd.

{M0 • Y : Mi • Y ≤ 0, i∈ [1:m], J0 • Y = 1} .

Dual SDP problem: slack matrix S(y) = M0 − y0J0 +m∑i=1

uiMi,

z∗SDP,dual := sup{y0 ∈ R : S(y) psd., y = [y0, u

>]> ∈ R× Rm+}.

Page 88: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

COP formulation of Semi-Lagrangian bounds

Theorem [Shor <’83, ’87]:

z∗dual = z∗SDP,dual ≤ z∗SDP,primal ≤ z

∗ .

>

Page 89: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

COP formulation of Semi-Lagrangian bounds

Theorem [Shor <’83, ’87]:

z∗dual = z∗SDP,dual ≤ z∗SDP,primal ≤ z

∗ .

Further, no duality/relaxation gap,

z∗dual = z∗ if and only if

Page 90: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

COP formulation of Semi-Lagrangian bounds

Theorem [Shor <’83, ’87]:

z∗dual = z∗SDP,dual ≤ z∗SDP,primal ≤ z

∗ .

Further, no duality/relaxation gap,

z∗dual = z∗ if and only if S(y) is psd.

for y> = [q0(x), u>] at a KKT pair (x, u) ∈ F × Rm+ for z∗.

Page 91: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

COP formulation of Semi-Lagrangian bounds

Theorem [Shor <’83, ’87]:

z∗dual = z∗SDP,dual ≤ z∗SDP,primal ≤ z

∗ .

Further, no duality/relaxation gap,

z∗dual = z∗ if and only if S(y) is psd.

for y> = [q0(x), u>] at a KKT pair (x, u) ∈ F × Rm+ for z∗.

... sufficient condition for global optimality of KKT point x.

Page 92: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

COP formulation of Semi-Lagrangian bounds

Theorem [Shor <’83, ’87]:

z∗dual = z∗SDP,dual ≤ z∗SDP,primal ≤ z

∗ .

Further, no duality/relaxation gap,

z∗dual = z∗ if and only if S(y) is psd.

for y> = [q0(x), u>] at a KKT pair (x, u) ∈ F × Rm+ for z∗.

... sufficient condition for global optimality of KKT point x.

Theorem [B.’14]: For the Semi-Lagrangian bound we have

z∗semi = z∗COP,dual

with the copositive optimization representation

z∗COP,dual := sup{y0 ∈ R : S(y) copositive, y = [y0, u

>]> ∈ R× Rm+}.

Page 93: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Copositive optimization - conic primal/dual pair

Lifting again: if x ∈ Rn+ and y> = [1, x>], then

Y = yy> is completely positive (cp.)

Page 94: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Copositive optimization - conic primal/dual pair

Lifting again: if x ∈ Rn+ and y> = [1, x>], then

Y = yy> is completely positive (cp.), so reformulation of z∗+:

z∗+ = infY cp.

{M0 • Y : Mi • Y ≤ 0, i∈ [1:m], J0 • Y = 1, rk Y = 1} ,

Page 95: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Copositive optimization - conic primal/dual pair

Lifting again: if x ∈ Rn+ and y> = [1, x>], then

Y = yy> is completely positive (cp.), so reformulation of z∗+:

z∗+ = infY cp.

{M0 • Y : Mi • Y ≤ 0, i∈ [1:m], J0 • Y = 1, rk Y = 1} ,

and its natural relaxation: drop rank constraint but keep Y cp.

z∗COP,primal = infY cp.

{M0 • Y : Mi • Y ≤ 0, i∈ [1:m], J0 • Y = 1} .

Page 96: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Copositive optimization - conic primal/dual pair

Lifting again: if x ∈ Rn+ and y> = [1, x>], then

Y = yy> is completely positive (cp.), so reformulation of z∗+:

z∗+ = infY cp.

{M0 • Y : Mi • Y ≤ 0, i∈ [1:m], J0 • Y = 1, rk Y = 1} ,

and its natural relaxation: drop rank constraint but keep Y cp.

z∗COP,primal = infY cp.

{M0 • Y : Mi • Y ≤ 0, i∈ [1:m], J0 • Y = 1} .

The conic dual of this problem is exactly z∗COP,dual.

Page 97: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Strong duality for copositive primal/dual pair

Theorem [B.’14]: Consider the copositive relaxation of z∗+.

If Qi is strictly copositive for one i∈ [0:m] and if F ◦+ 6= ∅, then

z∗COP,dual = z∗COP,primal and both values are attained.

> > ∈ ׯ

Page 98: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Strong duality for copositive primal/dual pair

Theorem [B.’14]: Consider the copositive relaxation of z∗+.

If Qi is strictly copositive for one i∈ [0:m] and if F ◦+ 6= ∅, then

z∗COP,dual = z∗COP,primal and both values are attained.

Further, the Semi-Lagrangean/copositive relaxation is tight,

z∗semi = z∗+ if and only if

Page 99: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Strong duality for copositive primal/dual pair

Theorem [B.’14]: Consider the copositive relaxation of z∗+.

If Qi is strictly copositive for one i∈ [0:m] and if F ◦+ 6= ∅, then

z∗COP,dual = z∗COP,primal and both values are attained.

Further, the Semi-Lagrangean/copositive relaxation is tight,

z∗semi = z∗+ if and only if S(y) is copositive

for y> = [q0(x), u>] at a generalized KKT pair (x, u) ∈ F+ × Rm+:

Page 100: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Strong duality for copositive primal/dual pair

Theorem [B.’14]: Consider the copositive relaxation of z∗+.

If Qi is strictly copositive for one i∈ [0:m] and if F ◦+ 6= ∅, then

z∗COP,dual = z∗COP,primal and both values are attained.

Further, the Semi-Lagrangean/copositive relaxation is tight,

z∗semi = z∗+ if and only if S(y) is copositive

for y> = [q0(x), u>] at a generalized KKT pair (x, u) ∈ F+ × Rm+:

i.e., Hux = du + v with vjxj = 0 and uiqi(x) = 0 for all i, j,

but without requiring vj ≥ 0.

Page 101: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Strong duality for copositive primal/dual pair

Theorem [B.’14]: Consider the copositive relaxation of z∗+.

If Qi is strictly copositive for one i∈ [0:m] and if F ◦+ 6= ∅, then

z∗COP,dual = z∗COP,primal and both values are attained.

Further, the Semi-Lagrangean/copositive relaxation is tight,

z∗semi = z∗+ if and only if S(y) is copositive

for y> = [q0(x), u>] at a generalized KKT pair (x, u) ∈ F+ × Rm+:

i.e., Hux = du + v with vjxj = 0 and uiqi(x) = 0 for all i, j,

but without requiring vj ≥ 0.

Again have sufficient condition for global optimality of x.

Page 102: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ok. But how to employ it ?

So we know, under weak conditions,

z∗dual,+ ≤ z∗semi = z∗COP,dual = z∗COP,primal ≤ z

∗+ .

Page 103: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ok. But how to employ it ?

So we know, under weak conditions,

z∗dual,+ ≤ z∗semi = z∗COP,dual = z∗COP,primal ≤ z

∗+ .

[Dickinson/Gijben ’14]: the copositive cone C?d and its dual cone

Cd are both intractable, checking membership is NP-hard.

Page 104: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ok. But how to employ it ?

So we know, under weak conditions,

z∗dual,+ ≤ z∗semi = z∗COP,dual = z∗COP,primal ≤ z

∗+ .

[Dickinson/Gijben ’14]: the copositive cone C?d and its dual cone

Cd are both intractable, checking membership is NP-hard.

But can employ approximation hierarchies

Drd ⊂ C?d for all r = 0,1,2, . . .

of tractable cones Drd.

Page 105: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Ok. But how to employ it ?

So we know, under weak conditions,

z∗dual,+ ≤ z∗semi = z∗COP,dual = z∗COP,primal ≤ z

∗+ .

[Dickinson/Gijben ’14]: the copositive cone C?d and its dual cone

Cd are both intractable, checking membership is NP-hard.

But can employ approximation hierarchies

Drd ⊂ Dr+1d ⊂ C?d for all r = 0,1,2, . . .

of tractable cones Drd.

Page 106: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Approximation hierarchies by tractable cones

Dual cones [Drd]? shrink towards cp. cone Cd, e.g., if

[Drd]? = {Y ∈ Lrd : Y is psd.}

with Lrd polyhedral cones of d×d matrices with no negative entries

such that∞⋂r=0

Lrd = Cd .

Page 107: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Approximation hierarchies by tractable cones

Dual cones [Drd]? shrink towards cp. cone Cd, e.g., if

[Drd]? = {Y ∈ Lrd : Y is psd.}

with Lrd polyhedral cones of d×d matrices with no negative entries

such that∞⋂r=0

Lrd = Cd .

Theorem [B.’14]: The tractable bounds

z∗D,r = sup{y0 ∈ R : S(y) ∈ Drn+1, y = [y0, u

>]> ∈ R× Rm+}

increase with r

Page 108: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Approximation hierarchies by tractable cones

Dual cones [Drd]? shrink towards cp. cone Cd, e.g., if

[Drd]? = {Y ∈ Lrd : Y is psd.}

with Lrd polyhedral cones of d×d matrices with no negative entries

such that∞⋂r=0

Lrd = Cd .

Theorem [B.’14]: The tractable bounds

z∗D,r = sup{y0 ∈ R : S(y) ∈ Drn+1, y = [y0, u

>]> ∈ R× Rm+}

increase with r and are tighter than the Lagrangian bounds.

Page 109: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Survey of approximation constructions

Name symbol mode method remarksB./de Klerk E outer LP rational grid for ∆n

Yıldırım Y outer LP Y ⊂ E, gridBundfuss/Dur B outer LP simplicial partitionB./Dur/Teo B(M) outer LP M⊃ K∗Bundfuss/Dur D inner LP simplicial partitionSponsel et al. D(M) inner LP M⊂ K∗

Parrilo et al. I inner LP coeff p(d)S ≥ o

Parrilo et al. S inner SDP p(d)S is a s.o.s.

Pena et al. Q inner SDP I ⊂ Q ⊂ SLasserre L(µ, T ) outer SDP µ-moments over TDickinson/Povh L(µ, T ;A) outer SDP L(µ, T ;A) ⊂ L(µ, T )

Page 110: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Survey of approximation constructions

Name symbol mode method remarks

Bundfuss/Dur D inner LP simplicial partitionSponsel et al. D(M) inner LP M⊂ K∗

Parrilo et al. I inner LP coeff p(d)S ≥ o

Parrilo et al. S inner SDP p(d)S is a s.o.s.

Pena et al. Q inner SDP I ⊂ Q ⊂ S

Page 111: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Improvement even at approximation level zero !

Starting hierarchy with r = 0: take

L0d =

{X = X>d× d : min

i,jXij ≥ 0

}.

Then D0d = P+N is cone of nonnegatively decomposable (NND)

matrices [Jarre/Lieder ’13] and its dual [ (NND)? = DNN ]

[D0d ]? =

{X = X>d× d psd. : min

i,jXij ≥ 0

}is familiar doubly nonnegative (DNN) cone P ∩N .

Theorem [B.’14]: Even at zero approximation level, with abovechoice,

z∗dual,+ ≤ z∗D,0 ≤ z

∗semi = z∗COP ≤ z

∗+ .

Hopefully [experiments needed] first inequality is significant.

Page 112: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Selected references in chronological order

[Shor ’87] Quadratic optimization problems, Izv. Akad. Nauk SSSR Tekhn.Kibernet. 22, 128–139.

[Hotho et al.’06] Information retrieval in folksonomies: Search and ranking,in The Semantic Web: Research and Applications, 411-426. Springer,Heidelberg.

[Burer ’09] On the copositive representation of binary and continuous non-convex quadratic programs, Math. Programming 120, 479–495.

[Saha et al.’10] Dense Subgraphs with Restrictions and Applications to GeneAnnotation Graphs, in Research in Computational Molecular Biology,456–472, Springer, Berlin.

[Bhaskara et al.’12] On quadratic programming with a ratio objective, inProc.ICALP’12, 109-120. Springer, Berlin.

[Jarre/Lieder ’13] Computing a nonnegative decomposition of a matrix,talk presented on 1 August 2013 at ICCOPT, Lisbon.

Page 113: A fresh CP look at MINLP { new formulations and ...€¦ · A fresh CP look at MINLP {new formulations and relaxations Immanuel Bomze, Universit at Wien COST MINLP Workshop@IMUS Seville,

Selected references, continued

[Dickinson/Gijben ’14] On the computational complexity of membershipproblems for the completely positive cone and its dual,Comput. Optim. Appl. 57, 403–415.

[B. ’14] Copositive relaxation beats Lagrangian dual bounds in quadraticallyand linearly constrained QPs,Isaac Newton Institute Preprint series NI13064-POP, submitted.

[Amaral/B.’15] Copositivity-based approximations for mixed-integer fractio-nal quadratic optimization, Pacific J Opt., to appear.

[Pena/Vera/Zuluaga ’15] Completely positive reformulations for polynomialoptimization, Math.Programming, DOI 10.1007/s10107-014-0822-9, toappear.


Recommended