1
A Convex Polynomial that is not SOS-Convex
Amir Ali AhmadiPablo A. Parrilo
Laboratory for Information and Decision SystemsMassachusetts Institute of Technology
FRG: Semidefinite Optimization and Convex Algebraic Geometry
May 2009 - MIT
2
Deciding Convexity
42
321
22
212
31
41 1632248)( xxxxxxxxxp
Given a multivariate polynomial p(x):=p(x1,…, xn ) of even degree, how to decide if it is convex?
A concrete example:
Most direct application: global optimization
Global minimization of polynomials is NP-hard even when the degree is 4
But in presence of convexity, no local minima exist, and simple gradient methods can find a global min
3
Other Applications
In many problems, we would like to parameterize a family of convex polynomials that perhaps:
serve as a convex envelope to a non-convex function
approximate a more complicated function
fit data samples with “small” error
[Magnani, Lall, Boyd]
To address these questions, we need an understanding of the algebraic structure of the set of convex polynomials
4
Convexity and the Second Derivative
2221
21
2221
21
2221
21
2221
21
19219248969624
969624484812)(
xxxxxxxx
xxxxxxxxxH
But can we efficiently check if H(x) is PSD for all x?
Equivalently, H(x) is PSD if and only if the scalar polynomial yTH(x)y in 2n variables [x;y] is positive semidefinite (psd)
Back to our example:
Fact: a polynomial p(x) is convex if and only if its Hessian H(x) is positive semidefinite (PSD)
5
Complexity of Deciding Convexity
Checking polynomial nonnegativity is NP-hard for degree 4 or larger
However, there is additional structure in the polynomial yTH(x)y:
Quadratic in y (a “biform”)
H(x) is a matrix of second derivatives partial derivatives commute
Pardalos and Vavasis (’92) included the following question proposed by Shor on a list of the seven most important open problems in complexity theory for numerical optimization:
“What is the complexity of deciding convexity of a multivariate polynomial of degree four?”
To the best of our knowledge: still open
6
SOS-Convexity
p(x) sos-convex
p(x) convex
))(())(()()()( yxMyxMyxMxMyyxHy TTTT
As we will see, checking sos-convexity can be cast as the feasibility of a semidefinite program (SDP), which can be solved in polynomial time using interior-point methods.
Defn. ([Helton, Nie]): a polynomial p(x) is sos-convex if its Hessian factors as
for a possibly nonsquare polynomial matrix M(x).
)()()( xMxMxH T
7
SOS-convexity (Ctnd.)
sos-convexity in the literature: Semidefinite representability of semialgebraic sets [Helton, Nie]
Generalization of Jensen’s inequality [Lasserre]
Polynomial fitting, minimum volume convex sets [Magnani, Lall, Boyd]
Question that has been raised:
Q: must every convex polynomial be sos-convex?
NoOur main contribution
(via an explicit counterexample)
8
AgendaNonnegativity and sum of squares
A bit of history
Connection to semidefinite programming
SOS-matrices
Other (equivalent?) notions for sos-convexity
Our counterexample (convex but not sos-convex)
Ideas behind the proof
Several remarks
How did we find it?
Conclusions
9
Nonnegative and Sum of Squares Polynomials
m
ii xqxp
1
2 )()(
Defn. A polynomial p(x) is nonnegative or positive semidefinite (psd) if
Defn. A polynomial p(x) is a sum of squares (sos) if there exist some other polynomials q1(x),…, qm(x) such that
p(x) sos p(x) psd (obvious)
When is the converse true?
nxxp 0)(
10
Hilbert’s 1888 Paper
In 1888, Hilbert proved that a nonnegative polynomial p(x) of degree d in n variables must be sos only in the following cases:
n=1 (univariate polynomials of any degree)
d=2 (quadratic polynomials in any number of variables)
n=2 and d=4 (bivariate quartics)
In all other cases, there are polynomials that are psd but not sos
11
The Celebrated Example of Motzkin
The first concrete counterexample was found about 80 years later!
This polynomial is psd but not sos
13),( 22
21
41
22
42
2121 xxxxxxxxM
12
Sum of Squares and Semidefinite Programming
Unlike nonnegativity, checking whether a polynomial is SOS is a tractable problem
,)( Qzzxp T
Thm: A polynomial p(x) of degree 2d is SOS if and only if there exists a PSD matrix Q such that
where z is the vector of monomials of degree up to d
].,...,,,...,,,1[ 2121dnn xxxxxxz
Feasible set is the intersection of an affine subspace and the PSD cone, and thus is a semidefinite program.
13
SOS matrices
Therefore, can solve an SDP to check if P(x) is an sos-matrix.
Defn. ([Kojima],[Gatermann-Parrilo]):
A symmetric polynomial matrix P(x) is an sos-matrix if
for a possibly nonsquare polynomial matrix M(x).
)()()( xMxMxP T
Lemma: P(x) is an sos-matrix if and only if the scalar polynomial yTP(x)y in [x;y] is sos.
14
PSD matrices may not be SOS
Explicit “biform” examples of Choi, Reznick (and others), yield PSD matrices that are not SOS.
For instance, the biquadratic Choi form can be rewritten as:
However this example (and all others we’ve found), is not a valid Hessian:
15
Equivalent notions for convexity
)2
1(]1,0[,
))1(()()1()(
enoughyx
yxpypxp
yxxyxpxpyp T ,))(()()(
yxyxHyT ,0)(
Basic definition:
First order condition:
Second order condition:
16
Each condition can be SOS-ified
]1,0[))1(()()1()(),( SOSyxpypxpyxg
SOSxyxpxpypyxg T ))(()()(),(
SOSyxHyyxg T )(),(2
Basic definition:
First order condition:
Second order condition:
SOSyxpypxpyxg )()()(),(2
1
2
1
2
1
2
1
2
1 A’
B
C
A
AThm: CBA’
Proof: mimics the “standard” proof closely and uses closedness of the SOS cone
17
A convex polynomial that is not sos-convex
Without further ado...
]1,0[))1(()()1()(),( yxpypxpyxg
))(()()(),( xyxpxpypyxg T
yxHyyxg T )(),(2
B
C
A
Need a polynomial p(x) such that all the following polynomials
are psd but not sos.
18
Our Counterexample
83
63
22
43
42
23
62
82
63
21
43
22
21
23
42
21
43
41
23
22
41
42
41
23
61
22
61
81
30607044
162416335
43254011832)(
xxxxxxx
xxxxxxxxxxx
xxxxxxxxxxxp
Claim:
p(x) is convex: H(x) is PSD
p(x) is not sos-convex: H(x) ≠ MT(x)M(x)
A homogeneous polynomial in three variables, of degree 8.
19
Proof: H(x) is PSD
yxHyxxx T )()( 23
22
21
)()()()( 23
22
21 xMxMxHxxx T
Or equivalently the scalar polynomial
is sos.
Claim:
Proof: Exact sos decomposition, with rational coefficients.
Exploiting symmetries of this polynomial, we solve SDPs of significantly reduced size
44433322211123
22
21 )()( zQzzQzzQzzQzyxHyxxx TTTTT
20
Rational SOS Decomposition
21
Rational SOS Decomposition
22
Proof: H(x)≠MT(x)M(x)
Lemma: if H(x) is an sos-matrix, then all its 2n-1 principal minors are sos polynomials. In particular, all diagonal elements are sos.
Proof: follows from the Cauchy-Binet formula.
Therefore, it suffices to show that
is not sos.
We do this by a duality argument.
63
43
22
23
42
43
21
23
22
21
42
21
23
41
22
41
61
48326420516
300120035401792)1,1(
xxxxxxxxxx
xxxxxxxH
23
Separating Hyperplane
SOS
PSDH(1,1)
µ
SOSww
H
0,
0)1,1(,
24
A few remarks
)1,,(),( 2121 xxpxxp
Our counterexample is robust to small perturbations
Follows from inequalities being strict
A dehomogenized version is still convex but not sos-convex
Minimal in the number of variables
“Almost” minimal in the degree
25
How did we find this polynomial?
sosyxHyxxx
H
r
Tr )()(
0)1,1(,
,fix
23
22
21
parameterize H(x)
add Hessian constraints (partial derivatives must commute)
solve this sos-programSOS
PSDH(1,1)
µ
M
26
Messages to take home…
SOS-relaxation is a tractable technique for certifying positive semidefiniteness of scalar or matrix polynomials
We specialized to convexity and sos-convexity
Three natural notions for sos-convexity are equivalent
Not always exact
But very powerful (at least for low degrees and dimensions)
Proposed a convex relaxation to search over a restricted family of psd polynomials that are not sos
Open: what’s the complexity of deciding convexity?
Our result further supports the hypothesis that it must be a hard problem
27
• Want to know more? Preprint at http://arxiv.org/abs/0903.1287
Questions?