1

Computational Complexity(Ctnd.)

ORF 523Lecture 15

Instructor: Amir Ali AhmadiTA: G. Hall

Spring 2015

2

Open questions in complexity for optimization

3

Open questions in complexity for optimization

4

Open questions in complexity for optimization

5

Open questions in complexity for optimization

6

Convexity

This talk:-- Given a multivariate polynomial, can we efficiently decide if it is convex?

-- Given a basic semialgebraic set, can we efficiently decide if it is a convex set?

“In fact the great watershed in optimization isn't between linearity and nonlinearity, but convexity and nonconvexity.”

Rockafellar, ’93:

But how easy is it to distinguish between convexity and nonconvexity?

7

Data separationwith convex sublevel sets

Convex envelope approximation

Convex fitting

Search over Convex Polynomials: Applications

Other Applications:

- Convex Lyapunov functions in robust control

- Norms defined by polynomials

8

Complexity of deciding convexityInput to the problem: an ordered list of the coefficients (all rational)

Degree d odd: trivial

d=1, always convex

d>1 and odd, never convex

d=2, i.e., p(x)=xTQx+qTx+c : poly time

check if Q is PSD

d=4, first interesting case

Question of N. Z. Shor

9

What we are going to prove:

Thm: Deciding convexity of quartic polynomials is (strongly) NP-hard. This is true even if the polynomials are restricted to be homogeneous (i.e., forms).

10

Sequence of reductions

STABLE SET

Nonconvex QP

Nonnegativity of biquadratic forms

Convexity of quartic forms

CLIQUE

Any problem in NP

Circuit SAT

SAT 3SAT

Cook-Levin Theorem

Saw in class

Flip edges

Motzkin-Strauss Thm.

Next

Next

(straightforward, see [DPV])

(straightforward, see [DPV])

Any problem in NP Circuit SAT (review)

11

The Cook-Levin theorem.

In a way a very deep theorem.

At the same time almost a tautology.

An example of a problem in NP: STABLE SET

12

Sequence of reductions

STABLE SET

Nonconvex QP

Nonnegativity of biquadratic forms

Convexity of quartic forms

CLIQUE

Any problem in NP

Circuit SAT

SAT 3SAT

Cook-Levin Theorem

Saw in class

Flip edges

Motzkin-Strauss Thm.

Next

Next

(straightforward, see [DPV])

(straightforward, see [DPV])

13

NP-hardness of deciding convexity of quartics

Reduction from problem of deciding “nonnegativity of biquadratic forms”

Thm: Deciding convexity of quartic forms is NP-hard.

Biquadratic form:

These are degree-4 polynomials of very special structure.

So our previous proof that testing nonnegativity of quartic (homogeneous) polynomials is NP-hard does not suffice…

14

CLIQUEBiquadratic Nonnegativity

15

Why this problem? And why aren’t we done?

Biquadratic form:

Can write any biquadratic form as

where is a matrix whose entries are quadratic forms

Example: with

This is exactly how the Hessian of a quartic form looks like!

So why aren’t we done?

16

Why aren’t we done?

The Hessian has very special structure!

above is not a valid Hessian:

17

From biquadratic forms to biquadratic Hessian forms

We give a constructive procedure to go from any biquadratic form

to a biquadratic Hessian form

by doubling the number of variables, such that:

In fact, we construct the polynomial f(x,y) that has H(x,y) as its Hessian directly

Let’s see this construction…

18

The main reductionThm: Given any biquadratic form

Let Let

Let

Then

19

Observation on the Hessian of a biquadratic form

20

Proof of correctness of the reductionStart with

Let Let

Claim:

21

Reduction on an instance

A 6x6 Hessian with quadratic form entries

22

Questions for you

Why does the result imply testing convexity is NP-hard also for degree-6 polynomials? (and degree-8, etc.)

(Hint: add an independent variable)

Why does the result imply testing convexity of basic semialgebraic sets (i.e., sets defined by polynomial inequalities) is NP-hard?

(Hint: epigraphs)

23

Other notions of interest in optimizationStrong convexity

“Hessian uniformly bounded away from zero”

Appears e.g. in convergence analysis of Newton-type methods

Strict convexity

“curve strictly below the line”

Guarantees uniqueness of optimal solution

Convexity

Pseudoconvexity

“Relaxation of first order characterization of convexity”

Any point where gradient vanishes is a global min

Quasiconvexity

“Convexity of sublevel sets”

Deciding convexity of basic semialgebraic sets

24

Summary of complexity results

25

QuasiconvexityA multivariate polynomial p(x)=p(x1,…,xn ) is quasiconvex if all its sublevel sets

are convex.

Convexity Quasiconvexity

(converse fails)

Deciding quasiconvexity of polynomials of even degree 4 or larger is (strongly) NP-hard

Quasiconvexity of odd degree polynomials can be decided in polynomial time

26

Quasiconvexity of even degree forms

Proof outline:

A homogeneous quasiconvex polynomial is nonnegative (why?)

The unit sublevel sets of p(x) and p(x)1/d are the same convex set

p(x)1/d is the Minkowski (semi)-norm defined by this convex set and hence a convex function

A convex nonnegative function raised to a power d larger than one remains convex (why?)

Corollaries:-- Deciding quasiconvexity is NP-hard

-- Deciding convexity of basic semialgebraic sets is NP-hard

Lemma: A homogeneous polynomial p(x) of even degree d is quasiconvex if and only if it is convex.

27

Proof:

Show super level sets must also be convex sets

Only convex set whose complement is also convex is a halfspace

Quasiconvexity of odd degree polynomials

This representation can be checked in polynomial time

Thm: The sublevel sets of a quasiconvex polynomial p(x) of odd degree are halfspaces.

Thm: A polynomial p(x) of odd degree d is quasiconvex iffit can be written as

As we’ve seen several times by now…

28

The boundary between the tractable and the intractable is very delicate!

Testing quasiconvexity of odd vs. even degree polynomials

Mincut vs. Maxcut

Testing matrix positive semidefiniteness versus matrix copositivity

Robust stability of interval polynomials versus robust stability of interval matrices

Testing primality versus finding the factors (?)

Many many others…

Let’s recall what we are achieving by an NP-hardness proof

29[Garey, Johnson]

NP-complete problems are everywhere…

30

Today we have thousands of NP-complete problems. In all areas of science and engineering.

The domino effect

31

All NP-complete problems reduce to each other!

If you solve one in polynomial time, you solve ALL in polynomial time!

The $1M question!

32

• Most people believe the answer is NO!• Philosophical reason: If a proof of the Goldbach conjecture were to fly from

the sky, we could certainly efficiently verify it. But should this imply that we can find this proof efficiently? P=NP would imply the answer is yes.

Nevertheless, there are believers too…

33

• Over 100 wrong proofs have appeared so far (in both directions)! See http://www.win.tue.nl/~gwoegi/P-versus-NP.htm

Coping with NP-completeness

34

Solving special cases exactly

Approximation algorithms – suboptimal solutions with worst-case guarantees

At least certifying gap to optimality on an instance-by-instance basis Can easily be done by finding bounds using convex optimization (think e.g., SDP for STABLE SET)

Challenge whether worst-case analysis is the right notion But propose rigorous alternatives

Average-case analysis, provable behavior on random instances, etc.

Heuristics OK if you are truly solving a real-life problem

Easy for practitioners to become overly optimistic about their performance

On what fraction of the input instances do you think a poly-time heuristic should fail? Just a few contrived examples that theoreticians can construct?

Constant fraction?

Linearly many?

Polynomially many?

Results in complexity theory from the 80s and 90s: even the last answer would imply P=NP

Main messages…

35

Computational complexity theory beautifully classifies many problems of optimization theory as easy or hard

At the most basic level, when faced with a new optimization problem, you should try to prove membership to the class P or establish NP-hardness.

After that, you can still study the complexity of your problem in further detail (e.g., quality/hardness of approximation)

Complexity theory is important to the theory of convex optimizationArguably the most rigorous tool we have for proving that certain problems do not have an efficient convex reformulation

The boundary between tractable and intractable problems is very delicate:

MINCUT vs. MAXCUT, PSD-ness vs. copositivity, LP vs. IP, ...

P=NP?

Maybe one of you guys will tell us one day.

References

36

- NP-hardness of testing convexity: http://web.mit.edu/~a_a_a/Public/Publications/convexity_nphard.pdf

- NP-hardness of testing nonnegativity of biquadratic forms: http://www.math.ucsd.edu/~njw/PUBLICPAPERS/LNQY_july02.pdf

- Survey on performance of heuristic algorithms: http://arxiv.org/abs/1210.8099v1

- [DPV08] S. Dasgupta, C. Papadimitriou, and U. Vazirani. Algorithms. McGraw Hill, 2008.

- [GJ79] D.S. Johnson and M. Garey. Computers and Intractability: a guide to the theory of NP-completeness, 1979.