Approximability& Sums of Squares
Ryan O’DonnellCarnegie Mellon
Basic Optimization Problems
Minimum-Balanced-Separator:
Given G=(V,E), partition V into 2 parts,each of size at least n/3,
minimize # of edges crossing partition.
Basic Optimization Problems
Minimum-Balanced-Separator:
Minimum-Vertex-Cover:
Given G=(V,E), partition V into 2 parts,each of size at least n/3,
minimize # of edges crossing partition.
Given G=(V,E), choose the smallestsubset S ⊆ V such that each edge touches S.
Both are NP-hard
n-vbl 3SATformula F
O(n)-vtxgraph G
poly(n) time
, β
F satisfiable
F unsatisfiable
⇒
⇒
Min-BS(G) = β
Min-BS(G) > β
Distinguishing requires* at least
2Ω(n) time.⇒
Distinguishing requires* at least
2Ω(n) time.
Approximate Optimization
“C-approximation algorithm”
Guaranteed to find a solution with value at most C times the minimum.
“C-certification algorithm”
• Output form: “I certify the minimum is ≥ α”.• Must always be correct.• Guaranteed that α ≥ (true minimum) / C.
Stronger
Minimum Balanced-Separator
Is there a 1.01-approximationalgorithm running in O(n) time?
Is there a 10000-certification
algorithm running in 2n.99 time?
DON’TKNOW
DON’TKNOW
[AMS’11]: Cannot* 1.0000000000000001-certify in poly(n) time.
Minimum Vertex-Cover
Can 2-approximate in linear time.
Cannot* 1.17-certify even in 2n.99999 time.
Cannot* 1.36-certify even in 2n.000001 time.
Can you 1.5-certify in polynomial time?
DON’TKNOW
How could you show that you can’t 1.5-certify Min-VC in poly time?
n-vbl 3SATformula F
O(n)-vtxgraph G
poly(n) time
, β
F satisfiable
F unsatisfiable
⇒⇒
Min-VC(G) ≤ β
Min-VC(G) > 1.5 β
This would show 1.5-certifying Min-VCrequires* superpolynomial time.
DON’TKNOWHOW
How could you show that you can’t 1.5-certify Min-VC in poly time?
give evidence that
Show that known powerful poly-timeoptimization techniques fail to do it.
Prehistory: Linear programming can’t 1.999999-certify Min-VC.
[ABL’02]: Lovász-Schrijverd Super-LP
can’t 1.999999-certify Min-VC.
[GK’95]: Semidefinite programming can’t 1.999999-certify Min-VC.
[GMPT’07]: Lovász-Schrijverd Super-SDP
can’t 1.999999-certify Min-VC.
[BCGM’10]: Sherali-Adamsd Super-Duper-SDP
can’t 1.999999-certify Min-VC.
+++
nO(d) time
[KS’09], [RS’09]: Sherali-Adamsd Super-Duper-SDP
can’t 10000-certify Min-Bal-Sep.
For Min-Balanced-Separator, a similar situation:
Prehistory: Linear programming can’t 1.999999-certify Min-VC.
I.e., there are graphs G on n vertices such that:
• Min-VC(G) ≥ .999999 n• LP(G) = “I certify Min-VC(G) ≥ .500001 n”
α = minimize: ∑v∈V Xv
subject to: Xv ∈ {0,1} for all v∈V
Xu + Xv ≥ 1 for all (u,v)∈E
[0,1]
LP certif. alg. for Min-VC outputs α, where
I.e., there are graphs G on n vertices such that:
• Min-VC(G) ≥ .999999 n
• SAd(G) = “I certify Min-VC(G) ≥ .500001 n”
[BCGM’10]: Sherali-Adamsd Super-Duper-SDP
can’t 1.999999-certify Min-VC.
Specifically, this is true for “Frankl-Rödl graphs” [FR’87]:
V = {0,1}m, E = {(x,y) : ∆(x,y)=.999 m}
I.e., there are graphs G on n vertices such that:
• Min-BS(G) ≥ β
• SAk(G) = “I certify Min-BS(G) ≥ ”.
Specifically, this is true for “Khot-Vishnoi graphs” [KV’05].
[KS’09], [RS’09]: Sherali-Adamsd Super-Duper-SDP
can’t 10000-certify Min-Bal-Sep.
These are tough instances.
We, the mathematicians, can analyze their opt.
But our strongest poly-time algorithms cannot.
Actually…
There is one more algorithm…
It’s even stronger, but hard to analyze…
The “Lasserred Super-Duper-Ultra-SDP”…
Also known as…
SOSd
nO(d) time
Our Results
[OZ’13]: SOS4 is a C-certification algorithm
(for some small C, maybe 5)for Min-BS on Khot-Vishnoi graphs.
SOSd is also pretty good for Max-Cut
on Khot-Vishnoi graphs.
[KOTZ’13]: SOSd is essentially a 1-certif.
alg. for Min-VC on all but the‘hardest’ Frankl-Rödl graphs.
So your whole result is that
one particular algorithm
does well on one particular
instance?
An Old Joke
Q: Why did the complexity theorist work on algorithms?
A: To get lower bounds on his lower bounds.
SOSd is a dozen years old, but hard to analyze.
The Dream: it’s great certification alg. not justfor these known hard graphs, but for all graphs.
Our Inspiration:
STOC’12 paper of Barak, Brandão, Harrow, Kelner, Steurer, and Zhou.
• Showed SOS4 is good certification alg.
on known hard instances of “Unique-Games”.
• Somewhat demystified analysis of SOSd.
So what is SOSd?
“Min-Balanced-Separator(G) > α”
⇔
has no real solutions”
“
infeasibility certificate:
identity −1 = Q0 + Q1P1 + Q2P2+ ••• +QmPm
where each Qi is a “sum of squares”:
Qi = Ri12 + ••• + Rik
2
Positivstellensatz
Subject to some mild technical conditions,every infeasible system has such a certificate.
Caveat: Qi’s might need to have high degree.
SOSd algorithm: [Shor’87,Lasserre’00,Parrilo’00]
If there exists an infeasibility certificate
where all the Qi’s have degree ≤ d,
finds it in time nO(d).
E.g.: SOSd for Min-VC(G)
“Min-VC(G) > α” ⇔
Xv2 = Xv for all v∈V,
Xu+Xv ≥ 1 for all (u,v)∈E,
∑v Xv ≤ α
infeasible
−1 = Q0 + Q1 (α−∑ Xv) + ∑ Quv (Xu+Xv−1) + •••
existence of sum-of-squares Q’s such that
Find largest α such that degree-d Q’s exist.
⇐
Our Results
[OZ’13]: SOS4 is a C-certification algorithm
(for some small C, maybe 5)for Min-BS on Khot-Vishnoi graphs.
I.e., for Khot-Vishnoi graphs G, there are degree-4 SOS Q’s certifying
“Min-Bal-Sep(G) > α” for some α > (true Min-Bal-Sep) / C.
One Slide How-ToThm: Min-VC in this graph is ≥ .999nProof: … vertex isoperimetry…
… inductive argument…
Thm: Min-BS in this graph is ≥ blahProof: … hypercontractivity…
“Check out these polynomials.”
“Check out these polynomials.”
Tiny Taste
A bit of the analysis for Max-Cut:
Lemma: Let a,b,c ∈ {−1,1}. If a ≠ c then either a ≠ b or b ≠ c.
Formalization with polynomials:
SOS Proof:
Open Problems
Can you give an SOS proof of…
• Vertex Isoperimetric Theorem in {0,1}n:
If A, B ⊆ {0,1}n, |A|,|B| ≥ .1·2n,
then ∃x∈A,y∈B with ∆(x,y) ≤
• Central Limit Theorem
Thanks!
+