Recent Progress in Approximability

Administrivia

Most agreeable times:

Monday 2:30-4:00Wednesday 4:00-5:30Thursday 4:00-5:30Friday 1:00-2:30

Please Fill Up Survey: http://www.surveymonkey.com/s/9TSVQM7

Evaluation: 6-8 short homeworks and class participation.

Max Cut

10

15

3

7

11

Max CUTInput: A weighted graph G

Find:A Cut with maximum number/weight of crossing edges

Fraction of crossing edges

MaxCut is NP-complete

(Karp’s original list of 21 NP-complete problems (1971)

An algorithm A is an α-approximation for a problem if for every instance I,

A(I) ≥ α OPT(I)∙

--Vast Literature--

Approximation Algorithms

Max Cut

10

15

3

7

11

Max CUTInput: A weighted graph G

Find:A Cut with maximum number/weight of crossing edges

Trivial ½ Approximation

Assign each vertex randomly to left or right side of the cutAnalysis

For every edge e,

Probability[edge is cut] = ½

Fraction of edges cut = ½

Optimum MaxCut < 1

So,

Solution returned = ½ > ½ *Optimum MaxCut

Till 1994, this was the state of the art.

Many linear programming techniques were known to NOT get any better approximation.

The ToolsTill 1994,A majority of approximation algorithms directly or indirectly relied on Linear Programming.

In 1994,Semidefinite Programming based algorithm for Max Cut

[Goemans-Williamson]

Semidefinite Programming - A generalization of Linear Programming.

Semidefinite Programming is the one of the most powerful tools in approximation algorithms.

Eji

jiij vvw),(

2||4

1

Semidefinite Program

Variables : v1 , v2 … vn

| vi |2 = 1

Maximize

Max Cut SDP

Quadratic Program

Variables : x1 , x2 … xn

xi = 1 or -1

Maximize

10

15

3

7

11

1

1

1

-1

-1

-1

-1-1

-1

Eji

jiij xxw),(

2)(4

1

Relax all the xi to be unit vectors instead of {1,-1}. All products are replaced by inner products of vectors

1 -1

Semidefinite Program:[Goemans-Williamson 94]

Embedd the graph on the N - dimensional unit ball, Maximizing

¼ (Average Squared Length

of the edges)

Eji

jiij vvw),(

2||4

1

Semidefinite Program[Goemans-Williamson 94]

Variables : v1 , v2 … vn

|vi|2 = 1

Maximize

MaxCut

10

15

3

7

11

1

1

1

-1

-1

-1

-1-1

-11 -1

Max Cut ProblemGiven a graph G,Find a cut that maximizes the number of crossing edges

v1

v2

v3

v4

v5

MaxCut Rounding

v1

v2

v3

v4

v5

Cut the sphere by a random hyperplane, and output the induced graph cut.

-A 0.878 approximation for the problem.

[Goemans-Williamson]

Analysisv1

v2v3

v4

v5

SDP Optiumum

1015

3

711

OptimalMaxCut

v1

v2v3

v4

v5

Algorithm’sOutput

0 1

Rounding Ratio > 0.878

Integrality Gap

Algorithm Output > 0.878 X SDP Optimum > 0.878 X Optimum MaxCut

minimumover all instances

=

value of rounded solution

value of SDP solution

rounding – ratioA

(approximation ratio)≤ integrality gap

=

value of optimal solution

value of SDP solution

minimumover all instances

For any rounding algorithm A, and a SDP relaxation ¦

v1v2 v3

v4v5

SDP Optiumum

10153

711OptimalMaxCut

v1v2 v3

v4v5

Algorithm’sOutput

0 1

Rounding Ratio > 0.878

Integrality Gap

=“algorithm achieves the gap’’

InapproximabilityIs 0.878 the best possible approximation ratio for MaxCut?

Satisfiable

Unsatisfiable

MaxCut value = K

MaxCut value < K

10

15

3

7

11

1

1

1

-1

-1

-1-1

3-SAT InstancePolynomial time

reduction

What we need..

(Completeness)

Satisfiable

(Soundness)

Unsatisfiable

MaxCut value = K

MaxCut value < 0.9K

10

15

3

7

11

1

1

1

-1

-1

-1-1

3-SAT InstancePolynomial time

reduction

If we had a polytime 0.95 approximation algorithm for MaxCut

A polytime algorithm for 3-SAT

A probabilistically checkable proof (PCP)

Goal: Alice wants to prove to Bob that 3-SAT instance A is satisfiable

3-SAT Instance A

Alex Bob (polytime machine)Satisfying assignment

A probabilistically checkable proof (PCP)

Goal: Alice wants to prove to Bob that 3-SAT instance A is satisfiable

3-SAT Instance A

Alex Bob (polytime machine)

10

15

3

7

11

1

1

1

-1

-1

-1-1

Polynomial

time

reduction

3-SAT Instance A

10

15

3

7

11

1

1

1

-1

-1

-1-1

Polynomial

time

reduction

Probabilistically Checkable ProofA cut of value > 0.9

Verifier (Bob):

Sample a random edge in graph,

Accept if edge is cut.

Prob[Bob Accepts] =

Value of the Cut

Suppose,

(Completeness)

Satisfiable(Soundness)

Unsatisfiable

MaxCut value = 0.99

MaxCut value < 0.9

10

15

3

7

11

1

1

1

-1

-1

-1-1

3-SAT InstancePolynomial time

reduction

Completeness: There exists a ``proof” that Bob accepts with probability 0.99Soundness: No matter what Alex does, Bob accepts with probability < 0.9

Bob reads only 2 bits of the proof!!

Analogy to Math Proofs

Could you check the proof of a theorem with any reasonable confidence by reading only 3 bits of the proof???

Guess: Probably Not..

Max-SNP complexity class was defined, because it was believable that

we will never be able to get a Gap Reduction aka Probabilistically Checkable Proof for NP.

PCP Theorem: [Arora-Lund-Motwani-Sudan-Szegedy 1991]

Max-3-SAT is NP-hard to approximate better than 1- 10^{-100}.

Corollary:Max-Cut is NP-hard to approximate better than 1- 10^{-200}.

Long and very difficult proof, simplified over the years..(*Check out History of PCP Theorem:

http://www.cs.washington.edu/education/courses/cse533/05au/pcp-history.pdf)

Completely new proof by Irit Dinur in 2005.

Hastad’s 3-Query PCP [Håstad STOC97]

For any ε > 0, NP has a 3-query probabilistically checkable proof system such that:

• Completeness = (1 – ε) • Soundness = 1/2 + ε

Verifier reads only 3-bits, and checks a linear equation on them!

Xi + Xj = Xk + c (mod p)

Alternately,

Hastad’s 3-Query PCP [1997]

For any ε > 0, given a set of linear equations modulo 2 , it is NP-hard to distinguish between:

• (1 – ε) – fraction of the equations can be satisfied.• 1/2 + ε – fraction of the equations can be satisfied.

All equations are of the form Xi + Xj = Xk + c (mod p)

By Very Clever Gadget reductions, [Sudan-Sorkin-Trevisan-Williamson]

MaxCut is NP-hard to approximate beyond 0.94.

ALGORITHMS[Charikar-Makarychev-Makarychev 06]

[Goemans-Williamson][Charikar-Wirth]

[Lewin-Livnat-Zwick][Charikar-Makarychev-Makarychev 07]

[Hast] [Charikar-Makarychev-Makarychev 07]

[Frieze-Jerrum][Karloff-Zwick]

[Zwick SODA 98][Zwick STOC 98]

[Zwick 99][Halperin-Zwick 01]

[Goemans-Williamson 01][Goemans 01]

[Feige-Goemans][Matuura-Matsui]

[Trevisan-Sudan-Sorkin-Williamson]

Approximability of CSPsGap for MaxCUTAlgorithm = 0.878Hardness = 0.941

MAX CUT

MAX 2-SAT

MAX 3-SAT

MAX 4-SAT

MAX DI CUT

MAX k-CUT

Unique GamesMAX k-CSP

MAX Horn SAT

MAX 3 DI-CUTMAX E2 LIN3

MAX 3-MAJ

MAX 3-CSPMAX 3-AND

0 1

NP HARD

Given linear equations of the form:

Xi – Xk = cik mod p

Satisfy maximum number of equations.

x-y = 11 (mod 17)x-z = 13 (mod 17)

…….

z-w = 15(mod 17)

Unique Games Conjecture [Khot 02] [KKMO]

For every ε> 0, for large enough p,Given : 1-ε (99%) satisfiable system,

NP-hard to satisfyε (1%) fraction of equations.

Towards bridging this gap, In 2002, Subhash Khot introduced the

Unique Games Conjecture

Unique Games Conjecture

A notorious open problem.

Hardness Results: No constant factor approximation for unique games. [Feige-Reichman]

Algorithm On (1-Є) satisfiable instances

[Khot 02]

[Trevisan]

[Gupta-Talwar] 1 – O(ε logn)

[Charikar-Makarychev-Makarychev]

[Chlamtac-Makarychev-Makarychev]

[Arora-Khot-Kolla-Steurer-Tulsiani-Vishnoi]

)2/( p)loglog(1 pnO

)log(1 3 nO

))/1log((1 5/12 pO

1log1

Assuming UGCUGC Hardness

Results[Khot-Kindler-Mossel-O’donnell]

[Austrin 06][Austrin 07]

[Khot-Odonnell][Odonnell-Wu]

[Samorodnitsky-Trevisan]

NP HARDUGC HARD

0 1

MAX CUT

MAX 2-SAT

MAX 3-SAT

MAX 4-SAT

MAX DI CUT

MAX k-CUT

Unique GamesMAX k-CSP

MAX Horn SAT

MAX 3 DI-CUTMAX E2 LIN3

MAX 3-MAJ

MAX 3-CSPMAX 3-ANDFor MaxCut, Max-2-SAT,

Unique Games based hardness=

approximation obtained by Semidefinite programming!

The Connection

MAX CUT

MAX 2-SAT

MAX 3-SAT

MAX 4-SAT

MAX DI CUT

MAX k-CUT

Unique GamesMAX k-CSP

MAX Horn SAT

MAX 3 DI-CUTMAX E2 LIN3

MAX 3-MAJ

MAX 3-CSPMAX 3-AND

UGC Hard

GENERICALGORITHM

Theorem:Assuming Unique Games Conjecture, For every CSP, “the simplest semidefinite programs give the best approximation computable efficiently.”

Constraint Satisfaction Problems [Raghavendra`08][Austrin-Mossel]

MAX CUT [Khot-Kindler-Mossel-ODonnell][Odonnell-Wu]

MAX 2SAT [Austrin07][Austrin08]

Ordering CSPs [Charikar-Guruswami-Manokaran-Raghavendra-Hastad`08]

MAX ACYCLIC SUBGRAPH, BETWEENESS

Grothendieck Problems [Khot-Naor, Raghavendra-Steurer]

Metric Labeling Problems [Manokaran-Naor-Raghavendra-Schwartz`08]

MULTIWAY CUT, 0-EXTENSION

Kernel Clustering Problems [Khot-Naor`08,10]

Strict Monotone CSPs [Kumar-Manokaran-Tulsiani-Vishnoi`10]

VERTEX COVER [Khot-Regev], HYPERGRAPH VERTEX COVER

Assuming the Unique Games Conjecture,

A simple semidefinite program (Basic-SDP) yields the optimal approximation ratio for

Is the conjecture true?

Many many ways to disprove the conjecture! Find a better algorithm for any one of these problems.

The UG Barrier

Constraint Satisfaction Problems

Graph Labelling Problems

Ordering CSPs

Kernel Clustering Problems

Monotone Min-One CSPs

UGC HARD

If UGC is true,

Then Simplest SDPs give the best approximation possible.

If UGC is false,

Hopefully, a new algorithmic technique will arise.

What if UGC is false?

Could existing techniques ( LPs/SDPs) disprove the UGC?

What if UGC is false?

UGC is false New algorithms?

Unique Games

Constraint Satisfaction Problems [Raghavendra`08]MAX CUT, MAX 2SAT

Ordering CSPs [GMR`08]MAX ACYCLIC SUBGRAPH, BETWEENESS

Grothendieck Problems [KNS`08, RS`09]

Metric Labeling Problems [MNRS`08]MULTIWAY CUT, 0-EXTENSION

Kernel Clustering Problems [KN`08,10]

Strict Monotone CSPs [KMTV`10]VERTEX COVER, HYPERGRAPH VERTEX COVER

…

Problem X

UGC is false New algorithm for Problem X

Despite considerable efforts,No such reverse reduction known for any of the above problems

[Feige-Kindler-Odonnell,Raz’08, BHHRRS’08]

Graph Expansion

d-regular graph G

d

expansion(S) = # edges leaving S

d |S|

vertex set S

A random neighbor of a random vertex in S is outside of S with probability expansion(S)

ФG = expansion(S)minimum|S| ≤ n/2

Conductance of Graph G

Uniform Sparsest Cut Problem Given a graph G compute ФG and find the set S achieving it.

Approximation Algorithms:

•Cheeger’s Inequality [Alon][Alon-Milman] Given a graph G, if the normalized adjacency matrix has second eigen value λ2 then,

•A log n approximation algorithm [Leighton-Rao 98-99?].•A sqrt(log n) approximation algorithm using semidefinite programming [Arora-Rao-Vazirani 2004].

)1(22

)1(2

2

G

Extremely well-studied, many different contexts

pseudo-randomness, group theory, online routing,

Markov chains, metric embeddings, …

A Reverse Reduction

Graph (Social Network)

Close-knitcommunity

Finding Small Non Expanding Sets

Suppose there exists is a small community say

(0.1% of the population)

99% of whose friends are within the community..

Find one such close-knit community.

Theorem [R-Steurer 10]UGC is false New algorithms to approximate expansion of small sets in graphs

STILL OPEN:

Reverse reduction from Max Cut or Vertex Cover to Unique Games.

What if UGC is false?

Could existing algorithmic techniques (LPs/SDPs) disprove the UGC?

Could LPs/SDPs disprove the UGC?

Question I:

Could some small LINEAR PROGRAM give a better approximation for MaxCut or Vertex Cover

thereby disproving the UGC?

Probably Not!

[Charikar-Makarychev-Makarychev][Schoenebeck-Tulsiani]

For MaxCut, for several classes of linear programs,

exponential sized linear programs are necessary to even beat the trivial ½ approximation!

Question II:

Could some small SEMIDEFINITE PROGRAM give a better approximation for MaxCut or Vertex Cover

thereby disproving the UGC?

We don’t know.

v1

v2

v3

v4

v5

Max Cut SDP:

Embedd the graph on the N - dimensional unit ball,

Maximizing

¼ (Average squared length of

the edges)

In the integral solution, all the vectors vi are 1,-1. Thus they satisfy additional constraintsFor example : (vi – vj)2 + (vj – vk)2 ≥ (vi – vk)2

(the triangle inequality)

The Simplest Relaxation for

MaxCut

Does adding triangle inequalities improve approximation ratio?(and thereby disprove UGC!)

[Arora-Rao-Vazirani 2002]

For SPARSEST CUT, SDP with triangle inequalities gives approximation.

An -approximation would disprove the UGC!

[Goemans-Linial Conjecture 1997] SDP with triangle inequalities would yield -approximation for SPARSEST CUT.

[Khot-Vishnoi 2005]

SDP with triangle inequalities DOES NOT give approximation for SPARSEST CUT

SDP with triangle inequalities DOES NOT beat the Goemans-Williamson 0.878 approximation for MAX CUT

Until 2009:

Adding a simple constraint on every 5 vectorscould yield a better approximation for MaxCut, and disproves UGC!

Building on the work of [Khot-Vishnoi],

[Khot-Saket 2009][Raghavendra-Steurer 2009]

Adding all valid local constraints on at most vectors to the simple SDP DOES NOT improve the approximation ratio for MaxCut

[Barak-Gopalan-Hastad-Meka-Raghavendra-Steurer 2009]

Change to in the above result.As of Now:

A natural SDP of size (the round of Lasserre hierarchy) could disprove the UGC.

[Barak-Brandao-Harrow-Kelner-Steurer-Zhou 2012] round of Laserre hierarchy solves all known instances of Unique Games.

Constraint Satisfaction Problems

Max 3 SAT

Find an assignment that satisfies the maximum number of clauses.

))()()(( 145532532321 xxxxxxxxxxxx

VariablesFinite Domain Constraints

{x1 ,x2 , x3 , x4 , x5}{0,1}Clauses

Kind of constraints permitted Different CSPs

Deeper understanding of the UGC – why it should be true if it is.

Why play this game?

Connections between SDP hierarchies, Spectral Graph Theory and Graph Expansion.

New algorithms based on SDP hierarchies.

[Raghavendra-Tan] Improved approximation for MaxBisection using SDP hierarchies

[Barak-Raghavendra-Steurer]

Algorithms for 2-CSPs on low-rank graphs.

New Gadgets for Hardness Reductions:[Barak-Gopalan-Hastad-Meka-Raghavendra-Steurer]

A more efficient long code gadget.