33
1
11
22
IntroductionIntroduction
In this lecture we’ll cover:Definition of PCP Prove some classical hardness of approximation results
Review some recent ones
33
Review: Decision, Review: Decision, Optimization ProblemsOptimization Problems
A A decisiondecision problem problem is is
a a Boolean Boolean function function ƒ(X)ƒ(X), or alternatively, or alternatively
a a languagelanguage L L {0, 1} {0, 1}** comprising all strings for comprising all strings for which which ƒƒ is is TRUETRUE:: L = { X L = { X {0, 1} {0, 1}* * | ƒ(X) }| ƒ(X) }
An An optimizationoptimization problem problem is is
a function a function ƒ(X, Y)ƒ(X, Y) which, given which, given XX, is to be maximized , is to be maximized (or minimized) over all possible (or minimized) over all possible YY’s: ’s: maxmaxyy[ ƒ(X, Y) ][ ƒ(X, Y) ]
A A thresholdthreshold version of version of max-ƒ(X, Y)max-ƒ(X, Y) is is
the the languagelanguage LLtt of all strings of all strings XX for which there exists for which there exists YY such that such that ƒ(X, Y) ƒ(X, Y) t t
[transforming an optimization problem into [transforming an optimization problem into decision]decision]
44
Review: The Class NPReview: The Class NP
The classical definition of the class The classical definition of the class NPNP is: is:
A language A language L L {0, 1} {0, 1}* * belongsbelongs to the class to the class NPNP, if there exists a Turing machine , if there exists a Turing machine VVLL [referred to as a [referred to as a verifierverifier] ] such thatsuch that
X X L L there exists a there exists a witnesswitness YY such that such that VVLL(X, Y)(X, Y) acceptsaccepts, in time , in time |X||X|O(1)O(1)
That is, That is, VVLL can verify a membership-proof of can verify a membership-proof of XX in in L L in time polynomial in the length ofin time polynomial in the length of X X
55
Review: NP-HardnessReview: NP-Hardness
A language A language LL is said to be is said to be NP-hardNP-hard if an if an efficient (polynomial-time) procedure for efficient (polynomial-time) procedure for LL can be utilized to obtain an efficient can be utilized to obtain an efficient procedure for any procedure for any NP-languageNP-language
This definition allows efficient reduction This definition allows efficient reduction that use the more general, that use the more general, CookCook reduction. An efficient algorithm, reduction. An efficient algorithm, translating any translating any NPNP problem to a single problem to a single instance of instance of LL - thereby showing that - thereby showing that LL NP-hardNP-hard - is referred to as - is referred to as KarpKarp reduction reduction
66
Review: Characterizing NPReview: Characterizing NP
ThmThm [Cook,Levin][Cook,Levin]:: For For LL NP NP there’s an algorithm there’s an algorithm that, on input that, on input XX, constructs, in time , constructs, in time |X||X|O(1)O(1), a set , a set of of local-constraintslocal-constraints (Boolean functions) (Boolean functions)
L,X L,X = { = { 11ll } }
over variables over variables yy11,...,y,...,ymm s.t.: s.t.:
1.1. each of each of 11ll depends on depends on o(1)o(1) variables variables
2.2. X X L L there exists an assignment there exists an assignment A: { yA: { y11, ..., y, ..., ym m } } { 0, 1 } { 0, 1 } satisfying all satisfying all L,XL,X
[ note that [ note that mm and and ll must be at most polynomial in must be at most polynomial in |X||X| ] ]
77
NP characterizationNP characterization
y1y1
y2y2
yiyi
ym-1ym-1
ymym
1111
jjjj
llll
TT
TT
TT
TT
TT
FF
TT
FF
TT
TT
FF
TT
FF
TT
If X X L, L, all of
the local tests aresatisfied
88
Approximation - Some Approximation - Some DefinitionsDefinitions
DefDef: : gg-approximation-approximation
A A g-approximationg-approximation of a maximization (similar of a maximization (similar for minimization) function for minimization) function ff, is an algorithm , is an algorithm that on input that on input XX, outputs , outputs f’(X)f’(X) such that: such that:
f’(X) f’(X) f(X)/g(|X|). f(X)/g(|X|).
DefDef: : PTASPTAS (poly-time approximation scheme) (poly-time approximation scheme)
We say that a maximization function We say that a maximization function ff, has a , has a PTASPTAS, if for every , if for every gg, there is a polynomial , there is a polynomial ppgg and a and a g-approximationg-approximation for for ff, whose running , whose running time is time is ppgg(|X|)(|X|)
99
Approximation - NP-hardApproximation - NP-hard?? We know that by using Cook/Karp reductions, we We know that by using Cook/Karp reductions, we
can show many decision problems to be can show many decision problems to be NP-hardNP-hard..
Can an approximation problem be Can an approximation problem be NP-HardNP-Hard??
One can easily show, that if there is One can easily show, that if there is gg,for which ,for which there is a there is a g-approximating g-approximating for for TSPTSP, , P=NPP=NP..
1010
Characterization of NPCharacterization of NP
ThmThm [Cook,Levin][Cook,Levin]:: For For LL NP NP there’s an there’s an algorithm that, on input algorithm that, on input XX, constructs, in time , constructs, in time |X||X|O(1)O(1), a set of , a set of local-constraintslocal-constraints (Boolean (Boolean functions)functions)
L,X L,X = { = { 11ll } }
over variables over variables yy11,...,y,...,ymm s.t.: s.t.:
1.1. each of each of 11ll depends on depends on o(1)o(1) variables variables
2.2. X X L L there exists an assignment there exists an assignment A: { yA: { y11, ..., y, ..., ym m } } { 0, 1 } { 0, 1 } satisfying all satisfying all L,XL,X
AS,ALMSS
PCP
X X L L assignment assignment A: { y1, ..., ym } A: { y1, ..., ym } { 0, 1 } { 0, 1 } satisfies satisfies < ½ < ½ fraction of fraction of L,XL,X
1111
PCP NP characterizationPCP NP characterization
y1y1
y2y2
yiyi
ym-1ym-1
ymym
1111
jjjj
llll
TT
FF
TT
FF
TT
If X X L, L, at least
half of the local tests aren’t
satisfied !
1212
Probabilistically Checkable Probabilistically Checkable ProofsProofs
Hence, Hence, Cook-LevinCook-Levin theorem states that a verifier theorem states that a verifier can efficiently verify membership-proofs for any can efficiently verify membership-proofs for any NPNP language language
PCPPCP characterization of characterization of NPNP, in contrast, states , in contrast, states that a membership-proof can be verified that a membership-proof can be verified probabilisticallyprobabilistically by choosing randomly one by choosing randomly one local-constraintlocal-constraint,, accessing the accessing the small setsmall set of variables it depends of variables it depends
on,on, accept or reject accordinglyaccept or reject accordingly
erroneouslyerroneously accepting a non-member only accepting a non-member only with with small probabilitysmall probability
1313
Gap ProblemsGap Problems
A A gap-problemgap-problem is a maximization is a maximization (or (or minimization) minimization) problem problem ƒ(X, Y)ƒ(X, Y), and two , and two thresholds thresholds tt11 > t > t22
XX must be accepted if must be accepted if maxmaxYY[ ƒ(X, Y) ] [ ƒ(X, Y) ] t t11
XX must be rejected if must be rejected if maxmaxYY[ ƒ(X, Y) ] [ ƒ(X, Y) ] t t22
other other XX’s may be accepted or rejected ’s may be accepted or rejected (don’t care)(don’t care)
(almost a decision problem, relates to (almost a decision problem, relates to approximation)approximation)
1414
Reducing gap-Problems to Reducing gap-Problems to Approximation ProblemsApproximation Problems
Using an efficient approximation algorithm for Using an efficient approximation algorithm for ƒ(X, Y)ƒ(X, Y) to within a factor to within a factor gg,,one can efficiently solve the corresponding one can efficiently solve the corresponding gap problem gap problem gap-ƒ(X, Y)gap-ƒ(X, Y), as long as , as long as tt1 1 // tt2 2 > g> g22
Simply run the approximation algorithm.Simply run the approximation algorithm.The outcome clearly determines which side of The outcome clearly determines which side of the gap the given input falls in.the gap the given input falls in.
(Hence, proving a gap problem (Hence, proving a gap problem NP-hardNP-hard translates to its approximation version, for translates to its approximation version, for appropriate factors )appropriate factors )
1515
gap-SAT
DefDef: gap-SAT[D, v, : gap-SAT[D, v, ]] is as follows: is as follows: InstanceInstance: a set : a set = { = { ll } } of Boolean-of Boolean-
functions functions (local-constraints)(local-constraints) over variables over variables yy11,...,y,...,ymm of of range range 22VV
LocalityLocality: each of : each of 11ll depends on at most depends on at most DD variablesvariables
Maximum-Satisfied-FractionMaximum-Satisfied-Fraction is the fraction of is the fraction of satisfied by an assignment satisfied by an assignment A: { yA: { y11, ..., y, ..., ym m } } 2 2vv
if this fraction if this fraction
= 1= 1 accept accept
<< reject reject DD, , vv and and may be a function of may be a function of l l
1616
The PCP HierarchyThe PCP Hierarchy
DefDef:: L L PCP PCP[ [ DD, , VV, , ] ] if if LL is efficiently reducible to is efficiently reducible to gap-SATgap-SAT[ [ DD, , VV, , ] ]
ThmThm [AS,ALMSS] [AS,ALMSS] NP NP PCP[ O(1), 1, ½] PCP[ O(1), 1, ½] [ The PCP characterization theorem above ][ The PCP characterization theorem above ]
ThmThm [ RaSa ] [ RaSa ] NP NP PCP[ O(1), m, 2 PCP[ O(1), m, 2-m-m ] ] for for m m loglogcc n n for some for some c > 0c > 0
ThmThm [ DFKRS ][ DFKRS ] NP NP PCP[ O(1), m, 2 PCP[ O(1), m, 2-m-m ] ] for for m m loglogcc n n for any for any c < 1c < 1
ConjectureConjecture [BGLR][BGLR] NP NP PCP[ O(1), m, 2 PCP[ O(1), m, 2-m -m ] ] for for m m log n log n
1717
Optimal CharacterizationOptimal Characterization
One cannot expect the error-probability to be less One cannot expect the error-probability to be less than exponentially small in the number of bits each than exponentially small in the number of bits each local-test looks atlocal-test looks at since a random assignment would make such a since a random assignment would make such a
fraction of the local-tests satisfiedfraction of the local-tests satisfied One cannot hope for smaller than polynomially One cannot hope for smaller than polynomially
small error-probabilitysmall error-probability since it would imply less than one local-test since it would imply less than one local-test
satisfied, hence each local-test, being rather easy satisfied, hence each local-test, being rather easy to compute, determines completely the outcometo compute, determines completely the outcome
[ the BGLR conjecture is hence optimal in that [ the BGLR conjecture is hence optimal in that respect]respect]
1818
Approximating MAX-IS is NP-hardApproximating MAX-IS is NP-hard
We will reduce We will reduce gap-SATgap-SAT to to gap –Independent-Set. gap –Independent-Set.
Given an expression Given an expression = { = { ll } } of Boolean-of Boolean-functions over variables functions over variables yy11,...,y,...,ymm of range of range 22VV, Each , Each of of 11ll depends on at most depends on at most DD variables, we variables, we must determine whether all the functions can be must determine whether all the functions can be satisfied or only a fraction less than satisfied or only a fraction less than ..
We will construct a graph, We will construct a graph, GG , that has an , that has an independent set of size independent set of size rr there exists an there exists an assignment, satisfying assignment, satisfying r r of the local-constraints of the local-constraints yy11,...,y,...,ymm..
1919
((q,rq,r-)-)co-partite Graph co-partite Graph G=(QG=(QR, R, E)E)
Comprise Comprise q=|Q|q=|Q| cliques of size cliques of size r=|R|r=|R|:: E E {(<i,j{(<i,j11>, <i,j>, <i,j22>) | i>) | iQ, jQ, j11,j,j2 2 R}R}
2020
Gap Independent-SetGap Independent-Set
InstanceInstance:: an an (q,r)(q,r)-co-partite -co-partite graph graph G=(qG=(qR, E)R, E)
ProblemProblem:: distinguish distinguish betweenbetween GoodGood: : IS(G) = qIS(G) = q BadBad: every set : every set I I V V s.t. s.t. |I||I|
> > qq contains an edge contains an edge
ThmThm:: IS( r, IS( r, ) ) is NP-hard as long as is NP-hard as long as r r ( 1 / ( 1 / ))cc for some constantfor some constant c c
2121
gap-SAT gap-SAT gap-IS gap-IS
y1y1
y2y2
yiyi
ym-1ym-1
ymym
1111
jjjj
llll
TT
TT
TT
TT
TT
FF
TT
FF
TT
ll
Construct a graphConstruct a graph G G that has that has 11 clique clique i i ,,in which in which 11 vertex vertex satisfying assignment for satisfying assignment for ii
2222
gap-SAT gap-SAT gap-IS gap-IS
y1y1
y2y2
yiyi
ym-1ym-1
ymym
1111
jjjj
llll
TT
TT
TT
TT
TT
FF
TT
FF
TT
ll
Two vertices are connected if the assignments Two vertices are connected if the assignments they represent are inconsistentthey represent are inconsistent
2323
LemmaLemma::
(G(G) = k) = k (independent set of size k) (independent set of size k) X X L L
(There is an assignment that satisfies k clauses) (There is an assignment that satisfies k clauses) Consider an assignment Consider an assignment AA satisfying k clauses. For each satisfying k clauses. For each
clause clause ii consider consider AA's restriction to 's restriction to ii‘s variables ‘s variables
The corresponding The corresponding kk vertexes form an independent set vertexes form an independent set
in in GG
Any independent set of size Any independent set of size kk in in GG implies an implies an
assignment satisfying assignment satisfying kk of of 11ll
gap-SAT gap-SAT gap-IS gap-IS
Hence: Gap-IS is NP hard, and IS is NP-Hence: Gap-IS is NP hard, and IS is NP-hard to approximate!hard to approximate!
2424
Each of the following theorems gives a hardness of Each of the following theorems gives a hardness of approximation result of Max-IS:approximation result of Max-IS:
ThmThm [AS,ALMSS] [AS,ALMSS] NP NP PCP[ O(1), 1, ½] PCP[ O(1), 1, ½] ThmThm [ RaSa ] [ RaSa ] NP NP PCP[ O(1), m, 2 PCP[ O(1), m, 2-m-m ] ] for for m m
loglogcc n n for some for some c > 0c > 0 ThmThm [ DFKRS ][ DFKRS ] NP NP PCP[ O(1), m, 2 PCP[ O(1), m, 2-m-m ] ] for for m m
loglogcc n n for any for any c > 0c > 0 ConjectureConjecture [BGLR][BGLR] NP NP PCP[ O(1), m, 2 PCP[ O(1), m, 2-m -m ] ] for for
m m log n log n
Hardness of approximation of Hardness of approximation of Max-ISMax-IS
2525
Assuming the PCP theorem, we will show that if Assuming the PCP theorem, we will show that if PPNP,NP, Max-3SatMax-3Sat does not have a does not have a PTAS: PTAS:
Theorem: There is a constant Theorem: There is a constant C>0 C>0 so that computing so that computing (1+c) approximations to (1+c) approximations to Max-3SatMax-3Sat is is NP-hardNP-hard
Hardness of approximation forHardness of approximation forMax-3SATMax-3SAT
2626
Hardness of approximation forHardness of approximation forMax-3SATMax-3SAT
SAT formulaSAT formulaEquivalent Equivalent
3SAT 3SAT formulaformula
y1y1
y2y2
yiyi
ym-1ym-1
ymym
variablesvariables
1111
jjjj
llll
CC11CC11 11CCkk
CCkkCC22CC22 CC33
CC33
CC11CC11 jjCCkk
CCkkCC22CC22 CC33
CC33
CC11CC11 llCCkk
CCkkCC22CC22 CC33
CC33
Given an instance of gap-SAT, Given an instance of gap-SAT, = { = { ll } }, we will , we will
transform each of the transform each of the ii‘s into a ‘s into a 3-SAT3-SAT expression expression ii..
2727
Hardness of approximation forHardness of approximation forMax-3SATMax-3SAT
Given an instance of gap-SAT, Given an instance of gap-SAT, = { = { ll } }, there are , there are O(n)O(n) functions functions i i . Each of the . Each of the ii‘s depends on up to ‘s depends on up to D=O(1)D=O(1) variables variables..
y1y1
y2y2
yiyi
ym-1ym-1
ymym
1111
jjjj
llll
CC11CC11 11CCkk
CCkkCC22CC22 CC33
CC33
CC11CC11 jjCCkk
CCkkCC22CC22 CC33
CC33
CC11CC11 llCCkk
CCkkCC22CC22 CC33
CC33
Hence each function can be represented as Hence each function can be represented as a CNF formula a CNF formula ii::
ijj
i D ,21C
(a conjunction of (a conjunction of 2^D 2^D clauses, each of size at most clauses, each of size at most DD))Note that the number of clauses is still constant.Note that the number of clauses is still constant.Overall, we build a CNF formula: a conjunction of Overall, we build a CNF formula: a conjunction of i i (one for or each local test)(one for or each local test)..
2828
Hardness of approximation forHardness of approximation forMax-3SATMax-3SAT
Now rewrite every D-clause as a group of 3-clauses to Now rewrite every D-clause as a group of 3-clauses to obtain a 3-CNF:obtain a 3-CNF:
y1y1
y2y2
yiyi
ym-1ym-1
ymym
1111
jjjj
llll
CC11CC11 11CCkk
CCkkCC22CC22 CC33
CC33
CC11CC11 jjCCkk
CCkkCC22CC22 CC33
CC33
CC11CC11 llCCkk
CCkkCC22CC22 CC33
CC33
)(^)^()()...( 12412112121
jzjkjkkkk zlzzllzlllll
Note that this is still a constant blow up in the number Note that this is still a constant blow up in the number of clauses.of clauses.
2929
Hardness of approximation forHardness of approximation forMax-3SATMax-3SAT
y1y1
y2y2
yiyi
ym-1ym-1
ymym
1111
jjjj
llll
CC11CC11 11CCkk
CCkkCC22CC22 CC33
CC33
CC11CC11 jjCCkk
CCkkCC22CC22 CC33
CC33
CC11CC11 llCCkk
CCkkCC22CC22 CC33
CC33
In case In case is NOT satisfyable, some constant fraction of the is NOT satisfyable, some constant fraction of the are not satisfied, and for each, at least one clause are not satisfied, and for each, at least one clause in in ii isn’t satisfied. isn’t satisfied.
3030
Conclusion:Conclusion:In case the original In case the original SATSAT formula formula isn’t satisfied, a constant isn’t satisfied, a constant
number of number of 3SAT3SAT formula formula ii are not satisfied, and for each at are not satisfied, and for each at least one clause isn’t satisfied.least one clause isn’t satisfied.
Because each Because each ii contains a constant number of clauses, contains a constant number of clauses, altogether a constant number of clauses in the resulting altogether a constant number of clauses in the resulting 3SAT3SAT aren’t satisfied.aren’t satisfied.
This provides a gap, and hence This provides a gap, and hence 3SAT3SAT cannot be approximated to cannot be approximated to within some constant unless within some constant unless P=NPP=NP!!
Hardness of approximation Hardness of approximation forfor
Max-3SATMax-3SAT
3131
The PCP theorem has ushered in a new era of hardness The PCP theorem has ushered in a new era of hardness of approximation results. Here we list a few:of approximation results. Here we list a few:
We showed that We showed that Max-CliqueMax-Clique ( and equivalently ( and equivalently Max-Independent-Set Max-Independent-Set ) do not has a ) do not has a PTASPTAS. It is . It is known in addition, that to approximate it with a known in addition, that to approximate it with a factor of factor of nn1-1- is hard unless is hard unless co-RP = NPco-RP = NP..
Chromatic Number Chromatic Number - - It is It is NP-HardNP-Hard to approximate it to approximate it within a factor of within a factor of nn1-1- unless unless co-RP = NPco-RP = NP. There is a . There is a simple reduction from simple reduction from Max-CliqueMax-Clique which shows that which shows that it is it is NP-HardNP-Hard to approximate with factor to approximate with factor nn. .
Chromatic Number for 3-colorable graphChromatic Number for 3-colorable graph - - NP-HardNP-Hard to approximate with factor to approximate with factor 5/3-5/3- (i.e. to (i.e. to differentiate between 4 and 3). Can be differentiate between 4 and 3). Can be approximated within approximated within O(nO(nloglogO(1) O(1) n).n).
More Results Related to PCPMore Results Related to PCP
3232
Vertex Cover Vertex Cover – Very easy to approximate within a – Very easy to approximate within a factor of 2. factor of 2. NP-HardNP-Hard to approximate it within a to approximate it within a factor of 4/3.factor of 4/3.
Max-3-Sat Max-3-Sat – Known to be approximable within a – Known to be approximable within a factor of factor of 8/78/7. . NP-HardNP-Hard to approximate within a to approximate within a factor of factor of 8/7-8/7- for every for every >0>0
Set Cover Set Cover - - NP-HardNP-Hard to approximate it within a to approximate it within a factor of factor of ln nln n. Cannot be approximated within . Cannot be approximated within factor factor (1-(1-))ln nln n unless unless NP NP Dtime(n Dtime(nloglognloglogn))..
More Results Related to PCPMore Results Related to PCP
3333
Maximum Satisfying Linear Sub-SystemMaximum Satisfying Linear Sub-System - - The problem: Given a linear system The problem: Given a linear system Ax=bAx=b (A is (A is n x mn x m matrix ) in field matrix ) in field FF, find the , find the largest number of equations that can be largest number of equations that can be satisfied by some satisfied by some xx..
If all equations can be satisfied the problem is in If all equations can be satisfied the problem is in PP..
IfIf F=Q NP-Hard F=Q NP-Hard to approximate by factor to approximate by factor mm. Can . Can be approximated be approximated in O(m/logm)in O(m/logm)..
If If F=GF(q)F=GF(q) can be approximated by factor can be approximated by factor q q (even a random assignment gives such a factor). (even a random assignment gives such a factor). NP-HardNP-Hard to approximate within to approximate within q-q-. Also . Also NP-HardNP-Hard for equations with only 3 variables.for equations with only 3 variables.
For equations with only 2 variables. For equations with only 2 variables. NP-HardNP-Hard to to approximated within 1.0909 but can be approximated within 1.0909 but can be approximated within 1.383 approximated within 1.383
More Results Related to PCPMore Results Related to PCP
