Date post: | 17-Jan-2016 |
Category: |
Documents |
Upload: | sharlene-george |
View: | 230 times |
Download: | 8 times |
How to refute a random CSP
Sarah R. Allen Ryan O’Donnell David WitmerCarnegie Mellon University
Average-case complexity of CSPs.
E.g., random 3SAT.
How to refute a random CSP
Random 3SAT
n variables, m = m(n) constraints, each an OR of 3 uniformly random literals
m
4.267n
satisfiable (w.v.h.p.) unsatisfiable (w.v.h.p.)
Algorithmic task:
find a satisfyingassignment
Algorithmic task:
find a refutation(proof of unsatisfiability)
What is this “refutation” task?
1. Algorithm should (w.h.p.) output a proof, encoded in some formal language like ZFC, that the CSP instance is unsatisfiable.
2. Algorithm should output “unsatisfiable” or “no comment”. It should never be wrong, and it should output “unsatisfiable” w.h.p.
Potential example: Algorithm solves an LP/SDP relaxation,
outputs “unsatisfiable” iff its value < m.
Random 3SAT
n variables, m = m(n) constraints, each an OR of 3 uniformly random literals
m
4.267n
find a sat. assignment find a refutation
3.52n
provablydoable
[HS03,KKL03]
Random 3SAT
n variables, m = m(n) constraints, each an OR of 3 uniformly random literals
m
4.267n
find a sat. assignment find a refutation
3.52n
provablydoable
[HS03,KKL03]
heuristicallydoable (?)[MPR15]
provablydoable
[FGK01]
???
Perhaps when, say, m = n1.4,
there’s no poly-time refutation algorithm.
Cool! An efficient way to randomly generatesimple, hard-to-solve algorithmic tasks!
Potential application 1: Cryptography. [Gol00, ABW09, Applebaum…]
Potential application 2: Hardness of learning.
[Daniely–Linial–Shalev-Shwartz13+]
For applications, investigating random CSP(P)for different predicates P matters.
E.g., for certain kinds of k-ary predicates P,
if refuting random CSP(P) is hard when m = nC…
App. 1: Gives evidence for PRGs in NC0
with stretch n ↦ nC
App. 2: Shows hardness of PAC-learning concept classes “related to” P,
assuming C → ∞ as k → ∞
Refuting random CSPs is
heavily studied for kSAT,
less studied for other predicates.
Our main theorem:
A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random
CSP(P) instances, provided m ≫ n cmplx(P) / 2.
Our main theorem:
A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random
CSP(P) instances, provided m ≫ n cmplx(P) / 2.
The “complexity” of P, an integer we’ll define shortly.It’s always at most k, and “often” small, like 2, 3, 4.
Our main theorem:
A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random
CSP(P) instances, provided m ≫ n cmplx(P) / 2.
There’s some evidence that this is the “right” answer.We’ll say more at the end of the talk.
Our main theorem:
A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random
CSP(P) instances, provided m ≫ n cmplx(P) / 2.
Ignoring polylog(n) factors.
Our main theorem:
A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random
CSP(P) instances, provided m ≫ n cmplx(P) / 2.
Recovers the known n1.5 result for 3SAT;
cmplx(OR3) = 3.
Our main theorem:
A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random
CSP(P) instances, provided m ≫ n cmplx(P) / 2.
New result for kSAT, when k ≥ 5 is odd:
we refute provided m ≫ nk/2;
previous best was m ≫ n⌈k/2⌉.
Our main theorem:
A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random
CSP(P) instances, provided m ≫ n cmplx(P) / 2.
Our results also hold for non-Boolean predicates.
Our main theorem:
A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random
CSP(P) instances, provided m ≫ n cmplx(P) / 2.
Running time is nO(k).
The algorithm can be“k rounds of the SOS SDP hierarchy”.
Our main theorem:
A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random
CSP(P) instances, provided m ≫ n cmplx(P) / 2.
We even provide a δ-refutation,i.e., a proof that OPT ≤ (1−δ)m,for some constant δ = δ(P) > 0.
Our main theorem:
A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random
CSP(P) instances, provided m ≫ n cmplx(P) / 2.
If you take m ≫ nk/2, we provide “strong refutation”:
i.e., proof that OPT ≤ E[P]·m + o(m).E.g., that OPT ≤ ⅞·m + o(m) for 3SAT, OPT ≤ ½·m + o(m) for kXOR.
cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.
3SAT example: P = OR3 =
001010100011101110111
Does it support a 2-wiseuniform distribution? Yes.
cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.
3SAT example: P = OR3 =
001010100011101110111
Does it support a 2-wiseuniform distribution? Yes. Uniform distrib.
on these 4 strings.
cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.
3SAT example: P = OR3 =
001010100011101110111
Does it support a 2-wiseuniform distribution? Yes.
Does it support a 3-wiseuniform distribution? No.
∴ cmplx(OR3) = 3.
We can refute oncem ≫ n1.5
cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.
2-out-of-4-SAT: P =
001101010110100110101100
Does it support a 2-wise uniform distribution? No.
(If it did, by symmetrization that distribution could be the uniform distribution over weight-2, length-4 strings,
but that’s not 2-wise uniform.)
cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.
2-out-of-4-SAT: P =
001101010110100110101100
Does it support a 2-wise uniform distribution? No.
∴ cmplx(P) = 2.We can refute random instances once m ≫ n.
cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.
Does it support a 2-wise uniform distribution? No.
∴ cmplx(P) = 2.We can refute random instances once m ≫ n.
The exact same thing holds for any j-out-of-k-SAT.
(Refutation fact was already known [BB02].)
cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.
[DLS14]: A very strong conjecture implying that for many k-ary predicates P, random
CSP(P) instances with nωk→∞(1) constraints
are not refutable in polynomial time.
Instantiated with 3 families (Pk),
derived 3 hardness-of-learning results.
Unfortunately, we show all these
predicates have cmplx(Pk) ≤ 4.
cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.
[DLS14]: A very strong conjecture implying that for many k-ary predicates P, random
CSP(P) instances with nωk→∞(1) constraints
are not refutable in polynomial time.
Instantiated with 3 families (Pk),
derived 3 hardness-of-learning results.
Fortunately, [DS14,Dan15] mostly recovered the results based on more plausible assumptions.
A condition similar to m ≫ n cmplx(P) / 2 also arises
in work on finding satisfying assignmentsin random CSPs with a planted solution [FPV14].
Warning / Remark
Nevertheless, I assure you:solving planted CSPs & refuting random CSPs
are quite distinct algorithmic tasks.
A poly-time algorithm that, for any k-arypredicate P, refutes (whp) uniformly random
CSP(P) instances, provided m ≫ n cmplx(P) / 2.
Plan for the rest of the talk
Part 1: Strong refutation for kXOR
Part 2: Strong refutation for any k-CSP, provided m ≫ nk/2
Part 3: Proof of the main theorem
Part 1: Strong refutation for kXOR
Given a random kXOR CSP with m ≫ nk/2,
want to certify “OPT ≤ ½·m +o(m)” (whp).
• [CCF10] strongly refute any CSP with m ≫ n⌈k/2⌉.
• For kXOR specifically, m ≫ n⌈k/2⌉ follows pretty
easily from approximation algs literature.
• For 3SAT, strong refutation provided m ≫ n1.5
achieved by [CGL07]. Messy, but if you stare at it long enough, it’s clear it generalizes to 3XOR.
• This, as well as the generalization to kXOR, done independently by [BM15].
Given a random kXOR CSP with m ≫ nk/2,
want to certify “OPT ≤ ½·m +o(m)” (whp).
• The weaker bound m ≫ n⌈k/2⌉ follows from the
k = 2 case, and a very trivial trick.
• The m ≫ nk/2 bound requires adding in a very
clever trick (from [FGK01,CGL07]).
• Sadly, only time for a sketch of the k = 2 case.
Given a random 2XOR CSP with m ≫ n,want to certify “OPT ≤ ½·m +o(m)” (whp).
n variables, x1, …, xn ∈ {0,1}
m constraints, each like
Switch to ±1 notation.
Given a random 2XOR CSP with m ≫ n,want to certify “OPT ≤ ½·m +o(m)” (whp).
n variables, x1, …, xn ∈ {−1,+1}
m constraints, each like
Let OPT denote (# sat − # unsat) instead;i.e., sum of right-hand sides achieved.
Given a random 2XOR CSP with m ≫ n,want to certify “OPT ≤ o(m)” (whp).
n variables, x1, …, xn ∈ {−1,+1}
m constraints, each like
Let OPT denote (# sat − # unsat) instead;i.e., sum of right-hand sides achieved.
Given a random 2XOR CSP with m ≫ n,want to certify “|OPT| ≤ o(m)” (whp).
n variables, x1, …, xn ∈ {−1,+1}
m constraints, each like
Let OPT denote (# sat − # unsat) instead;i.e., sum of right-hand sides achieved.
Given a random 2XOR CSP with m ≫ n,want to certify “|OPT| ≤ o(m)” (whp).
n variables, x1, …, xn ∈ {−1,+1}
m constraints, each like
Let and instead include each
possible constraint independently with prob. p.
Given a random 2XOR CSP with p ≫ 1/n,
want to certify “|OPT| ≤ o(pn2)” (whp).
n variables, x1, …, xn ∈ {−1,+1}
m constraints, each like ~pn2
Let A be the random n×n symmetric matrix with
entries {−1,0,+1} depending on the xi,xj constr.
Given assignment x ∈ {−1,+1}n, it has
(# sat − # unsat) = xTAx ≤ n||A||
Given a random 2XOR CSP with p ≫ 1/n,
want to certify “|OPT| ≤ o(pn2)” (whp).
Let A be the random n×n symmetric matrix with
entries {−1,0,+1} depending on the xi,xj constr.
Given assignment x ∈ {−1,+1}n, it has
|# sat − # unsat| = |xTAx| ≤ n||A||
Done if ||A|| ≤ o(pn) whp. (Can efficiently certify.)
A good old-fashioned random matrix fact.||A|| ≤ whp, by the Trace Method [FK81].
Part 1: Strong refutation for kXOR
Part 2: Strong refutation for any k-CSP, provided m ≫ nk/2
✔
m
k
Def: Given an assignment x∈{0,1}n,
m
k
Def: Given an assignment x∈{0,1}n,
the table distribution Dx is the probability
distribution on {0,1}k given by choosing a
uniformly random row from the above table.
m
k
Def: Given an assignment x∈{0,1}n,
the table distribution Dx is the probability
distribution on {0,1}k given by choosing a
uniformly random row from the above table.
Note: Has nothing to do with the predicate P.
Def: Given an assignment x∈{0,1}n,
the table distribution Dx is the probability
distribution on {0,1}k given by choosing a
uniformly random row from the above table.
Def: An instance I is quasirandom if
for all assignments x∈{0,1}n,
the table distribution Dx is o(1)-close
to the uniform distribution on {0,1}k.
Fact: If a CSP(P) instance I is quasirandom, thenOPT(I) ≤ E[P]·m + o(m).
Given a random k-ary CSP with m ≫ nk/2,
want to certify it’s quasirandom (whp).
By “Vazirani XOR Lemma”, a table distribution
Dx is o(1)-close to uniform if and only if
Dx has o(1)-correlation with XOR of S coordinates
for all ∅ ≠ S ⊆ [k].
∴ can certify this using the |S|-XOR refutation
algorithm from Part 1, for all 2k−1 subsets S.
Part 1: Strong refutation for kXOR
Part 2: Strong refutation for any k-CSP, provided m ≫ nk/2
✔
✔
Part 3: Main theorem, refuting random CSP(P) provided m ≫ n
cmplx(P) / 2
cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution.
Suppose P does not supporta t-wise uniform distribution.
Suppose P does not supporta t-wise uniform distribution.
⇒ every t-wise uniform distribution is δ-far from being supported on P (for some constant δ = δ(P) > 0, by basic LP theory).
Given CSP(P) instance, instead of certifying quasirandomness by strongly refuting S-XOR for all S ⊆ [k]…
Suppose P does not supporta t-wise uniform distribution.
⇒ every t-wise uniform distribution is δ-far from being supported on P (for some constant δ = δ(P) > 0, by basic LP theory).
Given CSP(P) instance, strongly refute S-XOR for all |S| ≤ t.
This is doable (whp) provided m ≫ nt/2.
This certifies that all table distributions Dx
are o(1)-close to t-wise uniform, and hence (δ−o(1))-far from being supported on P.
Suppose P does not supporta t-wise uniform distribution.
⇒ every t-wise uniform distribution is δ-far from being supported on P (for some constant δ = δ(P) > 0, by basic LP theory).
Given CSP(P) instance, strongly refute S-XOR for all |S| ≤ t.
This is doable (whp) provided m ≫ nt/2.
This certifies that all table distributions Dx
are o(1)-close to uniform, and hence (δ−o(1))-far from being supported on P.
I.e., it certifies that OPT ≤ (1−δ+o(1))m.
Part 1: Strong refutation for kXOR
Part 2: Strong refutation for any k-CSP, provided m ≫ nk/2
✔
✔
Part 3: Main theorem, refuting random CSP(P) provided m ≫ n
cmplx(P) / 2 ✔
Open directions
Give evidence that n cmplx(P) / 2 is optimal.
I.e., if P supports a (t−1)-wise uniform distribution,try to show various efficient refutation systems
fail on random instances with m ≪ nt/2 constraints.
[BW99]: Resolution fails on kXOR and kSAT.[Sch08]: SOS fails on kXOR and kSAT.[BGMT12,TW13,OW14,MWW15]: Sherali–Adams+ fails.[FPV15]: Certain “statistical algorithms” fail.[BCK15]: For t=3 (pairwise unif.), “pruned” random
instances with m = O(n), SOS fails.
Thanks!