List Decoding Using the XOR Lemma

Luca Trevisan

U.C. Berkeley

Yao’s XOR Lemma

• If f:[N] -> {0,1} is weakly hard on average– every circuit of size s computes f correctly on

at most 1- fraction of inputs

• Define f+k:[N]k ->{0,1} asf+k(x1,…,xk) = f(x1)+…+f(xk) mod 2

• Then f+k is very hard on average– every circuit of size ~s computes f correctly on

at most ~1/2+ (1-2k fraction of inputs

Concatenation LemmaEssentially equivalent to XOR Lemma• If f:[N] -> {0,1} is weakly hard on average

– circuits of size s succed on at most 1- fraction of inputs

• Define fk:[N]k ->{0,1}k as fk(x1,…,xk) = f(x1),…,f(xk)

• Then fk is very hard on average– every circuit of size ~s computes f correctly

on at most ~(1-k fraction of inputs

This Talk

• We observe:– any black-box proof of Concatenation Lemma or

XOR Lemma gives way of converting • ECCs with weak unique-decoding algorithms into• ECCs with strong list-decoding algorithms

• Better codes if the Concatenation Lemma– Holds even if x1,…,xk not fully independent– Proof uses limited non-uniformity

This Talk• We give ‘almost uniform’ version of Impagliazzo’s

Concatenation Lemma for pairwise independent x1,…,xk

• We get codes with quadratic encoding length and quasi-linear time list-decoding(no polynomials in the construction)

• The uniform version of Impagliazzo’s result also gives a weak uniform version of O’Donnell’s amplification of hardness within NP(work in progress)

Concatenation Lemma

A black-box proof argues:• Let f:[N]->{0,1}, and define

– F(x1,…,xk)=f(x1),…,f(xk)• Let G have agreement with F• There is a circuit that computes f on 1- fraction

of inputs and– makes poly(1/,1/) oracle queries to G– has size poly(log N, 1/, 1/, k)

( can be as small as ~(1-)k)

Error-Correcting Code

• Let C:{0,1}m->{0,1}N be ECC with decoding algorithm that corrects up to N errors

• Define C’:{0,1}m->({0,1}k)Nk

– Given message M, compute C(M)– C’(M) has an entry for each k entries of C(M)– C’(M)[x1,…,xk]=C(M)[x1],C(M)[x2],…,C(M)[xk]

Decoding• Given a corrupted G that has agreement with

C’(M), think of– C(M) as f– C’(M) as F– G as A

• Enumerate all exp(log N, 1/,1/,k) circuits having oracle access to A. At least one defines string having agreement 1- with C

• Apply unique decoding algorithm for C() to each string in list

• List will contain M

How to Improve• Encoding length is Nk

– Shorter encoding length if x1,…,xk not fully independent

– Impagliazzo proves concatenation lemma for pairwise independent x1,…,xk. Proof inherently non-uniform

• List size and decoding time are quasi-polynomial– Shorter list / faster decoding if reduction is uniform– Levin’s and GNW’s proofs are uniform, but need full

independence of x1,…,xk

• We give almost uniform proof of pairwise-independent concatenation lemma– Quadratic encoding length– Quasi-Linear decoding time

Impagliazzo’s Proof

1. If f is hard to solve on more than 1-fraction of inputs.

– Then there is a set H containing fraction of inputs and f is hard to solve on more than ½+ fraction of H

2. If f is hard to solve on more than ½+fraction of set H of density

– then F(a,b)=f(a+b),…,f(a+kb)hard to solve on more than O(1/k) fraction of inputs

1st Part: Hard-Core Sets

• Thm: – If f cannot be solved on (1-) fraction of inputs with

circuits of size s,– Then there is set H of size N such that f cannot be

solved on (½+) fraction of H with circuits of size s*poly(,)

• Equivalently:– If for every H of size N there is circuit of size s that

solves f on (½+) fraction of H – Then there is circuit of size s*poly(1/,1/) that solves

f on (1-) fraction of inputs

Impagliazzo’s Proof

• Set/function Game. At step i:– Player A produces a set Hi of size N

– Player B produces a function gi that agrees

with fi on ½ + fraction of Hi

Player A wins at step t if g(x)=maj{g1(x),…,gt(x)}

agrees with f on 1- fraction of inputs

• Thm [Imp95]: there is a winning strategy for A that suceeds in poly(1/,1/) steps

Impagliazzo’s Proof

• If for every H there is circuit of size s and agreement ½+ with f on H– Let Player A play Impagliazzo’s strategy– Let Player B always reply with a circuit size s

• Construct size s*poly(1/,1/) circuit that computes f on 1- fraction of inputs

Uniform Version

• Thm: suppose there is distrib C over circuits such that for every H of size N– PrC~C[ Prx~H [C(x)=f(x)]> ½ +] > Then there is distrib C’ over circuits such that

– PrC~C’[ Prx [C(x)=f(x)]> 1 -] > poly(1/,1/)

• Proof: pick t=poly(1/,1/) ckts from C and take majority. There is at least probability t that it works.

2nd Part: Pairwise Independence

• Suppose f is (1-)-hard, but algorithm A computes– F(a,b)=f(a+b),f(a+2b),…,f(a+kb)On ~1/k fraction of inputs

• Let H be set of size k. – [Imp95]: can compute f on ½+’ frac of H.

• Let A(a,b)= A1(a,b),A2(a,b),…,Ak(a,b)• Define 0/1 random variables Z1,…,Zk

– Zi=1 iff Ai(a,b)=f(a+ib) AND a+ib is in H

Studying the Zi• A(a,b)= A1(a,b),A2(a,b),…,Ak(a,b)• Zi=1 iff Ai(a,b)=f(a+ib) AND a+ib is in H

• Note: E[Zi]= Pr[Ai(a,b)=f(a+ib)|a+ib in H]

• Suppose E[Zi]> (1/2 + ’) or E[Zi] < (1/2-’)Then easy to solve f on H better than ½

• Remains to consider case all E[Zi] ~ /2

Studying the Zi

• A(a,b)= A1(a,b),A2(a,b),…,Ak(a,b)• Zi=1 iff Ai(a,b)=f(a+ib) AND a+ib is in H• E[Zi] ~ /2 for all i

• Suppose the Zi are almost pairwise independent• Then sum of Zi concentrated around k/2• Number of a+ib in H concentrated around k• Impossible for A to have noticeable prob of

being correct on all inputs

Studying the Zi

• A(a,b)= A1(a,b),A2(a,b),…,Ak(a,b)• Zi=1 iff Ai(a,b)=f(a+ib) AND a+ib is in H

• There are i,j such that E[ZiZj] > /4 + ’’• Equivalent:

Pr[ Ai(a,b)=f(a+ib) AND Aj(a,b)=f(a+jb)]> ¼+’’conditioned on a+ib and a+jb are in H

• Equivalent: pick x,y in H Pr[ Ai(a,b)=f(x) AND Aj(a,b)=f(y)]> ¼+’’where a,b such that a+ib=x and a+jb=y

Using the Dependency

• Pick x,y in H Pr[ Ai(a,b)=f(x) AND Aj(a,b)=f(y)]> ¼+’’where a,b such that a+ib=x and a+jb=y

• Then one of:– Pr[ Ai(a,b)=f(x)]>1/2 +2’’/3– Pr[ Aj(a,b)=f(y)]>1/2 +2’’/3– Pr[ Ai(a,b) XOR Aj(a,b) XOR f(y)=f(x)]>1/2 +2’’/3

• In each case we get circuit for f on H

Uniform Version

• Let f be a function and A be an algorithm that computes– F(a,b)=f(a+b),f(a+2b),…,f(a+kb)on ~1/k fraction of inputs

• Then can define distribution of circuits such that for each set H there is prob at least poly(,,1/k) that circuit computes f on H on more than ½+’ inputs

• Proof: replace non-uniformity in Impagliazzo’s argument with random choices.

Everything Together

• Suppose function f and algorithm A are so that– F(a,b)=f(a+b),f(a+2b),…,f(a+kb)

agrees with A on ~1/k fraction of inputs• Then can sample distribution of circuits such that

there is prob. exp(1/,1/,k) of sampling a circuit that agrees with f on 1- frac of inputs

• Also can produce list of size exp(1/,1/,k) that contains whp circuit -close to f

Coding Application• From

– binary error-correcting code with codewords of length N and unique decoding algorithm for fraction of errors

• Error-correcting code with – alphabet of size 2k,

– codewords of length N2

– list decoding up to 1-O(1/k) errors, with list of size exp(1/,k)

– implicit representation of list is computed in polylog N time. Explicit representation in Npolylog N time.

Comparison to Previous Work• Sudan’97:

– linear encoding length +– quasi-linear encoding time +– polynomial list-decoding time – – list is polynomial in 1/ +

• Feng’99, Alenkkhnovich’02:– improve to quasi-linear decoding time =

• Guruswami-Indyk’01-02– Even better rates but quadratic decoding time +/-– Do not use polynomials =

Possible Improvement

• Prove a Concatenation Lemma for almost pairwise independent inputs.

• As in BSVW’03, let Ft be vector space and S be small bias space, then consider– F(a,b)=f(a+b),f(a+2b),…,f(a+kb)

Where a ranges in Ft and b ranges in S

• Whole argument works out up to the point– Pr[f(a+ib)=Ai(a,b) XOR Aj(a,b) XOR f(a+jb)]>½

Other Applications

• O’Donnell proves a (non-uniform) amplification of hardness result in NP using Impagliazzo’s hard-core sets.

• We can prove:– Let f be NP function, can construct f’ such that– If f’ has BPP algorithm that works on ½+

fraction of inputs– Then f has BPP algorithm that produces

exp(1/,1/) circuits, one of them solves f on 1- fraction of inputs

How to Choose the Circuit

• Suppose every problem in NP has BPP algorithm that works on ¾+’ fraction of inputs

• Let f be NP function, C1,…,Cl circuits such that one of them solves f on 1- fraction of inputs

• Define – F(x1,…,xt)=g(f(x1),…,f(xt))– Di(x1,…,xt)=g(Ci(x1),…,Ci(xt))

Where g() is noise-sensitive function

How To Choose The Circuit

• Define – F(x1,…,xt)=g(f(x1),…,f(xt))– Di(x1,…,xt)=g(Ci(x1),…,Ci(xt))

• If Ci -close to f then Di t-close to F

• If Ci ’-far from f then Di (½+’’)-far from F

• We have algorithm to compute F on > ¾ of inputs, can distinguish two cases

Uniform Result

• Suppose for every NP problem there is BPP algorithm that works on 1-1/(log n)c fraction of inputs

• Then for every NP problem there is BPP algorithm that works on ¾+1/(log n)c fraction of inputs

(Proof not completely written up)

Conclusions

``Everything's got a moral,

if only you can find it.’'

(Lewis Carroll, Alice's Adventures in Wonderland)

Conclusions

• An information-theoretic interpretation of black-box concatenation lemmas– Conversion of weaker to stronger error-correcting

codes

• In coding theory– Suggests a new technique for list-decoding– What is it?

• In complexity theory– Emphasis on two aspects of Concatenation Lemma

proofs: non-uniformity and derandomization