Date post: | 31-Mar-2015 |
Category: |
Documents |
Upload: | alana-wommack |
View: | 218 times |
Download: | 0 times |
LT Codes
Paper by Michael Luby FOCS ‘02
Presented by Ashish Sabharwal Feb 26, 2003 CSE 590vg
2
Binary Erasure Channel
Code distance d ) can decode d-1 erasures
Probabilistic Model Bits get erased with prob p (Shannon) Capacity of BEC = 1 – p
In particular, p>1/2 is decodable!
Input00101
Codeword10100101
BECReceived10?001??
Input00101
decodeencode
“Packet loss”
3
LT Codes: Encoding
1
11
01
1 XOR 0 = 1
input 1 code bit
degree d = 2
1. Choose degree d from a distribution2. Pick d neighbors uniformly at random3. Compute XOR
4
LT Codes: Encoding
1
11
01
1
11
11
1
inputcodeword
… …
0
5
LT Codes: Decoding
1. Identify code bit of remaining degree 12. Recover corresponding input bit
1
11
01
?
11
11
?
inputcodeword
0
6
LT Codes: Decoding
1
11
01
11
11
input codeword
1 = 0 XOR 1
3. Update neighbors of this input bit4. Delete edges5. Repeat
7
LT Codes: Decoding
1
11
01
11
11
input codeword
1
8
LT Codes: Decoding
1
11
01
01
0 = 1 XOR 11
input codeword
0
Decoding unsuccessful!
9
LT Codes: Features
Binary, efficient
Bits can arrive in any order
Probabilistic model
No preset rate Generate as many or as few code bits
as required by the channel
Almost optimal
RS inefficient
Tornado codes are optimal andlinear time, but have fixed rate
10
Larger Encoding Alphabet
Why? Less overhead
Partition input into m-bit chunks Encoding symbol is bit-wise XOR
We’ll think of these as binary codes
11
Caveat: Transmitting the Graph
Send degree + list of neighbors
Associate a key with each code bit Encoder and decoder apply the same
function to the key to compute neighbors
Share random seed for pseudo-randomgenerator
12
Outline
The Goal
All 1’s distribution: Balls and Bins case
LT Process; Probabilistic machinery
Ideal Soliton Distribution
Robust Soliton Distribution
13
The Goal
Construct a degree distribution s.t.
1. Few encoding bits required for recovery= small t
2. Few bit operations needed= small sum of degrees= small s
14
All 1’s distribution: Balls and Bins
All encoding degrees are 1
k bins
t balls
• t balls thrown into k bins
• Pr [can’t recover input] Pr [no green input bits] = k . (1 – 1/k)t
¼ k e-t/k
• Pr [failure] guaranteed if t k ln k/
k bitinput
t unerasedcode bits
15
All 1’s distribution: Balls and Bins
t = k ln (k/)
s = k ln (k/)
GOODOptimal!
BADToo much overhead
k + k ln2(k/) suffices
16
Why is s = k ln (k/) optimal?
k bins
s balls
• s balls thrown into k bins
• can’t recover input ) no green input bits
• Pr [no green input bits] k . (1 – 1/k)s
¼ k e-s/k
• Pr [failure] if s k ln k/
k bitinput s edges
NOTE: This line of reasoning isnot quite right for lower bound!Use coupon collector type argument.
17
The LT Process
covered = { }processed = { }ripple = { }released = { }
a1
a2a3
a4a5
c1
c2c3
c4c5
c6
STATE:
ACTION:
Init: Release c2, c4, c6
18
The LT Process
released = {c2,c4,c6} covered = {a1,a3,a5}processed = { }ripple = {a1,a3,a5}
c1
c2c3
c4c5
c6
STATE:
ACTION:
Process a1
a1
a2a3
a4a5
19
The LT Process
released = {c2,c4,c6,c1}covered = {a1,a3,a5}processed = {a1}ripple = {a3,a5}
STATE:
ACTION:
Process a3
a1
a2a3
a4a5
c1
c2c3
c4c5
c6
20
The LT Process
released = {c2,c4,c6,c1} covered = {a1,a3,a5}processed = {a1,a3}ripple = {a5}
STATE:
ACTION:
Process a5
a1
a2a3
a4a5
c1
c2c3
c4c5
c6
21
The LT Process
released = {c2,c4,c6,c1,c5}covered = {a1,a3,a5,a4}processed = {a1,a3,a5}ripple = {a4}
STATE:
ACTION:
Process a4
a1
a2a3
a4a5
c1
c2c3
c4c5
c6
22
The LT Process
released = {c2,c4,c6,c1,c5,c3}covered = {a1,a3,a5,a4,a2}processed = {a1,a3,a5,a4}ripple = {a2}
STATE:
ACTION:
Process a2
a1
a2a3
a4a5
c1
c2c3
c4c5
c6
23
The LT Process
released = {c2,c4,c6,c1,c5,c3} covered = {a1,a3,a5,a4,a2}processed = {a1,a3,a5,a4,a2}ripple = { }
STATE:
ACTION:
Success!
a1
a2a3
a4a5
c1
c2c3
c4c5
c6
24
The LT Process: Properties
Corresponds to decoding
When a code bit cp is released The step at which this happens
is independent of other cq’s The input bit cp covers
is independent of other cq’s
25
Ripple size
Desired property of ripple Not too large: redundant covering Not too small: might die prematurely
GOAL: “Good” degree distribution Ripple doesn’t grow or shrink 1 input bit added per step
Why??
26
Degree Distributions
Degrees of code bits chosen independently
(d) = Pr [degree = d]
All 1’s distribution: (1) = 1, (d1) = 0initial ripple = all input bits“All-At-Once distribution”
27
Machinery: q(d,L), r(d,L), r(L)
L = | unprocessed | k, k-1, …,1
q(d,L) = Pr [ cp is released at L | deg(cp)=d}
r(d,L) = Pr [ cp is released at L, deg(cp)=d} = (d) q(d,L)
r(L) = Pr [cp is released at L] = d r(d,L) r(L) controls ripple size
28
q(d,L)
29
Ideal Soliton Distribution, (.)
“Soliton Wave”: dispersion balances refraction
Expected degree = ln k r(L) = 1/k for all L = k, …, 1
30
Expected Behavior
Choose t = k
Exp(s) = t Exp(deg) = k ln k
Exp(Initial ripple size) = t (1) = 1
Exp(# code bits released per step) = t r(L) = 1
) Exp(ripple size) = 1
optimal
31
We expect too much…
What if the ripple vanishes too soon?
In fact, very likely!
FIX: Robust Soliton Distribution
Higher initial ripple size ¼ k log k/ Expected change still 0
32
Robust Soliton Distribution, (.)
R = c k ln k/
(d) = ((d) + (d)) / where
t = k
33
Robust Soliton Distribution, (.)
t is small t = k k + O(k ln2 k/)
Exp(s) is small Exp(s) = t d d(d) = O(k ln k/)
34
Robust Soliton Distribution, (.)
Initial ripple size is not too small Exp(Initial ripple size) = t (1)
¼ R ¼ k ln k/
Ripple unlikely to vanish Ripple size = random walk of length
k Deviates from it’s mean by k ln k/
with prob
35
Robust Release Probability
t r(L) L / (L – R) for L ¸ R, const
t L=R..2R r(L) R ln R/ for const > 0
Proofs on board…
36
Pessimistic Filtering
Let Z = ripple size when L bits unprocessed
Let h = Pr [released code bit covers input bit not in ripple] h should be around (L – Z) / L
If h is lowered to any value (L – Z)/Lthen Pr[success] doesn’t increase
37
Pessimistic Filtering
Applying to robust release probability:
t r(L) ¸ L/(L – R) turns into t r(L) = L/(L – R) for worst case analysis
Will use pessimistic filtering again later
38
Main Theorem: Pr[success] ¸ 1–
Idea: ripple size is like a random walk of length k with mean R ¼ k ln k/
1. Initial ripple size ¸ R/2 with prob ¸ 1–/3 Chernoff bound: # of code bits of deg 1
2. Ripple does not vanish for L ¸ Rwith prob ¸ 1–/3
3. Last R input bits are covered by (k/R) spikewith prob ¸ 1–/3
39
Ripple does not vanish for L ¸ R
Let XL = | {code bits released at L} | Exp(XL) = L / (L – R)
Let YL = 0-1 random variable with Pr [YL = 0] = (L – R) / L
Let I = any end interval of {R, …, k-1} starting at L
RipplesizeL = R/2 + (L’ 2 I XL’ YL’) – (k–L)
Filtered down init ripplesize
40
Ripple does not vanish for L ¸ R
|L’ 2 I XL’ YL’ – (k–L) |
· |L’ 2 I (XL’ YL’ – Exp(XL’) YL’) |
+ |L’ 2 I (Exp(XL’) YL’ – Exp(XL’) Exp(YL’)) |
+ |L’ 2 I (Exp(XL’) Exp(YL’)) – (k–L) |
¸ R/4 with prob · (6k)
= 0
Pr [|L’ 2 I XL’ YL’ – (k–L) | ¸ R/2] · /(3k)
41
Ripple does not vanish for L ¸ R
Recall RipplesizeL = R/2 + L’ 2 I XL’ YL’ – (k–
L) There are k–R intervals I
Pr [Summation ¸ R/2 for some I] · /3
0 < RipplesizeL < R with prob ¸ 1–/3
Ripple doesn’t vanish!
42
Main Theorem: Pr[success] ¸ 1–
Idea: ripple size is like a random walk of length k with mean R ¼ k ln k/
1. Initial ripple size ¸ R/2 with prob ¸ 1–/3 Chernoff bound: # of code bits of deg 1
2. Ripple does not vanish for L ¸ Rwith prob ¸ 1–/3
3. Last R input bits are covered by (k/R) spikewith prob ¸ 1–/3
43
Last R input bits are covered
Recall t L=R..2R r(L) R ln R/
By argument similar to Balls and Bins,
Pr [Last R input bits not covered] · 1 – /3
44
Main Theorem
With Robust Soliton Distribution,the LT Process succeeds with prob ¸ 1 –
t = k + O(k ln2 k/)
s = O(k ln k/)