1
Distributed LT Codes
Srinath Puducheri, Jörg Kliewer, and Thomas E. Fuja.
Department of Electrical Engineering,University of Notre Dame, Notre Dame, IN 46556, USA
IEEE International Symposium on Information Theory (ISIT), 2006.
2
Outline
• Introduction
• Description of LT Code
• Decomposition of LT Code
• Simulation Results
• Conclusions
• References
3
Introduction• Consider a scenario:
T: k information symbols
Distributed LT
Distributed LT
LTN
S1: k/2 information symbols
S2: k/2 information symbols
4
Description of LT Code
• Luby Transform (LT) Codes:– Encoding process
• For the ith encoded symbol, select degree di by Soliton distribution
• Choose di source symbols
• Perform XOR on chosen symbols
– Decoding process• Decode degree-one encoded symbols
• Remove degree-one edges iteratively…
x1 x2 x3 x4 x5 x6
y1 y2 y3 y4 y5
x1
x3
x2
x2
x5
x3
x5
x6
Degree 1 2 3 … k
probability μ1 μ2 μ3 … μk
x4
5
Description of LT Code
• LT process:
A B D
y1 y2 y4
A
B
A B
D
covered: in the ripple S
processed unprocessed: LC
The most likely degree of encoded symbols that get released: k/Ly3
A
C
D
6
Description of LT Code
• The intuition for the degree distribution:– More small degree than large degree.
• Easy to be decoded.
– One single degree-one encoded symbol can trigger the LT process.
• Ideal Soliton Distribution:
k: number of total original symbols
i: degree of encoded symbols
7
Description of LT Code• Ideal Soliton Distribution:
probability
degree
8
Description of LT Code
• In practice:
– Only one single degree-one encoded symbol is too fragile.
• Number of degree-one encoded symbol has to be more.
• A larger Ripple.
– Large degree encoded symbols are necessary evils.
• Because of a larger Ripple.
• To give a better coverage of original symbols.
– A modified Soliton Distribution, concerning the size of Ripple during the LT process.
• Robust Solition Distribution.
9
Description of LT Code
• LT process:
C FD
y5
C
D
E
F
covered: in the ripple S
processed unprocessed: LE…
When L = S, we expect all the original symbols to be covered.
We need the encoding symbols that get released then must cover S original symbols by degree: k/S
10
Description of LT Code
• Robust Soliton Distribution (RSD):
– Robust Soliton Distribution μ(·) is given by:
k: number of total original symbols
i: degree of encoded symbols
11
Description of LT Code• ρ(·) is a distribution over 1 ≦ i ≦ k, and for constants c >
0, 0 <δ< 1, the τ(·) is:
, where S define the size of Ripple:
12
Description of LT Code• Robust Soliton Distribution
K: number of total original symbolsd: degree of encoded symbols
τ( k/S )
probability
k = 10000
13
Decomposition of LT Code• Recall:
S1: k/2 information symbols
S2: k/2 information symbols
T: k information symbols
Distributed LT
Distributed LT
LT
Deconvolved Soliton Distribution
Deconvolved Soliton Distribution
Robust Soliton Distribution
N
14
Decomposition of LT Code• Proposed scenario:
– Each data set contains k/2 information symbols.
– Node N only performs one of two operations:
• Transmits the bitwise XOR of the symbols receives from S1 and S2.
• Transmits just one of them.
• The resulting encoded symbol received by T:
– The degree of the resulting symbol has two cases:
• The sum of the degrees (up to i = k).
• The individual degree of one of the received symbols.
15
Decomposition of LT Code• In determining p(·), we employ the “deconvolution” of the
RSD μ(·).
– If node N XORs the incoming symbols, the resulting degree is th
e sum of the in coming symbols.
– If the degrees are independently chosen according to p(·), their s
um will follow the distribution p * p.
• ‘*’ denotes the discrete-time convolution operation:
16
Decomposition of LT Code• However, a direct attempt at deconvolving μ(·) does not
yield satisfactory results.
– μ(0) = 0, it is necessary that p(0) = 0.
• We cannot permit a degree of zero.
– It implies that the smallest degree in p * p is two.• However, this does not match the RSD as μ(1) > 0.
• degree = 1 would have problem.
( )
17
Decomposition of LT Code• Even if we neglect μ(1), and attempt to deconvolve the re
maining part of μ(·) by solving for p(·) from:
– owing to the fact that μ(i) has a spike at i = k/S, we obtain negati
ve values of p(·) for some arguments.
• degree = k/S would have problem.
18
Decomposition of LT Code• An alternative approach (1/3):
: ρ: τ
Prob.
Deg.
Deg.Deg = 1 Deg = k/S
Deg = k/S
Deg = 1
μ(i)
: ρ: τ
Prob.
Deg.
μ’(i)
: ρ: τ
Prob.
μ’’(i)
19
Decomposition of LT Code• An alternative approach (2/3):
– Split RSD μ(·) into two distributions – μ’(·) and μ’’(·).
with the normalization factor β’ given by:
Deconvolving μ’(·) yields a valid probability distribution function:
20
Decomposition of LT Code• An alternative approach (3/3):
– Split RSD μ(·) into two distributions – μ’(·) and μ’’(·).
with the normalization factor β’’ given by:
and:
21
Decomposition of LT Code• ∵ β’ + β’’ = β, the RSD can be rewritten as:
Hence, the RSD μ(·) can be viewed as a mixture of the
distributions μ’(·) and μ’’(·) with mixing parameter β’/β’’.
22
Decomposition of LT Code• Then we deconvolve μ’(·) to yield a valid probability distri
bution f (·).– By solving:
– Using the same approach as earlier mentioned (recall), with p(·) replaced by f (·) and μ(·) by μ’(·), we obtain the following set of recursive equations:
23
Decomposition of LT Code• The Deconvolved Soliton Distribution (DSD) p(·) is given
by:
with the parameter, given by:
p(·) can be viewed as the mixture of f (·) and μ’’(·) with
mixing parameter λ.
24
Decomposition of LT Code• Notice that:
μ':1 2 … k/S k/2 k
f :1 2 … k/S k/2 k
1 2 … k/S k/2f * f :
k/2 + 1tail
Deg = k/S
: ρ: τ
Deg.
μ(i)Prob.
25
Decomposition of LT Code• Encoding procedure:
D1: k/2 information symbols
DSD : p(·)
encoded symbols: Y
Robust Soliton Distribution
N
D2: k/2 information symbols
S1
S2DSD p(·)
encoded symbols: X1
encoded symbols: X2
Node N knows the degrees and neighbors of X1 and X2
Y = X1 X2
orY = X1
orY = X2
T
DLT-2 Code
26
Decomposition of LT Code• Node N generates a symbol Y as follows:
otherwise (1/2):Generates independent random variables: U1 and U2
uniformly on the interval (0, 1). Based on the values of U1
and U2 , one of the following operations is performed.
27
Decomposition of LT Code• otherwise (2/2):
28
Decomposition of LT Code• The code obtained by combining the two DLT-2 Codes generat
ed by S1 and S2 is not, strictly speaking, an LT Code.
29
Simulation Results• An application of LT Code decomposition:
T: k information symbols
Distributed LT Distributed LT
LTN
S1: m information symbols
SJ: m information symbols
… … …J ≧ 2 source nodes -
k = Jm
Lossy channel
30
Simulation Results• Two schemes (1/2):
– Let every source employ an LT code for its data, and these LT
codes are multiplexed in time at N and sent to T.
• Works well except when source data blocks are small (500-1000
information symbols).
• Overhead is fairly large (about 15-20%).
31
Simulation Results• Two schemes (2/2):
– It would be beneficial if we could form an LT code over the entire
set of k = Jm information symbols at N and send this to T.
• J-fold deconvolution of the RSD.
– Due to the constraint that N has limited processing and storage capabili
ties.
• Code symbols from different sources are XORed together at N.
• It yields symbols that obey the RSD over the range of degrees 1 thr
ough k = Jm.
32
Simulation Results• Consider the case J = 4 (1/2):
– The DLT-2 code can be decomposed further into two more subc
odes called DLT-4 codes.
– By performing a deconvolution of the DSD p(·) similar to how the DSD was derived from the RSD:
• Write DSD p(·) as a mixture of two distributions p’(·) and p’’(·), whe
re p’’(·) contributes to the degrees 1 and k/S.
• The distribution p’(·) is then deconvolved to get g(·) and the final de
gree distribution q(·) is obtained by taking the appropriate mixture of
g(·) and p’’(·).
33
Simulation Results• Consider the case J = 4 (2/2):
– By performing a deconvolution of the DSD p(·) similar to how the DSD was derived from the RSD:
• The degree distribution q(·) is used by source nodes to generate encode symbols from their data, and these code symbols constitute th
e DLT-4 codeword.
• The node N performs a selective XOR of the four DLT-4 code symbols received from four sources, and forwards the resulting code sy
mbol to T.
• Node T needs to collect slightly greater than k = Jm encode symbol
s from N to recover all source data.
34
Simulation Results• Compare two schemes (1/2):
– In terms of the number of code symbols that must be received b
y the sink T to recover all the sources’ data, for J = 4, m = 500, k = Jm = 2000:
• Scheme A:– Each source generates LT code symbols from its 500-symbol d
ata set.
– The code symbols from different sources are time-multiplexed at N and transmitted to the sink T.
– The sink separately decodes each LT code to recover the data.
– Let NA denote the total number of code symbols that are neede
d by T to recover all the data.
35
Simulation Results• Compare two schemes (2/2):
• Scheme B:– Each source generates DLT-4 codewords from its 500-symbol
data set.
– At every instant, the four encode symbols generated by the sources are selectively XOR-ed at N and forwarded to the sink T.
– The sink thus receives a big LT code over all the sources’ data, which is then decoded to recover all the data simultaneously.
– Let NB denote the total number of code symbols that are neede
d by T to recover all the data.
36
Simulation Results• The average values of NA and NB obtained via simulations:
P.S. Probable Max Degree of RSD in Scheme A: up to 500 Probable Max Degree of RSD in Scheme B: up to 2000
37
Simulation Results
38
Conclusions• Presents a novel technique that allows distributed users t
o encode their data by selectively XORed, and results in
an efficient rateless code, similar to an LT Code, at a common destination.
• Exploits the improved efficiency for large blocklengths: by forming a single codeword at the destination representi
ng all the distributed data sets.
– A reduction in overhead compared with a system in which each data set is encoded with its own LT code having a smaller blocklength.
39
References• M. Luby, “LT Codes,” Proc. of the 43rd Annual IEEE Symp.
on Foundations of Comp. Sc., pp 271-280, Vancouver, Canada, November 2002.
• D.J.C. MacKay, “Fountain Codes”, IEE Proceedings – Commun. Vol. 152, No 6, December 2005.
• Ashish Khisti, “Tornado Codes and Luby Transform Codes,” Course Lecture of “6.454: Graduate Seminar in Area I” at MIT, Fall 2003.