Date post:  02Jan2016 
Category: 
Documents 
Upload:  dominiccherry 
View:  32 times 
Download:  0 times 
ON THE ANALYSIS AND APPLICATION OF LDPC CODES
OLGICA MILENKOVIC
UNIVERSITY OF COLORADO, BOULDER
A joint work with:
VIDYA KUMAR (Ph.D) STEFAN LAENDNER (Ph.D)
DAVID LEYBA (Ph.D) VIJAY NAGARAJAN (MS) KIRAN PRAKASH (MS)
OUTLINE
A brief introduction to codes on graphs An overview of known results: randomlike codes for
standard channel models Code structures amenable for practical implementation:
structured LDPC codes Codes amenable for implementation with good errorfloor
properties Code design for nonstandard channels
Channels with memory Asymmetric channels
Applying the turbodecoding principle to classical algebraic codes: ReedSolomon, ReedMuller, BCH…
Applying the turbodecoding principle to systems with unequal errorprotection requirements
THE ERRORCONTROL PARADIGMNoisy channels give rise to data errors: transmission or storage systems
Need powerful errorcontrol coding (ECC) schemes: linear or nonlinear
Linear EC Codes: Generated through simple generator or paritycheck matrix
Binary information vector (length k)
Code vector (word): (length n)
Key property: “Minimum distance of the code”, , smallest separation between two codewords
Rate of the code R= k/n
1 0 0 0 1 0 11 1 1 0 1 0 0
0 1 0 0 1 1 00 1 1 1 0 1 0
0 0 1 0 1 1 11 0 1 1 0 0 1
0 0 0 1 0 1 1
G H
mind
1 2 3 4( , , , )u u u u u
, 0Tx uG Hx
Binary linear codes:
LDPC CODES
More than 40 years of research (19481994) centered around
Weights of errors that a code is guaranteed to correct
“Bounded distance decoding” cannot achieve Shannon limit
Tradeoff minimum distance for efficient decoding
LowDensity ParityCheck (LDPC) Codes
Gallager 1963, Tanner 1984, MacKay 1996
1. Linear block codes with sparse (small fraction of ones) paritycheck matrix
2. Have natural representation in terms of bipartite graphs
3. Simple and efficient iterative decoding in the form of belief propagation (Pearl, 19801990)
mind
min( 1)/ 2d
THE CODE GRAPH AND ITERATIVE DECODING
Variable nodes Check nodes
(Irregular degrees/codes)1 1 1 0 1 0 0
0 1 1 1 0 1 0
1 0 1 1 0 0 1
H
Most important consequence of graphical description: efficient iterative decoding
Message passing:
Variable nodes: communicate to check nodes their reliability (loglikelihoods)
Check nodes: decide which variables are not reliable and “suppress” their inputs
Small number of edges in graph = low complexity
Nodes on left/right with constant degree: regular code
Otherwise, codes termed irregular
Can adjust “degree distribution” of variables/checks
Best performance over standard channels: long, irregular, randomlike LDPC codes
Have proportional to length of code, but correct many more errorsmind
DECODING OF LDPC CODES
Iterative decoding optimal only if the code graph has no cycles
Vardy et.al. 1997: All “good codes” must have cycles
• What are desirable code properties?
• Large girth (smallest cycle length): for sharp transition to waterfall region; large minimum distance;
• Number of cycles of short length as small as possible;
• Very low errorfloors (girthoptimized graphs are not a very good choice Richardson 2003);
• Make the performance capacity approaching: irregular degree distribution (depends on the channel, “safe” gap to capacity about 1dB);
(Density Evolution, Richardson and Urbanke, 2001)
• Practical applications: mathematical formula for positions of ones in H
HOW CAN ONE SATISFY THESE CONSTRAINTS?
For “standard channels” (symmetric, binaryinput) the questions are mostly wellunderstood
BEC – considered to be a “closed case”
(Capacity achieving degree distributions, complexity of decoding, stopping sets, have simple EXIT chart analysis…)
Errorfloor greatest remaining unsolved problem!
For “nonstandard” channels still a lot to work on
CODE CONSTRUCTION AND IMPLEMENTATION FOR STANDARD CHANNELS
STRUCTURED LDPC CODES WITH GOOD PROPERTIES
Structured LDPC: amenable for practical VLSI implementations
Construction issues: minimum distance, girth/small number of short cycles, errorfloor properties
Cyclerelated constraints
Trappingset related constrained
!
THE WATERFALL AND ERRORFLOOR REGION
Waterfall: Optimization of minimum distance and cycle length distribution (regular/irregular codes)
Waterfall: Optimization of degree distribution and code construction Error floor: Optimization of trapping sets or … maybe a different
decoding algorithm?
http://www.inference.phy.cam.ac.uk/mackay/codes/data.html
Encyclopedia of Sparse Codes on Graphs:
by D. MacKay
IIIPIP
IPIIIP
IIPIPI
PIIIPI
PPIIII
H
0000
0000
0000
0000
0000
22
22
22
22
22
Laendner/Leyba/Milenkovic/Prakash: Allerton 2oo3, 2004, ICC 2004, IT 2004, IT 2005;
Tanner 2000; Vasic and Milenkovic 2001; Kim, Prepelitsa, Pless 2002; Fossorier 2004
0...001
10...00
...0100
0...010
P
1432
121
321
...
...............
...
...
msmsmsms
ss
s
iiii
iiii
iiii
PPPP
PPPP
PPPP
H
00
00
0
0
0000
0000
0000
0000
0000
000
0000
5
65
76
87
8
198
198
198
198
519
6519
761
i
ii
ii
ii
i
iii
iii
iii
iii
iii
iiii
iii
P
PP
PP
PP
P
PPP
PPP
PPP
PPP
PPP
PPPP
PPP
H
nknknkn
n
n
,2,1,
,22,21,2
,12,11,1
...
............
...
...
PPP
PPP
PPP
H
)1)(1(,
jiji PP
111
1
rqr
q
PPI
PPI
III
)1)(1(,
ji crji PP
P P P
P P P
PPPP
PPPP
PPPP
PPPP
PPPP
PPPP
0000
0000
0000
0000
0000
0000
Masking…Irregular Codes
Codes with large girth
MATHEMATICAL CONSTRUCTIONS OF EXPONENTS
P P P P r1
P P P P r2
P P P P r3
P P P P r4
ci cj ck cl
LOW TO MODERATE RATE CODES:
Fan, 2000: Cycles exist if sum of exponents in closed path is zero modulo size (P)=q
qcrrcrrcrrlilii mod0)(...)()( 13221 21
Proper Array Codes (PAC): RowLabels {ri} form an Arithmetic Progression with common mean ri+1ri=a; Improper Array Codes (IAC): RowLabels {ri} do not form an Arithmetic Progression;
i+j2k=0 mod q (CW3) i+j2k=0 and i+2j3k=0 mod q (CW4)
2i+jk2l=0 i+jkl=0 2k+i3j=0 2jik=0
ANALYTICAL RESULTS ON ACHIEVABLE RATES
Number theoretic bounds for contructions: For PACs with girth g=8: For IACs with girth g=10: Results based on work of Bosznay, Erdos,Turan, Szemeredi,
Bourgain
qqqC log/loglog
q2
3/12
)1(21 )1(2
3,))12/(1(1
q
qoriq
qi
)1log(
log,
1)1( 2
dD
qk
k
d k
s=1 0,1,3,4,9,10,12,13,27,28,30,38,...
s=2 0,2,3,5,9,11,12,14,27,29,30,39,...
s=3 0,3,4,7,9,12,13,16,27,30,35,36,...
s=1 0,1,5,14,25,57,88,122,198,257,280,...
s=2 0,2,7,18,37,65,99,151,220,233,545,...
s=3 0,3,7,18,31,50,105,145,186,230,289,...
Mathematical constructions: Joint work with N. Kashyap, Queen’s , University, AMS Meeting, Chicago 2004
CYCLEINVARIANT DIFFERENCE SETS
Definition:V an additive Abelian group, order v. (v,c,1) difference set Ω in V is a csubset of V with exactly one
ordered pair (x,y) in Ω s.t. xy=g, for a given g inV. Examples: {1,2,4} mod 7, {0,1,3,9} mod 13. “Incomplete” difference sets – not every element covered CIDS: Definition Elements of Ω arranged in ascending order. operator:cyclically shifts sequence i positions to the right For i = 1,…, m, (componentwise)
are ordered difference sets as well Ω is an (m+1)fold cycle invariant difference set over V
Example: V=Z7 and Ω ={1,2,4}, m=2
iCvCi
i mod
}4,2,1{3}5,6,3{}4,2,1{}2,1,4{1 mod 7
CIDS – GENERALIZATION OF BOSE METHOD
S = constructed as siii ,...,, 21 )}(,10:{ 2 qGFqia a
where is the primitive element of GF(q2) , q odd prime
1432
121
321
...
...............
...
...
msmsmsms
ss
s
iiii
iiii
iiii
PPPP
PPPP
PPPP
H
)}(,10:{:)( qGFqiaqGF amm
Excellent performance for very low rate codes
HIGHRATE CODES: LATIN SQUARE CONSTRUCTION
IIIPIP
IPIIIP
IIPIPI
PIIIPI
PPIIII
H
0000
0000
0000
0000
0000
22
22
22
22
22
ERROR FLOOR ANALYSIS Near codewords (Neal/MacKay, 2003) Trapping sets (Richardson, 2003) Considered to be the most outstanding problem in the area!
Inherent property of both the decoding algorithm and code structure
APPROACH: Try to predict error floor (Richardson, 2003, 2004), or Try to change the decoding algorithm
Observations leading to new decoding algorithm:1) Different computer codes give very different solutions;
quantization has a very large influence on the error floor 2) Certain variables in the code graph show extremely large and
uncontrolled jumps in likelihood values 3) Strange things happen when message values are very close to
+1/1
OBJECT OF INVESTIGATION: MARGULIS CODE Elementary trapping set (ETS): small subset of variable nodes for
which induced subgraph has very few checks of odd degree INTUITION: channel noise configuration such that all variables in
ETS are incorrect after certain number of iterations; small number of odd degree checks indicates that errors will not be easily corrected
Margulis code: based on Cayley graphs (algebraic construction), has good girth
Frame error rate curve has floor at 10E7, SNR=2.4dB, AWGN
Iteration number:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ….
182 142 108 73 66 55 38 29 20 16 15 14 14 14 14 14 14 14 14 ….
Number of erroneous bits:At “freezing” point, all values of variable messages are +1/1 and significant oscillations happen +1 → 1
Frozen activity lasts for 13 iterations: then “bad messages” propagate through the whole graph; can have jumps from 0.33 to 49.67
ALGORITHM FOR ELIMINATING THE ERROR FLOOR “Oscillations” reminiscent to problems encountered in
theory of divergent series (G.H. Hardy, Divergent Series) Trick: use a lowcomplexity “extrastep” in sumproduct
algorithm based on result by Hardy Parameters tunable: when do you “start” using “extrastep”,
what “numerical biases” should one use Can do density evolution analysis for optimization purposes
– no loss expected in waterfall region (nor one observed)(Paper with S. Laendner, in preparation)
1000 good frames: Standard and modified message passing both take 7.4 iterations on average for correct decoding
Modified message passing never takes more than 16 iterations to converge
20 Bad frames (in MATLAB): Corrected 2/3 fraction of errors after 35, 41, 26, 119, 38, 98… iterations (standard failed even after 10000
iterations)
Can use slightly more complicated result from G.H.H to reduce number of iteration to 2030
VARIABLE AND CHECK NODE ARCHITECTURE(JOINT WORK WITH N. JAYAKUMAR, S. KHATRI)
Globecom 2004, TCAS 2005
Check node Variable node
PLA
AdderArray
Adder
Adder
PLA
PLA
PLA
PLA
To variablenodes


Adder
Adder
AdderArray


From channel
From check 1
From check 2
To checks
log(tanh) and arctanh PLAsLog(dv+1)+1 Stages of TwoInput Manchester Adders, Rabaey 2000
LDPC ENCODING: SHIFT AND ADDITION UNITS FOR QUASICYCLIC CODES
ADVANTAGES OF NETWORK OF PLA ARCHITECTURE
Logic function in 2level form: good delay characteristics as long as size of each PLA is bounded (Khatri, Brayton, SangoivaniVincentelli, 2000)
Standard cell implementation: considerable delay in traversing the different levels (i.e. gates)
Local wiring collapsed into compact 2level core: crosstalkimmunity local wiring delays are reduced
Devices in PLA core are minimumsized: compact layouts NMOS devices used exclusively: devices placed extremely
close together In a standard cell layout, both PMOS and NMOS devices
are present in each cell: PMOStoNMOS diffusion spacing requirement results in loss of layout density
PLAs dynamic: faster than static standard cell implementation
Can easily accommodate reconfigurability requirement
Clocking and Logic Control
S1 Bank
S4 Bank
S2 S3
Ring of C/V Node Clusters
Check Node
Variable Nodes
C/V Clusters
HILBERT SPACE FILLING CURVES
1 2 3 4, , ,S S S S
Group variable/check nodes based on distance in code graph and place on adjacent fields in HP curve: proximity preserving
216543
321654
541216
654321
543216
654321
543216
654321
000000
000000
000000
000000
000000
000000
000000
000000
iiiiii
iiiiii
iiiiii
iiiiii
iiiiii
iiiiii
iiiiii
iiiiii
PPPPPP
PPPPPP
PPPPPP
PPPPPP
PPPPPP
PPPPPP
PPPPPP
PPPPPP
H
CHIP PARAMETERS: 0.1 MICRON PROCESS
Estimates for n=28800
Estimates for n=7200
Throughput (Gbps)
Side of chip(mm)
Power(W)
Flatout (max. duty cycle of 77.14%)
1479.4
11.0923
104.5185
50% Duty cycle 958.9158 11.0923 83.6372
Lower clock for practical applications
2
11.0923
13.3214
Throughput (Gbps) Side of chip (mm) Power (W)
Flat out (max. duty cycle of 77.14%)
369.8537 5.5461 30.4711
50% duty cycle (max. performance)
239.7289 5.5461 20.9093
Lower clock for practical applications
2 5.5461 3.4406
LDPC CODES FOR
NONSTANDARD CHANNELS
PRIOR WORK
Partial analysis of the GilbertElliott’s (GE) channel: (Eckford, Kschischang, Pasupathy, 20032004)
Polya’s Urn (PC) Channel Model (Alajaji and Fuja, 1994): Simpler description, more accurate in terms of autocorrelation function for many fading channels
B G State transition probabilities:
p(B,G), p(G,B)
Binary symmetric channel transition probability
p(G), p(B) 1p(G)
1p(G)
FINITE MEMORY POLYA URN CHANNEL
Binary additive channel with memory Occurrence of error increases probability of future errors Output sample at time i (Yi) is modulotwo sum of input Xi and noise
sequence Zi , where {Xi} and {Zi} are independent sequences The noise process is formed based on the following scheme:
Noise samples are outcomes of consecutive draws from urn, and Zi is one if ith ball was red and zero if
black
Stationary, ergodic channel with memory M Three parameters: (σ, δ, M), where ρ=R/T < ½ , σ = 1 ρ = S/T and δ =
Δ/T
M=1 PC CHANNEL (DEMONSTRATIVE EXAMPLE)
Reduced 2parameter space a = (σ + δ)/(1+ δ) and b = (1 σ + δ)/( 1+ δ) A twostate Markov chain with states S0 and S1
transition probabilities 1a and 1b Channel transition matrix is
State S0 corresponds to a noise sample of value zero, and state S1 to one
Observation: this type of channel is not included in the GE analysis by Eckford et. al.Nagarajan and Milenkovic, IEEE CCEC, 2004
bb
aa
1
1
JOINT DECODING/CHANNEL STATE ESTIMATION
Use extrinsic information from the decoder to improve the quality of the estimate of the channel state and viceversa
BCJR EQUATIONS FOR PC CHANNEL ESTIMATION
Forward message vector
Backward message vector
Channel information passed to decoder
)2(),1(
)(1)2()(1)1()1(};;)({)2(
)()1()2()()1(};;)({)1(
11
10
qbqaYSiSP
qbaqYSiSP
i
i
))2(),1((
))(1()2()()1)(1(},)({)2(
))(1)(1()2()()1(},)({)1(
1
0
qbqbSiSYP
qaaqSiSYP
ni
ni
ba
baLLR iY
211)2(
121)1(log)1(
CODE DESIGN CONSTRAINTS: ESTIMATOR
AND DECODER
Avoid short cycles from channel state estimator
Avoid short cycles from LDPC graph
Optimize degree distribution
SIMULATIONS
Performance of CIDSbased structured code in PC channel with channel estimation as compared to random codes
δ = 0.2 and σ = 0.951 to generate pairs of (a,b)
ASYMMETRIC CHANNELS: DISTRIBUTION SHAPING
Gallager 1968: Linear codes can achieve capacity on any Discrete Memoryless Channel (DMC) when combined with “distribution shapers”
Mapping not 11
Rate loss?
McEliece, 2001: Attempts to use this scheme with RepeatAccumulate or LDPC codes failed
Biggest obstacle: cannot do “soft” demapping
Solution: Use soft Huffman encoding
(extension of idea by Hagenauer et.al. for joint sourcechannel coding)
In preparation: Leyba, Milenkovic 2005
ASYMMETRIC CHANNELS: DISTRIBUTION SHAPING
Symbol Time
Bit
Time
Completion of codeword
“Interior bits”
BCJR Equations for trellis
Largest achievable probability bias
ITERATIVE DECODING OF CLASSICAL ALGEBRAIC CODES
APPLICATIONS FOR UNEQUAL ERROR PROTECTION
GRAPHICAL REPRESENTATIONS OF ALGEBEBRAIC CODES
Consider a [7,4,3] Hamming code
Idea put forward by Yedidia, Fossorier Support Sets: S1={4,5,6,7}; S2={2,3,6,7}; Intersection Set
={6,7}
Set entries corresponding to in these two rows to zero; insert new column with nonzero entries in these two rows; call this an auxiliary variable
Insert a new row with nonzero entries at positions indexed by
11
1
00101
00110
1000
11
11
H
11100000
01010101
10000110
10011000
1H
BVBV
MAXIMUM CYCLE NUMBER (MC) ALGORITHM
Approach by Sundararajan and Vasic, 2004 (SVS): use
to identify all cyclesNo attempt to minimize
number of Auxiliary variables
Key Ideas: Always select the two rows i, j involved in maximum number of cycles
422
242
224THH
426
542
324THH
Determine the coordinates i, j of the (non diagonal) largest element in T
ww HH
Is 1)( ij
Tww HH
mmnn
HHw
ww
w
1
wHOutput
No
Yes
Determine . For set,
Set, and
.0)(,0)( jxwixw HH
1,1,1 11 wwww nnmmww
.1)(,1)(,1)(,1)( wwwww nmwxmwjnwinw HHHH
x
Determine the rows l with . Set .1)(,0)( wnlwlxw HHlS
Kumar, Milenkovic: CISS’2005, IEEE CTL 2005
CYCLE OVERLAP STRATEGY (COS)
COS is a variant of MCS algorithm
Consider two rows i, j with intersection set of their support sets
Let s be number of rows with a support set such that
Choose indices that maximize
22
)( sHH ijT
lS
lS
0
,2,1
0
00
D
ji
Determine B. Construct submatrix HB of with columns indexed by B.
wH
Compute s the cardinality of the set })()(),{( ij
Twwlm
TBB HHHHandmlml
Compute D= .
22
)( sHH ijT
If then, .
Update the values of sequentially.0DD 000 ,, jjiiDD
0. ijo
Dimensions of fourcycle free paritycheck representations
Code Hamming Golay BCH RS_bin
Original
(3,7)
(5,31) (11,23)
(8,15) (10,31)
(33,63) (36,60)
SVS (4,8)
(25,31)
(29,41)
(22,29) (57,78)
(300,327)
(158,182)
MCS (4,8)
(10,36)
(23,35)
(16,23) (29,50)
(166,193)
(110,134)
COS (4,8)
(10,36)
(22,34)
(16,23) (29,50)
(157,184)
(102,126)
EXTENSION: CODES OVER NONBINARY FIELDS
Motivation Applications to ReedSolomon (RS) codes Two Approaches: Binary image mapping: Kumar, Milenkovic, Vasic CTW
2004, Milenkovic, Kumar ISIT 2004
Choice of matrix field representation and primitive polynomial, initial choice of paritycheck matrix
followed by cycle elimination for the binary paritycheck matrix
Direct cycle elimination followed by vector message passing
1210
0100
0010
m
M
DIRECT CYCLE ELIMINATION Define two types of cycles Type 1 cycle:
If are the values taken by the three variable nodes in RCF then,
Type 2 cycle:
If are the values taken by the three variable nodes in RCF then,
Type 2 cycle elimination introduces sixcycle while eliminating fourcycle Number of Type1 cycles: Number of Type2 cycles:
1
00
100
yx
hCFhyhx
yx
RR
)(},,{ 321 qGFvvv
213 vvv yx
110
0
0wz
yx
CFwz
yx
DD
)(},,{ 321 qGFvvv
23 vv
3)1( q3)1)(2( qq
ALGEBRAIC CODES AND UNEQUAL ERROR PROTECTION BASED ON
ITERATIVE DECODING Codes which provide UEP: practical applications for
Nonuniform channels Joint coding for parallel channels Image transmission over wireless channels Page oriented optical memories
Fekri 2002 – LDPC codes for UEP based on optimizing degree distributionsNo guarantee/tuning possibility for degree of protection
Dumer 2000  Decoding algorithm for RM codes based on their Plotkintype structure
Decodes different parts of the codeword separately Inherently achieves UEP for different portions of the
codeword
PLOTKIN CONSTRUCTION
Paritycheck matrix of Depth1 Plotkintype code:
Global iterative decoder not a good option: H2 placed side by side, introduces at least
fourcycles
The UEP obtained for Plotkintype codes is an inherent characteristic of the proposed decoding algorithm
As in other UEP schemes involving LDPC codes, one can vaguely associate the UEP obtained here to the different degree distributions
Encoding: Individual for all different component codes
22
1 0
HH
HH
i
ic
2
C = {uu+v ; u in C1; v in C2 }
ADVANTAGES OF THE SCHEME:
Reducedcomplexity encoding Lowcomplexity softoutput decoders Flexibility of UEP code construction, levels of UEP error
protection and adaptability to channel conditions Excellent performance for best protected components
INTUITION
uu+v: Have “two copies” of u Only one copy of v, which is implicit
MS MultiStage TMS Threshold MultiStageMRMS MultiRound MultiStage
Kumar, Milenkovic Globecom 2004, TCOMM 2004
FLOWCHART OF MS, MSMR DECODER
1/
,0)1/(
142
1,
122
1,
aforj
aforj
chcomp
chcomp
v
'yiL ''y
iL
viL
Decode v to
''' ˆ yi
yi
ui LvLL
Channel Information
Decode v to u
Output ˆˆˆ vuu
v
j times
'yiL ''y
iL
viL
Decode v to
''' ˆ yi
yi
ui LvLL
Channel Information
Decode u to u
1
,0)1/(
122
1,
122
1,
afor
aforj
chcomp
chcomp
Equivalent noise variances
THANK YOU!