Code Design in the Short Block Length RegimeExample: (2048;1024) Codes Non-Binary LDPC code from C....

October 8, 2014

Code Design in the ShortBlock Length Regime

Gianluigi Liva, [email protected]

Institute for Communications and NavigationGerman Aerospace Center, DLR

Outline

1 Introduction

2 Overview: (Some of the) Best Known Short Codes

3 Binary Codes for Iterative Decoding

4 Non-Binary Codes for Iterative Decoding

5 Beyond Iterative Decoding

6 Summary and Open Challenges

Outline

1 Introduction






Outline

1 IntroductionMotivations

Preliminaries






Page 1/158 G. Liva · Short Codes Design · Introduction October 8, 2014

IntroductionMotivations: Why Short Codes?

1. Update on the race towards the best performance, in the short blocklength regime, with some hints on the code design

2. While for long block lengths well-established tools exist to designcapacity-approaching codes (mostly, iteratively-decodable), short codes(dimension 100 < k < 1000) lack of universal design guidelines.

3. Considerable gap from theoretical bounds for moderate-length and shortcodes... Challenge!












IntroductionUsual Paradigm for Iterative Code Design

Step 1:Identify a code ensemble with near Shannon-limit performance via densityevolution1 / extrinsic information transfer2 analysis

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

I E,V

,I A

,C

IA,V , IE,C

Eb/N0 = 0.56 [dB]

VN EXIT function

CN EXIT function

1T. Richardson, A. Shokrollahi, and R. Urbanke. “Design of Capacity-Approaching IrregularLow-Density Parity-Check Codes”. In: IEEE Trans. Inf. Theory (2001).

2S. ten Brink. “Convergence Behavior of Iteratively Decoded Parallel Concatenated Codes”. In:IEEE Trans. Commun. (2001).



Step 1b:Identify a code ensemble with near Shannon-limit performance via densityevolution (DE)3 / extrinsic information transfer (EXIT)4 analysis under givenconstraints on ensemble distance properties

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

I E,V

,I A

,C

IA,V , IE,C

Eb/N0 = 0.56 [dB]

VN EXIT function

CN EXIT function

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

x 10−3

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

3x 10

−5

ω

G(ω

)

3T. Richardson, A. Shokrollahi, and R. Urbanke. “Design of Capacity-Approaching IrregularLow-Density Parity-Check Codes”. In: IEEE Trans. Inf. Theory (2001).

4S. ten Brink. “Convergence Behavior of Iteratively Decoded Parallel Concatenated Codes”. In:IEEE Trans. Commun. (2001).



Step 2:For the block length of interest, perform the finite-length code designaccording to techniques that allow picking good ensemble members (e.g., interms of minimum distance).

• For turbo codes, interleaver design5

• For low-density parity-check codes, girth optimization techniques67

5S. Crozier and P. Guinand. “High-performance low-memory interleaver banks for turbo-codes”.In: IEEE VTC. 2001.

6X.-Y. Hu, E. Eleftheriou, and D.M. Arnold. “Regular and irregular progressive edge-growthTanner graphs”. In: IEEE Trans. Inf. Theory (2005).

7T. Tian et al. “Selective Avoidance of Cycles in Irregular LDPC Code Construction”. In: IEEETrans. Commun. (2004).



• Step 1 (ensemble design) typically relies on asymptotic resultsI DE/EXIT predict the ensemble iterative decoding threshold in the limit of

infinite block lengthI The ensemble distance properties are often captured by the growth rate of

the weight distribution8

G(ω) = limn→∞

1n

logE(N(bωnc))

where E(N(bωnc)) is the expected number of codewords of weight bωnc andwe define the typical minimum distance

ω∗ = inf{ω > 0 : G(ω) ≥ 0}• Step 1 may fail to capture the ensemble performance behavior when the

block length is fixed to a few hundred bits...

8R. Gallager. Low-density parity-check codes. 1963.



• Step 1 (ensemble design) typically relies on asymptotic resultsI DE/EXIT predict the ensemble iterative decoding threshold in the limit of

infinite block lengthI The ensemble distance properties are often captured by the growth rate of

the weight distribution8

G(ω) = limn→∞

1n

logE(N(bωnc))

where E(N(bωnc)) is the expected number of codewords of weight bωnc andwe define the typical minimum distance

ω∗ = inf{ω > 0 : G(ω) ≥ 0}• Step 1 may fail to capture the ensemble performance behavior when the

block length is fixed to a few hundred bits...

8R. Gallager. Low-density parity-check codes. 1963.


IntroductionExample: (2048,1024) Codes

Multi-Edge-Type LDPCCode from

J. Thorpe. Low-DensityParity-Check (LDPC)Codes Constructed fromProtographs. IPNProgress Report. 2003

1 2 3 4 5 6 7 810−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gB

ound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

Multi-Edge-Type LDPC, dmin = O(logn), (Eb/N0)∗ = 0.25 dB



Multi-Edge-Type LDPCCode from

J. Thorpe. Low-DensityParity-Check (LDPC)Codes Constructed fromProtographs. IPNProgress Report. 2003

Bad typical minimumdistance, excellentthreshold

1 2 3 4 5 6 7 810−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gB

ound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate




Spatially-CoupledProtograph-basedLDPC code from

M. Lentmaier et al.“Approaching Capacitywith AsymptoticallyRegular LDPC Codes”.In: Information Theoryand ApplicationsWorkshop (ITA). 2009

1 2 3 4 5 6 7 810−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gB

ound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate


SC-LDPC (4,8), dmin = O(n), (Eb/N0)∗ = 0.35 dB



Spatially-CoupledProtograph-basedLDPC code from

M. Lentmaier et al.“Approaching Capacitywith AsymptoticallyRegular LDPC Codes”.In: Information Theoryand ApplicationsWorkshop (ITA). 2009

Good typical minimumdistance, excellentthreshold

1 2 3 4 5 6 7 810−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gB

ound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate





Non-Binary LDPCcode from

C. Poulliat, M. Fossorier,and D. Declercq. “Designof regular (2, dc)-LDPCcodes over GF(q) usingtheir binary images”. In:IEEE Trans. Commun.(2008)

1 2 3 4 5 6 7 810−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gB

ound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate



LDPC Code, GF(256), dmin = O(logn), (Eb/N0)∗ = 0.48 dB



Non-Binary LDPCcode from

C. Poulliat, M. Fossorier,and D. Declercq. “Designof regular (2, dc)-LDPCcodes over GF(q) usingtheir binary images”. In:IEEE Trans. Commun.(2008)

Bad typical minimumdistance, goodthreshold

1 2 3 4 5 6 7 810−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gB

ound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate



LDPC Code, GF(256), dmin = O(logn), (Eb/N0)∗ = 0.48 dB


IntroductionWhy Short Code Design is Challenging

• While for long codes asymptotic design tools work reliably, at short blocklengths we miss a universal approach to the code design problem

• As a results, the short block length iterative code design is often thedomain of

I Ad-hoc code constructions, driven by intuitionI Experimental evidenceI Lack of deep understanding

















IntroductionRace to the Best Performance For Long Blocks


IntroductionRace to the Best Performance for Short Blocks


IntroductionWhy “Short” Codes?

• Short blocks are of interest for a number of emerging applications

1. Messaging2. Machine-to-Machine3. Wireless Sensor Networks4. Data traffic originated by signaling in cellular networks

• In this presentation, we will focus on short codes under iterativedecoding due to their technology readiness

• We shall however define what is “short” is for us...














k [bits]

Eb

N0, [dB]

101 102 103 1040

1

2

3

4

5

6

7

8

PB = 10−4

R = 1/2

Non-Achievable Region

Sphere Packing Bound

PB = 10−4

R = 1/2


k [bits]

Eb

N0, [dB]

101 102 103 1040

1

2

3

4

5

6

7

8

PB = 10−4

R = 1/2



PB = 10−4

R = 1/2

Very Short Codes


k [bits]

Eb

N0, [dB]

101 102 103 1040

1

2

3

4

5

6

7

8

PB = 10−4

R = 1/2



PB = 10−4

R = 1/2

Very Short Codes Short Codes


k [bits]

Eb

N0, [dB]

101 102 103 1040

1

2

3

4

5

6

7

8

PB = 10−4

R = 1/2



PB = 10−4

R = 1/2

Very Short Codes Short Codes Long Codes

PB = 10−4

R = 1/2


k [bits]

Eb

N0, [dB]

101 102 103 1040

1

2

3

4

5

6

7

8

PB = 10−4

R = 1/2



PB = 10−4

R = 1/2


PB = 10−4

R = 1/2

Golay Code

QR Code

eBCH Code


k [bits]

Eb

N0, [dB]

101 102 103 1040

1

2

3

4

5

6

7

8

PB = 10−4

R = 1/2



PB = 10−4

R = 1/2


PB = 10−4

R = 1/2

Golay Code

QR Code

eBCH Code

Very Short Codes

(7,1/2) Conv. Codes


k [bits]

Eb

N0, [dB]

101 102 103 1040

1

2

3

4

5

6

7

8

PB = 10−4

R = 1/2



PB = 10−4

R = 1/2


PB = 10−4

R = 1/2

Golay Code

QR Code

eBCH Code

Very Short Codes

(7,1/2) Conv. Codes

Very Short Codes

CCSDS Turbo Codes


k [bits]

Eb

N0, [dB]

101 102 103 1040

1

2

3

4

5

6

7

8

PB = 10−4

R = 1/2



PB = 10−4

R = 1/2


PB = 10−4

R = 1/2

Golay Code

QR Code

eBCH Code

Very Short Codes

(7,1/2) Conv. Codes

Very Short Codes

CCSDS Turbo Codes

Long Codes

CCSDS Turbo/LDPC Codes

PB = 10−4

R = 1/2


k [bits]

Eb

N0, [dB]

101 102 103 1040

1

2

3

4

5

6

7

8

PB = 10−4

R = 1/2



PB = 10−4

R = 1/2


PB = 10−4

R = 1/2

Golay Code

QR Code

eBCH Code

Very Short Codes

(7,1/2) Conv. Codes

Very Short Codes

CCSDS Turbo Codes

Long Codes


PB = 10−4

R = 1/2

Long Codes

CCSDS LDPC Codes(Up-Link)

PB = 10−4

R = 1/2


k [bits]

Eb

N0, [dB]

101 102 103 1040

1

2

3

4

5

6

7

8

PB = 10−4

R = 1/2



PB = 10−4

R = 1/2


PB = 10−4

R = 1/2

Golay Code

QR Code

eBCH Code

Very Short Codes

(7,1/2) Conv. Codes

Very Short Codes

CCSDS Turbo Codes

Long Codes


PB = 10−4

R = 1/2

Long Codes


PB = 10−4

R = 1/2

Short Codes

DVB-RCS2 Turbo Code


k [bits]

Eb

N0, [dB]

101 102 103 1040

1

2

3

4

5

6

7

8

PB = 10−4

R = 1/2



PB = 10−4

R = 1/2


PB = 10−4

R = 1/2

Golay Code

QR Code

eBCH Code

Very Short Codes

(7,1/2) Conv. Codes

Very Short Codes

CCSDS Turbo Codes

Long Codes


PB = 10−4

R = 1/2

Long Codes


PB = 10−4

R = 1/2

Short Codes

DVB-RCS2 Turbo Code

Short Codes

F256 Turbo/LDPC Codes

Outline

1 IntroductionMotivations

Preliminaries







Channel Model

• Memory-less binary-input additive white Gaussian noise (biAWGN)channel

• Block length n (bits), code dimension k (bits), R = k/n

Y 2 RX 2 X = {�1, +1}

N ⇠ N�0,�2

�

Es

N0=

1

2�2

Eb

N0=

1

2R�2


Maximum-Likelihood Decoding

• The block-wise maximum-likelihood (ML) decoding criterion decides forthe codeword maximizing the likelihood function p(y|x), i.e.,

x = arg maxx∈C

p(y|x)

• Over the biAWGN channel, it reduces to finding the codeword aminimum Euclidean distance from the observation

Voronoi regions in Rn


Complete vs. Incomplete Decoding

• The ML decoding rule leads to complete decoding algorithms• For short codes, it is often preferable to have incomplete decoding

algorithms to reduce the probability of undetected errors9 (here, the useof error detection codes is not an option due to overhead)

• Though, we shall accept a loss w.r.t. ML decoding...

Voronoi regions in Rn

Decision regions of the iterative decoder

9S. Dolinar et al. “Bounds on error probability of block codes with bounded-anglemaximum-likelihood incomplete decoding”. In: International Symposium on Information Theoryand Its Applications. 2008.


Binary Linear Block Codes

• A (n, k) binary linear block code C can be defined by its k × n binarygenerator matrix G, whose k rows span the k -dimensional code space

• Alternatively, it may be defined through its (n − k)× n binaryparity-check matrix H via the equation

xHT = 0

where 0 is the n-elements all-zero vector• The parity-check matrix rows span the subspace orthogonal to C. We

denote such subspace as C⊥. Thus, we may write

x⊥v ∀x ∈ C, v ∈ C⊥.

• The vectors in C⊥ form a linear block code of dimension n − k , referredto as dual code of C


Turbo CodesParallel Concatenation Ensembles

• For given component codes and for a given interleaver size k , the turbocode ensemble Ck = {C1, C2, . . . } is the set of codes obtained byselecting all possible interleavers with uniform probability

D D

⇧

u

p(1)

p(2)

u0

First RSC Encoder

Second RSC EncoderD D


Low-Density Parity-Check CodesGraphical Representation of the Parity-Check Matrix

• The H imposes a set of n − k constraints to the n codeword bits• Graphical representation via Tanner graphs10

I Codeword bits ≡ variable nodes (VNs) Vi , i = 1, . . . , nI Check equations ≡ check nodes (CNs) Ci , i = 1, . . . , n − kI An edge connects Vi to Cj if and only if hj,i = 1

H =

1 1 0 1 0 01 0 1 0 1 00 1 1 0 0 1

V1 V2 V3 V4 V5 V6

C1 C2 C3

check nodes

variable nodes

10M. Tanner. “A Recursive Approach to Low Complexity Codes”. In: IEEE Trans. Inf. Theory(1981).


Low-Density Parity-Check CodesGraphical Representation of the Parity-Check Matrix

• Graphical representation via Tanner graphs (cont’d)I Degree of a node = number of edges attached to itI Length-l cycle = closed path composed by l edges, with each node in the

cycle visited only onceI Girth of a graph = length of the shortest cycle

V1 V2 V3 V4 V5 V6

C1 C2 C3

check nodes

variable nodes

degree= 2

length�6 cycle


Low-Density Parity-Check CodesRegular LDPC Codes Ensemble

• The C(dv ,dc ),n regular unstructured LDPC ensemble is the set of binarylinear block codes defined by a Tanner graph with

I n variable nodes, having degree dvI ndv/dc check nodes with degree dcI Edge permutation picked uniformly at random

V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12

C1 C2 C3 C4 C5 C6 C7 C8 C9


Low-Density Parity-Check CodesIrregular LDPC Codes Ensemble

• An irregular LDPC code graph is often characterized by the distributionsof degrees of variable and check nodes

• The node-oriented variable node degree distribution is denoted byΛ = {Λi}, i = 1, . . . , dv,max where Λi is the fraction of VNs with degree i

• The node-oriented check node degree distribution is denoted byP = {Pj}, j = 1, . . . , dc,max where Pj is the fraction of CNs with degree j

V1 V2 V3 V4 V5 V6 V7 V8 V9 V10

C1 C2 C3 C4 C5

degree-4

degree-5



• Alternatively, edge-oriented degree distributions can be defined• The edge-oriented variable node degree distribution is denoted byλ = {λi}, i = 1, . . . , dv,max where λi is the fraction of edges connected toVNs with degree i

• The edge-oriented check node degree distribution is denoted byρ = {ρj}, j = 1, . . . , dc,max where ρj is the fraction of edges connected toCNs with degree j

λi =iΛi∑l lΛl

, ρi =iPi∑l lPl

V1 V2 V3 V4 V5 V6 V7 V8 V9 V10

C1 C2 C3 C4 C5

degree-4

degree-5



• Degree distributions are often given in polynomial form

Λ(x) =∑

i

Λix i and P(x) =∑

j

Pjx j

λ(x) =∑

i

λix i−1 and ρ(x) =∑

j

ρjx j−1

• We have that

λ(x) =Λ′(x)

Λ′(1)and ρ(x) =

P′(x)

P′(1)

withΛ′(x) =

dΛ(x)

dxand P′(x) =

dP(x)

dx.



• The C(Λ,P),n irregular unstructured LDPC ensemble is the set of binarylinear block codes defined by a Tanner graph with

I n variable nodes whose degrees are distributed according to ΛI m check nodes with degrees are according to PI Edge permutation picked uniformly at random

V1 V2 V3 V4 V5 V6 V7 V8 V9 V10

C1 C2 C3 C4 C5

degree-4

degree-5


Finite-Length Bounds

• We consider upper and lowerbounds as benchmarks

• Shannon’s 1959 sphere-packinglower bound ([Sha59]) on theblock error probability of(n,M = 2k ) codes withequal-energy codewords

• Gallager’s random coding upperbound ([Gal68]) on the averageblock error probability of(n,M = 2k ) binary codes

• See also [DDP98; SS06; PPV10]

0.5 1 1.5 2 2.5 3 3.510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

Outline

1 Introduction






Outline

1 Introduction

2 Overview: (Some of the) Best Known Short CodesModerate-length Codes (k = 1024)

Short Codes (k = 256)

Very Short Codes (k = 64)





Page 37/158 G. Liva · Short Codes Design · Overview: (Some of the) Best Known Short Codes October 8, 2014

Moderate-length CodesDimension k = 1024 bits

Binary LDPC Codes

Binary Turbo Codes

Product Codes

Non-binary LDPC Codes

Non-binary Turbo Codes

Gallager’s (3, 6) Code

R. Gallager. Low-density parity-checkcodes. 1963

0.5 1 1.5 2 2.5 3 3.510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC



Binary LDPC Codes

Binary Turbo Codes

Product Codes



Accumulator-based

Protograph LDPC Codes

D. Divsalar et al. “Capacity-approachingprotograph codes”. In: IEEE JSAC(2009)

0.5 1 1.5 2 2.5 3 3.510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A



Binary LDPC Codes

Binary Turbo Codes

Product Codes



Protograph Raptor Codes

T.-Y. Chen et al. Protograph-BasedRaptor-Like LDPC Codes. 2014. URL:http://arxiv.org/abs/1403.2111

0.5 1 1.5 2 2.5 3 3.510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A

Protograph Raptor

http://arxiv.org/abs/1403.2111



Binary LDPC Codes

Binary Turbo Codes

Product Codes



16-States Turbo Codes with

Dithered Relative Prime

Interleaver

S. Crozier and P. Guinand.“High-performance low-memoryinterleaver banks for turbo-codes”. In:IEEE VTC. 2001

0.5 1 1.5 2 2.5 3 3.510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A

Protograph Raptor

Turbo Code (16 states)



Binary LDPC Codes

Binary Turbo Codes

Product Codes



Protograph-based

Generalized LDPC Codes

G. Liva, W. Ryan, and M. Chiani.“Quasi-cyclic Generalized LDPC codeswith low error floors”. In: IEEE Trans.Commun. (2008)

0.5 1 1.5 2 2.5 3 3.510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A

Protograph Raptor


G-LDPC (Irregular)



Binary LDPC Codes

Binary Turbo Codes

Product Codes



Ultra-Sparse LDPC Codes

over F256

C. Poulliat, M. Fossorier, and D. Declercq.“Design of regular (2, dc )-LDPC codesover GF(q) using their binary images”.In: IEEE Trans. Commun. (2008)

0.5 1 1.5 2 2.5 3 3.510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A

Protograph Raptor


G-LDPC (Irregular)

LDPC Code, GF(256)



Binary LDPC Codes

Binary Turbo Codes

Product Codes



Memory-1 Turbo Codes over

F256

G. Liva et al. “Short Turbo Codes overHigh Order Fields”. In: IEEE Trans.Commun. (2013)

0.5 1 1.5 2 2.5 3 3.510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A

Protograph Raptor


G-LDPC (Irregular)

LDPC Code, GF(256)

Turbo Code, GF(256)

Outline

1 Introduction









Short CodesDimension k = 256 bits

Binary LDPC Codes

Binary Turbo Codes

Product Codes





0.5 1 1.5 2 2.5 3 3.5 4 4.5 510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC



Binary LDPC Codes

Binary Turbo Codes

Product Codes



Accumulator-based



0.5 1 1.5 2 2.5 3 3.5 4 4.5 510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A

Protograph (NASA-CCSDS)



Binary LDPC Codes

Binary Turbo Codes

Product Codes





0.5 1 1.5 2 2.5 3 3.5 4 4.5 510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor




Binary LDPC Codes

Binary Turbo Codes

Product Codes





Interleaver


0.5 1 1.5 2 2.5 3 3.5 4 4.5 510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor




Binary LDPC Codes

Binary Turbo Codes

Product Codes



Protograph-based



0.5 1 1.5 2 2.5 3 3.5 4 4.5 510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor


G-LDPC (Irregular)



Binary LDPC Codes

Binary Turbo Codes

Product Codes




over F256


0.5 1 1.5 2 2.5 3 3.5 4 4.5 510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor


G-LDPC (Irregular)

LDPC Code, GF(256)



Binary LDPC Codes

Binary Turbo Codes

Product Codes




F256


0.5 1 1.5 2 2.5 3 3.5 4 4.5 510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor


G-LDPC (Irregular)

LDPC Code, GF(256)

Turbo Code, GF(256)

Outline

1 Introduction









Very Short CodesDimension k = 64 bits

Binary LDPC Codes

Binary Turbo Codes

Product Codes





1 2 3 4 5 6 7 8 910−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC



Binary LDPC Codes

Binary Turbo Codes

Product Codes



Accumulator-based



1 2 3 4 5 6 7 8 910−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A




Binary LDPC Codes

Binary Turbo Codes

Product Codes





1 2 3 4 5 6 7 8 910−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor




Binary LDPC Codes

Binary Turbo Codes

Product Codes





Interleaver


1 2 3 4 5 6 7 8 910−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor




Binary LDPC Codes

Binary Turbo Codes

Product Codes



Protograph-based



1 2 3 4 5 6 7 8 910−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor


G-LDPC (Irregular)



Binary LDPC Codes

Binary Turbo Codes

Product Codes




over F256


1 2 3 4 5 6 7 8 910−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor


G-LDPC (Irregular)

LDPC Code, GF(256)



Binary LDPC Codes

Binary Turbo Codes

Product Codes




F256


1 2 3 4 5 6 7 8 910−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor


G-LDPC (Irregular)

LDPC Code, GF(256)

Turbo Code, GF(256)

Outline

1 Introduction






Outline

1 Introduction


3 Binary Codes for Iterative DecodingProtograph-based Low-Density Parity-Check Codes

Turbo Codes

Product and Generalized Low-Density Parity-Check Codes




Page 58/158 G. Liva · Short Codes Design · Binary Codes for Iterative Decoding October 8, 2014

LDPC Codes: Unstructured vs. Structured EnsemblesBasics

• Our definition of LDPC ensemble leads to graphs with limited (if any)structure

• Unstructured graphs are easy to analyze, butI Are difficult to implement in actual (hardware) decodersI Allow limited control on edge connectivity properties

• A refined definition of LDPC ensemble addressing the points aboveleads to structured LDPC ensembles

I Repeat-Accumulate-like Codes11

I Protograph Codes12

I Multi-Edge-Type Codes13

11H. Jin, A. Khandekar, and R. McEliece. “Irregular repeat-accumulate codes”. In: Proc. IEEEInt. Symp. Turbo Codes and Related Topics. 2000.

12J. Thorpe. Low-Density Parity-Check (LDPC) Codes Constructed from Protographs. IPNProgress Report. 2003.

13T. Richardson and R. Urbanke. Multi-edge type LDPC codes. unpublished. 2004.


LDPC Codes: Structured Ensembles

H =

H = Unstructured LDPC Code

Structured LDPC Code


LDPC Codes: Structured EnsemblesProtograph Codes

• Protograph: small Tanner graph used as template to build the actualcode graph

• Equivalent representation: base matrix

Type-1 VN Type-2 VN Type-3 VN

Type-1 CN Type-2 CN

B =

✓2 1 01 1 1

◆



• A protograph can be used to construct a larger Tanner graph by a copy &permute procedure

• The larger Tanner graph defines the designed code

• First step: protograph is copied Q times

Type-1 VNs Type-2 VNs Type-3 VNs

Type-1 CNs Type-2 CNs

B0 =

0BBBBBBBBBB@

2 0 0 0 1 0 0 0 0 0 0 00 2 0 0 0 1 0 0 0 0 0 00 0 2 0 0 0 1 0 0 0 0 00 0 0 2 0 0 0 1 0 0 0 01 0 0 0 1 0 0 0 1 0 0 00 1 0 0 0 1 0 0 0 1 0 00 0 1 0 0 0 1 0 0 0 1 00 0 0 1 0 0 0 1 0 0 0 1

1CCCCCCCCCCA



• Second step: permute edges among the replicas• Permutations shall avoid parallel edges



H =

0BBBBBBBBBB@

1 1 0 0 0 1 0 0 0 0 0 01 0 1 0 0 0 0 1 0 0 0 00 0 1 1 0 0 1 0 0 0 0 00 1 0 1 1 0 0 0 0 0 0 01 0 0 0 1 0 0 0 0 1 0 00 0 1 0 0 0 0 1 0 0 1 00 1 0 0 0 0 1 0 1 0 0 00 0 0 1 0 1 0 0 0 0 0 1

1CCCCCCCCCCA




⇧1,1 ⇧1,2 ⇧2,1 ⇧2,2 ⇧2,3



H =

0BBBBBBBBBB@

1 1 0 0 0 1 0 0 0 0 0 01 0 1 0 0 0 0 1 0 0 0 00 0 1 1 0 0 1 0 0 0 0 00 1 0 1 1 0 0 0 0 0 0 01 0 0 0 1 0 0 0 0 1 0 00 0 1 0 0 0 0 1 0 0 1 00 1 0 0 0 0 1 0 1 0 0 00 0 0 1 0 1 0 0 0 0 0 1

1CCCCCCCCCCA

• A protograph defines structured LDPC code ensemble: The iterativedecoding threshold and distance properties follow from the protograph




⇧1,1 ⇧1,2 ⇧2,1 ⇧2,2 ⇧2,3



H =

0BBBBBBBBBB@

1 1 0 0 0 1 0 0 0 0 0 01 0 1 0 0 0 0 1 0 0 0 00 0 1 1 0 0 1 0 0 0 0 00 1 0 1 1 0 0 0 0 0 0 01 0 0 0 1 0 0 0 0 1 0 00 0 1 0 0 0 0 1 0 0 1 00 1 0 0 0 0 1 0 1 0 0 00 0 0 1 0 1 0 0 0 0 0 1

1CCCCCCCCCCA

• A protograph defines structured LDPC code ensemble: The iterativedecoding threshold and distance properties follow from the protograph



• Depending on code length, the expansion can be done in more steps• In each step, girth optimization techniques1415 are used• The final expansion is usually performed by means of circulant

permutation matrices (quasi-cyclic code)1617



H =

0BBBBBBBBBB@

1 1 0 0 0 1 0 0 0 0 0 01 0 1 0 0 0 0 1 0 0 0 00 0 1 1 0 0 1 0 0 0 0 00 1 0 1 1 0 0 0 0 0 0 01 0 0 0 1 0 0 0 0 1 0 00 0 1 0 0 0 0 1 0 0 1 00 1 0 0 0 0 1 0 1 0 0 00 0 0 1 0 1 0 0 0 0 0 1

1CCCCCCCCCCA



16W. Ryan and S. Lin. Channel codes – Classical and modern. Cambridge Univ. Press, 2009.17D. Declercq, M. Fossorier, and E. Biglieri. Channel Coding: Theory, Algorithms, and

Applications. Academic Press Library, 2014.



• Depending on code length, the expansion can be done in more steps• In each step, girth optimization techniques1819 are used• The final expansion is usually performed by means of circulant

permutation matrices (quasi-cyclic code)2021



H =

1 1 0 0 0 1 0 0 0 0 0 01 0 1 0 0 0 0 1 0 0 0 00 0 1 1 0 0 1 0 0 0 0 00 1 0 1 1 0 0 0 0 0 0 01 0 0 0 1 0 0 0 0 1 0 00 0 1 0 0 0 0 1 0 0 1 00 1 0 0 0 0 1 0 1 0 0 00 0 0 1 0 1 0 0 0 0 0 1



20W. Ryan and S. Lin. Channel codes – Classical and modern. Cambridge Univ. Press, 2009.21D. Declercq, M. Fossorier, and E. Biglieri. Channel Coding: Theory, Algorithms, and

Applications. Academic Press Library, 2014.



• Punctured (state) and degree-1 variable nodes are allowed, while forunstructured designs they may prevent the iterative decodingconvergence process to succeed

• W.r.t. unstructured ensembles, near-capacity thresholds can beachieved with lower average degrees→larger girth

• Example: Accumulate-Repeat-3-Accumlate (AR3A), R = 1/2,(Eb/N0)∗ = 0.475 dB, only 0.3 dB from Shannon limit

Punctured VN



• Protograph ensembles allow controlling in finer way the edgeconnectivity properties of the codes22

• Example: consider the based matrices

B1 =

1 2 1 1 02 1 1 1 01 2 0 0 1

B2 =

1 3 1 0 02 1 1 1 01 1 0 1 1

They describe sub-ensembles of the unstructured ensemble defined by

Λ(x) = 0.2x + 0.4x2 + 0.2x4 + 0.2x5, P(x) = 0.333x4 + 0.667x5

with degree-5 VNs punctured. However, (Eb/N0)∗1 = 0.475 dB while(Eb/N0)∗2 = +∞ dB.

22G. Liva and M. Chiani. “Protograph LDPC codes design based on EXIT analysis”. In: Proc.IEEE Global Telecommun. Conf. 2007.


LDPC Codes: Protograph EnsemblesExtrinsic Information Transfer Analysis

B =

✓2 1 01 1 1

◆

Ich Ich Ich

IEv(1, 1) IEc(2, 3)

• Ich: channel mutual information (MI) at the input ofthe j-th protograph VN

• IEv (i, j): the MI between the message sent by Vj to Ci

and the associated bit, on one of the bi,j edgesconnecting Vj to Ci

• IEc(i, j): the MI between the message sent by Ci to Vj

and the associated codeword bit, on one of the bi,j

edges connecting Ci to Vj

• IAPP(j): the MI between the APP LLR and theassociated codeword bit



• Each message (LLR) is modeled as a r.v. M with Gaussian distributionconditioned on the associated codeword symbol r.v. X

p(m| ± 1) =1√

2πσ2m

exp(

12σ2

m

(m ∓ σ2

m

2

))• We denote by J(σm) := I(M; X )

• Note that

J(x) = 1−

∫+∞

−∞

e−(y−x2/2)2

2x2

√2πx2

· log2

(1 + e−y) dy .



The EXIT recursions of unstructuredLDPC ensembles

IEv =∑

d

λd J(√

(J−1 (Ich))2

+(d − 1)(J−1

(IAv))2

IEc ≈ 1−∑

d

ρd J(√

(d − 1)(J−1

(1− IAc

))2)

need to be modified to account for theprotograph structure...

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

I E,V

,I A

,C

IA,V , IE,C

Eb/N0 = 1.1 [dB]

VN EXIT function

CN EXIT function



B =

✓2 1 01 1 1

◆

Ich Ich Ich

IEv(1, 1) IEc(2, 3)

Initialize Ich = J (σch) with

σ2ch = 8R

Eb

N0

For punctured VNs, obviously Ich = 0



B =

✓2 1 01 1 1

◆

Ich Ich Ich

IEv(1, 1) IEc(2, 3)

Variable to check update∀j, i , if bi,j 6= 0,

IEv (i, j) ≈

J

(√∑s 6=i

bs,j [J−1 (IAv (s, j))]2 +

+ (bi,j − 1) [J−1 (IAv (i, j))]2 +[J−1

(I(j)ch

)]2)

∀j, i set IAc(i, j) = IEv (i, j)



B =

✓2 1 01 1 1

◆

Ich Ich Ich

IEv(1, 1) IEc(2, 3)

Check to variable update∀j, i , if bi,j 6= 0,

IEc(i, j) ≈

1− J

(√∑s 6=j

bi,s [J−1 (1− IAc(i, s))]2 +

+ (bi,j − 1)[J−1 (1− IAc(i, j))

]2)

∀j, i , set IAv (i, j) = IEc(i, j)



B =

✓2 1 01 1 1

◆

Ich Ich Ich

IEv(1, 1) IEc(2, 3)

APP-LLR mutual information evaluation

IAPP(j) ≈ J

(√∑s

bs,j [J−1 (IAv (i, j))]2 +[J−1

(I(j)ch

)]2)

The steps are iterated until IAPP(j)→ 1, ∀j , or a maximumnumber of iterations (Imax ) is reached

The iterative decoding threshold, (Eb/N0)∗, is thesmallest signal-to-noise ratio such that IAPP(j)→ 1 ∀j


Protograph Ensembles: Accumulator-basedRepeat-Accumulate

• Accumulator-based LDPC codes admit simple protograph representation• Very low decoding thresholds with low average degrees→ large girth• A clear example is given by repeat-accumulate (RA) codes23

Repetition-3

(Tail-biting)Accumulator

Information bits

Parity bits

23D. Divsalar et al. “Constructing LDPC codes from simple loop-free encoding modules”. In:Proc. IEEE Int. Conf. Commun. (ICC). 2005.


Protograph Ensembles: Accumulator-basedExample: AR3A Protograph

Accumulate-Repeat-3-Accumulate (AR3A) Protograph24

Punctured VN

dmin Threshold, (Eb/N0)∗ EncodingO(log n) 0.475 dB O(n)

24D. Divsalar et al. “Capacity-approaching protograph codes”. In: IEEE JSAC (2009).


Protograph Ensembles: Accumulator-basedExample: AR4A Protograph

Accumulate-Repeat-4-Accumulate (AR4A) Protograph25

Punctured VN

dmin Threshold, (Eb/N0)∗ EncodingO(log n) 0.550 dB O(n)

25D. Divsalar et al. “Capacity-approaching protograph codes”. In: IEEE JSAC (2009).


Protograph Ensembles: Accumulator-basedARA Protograph Thresholds

R AR3A, (Eb/N0)∗ AR4A, (Eb/N0)∗ Shannon Limit7/8 3.139 dB 3.196 dB 2.845 dB4/5 2.283 dB 2.391 dB 2.040 dB3/4 1.845 dB 1.977 dB 1.626 dB2/3 1.295 dB 1.417 dB 1.059 dB1/2 0.475 dB 0.550 dB 0.187 dB1/3 −0.050 dB 0.020 dB −0.495 dB1/4 −0.554 dB −0.508 dB −0.794 dB1/5 −0.755 dB −0.550 dB −0.964 dB1/6 −0.791 dB −0.778 dB −1.073 dB1/8 −0.962 dB −0.877 dB −1.204 dB


Protograph Ensembles: Accumulator-basedEnsembles with Positive Typical Minimum Distance

• ARA codes suffer for dmin growing sublinearly with n• By limiting the number of degree-2 VNs in the protograph to be strictly

less than the number of parity VNs, it is possible to guarantee a lineargrowth of dmin

• Accumulate-Repeat-Jagged-Accumulate (ARJA) codes26

dmin (Eb/N0)∗ EncodingO(n) 0.628 dB O

(n2)

Punctured VN

26D. Divsalar, S. Dolinar, and C. Jones. “Construction of protograph LDPC codes with linearminimum distance”. In: IEEE International Symposium on Information Theory. 2006.


Protograph Ensembles: Accumulator-basedDimension k = 1024 bits

0.5 1 1.5 2 2.5 3 3.510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A



0.5 1 1.5 2 2.5 3 3.5 4 4.5 510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A




1 2 3 4 5 6 7 8 910−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A



Protograph Ensembles: Raptor-like

• Serial concatenation of a high-rate protograph-based outer LDPC code,and a protograph-based Luby-Transform code27

B =

(Bo 0

BLT

)• Although the construction targets short block lengths, the outer code

parity-check matrix density prevents from obtaining large girths at veryshort block lengths

27T.-Y. Chen et al. Protograph-Based Raptor-Like LDPC Codes. 2014. URL:http://arxiv.org/abs/1403.2111.



Protograph Ensembles: Raptor-like

Large flexibility of rates, with thresholds within 0.5 dB from the Shannon limit

B =

0BBBBBBBBBBBBBBBBBBBB@

2 1 2 1 2 1 2 1 0 0 0 0 0 0 0 0 0 0 01 2 1 2 1 2 1 2 0 0 0 0 0 0 0 0 0 0 02 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 02 0 1 0 1 0 1 0 0 1 0 0 0 0 0 0 0 0 01 0 1 1 1 0 1 1 0 0 1 0 0 0 0 0 0 0 01 0 1 0 1 0 1 1 0 0 0 1 0 0 0 0 0 0 01 0 1 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 01 0 1 0 1 0 0 1 0 0 0 0 0 1 0 0 0 0 01 0 1 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 01 0 1 0 1 0 1 1 0 0 0 0 0 0 0 1 0 0 01 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 1 0 01 0 1 0 0 1 0 1 0 0 0 0 0 0 0 0 0 1 01 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1

1CCCCCCCCCCCCCCCCCCCCA

R (Eb/N0)∗ Shannon Limit

6/7 3.077 dB 2.625 dB6/8 1.956 dB 1.626 dB6/9 1.392 dB 1.059 dB6/10 1.078 dB 0.679 dB6/11 0.798 dB 0.401 dB6/12 0.484 dB 0.187 dB6/13 0.338 dB 0.018 dB6/14 0.144 dB −0.122 dB6/15 0.072 dB −0.238 dB6/16 0.030 dB −0.337 dB6/17 −0.024 dB −0.422 dB6/18 −0.150 dB −0.495 dB

Other constructions28 allow to trade threshold for better error floorperformance

28T.-Y. Chen et al. Protograph-Based Raptor-Like LDPC Codes. 2014. URL:http://arxiv.org/abs/1403.2111.



Protograph Ensembles: Raptor-likeDimension k = 1024 bits

0.5 1 1.5 2 2.5 3 3.510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A

Protograph Raptor



0.5 1 1.5 2 2.5 3 3.5 4 4.5 510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor



1 2 3 4 5 6 7 8 910−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor

Outline

1 Introduction



Turbo Codes






The Parallel Concatenated Convolutional Code Ensemble

• Turbo codes, in their parallel concatenation form, are excellentcandidates in the short block length regime

• Turbo codes with 16-states component codes provide the best trade-offbetween minimum distance and decoding threshold2930

• Interleaver design is crucial31

FB/FFW Polynomial (Octal) (Eb/N0)∗, R = 1/2 Notes

27/37 0.56 dB 16-states23/35 0.62 dB 16-states15/13 0.70 dB 8-states

29C. Berrou, A. Glavieux, and P. Thitimajshima. “Near Shannon limit error-correcting coding anddecoding: Turbo-codes”. In: Proc. IEEE Int. Conf. Commun. (ICC). 1993.

30H. El-Gamal and Jr. Hammons AR. “Analyzing the turbo decoder using the Gaussianapproximation”. In: IEEE Trans. Inf. Theory (2001).



The Parallel Concatenated Convolutional Code EnsembleFactor Graph

Turbo codes factor graphs32 are characterized by large girths

⇧

32N. Wiberg. “Codes and Decoding on General Graphs”. PhD thesis. Linköping University, 1996.


The Parallel Concatenated Convolutional Code EnsembleInterleavers

• The interleaver is the main responsible for a large girth and for a largespread (essential for large dmin)

• Still, dmin = O(log n)

• Among the best-known constructionsI Dithered-Relative-Prime (DRP)33

I Quadratic permutation polynomial (QPP) - LTE34


34O. Takeshita. “On maximum contention-free interleavers and permutation polynomials overinteger rings”. In: IEEE Trans. Inf. Theory (2006).


The Parallel Concatenated Convolutional Code EnsembleDithered Relative Prime Interleavers

• Relative-prime interleavers are characterized by excellent spread• Though, they are too regular→ large multiplicity of minimum-weight

codewords→ local “dither” to reduce multiplicity

Relative-prime interleaver

⇡⇡

⇡⇡

Input word

Input dither

Output dither

Output word

M bits

⇡i1 ⇡i

2 ⇡i3 ⇡i

4 ⇡i5 ⇡i

k/M

⇡ok/M⇡o

5⇡o4⇡o

3⇡o2⇡o

1

...

...


The Parallel Concatenated Convolutional Code EnsembleDimension k = 1024 bits

0.5 1 1.5 2 2.5 3 3.510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A

Protograph Raptor




0.5 1 1.5 2 2.5 3 3.5 4 4.5 510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor




1 2 3 4 5 6 7 8 910−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor


Outline

1 Introduction



Turbo Codes






Product CodesBasics

• Product Codes35 are the ancestors of any iterative coding scheme• Simplest case: 2-dimensional product code C = C1 × C2

I Obtained from two binary linear block codes C1, C2 with parameters (n1, k1)and (n2, k2)

I k Information bits are organized in a k2 × k1 array U (with k1k2 = k )I Each row of U is then encoded via the (n1, k1) binary linear block code C1I The resulting k2 × n1 array

is then encoded column-wise through an (n2, k2) binary linear block codeC2, leading to an n2 × n1 array C with the structure

C =

[U P(1)

P(2) P(12)

]

35P. Elias. “Error-free coding”. In: IRE Trans. on Inform. Theory (1954).


Product CodesGraphical Representation

• Decoding takes place iteratively, row and column-wise (block turbocodes)

• Component code SISO decoding based on its trellis (BCJR) or onmodified-Chase algorithms

information array Uk2 ⇥ k1 n2 ⇥ n1

codeword array C

codeword of C1

codeword of C2

checks-on-checks, P(1,2)

parity of C1, P(1)

parity of C2, P(2)

1

0

0

0

0

0

0

0

0

0

0

0 0 0 0

1

1

1

1 10

0 1 1 1

1

01 1

0 0

1 0 0 0

0

0

1

1

0

11

Column Check Nodes

Row Check Nodes


Product CodesDistance Properties

• Code parameters:

n = n1n2 k = k1k2 R = (k1/n1)(k2/n2) = R1R2

• The minimum distance is dmin = d1d2

• The multiplicity of codeword with minimum Hamming weight is given by

Amin = A(1)d1

A(2)d2,

i.e., the minimum distance multiplicity of the product code is the productof the minimum distance multiplicities of the component codes!



• Example: (1024, 676) product code based a (32, 26) extendedHamming code for both rows and columns (IEEE 802.16)

• The (32, 26) extended Hamming code has weight enumerator function(WEF) given by

A1(X ) = 1 + 1240X 4 + 27776X 6 + 330460X 8 + 2011776X 10

+ 7063784X 12 + 14721280X 14 + 18796230X 16

+ 14721280X 18 + 7063784X 20 + 2011776X 22

+ 330460X 24 + 2777X 26 + 1240X 28 + X 32.

• Component codes minimum distance is d1 = d2 = 4 resulting in aproduct code minimum distance dmin = 16

• The multiplicity of codewords with Hamming weight 16 isAmin = 12402 = 1537600



• The high multiplicity of minimum-weight codewords leads to high errorfloors

PB ≈12

Aminerfc

(√dminR

Eb

N0

)(n, k ) C1 C2 dmin Amin

(256, 121) eH (16, 11) eH (16, 11) 16 19600(256, 165) eH (16, 11) SPC (16, 15) 8 16800(256, 225) SPC (16, 15) SPC (16, 15) 4 14400

(1024, 676) eH (32, 26) eH (32, 26) 16 1537600(1024, 806) eH (32, 26) SPC (32, 31) 8 615040(1024, 961) SPC (32, 31) SPC (32, 31) 4 246016

• The large multiplicity is mainly due to the highly-regular graph structure...


Generalized LDPC CodesBasics

• Generalized LDPC (G-LDPC)3637 codes extends both LDPC and productcode constructions

• Tanner graph includes generalized check nodes based on arbitrary linearblock codes

V1 V2 V3 V4 V5 V6 V7 V8 V9 V10

C1 C2 C3 C4 C5

Ca CaCb

Ha =

✓0 1 1 01 1 0 1

◆Hb =

0@

1 0 1 1 01 0 1 1 10 1 1 0 1

1A

single-parity-check

36M. Tanner. “A Recursive Approach to Low Complexity Codes”. In: IEEE Trans. Inf. Theory(1981).

37M. Lentmaier and K. S. Zigangirov. “Iterative decoding of generalized low-density parity-checkcodes”. In: Proc. IEEE Int. Symp. Inf. Theory (ISIT). 1998.


Generalized LDPC CodesStructured Designs

• One can target G-LDPC codes having same component codes asproduct codes, but with a graph structure that prevents large minimumweight multiplicity38

• Protograph G-LDPC codes39 as protograph LDPC codes, but withgeneralized check nodes

(32,6) extended Hamming code (32,6) extended Hamming code

38M. Lentmaier et al. “From Product Codes to Structured Generalized LDPC Codes”. In: Proc.Chinacom. 2010.

39G. Liva, W. Ryan, and M. Chiani. “Quasi-cyclic Generalized LDPC codes with low error floors”.In: IEEE Trans. Commun. (2008).


Generalized LDPC CodesExample - Union Bounds

Product and G-LDPC codesbased on (32, 26) extendedHamming component codes

For the G-LDPC code it ispossible to design a graphleading to the same girth andminimum distance of theproduct code

Two G-LDPC ensembles:The complete ensemble, andthe one expurgated from allcodes having dmin < 16

It is possible to show that atleast 50% of the codes in theoriginal ensemble havedmin = 16 or more

1 2 3 4 5 6 710−14

10−12

10−10

10−8

10−6

10−4

10−2

100

Eb/N0 [dB]

Blo

ckE

rror

Rate

Truncated UB, Product Code

UB G-LDPC Expurgated Ensemble

UB G-LDPC Non-Expurgated Ensemble

Divsalar Bound, G-LDPC Expurgated Ensemble


Generalized LDPC CodesExample

Product and G-LDPC codesbased on (32, 26) extendedHamming component codes

For the G-LDPC code graph,girth equal to the productcode one

G-LDPC obtained bycirculant permutation matrixexpansion (quasi-cyclic code)

1 1.5 2 2.5 3 3.5 4 4.5 510−6

10−5

10−4

10−3

10−2

10−1

100

Eb/N0 [dB]

Blo

ckE

rror

Rate

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Product Code

G-LDPC Code


Generalized LDPC CodesDimension k = 1024 bits

0.5 1 1.5 2 2.5 3 3.510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A

Protograph Raptor


G-LDPC (Irregular)



0.5 1 1.5 2 2.5 3 3.5 4 4.5 510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor


G-LDPC (Irregular)



1 2 3 4 5 6 7 8 910−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor


G-LDPC (Irregular)

Outline

1 Introduction






Outline

1 Introduction



4 Non-Binary Codes for Iterative DecodingLow-Density Parity-Check Codes

Parallel Concatenated Convolutional Codes



Page 107/158 G. Liva · Short Codes Design · Non-Binary Codes for Iterative Decoding October 8, 2014

Non-Binary LDPC CodesBasics

• Defined40 by M × N sparse parity-check matrix H via the equation

xHT = 0

where x ∈ Fnq and the elements of H belong to Fq , q > 2

• We are mainly interested in q = 2p

• Example: parity-check matrix over F16

H =

[α3 α7 α3 0α0 0 α12 α11

]where the field elements are expressed as powers of the primitiveelement α and where the field primitive polynomial is p(x) = 1 + x + x4

40M. Davey and D. MacKay. “Low density parity check codes over GF(q)”. In: IEEE Commun.Lett. (1998).


Non-Binary LDPC CodesBasics

• Tanner graph representationI Codeword symbols ≡ variable nodes (VNs) Vi , i = 1, . . . ,NI Non-Binary Check equations ≡ check nodes (CNs) Ci , i = 1, . . . ,MI An edge connects Vi to Cj if and only if hj,i 6= 0

H =

α2 α0 0 α7 0 0α5 0 α11 0 α0 00 α2 α3 0 0 α10

V1 V2 V3 V4 V5 V6

C1 C2 C3

check nodes

variable nodes

• Binary image of the code: in the codeword, replace each symbol in Fq

with its length-p, p = log2 q, binary representation• Next, we will always refer to the binary block length n = Np bits


Non-Binary LDPC CodesBelief Propagation Decoding

• Messages are vectors of q probabilities (probability mass functions,p.m.f.s)

• At a degree-d CN, the p.m.f.s incoming on d − 1 edges are convolved toproduce the extrinsic estimate (p.m.f.) of the remaining edge

• Decoding complexity scales as quadratically in q. It can be reduced toO(q log2 q) by performing the CN operation via fast Fourier transform41

⇤

min1

min2

mout3 = min

1 ⇤ min2

min1

min2

FF F�1

mout3

mout1 mout

2mout

3 = mout1 · mout

2

⇥

m =⇥p(0), p(1), p(↵), p(↵2), . . . , p(↵q�1)

⇤

41L. Barnault and D. Declercq. “Fast decoding algorithm for LDPC over GF(2q )”. In: Proc. IEEEInf. Theory Workshop (ITW). 2003.


Non-Binary LDPC CodesUltra-Sparse Ensembles: Cycle LDPC Codes

• Non-Binary LDPC codes constructed on relatively-large fields (q ≥ 64)perform particularly well when the graph is ultra-sparse42

I Constant variable node degree dv = 2I Uniform check node degree

• Ultra-sparse LDPC Codes are actually instances of cycle codes43

C1 C2 C3 C4

C1

C2 C3

C4

Tanner graph perspective Cycle (circuit) code perspective

girth ÷2

girth ⇥2

42C. Poulliat, M. Fossorier, and D. Declercq. “Design of regular (2, dc)-LDPC codes over GF(q)using their binary images”. In: IEEE Trans. Commun. (2008).

43S. Hakimi and J. Bredeson. “Graph theoretic error-correcting codes”. In: IEEE Trans. Inf.Theory (1968).


Cycle LDPC CodesGraphs and Cages

• Thanks to the sparse nature of the graph, large girths can be attainedeven for short block lengths

• Optimum graphs can be either found in literature44, or constructed bymeans of progressive-edge-growth techniques4546

44P.K. Wong. “Cages – A survey”. In: J. Graph Theory (1982).45A. Venkiah, D. Declercq, and C. Poulliat. “Design of Cages with a Randomized Progressive

Edge Growth Algorithm”. In: IEEE Commun. Lett. (2008).46W. Chen, C. Poulliat, and D. Declercq. “Structured High-Girth Non-Binary Cycle Codes”. In:

Proc. of APCC. 2009.



• A graph G is composed by a set V = {vi} of m vertices and by a setE = {ej} of n edges

• The order of a graph is its number m of vertices• For a (v , g)-graph, where g is the girth,

m ≥ m0(v , g)

with

m0(v , g) =

v(v−1)r−2

v−2 if g = 2r + 1,

2(v−1)r−2v−2 if g = 2r .

• The (v , g)-graph of smallest order is referred to as (v , g)-cage• In general the bound is not achieved with equality even for cages• When the bound is achieved with equality, the (v , g)-cage is referred to

as Moore (v , g)-graph



(a) The Robertson graph. (b) The Balaban-11 cage.

(c) The Petersen graph. (d) The Coxeter graph.

Fig. 1. Examples of Hamiltonian (a,b) and Hypohamiltonian (c,d) graphs.

graph is the length of its smallest cycle. For a (!, #)-graph, itturns out that [9]

$ ≥ $0(!, #), (1)

with

$0(!, #) =

⎧⎨⎩

!(!−1)!−2!−2 if # = 2% + 1,

2(!−1)!−2!−2 if # = 2%.

The (!, #)-graph of smallest order is referred to as (!, #)-cage[9]. Note that in general (1) is not achieved with equality evenfor cages. When (1) is achieved with equality, the (!, #)-cageis referred to as Moore (!, #)-graph.

A Hamiltonian cycle C in a graph G is a closed path visitingeach graph vertex once. A graph possessing a Hamiltoniancycle is called Hamiltonian graph. A graph G is called Hypo-hamiltonian if every graph obtained by removing a vertex fromG , along with all edges incident on this vertex, is Hamiltonian[10]. Examples of Hamiltonian and Hypohamiltonian graphsare provided in Figure 1.

III. GRAPH REPRESENTATION AND ENCODING

We denote by Ψ the binary ($× ') incidence matrix of aHamiltonian graph and refer to its entries by (",$ . To constructΨ, we associate the last $ columns of Ψ with the edgesforming the Hamiltonian cycle. More specifically, we assumethat the edges forming the Hamiltonian cycle are numberedin sequence, i.e. that the Hamiltonian cycle is obtained byfollowing the path ()%−&, )%−&+1, )%−&+2, . . . , )%−1). Sim-ilarly, the numbering of the vertices is defined such that edge)%−& connects vertices !0, !1, edge )%−&+1 connects vertices!1, !2, edge )%−&+2 connects vertices !2, !3 and so on. Anexample of edge labeling for the Heawood graph is providedin Figure 2. By associating the *-th column of Ψ with the *-thedge, and the +-th row of Ψ with the +-th vertex, Ψ takes the

form Ψ = [Ψ'∣Ψ(] where Ψ( is an ($×$) double diagonalbinary matrix,

Ψ( =

⎡⎢⎢⎢⎢⎢⎢⎢⎣

1 0 0 . . . 0 11 1 0 . . . 0 00 1 1 . . . 0 00 0 1 . . . 0 0...

......

. . ....

...0 0 0 . . . 1 1

⎤⎥⎥⎥⎥⎥⎥⎥⎦.

The matrix Ψ can be used as parity check matrix for an(', ,) binary linear block (cycle [6], [22]) code, where , isthe code dimension, , = ' − $ + 1, being $ − 1 the rankof Ψ by construction.2 Denoting the '-symbol codeword rowvector by c, Ψ imposes a set of $ linear parity constraintsto c, cΨ) = 0. Due to the above-presented construction, the+-th codeword symbol, -", is associated with the +-th graphedge, )". The resulting code parity check matrix is low-density,and thus iterative decoding is possible. The double-diagonalstructure of Ψ( allows adopting efficient encoding, being Ψde facto the parity check matrix of a tail-biting IRA code [23].However, the low degree of connectivity of the correspondingTanner graph, which has regular VN degree 2 and regular CNdegree !, leads to codes with poor performance both in termsof decoding threshold and (especially) of minimum distance[1]. In fact, the minimum distance equals the girth of the graph.

On the contrary, Ψ can be well suited to design parity checkmatrix for non-binary IRA codes. To do so, we construct an($ × ') parity check matrix over a finite field !*, . > 2,by setting its entries ℎ",$ = 0 whenever (",$ = 0, andℎ",$ = 1",$ ∈ !∗

* whenever (",$ = 1, where !∗* = !*∖{0}

[24]. The code parameters are thus given by (', ,) with, = ' − $ if the coefficients {1",$} are selected to avoidlinear dependencies among the rows of H. The $ edgesbelonging to the Hamiltonian cycle are thus associated withthe $ parity symbols, whereas the inner edges are associatedto the , information symbols. If the finite field order islarge enough (e.g., . > 16), powerful codes can be obtainedeven by selecting the coefficients {1",$} randomly. Improvedperformances can be attained by a judicious selection of thenon-zero coefficients, to give an example, by optimizing theminimum Hamming distance associated with the binary imageof the code parity-check equations [5].

Examples of Hamiltonian/Hypohamiltonian graphs withlarge girths are given in Table I. For each graph, we provide thevalency, the girth, the order and the nominal code parameters,i.e. the block length ', the code dimension , (both in symbols)and the code rate 2 = ,/' (assuming linearly independentequations in the derived code parity check matrix). A morecomplete list of Hamiltonian graphs covering a broader set ofcode parameters can be found e.g. in [25].

A. On Cubic Hamiltonian Graphs

The special case of cubic, i.e. valency-3, graphs representsa particularly rich class of large-girth Hamiltonian graphs.Remarkably, many cages belonging to this class can be derived

2Note that the column weight of Ψ is uniformly 2, thus Ψ is rank deficient.



Example: (120, 40)Non-Binary LDPCCode over F256 basedon the Petersen graph

The code turns to beactually a rate-1/3non-binary turbo code

G. Liva et al. “ShortTurbo Codes over HighOrder Fields”. In:IEEE Trans. Commun.(2013)

2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 710−8

10−7

10−6

10−5

10−4

10−3

10−2

10−1

100

Random

Coding

Bound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

3GPP Turbo Code

Turbo Code, GF(256)


Cycle LDPC CodesDistance Properties

• The cycle code ensemble is characterized by dmin = O(log n)independently from the field order

• Beyond girth optimization, the minimum distance of the binary image ofthe code can be kept sufficiently large by a proper choice of theparity-check matrix coefficients


Cycle LDPC CodesCoefficient Optimization

• Choosing the coefficients in a parity-check matrix which optimizes theminimum distance of the binary image of the code is a formidable task

• Less complex (sub-optimum): optimize the coefficients row-wise47

• Example: Parity-check matrix over F256, p(x) = x8 + x4 + x3 + x2 + 1

H =

[α8 α80 α0 0α0 α41 α122 α113

]The first row imposes the equation c1α

8 + c2α80 + c3α

0 = 0, i.e.

[c1, c2, c3] hT1 = 0 h1 =

[α8, α80, α0

](non-binary SPC code)

47C. Poulliat, M. Fossorier, and D. Declercq. “Design of regular (2, dc)-LDPC codes over GF(q)using their binary images”. In: IEEE Trans. Commun. (2008).



• Let us analyze the binary code associated with the non-binary SPC codewith parity-check matrix h1 =

[α8, α80, α0]

• Companion matrix of p(x) =∑8

i=0 pix i = x8 + x4 + x3 + x2 + 1

M =

0 1 0 0 0 0 0 00 0 1 0 0 0 0 00 0 0 1 0 0 0 00 0 0 0 1 0 0 00 0 0 0 0 1 0 00 0 0 0 0 0 1 00 0 0 0 0 0 0 1p0 p1 p2 p3 p4 p5 p6 p7

=

0 1 0 0 0 0 0 00 0 1 0 0 0 0 00 0 0 1 0 0 0 00 0 0 0 1 0 0 00 0 0 0 0 1 0 00 0 0 0 0 0 1 00 0 0 0 0 0 0 11 0 1 1 1 0 0 0

• Denote by c1 the 8-bits representation of the symbol c1, and by s1 the

8-bits binary image of s1 = c1αi . Then

s1 = c1Mi



• The binary code associated with the non-binary SPC code havingh1 =

[α8, α80, α0] has parity-check matrix given by

Hbr =

[M8|M80|I

]i.e.

12 SUBMITTED TO IEEE TRANSACTIONS ON COMMUNICATIONS

denote by Cbr the binary image of Cr. Clearly, Cb

r has dimension 2m and block length 3m, where

m = log2 q. The weight enumerator function (WEF) of Cbr is referred to as Ab

r(x).

The enumeration of the codeword weights for Cbr may be computationally expensive for large

m, since the cardinality of Cbr is 22m. A simpler approach relies on the WEF of the (3m, m) dual

code, which can be derived in an easier way since it involves the calculation of 2m codeword

weights. Given the binary dual (3m, m) code WEF Bbr (x), the binary WEF Ab

r(x) of Cbr can be

derived applying the MacWilliams identity [35], i.e.,

Abr(x) =

(1 + x)3m

2mBb

r

(1 − x

1 + x

).

We seek for coefficient combinations which optimize locally the minimum distance of the

component code Cbr with the purpose of improving the overall code distance properties.5

Example 3: Binary WEF of a (3, 2) SPC code Cr over F256 with parity-check matrix Hr =

[α8 α80 α0]. The binary image of the SPC code parity-check matrix is given by Hbr = [M8|M80|I],

being M the 8 × 8 companion matrix [35] of the field primitive element α, and I the 8 × 8

identity matrix. The field is generated by the polynomial p(x) = x8 + x4 + x3 + x2 + 1 so that

Hbr =

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

1 0 0 0 1 1 1 0 1 1 1 1 1 0 1 1 1 0 0 0 0 0 0 0

0 1 0 0 0 1 1 1 0 1 1 1 1 1 0 1 0 1 0 0 0 0 0 0

1 0 1 0 1 1 0 1 1 1 0 0 0 1 0 1 0 0 1 0 0 0 0 0

1 1 0 1 1 0 0 0 1 0 0 1 1 0 0 1 0 0 0 1 0 0 0 0

1 1 1 0 0 0 1 0 1 0 1 1 0 1 1 1 0 0 0 0 1 0 0 0

0 1 1 1 0 0 0 1 1 1 0 1 1 0 1 1 0 0 0 0 0 1 0 0

0 0 1 1 1 0 0 0 1 1 1 0 1 1 0 1 0 0 0 0 0 0 1 0

0 0 0 1 1 1 0 0 1 1 1 1 0 1 1 0 0 0 0 0 0 0 0 1

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

.

The dual code C⊥r has generator matrix Hr = [α8 α80 α0]. Thus, its binary image has generator

matrix Hbr and its WEF is given by

Bbr (x) = 1+15x8 +17x9 +32x10 +46x11 +56x12 +32x13 +14x14 +18x15 +8x16 +15x17 +2x18.

5As mentioned in Section III, the use of optimized check equation coefficients also helps to improve the iterative decoding

threshold. The improvement is evident for moderate-small field orders, whereas for larger fields (e.g., F256) the threshold gain

is marginal.

DRAFT July 26, 2012

• This is the parity-check matrix of a (24, 16) binary linear block code



• Distance spectrum of the code with parity-check matrix given by

Hbr =

[M8|M80|I

]via MacWilliams identity48

A(x) =1

2n−k (1 + x)n B(

1− x1 + x

)where A(x) is the WEF we are interested in, and B(x) is the one of thedual code (having Hb

r as generator matrix)

48J. MacWilliams and N. Sloane. The theory of error-correcting codes. North HollandMathematical Libray, 1977.



• Distance spectrum of the code with parity-check matrix given by

Hbr =

[M8|M80|I

]is

A(x) = 1 + 40x4 + 202x5 + 502x6 + 1307x7 + 2980x8 + 5082x9 + 7486x10+

+ 9854x11 + 10698x12 + 9686x13 + 7618x14 + 5076x15 + 2875x16+

+ 1406x17 + 522x18 + 146x19 + 46x20 + 8x21 + x23.

• Thus, dmin = 4 and Amin = 40



List of good row coefficients, F256, check node degree 3

Coefficients,[αi αj α0] dr,min Ar,min[

α8 α183 α0] 4 36[α10 α82 α0] 4 37[α16 α167 α0] 4 37[α41 α127 α0] 4 37[α41 α128 α0] 4 37[α42 α128 α0] 4 37[α72 α80 α0] 4 36[α72 α245 α0] 4 37[α86 α213 α0] 4 37[α86 α214 α0] 4 37


Cycle LDPC CodesExample: New CCSDS Telecommand Standard

• (128, 64) code for satellite (CCSDS) telecommand• Requirement: very low undetected error rate• Competitor: binary protograph code

H =

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

α182 0 0 0 α8 0 0 0 α173 0 0 0 0 0 0 α0

0 α89 0 0 0 α0 0 0 0 α9 0 0 α81 0 0 00 0 α0 0 0 0 α173 0 0 0 α182 0 0 α8 0 00 0 0 α8 0 0 0 α182 0 0 0 α173 0 0 α0 00 0 α88 0 0 0 0 α80 α0 0 0 0 α8 0 0 00 0 0 α169 α0 0 0 0 0 α127 0 0 0 α40 0 0

α169 0 0 0 0 α128 0 0 0 0 α40 0 0 0 α0 00 α8 0 0 0 0 α80 0 0 0 0 α88 0 0 0 α0

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

. (1)

on fields of larger orders. As an example, Figure 6 shows theperformance of a (160, 80) code constructed over F1024, whichshares its bipartite graph structure with the one of the (128, 64)code over F256. Also in this case, a performance within 1 dBfrom the SPB is achieved down to CER ≃ 3 · 10−7.

V. INTEGRATION IN THE CCSDS PROTOCOL STACK

As emerged in the previous sections, the robustness andeffectiveness of the proposed code is such that their inclusionin standardization of next generation CCSDS TC is desirable.From this standpoint, a particular attention has to be devotedto the implementation issues this integration may introducein a protocol perspective. As already introduced, the CCSDSTC Coding and Synchronization sublayer receives CLTUframes from the CCSDS TC Datalink sublayer interface [1].Depending on the frame size, aggregation of multiple frames,splitting and eventually bit-padding are performed so that 56-bits blocks are filled. In turn, BCH encoding is carried out,resulting in the generation of 63-bits codewords. Actually, thesame principle can be also applied to the case of non-binaryLDPC/Turbo Codes, provided that the CCSDS TC sublayeris able to fill blocks larger than 56 bits. An additional pointis represented by the sequence tail, which must be as largeas the codeword. Hence, in the case of non-binary codes, ithas to be updated accordingly, in order to avoid mismatcheswith the code configuration specifically used. An example ofhow a non-binary (128, 64) code can work with two CLTUframes carrying 25 and 15 bytes, respectively, is depicted inFigure 7. The frames are split in such a way to fill blocks of64 bits; padding is not needed in this case as 40 bytes can fit 564-bits blocks. In turn, non-binary coding generates 128-bitscodewords, to which sequence start and tail are appended.

VI. CONCLUSIONS

In this paper, the performance of the non-binaryturbo/LDPC codes proposed within the NGU working groupof the CCSDS have been presented and discussed. Their inte-gration into the CCSDS uplink protocol has been addressed.The proposed codes allow achieving remarkable gains (∼ 1.5dB) w.r.t. the binary LDPC codes considered up to now withinthe CCSDS.

REFERENCES

[1] “TC Synchronization and Channel Coding,” Blue Book, Issue 2, Con-sulative Committee for Space Data Systems (CCSDS), Newport Beach,CA, Sep. 2010.

2 3 4 5 6 7 8 910

−10

10−9

10−8

10−7

10−6

10−5

10−4

10−3

10−2

10−1

100

Eb/N

0 [dB]

CE

R

Sphere Packing Bound, k=64 bits, R=1/2Truncated Union Bound, (128,64) IRA Code(128,64) IRA Code − F256(128,64) Binary Protograph(63,56) BCH Code, BCJR(63,56) BCH Code, SEC

Fig. 4. CER for the (128, 64) binary LDPC code from the CCSDS OrangeBook [7] compared with the CER for the (128, 64) IRA code over F256. Theperformance of the (63, 56) BCH code in SEC mode is reported, as well asunder soft-decision BCJR decoding.

3 3.5 4 4.5 5 5.5 6 6.5 710

−9

10−8

10−7

10−6

10−5

10−4

10−3

10−2

10−1

100

Eb/N

0 [dB]

FR

P

F256

IRA

Protograph LDPC

L=1

L=2

L=4

L=8

L=16

L=32

L=64

L=128

L=256

L=1

L=2

L=4

L=8

L=16

L=32

L=64

L=128

L=256

Fig. 5. FRP for the (128, 64) binary LDPC code from the CCSDS OrangeBook [7] compared with the CER for the (128, 64) IRA code over F256.


Cycle LDPC CodesExample: New CCSDS Telecommand Standard

1 2 3 4 5 6 7 8 910−10

10−8

10−6

10−4

10−2

100

Random

Coding

Bound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate


LDPC Code, GF(256)


Cycle LDPC CodesLow-Rate via Multiplicative Repetition

• Non-Binary LDPC codes admit largeflexibility in the code rate

• A low-rate non-binary LDPC code can beobtained from a higher-rate one by simplyrepeating some codeword symbols, andmultiply them by a randomly-pickednon-zero field element

K. Kasai et al. “Multiplicatively repeatednonbinary LDPC codes”. In: IEEE Trans.Inf. Theory (2011)

• Non-binary LDPC codes are almostrate-less

Rate-1/3 code graph

Rate-1/9 code graph


Cycle LDPC CodesExample: Low-Rate via Multiplicative Repetition

k = 1024 codes

Non-binary LDPC overF256, mother code hasrate R = 1/3

-1 -0.5 0 0.5 1 1.510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound,R

=1/6

Sphere-P

ack

ing

Bound,R

=1/6

Random

Codin

gBound,R

=1/4

Sphere-P

ack

ing

Bound,R

=1/4

Eb/N0 [dB]

Blo

ckE

rror

Rate

AR4A, R = 1/6

G-LDPC Code, R = 1/4

G-LDPC Code, R = 1/6

LDPC Code GF(256), R = 1/6


Protographs for Moderate-Order Fields

• For moderate-order fields (e.g., 4 ≤ q ≤ 32), the cycle code constructiontends to exhibit high error floors and poor thresholds

• In49, protograph-based non-binary LDPC codes have been proposed formoderate field orders

• Example: rate-1/2 protograph over F16, (Eb/N0)∗ = −0.35 dB

49L. Dolecek et al. “Non-Binary Protograph-Based LDPC Codes: Enumerators, Analysis, andDesigns”. In: IEEE Trans. Inf. Theory (2014).


Protographs for Moderate-Order Fields

Non-binary LDPCcode over F16 withprotograph

Coefficients have notbeen optimized

0.5 1 1.5 2 2.5 3 3.5 4 4.5 510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

LDPC Code, GF(256)


Protograph LDPC Code, GF(16)

Outline

1 Introduction



4 Non-Binary Codes for Iterative DecodingLow-Density Parity-Check Codes

Parallel Concatenated Convolutional Codes




Non-Binary Turbo CodesBasics

• Turbo codes on high-order fields50 can be constructed, with a structurethat closely follows that on their LDPC codes counterpart51

• Memory-1 (in field symbols), time-variant convolutional codes

LIVA ET AL.: SHORT TURBO CODES OVER HIGH ORDER FIELDS 23

gi

fi

vi

pi

Si−1 = pi−1Si = piwi

D

ui

D

u

p(1)

p(2)

g(1)

f (1)

g(2)

f (2)

a) PCCC encoder structure

D

b) RSC encoder

Π

Fig. 2. Structure of the encoder for a memory-1 time-variant PCCC (a) and of the component code RSC encoder (b).

July 26, 2012 DRAFT

LIVA ET AL.: SHORT TURBO CODES OVER HIGH ORDER FIELDS 23

gi

fi

vi

pi

Si−1 = pi−1Si = piwi

D

ui

D

u

p(1)

p(2)

g(1)

f (1)

g(2)

f (2)

a) PCCC encoder structure

D

b) RSC encoder

Π

Fig. 2. Structure of the encoder for a memory-1 time-variant PCCC (a) and of the component code RSC encoder (b).

July 26, 2012 DRAFT

Turbo encoder Detail of the RSCC encoder

50J. Berkmann and C. Weiss. “On dualizing trellis-based APP decoding algorithms”. In: IEEETrans. Commun. (2002).

51G. Liva et al. “Short Turbo Codes over High Order Fields”. In: IEEE Trans. Commun. (2013).


Turbo Codes based on Memory-1 Time-Variant RCCsDecoding

• Decoding over the q-states trellis of the component codes• q branches leaving/entering a state in each section• Decoding complexity scales as O(q2)

• Can be reduced to O(q log2)by decoding through fast Fourier transform522

C1 ⇡ C2 BPSK

(n1, k1) binarySPC code

(n2, k2) binarylinear code

antipodalmodulation

Cq M

(n1, k1) SPCcode over Fq

sequencemodulator

c 2 Cq x 2 X n1

Fig. 1. Relation between the conventional product code encoder perspectiveand the the coded modulation one.

III. EFFICIENT ML DECODING OF PRODUCT CODES

For the product code class introduced in Section II ef-ficient (block-wise) ML decoding can be implemented byintroducing a compact trellis representation of the code [2].In the following, we revisit the approach of [2] from a codedmodulation perspective. The vectors mT

1 ,mT2 , . . . ,mT

n1can be

regarded as the binary vector representations of the elementsof a finite field Fq of order q “ 2k2 . We denote the elementscorresponding to mT

1 ,mT2 , . . . ,mT

n1by µ1, µ2, . . . , µn1

. Thus,(2) can be written as

n1ÿ

i“1

µi “ 0 (3)

which defines the parity-check equation of an pn1, n1 ´ 1qSPC code Cq over Fq with parity-check matrix

H “ r1 1 . . . 1s . (4)

We denote by µ “ rµ1 µ2 . . . µn1s the n1-symbols codeword.

The symbols are then mapped onto a constellation X , whichis composed of q real-valued vectors equal to the q “ 2k2

BPSK-modulated codewords of C2. The mapping (modulation)function is M : Fq fiÑ X and it is constrained by the productcode construction, being the real-valued vector associated witha generic symbol µi given by the BPSK-modulated binaryvector ci “ miG2. The relation between the conventionalproduct code encoder perspective and the coded modulationone is summarised in Figure 1. The code Cq can be decodedefficiently over its trellis, which comprises q states and where(apart from the terminations) each state is section i ´ 1 isconnected through q branches to the states in section i. Anexample of a trellis section for a SPC code over F4 is depictedin Figure 2, in which ↵ is a primitive element of F4. We denoteby Si the trellis state at time i. The label associated with thetrellis edge connecting the state s at time i ´ 1 to the state s1at time i is given by s ` s1, with s, s1 P Fq .

The Viterbi algorithm is then applied over the trellis, withthe branch (transition) metrics given by xxi,yiy , @xi P X . Foreach trellis section and at each node q, accumulated metricsneed to be computed and compared, resulting in q2 operations.Neglecting the terminations, the overall complexity is thusproportional to k1q

2 “ k122k2 as in (1).

0

1

↵2

0

1

↵2

Si�1 Si

0

1

↵

↵2

↵ ↵

Fig. 2. Example of a trellis section for a non-binary SPC code with parity-check matrix in the form (4).

IV. SYMBOL-WISE MAP DECODING

In this section, we turn the decoding problem into a symbol-wise MAP decoding for the class of product codes introducedin Section II. The modification ensures complexity savingswith respect to (1). The proposed algorithm is optimum inthe sense of maximizing the a posteriori probability for eachsymbol given the observation y, i.e.,

µi “ arg max!PFq

Pr tµi “ !|yu .

Note that the main contribution here lies in the changeof perspective in the decoding algorithm (i.e., from ML tosymbol-wise MAP). Though, the change of perspective allowsemploying efficient MAP decoding for non-binary codes [8]as instantiated next for the case on non-binary SPC codes.

The a posteriori probability mass function (p.m.f.) vectorfor the symbol µi given the channel output y is denotedby Li “ rLip0q, Lip1q, Lip↵q, . . . , Li

`↵q´2

˘s with Lir!s “Prtµi “ !|yu, ! P Fq . We further introduce the notationyri:js “ pyi,yi`1, . . . ,yjq, 1 § i † j § n1. The a posterioriprobability for symbol µi is

Lip!q “ Prtµi “ !|yu “ÿ

s,s1“s`!

'i´1psq�ips, s1q�ips1q.

'i´1psq denotes the forward metric for the state s at time i´1,�ips1q is the backward metric for the state s1 at time i, and�ips, s1q is the transition metric between states s, s1 at time i.We normalize the metrics such that

'ipsq “ PrtSi “ s|yr1:isu, '1p0q “ 1,

�ipsq “ PrtSi “ s|yri`1:n1su, �n1p0q “ 1,

�ips, s1q “ PrtSi´1 “ s, Si “ s1|yiu. (5)

Defining s1 “ s`!, we have that the branch transition metriccan be computed as

�ips, s1q “ Prtµi “ !|yiu 9 ppyi|µi “ !q“ `

2⇡�2˘´n2{2

exp

ˆ´ 1

2�2xyi, Mp!qy

˙. (6)

Since (6) involves a correlation between length-n2 vectors,and since it has to be computed for each ! P Fq withq “ 2k2 , the complexity of the branch metrics calculation

52J. Berkmann and C. Weiss. “On dualizing trellis-based APP decoding algorithms”. In: IEEETrans. Commun. (2002).


Non-Binary Turbo CodesMemory-1 Time-Variant Recursive Convolutional Codes

• The codes admit a non-binary protograph LDPC code representation

22 SUBMITTED TO IEEE TRANSACTIONS ON COMMUNICATIONS

a) Parallel concatenation

b) Serial concatenation

C0

C1

C1 C0

V0

V0

V1

V2

V2 V1

✐

✙

accumulators on Fq

❄❄

differentiator on Fq

Fig. 1. Protograph representation of a parallel (a) and serial (b) concatenation.

DRAFT July 26, 2012

• The protograph ensemble is a sub-ensemble of the unstructured (3, 2)regular LDPC ensemble

• Same distance properties dmin = O(log n)

• Similar decoding threshold. E.g., on F256 the rate−1/3 protographensemble possesses (Eb/N0)∗ = −0.214 dB vs. (Eb/N0)∗ = −0.225 dBof the unstructured one


Turbo Codes based on Memory-1 Time-Variant RCCsInterleaver Design and Graph Interpretation

• For binary turbo codes, a large-spread regular interleaver is not anoption, due to the large multiplicity of minimum-weight codewords thatarises from the combination of the regular interleaver structure and ofthe periodic trellis for the component codes

• Thus, the interleaver shall sacrifice spread for irregularity (randomness)• For non-binary turbo codes, the component codes do not have anymore

a periodic trellis (time-variant FB/FFW coefficients), thus large-spreadregular interleavers can be used53

53C. Radebaugh, C. Powell, and R. Koetter. “Wheel codes: Turbo-like codes on graphs of smallorder”. In: Proc. IEEE Inf. Theory Workshop (ITW). 2003.



• As for non-binary LDPC codes, non-binary turbo codes can be definedby the graph of a cycle code, with specific structure

• The graph structure directly specifies the interleaver

C8

C7

C9

C6

C0

C5

C1

C4

C2

C3 C3

C4

C2

C0

C1

Petersen Graph (Cage)

Tail-biting trellis (1st component code)

Tail-biting trellis (2nd component code)

Information bits



• As for non-binary LDPC codes, non-binary turbo codes can be definedby a graph of a cycle code, with specific structure


C8

C7

C9

C6

C0

C5

C1

C4

C2

C3 C3

C4

C2

C0

C1




Information bits





C8

C7

C9

C6

C0

C5

C1

C4

C2

C3 C3

C4

C2

C0

C1




Information bits





C8

C7

C9

C6

C0

C5

C1

C4

C2

C3 C3

C4

C2

C0

C1




Information bits


Non-Binary Turbo and LDPC Codes, F256Dimension k = 1024 bits

0.5 1 1.5 2 2.5 3 3.510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A

Protograph Raptor


G-LDPC (Irregular)

LDPC Code, GF(256)

Turbo Code, GF(256)



0.5 1 1.5 2 2.5 3 3.5 4 4.5 510−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-P

ack

ing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor


G-LDPC (Irregular)

LDPC Code, GF(256)

Turbo Code, GF(256)



1 2 3 4 5 6 7 8 910−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor


G-LDPC (Irregular)

LDPC Code, GF(256)

Turbo Code, GF(256)

Outline

1 Introduction






Page 139/158 G. Liva · Short Codes Design · Beyond Iterative Decoding October 8, 2014

Ordered Statistics DecodingPrinciple

• Compute, on-the-fly, a subset of the candidate codewords among whichthe one at minimum Euclidean distance from the observation isselected54

• Complete decoding algorithm, though not maximum-likelihood• Performance depends on the cardinality of the candidates subset, and

on how the set is formed• Can be applied in principle to any block code• However, since complexity is not linear in n, application is limited to short

codes

54M. Fossorier and S. Lin. “Soft-decision decoding of linear block codes based on orderedstatistics”. In: IEEE Trans. Inf. Theory (1995).


Ordered Statistics DecodingAlgorithms

• Example: Most Reliable Basis (MRB)55. Given the order i ,

1. Find the k most reliable channel observations in y and collect them in avector v

2. Perform Gaussian elimination on the generator matrix G with respect to theirpositions, obtaining a systematic generator matrix Gsys with respect to v

3. Encode c0 = vGsys

4. Generate the possible error patterns of length k and weight w ≤ i5. For the generic l th error pattern el , encode cl = (v + el )Gsys

6. Select the codeword in {cl} at minimum Euclidean distance from y

• The complexity is linear in the number of error patters, Ne =∑i

l=0

(kl

),

and polynomial in k• The larger i , the closer to ML, to higher the complexity. Typical values of

i are ∈ [2, 4]55Y. Wu and C. Hadjicostis. “Soft-decision decoding using ordered recodings on the most reliable

basis”. In: IEEE Trans. Inf. Theory (2007).


Ordered Statistics DecodingExamples

MRB decoding withi = 4

(128, 64) extendedBCH code, dmin = 22

1 2 3 4 5 6 7 8 910−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor


G-LDPC (Irregular)

LDPC Code, GF(256)

Turbo Code, GF(256)

BCH Code, OSD


Ordered Statistics DecodingExamples

MRB decoding withi = 4

(128, 64) extendedBCH code, dmin = 22

(128, 64) non-binaryLDPC code

1 2 3 4 5 6 7 8 910−6

10−5

10−4

10−3

10−2

10−1

100

Random

Codin

gBound

Sphere-Packing

Bound

Eb/N0 [dB]

Blo

ckE

rror

Rate

(3,6) Gallager LDPC

ARJA

AR3A


Protograph Raptor


G-LDPC (Irregular)

LDPC Code, GF(256)

Turbo Code, GF(256)

BCH Code, OSD

LDPC Code, GD(256), OSD


Efficient Symbol-wise MAP DecodingExamples

Performed byexploiting an efficienttrellis representationavailable for certainproduct codes

J.K. Wolf. “Efficient maximumlikelihood decoding of linearblock codes using a trellis”.In: IEEE Trans. Inf. Theory(1978)

G. Liva, E. Paolini, andM. Chiani. “On OptimumDecoding of Certain ProductCodes”. In: IEEE Commun.Lett. (2014)

2 2.5 3 3.5 4 4.5 5 5.5 610−6

10−5

10−4

10−3

10−2

10−1

100

Eb/N0 [dB]

Blo

ckE

rror

Rate

Product Code - Iterative

Product Code - Symbol-by-Symbol MAP

Product Code - Truncated UB

CC - Symbol-by-Symbol MAP

CC Ensemble - Truncated UB

Outline

1 Introduction






Page 144/158 G. Liva · Short Codes Design · Summary and Open Challenges October 8, 2014

The Short Code Design ToolkitBenchmarking Solutions

Code Class Performance (Waterfall) Performance (Error floor) Complexity Flexibility (rates) NotesBinary LDPC Good (Tunable) Good (Tunable) Low Excellent -Binary Turbo Excellent ? Low Good High-rate

design?Product/G-LDPC Uninspiring Good (Tunable) Low Limited -Non-Binary Excellent Excellent High Excellent log-domainTurbo/LDPC decoding?

• In general, short code design still relies heavily on optimization of knowncode classes

• A good asymptotic decoding thresholds seems to be essential for goodwaterfall performance, but it shall be attained while maintaining thegraph sparse (large girth) and reasonable minimum distance

• Non-binary LDPC and turbo codes satisfy these heuristic principles, andare nowadays probably the best solution from a performance view-point


Open ChallengesJust some...

1. In general, finite-length design tools that can be efficiently used for thedesign in the short/moderate length regime are missing. They shall beable to capture both the finite-length scaling of belief propagation, aswell as the error floor behavior, with limited computational complexity

2. 16-states turbo codes have great potential at short blocks, but their errorfloor performance shall be improved, as well as their performance athigh code rates

3. Regarding non-binary codes, would be of some importance to devise anefficient log-domain decoding algorithm (beyond extended min-sumalgorithms)

4. This tutorial is quite iterative-centric: other solutions?


Open ChallengesFeedback?

S Encoder Modulator

Channel

SoftDemodulator

DecoderD

Feed-backPolicy



Es/N0 [dB]

Rate

,bits/

channel

use

-10 -5 0 5 10 150

1

2

3Unconstrained Channel Capacity

256-QAM Symmetric Information Rate

NB-LDPC, GF(256), k = 40, 256-QAM

NB-LDPC, GF(256), k = 128, 256-QAM

NB-LDPC, GF(256), k = 128,, 256-APSK



-1 0 1 2 3 4 5 6

10−4

10−3

10−2

10−1

100

Eb/N0 [dB]

Blo

ckE

rror

Rate

0.7760.907

1.0515 maximum iterations w/o P

0.760.91.042

5 maximum iterations w/ P

0.7370.864

1.0034 maximum iterations w/o P

0.6840.802

0.933

3 maximum iterations w/o P


Thank you!

Page 150/158 G. Liva · Short Codes Design · Bibliography October 8, 2014

Bibliography I

Finite-Length Performance BoundsC. Shannon. “Probability of error for optimal codes in a GaussianChannel”. In: Bell System Tech. J. (1959).

R. Gallager. Information theory and reliable communication. Wiley, 1968.

S. Dolinar, D. Divsalar, and F. Pollara. “Code performance as a function ofblock size”. In: TMO progress report (1998).

I. Sason and S. Shamai. Performance analysis of linear codes undermaximum-likelihood decoding: A tutorial. Now Publishers Inc, 2006.

S. Dolinar et al. “Bounds on error probability of block codes withbounded-angle maximum-likelihood incomplete decoding”. In:International Symposium on Information Theory and Its Applications. 2008.

Y. Polyanskiy, V. Poor, and S. Verdú. “Channel coding rate in the finiteblocklength regime”. In: IEEE Trans. Inf. Theory (2010).


Bibliography II

Binary LDPC CodesR. Gallager. Low-density parity-check codes. 1963.

H. Jin, A. Khandekar, and R. McEliece. “Irregular repeat-accumulatecodes”. In: Proc. IEEE Int. Symp. Turbo Codes and Related Topics. 2000.

T. Richardson, A. Shokrollahi, and R. Urbanke. “Design ofCapacity-Approaching Irregular Low-Density Parity-Check Codes”. In:IEEE Trans. Inf. Theory (2001).

J. Thorpe. Low-Density Parity-Check (LDPC) Codes Constructed fromProtographs. IPN Progress Report. 2003.

T. Richardson and R. Urbanke. Multi-edge type LDPC codes.unpublished. 2004.

T. Tian et al. “Selective Avoidance of Cycles in Irregular LDPC CodeConstruction”. In: IEEE Trans. Commun. (2004).


Bibliography III

D. Divsalar et al. “Constructing LDPC codes from simple loop-freeencoding modules”. In: Proc. IEEE Int. Conf. Commun. (ICC). 2005.

X.-Y. Hu, E. Eleftheriou, and D.M. Arnold. “Regular and irregularprogressive edge-growth Tanner graphs”. In: IEEE Trans. Inf. Theory(2005).

D. Divsalar, S. Dolinar, and C. Jones. “Construction of protograph LDPCcodes with linear minimum distance”. In: IEEE International Symposiumon Information Theory. 2006.

G. Liva and M. Chiani. “Protograph LDPC codes design based on EXITanalysis”. In: Proc. IEEE Global Telecommun. Conf. 2007.

D. Divsalar et al. “Capacity-approaching protograph codes”. In: IEEEJSAC (2009).


Bibliography IV

M. Lentmaier et al. “Approaching Capacity with Asymptotically RegularLDPC Codes”. In: Information Theory and Applications Workshop (ITA).2009.

W. Ryan and S. Lin. Channel codes – Classical and modern. CambridgeUniv. Press, 2009.

T.-Y. Chen et al. Protograph-Based Raptor-Like LDPC Codes. 2014. URL:http://arxiv.org/abs/1403.2111.

D. Declercq, M. Fossorier, and E. Biglieri. Channel Coding: Theory,Algorithms, and Applications. Academic Press Library, 2014.



Bibliography V

Non-Binary LDPC & Turbo CodesM. Davey and D. MacKay. “Low density parity check codes over GF(q)”.In: IEEE Commun. Lett. (1998).

J. Berkmann and C. Weiss. “On dualizing trellis-based APP decodingalgorithms”. In: IEEE Trans. Commun. (2002).

L. Barnault and D. Declercq. “Fast decoding algorithm for LDPC overGF(2q)”. In: Proc. IEEE Inf. Theory Workshop (ITW). 2003.

C. Poulliat, M. Fossorier, and D. Declercq. “Design of regular (2, dc)-LDPCcodes over GF(q) using their binary images”. In: IEEE Trans. Commun.(2008).

A. Venkiah, D. Declercq, and C. Poulliat. “Design of Cages with aRandomized Progressive Edge Growth Algorithm”. In: IEEE Commun.Lett. (2008).


Bibliography VI

W. Chen, C. Poulliat, and D. Declercq. “Structured High-Girth Non-BinaryCycle Codes”. In: Proc. of APCC. 2009.

K. Kasai et al. “Multiplicatively repeated nonbinary LDPC codes”. In: IEEETrans. Inf. Theory (2011).

G. Liva et al. “Short Turbo Codes over High Order Fields”. In: IEEE Trans.Commun. (2013).

L. Dolecek et al. “Non-Binary Protograph-Based LDPC Codes:Enumerators, Analysis, and Designs”. In: IEEE Trans. Inf. Theory (2014).


Bibliography VII

Turbo Codes & Generalized Iterarive CodesP. Elias. “Error-free coding”. In: IRE Trans. on Inform. Theory (1954).

M. Tanner. “A Recursive Approach to Low Complexity Codes”. In: IEEETrans. Inf. Theory (1981).

C. Berrou, A. Glavieux, and P. Thitimajshima. “Near Shannon limiterror-correcting coding and decoding: Turbo-codes”. In: Proc. IEEE Int.Conf. Commun. (ICC). 1993.

N. Wiberg. “Codes and Decoding on General Graphs”. PhD thesis.Linköping University, 1996.

M. Lentmaier and K. S. Zigangirov. “Iterative decoding of generalizedlow-density parity-check codes”. In: Proc. IEEE Int. Symp. Inf. Theory(ISIT). 1998.

S. ten Brink. “Convergence Behavior of Iteratively Decoded ParallelConcatenated Codes”. In: IEEE Trans. Commun. (2001).


Bibliography VIII

S. Crozier and P. Guinand. “High-performance low-memory interleaverbanks for turbo-codes”. In: IEEE VTC. 2001.

H. El-Gamal and Jr. Hammons AR. “Analyzing the turbo decoder using theGaussian approximation”. In: IEEE Trans. Inf. Theory (2001).

C. Radebaugh, C. Powell, and R. Koetter. “Wheel codes: Turbo-like codeson graphs of small order”. In: Proc. IEEE Inf. Theory Workshop (ITW).2003.

O. Takeshita. “On maximum contention-free interleavers and permutationpolynomials over integer rings”. In: IEEE Trans. Inf. Theory (2006).

G. Liva, W. Ryan, and M. Chiani. “Quasi-cyclic Generalized LDPC codeswith low error floors”. In: IEEE Trans. Commun. (2008).

M. Lentmaier et al. “From Product Codes to Structured Generalized LDPCCodes”. In: Proc. Chinacom. 2010.


Bibliography IX

OtherS. Hakimi and J. Bredeson. “Graph theoretic error-correcting codes”. In:IEEE Trans. Inf. Theory (1968).

J. MacWilliams and N. Sloane. The theory of error-correcting codes.North Holland Mathematical Libray, 1977.

J.K. Wolf. “Efficient maximum likelihood decoding of linear block codesusing a trellis”. In: IEEE Trans. Inf. Theory (1978).

P.K. Wong. “Cages – A survey”. In: J. Graph Theory (1982).

M. Fossorier and S. Lin. “Soft-decision decoding of linear block codesbased on ordered statistics”. In: IEEE Trans. Inf. Theory (1995).

Y. Wu and C. Hadjicostis. “Soft-decision decoding using ordered recodingson the most reliable basis”. In: IEEE Trans. Inf. Theory (2007).

G. Liva, E. Paolini, and M. Chiani. “On Optimum Decoding of CertainProduct Codes”. In: IEEE Commun. Lett. (2014).

Date post:	18-Mar-2020
Category:	Documents
Upload:	others
View:	8 times
Download:	0 times

Code Design in the Short Block Length RegimeExample: (2048;1024) Codes Non-Binary LDPC code from C....

Documents