Home > Documents > incompletepreliminaryversion - TU Wien · 292 Chapter 7. LDPC Codes 7.1 Introduction and Outline...

# incompletepreliminaryversion - TU Wien · 292 Chapter 7. LDPC Codes 7.1 Introduction and Outline...

Date post: 02-Sep-2019
Category:
View: 4 times
Embed Size (px)
of 28 /28
7 LDPC Codes incomplete preliminary version You do not have to study this chapter for the exam. Contents 7.1 Introduction and Outline ............................. 292 7.2 Deﬁnition and Properties of LDPC Codes .................. 292 7.2.1 Deﬁnition of LDPC Codes .............................. 293 7.2.2 Regular LDPC Codes ................................ 295 7.2.3 Tanner Graphs .................................... 297 7.2.4 Irregular LDPC Codes ................................ 301 7.2.5 Performance ..................................... 303 7.2.6 Encoding ....................................... 304 7.2.7 Problems ....................................... 306 7.3 Some Constructions of LDPC Codes ...................... 306 7.3.1 A Simple Random Construction (MacKay–Neal Construction) ......... 306 7.3.2 Gallager’s Random Construction .......................... 306 7.3.3 Random Construction Based on Permutation Matrices .............. 307 7.3.4 Algebraic Construction Based on Cyclic Shift Matrices .............. 308 7.3.5 Algebraic Constructions Based on Finite Geometries ............... 308 7.3.6 Repeat-Accumulate Codes .............................. 310 7.3.7 Further Design Techniques ............................. 314 7.3.8 Problems ....................................... 315 7.4 Iterative Decoding ................................. 315 7.4.1 Bit-ﬂipping Algorithm ................................ 315 7.4.2 Belief Propagation (Sum-Product) Algorithm ................... 316 7.4.3 EXIT Charts and Density Evolution ........................ 316 7.4.4 Problems ....................................... 317 291
Transcript 7 LDPC Codes

incomplete preliminary version

You do not have to study this chapter for the exam.

Contents

7.1 Introduction and Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292

7.2 Definition and Properties of LDPC Codes . . . . . . . . . . . . . . . . . . 292

7.2.1 Definition of LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293

7.2.2 Regular LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295

7.2.3 Tanner Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

7.2.4 Irregular LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

7.2.5 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

7.2.6 Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304

7.2.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306

7.3 Some Constructions of LDPC Codes . . . . . . . . . . . . . . . . . . . . . . 306

7.3.1 A Simple Random Construction (MacKay–Neal Construction) . . . . . . . . . 306

7.3.2 Gallager’s Random Construction . . . . . . . . . . . . . . . . . . . . . . . . . . 306

7.3.3 Random Construction Based on Permutation Matrices . . . . . . . . . . . . . . 307

7.3.4 Algebraic Construction Based on Cyclic Shift Matrices . . . . . . . . . . . . . . 308

7.3.5 Algebraic Constructions Based on Finite Geometries . . . . . . . . . . . . . . . 308

7.3.6 Repeat-Accumulate Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

7.3.7 Further Design Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

7.3.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315

7.4 Iterative Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315

7.4.1 Bit-flipping Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315

7.4.2 Belief Propagation (Sum-Product) Algorithm . . . . . . . . . . . . . . . . . . . 316

7.4.3 EXIT Charts and Density Evolution . . . . . . . . . . . . . . . . . . . . . . . . 316

7.4.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317

291 292 Chapter 7. LDPC Codes

7.1 Introduction and Outline

Besides turbo codes, low-density parity check (LDPC) codes are a class of linear block codes that

allow communication close to channel capacity and are decoded iteratively. LDPC codes are among

the most powerful codes currently available. The sparse structure of their check matrix potentially

leads to excellent distance properties and enables suboptimum iterative decoding with a complexity

that grows only linearly with the blocklength.

LDPC codes, along with an iterative decoding algorithm, were introduced by Robert Gallager

already in the early 1960s. The method was then more or less ignored during more than 30 years.

This changed only in the mid-1990s, after the advent of turbo codes, when researchers rediscovered

the advantages of codes with sparse (low-density) check matrices and became interested in codes on

graphs and iterative decoding.

In this chapter, we discuss some basic aspects of LDPC codes and present two iterative decoding

algorithms with different performance and complexity. The chapter is organized as follows.

• In Section 7.2, we present the definition of regular and irregular LDPC codes and their description

in terms of row and column weights, degree distributions, and Tanner graphs. We also discuss

basic properties and performance aspects of LDPC codes and describe an efficient encoding

method.

• Section 7.3 presents some systematic random and algebraic constructions of LDPC codes, in-

cluding constructions based on finite geometries and repeat-accumulate codes.

• In Section 7.4, we discuss two iterative decoding algorithms for LDPC codes, the bit-flipping

algorithm and the belief propagation algorithm. We also briefly consider EXIT charts and the

density evolution scheme, which can be used to analyze the performance of iteratively decoded

LDPC codes.

7.2 Definition and Properties of LDPC Codes

Roughly speaking, an LDPC code is a linear block code for which there exists a check matrix H

that is “sparse” in a specific sense. To present the basic idea of LDPC codes, we temporarily assume

transmission of a codeword c ∈ C over a HIHO channel, at whose output a senseword v is received.

Then, the essential implication of the sparse check matrix H is that each senseword symbol vn is

involved in only very few parity check equations, and each parity check equation involves only very

few senseword symbols vn. For a sufficiently large blocklength N , this sparsity structure potentially

leads to extremely good distance properties; in particular, the code’s minimum distance increases

linearly with the blocklength. Furthermore, a sparse H enables an iterative probabilistic decoding

scheme—the belief propagation or sum-product algorithm—whose complexity scales only linearly

with the blocklength. It has been demonstrated that appropriately designed LDPC codes with very 7.2 Definition and Properties of LDPC Codes 293

large blocklength N , when decoded iteratively using the belief propagation algorithm, perform only a

fraction of a decibel away from the channel capacity.

LDPC codes have certain advantages over turbo codes:

• They tend to have a better block error performance, and a better performance on bursty channels.

• They are more amenable to high rates, and in fact can be designed for almost any rate and

blocklength. (In contrast, the rate of turbo codes is usually adjusted by means of a puncturing

scheme, which necessitates an additional design step.)

• Their error floor tends to occur at a lower BER.

• The encoder and decoder do not require interleavers.

• A single LDPC code can be universally good for a collection of channels.

• There exist iterative LDPC decoding algorithms that are easy to implement, have moderate

complexity (which scales linearly with the blocklength), and are parallelizable in hardware. In

particular, LDPC decoding using the belief propagation (sum-product) algorithm tends to be

less complex than turbo decoding using the BCJR algorithm.

• LDPC decoders inherently check if a codeword satisfying the check equations has been found,

and otherwise declare a decoding failure. (In contrast, turbo decoders usually need to perform

additional operations to compute a stopping criterion, and even then it is not clear if the decoding

result corresponds to a codeword satisfying the check equations.)

LDPC codes also have certain disadvantages relative to turbo codes:

• In general, the encoding complexity is higher than for turbo codes. (However, there are special

LDPC code constructions with low encoding complexity.)

• Iterative LDPC decoding typically requires many more iterations than iterative turbo decoding,

which may lead to a higher latency. (The complexity per iteration is much lower, though.)

7.2.1 Definition of LDPC Codes

Check matrix and parity checks. The central object in the definition, analysis, and design of

LDPC codes is the check matrix H. Consider a linear (N,K) block code C with a check matrix H.

According to Section 3.3.2, H has N rows rn, m = 0, . . . , N−1 and M = N−K linearly independent

columns hm, m = 0, . . . , N−K−1. The columns hm span the orthogonal space C⊥, which equals the

dual code; the code C itself is the linear space that is orthogonal to the linear space spanned by all

the columns hm. Equivalently, C is the null space of the matrix HT, i.e., the set of all the vectors

v ∈ [GF(q)]N for which HTv = 0. Furthermore, the rank of H is equal to N−K.

We recall from Equation (3.58) that the syndrome associated with a received senseword v is the

vector of length M = N−K defined as s = HTv. Furthermore, v ∈ C if and only if s = 0. The mth

syndrome component is 294 Chapter 7. LDPC Codes

sm = hTmv =

N−1∑

n=0

(hm)nvn =

N−1∑

n=0

Hn,mvn , m = 0, . . . ,M−1 , (7.1)

where hm is the mth column of H (note that we index the columns from 0 to M−1). Thus, we have

the following necessary and sufficient condition for v being a codeword:

v ∈ C ⇔ sm = hTmv = 0 , for all m = 0, . . . ,M−1. (7.2)

Therefore, we will call the mth syndrome component sm = hTmv the mth (parity) check.

In the context of LDPC codes, it is convenient to allow the possibility of linearly dependent columns

hm. If H contains Q additional columns that are linearly dependent on the N−K linearly independent

columns, then the size of H is N×M with

M = N−K+Q ≥ N−K .

We now have M ≥ N−K checks sm = hTmv and as many check equations (7.2), although only N−K of

them are linearly independent. Note that C is still the null space of HT, because if we add to a given

set of vectors a set of vectors that is linearly dependent on the first set, then the linear space spanned

by the vectors is unchanged. Also, the rank of H is still equal to N−K. In fact, the dimension K of

the code defined by H is now determined not via the dimension N×M of H but via the rank of H.

We furthermore recall from Section 3.3.2 the following obvious but important facts, which are a

straightforward consequence of (7.1):

• The weight of the mth column hm (i.e., the number of nonzero elements of hm) equals the

number of senseword symbols vn that are involved in the mth check sm. The positions of the

nonzero elements of hm equal the indices n of those vn that are involved in check sm = hTmv.

• The weight of the nth row rn (i.e., the number of nonzero elements of rn) equals the number of

checks sm = hTmv involving the nth senseword symbol vn. The positions of the nonzero elements

of rn equal the indices m of those checks sm that involve vn.

Note that, in the sense discussed above, the mth check sm is related to the mth column hm and the

nth senseword symbol vn is related to the nth row rn.

Definition of an LDPC code. An LDPC code is, essentially, a linear block code for which there

is a very sparse check matrix H. Here, “very sparse” means more specifically that the weight of each

row rn and of each column hm is much smaller than the matrix dimensions N and M . That is, each

check sm involves only very few senseword symbols vn, and each senseword symbol vn is involved in

only very few checks sm. The sparse check matrix H is called a low-density parity check matrix. (The

density of H is defined as the number of nonzero elements Hn,m 6= 0 divided by the total number of

elements, NM .) For reasons that will become clear in Section 7.2.3, we additionally require that any

two rows rn, rn′ with n 6= n′ have at most one “nonzero position” in common, i.e., there is at most

one index m ∈ {0, . . . ,M−1} such that the mth elements of rn and rn′ are both nonzero. This means

that there are not two senseword symbols vn , vn′ that are both involved in two different checks sm1

and sm2 simultaneously. 7.2 Definition and Properties of LDPC Codes 295

The LDPC code is then formally defined—consistent with Section 3.3.2—as the linear space that

is orthogonal to the linear space spanned by all the columns hm. That is, the LDPC code is the null

space of the matrix HT. Note that the columns hm may be linearly dependent.

The sparse check matrix H is typically nonsystematic. As any linear code, an LDPC code has

many check matrices besides H, including a systematic check matrix; these will not be sparse in

general. Similarly, the generator matrices G are not sparse in general.

7.2.2 Regular LDPC Codes

Column weight and row weight. An LDPC code whose sparse check matrix H is such that all

its columns hm have the same weight wc and all its rows rn have the same weight wr is called a regular

LDPC code with column weight wc and row weight1 wr. More formally, the sparse check matrix H of

a regular LDPC code satisfies

wH(hm) = wc , m = 0, . . . ,M−1and

wH(rn) = wr , n = 0, . . . , N−1 .

Thus, each check sm involves wc senseword symbols vn, and each senseword symbol vn is involved in

wr checks sm. For later use, we define the set Nm ⊆ {0, . . . , N−1} whose elements are the indices n

of those senseword symbols vn that are involved in the mth check sm. These indices correspond to

the positions of the nonzero elements of hm, i.e.,

Nm ,{n : (hm)n 6= 0

}={n : Hn,m 6= 0

}.

Note that if sm = hTmv 6= 0, then at least one of the senseword symbols vn with n ∈ Nm must have

been changed by the channel. Similarly, we define the set Mn ⊆ {0, . . . ,M−1} whose elements are

the indices m of those checks sm that involve the nth senseword symbol vn. These indices correspond

to the positions of the nonzero elements of rn, i.e.,

Mn ,{m : (rn)m 6= 0

}={m : Hn,m 6= 0

}.

Evidently, the cardinality of the set Nm is equal to the column weight wc, i.e.,

|Nm| = wc , m = 0, . . . ,M−1 ,

and the cardinality of the set Mn is equal to the row weight wr, i.e.,

|Mn| = wr , n = 0, . . . , N−1 .

A schematic illustration is given in Figure 7.1. Note that in the binary case (where Hn,m ∈ {0, 1}),the mth check sm = hTmv =

∑N−1n=0 Hn,mvn can be written as

sm =∑

n∈Nm

vn . (7.3)

1In the literature, a check matrix H is sometimes defined as the transpose of H, and the syndrome is expressed as

s = vHT (where s = sT and v = vT are row vectors) rather than s = HTv. Within this definition, the meanings of

column and row are interchanged, and so are the meanings of column weight wc and row weight wr, i.e., the column

(row) weight is our row (column) weight. 296 Chapter 7. LDPC Codes

Figure 7.1: S hemati representation of H, hm, rn, Nm, and Mn for a regular binary LDPC ode with

N= 6, M= 4, wc = 3, and wr = 2. (The representation is only s hemati be ause the density of H is too

high for a true LDPC ode.)

The sparsity of H helps spread out the senseword symbols vn into checks sm such that two different

checks are unlikely to involve similar sets of senseword symbols (i.e., Nm and Nm′ tend to be very

different for m 6= m′) and two different senseword symbols are unlikely to be involved in similar sets

of checks (i.e., Mn and Mn′ tend to be very different for n 6= n′).

Consider a regular (N,K) LDPC code with a sparse check matrix H of size N×M , where M =

N−K+Q. Let E denote the total number of nonzero elements of H. Clearly

E = Nwr = Mwc . (7.4)

The second equation is a relation between the parameters N ,M , wr, and wc. Thus, e.g., if blocklength

N , column weight wc, and row weight wr have been chosen, the number of columns of H follows as

M = Nwrwc

.

Since typically M ≤ N , this implies wr ≤ wc. Furthermore, note that

wrwc

=M

N. (7.5)

The density of H is defined as the number of nonzero elements divided by the total number of elements,

i.e.,

ρ ,E

NM.

Using (7.4), we obtain

ρ =wrM

=wcN. (7.6)

Rate. Let us next consider the rate of a regular LDPC code, R = K/N . If Q = 0 (i.e., all the

columns of H are linearly independent), then M = N−K, or equivalently K = N−M . That is, the

dimension K equals the difference of the two matrix dimensions of H, N−M . Formally replacing in

R = K/N the dimension K by N−M even if Q> 0 yields the design rate 7.2 Definition and Properties of LDPC Codes 297

R′ ,N−MN

= 1− M

N= 1− wr

wc, for Q ≥ 0 . (7.7)

Because M = N−K+Q ≥ N−K, we have N−M ≤ K, and hence

R′ ≤ R ,

with equality iff Q = 0. Thus, the actual rate R = K/N may be higher than the design rate R′ =

(N−M)/N . Determination of R requires determination of the rank of H (which equals N−K): we

have

R =K

N=N − (N−K)

N=N − rank{H}

N= 1− rank{H}

N.

In many cases, N is chosen quite large, from thousands to hundreds of thousands, while wr is

chosen as 3 or 4. It has been shown that wr = 2 leads to poor performance, whereas for wr ≥ 3 the

minimum distance of a “typical” regular LDPC code increases linearly with the blocklength N . In

other words, each senseword symbol should be involved in at least three checks.

7.2.3 Tanner Graphs

Any binary linear code C—note that binary implies Hn,m ∈ {0, 1}—can be represented by a Tanner

graph (TG). There are many different TGs for a code C, each one corresponding in a one-to-one fashion

to a check matrix H for C.A graph consists of a set of nodes (or vertices) and a set of edges connecting two nodes. Two

nodes connected by an edge are said to be adjacent. The number of edges connected to (or incident

on) a given node is called the degree of that node. The state diagram and trellis diagram previously

encountered in Sections 5.5.1 and 5.5.2 are two examples of graphs.

Definition and interpretation of TGs. The TG corresponding to a given N ×M check matrix

H contains two types of nodes: N bit nodes and M check sum nodes or briefly check nodes. Each bit

node corresponds to one of the N code bits cn or senseword bits vn, and each check node corresponds

to one of the M checks sm. Bit node n is adjacent to check node m—i.e., there is an edge between

these two nodes— iff Hn,m = 1, i.e., iff bit vn is involved in check sm. Thus, the degree of bit node n

equals the number of checks sm involving senseword bit vn, and the degree of check node m equals the

number of senseword bits vn involved in check sm. While there may be an edge between a bit node

and a check node, there are no edges between two bit nodes, nor between two check nodes. Thus, TGs

are bipartite graphs.2

Evidently, the TG describes which bits are involved in which checks, and thus it provides a graphical

description of the check matrix H to which it corresponds. More specifically, each bit node in the TG

corresponds to a row of H, each check node in the TG corresponds to a column of H, and each edge

in the TG corresponds to a nonzero element of H. Since the code C is the null space of H, it follows

that a TG provides a complete (though nonunique) description of C.2A bipartite graph is a graph with the following two characteristics: (i) it contains two different types of nodes, and

(ii) edges may only exist between two nodes of different type. 298 Chapter 7. LDPC Codes

Figure 7.2: TG orresponding to the he k matrix in (7.8), depi ted in two alternative but equivalent

graphi al representations.

Example. We consider the linear binary (7, 4) block code defined by the check matrix

H =

1 1 11 0 01 0 00 1 00 1 00 0 10 0 1

. (7.8)

There are seven code bits c0, . . . , c6 or senseword bits v0, . . . , v6 and three checks s0, s1, s2. For example,

check s0 corresponds to the first column of H. Thus, it involves the senseword bits v0, v1, v2 and equals

s0 = hT0v = v0 + v1 + v2. Furthermore, bit c0 or v0 corresponds to the first row of H and is involved

in all three checks s0, s1, and s2. Note that this code cannot be considered an LDPC code since the

density ρ is not very low (due to the small matrix dimensions). Note also that this is an irregular code

since the weight of the first row is different from the other row weights.

The TG corresponding to this check matrix is shown in Figure 7.2 in two different but equivalent

graphical representations. The first representation emphasizes the bipartite nature of the TG, while the

second emphasizes its tree structure (further discussed below). For example, check node s0 is adjacent

to bit nodes v0, v1, v2; bit node v0 is adjacent to check nodes s0, s1, s2; and the edge connecting check

node s0 and bit node v0 corresponds to the check matrix element H0,0 = 1. �

TGs can be established for any binary linear code. However, the TG description is most efficient

for LDPC codes, because there are only few edges. For a regular binary LDPC code, evidently, the

bit node degrees (= numbers of checks in which the various bits are involved) are all equal and given

by the row weight wr, and the check node degrees (= numbers of bits involved in the various checks)

are all equal and given by the column weight wc. This means that each bit node is adjacent to wr

check nodes and each check node is adjacent to wc bit nodes.

Cycles. A finite sequence of connected edges starting and ending in the same node, and not con-

taining a node more than once, is called a cycle (or loop). The length of a cycle is the number of

edges it contains. TGs may or may not have cycles. The TG of Figure 7.2 does not have cycles; it is 7.2 Definition and Properties of LDPC Codes 299

Figure 7.3: TG orresponding to the he k matrix in (7.9), depi ted in two alternative but equivalent

graphi al representations. One y le of length 4 is emphasized by boldfa e edges.

thus an acyclic or cycle-free graph. Any acyclic graph is a tree. The tree structure is visualized by

the right-hand part of Figure 7.2; note that each one of the nodes can be interpreted as the root of

the tree. The next two examples present codes whose TGs have cycles.

Example. We consider the linear binary (7, 4) block code defined by the check matrix3

H =

1 1 11 0 00 0 10 1 01 0 11 1 00 1 1

. (7.9)

As in the previous example, there are seven code bits c0, . . . , c6 or senseword bits v0, . . . , v6 and three

checks s0, s1, s2. Again, this is an irregular code. The TG corresponding to this check matrix is shown

in Figure 7.3. This TG contains several cycles of length 4 and 6. One cycle of length 4 is emphasized

in the graph by boldface edges; the elements in H corresponding to these edges are emphasized in

(7.9) by boldface 1s. �

Evidently, a TG—just as any bipartite graph—cannot have cycles of length less than 4, and it

cannot have cycles of odd length. The minimum length of cycles in a graph is referred to as the girth

of the graph. Thus, the girth of a TG is at least 4. Formally, the girth of an acyclic TG is infinite.

TGs are of interest not only because they provide an intuitively appealing graphical description of

an LDPC code (or, more specifically, of a check matrix). More importantly, they provide a basis for

iterative belief propagation decoding algorithms, to be discussed in Section 7.4.2. The performance of

these algorithms depends on the presence and length of the cycles in the TG. For good performance,

short cycles—especially cycles of length 4 or 6—should be avoided. This is an important criterion in

the design of LDPC codes. In particular, as demonstrated by our previous example, a cycle of length

4 in the TG corresponds to four 1s in H that lie on the corners of a submatrix of H. This means

3This code is equivalent to the (7, 4)2 Hamming code presented in Section 3.4.3. That is, H is a row-permuted version

of the check matrix in (3.73); the row permutation results in a simpler graphical representation of the associated TG. 300 Chapter 7. LDPC Codes

Figure 7.4: TG orresponding to the he k matrix in (7.10). One y le of length 6 is emphasized by

boldfa e edges.

that two different rows rn, rn′ have more than one 1-position in common. Note that our definition of

LDPC codes in Section 7.2.1 postulated that any two rows have at most one 1-position in common,

which excludes the presence of cycles of length 4 in the TG. Next, we give an example of the TG of a

regular code with girth 6.

Example. We consider the linear binary block code of length N = 6 that is defined by the check

matrix

H =

1 0 1 01 1 0 00 1 0 11 0 0 10 1 1 00 0 1 1

. (7.10)

There are six code bits c0, . . . , c5 or senseword bits v0, . . . , v5 and four checks s0, . . . , s3. However,

one check is redundant in the sense that the rank of H is only 3; indeed, it is easily verified that the

last column of H is the sum of the first three columns. Hence, while M = 4, the dimension is only

K= 3 (because rank{H} = N−K = 6−K). This is a regular code with wr=2 and wc=3. The TG

corresponding to H is shown in Figure 7.4. This TG contains several cycles of length 6 but no cycle

of length 4. Hence, the girth is 6. One cycle of length 6 is emphasized in the TG by boldface edges,

and in H by boldface 1s. �

Within the set of TGs associated with the various check matrices for the same code, it is possible

that some contain cycles whereas others are acyclic; furthermore, those TGs that contain cycles may

have different girths. Unfortunately, the minimum distance dmin of binary codes that possess an acyclic

TG is poor: it has been shown that if R ≥ 1/2, then dmin ≤ 2, and if R < 1/2, then dmin ≤ 2/R. For

growing blocklength N , short cycles become increasingly unlikely. In fact, asymptotically as N→∞,

the cycle-free assumption can be made. This fact is exploited by the density evolution technique

considered in Section 7.4.3.

Besides the girth of the TG, a related concept relevant to the performance of iterative belief

propagation decoders is the expansion of the TG. A TG with a good expansion is such that a given

subset of bit nodes is adjacent to a relatively large subset of check nodes (even though each individual

bit node is adjacent only to few check nodes), which in turn is adjacent to a large subset of other 7.2 Definition and Properties of LDPC Codes 301

bit nodes (even though each individual check node is adjacent only to few bit nodes). Thus, a given

set of erroneous bits is checked by many check nodes, which in turn receive information from a large

number of correct bits.

7.2.4 Irregular LDPC Codes

An LDPC code that is not regular is called irregular. Irregular LDPC codes have a very sparse check

matrix H in which the column weight may vary from column to column and the row weight may vary

from row to row. The ability to allocate the column and row weights flexibly provides a useful design

freedom. In fact, properly designed irregular LDPC codes tend to outperform regular LDPC codes for

large block lengths; gains of up to 0.5 dB can be obtained. In the following description, we consider

binary irregular LDPC codes for simplicity.

Degree distribution. We recall that the nth row of H corresponds to code bit cn and an associated

bit node in the TG, and the weight of the row equals the degree (number of incident edges) of that

bit node. Similarly, the mth column of H corresponds to check sm and an associated check node in

the TG, and the weight of the column equals the degree of that check node. In this sense, the terms

weight and degree can be used interchangeably. Furthermore, we recall that each 1 in H corresponds

to an edge in the TG.

Let Ni denote the number of rows of H with weight i (number of bit nodes in the TG with degree

i), and let Mi denote the number of columns of H with weight i (number of check nodes in the TG

with degree i). For the special case of a regular LDPC code with column weight wc and row weight

wr, we have Ni = Nδi,wr and Mi =Mδi,wc. Furthermore, let wr,max denote the maximum row weight

(maximum bit node degree), and let wc,max denote the maximum column weight (maximum check

node degree). Thenwr,max∑

i=1

Ni = N ,

wc,max∑

i=1

Mi = M. (7.11)

Moreover, the total number of 1s in H (total number of edges in the TG) can be expressed as

E =

wr,max∑

i=1

iNi =

wc,max∑

i=1

iMi . (7.12)

Note that this generalizes relation (7.4), which is reobtained for a regular LDPC code. We also consider

the fraction of rows of H with weight i (fraction of bit nodes with degree i),

νi ,Ni

N, (7.13)

and the fraction of columns of H with weight i (fraction of check nodes with degree i),

µi ,Mi

M.

Note that∑wr,max

i=1 νi =∑wc,max

i=1 µi = 1. The pair (ν,µ) of vectors ν , (ν1 · · · νwr,max)T and µ ,

(µ1 · · · µwc,max)T is called the degree distribution of the LDPC code. It is an important characteristic

of an irregular LDPC code. For a regular code with column weight wc and row weight wr, we have

νi = δi,wr and µi = δi,wc . 302 Chapter 7. LDPC Codes

Edge-related degree distribution. An alternative—but equivalent—definition of degree distri-

bution is related to the edges in the TG. Let Er,i denote the total number of 1s in H that occur in all

the rows with weight i (total number of edges in the TG that are incident on all the bit nodes with

degree i, and also total number of check nodes adjacent to all the bit nodes with degree i). Similarly,

let Ec,i denote the total number of 1s in H that occur in all the columns with weight i (total number

of edges in the TG that are incident on all the check nodes with degree i, and also total number of

bit nodes adjacent to all the check nodes with degree i). For the special case of a regular LDPC code

with column weight wc and row weight wr, we have Er,i = Eδi,wr and Ec,i = Eδi,wc . We note that

wr,max∑

i=1

Er,i = E ,

wc,max∑

i=1

Ec,i = E . (7.14)

Evidently, Er,i is equal to i multiplied by the number of rows with weight i (bit nodes with degree i),

i.e.,Er,i = iNi , (7.15)

and similarly

Ec,i = iMi .

Furthermore, we consider the fraction of 1s in H that occur in all the rows with weight i (fraction of

edges incident on all the bit nodes with degree i),

ǫr,i ,Er,iE

, (7.16)

and the fraction of 1s in H that occur in all the columns with weight i (fraction of edges incident on

all the check nodes with degree i),

ǫc,i ,Ec,iE

.

Note that∑wr,max

i=1 ǫr,i =∑wc,max

i=1 ǫc,i = 1. The pair (ǫr, ǫc) of vectors ǫr , (ǫr,1 · · · ǫr,wr,max)T and

ǫc , (ǫc,1 · · · ǫc,wc,max)T is an alternative version of the degree distribution of the LDPC code. It is

sometimes convenient to express the vectors ǫr and ǫc by the polynomials

ǫr(x) ,

wr,max∑

i=1

ǫr,i xi−1, ǫc(x) ,

wc,max∑

i=1

ǫc,i xi−1.

For a regular code with column weight wc and row weight wr, we have ǫr,i = δi,wr and ǫc,i = δi,wc or,

equivalently, ǫr(x) = xwr−1 and ǫc(x) = xwc−1.

The parameters N and M and, in turn, the design rate R′ (see (7.7)) can be expressed in terms

of the degree distribution (ǫr, ǫc). First, using (7.11), (7.15), and (7.16), we obtain

N =

wr,max∑

i=1

Ni =

wr,max∑

i=1

Er,ii

= E

wr,max∑

i=1

ǫr,ii, (7.17)

and similarly

M = E

wc,max∑

i=1

ǫc,ii. (7.18) 7.2 Definition and Properties of LDPC Codes 303

Then, inserting (7.17) and (7.18) into (7.7) yields for the design rate

R′ = 1− M

N= 1−

∑wc,max

i=1 ǫc,i/i∑wr,max

j=1 ǫr,j/j.

It is easily verified that this can also be expressed as

R′ = 1−∫ 10 ǫc(x) dx∫ 10 ǫr(x) dx

.

The two definitions of degree distribution (ν,µ) and (ǫr, ǫc) are equivalent in that one can be

obtained from the other. Indeed, using (7.13), (7.15), (7.16), and (7.17), we obtain

νi =Ni

N=

Er,i/i

N=

ǫr,i/i∑wr,max

j=1 ǫr,j/j,

and similarly

µi =ǫc,i/i∑wc,max

j=1 ǫc,j/j.

Conversely, carrying out the above steps in reverse order yields

ǫr,i =iνi∑wr,max

j=1 jνj, ǫc,i =

iµi∑wc,max

j=1 jµj.

The “edge-related” degree distribution (ǫr, ǫc) is closely related to the density evolution technique

for irregular LDPC codes, which will be sketched in Section 7.4.3. The density evolution technique is

an important design tool that allows the determination of an “optimal” degree distribution. From the

ensemble of LDPC codes having the optimal degree distribution, a specific LDPC code is then chosen.

Using this approach, an irregular LDPC code of blocklength N = 107 and rate R = 1/2 has been

designed whose simulated performance was within 0.045 dB of the capacity limit for a binary-input

AWGN channel.

7.2.5 Performance

Suitably designed LDPC codes with large blocklength provide near-capacity performance on many

data transmission and storage channels while at the same time admitting implementable decoders.

LDPC codes can be roughly categorized as random and algebraic constructions. The performance of

randomly constructed LDPC codes is good (with high probability) if the blocklength N is large enough,

but may not be sufficiently good for moderate values of N . On the other hand, randomly constructed

LDPC codes typically do not have a sufficient structure to allow efficient encoding. (Note that with

LDPC codes, the complexity of decoding is a lesser issue because iterative message passing algorithms

allow efficient decoding with a complexity that is only linear in N , see Section 7.4.) Algebraic LDPC

codes, due to their strong structure, can usually be encoded more efficiently than random LDPC codes,

and for moderate values of N their performance is often better. However, they tend to perform less

well than random LDPC codes when N is very large.

The BER-versus-SNR performance of LDPC codes for the AWGN channel exhibits an error floor

that depends on the minimum distance dmin; it is low if dmin is large (cf. Section 6.2.3). LDPC codes

tend to have a low error floor— in particular, lower than turbo codes. This is because for a very large 304 Chapter 7. LDPC Codes

blocklength N and a very sparse check matrix H, it is relatively easy to pseudorandomly construct a

code with a large dmin. The sparsity of H guarantees the linear growth of dmin with N for suitably

constructed LDPC code ensembles. In particular, it has been shown that for N sufficiently large, there

exists an LDPC code with rate R ≥ 1− r and fractional minimum distance δ = dmin/N ≥ δ0, for any

δ0 < 1/2 satisfying H2(δ0) < r. (Here, as in Section 2.4.3, H2(·) is the binary entropy function, i.e.,

H2(p) = −p ld p − (1−p) ld (1−p).) In addition, the sparsity of H is essential for a low complexity

of iterative message-passing decoding (see Section 7.4). In fact, the sparsity of H enables both a

minimum distance that grows linearly with the blocklength and a decoding complexity that grows

only linearly with the blocklength.

For very large blocklengths, randomly constructed irregular LDPC codes can perform a small

fraction of a decibel away from the channel capacity. However, such codes tend to have a higher

error floor. Conversely, LDPC codes with an extremely low error floor tend to perform far away from

capacity. Thus, there is a tradeoff between capacity-approaching performance and a low error floor.

For small and medium blocklengths, algebraic constructions can outperform random ones, and

irregular codes are not necessarily better than regular ones. In addition, the structure inherent to

algebraic constructions often leads to much simpler implementations, especially for encoding, and to

minimum distance and girth properties that are difficult to achieve with a random construction for

short and medium blocklengths. In particular, for reasonably small blocklengths, algebraic construc-

tions with a relatively high row weight are able to achieve very low error floors, the downside being a

larger gap to capacity. Some random and algebraic constructions of LDPC codes will be presented in

Section 7.3.

7.2.6 Encoding

LDPC codes are usually designed by first constructing a sparse check matrix H and then determining

a corresponding generator matrix G. Because G is not sparse, the complexity of encoding via the

generic relation c = Gu is quadratic in the blocklength N . However, there exist various classes

of LDPC codes that possess a deterministic structure allowing encoding with reduced complexity.

Examples are certain LDPC code designs based on finite geometries, which lead to cyclic or quasi-

cyclic LDPC codes that can be encoded using shift register circuits (see Section 7.3.5), and the class

of repeat-accumulate codes, which use encoders composed of simple blocks (see Section 7.3.6). On

the other hand, the efficient encoding enabled by a deterministic structure may come at the cost of a

poorer error performance for large blocklengths.

Low-complexity encoding for unstructured LDPC codes. In the absence of a deterministic

structure, it is still possible to significantly reduce the encoding complexity by an encoding method

that is based not on the generator matrix G but directly on the check matrix H. The codeword is

constructed in the systematic form (3.33), i.e., c = (uT pT )T. Thus, encoding a dataword u amounts

to determining the corresponding parity word p. This can be done by solving the check equation

HTc = 0, i.e.,

HT

(up

)= 0 , (7.19) 7.2 Definition and Properties of LDPC Codes 305

for p. Here, H is a check matrix that is consistent with the systematic form of c. We also assume for

simplicity that M = N−K, i.e., all columns of H are linearly independent.

In general, the complexity of solving (7.19) is still quadratic in the blocklength N . However, using

elementary column operations and row permutations, it is possible to transform H into an equivalent

check matrix H in block form, where one of the blocks is zero and a further block is triangular.

Thereby, the complexity of solving (7.19) (suitably transformed) is reduced significantly. Since many

of the steps in the transformation H → H are merely column and row permutations, which do not

change the sparsity of H, the resulting check matrix H preserves the sparsity of H to a large extent.

More specifically, as a first step, column and row permutations are used to obtain the transpose

of an equivalent check matrix HD as

HTD =

(A B TC′ D′ E

),

where A is of size (M−g)×K (for some g<M ; note also that M =N−K), B is of size (M−g) × g,

T is a lower triangular (M−g) × (M−g) matrix with 1s on its diagonal (hence, T−1 exists), C′ is of

size g ×K, D′ is of size g × g, and E is of size g × (M−g). Note that HD is exactly as sparse as H.

Next, HTD is multiplied from left by the matrix

(IM−g 0

−ET−1 Ig

). This amounts to performing elementary

row operations and column permutations in HTD and leaves the blocks A, B, and T unchanged, while

the block E is cleared. Thus, the result of this multiplication is of the form

HT =

(A B TC D 0

),

where C = −ET−1A+C′, D = −ET−1B+D′, and 0 is the g × (M−g) zero matrix.

The equivalent check matrix H is now used instead of H in (7.19). Let us partition the codeword

as c = (uT pT1 pT2 )T, where p1 and p2 are of length g and M−g, respectively. Then the new check

equation HTc = 0 is equivalent to the following two linear equations in p1 and p2:

Au+Bp1 +Tp2 = 0 (7.20)

Cu+Dp1 = 0 . (7.21)

p1 = −D−1Cu . (7.22)

(If D is singular, then the columns of HT can be permuted until D becomes nonsingular.) Next, (7.20)

gives

p2 = −T−1(Au+Bp1) . (7.23)

Thus, the parity blocks p1 and p2 have been calculated from the dataword u. We note that the

complexity of the matrix-vector multiplication in (7.22) is of order gK, and hence it is desirable to

keep g as small as possible. Furthermore, because T is lower triangular, (7.23) can be calculated by

first calculating d , −(Au+Bp1) and then using back-substitution to solve the equation Tp2 = d.

The complexity of this calculation is low due to the sparsity of A, B, and T. 306 Chapter 7. LDPC Codes

7.2.7 Problems

to be done

7.3 Some Constructions of LDPC Codes

The most obvious way to design an LDPC code is to construct a sparse check matrix H with suit-

able properties. Important design criteria include efficient encoding, near-capacity performance, and

low error floor. For long codes, randomly choosing a check matrix almost always produces a good

code. This is guaranteed by the concentration property, which states that the behavior of long codes

randomly drawn from an ensemble concentrates around the ensemble average. However, in practical

applications the block lengths may not be large enough, possibly due to latency constraints. Most

LDPC codes are constructed pseudorandomly, where the basic construction is random but also features

some structure, and certain bad configurations—such as 4-cycles—are avoided by the construction or

removed afterwards.

Many (pseudo)random and algebraic constructions of LDPC codes have been proposed. In this

section, we present or sketch some of them. For simplicity and practical relevance, we consider only

binary codes.

7.3.1 A Simple Random Construction (MacKay–Neal Construction)

A rather straightforward algorithm for randomly constructing an N×M check matrix H with column

weight wc and row weight wr (where M = Nwr/wc) proceeds as follows. The rows r0, r1, etc. are

randomly generated one after the other. More precisely, for generating the nth row, a binary candidate

vector of length M and weight wr is randomly drawn. It is then checked whether (i) the weight of

each partial column up to this point is at most wc and (ii) any two rows up to this point have at most

one “1-position” in common. If these two properties are satisfied, the candidate vector is accepted

and thus becomes the nth row, and the algorithm proceeds to generate the next row. Otherwise, the

candidate vector is rejected and a new candidate vector for the nth row is generated and checked, etc.

Note that this construction avoids cycles of length 4 in the TG (cf. Section 7.2.3).

With this algorithm, in the final check matrix H, the weight of each column is either wc or smaller

than wc. Thus, in general, an irregular LDPC code is obtained. If a regular code is required, the

algorithm has to be modified in a suitable way. A disadvantage of this code construction is that it

does not offer any specific structure that facilitates efficient encoding.

7.3.2 Gallager’s Random Construction

Gallager proposed the following random construction of a regular LDPC code. The code parameters

are chosen such that N = pwc and M = pwr with an integer factor p, so that N/wc = M/wr = p.

(Note that condition (7.4) is satisfied.) The density of H follows from (7.6) as ρ = wr/M = 1/p.

Thus, the code is an LDPC code if p is chosen sufficiently large. Then, the N×M check matrix H is 7.3 Some Constructions of LDPC Codes 307

constructed as a block-row matrix composed of wr ≥ 3 blocks Hi, i = 1, . . . , wr of dimension N × p

each, i.e.,H =

(H1 · · · Hwr

). (7.24)

Here, the blocks Hi are constructed such that each row of Hi has one 1 and each column has wc 1s, all

other elements being 0. This construction guarantees that H in (7.24) has row weight wr and column

weight wc, as desired.

More specifically, the first block H1 is chosen as

H1 =

1 0 0 · · · 00 1 0 · · · 0.........

...0 0 0 · · · 1

,

where 1 is the all-one vector of length wc, 0 is the zero vector of length wc, and there are p block rows

and p columns. It is seen that, as desired, H1 is an N×p matrix in which each row has one 1 and each

column has wc 1s (in the mth column of H1, these 1s occur at positions n = mwc, mwc +1, . . . , (m+

1)wc − 1; note that m ∈ {0, . . . , p−1} and n ∈ {0, . . . , N−1}). Furthermore, no two columns of H1

have any “1-position” in common, and no two rows of H1 have more than one “1-position” in common.

It is easily verified that the latter two properties remain true if the N rows of H1 are permuted in

an arbitrary manner. Therefore, the remaining blocks Hi, i = 2, . . . , wr are derived from H1 by row

permutations, i.e., certain rows are swapped. These row permutations are chosen randomly. Gallager

showed that if each permutation is chosen randomly from among the N ! possible permutations, then

the average minimum distance (averaged over the entire code ensemble) increases linearly with the

blocklength N provided that wr ≥ 3 and wc > wr. This can be viewed as a constructive realization of

Shannon’s idea of a random code. Furthermore, these codes allow for low-complexity encoding since

parity bits can be calculated from the data bits via the check matrix.

For different specific choices of row permutations, the weight distributions and, in particular,

minimum distances of the resulting codes will generally be different. Also, the property that any two

rows rn, rn′ of H have at most one “1-position” in common (i.e., cycles of length 4 in the TG are

avoided) is only satisfied for certain choices of row permutations. In practice, a computer search is

required to find good row permutations.

7.3.3 Random Construction Based on Permutation Matrices

Another random construction starts with the N0×M0 check matrix H0 of a good “core” regular LDPC

code. Let wc (wr) be the column (row) weight of H0. A longer regular LDPC code is now obtained

by replacing each 1 in H0 with a randomly chosen p1 × p1 permutation matrix4 and replacing each

0 in H0 with the p1 × p1 zero matrix. The result is a new check matrix H1 of increased dimension

N1 ×M1, where N1 = p1N0 and M1 = p1M0. It is easily verified that the column (row) weight of

H1 is unchanged, i.e., still wc (wr). Next, each 1 in H1 is replaced with a randomly chosen p2 × p2

permutation matrix and each 0 with the p2 × p2 zero matrix. This yields a new check matrix H2 of

4A permutation matrix is a square matrix that has a single 1 in each row and a single 1 in each column, all other

elements being zero. 308 Chapter 7. LDPC Codes

increased dimension N2×M2, where N2 = p2N1 = p1p2N0 andM2 = p2M1 = p1p2M0, and of the same

column (row) weight wc (wr) as previously. Evidently, this “inflation” procedure can be repeated an

arbitrary number of times. It has been shown that this construction preserves the girth and minimum

distance of the core code.

7.3.4 Algebraic Construction Based on Cyclic Shift Matrices

A simple algebraic construction of a check matrix H with column weight wc and row weight wr is as

follows. For an arbitrary p ≥ (wc−1)(wr−1) + 1 and an arbitrary l ∈ {0, . . . , p−1}, consider the p× p

cyclic shift matrix Jl that is obtained by cyclically shifting each column of the p × p identity matrix

I down by l positions.5 For example, for l= 1,

J1 =

0 0 0 · · · 0 11 0 0 · · · 0 00 1 0 · · · 0 00 0 1 · · · 0 0...

......

......

0 0 0 · · · 1 0

.

Note that Jl = Jl1 and J0 = I. Then, the check matrix constructed as

H =

I I I · · · I

I J1 J2 · · · Jwr−1

I J2 J4 · · · J2(wr−1)

I J3 J6 · · · J3(wr−1)...

......

...

I Jwc−1 J2(wc−1) · · · J(wc−1)(wr−1)

has N = pwc rows and M = pwr columns; furthermore, each row (column) has wr (wc) 1s. It can

also be shown that two different rows have at most one “1-position” in common, i.e., the TG does not

have cycles of length 4.

7.3.5 Algebraic Constructions Based on Finite Geometries

A finite geometry G is a mathematical object that consists of NP points pi, i ∈ {0, . . . , NP−1} and NL

lines lj, j ∈ {0, . . . , NL−1}. Each line lj is a set of some of the points pi, and the following structural

properties can be postulated:

1. Each line consists of nP points.

2. Each point belongs to nL lines.

3. Two points simultaneously belong to exactly one line.

4. Two lines have either no or one point in common.5We note that Jl is a special case of a permutation matrix. Multiplying a p-dimensional vector by Jl produces a

cyclic shift of the vector elements by l positions. 7.3 Some Constructions of LDPC Codes 309

The incidence vector of line lj is defined as the NP-dimensional binary vector ψL,j ∈ {0, 1}NP with

elements(ψL,j

)i=

{1 , if line lj contains point pi

0 , else ,

for i ∈ {0, . . . , NP−1}. Thus, ψL,j indicates which points belong to line lj . Due to Property 1 above,

the weight of ψL,j is nP.

Similarly, the incidence vector of point pi is defined as the NL-dimensional binary vector ψP,i ∈{0, 1}NL with elements

(ψP,i

)j=

{1 , if point pi belongs to line lj

0 , else ,

for j ∈ {0, . . . , NL−1}. Thus, ψP,i indicates which lines lj contain point pi. Due to Property 2 above,

the weight of ψP,i is nL. Note that the set {ψP,i}NP−1i=0 contains exactly the same information as the

set {ψL,j}NL−1j=0 .

For a given finite geometry G, the check matrix HI of the type-I geometry-G LDPC code has the

incidence vector ψL,j as its jth column and the incidence vector ψP,i as its ith row. Thus, we obtain

a regular LDPC code with N = NP (there are NP rows) and M = NL (there are NL columns). With

this type-I construction, the nth senseword bit vn (or, the nth row) corresponds to the point pn, and

the mth check sm (or, the mth column) corresponds to the line lm. Furthermore, the above properties

1–4 translate into the following properties of the check matrix HI:

1. The column weight is wc = nP (because each ψL,j has weight nP).

2. The row weight is wr = nL (because each ψP,i has weight nL).

3. Two rows have one “1-position” in common (because the two rows correspond to two points,

which simultaneously belong to one line).

4. Two columns have at most one “1-position” in common (because the two columns correspond

to two lines, which have at most one point in common).

By (7.6), the density of HI is ρ = wr/M = nL/NL and simultaneously ρ = wc/N = nP/NP; thus, for

an LDPC code, nL and nP must be sufficiently small. The Tanner graph has no cycles of length 4

(due to Property 3); however, it can be shown to have cycles of length 6. The minimum distance of

the code can be shown to be at least nL + 1.

Similarly, the check matrix HII of the type-II geometry-G LDPC code has ψL,j as its jth row and

ψP,i as its ith column. This results in a regular LDPC code with N = NL and M = NP. Here, the

nth senseword bit vn (or, the nth row) corresponds to the line ln, and the mth check sm (or, the mth

column) corresponds to the point pm. The above properties 1–4 remain valid with the roles of column

and row interchanged and with wc and wr interchanged. We have HII = HTI , and thus, the density

and rank of HII equal those of HI. Since the ranks of HI and HII are equal, the numbers of linearly

independent check equations for the type-I and type-II geometry-G codes are equal too. Furthermore,

the TGs associated with HI and HII are dual graphs in the sense that the bit nodes (check nodes)

of one graph become the check nodes (bit nodes) of the other graph. The minimum distance of the

type-II code is at least nP + 1. 310 Chapter 7. LDPC Codes

Two standard constructions of finite geometries (not discussed here) are Euclidean geometries

(EGs) and projective geometries (PGs). For the m-dimensional EG over GF(2s),

NL =2(m−1)s(2ms−1)

2s−1, NP = 2ms, nP = 2s, nL =

2ms−1

2s−1.

For the m-dimensional PG over GF(2s),

NL =ξs(m)ξs(m−1)

2s+1, NP =

2(m+1)s−1

2s−1, nP = 2s+1, nL =

2ms−1

2s−1,

where ξs(m) ,∑m

µ=0 2µs = 1 + 2s + . . . + 2ms. We now obtain four classes of finite-geometry LDPC

codes: type-I EG-LDPC codes, type-II EG-LDPC codes, type-I PG-LDPC codes, and type-II PG-

LDPC codes. The parameters N , M , wc, wr, and ρ of these code classes follow easily from our

discussion above. Note that although the EG and PG are “over GF(2s),” the resulting LDPC codes

are over GF(2), since the incidence vectors are still binary vectors.

A suitable modification of the type-I EG-LDPC codes results in cyclic codes, and a suitable modi-

fication of the type-II EG-LDPC codes results in quasi-cyclic codes. (For a quasi-cyclic code, a cyclic

shift of a codeword by any integer multiple of a certain number l of positions is another codeword.

Thus, in particular, cyclic codes are quasi-cyclic codes with l= 1.) These codes can be easily encoded

with moderate complexity—linear in the blocklength N—using one or several shift-register circuits.

Their parameters are somewhat different from those of the original EG-LDPC codes. The type-I PG-

LDPC codes are themselves cyclic codes, and a suitable modification of the type-II PG-LDPC codes

yields quasi-cyclic codes.

For small blocklength N , these finite-geometry LDPC codes perform generally better than LDPC

codes based on random constructions. They have relatively good minimum distances, and their TGs

do not contain short cycles. Their performance can be further improved by row splitting (see Section

7.3.7). They can be decoded in various ways ranging from low to high complexity and from reasonably

good to very good error performance. They can be encoded with linear complexity. On the other hand,

there are certain restrictions in the choice of blocklength and rate. Furthermore, the values of wc and

wr tend to be relatively high, which is undesirable since the complexity of iterative message-passing

decoders is proportional to these values.

7.3.6 Repeat-Accumulate Codes

A special type of irregular binary LDPC codes is the class of repeat-accumulate codes (RA codes),

which have characteristics of both LDPC codes and serial turbo codes. A major advantage of RA

codes is their efficient encoding, whose complexity is linear in the blocklength N . In spite of their

simplicity and linear encoding complexity, RA codes perform very well over the AWGN channel and

are capable of operation near capacity. However, a potential drawback is their low rate (typically 1/2

or lower).

Encoding. For a systematic (q, a)-regular RA code, the codeword is constructed in the systematic

form (3.33), i.e., c = (uT pT )T. The encoder is shown in Figure 7.5. We see that the parity word p = 7.3 Some Constructions of LDPC Codes 311

Figure 7.5: Blo k diagram of an en oder for a (q, a)-regular RA ode.

(p0 · · · pM−1)T of length M = N −K is calculated from the length-K dataword u = (u0 · · · uK−1)

T by

the serial concatenation of a binary repetition encoder of blocklength q (or rate 1/q), a pseudorandom

interleaver of length Kq, a combiner summing over blocks of length a, and an accumulator. These

four components are discussed in the following.

1. The repetition encoder produces q copies of each data bit uk. Thus, its output corresponding to

the entire length-K dataword u = (u0 u1 · · · uK−1)T is the length-qK word

b = (b0 · · · bqK−1)T = (u0 · · · u0︸ ︷︷ ︸

q times

u1 · · · u1︸ ︷︷ ︸q times

· · · uK−1 · · · uK−1︸ ︷︷ ︸q times

)T .

Typical values of q are 2 or 3.

2. The pseudorandom interleaver permutes the bits in b, resulting in a new word d = (d0 · · ·dqK−1)

T of the same length qK.

3. In the combiner, the bits in each successive length-a block of d are summed (modulo 2), which

produces one bit for each block of a bits. Thus, the output of the combiner is a binary word of

length6 qK/a, r = (r0 · · · rqK/a−1)T, whose bits are given by the mod-2 sums

rm =a−1∑

i=0

dma+i , m = 0, . . . ,qK

a−1 .

4. Finally, the accumulator produces the parity word p = (p0 · · · pM−1)T of length M = N − K.

The mth parity bit pm is the mod-2 sum of the previous parity bit pm−1 and the mth combiner

output bit rm, i.e.,

pm = pm−1 + rm , m = 0, . . . ,qK

a−1 .

This recursion is initialized (for m = 0) with p−1 = 0. Note that pm =∑m

m′=0 rm′ . The

accumulator can be viewed as a rate-1 recursive convolutional encoder described by the 1×1

generator matrix G(x) =(g(0,0)(x)

)with g(0,0)(x) = 1

1+x .

Parameters. From the above construction of the encoder for a systematic (q, a)-regular RA code,

it is clear that the number of parity bits pm is given by qK/a, i.e., we have

M =qK

a.

6The parameters K, q, and a are chosen such that qK/a is integer. 312 Chapter 7. LDPC Codes

Furthermore, the blocklength of a (q, a)-regular RA code is obtained as

N = K+M = K +qK

a= K

a+ q

a,

and the rate follows as

R =a

a+ q.

There are also nonsystematic RA codes, whose codewords do not include the data bits uk. In other

words, the codewords are simply given as c = p. Here, the blocklength is N = qK/a and the rate is

R = a/q. In what follows, we will only consider systematic RA codes.

Check matrix. It can be shown that for a systematic (q, a)-regular RA code, a sparse check matrix

is given by the N×M matrix

H =

(H1

H2

),

where H1 is an (N−M)×M = K×M matrix with (small) column weight a and row weight q, and

H2 is an upper triangular M×M matrix of the following special form:

H2 =

1 1 0 0 · · · 0 0 00 1 1 0 · · · 0 0 00 0 1 1 · · · 0 0 00 0 0 1 · · · 0 0 0...

......

......

......

0 0 0 0 · · · 1 1 00 0 0 0 · · · 0 1 10 0 0 0 · · · 0 0 1

.

Note that although the RA code is encoded systematically, H is not in systematic form (cf. (3.48)).

Inspection of H also shows that a (q, a)-regular RA code is actually an irregular LDPC code— in

particular, the rows in the H2 block have weight 2 except the last row, which has only weight 1, and

the rows in the H1 block have weight q. Also, the columns of H have weight a + 2 except the first

column, which has only weight a+1.

Because of the systematic encoding, one can distinguish two types of code bits: K data (message)

bits and M parity bits. The upper K rows of H—which make up the upper matrix block H1 —

correspond to the data bits whereas the remaining lower M rows of H—which make up the lower

matrix block H2 —correspond to the parity bits. Note also that H1 is related to the joint operation

of repetition encoder, interleaver, and combiner, whereas the special structure of H2 corresponds to

the operation of the accumulator.

There are also irregular RA codes. Their check matrix H is constructed as above, except that

the row weights of the matrix block H1 are not all equal. This means that instead of using a single

repetition encoder on all data bits uk, different subsets of data bits are repeated different numbers of

times, i.e., some bits are repeated more often than others. Irregular RA codes are capable of operation

even closer to capacity, and they permit higher rates. 7.3 Some Constructions of LDPC Codes 313

Figure 7.6: TG of a parti ular (3, 2)-regular RA ode of blo klength N = 10, orresponding to the he k

matrix in (7.25).

Tanner graph. The TG of a systematic (q, a)-regular RA code has a special structure. The N bit

nodes consist of K data (message) bit nodes and M parity bit nodes. These nodes are connected as

follows:

• Each data bit node is adjacent to q check nodes (this is due to the length-q repetition encoder).

The choice of these q check nodes depends on the interleaver and the combiner.

• Each parity bit node is adjacent to a corresponding check node and, except for the last parity

bit node, to the respective next check node (this is due to the accumulator).

• Each check node is adjacent to a corresponding parity bit node and, except the first check node,

to the respective previous parity bit node (this is due to the accumulator). In addition, each

check node is adjacent to a data bit nodes (due to the combiner), whose choice depends on the

repetition encoder and the interleaver.

This structure of the TG will be made more concrete by considering a simple example.

Example. Consider a (3, 2)-regular RA code (i.e., q = 3 and a = 2) of blocklength N = 10. The

rate of this code is R = a/(a+ q) = 2/5, the dimension is K = RN = 4, and the number of checks is

M = N −K = 6. A possible check matrix is

H =

1 0 1 0 1 00 1 1 0 0 11 0 0 1 1 00 1 0 1 0 1− − − − − −1 1 0 0 0 00 1 1 0 0 00 0 1 1 0 00 0 0 1 1 00 0 0 0 1 10 0 0 0 0 1

. (7.25)

The corresponding TG is shown in Figure 7.6. For easier interpretation of the TG, the K = 4 data bit

nodes and M = 6 parity bit nodes are depicted above and below the M = 6 check nodes, respectively. 314 Chapter 7. LDPC Codes

For example, the first (leftmost) check node in the TG corresponds to the first column of H, and

thus the first check involves the first and third data bit and the first parity bit. Accordingly, in the

TG, the first check node is adjacent to the first and third data bit nodes and the first parity bit node.

We can now verify the general rules stated further above: Each data bit node is adjacent to q = 3

check nodes; each parity bit node is adjacent to a corresponding check node and, except for the last

parity bit node, to the respective next check node; and each check node is adjacent to a corresponding

parity bit node, to the respective previous parity bit node (except for the first check node), and to

a= 2 data bit nodes. �

7.3.7 Further Design Techniques

Many further designs of LDPC codes have been proposed. We briefly describe two classes of construc-

tions.

Graph-based constructions. Cycles and other undesired configurations in the TG can be removed

by row splitting and/or column splitting. In row splitting, a row of H is replaced by two or more rows,

such that each 1 of the original row is allocated to one of the new rows. Thereby, the blocklength N

(number of bits) is increased, and H is made more sparse. Similarly, in column splitting, a column of

H is replaced by two or more columns, such that each 1 of the original column is allocated to one of

the new columns. Here, the number of checks is increased, and H is again made more sparse. Both

row splitting and column splitting reduce the degrees of certain nodes in the TG and thus tend to

reduce the number of cycles.

In the bit filling construction, bit nodes are added to the TG one at a time. The edges incident

on a new bit node are chosen with the aim of avoiding cycles of length l or less. More specifically, the

wc check nodes adjacent to the new bit node are chosen such that each check node is (i) a “distance”

of at least l/2 edges away from all the check nodes already chosen, and (ii) the check node least used

so far (i.e., with the lowest degree). Similarly, in the progressive edge growth construction, edges are

added to the TG one at a time. Here, the goal is to maximize the local girth at the current bit node.

These graph-based techniques can also be used to modify an existing TG that was devised using

some structured design, e.g., the TG of a finite-geometry LDPC code or of an RA code.

Combinatorial constructions. Good LDPC codes may be designed by using concepts and tech-

niques from combinatorial mathematics. A combinatorial design is an assignment of a set of entities

(bits) to subsets (which define checks) subject to certain constraints such as the number of bits involved

in each check or the absence of cycles of specified lengths. The incidence matrix of the combinatorial

design indicates which bits belong to which checks, and is thus used as the check matrix H or the

transposed check matrix HT of the LDPC code. The finite geometry based LDPC codes discussed in

Section 7.3.5 are examples of combinatorial constructions. Other designs are based on combinatorial

objects known as Steiner systems and Kirkman systems.

LDPC codes based on combinatorial designs offer a deterministic (algebraic) construction and

guaranteed lower bounds on girth and minimum distance. They may also allow for efficient encoding. 7.4 Iterative Decoding 315

For example, the EG-LDPC and PG-LDPC code designs can produce cyclic or quasi-cyclic codes as

mentioned in Section 7.3.5. Efficient encoders can also be obtained in other cases by exploiting the

structure of the design. Combinatorial designs lead to check matrices that are both sparse and highly

redundant in the sense that they contain many linearly dependent columns, i.e.,M is much larger than

N−K. The linearly dependent check equations help achieve good performance of iterative decoding

even for moderate blocklengths.

7.3.8 Problems

to be done

7.4 Iterative Decoding

For LDPC codes, several decoding methods with different performance and complexity are available.

We will discuss two methods, which are both iterative: the bit-flipping algorithm and the belief prop-

agation algorithm (or sum-product algorithm). The former is a nonprobabilistic hard-input decoding

algorithm, the latter is a probabilistic soft-input decoding algorithm. The belief propagation algorithm

has a better performance than the bit-flipping algorithm but also a higher complexity. In general, the

complexity of iterative decoding is lower for a sparser check matrix.

7.4.1 Bit-flipping Algorithm

Consider a binary regular LDPC code. A codeword c ∈ C is transmitted over a HIHO channel, at

whose output a binary senseword v is received. Hard-decoding of v by means of bit flipping is based

on the M checks sm = hTmv =∑

n∈Nmvn , m= 0, . . . ,M−1 (see (7.3)). If v is a codeword, then all the

sm are zero. However, if a detectable error occurred, then v is not a codeword and some of the checks

sm are 1. Such checks will be called “parity failures.” According to Section 7.2.2, a given senseword

bit vn is involved in the checks sm for m ∈ Mn. Some of these checks may be parity failures. In

particular, if vn is the only incorrect senseword bit, then all the checks sm, m ∈ Mn are parity failures.

The bit-flipping algorithm is based on this observation; its principle is that a senseword bit vn that is

involved in many parity failures is most likely incorrect and should therefore be flipped (inverted). Of

course, flipping the bit vn will change the number of parity failures for some of the other senseword

bits. Therefore, this selective bit flipping is repeated in an iterative manner.

More specifically, consider the checks involving senseword bit vn, i.e., sm for m ∈ Mn, and denote

by Pn (with 0 ≤ Pn ≤ |Mn|) the number of parity failures among these checks. In the bit-flipping

algorithm, the Pn are calculated for all n ∈ {0, . . . , N−1}, and those bits vn for which Pn is above

a predefined threshold T are flipped.7 This results in a modified senseword v′. The checks are

recomputed for v′, yielding s′m = hTmv′ =

∑n∈Nm

v′n and new numbers of parity failures P ′n. These are

7In other versions of the algorithm, the bit(s) vn for which Pn is maximum is/are flipped, or the bits for which more

checks are 1 than 0 (i.e., Pn > |Mn| − Pn) are flipped. 316 Chapter 7. LDPC Codes

again compared to T , and those bits v′n for which P ′n > T are flipped. This procedure is continued

in an iterative manner. If after a predefined number of iterations there are no more parity failures,

i.e., all the M checks are zero, then the modified senseword is a codeword, and this codeword is

used as the result of the decoding algorithm. On the other hand, if there are still some parity failures,

there are two possible ways to proceed: if additional processing time is available, then further decoding

iterations may be performed with a reduced threshold T ; if no more time is available, then the decoding

procedure is terminated and a decoding failure (uncorrectable error) is declared.

This bit-flipping algorithm can be interpreted as message passing on the TG of the code. Each

bit node passes its bit value as a binary message to each adjacent check node, and each check node

passes its check value as a binary message to each adjacent bit node.

Significant overlap between two rows of H or, equivalently, short cycles in the TG reduce the effec-

tiveness of the bit-flipping algorithm and may even prevent the algorithm from converging. For exam-

ple, assume that two rows of H have two “1-positions” in common. This means that two senseword

bits are simultaneously involved in two checks—thus, when both of the checks are 1, it is impossible to

decide which bit is in error—and, also, that there is a cycle of length 4 in the TG. It is then possible

that in each iteration of the bit-flipping algorithm, the two bits are flipped simultaneously and, over

the iterations, each of them is alternatingly correct and incorrect. Hence, in that case, the algorithm

fails to converge.

7.4.2 Belief Propagation (Sum-Product) Algorithm

Presentation and discussion of the belief propagation (BP) decoding algorithm: to be done

Cycles in the TG adversely affect the performance and convergence of iterative BP decoders. In

fact, cycles lead to correlations in the probabilities passed by the BP decoder; the smaller the cycles the

fewer the number of iterations that are correlation-free. Large improvements in decoding performance

can be achieved by removing cycles of length 4 and 6. However, as the the girth of the TG is increased

further, the additional improvements tend to diminish. For LDPC codes with very large blocklength,

the TG can be assumed to be acyclic.

The number of iterations needed to achieve convergence of BP decoding increases when the channel

SNR is reduced. Below a threshold SNR, the BP decoder does not converge at all (cf. Section 7.4.3).

7.4.3 EXIT Charts and Density Evolution

EXIT charts. The extrinsic information transfer (EXIT) chart was originally developed for turbo

codes, as described in Section 6.3.9. However, an EXIT chart can also be established for iteratively

decoded LDPC codes. In an iterative message passing decoder, in each iteration j, there are messages

from check nodes to bit nodes (“check→bit messages”) and messages from bit nodes to check nodes

(“bit→ check messages”). Accordingly, the two types of mutual information used in the EXIT chart

are (i) the mutual information between a check→bit message for a fixed bit and the value of that bit,

denoted IC→B[j], and (ii) the mutual information between a bit→ check message for a fixed bit and

the value of that bit, denoted IB→C[j]. These mutual informations are estimated based on histogram 7.4 Iterative Decoding 317

estimates of the pdf’s of the involved messages. The EXIT chart is then obtained by plotting IC→B[j]

and IB→C[j] for growing iteration index j in a 2-D diagram, similarly to the mutual informations

I(i)a [j] and I

(i)e [j] in Section 6.3.9. The interpretation of this diagram and the conclusions drawn from

it are analogous to those discussed in Section 6.3.9. Again, EXIT chart analysis reveals the existence

of an SNR threshold and provides a rough characterization of the speed of convergence of iterative

BP message passing decoding for different SNRs above the threshold.

Density evolution. Besides EXIT charts, density evolution is another technique for analyzing the

performance of an iterative message passing decoder. For a given LDPC code described by a TG,

we would like to know for which channel noise levels the decoder performs successfully in the sense

of converging and correcting the errors. This question can be difficult to answer analytically for a

specific LDPC code. However, density evolution is able to answer it for classes (ensembles) of regular

LDPC codes with a given column weight wc and row weight wr, under the simplifying assumptions

of infinite blocklength (N→∞) and, hence, a cycle-free TG. By tracking the evolution of pdf’s over

the message passing iterations and studying the asymptotic behavior of these pdf’s as the number

of iterations tends to infinity, density evolution characterizes the maximum noise level (equivalently,

minimum SNR in the case of an AWGN channel) such that iterative decoding is likely to succeed.

This “decoding threshold” noise level is a clear dividing line between successful and unsuccessful de-

coding. Unfortunately, empirical evidence suggests the existence of a tradeoff between a high decoding

threshold noise level and a low error floor.

The decoding threshold noise level is a convenient characterization of performance in terms of a

single parameter. This characterization allows a designer to search for the column weight wc and row

weight wr maximizing the decoding threshold noise level. From the class of all LDPC codes with the

optimum wc and wr, a specific code can then be selected based on other criteria.

Density evolution can also be used for irregular LDPC codes. Here, classes of codes with a given

degree distribution (ǫr, ǫc) as defined in Section 7.2.4 are considered, and the goal is to determine the

degree distribution maximizing the decoding threshold noise level. Using this approach, an irregular

LDPC code of blocklength N = 107 and rate R = 1/2 has been designed whose simulated performance

was within 0.045 dB of the capacity limit for a binary-input AWGN channel.

The design of LDPC codes using the density evolution technique is best suited to codes with large

blocklength (due to the underlying N→∞ assumption). Applying the optimum degree distribution

(ǫr, ǫc) to short or medium-length codes tends to result in a high error floor.

7.4.4 Problems

to be done 318 Chapter 7. LDPC Codes

Recommended