Distance spectrum of right-regular low-densityparity-check codes: derivation and discussion ofnumerical results

M. Rashidpour and S.H. Jamali

Abstract: The asymptotic distance distribution of regular low-density parity-check (LDPC) codesderived by Litsin and Sherlev (2002) is generalised for the case of irregular LDPC codes withconstant check node degree. Numerical analysis of simple irregular LDPC codes with weight-twovariable nodes shows that their minimumHamming distance can be increased linearly with the codelength when the maximum variable node degree of the code is kept below a critical value. This resultis consistent with the stability condition of the LDPC codes, at least for the simple irregular case.

1 Introduction

Irregular low-density parity-check (LDPC) codes wereintroduced in [1]. The asymptotic design of these codeswith the density evolution technique for different symmetricchannels has been discussed in a landmark paper [2]. Welldesigned irregular LDPC codes outperform the regularones, introduced by Gallager [3] and rediscovered in [4].

Recently, the problem of estimation of the averagedistance distribution for several ensembles of regular LDPCcodes has been presented in [5]. In this paper we extend theresults of [5] to an ensemble of the irregular LDPC codes.The average distance distribution analysis enables us toestimate the minimum Hamming distance of the irregularLDPC codes, and estimate the gap between the perfor-mance of these codes under the maximum likelihoodand the conventional sum–product iterative decodingalgorithms.

As is discussed in [1, p. 592] and [6, Theorem 7.6], a welldesigned irregular LDPC code has a concentrated checknode degree distribution, and ideally a constant check nodedegree [7]. Therefore, we consider an ensemble of right-regular LDPC codes, where the check node degree isconstant.

As is originally claimed by Gallager [3], proved byMacKay [8, Theorem 3] and shown in [5, Ensemble A], theminimum Hamming distance of the regular LDPC codes islinearly increased with the code length, provided that thevariable node degrees are kept larger than two. Our specificanalysis of the simple irregular LDPC codes, with only twodifferent variable node degrees, shows that this group withdegree-two variable nodes can have a minimum Hammingdistance growing linearly with the code length provided thatthe maximum variable node degree is kept below a criticalvalue. Interestingly, this feature is consistent with thestability condition of the LDPC codes [2].

2 Preliminaries

Any irregular LDPC code is specified by a pair of edgedegree distribution polynomials, which are denoted by l(x)and r(x) for variable and check nodes, respectively. Thesedistributions are defined as [2]:

lðxÞ ¼Xdv

j¼2ljxj�1; lj � 0; lð1Þ ¼ 1 ð1Þ

rðxÞ ¼Xdc

k¼2rkxk�1; rk � 0; rð1Þ ¼ 1 ð2Þ

where dv(dc) is the maximum variable (check) nodes degree,and lj(rk) is the fraction of edges emanating from variable(check) nodes of degree j(k). The rate of the above irregularLDPC code can be stated as [2]

R ¼ 1�Z 1

0

rðxÞdx=Z 1

0

lðxÞdx ð3Þ

2.1 Ensemble of right-regular LDPC codesAn ensemble of right-regular LDPC codes can be definedby the following edge degree distribution pair:

lðxÞ ¼XJ

j¼1l‘j x

‘j�1; l‘j40;

XJ

j¼1l‘j ¼ 1; 2 � ‘1o‘2o � � �o‘J ¼ dv

ð4Þ

rðxÞ ¼ xk�1 ð5ÞLet us denote the low-density parity-check matrix of thisright-regular ensemble by a binary matrix A ¼ aij

�� ��m;n,

where m and n are the number of parities and the codewordlength, respectively. The Hamming weight of a generalcodeword in the ensemble is denoted by w. The normalisedor the relative distance is defined by y:¼w/n. Let nj standfor the number of columns with cj ones. The followingparameters are defined: a:¼m/n, aj:¼m/nj, bj¼ aj/a. Fromthe definition of the edge degree distribution we have l‘j ¼ðnj‘jÞ=ðkmÞ or ‘j ¼ kabjl‘j , where k is the constant check

The authors are with the Wireless Research Laboratory, University of Tehran,Department of Electrical and Computer Engineering, North Kargar Avenue,Tehran, Iran

r IEE, 2005

IEE Proceedings online no. 20040964

doi:10.1049/ip-com:20040964

Paper first received 26th June 2003 and in revised form 13th July 2004

IEE Proc.-Commun., Vol. 152, No. 2, April 2005 157

node degree of the right-regular ensemble in (5). It is

noticeable that 1/bj¼ nj/n,PJ

j¼1 1=bj ¼ 1, and 100/bj can

be interpreted as the percentage of columns with weight cj.

2.2 Simple irregular LDPC codesThe simple irregular LDPC codes are a special case of theright-regular LDPC codes with J¼ 2 and the followingvariable node degree distribution:

lðxÞ ¼ lI xI�1 þ ldvxdv�1; dv4I ; ldv ¼ 1� lI ð6ÞFor the simple irregular LDPC codes in (6) with rate Rsatisfying (3), we have:

lI ¼I � dv

ðdv� IÞkð1� RÞ �I

ðdv� IÞ ð7Þ

0olIo1) I1� R

okodv

1� R) kmin

¼ I1� R

� �; kmax ¼

dv1� R

� �ð8Þ

2.3 Ensemble A of regular LDPC codes [5]The parity-check matrix of the ensemble A in [5] is chosenwith uniform probability from the m� n (0, 1)-matrixensemble having k ones in each row and c ones in eachcolumn. In fact, this ensemble is a special case of the right-regular LDPC codes ensemble with J¼ 1 and lðxÞ ¼ x‘�1.

2.4 Asymptotic distance distribution ofensemble AOur procedure for generalising the asymptotic distancedistribution of the regular LDPC codes to the right-regularones is the same as [5]. Therefore, we will try to use the samenotations as in [5] for consistency. When it is possible wemay use a compact form or simpler notation.We willreiterate only essential material from [5]; see [5] for theremainder.

The average ensemble distance distribution in anensemble of a code with length n, defined by binarymatrices of size m� n, is denoted by B¼ {Bw7w¼ 0,1,y,n},where Bw is the ratio of the number of all codewords withHamming-weight w to the number of all codewords in theensemble. The asymptotic (average) distance distributionwith w¼ ny is defined as

by :¼ limn!1

1

nlnBny ð9Þ

Theorem 1 (theorem 1 of [5]): Let a:¼m/n, aA(0, 1). ForyA(0, 1) the asymptotic distance distribution in ensemble A

is by ¼ Hby þ pa

y with:

Hby ¼ HðyÞ ð10Þ

pay ¼

a ln ð1þtÞkþð1�tÞk2tyk

� �ify 2 0; 2½k=2�k

� ��akHðyÞ

�1 otherwise

8><>: ð11Þ

where [x] is the greatest integer part of x, H(y)¼�y1ny�(1�y)1n(1�y) is the natural entropy, and tis the only positive root of

ð1þ tÞk�1 þ ð1� tÞk�1

ð1þ tÞk þ ð1� tÞk¼ 1� y ð12Þ

Let K¼ (k1,k2,y,km), L¼ (c1,c2,ycn), and LK;Lm;n be the

ensemble of rectangular m� n matrices mrn, with rowsums ki, i¼ 1, 2,y,m, and column sums cj, j¼ 1, 2,y,n.

Then the generalisation of the results by O’Neil ([9] andtheorem 2 of [5]) can be stated as (theorem 1 of [5]) follows.Theorem 2: (theorem 3 of [5]) Let m-N.

max max ki;1�i�m

max ‘j1�j�n

� � ðln nÞ

14�e; e40 ð13Þ

Then for d 40

LK;Lm;n

��� ��� ¼Pm

i¼1 ki �

!

�mi¼1ki! �n

j¼1‘j!

� exp�1

2Pm

i¼1 ki �2

Xm

i¼1kiðki � 1Þ

Xn

j¼1‘jð‘j � 1Þ

! !

� 1þ o n�1þd � �

ð14ÞCorollary 1: LetXm

i¼1 ki ¼an; bn �Xm

i¼1 kiðki � 1Þ � cn andXn

j¼1 ‘jð‘j � 1Þ ¼ dn

where a, b, c and d are positive (and bounded) valuesindependent of n. Then for n-N we have

ln LK;Lm;n

��� ��� � ln

Pmi¼1 ki

�!

�mi¼1ki! �n

j¼1‘j!

, LK;Lm;n

��� ���ln�

Pmi¼1 ki

�!

�mi¼1ki!�

nj¼1‘j!

ð15Þ

That is, the expression in (14) is logarithmically equivalentto the first ratio, and the remainder terms in (14) have no

effect on the logarithmic equivalency of LK;Lm;n

��� ���.Proof: Denote the first ratio in (14) by f(n) and theremainder terms by g(n). We have

exp�cd2a2

� 1þ o n�1þd

� �� gðnÞ

� exp�bd2a2

� 1þ o n�1þd

� �ð16Þ

Therefore, limn!1ðln gðnÞÞ=n ¼ 0, which implies our assertion

in (15). &

3 Asymptotic distance distribution of ensembleof right-regular LDPC codes

In this Section we introduce our main result on theasymptotic distance distribution for the ensemble of right-regular LDPC codes, defined in Section 2.1.Proposition 1: Let a:¼m/n, aA(0, 1). For yA(0, 1) theasymptotic (average) distance distribution in the ensembleof the right-regular LDPC codes defined in (4), (5) is

by ¼ maxG

Hby þ qa

y

n oð17Þ

if Y 2 0; 2½k=2�=kð Þ, otherwise by ¼ �1, where

Y :¼XJ

j¼1 ygjbjl‘j ; qay ¼ pa

Y ð18Þ

in which paY is defined in (11) and t¼ t(Y) is defined in (12).

H ay is defined as

H ay ¼

XJ

j¼1

1

bjHðygjbjÞ ð19Þ

158 IEE Proc.-Commun., Vol. 152, No. 2, April 2005

The set G is defined as

G ¼ fg1; . . . ; gJ j0 � gj � minf1; 1=ðybjÞg;XJ

j¼1 gj ¼ 1g:

ð20ÞProof: We sketch the proof similar to the approach of [5].For an ensemble of right-regular LDPC codes, a set ofconditions is imposed on the row and column weights of theparity-check matrix. Corresponding to each codeword ofweight w there exist w columns in parity-check matrix withzero modulo two addition. Therefore, to count the numberof codewords with weight w, we should find all the possibleways of selecting w columns of the parity-check matrix, andthen find all the possible ways in which the modulo twoaddition of these columns vanishes. This strategy will beaccomplished in the following.

Hby in theorem 1 of [5] results from the selection of w¼ ny

columns from the set of all n columns of the parity-checkmatrix, i.e.

Hby ¼ lim

n!1

1

nln

nny

� ¼ HðyÞ ð21Þ

while for our right-regular ensemble we should select wjZ0columns from the set of nj columns with column weight cj.This selection is done conditioned on

XJ

j¼1wj ¼ w; 0 � wj � minfw; njg;

XJ

j¼1nj ¼ n ð22Þ

In another words, we should select wj, j¼ 1, 2,y,J, fromthe set W, which is defined as

W ¼ fw1; . . . ;wJ j0 � wj �minfw; njg;Pj

j¼1 wj ¼ wg, or

the equivalent set

G ¼ fg1; . . . ; gJ j0 � gj � minf1; 1=ðybjÞg;Xj

j¼1 gj ¼ 1g

ð23Þwhere gj:¼wj/w. The number of ways for such a selection is

S ¼ Sðw1;w2; . . . ;wJ Þ ¼#Jj¼1

nj

wj

� ð24Þ

Following the approach of [5], we study the case of even kand odd k, separately. First, we consider the case of even k.

We partition the matrix A into two sub-matrices Aleftm�w and

Arightm�ðn�wÞ, where the left matrix is devoted to the w selected

columns as in (22). Note that the modulo two addition ofthe first w columns is vanished to produce a codeword ofweight w. The number of rows with even row-sum (2j) inthe left sub-matrix is denoted by m2j. Therefore,

m0 þ m2 þ � � � þ mk ¼ m ¼ an ð25ÞCounting the number of ones in the left and right matricesand taking into account wj ¼ gjw ¼ gjny and ‘j ¼ kabjl‘lj

result in:

2m2 þ 4m4 þ . . .þ kmk

¼XJ

j¼1 wj‘j ¼XJ

j¼1 gjnykabjl‘j ; ¼ nkaYð26Þ

Y :¼XJ

j¼1 ygjbjl‘j ð27Þ

km0 þ ðk � 2Þm2 þ . . .þ 2mk�2 ¼ nkað1�YÞ ð28ÞFor the specific set of (w1,w2,y,wJ)AW in (22), we denotethe number of possible matrices Aleft, Aright and Aconditioned on (25)–(28) by 7L7, 7R7 and 7LS7, respectively.The number of all matrices A in the ensemble is also

denoted by 7L07. Hence, we have

Lsj j ¼X

M

anm0;m2; . . . ;mk

� Lj j Rj j ð29Þ

where M is the set of all solutions m0,m2,y,mk of (25)–(28).Denote the total number of matrices, which take part in theformation of weight w codeword, by 7L7. Therefore, wehave

Lj j ¼X

GS � Lsj j

¼XG

S �X

M

an

m0;m2; . . . ;mk

� Lj j Rj j ð30Þ

Let P¼ 7LS7/7L07; we are going to compute the proportion:

B ¼ Lsj j= L0j j ¼X

GS � P ð31Þ

Using Theorem 2, we can count the number of left and rightmatrices as follows:

Lj j ¼ ðnkaYÞ!‘1!w1 . . . ‘J !wJ 2!m24!m4 . . . k!mk

� gðnÞ ð32Þ

nkaYoXm

j¼1 kjðkj � 1Þok2na ð33Þ

Xw

j¼1‘jð‘j � 1Þ ¼

XJ

j¼1wj‘jð‘j � 1Þ ¼ nka

�XJ

j¼1gjbjl‘jð‘j � 1Þ ð34Þ

Considering (32)–(34), Corollary 1 implies that g(n) has noeffect on the logarithmic equivalency of 7L7 for large n.Similarly, we have

Rj j ¼ ðnkað1�YÞÞ!‘1!ðn1�w1Þ . . . ‘J !ðnJ�wJ Þk!m0ðk � 2Þ!m2 . . . 2!mk�2

� hðnÞ ð35Þ

For the same reason. h(n) has no effect in the logarithmicequivalency of 7R7 for large n. The number of all matrices inthe ensemble is obtained as

L0j j ¼ðnkaÞ!

‘1!n1 . . . ‘J !nJ ðk!Þna

exp �ðk� 1ÞXJ

j¼1 l‘jð‘j � 1Þ=2� �

ð1þ oðn�1þdÞÞ

ð36Þ

This term is logarithmically equivalent to the first ratio.Therefore, P is logarithmically equivalent to

p ln�1

nka

nkaY

� XM

na

m0;m2; . . . ;mk

�

k

2

� m2 k

4

� m4

� � �k

k � 2

� mk�2

ð37Þ

From this point all the procedures are equivalent to [5] withY replaced by y. Similarly, this procedure is completelyapplicable to the case of odd k. Therefore, we can write

by :¼ limn!1

1

nlnB ¼ lim

n!1

1

nlnXG

S � expðnpaYÞ ð38Þ

IEE Proc.-Commun., Vol. 152, No. 2, April 2005 159

Denote the maximum value of the summand over the set Gby Q. Therefore,

limn!1

1

nlnQobyo lim

n!1

1

nlnðð1þ nyÞJ QÞ

¼ limn!1

1

nlnQ ð39Þ

Considering the squeezing (sandwich) theorem [10], weobtain

by ¼ limn!1

1

nlnQ ð40Þ

Since we need the maximum value for large n (n-N), wecompute if for this case. Hence,

by ¼ limn!1

1

nlnQ ¼ max

Gf lim

n!1

1

nln S þ pa

Yg ð41Þ

Consider (21), (24), and let

Hby :¼ lim

n!1

1

nS ¼ lim

n!1

1

n

XJ

j¼1 lnn=bjgjny

�

¼XJ

j¼1 HðbjgjyÞ=bj ð42Þ

Therefore, we should maximise

by ¼ maxGfHb

y þ paYg ð43Þ

This completes the proof. &Corollary 2: Theorem 1 is deduced from (43) with J¼ 1.Proof: For J¼ 1, we have g1 ¼ b1 ¼ l‘1 ¼ 1;Y ¼ y, and(43) is reduced to by ¼ HðyÞ þ pa

y . &Theorem 3: by¼ b1�y when k is even. It means that for evenk, by is symmetric with respect to y¼ 1/2.

Proof: Let F ðy; g1; . . . ; gjÞ :¼ Hby þ pa

Y. We show that for

any set of gj 2 G; j ¼ 1; . . . ; J , there is a unique set of

ggj 2 G; j ¼ 1; . . . ; J , such that F ð1� y; g1; . . . ; gJ Þ ¼F ðy; gg1; . . . ; ggJ Þ. Let

gjbjð1� yÞ ¼ 1� ggjbjy) ggj ¼1

bj� gj

!1

yþ gj ð44Þ

It is evident that if 0rgjrmin{1, 1/(bj(1�y))}, then wewill have 0 � ggj � minf1; 1=ðbjyÞg. ConsideringPJ

j¼1 1=bj ¼PJ

j¼1 gj ¼ 1, and applying them on the above

definition of ggj in (44), we can writePJ

j¼1 ggj ¼ 1. Now

consider Y as Y¼Y (y,g1,y, gJ). Therefore, we can write

Yð1� y; g1; . . . ; gJ Þ

¼XJ

j¼1ð1� yÞgjbjllj ¼

XJ

j¼1ð1� yggjbjÞllj

¼1�Yðy; gg1; . . . ; ggJ Þ

ð45Þ

Considering H(x)¼H(1�x), and (45), we will obtain

F ð1� y; g1; . . . ; gJ Þ ¼ Hby g!ggj

þ pa1�Yðy;gg1;...;ggJ Þ

��� ð46Þ

Consider that pay ¼ pa

1�y when k is the even ([5], theorem4). Therefore, we have F ð1� y; g1; . . . ; gJ Þ ¼F ðy; gg1; . . . ; ggJ Þ. Finally, we can deduce that

b1�y ¼ maxG

F ð1� y; g1; . . . ; gJ Þ

¼ maxG

F ðy; gg1; . . . ; ggJ Þ ¼ by ð47Þ

This completes the proof. &Lemma 1: A suitable set of initial conditions for theoptimisation problem in (43), satisfying the constraints of

the set G in (20), is

gj ¼1

ybbjðexpðm‘jÞ þ 1Þ; j ¼ 1; 2; . . . ; J ð48Þ

Where m is the unique solution of:

hðmÞ :¼XJ

j¼1

1

bbjðexpðm‘jÞ þ 1Þ¼ y ð49Þ

and bbj ¼ maxfbj; 1=yg; j ¼ 1; 2; . . . ; J :Proof: Simplify the condition gjrmin{1, 1/(ybj)}, as

gj � 1=ðybbjÞ, where bbj :¼ maxfbj; 1=yg.Let

ln1

bbjgjy� 1

!¼ m‘j ) gj ¼

1

ybbjðexpðm‘jÞ þ 1Þ

� 1

ybbj

; j ¼ 1; 2; . . . ; J ð50Þ

Now applying the constraintPJ

j¼1 gj ¼ 1 results in (49), by

which we can compute the constant value of m. Since

@h(m)/@mo0, hð�1Þ ¼P

bbj¼bj1=bj þ

Pbbj¼l=y y4y, and

h(+N)¼ 0, the equation in (49) has a unique solution forall yA(0, 1). This completes the proof. &

Remark 1: Partial derivatives of F ðy; g1; . . . gJ Þ :¼ Hby þ pa

Ywith respect to gj for j¼ 1, 2,y, J�1 can be written as

bjy :¼

@F ðy; g1; . . . ; gjÞy@gj

¼ ln1

ygjbj� 1

!� ln

1

ygJbJ� 1

�

� ð‘j � ‘J Þ ln t1

Y� 1

� � ð51Þ

where we considered that gJ ¼ 1�Pj�1

J¼1 gj, and

@paY=@gj ¼ ð@Y=@gjÞ � ð@pa

Y=@YÞ. Also, it is noticeable

that with Zj:¼ 1/(bjgjy)�1Z0 we have

@2Fy@g2j

¼ �1Zjybjg2j

� 1

ZJybJg2Jþ ‘j � ‘J

ka

� 2

� @2pa

Y

@Y2ð52Þ

Since by [5, Theorem 7] and [5, Remark 5], the secondderivative of pa

Y is negative for odd k and Y41/2, we have

@2F =@g2jo0 for this case.

Remark 2: One set of ‘possible’ solutions of the equations,qF/qgj¼ 0 at y¼ 1/2 is gj¼ 1/bj for j¼ 1, 2,y,J.

Proof: Considering gj¼ 1/bjAG, and Y ¼PJ

j¼1 ygjbjlj

¼ y ¼ 1=2, we will have t¼ t(Y)¼ 1, and all equations in(51) vanish. If there is not a global maximum on theboundary, then this set of gjs produces the maximum valueat y¼ 1/2, which coincides with the value of by¼ 0.5 for theregular case (J¼ 1) with the same value of k and a. i.e.H(1/2)�a ln 2¼ (1�a) ln 2¼R ln 2, where R is the coderate. &

Note that the optimisation problem in Proposition 1belongs to the constrained optimisation problems [11, 12],where the constraints are linear in the form of lower andupper bounds, and a nonlinear function should bemaximised.

160 IEE Proc.-Commun., Vol. 152, No. 2, April 2005

4 Numerical results

It is difficult to illustrate all the possible cases in Proposition1. However, we consider a few cases, which could beinteresting.

As is proved in Rashidpour et al. [Note 1], the constant-rate optimal regular LDPC codes on the binary erasurechannel (BEC) have a minimum possible node degree (42).This optimality is based on the asymptotic threshold valuesattained under the iterative decoding techniques, Numericalanalyses (Table 1) show that the theorem [Note 1] is alsotrue for the additive white Gaussian noise (AWGN)channel. Table 1 shows the threshold values on the binaryinput AWGN channel for the low-rate (1/5) to high-rate (4/5) codes. These threshold values are computed by using theGaussian approximation [13] of the density evolutiontechnique [2]. Distance spectra of the codes in Table 1 areshown in Fig. 1a–d. As is mentioned in [3, 5, 8], we see thatfor all rates there is an effective minimum Hammingdistance which grows linearly with the block length if thevariable node degree is 42. However, when the variablenode degree is two (as shown in Fig. 1c, d), the minimumHamming distance tends to zero for large lengths. TheseFigures also show that increasing the node degrees for aconstant rate reduces the number of codewords with lowHamming weights. This improves the error floor of the codeensemble, as illustrated in [14].

In view of the distance spectrum, we see that for aconstant rate the larger the degree of the variable (check)nodes the better the distance spectrum attained. This makesthe ensemble of codes approach the Shannon limit undermaximum likelihood (ML) decoding. In view of theasymptotic threshold values, the smaller the degree ofvariable (check) nodes, the better the threshold valuesattained under iterative decoding. The newest quantitativeresults in [15] states that: ‘in order to approach the channelcapacity with vanishing bit error probability, LDPC codesshould not have too sparse parity-check matrices, asotherwise their inherent gap to capacity becomes large’. In[15] the quantitative value of sparseness is stated asD¼ k(1�R)/R, in which R is the code rate and k is theaverage check node degree. It shows that to approach thechannel capacity a large degree for the variable (check)nodes is required. It implies that the gap between theperformance under ML and iterative decoding of LDPCcodes can become very large with increasing node degrees.

In the following we consider two groups of ‘comparable’simple irregular and regular LDPC codes, and comparetheir asymptotic distance distributions. In both groups, weconsider constant rate codes with the same check nodedegrees for both the regular and the simple irregular LDPCcodes. Denoting the rate and the constant check nodedegree by R and K, respectively, the variable node degree ofthe regular LDPC code will be (1�R)K. As we see inFig. 1a–d, the spectrum variations for low rate ensemblesare more considerable than those of the high rate ones.Therefore, we continue our discussions based on the low-rate (1/8) or the medium-rate (1/2) codes.

Note that there are some situations in which we cannotfind a comparable regular LDPC code. For example,consider the parameters reported in Table 2, and thecorresponding distance spectrum in Fig. 1e, f. The groupof codes in Fig. 1e is used in [16–18] to increase the capacityof CDMA systems. For the codes of Fig. 1e, since the check

node degree and rate are constant, the sparseness factor D isconstant, and according to [15] we do not expect that thesecodes approach the Shannon limit. Another importantobservation is about the minimum Hamming distance,which can grow linearly with the block length if themaximum variable node degree is kept below 4, i.e. dvo4.This Figure also explains the difficulties in the control of theerror-floor [18] when dvZ4. However, for dvZ4 thespectrum is changed in such a way that the occurrence ofall possible weights in the ensemble is nearly equalised. Inother words, the nearest-neighbour multiplicity is reducedand consequently better performance is expected at lowersignal-to-noise ratios (SNRs), as observed in [18]. Figure 1falso shows similar behaviour when K is even (K¼ 4). Let uscompute the value of l�2GA ¼ expð1=2s2GAÞ=ðK � 1Þ [13] to

Table 1: Asymptotic threshold values with Gaussianapproximation from low-rate to high-rate for Fig. 1a–d

L Fig. 1a,R¼1/5

Fig. 1b,R¼ 2/5

Fig. 1c,R¼ 3/5

Fig. 1d,R¼4/5

(4L,5L) (3L, 5L) (2L, 5L) (L, 5L)

sGA sGA sGA sGA

1 1.1764 1.0002 0.6182 F

2 0.8869 0.8279 0.7439 0.4918

3 0.7840 0.7436 0.6904 0.5850

4 0.7285 0.6949 0.6529 0.5789

5 0.6859 0.6627 0.6262 0.5664

6 0.6531 0.6395 0.6059 0.5541

7 0.6281 0.6166 0.5902 0.5434

Table 2: Asymptotic threshold values with Gaussianapproximation for rate 1/8 codes in Fig. 1e, f

Fig. 1e, R¼1/8, I¼ 2, c¼3dv, sGA, (l2, l

�2GA)

Fig. 1f, R¼1/8, I¼ 2, c¼ 4dv, sGA, (l2, l

�2GA)

3,1.7932, (0.2857,0.5841) 4,1.4568, (0.1429,0.4219)

4,1.9322, (0.5238,0.5717) 5,1.5498, (0.2857,0.4105)

5,1.6701, (0.6032,0.5982) 6,1.6385, (0.3571,0.4016)

6,1.4451, (0.6429,0.6353) 7,1.6963, (0.4000,0.3966)

7,1.3514, (0.6667,0.6575) 8,1.4475, (0.4286,0.4232)

8,1.2998, (0.6825,0.6722) 9,1.3303, (0.4490,0.4422)

9,1.2670, (0.6939,0.6827) 10,1.2619, (0.4643,0.4563)

Table 3: Parameters of ensembles of codes in Fig. 4

Fig. 4: J¼3 R¼ 1/8 c, sGA ðl‘1 ; l‘2 ; l‘3 Þ

4, 1.7588 (l2¼ 0.3750, l5¼ 0.3795m l11¼ 0.2455)

8, 1.0549 (l2¼ 0.0970, l5¼ 0.4060, l10¼0.86240)

8, 1.2061 (l2¼ 0.1400, l5¼ 0.1990, l20¼0.6610)

8, 1.0345 (l3¼ 0.1570, l7¼ 0.1452, l10¼0.6978)

8, 1.0576 (l3¼ 0.1510, l7¼ 0.4715, l15¼0.3775)

10, 0.9906 (l3¼0.1460, l9¼ 0.3535, l19¼ 0.5005)

24, 0.7133 (l11¼ 0.2380, l21¼ 0.3481, l44¼0.4139)

24, 0.7207 (l11¼ 0.3300, l30¼ 0.4073, l65¼0.2627)

Note 1: Rashidpour M., Shokrollahi A., and Jamali S.H.: ‘Optimal regularLDPC codes for the binary erasure channel’, IEEE Commun. Lett., Submittedfor publication, 2004

IEE Proc.-Commun., Vol. 152, No. 2, April 2005 161

check the stability condition of these LDPC codes. Thesevalues are reported in Table 2. As we know, the stabilitycondition limits the value of l2 ðl2ol�2Þ to guaranteesuccessful decoding. On the other hand, the value of l2 in(7) is an increasing function of dv. Therefore, we expect thatthe stability condition will be violated beyond a criticalvalue of dv (see Table 2). This is completely consistent withour observation about the minimum Hamming distance onthe distance spectrum. This consistency is seen for the othersimple irregular codes in Figs. 2 and 3.

Figure 2 shows the ‘variable dv-curves’ of the simpleirregular LDPC codes and a comparable regular LDPCcode. As is shown in [18] the simple irregular LDPC codeswith I¼ 2 have a different behaviour compared to the caseI42.This different behaviour is seen in the distancespectrum again. For I¼ 2 (Fig. 2a, b, e), the minimumHamming distance can linearly increase with the codelength provided that the value of dv is kept below a criticalvalue. After this critical value, the minimum Hammingdistance tends to zero for large block lengths, while the

R = 1/5, (4L,5L)-regular

0 0.2

L1L2L3L4L5

0.4 0.6 0.8 1.0−0.6

−0.5

−0.4

−0.3

−0.2

−0.1

0.1

0

−0.4

−0.3

−0.2

−0.1

0.1

0.2

0

a

0 0.2 0.4 0.6 0.8 1.0

b

0 0.2 0.4 0.6 0.8 1.0

−0.20

0.30

0.40

−0.10

0.10

0.20

0

−0.7

−0.6

−0.5

−0.4

−0.2

−0.1

0.3

0

c

0 0.2

0.2

0.1

0.3

0.4

0.5

0.4 0.6 0.8 1.0

d

0 0.2 0.4 0.6 0.8 1.0

e

0 0.2 0.4 0.6 0.8 1.0

−0.04

−0.02

0

0.02

0.04

0.06

I = 2,K = 4I = 2,K = 3 0.08

f

R = 2/5, (3L,5L)-regular

L1L2L3L4L5

R = 1/5, (L,5L)-regularR = 3/5, (2L,5L)-regular

L1

R =1/8, simple irregular

R = 1/8, simple irregular

dv4

dv3

L5L4L3L2

L5L4L3L2

dv8dv7dv6dv5

(7,8)dv7dv6dv5dv4

b �

�

b �

�

b �

�

b �

�

b �

�

b �

�

Fig. 1 Distance spectraa–d Distance spectrum at constant-rate R of (dv,dc)-regular LDPC codes with variable (check) node degree dv(dc). In each constant-rate Figure thevariations of the node degrees are illustratede Distance spectrum of simple irregular LDPC codes of rate 1/8 used in CDMA applications, with K¼ 3f Corresponds to e for K¼ 4 as an instance of even K (see also Tables 1 and 2)

162 IEE Proc.-Commun., Vol. 152, No. 2, April 2005

variations of the spectrum cause the nearest-neighbourmultiplicity to enhance, by which the improvement in theerror performance at low SNRs is expected. However, forthe case I42 (Fig. 2c, d, f), the minimum Hammingdistance can linearly increase with block length and withoutany limitation on dv. Other variations are nearly similar tothe case I¼ 2. Since the rate (R) and the check node degree(K) are constant in these Figures, the sparseness factor isconstant, and these codes cannot approach the Shannonlimit closely, according to [15]. However, it is possible to

find how near we can approach the Shannon limit for aspecific target bit (block) error probability by using theguidance for designing codes of a finite-length in [15,Theorem 2.4].

Figure 3 shows the ‘variable I-curves’ of the simpleirregular LDPC codes and a comparable regular LDPCcode. Again, we see the different behaviour of the case I¼ 2.Also, it seems that for a constant dv, by increasing the valueof I the spectrum approaches those of the regular ones. Infact, if we need a good minimum Hamming distance the

I = 2,R = 1/8

0 0.2

dv 8_20dv 30_100dv 1000_10000K8-regular

dv 8_18dv 30_100dv 1000_10000K8_regular

dv 15_25dv 30_100dv 1000_10000K16-regular

dv 15_20dv 30_100dv 1000_10000K16_regular

dv 9_10000K16_regular

dv 9_10000K16_regular

0.4 0.6 0.8 1.0

−0.15

−0.10

−0.05

0.05

0.1

0.15

0

a

I = 2,R = 1/8

0 0.2 0.4 0.6 0.8 1.0

−0.25

−0.15

−0.10

−0.20

−0.05

0.05

0.15

0.10

0

b

l = 3,R = 1/8

0 0.2 0.4 0.6 0.8 1.0

−0.15

0.10

−0.10

−0.05

0.05

0

c

i = 3,R = 1/8

0 0.2 0.4 0.6 0.8 1.0

−0.15

−0.20

−0.25

−0.10

−0.05

0.10

0.05

0

d

I = 9,R = 1/2

0 0.2 0.4 0.6 0.8 1.0

−0.05

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0

e

I = 3,R = 1/2

0 0.2 0.4 0.6 0.8 1.0

−0.05

0

0.05

0.10

0.15

0.20

0.25

0.30

0.35

f

b �

�

b �

�

b �

�

b �

�

b �

�

b �

�

Fig. 2 Variable-dv curves of simple irregular LDPC codes for rate 1/8 (a–d) and rate 1/2 (e, f)a, b, e Case I¼ 2; c, d, f I¼ 3 as an instance of case I42

IEE Proc.-Commun., Vol. 152, No. 2, April 2005 163

regular LDPC code is a good candidate. However, bychanging the value of I we can gain better nearest-neighbour multiplicity in each group.

Figure 4 shows some examples of the case J¼ 3, with theparameters reported in Table 3.

5 Conclusions

In this paper we have generalised the result of the distancespectrum of the regular LDPC codes in [5] to the mostimportant ensembles of the irregular LDPC codes (right-

regular LDCP codes), where the check node degree isconstant.

Numerical results showed that, for the simple irregularLDPC codes with I¼ 2, the ensembles can have a minimumHamming distance growing linearly with the block length,provided that the value of dv is kept below a critical value.This result is completely consistent with the stabilitycondition of the LDPC codes. Beyond this critical valueof dn the nearest-neighbour multiplicity of the ensembles isimproved, which results in a better error performance at lowSNRs.

dv = 8,R =1/8

0 0.2

I2I3I4I5I6K8

0.4 0.6 0.8 1.0

−0.15

−0.10

−0.05

0.05

0

a

dv = 22,R = 1/8

0 0.2

I2I3I4I5I6K8

0.4 0.6 0.8 1.0

−0.15

−0.10

−0.05

0.05

0

b

dv = 8,R =1/8

0 0.2

I2I4

I8I6

K16

0.4 0.6 0.8 1.0

−0.15

−0.20

−0.25

−0.10

−0.05

0.05

0

c

dv = 22,R = 1/8

0 0.2

I2I3I4I5I6I7I8K16

0.4 0.6 0.8 1.0

−0.15

−0.20

−0.25

−0.10

−0.05

0.05

0

d

dv = 9,R =1/2

0 0.2

I2I4I6K16

0.4 0.6 0.8 1.0

−0.05

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0

e

dv = 28,R =1/8

0 0.2

I2I3I4I5I6

K16I7

0.4 0.6 0.8 1.0

−0.05

0

0.05

0.10

0.15

0.20

0.25

0.30

0.35

f

b �

�

b �

�

b �

�

b �

�

b �

�

b �

�

Fig. 3 Variable-I curves of simple irregular LDPC codes for rate 1/8 (a–d) and rate 1/2 (e, f), where Ix denotes I = x

164 IEE Proc.-Commun., Vol. 152, No. 2, April 2005

Combining the guidance for designing of codes with afinite-length in [15] and investigating their distance spectracould be a good approach for selecting a good code from anensemble of right-regular LDPC codes for finite-lengthapplications over a desirable range of SNRs and a giventarget error rate.

6 Acknowledgment

The authors wish to thank Professor Amin Shokrollahi forinforming them about one of the newest papers on the

LDPC codes [15], which helped them to give a betterconclusion and enrich their results and paper.

7 References

1 Luby, M.G., Mitzenmacher, M., Shokrollahi, M.A., and Spielman,D.A.: ‘Improved low-density parity-check codes using irregulargraphs’, IEEE Trans. Inf. Theory, 2001, 47, pp. 585–598

2 Richardson, T.J., Shokrollahi, M.A., and Urbanke, R.L.: ‘Design ofcapacity-approaching irregular low-density parity-check codes’, IEEETrans. Inf. Theory, 2001, 47, pp. 619–637

3 Gallager, R.G.: ‘Low density parity check codes’ Research Mono-graph Series, No. 21’ (MIT Press, Cambridge, MA, USA, 1963)

4 Mackay, D.J.C., andNeal, R.M.: ‘Near Shannon limit performance oflow density parity check codes’, Electron. Lett., 1996, 32, pp. 1645–1646

5 Litsyn, S., and Shevlev, V.: ‘On ensembles of low-density parity-checkcodes: asymptotic distance distributions’, IEEE Trans. Inf. Theory,2002, 48, pp. 887–908

6 Chung, S.-Y.: ‘On the construction of some capacity-approachingcoding schemes’. PhD Thesis, Massachusetts Institute of Technology,Cambridge MA, USA 2000

7 Oswald, P., and Shokrollahi, A.: ‘Capacity-achieving sequences forthe erasure channel’, IEEE Trans. Inf. Theory, 2002, 48, (12), pp.3017–3028

8 Mackay, D.J.C.: ‘Good error-correcting codes based on very sparsematrices’, IEEE Trans. Inf. Theory, 1999, 45, (2), pp. 399–431

9 O’Neil, P.E.: ‘Asymptotics and random matrices with row-sumand column-sum restrictions’, Bull. Am. Math. Soc., 1969, 75, pp.1276–1282

10 ‘Squeezing (sandwich) theorem’: http://mathworld.wolfram.com/Squeezing Theorem.html

11 Zeidler, E., and Boron, L.F.: ‘Nonlinear functional analysis and itsapplications III: Variational methods and optimization’ (SpringerVerlag, New York, USA, 1985)

12 Luenberger, D.G.: ‘Linear and nonlinear programming’ (Addison-Wesley Inc., Reading, MA, USA, 1984, 2nd edn.)

13 Chung, S.-Y., Richardson, R.J., and Urbanke, R.L.: ‘Analysis ofsum-product decoding of low-density parity-check codes usinga Gaussian approximation’, IEEE Trans. Inf. Theory, 2001, 47, pp.657–670

14 Sason, I., and Shamai, S.: ‘Improved upper bounds on the ensembleperformance of ML decoded low density parity check codes’, IEEECommun. Let., 2000, 4, (3), pp. 89–91

15 Sason, I., and Urbanke, R.: ‘Parity-check density versus performanceof binary linear block codes over memoryless symmetric channels’,IEEE Trans. Inf. Theory, 2003, 49, (6), pp. 1611–1635

16 Sorokine, V., Kschishang, F.R., and Pasupathy, S.: ‘Gallager codesfor CDMA applications II. Implementations, complexity, and systemcapacity’, IEEE Trans. Commun., 2000, 48, (11), pp. 1818–1828

17 Haghighat, J., Jamali, S.H., Behroozi, H., and Nasiri-Kenari, M.:‘High-performance LDPC codes for CDMA applications’.Proc. IEEE 4th Int. Workshop on Mobile and Wireless Commu-nications Networks, Stockholm, Sweden, 9–11 September 2002, pp.105–109

18 Rashidpour, M., and Jamali, S.H.: ‘Design and analysis ofsimple irregular LDPC codes with finite-lengths and their appli-cation in CDMA systems’, Wirel. Commun. Mob. Comput., 2004, 4,pp. 1–14

19 Litsyn, S., and Shevlev, V.: ‘Distance distribution in ensembles ofirregular low-density parity-check codes’, IEEE Trans. Inf. Theory,2003, 49, pp. 3140–3159

8 Appendix

In this paper, we have generalised the result of [5] to themost powerful class of irregular LDPC codes with constantcheck node degree. While our paper was under review, amore complete result for the general irregular LDPC codeswas published in [19] by the authors of [5]. However, ourresult is based on an accurate extension of the result in [5],and we have focused on the numerical discussions which issuitable for code design. In the following we show theconsistency of our results in Proposition 1 with the resultspresented in ([19] theorem 1).

Theorem 1 of [19]: Let yA(0, 1) r1, r2,y,rg, s1, s2,y,sh, bepositive integers, g¼m/n. v1; v2; . . . ; vg; Z1; Z2; . . . ; Zh, be

numbers from (0, 1), such thatPg

i¼1 vi ¼ 1;Ph

j¼1 Zj ¼ 1,

and let s ¼Ph

i¼1 Zisi,

~yy ¼ 1� gs

Xg

i¼1 viriri�1ðtÞriðtÞ

;

rjðtÞ ¼ ð1þ tÞj þ ð1� tÞjð53Þ

0 0.2 0.4

b �

0.6�

0.8 1.0−0.35

−0.30

−0.25

−0.20

−0.15

−0.10

−0.05

0.05

0R = 1/8,J = 3

R = 1/8,J = 3

K(24)-L(1l,21,44)K(24)-L(1l,30,65)K(24)-L(21)

K(8)-L(3,7,0)K(8)-L(3,7,5)K(10)-L(3,9,19)K(8)-L(7)

0 0.2 0.4 0.6 0.8 1.0

−0.15

−0.10

−0.05

0.05

0

R = 1/8,J = 3

K(8)-L(2,5,10)K(8)-L(2,5,20)K(4)-L(2,5,1l)K(8)-L(7)

0 0.2 0.4 0.6 0.8 1.0

−0.15

−0.10

−0.05

0.05

0

a

b

c

b �

�

b �

�

Fig. 4 Case J¼ 3 with rate 1/8Code parameters are collected in Table 3, where in this Figure thenotation K(x)�L(a, b, c) means that check node degree is K¼ x andvariable degrees are a, b and c

IEE Proc.-Commun., Vol. 152, No. 2, April 2005 165

Let

xi ¼ xiðt; xÞ ¼Zi

1þ z�six�1; z ¼

~yy

tð1� ~yyÞð54Þ

Let a finite discrete set T¼ {t1,y} be the set of positive firstcomponents of solutions (ti, zi) to the system

Pii¼1 xi ¼ yPhi¼1 xisi ¼ ~yys

�ð55Þ

Then, if

~yy � 1�Pg

i¼1 pðriÞPgi¼1 ri

ð56Þ

where p(a) is the parity funcation, p(a)¼ 0 if a is even, andp(a)¼ 1 otherwise, then

by ¼maxt2T

�sHð~yyÞ þ gXg

i¼1vi ln rri

ðtÞ � g ln 2

(

�s~yy ln t þXh

i¼1ZiH

xi

Zi

� ) ð57Þ

If (56) does not hold, then b(y)¼�N.Our corresponding notation with related notation in [19]

is presented in Table 4. Starting with the terms in (57),considering (11) and (19), we can re-write the argument in

(57) based on our terminology and notation as

�ðkaÞHðYÞ þ arkðtÞ � a ln 2� ðkaÞY ln t

þXJ

j¼1

1

bjHðygjbjÞ ¼ pa

Y þ Hby

which shows consistency of our results with [19].

Table 4: Notation in [19] and corresponding notation usedin this paper

Notation in [19] Corresponding notation

g a

g dc¼ 1

r1,yrg r1¼ k

h J

s1,y,sh c1,y, cJ

v1,y,vg v1¼1

ZjPj Zj ¼ 1

1=bjPj 1=bj ¼ 1

s ¼P

j Zj sj ka ¼P

j ð1=bj Þ‘j~yy Y

xjPj xj ¼ yPj xjsj ¼ ~yys

( ygjPj ðygj Þ ¼ yPj ðygj Þ‘j ¼ Y� ðkaÞ

(

maxti

$ maxxj

maxgj

166 IEE Proc.-Commun., Vol. 152, No. 2, April 2005