Analysis of LDPC Convolutional Codes Derived From LDPC Block Codes_by_Pusane

ANALYSIS OF LDPC CONVOLUTIONAL CODES DERIVED FROM LDPC

BLOCK CODES

A Dissertation

Submitted to the Graduate School

of the University of Notre Dame

in Partial Fulfillment of the Requirements

for the Degree of

Doctor of Philosophy

by

Ali Emre Pusane, B.S., M.S., M.S., M.S.

Daniel J. Costello, Jr., Director

Graduate Program in Electrical Engineering

Notre Dame, Indiana

April 2008

c© Copyright by

Ali Emre Pusane

2008

All Rights Reserved

ANALYSIS OF LDPC CONVOLUTIONAL CODES DERIVED FROM LDPC

BLOCK CODES

Abstract

by

Ali Emre Pusane

LDPC convolutional codes have been shown to be capable of achieving the

same capacity-approaching performance as LDPC block codes with iterative mes-

sage passing decoding. In this dissertation, we present several methods of deriv-

ing families of time-varying and time-invariant LDPC convolutional codes from

LDPC block codes. We demonstrate that the derived LDPC convolutional codes

significantly outperform the underlying LDPC block codes, and we investigate the

reasons for these “convolutional gains”.

It is well known that cycles in the Tanner graph representation of a sparse code

affect the iterative decoding algorithm, with short cycles generally pushing its

performance further away from optimum. Hence it is common practice to design

codes that do not contain short cycles, so as to obtain independent messages in

at least the initial iterations of the decoding process. We show that the derived

LDPC convolutional codes have better graph-cycle properties than their block

code counterparts. In particular, we show that the unwrapping process that is

used to derive LDPC convolutional codes from LDPC block codes can “break”

some cycles of the underlying LDPC block code, but cannot create any shorter

cycles. We prove that any cycle in an LDPC convolutional code always maps to

a cycle of the same or smaller length in the underlying LDPC block code.

Ali Emre Pusane

Minimum distance is an important code design parameter when the channel

quality is high, since codewords that are minimum distance apart are the most

likely to cause errors with ML or near-ML decoding. In this case, the minimum

distance determines the so-called error floor behavior in the performance curve

corresponding to high channel signal-to-noise ratios. Thus studying the minimum

distance properties of a code family gives insight into its error floor behavior.

We use asymptotic methods to calculate a lower bound on the free distance of

several ensembles of asymptotically good LDPC convolutional codes derived from

protograph-based LDPC block codes. Further, we show that the free distance to

constraint length ratio of the LDPC convolutional codes exceeds the minimum

distance to block length ratio of the corresponding LDPC block codes.

Message-passing iterative decoders for LDPC block codes are known to be

subject to decoding failures due to so-called pseudo-codewords. These failures

can cause the large signal-to-noise ratio performance of message-passing iterative

decoding to be worse than that predicted by the maximum-likelihood decoding

union bound. We address the pseudo-codeword problem from the convolutional

code perspective. In particular, we show that the minimum pseudo-weight of

an LDPC convolutional code is at least as large as the minimum pseudo-weight

of an underlying LDPC block code. This result, which parallels a well-known

relationship between the minimum Hamming weight of convolutional codes and

the minimum Hamming weight of their quasi-cyclic counterparts, is due to the fact

that every pseudo-codeword in the LDPC convolutional code induces a pseudo-

codeword in the LDPC block code with pseudo-weight no larger than that of the

convolutional pseudo-codeword. More generally, we demonstrate a difference in

the weight spectra of LDPC block and convolutional codes that leads to improved

Ali Emre Pusane

performance at low-to-moderate signal-to-noise ratios for the convolutional codes,

a conclusion supported by simulation results.

To my wife Ozlem and my parents Ayse and Ozkan Sendir

ii

CONTENTS

FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi

TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

AUTHOR’S NOTE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

CHAPTER 1: INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . 11.1 A Simple Digital Communication System . . . . . . . . . . . . . . 11.2 Channel Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Channel Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.3.1 Block codes . . . . . . . . . . . . . . . . . . . . . . . . . . 51.3.2 Convolutional codes . . . . . . . . . . . . . . . . . . . . . 81.3.3 Low-density parity check block codes . . . . . . . . . . . . 13

1.4 Channel Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . 171.4.1 Decoding of LDPC block codes . . . . . . . . . . . . . . . 18

1.5 Dissertation Outline . . . . . . . . . . . . . . . . . . . . . . . . . 21

CHAPTER 2: LDPC CONVOLUTIONAL CODES . . . . . . . . . . . . . 232.1 An Introduction to LDPC Convolutional Codes . . . . . . . . . . 242.2 Improvements to the Pipeline Decoder . . . . . . . . . . . . . . . 30

2.2.1 A stopping rule for the pipeline decoder . . . . . . . . . . 302.2.2 On-demand variable node activation schedule . . . . . . . 322.2.3 Compact pipeline decoder architecture . . . . . . . . . . . 35

2.3 Implementation Complexity Comparisons with LDPC Block Codes 362.3.1 Computational complexity . . . . . . . . . . . . . . . . . . 362.3.2 Processor (hardware) complexity . . . . . . . . . . . . . . 372.3.3 Storage requirements . . . . . . . . . . . . . . . . . . . . . 382.3.4 Decoding delay . . . . . . . . . . . . . . . . . . . . . . . . 38

iii

CHAPTER 3: DERIVING LDPC CONVOLUTIONAL CODES BY UN-WRAPPING LDPC BLOCK CODES . . . . . . . . . . . . . . . . . . 423.1 Deriving Time-Invariant LDPC Convolutional Codes by Unwrap-

ping QC-LDPC Block Codes . . . . . . . . . . . . . . . . . . . . . 433.2 Deriving Time-Varying LDPC Convolutional Codes by Unwrapping

Randomly Constructed LDPC Block Codes . . . . . . . . . . . . . 463.3 A Unified Approach to Unwrapping . . . . . . . . . . . . . . . . . 47

CHAPTER 4: PERFORMANCE OF DERIVED LDPC CONVOLUTIONALCODES AND THE CONVOLUTIONAL GAIN . . . . . . . . . . . . . 544.1 Unwrapped LDPC Convolutional Code Examples . . . . . . . . . 544.2 Unwrapping with Different Step Sizes . . . . . . . . . . . . . . . . 60

CHAPTER 5: GRAPH STRUCTURE OF DERIVED LDPC CONVOLU-TIONAL CODES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645.1 A Bound on the Girth of Unwrapped LDPC Convolutional Codes 64

CHAPTER 6: MINIMUM DISTANCE GROWTH RATES OF DERIVEDLDPC CONVOLUTIONAL CODE FAMILIES . . . . . . . . . . . . . 686.1 An ensemble of protograph-based LDPC block codes . . . . . . . 686.2 Protograph Weight Enumerators . . . . . . . . . . . . . . . . . . . 726.3 Free Distance Bounds . . . . . . . . . . . . . . . . . . . . . . . . . 73

6.3.1 Tail-biting convolutional codes . . . . . . . . . . . . . . . . 736.3.2 A tail-biting LDPC convolutional code ensemble . . . . . . 756.3.3 A free distance bound . . . . . . . . . . . . . . . . . . . . 766.3.4 The free distance growth rate . . . . . . . . . . . . . . . . 77

6.4 Distance Growth Rate Results . . . . . . . . . . . . . . . . . . . . 78

CHAPTER 7: PSEUDO-CODEWORD STRUCTURE OF DERIVED LDPCCONVOLUTIONAL CODES . . . . . . . . . . . . . . . . . . . . . . . 877.1 Background and Notations . . . . . . . . . . . . . . . . . . . . . . 887.2 Pseudo-Codeword Structures of Time-Invariant LDPC Convolu-

tional Codes Derived From QC-LDPC Block Codes . . . . . . . . 927.2.1 The fundamental cone . . . . . . . . . . . . . . . . . . . . 997.2.2 Minimum pseudo-weights . . . . . . . . . . . . . . . . . . . 111

7.3 Pseudo-Codewords in Time-Varying LDPC Convolutional Codes . 123

CHAPTER 8: CONCLUSIONS AND RECOMMENDATIONS FOR FU-TURE RESEARCH . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

iv

APPENDIX A: GRAPH CYCLE HISTOGRAMS OF LDPC BLOCK ANDCONVOLUTIONAL CODES . . . . . . . . . . . . . . . . . . . . . . . 132

BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

v

FIGURES

1.1 A binary communication system model. . . . . . . . . . . . . . . . 2

1.2 The binary symmetric channel. . . . . . . . . . . . . . . . . . . . 4

1.3 The binary erasure channel. . . . . . . . . . . . . . . . . . . . . . 5

1.4 A rate R = 1/2 convolutional encoder with memory 2. . . . . . . 10

1.5 Bipartite Tanner graph corresponding to the (10,3,6)-regular LDPCblock code of Example 1.3.8. . . . . . . . . . . . . . . . . . . . . . 15

1.6 Cycles of length 4 (solid) and 6 (dashed). . . . . . . . . . . . . . . 16

2.1 Shift-register based encoder. . . . . . . . . . . . . . . . . . . . . . 27

2.2 Tanner graph of an R=1/3 LDPC convolutional code and an illus-tration of pipeline decoding. . . . . . . . . . . . . . . . . . . . . . 29

2.3 Bit error rate performance of on-demand variable node schedulingand the compact decoder. . . . . . . . . . . . . . . . . . . . . . . 34

2.4 Performance comparison of LDPC block and convolutional codes. 40

3.1 Tanner graph of (a) a QC block code of length n = r ·c = 7 ·3 = 21,(b) a QC block code of length n = r ·c = 14 ·3 = 42, (c) the derivedLDPC convolutional code with ms = 2 and νs = (ms+1)·c = 3·3 = 9. 45

3.2 Tanner graph of (a) the original LDPC block code, (b) LDPC blockcode after cutting along the diagonal, (c) after appending the sub-matrices for fast encoding, and (d) the resulting LDPC convolu-tional code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.3 Deriving a time-invariant LDPC convolutional code from a QC-LDPC block code: (a) QC-LDPC code, (b) after reordering of rowsand columns, (c) resulting time-varying LDPC convolutional code,(d) resulting time-invariant LDPC convolutional code. . . . . . . 51

4.1 Performance of three (3,5)-regular QC-LDPC block codes and theirassociated time-invariant and time-varying LDPC convolutional codes. 56

vi

4.2 Performance of a [4608,2304] QC-LDPC block code and the asso-ciated time-invariant and time-varying LDPC convolutional codes. 58

4.3 Performance of a family of randomly constructed protograph-basedLDPC block codes and the associated time-varying LDPC convo-lutional codes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

4.4 Performance of LDPC convolutional codes unwrapped from the[400,162] QC-LDPC block code using different step sizes. . . . . . 61

4.5 The shape of the convolutional code parity-check matrix non-zeroregion for (a) k = r (left), (b) k < r (right). . . . . . . . . . . . . 63

6.1 The copy-and-permute operation for a protograph. . . . . . . . . 70

6.2 Tanner graph representation of a protograph-based LDPC blockcode ensemble. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

6.3 Distance growth rates for Example 6.4.1. . . . . . . . . . . . . . . 80



6.6 Simulation results for Example 6.4.3. . . . . . . . . . . . . . . . . 85

7.1 The Tanner graph of H , 1|i−j| < 2. . . . . . . . . . . . . . . . . 89

7.2 (110 110 110) is a valid codeword in a cubic cover. . . . . . . . . . 90

7.3 Number of iterations needed for a sum-product-algorithm-type iter-ative message-passing decoder to decide for the all-zero codeword.(1000 corresponds to no convergence.) Here the β values 1.00, 2.00,3.00, 4.00, 5.00 correspond to signal-to-noise ratios Eb/N0 of, re-spectively, −2.04 dB, 0.96 dB, 2.73 dB, 3.97 dB, 4.94 dB. . . . . . 94

7.4 Number of iterations needed for a min-sum-algorithm-type iterativemessage-passing decoder to decide for the all-zero codeword. (2000corresponds to no convergence.) We did not observe convergenceto the all-zero codeword for α > 0.50 either for the sum-product-algorithm-type iterative message-passing decoder or for the min-sum-algorithm-type iterative message-passing decoder. . . . . . . 94

7.5 The fundamental polytope for H = [1 1 1]. . . . . . . . . . . . . . 101

7.6 The performance of a rate R = 1/4 (3, 4)-regular LDPC convo-lutional code and three associated (3, 4)-regular QC-LDPC blockcodes. Note that the horizontal axis is Es/N0 and not the morecommon Eb/N0 = (1/R) · Es/N0. . . . . . . . . . . . . . . . . . . 121

vii

TABLES

2.1 COMPUTATIONAL AND MEMORY REQUIREMENTS OF THEPROPOSED DECODERS . . . . . . . . . . . . . . . . . . . . . . 39

7.1 THE PSEUDO-WEIGHTS OF THE PSEUDO-CODEWORDS INEXAMPLE 7.2.9 . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

7.2 THE MINIMUM PSEUDO-WEIGHTS OF THE CODES C(r)QC, GIVEN

BY THE PARITY-CHECK MATRIX H(r)QC(X), for r = 1, 2, 3, 4 . 122

A.1 NORMALIZED GRAPH CYCLE HISTOGRAM FOR LDPC CODESDERIVED FROM THE [155,64] QC CODE . . . . . . . . . . . . 133

A.2 NORMALIZED GRAPH CYCLE HISTOGRAM FOR LDPC CODESDERIVED FROM THE [4608,2304] QC CODE . . . . . . . . . . 134

A.3 NORMALIZED GRAPH CYCLE HISTOGRAM FOR LDPC CODESDERIVED FROM THE RATE R = 1/2 PROTOGRAPH-BASEDLDPC CODE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

viii

ACKNOWLEDGMENTS

When I arrived on the Notre Dame campus six years ago, I had no idea that

the time I would spend here would be the most rewarding years of my life. I

met some of the most amazing people I know on this campus, including my wife

Ozlem, and this dissertation would not have been possible without their help and

support.

First of all, I would like to express my deepest gratitude to my advisor Dr.

Daniel J. Costello, Jr. for his guidance during my research and study. Throughout

my doctoral work he encouraged me to develop independent thinking and research

skills. He continually stimulated my analytical thinking and greatly assisted me

with scientific writing. He was always accessible and willing to help his students

with their research. I am very fortunate to have him as my advisor.

When Dr. Kamil Sh. Zigangirov visited the Coding Research Group in the

Spring of 2003, I was the newest member in the group. This gave me a unique

opportunity to start working on several problems with Dr. Zigangirov. His endless

string of ideas and our long discussions were very helpful in shaping my research

as well as developing my research techniques. I would like to thank Dr. Thomas

E. Fuja for his help and support throughout the years and for being part of my

committee. I would also like to thank Dr. J. Nicholas Laneman and Dr. Robert L.

Stevenson for serving on my committee. Alongside my doctoral work, I received

an M.S. in applied mathematics under the supervision of Dr. Joachim Rosenthal

ix

in the Department of Mathematics. Although, we did not have much chance to

collaborate more due to geographical distance, I enjoyed working with him and I

thank him for his help and support.

My dissertation is a compilation of results from several collaborations I made

with leading researchers in the field. I have very much enjoyed working with

Dr. Stephen Bates, Zhengang Chen, Dr. Dariush Divsalar, Dr. Norbert Goertz,

Logan Gunthorpe, Dr. Alberto Jimenez-Feltstrom, Dr. Christopher R. Jones, Dr.

Michael Lentmaier, David G. M. Mitchell, Dr. Roxana Smarandache, Dr. Arvind

Sridharan, Dr. Pascal O. Vontobel, Dr. Dmitri K. Zigangirov on several projects.

I would like to acknowledge and thank them for their contribution to my research

during my doctoral studies. Dr. Michael Lentmaier, Dr. Roxana Smarandache,

and Dr. Pascal O. Vontobel deserve special thanks for their contribution to my

dissertation.

I wish to thank my friends and colleagues at Notre Dame; Ajit, Arvind, Ching,

Christian, Deepak, Junying, Marcin, Ralf, Wei, Xiaowei and many others. I also

thank David Mitchell for being a great research partner. Mehtap and Demirhan

Tunc have been our best friends since they arrived at Notre Dame three years

ago. I would like to thank them for their continuous support. Especially since the

arrival of their son Aslan Tunc in August 2007, Ozlem and I have spent pretty

much all our free time trying to make Aslan smile.

Prior to coming to Notre Dame, I received an M.S. degree from Istanbul Tech-

nical University under the supervision of Dr. Umit Aygolu. My heartfelt thanks

go to Dr. Aygolu whose guidance and support prepared me for my doctoral work.

I would also like to thank Dr. Erdal Panayırcı for his help and support in the

same period.

x

Finally, I would like to thank my wife Ozlem. Her love, encouragement and

support have inspired me and given me strength since the day we met. I thank

my parents Ayse and Ozkan Sendir. They have always supported and encouraged

me to do my best in all matters of life. Ozlem’s parents Ulker and Cevdet Kayhan

have always been there for us when we needed their support. This dissertation is

dedicated to Ozlem and our parents.

xi

AUTHOR’S NOTE

This dissertation includes results from several collaborations with leading re-

searchers in the field from across the globe. I would like to acknowledge and thank

my collaborators for their contribution to my research during my Ph.D. studies.

On this note, I will list our collaborations and acknowledge my co-authors on the

published papers.

Chapter 2 of this dissertation deals with the background for LDPC convolu-

tional codes and it is an essential part of the dissertation leading into the work

presented in the following chapters. Section 2.2 deals with several improvements

we have proposed for the pipeline decoder that is used to decode LDPC convolu-

tional codes. This has been joint work with Dr. Michael Lentmaier, Dr. Kamil Sh.

Zigangirov, and my dissertation advisor, Dr. Daniel J. Costello, Jr.. Our results

were published in [1]. This work was later extended to include block transmission

of LDPC convolutional codes and comparisons between LDPC block and convo-

lutional codes. These extensions were aided by the contributions of Dr. Alberto

Jimenez-Feltstrom and Dr. Arvind Sridharan and resulted in [2]. The LDPC

block vs. LDPC convolutional code comparisons were further developed with the

help of Dr. Stephen Bates, Dr. Christopher R. Jones, and Dr. Dariush Divsalar

and resulted in [3, 4].

Chapters 3, 4, and 5 constitute the heart of this dissertation and were published

in [5]. Dr. Roxana Smarandache, Dr. Pascal O. Vontobel, and Dr. Daniel J.

xii

Costello, Jr.’s contributions are greatly appreciated. Our collaboration also lead

to the pseudo-codeword analysis presented in Chapter 7, which was published in

[6, 7].

The distance growth rate analysis presented in Chapter 6 is a collaboration

with David G. M. Mitchell, Dr. Kamil Sh. Zigangirov, and Dr. Daniel J. Costello,

Jr.. Our results have been submitted for publication in [8].

There are a couple of projects I have participated in that are not included

in this dissertation. I would also like to acknowledge my collaborators on these

projects.

• A code construction algorithm for irregular LDPC convolutional codes was

presented in [9]. This was joint work with Dr. Kamil Sh. Zigangirov and

Dr. Daniel J. Costello, Jr..

• A bandwidth efficient LDPC convolutional code construction technique was

proposed in [10]. This was joint work with Dr. Michael Lentmaier, Dr.

Thomas E. Fuja, Dr. Kamil Sh. Zigangirov, and Dr. Daniel J. Costello, Jr..

• Serial decoding architectures were considered for the decoding of LDPC

convolutional codes in [11–13]. This was joint work with Dr. Stephen Bates,

Logan Gunthorpe, Zhengang Chen, Dr. Kamil Sh. Zigangirov, and Dr.

Daniel J. Costello, Jr..

• Finally, an analysis of the error correcting capability of LDPC block codes

was presented in [14]. This was joint work with Dr. Dmitri K. Zigangirov,

Dr. Kamil Sh. Zigangirov, and Dr. Daniel J. Costello, Jr..

xiii

CHAPTER 1

INTRODUCTION

1.1 A Simple Digital Communication System

A digital communication system block diagram is given in Figure 1.1. A binary

information source produces the data to be transmitted. The original message can

be digital, as in computers, or digitized versions of analog signals, as in voice com-

munications. In either case, a source encoder is employed in order to eliminate the

redundancy in the digital data. This operation is generally known as compression

and it increases the throughput of the communication system by saving valuable

signaling time. The compressed data is expected to be restored by the source

decoder at the receiver side. Ideally, it is desirable for a source decoder to decode

the compressed data perfectly (lossless data compression); however, some sacrifice

in performance is sometimes made to increase the compression ratio (lossy data

compression). An ideal source encoder minimizes the number of symbols needed

to represent the data, such that the symbols in the information sequence at the

output of the source encoder are independent and equally likely. The channel

encoder takes fixed length blocks u = [u1, u2, . . . , uk] of the information sequence

and adds redundancy to form a codeword v = [v1, v2, . . . , vn]. The ratio R = k/n,

k < n, is called the code rate. The channel decoder utilizes this redundancy in the

noisy received block r = [r1, r2, . . . , rn] to try and recover the original information

1

block u. Finally, the modulator uses the codeword v to generate a waveform x(t)

that is suitable for transmission over the physical transmission channel. The noisy

received waveform r(t) is demodulated to arrive at the received block r, which is

decoded to produce an estimate u = [u1, u2, . . . , uk] of the original information

block. While the information block u and the codeword v are assumed to be bi-

nary, depending on the channel model, the received block r may be unquantized.

SourceEncodermessage Channel

Encoder Modulator

Channel

DemodulatorSourceDecoder

ChannelDecoder

messageestimate

u

u v

r

x(t)

r(t)

Binary Information Source

Binary Information Sink

DiscreteChannel Noise

Transmitter

Receiver

Figure 1.1. A binary communication system model.

One major result of the pioneering paper by Claude Shannon [15] is that, for a

broad class of channel codes, arbitrarily low error probabilities can be achieved so

long as the code rate R is less than the “channel capacity” C, which is determined

by the physical properties of the channel. The converse theorem states that if

R > C, reliable transmission with arbitrarily low error probabilities is not possible.

Another major result of [15] is that one can design the source and channel codes

2

separately and still achieve optimality. Based on this fact, we focus only on the

channel coding part of the communication system in this dissertation.

1.2 Channel Model

Throughout the dissertation, binary phase shift keying (BPSK) modulation is

assumed. In BPSK, a binary symbol vi ∈ 0, 1 is modulated to obtain a BPSK

symbol xi, where xi = 2vi − 1 ∈ +1,−1. xi then modulates a sinusoidal pulse

to produce the signal x(t), which is transmitted over an additive white Gaussian

noise (AWGN) channel. The effect of the AWGN channel on a received symbol ri

takes the form of additive noise, i.e.,

ri = xi + ni, i = 1, 2, . . . , n

where the noise variables ni are independent, identically distributed, Gaussian

random variables with zero mean and variance σ2 = N0/2, and N0 is the single

sided power spectral density of the noise process. The relative strength of a

transmitted channel symbol is given in terms of the bit signal-to-noise (SNR) ratio,

which is defined as Eb/N0, where Eb denotes the average energy per information

symbol.

Given the above definition of SNR, Shannon’s noisy-channel coding theorem

can now be restated in a way that applies to designing a code with a certain target

rate. For a given code rate R, the noisy-channel coding theorem determines a

minimum channel quality SNR∗ such that reliable transmission is possible if and

only if SNR > SNR∗. Code families that achieve reliable communications at SNRs

very close to SNR∗ with asymptotically large block lengths are conveniently named

capacity-approaching codes.

3

The output of the AWGN channel described above is continuous valued, i.e.,

soft demodulation is assumed. However, in some communication scenarios, hard

demodulation may be an option. Hard demodulation quantizes the output of the

channel into a pre-determined set of symbols. The simplest hard demodulation

example is the binary symmetric channel (BSC) model depicted in Figure 1.2.

0

1

0

1

!

!

1! !

1! !

Figure 1.2. The binary symmetric channel.

The BSC is described by a single parameter – its crossover probability ε.

With probability ε, a symbol is inverted during transmission over a BSC. The

hard demodulation employed on a BSC assumes a certain decision boundary to

distinguish between the two possible values. The closer the received symbol is to

this decision boundary, the larger probability a transmission error has occurred.

The binary erasure channel (BEC) model utilizes this approach to arrive at the

transmission diagram shown in Figure 1.3. Each symbol is assumed to be erased

with probability δ. The biggest difference from the BSC model is that the received

symbol cannot be erroneous, i.e., a symbol transmitted over a BEC is either

received correctly or an erasure occurs.

4

0

1

0

1

!!

!

1! !

1! !

Figure 1.3. The binary erasure channel.

1.3 Channel Encoding

Symbols of a sequence transmitted over a channel are prone to random channel

errors. Channel codes are employed to introduce redundancy to the information

sequence and to use this redundant information to recover the information se-

quence from a received noisy sequence. Channel codes can be categorized into two

broad classes, namely block codes and convolutional codes. Block codes operate

on blocks of the information sequence and encode/decode these blocks indepen-

dently, whereas convolutional codes work continuously at the symbol level. We

will review these two approaches in this section.

1.3.1 Block codes

A linear block code C of rate R = k/n takes a block of information symbols

u = [u1, u2, . . . , uk] and produces a codeword v = [v1, v2, . . . , vn]. This dissertation

focuses only on linear block codes defined over the binary field F2. In this case,

there is a one-to-one correspondence between the 2k possible distinct information

blocks and 2k codewords of length n. The linearity condition yields a compact

description of the block code as a k-dimensional subspace of the vector space Fn2 .

5

Using this description, it is possible to represent each codeword v corresponding to

an information block u as a linear combination of k linearly independent codewords

v =k∑i=1

uigi, (1.1)

where gi = [gi,1, gi,2, . . . , gi,n], 1 ≤ i ≤ k, and modulo-2 addition is assumed. These

k linearly independent codewords can therefore be arranged to form a generator

matrix G for the block code C, where G is given as

G =

g1

g2

...

gk

=

g1,1 g1,2 · · · g1,n

g2,1 g2,2 · · · g2,n

......

. . ....

gk,1 gk,2 · · · gk,n

. (1.2)

A block code can be defined in terms of its generator matrix as

v1×n = u1×kGk×n. (1.3)

Different sets of linearly independent codewords lead to the same block code.

However, the mapping of information blocks to codewords is changed in this case.

Also, a block code is called systematic if the codeword symbols can be arranged

in such a way that the first k symbols correspond to the information block u.

Example 1.3.1 Consider a rate R = 4/7 block code of length n = 7. This code

maps information blocks of length k = 4 to codewords of length n = 7 using a

6

generator matrix G given as

G =

1 0 0 0 1 0 1

0 1 0 0 1 1 1

0 0 1 0 1 1 0

0 0 0 1 0 1 1

4×7

. (1.4)

The information block u = [0, 1, 1, 0] can be encoded to produce

v = uG =

(0 1 1 0

)

1 0 0 0 1 0 1

0 1 0 0 1 1 1

0 0 1 0 1 1 0

0 0 0 1 0 1 1

(1.5)

=

(0 1 1 0 0 0 1

). (1.6)

The 4× 4 identity matrix that constitutes the first 4 columns of G results in a

systematic block code.

The Hamming distance between two codewords v1 and v2 of C is the number

of positions in which they differ, which equals the Hamming weight1 of the mod-2

sum of v1 and v2, i.e., v1 ⊕ v2. The minimum distance dmin of C is the minimum

Hamming distance between any two codewords of C. Since, the linearity of the

code guarantees that the summation of any two codewords is still a codeword in

the code, the minimum distance dmin equals the minimum weight of any non-zero

codeword in C.

A linear block code can also be described by its parity-check matrix. A parity-

1The Hamming weight of a vector is the number of non-zero components.

7

check matrix H is the null space of the generator matrix G, i.e.,

GHT = 0. (1.7)

Therefore, for any codeword v in C,

vHT = 0, (1.8)

where HT (also called the syndrome former matrix) is the transposed parity-check

matrix. For a rate R = k/n linear block code of length n, the parity-check matrix

is an (n− k)× n matrix.

Example 1.3.2 A parity-check matrix for the linear block code given in Example

1.3.1 is given by

H =

1 1 1 0 1 0 0

0 1 1 1 0 1 0

1 1 0 1 0 0 1

3×7

. (1.9)

Similar to the case for the generator matrix, several parity-check matrices for

the same code C can be formed by taking different sets of n − k basis vectors of

the null space of G.

1.3.2 Convolutional codes

Convolutional coding takes a very different approach to encoding informa-

tion sequences. Rather than taking large blocks of the information sequence and

encoding/decoding them separately, convolutional codes continuously encode an

8

information sequence to a code sequence. We use a different notation for con-

volutional codes. The number of information symbols that enter a convolutional

encoder per unit time is denoted by b, while the number of code symbols at the

output is denoted by c. Thus, the code rate of a convolutional code is given as

R = b/c. At each time instant, in order to produce a c-tuple, a convolutional

encoder processes b incoming information symbols as well as several information

symbols that were encoded and stored in previous time instants. The encoder

memory m determines how long previous information symbols can be stored and

used in the encoder. We now introduce a simple example to demonstrate the

encoding process.

Example 1.3.3 Consider a rate R = b/c = 1/2 convolutional code with memory

m = 2. The block diagram for the convolutional encoder is given in Figure 1.4,

where the squares represent the delay elements. At each time instant t, the convo-

lutional encoder takes b = 1 information symbol ut from the information sequence

u and produces c = 2 code symbols v1t and v2

t to form the code sequence v. The

input-output relation described in Figure 1.4 is given as

v1t = ut, (1.10)

v2t = ut + ut−2. (1.11)

The convolutional encoder in Example 1.3.3 is said to be time-invariant, since

the input-output relations given in (1.10) and (1.11) do not change with time.

In the following chapters, we will introduce and analyze both time-invariant and

time-varying convolutional codes.

9

ut

v1t

v2t

Figure 1.4. A rate R = 1/2 convolutional encoder with memory 2.

Similar to the notation of block codes, convolutional codes can also be de-

scribed using generator matrices. In general, a rate R = b/c convolutional encoder

takes an information sequence u in the form

u = [. . . ,u0,u1, . . . ,ut, . . .], (1.12)

where ui = [u1i , u

2i , . . . , u

bi ], for all i ∈ Z. This information sequence is mapped to

a code sequence v given by

v = [. . . ,v0,v1, . . . ,vt, . . .], (1.13)

where vi = [v1i , v

2i , . . . , v

ci ]. For a convolutional encoder without feedback, the

relation between u and v is given by

vt = utG0(t) + ut−1G1(t) + · · ·+ ut−mGm(t). (1.14)

Here, each binary matrix Gi(t), i = 0, 1, . . . ,m, of size b×c determines which of the

b symbols in the corresponding information b-tuple takes part in the computation

10

of each of the c output symbols at time instant t. For time-invariant codes, the

Gi(t) matrices are the same for each time unit and can therefore simply be denoted

as Gi.

The encoding operation given in (1.14) can be rewritten as v = uG using the

generator matrix

G =

. . . . . .

G0(0) · · · Gm(m)

G0(1) · · · Gm(m+ 1)

. . . . . .

G0(t) · · · Gm(m+ t)

. . . . . .

. (1.15)

Example 1.3.4 The generator matrix G corresponding to the convolutional code

of Example 1.3.3 is given as

G =

. . . . . . . . .

11 00 01

11 00 01

11 00 01

. . . . . . . . .

. (1.16)

As seen in (1.16), the generator matrix has a repetitive structure for time-

invariant convolutional codes. A shorthand notation can be defined using the

delay variable D. The information and code sequences can be described in terms

11

of the delay variable as

u(D) = . . .+ u0 + u1D + . . .+ utDt + . . . , (1.17)

v(D) = . . .+ v0 + v1D + . . .+ vtDt + . . . . (1.18)

The encoding operation can then be described as

v(D) = u(D)G(D), (1.19)

where the generator matrix G(D) is given as

G(D) = G0 + G1D + . . .+ GmDm. (1.20)

Similar to the scalar representation given in (1.12)-(1.14), the coefficients ui,

vi, and Gi are of size 1× b, 1× c, and b× c, respectively.

Example 1.3.5 The generator matrix G(D) corresponding to the convolutional

code of Example 1.3.3 is given by

G(D) = [1 1] + [0 0]D + [0 1]D2 = [1 1 +D2]. (1.21)

Each codeword v(D) satisfies

v(D)HT(D) = 0, (1.22)

where HT(D) (also called the syndrome former matrix) is the transposed parity-

check matrix of size c× (c− b).

12

Example 1.3.6 The syndrome former matrix HT(D) corresponding to the gen-

erator matrix G(D) given in Example 1.3.5 is given by

HT(D) = [1 +D2 1]. (1.23)

Finally, similar to the minimum distance dmin of block codes, the free dis-

tance dfree of a convolutional code is defined as the minimum Hamming distance

between any two finite-length code sequences. If one of the two code sequences

is shorter than the other, zeros can be added to the shorter sequence so that

the corresponding codewords have equal lengths. Since, the linearity of the code

guarantees that the summation of any two code sequences is still a valid code

sequence, the free distance dfree equals the minimum weight of any finite-length

non-zero code sequence.

1.3.3 Low-density parity check block codes

In the last fifteen years, the area of channel coding has been revolutionized

by the practical realization of capacity-approaching codes, initiated by the in-

vention of turbo codes in 1993 [16]. A few years after the invention of turbo

codes, researchers became aware that Gallager’s low-density parity-check (LDPC)

block codes, first introduced in [17], were also capable of capacity-approaching

performance. We now give a formal definition of LDPC block codes.

Definition 1.3.7 An LDPC block code is a linear block code whose parity-check

matrix H has a relatively low number of ones in its rows and columns. In partic-

ular, an (n, J,K)-regular LDPC block code is a linear block code of length n and

13

rate R ≥ 1−J/K whose parity-check matrix H has exactly J ones in each column

and K ones in each row, where J,K << n.

Although the term “relatively low” is imprecise, we generally expect to keep

the density of ones in each row and column low as we increase the block length

of an LDPC block code. This results in a very sparse parity-check matrix which

allows very low complexity decoding, as will be discussed later in this section. It

should be noted that, while the majority of block codes are defined in terms of

their generator matrices, as a result of their unique sparsity property, LDPC block

codes are generally defined in terms of their parity-check matrices. In some cases,

it is even possible to avoid the generator matrix altogether and use the parity-

check matrix for both encoding and decoding, as will be seen in the following

chapter. First, we give a simple example of LDPC block codes from [18].

Example 1.3.8 Consider a randomly constructed rate R = 1/2, (10,3,6)-regular

LDPC block code. The corresponding parity-check matrix is given as

H =

1 0 1 1 1 0 1 1 0 0

0 1 1 0 0 1 1 0 1 1

1 1 0 1 1 1 0 1 0 0

0 1 1 0 1 0 1 0 1 1

1 0 0 1 0 1 0 1 1 1

.

The parity-check matrix H consists of 10 columns and 5 rows corresponding to

the 10 code symbols participating in 5 parity-check equations. Each code symbol is

included in J = 3 parity-check equations (each column has weight J = 3) and each

parity-check equation contains K = 6 code symbols (each row has weight K = 6).

14

We can associate a bipartite graph, called a Tanner graph [19], with the parity-

check matrix of an LDPC block code.

Example 1.3.9 Consider the rate R = 1/2, (10,3,6)-regular LDPC block code

whose parity-check matrix H is given in Example 1.3.8. The associated Tanner

graph has n = 10 variable nodes and 5 check nodes representing the columns and

rows of H, respectively. The Tanner graph for this example is given in Figure

1.5. Each variable node is connected to the J = 3 check nodes with “1”s in its

corresponding column, and each check node is connected to the K = 6 variable

nodes with “1”s in its corresponding row.

variable node check node

Figure 1.5. Bipartite Tanner graph corresponding to the (10,3,6)-regularLDPC block code of Example 1.3.8.

15

A variable node and a check node have a common edge in the Tanner graph if the

associated code symbol takes part in the associated parity-check equation. The

total number of edges in a Tanner graph is equal to the number of ones in the

parity-check matrix. Therefore, the relatively low density of ones in a parity-check

matrix translates into a sparse associated Tanner graph.

We now define two properties of Tanner graphs that will be important in the

following sections.

Definition 1.3.10 A cycle of length 2L in a Tanner graph is a path consisting of

2L edges such that the start node and end node are the same.

Definition 1.3.11 The length of the shortest cycle in a Tanner graph is called

its girth. The girth is always even and greater than 2.

Example 1.3.12 The Tanner graph given in Figure 1.5 has girth 4. Two cycles,

one of length 4 and the other of length 6 are shown in Figure 1.6.

Figure 1.6. Cycles of length 4 (solid) and 6 (dashed).

16

1.4 Channel Decoding

Since there is a one-to-one correspondence between each information block u

and codeword v, a similar correspondence also exists for the message estimate u

and the codeword estimate v produced by the channel decoder. How this codeword

estimate is computed is the subject of this section. We start by defining the goals

of decoding. A channel code is employed to minimize the errors introduced by

the channel. The block error rate (BLER) PB of a channel code is the probability

that the codeword estimate v is different from the transmitted codeword v, i.e.,

PB = Pr(v 6= v) =∑

r

Pr(v 6= v|r)Pr(r), (1.24)

where Pr(r) is the probability of the received sequence r, which is independent

of the decoding algorithm. Minimizing PB is therefore equivalent to minimizing

Pr(v 6= v|r), or maximizing Pr(v = v|r), for each received sequence r. In order

to minimize the block error probability, the decoding algorithm then chooses v

according to

v = arg maxv

Pr(v|r), (1.25)

which is equivalent to choosing the message estimate u according to

u = arg maxu

Pr(u|r). (1.26)

This decision rule is called the maximum a posteriori (MAP) decoding rule. If all

information sequences u are equally likely, the MAP decision rule is equivalent to

u = arg maxu

Pr(r|u), (1.27)

17

which is called the maximum likelihood (ML) decoding rule.

Channel decoding can also be designed to minimize the bit error probability

(BER) Pb. In this case, the MAP decoding rule can be rewritten as

ui = arg maxu∈0,1

Pr(ui = u|r). (1.28)

1.4.1 Decoding of LDPC block codes

The sparse parity-check matrix representation of LDPC block codes leads nat-

urally to a low-complexity iterative decoding algorithm based on message pass-

ing. The most common iterative decoding algorithm is belief-propagation or sum-

product decoding. This algorithm belongs to a class of message-passing iterative

decoding algorithms and can be conveniently described in terms of the Tanner

graph. On the Tanner graph, a message-passing iterative decoding algorithm

refers to nodes exchanging messages along the edges. Each node in the Tanner

graph receives incoming messages along its neighboring edges and computes an

outgoing message for each neighboring edge based on what was received from the

other neighboring edges. The variable nodes are assumed to also have an incoming

message from the channel, which carries a priori information about the value of

the associated codeword symbol.

Depending on the actual message-passing decoding algorithm employed and

on the channel model, various kinds of messages are possible. They can be hard

decision values for the associated code symbols, i.e., 0 or 1 for binary codes, or

soft decision values, i.e., a probability p0 that the corresponding code symbol is a

0 or p1 that the symbol is a 1. The most common message type is the log likelihood

ratio (LLR) of a symbol. This is calculated using the probabilities of the symbol

taking on the two values 0 and 1 and is given by log(p0/p1).

18

In the belief propagation algorithm, an outgoing message from the i-th variable

node to the j-th check node mij is computed using

mij = ci +∑

j′∈N(i)\j

nj′i, j ∈ N(i) (1.29)

where the set N(i) includes the indices of the check nodes that are connected

to the variable node i, i.e., the neighborhood of the variable node i. ci denotes

the a priori channel LLR of the variable node i and the messages nj′i represent

the incoming messages from the check nodes in N(i). The message from the j-th

check node is excluded from the outgoing message to the same check node in order

to reduce the correlation between successive messages.

The computation of outgoing messages for a check node is a little bit more

complicated. An outgoing message from the j-th check node nji is computed

using

nji = log

(1 +

∏i′∈N(j)\i tanh(mi′j)

1−∏i′∈N(j)\i tanh(mi′j)

), i ∈ N(j) (1.30)

where N(j) represents the neighborhood of the j-th check node.

The standard message-passing schedule for LDPC block codes calls for, in each

iteration, all the variable nodes, and then all the check nodes, to be activated.

During a variable node activation, all the incoming messages to a variable node

from the neighboring check nodes are collected and the new outgoing messages

are calculated according to (1.29). Then the new messages are sent to each neigh-

boring check node. Correspondingly, a check node activation is the step where

the check node collects all the incoming messages from its neighboring variable

nodes, calculates the outgoing messages according to (1.30), and sends the new

messages to each neighboring variable node. Since the variable node activations

19

must have access to the check node messages from the previous iteration, at the

first iteration the check-to-variable node messages are assumed to be zero.

Following the initial work by Gallager [17, 18], the analysis and design of LDPC

block codes has attracted considerable recent attention in the literature, begin-

ning with the work of Wiberg [20], MacKay and Neal [21], and many others. The

irregular version of these codes was first introduced by Luby et al. in [22, 23], and

analytical tools were presented in [24, 25] to compute iterative decoding thresh-

olds for LDPC block codes when decoded using the iterative message-passing

algorithms suggested by Tanner [19]. While Shannon’s channel capacity is a func-

tion of the physical properties of the channel only, the iterative decoding threshold

takes into account the structure of a code ensemble and the associated decoding

algorithm as well. The most popular technique to determine the iterative decod-

ing threshold for a given LDPC code ensemble is called density evolution [24].

Density evolution tracks the probability density functions of messages in a Tanner

graph and determines whether the iterative message-passing algorithm converges

at a pre-determined channel SNR. Analogous to the channel capacity, it can be

seen as the ultimate performance limit of an LDPC code ensemble. By definition

the iterative decoding threshold cannot exceed the channel capacity; however,

irregular codes that achieve near capacity performance were designed in [26, 27].

One important characteristic of LDPC code performance is the sudden changes

in the slope of the performance curve at moderate to high SNRs. This undesirable

characteristic, commonly known as an error floor, is due to non-convergence of the

iterative message-passing decoding algorithm. One possible explanation for the

error floor is pseudo-codewords, which are error patterns that cause the decoding

algorithm to converge to a non-codeword.

20

1.5 Dissertation Outline

This dissertation focuses on the analysis of the convolutional counterparts

of LDPC (block) codes. In addition to presenting a unified theory of deriving

LDPC convolutional codes from LDPC block codes, we develop several analytical

techniques to understand the advantages and disadvantages of the constructed

codes. The dissertation is organized as follows.

We begin with a short introduction to LDPC convolutional codes (Chapter 2)

and propose several improvements to the pipeline decoder that is used to decode

these codes. Comparisons of LDPC block and convolutional decoders on the basis

of computational complexity, hardware complexity, memory requirements, and

decoding delay (latency) are also given in this chapter.

We devote Chapter 3 to code construction. We use the term unwrapping to

describe the process of modifying a block code to yield a time-varying or time-

invariant convolutional code, depending on the parity-check matrix of the block

code that is chosen from certain equivalent representations. We then show how

the two construction methods can be unified under the same umbrella.

In Chapter 4, we demonstrate the performance capabilities of unwrapped

LDPC convolutional codes for quasi-cyclic, protograph-based, and randomly con-

structed underlying LDPC block codes. We show, for all practically interesting

block lengths, that the constructed time-invariant and time-varying LDPC convo-

lutional codes typically outperform the underlying LDPC block codes by a large

margin, and we term this performance improvement a “convolutional gain”2.

We devote the rest of the dissertation to an analysis of the derived codes and

we provide several possible explanations for the observed convolutional gain. In

2The author would like to acknowledge Dr. Christopher Jones of the Jet Propulsion Labora-tory for recommending the term “convolutional gain” for the observed performance improvement.

21

addition, in Chapter 5, based on the effect that the graph-cycle structure of a code

has on its iterative decoding performance, we analyze the graph-cycle structure

of the derived time-varying and time-invariant LDPC convolutional codes with

respect to their underlying LDPC block codes. In Chapter 6 we use ensembles of

tail-biting LDPC convolutional codes derived from “protograph-based” ensembles

of LDPC block codes to obtain a lower bound on the free distance of untermi-

nated, asymptotically good, periodically time-varying LDPC convolutional code

ensembles, i.e., ensembles whose average free distance grows linearly with con-

straint length. Finally, in Chapter 7 we present an analytical treatment of the

pseudo-codeword problem for LDPC convolutional codes.

Concluding remarks and some areas for future work are included in Chapter

8.

22

CHAPTER 2

LDPC CONVOLUTIONAL CODES

The convolutional counterparts of LDPC block codes, LDPC convolutional

codes, were first proposed1 by Jimenez-Feltstrom and Zigangirov in [29]. Anal-

ogous to LDPC block codes, LDPC convolutional codes are defined by sparse

parity-check matrices, which allow them to be decoded using iterative message-

passing algorithms. Recent studies have shown that these codes are suitable for

practical implementation in a number of different communication scenarios, in-

cluding continuous transmission and block transmission in frames of arbitrary

size [11, 30, 31].

We begin this chapter with a short introduction to LDPC convolutional codes.

We then give an overview of the encoding and decoding processes and describe

the pipeline decoding architecture that is used to decode these codes. In Section

2.2, we propose several improvements to the pipeline decoder architecture that

lead to savings in decoding delay and decoder storage requirements. Finally, we

present comparisons of LDPC block and convolutional decoders based on several

practical decoding parameters.

1The basic concept of LDPC convolutional codes was independently presented by Tanner ina patent application [28].

23

2.1 An Introduction to LDPC Convolutional Codes

Definition 2.1.1 An LDPC convolutional code is a linear convolutional code

whose parity-check matrix Hconv has a relatively low number of ones in its rows

and columns. In particular, an (ms, J,K)-regular LDPC convolutional code is a

linear convolutional code of syndrome former memory ms and rate R ≥ 1− J/Kwhose parity-check matrix Hconv has exactly J ones in each column and K ones

in each row, where J,K << ms.

An LDPC convolutional code can be described as the set of sequences v sat-

isfying the equation vHTconv = 0, where HT

conv is the (time-varying) semi-infinite

syndrome-former (transposed parity-check) matrix corresponding to the parity-

check matrix

Hconv =

H0(0)

H1(1) H0(1)

......

. . .

Hms (ms) Hms−1(ms) . . . H0(ms)

Hms (ms + 1) Hms−1(ms + 1) . . . H0(ms + 1)

. . .. . .

. . .

Hms (t) Hms−1(t) . . . H0(t)

. . .. . .

. . .

.

(2.1)

For a rate R = b/c LDPC convolutional code (b < c), the elements Hi(t),

i = 0, 1, · · · ,ms, t ∈ Z, are binary (c − b) × c submatrices. Starting from the

(ms + 1) · (c − b)-th row, Hconv has K ones in each row, and starting from the

first column, Hconv has J ones in each column. The syndrome former memory

ms is determined by the maximum width (in submatrices) of the nonzero area

in Hconv, and the associated constraint length (in symbols) is defined as νs =

24

(ms + 1) · c. Periodic syndrome formers are said to have period Ts if they satisfy

Hi(t) = Hi(t + Ts), i = 0, 1, . . . ,ms, t ∈ Z. If the number of ones in each row

and/or column of Hconv varies, the code is irregular. Finally, if the period Ts = 1,

the code is called time-invariant, and the parity check matrix can be written as

Hconv =

H0

H1 H0

......

. . .

Hms Hms−1 . . . H0

Hms Hms−1 . . . H0

. . . . . . . . .

. (2.2)

A significant advantage of LDPC convolutional codes compared to their block

code counterparts is the so-called “fast encoding” property. As a result of the

diagonal shape of their parity check matrices, LDPC convolutional codes enjoy

simple shift register based encoders. Even randomly constructed LDPC convo-

lutional codes can be formed in such a way as to achieve this feature without

any loss in performance (see, e.g., [9, 29]). On the other hand, in order to have

a simple encoding procedure for LDPC block codes, either the block code must

have some sort of structure [32] or the parity-check matrix must be changed to a

more “encodable” form [33].

We now summarize the encoding of LDPC convolutional codes for a rate R =

b/c code. Let

u = (u0,u1, . . . ,ut) (2.3)

be the information sequence, where ut = (u(1)t , . . . , u

(b)t ), u

(i)t ∈ GF(2), i = 1, . . . , b.

This sequence is mapped by a convolutional encoder into the code sequence

v = (v0,v1, . . . ,vt), (2.4)

25

where vt = (v(1)t , . . . , v

(c)t ), v

(i)t ∈ GF(2), i = 1, . . . , c. As a result of the convolu-

tional code structure, a codeword v must satisfy

vtHT0 (t) + vt−1H

T1 (t) + . . .+ vt−msH

Tms

(t) = 0, t ∈ Z. (2.5)

Assuming that all submatrices HT0 (t), t ∈ Z, along the diagonal have full rank,

(2.5) can be used to define a shift-register based encoder. The encoder is system-

atic if the last c− b rows of the submatrices HT0 (t) are linearly independent, and

in this case the first b symbols of vt coincide with the information symbols, i.e.,

v(j)t = u

(j)t , j = 1, . . . , b.

As a special case of this structure, when the last c − b linearly independent

rows of HT0 (t) are chosen as the (c− b)× (c− b) identity matrix, the calculation

of the parity symbols is simplified and the encoding equations are defined using

(2.5) by the expressions

v(j)t = u

(j)t , j = 1, . . . , b, (2.6)

v(j)t =

b∑k=1

v(k)t h

(k,j−b)0 (t) +

ms∑i=1

c∑k=1

v(k)t−ih

(k,j−b)i (t), j = b+ 1, . . . , c . (2.7)

The corresponding shift-register based encoder is shown in Figure 2.1 for the

case R = b/c = 1/2. Such an encoder realization requires c · ms + b memory

units and the encoding complexity per encoded symbol is O(1), independent of the

codeword length (t+1)·c and the syndrome former memory ms. A straightforward

encoder for a length n LDPC block code that multiplies the information sequence

by the generator matrix has a complexity per encoded symbol of O(n) and the

best known algorithm to encode randomly constructed LDPC block codes has

26

a complexity per encoded symbol of O(g) with g << n. Therefore, the special

structure of the convolutional code gives it an advantage compared to LDPC block

codes in terms of encoding complexity.

h(1,1)0 h

(2,1)1 h

(1,1)1 h

(2,1)ms h

(1,1)ms

ut

v1t , v2

t

Figure 2.1. Shift-register based encoder.

The difficulty in constructing and decoding LDPC convolutional codes is deal-

ing with the unbounded size of the parity-check matrix. This is overcome at the

code construction step by considering only periodically time-varying codes. The

code construction problem is therefore reduced to designing just one period of the

parity-check matrix. For decoding, the most obvious approach is to terminate

27

the encoder2 and employ message-passing decoding based on the complete parity-

check matrix of the code. Although this would be straightforward to implement

using a standard LDPC block code decoder, it would be wasteful of resources,

since the resulting (very large) block decoder would not be taking advantage of

two important aspects of the convolutional structure: namely, that decoding can

be done continuously without waiting for an entire terminated block to be received

and that the distance between two variable nodes that are connected to the same

check node is limited by the size of the syndrome former memory. To take advan-

tage of the convolutional nature of the parity-check matrix, a continuous sliding

window decoder that operates on a window of size I · νs variable nodes, where I

is the number of decoding iterations to be performed, can be implemented, sim-

ilar to a Viterbi decoder with finite path memory [34]. A further simplification

is achieved by exploiting the fact that the decoding of two variable nodes that

are at least (ms + 1) time units apart can be performed independently, since the

corresponding bits cannot participate in the same parity-check equation. This

allows the parallelization of the I iterations by employing I independent identical

processors working on different regions of the parity-check matrix simultaneously,

resulting in the pipeline decoding architecture introduced by Jimenez-Feltstrom

and Zigangirov in [29]. The pipeline decoder outputs a continuous stream of de-

coded data after an initial decoding delay of I ·νs received symbols. The operation

of this decoder on the Tanner graph of a simple time-invariant rate R = 1/3 LDPC

convolutional code with syndrome former memory ms = 2 and constraint length

νs = 9 is illustrated in Figure 2.2. (Note that, to achieve capacity-approaching

performance, an LDPC convolutional code must have a large value of ms.)

2In termination, a pre-determined sequence of symbols are appended to the informationsequence in order to drive the encoder state to all-zero. The tail is generally computed as afunction of the encoder state at the end of encoding the information sequence.

28

ms + 1

Proc. Proc. Proc. Proc.

Decoding window(size= I · (ms + 1))

variable nodecheck node

r1t

r0t

r2t

v2t!I(ms+1)+1

v1t!I(ms+1)+1

v0t!I(ms+1)+1

(I ! 1)1 2 I

Figure 2.2. Tanner graph of an R=1/3 LDPC convolutional code and anillustration of pipeline decoding.

When the first c received symbols of the transmitted sequence enter the pipeline

decoder, it takes I · (ms + 1) time units for this c-tuple to reach the output of the

decoder where the decision on these symbols is made. Once this initial decoding

delay of the pipeline decoder has elapsed, the decoder produces a continuous

output stream, i.e., at each time unit, c newly received values enter the decoder

and c decoded bits leave it. The I processors perform the I decoding iterations

simultaneously. At each time unit, the i-th processor begins by activating the first

column of (c−b) check nodes in its operating region and then proceeds to activate

the last column of c variable nodes that are about to leave the operating region.

Although the pipeline decoder is capable of fully parallelizing the iterations

by employing I independent identical processors, employing a large number of

hardware processors might not be desirable in some applications. In such cases,

fewer processors (even one processor) can be scheduled to perform subsets of

iterations, resulting in a serial looping architecture [13] with reduced throughput.

29

This ability to balance the processor load and decoding speed dynamically is

especially desirable where very large LDPC convolutional codes must be decoded

with limited available on-chip memory.

2.2 Improvements to the Pipeline Decoder

To insure the independence of consecutive decoding iterations, the operating

regions of neighboring processors of the pipeline decoder do not overlap. As noted

above, this results in a fairly long initial decoding delay of I · (ms + 1) time units.

Also, since each variable node must store a channel value and one storage element

is needed for each edge in the Tanner graph, in order to decode a (J,K)-regular

code a total memory of (J + 1) · I · (ms + 1) · c = (J + 1) · I · νs storage elements

is required. Both quantities are linear functions of the number of iterations I and

the processor separation (ms +1). In this section, we present several reduced com-

plexity decoding strategies to decrease the initial decoding delay and the storage

requirements without significantly reducing the performance of the decoder.

These strategies can also be used with the serial looping architecture of [13].

2.2.1 A stopping rule for the pipeline decoder

For LDPC block codes, the decoded sequence is a valid codeword if and only

if all of the parity-check equations are satisfied after some number of iterations,

or, equivalently, all components of the syndrome vector are zero. When using

belief-propagation decoding, it almost never occurs that the decoder converges

to a different codeword after further iterations. This suggests stopping decoding

once the decoder output converges to a codeword. This is a well-known tool that

saves both time and power consumption in the decoding process.

30

However, this stopping rule cannot be used in the continuous pipeline con-

volutional decoder because of the finite size operating window, semi-infinite size

parity-check matrix, and the undetermined length of the input sequence. For such

an LDPC convolutional code decoder, it would be cumbersome to check whether

all the parity-check equations are satisfied at the completion of decoding. In this

section, we present a way of allowing the individual processors to determine when

to stop and restart.

Since decoding is performed over a finite size window, instead of trying to

determine whether the entire decoded sequence is a codeword or not, one can

consider segments of the decoded sequence and determine whether they match

a segment of a valid codeword, i.e., whether the corresponding components of

the syndrome vector are zero. This control of the processors is realized by using

individual counters for each processor. If the c-tuple that enters the operating

region of the i-th processor is observed to satisfy the first column of check nodes

in this operating region, i.e., the corresponding component of the syndrome vector

is zero, the i-th counter is incremented by one. If the counter value exceeds a

stopping parameter Ps, the processor enters the sleep mode and stops processing.

(The stopping condition is still checked in this mode.) When a c-tuple that does

not satisfy the first column of check nodes enters the decoder, the counter is

reset to zero and the processor resumes operation. The complexity overhead the

stopping rule introduces is quite small compared to the total complexity of the

tasks the processor performs at each time unit.

Intuitively, the stopping parameter Ps can be chosen equal to the syndrome-

former memory ms. This insures that all the bits entering a decoding region in

succeeding time intervals will lie on a valid codeword until a non-zero syndrome

31

component is observed. Simulation results for the BER performance of the pro-

posed stopping rule for unterminated LDPC convolutional codes with J = 3,

K = 6, Ps = ms, and different syndrome former memories over an AWGN channel

with SNR Eb/N0 per bit have been obtained. Compared to simulation results for

the conventional pipeline decoder, no performance degradation is observed.

The stopping rule performs very well for various memory lengths. For the

(512, 3, 6) code and a maximum allowable number of iterations I = 100, the

average number of iterations Iav performed per bit at 1.00dB SNR is 64, and this

value drops to 16 at 1.25dB. Although the stopping decoder does not reduce the

required decoder memory, depending on the quality of the received sequence and

the value of ms, it performs many fewer iterations than the non-stopping decoder

without any loss in BER performance. The realized savings in power consumption

can be translated into decoding time if, instead of I different processors, a serial

looping architecture (such as in [13]) is employed. It is also possible to use the

stopping rule with Ps < ms. However, if Ps is too small, the BER will increase.

2.2.2 On-demand variable node activation schedule

For decoding LDPC block codes, the standard message-passing schedule calls

for, at each iteration, all the variable nodes, and then all the check nodes, to be

activated. This scheduling corresponds to the message-passing schedule of the

pipeline decoder introduced in [29], in which, within a processing region, the c− bcheck node activations are performed just after a set of c variable nodes enters

the region, and the c variable node activations are performed just before a set of c

variable nodes leaves the region. However, using this scheduling, the decoder does

not make use of the most recent information at the variable nodes. The incoming

32

information to a variable node is included in its outgoing messages only after an

iteration is completed. In this subsection, we propose an on-demand variable node

activation schedule for the pipeline decoder.

According to the proposed schedule, the activation order of the check nodes

does not change, but instead of activating the variable nodes that are about to

leave an operating region, we activate a variable node whenever its message is

required by an active check node. Using this approach, the check nodes are able

to access the most recent information available from the variable nodes. Also,

the messages generated during a check node activation are immediately available

for further check node activations within the same iteration. This on-demand

scheduling allows the effects of message passing to flow faster through the Tanner

graph.

BER performance simulation results are presented in Figure 2.3 for codes of

various syndrome former memory values. These results show that allowing some

messages to flow faster through the Tanner graph improves performance, as long as

these messages have high enough reliabilities. The (128, 3, 6) code results indicate

that weaker codes may not take full advantage of this scheme, whereas the higher

memory order codes exhibit a 0.05− 0.10dB gain at high SNR.

From the computational point of view, the standard and on-demand variable

node activation schedules have exactly the same complexity, since the number of

computations is identical for both schemes. The check node activation complex-

ity is also clearly the same for the two schemes. On the one hand, on-demand

scheduling activates each variable node J times per iteration, which is the num-

ber of neighboring check nodes that require new information, whereas standard

scheduling activates each variable node just once per iteration. But, in the stan-

33

10-1

10-2

10-3

10-4

10-5

1 1.1 1.2 1.3 1.4 1.5

BER

Eb/N0 (dB)

(128,3,6) standard schedule(128,3,6) on-demand variable node activation(512,3,6) standard schedule(512,3,6) on-demand schedule with S=128(512,3,6) on-demand schedule with S=256(512,3,6) on-demand variable node activation(2048,3,6) standard schedule(2048,3,6) on-demand variable node activation

Figure 2.3. Bit error rate performance of on-demand variable nodescheduling and the compact decoder.

34

dard schedule, once a variable node is activated, all J outgoing messages are

calculated, whereas the variable node activation in on-demand scheduling corre-

sponds only to a single outgoing message calculation from the variable node to

the check node that has triggered the activation.

Compared to standard scheduling, the on-demand variable node activation

schedule achieves a lower BER for a fixed number of iterations. This advantage

can be translated into reduced delay and storage requirements by noting that the

on-demand schedule requires fewer iterations to achieve a given fixed BER, thus

reducing the initial decoding delay and requiring fewer processors and memory

units in the decoder implementation.

2.2.3 Compact pipeline decoder architecture

The requirement that the processing regions of the pipeline decoder cannot

overlap follows directly from the independence of the operations performed in

the standard schedule. But, once on-demand scheduling is employed, we can let

neighboring operating regions overlap to decrease the total storage requirements

of the decoder. As a result of the overlap, the information generated in the i-th

processor’s region will be immediately available to the (i+ 1)-th processor. These

two processors will both have access to the information stored in the overlapping

region. In such an arrangement, the BER is expected to increase, since the bits

that belong to the overlapping region are not ready for the next iteration. There-

fore, there will be a trade-off between the distance between neighboring processors

and BER performance. We introduce the overlap by denoting the processor sepa-

ration (which is equal to (ms + 1) in the standard decoder) as a variable S. This

implementation is called the reduced-memory (compact) decoder.

35

The simulation results given in Figure 2.3 show that, as long as S is chosen to

satisfy S > ms/2, the compact decoder causes negligible performance loss while

reducing storage requirements and initial decoding delay by almost a factor of

two. One reason for the performance loss when S < ms/2 is that even the non-

neighboring (i− 1)-th and (i+ 1)-th regions overlap, i.e., messages that have not

even been processed completely by the (i−1)-th processor are used in the (i+1)-th

iteration.

Although the computational complexity of the proposed decoder is the same

as the standard one, the storage requirement is much less. The total number of

memory units required for the compact decoder is just (I ·S+(ms+1−S))·c·(J+1).

(The additional memory units corresponding to the (ms + 1− S) additional time

units are required by the last processor to successfully decode the c bits leaving

the decoder. This amount is usually very small compared to I · S.)

2.3 Implementation Complexity Comparisons with LDPC Block Codes

In this section, we compare several aspects of decoding LDPC convolutional

codes to LDPC block codes.

2.3.1 Computational complexity

Let Ccheck (Cvar) denote the number of computations required for a check

(variable) node update for a check (variable) node of degree K (J). Regardless of

the code structure, Ccheck and Cvar only depend on the values J and K.

For a rate R = b/c, (ms, J,K)-LDPC convolutional code decoded using a

pipeline decoder with I iterations/processors, at every time instant each processor

activates c − b check nodes and c variable nodes. The computational complexity

36

per decoded bit is therefore given by

Cconvbit = ((c− b) · Ccheck + c · Cvar) · I/c (2.8)

= ((1−R) · Ccheck + Cvar) · I,

which is independent of the constraint length νs.

Similarly, the decoding complexity for an (n, J,K)-LDPC block code is calcu-

lated as

Cblockbit = (n · J

K· Ccheck + n · Cvar) · I/n (2.9)

= (J

K· Ccheck + Cvar) · I

= ((1−R) · Ccheck + Cvar) · I,

which is again independent of the code length n. Thus, there is no difference be-

tween block and convolutional LDPC codes with respect to computational com-

plexity.

2.3.2 Processor (hardware) complexity

The pipeline decoder implementation of an LDPC convolutional code operates

on I · νs symbols. However, as described in the previous sections, decoding can be

carried out by using I identical independent parallel processors each capable of

handling only νs symbols. Hence, it is sufficient to design the processor hardware

for νs symbols. For an LDPC block code of length n, the processor must be

capable of handling all n symbols. Therefore, for the same processor complexity,

the block length of an LDPC block code must be chosen to satisfy n = νs.

37

2.3.3 Storage requirements

For the pipeline decoder, we need a storage element for each edge in the cor-

responding Tanner graph. Each variable node also needs a storage element for

the channel value. Thus a total of I · (J + 1) · νs storage elements are required

for I iterations of decoding. Similarly, we need n · (J + 1) storage elements for

the decoding of an LDPC block code of length n. Thus, for the same storage

requirements, an LDPC block code must satisfy n = I · νs.

2.3.4 Decoding delay

Let Ts denote the time between the arrival of successive symbols, i.e., the

symbol rate is 1/Ts. Then the maximum time from the arrival of a symbol until

it is decoded is given by

∆convio = ((c− 1) + (ms + 1) · c · I) · Ts. (2.10)

The first term (c− 1) in (2.10) represents the time between the arrival of the first

and last of the c encoded symbols output by a rate R = b/c convolutional encoder

in each encoding interval. The dominant second term (ms + 1) · c · I is the time

each symbol spends in the decoding window. Since c symbols are loaded into the

decoder simultaneously, the pipeline decoder also requires a buffer to hold the first

(c− 1) symbols.

With LDPC block codes, data is typically transmitted in a sequence of blocks.

Depending on the data rate and the processor speed, several scenarios are possible.

We consider the best case for block codes, i.e., each block is decoded by the time

the first bit of the next block arrives. This results in a maximum input-output

38

delay of ∆blockio = n · Ts. 3 Thus, for equal decoding delays, the block length must

satisfy n = (c− 1) + νs · I, assuming the least possible delay for block codes.

We summarize our comparison results in Table 2.1.

TABLE 2.1

COMPUTATIONAL AND MEMORY REQUIREMENTS OF THE

PROPOSED DECODERS

Implementation Type Memory Requirement Computational Complexity

Standard I · (J + 1) · (ms + 1) · c I · (c− b) · Ccheck + I · c · CvarOn-demand I · (J + 1) · (ms + 1) · c I · (c− b) · Ccheck + I · c · Cvar

On-demand/compact I · (J + 1) · S · c+ (J + 1) · (ms + 1− S) · c I · (c− b) · Ccheck + I · c · CvarOn-demand/stopping I · (J + 1) · (ms + 1) · c Iav · (c− b) · Ccheck + Iav · c · Cvar + I · Cstop

In Figure 2.4, we plot the performance of a rate R = 1/2, (2048,3,6)-LDPC

convolutional code with I = 50 iterations on an AWGN channel. Also shown

is the performance of two J = 3, K = 6 LDPC block codes with a maximum

number of 50 iterations. The block lengths were chosen so that in one case the

decoders have the same processor complexity, i.e., n = νs, and in the other case

the same decoding delay (assuming the least possible delay for the block codes),

i.e., n = (c − 1) + νs · I. For the same processor complexity, the convolutional

3Note that the block decoder does not need any buffering under such conditions. Let Tdec(n, I)denote the time required to perform I iterations of message-passing decoding on a block of nsymbols. Thus we require Tdec(n, I) ≤ Ts, i.e., this scenario requires extremely fast processorsor very low data rates. By contrast, Tdec(c, I = 1) ≤ c · Ts for convolutional codes.

39

code outperforms the block code by about 0.6 dB at a bit error rate of 10−5. For

the same decoding delay, the convolutional and block code performance is nearly

identical. Note that in the case of equal decoding delay, the storage requirements

for the block code are also almost the same as for the convolutional code.

10-1

10-2

10-3

10-4

10-5

1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8

BER

Eb/N0 (dB)

(4096,3,6) LDPC block code(204800,3,6) LDPC block code(2048,3,6) LDPC convolutional code

Figure 2.4. Performance comparison of LDPC block and convolutionalcodes.

Typically, the data rates and decoding speeds are such that some buffering is

required to decode block codes. There are several different decoding architectures

40

possible in this case. However, the delay in all of these scenarios is at least as

much as in the scenario we have considered earlier, so that the convolutional code

would outperform the block code in each of these cases for the same decoding

delay.

An LDPC block code can also be decoded by a decoder with an architecture

similar to the convolutional code, by treating it as a convolutional code with

memory zero. Such a decoder would consist of a buffer to store n symbols along

with I processors, each performing one iteration of message-passing. In this case,

∆blockio = (n − 1 + I · n) · Ts and Tdec(n, I = 1) ≤ n · Ts. (The block length n is

analogous to c in this case.) This decoder structure for block codes is appropriate

when transmitting a large amount of data (>> n) at high data rates.

41

CHAPTER 3

DERIVING LDPC CONVOLUTIONAL CODES BY UNWRAPPING LDPC

BLOCK CODES

In Chapter 2, parity-check matrices for time-varying and time-invariant LDPC

convolutional codes were given in (2.1) and (2.2), respectively. However, no tech-

nique to construct such parity-check matrices were considered. In this chapter, we

first discuss two techniques to construct these codes and then propose a unified

approach that provides a better understanding of these code construction tech-

niques. The unified code construction approach also provides the basis for the

graph and decoding structure analyses presented in Chapters 5 and 7.

We mentioned in Chapter 2 that, during the code construction step, it is

enough to consider only one period of a periodically time-varying code. However,

it is not an easy task to directly design one period of the parity-check matrix, since

the process of repeating and stacking periodic sections on top of each other in order

to construct a semi-infinite parity-check matrix results in a dependancy between

the initial and final sets of columns. In the LDPC convolutional code literature,

this dependancy is usually avoided by “deriving” the LDPC convolutional code

parity-check matrix from an LDPC block code parity-check matrix, thus avoiding

the dependancy problem altogether. Two different techniques that exploit this

idea are presented in Sections 3.1 and 3.2.

42

3.1 Deriving Time-Invariant LDPC Convolutional Codes by Unwrapping QC-

LDPC Block Codes

The term unwrapping, in particular, unwrapping a quasi-cyclic (QC) block

code to obtain a time-invariant convolutional code, was first introduced in a paper

by Tanner [35]. The paper describes a link between QC block codes and time-

invariant convolutional codes and shows that the free distance of the unwrapped

convolutional code cannot be smaller than the minimum distance of the underlying

QC code. This idea was later extended in [36, 37]. We start with a brief definition

of QC block codes.

Definition 3.1.1 A linear block code is called quasi-cyclic with period c if cycli-

cally shifted version of any codeword by c symbol positions is another codeword.

Any length n = rc linear QC block code CQC , C(r)QC with period c has an

r(c−b)×rc parity-check matrix H(r)QC that consists of r×r circulant matrices, and

equivalently, a (c−b)×c polynomial parity-check matrix H(r)QC(X), with polynomial

operations performed modulo Xr − 1. The equivalence of H(r)QC and H

(r)QC(X) is

preserved as a result of the isomorphism between the ring of polynomials modulo

Xr − 1 and the set of r × r circulant matrices.

With any QC block code C(r)QC described by the polynomial parity-check matrix

H(r)QC(X), we can associate a rate R = b/c time-invariant convolutional code Cconv

given by the same polynomial parity-check matrix Hconv(D), where the change of

variables indicates the lack of modulo Dr−1 operations [38]. The scalar version of

Hconv(D) results in the semi-infinite parity-check matrix Hconv described in (2.2)

(see also [39]).

43

Example 3.1.2 Consider the Tanner graph corresponding to a rate R = 1/3 QC

block code C(r)QC with

H(r)QC(X) =

1 +X 1 X

X2 1 +X 1

(3.1)

and r = 7 is shown in Figure 3.1(a). A longer code with the same code structure

and r = 14 is also given in Figure 3.1(b). The QC structure of this family of

codes with different circulant sizes can be seen from the figures. It consists of

an inner structure (observed closer to the middle of the Tanner graph) where the

node connections repeat themselves and the outer structure (observed closer to the

beginning and the end of the Tanner graph) where the connections wrap around.

The derived LDPC convolutional code has the polynomial parity-check matrix

Hconv(D) =

1 +D 1 D

D2 1 +D 1

. (3.2)

For this code, the syndrome former memory ms is equal to the largest power of

the delay variable D which is 2 and the constraint length νs equals 9.

Analyzing the Tanner graph corresponding to the derived LDPC convolutional

code reveals that this graph possesses the same inner structure of the underlying

QC block code. Furthermore, the derived convolutional code can be seen as a QC

block code with circulant size r →∞.

44

(a)

(b)

(c)

Figure 3.1. Tanner graph of (a) a QC block code of lengthn = r · c = 7 · 3 = 21, (b) a QC block code of length

n = r · c = 14 · 3 = 42, (c) the derived LDPC convolutional code withms = 2 and νs = (ms + 1) · c = 3 · 3 = 9.

45

3.2 Deriving Time-Varying LDPC Convolutional Codes by Unwrapping Ran-

domly Constructed LDPC Block Codes

In [29], Jimenez-Feltstrom and Zigangirov proposed a matrix-based unwrapping

method of deriving periodically time-varying LDPC convolutional codes from ran-

domly constructed LDPC block codes. The r(c − b) × rc parity-check matrix H

of the block code is first cut along its diagonal in steps of size (c− b)× c and the

upper-right portion is “pasted” below the remaining block. This gives the ma-

trix the desired diagonal shape while preserving the row and column weights.

As an additional step (c − b) × c submatrices of all-ones can be attached to

the top of this diagonal shape to ensure fast encoding, although not required

in general. The matrix is then repeated indefinitely to arrive at a semi-infinite

parity-check matrix of a periodically time-varying LDPC convolutional code of

rate R = b/c with parity-check matrix Hconv given in (2.1), whose constraint

length νs = (ms + 1) · c = (r + 1) · c and period Ts = ms = r is determined by the

length rc of the LDPC block code. We now demonstrate this code construction

technique on an example code.

Example 3.2.1 Consider a randomly constructed rate R = b/c = 1/2 LDPC

block code described by the 4× 8 scalar parity-check matrix

H =

1 0 0 1 1 0 1 1

0 1 1 1 0 1 1 0

1 1 0 0 1 0 1 1

1 0 1 1 0 1 0 1

. (3.3)

The steps of the matrix-based unwrapping of this LDPC block code with parameters

r = 4, b = 1, and c = 2 are given in Figure 3.2. The resulting LDPC convolutional

46

code of Figure 3.2(d) has a constraint length νs = 10 and syndrome former period

Ts = 4.

3.3 A Unified Approach to Unwrapping

The QC-based unwrapping method of [35] and the matrix-based unwrapping

method of [29] can be efficiently unified under the same theory (see [5]). In

particular, by permuting the rows and columns of the scalar parity-check matrix

H(r)QC,1 we obtain an equivalent parity-check matrix representation H

(r)

QC for the

QC code C(r)QC (see (3.4), left),

H(r)

QC ,

H0 Hr−1 · · · H1

H1 H0 · · · H2

......

. . ....

Hr−1 Hr−2 · · · H0

,

where the scalar (c− b)× c-matrices H0,H1, . . . ,Hr−1 define a polynomial parity-

check matrix H(r)QC(X) = H0 + H1X + · · · + Hr−1X

r−1. The associated time-

invariant convolutional code Cconv defined by the same polynomial matrix Hconv(D)

= H(r)QC(X)|X=D has the semi-infinite scalar parity-check matrix Hconv (see (3.4),

right), described also in (2.2). We note that the matrix Hconv can be obtained

from the matrix H(r)

QC by cutting along the diagonal in steps of (c − b) × c and

then pasting the upper-diagonal portion below the lower-diagonal portion, with

the resulting diagonally-shaped matrix repeated indefinitely to obtain the matrix

Hconv. The resulting rate R = b/c time-invariant convolutional code has syndrome

1I.e., by taking the first row in the first block of r rows, the first row in the second block of rrows, etc., then the second row in the first block, the second row in the second block, etc., andsimilarly for the columns.

47

1 0 0 1 1 1 100 0 01111 11 1 1 111 1 1 1 1

0000

0 0

0 11 11 1 1 11 0 0 1 1 1 10

0 011111 11

10

0

000 0

(a) (b)

0 11 11 1 1 1

00

11 11

1 1 1 1 000 1111001

000 0

1 11 1

1 11 1

1

00

11 11

1 1 000 111

011

11

0 11 11 1 1 1

00

11 11

1 1 1 1 000 1111001

000 0

1111

1111

0 11 11 1 1

1 11001

00 0

1111

(c) (d)

Figure 3.2. Tanner graph of (a) the original LDPC block code, (b)LDPC block code after cutting along the diagonal, (c) after appending

the submatrices for fast encoding, and (d) the resulting LDPCconvolutional code.

48

former memory ms ≤ r − 1.

H(r)

QC ,

H0 Hr−1 · · · H1

H1 H0 · · · H2

......

. . ....

Hr−1 Hr−2 · · · H0

, Hconv ,

H0

H1 H0

......

. . .

Hr−1 Hr−2 . . . H0

Hr−1 Hr−2 . . . H0

. . . . . . . . . . . .

(3.4)

Motivated by the results of [29] and [7], we now consider an alternative version of

unwrapping a QC-LDPC block code, without prior row and column reordering.

In this approach, we directly unwrap the r(c− b)× rc binary parity-check matrix

H(r)QC =

P1,1 P1,2 . . . P1,c

P2,1 P2,2 . . . P2,c

...... . . .

...

Pc−b,1 Pc−b,2 . . . Pc−b,c

, (3.5)

where the entries Pi,j are r×r circulant matrices, in steps of (c−b)×c, to obtain the

parity-check matrix Hconv, described also in (2.1), of a periodically time-varying

rate R = b/c convolutional code Cconv. A convenient feature of this time-varying

unwrapping is that a family of time-varying convolutional codes can be derived

by choosing different values for the circulant size r of the QC block code. The

resulting time-varying codes have syndrome former memory ms ≤ r−1 (the same

as the time-invariant codes) and period Ts = r. This property is not shared by the

49

time-invariant convolutional codes obtained above, since for any circulant size r

the resulting time-invariant code is the same. We now introduce a simple example

to demonstrate the different unwrapping methods.

Example 3.3.1 The parity-check matrix H(r)QC of a rate R = 1/3 (2,3)-regular

QC-LDPC block code of length n = rc = 21 with J = 2 ones in each column and

K = 3 ones in each row is shown in Figure 3.3(a). The r(c − b) × rc (14 × 21)

matrix H(r)QC was constructed using a (c − b) × c (2 × 3) array of r × r (7 × 7)

shifted identity (circulant) matrices, and the corresponding block (convolutional)

polynomial parity-check matrices H(r)QC(X) (Hconv(D)) are given by

H(r)QC(X) =

X X2 X4

X6 X5 X3

, Hconv(D) =

D D2 D4

D6 D5 D3

. (3.6)

For the time-invariant unwrapping, the rows and columns of this matrix are re-

ordered to arrive at the matrix H(r)

QC shown in Figure 3.3(b) and then unwrapped

to obtain the rate R = 1/3 (2, 3)-regular time-invariant LDPC convolutional code

parity-check matrix shown in Figure 3.3(d), which is identical to the scalar version

Hconv of Hconv(D). For the time-varying unwrapping, the only difference is that

the rows and columns of the matrix H(r)QC are not reordered prior to the unwrapping

step, resulting in the rate R = 1/3 (2, 3)-regular time-varying LDPC convolutional

code parity-check matrix Hconv shown in Figure 3.3(c).

Although the time-invariant unwrapping usually gives convolutional codes with

syndrome former memory ms = r − 1, it is clear (see (3.4), right) that the actual

maximum width (in submatrices) of the nonzero region in Hconv depends on the

particular (c − b) × c matrices H0,H1, . . . ,Hr−1. If one or more of the latter

matrices in this sequence is all-zero, then the convolutional code has a syndrome

50

0 00 0 0 0 0 00 0 0 0 0 00 0 0 00 0 0 0 0 0 0 00 0 0 0 0 00 0 0 0

0 0 0 0 0 0 0 0 00 0 0 00 00 0 0 0 0 0 00 0 0 00 00 0 0 0 0 0 0 0 0 0 00 0 00 0 0 0 00 0 0 0 0

0 00 0 0 0 0 00 0 0 0 00 0 0 0 000 0 0 0 0 00 0 0 0 0 0 00 0 0 0

00 0 0 0 0 0 00 0 0 0 0 00 0 0 00 00 0 0 0 0 00 0 0 0 0 00 0 0 00 00 0 0 0 0 00 0 0 0 0 0 0 0 0 00 00 0 0 0 0 00 0 0 0 00 0 0 0 00 00 0 0 0 0 0 0 0 0 0 00 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0

11

1

11

11111

1111

111

111111

1

11111

11

111

1111

00 0 0 000 0 0 00 00 0 00 00 0 0

1111

H(r)QC

(a)

0 0 0 0 00 0 0 0 0 00 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0

0 0 0 0 0 0 0 0 00 0 0 00 00 0 0 0 0 0 00 0 0 0

0 0 0 0 0 0 0 0 0 00 0 0 0 00 0 0 0 0

0 0 00 0 0 0 00 0 0 00 0 0 0

0 0 0 0 00 0 0 0 0

0 0 00 0 0

1 11

1111

1111

111

111

0 000 000 0 0 0

00 0 0 00 00 0 0 0 000 00 0 0 0 000 00 0 0 0 0 00 000 0 0 0 0 00 0 0

00 0 0 0 0 0 00 0 0 0 00 00 0 0 0 0 00 0 0 0 00 00 0 0 0 0 00 0 0 0 0 0 00 00 0 0 0 0 00 0 0 0 00 00 00 0 0 0 0 0 0 0 0 0 00 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0

111111 1

111111

1

11111

11

1111

Hconv

(c)

0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 00 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 01 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 00 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 00 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 00 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 10 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 00 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 00 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 00 0 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 01 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0

H(r)

QC

(b)0 0 00 0 01 0 0 0 0 00 0 0 0 0 00 1 0 1 0 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 1 0 1 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 1 0 1 0 0 0 0 00 0 0 0 0 1 0 0 0 0 0 0 0 0 00 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 00 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 01 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 01 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0

0 0 0 0 0 0 0 0 1 0 0 0 0 1 01 0 0 0 1 0 0 0 0 0 0 1 0 0 0

0 0 0 0 0 0 0 0 1 0 0 01 0 0 0 1 0 0 0 0 0 0 1

0 0 0 0 0 0 0 0 11 0 0 0 1 0 0 0 0

0 0 0 0 0 01 0 0 0 1 0

0 0 01 0 0

Hconv

(d)

Figure 3.3. Deriving a time-invariant LDPC convolutional code from aQC-LDPC block code: (a) QC-LDPC code, (b) after reordering of rowsand columns, (c) resulting time-varying LDPC convolutional code, (d)

resulting time-invariant LDPC convolutional code.

51

former memory ms strictly less than r − 1. It may also be possible to reduce

the syndrome former memory, since dividing or multiplying the rows of Hconv(D)

by the delay variable D does not change the convolutional code associated with

Hconv(D). In particular, we can divide each row of Hconv(D) by a power of D such

that the smallest power of D in each row is 0, i.e., there is a 1 in each row. The

maximum power of D in the resulting equivalent polynomial parity-check matrix

is the minimum syndrome former memory for the convolutional code. We now

illustrate this syndrome former memory reduction for the small convolutional code

example of (3.6).

Example 3.3.2 The circulant size is r = 7 in this case, and the polynomial

parity-check matrix Hconv(D) in (3.6) has syndrome former memory ms = 6.

But, following the above procedure, the first and second rows of Hconv(D) can be

divided by D and D3, respectively, to yield an equivalent convolutional code with

syndrome former memory ms = 3 and polynomial parity check matrix

Hconv(D) =

1 D D3

D3 D2 1

. (3.7)

For large circulant-based QC-LDPC codes, however, this additional reduction in

syndrome former memory is usually relatively small, and for the time-invariant

LDPC convolutional codes considered in this dissertation we assume that the syn-

drome former memory is simply the largest power of D in the original polynomial

parity-check matrix.

In the case of periodically time-varying LDPC convolutional codes with period

Ts, the probability of having all Hms(ms + i) = 0, i = 0, 1, . . . , Ts− 1, is extremely

52

low, and therefore no reduction in the syndrome former memory is possible.

This visualization method of cutting and pasting parts of H(r)

QC to obtain the

matrix Hconv, referred to in [29] as “unwrapping the matrix”, reinforces the notion

of unwrapping a QC code to obtain a time-invariant convolutional code introduced

in [19]. Therefore, the methods of obtaining time-varying convolutional codes from

random LDPC block codes (see [29]) and time-invariant convolutional codes from

QC-LDPC block codes (see [19]) can be unified under the same theory of unwrap-

ping a block code specified by a given parity-check matrix. In fact, any periodically

time-varying convolutional code, including time-invariant convolutional codes as

a special case, can be obtained from some block code by unwrapping its par-

ity check-matrix. Starting with block codes of a known structure, we can unwrap

them to obtain various convolutional codes, both time-varying and time-invariant,

depending on the original parity-check matrix representation of the block code.

In particular, different parity-check matrices for the same block code, or parity-

check matrices of equivalent codes, generate different unwrapped convolutional

codes. For example, in Figure 3.3, two different convolutional codes are obtained

by unwrapping equivalent parity-check matrices of the same QC block code. The

original block code and the two convolutional codes have the same row and col-

umn weights, J = 2 and K = 3, the same rate2 R = b/c = 1/3, and the constraint

length νs = (ms + 1)c = 21 of the two convolutional codes equals the block length

n = 21 of the block code. As will be seen in Chapter 4, the simulated performance

of the two convolutional codes can be vastly different, but both are much better

than the original block code.

2The actual rate of the block code can be slightly higher than the design rate if there aredependent rows in the parity-check matrix. In this example, the block code rate is R = 8/21.

53

CHAPTER 4

PERFORMANCE OF DERIVED LDPC CONVOLUTIONAL CODES AND

THE CONVOLUTIONAL GAIN

In this chapter, we demonstrate the performance capabilities of unwrapped

LDPC convolutional codes for quasi-cyclic, protograph-based, and randomly con-

structed underlying LDPC block codes. We also investigate the relation between

the unwrapping step size and the performance of the derived LDPC convolutional

codes.

4.1 Unwrapped LDPC Convolutional Code Examples

As our first example, we consider a [155,64] (3,5)-regular QC-LDPC block code

whose parity-check matrix contains 3 ones per column and 5 ones per row with

circulant size r = 31. We apply the methods described in Chapter 3 to construct

a variety of time-invariant and time-varying convolutional codes. The block code

has a polynomial parity-check matrix given by

H(r)QC(X) =

X X2 X4 X8 X16

X5 X10 X20 X9 X18

X25 X19 X7 X14 X28

.

54

The rate R = b/c = 2/5 time-invariant convolutional code obtained from this

block code has syndrome former memory ms = 28, corresponding to a constraint

length of νs = (ms+1)·c = 145. We can also obtain a family of rate R = 2/5 time-

varying convolutional codes with syndrome former memories ms = 30, 47, and 79

by performing the time-varying unwrapping (cutting along the diagonal of H(r)QC

in 3× 5 steps) on QC block codes with the same polynomial parity-check matrix

and circulant sizes of 31, 48, and 80, respectively. It is important to note that the

time-invariant unwrapping for each of the QC codes with different circulant sizes

gives the same ms = 28 convolutional code, since the time-invariant unwrapping

utilizes the polynomial parity-check matrix structure only and not the circulant

size. Simulation results for these codes were obtained assuming BPSK modulation

and an AWGN channel. All decoders were allowed a maximum of 50 iterations,

and the block code decoders employed a syndrome-check based stopping rule. The

resulting BER performance is shown in Figure 4.1.

We note that the time-invariant LDPC convolutional code performs between

0.5dB and 1.0dB better than the QC-LDPC block code in the low-to-moderate

SNR region and that the gain drops to only about 0.2dB in the high SNR region.

On the other hand, the time-varying LDPC convolutional codes achieve signifi-

cantly better performance, with gains ranging between 1.9dB and 2.8dB at a BER

of 10−6 compared to their corresponding QC-LDPC block codes. We also note

that these “convolutional gains” increase with increasing circulant size.

For a comparison based on processor complexity, we require that n = νs. The

time-varying convolutional code derived from the QC code with circulant size

r = 31 has constraint length νs = (ms + 1)c = 155, and hence approximately

the same processor complexity as the QC block code of length n = 155 and the

55

100

10-1

10-2

10-3

10-4

10-5

10-6

1 2 3 4 5 6

BER

Eb/N0 (dB)

[155,64] QC code[240,98] QC code[400,162] QC codeR=2/5 time-invariant conv. code with !s=145R=2/5 time-varying conv. code with !s=155R=2/5 time-varying conv. code with !s=240R=2/5 time-varying conv. code with !s=400

Figure 4.1. Performance of three (3,5)-regular QC-LDPC block codesand their associated time-invariant and time-varying LDPC

convolutional codes.

56

time-invariant convolutional code with νs = 140, but it achieves large BER gains

compared to both of these codes. We note, in addition, that the performance

of the time-varying convolutional code with syndrome former memory ms = 80

and constraint length νs = 400 is quite remarkable, since, at a BER of 10−5, it

performs within 1dB of the iterative decoding threshold of 0.965dB, while having

the same processor complexity as a block code of length only n = 400.

As a second example, we unwrap a rate R = 1/2 (3,6)-regular QC-LDPC block

code of length n = 4608 constructed based on finite fields [40]. The polynomial

parity-check matrix of this code is of size 32× 64. The time-varying unwrapping

with a step size of 1 × 2 gives a periodically time-varying rate R = 1/2 LDPC

convolutional code with syndrome former memory ms = 2303 and constraint

length νs = 4608. For the time-invariant unwrapping, taking the step size equal

to the size of the polynomial parity-check matrix 32× 64 yields a rate R = 32/64

time-invariant LDPC convolutional code with syndrome former memory ms = 71

and constraint length νs = 4608. The BER performance of these codes is shown

in Figure 4.2, where both LDPC convolutional codes perform almost identically

and outperform the underlying QC block code by about 0.6dB.

The time-varying unwrapping method described above can also be applied to

codes with randomly constructed or irregular parity-check matrices. For example,

in [4] we applied the time-varying unwrapping to some irregular protograph-based

LDPC block codes constructions (see also [41]) with optimized iterative decoding

thresholds. In particular, punctured versions of the protograph codes were chosen

for comparison. They each had a block length of about 2500, with code rates

ranging from 1/2 to 4/5. Again, using the time-varying unwrapping, “convolu-

tional gains” ranging from 0.6dB to 0.9dB were obtained. (See Figure 4.3.) This

57

10-1

10-2

10-3

10-4

10-5

10-6

0 0.5 1 1.5 2

BER

Eb/N0 (dB)

[4608,2304] QC codeR=1/2 time-varying conv. code with !s=4608R=32/64 time-invariant conv. code with !s=4608

Figure 4.2. Performance of a [4608,2304] QC-LDPC block code and theassociated time-invariant and time-varying LDPC convolutional codes.

58

example demonstrates the applicability of the unwrapping method to the class of

irregular codes. Similarly, we had shown in [9] that an LDPC convolutional code

derived from a randomly constructed irregular LDPC block code outperformed

the underlying code by almost 0.80 dB at a BER of 10−5. The degree distribu-

tion of the underlying LDPC block code was fully optimized and had an iterative

decoding threshold of 0.3104 dB [26].

100

10-1

10-2

10-3

10-4

10-5

0 0.5 1 1.5 2 2.5 3 3.5 4

BER

Eb/N0 (dB)

n=2500 R=1/2 protographUnwrapped conv. code with !s=2500n=2520 R=2/3 protographUnwrapped conv. code with !s=2520n=2520 R=3/4 protographUnwrapped conv. code with !s=2520n=2508 R=4/5 protographUnwrapped conv. code with !s=2508

Figure 4.3. Performance of a family of randomly constructedprotograph-based LDPC block codes and the associated time-varying

LDPC convolutional codes.

59

The example code families we have presented in this section demonstrate the

universality of the observed “convolutional gains”. It is interesting to note that the

gains vary from about 0.5 dB to several dB, depending on the underlying LDPC

block code. It should be noted, however, that the gains are a function of the block

length n of the underlying code. An extreme case would be to start with a very

large value of n, such that the underlying code already performs very close to its

iterative decoding threshold. In this case, although some additional gain is possible

by unwrapping the block code, the resulting convolutional code performance would

essentially be bounded between the iterative decoding threshold and the block

code performance, resulting in only a small “convolutional” gain. For short to

moderate length codes, however, the “convolutional gain” is significant. In the

following chapters we discuss the reasons for this “convolutional gain” and its cost

in terms of implementation complexity.

4.2 Unwrapping with Different Step Sizes

In the QC block code case, although the unwrapping step size of (c − b) × cis fixed for the time-invariant unwrapping, there is no such constraint on the step

size of the time-varying unwrapping. We can choose any step size (c− b)k× ck as

long as the coefficient k satisfies 0 < k ≤ r and k ∈ Z. Among the possible choices

of k, the special case of k = r is of no practical interest, since it corresponds to

repeating the original QC block code indefinitely and is therefore equivalent to

decoding the QC block code with the pipeline decoder (which would not change

its performance). In Figure 4.4 we give examples of unwrapping the rate R = 2/5

QC-LDPC block code with circulant size r = 80 for different step sizes. (Note

that the k = r = 80 case gives an unwrapping size of 240× 400 and the resulting

60

convolutional code has the same performance as the underlying block code given

in Figure 4.1.) Smaller values of k yield performance comparable to the smallest

possible unwrapping size of 3× 5.

100

10-1

10-2

10-3

10-4

10-5

10-6

1 2 3 4 5 6

BER

Eb/N0 (dB)

[400,162] QC code unwrapped with step size 3x5[400,162] QC code unwrapped with step size 6x10[400,162] QC code unwrapped with step size 12x20[400,162] QC code unwrapped with step size 24x40[400,162] QC code unwrapped with step size 48x80[400,162] QC code unwrapped with step size 240x400

Figure 4.4. Performance of LDPC convolutional codes unwrapped fromthe [400,162] QC-LDPC block code using different step sizes.

The ability to choose different step sizes in the time-varying unwrapping is not

specific to the quasi-cyclic structure of the underlying block code. Indeed, similar

results have been obtained using the other example code families in the paper.

61

In all cases, we observed that, while the ordering of the error curves for k < r

varies depending on the actual code used, the difference between the k = r and

k < r cases is always significant. We illustrate the causes of this difference in

Figure 4.5, where the shape of the non-zero region of the parity-check matrix of

the unwrapped convolutional code is sketched for both cases. As noted earlier, the

matrix resulting from the cutting and pasting operation is repeated periodically

to form an infinitely long parity-check matrix. The step size k = r corresponds

to repeating the block code indefinitely (see Figure 4.5(a)) and therefore results

in the same performance as the block code. On the other hand, for any value

of k smaller than the circulant size r, the resulting convolutional code has inter-

connections among the repeated periods (see Figure 4.5(b)), which allow messages

to propagate over large regions of the graph, rather than over only one constraint

(block) length, as for the k = r case. The convolutional codes unwrapped with

size (c− b)k × ck, k < r, therefore have improved iterative decoding performance

compared to the underlying block code.

Convolutional codes unwrapped with different step sizes (k < r) share the

same constraint length νs, but differ in the syndrome former memory ms. This,

however, does not cause a difference in the processor size or in the computational

complexity. The only difference among these convolutional codes is in encoding,

where for larger values of k, larger size (ck) tuples must be encoded. But this is not

a difficult implementation issue, and we usually can choose the actual unwrapping

size based on the structure of the underlying block code.

For the time-invariant unwrapping of QC block codes, we are forced to choose

the size of the polynomial parity-check matrix H(r)QC(X) as the default unwrap-

ping size, but (as noted earlier) there is no such constraint for the time-varying

62

Figure 4.5. The shape of the convolutional code parity-check matrixnon-zero region for (a) k = r (left), (b) k < r (right).

unwrapping. As a general rule, however, for block codes of rate R = b/c, we

simply choose a step size (c − b) × c for time-varying unwrappings, with the ex-

ception of time-varying unwrappings of protograph-based codes. For these codes,

the step size can be conveniently chosen as the size of the underlying protograph1

used to construct the block code, since this choice allows a parallelization in the

implementation of the pipeline decoder.

1Protograph-based codes are constructed by copying a relatively small projected graph (pro-tograph) many times and interleaving the connections (edges) of the resulting Tanner graph[41].

63

CHAPTER 5

GRAPH STRUCTURE OF DERIVED LDPC CONVOLUTIONAL CODES

It is well known that cycles in the Tanner graph representation of a sparse code

affect the iterative decoding algorithm, with short cycles generally pushing its

performance further away from optimum. (Indeed, an attempt to investigate and

minimize these effects has been made in [42] and [43], where the authors propose

LDPC code construction procedures to maximize the connectivity of short cycles

to the rest of the graph, thus also maximizing the independence of the messages

flowing through these cycles.) Hence it is common practice to design codes that

do not contain short cycles, so as to obtain independent messages in at least the

initial steps (iterations) of the decoding process.

5.1 A Bound on the Girth of Unwrapped LDPC Convolutional Codes

We now show that unwrapped LDPC convolutional codes have better graph-

cycle properties than their block code counterparts. In particular, we show that

the unwrapping process can “break” some cycles of the underlying block code, but

cannot create any shorter cycles. We start with a theorem that shows that any

cycle in a convolutional code derived by unwrapping a block code always maps to

a cycle of the same or smaller length in the underlying block code.

64

Theorem 5.1.1 Consider a convolutional code Cconv derived by unwrapping a

block code C with block length n. Let the path (vI(0), vI(1), . . . , vI(L−1)) be a cycle of

length 2L in the Tanner graph of the convolutional code1, where I =(I(0), I(1), . . . ,

I(L − 1))

is an index vector of length L and all of its elements are unique. Let

the elements of the modulo index vector I be defined as I(i) = I(i) mod (n),

i = 0, 1, . . . , L− 1. Then the path (vI(0), vI(1), . . . , vI(L−1)) is a path in the Tanner

graph of the underlying block code. Moreover, it is either a cycle of length 2L or

contains a cycle of length smaller than 2L.

Proof:

The path (vI(0), vI(1), . . . , vI(L−1)) must satisfy one of the following two cases:

(a) All elements of this path are unique. Then the path is a cycle of length 2L

in the Tanner graph of the underlying block code.

(b) There is at least one repetition in the vector I, i.e., it cannot be a cycle itself,

although it can be partitioned into shorter cycles. Then, for any repetition

in I, let i and j be the two positions such that I(i) = I(j), with j > i. This

repetition yields a cycle of length 2 min(j − i, L+ i− j). It follows that

2 ≤ 2 min(j − i, L+ i− j) ≤ 2

⌊L

2

⌋≤ L < 2L.

The only case where the lower bound is achieved with equality is when

j = i + 1. However, vI(i) and vI(i+1) cannot follow each other in a cycle

in the convolutional code graph, since by construction two distinct copies

of a variable node in the block code cannot participate in a common check

1I.e., the sequence of L variable nodes, in cyclic order, vI(0) → vI(1) → · · · → vI(L−1) → vI(0)

constitutes a cycle in the Tanner graph associated with the convolutional code.

65

equation of the unwrapped convolutional code. Therefore

2 < 2 min(j − i, L+ i− j) ≤ 2

⌊L

2

⌋≤ L < 2L,

and a cycle in the unwrapped convolutional code maps to a cycle of smaller

length in the block code.

This result holds for time-varying convolutional codes derived from randomly

constructed, protograph-based, and quasi-cyclic LDPC block codes as well as for

time-invariant convolutional codes derived from QC-LDPC block codes. (Note

that the additional step of row/column reordering in the time-invariant unwrap-

ping does not affect the cycle structure in the Tanner graph of the underlying

block code.) A trivial corollary of the above theorem is that the girth of the de-

rived LDPC convolutional code is at least as large as that of the underlying LDPC

block code.

We now revisit some example codes from Section 4 and analyze their graph

cycle structure. The complete graph cycle histograms of these example codes are

given in Appendix A.

Example 5.1.2 Consider the convolutional codes obtained from the QC-LDPC

block code of length n = 155, whose BER performance was plotted in Figure 4.1.

The girth2 of the underlying block code is 8 and it has an average3 of 7 cycles

of length 8 and 10.65 cycles of length 10. The corresponding time-invariant con-

volutional code has 5.8 cycles of length 8 and 9.6 cycles of length 10, and the

2The class of QC-LDPC codes constructed in [38] is known to have good girth properties.3In order to accurately compare the graph cycle spectra of two codes with differ-

ent block/constraint lengths, we compute the number of cycles of a given length over ablock/constraint length and divide it by the block/constraint length.

66

time-varying convolutional code has only 3.57 cycles of length 8 and 10.32 cycles

of length 10.

Example 5.1.3 Consider the rate R = 1/2 (3,6)-regular QC-LDPC block code of

length n = 4608 whose BER performance was plotted in Figure 4.2. As a result

of its strong algebraic structure, the underlying block code has few short cycles

– an average of 0.46 cycles of length 6 and 1.95 cycles of length 8, whereas the

time-invariant convolutional code has an average of only 0.28 cycles of length 6

and 1.23 cycles of length 8. The time-varying convolutional code also has fewer

short cycles – an average of 0.39 cycles of length 6 and 1.29 cycles of length 8.

Thus, both convolutional codes have better cycle properties than the underlying

block code, as the analysis suggests.

Example 5.1.4 In the case of the randomly constructed protograph-based codes,

whose BER performance was plotted in Figure 4.3, the rate R = 1/2 block code

has girth 4 and it has an average of 0.02 cycles of length 4 and 0.87 cycles of

length 6, whereas the unwrapped time-varying convolutional code has an average

of 0.01 cycles of length 4 and 0.50 cycles of length 6.

From Examples 5.1.2-5.1.4, we see that many of the short cycles in the block

codes are broken to yield cycles of larger length by the time-varying and time-

invariant unwrappings. The better graph-cycle structures of the unwrapped con-

volutional codes, illustrated by the above examples, provide a possible explanation

for their improved performance, compared to the underlying block codes.

67

CHAPTER 6

MINIMUM DISTANCE GROWTH RATES OF DERIVED LDPC

CONVOLUTIONAL CODE FAMILIES

In this chapter, we use ensembles of tail-biting LDPC convolutional codes

derived from a protograph-based ensemble of LDPC block codes to obtain a lower

bound on the free distance of unterminated, asymptotically good, periodically

time-varying LDPC convolutional code ensembles, i.e., ensembles that have the

property of free distance growing linearly with constraint length.

6.1 An ensemble of protograph-based LDPC block codes

Irregular LDPC code ensembles are described in terms of their degree distri-

butions in the Tanner graph representation. For a code with maximum variable

(check) node degree dv (dc), the degree distribution polynomials are given by

λ(X) =dv∑i=2

λi ·X i−1, (6.1)

ρ(X) =dc∑i=2

ρi ·X i−1. (6.2)

Here, λi denotes the fraction of edges of degree i among the variable nodes, whereas

ρi denotes the fraction of edges of degree i among the check nodes. Degree distri-

butions are widely employed in the literature, since they can be used to calculate

68

many important properties of the code ensemble.

On the code construction side, however, knowing only the degree distribution

and the block length of a desired LDPC block code is not enough to describe the

code. This is because of the fact that these quantities themselves do not dictate

any structure on the code design. A very efficient solution to this problem was

proposed in the form of projected-graphs (protographs) in [44]. A protograph is

in simple terms a Tanner graph with a small number of nodes. However, how the

few nodes of this graph are connected is of great importance. It gives a structure

to the protograph and longer LDPC block codes that can be constructed based

on this protograph.

Suppose a given protograph has nv variable nodes and nc check nodes. An

ensemble of protograph-based LDPC block codes can be created by the copy-and-

permute operation [44]. The Tanner graph obtained for one particular member of

an ensemble created using this method is illustrated in Figure 6.1. In the copy-

and-permute operation, the protograph is first copied N times and the edges of the

resulting large graph are permuted. One important restriction on this operation

is that the edges can be permuted with ones that have the same connectivity.

This is demonstrated in Figure 6.1, where the connections of an edge between the

variable node “0” and check node “A” of the first copy can only be replaced with

another edge that has the same “0”-“A” connectivity.

The protograph parity-check matrix P corresponding to the protograph given

in Figure 6.1 can be written as

P =

!"1 1 0 00 1 1 11 1 1 0

#$.

The code created by applying the copy-and-permute operation to an nc × nv

69

0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3

0 1 2 3 0 1 2 3 0 1 2 3

A B C A B C A B C A B C

A B C A B C A B C

Figure 6.1. The copy-and-permute operation for a protograph.

protograph parity-check matrix P has block length n = Nnv. In addition, the

code has the same rate and degree distribution for each of its variable and check

nodes as the underlying protograph code. Therefore, the constructed LDPC block

code possesses the same iterative decoding threshold as the underlying protograph,

since the iterative decoding threshold is a function of only the degree distribution.

The parity-check matrix H corresponding to the entire ensemble of protograph-

based LDPC block codes can be obtained by replacing ones with N ×N permuta-

tion matrices and zeros with N×N all zero matrices in the underlying protograph

parity-check matrix P , where the permutation matrices are chosen randomly and

independently. This procedure can be shown on a Tanner graph as in Figure 6.2

According to this notation, the copy-and-permute operation of Figure 6.1 is

nothing else than a particular selection of the permutors on the Tanner graph

drawn in Figure 6.2.

If a variable node and a check node in the protograph are connected by r

70

!

!

!!

!

!

!

Figure 6.2. Tanner graph representation of a protograph-based LDPCblock code ensemble.

71

parallel edges, then the associated entry in P equals r and the corresponding

block of H consists of a summation of r N × N permutation matrices. The

sparsity condition of an LDPC parity-check matrix is thus satisfied for large N .

6.2 Protograph Weight Enumerators

Combinatorial methods of calculating ensemble average weight enumerators

have been presented in [45] and [46]. We now briefly describe the methods pre-

sented in [45].

In some code constructions, it may be desirable to “puncture” some variable

nodes, i.e., these variable nodes are not transmitted over the channel and at the

decoder side they assume zero a priori log-likelihood ratios. This is useful for

increasing the effective code rate. Suppose a protograph contains m variable

nodes to be transmitted over the channel and nv −m punctured variable nodes.

Also, suppose that each of the m transmitted variable nodes has an associated

weight di ∈ 0, 1. Let Sd = (d1, d2, . . . , dm) be the set of all possible weight

distributions such that d1 + . . . + dm = d, and let Sp be the set of all possible

weight distributions for the remaining punctured nodes. The ensemble weight

enumerator for the protograph is then given by

Ad =∑dk∈Sd

∑dj∈Sp

Ad, (6.3)

where Ad is the average number of codewords in the ensemble with a particular

weight distribution d = (d1, d2, . . . , dnv).

The normalized logarithmic asymptotic weight distribution of a code ensemble

can be written as r(δ) = limn→∞ sup rn(δ), where rn(δ) = ln(Ad)n

, δ = d/n, d is the

72

Hamming distance, n is the block length, and Ad is the ensemble average weight

distribution.

Suppose the first zero crossing of r(δ) occurs at δ = δmin. If r(δ) is negative in

the range 0 < δ < δmin, then δmin is called the minimum distance growth rate of

the code ensemble. By considering the probability

P(d < δminn) =

δminn−1∑d=1

Ad,

it is clear that, as the block length n grows, if P(d < δminn) << 1, then we can say

with high probability that the majority of codes in the ensemble have a minimum

distance that grows linearly with n and that the distance growth rate is δmin.

6.3 Free Distance Bounds

In this section we present a method for obtaining a lower bound on the free

distance of an ensemble of unterminated, asymptotically good, periodically time-

varying LDPC convolutional codes derived from protograph-based LDPC block

codes. To proceed, we will make use of a family of tail-biting LDPC convolutional

codes with incremental increases in block length. The tail-biting codes will be used

as a tool to obtain the desired bound on the free distance of the unterminated

codes.

6.3.1 Tail-biting convolutional codes

As a result of the very small size of protograph parity-check matrices, we

introduce a modified version of the unwrapping procedure for the purposes of this

chapter. Suppose that we have an nc × nv protograph parity-check matrix P ,

73

where gcd(nc, nv) = y. We then partition P as a y × y block matrix as follows:

P =

P1,1 . . . P1,y

......

Py,1 . . . Py,y

,

where each block Pi,j is of size nc/y× nv/y. P can thus be separated into a lower

triangular part, Pl, and an upper triangular part minus the leading diagonal, Pu.

Explicitly,

Pl =

P1,1

P2,1 P2,2

......

. . .

Py,1 Py,2 . . . Py,y

and Pu =

P1,2 . . . P1,y

. . ....

Py−1,y

, (6.4)

where blank spaces correspond to zeros.

Rearranging the positions of these two triangular matrices and repeating them

indefinitely results in a parity-check matrix Hcc of an unterminated, periodically

time-varying convolutional code with constraint length νs = nv and period T = 1

given by

Hcc =

Pl

Pu Pl

Pu Pl

. . . . . .

. (6.5)

Note that if gcd(nc, nv) = 1, we cannot form a square block matrix larger than

1 × 1 with equal size blocks. In this case, Pl = P and Pu is the all zero matrix

74

of size nc × nv. This trivial unwrapping results in a convolutional code with

syndrome former memory zero, with repeating blocks of the original protograph

on the leading diagonal. It is necessary in this case to create a larger protograph

parity-check matrix by using the copy and permute operation on P . This results

in an Mnc ×Mnv = n′c × n′v parity-check matrix for some small integer M . The

n′c × n′v protograph parity-check matrix can then be unwrapped following the

procedure outlined above. In effect, the choice of M × M permutation matrix

creates a mini ensemble of block codes suitable to be unwrapped to an ensemble

of convolutional codes.

We now introduce the notion of tail-biting convolutional codes by defining an

‘unwrapping factor’ λ as the number of times the sliding convolutional structure

is repeated. For λ > 1, the parity-check matrix H(λ)tb of the desired tail-biting

convolutional code can be written as

H(λ)tb =

Pl Pu

Pu Pl

Pu Pl

. . . . . .

Pu Pl

λnc×λnv

.

Note that the tail-biting convolutional code for λ = 1 is simply the original block

code.

6.3.2 A tail-biting LDPC convolutional code ensemble

Given a protograph parity-check matrix P , we generate a family of tail-biting

convolutional codes with increasing block lengths λnv, λ = 1, 2, . . ., using the

75

process described above. Since tail-biting convolutional codes are themselves block

codes, we can treat the Tanner graph of H(λ)tb as a protograph for each value of

λ. Replacing the entries of this matrix with either N × N permutation matrices

or N × N all zero matrices, as discussed in Section 6.2, creates an ensemble of

LDPC codes that can be analyzed asymptotically as N goes to infinity, where

the sparsity condition of an LDPC code is satisfied for large N . Each tail-biting

LDPC code ensemble, in turn, can be unwrapped and repeated indefinitely to

form an ensemble of unterminated, periodically time-varying LDPC convolutional

codes with constraint length νs = Nnv and period T = λN .

6.3.3 A free distance bound

Tail-biting convolutional codes can be used to establish a lower bound on the

free distance of an associated unterminated, periodically time-varying convolu-

tional code by showing that the free distance of the unterminated code is lower

bounded by the minimum distance of any of its tail-biting versions.

Theorem 6.3.1 Consider a rate R = (nv − nc)/nv unterminated, periodically

time-varying convolutional code with constraint length Nnv and period λN . Let

dmin(λNnv) be the minimum distance of the associated tail-biting convolutional

code with length λNnv and unwrapping factor λ > 0. Then the free distance dfree

of the unterminated convolutional code is lower bounded by dmin(λNnv) for any

unwrapping factor λ.

Proof: It can be shown that any codeword in a rate R = (nv − nc)/nv untermi-

nated, periodically time-varying convolutional code with constraint length Nnv

and period λN can be wrapped back to a codeword in a tail-biting convolutional

76

code of length λNnv, for any λ. The ‘wrapping back’ procedure consists of divid-

ing the convolutional codeword into segments of length λNnv and superimposing

these segments through a modulo summation. The Hamming weight of the re-

sulting codeword in the tail-biting code must be equal to or less than that of the

codeword in the unterminated code. The result is then obtained by wrapping back

the unterminated codeword with minimum Hamming weight. For a formal proof,

see [47].

A trivial corollary of the above theorem is that the minimum distance of a

protograph-based LDPC block code is a lower bound on the free distance of the as-

sociated unterminated, periodically time-varying LDPC convolutional code. This

can be observed by setting λ = 1.

6.3.4 The free distance growth rate

The distance growth rate δmin of a block code ensemble is defined as its min-

imum distance to block length ratio. For the protograph-based tail-biting LDPC

convolutional codes defined in Section 6.3.1, this ratio is therefore given as

δmin =dmin

n=

dmin

λNnv=dmin

λνs

. (6.6)

Using Theorem 6.3.1 we obtain

δmin ≤ dfree

λνs

, (6.7)

where dfree is the free distance of the associated unterminated, periodically time-

varying LDPC convolutional code. It is important to note that, for convolutional

codes, the length of the shortest codeword is equal to the encoding constraint

77

length νe, which in general differs from the decoding constraint length νs. Assum-

ing minimal encoder and syndrome former matrices, the relation between νe and

νs can be expressed as

νe =1−RR

νs, (6.8)

which implies that, for code rates less than 1/2, the encoding constraint length is

larger than the decoding constraint length.

Combining (6.7) and (6.8) gives us the desired bound on the free distance

growth rate

δfree ≥ R

1−Rλδmin, (6.9)

where δfree = dfree/νe is the convolutional code free distance growth rate1.

6.4 Distance Growth Rate Results

We now present distance growth rate results for several ensembles of asymp-

totically good LDPC convolutional codes based on protographs.

Example 6.4.1 Consider a (3, 6) regular LDPC code with the following proto-

graph:

.

For this example, the minimum distance growth rate is δmin = 0.023, as originally

shown by Gallager [17].

1If the syndrome former matrix is not in minimal form, (6.8) results in an upper bound onνe, which implies that δfree is underestimated in this case.

78

A family of tail-biting LDPC convolutional code ensembles can be generated

according to the following cut:

P =

!"1 1 1 1 1 11 1 1 1 1 11 1 1 1 1 1

#$.

For each λ, the minimum distance growth rate δmin was calculated for the tail-

biting LDPC convolutional codes using the approach outlined in Section 6.3.2.

These growth rates are shown in Figure 6.3, along with the corresponding lower

bound on the free distance growth rate δfree of the associated ensemble of untermi-

nated, periodically time-varying LDPC convolutional codes.

Intuitively, as λ increases, the tail-biting code becomes a better representation

of the associated unterminated convolutional code, with λ → ∞ corresponding to

the unterminated convolutional code itself. This is reflected in the weight enumer-

ators and increasing λ provides us with distance growth rates that converge to a

lower bound on the free distance growth rate of the unterminated convolutional

code. We observe that, once the unwrapping factor λ of the tail-biting convolu-

tional codes exceeds 3, the lower bound on δfree levels off at 0.086 and agrees with

the results presented in [47] and [48]. In this case, the minimum weight codeword

in the unterminated convolutional code also appears as a codeword in the tail-biting

code.

Example 6.4.2 The following irregular protograph is from the Repeat Jagged

Accumulate [49] (RJA) family. It was shown to have a good iterative decod-

ing threshold (γiter = 1.0 dB) while maintaining linear minimum distance growth

(δmin = 0.013). We display below the associated P matrix and cut used to generate

79

0

0.02

0.04

0.06

0.08

0.1

1 2 3 4 5 6 7 8

Dist

ance

gro

wth

rate

s fo

r !m

in a

nd !

free

Protograph unwrapping factor "

Lower bound to the convolutional growth rate !freeTail-biting growth rates !min

Figure 6.3. Distance growth rates for Example 6.4.1.

80

the family of tail-biting LDPC convolutional code ensembles.

!P =

!2 2 1 11 1 3 1

".

We observe that, as in Example 6.4.1, the minimum distance growth rates calcu-

lated for increasing λ provide us with a lower bound δfree ≥ 0.057 (for λ ≥ 5) on

the free distance growth rate of the convolutional code ensemble that is significantly

larger than the minimum distance growth rate δmin of the underlying block code

ensemble. These growth rates are shown in Figure 6.4.

81

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

1 2 3 4 5 6 7 8

Dis

tance g

row

th r

ate

s for !

min

and !

free




82

Example 6.4.3 The following irregular protograph is from the Accumulate Repeat

Jagged Accumulate family (ARJA) [49]:

,

where the empty circle represents a punctured variable node. This protograph is of

significant practical interest, since it was shown to have δmin = 0.015 and iterative

decoding threshold γiter = 0.628, i.e., pre-coding the protograph of Example 6.4.2

provides an improvement in both values.

In this ARJA example, the protograph matrix P is of size nc×nv = 3× 5. We

observe that gcd(nc, nv) = 1, and thus we have the trivial cut mentioned in Section

6.3.1. We must then copy and permute P to generate a mini ensemble of block

codes. Results are shown for one particular member of the mini ensemble with

M = 2, but a change in performance can be obtained by varying the particular

permutation chosen. Increasing λ for the chosen permutation results in a lower

bound δfree ≥ 0.0353 for λ ≥ 4. Again, we observe a significant increase in δfree

compared to δmin. These growth rates are shown in Figure 6.5.

Simulation results for LDPC block and convolutional codes based on the proto-

graph of Example 6.4.3 were obtained assuming BPSK modulation and an AWGN

channel. All decoders were allowed a maximum of 100 iterations, and the block

code decoders employed a syndrome-check based stopping rule. The resulting

BER performance is shown in Figure 6.6.

We note that the protograph-based tail-biting LDPC convolutional codes out-

perform the underlying protograph-based LDPC block code (which can also be

83

0

0.01

0.02

0.03

0.04

0.05

1 2 3 4 5 6 7 8

Dis

tance g

row

th r

ate

s for !

min

and !

free




84

10-5

10-4

10-3

10-2

10-1

100

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

BER

Eb/N0 (dB)

Protograph-Based LDPC-BC with n=10000Protograph-Based Tailbiting LDPC-BC with !s=10000 and "=2Protograph-Based Tailbiting LDPC-BC with !s=10000 and "=4Protograph-Based Tailbiting LDPC-BC with !s=10000 and "=8Protograph-Based LDPC-CC with !s=10000

Figure 6.6. Simulation results for Example 6.4.3.

85

seen as a tail-biting code with unwrapping factor λ = 1). Larger unwrapping fac-

tors yield improved error performance, eventually approaching the performance

of the unterminated convolutional code, which can be seen as a tail-biting code

with an infinitely large unwrapping factor. We also note that no error floor is

observed for the convolutional code, which is expected, since the code ensemble

is asymptotically good and has a relatively large (δfree ≥ 0.0353) distance growth

rate.

We also note that the performance of the unterminated LDPC convolutional

code is consistent with the iterative decoding threshold computed for the under-

lying protograph. At a moderate constraint length of 10000, the unterminated

code achieves 10−5 BER at roughly 0.12 dB away from the threshold, and with

larger block (constraint) lengths, the performance will improve even further. This

is expected, since both the unterminated and the tail-biting convolutional codes

preserve the same degree distribution as the underlying protograph.

86

CHAPTER 7

PSEUDO-CODEWORD STRUCTURE OF DERIVED LDPC

CONVOLUTIONAL CODES

Although LDPC block codes have very good performance under message-

passing iterative decoding, they are known to be subject to decoding failures due

to so-called pseudo-codewords. These are real-valued vectors that can be loosely

described as error patterns and cause non-convergence in iterative decoding due

to the fact that the algorithm works locally and can give priority to a vector that

fulfills the equations of a graph cover rather than the graph itself. Indeed, it has

been shown [20, 50, 51] that the performance of message-passing iterative decoding

schemes for LDPC block codes is largely dominated not by minimum Hamming

weight considerations but by minimum pseudo-weight considerations. The mini-

mum pseudo-weight is a measure of the effect that decoding failures have on the

performance of the code. The minimum Hamming weight may still be a key factor

in performance analysis; however, the larger the gap between minimum Hamming

weight and minimum pseudo-weight, the greater role the minimum pseudo-weight

plays. In any case, the minimum Hamming weight is important for quantifying

the impact of undetectable errors. As a consequence, the large SNR performance

of message-passing iterative decoding can be worse than that predicted by the

maximum-likelihood decoding union bound, which constitutes a major problem

when trying to determine performance guarantees.

87

The large SNR performance of iterative message passing decoding is a key as-

pect in code design, since error floors have prevented LDPC codes from gaining

acceptance in some applications. The presence of error floors in the performance

curves was first linked to the existence of near-codewords, trapping sets [52, 53],

and, in the case of iterative decoding for the BEC, stopping sets [54]. The computa-

tion tree and pseudo-codewords introduced in [20] and further developed in [51, 55]

offer a general setting for error-floor analysis over any memoryless channel.

In this chapter, we analyze the pseudo-codeword spectra of LDPC convolu-

tional codes derived from LDPC block codes. We start our analysis using time-

invariant LDPC convolutional codes derived from QC-LDPC block codes for ease

of presentation, since these codes have nice polynomial representations. We then

extend our results to time-varying LDPC convolutional codes.

7.1 Background and Notations

Pseudo-codewords [20, 50, 51, 55] are real valued vectors loosely described

as error patterns that cause non-convergence in iterative decoding. Pseudo-

codewords correspond to codewords, not of the original code, but of a finite cover

of the original graph.

The simple example below of the trivial [3,0]-code C = (0 0 0) defined by the

matrix H , 1|i−j| < 2, whose Tanner graph is drawn in Figure 7.1, will help to

visualize pseudo-codewords. The vector (110 110 110) is a valid codeword in the

cubic cover in Figure 7.2. This vector corresponds to the real pseudo-codeword

ω =(

23, 2

3, 2

3

)for the graph of the code C.1 In general, if G is an m-cover of a

1Locally operating decoding algorithms cannot distinguish if they are operating on a Tannergraph G or any finite cover of G, e.g., the cubic cover in Figure 7.2. The algorithm considersall possible codewords in covers of the original graph. I.e., the binary configuration in Figure7.2 competes for the best solution along with all other valid configurations in the union of all

88

Tanner graph G of a code C of length N , corresponding to a code C of length mN ,

and if c is a codeword in C, we define ωi(c) as ωi(c) , |l | ci,l=1|m

, i = 0, . . . , N−1,

i.e., the fraction of times a variable in G assumes the value 1. The real vector

ω(c) = (ω0(c), ω1(c), . . . , ωN−1(c)) is called a pseudo-codeword and it plays a

crucial role in characterizing the behavior of codewords in C [50].

c0

c1

c2

Figure 7.1. The Tanner graph of H , 1|i−j| < 2.

Example 7.1.1 Consider the trivial length-3 and dimension-0 code C = (0, 0, 0)defined by the 3× 3 parity-check matrix

H ,

1 1 0

1 1 1

0 1 1

(7.1)

covers.

89

and whose Tanner graph is shown in Figure 7.1. A possible cubic cover of this

Tanner graph is depicted in Figure 7.2. Because (1:1:0, 1:1:0, 1:1:0) is a valid

configuration in this cubic cover, the vector ω =(

23, 2

3, 2

3

)is a pseudo-codeword of

the code C defined by H (see [50, 56]).

c0,0

c0,1

c0,2

c1,0

c1,1

c1,2

c2,0

c2,1

c2,2

c0

c1

c2

0

1

1

0

1

1

0

1

1

Figure 7.2. (110 110 110) is a valid codeword in a cubic cover.

90

Throughout this chapter we mainly take the approach of [50, 56], which con-

nects the presence of pseudo-codewords in iterative message-passing decoding and

linear programming (LP) decoding. LP decoding was introduced by Feldman,

Wainwright, and Karger [57, 58] and can be seen as a relaxation of the maximum-

likelihood decoding problem. More precisely, maximum-likelihood decoding can

be formulated as the solution to an optimization problem, where a linear cost func-

tion is minimized over a certain polytope, namely the polytope that is spanned

by the set of all codewords. For general codes, there is no efficient description

of this polytope and so Feldman et al. suggested replacing it with an (efficiently

describable) relaxed polytope, which in the following will be called the “funda-

mental polytope”. In other words, the decoding result of the LP decoder is the

point in the fundamental polytope that minimizes the above-mentioned linear cost

function.

Throughout this chapter we will use the standard way of associating a Tan-

ner graph with a parity-check matrix and vice-versa. We will use the following

notation. We let F2, R, R+, and R++ be the Galois field of size 2, the field of

real numbers, the set of non-negative real numbers, and the set of positive real

numbers, respectively. If a(X) is a polynomial over some field and r is some

positive integer then(a(X) mod (Xr − 1)

)denotes the polynomial b(X) of degree

smaller than r such that b(X) = a(X) (mod (Xr − 1)). We say that a poly-

nomial ω`(D) =∑

i ω`,iDi with real coefficients is non-negative, and we write

ω`(D) ≥ 0, if all its coefficients ω`,i satisfy ω`,i ≥ 0. Similarly, a polynomial vector

ω(D) =(ω1(D), ω2(D), . . . , ωL(D)

)is non-negative, and we write ω(D) ≥ 0, if

all its polynomial components ω`(D) satisfy ω`(D) ≥ 0 for all ` ∈ 1, . . . , L.Moreover, a polynomial matrix A(D) is non-negative, and we write A(D) ≥ 0, if

91

all its entries are non-negative polynomials. Finally, for any positive integer r and

for any r′ ∈ 0, 1, . . . , r − 1, Ir′ will represent the r′-times cyclically left-shifted

identity matrix of size r × r.

7.2 Pseudo-Codeword Structures of Time-Invariant LDPC Convolutional Codes

Derived From QC-LDPC Block Codes

As a motivational example we reuse the simulation results of a rate R = 2/5

(3, 5)-regular LDPC convolutional code with constraint length νs = 145, together

with three wrapped block code versions with parity-check matrices of increasing

circulant sizes r = 31, r = 48, and r = 80, respectively, while keeping the same

structure within each r×r circulant. (Note that increasing the circulant size of the

QC code increases its complexity, i.e., its block length. Also note that each of the

three block codes has rate slightly greater than 2/5.) The sliding-window iterative

message-passing decoder was used to decode the convolutional code. Conventional

LDPC block code iterative message-passing decoders were employed to decode the

QC-LDPC block codes. All decoders were allowed a maximum of 50 iterations.

The resulting BER performance of these codes on a binary-input AWGN channel

is shown in Figure 4.1.

We note that, particularly in the low-to-moderate SNR region, where the com-

plete pseudo-weight spectrum plays an important role, the unwrapped LDPC con-

volutional code performs between 0.5 dB and 1.0 dB better than the associated

QC-LDPC block codes. Also, as the circulant size increases, the performance of

the block codes approaches that of the convolutional code. These performance

curves suggest that the pseudo-codewords in the block code that result in decod-

ing failures may not cause such failures in the convolutional code, which suggests

92

that LDPC convolutional codes may have better iterative decoding thresholds

than comparable LDPC block codes (see also [59] and [60]).

In order to underline the influence of pseudo-codewords under iterative message-

passing decoding, we consider the following experiment. Let ω be a minimal

pseudo-codeword [56] for the above-mentioned R = 2/5 (3, 5)-regular LDPC con-

volutional code, i.e., a pseudo-codeword that corresponds to an edge of the fun-

damental cone of that LDPC convolutional code. Moreover, we define the log-

likelihood ratio vector λ(α, β) to be2

λ(α, β) , β ·(

1− 2α‖ω‖1‖ω‖22

ω

).

We then run the iterative message-passing decoder that is initialized with λ =

λ(α, β) and count how many iterations it takes until the decoder decides for the

all-zero codeword as a function of α and β. The results are shown in Figures 7.3

and 7.4.

The meaning of λ(α, β) is the following. If α = 0, then λ corresponds to the

log-likelihood ratio vector that the receiver sees when the communication system

operates at a signal-to-noise ratio Eb/N0 = β/(4R) and when the noise vector that

is added by the binary-input AWGN channel happens to be the all-zero vector

(see, e.g., the discussion in [56, Sec. 3]). For non-zero α the expression for λ(α, β)

has been set up such that the LP decoder has a decision boundary at α = 0.5:

for α < 0.5 the all-zero codeword wins against the pseudo-codeword ω whereas

for α > 0.5 the all-zero codeword loses against the pseudo-codeword ω under LP

decoding.

2Note that the numerator in the ratio ‖ω‖1/‖ω‖22 is not squared, therefore this ratio doesnot correspond to the AWGN channel pseudo-weight of ω (see Sec. 7.2.1), although it is closelyrelated to that value.

93

0 0.1 0.2 0.3 0.4 0.50

500

1000

#ite

r @ !

= 1

0 0.1 0.2 0.3 0.4 0.50

500

1000

#ite

r @ !

= 2

0 0.1 0.2 0.3 0.4 0.50

500

1000

#ite

r @ !

= 3

0 0.1 0.2 0.3 0.4 0.50

500

1000

#ite

r @ !

= 4

0 0.1 0.2 0.3 0.4 0.50

500

1000#i

ter @

! =

5

"

Figure 7.3. Number of iterations needed for asum-product-algorithm-type iterative message-passing decoder to decidefor the all-zero codeword. (1000 corresponds to no convergence.) Herethe β values 1.00, 2.00, 3.00, 4.00, 5.00 correspond to signal-to-noiseratios Eb/N0 of, respectively, −2.04 dB, 0.96 dB, 2.73 dB, 3.97 dB,

4.94 dB.

0.4 0.41 0.42 0.43 0.44 0.45 0.46 0.47 0.48 0.49 0.50

200

400

600

800

1000

1200

1400

1600

1800

2000

!

#iterations

Figure 7.4. Number of iterations needed for a min-sum-algorithm-typeiterative message-passing decoder to decide for the all-zero codeword.

(2000 corresponds to no convergence.) We did not observe convergenceto the all-zero codeword for α > 0.50 either for the

sum-product-algorithm-type iterative message-passing decoder or for themin-sum-algorithm-type iterative message-passing decoder.

94

The simulations in Figure 7.3 and 7.4 were obtained using a search algorithm

that looked for a low-pseudo-weight minimal pseudo-codeword ω in the funda-

mental cone of the above-mentioned LDPC convolutional code. The ω that we

found has AWGN channel pseudo-weight 18.1297, which happens to be smaller

than the free distance. Secondly, we ran the iterative message-passing decoder

for various choices of α and β: Figure 7.3 shows the number of iterations needed

using a sum-product-algorithm-type iterative message-passing decoder whereas

Figure 7.4 shows the number of iterations needed using a min-sum-algorithm-type

iterative message-passing decoder. (Note that the decisions reached by the latter

are independent of the choice of β, β > 0.) Because of the more oscillatory be-

havior of the min-sum-algorithm-type iterative message-passing decoder close to

decision boundaries of the LP decoder, observed empirically, it is advantageous to

run that decoder for many iterations in our scenario, whereas in the case of the

sum-product-algorithm-type iterative message-passing decoder it hardly pays to

go beyond 150 iterations.

In this section, we provide a possible explanation for the performance difference

observed in the motivational example above. Based on the results of [50, 56] that

relate code performance to the existence of pseudo-codewords, we examine the

pseudo-codeword weight spectra of QC-LDPC block codes and their associated

convolutional codes. We will show that for a non-trivial LDPC convolutional code

derived by unwrapping a non-trivial QC-LDPC block code,3 the minimum pseudo-

weight of the convolutional code is at least as large as the minimum pseudo-weight

3Non-trivial means here that the set of pseudo-codewords contain non-zero pseudo-codewords.

95

of the underlying QC code[6], i.e.,

wminp

(H

(r)QC

)≤ wmin

p (Hconv) .

This result, which parallels the well-known relationship between the free Ham-

ming distance of non-trivial convolutional codes and the minimum Hamming dis-

tance of their non-trivial quasi-cyclic counterparts [35],4 is based on the fact that

every pseudo-codeword in the convolutional code induces a pseudo-codeword in

the block code with pseudo-weight no larger than that of the convolutional code’s

pseudo-codeword. This difference in the weight spectra leads to improved BER

performance at low-to-moderate SNRs for the convolutional code, a conclusion

supported by the simulation results presented in Figure 4.1.

We first turn our attention to codewords in these codes. Codewords in C(r)QC

will be denoted by a polynomial vector like c(X) and codewords in Cconv will be

denoted by a polynomial vector like c(D) (when the code is represented by a scalar

parity-check matrix Hconv, then it will be denoted by a vector like c). (We will

later use a similar notation for pseudo-codewords.)

For any non-zero codeword c(D) with finite support in the convolutional code,

its r wrap-around, defined by the vector c(X) mod (Xr−1) ∈ (F2[X]/〈Xr−1〉)L, is

a codeword in the associated QC-code, since H(r)QC(X) ·c(X) = 0 in

(F2[X]/〈Xr−

1〉)L.

In addition, the Hamming weight of the two codewords is linked by the fol-

lowing inequality wH

(c(X) mod (Xr − 1)

) ≤ wH(c(D)), which gives the inequal-

4Non-trivial means here that the set of codewords contain non-zero codewords.

96

ity [35, 36] (we assume all codes are non-trivial)

dmin

(C

(r)QC

)≤ dfree (Cconv) for all r ≥ ms + 1.

Moreover, we have the following inequalities.

Lemma 7.2.1

dmin

(C

(r)QC

)≤ dmin

(C

(2r)QC

)≤ dmin

(C

(4r)QC

)≤ · · · for all r ≥ ms + 1,

and

limr→∞

dmin

(C

(r)QC

)= dfree (Cconv) .

Proof: Let H(2r)QC (X) be a parity-check matrix of C

(2r)QC and H

(r)QC(X) , H

(2r)QC (X)

mod (Xr − 1) be the corresponding parity-check matrix of C(r)QC. Let c(X) be a

nonzero codeword in C(2r)QC of weight equal to the minimum distance dmin

(C

(2r)QC

).

We have H(2r)QC (X) · c(X) = 0 in

(F2[X]/〈X2r − 1〉)L and since Xr − 1|X2r − 1, it

follows that H(r)QC(X) · c(X) = 0 in

(F2[X]/〈Xr − 1〉)L.

If c(X) mod (Xr−1) 6= 0 then we obtain the inequality dmin

(C

(r)QC

)≤ dmin

(C

(2r)QC

).

If c(X) = 0 mod (Xr − 1) we can write c(X) = (Xr − 1)c1(X) with c1(X) 6=0 mod (Xr − 1) (otherwise c(X) = 0 mod (X2r − 1).) Hence H

(r)QC(X) · (Xr −

1) · c1(X) = 0 mod (X2r − 1) or equivalently, H(r)QC(X) · c1(X) = 0 mod (Xr − 1),

which implies that c1(X) is a nonzero codeword in C(r)QC. The Hamming weight

dmin

(C

(2r)QC

)= wH

(c(X)

)= wH

((Xr − 1)c1(X)

) ≥ 2 · wH(c1(X) mod (Xr − 1)) >

wH(c1(X) mod (Xr − 1)) ≥ dmin

(C

(r)QC

). Mimicking this proof we also obtain

97

dfree (Cconv) ≥ dmin

(C

(r)QC

), and hence the desired inequality

dmin

(C

(r)QC

)≤ dmin

(C

(2r)QC

)≤ dmin

(C

(4r)QC

)≤ · · · ≤ dfree (Cconv) , for all r ≥ ms + 1.

From the way we construct the semi-infinite sliding matrix of the convolutional

by unwrapping the scalar parity-check matrix of the QC block code versions, we

can see that there exists a QC code of circulant size r large enough, so that its

minimum distance is equal to the free distance of the convolutional code. This

assures the limit equality above.

If we denote by G , G(r), G(2r), G(4r), . . . the Tanner graphs associated with

the parity-check matrices of C(r)QC, C

(2r)QC , C

(4r)QC , . . . of increasing size (which are

associated with the same polynomial matrix H(D)), it is easy to see that we obtain

a tower of covers of the graph G. The above relationship between the minimum

distances of the codes in the tower is then easily verified in graph language, since a

codeword c(D) in a larger graph, say C(4r)QC , when projected onto the graphs of C

(r)QC

and C(2r)QC using the formula c(X) mod (Xr − 1), respectively, c(X) mod (X2r − 1),

gives another codeword. Finally, the graph of the associated convolutional code

is an infinite cover of each of the graphs in the tower.

Example 7.2.2 Using the graphs associated with the trivial code C = (0, 0, 0)and its cubic cover C(3) having parity-check matrix H

(3)QC (see Example 7.1.1)

corresponding to the codeword (0, 0, 0) in C, we can identify the codeword c ,

(1:1:0, 1:1:0, 1:1:0) ∈ Nullsp(H(3)QC) in C(3). The polynomial description of c is

c(X) , (1 + X, 1 + X, 1 + X) ∈ Nullsp(H(3)QC(X)). The wrap-around of c(X)

mod (X − 1), c(X) mod (X − 1) ∈ (F2[X]/〈Xr − 1〉)3, r = 1, gives the codeword

(0, 0, 0) in C. The associated convolutional code is trivially zero.

98

7.2.1 The fundamental cone

In this section we introduce our main object of study, the fundamental cone

of a matrix, a set that contains all relevant pseudo-codewords [20, 50, 56–58, 61].

Definition 7.2.3 (see [50, 56–58]) Let H be a binary matrix of size m× n, let

I , I(H) , 0, . . . , n−1 be the set of column indices of H, and let J , J (H) ,

0, . . . ,m − 1 be the set of row indices of H, respectively. For each j ∈ J , we

let Ij , Ij(H) ,i ∈ I | hji = 1

be the support of the j-th row of H. The

fundamental polytope P , P(H) of H is then defined as [50, 56]

P ,m⋂j=1

ConvexHull(Cj) with Cj ,c ∈ 0, 1n ∣∣ rjc

T = 0 mod 2,

where rj is the j-th row of H, and ConvexHull(Cj) is the convex hull of Cj, defined

as the set of convex combinations of points in Cj when seen as points in Rn. The

fundamental cone K(H) of H is defined as the conic hull of the fundamental

polytope P, which includes the vertex 0, stretched to infinity. Note that if ω ∈K(H), then also α · ω ∈ K(H) for any real α > 0. Moreover, for any ω ∈ K(H),

there exists an α > 0 (in fact, a whole interval of α’s) such that α · ω ∈ P(H).

Vectors in P(H) are called pseudo-codewords of H, and we will call any vector

in K(H) a pseudo-codeword and two pseudo-codewords that are equal up to a

positive scaling constant will be considered to be equivalent. Clearly, not all

pseudo-codewords are codewords, but all codewords are pseudo-codewords.

The fundamental cone and polytope can both be described by certain sets of

inequalities that are computationally useful [56, 58]. Since our study of pseudo-

codewords will rely heavily on the fundamental cone, we now describe it in more

detail.

99

A vector ω = (ω1, . . . , ωn) ∈ Rn is in the fundamental cone K , K(H) if and

only if

ωi ≥ 0 for all i ∈ I(H), (7.2)∑i∈(Ij\i′)

ωi ≥ ωi′ for all j ∈ J (H) and for all i′ ∈ Ij(H). (7.3)

Example 7.2.4 Let H , [1, 1, 1] be a parity-check matrix for the code C =

(0, 0, 0), (1, 1, 0), (1, 0, 1), (0, 1, 1). The fundamental polytope P(H) associated

with H is (see Figure 7.5)

P(H) =

(ω1, ω2, ω3) ∈ R3

∣∣∣∣∣∣∣∣∣∣∣∣∣

0 ≤ ω1 ≤ 1

0 ≤ ω2 ≤ 1

0 ≤ ω3 ≤ 1

−ω1 + ω2 + ω3 ≥ 0

+ω1 − ω2 + ω3 ≥ 0

+ω1 + ω2 − ω3 ≥ 0

+ω1 + ω2 + ω3 ≤ 2

,

from which we obtain the fundamental cone K(H) associated with H:

K(H) =

(ω1, ω2, ω3) ∈ R3

∣∣∣∣∣∣∣∣∣∣0 ≤ ω1

0 ≤ ω2

0 ≤ ω3

−ω1 + ω2 + ω3 ≥ 0

+ω1 − ω2 + ω3 ≥ 0

+ω1 + ω2 − ω3 ≥ 0

.

We see that the pseudo-codeword ω =(

23, 2

3, 2

3

)satisfies the inequalities of both the

fundamental polytope and the fundamental cone. Also, the pseudo-codeword ω =

(1, 1, 1) and its positive multiples are in the fundamental cone but not necessarily

in the fundamental polytope.

As briefly explained earlier, pseudo-codewords can also be described using

graph theory language as vectors corresponding to codewords in a cover graph.

100

!2

!1

!3

(0, 1, 1)

(1, 0, 1)

(1, 1, 0)

(0, 0, 0)

Figure 7.5. The fundamental polytope for H = [1 1 1].

101

Looking again at the cover tower G , G(r), G(2r), G(4r), . . ., a codeword c(D) of

a larger graph, say C(4r)QC , seen as a vector with real components, when projected

over R onto the graphs of C(r)QC and C

(2r)QC , using the formula c(X) mod (Xr − 1),

respectively, c(X) mod (X2r− 1), gives a pseudo-codeword. Similarly, a codeword

c(D) of the unwrapped convolutional code Cconv, when projected onto the graphs

of C(r)QC C

(2r)QC , C

(4r)QC , . . . , using the formulas c(X) mod (Xr−1), c(X) mod (X2r−1),

c(X) mod (X4r − 1), . . ., gives pseudo-codewords in the QC codes.

Example 7.2.5 Let c , (1:1:0, 1:1:0, 1:1:0) ∈ Nullsp(H(3)QC) be the codeword

identified in Example 7.2.2 for the code C(3), with corresponding polynomial de-

scription c(X) , (1+X, 1+X, 1+X) ∈ Nullsp(H(3)QC(X)). When projecting c(X)

over R onto the graph of C = (0, 0, 0) we obtain c(X) mod (X − 1) = (2, 2, 2),

which is a pseudo-codeword in C, equivalent to the pseudo-codeword ω =(

23, 2

3, 2

3

)presented in Example 7.1.1.

In the above example we made the following important connection between

two graphical ways of representing pseudo-codewords ω of a QC code C of length

rL as vectors corresponding to codewords c(X) in some QC code (or convolutional

code) associated with an m-cover (or an infinite cover) of the graph of C. We can

see them as vectors with each entry representing an average of the ones attributed

to the variable nodes of the cover graph that are in the “cloud” of that entry,

e. g., ω =(

23, 2

3, 2

3

), corresponding to pseudo-codewords in the polytope of C, or

we can see them as vectors that are the projections of c(X) onto the code C by the

formula c(X) mod (Xr−1), e.g., ω = (2, 2, 2), corresponding to pseudo-codewords

in the fundamental cone of C that are not necessarily in the polytope.

In fact a similar statement can be made about pseudo-codewords in any block

code C of length rL, not having necessarily a QC structure. A pseudo-codeword

102

ω in C is a vector corresponding to some codeword c in some block code (or

convolutional code) associated with an m-cover (or an infinite cover) of the graph

of C with each entry representing an average of the ones attributed to the variable

nodes of the cover graph that are in the “cloud” of that entry. However, this

is the same, under the equivalence between different representations of a pseudo-

codeword in the fundamental cone, as defining a pseudo-codeword ω as a vector for

which there exists a cover corresponding to a block code or to a convolutional code,

C, of the code C, and there exists some codeword, c in C, such that ω = c mod (rL)

where the vector c is equal to a permutation of the vector c (where the components

of c are taken in the following order 1,m+ 1, . . . , (L− 1)m+ 1, 2,m+ 2, . . . , (L−1)m + 2, . . . ,m − 1, 2m − 1, . . . , Lm − 1). The vector c it is seen as a vector

with real entries and the modular operation mod (rL) is performed over the real

numbers.

The above description of the fundamental polytope and cone provides compu-

tationally easy descriptions of the pseudo-codewords, especially in the case of QC

and convolutional codes. From (7.2) and (7.3) it follows that for any parity-check

matrix H there is a matrix K such that a vector ω is a pseudo-codeword in the

fundamental cone K(H) if and only if K ·ωT ≥ 0T. In the case of QC and convo-

lutional codes, the fundamental cone can be described with the help of polynomial

matrices. Namely, for any H(r)QC(X) there is a polynomial matrix K

(r)QC(X) such

that ω(X) ∈ K(H(r)QC(X)

)if and only if K

(r)QC(X) ·ω(X)T ≥ 0T. Similarly, for any

Hconv(D) there is a polynomial matrix Kconv(D) such that ω(D) ∈ K(Hconv(D))

if and only if Kconv(D) · ω(D)T ≥ 0T.

This fundamental cone description is particularly simple and hence useful in

the case of monomial parity-check matrices Hconv (see [38]), since the fundamen-

103

tal cone inequalities can then be stated very simply as follows. A vector ω is a

pseudo-codeword in the fundamental cone of a monomial matrix Hconv if and only

if the associated polynomial vector ω(D) =(ω1(D), ω2(D), . . . , ωL(D)

)satisfies:

ω`(D) ≥ 0, ∀` ∈ 1, . . . , L,∑`∈1,...,L\`′

hj`(D)ω`(D) ≥ hj`′(D)ω`′(D), ∀j ∈ 1, . . . , J, ∀`′ ∈ 1, . . . , L.

Equivalently, if Ω(D) is an L× L matrix with `-th row vector

(+ω1(D) · · · +ω`−1(D) −ω`(D) +ω`+1(D) · · · ωL(D)

)

for j ∈ 1, . . . , L, then ω(D) is a pseudo-codeword if and only if ω`(D) ≥ 0 for

all ` ∈ 1, . . . , L and

Hconv(D) ·Ω(D)T ≥ 0.

Example 7.2.6 Let

Hconv(D) ,

1 1 1 1

1 D D2 D3

1 D4 D3 D2

be a polynomial parity-check matrix of a rate-1/4 convolutional code. The following

vector

ω(D) ,

(3D2 +D3 4D +D2 3 +D + 4D2 +D3 3 + 4D +D2

)=

(0 0 3 3

)+

(0 4 1 4

)D +

(3 1 4 1

)D2 +

(1 0 1 0

)D3,

104

corresponding to the scalar vector

ω = (0, 0, 3, 3, 0, 4, 1, 4, 3, 1, 4, 1, 1, 0, 1, 0, · · · ) ,

is a pseudo-codeword for the convolutional code since ω(D) ≥ 0 and the matrix

Ω(D) ,

−3D2 −D3 +4D +D2 +3 +D + 4D2 +D3 +3 + 4D +D2

+3D2 +D3 −4D −D2 +3 +D + 4D2 +D3 +3 + 4D +D2

+3D2 +D3 +4D +D2 −3−D − 4D2 −D3 +3 + 4D +D2

+3D2 +D3 +4D +D2 +3 +D + 4D2 +D3 −3− 4D −D2

gives

Hconv(D) ·Ω(D)T =

6 + 9D + 3D2 4D2 + 4D3 + 8D4 + 2D5 6D3 + 2D4 + 8D5 + 2D6

6 +D +D2 + 2D3 2D2 + 4D3 + 8D4 + 2D5 6D2 + 8D3 + 2D4 + 2D6

7D +D2 4D2 + 4D3 6D2 + 2D3

D + 7D2 + 2D3 10D2 8D5 + 2D6

T

≥ 0,

i.e., Hconv(D) ·Ω(D)T has all polynomial entries with non-negative coefficients.

Similarly, for a QC code that is described by a monomial J × L parity-

check matrix H(r)QC(X), a vector ω(X) is a pseudo-codeword in the correspond-

ing fundamental cone if and only if the associated polynomial vector ω(X) =(ω1(X), ω2(X), . . . , ωL(X)

)satisfies

(ω`(X) mod (Xr − 1)

) ≥ 0 for all ` ∈ 1, . . . , L,

105

and

(H

(r)QC(X)Ω(X)T mod (Xr − 1)

)≥ 0.

Example 7.2.7 Let H(5)QC(X) be the r = 5 parity-check matrix obtained from

Hconv(D) in Example 7.2.6 for a QC block code of length n = 20. Then, for the

polynomial vector

ω(5)(X) ,

(3X2 +X3 4X +X2 3 +X + 4X2 +X3 3 + 4X +X2

),

we obtain

H(5)QC(X)Ω(X)T mod (X5 − 1)

=

6 + 9X + 3X2 2 + 4X2 + 4X3 + 8X4 8 + 2X + 6X3 + 2X4

6 +X +X2 + 2X3 2 + 2X2 + 4X3 + 8X4 2X + 6X2 + 8X3 + 2X4

7X +X2 4X2 + 4X3 6X2 + 2X3

X + 7X2 + 2X3 10X2 8 + 2X

T

.

Since ω(5)(X) ≥ 0 and (H(5)QC(X)Ω(X)T mod (X5−1)) ≥ 0, we know that ω(5)(X)

is a pseudo-codeword.

Projecting the pseudo-codeword ω(D) = (3D2 +D3, 4D+D2, 3 +D+ 4D2 +

D3, 3 + 4D + D2) in the convolutional code onto the r-wrapped QC code, for

r = 5, we obtained the pseudo-codeword ω(X) = (3X2 +X3, 4X +X2, 3 +X +

4X2 + X3, 3 + 4X + X2) in the QC code. This is not a mere coincidence as the

following lemma shows.

Lemma 7.2.8 Let ω(D) be a pseudo-codeword in the convolutional code defined

106

by Hconv(D), i.e., ω(D) ∈ K(Hconv(D)). Then its r wrap-around polynomial

vector is a pseudo-codeword in the associated QC-code defined by H(r)QC(X), i.e.,

ω(X) mod (Xr − 1) ∈ K(H(r)QC(X)).

Proof: We have seen that for any Hconv(D) describing a convolutional code there is

a matrix Kconv(D) such that a polynomial vector ω(D) is a pseudo-codeword in the

fundamental cone K(Hconv(D)) if and only if Kconv(D)ω(D)T ≥ 0T. By reducing

Kconv(D) modulo Dr−1, we obtain a matrix K(r)QC(X) = (Kconv(X) mod (Xr − 1))

with the property that a polynomial vector ω(X) is a pseudo-codeword in the

fundamental cone K(H(r)QC(X)) if and only if K

(r)QC(X)ω(X)T ≥ 0T. Reducing

Kconv(D)ω(D)T ≥ 0T modulo Dr−1, we obtain K(r)QC(X)(ω(X) mod (Xr−1))T ≥

0T, which proves the claim.

This result can be easily deduced using the graph-theory language. Indeed,

looking again at the cover tower G , G(r),G(2r),G(4r), . . ., a pseudo-codeword ω(D)

of a larger graph, say C(4r)QC , when projected onto the graphs of C

(r)QC and C

(2r)QC ,

using the formula ω(X) mod (Xr − 1), respectively, ω(X) mod (X2r − 1), gives

another pseudo-codeword. Similarly, a pseudo-codeword ω(D) of the unwrapped

convolutional code Cconv, when projected onto the graphs of C(r)QC C

(2r)QC , C

(4r)QC , . . . ,

using the formulas ω(X) mod (Xr − 1), ω(X) mod (X2r − 1), ω(X) mod (X4r −1), . . . , gives pseudo-codewords in the QC codes.

Example 7.2.9 The pseudo-codeword

ω(D) ,

(3D2 +D3 4D +D2 3 +D + 4D2 +D3 3 + 4D +D2

)

of the convolutional code in Example 7.2.6 projects onto the following pseudo-

codewords in the QC wrapped block code versions of parity-check matrix H(r)QC(X) =

107

Hconv(X) mod (Xr − 1), for all r ≥ 0:5

ω(r)(X) , ω(D) mod (Xr − 1)

,

(3X2 +X3 4X +X2 3 +X + 4X2 +X3 3 + 4X +X2

),

for all r ≥ 4, (r = 5 case above is included, )

ω(3)(X) , ω(D) mod (X3 − 1)

,

(1 + 3X2 4X +X2 4 +X + 4X2 3 + 4X +X2

),

ω(2)(X) , ω(D) mod (X2 − 1)

,

(3 +X 1 + 4X 7 + 2X 4 + 4X

),

ω(1)(X) , ω(D) mod (X − 1)

,

(4 5 9 8

).

We now define various channel pseudo-weights and continue with the main

result that the minimum AWGN channel, BEC, BSC, and max-fractional pseudo-

weights of a convolutional code are at least as large as the corresponding pseudo-

weights of a wrapped QC block code.

Definition 7.2.10 [20, 50, 51, 56–58] Let ω = (ω0, . . . , ωn−1) be a nonzero

vector in Rn+. The AWGN channel pseudo-weight and the BEC pseudo-weight of

the vector ω are defined to be, respectively,

wAWGNCp (ω) ,

‖ω‖21‖ω‖22

, wBECp (ω) =

∣∣ supp(ω)∣∣,

5If the convolutional code has a monomial polynomial parity-check matrix then we do notneed the earlier assumption r ≥ ms + 1, as Lemma 7.2.8 holds for each r ≥ 1.

108

where ‖ω‖1 and ‖ω‖2 are the 1-norm, respectively 2-norm, of ω, and supp(ω) is

the set of all indices i corresponding to nonzero components ωi of ω. In order to

define the BSC pseudo-weight wBSCp (ω), we let ω′ be the vector of length n with

the same components as ω but in non-increasing order. Now let

f(ξ) , ω′i (i < ξ ≤ i+ 1, 0 < ξ ≤ n), F (ξ) ,∫ ξ

0

f(ξ′) d ξ′, and e , F−1

(F (n)

2

).

Then the BSC pseudo-weight wBSCp (ω) is wBSC

p (ω) , 2e. Finally, the fractional

weight of a vector ω ∈ [0, 1]n and the max-fractional weights of a vector ω ∈ Rn+

are defined to be, respectively,

wfrac(ω) = ‖ω‖1, wmax−frac(ω) ,‖ω‖1‖ω‖∞ ,

where ‖ω‖∞ is the infinite or max norm. For ω = 0 we define all of the above

pseudo-weights, fractional weights, and max-fractional weights to be zero.

A discussion of the motivation and significance of these definitions can be found

in [56]. Note that whereas the fractional weight has an operational meaning only

for vertices of the fundamental polytope, the other measures have an operational

meaning for any vector in the fundamental polytope or cone. Note also that here

wfrac and wmax−frac are defined for any vector in Rn+, whereas wfrac and wmax−frac

in [57] are what we will call wminfrac and wmin

max−frac.

Example 7.2.11 Let

ω(D) =

(3D2 +D3 4D +D2 3 +D + 4D2 +D3 3 + 4D +D2

)

109

be the pseudo-codeword in Example 7.2.6 and let

ω = (0, 0, 3, 3, 0, 4, 1, 4, 3, 1, 4, 1, 1, 0, 1, 0, 0, 0, 0, 0, . . .)

be its scalar vector description. Then

wAWGNCp (ω) =

‖ω‖21‖ω‖22

=(3 · 4 + 3 · 3 + 5 · 1)2

3 · 16 + 3 · 9 + 5 · 1 = 8.45, wBECp (ω) = | supp(ω)| = 11.

In order to compute wBSCp (ω), we let

ω′ = (4, 4, 4, 3, 3, 3, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, . . .)

be the vector that lists the components of ω in non-increasing order. We obtain

wBSCp (ω) = 2e = 20

3= 6.67, since we need to add up e = 10

3ordered components

of ω′ to obtain ‖ω‖12

= 13. Finally ‖ω‖∞ = max16i=1 ωi, from which it follows that

wmax−frac(ω) = ‖ω‖1‖ω‖∞ = 26

4= 6.5.

A measure of the effect that the pseudo-codewords have on the performance

of a code is given by the minimum pseudo-weight [50, 56–58]

wminp (H) , min

ω∈V(P(H))\0wp(ω),

where V(P(H))\0 is the set of all non-zero vertices in the fundamental polytope

P(H) and the pseudo-weights are the appropriate ones for each channel (AWGN

channel, BSC, and BEC pseudo-weights) and the minimum fractional and max-

fractional weights.

Computing these values can be quite challenging, since the task of finding

the set of vertices of P(H) is in general very complex. However, in the case of

110

four of the above pseudo-weights (the minimum AWGN channel, BSC, and BEC

pseudo-weights and the minimum max-fractional weight) there is a computation-

ally simpler description given by

wminp (H) = min

ω∈K(H)\0wp(ω),

for the appropriate pseudo-weight. (Note that there is no such statement for the

minimum fractional weight; see, e.g., [56, 58]).

Example 7.2.12 Let H(5)QC(X) be the matrix of Example 7.2.7 and let C

(r)QC be

the QC block code of length n = 20, with r = 5. The minimum distance is

6 and it is equal to the wAWGNC,minp (H

(5)QC). (This was obtained using a vertex

enumeration program for polytopes that lists all the minimal pseudo-codewords of

a code, see [62].)

In [56] it was shown that for any code defined by a parity-check matrix H, the

following inequalities hold:

wminfrac(H) ≤ wmin

max−frac(H)≤ wAWGNC,minp (H) ≤ wBEC,min

p (H), (7.4)

wminfrac(H) ≤ wmin

max−frac(H) ≤ wBSC,minp (H) ≤ wBEC,min

p (H). (7.5)

Therefore, wminfrac(H) and wmin

max−frac(H) can serve as lower bounds for wAWGNC,minp (H),

wBSC,minp (H), and wBEC,min

p (H).

7.2.2 Minimum pseudo-weights

In what follows, we compare the minimum pseudo-weights and minimum max-

fractional weight of a QC block code to those of its corresponding convolutional

111

code which we assume to have a fundamental cone containing non-zero vectors. In

order to analyze the minimum pseudo-weight and minimum max-fractional weight,

it is sufficient to analyze the weights of the non-zero vectors in the fundamental

cone. Throughout this section, without loss of generality, all pseudo-codewords

ω(D) are assumed to have finite support.6

Theorem 7.2.13 For the AWGN channel, BEC, and BSC pseudo-weights, if

ω(D) ∈ K(Hconv(D)), then

wp

(ω(X) mod (Xr − 1)

) ≤ wp

(ω(D)

).

Therefore, if the fundamental cone of the convolutional code is not trivial (i.e., it

contains non-zero vectors) we obtain

wminp

(H

(r)QC(X)

)≤ wmin

p

(Hconv(D)

).

Proof: In the following, we need to analyze separately the AWGN channel, BEC,

and BSC pseudo-weights of ω(D) and of its r wrap-around ω(X) mod (Xr − 1).

Let ω(D) =(ω1(D), . . . , ωL(D)

)be a pseudo-codeword. By assumption ω(D)

has finite support, i.e., there exists an integer t such that the maximal degree of

any ω`(D), ` ∈ 1, . . . , L, is smaller than t.

6With suitable modifications, this can easily be generalized to ω(D) with ‖ω(D)‖1 < ∞.Note that such polynomial vectors also fulfill ‖ω(D)‖2 <∞.

112

Case 1 (AWGN channel): Since ‖ω(D)‖1 = ‖ω(X) mod (Xr − 1)‖1 and

∥∥ω(X) mod (Xr − 1)∥∥2

2=

L∑`=1

r−1∑i=0

b(t−1)/rc∑i′=0

ω`,i+i′r

2

≥L∑`=1

r−1∑i=0

b(t−1)/rc∑i′=0

ω2`,i+i′r =

∥∥ω(D)∥∥2

2,

we obtain wAWGNCp (ω(D)) ≥ wAWGNC

p (ω(X) mod (Xr − 1)).

Case 2 (BEC): Since the components of the vector ω(X) mod (Xr − 1) are

obtained by adding in R+ certain non-negative components of ω(D), it follows

that

| suppω(D)| ≥ | supp (ω(X) mod (Xr − 1)) |

and we obtain wBECp

(ω(D)

) ≥ wBECp

(ω(X) mod (Xr − 1)

).

Case 3 (BSC): In order to compare the BSC-pseudo-weight of the two vec-

tors, we first need to arrange the components in decreasing order. Let M0 ≥M1 ≥ . . . ≥ MtL−1 and m0 ≥ m1 ≥ . . . ≥ mrL−1 be listings of all the coeffi-

cients of all the components of ω(D) and ω(X) mod (Xr − 1), respectively, in

non-increasing order. Since ‖ω(D)‖1 = ‖ω(X) mod (Xr − 1)‖1, we obtain that

‖ω(D)‖12

= ‖ω(X)mod (Xr−1)‖12

, M , which gives∑tL−1

i=0 Mi =∑rL−1

i=0 mi = 2M .

Hence the two sequences of non-negative integers form two partitions, λ and µ,

respectively, of 2M . We fill the shorter partition with zeros so that both partitions

have the same length, say P . It is enough to show that∑i′−1

i=0 Mi ≤∑i′−1

i=0 mi for

all i′ = 1, 2, . . . , P , i.e., that µ majorizes λ [63].

We show first that m0 ≥M0. Suppose the contrary, i.e., m0 < M0. Since mi ≤m0 for all i ∈ 0, . . . , P − 1, we obtain that mi < M0 for all i ∈ 0, . . . , P − 1.

113

But mi, i ∈ 0, . . . , P − 1, was obtained by adding over R+ a certain subset of

the setMi′ | i′ ∈ 0, . . . P − 1. So there should be at least one mi′′ that has

M0 in its composition, and hence mi′′ ≥ M0. This is a contradiction, from which

we obtain m0 ≥M0.

We finish the proof by induction. Namely, we want to show that from

i′−1∑i=0

Mi ≤i′−1∑i=0

mi

for some i′ ∈ 1, . . . P − 1, it follows that

i′∑i=0

Mi ≤i′∑i=0

mi.

If Mi′ ≤ mi′ then this induction step clearly holds. So, assume that Mi′ > mi′ .

Since mP−1 ≤ . . . ≤ mi′ < Mi′ ≤ Mi′−1 ≤ . . . ≤ M0, we can deduce that mi′ , and

in fact all mi with i′ ≤ i ≤ P − 1, cannot contain any Mi with 0 ≤ i ≤ i′ in its

composition. Hence all possible Mi, 0 ≤ i ≤ i′, have occurred in the composition

of mi for 0 ≤ i ≤ i′ − 1, which gives

]i′∑i=0

mi ≥i′−1∑i=0

mi ≥i′∑i=0

Mi.

This proves that µ majorizes λ and we obtain

wBSCp

(ω(D)

) ≥ wBSCp

(ω(X) mod (Xr − 1)

).

Theorem 7.2.13 implies that low-pseudo-weight vectors in the block code may

correspond to higher pseudo-weight vectors in the convolutional code, but the op-

114

posite is not possible. This suggests that the pseudo-codewords in the block code

that result in decoding failures may not cause such failures in the convolutional

code, thereby resulting in improved performance for the convolutional code at

low-to-moderate SNRs.

A similar bound also holds for the max-fractional weight, as shown in the next

theorem.

Theorem 7.2.14 If ω(D) ∈ K(Hconv(D)), then

wmax−frac

(ω(X) mod (Xr − 1)

) ≤ wmax−frac(ω(D)).

Therefore,

wminmax−frac

(H

(r)QC(X)

)≤ wmin

max−frac

(Hconv(D)

).

Proof: We have ‖ω(D)‖1 = ‖ω(X) mod (Xr − 1)‖1 and

∥∥ω(X) mod (Xr − 1)∥∥∞ =

Lmax`=1

r−1maxi=0

b(t−1)/rc∑i′=0

ω`,i+i′r

≥ Lmax`=1

r−1maxi=0

b(t−1)/rcmaxi′=0

ω`,i+i′r

≥ ∥∥ω(D)∥∥∞,

which leads to wmax−frac(ω(X) mod (Xr − 1)) ≤ wmax−frac(ω(D)). It now follows

that

wminmax−frac(H

(r)QC) ≤ wmin

max−frac(Hconv).

115

In the case of the fractional weight, it is easy to see that for any ω(D) ∈V(P(Hconv(D))) \ 0, we have ‖ω(D)‖1 = ‖ω(X) mod (Xr − 1)‖1, and hence

wfrac(ω(X) mod (Xr − 1)) = wfrac(ω(D)). When comparing the minimum frac-

tional weight of the convolutional and QC codes, we encounter a computationally

harder case, as these values must be computed over the set of nonzero pseudo-

codewords that are vertices of the fundamental polytope. This is not an easy task,

because a vertex pseudo-codeword in the convolutional code might not map into

a vertex pseudo-codeword in the QC code.

The theorem below, however, can be established. For its better understanding,

we recall that wminfrac(H

(r)QC) has the following meaning [57, 58]. Let E (r)

QC ⊆ I(H(r)QC)

be the set of positions where bit flips occurred when using C(r)QC for data transmis-

sion over a BSC with crossover probability p, 0 ≤ p < 1/2. If |E (r)QC| < 1

2wmin

frac(H(r)QC),

then LP decoding is correct. Similarly, wminfrac(Hconv) implies the following. If

Econv ⊆ I(Hconv) is the set of positions where bit flips occurred when using Cconv

for data transmission over a BSC, then |Econv| < 12wmin

frac(Hconv) guarantees that LP

decoding is correct.

Theorem 7.2.15 Assume that we are using Cconv for data transmission over a

BSC with cross-over probability p, where 0 ≤ p < 1/2, and that bit flips occur at

positions Econv ⊆ I(Hconv). If |Econv| < 12wmin

frac(H(r)QC), then LP decoding is correct.

(Note that on the right-hand side of the previous inequality we have wminfrac(H

(r)QC)

and not wminfrac(Hconv).)

Proof: We know that

wminfrac

(H

(r)QC

) (∗)≤ wmin

max−frac

(H

(r)QC

) (∗∗)≤ wmin

max−frac (Hconv) , (7.6)

116

where step (∗) follows from [57, 58] (see also [56]) and step (∗∗) follows from

Theorem 7.2.14.

Remember that wminmax−frac(H

(r)QC) has the following meaning [57, 58]. Let Econv ⊆

I(Hconv) be the set of positions where the bit flips occurred when using Cconv for

data transmission over a BSC. If |Econv| < 12wmin

max−frac(Hconv), then LP decoding is

correct.

Now, because the theorem statement assumes that |Econv| < 12wmin

frac(H(r)QC), us-

ing (7.6) we have |Econv| < 12wmin

max−frac(Hconv) and so, according to the meaning of

wminmax−frac(Hconv), LP decoding is correct.

It is not difficult to adapt Theorem 7.2.13 and Theorem 7.2.14 such that similar

conclusions similar can be drawn with respect to a QC block code with the same

structure but a larger circulant size that is a multiple of r. In fact, most QC block

codes with the same structure but a larger circulant size, even if not a multiple of

r, behave according to Theorem 7.2.13. Using similar arguments to the ones in the

proofs of these theorems we obtain the following more general inequalities that

hold for the AWGN channel, BEC, BSC max-fractional and fractional pseudo-

weights. If ω(D) ∈ K(Hconv(D)), then

wp

(ω(X) mod (Xr − 1)

) ≤ wp

(ω(X) mod (X2r − 1)

)(7.7)

≤ wp

(ω(X) mod (X4r − 1)

)(7.8)

≤ . . . ≤ wp

(ω(D)

), (7.9)

117

for all r ≥ 1. In addition, for the AWGN channel, BEC, BSC and max-fractional

minimum pseudo-weights the following holds for any m ≥ 1:

wp

(ω(X) mod (Xr − 1)

) ≤ wp

(ω(X) mod (Xmr − 1)

).

Example 7.2.16 To illustrate how the pseudo-weights of the pseudo-codeword

in the convolutional code and its projections onto the QC versions satisfy the

pseudo-weight inequalities given above for all the defined pseudo-weights wAWGNCp ,

wBECp , wBSC

p , wmax−frac and wfrac, we computed the pseudo-weights of the pseudo-

codewords in Example 7.2.9. Table 7.1 contains these results.

Next we exemplify some of the bounds on the minimum pseudo-weight of codes

proved in this section. We take a tower of three QC codes together with their con-

volutional version and compute their minimum pseudo-weights which, according to

the bounds derived above, form a sequence of increasing numbers, upper bounded

by the minimum pseudo-weight of the convolutional version. However, due to the

large code parameters, we were able to compute only the minimum pseudo-weight

of the smaller code of length 20. For the other QC codes we used the methods

of [50, 56] to give lower and upper bounds.

Example 7.2.17 Consider the (3, 4)-regular QC-LDPC code of length 4r given

by the scalar parity-check matrix, or polynomial parity check matrix, respectively,

H(r)QC =

I0 I0 I0 I0

I0 I1 I2 I3

I0 I4 I3 I2

, H(r)QC(X) =

1 1 1 1

1 X X2 X3

1 X4 X3 X2

.

118

TABLE 7.1

THE PSEUDO-WEIGHTS OF THE PSEUDO-CODEWORDS IN

EXAMPLE 7.2.9

wAWGNCp wBEC

p wBSCp wmax−frac wfrac

ω(D) 8.45 11 6.67 6.5 26

ω(X) mod (Xr − 1)

for all r ≥ 4 8.45 11 6.67 6.5 26

ω(X) mod (X3 − 1) 7.86 10 6.5 6.5 26

ω(X) mod (X2 − 1) 6.09 8 5 3.71 26

ω(X) mod (X − 1) 3.63 4 3 2.89 26

119

For r = 5 we obtain a [20, 7, 6] code with rate R = 0.35. By increasing r we

obtain other QC codes. By taking powers of r we obtain a tower of QC codes whose

graphs form a sequence of covers over the Tanner graph of the [20, 7, 6] code. For

r ≥ 9, all the codes have minimum distance 10, and hence the free distance of the

associated rate R = 1/4 convolutional code is dfree = 10, strictly larger than the

minimum distance 6 of the [20, 7, 6] code.

For comparison we simulated these codes together with the associated convolu-

tional code. The results for an AWGN channel are given in Figure 7.6. For the

[20, 7, 6] code we ran a vertex enumeration program [62] for polytopes that lists all

the minimal pseudo-codewords and found that the minimum pseudo-weight of the

[20, 7, 6] code is wAWGNC,minp = 6.00. The larger parameters of the other three codes

allowed us to only lower and upper bound the minimum pseudo-weights.7 For the

[40, 12, 10] QC code, we have 6.05 ≤ wAWGNC,minp ≤ 9.09 and for the [80, 22, 10]

and the [160, 42, 10] codes, we obtained the bounds: 6.20 ≤ wAWGNC,minp ≤ 9.09.8

The slight increase in the lower bound from 6, which is the minimum weight of the

[20, 7, 6] code, to 6.05 and 6.20 shows that the minimum AWGN channel pseudo-

weight of the [20, 7, 6] code is less than that of the [40, 12, 10], [80, 22, 10], and

[160, 42, 10] codes. The slight increase in the lower bounds from 6.05 to 6.20 only

suggests, but unfortunately gives no further evidence of, the existence of an in-

creasing sequence of minimum AWGN channel pseudo-weights for these codes,

according to the results of this section.

For completeness, we mention that the techniques in [57, 58] allow us to ef-

ficiently compute the minimum max-fractional weight for the above-mentioned

7The lower bounds are obtained by applying the techniques that were presented in [64].8Using some more sophisticated lower bounds from [64], one can actually show that 7.19 ≤

wAWGNC,minp ≤ 9.09 for the length-40 code. This implies that the lower bounds for the length-80

and length-160 codes can also be tightened.

120

10-8

10-7

10-6

10-5

10-4

10-3

10-2

10-1

-4 -3 -2 -1 0 1 2 3 4 5

BER

Es/N0 (dB)

[20,7] QC code[40,12] QC code[80,22] QC code[160,42] QC codeR=1/4 convolutional code with ms=4

Figure 7.6. The performance of a rate R = 1/4 (3, 4)-regular LDPCconvolutional code and three associated (3, 4)-regular QC-LDPC block

codes. Note that the horizontal axis is Es/N0 and not the more commonEb/N0 = (1/R) · Es/N0.

121

codes: we obtain wminmax−frac = 4.67 for the length-20 code, wmin

max−frac = 5.31 for

the length-40 code, wminmax−frac = 5.33 for the length-80 code, and wmin

max−frac = 5.33

for the length-160 code. Applying the results given in (7.4) and (7.5), we see that

these values yield weaker lower bounds on wAWGNC,minp than the ones given in the

previous paragraph.

Using the same vertex enumeration program [62] we could compute as well the

minimum pseudo-weights of the QC codes of length 4r given by the parity-check

matrix H(r)QC(X) in Example 7.2.17 for r = 1, 2, 3, 4. We present the results in

Table 7.2.

TABLE 7.2

THE MINIMUM PSEUDO-WEIGHTS OF THE CODES C(r)QC, GIVEN

BY THE PARITY-CHECK MATRIX H(r)QC(X), for r = 1, 2, 3, 4

wminp (C

(r)QC) wAWGNC,min

p wBEC,minp wBSC,min

p wminmax−frac

r = 1 2 2 2 2

r = 2 2 2 2 2

r = 3 4 4 4 3

r = 4 4 4 4 4

122

7.3 Pseudo-Codewords in Time-Varying LDPC Convolutional Codes

In this section, we analyze the pseudo-codeword weight spectra of the un-

wrapped time-varying LDPC convolutional codes in comparison to that of their

underlying LDPC block codes. We start by analyzing the relation between the

minimum distance of an LDPC block code and the free distance of the associ-

ated time-varying LDPC convolutional codes and prove that the free distance of

the unwrapped LDPC convolutional code cannot be smaller than the minimum

distance of the underlying LDPC block code. We then generalize this result and

prove that this statement is also valid for the pseudo-weights of these codes on

well-known channels such as the AWGN channel, BEC, and BSC.

Similar to the approach taken in the previous section to compare the minimum

Hamming weights and the minimum pseudo-weights of a QC block code and an

associated unwrapped time-invariant convolutional code, we can also compare

a QC block code, and, more generally, any block code, described by a parity-

check matrix H to an associated periodically time-varying convolutional code

Cconv described by the parity-check matrix Hconv obtained by unwrapping the

matrix H. Let t ≥ 0 be an integer and v , (v0,v1, . . . ,vt−1) ∈ Ftrc, with

vi , (vi,0, vi,1, . . . , vi,rc−1) ∈ Frc, 0 ≤ i ≤ t− 1 , be a codeword with finite support

in the time-varying convolutional code. The vector

vr−wrapped ,t−1∑i=0

vi ∈ Frc

then denotes its r wrap-around vector in the associated block code. We have that

v is in the null-space of the matrix Hconv. Any row vector r of the matrix H

can be obtained by patching together two row vectors r′ and r′′ of the matrix

123

Hconv that have disjoint support (no overlapping ones). (For example, note that

the first and the fifteenth rows of the convolutional code matrix Hconv in Figure

3.3(c) can be patched together to form the first row of the block code matrix H

in Figure 3.3(a).) Since v is in the null space of each of the vectors r′ and r′′, and

these vectors have disjoint support, we obtain that the vector vr−wrapped is in the

null space of the vector r. We can see this easily if we denote by A and B the

lower and upper, respectively, triangular matrices of size r(c− b)× rc, having the

property that H = A + B disjointly (no overlapping nonzero entries), and

Hconv =

A

B A

B A

. . .

B A

B

. (7.10)

Since v , (v0,v1, . . . ,vt−1)T is in the null space of Hconv, we obtain that

AvT0 = 0, BvT0 + AvT1 = 0, BvT1 + AvT2 = 0, . . . , BvTt−2 + AvTt−1 = 0, BvTt−1 = 0,

from which it follows easily that

(A + B)vTr−wrapped = AvT0 + (BvT0 + AvT1 ) + . . .+ (BvTt−2 + AvTt−1) + BvTt−1 = 0,

and hence the vector vTr−wrapped is in the null space of H. Therefore any codeword

v with finite support in the time-varying convolutional code wraps to a codeword

vr−wrapped in the associated block code, and their Hamming weights are linked

124

through wH

(vr−wrapped

) ≤ wH(v). Similar to the time-invariant case [35, 36], we

obtain the following relationship between the minimum Hamming distances of the

two codes.

Theorem 7.3.1 Let C be a block code with r(c − b) × rc parity-check matrix

H = A + B disjointly and let Cconv be the periodically time-varying convolutional

code given by the parity-check matrix Hconv formed by unwrapping H in steps of

size (c− b)× c. Then the following inequality holds:

dmin(C) ≤ dfree(Cconv), for all r ≥ 1.

Proof: Suppose v , (v0,v1, . . . ,vt−1) is a codeword with finite support in the

time-varying convolutional code having Hamming weight equal to the free distance

of code.

If vr−wrapped 6= 0, we have that dmin

(C) ≤ wH

(vr−wrapped

) ≤ wH(v) = dfree

(Cconv

),

which gives the desired inequality.

If, however, vr−wrapped = 0, we obtain that the vector v(1) , (v0,v0 + v1,v0 +

125

v1 + v2, . . . ,v0 + v1 + . . .+ vt−2, 0) will also be a codeword in Cconv, since

AvT0 = 0,

BvT0 + A(vT0 + vT1 ) = AvT0 + (BvT0 + AvT1 ) = 0

B(vT0 + vT1 ) + A(vT0 + vT1 + vT2 ) = AvT0 + (BvT0 + AvT1 ) + (BvT1 + AvT2 ) = 0

...

B(vT0 + vT1 + . . .+ vTt−3) + A(vT0 + vT1 + . . .+ vTt−2) =

AvT0 + (BvT0 + AvT1 ) + (BvT1 + AvT2 ) + . . .+ (BvTt−3 + AvTt−2) = 0

B(vT0 + vT1 + . . .+ vTt−2) + A · 0 = B · vTt−1 = 0

and

B · 0 = 0,

where we have used the fact that vr−wrapped = vT0 + vT1 + . . . + vTt−2 + vTt−1 = 0

implies vT0 + vT1 + . . .+ vTt−2 = vTt−1.

Computing the Hamming weight of the wrap-around vector v(1)r−wrapped of v(1),

we obtain

wH

(v

(1)r−wrapped

)= (7.11)

wH

( t−2∑i=0

(v0 + v1 + . . .+ vi))

= wH ((t− 1)v0 + (t− 2)v1 + . . .+ 2vt−2 + vt−1) =

(7.12)

=

wH

(v1 + v3 + . . .+ vt−3 + vt−1

), if t odd

wH

(v0 + v2 + . . .+ vt−3 + vt−1

), if t even

≤ wH(v) = dfree

(Cconv

).

(7.13)

126

We now repeat inductively this procedure until we obtain a nonzero codeword in C,

which, as above, implies that the inequality dmin

(C) ≤ dfree

(Cconv

)is satisfied. The

case in which a nonzero codeword does not exist can appear only if all codewords

v , (v0,v1, . . . ,vt−1) in Cconv of minimum weight have weight 2 and are of the

form (v0,v1, . . . ,vt−1) with vi = (0, . . . , 0, 1, 0, . . . , 0), with 1 in position i, vj =

(0, . . . , 0, 1, 0, . . . , 0), with 1 in position j, for some 0 ≤ i < j ≤ t − 1, and

vl = (0, . . . , 0, 0, 0, . . . , 0) for all l /∈ i, j, which is a trivial case of no practical

interest.

In order to compare the minimum pseudo-weights of the two codes, we recall

from [7] that the pseudo-codewords in a block code C ⊂ Fn with parity-check

matrix H can be viewed as elements of the fundamental cone K(H) of H.

Hence a pseudo-codeword must satisfy certain inequalities associated with

each row of the parity-check matrix. Now let t ≥ 0 be an integer and ω ,

(ω0,ω1, . . . ,ωt−1) ∈ Rtrc, with ωi , (wi,0, wi,1, . . . , wi,rc−1) ∈ Rrc, 0 ≤ i ≤ t− 1 ,

be a pseudo-codeword with finite support in the time-varying convolutional code.

The vector ωr−wrapped ,∑t−1

i=0 ωi ∈ Rrc will be called its r wrap-around vector in

the associated block. It follows that ω satisfies the fundamental cone inequali-

ties described above associated with each of the rows of the matrix Hconv. Since

any row vector r of the matrix H can be obtained by patching together two row

vectors r′ and r′′ of the matrix Hconv that have disjoint support, we obtain that

the vector ωr−wrapped satisfies the inequalities corresponding to the row r. Hence

any pseudo-codeword with finite support ω in the time-varying convolutional code

wraps to a pseudo-codeword ωr−wrapped in the associated block code. Moreover, in

the case of time-invariant convolutional codes obtained by unwrapping QC block

codes, we showed in the previous section that the AWGN channel, BEC, and BSC

127

pseudo-weights are linked by the inequality wp

(ωr−wrapped

) ≤ wp

(ω). This led to

the inequality

wminp

(H

(r)QC

)≤ wmin

p

(Hconv

).

The corresponding proof we developed in the previous section for a time-invariant

convolutional code obtained by unwrapping a QC block code is a perfectly valid

proof for a time-varying unwrapping of any block code also, provided that the

association between the vector ω , (ω0,ω1, . . . ,ωt−1) and the polynomial vector

ω(X) , ω0(X)+ω1(X)Xrc+. . .+ωt−1(X)X(t−1)rc, where ωi(X) , wi,0+wi,1X+

. . .+ wi,rc−1Xrc−1, is used. Hence, we obtain the following result.

Theorem 7.3.2 Let C be a block code with r(c − b) × rc parity-check matrix

H = A + B disjointly and let Cconv be the periodically time-varying convolutional

code given by the parity-check matrix Hconv formed in steps of size (c− b)× c by

unwrapping H. The following inequality holds:

wminp (H) ≤ wmin

p

(Hconv

).

128

CHAPTER 8

CONCLUSIONS AND RECOMMENDATIONS FOR FUTURE RESEARCH

We showed that it is possible to unify two known techniques for deriving

LDPC convolutional codes from LDPC block codes, one based on deriving time-

invariant LDPC convolutional codes from QC-LDPC block codes and the other

on deriving time-varying LDPC convolutional codes from randomly constructed,

quasi-cyclic, and protograph-based LDPC block codes, under the same theory

of unwrapping. We then derived several families of LDPC convolutional codes,

both time-varying and time-invariant, from LDPC block codes. We demonstrated

that the constructed convolutional codes significantly outperformed the underly-

ing block codes, and we investigated the reasons for these “convolutional gains”.

In particular, we analyzed the pseudo-weight spectra of the LDPC convolu-

tional codes and showed that the minimum free pseudo-weight of the convolutional

code is at least as large as the minimum pseudo-weight of the underlying block

code, in both the time-varying and time-invariant cases. We also proved that the

unwrapped LDPC convolutional codes have better graph-cycle properties, as a

result of some of the cycles in the underlying block code breaking to yield larger

cycles during the unwrapping process. These facts lead to significantly improved

BER performance at low-to-moderate SNRs for the convolutional codes, a con-

clusion supported by simulation results. Finally, we used asymptotic methods to

calculate a lower bound on the free distance that grows linearly with constraint

129

length for several ensembles of unterminated, protograph-based periodically time

varying LDPC convolutional codes. It was shown that the free distance growth

rates of the LDPC convolutional code ensembles exceed the minimum distance

growth rates of the corresponding LDPC block code ensembles. Further, we ob-

served that the performance of the LDPC convolutional codes is consistent with

the iterative decoding thresholds of the underlying protographs.

What follows is a set of future research topics:

• Connections between pseudo-codewords, trapping sets, and near-

codewords

Besides pseudo-codewords, trapping sets and near-codeword can be studied

in order to assess the sub-optimal behavior of iterative decoders. The so-far

somewhat imprecise relationship between these concepts can be investigated

in the context of LDPC convolutional codes.

• Graph cycle structure comparison of time-varying and time-invariant

LDPC convolutional codes

The graph cycle structure analysis can be extended to investigate the rela-

tion between the cycle structures of time-varying and time-invariant LDPC

convolutional codes derived from the same underlying LDPC block code.

• Unwrapping procedures for protographs of size nc × nv with

gcd(nc, nv) = 1

For a protograph parity-check matrix of size nc×nv with gcd(nc, nv) = y = 1,

the standard unwrapping procedure summarized in Chapter 6 cannot be

employed. However, we are able to analyze this class of protographs by

constructing and analyzing slightly larger protographs based on the original

protograph. This results in a number of possible code ensembles based on

130

the larger protographs. An interesting research problem is to determine

which of the larger protographs gives the best LDPC convolutional code

ensemble in terms of distance growth rate.

• LDPC convolutional codes with large periods

While unwrapping protograph-based LDPC block codes consisting of N

copies of a base protograph results in periodically time-varying LDPC-CCs

with period T = N , unwrapping tail-biting LDPC convolutional codes with

an unwrapping factor λ > 1 results in LDPC convolutional codes with pe-

riod T = λN . This family of time-varying LDPC convolutional codes with

periods larger than the syndrome former memory has not been previously

considered in the literature, and our distance growth rate analysis suggests

that these codes have better distance properties than time-varying LDPC

convolutional codes with λ = 1. Investigating the design and analysis of

periodically time-varying LDPC convolutional codes with large periods is

an open research problem.

131

APPENDIX A

GRAPH CYCLE HISTOGRAMS OF LDPC BLOCK AND

CONVOLUTIONAL CODES

In this chapter we give the exact graph cycle histograms for the example codes

considered in Chapter 5.

132

TABLE A.1

NORMALIZED GRAPH CYCLE HISTOGRAM FOR LDPC CODES

DERIVED FROM THE [155,64] QC CODE

Cycle Block Code Time-invariant Time-varying

Length Convolutional Code Convolutional Code

8 7.00 5.80 3.57

10 10.65 9.60 10.32

12 6.44 8.00 8.01

14 9.52 8.60 8.14

16 7.80 8.60 8.14

18 9.29 8.60 8.14

20 7.87 8.60 8.14

22 9.26 8.60 8.14

24 7.90 8.60 8.14

133

TABLE A.2


DERIVED FROM THE [4608,2304] QC CODE

Cycle Block Code Time-invariant Time-varying

Length Convolutional Code Convolutional Code

6 0.46 0.28 0.39

8 1.95 1.23 1.29

10 11.59 8.82 8.76

TABLE A.3


DERIVED FROM THE RATE R = 1/2 PROTOGRAPH-BASED

LDPC CODE

Cycle Block Code Time-varying

Length Convolutional Code

4 0.025 0.0192

6 0.871 0.506

8 3.559 2.214

10 11.791 10.614

134

BIBLIOGRAPHY

1. A. E. Pusane, M. Lentmaier, K. Sh. Zigangirov, and D. J. Costello, Jr. Re-duced complexity decoding strategies for LDPC convolutional codes. In Proc.IEEE Intl. Symposium on Inform. Theory, page 490, Chicago, IL, USA, June2004.

2. A. E. Pusane, A. Jimenez-Feltstrom, A. Sridharan, M. Lentmaier, K. Sh.Zigangirov, and D. J. Costello, Jr. Implementation aspects of LDPC convo-lutional codes. IEEE Trans. Commun., to appear, 2008.

3. D. J. Costello, Jr., A. E. Pusane, S. Bates, and K. Sh. Zigangirov. A com-parison between LDPC block and convolutional codes. In Proc. InformationTheory and Applications Workshop, San Diego, CA, USA, Feb. 6-10, 2006.

4. D. J. Costello, Jr., A. E. Pusane, C. R. Jones, and D. Divsalar. A compari-son of ARA- and protograph-based LDPC block and convolutional codes. InProc. Information Theory and Applications Workshop, San Diego, CA, USA,January 29-February 2, 2007.

5. A. E. Pusane, R. Smarandache, P. O. Vontobel, and D. J. Costello, Jr. Onderiving good LDPC convolutional codes from QC LDPC block codes. In Proc.IEEE Intl. Symposium on Inform. Theory, pages 1221–1225, Nice, France,June 24-29, 2007.

6. R. Smarandache, A. E. Pusane, P. O. Vontobel, and D. J. Costello, Jr. Pseudo-codewords in LDPC convolutional codes. In Proc. IEEE Intl. Symposium onInform. Theory, pages 1364–1368, Seattle, WA, USA, July 9-14, 2006.

7. R. Smarandache, A. E. Pusane, P. O. Vontobel, and D. J. Costello, Jr. Pseudo-codeword performance analysis of LDPC convolutional codes. IEEE Trans.Inform. Theory, to appear, 2008.

8. D. G. M. Mitchell, A. E. Pusane, K. Sh. Zigangirov, and D. J. Costello,Jr. Asymptotically good LDPC convolutional codes based on protographs.submitted to IEEE Intl. Symp. on Inform. Theory, Jan. 2008.

135

9. A. E. Pusane, K. Sh. Zigangirov, and D. J. Costello, Jr. Construction of irreg-ular LDPC codes with fast encoding property. In Proc. IEEE Intl. Conferenceon Commun., Istanbul, Turkey, June 11-15, 2006.

10. A. E. Pusane, M. Lentmaier, T. E. Fuja, K. Sh. Zigangirov, and D. J. Costello,Jr. Multilevel coding/modulation using LDPC convolutional codes. In Proc.Intl. Symposium on Information Theory and its Applications, pages 685–689,Parma, Italy, Oct. 2004.

11. S. Bates, L. Gunthorpe, A. E. Pusane, Z. Chen, K. Sh. Zigangirov, andD. J. Costello, Jr. Decoders for low-density parity-check convolutional codeswith large memory. In Proc. 12th NASA Symposium on VLSI Design, Coeurd’Alene, Idaho, U.S.A., Oct. 2005.

12. S. Bates, L. Gunthorpe, A. E. Pusane, Z. Chen, K. Sh. Zigangirov, and D. J.Costello, Jr. Decoders for low-density parity-check convolutional codes withlarge memory. In Proc. IEEE Symposium on Circuits and Systems, Kos,Greece, May 21-24, 2006.

13. S. Bates, Z. Chen, L. Gunthorpe, A. E. Pusane, K. Sh. Zigangirov, and D. J.Costello, Jr. A low-cost serial decoder architecture for low-density parity-check convolutional codes. IEEE Trans. Circuits and Systems I, to appear,2008.

14. K. Sh. Zigangirov, A. E. Pusane, D. K. Zigangirov, and D. J. Costello, Jr.On the error correcting capability of LDPC codes. Problems of InformationTransmission (Problemy Peredachy Informatsii), to appear, 2008.

15. C. E. Shannon. A mathematical theory of communication. Bell SystemsTechnical Journal, 27:379–423, July 1948.

16. C. Berrou, A. Glavieux, and P. Thitimajshima. Near Shannon limit error cor-recting coding and decoding: turbo codes. In Proc. of the IEEE InternationalConference on Communications, pages 1064–1070, Geneva, Switzerland, May1993.

17. R. G. Gallager. Low-density parity-check codes. IRE Trans. Inform. Theory,IT-8:21–28, Jan. 1962.

18. R. G. Gallager. Low-density parity-check codes. M.I.T. Press, Cambridge,MA, 1963.

19. R. M. Tanner. A recursive approach to low complexity codes. IEEE Trans.Inform. Theory, IT-27:533–547, Sept. 1981.

136

20. N. Wiberg. Codes and Decoding on General Graphs. PhD thesis, LinkopingUniversity, Sweden, 1996.

21. D. J. C. MacKay and R. M. Neal. Near Shannon limit performance of lowdensity parity check codes. Electronics Letters, 32(18):1645–1646, Aug. 1996.

22. M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. A. Spielman. Anal-ysis of low density codes and improved designs using irregular graphs. In Proc.30th Annual ACM Symp. Theory of Computation, pages 249–258, May 1998.

23. M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. A. Spielman. Im-proved low-density parity-check codes using irregular graphs. IEEE Trans.Inform. Theory, IT-47:585–598, Feb. 2001.

24. T. J. Richardson and R. L. Urbanke. The capacity of low-density parity-checkcodes under message-passing decoding. IEEE Trans. Inform. Theory, IT-47:599–618, Feb. 2001.

25. S. Y. Chung, T. J. Richardson, and R. L. Urbanke. Analysis of sum-productdecoding of low-density parity-check codes using a Gaussian approximation.IEEE Trans. Inform. Theory, IT-47:657–670, Feb. 2001.

26. T. J. Richardson, M. A. Shokrollahi, and R. L. Urbanke. Design of capacity-approaching irregular low-density parity-check codes. IEEE Trans. Inform.Theory, IT-47:619–637, Feb. 2001.

27. S. Y. Chung, G. D. Forney, Jr., T. J. Richardson, and R. L. Urbanke. Onthe design of low-density parity-check codes within 0.0045 db of the Shannonlimit. IEEE Communications Letters, 5:58–60, Feb. 2001.

28. R. M. Tanner. Error-correcting coding system. U.S. Patent # 4,295,218, Oct.1981.

29. A. Jimenez-Feltstrom and K. Sh. Zigangirov. Time-varying periodic convo-lutional codes with low-density parity-check matrix. IEEE Trans. Inform.Theory, IT-45:2181–2191, Sept. 1999.

30. S. Bates, D. Elliot, and R. Swamy. Termination sequence generation circuitsfor low-density parity-check convolutional codes. IEEE Trans. Circuits andSystems I, 53:1909–1917, Sept. 2006.

31. S. Bates, Z. Chen, and X. Dong. Low-density parity check convolutional codesfor Ethernet networks. In Proc. IEEE Pacific Rim Conference on Commu-nications, Computers and Signal Processing, Victoria, B.C., Canada, Aug.2005.

137

32. L. Zongwang, C. Lei, Z. Lingqi, S. Lin, and W. H. Fong. Efficient encoding ofquasi-cyclic low-density parity-check codes. IEEE Trans. Commun., COM-54:71–81, Jan. 2006.

33. T. J. Richardson and R. L. Urbanke. Efficient encoding of low-density parity-check codes. IEEE Trans. Inform. Theory, IT-47(2):638–656, Feb. 2001.

34. S. Lin and D. J. Costello, Jr. Error Control Coding. Prentice-Hall, EnglewoodCliffs, NJ, 2nd edition, 2004.

35. R. M. Tanner. Convolutional codes from quasi-cyclic codes: a link betweenthe theories of block and convolutional codes. University of California, SantaCruz, Tech Report UCSC-CRL-87-21, Nov. 1987.

36. Y. Levy and D. J. Costello, Jr. An algebraic approach to constructing convolu-tional codes from quasi-cyclic codes. in Coding and Quantization (Piscataway,NJ, 1992), vol. 14 of DIMACS Ser. Discrete Math. Theoret. Comput. Sci., pp.189–198, Providence, RI: Amer. Math. Soc., 1993.

37. M. Esmaeili, T. A. Gulliver, N. P. Secord, and S. A. Mahmoud. A link betweenquasi-cyclic codes and convolutional codes. IEEE Trans. Inform. Theory, IT-44:431–435, Jan. 1998.

38. R. M. Tanner, D. Sridhara, A. Sridharan, T. E. Fuja, and D. J. Costello,Jr. LDPC block and convolutional codes based on circulant matrices. IEEETrans. Inform. Theory, IT-50:2966–2984, Dec. 2004.

39. R. Johannesson and K. Sh. Zigangirov. Fundamentals of Convolutional Cod-ing. IEEE Press, Piscataway, NJ, 1999.

40. L. Lan, L. Zeng, Y. Y. Tai, L. Chen, S. Lin, and K. Abdel-Ghaffar. Construc-tion of quasi-cyclic LDPC codes for AWGN and binary erasure channels: Afinite field approach. IEEE Trans. Inform. Theory, 53:2429–2458, July 2007.

41. D. Divsalar, C. R. Jones, S. Dolinar, and J. Thorpe. Protograph based LDPCcodes with minimum distance linearly growing with block size. In Proc. IEEEGlobal Telecommun. Conf., volume 3, St. Louis, MO, USA, November 28 -December 5, 2005.

42. T. Tian, C. R. Jones, J. D. Villasenor, and R. D. Wesel. Selective avoidance ofcycles in irregular LDPC code construction. IEEE Trans. Commun., COM–52(8):1242–1247, 2004.

43. A. Ramamoorthy and R. D. Wesel. Analysis of an algorithm for irregularLDPC code construction. In Proc. IEEE Intl. Symposium on Inform. Theory,page 69, Chicago, IL, USA, June 27–July 2, 2004.

138

44. J. Thorpe. Low-density parity-check (LDPC) codes constructed from pro-tographs. JPL INP Progress Report, 42-154, Aug. 2003.

45. D. Divsalar. Ensemble weight enumerators for protograph LDPC codes. InProc. IEEE Intl. Symposium on Inform. Theory, Seattle, WA, USA, July 9-14,2006.

46. S. L. Fogal, R. J. McEliece, and J. Thorpe. Enumerators for protographensembles of LDPC codes. In Proc. IEEE Intl. Symposium on Inform. Theory,Adelaide, Australia, Sept. 2005.

47. D. V. Truhachev, K. Sh. Zigangirov, and D. J. Costello, Jr. Distance boundsfor periodic LDPC convolutional codes and tail-biting convolutional codes.submitted to IEEE Trans. Inform. Theory, Jan. 2008.

48. A. Sridharan, D. V. Truhachev, M. Lentmaier, D. J. Costello, Jr., and K. Sh.Zigangirov. Distance bounds for an ensemble of LDPC convolutional codesconstructed from permutation matrices. IEEE Trans. Inform. Theory, IT-53:4537–4555, Dec. 2007.

49. D. Divsalar, S. Dolinar, and C. R. Jones. Construction of protograph LDPCcodes with linear minimum distance. In Proc. IEEE Intl. Symposium onInform. Theory, Seattle, WA, USA, July 9-14, 2006.

50. R. Koetter and P. O. Vontobel. Graph covers and iterative decoding of finite-length codes. In Proc. 3rd Turbo Symp. on Turbo Codes and Related Topics,Brest, France, September 1-5, 2003.

51. G. D. Forney, Jr., R. Koetter, F. Kschischang, and A. Reznik. On the effec-tive weights of pseudocodewords for codes defined on graphs with cycles. InB. Marcus and J. Rosenthal, editors, Codes, Systems, and Graphical Models,volume 123, pages 101–112. Springer Verlag, New York, USA, 2001.

52. D. J. C. MacKay and M. S. Postol. Weaknesses of Margulis and Ramanujan-Margulis low-density parity-check codes. Electronic Notes in Theoretical Com-puter Science, 74, 2003.

53. T. Richardson. Error floors of LDPC codes. In Proc. 41st Allerton Confer-ence on Communication Control, and Computing, Allerton House, Monticello,Illinois, USA, October 1–3 2003.

54. C. Di, D. Proietti, I. E. Telatar, T. Richardson, and R. L. Urbanke. Finite-length analysis of low-density parity-check codes on the binary erasure chan-nel. IEEE Trans. Inform. Theory, IT–48(6):1570–1579, 2002.

139

55. B. J. Frey, R. Koetter, and A. Vardy. Signal-space characterization of iterativedecoding. IEEE Trans. Inform. Theory, IT-47:766–781, Feb. 2001.

56. P. O. Vontobel and R. Koetter. Graph-cover decoding and finite-length analysis of message-passing iterative decoding of LDPC codes.submitted to IEEE Trans. Inform. Theory, available online underhttp://www.arxiv.org/abs/cs.IT/0512078, 2005.

57. J. Feldman. Decoding error-correcting codes via linear programming. PhDthesis, Massachusetts Institute of Technology, Cambridge, MA, 2003.

58. J. Feldman, M. J. Wainwright, and D. R. Karger. Using linear programmingto decode binary linear codes. IEEE Trans. Inform. Theory, IT–51:954–972,Mar. 2005.

59. A. Sridharan, M. Lentmaier, K. Sh. Zigangirov, and D. J. Costello, Jr. Con-vergence analysis of LDPC convolutional codes on the erasure channel. InProc. 42nd Allerton Conference on Communication Control, and Computing,pages 685–689, Monticello, Illinois, USA, Sept. 2004.

60. A. Huebner, M. Lentmaier, K. Sh. Zigangirov, and D. J. Costello, Jr. Lami-nated turbo codes. In Proc. IEEE Intl. Symposium on Inform. Theory, pages597–601, Adelaide, Australia, September 4-9, 2005.

61. P. O. Vontobel and R. Koetter. On the relationship between linear program-ming decoding and min-sum algorithm decoding. In Proc. Intl. Symposiumon Information Theory and its Applications, pages 991–996, Parma, Italy,October 10-13, 2006.

62. D. Avis. lrs: A revised implementation of the reverse search vertex enumera-tion algorithm. In G. Kalai and G. M. Ziegler, editors, Polytopes – Combina-torics and Computation, pages 177–198. Birkhauser-Verlag, 2000. Programsare available online under http://cgm.cs.mcgill.ca/~avis/C/lrs.html.

63. A. Marshall and I. Olkin. Inequalities: Theory of majorization and its appli-cations. Academic Press, San Diego, CA, 1979.

64. P. O. Vontobel and R. Koetter. Lower bounds on the minimum pseudo-weightof linear codes. In Proc. IEEE Intl. Symposium on Inform. Theory, page 70,Chicago, IL, USA, June 27–July 2 2004.

140

This document was prepared & typeset with pdfLATEX, and formatted withnddiss2ε classfile (v3.0[2005/07/27]) provided by Sameer Vijay.

141

Date post:	22-Nov-2014
Category:	Documents
Upload:	ihsanmarwat
View:	210 times
Download:	2 times

Analysis of LDPC Convolutional Codes Derived From LDPC Block Codes_by_Pusane

Documents