Telex Magloire NgatchedTelex Magloire Ngatched

Centre for Radio Access TechnologiesCentre for Radio Access TechnologiesUniversity Of NatalUniversity Of Natal

Durban, South-AfricaDurban, South-Africa

Telex Magloire NgatchedTelex Magloire Ngatched

Centre for Radio Access TechnologiesCentre for Radio Access TechnologiesUniversity Of NatalUniversity Of Natal

Durban, South-AfricaDurban, South-Africa

Stopping Criteria for Turbo Decoding

and Turbo Codes for Burst Channels

University Of Natal

2

Introduction

Turbo Encoder and Decoder

Stopping Criteria for Turbo Decoding

Comparison of Coding Systems

Turbo Codes for Burst channels

Conclusions

OverviewOverview

University Of Natal

3

IntroductionIntroduction

Fig. 1: Generalized Communication System

University Of Natal

4

Turbo Encoder Turbo Encoder

Turbo codes, introduced in June 1993, represent the most recent successful attempt in achieving Shannon’s theoretical limit.

N-bitInterleaver

RSC 1

RSC 2

PuncturingMechanism

data dk

y1k

y2k

xk

y1k y2k

Fig. 2: A turbo encoder

University Of Natal

5

Turbo DecoderTurbo Decoder

Two component decoders are linked by interleavers in a structure similar to that of the encoder.

MAPDecoder1

InterleaverMAPDecoder2

Deinterleaver

Deinterleaver

py12

2

sy

2

2

u1L

u1eL

py22

2

u1eL

ue2L

u2L

Fig. 3: Turbo Decoder

University Of Natal

6

Turbo DecoderTurbo Decoder

Each decoder takes three types of soft inputs

• The received noisy information sequence.

• The received noisy parity sequence transmitted from the associated component encoder.

• The a priori information, which is the extrinsic information provided by the other component decoder from the previous step of decoding process.

University Of Natal

7

Turbo DecoderTurbo Decoder

The soft outputs generated by each constituent decoder also consist of three components:

• A weighted version of the received information sequence

• The a priori value, i.e. the previous extrinsic information

• A newly generated extrinsic information, which is then provided as a priori for the next step of decoding.

kis

kki

ki uyuu ˆL

2ˆLˆL

12 e21

e1

kis

kki

ki uyuu ˆL

2ˆLˆL

21 e2e2

(1)

(2)

University Of Natal

8

Turbo DecoderTurbo Decoder

The turbo decoder operates iteratively with ever-updating extrinsic information to be exchanged between the two decoder until a reliable hard decision can be made.

Often, a fixed number, say M, is chosen and each frame is decoded for M iterations.

Usually, M is set with the worst corrupted frames in mind.

Most frames, however, need fewer iterations to converge

It is therefore important to terminate the iterations for each individual frame immediately after the bits are correctly estimated

University Of Natal

9

Stopping Criteria for Turbo Decoding Stopping Criteria for Turbo Decoding

Several schemes have been proposed to control the termination:

• Cross Entropy (CE)

• Sign Change ratio (SCR)

• Hard Decision-Aided (HDA)

• Sign Difference Ratio (SDR)

• Improved Hard Decision-Aided (IHDA) (Ngatched scheme)

University Of Natal

10

Stopping Criteria for Turbo DecodingStopping Criteria for Turbo Decoding

Cross Entropy (CE)

• Computes

• Terminates when drops to

Sign Change Ratio (SCR)• Computes the number of sign changes of the extrinsic

information from the second decoder between two consecutive iterations and .

• Terminates when , ; N is the frame size.

k ki

kie

u

uiT

ˆLexp

ˆL

1

2

2 (3)

iT 1to1010 -4-2 T

iC

1 i i qNiC 03.0005.0 q

University Of Natal

11

Stopping Criteria for Turbo DecodingStopping Criteria for Turbo Decoding

Hard-Decision-Aided (HDA)

• Terminates if the hard decision of the information bits based on at iteration agrees with the hard decision based on at iteration for the entire block.

Sign Difference Ratio (SDR)

• Terminates at iteration if the number of sign difference between , , satisfies

, is the frame size.

ki uL 12 1i

ki uL2 i

i kiej uL jiD NpD ji

01.0001.0 p N

University Of Natal

12Stopping Criteria for Turbo DecodingStopping Criteria for Turbo Decoding

The influence of each term on the a-posteriori LLR depends on whether the frame is “good” (easy to decode) or “bad” (hard to decode).

For a “bad” frame, the a-posteriori LLR is greatly influenced by the channel soft output.

For a “good” frame, the a-posteriori LLR is essentially determined by the extrinsic information as the decoding converges.

These observations, together with equations (1) and (2) led us to the following stopping criterion.

University Of Natal

13

Iteration 1

-8

-6

-4

-2

0

2

4

6

0 50 100 150 200 250 300 350 400 450 500

Bit number

Valu

es

Sum

Output

Iteration 3

-8

-6

-4

-2

0

2

4

6

0 50 100 150 200 250 300 350 400 450 500

Bit number

Valu

es

Sum

Output

Fig. 4.a: Outputs from the decoder for a transmitted “bad” stream of -1

University Of Natal

14

Iteration 5

-8

-6

-4

-2

0

2

4

6

0 50 100 150 200 250 300 350 400 450 500

Bit number

Valu

es

Sum

Output

Iteration 7

-8

-6

-4

-2

0

2

4

6

0 50 100 150 200 250 300 350 400 450 500

Bit number

Valu

es

Sum

Output

Fig. 4.b: Outputs from the decoder for a transmitted “bad” stream of -1

University Of Natal

15

Iteration 1

-16-14-12-10-8-6-4-20246

0 100 200 300 400 500

Bit number

Valu

es

Sum

Output

Iteration 3

-30

-25

-20

-15

-10

-5

0

0 100 200 300 400 500

Bit number

Valu

es

Sum

Output

Fig. 5.a: Outputs from the decoder for a transmitted “good” stream of -1

University Of Natal

16

Fig. 3.b: Outputs from the decoder for a transmitted “bad” stream of -1

Iteration 5

-50

-40

-30

-20

-10

0

0 50 100 150 200 250 300 350 400 450 500

Bit number

Valu

es

Sum

Output

Iteration 7

-80

-60

-40

-20

0

0 50 100 150 200 250 300 350 400 450 500

Bit number

Val

ues Sum

Output

Fig. 5.b: Outputs from the decoder for a transmitted “good” stream of -1

University Of Natal

17

Stopping Criteria for Turbo DecodingStopping Criteria for Turbo Decoding

Improved Hard-Decision-Aided (IHDA)

• Terminates at iteration if the hard decision of the

information bit based on agrees

with the hard decision of the information bit based on

for the entire block.

i

k

isk uy ˆL

21e2

ki uL2

University Of Natal

18Comparison of Stopping Criteria for Turbo Decoding

Comparison of Stopping Criteria for Turbo Decoding

Simulation Model• Code of rate 1/3, rate one-half RSC component encoders

of memory 3 and octal generator (13, 15).

• Frame size 128, AWGN channel.

• MAP decoding algorithm with a maximum of 8 iterations.

• Five terminating schemes are studied:• CE ( ), SCR ( ), HAD, SDR ( ) and IHDA.

• The “GENIE” case is shown as the limit of all possible schemes.

iT 110 3T 310q410p

University Of Natal

19

Results: BER PerformanceResults: BER Performance

1.00E-05

1.00E-04

1.00E-03

1.00E-02

1.00E-01

0 0.5 1 1.5 2 2.5 3

Eb/No (dB)

BE

R

GENIE HDA

SCR SDR

IHDA CE

Graph 1: Simulated BER performance for six stopping schemes.

University Of Natal

20

Results: Average number of iterationsResults: Average number of iterations

1

2

3

4

5

6

7

8

0 0.5 1 1.5 2 2.5 3

Eb/No (dB)

Av

era

ge

nu

mb

er

of

ite

rati

on

s

GENIE HDA

SCR SDR

IHDA CE

Graph 2: Simulated Average number of iteration for the six schemes

University Of Natal

21Comparison of Stopping Criteria for Turbo Decoding

Comparison of Stopping Criteria for Turbo Decoding

All six schemes exhibit similar BER performance. The HDA, however, presents a slight degradation at high SNR.

The IHDA saves more iterations for small interleaver sizes.

The CE, SCR and HDA require extra data storage. The SCR and HDA however require less computation than the CE.

Both the IHDA and the SDR have the advantage of reduce storage requirement.

The IHDA has the additional advantage that its performance is independent of the choice of any parameter.

University Of Natal

22

Comparison of Coding SystemsComparison of Coding Systems

University Of Natal

23

Turbo Codes for Burst ChannelsTurbo Codes for Burst Channels

Studies of the performance of error correcting codes are most often concerned with situations where the channel is assumed to be memoryless, since this allows for a theoretical analysis.

For a channel with memory, the Gilbert-Elliott (GE) channel is one of the simplest and practical models.

We model a slowly varying Rayleigh fading channel with autocorrelation function by the Gilbert-Elliott channel model.

We then use this model to analytically evaluate the performance of Turbo-coded system.

m0 fJR 2

University Of Natal

24

The Gilbert-Elliott Channel ModelThe Gilbert-Elliott Channel Model

The GE channel is a discrete-time stationary model with two states: one bad state which generally has high error probabilities and the other state is a good state which generally has low error probabilities.

BG

b

g

1-g1-b

Fig. 6: The Gilbert-Elliott channel model.

University Of Natal

25

The Gilbert-Elliott Channel ModelThe Gilbert-Elliott Channel Model

The dynamics of the channel are modeled as a first-order Markov chain.

In either state, the channel exhibits the properties of a binary symmetric channel.

Important statistics

• Steady state probabilities.

• Average time units in each state

gb

g

G gb

b

B

b

1GTEGT

g

1BTEBT

University Of Natal

26Matching the GE channel to the Rayleigh fading channel

Matching the GE channel to the Rayleigh fading channel

We let the average number of time unit the channel spends in the good (bad) state to be equal to the expected non-fade (fade) duration, normalized by the symbol time interval.

In doing this, we obtain the following transition probabilities:

2Tb SDf

1e

2Tg SD

f

University Of Natal

27Matching the GE channel to the Rayleigh fading channel

Matching the GE channel to the Rayleigh fading channel

The error probabilities in the respective states in the GE channel are taken to be the conditional error probabilities of the Rayleigh fading channel, conditioned on being in the respective state. For BPSK, the simplified expressions are:

TTe erfc1

1

experfc

2

1GP

-exp-12

erfc11

1

1-experfc1

BP

TT

e

University Of Natal

28Performance Analysis of Turbo-Coded System on a Gilbert-Elliott Channel

Performance Analysis of Turbo-Coded System on a Gilbert-Elliott Channel

There are two main tools for the performance evaluation of turbo codes: Monte Carlo simulation and standard union bound.

Monte Carlo simulation generates reliable probability of error estimates as low as 10-6 as is useful for rather low SNR.

The union bound provides an upper bound on the performance of turbo codes with maximum likelihood decoding averaged over all possible interleavers.

The expression for the average bit error probability is given as

3N

dd i d d221bit

min 1 2

dPidPidPi

N

N

iP

University Of Natal

29Performance Analysis of Turbo-Coded system on a Gilbert-Elliott Channel

Performance Analysis of Turbo-Coded system on a Gilbert-Elliott Channel

and are the distribution of the parity sequences and are given as

P2(d) is the pairwise-error probability and depends on the

channel.

We derive and expression for P2(d) for the Gilbert-Elliott

channel.

idP 1 idP 2

i

N

di,N,t

di,N,t

di,N,tidP

p

d p

pp

p

University Of Natal

30Pairwise-error Probability for the Gilbert-Elliott Channel Model

Pairwise-error Probability for the Gilbert-Elliott Channel Model

If the channel state is known exactly to the decoder we assume that amongst the d bits in which the incorrect path and the correct path differ, there are dB in the bad state and

dG = d-dB in the good state.

Amongst the dB bits, there are eB bits in error and amongst

the dG bits, eG are in error.

Let CM(1) and CM(0) be the metric of the incorrect path and the correct path respectively. GlogPe-dGP-1logeBlogPedBP-1logeCM eGGeGeBBeB

1

GlogPeGP-1loge-dBlogPeBP-1logedCM eGeGGeBeBB0

University Of Natal

31Pairwise-error probability for the Gilbert-Elliott Channel Model

Pairwise-error probability for the Gilbert-Elliott Channel Model

The probability of error in the pairwise comparison of CM(1) and CM(0) is:

where C is the metric ratio defined as

To evaluate P2(d), we need the probability distribution of being in the bad state dB times out of d and the distribution for being in the good state dG times out of d.

01r2 CMCMPdP BGBGr CeeCddP 2

GPGP-1log

BPBP-1logC

ee

ee

University Of Natal

32Pairwise-error Probability for the Gilbert-Elliott Channel Model

Pairwise-error Probability for the Gilbert-Elliott Channel Model

We show that:

and

dd g

dd1 ,BBdPBGdPGBdPGGdP

0d ,b

dP

BB1-d

BBBdBdGBdBd

BG1-d

Bd

,1

1

dd b

dd1 ,BBdPBGdPGBdPGGdP

0d ,g

dP

GG1-d

GBGdGdGGdGd

GB1-d

Gd

,1

1

University Of Natal

33Pairwise-error Probability for the Gilbert-Elliott Channel Model

Pairwise-error Probability for the Gilbert-Elliott Channel Model

where

1-i1idd-ddmin

2i

1-iid-dBBBd ggbb

2-i

d

1-i

d-dGGdP B

BB

B

11

11,1

1-iidd-ddmin

1i

iid-dBBBd ggbb

1-i

d

1-i

d-dGBdP B

BB

B

11

11,

iidd-ddmin

1i

1-iid-dBBBd ggbb

1-i

d

1-i

d-dBGdP B

BB

B

11

11,

1-iidd-ddmin

2i

1-i1id-dBBBd ggbb

1-i

d

2-i

d-dBBdP B

BB

B

11

111,

University Of Natal

34Pairwise-error Probability for the Gilbert-Elliott Channel Model

Pairwise-error Probability for the Gilbert-Elliott Channel Model

and

1-i1d-dd-ddmin

2i

1-iidGGGd ggbb

1-i

d

2-i

d-dGGdP G

GG

G

11

111,

1-iid-dd-ddmin

1i

iidGGGd ggbb

1-i

d

1-i

d-dGBdP G

GG

G

11

11,

iid-dd-ddmin

1i

1-iidGGGd ggbb

1-i

d

1-i

d-dBGdP G

GG

G

11

11,

1-iid-dd-ddmin

2i

1-i1i-dGGGd ggbb

2-i

d

1-i

d-dBBdP G

GG

G

11

11,1

University Of Natal

35Pairwise-error Probability for the Gilbert-Elliott Channel Model

Pairwise-error Probability for the Gilbert-Elliott Channel Model

Thus

We apply this union bound technique to obtain upper bounds on the bit-error rate of a turbo-coded DS-CDMA system.

ddGd eGd

ede

ee

G

G

eBd

ede

ee

B

B2

B B

GGG

G

BBB dPGPGPe

ddPBPBP

e

ddP 11

University Of Natal

36Performance Analysis of Turbo-Coded DS-CDMA System on a Gilbert-Elliott Channel Model

Performance Analysis of Turbo-Coded DS-CDMA System on a Gilbert-Elliott Channel Model

System model

• We consider an asynchronous binary PSK direct-sequence CDMA system that allow K users to share a channel. The received signal at a given receiver is given by

• The output of the matched filter at each sampling instant is

tnttbtatPtrK

kkckkkkk

1

cos2

ii,0i TbP

Z 2

K

ikk

kkik,k,0kik,k,-1k RbRbP

1

cosˆ2

tn'

University Of Natal

37Performance Analysis of Turbo-coded DS-CDMA System on a Gilbert-Elliott Channel Model

Performance Analysis of Turbo-coded DS-CDMA System on a Gilbert-Elliott Channel Model

Using gausssian approximation, the SNR at the output of the receiver is

and its expected value is

ii

ii

Z

ZE

var

2

2P.T

NE

3N

1 0K

ikk

2k

2i

1

2

i

0

2P.T.E

N

3N

1-KE

1

University Of Natal

38Performance Analysis of Turbo-Coded DS-CDMA System on a Gilbert-Elliott Channel Model

Performance Analysis of Turbo-Coded DS-CDMA System on a Gilbert-Elliott Channel Model

Simulation model

• Code of rate 1/3, rate one-half RSC component encoders of memory 2 and octal generator (7, 5).

• Gold spreading sequence of length N = 63.

• The number of users is 10 and the frame size is 1024.

• Perfect channel estimation and power control.

• The product is considered as an independent parameter, fm , and simulations are performed for fm = 0.1, 0.01and 0.001.

• The threshold is 10 dB for fm = 0.1, 0.01 and 14 dB for fm =

0.001.

SDTf

University Of Natal

39

ResultsResults

1.00E-18

1.00E-16

1.00E-14

1.00E-12

1.00E-10

1.00E-08

1.00E-06

1.00E-04

1.00E-02

1.00E+00

0 4 8 12 16 20 24 28 32 36 40 44

Eb/No (dB)

Bo

un

d o

n P

b

N1 = 10

N1 = 100

N1 = 1024

fm = 0.1

fm = 0.01

fm = 0.001

Graph 3: Bounds on the BER for different values of N1 and fm

University Of Natal

40

ResultsResults

1.00E-18

1.00E-16

1.00E-14

1.00E-12

1.00E-10

1.00E-08

1.00E-06

1.00E-04

1.00E-02

1.00E+00

0 4 8 12 16 20 24 28 32 36 40

Eb/No (dB)

Bit

erro

r ra

te

fm = 0.1

fm = 0.001

fm = 0.01

Graph 4: Transfer function bound (solid lines) versus simulation for various values of fm.

University Of Natal

41The Effect of Imperfect Interleaving for the GE Channel

The Effect of Imperfect Interleaving for the GE Channel

An effective method to cope with burst errors is to insert an interleaver between the channel encoder and the channel.

How effective an interleaver is depends on its depth, m.

The size of the interleaver is typically determined by how much delay can be tolerated.

We show that interleaving a code to degree m has exactly the same effect as transmitting at a lower rate or increased symbol duration of T.m.

Thus, the GE channel with an interleaver will be equivalent to a GE channel where the corresponding transition probabilities are the m-step transition probabilities of the original model.

University Of Natal

42The Effect of Imperfect Interleaving for the GE Channel

The Effect of Imperfect Interleaving for the GE Channel

The m-step transition probabilities are obtained by applying the Chapman-Kolmogrov equation to our two-state Markov chain.

We obtain:

mg-b1gb

b

gb

gGGPm

mg-b1

gb

g

gb

gBGPm

mg-b1gb

b

gb

bGBPm

mg-b1

gb

g

gb

bBBPm

University Of Natal

43

ResultsResults

1.00E-08

1.00E-07

1.00E-06

1.00E-05

1.00E-04

1.00E-03

1.00E-02

1.00E-01

1.00E+00

0 2 4 6 8 10 12 14 16 18 20 22 24 26

Eb/ No (dB)

Bit

err

or

rate

m = 1 (No int.)

m = 10m = 20

m = 40m = 60

m = 80m = 100

Graph 5: Simulated bit error rate on the interleaved Gilbert-Elliott channel model for different values of the interleaver depth. fm = 0.001.

University Of Natal

44ResultsResults

1.00E-08

1.00E-07

1.00E-06

1.00E-05

1.00E-04

1.00E-03

1.00E-02

1.00E-01

1.00E+00

0 2 4 6 8 10 12 14 16 18 20 22 24 26

Eb/No (dB)

Bit

err

or

rate

N1 100

N1 = 100 ( m = 50)

N1 = 1024

N1 = 4096

N1 10000

N1 = 60000

Graph 6: Comparison of the performance of a combined small code interleaver with channel interleaver and larger code interleavers of various sizes.

University Of Natal

45ConclusionsConclusions

In this presentation, we present stopping criteria for Turbo decoding.

We model a slowly varying Rayleigh fading channel by the Gilbert-Elliott channel model.

We then use this model to analytically evaluate the performance of a Turbo-coded DS-CDMA system.

We analyze the effect of imperfect interleaving for the Gilbert-Elliott channel model.

We show that a combination of a small code interleaver with a channel interleaver could outperform codes with very large interleavers, making Turbo codes suitable for even delay-sensitive services.