III. Turbo Codes

© Tallal Elshabrawy

Applications of Turbo Codes

Worldwide Applications & Standards Data Storage Systems

DSL Modem / Optical Communications

3G (Third Generation) Mobile Communications

Digital Video Broadcast (DVB)

Satellite Communications

IEEE 802.16 WiMAX

…

© Tallal Elshabrawy

Basic Concept of Turbo Codes

Invented in 1993 by Alian Glavieux, Claude Berrou and Punya Thitimajshima

Basic Structure:

a) Recursive Systematic Convolutional (RSC) Codes

b) Parallel concatenation and Interleaving

c) Iterative decoding

© Tallal Elshabrawy

Recursive Systematic Convolutional Codes

+

+

Non Systematic Convolutional Code

Recursive Systematic Convolutional Code

+

+

c1

c2

g1=[1 1 1] g2=[1 0 1]

G=[g1 g2]

+

c1

c2

g1=[1 1 1] g2=[1 0 1]

G=[1 g2/g1]

© Tallal Elshabrawy

Non-Recursive Encoder Trellis Diagram

s0 (0 0)

s1 (1 0)

s2 (0 1)

s3 (1 1)

00 00 00

11 11 11

10

01 01

01

10 10

11

00

00 00 00

11 11 11

10 10 10

01 01 01

01 01 01

10 10 10

11 11 11

00 00 00

© Tallal Elshabrawy

Systematic Recursive Encoder Trellis Diagram

s0 (0 0)

s1 (1 0)

s2 (0 1)

s3 (1 1)

00 00 00

11 11 11

10

01 01

01

10 10

11

00

00 00 00

11 11 11

10 10 10

01 01 01

01 01 01

10 10 10

11 11 11

00 00 00

© Tallal Elshabrawy

Non-Recursive & Recursive Encoders

Non Recursive and recursive encoders both have the same trellis diagram structure

Generally Recursive encoders provide better weight distribution for the code

The difference between them is in the mapping of information bits to codewords

© Tallal Elshabrawy

Parallel Concatenation with Interleaving

Two component RSC Encoders in parallel separated by an Interleaver The job of the interleaver is to de-correlate the encoding process of the

two encoders Usually the two component encoder are identicalxk : Systematic information bit

yk(1) : Parity bit from the first RSC encoder

yk(2) : Parity bit from the second RSC encoder

RSC Encoder 1

RSC Encoder 2

π

xk

yk(1)

yk(2)

xk

1/3 Turbo Encoder

© Tallal Elshabrawy

InterleavingPermutation of data to de-correlate encoding process of both encodersIf encoder 1 generates a low weight codeword then hopefully the interleaved input information would generate a higher weight codewordBy avoiding low weight codewords the BER performance of the turbo codes could improve significantly

© Tallal Elshabrawy

Turbo Decoder

Iteratively exchanging information between decoder 1 and decoder 2 before making a final deciding on the transmitted bits

rkRSC Decoder 1

RSC Decoder 2

ππ-1

xk*

© Tallal Elshabrawy

Turbo Codes Performance

Principles of Turbo Codes: Keattisak Sripiman http:www.kmitl.ac.th/dslabs/TurboCodes

© Tallal Elshabrawy

Turbo Decoding

Turbo decoders rely on probabilistic decoding of component RSC decoders

Iteratively exchanging soft-output information between decoder 1 and decoder 2 before making a final deciding on the transmitted bits

As the number of iteration grows, the decoding performance improves

12

© Tallal Elshabrawy

Turbo Coded System Model

u = [u1, …, uk, …,uN] : vector of N information bits

xs = [x1s, …, xk

s, …,xNs] : vector of N systematic bits after turbo-coding of u

x1p = [x11p, …, xk

1p, …,xN1p] : vector of N parity bits from first encoder after turbo-coding of u

x2p = [x12p, …, xk

2p, …,xN2p] : vector of N parity bits from second encoder after turbo-coding of u

x = [x1s, x1

1p, x12p, …, xN

s, xN1p, xN

2p] : vector of length 3N of turbo-coded bits for u

y = [y1s, y1

1p, y12p, …, yN

s, yN1p, yN

2p] : vector of 3N received symbols corresponding to turbo-coded bits of u

ys = [y1s, …, yk

s, …,yNs] : vector of N received symbols corresponding to systematic bits in xs

y1p = [y11p, …, yk

1p, …,yN1p] : vector of N received symbols corresponding to first encoder parity bits in x1p

y2p = [y12p, …, yk

2p, …,yN2p] : vector of N received symbols corresponding to second encoder parity bits in x2p

u* = [u1*, …, uk*, …,uN*] : vector of N turbo decoder decision bits corresponding to u

13

k k*k

1 Pr u 1 Pr u 1u

1 otherwise

y y Pr[uk|y] is known as the a posteriori probability

of kth information bit

RSC Encoder

1

RSC Encoder

2

MU

X

π

Channel

DEM

UX

TurboDecoding

u=[u1, …, uk, …,uN]

x y

ys

y1p

y2p

u*

N-bitInterleaver

xs

x1p

x2p

© Tallal Elshabrawy

Log-Likelihood Ratio (LLR)

The LLR of an information bit uk is given by:

14

kk

k

Pr u 1L u ln

Pr u 1

Given that Pr[uk=+1]+Pr[uk=-1]=1

kL uk

k

Pr u 1e

1 Pr u 1

kL uk kPr u 1 e 1 Pr u 1

k

k k

L u

k L u L u

e 1Pr u 1

1 e 1 e

k

k

L u

k L u

ePr u 1

1 e

© Tallal Elshabrawy

Maximum A Posteriori Algorithm

15

k

k

k

Pr u 1L u ln

Pr u 1

yy

y

kk k

1 L u 0u * sign L u

1 otherwise

yy

RSC Encoder

1

RSC Encoder

2

MU

X

π

Channel

DEM

UX

TurboDecoding

u=[u1, …, uk, …,uN]

x y

ys

y1p

y2p

u*

N-bitInterleaver

xs

x1p

x2p

© Tallal Elshabrawy

Some Definitions & Conventions

16

s 1p 2p s 1p 2pa a a b b b[y ,y ,y ,...,y ,y ,y ]b

ays0

s1

s2

s3

s0

s1

s2

s3

Sk-1 Skuk yk

p

RSC Encoder

1

RSC Encoder

2

MU

X

π

Channel

DEM

UX

TurboDecoding

u=[u1, …, uk, …,uN]

x y

ys

y1p

y2p

u*

N-bitInterleaver

xs

x1p

x2p

uk=+1

uk=-1

k 1 ks',s S s',S s

k k 1 k s ',sPr S s S s' Pr u

N1y y

© Tallal Elshabrawy

Derivation of LLR

17

Define S(i) as the set of pairs of states (s’,s) such that the transition from Sk-1=s’ to Sk=s is caused by the input uk=i, i=0,1, i.e.,

S(0) ={(s0 ,s0), (s1 ,s3), (s2 ,s1), (s3 ,s2)}

S(1) ={(s0 ,s1), (s1 ,s2), (s2 ,s0), (s3 ,s3)}

( i )k k 1 k

S

Pr u i Pr S s',S s N N

1 1y y Remember Bayes’ Rule

Pr A,BPr B A

Pr A

( i )k 1 k

Sk

Pr S s',S s,

Pr u iPr

N1

N1 N

1

y

yy

s0

s1

s2

s3

s0

s1

s2

s3

Sk-1 Skuk yk

p

uk=+1

uk=-1

© Tallal Elshabrawy

Derivation of LLR

18

Define

k k 1 kσ s',s Pr S s',S s, N1y

k

k

k

Pr u 1L u ln

Pr u 1

N

1N

1 N

1

yy

y

( i )k 1 k

Sk

Pr S s',S s,

Pr u iPr

N1

N1 N

1

y

yy

(1) (1)

(0 )(0 )

k kS S

kkk

SS

σ s',s Pr σ s',s

L u ln lnσ s',sσ s',s Pr

N1

N

1 N1

y

yy

© Tallal Elshabrawy

Derivation of LLR

19

k k 1 kσ s',s Pr S s',S s, N1y

k k 1 k kσ s',s Pr S s',S s, ,y , k-1 N1 k+1y y

k k 1 k k k 1 k kσ s',s Pr S s',S s, ,y Pr S s',S s, ,y

N k-1 k-1k+1 1 1y y y

depends only on Sk and is independent on Sk-1, yk

and

k-11yN

k+1y

k k 1 k k kσ s',s Pr S s',S s, ,y Pr S s k-1 N1 k+1y y

k k 1 k k k 1 kσ s',s Pr S s', Pr S s,y S s', Pr S s

k-1 k-1 N1 1 k+1y y y

k k 1 k k k 1 kσ s',s Pr S s', Pr S s,y S s' Pr S s k-1 N1 k+1y y

Sk , yk depends only on Sk-1 and is independent onk-11y

© Tallal Elshabrawy

Derivation of LLR

20

k k 1 k k k 1 kσ s',s Pr S s', Pr S s,y S s' Pr S s k-1 N1 k+1y y

k kα s Pr S s, k1y

k kβ s Pr S s Nk+1y

k k k k 1γ s',s Pr S s,y S s'

k k 1 k kσ s',s α s' γ s',s β s

(1)

(0 )

k 1 k kS

kk 1 k k

S

α s' γ s',s β s

L u lnα s' γ s',s β s

N

1y

© Tallal Elshabrawy

Derivation of LLR

21

S0 Sk-1 Sk Sk+1 SN

y= y1 yk-1 yk yk+1 yN

k 1α s' kγ s',s kβ s

s0

s1

s2

s3

© Tallal Elshabrawy

Computation of αk(s)

22

k k k 1 kall s '

α s Pr S s, Pr S s',S s, k k1 1

y y

k 0 k 0 k 1 k 0all s '

α s Pr S s , Pr S s',S s , k k1 1

y y

Example

k 0 k 0 k 1 0 k 0 k 1 2 k 0α s Pr S s , Pr S s ,S s , Pr S s ,S s , k k k1 1 1y y y

k k 1 k k k 1all s '

α s Pr S s', Pr S s,y S s', k-1 k-1

1 1

y y

k k 1 k k k 1all s '

α s Pr S s', Pr S s,y S s' k-11

y

k k 1 kall s '

α s α s' γ s',s

© Tallal Elshabrawy 23

k k 1 kall s '

α s α s' γ s',s

Given the values of γk(s’,s) for all index k, the probability αk(s) can be forward recursively computed. The initial condition α0(s) depends on the initial state of the convolutional encoder

Computation of αk(s)

Forward Recursive Equation

0 0 0α s Pr S s, Pr S s 01y

0 0

0 0

α s 1

α s 0 s s

The encoder usually starts at state 0

© Tallal Elshabrawy

Backward Recursive Equation

Computation of βk(s)

24

k kβ s Pr S s Nk+1y

k k 1 k 1 k k k 1 k 1s ''

β s Pr S s'',y S s Pr S s,S s'',y Nk+2y

k k 1 k 1 k 1 ks ''

β s Pr S s'' Pr S s'',y S s Nk+2y

k k 1 k 1s ''

β s β s'' γ s,s''

Given the values of γk(s’,s) for all index k, the probability βk(s) can be backward recursively computed. The initial condition βN(s) depends on the final state of the trellis

First encoder usually finishes at state s0 N 0 Nβ s 1, β s 0 s 0

Second encoder usually has open trellis Nβ s 1 s

© Tallal Elshabrawy

Computation of γk(s’,s)

25

k k k k 1γ s',s Pr S s,y S s'

k k k 1 k k k 1γ s',s Pr y S s',S s Pr S s S s'

ak k k 1 k kγ s',s Pr y S s',S s Pr u

ak k k kγ s',s Pr y x Pr u

Note thats p s p

k k k k k ky y y ,x x x

s s p pk k k k k kPr y x Pr y x Pr y x

© Tallal Elshabrawy

Computation of γk(s’,s)

26

2 2s s p pk k k k

2 2

y x y x

2σ 2σk k 2 2

1 1Pr y x e e

2πσ 2πσ

2 2s s p pk k k k

2 2

y x y x

2σ 2σk k 2

1Pr y x e e

2πσ

kak

k

Pr u 1L u ln

Pr u 1

ak

a ak k

L ua a

k kL u L u

1 ePr u 1 ,Pr u 1

1 e 1 e

ak a

k ka

k

L u 2u L u 2a

k L u

ePr u e

1 e

2 2s s p pak k k k

k a2 2 k ka

k

y x y x L u 2u L u 22σ 2σ

k 2 L u

1 eγ s',s e e e

2πσ 1 e