Introduction to Turbo Codes · One Symbol in Two Codes More Codes Decoding a Turbo-Code Remarks,...

Three InventorsEncoder Structure

Decoding ProcedureRemarks, Exercises

Introduction to Turbo Codes

Frans M.J. Willems, Technische Universiteit Eindhoven

October 13, 2009

Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes



Three Inventors

Encoder StructureA Systematic Convolutional CodeParallel ConcatenationInterleaver

Decoding ProcedureBCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code

Remarks, ExercisesTrellis TerminationPuncturing a CodeExercises




Berrou, Glavieux, and Thitimajshima

Figure: Berrou and Glavieux.

1993 - ”Near Shannon Limit Error-Correcting Coding andDecoding: Turbo Codes,”Claude Berrou and Alain Glavieux, professors in ElectricalEngineering at Ecole Nationale Superieure desTelecommunications de Bretagne, invented in 1991 (withPunya Thitimajshima that developed later similar methodspublished in a doctorate thesis when he was a student inthe same research lab) of a groundbreaking quasi-optimalerror-correcting coding scheme called Turbo codes.Berrou, Glavieux, and Thitimajshima are the recipients of a1998 IEEE Information Theory Society Golden JubileeAward.Glavieux died on 25 September 2004 at the age of 55 fromillness. Thitimajshima died on 9 May 2006 at the age of 51from illness.




A Systematic Convolutional CodeParallel ConcatenationInterleaver

A Systematic Convolutional Code

D� � � �

� �- -

?

-

-

-

-

�

�

-

�

�

-

-

-

?

-

?

D

⊕

⊕

Du(t) u(t − 1)

v1(t)

v2(t)

u(t − 2)

x(t)

p(t)

x(t)

u(t − 2)u(t − 1)u(t)D

⊕

⊕� �

We can turn the non-systematic (feedforward) code

v1(t) = u(t)⊕ u(t − 2)

v2(t) = u(t)⊕ u(t − 1)⊕ u(t − 2) (1)

into a systematic (feedback) code by setting the first output v1(t) ≡ x(t). Then

u(t) = x(t)⊕ u(t − 2)

p(t) = v2(t) = u(t)⊕ u(t − 1)⊕ u(t − 2). (2)

We now call the second output v2(t) ≡ p(t), the parity output.





Transition Table

The transition table of the new code is:

u(t) s(t) = u(t − 1)u(t − 2) x(t) p(t) s(t + 1) = u(t)u(t − 1)0 0 , 0 0 0 0 , 01 0 , 0 1 1 1 , 01 0 , 1 0 0 1 , 00 0 , 1 1 1 0 , 00 1 , 0 0 1 0 , 11 1 , 0 1 0 1 , 11 1 , 1 0 1 1 , 10 1 , 1 1 0 0 , 1

Observe that u(t) is the input which results in the u(t)⊕ u(t − 2) = x(t).Note that this systematic feedback code corresponds to a state-diagram and trelliswith four states. The code sequences of the new code are identical to that of itsfeedforward counterpart.





Parallel Concatenation of Two Systematic Codes

Π

��

�� -

-

?

-

-

-

?- -

-?�

�?

-

--?

⊕

⊕

⊕

⊕

D D

DD

xt

p′′t

p′t

xt

��

The structure above is that of the parallel TURBO-encoder as proposed by Berrou,Glavieux, and Thitimajshima at the ICC in 1993. Observe two rate-1/2 systematicconvolutional encoders that produce two parity streams. Important is that thesystematic input sequence x1, x2, · · · , xT is interleaved before it is passed on to thesecond encoder. The rate of this turbo encoder is 1/3.





Interleaver

1

-

-

��)

��)

��)

-

��

��

��

? ??? 16

12

8

4

15

11

7

3

14

10

6

2

13

9

5

-

The systematic symbols are interleaved (permuted). We can use e.g. ablock-interleaver, here shown for T = 16. The sequence x1, x2, x3, · · · , x15, x16

(horizontal) is interleaved in order to get the sequence x1, x5, x9, x13, x2, x6, x10, x14, x3,x7, x11, x15, x4, x8, x12, x16 (vertical).Objective is that symbols which are close in one ordering are far apart in the otherordering.




BCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code

BCJR for a Systematic Code

Probabilistic structure:

sT+3

6

- - - - -��-

@@@R -

yptyx

t

ptxt

s1 s2 · · · st st+1 · · ·

6

Note that yxt and yp

t are the channel outputs resulting from inputs xt and pt .

Moreover we define yt

∆= (yx

t , ypt ).

We are interested in the a-posteriori symbol probability

P(Xt = 1|y1, · · · , y

t−1, y

t, y

t+1, · · · , y

T)

=P(y

1, · · · , y

t−1,Xt = 1, y

t, y

t+1, · · · , y

T)

P(y1, · · · , y

t−1, y

t, y

t+1, · · · , y

T)

. (3)





Since the denominator in (3) can be obtained from

P(y1, · · · , y

t−1, y

t, y

t+1, · · · , y

T)

= P(y1, · · · , y

t−1, xt = 0, y

t, y

t+1, · · · , y

T)

+P(y1, · · · , y

t−1, xt = 1, y

t, y

t+1, · · · , y

T). (4)

we are interested in the enumerator in (3). We can write

P(y1, · · · , y

t−1,Xt = 1, y

t, y

t+1, · · · , y

T)

=∑

s,p,s′P(y

1, · · · , y

t−1, St = s,Xt = 1,Pt = p, y

t, St+1 = s′, y

t+1, · · · , y

T)

=∑

s,p,s′P(y

1, · · · , y

t−1, St = s)P(Xt = 1)P(yx

t |Xt = 1)P(Pt = p|St = s,Xt = 1)

·P(ypt |Pt = p)P(St+1 = s′|St = s,Xt = 1)P(y

t+1, · · · , y

T|St+1 = s′)

=∑

s

P(y1, · · · , y

t−1, St = s)P(Xt = 1)P(yx

t |Xt = 1)P(ypt |Pt = p(s, 1))

·P(yt+1

, · · · , yT|St+1 = s∗(s, 1)). (5)

where p(s, x) is the parity function and s∗(s, x) the next-state function if s is thecurrent state and x the input and systematic output.





Observing that

αt(s) = P(y1, · · · , y

t−1,St = s)

βt+1(s′) = P(yt+1

, · · · , yT|St+1 = s′), (6)

and rearranging terms we obtain

P(y1, · · · , y

t−1,Xt = 1, y

t, y

t+1, · · · , y

T)

= P(Xt = 1)P(yxt |Xt = 1)

∑s

αt(s)P(ypt |Pt = p(s, 1))βt+1(s∗(s, 1))

P(y1, · · · , y

t−1,Xt = 0, y

t, y

t+1, · · · , y

T)

= P(Xt = 0)P(yxt |Xt = 0)

∑s

αt(s)P(ypt |Pt = p(s, 0))βt+1(s∗(s, 0)). (7)

The (a) factor P(Xt = x) is the a-priori probability, (b) factor P(yxt |Xt = x) is the

systematic channel probability, and the (c) factor∑s αt(s)P(yp

t |Pt = p(s, x))βt+1(s′(s, x)) is called extrinsic probability. Later we seethat it is crucial that we can split up P(y

1, · · · , y

t−1,Xt = x , y

t, y

t+1, · · · , y

T) in this

way.





The α- and β-recursions are standard, i.e.:

αt+1(s′) =∑

s,x→s′αt(s)P(Xt = x)P(yx

t |Xt = x)P(ypt |Pt = p(s, x)),

βt(s) =∑

x

P(Xt = x)P(yxt |Xt = x)P(yp

t |Pt = p(s, x))βt+1(s∗(s, x)). (8)





When a Systematic Symbol xij is in Two Codes

x′1

codevertical

horizontal code

x′′T−1

x′′j+1

x′′j−1

x′i+1x′i−1

.

.

.

.

.

.

· · ·· · ·

x′′1

x′T

x′′T

xijx′2

The symbols x ′1, x′2, · · · , x ′i−1, xij , x

′i+1, · · · , x

′T are the systematic symbols of a first

(horizontal) convolutional code, see figure.The symbols x ′′1 , x

′′2 , · · · , xj−1, xij , x

′′j+1, · · · , x

′′T are the systematic symbols of a second

(vertical) convolutional code.The rate of both codes is 1/2, hence the first and second code produce parity symbolsp′1, p

′2, · · · , p′T and p′′1 , p

′′2 , · · · , p′′T respectively. Note that there are two parity

symbols for xij .





A-Posteriori Probabilities for Systematic Symbol xij

We can write:

P(y ′1, · · · , y ′

i−1, y ′′

1, · · · , y ′′

j−1,Xij = 1, yx

ij , yp′

ij , yp′′

ij , y ′i+1, · · · , y ′

T, y ′′

j+1, · · · , y ′′

T)

=∑s′α′i (s

′)∑s′′

α′′j (s′′)P(Xij = 1)P(yxij |Xij = 1)

·P(yp′

ij |P′ij = p(s′, 1))P(yp′′

ij |P′′ij = p(s′′, 1))β′i+1(s∗(s′, 1))β′′j+1(s∗(s′′, 1))

= P(Xij = 1) · P(yxij |Xij = 1) ·

[∑s′α′i (s

′)P(yp′

ij |P′ij = p(s′, 1))β′i+1(s∗(s′, 1))

]

·[∑

s′′α′′j (s′′)P(yp′′

ij |P′′ij = p(s′′, 1))β′′j+1(s∗(s′′, 1))

]. (9)

Here we can see an (a) a-priori probability factor, a (b) systematic channel probabilityfactor, and (c) the extrinsic probability factor corresponding to the first code and (d)an extra extrinsic probability factor related to the second code. The last factor canbe considered as an adjustment factor for the a-priori probability.Note that α′i (s

′), α′′j (s′′), β′i+1(s′), and β′′j+1(s′′) are computed in the standard way.





Non-Standard α-Computation

If we want to compute other a-posteriori probabilities, for e.g. x ′i+1 then, as usual, wecan write:

P(y ′1, · · · , y ′

i−1, y ′′

1, · · · , y ′′

j−1, yx

ij , yp′

ij , yp′′

ij , y ′′j+1, · · · , y ′′

T,

X ′i+1 = 1, yx′i+1, y

p′

i+1, y′i+2, · · · , y ′

T)

=∑

s

α′i+1(s)P(X ′i+1 = 1)P(yx′i+1|X

′i+1 = 1)P(yp′

i+1|P′i+1 = p(s, 1))β′i+2(s∗(s, 1))

= P(X ′i+1 = 1) · P(yx′i+1|X

′i+1 = 1)

·[∑

s

α′i+1(s)P(yp′

i+1|P′i+1 = p(s, 1))β′i+2(s∗(s, 1))

], (10)

where α′i+1(s) is computed in a non-standard way as follows:

α′i+1(s′) =∑

s,x→s′α′i (s)P(Xij = x)P(yx

ij |Xij = x)P(yp′

ij |P′ij = p(s, x))

[∑s′′

α′′j (s′′)P(yp′′

ij |P′′ij = p(s′′, x))β′′j+1(s∗(s′′, x))

], (11)

so in this part of the horizontal α′-recursion the a-priori probability is ”adjusted” bythe extrinsic probability of the vertical code for symbol Xij = x .





All Systematic Symbols of the First Code are in Different Second Codes

x1

.

.

.

xT

x4

x3

x2

PROCEDURE:

1. Extrinsic probabilities for all systematic symbols x1, x2, · · · , xT are obtained fromT horizontal BCJR-decoding actions.

2. These extrinsic probabilities are fed to the vertical BCJR-decoding action, wherethey are used as (adjustments of the) a-priori probabilities.

3. This leads to α- and β-recursions as in (11) and a-posteriori computations as in(9).





A Third Series of Codes, Now Vertical Again, ...

xT

x1

x2

.

.

.





All Codes are Assumed to be Different!

We say that the final (vertical) code is at level 0. The horizontal codes are at level 1,and the vertical codes at level 2.PROCEDURE:

1. First the vertical codes at level 2 are BCJR-decoded to obtain the extrinsicprobabilities for all the symbols which are in a horizontal code at level 1. Notethat this does not include the symbols x1, x2, · · · , xT in the (final) vertical codeat level 0.

2. These extrinsic probabilities are then fed to the horizontal BCJR-decoders, wherethey are used as (adjustments of the) a-priori probabilities. These horizontaldecoding actions result again in extrinsic probabilities, now for the symbolsx1, x2, · · · , xT .

3. The extrinsic probabilities from the horizontal actions are used in the final verticalBCJR-decoding action as (adjustments of the) a-priori probabilities.

Note that similar procedures apply to systems with more levels !!!





Decoding a Turbo Code (Parallel Concatenation)

First note that the systematic input sequence x1, x2, · · · , xT can be decoded in twodifferent ways:

(i) using the channel-output streams (yx1 , y

p′

1 ), (yx2 , y

p′

2 ), · · · , (yxT , y

p′

T ) (horizontalcode) or

(ii) using the channel-output streams (yx1 , y

p′′

1 ), (yx2 , y

p′′

2 ), · · · , (yxT , y

p′′

T ) (verticalcode).A turbo decoder iterates between the horizontal and vertical decoding actions.Decoding is done exactly as we have described on the previous slide.Although there we have assumed that all codes are different, in reality we have onlyone horizontal code and one vertical code. The interleaver breaks up relations betweensymbols and makes the assumption that all codes are different reasonable.The fact that not all codes are different, makes the suggested procedure suboptimal.In practise the performance is excellent however.





Turbo Decoding Procedure:

1. Do a first horizontal BCJR decoding-action. All a-priori probabilities are assumedto be equal. Send the computed ’horizontal’ extrinsic probabilities to the verticalBCJR-decoder.

2. Use the extrinsic probabilities from the previous horizontal decoding action (afterinterleaving) as a-priori probabilities in a next vertical BCJR decoding-action.Now compute the ’vertical’ extrinsic probabilities and send them to the horizontalBCJR-decoder.

3. Use the extrinsic probabilities from the previous vertical decoding action (afterinterleaving) as a-priori probabilities in a next horizontal BCJR decoding-action.Compute the ’horizontal’ extrinsic probabilities and send them to the verticalBCJR decoder. Goto 2 for another iteration or to 4 to stop.

4. Use the extrinsic probabilities from the previous horizontal decoding action (afterinterleaving) as a-priori probabilities in a last vertical BCJR decoding-action.Compute the ’horizontal’ a-posteriori probabilities for all systematic symbols andmake decisions.





Decoder Architecture:

x

-

-

-

�

-

-

� �

-

�

--

DECODER

CHANNEL

Π−1

a-post.vert.

a-pri.a-pri.

vert.ext.

hor.ext.Π

Π

(modified)

BCJR

BCJR

modified

yp′′

yp′

yx

ch.

ch.

ch.

p′′

p′

-




Trellis TerminationPuncturing a CodeExercises

Trellis Termination

How can we terminate the two trellises simultaneously?

To see how this can be done, observe that each weight-one input sequence results insome final state-pair. For an arbitrary input sequence the final state-pair is a sum ofsuch pairs (linearity).Now we can select a set S of weight-one input sequences that spans the space of allpossible final state-pairs. This set can not be larger than twice the number ofdelay-elements of the component codes.Now symbols at the positions corresponding to the set S of weight-one inputsequences can be used to compensate every state-pair resulting from the symbols atthe remaining positions.





Puncturing our Turbo Code

Punture-table:1 11 00 1

The top row of this matrix shows that all systematic symbols (at odd and even t) areactually transmitted. The second row corresponds to the first parity symbol. Only forodd t this first parity symbol is transmitted, for even t it is punctured. The third rowindicates that the second parity symbol is punctured for odd t and transmitted foreven t.Punctured symbols are not actually transmitted. We may assume that these symbolsalways result in an erasure ”?” as channel-output, i.e.

P(Y =?|P = 1) = P(Y =?|P = 0) = 1.

Note that the rate is increased by puncturing. For our turbo code it brings the ratefrom 1/3 up to 1/2.





Two Exercises

1. Also the β′i (s) for all s ∈ {00, 01, 10, 11} are not computed in a standard way.Give the formula analogous to (11) that describes how this is done.

2. A single-parity-check code of length 4 is our horizontal code. Our vertical code isa single parity-check code of length 3. The second symbol of both codes is thesame, see below. The parity-symbol of the horizontal code is x ′4 hencex ′4 = x ′1 ⊕ x22 ⊕ x ′3. The parity-symbol of the vertical code x ′′3 = x ′′1 ⊕ x22. Theindependent information symbols x ′1, x

′′1 , x22, and x ′3 are a-priori uniform.

x ′′1x ′1 x22 x ′3 x ′4

x ′′3

Now suppose that all code-symbols are transmitted via a BSC with cross-overprobability p = 0.1 and that

01 0 0 0

0

is received. Compute the a-posteriori probabilities for symbol X22.


Date post:	19-Mar-2020
Category:	Documents
Upload:	others
View:	35 times
Download:	0 times

Introduction to Turbo Codes · One Symbol in Two Codes More Codes Decoding a Turbo-Code Remarks,...

Documents