Post on 19-Mar-2020
transcript
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
Introduction to Turbo Codes
Frans M.J. Willems, Technische Universiteit Eindhoven
October 13, 2009
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
Three Inventors
Encoder StructureA Systematic Convolutional CodeParallel ConcatenationInterleaver
Decoding ProcedureBCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code
Remarks, ExercisesTrellis TerminationPuncturing a CodeExercises
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
Berrou, Glavieux, and Thitimajshima
Figure: Berrou and Glavieux.
1993 - ”Near Shannon Limit Error-Correcting Coding andDecoding: Turbo Codes,”Claude Berrou and Alain Glavieux, professors in ElectricalEngineering at Ecole Nationale Superieure desTelecommunications de Bretagne, invented in 1991 (withPunya Thitimajshima that developed later similar methodspublished in a doctorate thesis when he was a student inthe same research lab) of a groundbreaking quasi-optimalerror-correcting coding scheme called Turbo codes.Berrou, Glavieux, and Thitimajshima are the recipients of a1998 IEEE Information Theory Society Golden JubileeAward.Glavieux died on 25 September 2004 at the age of 55 fromillness. Thitimajshima died on 9 May 2006 at the age of 51from illness.
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
A Systematic Convolutional CodeParallel ConcatenationInterleaver
A Systematic Convolutional Code
D� � � �
� �- -
?
-
-
-
-
�
�
-
�
�
-
-
-
?
-
?
D
⊕
⊕
Du(t) u(t − 1)
v1(t)
v2(t)
u(t − 2)
x(t)
p(t)
x(t)
u(t − 2)u(t − 1)u(t)D
⊕
⊕� �
We can turn the non-systematic (feedforward) code
v1(t) = u(t)⊕ u(t − 2)
v2(t) = u(t)⊕ u(t − 1)⊕ u(t − 2) (1)
into a systematic (feedback) code by setting the first output v1(t) ≡ x(t). Then
u(t) = x(t)⊕ u(t − 2)
p(t) = v2(t) = u(t)⊕ u(t − 1)⊕ u(t − 2). (2)
We now call the second output v2(t) ≡ p(t), the parity output.
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
A Systematic Convolutional CodeParallel ConcatenationInterleaver
Transition Table
The transition table of the new code is:
u(t) s(t) = u(t − 1)u(t − 2) x(t) p(t) s(t + 1) = u(t)u(t − 1)0 0 , 0 0 0 0 , 01 0 , 0 1 1 1 , 01 0 , 1 0 0 1 , 00 0 , 1 1 1 0 , 00 1 , 0 0 1 0 , 11 1 , 0 1 0 1 , 11 1 , 1 0 1 1 , 10 1 , 1 1 0 0 , 1
Observe that u(t) is the input which results in the u(t)⊕ u(t − 2) = x(t).Note that this systematic feedback code corresponds to a state-diagram and trelliswith four states. The code sequences of the new code are identical to that of itsfeedforward counterpart.
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
A Systematic Convolutional CodeParallel ConcatenationInterleaver
Parallel Concatenation of Two Systematic Codes
Π
�� ���� ��
�� �� �� ��-
-
?
-
-
-
?- -
-?�
�?
-
--?
⊕
⊕
⊕
⊕
D D
DD
xt
p′′t
p′t
xt
�� ��
The structure above is that of the parallel TURBO-encoder as proposed by Berrou,Glavieux, and Thitimajshima at the ICC in 1993. Observe two rate-1/2 systematicconvolutional encoders that produce two parity streams. Important is that thesystematic input sequence x1, x2, · · · , xT is interleaved before it is passed on to thesecond encoder. The rate of this turbo encoder is 1/3.
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
A Systematic Convolutional CodeParallel ConcatenationInterleaver
Interleaver
1
-
-
���������)
���������)
���������)
-
����������
����������
����������
? ??? 16
12
8
4
15
11
7
3
14
10
6
2
13
9
5
-
The systematic symbols are interleaved (permuted). We can use e.g. ablock-interleaver, here shown for T = 16. The sequence x1, x2, x3, · · · , x15, x16
(horizontal) is interleaved in order to get the sequence x1, x5, x9, x13, x2, x6, x10, x14, x3,x7, x11, x15, x4, x8, x12, x16 (vertical).Objective is that symbols which are close in one ordering are far apart in the otherordering.
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
BCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code
BCJR for a Systematic Code
Probabilistic structure:
sT+3
6
- - - - -����-
@@@R -
yptyx
t
ptxt
s1 s2 · · · st st+1 · · ·
6
Note that yxt and yp
t are the channel outputs resulting from inputs xt and pt .
Moreover we define yt
∆= (yx
t , ypt ).
We are interested in the a-posteriori symbol probability
P(Xt = 1|y1, · · · , y
t−1, y
t, y
t+1, · · · , y
T)
=P(y
1, · · · , y
t−1,Xt = 1, y
t, y
t+1, · · · , y
T)
P(y1, · · · , y
t−1, y
t, y
t+1, · · · , y
T)
. (3)
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
BCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code
Since the denominator in (3) can be obtained from
P(y1, · · · , y
t−1, y
t, y
t+1, · · · , y
T)
= P(y1, · · · , y
t−1, xt = 0, y
t, y
t+1, · · · , y
T)
+P(y1, · · · , y
t−1, xt = 1, y
t, y
t+1, · · · , y
T). (4)
we are interested in the enumerator in (3). We can write
P(y1, · · · , y
t−1,Xt = 1, y
t, y
t+1, · · · , y
T)
=∑
s,p,s′P(y
1, · · · , y
t−1, St = s,Xt = 1,Pt = p, y
t, St+1 = s′, y
t+1, · · · , y
T)
=∑
s,p,s′P(y
1, · · · , y
t−1, St = s)P(Xt = 1)P(yx
t |Xt = 1)P(Pt = p|St = s,Xt = 1)
·P(ypt |Pt = p)P(St+1 = s′|St = s,Xt = 1)P(y
t+1, · · · , y
T|St+1 = s′)
=∑
s
P(y1, · · · , y
t−1, St = s)P(Xt = 1)P(yx
t |Xt = 1)P(ypt |Pt = p(s, 1))
·P(yt+1
, · · · , yT|St+1 = s∗(s, 1)). (5)
where p(s, x) is the parity function and s∗(s, x) the next-state function if s is thecurrent state and x the input and systematic output.
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
BCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code
Observing that
αt(s) = P(y1, · · · , y
t−1,St = s)
βt+1(s′) = P(yt+1
, · · · , yT|St+1 = s′), (6)
and rearranging terms we obtain
P(y1, · · · , y
t−1,Xt = 1, y
t, y
t+1, · · · , y
T)
= P(Xt = 1)P(yxt |Xt = 1)
∑s
αt(s)P(ypt |Pt = p(s, 1))βt+1(s∗(s, 1))
P(y1, · · · , y
t−1,Xt = 0, y
t, y
t+1, · · · , y
T)
= P(Xt = 0)P(yxt |Xt = 0)
∑s
αt(s)P(ypt |Pt = p(s, 0))βt+1(s∗(s, 0)). (7)
The (a) factor P(Xt = x) is the a-priori probability, (b) factor P(yxt |Xt = x) is the
systematic channel probability, and the (c) factor∑s αt(s)P(yp
t |Pt = p(s, x))βt+1(s′(s, x)) is called extrinsic probability. Later we seethat it is crucial that we can split up P(y
1, · · · , y
t−1,Xt = x , y
t, y
t+1, · · · , y
T) in this
way.
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
BCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code
The α- and β-recursions are standard, i.e.:
αt+1(s′) =∑
s,x→s′αt(s)P(Xt = x)P(yx
t |Xt = x)P(ypt |Pt = p(s, x)),
βt(s) =∑
x
P(Xt = x)P(yxt |Xt = x)P(yp
t |Pt = p(s, x))βt+1(s∗(s, x)). (8)
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
BCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code
When a Systematic Symbol xij is in Two Codes
x′1
codevertical
horizontal code
x′′T−1
x′′j+1
x′′j−1
x′i+1x′i−1
.
.
.
.
.
.
· · ·· · ·
x′′1
x′T
x′′T
xijx′2
The symbols x ′1, x′2, · · · , x ′i−1, xij , x
′i+1, · · · , x
′T are the systematic symbols of a first
(horizontal) convolutional code, see figure.The symbols x ′′1 , x
′′2 , · · · , xj−1, xij , x
′′j+1, · · · , x
′′T are the systematic symbols of a second
(vertical) convolutional code.The rate of both codes is 1/2, hence the first and second code produce parity symbolsp′1, p
′2, · · · , p′T and p′′1 , p
′′2 , · · · , p′′T respectively. Note that there are two parity
symbols for xij .
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
BCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code
A-Posteriori Probabilities for Systematic Symbol xij
We can write:
P(y ′1, · · · , y ′
i−1, y ′′
1, · · · , y ′′
j−1,Xij = 1, yx
ij , yp′
ij , yp′′
ij , y ′i+1, · · · , y ′
T, y ′′
j+1, · · · , y ′′
T)
=∑s′α′i (s
′)∑s′′
α′′j (s′′)P(Xij = 1)P(yxij |Xij = 1)
·P(yp′
ij |P′ij = p(s′, 1))P(yp′′
ij |P′′ij = p(s′′, 1))β′i+1(s∗(s′, 1))β′′j+1(s∗(s′′, 1))
= P(Xij = 1) · P(yxij |Xij = 1) ·
[∑s′α′i (s
′)P(yp′
ij |P′ij = p(s′, 1))β′i+1(s∗(s′, 1))
]
·[∑
s′′α′′j (s′′)P(yp′′
ij |P′′ij = p(s′′, 1))β′′j+1(s∗(s′′, 1))
]. (9)
Here we can see an (a) a-priori probability factor, a (b) systematic channel probabilityfactor, and (c) the extrinsic probability factor corresponding to the first code and (d)an extra extrinsic probability factor related to the second code. The last factor canbe considered as an adjustment factor for the a-priori probability.Note that α′i (s
′), α′′j (s′′), β′i+1(s′), and β′′j+1(s′′) are computed in the standard way.
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
BCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code
Non-Standard α-Computation
If we want to compute other a-posteriori probabilities, for e.g. x ′i+1 then, as usual, wecan write:
P(y ′1, · · · , y ′
i−1, y ′′
1, · · · , y ′′
j−1, yx
ij , yp′
ij , yp′′
ij , y ′′j+1, · · · , y ′′
T,
X ′i+1 = 1, yx′i+1, y
p′
i+1, y′i+2, · · · , y ′
T)
=∑
s
α′i+1(s)P(X ′i+1 = 1)P(yx′i+1|X
′i+1 = 1)P(yp′
i+1|P′i+1 = p(s, 1))β′i+2(s∗(s, 1))
= P(X ′i+1 = 1) · P(yx′i+1|X
′i+1 = 1)
·[∑
s
α′i+1(s)P(yp′
i+1|P′i+1 = p(s, 1))β′i+2(s∗(s, 1))
], (10)
where α′i+1(s) is computed in a non-standard way as follows:
α′i+1(s′) =∑
s,x→s′α′i (s)P(Xij = x)P(yx
ij |Xij = x)P(yp′
ij |P′ij = p(s, x))
[∑s′′
α′′j (s′′)P(yp′′
ij |P′′ij = p(s′′, x))β′′j+1(s∗(s′′, x))
], (11)
so in this part of the horizontal α′-recursion the a-priori probability is ”adjusted” bythe extrinsic probability of the vertical code for symbol Xij = x .
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
BCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code
All Systematic Symbols of the First Code are in Different Second Codes
x1
.
.
.
xT
x4
x3
x2
PROCEDURE:
1. Extrinsic probabilities for all systematic symbols x1, x2, · · · , xT are obtained fromT horizontal BCJR-decoding actions.
2. These extrinsic probabilities are fed to the vertical BCJR-decoding action, wherethey are used as (adjustments of the) a-priori probabilities.
3. This leads to α- and β-recursions as in (11) and a-posteriori computations as in(9).
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
BCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code
A Third Series of Codes, Now Vertical Again, ...
xT
x1
x2
.
.
.
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
BCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code
All Codes are Assumed to be Different!
We say that the final (vertical) code is at level 0. The horizontal codes are at level 1,and the vertical codes at level 2.PROCEDURE:
1. First the vertical codes at level 2 are BCJR-decoded to obtain the extrinsicprobabilities for all the symbols which are in a horizontal code at level 1. Notethat this does not include the symbols x1, x2, · · · , xT in the (final) vertical codeat level 0.
2. These extrinsic probabilities are then fed to the horizontal BCJR-decoders, wherethey are used as (adjustments of the) a-priori probabilities. These horizontaldecoding actions result again in extrinsic probabilities, now for the symbolsx1, x2, · · · , xT .
3. The extrinsic probabilities from the horizontal actions are used in the final verticalBCJR-decoding action as (adjustments of the) a-priori probabilities.
Note that similar procedures apply to systems with more levels !!!
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
BCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code
Decoding a Turbo Code (Parallel Concatenation)
First note that the systematic input sequence x1, x2, · · · , xT can be decoded in twodifferent ways:
(i) using the channel-output streams (yx1 , y
p′
1 ), (yx2 , y
p′
2 ), · · · , (yxT , y
p′
T ) (horizontalcode) or
(ii) using the channel-output streams (yx1 , y
p′′
1 ), (yx2 , y
p′′
2 ), · · · , (yxT , y
p′′
T ) (verticalcode).A turbo decoder iterates between the horizontal and vertical decoding actions.Decoding is done exactly as we have described on the previous slide.Although there we have assumed that all codes are different, in reality we have onlyone horizontal code and one vertical code. The interleaver breaks up relations betweensymbols and makes the assumption that all codes are different reasonable.The fact that not all codes are different, makes the suggested procedure suboptimal.In practise the performance is excellent however.
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
BCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code
Turbo Decoding Procedure:
1. Do a first horizontal BCJR decoding-action. All a-priori probabilities are assumedto be equal. Send the computed ’horizontal’ extrinsic probabilities to the verticalBCJR-decoder.
2. Use the extrinsic probabilities from the previous horizontal decoding action (afterinterleaving) as a-priori probabilities in a next vertical BCJR decoding-action.Now compute the ’vertical’ extrinsic probabilities and send them to the horizontalBCJR-decoder.
3. Use the extrinsic probabilities from the previous vertical decoding action (afterinterleaving) as a-priori probabilities in a next horizontal BCJR decoding-action.Compute the ’horizontal’ extrinsic probabilities and send them to the verticalBCJR decoder. Goto 2 for another iteration or to 4 to stop.
4. Use the extrinsic probabilities from the previous horizontal decoding action (afterinterleaving) as a-priori probabilities in a last vertical BCJR decoding-action.Compute the ’horizontal’ a-posteriori probabilities for all systematic symbols andmake decisions.
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
BCJR for a Systematic CodeOne Symbol in Two CodesMore CodesDecoding a Turbo-Code
Decoder Architecture:
x
-
-
-
�
-
-
� �
-
�
--
DECODER
CHANNEL
Π−1
a-post.vert.
a-pri.a-pri.
vert.ext.
hor.ext.Π
Π
(modified)
BCJR
BCJR
modified
yp′′
yp′
yx
ch.
ch.
ch.
p′′
p′
-
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
Trellis TerminationPuncturing a CodeExercises
Trellis Termination
How can we terminate the two trellises simultaneously?
To see how this can be done, observe that each weight-one input sequence results insome final state-pair. For an arbitrary input sequence the final state-pair is a sum ofsuch pairs (linearity).Now we can select a set S of weight-one input sequences that spans the space of allpossible final state-pairs. This set can not be larger than twice the number ofdelay-elements of the component codes.Now symbols at the positions corresponding to the set S of weight-one inputsequences can be used to compensate every state-pair resulting from the symbols atthe remaining positions.
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
Trellis TerminationPuncturing a CodeExercises
Puncturing our Turbo Code
Punture-table:1 11 00 1
The top row of this matrix shows that all systematic symbols (at odd and even t) areactually transmitted. The second row corresponds to the first parity symbol. Only forodd t this first parity symbol is transmitted, for even t it is punctured. The third rowindicates that the second parity symbol is punctured for odd t and transmitted foreven t.Punctured symbols are not actually transmitted. We may assume that these symbolsalways result in an erasure ”?” as channel-output, i.e.
P(Y =?|P = 1) = P(Y =?|P = 0) = 1.
Note that the rate is increased by puncturing. For our turbo code it brings the ratefrom 1/3 up to 1/2.
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes
Three InventorsEncoder Structure
Decoding ProcedureRemarks, Exercises
Trellis TerminationPuncturing a CodeExercises
Two Exercises
1. Also the β′i (s) for all s ∈ {00, 01, 10, 11} are not computed in a standard way.Give the formula analogous to (11) that describes how this is done.
2. A single-parity-check code of length 4 is our horizontal code. Our vertical code isa single parity-check code of length 3. The second symbol of both codes is thesame, see below. The parity-symbol of the horizontal code is x ′4 hencex ′4 = x ′1 ⊕ x22 ⊕ x ′3. The parity-symbol of the vertical code x ′′3 = x ′′1 ⊕ x22. Theindependent information symbols x ′1, x
′′1 , x22, and x ′3 are a-priori uniform.
x ′′1x ′1 x22 x ′3 x ′4
x ′′3
Now suppose that all code-symbols are transmitted via a BSC with cross-overprobability p = 0.1 and that
01 0 0 0
0
is received. Compute the a-posteriori probabilities for symbol X22.
Frans M.J. Willems, Technische Universiteit Eindhoven Introduction to Turbo Codes