-,
... 1
r -,
\ 1
"
"
o
"
"rACK t\LtivtU 'l"rlMS FOR DECODING CONVOLUTJ ONAL CODES
-- ..
, \
1
(
.. '
/
'.
, , ~ .:.,...- .";.':.-~.~_~'.L_'.~:jJrtrnttr~~;
...................... ------------~--~--
\
.'
r
MJjLTIPlE - PATH' STAK ALGORITHMS FOR , ,
~ DECODING CONVOlUT10NAl.. CODES
by
Da,:id Haccoun, B. Sc. A. & Ing. Phys. (U of M) S. M. (M. 1: T. )
Deportment of Electrical Engineerrn('
A thesis ~ubmitted to the Faculty of Gradua te Studies and Research
in' portial fulfjllm~nt of tt'e requiremenh for the degree of
Doctor of Philo50P~Y. ! : (
- \
\ . Depaftment of Electreol Engineering
.... /"
( " - 1
\
,.
,
j
, ..
"
MUlTIPlE-PATH STAK AlGORITHMS FOR
DECODING CONVOLUTlONAl CODES
by,
David Hoccoun, B.Sc'.A, & Ing, Phys. (Universitde Montral)
S . M. (M. 1. T . )
Def?Ortment of Electricel En9in~erin9
.. , (
"
-'
ABSTRACT
~
. A new class of generalized stack a/gorithms for decodi'1g convol-
utional codes is presented. It is based on th: Zigangirov-JeHnek olgo.rit~m but
insteqd of extending just th .top node of the st&ck at ail times, a nu~ber of th
"--mest likely paths are simultaneously extended. This number of paths moy be 1 ) ,
'
Il
J ,. , .. SOMMAIRE ..
, Une classe nouvelle d'algorithmes gnraliss utilisant une pile
pour le dcodage des c-?des de convolution est propose. 'Ces algorithmes drivent
de l'algorithme de Zigangirov et Jelinek, mais au lieu de prolonger seulement le , -
" .sommet de la pile, on prolonge simultanment un sous-ensemble des chemins les
plus vraisemblables. Ce nombre de chemins peut demeurer fixe ou peut tre vari , 'J
pour adopter l'effort de.dco'dage courant a Ptat actuel du bruit aans 'lb voie de
transmission. De plus la structure en treillis des codes de convolution est mise en
va'ieur par la reconnaissance et l'exploitatim de tOute convergence entre les chemins
. c:;xplors. La variabilit du nombre de calculs peut ainsi tre rduite jusqu ' une \
limite the'orique correspondant l'algorithme idal. De plus la probabilit d'erreur
est borne (borne suprieure) par cetle u dcodeur squentiel. Ces algorithmes , r
font le 1 ien entre le dcodeur squentieJ et le dcodeur Ide Viterbi.
Le chemin correct tant prolong avant d'atteindre le sommet de la o
pife, la chut apparente de sa mtrique cumulee est rduite'. Bien que la queue de la , 1
distribution de l'effort de, calcul suive la loi Pareto, 'on dmontre et on observe pai' . . simulatiol;l avec des codes de taux 1/2 et de IOl1guurs de contrainte courtes
,t
(K s; 9)/que ,compares ou dcodeur squentieJ,les'distributions du nombre de calculs
par bit de
"
\ i ii , <
, ACKNOWlEDGEMENT
, 1 am greatly ~.in,debted to Professor Mic~ael J. Ferguson {or h is - ,
\
guidance th~ughout *is resebrch. 1 wish Id express my sincere thanks for his
stimulating discussions anq criticisms, and for his many helpful suggestions during
the preparation of this dissertation.
The financial support of the E.cole Polytechnique de Montral ~ - ,
and Hydro-Qubec is gratefully acknowledged. Sp,cial tha~ks are also due to
my wife, Lyson for her encouragement during the course df this research. '.
/
, ,
\ '
(1,
"
.. : " . -, " .
:
..
l' 1 1 1
. -
:
o
...
., 4 . ~ -" , iv
e' ~ ~ABLE OF CONTENTS .?
/ - Page ...
~StRACt
SOM~AIRE ii
ACKNOWLEDGEMENT iii
TABLE OF CONTENTS IV
CHAPTER- . INTRODUC'HON . " 1.1 Outline of the Thesis' , t 6
, "1 .. 2 Claim of Cont;ibutions of this. Research 8
* C:HAPTER " STRUCTURE OF 'CONVOLUTIONAL ,CODES ln 2. 1 Conv.plutionaJ Encoder 10 2.2 The Structure of Convo'uti~nal Codes 17 2.2. 1 T ree StructUTe
\ 2'.2.2 State DiogrQ'hl 21 ,-1 2.2.3 T re" is Structure 24
~ t CHAPTER Iii DECODI NG OF CONVOlUTlONAL CODES 27
. \. 3.1 Introduction . ~ 27 3.2 Maximum likelihood Decoding of Convolutiont;J1
.. .6l "Codes. 31 :' ' .
~.(.,I,' . 3.3 35 Suboptimum DeC~ing . "l 3.4 Probabilistic Dec ding Algorithms 37 ',' 1 -" .. ~ . ...... ~< "}Ii: 3.4. 1 Tree Search: Se ential Oecoding 38 ')
3.4.2 Treflis Sarch: Viterbi\ Oecoding 55 3.4.3 Compa~iSO!:l5 and limitatio~s of Sequentil and
Viterbi Decoding 63
CHAPTER ' IV MULTIPLE l'ATH' SJACK OECODING\AlGORITHM 67 .,
4. 1 Introduction: Generalized Stock Algorith~ 67 4.2 Exploitation of the Remergers 70 4.2.1 Z-J,Algorithm With a Purging Cycle 75 ~.2.2 "" 79 c Purgi.ng Procedure ~ , 4.2.3 Speeding Up the Procedur'e 81 e, 4.3 Z-) Algorithm With a Multiple Path Extension Cycle 84 ~
" "
~.
,r . -v 0 .
/
" Pag~ ..
4.~. 1 Search Prope/fies ofS'equ~n~ol Deco"ding 85 4.3.2 . Mu Itiple Poth E~ten5ion f" 91 4.3.-3 An' M-pth Generalized Stock AlgOr'ithm 93 ,. 4.3.4 f;~mputotionol B~haviour of ~he M-.Pa"th Algbrithm 95-4.3.5 rror'Probobi 1 ity 104 4:4' '- CCI1}pUter Simulation Results 110 4.4. l
b ,. , 110 Sysfem Description .
...., 4.41:'2 Results of th Simulation ,; 115 , . . ..
/ CHAPTER V "ADAPTIVE SEQUENTIAL DECODING 135
\ 5. l Variants to th.e M-path AJgorithm 135 .5.2 Ideol Adoptive Stock Algorithm 143 5.3 An Adoptive Stock Algorithm 149 5.4 Model of the Adoptive Stock Algo,ithm 152 5.4.1 Average Distance Between No-Search Node.s 155
> . 5.4.2 SE"orch Mode Probobility .. 164 5.5 Computer Simulation Results 170
CIiAPTER VI CONCLUSIONS AND SUGGESTIONS FOR , FURTHER RESEARCH 185
APPENDIX THE GENERALIZED STACK DECODER' 189
APPENDIX /1 MARKOV CHAIN MODEL OF THE C'oRRECT .-PATH METRIC 193 ..
J
. APPENOIX "' BRANCHING PROCESS.MODEL OF THE INCORRECT PATHS 204 APPENOIX IV. SEARCH MODE PROBABILITY Z14
.-1
REFERENCES 218
.",. , .'" ~ l' ,t' .. '
" ~ '\ ~ . .
#
J
'e ,. .. ~
\
" } J
,.
'- ,
'tI CHAPTER
INTRODUCTION
.. Communication i,s essentially the process of transmitting information , from one point t~ another through a noisy channel: ln a generol communi~atton , . s;stem,shown i~ Fig. (1.1), a message sou.rce generates m~ssages that must b~ transmifted to a user over the channel. A specifie signol is assigned to each of
,1
the possib)e M messotles wh ich the systm can transmit, nd the selection rU,le . , ._ ~ thot Jssigns a transmitted signol tcf'each- possible messa9~ is cailed the code. , --
Beca~se of the channel noise th~ tra~smitted signal does ~ot ~rrive at the receiver "'. " l
_ ~xactly as transmitted, and hence rt'brs ore made in cOflveying the source message
, to the user. The coding theorem of Shannon (1948) demonstrates the existence of
~ .. (,
codes that achieve reliable communication if and oilly if the infornption transmission
rate R is less than some maximum rate C - cal/ed th~ channel capacity.' If ~
Since Shannon 's r.sult ~ppe~red, ofnsiderob'e omoun! of work hos been concerned with the se~rch of coding-deoding tchniques that permit commun-
"1\ ication with low error probobility, and which are simple to implemnt. The first code "
investigated were cal/ed bloGk codes. In these codes, a s~uence Jf K information
symbols is encoded in a bl9ck of N ;ymbols to b'e transmitted over the channel. ,
Block codes h~ve been studied extensively (8erlekomp ~ 968, Gal/oger 1968).
'Atthough Shannon prpved that block codes exist for which the error probability 1
',J " d - ,:fecrease5 exponentially with the black length N, th long codes which are simply ,~ -
~ ,,', ,..." ~ .. '" ": d''coded yi~ld a significantly greater ~rror probability than predicted by the theory.
, ,
l '
..
/-~ 2 '~ ,
" 1 1 e. '.
"
... II) III
" ::>
... ( , , , p
,. -;, . .. . f1
... ., f " / 0
" , 0 U II)
q If . , / " ~
", . ~ ;JI i' "
1 .
, (., ~
E -' II) ~ .... .., ~. ~I III ~
,$
~~
c: -1 ~Q -, ....
II) 0,
. 0 ..s: , U
~ c: ,~
"
E .. ~
Q U , .
, '0 ... QI . c: QI
C> . , ... II)
"8 -u ... c:
w (
... -II) ... :> , .2> . u.
... .
. e
...
. 1
..
..
, .
, . p
. ,
.,.. .
.. . ~.. :\. ~ Convolutional cofes ore codes which are par'ticu1orly suitoble ,. ,-
when the inf~rmation symbols to be tronsmitted arr.~v~ seria'lIy rother t~an .in r-
.. ' 'arge I:>I9cks. Randomly chosen 'convolutional ~odes are known which achieve
" . . . er r probobiliHS decreasing xponentially with the constr.nt iength-of the
-
..
." <
"
, encod. Jn g~neral convolutional codFs outperform block c~des df the, sortie' ) ... .'
ontr of d c.oder complexi ty. _ . -
.. . . -In 1957 Wozencraf( (1957) proposed a decoding proced~re colled
, " , (J "
. sequential decoding wherel;>y the tree-lilte str.ucture of the co'nv~utional code ) .
/ ,
is used in a step >by- step ,search of ~he most Jikely ~ransmitt~d ~$~age. As long
. ~ . as the fote does "not eX'Ced a qlJontity caHed R , which ;9 less thon the
, . .: comp ft . - . chonn~1 copocity, th average number of comgutotions necessary tp ~code one ,
infor~tion digit is small and independent of the constrciint length of the code . . " .
There are two principal sequential decoding algprithms:~e Fano . ~
algorithm (Fno 1903) and th~ Zigongirov-J~linek (Z-d) stQcj< olgorithm introdu,ced .,. "
,independently by Zigongirov (1966) and Jelinek tl969a~ Regordls~ of'the olgorithm', d
a maior problem with sequ~n'tial 'decoding is the variability in 'the n~mbe( of computotio~s . . - .. ~ ,
required per decoded'digit. The cumulative distribution P (.~ N ) of the number of . . o .,
, co.mputations performed per decoded digit is upper and lower bounded for the
/' .r 1,.
l'
r
.'
. .
J
..
J
J'
. , ',4
, L , ,
r" 0' J ' 1 inearly with the cOl]sfraint le~gth. ~onsquently, convolutional ~ncoding and
~:
seq\,Jential decoding are' among th;-mOst attractive means of a~hieving digita,l l' 1 ~ f -y r) l'
communicati~n over memoryless chann~ls at the low er~r pro bah 1 il ity predicted' ,0
, , .' by the theory.
,In the tree represnta~ of a c~"'rivalut,ional cod~ a ~t,~ is . .spedfied by,the input message th,Qt entered the encoder. But' in ,a convdlutional
, ~lof. 1
1
encoder of finite constfaint fength' K, the encoder outputs depend onIY"upon the
..
span of K', K' ,
e.
\
1
. ' $
.. .. 5
Given th'e received sequence hom the channel, sequentiol and
Viterbi decoding apper te be at the extremes for determining the most likely
transmitte,d message. Thefprime motivation for this researoo h~: been the desire '~1 fi, ~ ~ , ,.,
to alleviate the variability of the computational effort of sequentiol decoding .~
., , , " fY,closing the 9C;1P between the 'single-path seqvential decodlng and the ,,11-
paths Vit~rbi decodil19.
or Keeping the sealich properties oT sequentiol decoding while using
" . some of th~ concepts of Viterbi decoding, we develop a c10ss of generolized
stock decoding a 19orithms which unify these two seemingly different decoding' f '
techniques. The variabifity of the computati~nal effort of sequentiel decodir'lg is
shown to be reduced without degrading the error performance.
The fundamentol idea of the generolized stock olgorithm is to ovoid
b ' back-searching for the most likely tronsmitted sequenc.e th!:ough the si multaneous
extension of a subset of the most likely paths., Moreover some redundant and useless li, '
'decoding effort is elimi~'y usin'g the trall.is structure of the code. In the process, ~ . -,...
_" 1. 1 (~ .vents which could lead to erroTs-for sequential decoding are shown to be corrected
. .
/ , ,
by this technique. However, unless 011 distinct paths are constantly extended, a
:"1earch mode of operation si mi lor to that of seq~ntjal decoding appears to be , ~
~
asymptotica.lly inescapoble .
f The generalized stock algorithm is further refined by exploiting the -------~ ___ ,
information about post c{ebding behavioul"l and current stote of the, chann'el noise
. in deciding o!, the actuol number of paths that must be extende~. Hence the d~coding
J ( s
~ .. ----------------~, ~~-----I
',.
r
, \ 6
effort tends to ,be adapted to the particular requireme~ts imposed by the chQnnel
noise, As a co~:equepce both the computational' behaviour and error pe.rJormOnce l' r:' j ..
can be furt~,~;:mprov~d. The limit to this adoptive ~roced~~e is shown jo b~~given by an optirp6)m unrealizable ideel stock algorithm..wt.ich lower bounds fhe compu-
Q
----- . , ~
1 / .,~
tational 'Wformance d~ any stac~ decoding algorithm (including Viterbi and ,y'
sequential decoding).
"'" 1. l Outline of the Thesi!'
Chapter Il is a review of the structure of convolutionol codes. The
essentiel concepts,error performance and computotional behoviour of the sequential
(Z-J) and Viterbi decoding algorithms are presented in Chopter III. The two .
decoding algorithms br~ compared, and inethods to alleviote the computationol
variability of sequential decoding ore introduced.
- ln Chapter IV we first show that the Z-J and Viferpi decoding
algorithms are in fac~ only particular cases of a more general dass of stock decoders.
We th~n proceed ta bridge the gap between these two algorithms by generalizing~the,
Z-J decoding algorithm. A new M-path stock deGoding lgorithm is described,
and its computational behaviour and errOr' performance ore anolyzed. In this
algorifhm the useless computations ohequentiol decoding are eliminated by
exploiting the reconvergence of the explored poth1, andJhe variability of th~
computational effort is reduced by the simultaneous extension of the set of M,
M}I-l- most Iikely pat~s. Results of a simulation of the M'-path algorithm with fI' ~ .
1
7
1
codes of constraint length K ~ 7, over a discrete memoryless gaussian
'channel are presented md compored to the Z-J algorithm.
The e'xploittion of some of the information imbedded in the stock,
ta help reduce further the voriobility of the computationol effort without unduly
incresing the average amountJof computation is exomined in Chapter V
. Simulation results for variants of the M-poth algorithm are presented in Section
( 5.1. An ideal adaptive stock decoding algorithm is proposed in Section 5.2 and . shbwn to be the best algorith~' in the class of generalizea stock algorithms. A
practical adoptive procedure is then described, modelled and its modes of operation
o onalyzed by Markov chain techn iques. The chapter ends with a discussion of
,simulation results for sorne implementotions of the Qdaptive algorithm with short.
constrant length cpdes (K:s: 9 ).
Chapter VI conta ins" fina 1 co~clusions and 'Suggestions for future
research.
ri' ! Appendix 1 contains a bri~f description of the generalized stock
decoding agorith'm as we have programmed it: Massey's Markov Chain model of ~I '
the correct path metric differences is r~iewed in Appendix Il. This model is used
ta determine the average separation between breakout nodes of the correcf poth.
ln Appendix III a branching process mode 1 of the incorrect paths is used for a
newapproach in upper bounding th, average decoding effort of sequentiel qecocHng ....,.
Appendix IV is an altemative method of determining the probability of entering
a search mode of opefation presented in Section 5.4. '
..
;. ,
1. 2 Claim of Contributions of this Research
Some spec ific contributions of this research are summarized below.
A new clos! of generaljzed stack decoding algorithrns is proposed.
ln this 'class, depending on whether the poth extension decision rul~ fixed o.r
variable we distinguish two important subclasses of algotithms.
(o) The M-poth algorithm which unifies the l-J and Viterbi decodi'ng
algorithms.
(b) The adoptive algorithm where~ variable number of paths are' extended
" depending on how much the maximum stock metric has dipped. In
{a}
this subclass the ideol algorithm is shown to be the best possib..le stock , decoding algorithm.
ln thes~ algorithms we developped:
A practicat procdur!! to recognize ~nd exp,loit the convergences of , \
the poths.
(b) li A simple rMthod of extracting the exa~l top node of the stac~
We showed and verifieEl through simylation with short constroint length
codes (K ~ 9) thot:
(o)
\
Compared to squential decoding, the vQriability of the computotionQI ".., . 1 ~
~ffort is redued .. .IIowever 1 the tait of the computational distribution
is sti /1 Pareto
\ " , ..
,
--------.. ----------~--~----------
/
9
"(b) The errOr probability i, upper bounded by that of the ordinary
sequentfol decoder and lower bounded b)C. that of the Viterbf :
decoder. t
il
, J
., .' 2.1
CHAPTER Il
i' .
STRUCTURE OF CONVOLUTIONAL CODES
Convolutional Encoder
A binary convolutional code of rote l/V bits/symbol moy be
10
generoted by a lineir ~inite-state machine consisting of a K-stage $hift register,
V modulo-2 adders connected to some of the shift rE'gistet stoges, and a commu-
tator thot scons the output of the modulo-2 adders. The mchine is ca"ed a
convolutionol enc~deT and is sketched in Fig. (2. la).'
'Let us assume that the information to be encoded is a sequence
(2. 1)
of binary letters, O.or 1. Assuming the shift register to be initially filled with
an ali-zerQ sequence (or any other known sequence), the first binary digit u1 . . ,
is fed into the first stage of the shift register. The V rnodulo-2 odders are then
sampled in sequence by the commutator, produc ing V output caded symbo Is
(1) (2)' (V) Xl ' Xl , .. Xl (2.2) .
, " where the output of ~n adder is "1" j~.9pd'-~nly if the totqJ. number of l's in
.. ,,~- ..
the"shift register stages conne:ted to the odder is odd. Otherwise the adder
output is "0". After the Vth output 'symbol X~) 150btained, the second
ir,:lput 42 is}h i fted ir1to the first stage of the sh ift register cQUsing the contents
of 011 other stages to move one step to the right. The right-most di~it leaving
,'~
.. ~
,.
1
, .
, 11
1
the encoder is lost. The V modulo-2 adders are again sampled, yielding V new<
output symlls x~l), ~~2) , ... x~V). ~his procedure continues unt~ the lost . --.,
input di~it u L enters the shift register. In order to retum the shift registrno
il, in;t~al zero 'Ial~, wilh'eacf! ,hm of uL a zero 'Y~~I i. fed-in unlil '~l " " . leav~s the shjft' registr. Th is terminating sequence.of (K; 1) zeros is calJed '1 ~,' ...
"
the tail of the message.
The L -bit message sequence U of Eq. (2.1) produces the output
sequence or codeword
, . r (1) (2) xc' { )Q _ \ 1 ' Xl '
of length (K +L-l ) V binary digits. The rate of the code is
and for l K
l ~ = (L +K-l ) V bits/symbol
('"
1 l' R::.:f V bits symbol.
let the row vector
G j '= (g1j' g2j' ... gVi)' i = 1,2" . K ..
specify the connection between the stage of the shift register and the V
, (2.4)
(2.5)
(2.6)
modulo-2 adders. The component g.. is a
" 1 if the i ~h modu 10-2 aerder is
connected to stage j, otherwise it is a O. For example, the rate 1/3
,1
.'
u
\, /
, '04
K. stages
V modulo-2 add~rs -
/ ~~ ___ ~_. ___________ . x
Figure (2. la) Convolutional encoder.
."
u
~----___________ X
~
G1
= 111
G 2 = 1"10 ~
G3 = 10l
"Figure (2. lb) Convolutional encoder with K = 3, V = 3,.d = 1.
\.
! .
'. 1
J2
\ /
1
J
,,J
. , , ( 13
.( , , , convolutionol encoder encoderrshown in Fig. (2. lb) hos the following connection
"
vectors '-
(2.7)
G 3 = ( l,? 'n l ) "
From the operation of the encoder, the V output digits (l) (2) (V)
x. ,x" 1 X. 1 1 1
produced whe'n the input binary digjt u. is first shifted in the shift register are 1
(2.8)
where the sign Gl represents the modulo-2 addition. Relation (2.8) is the ,
convolution of,the vectors GiS and the K-bit long input sequence u., u. l' .. -- 1 1-
ui-
K+
1, The term "convolutional" is taken From the frm of the relation (2.8)
and c learly the vectors GiS specify the cpde. We observe that the V outp~t
symbols corresponding to a particular 'input digit depend 'Ipon that digit and the
( K-l ) preceding digits that entered th,e encoder., The constraiRt length of the
code is defined as the number of shifts ov~r which Q' single- information symbol can - . \
influence the encoder output. For the simple binary c~nvolutional code, the
constr~int lef1gth 4S equalnto, K, the length of the shift register. ~
, Considering the input binary sequence OS' on l-dimensional row ..
vector U, the (l+K-l) V - component output X con be written, usin~ modulo~ -
arithm~fjc, as
,
\
\ \
14.
X==U[Gl (2.9)'
where
( L +K-l ) V columns ... ,
G 1 G2 G3 GK , .. G1 cf. G3 ~K -2' ..
" G1 G2 G3 G K ,
[G)= l rows
.. l
l r , G ;. G3 GK (2.10) 1 -2 The matrix [G} is cal/ed the. generator matrix of the-code, and the KV"
component vector r -
:G* = ( G1, G2, G 3,. (2.'11)
is cal/ed the generotor of th code. The first row of the matrix, [G J is the , ,
generator G* followed by (l_-l ) V zeros, 'and each succeeding row is the . .. . , , previ~us row shifted V places to the right, with ail elements to the left of G
1
equol to zero. The number of rows is the length L of the input sequence, and ,
the number of columns is (K+L-l ) V. Every codeword X con be expres$ed os \. -' . .
a linear combination of the rws of [] and the code may thus be seen os the' -
..set of aU 2l
I-inear combinotions of the rows of [G,. for the exomple of .
Fig. (2. lb), the generator of the code is , . -G* :;: (111 110 101) 'f
1 1
, i
'.r' '" .
1
15
and ~r the 4 bit long inp'ut sequence
U t:: (1011 )
x = ( 11 L 11 0, 01 ()., 00 1, 011, 1 01 )
" , One can generalize this binary convo'utional encocrer by
aLiowing the K-stage shif~ register to rec\ive groups of d q-a'ry information . . \. -
symbols of an input sequence whose components ore element; of a finite Galois . -fi~ld ' GF ( q )., Then the input to the encoder is the sequence
(2. 12)
"
whre
J _ (1) (2) (d) u, - (u. 1 U. 1'" U. ), -1 1 1 1
u~j) E GF (q ) 1 i = 1, 21" ,' d, i = ], 2, " l 1 C
4
The V moCiulo-2 odders,' V> d and their connections are reploced by V
inner produ.ct computers, eoch of which compute the inner p,roduct in GF (q )
of the shift register con~ents anq so~fie~ vec~r. In o.ther'words, the
lements of the generator matrix [G 1 bel\"9 to GF ( q)/ and the operatIons
of Eq. (2.9) are performed by arithmetic operations in' GF ( q). The length
K ~$ ch~sen to be 0 multiple of the integer d,
,,'
, ,
.. . . , . . , . ..
-,
. \
.'
1 ,.
16
K = kd (2.14)
. , and-~e constroint length of the code' bec~mes the numb~! J = K/d of groups of
d shifts over which ~ 5ingte ih~rmation S~rF' dqn inf!~ence the encoder output. "i'>
:''---~~p~~.~.--------------~----~ .... ~( ............... ........ (
. , ~ .. 17 I-er .. ,
G * :::' ( 1 01 00 1 111 011) -2 " ...
;
.. ' ~ is sho~n in Fig.
encod~d as 010
(2.2). The input seque~ce sta~ing W(ith 11001001
010 O 1, ~1 .. , .,' . . . -
"'/ wiltbe , ,
The Structure of Convol~io~al Cod!!s. ,
From the above description of. the operation of'a convolu'tionaJ ,"
encoder, we observe that
(1) ;;, ~ l'; l ,
The code symbols depend on post values of the lnputs. " . , '.
~ (2) The post dependency does not extend to the infin ite post but s
limited to the length of the shift register 1
. ~ (3) The code is line~r sin,ce the out,put symbols are linear combinations
of the post inputs, , ... ... ~
We now sho~ that the first tv.y,bservations leJd ta two d~ffere~t
tations of the convoJutional codes.
2.2,1 ee Structure ..
" o . .) ," d
:'The .fact that ,at any time, an input may ta'ke q values hen,ce
leading te, qd ~utput sequences, ~Ubge~S ~e~~esenting the out~ut Qf a ~onvolutio~al 1 t ., '10 ,',
'1
od b th dn 'bl 'enc er y a tree WI q,' poSSI e sequences, corresporlding ta inplJts. The. .
n ~
r
~ ., .' ,
~ . CI .q
1
\
t> ~
;\ e ' " ."" " ..
.> a.. ." + """-- ~ 18 ~ '\: .. "
/ 'c e" / '-. ~ , t '" .. "r 0 ' ,,, 0 ~
'C ., ~ . 1 ,1 "tJ> . '() 11. d'_ .. l4> , ~
, 0 " " .. (l'Y> (1) (1) u3 u2 u , ~
,~ o .~ ,
\
x3 ~ . , J) c;o;
x2
M
X .. , y--- " ~ -,. . 0 , "' ~,
. ' ~1
~ , , . .. (2) (2~ (2) ", ~ , "
'~3 ~2 u 1 . ~ , .
~
'.
\/ ". .. " . . " .... ~
, \~ ..
, ,
- ,
19
\ ." -root node of the tree hos no predecessor whi le every other node hos one predecessor. " \
There ore qd branches stf:!~ming out of eachl:node, with the exception of the nodes
corresponding to the taU oflhe message which hav only a single branch extension.
"' Ail bronches corry V coded symbols. Fig. (2.3) depicts the tree representatio"\
for the outputs of the encoder of I\ig. (2. lb).
. .... A poth . in the tr~ is troced from Jeft to ri~bt occordin~/o the .
input sequence lhot speci fies i t. For the b inary tree of F i~. '(2,.3), 0 0 input , means toking the upper bronch leoving a node, and a 1 meo~~ tcrking the low~.r
, "
branch. Hence a message sequence ~1' ~2' ~3' ". traces 0 poth throJ.!g~ the _1 ~_ , ,
i'>
tree, and the correspoAding coded ~Is of fh( bronches of the poth ore the
channel inputs that ore\transmitteQ, ' ln' Fig. (2.f>, the input message storting . t. 1 ('
'with the sequence 01100 .determines the path in~iate~ by the thic!f'U'}e having
tht'! encoded sequence 000 III 001 0) l 101. The tree of Fig. (2.3) is said to hav , r
fi.ve levels, 'one for eoch bronc'j99 olong a possibl-e patb. The number of fevels in
a tree con be extended ind~f,Lt~I;, and noturally there iSf{] one-to~one corres-'dl 1 \ ~pondence between the set of al! q informti?n sequences of ength dl an
l'
'Set of poths through the L levels of the tree. A code thot con be represented by .., , , a tree IS calred a't.ree code,
. ! .. ln general, tree ~odes ore codes where coded symbals
1
depenCl on post input sequences and where poths moy be re~resented as branch
cho ices on a tree.
..
f
\ . . ,
20
...
..
,j
000 000
111 . 000 - S -0' 111 1
110 (Jo'l " 000 S
-0 110 lUl
111 010
~1 001 1 011 100
S .yOOO 101 1 -0 " . 110 {;I_ I 111
,~. ~2 010 110 001 - 111 01
~1 101 - . 011 .-010 0', ' 001 ~3 011 - , 100 1 -100 .
Root node 000 . 000 1 1 ! ,l .. - 15 111 , 101 ~ 110 111 001 H.,9 -.
~2 101 1101 010
,J - 010 " . --. ~1 001 011 100 ~1 000 101 " . 111, -Olt . ,
~2' " 010 11 () 001 001 , 23 OlT
101 4 . .
100 1 010 S 011 , -3 100
, 100 ,
\ "
Figure (2'.3) Trec representation of encoder of Fig. (2.1b)
. . \
..
21,
2.2.2 :Stote Piogram,
'In the causal transformatian of the input seque'nce U into the
coded sequence X, at any given time the V output digits are ,completely
determined by the most recent k groups (inc/uding the present one) of d
information symbols that entered the encoder, and are residing in the sh ift r;gister. , .
For every new shift, of input symbols, the output will d'epend upen the previous
( k-1 ) shifts of input data, and c1early, if the first nd components of 2 or
more input sequences are equal, the correspending output sequences wi Il be identicol
in their first nV components.
, A cOJlvolutional enc:;.od~r w.as defined as a fi.nite stote machine,
and since a fin ite state macit,ine changes From one stote to another occording to .the
input it receives, the state of a convolutional encoder is then simply the (1
22
The toto 1 nomber of state's is then
( k-l ) d 'K-d q , =, '
d and From one state th~ machine moy move to q other, states. Naturally 1 sorne.
input symbols moy cause the mach ine to stay in the same state .
... This descripti0'f 5IJggests representing the encoder by astate
diagram with q~-d states or nodes and qd branche's leaving and entering each
state. These branhes represent the transitions of the encoder from one state fo
another, and therefore they cerry the V coded symbols delivered by the encoder
in the transi tion. For example, the state diagram representotion of the encoder
of'Fig. (2.10) is shown in Fig. (2.4). \t hos 4 states and~ach branch is labelled
by the corresponding input (above) and output V-tuple (below).
The stote diagrom is a very compact representotion of the encoder.
Storting from the initial stote S := ( 0, 0, ... 0), the coded output corresponding -0
to the, input sequence U =: (u1, u2
' u3
, ... ) is reodily obtoined by writing the
branch symbols corresponding ta the transitions due to u1, u2, u
3, ... For exomple,
from Fig. (2.4), the output ta the message 110100 i~ 111, 001, 011, 010, 110, 101.
A potentially infinlte tree hos been reduced ta a mere 4-stote diagraln~
Regarding the sfote diogrol11 a a signol flow graph, particular codes
can be analyzed via the transfer function of eir dia9fom (Viterbi 1971). However, , th~ greot diffiulty in finding the transfer fu ction for long constraint length codes
limits t~e'use of this technique to very short constmint length codes.
2
u
(1)
1
.. ,.
24
2.2.3 Trellis Structure
Using the definition of astate, to each node of the tree tnere
corresponds an encoder state. For~ input sequence of dl symbols, there are
qdl terminal nodes in the L-Iever tree, but only qK-d different states. There-
fore 1 for L ~ (KI d ) 1 the number of nodes in the tree exceeds the numbe r of p
states of the encoder. This means tha,t sevial nodes must correspond to the some ..
encoder state~ and hence the coded tree paths are not distinct over their entire j
length. From our discussion on the state diagram we know that after the input
symbols have been identical for (K-d) consecutive symbols, the encoded
symbols are the same: the paths remerge. However, th is fact is obscured in the
tree'representation of the code since the tree keeps an branching off with q~
branches From each node, as if the states were ail different. This behaviour is
equivalent to considering an infinite number of states or a dependenc~ over the
infinite post. Realizing that the number of sta,tes is finite, we see that the tree
contqins much redundant information which con be eliminated by merging together 1
at any some level, ail nod~s corr'esponding ta the some encoder state. The
redrawing of the tree with remerging paths is called a trellis.
Taking the example of Fig. (2.3), let the four states by ~ :=: (0,0)
~1 = (0,1 ); ~2 =: ( 1,0); ~3 = ( 1,1 ) Xfter the third level, the data sequence
100 abc ... , and 000 abc. .. generate the same code symbols, and hence both
-nodes in the figure labelled ~ can be ioinecl together: The Same reason,ing applies
for the nodes labelled ~1' ~2 and ~3' and the tree collapses in the trellis of
25
Fig. (2.5). More insight about the remergng of the tree paths in general is l"
,
1 given by the following properties (Viterbi 1967).
r Property 1 :
If tw~ sequences of i{lfbrmotion symbols are identical except for .'
n consecutive groups of d information symbols, the corresponding codewords
w'nl be distinct over (n+k-1 ) consecutive branches.
Proof:
The paths in the tree code, corresponding to the 2 codewords , ..
will be identical as long as the input sequences are the sorne (prior to the nd
different symbols), and will diverge, i.e. will be different, for the n
consecutive branches where the input symbols are different. Thereafter, although
the input symbols are the same for the two data sequences, the code symbols will
be different as long as the encoder states are different. Hence, (k-1) additional
shifts ore necessary before obtaining identical outputs, causing the paths to diverge
over a total of (n+k-1) branches.
Property 2:
If the information symbols of two paths agree at sorne point ,in r
(k-l ) branches, the $ubtrees extending From these two paths thereafter must be \ .
identical.
.'
.. )
26
The trelli-s representotion of a convolutionol code is more
instructive thon the tree representation for it explicitly uses the foct that the ... encoder is a finite state machine. Each stote of the trellis hos q~ possible
d J. . successon, q possible predecessors, and From level to levl the struct.ure is
repetitive. "".
This redution of the tree str~ture of the code into a trellis
is c;redited to Fomey, (1967) and the trellis structure is at the heart of an
optimum decoding tec~nique for convolutionol codes due to Viterbi (19'67).
This decoding technique will be present~d in section 3.4.
" 1
"
,.., "
"
..
~
,
27
CHAPTER HI
DECODING OF CONVOLUTIONAL CODES
. 3. 1 Introduction
" ln a terminated convolutional code, the message input sequence !
U is enc6ded as a sequence X and is represented by a porticular poth through , - ~ . the tree or trellis of the code. This codeword 'X (0150 called the transmtted
s~quence) is sent through a noisy memoryless channe,l ta a decoder and a use'r. 0
A discrete memoryless cflannel (OMC) is an idealized model of
a noisy channel with digital input and quantized or digital outputs. The input
ta a DMC is a sequence of letfers From a Q-symbol alphabet, and the output is
a sequenc~ of letters from a J-symbol alphabet. During each channel use, a
letter of the input sequence is fransmitted, and the corresponding output letter > 1
is received. A r~,ceived letter j, \ l, 2, ... J is assumed to be statistically
dependent only on the corresponding input letter i, i = l, 2, ... Q, and is
determined by a fixed conditional probobility a~ignment P ( i/i ). S!Jccessive 1
inpuf-output transitions are random and statistically independe~t.
The decoder observes the output sequence of the DMC corres-
pondi'ng to th'e transmitted sequence, and determines which of the possible qL
q-ary data sequences of length l entered the encoder. Fig. (3. 1) shows a G
comrrlunicatiol'l system employing a convolutional code and (t . DMC.
l The encoder simply maps the integers 1 to M = q onta the
codewords X(l) ta X(M}. As message m enters the encoder, ~(m) is
, ,e, ('
-Data
y
.-./
.. .. Trnsmitted
Convolutiono 1
.
" ;0
Encoder
binary input
(Q = 2)
.sequence X - --
,
~
.
.. D.M.C.
" ~
Received sequence
Y - .. , . .... ,
. .,
' .. ..
,....
Decoder .'
8 outputs
DMC with
Q = 2; J = 8
Figure (3. 1) Communication system and Discrete Memoryless channel. ~ ,
tf'
1':1
,
--~
...
r
'"
'"
User
" U
;;
~
\ tr~l')smitted, and on the basis of the corres~nding receivd sequence '1., the
, decoder pro(juces an integer m', Decoding errors occur if m t ~', "
"
30
Likewise let
v. ::;: (y.'" Y'2' ." y. ) LI 'l, 1 IV (3.4) c
"
represent ,l'he V received symbols when x~U) is transmitted. Then~ E~. (3.1) \} -Y' \ .. .
the like/ihood function for the jth branch of that poth s.
, , "
(3.5)
;'" 't' since the channel is ossumed memoryl~ss, .
(.
. Gen;r~lIy it is more convenient to use the logarithm of the
likelihood function since the logorithm i5 a monotonely, increasing function, and i: ~ \.
, . , tHerefore does, not alter the final result. Defining the log-liklihood Junction
'-
;y ~U) for th'e jth branch on the path speciHed by the input se~~nce b -as cl,
, . (U) A
y.- = 1
log
, 0 V ,
1
1
1 ~
1
'1' f
,.
1
, ,
, 1>
....
, J
, . ~ 0 a
31
. . maximum likelihood decoding pf convolutional .codes may be ~imply regarded as
(J 1 ~ n
~~ finding of the poth with tJi'e I~rg~st accumulated me!ric, giw;n the received 'l-' , ~. , p ,
sequence. However; dePending on whether the tr~e or trellis'structure of the 1-
code is used, the d.terrrlinatior'l of t~ MOst likel};' poth will lead to either an
. imp~ical or practic~1 de~oding technique. ., " '-,'< , J .. '--
, Maxim(Jm tikelihood Deeoding of C?nvolutionol aCod~ ..
By d~finition, n;lOximum likelihood decoding implies comporing
/ , ' "the reeeived sequen-ce with ail possible transmittd seqences before making 0
-) , J _
decision. ije,rc;~" in g~ne'raJ for memoryless chonnels, the optImal decoding 'of (1' ., - fi
an l-bit long bina'ry seque,nce requl'tes c~parin,g the ',2L ~'Cc'umulated metrics
of the 2L
di,fferent c'odew?rdsJh~t .;cQuld have ~een transmitted~ and picking the . . 1
best one .. 6ecouse of this exponential inc'rease of the decoding effo~t with the . ' . Jength t of the sequen~~1 maximum likeli~ood ~ecoding is u;~o!t difficult to
, , \
, . 'its use as a standard with wh1ch other practica'i suboptimum deodi~~ techniq~es mey
, 'S- -'>- r _"'"
be coh1pored. Mor.eover"..it is used to q~rer!"ine thebperformance of codes, since a
meas~~ of ~er!~rmance fOl\apy c>de is the probability ~f error witp the optimum ..
. . ~ :~ ~/ , 1
,For the decodi~9 of 'ton vo lvtiono 1 codes, either the tr~ or trellis c. '" ';
decoder.'
, structur~ of -the c0cte may be used. Ir the tfee representati~n of. th~ ~ode, \the fact
Q \
, \ that the paths remerge is entirely ignored, and hence the decodin-g of an L-bV
, C" \ '\ , .., 1 . -'.
b
Q
1
. .
L
,>
...
32
:-.. seq,:,ence reqyires the exhaustive c'tmpariSon of 2
L ~~cumt.itd ~etrics. Conse"" l'
. " qumtly over the tree structure of th~ ~ode: optimal decoding is not proctical, , ,
and g'i~en th~TVed sequence, the estimation of th~ most Ukely' data. seque'nce would coll for a suboptimum but more practicol decoding technique.
~onsl..der now the trellis st?ucture of the ~ode where the rfd~ncioncy "
of-the trie was elimmoted through merging. -In princ iple this redundoncy o;ouid be 1 ,
\
" exploited by a clever decoder which would consider only those paths thot could 1 '"
'ever maximize the Ifkelihood function ov~r the whole set of paths. In th~ ~~o(jing ~"j
#'
. " process, iF at some point it is realized thaf a poth connot possib~y yield-the lorgest
, ' , '/
.. C'> 1. ....
metric, thn that.pa!h would be ignored' by the decoder. The decoded pot~ would , "".1 ',' 0
then b -ho;e~: From amon9 the set of rerhG:lining Or "surV'fving ll poths that reoched ~" ~ l'
"the lost level l. Such o'decoder wou Id st~e optimum in the sense thot the 1 ~
decoded poth would be the some os the decoded ROth of 0 IIbrute force" maximum
.'kelihood de~er~""1 the eorly rejeclion ~f'unlikely polh. WOu Id r.duce Ih~ decoding effort. 'Th, objective is naturo lIy to fj~d a procedure that breaks the ~
, p
exponerttiaJ message sequence-Iength dependency of the decoding effor~ ond yet . '
~ields maximum likelihoo9 decisions. We now show that the trellrs structure of
\ the code allo~sosuch ~ ~ec
e
,/
(
..
/
/ l ',J;
l,
l 33
1
this set, the path having the '?rgest accumulated metric is c,al'ed the survivor at
s"( t ) d Il h \ h . '"\... h . S (-t) 1 . state . , on a ,ot er pat s constlture t e non-survlvor set at . . t IS -1 -1
readi Iy ~that an optimum decoder con d-t$card thJ non-survivor set~f poths ',' L . withbufaltering the optimum decsion. /.' l,' . '
~ --' , Since the sub rees issud From a set of converging nnfhs are / r":,,
'
, '{
Poth metrics.
1
"
Remerging level
, /
, , ,
34
Identicb 1 subtrees.
. ........ .. -----.... ...--e
.:..__e
;-------+-------+-----~~-----+------~------- -Tree levels
~I . ...... l t -1 t+1
1
Figure (3.2) Poth metrics of remerging poths. .
, -.
35
A very proctical optimu.m decoding technique is the Viterbi
decoding aigorithm1tlVented by Viterbi (1967). This.deco~n9 procedure uses (
~tre/lis structure of the code, and is very efficient in the way the periodic , ~- ),
remerging of the poths is sysklmaticlly exploited to determine the survivors.
The optimolity of the Viterbi decoder wos demorstrated by Forney (1967) and
Omura (1969). Forney showed that for cny convoJutionaLcode, the o~tput of
the decoder is a sequence of estimated information digits whic~ is maximum lik~-
lihood conditioned upon the received sequence, and Omura showed that the '\
Viterbi olgorithm wos in foct a forword dynamic progromming technique.
(,
,~
3.3 Suboptimum Decodiog
Since the number of states increoses exponentiafly with the , .
constraint length, !JIOximum likelihood decoding becomes v~ry ropidly impractital.
t
However, -given ,the received sequence, the seorch for the moSt likely poth con be ,
attempted sequentiallYi one branch at 0 time, making tentative decisions (or \ \ .,
hYpQtheses) pn successive branches of explored poths, in such a woy that the poth
being followed is the most li ke t'y omong the subset of paths currently examined.
Whene.ver a subsequent choice indicates an earlier'incorrect decision, tflis, j
decision is properly modified, and hence at each decoding step, the cur,rent mast
likely poth is chosen among dfferent paths Iyin~ ~t different levels in the tree
(or trellis). Clearly the exploNtdpath is o'nly locolly most IikelY- .
t, p
1
> \
... '"
. '
J
36
Th is step by step procedu re is faci 1 itated by the tree (or he Il is)
structure of the ~ode, and by following only the current most likely path, unlikely
s~quences ore eliminated until 0 single sequence of length l remains; this
sequence is occepted os the decoded poth.
The elimination of on unlikely poth ba~sed on the observation of " ,
its first m bronches, m < l, is of course equivalent ,to' the elimination of every
poth beginning with,these m bronches, i.e., 0 whole subtree of length (L-m).
The earlier the rejection, the larger (exponentiol/y) the ,number of paths discarded,
re
~ .......... ------~--------~--------1
1
1
1
1
37
decoded sequence is obtained by probabilistic considerations 'rather than bya
Fixed sequence of algebraic operations. ( These two techniques are presented in the next section, whereos..
the new decoding technique that extends the M-most likely poths will be
pre.sented in the next chapter.
There exists also a number of non-probabilistic (or algebraic)
suboptimum decoding techniques for convolutional codes, for example Mossey's
threshold decoding (Massey 1963), suitable for codes having certain algebraic
(orthogonal) properties. In general these decoding techniques lead to inferior
. ,
performance compared to Viterbi or sequential decoding, and wi Il not be discussed-, \
in this thesis.
3.4 Probabilistic Decoding Algorithms
This section is a brief introduction to the two main probobilistic 1
decoding techn iques introduced in the preceding sections: ,the tree search of
sequ.entiol decoding and the trellis search of Viterbi decoding. ,
For both decoding techniques, the decoder is assumed to have a . replica of the encoder and hence is able to duplicate ail possible transmit~ed
sequences that constitute the code. Successive sequences of V channel output
symbols. caUed "received branches" are received From the channel, a,nd from
. th~se the corresponding bronch metries con be computed." Assu~ing 0 DMC" the,
metric along the poths i$ cumulative and th~refore, for both techniques, the
f
objective oF the deoder is to find the poth that yields the largest accumulated \
t
metric, given the received sequence. Viterbi (or optimum) deco~g represents
C?n exhaustive search that takes advontage of the code topolC;;gy 1 whereas
sequentlQI decoding uses an intuitive trial-and-en-or search method to reduce
the average decoding effort.
1 3.4. 1 Tree Search: Sequential Decoding
The central idea of sequentia 1 decoding is the decoding of the
received message one symbOl at a time rathe~han decoding 011 information \
symbols simultaneously as in maximum likelihood decoding. Intuitively, one'
38
could suspect that in many instances, the correct poth has a larger metric thon the
incorrect Paths diverging From it, and hence could be accurately estimoted by
considering only the succeeding Few branches. Starting From the origin of th~
tree, the poth se /ected to be- searched one step Fu rther (i. e. , extended one level '~
deeper in its qd branches) is the pcth that has the largest accumulated metric
amang those already examined. Consequently, by extending only the poth that
appears the most promising, most of the computations necessary for an optimum
decoding can be avoided. This idea is common ta various olgorithms known as
sequential decoding algorithms, the specifie method of,searching and selecting
the poth to be' extended depending on the particulor algorithm . Among the subset
of exp/ored paths, the poth that reaches the last 1evel of the tree (including the
tail) with the highest metric is accepted as the decoded pa!h.
, Jacobs ~nd Berlekamp (1967) have given two conditions which
.
39
a decoding algor7thm must satisfy in order to be a sequential decoding "0 1 gO ri thm'. '
(1) The branches of the explored part of the tree are examined ) ..
sequentia Ily, and the deci,siQn on wh ich poth to extend is based
on Iy On those paths already examined. Each new poth is thus
an extension of a previously examined one.
(2) The decode40.performs at least dne computation for each of every
examined path.
Algorithms which do not have these two properties are not considered to be,
s,equential decoding algorithms.
The interesting feature of su ch a decoding procedure is ,that the
rejection of a branch ;5 in fact the rejection of a whole 'subtree whose root node
is the end node of the rejected branch.
J As the decoding. proceeds, the number of poths in the subset of
explored paths grows, and,occasionally the decoder goes boek in the tree and , 1
extend early ~nd posSib!y incorrect paths. A fundamental idea of sequential is
to bias the metries (on which the seareh deeision5 are ba.sed) in ,5uch a way that .:
the backing-up and, extension of unlikely paths is reduced to a minimum. for a
, 1",
memoryless chonne l, the bronch metric (U)
y.-, of the branch (U) (U)
( xiT ' xi2 ' x~~) ) of a poth ~,.i'v
,
U is generally taken os
(U)
40 . ,
(U) ::: X.--1
y~U)::: L [log p ( y .. 'x.-:- ) Il ,I f( y .. ) JI
- B] (3.8)
i=l
\
where r. = (y. l' y'2' ... y,V) is the sequence f V received symbols , 1 1 1
corresponding to the transmi"tted branch X~U), and wh~re f ( i ) is the nominal -1 .
probobility of output.j for a DMC with an input probabili,ty ossignment w ( i ),
i::: 1,2, ... Q and transitiOj1 probabilities P (i/i), i::: 1,2, ... J. That is
Q f ( i )::: L w ( i ), P ( i/i ) (3.9)
i=l
the bias term B is chosen in such a way that ,_ on the 6verage 1 the branch me trics
ore positive along the correct poth and negotive olong the incorrect paths. As
the-"\'netric ossigned to eoch node on a poth is the sm of the branch metrics olong
the path leading to thot node, the accumuloted metric for node UN is ..
N
= l i=l
(U) y.-
1
, N = 1, 2, 3, (3. 10)
The metric of Eq. (3.8) is then a "tilting" of a modified form of the log-likelihood ,. .
function of Eq. (3.6). This biosirfg 'or "tilting" of the likelihood function is ...
" common to 011 sequentiol decoding algorithms, and the value of the bias which
)
rninimizes the average computat~onol effort per dec4ed dfgit is equal to the rate
'.
"
!. ~
".
41
. of the code, (Fano 1963, Bucher 1970), Clearly then, the consequence of biasing
the likelihood function is that aJ.ong the correc.t poth the accumulated metric tends to
incras, while alon9 any incorrect path it tends to decrease. Therefore, although
the objective of any sequential decoder is to firid the poth that yields th~ largest
accumulated metric, the strategy is to searc!l and follow the path of increasing
metric value.
There ore two main sequential decoding oJgorithms: the Fano algorith~(Fano 1963) and a ~arch algorithm introduced independently by Jelinek
(1969a)andZigangirov (1966). Although seemingly different, Qoth algorithms
search the tree From its root node out, trying to "match" sequentially the beginning
~egment ~i 1 i = l, 2, . '.' L of the received sequence ~l ta the corresponding
initial segments of the various paths of the tree. The Fano al
by, Fano in 1963 is a modification of the original dlgorithm present d by Wozencraft in . 1957. A detai/ed descriptTon of the algorithm may be found in ei ner Wozencraft
and Jacobs (1965) or Gallager (1968), and will not be given here. The Fano
oJgorithm is quite complex, the simplicity of the decoding principles being somewhat - "
lost in the details of the rules thot govern the motion of the decoder. It is however 1 , .
quite populor, and has been used ta derive mast of the theoretical results on
s~quential decoding. ."
, ,
...................... ~----------------------~.~
'
42
(a) The Zigangirov-Jelinek (Z-J) Algorithm
_ Th~ l-J algoritf is a very simple algorithm y.'here the es'intial
concepts of sequential decoding are ra(lily apparent. The decoder consists of a Iist , 1.
~
or stock of the olready searched paths ordered in decreasing order of their metric -- .' valves. The "top" of the stock is th poth of largest occumuloted metric among ,
-the poths,in the stock, and wi Il be seorched fu~ther (extended one lev~ further J
. in its qd branches) since 7t is the one which is the most likely to be the correct
" t
,path. As,the stac:k is reordered after eoch extension, 0 poth w,hose metric is ever-
! increasing wi Il continue te) be seorched further. Should the metric decreose al
drop from the top position, thot path will be properly stored in the stock and the
new top node wi Il be extended.
Denoting eoch explored path by the .npde of its extremity, the
stock con equivalentfy be considered as a list of nOdes ordered according to their ;'
~
metric values. The objective of the decoder is the finding of the top node, the
extension of its successors, and the proper rerdering of the stack. After initially ~
. ~ loading the stock with the origin node whose metric is token as zero, the decoding \
olgorithm consists of the foJ/owing rules:
:'(1) Compute the metrics of 011 successors of the top node and enter
_ them in th'ttr proper place in the stock.
(2) . R.emove From the stock the node whose successors were just . . . " ,inserted .
") ,
< ..
(3) If th new tt ~ode is the final node, stop. Otherwise go to (1).
, 1
Wh~n thE\aJgorithm gets out of the loop, the top ':'Iode is m' 1
end node of the decoded path. Tf,e who le- path' is then e~si1y',recovered From the ", <
, ,
information stored in the stock .
43
A decoding example for a binaJ}ree\;s illustrated in Fjg. (3.3). ,',
,
The paths are associated with their termjn~1 nodes wh'ich are ~umbered in the ,
figure. 'The, numbers Ylritten on top of the branches represent the 'corresponding
" ' \ branch metrics for som~ received sequence ~4 ~f length 4 branches. Ordering.
the nod'es From Jeft to right 1 the contents of the sto~ during decoding is then:
Stee No.
Initiafization 1
2
3
4
5
. " . ' \ ... .
. .
'Stock contMts,: node (metric) . ' 0(0)., t' .
2(-1),1(-7) , 5 ( -5 ), 6 ( -5 ), 1. ( -7 ) ~(-5),11 ,(-6),1 (-7), 12(-12)
, 11 ( -6 ), 1 ( -7 ), 13 ( -9 ), 14 ( -9 ), 12 ( - 12 ) . - o
)
6 1 ( -7 ), 13 ( -9/), 14 ( -9 ), 23 ( -10 ), 24 ( -10 ), 12 ( -12 ) - . ~
" A
7 4 ( -8 ), 13 ( -9 ), 14 ( -9 ), 23 ( - 10), 24 (- 10),: 12 ( - 12), 3 ( - 14 ) , 8
. 9
1 0 ( - 8 ), 13 ( - 9 ), 14 ( - 9 ), 23 ( - 1 0 ), 24 ( - 10 ), 9 ( - 10 ), 12(-12),3(-14)
21 ( -6 ), 13 ( -9 ), 14 ( -9 ), 23 ( -10 ), 24 ( -10 ), 9 ( -10 ), 12 (f'" 12 ), 3 ( -14 ), 22 ( -18 ) l . .
1
The decoded path iSa thu&s specified by t~e doto sequence,. terminat,ing ot nqde 21, thot,
is (0, 1, l, 0). (
a
J J 44
"
1 It ;~, 15 /~ , '- ... ., " . -7
16 Ci
" "1 .y
~
-7 ..
1 ~ 4
"11\ .- , , 19 -2
~
'" -1 0 ,- 1
0i-
J
1 . '2 21
l
-10 22 .,.
, )} -4
./
11 -1 '~ \
\ -4 24 .. -4'
~
25 ''''''7-. . . t 26 1,1-:-
~ t 1 ~ -1 ~ ,,," 27
"'4 "
28 -4
1 ->-4
'fi 1
~ "
45 1
" WHO. the keJp .of this exal'nple ,1) the Z-J 'afgorithm n'loy be . \.,
rgorded as th most natural way ta extend thct explored path having the largest
biased metric, By olways 'extending the most fikely'poth omongthose ,a 1 reody ,
" explored (~hot is, stor~d ~n the. st?) 1 t~e ~Igorithm ~x~mizes' the -probabii~y
that the next step will be token alorlg the correct poth in the encoded 'tr~, ~ " .. ~~~-~
The m~~han ics of the seorh 'ore quite s~mple in principle. After . . 1
" d .. the'top node\is eliminoted, th- q new successors are inserted in the stock ot 1 _
th place ossigned by their totQI metr}os .. The sto~k grows by (q d -1 ) entries ~o~h d~codin~ step,' so thot of~er, i steps, it contoins (1 +j ( q d -1 )) paths
of vorious le~gths terminoting in different nodes in the tree. Defin~ng a compu-
tation as the exe'cution of step l of the algorthm, the toto 1 number qf computotiorRl ,()
. ~ , .to decode a binary tree is equol to (W-1) where W is the size of the stock when
" , dec~din9 stops.
, , . .
J\ groph of~e metric values of the poths called the "received \
) '- p
value tree U may be constructed as in Fig, {3.4) for the example of Fig. (3.3). At , '
the 4th levet' obvilsly' the decOd~ poth ~os ~e highest value. However, t~i~ is . not th~ ca~e for any"'other interrnediate no,de lev~1 and by_ the rul~~ of the olgorlthm,
011 poths whose metric is higher thon the correct path metric must be ev-entuolly . ~ 1
extended. A typical example of th~ plot of the ~orrect poth wet,t.lc is shown in
, 1
Fig. (3.5), A span of bod noise causes the metric to.dro~, and therefore some
incorrect path~ may occupy the top of the stack and be extended. OI~r1y, before o
the~ correct.path reoches the top'of the stock and is e~ended anew (henc-e hos
.-
"
-6
-8
-JO , / / ! 1
/-12 1
1
1
j f 14 ! -
1
/
'r .
:\' Metric valu.es ai the explored paths. Figu~(3.~)
.~/;'; 'L, Cor~:; ~~, i~
T', min
Incorrect poth, ,
.J, __________ -_
" / tSOd ~i~ \ \
, " ,
\ \ ,
~
"
\ \
... , t , , , ,
~'~~ , ,
Figure (3,5) \T.)1)cal correct and jncor;e~{~ths metria.
(
. . / . .
( ,1'\
~
'"
----------------~~~--~-~-
)
.)
1
, . ./
/
) 47
,possed ~e region of ~d noise), ail incorrect paths with a metric hj~eI thon the smal/est value of the correct path metric must have been extended. Th is .obser-
vation i5 fundamental ta understc:;"ding the computational behaviour of sequential
decoding.' ft is the occasional drop in the correot path metric and the conco~itant
extension of inco;rect paths that is responstble for the ron~om nature of the decoding
, effort. ! '
(b) Quantized Z-J Algorithm
The simplicity of the Z-J a/gorithm is paid for by requiring a
large memory for storing ait the information about the exp/ored paths. However, L 1,_ '
the rea/ ptob/em is keeping the stock exoctly ordered. An exact ordering of the
nodes is 50 time consuming thot the algorithm becomes practieally worth/esse To 'p
overcome ~is difficulty; Jelinek (19690) pro'posed a quantized version of the
algorithm in which the nodes ore p/aced at random into substacks (aise ca/led
bins) accorQing to their metric va~ues. In each substack are stored ail l'hose nodes Q
. ~hose ~tric value ~ie5 with;n a ce!1.tfift range. ThOt is, anOde m of metric r m
is inserted at random into substack ci if
c
Q H $ rm
.. ,..
\
48
. . #
The node which is to be extended is thn taken ot rondo';; From this top substack,
usuolly in the lost-in first-out mode. Jhis quantized version of the l-J algorithm
is sometimes called the Jelinek algorithrn and the rules becorne:
(1 )
(2)
(3)
Compute the metric$ of the successors of any node from the r~ ,
highesfn-emPt~ substack and enter them in. their proper
s~ck.. 1 ~emove f1m+he stock the node whose successors were just
Inseoetr:'
If any node in the highest non-emplty substack is the final node,
stop. Otherwisegoto (1).
Under this quantized form, the Z-J algorithm becomes practicat ..
and competitive wah the ~Ider Fano ~Igorithm. In a compprative study of the two . , algorithms,' Geist (1970) has show ... that although quite different, both algorithms
essentiafly folfow the some rules of poth extension. Considering the decoding time
as a measure ~f the decoding effort, Geist observed by simulation that foncoses of
low noise, the)no decoder perfo{"" better thon the S~J,.it.coder, but the advantage goes rapidly to the stock decoderas the l')oise increases. Hence the
stack decq-r would be superior than the Fano decoder for periods of high chann~f . "- . noise and vic-verso for periods of calm channel noise. The choice of the
../
"r 1 partiular decoder will df!IJpend on a trade-off between storage; speed o:nd sofOtwore
soph istication. -/
l , --~----
49
(c) The Computational Problem of Sequential Decoding
Regardless of the algorithm used, sequential decoding involves a
random motion into the tree, where the exact moves of the decoder are determined . -by the re~j"ed seque~ce and the particular algorithm. Since any decoder move
implies a computation, the num~rr of computgtions to decode a given block of
~-' , information symbols is a random variable. Conseqtlently, the analysis of sequential
~o decoding is not only concemed wJth the error probability but al50 with the distribution
of the computationaf effort. This variability of the computation is one of the draw-
bocks of sequential decoding .
A combination of asymptotic resulls by Jacobs and Berlekamp (1967) 1
Savage (1965), Falconer (1967), and Jelinek (1969b) ~hows that the number of
computations required to .decode one information symbol for any sequential
decoding algorithm has a Pareto distribution (under the assumption of an infin ite
constraint length code). Thus ignoring the converge'nces of paths,
P (C ~ N) 1=:1 k1
N -0
(3. 12)
yr large N. k1
i$ a constant. The exponent a is cal/ed the Pareto exponent
and is giveQ by.the porametric equation
tE (a) o
R=---a
(3.13)
whttre R is the information rote of the code and E~ ( a) is the Gal/oger. fU'hctic:>n
(Gallager 1968) deFined os
... "
.'
50
(3. 14)
for a discrete memoryless channel with Q inputs having a distribution L) - -'
fw (i ) , i = 1, 2, ... Q}, J outputs, and transition probabilities ! ( Vi ).
E (a) is a concave monoton ic non-decreasing function of a "and is determined \ .0 .1 by the channel statistics. This fU!1ction has the properties that
E, (0)=0 ()
E (1) A R o ,. comp 1
d (E (a o
lim da
a .... 0 =c
(3. 15)
Where C is the channel capacity 1 and the rate R = E (1) is called the comp 0
,
computationai ,~ut-off ~te of sequential decoding. FOr a discrete memoryless , ,
1 channel R
, ...
..
51
The theoretical values of the exponent 0 and R/R moy be comp
obtained by a graphical construction On the graph of Eo (0) as shown i,:" Fig. (3.6).
Observing that E (a) is monotone non-decreasing, then o
E (0) ~ R comp'
o ~ 1 " 0 , (3. 16)
and from Eq. (3. 13) we obtain
R o ~ com~ for R s: R
R comp (3. 17)
Simi larly for o s:
R o ~ comE for R ~ R
R comp (3. 18)
"
Observe that a larger Pareto-exponent wi" moke the tai! of
the ,distribution of the computation'" ~ C ~ N) dec!ease more rapidly. This (s
a desirable situation which for a given MC can be obtained only by decreasing
the in'tormotion rate of the code.
Many experimentaf 'fhvestigat'ons '(Jordan & 8lu~stein 1'963,
Niessen 1965) confirm the Paretean nature of the distribution of the computation
f
for the coherent and incoherent di1rete memotyle channel (Ha.ccoun 1966, , 1
Helier 1967), and the fading :Iispersive ch'-nel
1
. , ..
E (p) o
R(a) comp
R(a)
E (p) o
R (a)
R(a) comp
"/ /
/ /,
/
,/ /
. 'O.
- - --jf 1 /1 1 / 1 1"'/
---A / 1
/ 1 /
R
;'
/
comp a R ..
------------------ -If /1
/ 1 / J ----------------. ---
1 1 1
1 1 ~.
Z
- ~ ,
'1 /
R cony R
, , . 1
a > 1
a
'-
53
with the penetration into the subset, the number of required computations will also \
~ ,
grow exponentially with the length of the correct poth metric dip. How~ver, on
the discrete mernoryless channel, the interval of high noise thot caused the cOrrect
poth metric to dip, occurs with a probabi lit y that is exponentially decreasing with
- its length. The combined effect of these two exponential behaviours results in a
distribution which decreases at rnost algebraicolly.
. Another unpleasant consequence of the variabi 1 itr of, the comi
tation is that incoming dota may l')ot be always immed'iately processed as it arr.ives,
and buffer storage must be provided. Regardless of the size of the buffer, there is
a non-zero probabi 1 ity that it may fi II-up, leading to on overflow and complete
communication breakdown. This overflow problem is the most serious problem with \.
\ sequentiaJ/decoding on the discrete memoryless channel. The probability of over-
flow has been bounded by Savage (1965) os
(3. 19)
where S i~ the decoder speed factor in computations per informatipn digit inter-
arrivai time, B is the size of the buffer and a is the Paret~ exponent. -The
probobility"of overfJow is rather difficult to combat because of its relative
insensitivity to buffer size and defoder speed.
To overcome this major diffieulty, dota s sent in blocks, usuolly
in thel order of 500 to 1000 branches, and known sequences (the toil of thl
messag.) cleor the shift register of the convoJutional encoder of ter each block.
"
\ \
1 ! i
54 \
~
ln case of an overflow, the entire block is lost, but the buffer is cleared and
decoding can be re~umed in the following block.
The sequential decoder is soid to have made an error if it makes
an incorrect decision On an information bit and never returns to correct it. The
error probability in sequential decoding decreases exponentially with the constraint
length of the code and hence can be made arbitrarily sma". Sequential decoding
being a subopti,mal procedure, a true lower bound on its error probability is
obviously the lower bound on the errOr probabi lit y for'the optimal decoding o~
Viterbi '(1967). Asymptotically the bound is
E (a) o
-K E (a )/R o
e 1 R""" P ( E ) > k2 a (3.20)
where k2
i5 a constant, ,K the constraint length of the code, R the rate and
E (a) the Gallager function of Eq. (3.14). o
Long constraint lengths present no problem in actual sequential
decoders bec ouse the coder complexity varies only tinearly with the constraint
len9th and the prObabilit~verflow is inoensitive ta it. Th~refore, t'I.e real problems with sequential decoding are the variability of the computational effort
and the buffer overflow.
These difficulties have somewhat been alleviated by a hybrid
scherne due to Falconer (1967), where instead of using one sequential decoder,
a mixture of several sequential decoders working in parallel and an algebraic
block decoder are used. The system 'works in such a way that periodically the / /
1
\ 1
55
algebraic decoder helps those sequential decoders involved in long searches by
putting them in the right poth. Compored fo straight sequential decoding" the
computational variability of Falconer's scheme is reduced, but the distribution
of the computation remains osymptoticolly of a Pareto-type. The increosed'
complexify of the hybdd scheme has so far prevented ifs implementation in
proctice.
A new method to reduce the computotional voriobility is to allow
the decoder to ex tend sever~1 poths simultoneously, and by using the trellis of
the code exploit the convergence of the pcths and eliminate unnecessary
compufptions and storage. This method which will be presented i~ the next
chapter, involves only a slight modification of the Z-J algorithm, and in
'addition mak~s sequential decoding att+ctive for short constrainf' length codes.
3.4.2 Trellis Search: Viterbi Decoding
The Viterbi decoding algorithm (~i~erbi 1967) is a slmple decoling
procedure, which delivers a sequence of estimated ilformation digits, maximum-
~~ 1 < likelihood conditioned upon the received sequence. In contrast with sequential
decodini' the Viterbi de~oded sequence corresponds to the poth hoving the lorgest
occumulated metrie of:e.!..! possible distinct paths.
We recal! From section 3.2 thpt of 011 the paths- remerging ot a
given node, only the survivor at thaf no de needs to be retoined by the decoder,
~nd consequently the trellis is the structure to use for decoding. At ony trellis
\
depth (or level) t furtherthanoneconstraint length, thesurvi,vorforeach of the
qK-d distinct states must be determi~ed, which implies that ~ distinct paths of
length t branches must have been examined, Since none of the possibly volid
paths are discarded, there is no need to bias the liklihood function, and hence
the metrie will be the log-li.kelihood function of Eq; (3.6).
56
ln the trellis structure, anode is entirely specified by its stote
and its level. Thot is, s~t) 'specifies- anode at level t, t= 1, 2, ... , having -1
~, . 'state S., i = l, 2, .. , ttK- d . Given the survivor for each node cff some level t,
-1
the survivor at any ,node ~~t+l) is eosily obtained by considering only those- qd
l-branch ~xtensions ~,merging from th~ qd survivors ot level t) that merge ('{'+1) . (t+l) . .
onto node S. . Consequently, the survlvor at node S. IS the extension of -1 - -1 .
that survivor which yields the largest accumulated metric at node s~t+l). For -\
exomple in Fig. (3.7), the sl,Irvivor ot ~~t+)) 'is the extension of the sur~r;~r --,
at node S(t) and not ot the survivors ot either Sl(,{,) Or S (.t) l" -3 - -0
~--
ln case none of the extensions of a survivor yi~td a su~Jvivor ot the
next le~el, then cleorly thot svrvivor connot become port of the decoded path and
hence may be discorded. This situation is illustroted in Fig. (3.-7), where none of
. (t) (t+l) , the successors of the survlvors ot nodes ~1 and ~3 were chosen, and hence
.. these survivors are etiminoted. Summorizing we see thot a surv'iving path ot ~ny
level is alwoys a surviving poth ot:all of its precedin~ levels, but it, 1 not necessorily o part of 0 surviving path at any following level. rThis observation is at the heart of
the Viterbi decoding olgorithm which 'can be regorded os a set of rules for obtoining ,?
the survivor ot eoch stote of every trellis level. -1
1
S -0 ('\ ,.
~1
~5
.. ,
r=15 T"=19
r=19
t +1 t+2'
The heavy lines represent the surviving paths.
Figure (3.7) Surviving and non-surviving paths in the tre"is. ~
-,
57
_ 0
, -
"
o
1
(a) The Viterbi De'~oding lgorithm p" ~ 0 " \
The Viterbi decoding algorithm operates os follows where' we may\ , ,
assume d = 1 without loss of generality. Given the first (K-1) branches of the ~ ~ J 1
received sequence, ~K-1 (= 11' 2::2' ... 1K-l ), starf by e~amining, fro~ the \_
origin out, 011 qK-1 paths of length (K-'l) ~ro.nGhes and compute their total me~ics
r (i) K71 ' K -1 ' i = 1, 2, ... q Since no merging takes place for the first (K: 1 ) con-
secutive branchesi9f the, tree, those q K-1 poths are distinct and identical f?r bath
\ ,
-59
J
factor"of q, resulting in a constant nu-mber of survivors. From our ea~lier
. ~
1
.',
O( . ~\
.) 0 ( ..
, ~ be performed at every state, the cOl1'lplexity of the decoder is proportional to the
number of ~tates, hence groQws exponentiolly .with the constllOint length. To
decode a block of L information symbo'ls the total number of Vitethi op~rations
/ 60
... . ... , 1
K-l ~ 1\....5L L 15 L q ,and for L >;>- K l.i' is far smoller ,thcr the tot~umber ~ 1 (q -1 )
. ',. ..l.. tf of operations that would b~ required to perir;;'~ ma~"),um l''kelihood decoding
on a tree. However, as ci practical decoc{,ing.teohnique, clearly the eXpOnential ~ ~ ... 1) )
, , . growth of the number of ~tates with K will lirn,it Vherbi deco~ing to convolutional
... codes of short constraint lengths (K ~ 7 ). . r
, "_1 . .,.
>
.. t, 1 ) ~. J
The probability of error fur the optimal decodjng of a convolutional
code of rate R over a memoryless channe! has been boutrd~ Viterbi (1967) as 1) \
'\
\ 'where L
\
..,
/.7 P(E)~ l(q q e-NE(R) / ) -E/tR
,: l-q 7 1
1 " j
rrllle number of inforrhation symbofs thot ore encode', ~ "
N = KV is' , 0 \'
the symbol constraif,'lt lengfh of the COde Glnd where \
O
Using computer searcn techniques, good short constraint length
codes of rates 1/2 and 1/3 have been ob(ained by Odenwalder (1970) for .
K s: 9, and excelle~t decoding performance 'w'jth these codes was reported~Heller & Jacobs, 1971). More recent/y Larsen (1973) extended the list of good codes
1\ ~
~ith constraint lengths up to K = 14 and ;ates 1/2, 1/3 and 1/4 .
. Iifications of the Algorithm
. , The great odvontage.of the Viterbi decoder is that the number of
decoding op~rations performed per received branch is constant.. These operations
-ore always of the sorne nature and do not require a sophisticoted, logie. The main
.. 'disadva~s the huge memory necssary to ~tore ail the paths. Moreover,
forcing the encoder into a known final state by using Cl toi! of o,known sequence
of symbo fs !s equivalent to waiting until al/ L information symbols have been ,:
,. D
decoded before starting the delivery of the, fil'St syml;>ol ta the user. In a real , -
... system th t~ delay 'may be operationa!ly undeshab/e.
, ft is observed ~t ail the survl~in~ ~ths do not stay distinct over
61
{ll ~. , . ' -
their entfre length but ha\l"e a tend~cy te stemJrom a single node s~veral constraint i "
fengths ebrlier (Hell~~CobS, 19711. For/example in Fig. ,(3.7) te survivors at , . (t +2). ( t + ( .( +2) ( t) . . nodes ~ , $1 c' ~4' merge together at node ~3 . Supposmg that , . . ail the suro,rs ot the present ~eeoding depth n merge togethe.r ot ~me node
~
S~n-M) JYin9,~Vel~ back~, M"> K, the'; ;!gordless of whch poth ;5 'fi~Qlly -,1. _
'h's~n as the ~odec:f path,' 'the survi~in9 poth ~eadin9 to node S~n-M) mu~t be ~ -1
(
..
................... ------------~~-----,----
)
62
. ,J--a part of it. Hence, it is not necessary te wait until the unique decoded path has
been obtained before starting to deliver the decoded symbo/s. A final decision may be made on these branches prior to the point of common convergence, ahd the
decoder con safely deliver ,the information (with a small delay corresponding to
M) as i t progresses in the tre"is. With th is procedure, clearly the decoder con , ~
eliminote the toi 1 altogether.
Ge~tin~ rid of the tail in this way leads to a v,ery attractive situation
with respect to the memory requirement. C/early instead of storing the entire,pa)s
hiS\ry for the total length l, the decoder needs to s!ore only the history of the
poth~. to and inc lding the point of comman convergence. If the point of .... common conv~e lies M leve Is bock (M is ca lied the ~mory of the decoder),
then the total amount W of path storoge required is
(3.23)
A practical decoder hos a given amount of storage ovailable, 50
that in fact M becomes fixed. A \practical refinement of the above method of
symbol decision consists in deciding on the oldest symbol of the most likely of the
q K -1 reta ined poths. It hos been shown theoreticolly by Forney (1967) th'il t with J , ,
this procedure, asymptotically the expected value of the~vror probability on that ~
oldest symbol is not increosed if the memory M is large enogh. It has been found , , '
through simulation (Odenwold~r 1970, Hel./er & Jac6bs 1971) thot a memory of . . ~
4, Or 5 constraint lengths is sufficient to insure a negligible degradotion from the
error probab il ity of the optimum dec6der. Should an error o~cur by this method it
...
....
\.
, \
63
does not I~od to catostrophy, for although- the Viterbi decoder was assumed to start
operating from a known starting,state, it has been found through simulation (Helle
& Jacobs 197"1) that if mey in fact start deeoding From any state. The fi rst 3 or 4
constraint lengths of data symbols may be unreliable but norrnatopefations will
. prevarJ th~reofter. Therefore, for a practical decoder the tail con be eliminated
and the memory reduced without altering the performmce of 0 (theoretical) optimum
dQcoder.
Fina/ly a quontized integer metrie may be used instead of the
, exact log-like/ihood function of Eq. (3.6) without much difference in error
performanq,e: the Viterbi decoder is relatively insensitive to metric quantization.
3.4.3 Comparisons and Limitatiol'ls of Sequential and Viterbi Decoding ,
"
'.
64
distribution of the number of computations to decode one branch prevai 15 regardless
of the rate (Falconer 1967). Stoorage requirements and decoding speed must be
traded in suc'h a way thet a decrease in buffer storage, must be compenseted by
an increase of the decoding speed over the bit rate, thu5 limiting the maximum 0
bit rate capability. Below R , if the number of computations that the decoder comp .
con perform per unit time is greater thon the mean value of the corresponding
. Pareto distribution of computations, then on the average the decoder con keep-up
with the data, although buffering is necessary. Regardless of the buffer storage "
provided and even for rates below R , occasional long searches do occur . camp ,
~
resulting in a pos~j'ble buffer overftow and the consequent erasing of long sequ~nces.
For ;equential decoders using long constraint lengths the probabi 1 ity of undetected , ,
errer is indeed very small, and usually the main contribution of errors cornes From
the buffer overflow.
For the Viterbi decoder such a situation obviously does not occur
due to the fixed nature of the computationol effort. Storage history and decoding
speed requirements may be definitely set for a pcirticular application.
, Since both the number of op~rations per decoded branch and the
total storage are prolfrtional to the l'}umber of s~ates, the complexity of the Viterbi
decoder is' Pfoportional to the number of states, and therefore it increases expmentio lIy ~
with the constraint length of the code. A,.lthough the errer probability decreoses
exponentially with the constraint Jength, clearlyaoy improvement on' the error
performance by increasing K, is expnsive in terms of computationol effort and
- -- - - - -- ~--------------
65
storage. Therefore., Viterhi decoders are limited to small (K 's 7) constrajnt ,
length codes, and consequently where very moderate error probability is sufficient.
J However, the very simple nature of a Viterbi operation and its identical repetition
, 4
from level to level permits a porallel implementation of a decoder and hence
decoding at very h igh data rotes in the tens of Megabits/sec ronge.
')
Decoding delays are inescepoble for both sequentia', and Vit~rbi ,
decoders. For sequential decoders the deloy is a direct consequence of the 1
variability of the computational effort, and for Viterbi decodrs it corresponds
to the poth memory storage of each state. Sequentiel decoders tend to have ,<
buffers longer thon the poth memory storage of Viter!;>i decoders. However, in a
real-time situation where a fixed time deloy is required for putting ouf data, the
total number of computations required in one delay g~s only lineady with ~he
constraint length K for sequentio 1 decodrs wherees it grows exppnentially with
K for Vi terbi decoders.
ln generel, perforrTlQnce end complexity combine in such a way
that sequential decoding is chosen over Viterbi decoding ;"hen high performance
;s required (P.( E ) s: 10-7 ), provided the information rate is sufficiently low that
operation below R is assured. M,?reover 1 since sequential decode'rs will be comp
extrerhely sensitive to bursty noise patterns, they wi" usually be restricted to
weil beheved rflemoryJess channels suc~ as the $pace cha~~el.
" , _ Viterbi decoders mey be preferred when.the performance demanded is more modest but Jhe dota rate very high. The crossover point occurs for bit error . .
. " -\
J
- ---~ - ~-------------------
~....... #
\ \
p~bability in the' 10-3 to 10.-5 range and constraint length between 6 1 .....
aod 8. "'
Most of the drawbacks of sequential decoding are th~ conse- currently
\ ' \'
most likely paths.
.,
\ \ "
1(
~
c l~ /
l \
\"\ \ )
1 .. ..-:
/'
\
J
"
'\
67
CHAPTER IV
MUlTIPl{ PATH STACK DECODING AlGORITHM
4.1 1 nt-roduct ion: Generalized Stock A/gorithm 1
The two a/gorithms presented in the preceding chapter appear to b~e
the extremes for determining the most likely transmitted sequence of a convolutionol
code, given the received sequence. The Viterbi algoritMm examines ~ distinct
paths ot every level while a sequentiel decoding olgorithm follows only that poth
'" thot appears to be the most likely. The step by step procedure of sequential decoding ,tends to reduce to a minimum the average value of the number of computations to
decode one information digit. Unfortunate/)'l( the procedure mokes this computational
effort variable with 0 Pareto distribution. By comporison, a Viterbi decoder carries
~Iong the one correct rth and ail incorrect poths of the trellis. As a consequence
'the computationol eff1rt becomes rather large but remains constant for the enti re decoding.
<
We now tjJm our attention to improving the computotionol behd'viour. j .~~ J ' - ,
of sequential decodin~ by reducing the yoriability of the" cdmputational effort without
degreding the efr?r performance. Th is variabi lit Y wi" be reduced when the distribution
function P ( C ~ t) is ltself reduced, mostly when t becomes relatively large. '-
The fundomentol idea is to keep the search proper~ies of sequentiel decoding and
the notion of a stock while using sorne cf the features of Viterbi decoding.
1
Denoting each explored poth by the node of its extremity, we recall ,." 1
tht a stock can be considered as a 1 tst 9f nodes ordere~ (in decreasing order)
'.
1
\ ,
;
68
according to their metric values. Allowing for the extension of sJt.eral paths From
the stock we now generalize the concepts M the stock algorithm of Zigangirov , /
, 1
and Jelinek.
Let a poth extension cycle consist of omputing the mefrics of the
successors of the extended poths, entering them in their proper pliee in the stock,
, and removing from the stac~ the nodes just extended. In a poth extension cycle,
the extension of the Jth node of the stock, J ~ 1, implies th~simultaneous
extension of a Il nodes stored h igher in the stac k. In other words if J paths are --; ,
extended, they ore the J most likely explored poths. The actual number of
extended -paths in 0 poth extension cycle is assumed determined by 0 decision
rule which is specifid in the algorithm.
Following the poth exten~'ion cycle a purging cycle eliminates from 1
/ -the stock some unwonted nodes and r rders the stock. The purging cycle is
directed by sorne pUfging rule 0150 s ecified by the user. The decision ond purging
rule; need not be kept fixed over e entire decoding, but" may be vori~ according
to any information pertinent ta the decoding as it progresses (post stpck contents,
metric behoviours, channel measurements, etc.).
The set of operations comprising a path extension cycle and a purging
cycle is defined as a decoding cycle. An ~Iso'rithm that uses this decoding cycle . at every step is called a generolized stock algorithm.
If is now easy to show that bWh the 'Z-J _cOO
&
e.
Il
\
l'
69
ln the Z-J algorithm ~e decoding eye le is degenerate since it eontains no purging
cycle. The decision rule is th~simplest possible as it directs the extension of only
the top node of the stock for eoch path extension cycle. (The quontized form of
the l-J algorithm introduces only ~~proctical, detail in the poth extension cyc,le).
Now: suppose the Viterbi decod~r uses a stack to store and order ail
explored paths. For a binary code the decision rule is.simply to extend the"
M = 2K- 1 ,v
(4.1)
, highest metric paths (the survivors at any depth) in the stack at each poth extension
cycle, whe'reas the purging rule is recognized as the elimination of ail non-surviving
paths. Ali redundancy is eliminated from the stock whose size remoins constant. , l
The M -path extension together with the purging fuIe c1early guarantees the v ..