IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 37, NO. 11, NOVEMBER 1989
1113
HighRate Punctured Convolutional Codes for Viterbi and Sequential DecodingDAVID HACCOUN,SENIOR
MEMBER, IEEE,
AND
GUY BEGIN,
STUDENT MEMBER, IEEE
AbstructThis paper investigates highrate punctured convolutional codes suitable for Viterbi and sequential decoding. Results on known shortmemory codes M I 8 discovered by others are extended. Weight spectra and upper bounds on the bit error probability of the best known punctured codes having memory 2 I M I 8, and coding rates 2/3 5 R I 7/8 are provided. Newly discovered rate2/3 and 3/4 longmemory punctured convolutional codes with 9 % M s 23 are provided together with the leading terms of their weight spectra and their bit error performance bounds. Some results of simulation with sequential decoding are given.
I. INTRODUCTION OR discrete memoryless channels where the noise is essentially white (such as the space and satellite channels), systems using convolutional encoding at the transmitting end of the link and probabilistic decoding at the receiving end are among the most attractive means of approaching the reliability of communication predicted by the Shannon theory; these systems provide substantial coding gains while being readily implementable. By far, error control techniques using convolutional codes have been dominated by lowrate R = l / u codes [lJ[3]. Optimal lowrate codes providing large coding gains are available in the literature [4][ lo], and practical implementations of powerful decoders such as Viterbi or sequential exist for data rates in the range of 1040 Mbitsls [11][13]. However, as the trend for ever increasing data transmission rates and high error performance continues while conserving bandwidth, the needs arise for good highrate R = b/u convolutional codes as well as practical encoding and decoding techniques for these codes. Unfortunately, a straightforward application of Viterbi and sequential decoding to highrate codes becomes very rapidly impractical as the coding rate increases. Furthermore, a conspicuous lack of good nonsystematic longmemory (M > 9) convolutional codes with rates R larger than 2/3 prevails in the literature. A significant breakthrough occured recently with the advent of highrate punctured convolutional codes [ 141 where the inherent difficulties of coding and decoding of highrate codes can be almost entirely circumvented. Viterbi or sequential decoding of rateb/u punctured convolutional codes is hardly more complex than for ratellu codes and, furthermore, either technique may be easily applicable to adaptive and variablerate decoding [lo], [14][21]. In this paper, we present highrate punctured convolutional codes suitable for Viterbi and sequential decoding. Extending results on good shortmemory codes (M < 9) discovered by others 1141[16], we provide the weight spectra and upper
F
bounds on the bit error probability of the best known punctured codes having memory 2 5 M 5 8 and coding rates 2/3 5 R 5 7/8. Using computer search techniques, newly discovered rate213 and 314 longmemory punctured convolutional codes with 9 5 M 5 23 are derived from the best known rate112 codes of the same memory. These codes, which are suitable for sequential decoding, are provided together with the leading terms of their weight spectra and their bit error performance bounds. The paper is structured as follows. Section I1 introduces the encoding of punctured codes and their perforation patterns. Section 1 1 presents Viterbi decoding for punctured codes and 1 the extension to sequential decoding. The search for good punctured code is the object of Section IV. This section contains the principal new results of the paper: weight spectra of known shortmemory codes and new good rate213 and 314 longmemory punctured codes, together with their weight spectra and bit error performance bounds. Accuracy of these results is verified by theoretical evaluation of the transfer functions of rate213 and 415 codes. Finally, some simulation results for sequential decoding are given in Section V, demonstrating the advantages of using highrate punctured convolutional codes over usual codes of the same rate. 11. BASIC CONCEPTS OF PUNCTURED CONVOLUTIONAL CODES A . Encoding of Punctured Codes A punctured convolutional code is a highrate code obtained by the periodic elimination (i.e., puncturing) of specific code symbols from the output of a lowrate encoder. The resulting highrate code depends on both the lowrate code, called the original code, and on both the number and specific positions of the punctured symbols. The pattern of punctured symbols is called the perforation pattern of the punctured code, and is conveniently described in a matrix form called the perfora
tion matrix.Consider constructing a highrate R = b/u punctured convolutional code from a given original code of any lowrate R = l/uo. From every uob code symbols corresponding to the encoding of b information bits by the original encoder, a number S = (uob  U) symbols are deleted according to some chosen perforation pattern. The resulting rate is then R = b/(uob  S), which is equal to the desired target rate R = b/u. By a judicious choice of the original lowrate code and perforation pattern, any rate code may be thus obtained [20], 1211. For example, Fig. 1 shows the trellis diagram of a rate1/2, memory M = 2 code where every fourth symbol is punctured (indicated by X on every second branch on the diagram). Reading this trellis two branches at a time and redrawing it as in Fig. 2, we see that it corresponds to a rate213, memory M = 2 code. A punctured rate213 code has therefore been obtained from an original rate112 encoder.
Paper approved by the Editor for Coding Theory and Applications of the IEEE Communications Society. Manuscript received February 9, 1888; revised March 25, 1988. This work was supported in part by the Natural Sciences and Engineering Research Council of Canada and by the Fonds FCAR of Quebec. This paper was presented in part at the 1988 IEEE International Symposium on Information Theory, Kobe, Japan, June 1988. The authors are with the Department of Electrical Engineering, Ecole Polytechnique de Montreal, Montreal, P.Q. H3C 3A7, Canada. IEEE Log Number 8930918.
B. Perforation Patterns An encoder for rate R = b/u punctured codes may be visualized as consisting of an original lowrate R = l/uo convolutional encoder followed by a symbol selector or sampler which deletes specific code symbols according to a given perforation pattern. The perforation pattern may be
00906778/89/11001113$01 .OO 0 1989 IEEE
1114
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 37, NO. 11, NOVEMBER 1989
rli Fig. 1. Tels for M = 2, R = 1/2 convolutional code.
rli Fig. 2. Tels for punctured M = 2, R = 2/3 code.
expressed as a matrix [PIhaving uo rows and b columns, with only binary elements 0s and ls, corresponding to the deleting or keeping of the corresponding code symbol of the original encoder. Clearly, both the punctured code and its rate can be varied by suitably modifying the elements of the perforation matrix. For example, starting from an original rate1/2 code, the perforation matrix of the rate213 punctured code of Fig. 1 is given bvr
whereas a r a t e 4 5 code could be obtained using the perforation matrix
code. With an orthogonal perforation pattern, the perforation matrix has uo = U rows and b columns, with each row containing only one element 1. Orthogonal perforation patterns ensure that all the different generators of the original lowrate Vuo code are used in deriving the desired punctured rateb/u code. In particular, it can be shown that any punctured code can be obtained by means of an orthogonal perforation pattern. Using this concept punctured codes identical to the best known usual rate2/3 and 3/4 codes have been obtained by the authors [2 13. The basic notions of encoding punctured codes having been established, the problem of their decoding by both the Viterbi algorithm and sequential decoding is examined in the next section.
m. DECODING PUNCTURED CODES OFVariablerate coding is readily obtained if all punctured rates of interest are obtained from the same lowrate encoder. Only the perforation matrices need to be modified accordingly as illustrated by [Pi] and [Pz]. Variablerate coding may be further specialized by adding the restriction that all the code symbols of the higher rate punctured codes are required by the lower rate codes. This iia restriction implies m n m l modifications of the perforation matrix as the code rates vary. Punctured codes satisfying this restriction are said to be ratecompatible. Ratecompatible punctured codes are especially useful in some rateadaptive ARQIFEC applications since only the incremental redundancy needs to be transmitted as the coding rate is decreased. Families of good noncatastrophic shortmemory ratecompatible punctured codes with rates varying from 8/9to 114 have been found by Hagenauer [191. Finally, another class of perforation patterns called orthogonal perforation patterns plays an important part in the search of specific punctured codes [21]. An orthogonal perforation pattern is a pattern in which any code symbol that is not punctured on one of the b branches is punctured on all the other (b  1) branches of the resulting rateb/u punctured
A. Viterbi DecodingGiven the received sequence from the channel, Viterbi decoding consists essentially of computing the likelihood (that is the metric) that a particular sequence has been transmitted for every possible encoder state. For rate R = b/u codes, there are 2 paths merging at every state and only the path with the largest metric is selected at each state. The process is repeated for each of the encoder states, so that clearly, as b increases, the complexity of decoding increases also very rapidly. Now for punctured highrate b/u codes, Viterbi decoding is hardly more complex than for the original lowrate l/uo code from which the punctured codes are derived. The decoding is performed on the trellis of the original lowrate code where the only modification consists of discarding the metric increments corresponding to the punctured code symbols. Given the perforation pattern of the code, this can be readily performed by inserting dummy data into the positions corresponding to the deleted code symbols. In the decoding process this dummy data is discarded by assigning them the same metric value (usually zero) regardless of the code symbol, 0 or 1. For either hard or softquantized channels, this procedure in effect
HACCOUN AND BEGIN: CODES FOR VITERBI AND SEQUENTIAL DECODING USUAL CODE
1115
inhibits the conventional metric calculation for the punctured symbols. Therefore, Viterbi codecs for highrate punctured codes involve none of the complexity of the straightforward decoding of highrate b / u codes. They can be implemented by adding relatively simple circuitry to the codecs of the original lowrate l/uo code. Furthermore, since a given lowrate l / u o code can give rise to a large number of highrate punctured codes, the punctured approach leads to a very attractive realization of variablerate Viterbi decoding. In addition to the metric inhibition mentioned above, the only coding ratedependent modification in a variablerate codec is the truncation path length, which must be increased with the coding rate. All other operations of the decoder remain essentially unchanged [14][2 1).
U3
UI W
U
I
InI
Decoded Path Paths Examined by Decoder
 Incorrect
I
1
1
2
3
4DEPTH (a)
5
6
PUNCTURED CODE
B. Sequential Decoding In sequential decoding, the decoding of the received message is performed one branch at a time without searching v) the entire tree. The exploration of the mostlikely fraction of I ! the tree is performed along the path having the current largest U Decoded Path I likelihood function or Fano metric 121. Starting from the root W r  Incorrect Paths node of the tree, the path selected to be extended one step Examined by Decode1 further in the tree is the path whose metric is the largest among all previously examined paths. For practical reasons, the tree is limited to a depth of a few hundreds to a few thousands branches, and the path that first reaches the end of the tree is 1 1' 2 2' 3 3 4 4 5 5 6 6' ' ' ' accepted as the decoded path. DEPTH For binary convolutional codes of rate R = b/u, 2 h (b) branches each with U code symbols emerge from each node of Fig. 3. Paths explored by the stack algorithm: a) usual R = 2/3 code; b) the encoding tree. Since decoding is an exploration of tree punctured R = 2/3 code. paths, a straightforward application of the decoding algorithm to codes of highrate b / u may become very quickly unacceptable. Several methods using discarding thresholds to eliminate stack algorithm for both an usual and punctured rate2/3 code unlikely paths have been proposed to alleviate this difficulty are compared. Since each decoding cycle of the algorithm [22], 1231. However, even with discarding thresholds, the use involves 22 = 4 stack entries for the usual code, but only 2 for of sequential decoding for highrate codes is somewhat limited the punctured code, then clearly, the small metric dips of the by the lack of suitable long memory codes [24][26]. By correct path are far more costly, in terms of computations and considering punctured codes instead, both difficulties of stack storage, for the usual than for the punctured code. For computational complexity and finding good highrate codes example, Fig. 3 shows that in reaching the same decoded path, a substantially larger number of nodes must be explored with can be circumvented. Just as for Viterbi decoding, the punctured approach to the usual code than with the punctured code. An additional advantage of the punctured approach is that highrate codes can, as easily, be applied to sequential decoding. Again, decoding is performed on the tree of the the flag procedure usually used to save stack entries and speedoriginal lowrate code rather than on the highrate code, and in up the decoding of ratel/u codes can be readily implemented principle any sequential decoding algorithm could be used. here at no additional cost [ 171, [ 181, [20]. The use of rejection However, only the ZigangirovJelinek or stack algorithm will thresholds to discard unlikely branch extensions can even be be considered here [27], [28]. Since it is very simple and added up at hardly any cost. Therefore, the stack algorithm readily amenable to variants and generalizations [29], [30], can be very easily adapted for the decoding of punctured this algorithm is a good candidate for the decoding of codes, and the method can lead to substantial simplifications punctured codes since it requires only minimal modifications and stack storage savings over the direct approach. Finally, as [ 171. Given the perforation pattern the decoder proceeds one with variablerate Viterbi decoding, variablerate sequential branch at a time just like for the decoding of the original low decoding can be easily implemented. rate l/uo code. The algorithm is repeated through (b  1) Longmemory codes present no problem in sequential elementary intermediate steps involving the original lowrate decoding. The achievement of low error probabilities, fast code. At each intermediate step, the top node of the stack is decoding speeds, and low overflow probabilities require the extended into its two uosymbol branches from which specific selection of long memory codes having a large free distance symbols are discarded. Again this is accomplished by inhibit and a good distance profile. Extensive lists of such codes exist ing the metric evaluation of these punctured symbols, that is, for lowrate l / u codes [25], [31]. However, with few by assigning them the same metric increment (usually zero). exceptions optimal longmemory codes suitable for sequential The advantages of the procedure is that the likelihood that decoding and with rates higher than 314 are not known [24], the decoder is following the correct path is evaluated on a bit [26]. By taking the punctured approach, the search for good bybit basis rather than on the basis of blocks of b bits like in a highrate codes becomes much simplified. The search probbruteforce decoding. Hence, the decision to retreat in the tree lems for punctured codes is examined in Section IV. and follow another path may be taken earlier, at a possible reduction of the computational variability and required stack C. Error Performance storage. In addition, since each elementary path extension For discrete memoryless channels an upper bound on the bit involves only two branches, for a rateb/u code, the forward error probability of a convolutional code can be obtained. The motion of the decoder involves 2b entries in the stack rather derivation of the bound is based on an union bound argument than the 2 b entries required of a straightforward decoding. on the transfer function T(D, B) of the code which describes This is illustrated in Fig. 3 where the paths explored by the the weight distribution, or weight spectrum, of the incorrect

1
1
)
1
"
'
1
~
'
1
'
'
1116
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 37, NO. 11, NOVEMBER 1989
codewords and the number of bit errors on these paths [l], [32]. The entire transfer function of the code is rarely known but an upper bound can still be calculated using only the first few terms of two series expansions related to the transfer function T ( 0 , B), that is
and
In these expressions, d is the free distance of the code and
aj is the number of incorrect paths or adversaries of Hammingweight j, j 5 dh, that diverge from the correct path and remerge with it sometime later. As for cj, it is simply the total number of bit errors in all the adversaries having a given Hamming weight j . Using the weight spectrum, an upper bound on the bit error probability PE of a code of rate R = b / u is given by
(3)The evaluation of this bound depends on the actual expression of the pairwise error probability Pj, which in turn depends on the type of modulation and channel parameters used [13, [32]. For coherent PSK modulation and unquantized additive white Gaussian noise channels, the error probability between two codewords that differ over j symbols is given by
Pi= Q(d2jREb/No)where
(4)
QW=
s, 6 (z 1
exp
1
z2) dz
(5)
and where Eb/No is the energy per bittonoise density ratio. A good evaluation of the bound on PErequires knowledge of the transfer functions (1) or (2). However, for the vast majority of codes only the first few terms of either functions are known, and very often only the leading coefficients a d f , and c d f , are available. But for channels with large EJN0 values such as those usually used with highrate codes, the bound on PEis dominated by the first term, c d k . Naturally, bound (3) is also applicable for punctured codes whether decoded by a Viterbi or a sequential decoder. Therefore, in deriving the bit error performance of punctured codes, the free distances and at least the first few terms of the weight spectra of these codes must be obtained. Such partial weight spectra are provided in this paper for the best known shortmemory codes given by others [l5], [16] as well as for the new longmemory codes discovered by the authors.
IV. SEARCH GOOD PUNCTURED CODES FOR Since punctured coding was originally devised for Viterbi decoding, the criterion of goodness for these codes was the free distance, and hence the maximal free distance punctured codes that first appeared in the literature were all shortmemory codes [141[ 161. For sequential decoding, good longmemory punctured codes should have both a large free distance and a good distance profile. In searching for good punctured codes of rate b / u and memory M , one is confronted with the problem of finding both an original code of memory M and rate R = 1/UO, and its accompanying perforation pattern. Not unlike the search for usual convolutional codes, the search for punctured codes is often based on intuition and trial and error rather than on a strict mathematical construction 1211. An approach that
yielded good results is based on the intuition that good codes generate good codes. Consequently one could choose a known good code of memory M and rate R = l/uo, (e.g.. R = 1/2, 1/3, 1/4,  ) and exhaustively try out all possible perforation patterns to generate sood punctured codes of rates R = b / u and memory M . Naturally, if families of variablerate codes are desired, then all the perforation patterns must be applied to the same lowrate code. Furthermore, if the codes are to be ratecompatible, then the perforation patterns must be selected accordingly [191. Obviously puncturing a code reduces its free distance, and hence a punctured code cannot achieve the free distance of its original code. Although the freedistance of a code increases as its rate decreases, using original codes with rates l/uo lower than 1/2 does not always warrant punctured codes with larger free distances since, for a given b and U, the proportion of deleted symbols, S/uo b = 1  (u/u,,b), also increases with UO. Consequently, good results and ease of implementation tend to favor the use of rate1/2 original codes for generating good punctured codes with coding rates of the form R = (U  l)/u that may be suitable for either Viterbi decoding or sequential decoding. Further results on both short(M 5 8) and longmemory punctured codes (M 2 9) that are derived from R = 1/ 2 codes are provided in this paper. Although one could select the punctured code on the basis of its free distance only, a finer method consists of determining the weight spectrum of the punctured code according to (1) and (2) and then calculating the bit error probability bound (3). The code yielding the best error performance may be thus selected as the best punctured code, provided it is not catastrophic. Clearly then, starting from a known optimal lowrate code, a successful search for good punctured codes hinges on the ability of determining the weight spectrum corresponding to each possible perforation pattern. Although seemingly simple, finding the weight spectrum of punctured codes turned out to be a very difficult task. This is due to the fact that even if the spectrum of the lowrate original code were available, the spectrum of the punctured code cannot be derived from it. One has to go back exploring the tree or trellis of the lowrate original code and apply to each path of interest the perforation pattern. For the wellknown shortmemory codes, the procedure is at best a rediscovery of their weight spectra, whereas for longmemory codes where often only the free distance is known, it is a novel determination of their spectra. The problem is further compounded by the fact that since puncturing a path reduces its Hamming weight, then in obtaining a given number of spectral terms, a larger number of paths must be explored over much longer lengths for a punctured than for a usual code. In our search we have used recently developed algorithms for the spectral determination of convolutional codes [33][35]. Each algorithm has been designed to be especially efficient within a given range of memory lengths, making it possible to obtain substantial extentions of the known spectra of the best convolutional codes. Obviously, the spectral terms obtained by these exploration methods must match exactly the corresponding terms derived from the transfer function of the code. This verification has been conducted for two different codes by evaluating their transfer function as detailed in the Appendix.

A . ShortMemory Punctured Codes A number of shortmemory (M 5 8) punctured codes of rates R = (U  l)/u varying from 213 to 16/17 have been proposed by Cain et al. [14], Yasuda et al. 1151. [16], and more recently by Hagenauer [19] for ratecompatible codes. In particular, in [15] all the memory6 punctured codes of rates varying from 213 to 16/17 have been derived from the same original memory6, rate112, optimal convolutional code due
HACCOUN AND BEGIN: CODES FOR VITERBI AND SEQUENTIAL DECODING TABLE I WEIGHT SPECTRA OF YASUDA et al. PUNCTURED CODES WITH R 2 s M s 8original code
1117TABLE 1 1 1 WEIGHT SPECTRA OF YASUDA et al. PUNCTURED CODES WITH R 2 5 M 1 8original code M GI G ,
P10
I 3 1
1
d,
punctured code (a.,n=d,.d,+l,dj+2, [c,,n = d,,d, l,d, + 2 ,
+
..)
. . .]
(1.4,14.40.116.339,991,2897,
18468,24752) [ I , 10,54,226,856,3072,10ffl7,35998, 119478,3909041(3,11,35,114,381,1276,4257.14208,
1
=
2/3,
=
4/5,
P10 11
dj
47413,158245)
#2
I
5
I
7
1100
[ 10,43.200,826,3336,13032,49836,187480,10
696290,25595211 f 1.0,27.0.345,0.4528.0. 59435.0)
.
[1,0,124,0,2721,0.50738,0,
862127,0]10 11
6
(19.0,220,0,3089,0,427~,0.
588022,O) [96,0.1904,0,35936.0.638393. 0,10657411] 6(1,16,48.158,642,2435,9174,
1000 1111
11 10
34705,131585.499608)[3,70,285,1276.6160,27128,117019,
498860,2103891,8784123]10 11
7
(9,35,104,372,1552,5905.
22148,85189,323823,1232139) l47.237.835.3637.17770.76162, 322120,1374323,5730015,23763275]
U1 I I PO7 247 37110 10
I i[1,36,309,2060,12320,69343,375784, 1983150,10262827,52316541 5,36,200,1070,5919,32721,180476,
punctured code (o,,n= d,,d, l,d, + 2 , . . .) [c,,n = d,,d, 1,dl + 2 , . . .] 2 I (1,12,53,238,1091,4947,22459 102030,463451,2105128)
+ +
995885,5495386,30323667)
[14,194,1579,11313,77947,514705,3305113, 3,16,103,675,3969,24328.147313,
20808587,129003699,790098445
I 19259760,132078031,896198879]I (7,54.307,2005,12970,83276,
897523,5447618,33133398) I11.78.753.6901.51737.386465.2746036.
4
~34556.343l703.22040ll0.l41546111 ) [40,381,325 1,27123.2 1345 1 , I fi2 1873, 1201l339,87380826,627189942,445L640~18] 3,24,I72.lI58.7409,4U729. 319861,209797113765538,90315667) 112.188.1732.15256.121372.945ffl5,(20.115.694 4816.32027.210909. l3928669223171.610132364036650541
5
[ l68,1232,9120,78715,626483,4758850,663936,4431049,29536078,197041 141)
8
561
753
1 10 1[31,469.4205,34011,268650,2113955,
I 1 127923,544616,2313272,9721227]TABLE I1 WEIGHT SPECTRA OF YASUDA et al. PUNCTURED CODES WITH R 2 5 M s 8Moriginal code GI GZ2=
I11.46.324.1594.6425.29069.
3/4,
TABLE IV WEIGHT SPECTRA OF YASUDA et al. PUNCTURED CODES WITH Roriginal code punctured code (a,,n=d,,d,+l,d,+Z, ..) [ + , n = dj,d, 1,dl + 2 ,...]
= 5/6,
PIO 1 1 10
d, 3
punctured code (a,,n=d,,d,+l,d,+Z, .) [c,,,n = d,,d, l,d, + 2 ,...]
+
P10111 110 0 0
d,
+
5
7
(6,23,80,290,1050,3804,13782,
2
(2,26,129,633,3316,17194,88800,
j + $3 156
49930.180890.655342)[15,104,540,2557,11441,49340,207335,
459295,2375897,12288610)[2,111,974,6857,45555,288020,1758617, 10487425,61445892,355061333]
854699,3471621,139363811 17110 10 1
4
(29,0,532,0,10059,0,190112,0,
3593147.0) [124,0,4504,0,126M9,0,3156062,0,
10 10 0 1 10 1 1
3
(l5,96,601,3918,2539l,lffl481, 1065835,6906182,44749517,289960852) [63,697,6367,53574,42ffl71,3277878, 24573195,180823448,131 1630186,9405755195](5,37,309,2282,16614,122308,
110[1,7,125,936,5915,36608,216972,
101 1 1 1 I 000
3
900991,6634698,48853474,359716572) [20,265,3248,32328,297825.2638257,22710170,191432589,1587788458,13003147461]
100 111
42626,216131,1095495,5557252)13,85.490.3198,20557.123384,
10 0 0 0 11 1I 1
4
(19,171,1251,9573,75167,585675, 4558463,35513472,276598647,2154'249884)[100,1592.17441,166331,1591841,14627480, 131090525,1155743839,10049038363,86420307777] (14,69,654,4996,39699,315371,
133
171
1 I O
5
(8,31.160.892.451223307121U~7.625059.32.14RR6.16753077I2253373,13073811,75152755,428005675]
I 0I
11010 10 10 1
4
2507890,19921920,158275483,1257455600) [92,528.8694,79453,792114,7375573, 67884974,610875423,5427275376,4766421 5639) 4(2,51,415,3044,25530,200878,
1 1 100 10 01 1[239,0,11165,0,422030,0,14812557.
1628427,12995292,104837990,839661288)[7,426,5244,49920,514857,4779338, 44929071,40647031 1,3672580016,32529952038]
0,493081189,Ol 10 0 236294,1236990,6488527,34056195)[52,659,3265,21442,133697,805582, 4812492,28107867,162840763,935232173] 10 110 1 10 0 1
5
(19,187,1499,11809,95407,775775, 6281882,50851245,411841549,3335453320) [ 168,2469,25174,242850,232M29.21768364.199755735,1807353406,16178271375,143502720686~
to Odenwalder [32]. A more complete list of rate 2/3 to 13/14 punctured codes has been derived from the optimal rate 112 codes with memory M varying from 2 to 8 and compiled by Yasuda et al. [16]. In this list, for each code the perforation matrix is provided but the weight spectrum is limited to the first term only, that is, the term corresponding to dfree. Using the given perforation patterns, we have extended Yasuda's et al. results by determining the first 10 spectral terms for all the codes having memory lengths 2 5 M 5 8 and coding rates 213, 314,415, 516, 617, and 718. These results are given in Table ITable VI, respectively. Each table lists the generators of the original lowrate code, the perforation matrix, the free distance of the resulting punctured code, and the coefficients a, and c, of the series expansions (1) and (2) of the corresponding weight spectra. These spectra determina
tions have been conducted on SUN13 and APPOLO microcomputers and turned out to be quite time consuming. The amount of CPU time required for determining each further spectral coefficient increases with the coding rate by a factor varying from 3.8 to 11 as the coding rate varies from 2/3 to 7/8. Beyond 10 spectral coefficients the computer time becomes prohibitive, reaching several hundred hours with these computers. Now in order to verify the accuracy and validity of these results the entire transfer functions T ( D , B) have been evaluated for both rate213 and rate415 memory2 punctured codes. As detailed in the Appendix, the transfer functions of the codes have been determined by solving the matrix equations describing the transitions between the different states of the encoder. As expected all the spectral terms
1118
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 37, NO. 11, NOVEMBER 1989
TABLE VWEIGHT SPECTRA OF YASUDA et al. PUNCTURED CODES WITH R = 6/7,
25M18P25 dj 2 ( % , n = d j , d j + l , d t + Z ,...) [%,n = d j , d j l , d , Z,.. .]
+
+
7
I4 I 23 I 35
101111 1 1000 0
I
I
Un
(4,39,221,1330,8190,49754. 302405.1~129.11194714.68101647~ [5,186,1942,16642,131415,'981578, ' 7076932,49784878,343825123~0813323] 1000 1 1 2 (1,25,188,1416,10757,81593, 111100 619023,46~330,3~3844,27w85463) [2,134,1696,18284,179989,1678667, 15082912,132368246,1140378555,9~489~26] 1 o 1 o 1 o I 3 I (14.100.828.7198.60847.513573. 1 1010 1 4~760,~675l720,31062M41,i628803066) [69,779,9770,113537,1203746,1~17198, 12070.1682,1167799637,11118330624,104537090090] 1 10 1 10 3 (5,55,517,4523,40476,~2074~2~848, 101001 288725n,2578984.L2,2303819072) 125,475,6302,73704,82~,8816834,
I
I
6
133
171
I7
I
I
I 247 I 371
B I I [5,169,n25.32233.37~1,4169788,454174~, B 1 o 1 o o 1 I 4 II483171499,5063039490,5239471~1,536645404278] I (11,155,1399,13018,1~560,1154C87,
11101 0 10010 1
3
(1,20,~3,19a1,1~3.169175,15761~, 14656816.136394365.1269388805.11812597193~
TABLE VIWEIGHT SPECTRA OF YASUDA et al. PUNCTURED CODES WITH R = 718,
2 s M s 8P2 5 7 1 1 00000 793483,5293846,35318216,235589046) [8,393,4248,38142,325739,2847528, dj punctured code ( o . , n = d j , d j + l , d j + Z ,...) [ k , n = d j , d j + 1,dj +2,. .]
.
3
15
17 11 11 1 0 1 1717289,14547758,123478503,1047166985) [4,219,3456,38973,437072,4492~,
453031M,4rl~B8,4265246076,40445094'4 23 35 1010011 1 101 10 0
I
1I
3
I
I 14019214,1~7So3~,1373401554,13593428075)(49,1414,21358,284324,3544718,4m8392,~9.l~.l195.12139.123889.1259682.489726840,5552764343,6 1942612828,682203758430]
f13.145,1471.1~73.143110*1416407.
I
the rate 314 code than for the rate 415 code. Clearly, it is the coefficients e,,which adversely affect the error performance of the rate 314 code. This anomaly illustrates the fact that selecting a code according to the free distance only is good in general but may sometimes be insufficient. Knowledge of further terms of the spectrum will always provide more insight on the code performance. The theoretical bound on PB,for the original M = 6, R = 1/2 optimalfreedistance code is plotted in Fig. 4, together with all the punctured codes with rates R = 2/3, 3/4, 4/5, 5/6, 6/7, and 718 derived from it. For cornparison purposes Fig. 4 includes also the performance curve of the uncoded coherent PSK modulation. It shows that the performance degradation from the original rate1/2 code is rather gentle as the the coding rate increases from 112 to 718. At PB = coding gains for the punctured rate2/3 and 314 codes are 5.1 dF3 and 4.6 dF3, respectively. These results indicate that these codes are indeed very good, even though their free distances, which are equal to 6 and 5 , respectively, are slightly smaller than the free distances of the best known usual rate213 and 314 codes which are equal to 7 and 6, respectively [24]. The error performance of these codes has been verified using an actual punctured Viterbi codec 1151, [16], and independently, using computer simulation [36]. Both evaluations have been performed using 8level soft decision Viterbi decoding with truncation path lengths equal to 50,56,96, and 240 bits for the coding rates 213, 314, 718, and 15/16, respectively. Both hardware and software evaluations have yielded identical error performances which closely match the theoretical upper bound. At rate 15/16, the coding gain of the M = 6 code reaches a The fact that such a coding gain substantial 3 dB at PB = can be achieved with only a 7 percent redundancy and a Viterbi decoder that is hardly more complex than for a rate112 code makes the punctured coding technique very attractive for shortmemory codes. For longer codes and larger coding gains, Viterbi decoding becomes impractical and sequential decoding should be considered instead. Newly discovered longmemorylength punctured codes and their performance with sequential decoding are presented next.
B. Long Memory Punctured Codes izs34~i~,i30~ws~~,i331j13258,13562o351~~~ Following the same approach as for the shortmemory codes, one could choose a known optimal longmemory code [So, 1360,18971,252751,3165885,3822720, 450898174,5217969030,59518129067,671110852141] of rate 112 and exhaustively try out all possible perforation 6 133 171 1 1 1 1 0 1 0 3 (2,46,499,5291,56179,599557. patterns to generate all punctured codes of high rate R = b / u . 6387194,~117821,72~~96,~41~67~) 10 00 10 1 [9,500,7437,105707,14~43,179092~, The selection of the punctured code is again based on its bit 222282289,27w822556,32434972565~83973015410] error performance, which is calculated from the series 7 247 371 1 0 1 0 1 0 0 4 (26,26L,~32,30389,32927,3571So7, expansion of its transfer function. Here, one of the difficulties 1 10101 1 38799203,421331658,457~8~2,49898647357) [258,3652,52824,746564,9825110,125472545, is that for the original lowrate and longmemory codes of 1567656165,192~335327,23319~14828,~948799~442] interest, only very partial knowledge of their weight spectra is 8 561 753 1 1 0 1 0 1 1 4 (6,132,1289,13986,1539,1694634, 18532588,2028~21,2~74919,24305050429) 1010100 available. In fact, beyond constraint length M = 15, very [70,1842,2~,337514,4548~4,586~237, often only the free distances of these codes are available in the 738611595,9148958248,111799479417,1351261813545] literature [34]. Extending previous work in this area [20], [2 11, a computer obtained by the search algorithms match exactly the corres search for the best rate213 and 314 punctured codes with memory length extending from 9 to 23 that are derived from ponding terms of the transfer functions. The bit error probability upper bound PB over both the best known rate112 has been conducted. For each code the unquantized and hard quantized additive white Gaussian noise first few terms of the weight spectrum have been obtained for channels has been evaluated for all the codes listed in Tables I each possible distinct perforation pattern. The obviously bad VI [36], using all the listed weight spectra terms. For a l these codes are discarded and the codes to be investigated further are l codes the error performance improves as the coding rate those that provide the largest free distance and the smallest decreases, indicating wellchosen perforation patterns. A number of bit errors in their weight spectra. However, the notable exception concerns the memory M = 3 code where h a l selection of the best punctured codes is based on the the error performance turned out to be slightly better at rate evaluation of the upper bound on the bit error probability. 415 than at rate 314. This anomaly may be explained by an Naturally, the codes obtained with this approach are suitable examination of the spectra of these two codes. As shown in for variablerate decoding with sequential decoding. The Tables II and III, although the free distances of the rates 314 search has been initially limited to these two rates only since a and 415 codes are df = 4 and df = 3 respectively, the number fine comparison of the punctured codes with the best known terms are _ ___~ for nonsystematic highratecodes of the same memory length is of bit errors C , on the various s~ectral ~ far larger . ~ 5 53 75 10 I1 10 I 1 1 o o o 1o 3
I
~~~~~
HACCOUN AND BEGIN: CODES FOR VITERBI AND SEQUENTIAL DECODING
1119TABLE VUI BEST RATE314, 9 5 M 5 23 PUNCTURED CODES WITH THEIR WEIGHT SPECTRA, PERFORATION MATRIX, AND ORIGINAL R = 1/2 CODESpunctured code
BEST RATE213, 9 Sr M
TABLE VI1 23 PUNCTURED CODES WITH THEIR WEIGHT SPECTRA. PERFORATION MATRIX. AND ORIGINAL R = 1/2 CODES5origind codeponctllred code
P I dt I ( o . . n = d , ..d , + l , d t +. 2 .1
I I [%,n=df,d,+l,dl+Z,7
,...)]IO01 I I
0
(1,10.29,94,415,1589, 5956.23060.88731.338932)
I6
1 [%,a=d,,dj+l.d,+2.(4,31.15L.774,3967,2I140,
]
[3,70,207,836.4411.19580, 82154,353734,1504013,6283078] 0 I 8 I 11 21 65.226 907 3397.
;3z23,~io~~;194iz~.7i8110)[8,165,560.2321.10932,4692l, 204372,869373,3635291,15l75050]
I O 1 I 6 I 12.7.59.338.1M6.8918. M i i o m i , ~ ~ r ~ m ) [ 9,40,517.3731,22869,lU233, 864435,5134427,30224375,175613909I2,55,236,1271,6853,364l8, 192287.1012294.5346233.28215351 1
; ;I I
111674,586630,3087541.16251939~
[38,27O,l~O.l05M,6360l,387227,'2314232.135472~,78566879,451980738]
28907,111733,428615,1648700) [86.326.1379,6350,27l94,114590.492275,2078228,8657889,35986562]
4,8,45,193,604,2383,
[ 25.65,411.1991.6925,11304,22003,84614,324859,1248828) [ 46,268,1066,4344,19992,87284,368272,1554340,6487550.26934394]
(34,182,965,6294.3~61.230668,' 1370985,8068081,47123543,272477579] 1 1 0 I 7 I 11 11 41 202.1334 6982.
14
55367
63121
I1 I0
10
(2,6.37,153,582,2184, 8329,32111.124791.476203)[13,62.334.1606,7321,30635, 131195,555564,2366161,9790577]
3,19.95,529,2800,14824, 77841,413026,2182612,Il529739)[28,159,1186,7461,46689.282212, 1659671,9787540,56862563,327592538]
15
111653
145665
II I0
10
(3,0,46,0,683.0. 10135.0.151488.0) (28.0.397.0.7735.0, 150284.0,2734434,0](8,45,145,567,2182,8242,
16
347241
246277
I 1 I 0
12
31574,122351,470446,1808049) [ 68,495,1569,7112,31556.131760,554286,2349061,9797200,405872l7]
[5,49,311,2266.12976.78'198, 464998,2729868,15946528.92079794]
17
506477
673711
I 011
12
(2,24,79,320.1251.4784, 17972,69040,267659,1029502) [l I ,253,889,3978,18056.76298,316866,1326374,5565438,23081423]
18
1352755
1771563
11 10
12
(2.11,27,137,514.1940, 7486,28593,110382,425046)[18,105.276,1679,7028,29848,
I111
I
I 1552898,9028366,52204067,299949760]IO
[ 5,142,1375,6842,44160,265938,(6,51,217,1014,5622,29034,
I
126696,529543,2222988,9245 160) 19 2451321 354671311 1012
(1,3,14,71.249.995,3841,
14926,57232,21922'11
[ 9,21,139,715.2940,13629,59239,254614,1068299.4455654]
[40.240,1219,6934,472l3,283532, l645614,9598742,55146996,315652I79]
19
2142513
3276171[99,425.1425,6158.25037,105225,
443232,1836907,7578943,312l9806100
70561,374042,1975961,10442453) [48.202,1248,7445,4698l,279184. 1640495,9587002,55315528,316950355
[40.249.1510,8120,53164]
[11,5,231,736,3096]
[143,266,2038,12128,73739]
[17,163,927.3614,14459]I I
[79.1275,5279,32916,200731]
IO0[39,170,852 ]111
13
(2l,I4l,707) [292.2340,13196 ]
limited to the rate213 and 314 codes. This is due to the fact that with very few exceptions, optimal long codes for sequential decoding are known for rates 213 and 314 only. Tables VI1 and VI11 list the characteristics of the best punctured codes of rate 213 and 314, respectively, with memory lengths varying from 9 to 23, that are derived from the best nonsystematic rate112 codes. From M = 9 to M = 13 the original codes are the maximalfreedistance codes discovered by Larsen [37], whereas for 14 5 M 5 23 the original codes are those of Johannesson and Paaske [25]. In both tables, for each memory length the generators of the original code and its perforation matrix are given, together with the free distance of the resulting punctured code. Just as with shortmemory codes, the first few terms a, and c,,, n = dbe, dbe + 1, df,, + 2, . of the series expansions of the weight spectra are also given for each punctured code. In deriving these spectral coefficients, the exploration of the original code had to be performed over a considerable length. In the search for the best punctured codes, the perforation patterns were chosen as to yield both a maximal free distance
and a good distance profile. Although all perforation patterns were exhaustively examined, the search was somewhat reduced by exploiting equivalences of the perforation patterns under cyclical shifts of their rows [21]. Among all the codes that were found, Tables VI1 and VILI list only those having the smallest number of bit errors cdfie at the free distance dh, and obviously, all the codes listed are noncatastrophic. Table VII lists two M = 19, R = 2/3 punctured codes derived from two distinct good M = 19, R = 1/2 original codes. The free distances of these two punctured codes are equal to 12 and 13, but an examination of the spectral coefficients shows that the coefficients c, are far larger for the d , = 13 code than they are for the dfre = 12 code. f Consequently, as confirmed by the calculation of the bit error bound PB,the code with dfre= 13 turned out to be worse than the code with df, = 12, by approximately 0.35 dB. This anomaly confirms further the need to determine at least the first few terms of the weight spectra when searching for the best codes. Fig. 5 plots the free distances of the original rate 112 codes
1120BIT ERROR PERFORMANCE BOUNDS UNQUANTIZED CHANNEL M=0 CONVOLUTIONALCODES
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 37, NO.BIT ERROR PERFORMANCE BOUNDSUNOUANTlZED CHANNEL RATE = 2/3 CONVOLUTIONAL CODES
11, NOVEMBER 1989
PE
EblNO ( dE )
EblNO ( dB )
Fig. 4. Bit error performance bounds for the M = 6, R = 1/2 original and punctured rates 213, 314, 415, 516, 617, and 718 codes.
Fig. 6. Upper bounds on the bit error probability over unquantkd channel for R = 2/3 punctured codes with 2 5 M s 22, M even.
FREE DISTANCE OF PUNCTURED CODESOriginal
18
f0 
15 12 9
6
324 6
0
10 12 14 16 18 2 0 22
M Fig. 5. Free distancesof original rate112codes and punctured R = 2/3 and 314 codes derived f o them as a function of M , 2 I M I22, and upper rm bounds.
and the punctured rate2/3 and 314 codes derived from them as a function of the memory length. Except for the two anomalies mentioned above, the behavior of the free distances is as expected: the free distance of the punctured codes of a given rate is generally nondecreasing with the memory, and for a given memory length the free distance decreases with increasing code rates. When the punctured codes of rate b / u are determined from the best original lowrate l / v code, an upper bound on the free distance of the punctured code can be derived [21]. This derivation, which is based on an analysis of the effect of the different perforation patterns on the spectrum of the original code, yields the bound where d m ) and d&(o) are the free distances of the punctured and best codes of rate l l v , respectively. This bound is also plotted on Fig. 5 for the rate213 and 3/4 codes with memory M I 13. The upper bounds on the bit error probability have been evaluated for all the punctured codes of rates R = 2/3 and 3/4 and are shown for even values of memory in Figs. 6 and 7. This evaluation has been performed according to (3) for q u a n t i z e d Gaussian channels. These bounds indicate a normal behavior for all the punctured codes listed in Tables V I and VIII. The bit error performances all improve as the I coding rate decreases and/or as the memory increases, with approximately 0.4 dB improvement for each unit increase of
the memory length. At Pe = the M = 22, R = 2/3, and R = 3/4 punctured codes can yield impressive coding gains of 8.3 dB and 7.7 dB, respectively. The selection of the best punctured codes listed in Tables V I and VIII has been based on both the maximal free distance I and the calculated bit error probability bound. However, the choice of the best punctured code was often not a very clearcut one. It was often observed that different puncturing patterns yielded only marginally different error performances [36]. In some cases the performance curves were almost undistinguishable, and, hence, several best punctured codes having the same rate and memory may be obtained. However, since these codes are for sequential decoding applications, the final selection must also be based on the distance profile and computational performance. Short of analyzing the computational behavior, when in doubt, the h codes finally selected and listed in the Tables had t e fastest growingcolumn distance function. In the search for punctured codes, the above approach produces good but not optimal codes since the original lowrate code is imposed at the outset. A measure of the discrepancy between optimal and punctured codes of the same rate and memory is provided in Fig. 8, which shows the bit error performance bound curves of both the memory9 punctured and maximal free distance codes of rates 213 and 3/4. These bounds have been computed using only the term at d h for the MFD codes and using both terms at d h and d h +1 for the punctured codes. Based on these terms only, the two MFD codes appear to be only slightly better than the punctured codes. Therefore, it may be concluded that although not optimal, the error performances of the rate2/3 and 3/4 punctured codes of memory 9 closely match those of the MFD codes of the same rates and memory lengths. The same general conclusions may be made for the other punctured codes with different memory lengths [36]. Given an optimal usual highrate code of rate R = b/v and memory length M, one could attempt to determine the lowrate l/v code which, after puncturing will yield a punctured code that is equivalent to that optimal code. This approach, which is the converse of the usual code searching method, can be used to find the punctured code equivalent to any known usual highrate code. Based on this approach and using the notion of orthogonal perforation patterns, a systematic construction technique has been developed by the authors [21].
HACCOUN AND BEGIN: CODES FOR VITERBI AND SEQUENTIAL DECODINGBIT ERROR PERFORMANCE BOUNDS UNQUANTIZED CHANNELRATEi
1121BIT ERROR PERFORMANCE BOUNDSUNQUANTIZED CHANNELM= 9
314
CONVOLUTIONAL CODES
PB
USUAL AND PUNCTURED CODES
PB
1oaL
4
5EblNO ( dB 1
6
1 Fig. 7. Upper bounds on the bit error probability over unquantized channel for R = 3/4 punctured codes with 2 5 M 5 22, Meven.EblNO ( d B
Fig. 8. Bit error performance bounds for maximal free distance (MFD) codes and punctured codes of rates 213 and 314 with memory M = 9.
Using this technique punctured codes equivalent to the best known nonsystematic rate2/3 codes with memory lengths up to M = 24 have been found. Likewise, punctured codes equivalent to the best known rate3/4 codes with memory lengths up to M = 9 have been tabulated [21]. Therefore, optimal highrate codes may be obtained as punctured codes, but it should be pointed out that the punctured codes generated by this approach are not suitable for variablerate applications since each punctured code has its own distinct lowrate original code. V. COMPUTER SIMULATION RESULTS The stack algorithm for sequential decoding of punctured codes has been simulated on a computer. In these simulations the results of interest concern both the computational effort and the bit error performance. For R/R,,,, values ranging between 0.85 and 0.99, meaningful error events could be observed with shortmemorylength codes only. In these cases the resulting error probabilities were always found to satisfy the upper bound computed from the first terms of the code spectra. As the memory lengths of the codes increase, no errors were collected within a reasonable simulation time. Therefore, as usually expected with sequential decoding of longmemory codes, the simulation results will concern mainly the computational effort. Fig. 9 shows a typical distribution of computation obtained for the memory23 rate2/3 punctured code equivalent to the best known usual code of the same memory and rate discovered by Johannesson and Paaske [25]. For comparison purposes, the computational distribution for the usual code is also plotted. Both punctured and usual codes were simulated over identical additive white Gaussian noise binary symmetric channels with &,/No= 5.76 dB corresponding to R/R,,,, = 0.94. In either case, no decoding errors were observed. The distributions of the computational efforts appear to be very nearly identical for both the punctured and usual codes. However, an examination of the decoding parameters demonstrate the advantages of using the punctured approach for the decoding of highrate codes. The average number of stack entries, SAV, is equal to 1057 for the usual code, whereas it is only 656 for the punctured code. Counting a computation as the extension of the top node of the stack into all its successors, then clearly the decoding of punctured codes requires on the average more computations than the decoding of usual high
rate codes, but these computations are far simpler. For our example, as indicated in Fig. 9 the average number of computations CAV is equal to 1.32 and 1.12 for the punctured and usual codes, respectively. A fairer comparison of the average computational effort using punctured and usual codes may be based on the overall simulation times, since using both the punctured and usual codes the decoding algorithms are identical with the exception of the branch extensions and metric inhibition of the punctured symbols. For example, for rate2/3 codes, the decoding time of the punctured codes was observed to be approximately 16 percent of that required for the usual codes, whereas for rate7/8 codes this proportion of decoding time drops to 5 percent. Based on extensive simulation results involving a large number of codes having different coding rates over a wide range of Eb/No values, it was observed that these decoding advantages of the punctured codes over the normal codes increase as both the coding rates and Eb/Noincrease [38]. VI. CONCLUSIONS In this paper, we have presented the encoding as well as Viterbi and sequential decoding of highrate punctured convolutional codes. These codes are derived from wellknown optimal lowrate convolutional codes, and depending on the choice of the perforation patterns they may yield easy implementations of variablerate and ratecompatible coding. Using a computer search, we have extended results on previously discovered shortmemorylength punctured codes by providing up to ten terms of their weight spectra. We have shown that sequential decoding can be easily adapted for the decoding of punctured codes and have provided new longmemory punctured codes with 9 5 M I23 and rates 2/3 and 314 that have been derived from the best known rate1/2 codes of the same memory length. The first few terms of these code spectra have been provided together with their bit error probability upper bounds. The substantial advantages of using highrate punctured codes over the usual highrate codes open the way for powerful, yet practical, implementations of variablerate codecs, extending from very low to very high coding rates. APPENDIX In this Appendix, the weight spectra for two of Yasudas et al. punctured codes are derived using the transfer function
1122
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 37, NO. 1 1 , NOVEMBER 1989
over the input, yielding
T(D,B)=Xb/X,=G[ZA]F.
(A.7)
Inverting matrix [I  A] this equation yields the transfer function of the state diagram in closed form. For the purpose of comparing these results with those of the exploration technique, the expansion of T(D, B) as a power series in D is required. This is readily obtained since [I  A ]  may be expanded as an infinite series of power of matrices
[IA]
= I + [ A ]+ [ A 2 ] [ A 3 ] [ A 4 ] * + + +
*.
(A.8)
Substituting in (A.7), we get
T(D,B)=GF+ G [ A ] F + G [ A Z ] F + G [ A 3 ] F + *
(A.9)
loo
2
5
10
2
5
lo2
2
lo3
N Fig. 9. Empirical distribution of the number of computations per decoded branch for equivalent usual and punctured codes having R = 2/3 and M = 24 over a hardquantized channel at EbNo = 5.76 dB.
which is the desired series expansion of T(D, B). One can easily obtain the terms U, and cj of the weight spectrum from T(D, B) as
technique [11, [32]. These results provide a validation for all the weight spectra that were obtained by the search technique. Punctured codes are considered highrate codes in the true sense and hence their performance is evaluated assuming an optimal (i.e., Viterbi) decoder that makes comparisons only on true trellis nodes of the highrate code, that is, nodes whose depths are a multiple of b for a rateb/v code. The transfer function of a convolutional code may be evaluated by solving equations describing the transitions between the states of the finitestate encoder. In Fig. 10 the state diagram of the rate 213, memory2 punctured code of Table I is given. Following the usual procedure, the selfloop at state zero is eliminated by splitting that state into two states X , and x b . The remaining states are arbitrarily labeled X I , X2, and X,. The state transitions are labeled with the indeterminate monomials that describe them: the exponent of B indicating the number of information bits 1 causing the transition and the exponent of D indicating the Hamming weight on the transition. From Fig. 10, the transition behavior within the state diagram may be described by the matrix equation
T ( D ,B ) J B = I =
j = d k
2
UjDj
(A.10)
dTg
)IB=I
=j = d f r e e cjDJ.
9
(A.11)
For the rate213 code of memory M = 2 described by (A. l), the expansion (A.9) yields
T(D, B ) =D 3 B +D4(3B2 B4)
+ +Ds(B+7 B 3+ 5Bs + B7) +D6(B2 14B4+ 17B6+ 7B8+Bl0) + +D7(2B3+ 26B5+ 47B7+ 31 B9+ 9B11+ B13) +D8(Bz+ 6B4+ 48B6+ 1 15B8+ 108BO + 49BIZ+ 1 1B14+ B16)+D9(3B3+ 19B5+93B7+262B9
+ 324B11+205B13+71 B15+ 13B17+B19)+  .*
B2D2 (A.1 )or in matrix notation,
(A.12)From (A.12) the spectral coefficients aiare given by
T ( D , B)(B=l=
2
ajDj
x= [ A ] X + F X ,from which it follows that
(A3 (A.3)
J=d&
= D 3 + 4 D 4 +14D5+40O6
x = [ I  A ] FX,
+ 116D7+ 339D8+ 991D 9 + and
*
(A.13)
where the meaning of the symbols [A], I, and F is obvious from (A.1). W e also have
[xb]=[D2 2 D2] Dor,x =G x . b
[$1
(A.4)
=D 3
+ l o o 4 + 540 + 226D6. e  .
+856D7+3072D8+10647D9+ (A.5)
(A.14)
Comparing to the results of Table I, we see that the spectral Finally, combining (A.3) through (AS), we get the scalar coefficients obtained via the transfer function are identical to equation those obtained by our search algorithms. In order to verify our results for a still higher rate code, the X b = G [ I  A ] FX,. (A.6) same calculation has been applied to the memory2, r a t e 4 5 The transfer function is given by the ratio of the output punctured code of Table III. The state diagram of the code is
HACCOUN AND BEGIN: CODES FOR VITERBI AND SEQUENTIALDECODING
1123
BO
D2
Fig. 10. State diagram of the rate2/3, memory2 punctured code of Table I
Fig. 11. State diagram of the r a t e 4 5 , memory2 punctured code of Table 111. In this diagram, each state transition pertains to all branch transitions between the encoder states
depicted in Fig. 11 and its matrix equation is given by
BD3+ B2D + B3D3+ B2D3 BO3+ B2D + B 3 D 3 B2D3 +
BD3+ B2D BD3+ 2B2D3 + B3D3+ B2D3 + B 3 D BD3+ B2D BD3+ B 2 D + B3D3+ B2D3 + B2D5+R 3 D

B2D4+2B3D2 B4+B3D2 B2D4 2B3D2 + + B4D2 +B3D4+B2D4 +B4D2X=[A]X+FX,
1

j" 3
(A. 15)
[Xb] [2BD2+ B2D4 D 2 D 2+ BD4+ B2D2 BD2 2BD2+ B2D4+ D 2 ] X 2 + ( D 2 B + D 4 ( B + B 2 ) ) X , (A.17) = + +
"'Ix 3
Xb = GX + hX,.
+
(A. 16)
(A.18)
1124 As above, the transfer function is given by T ( D , B ) =  = G [ Z  A ] IF+ h Xl Land hence
IEEE TRANSACTIONSON COMMUNICATIONS,VOL. 37, NO. 11, NOVEMBER 1989 REFERENCES
x b
( A .19)
T ( D , B ) = h + GF+ GAF+ GA2F+CA 3F+

*.
(A.20)
Following the same approach as above, (A.20) for this code yields
T(D, B ) = D 2 B +D3(2B+4B2+ 3B3+ B5+ 2B6)+ D 4 ( B + 2 B 2 + 6 B 3 1 1 B4+ 10B5+2B6 + 4 B 7 + 9B8+5B9+BI1+2BlZ)
+
+ D5(3B2+5B3+ 1 1 B4+ 19Bs+ 30B6 + 33B7+ 18B8+20B9+33B10+29B11 +7BI2+ 6BI3+ 14B14+7BI5+B17 + 2 B 9 +D6(4B2+13 B3+ 23B4+25B5+46B6+ 72B7 + 97B8+ 101 B9+ 88B*O+938 + 122BI2 + 124BI3+ 63BI4+43B + 67BI6+57BI7 + 14B18+8B19+ 19Bm+9BZ1 + BZ3 2B) + D7(B2+ 8B3+ 26B4 ++ 6 8 B 5 + 105B6+ 117B7+ 173B8+274B9
+ 357B1+ 352BI1+ 366BI2+409B13 + 477B14+ 477B15+ 350B16+ 268BL7 + 309BI8+ 305BI9+ 150Bm+77B21+ 113BU+94B23+23B24+ lOBZS*... +24BZ6+ 11B27+B29+2B30)+(A.21)The spectra coefficients uj and cj are therefore given by
T ( D , B ) J B = ~ = UjDJj=d*
= D 2 + 12D3+53D4+238D5
+ 1091D6+4947O7+ m
* *
(A.22)
= D 2 36D3 309D4 2060D5*e.
+ + + 12320D6+ 69343D7+ +
(A.23)
Once again, the spectral coefficients determined by the transfer function approach match perfectly those obtained by the tree searching algorithms, thus validating all the results provided in Tables I through VII.ACKNOWLEDGMENT The authors wish to thank C. Paquin, a graduate student at Ecole Polytechnique de MontrM, for her help in running the computer programs for the search of punctured codes.
A. J. Viterbi, Convolutional codes and their performance in communications systems, IEEE Trans. Commun. Technol., vol. COM19, Oct. 1Wl. R. M. Fano, A Heuristic discussion of probabilistic decoding, IEEE Trans. Inform. Theory, vol. IT9, Apr. 1962. I. M. Jacobs and E. R. Berlekamp, A lower bound to the distribution of computation for sequential decoding, IEEE Trans. Inform. Theory, vol. IT13, pp. 167174, Apr. 1967. I. M. Jacobs, Practical applications of coding, IEEE Trans. Inform. Theory, vol. IT20, pp. 305310, May 1974. E. R. Berlekamp, The technology of errorcorrecting codes, Proc. IEEE, vol. 68, pp. 564593, May 1980. V. K. Bhargava, D. Haccoun, R. Matyas, and P. Nuspl, Digital Communications by Satellite. New York: Wiley, 1981. S. Lin and D. J. Costello, Jr., Error Control Coding. Englewood Cliffs, NJ: PrenticeHall, 1983. W. W.Wu,Elements of Digital Satellite Communications, Volume II. Rockville, MD: Computer Science, 1985. A. Michelson and A. Levesque, ErrorControl Techniques for Digital Communication. New York: Wiley, 1985. W. W. Wu, D. Haccoun, R.Peile, and Y.Hirata, Coding for satellite communication, IEEE J. Select. Areas. Commun., vol. SAC5, pp. 724748, May 1987. A. S. Acampora and R. Gilmore, Analog Viterbi decoding for high speed d g t l satellite channels, NTC 77 Conf. Rec., Los Angeles, iia CA, Dec. 1977, pp. 34.6.134.6.5. R. T. Clark and R. D. McCallister, Development of an LSI maximumlikelihood convolutional decoder for advanced forward error correction capability on the NASA 30120 GHz program, in Proc. AIAA 9th Commun. Satellite Syst. Conf., San Diego, CA, Mar. 1982, pp. 142144. R. M. Orndoff et al., Viterbi decoder VLSI integrated circuit for bit e m r correction, in Proc. Nut. Telecommun. Conf., New Orleans, LA, Dec. 1981, pp. E1.7.1E1.7.4. J. B. Cain, G. C. Clark, and J. Geist, Punctured convolutional codes of rate (n  l)/n and simplified maximum likelihood decoding, IEEE Trans. Inform. Theory, vol. IT25, pp. 97100, Jan. 1979. Y. Yasuda,Y. Hirata, K. Nakamura, and S. Otani, Development of a variablerate Viterbi decoder and its performance characteristics, 6th Int. Conf. Digital Satellite Commun., Phoenix, AZ, Sept. 1983. Y. Yasuda, K. Kashiki, and Y. Hirata, Highrate punctured convoluot tional codes for s f decision Viterbi decoding, IEEE Trans. Commun., vol. COM32, pp. 315319, Mar. 1984. G. Begin and D. Haccoun, Sequential decoding of punctured convolutional codes, in Proc. 13th Biennial Symp. Commun., Kingston, Ont., Canada, June 1986, pp. A.3.5A.3.8. , Decoding of punctured convolutional codes by the stack algorithm, Abstracts of Papers, 1986 IEEE Int. Symp. Inform. Theory, Ann Arbor, MI,Oct. 1986, p. 159. J. Hagenauer, Rate compatible punctured convolutional codes and their applications, IEEE Trans. Commun., vol. 36, pp. 389400, Apr. 1988. D. Haccoun and G. Begin, Codage et dkodage skquentiel de codes convolutionnels perforbs, in Proc. 11th GRETSI Conf., Nice, France, June 1987, pp. 221224. G. Begin and D. Haccoun, High rate punctured convolutional codes: Structure properties and construction techniques, IEEE Trans. Commun., to appear Dec. 1989. D. Haccoun and C. Naiyun, Variants of the stack algorithm for the decoding of highrate codes by sequential decoding, in Proc. Int. Satellite Commun. Conf., Ottawa, Canada, June 1983, pp. 21.4.121.4.4. D. Haccoun, Dkodage dquentiel des codes convolutionels de taux ClevCs, Traitement du Signal, Paris, France, vol. 4, no. 6, pp. 471478, Dk. 1987. re E. Paaske, Short binary convolutional codes with maximal f e distance for rates 213 and 314, IEEE Trans. Inform. Theory, vol. IT20, pp. 683686, Sept. 1974. R. Johannesson and E. Paaske, Further results on binary convolutional codes with an optimum distance profile, IEEE Trans. Inform. Theory, vol. IT24, pp. 264268, Mar. 1978. J. Hagenauer, High rate convolutional codes with good profiles, IEEE Trans. Inform. Theory, vol. IT23, pp. 615618, Sept. 1977. F. Jelinek, A fast sequential decoding algorithm using a stack, IBA4 J. Res. Develop., vol. 13, pp. 675685, Nov. 1969. K. Zigangirov, Some sequential decoding procedures, Problemii Peredachi Informatsii, vol. 2, pp. 1315, 1966.
HACCOUN AND BEGIN: CODES FOR VITERBI AND SEQUENTIAL DECODING
1125and is currently Head of the Communication Department. From June 1984 to May 1985 he was a Research Visiting Professor at Concordia University, Montrkal, P.Q., Canada. His current research interests include theory and applications of errorcontrol coding, VLSI implementation of powerful codecs, mobile communications, and digital communications systems by satellite. Dr. Haccoun is member of the Order of Engineers of Quebec and Sigma Xi. He was Treasurer of the 1983 IEEE International Symposium on Information Theory, St. Jovite, P.Q., Canada, September 1983, the Chairman of the Communications Chapter (19761978) and the founder of the Vehicular Communications Chapter (19841985) in the Montreal section of the IEEE, and a cofounder and President of the Canadian Society of Information Theory (19861987). He has been serving as a consultant to several government agencies and industries. He is the author or coauthor of several papers in these areas and the coauthor of the book Digital Communications by Satellite: Modulation, Multiple Access and Coding (New York: Wiley, 1981). A Japanese translation of the book was published in 1984.
D. Haccoun and M. J. Ferguson, Generalized stack algorithms for decoding convolutional codes, IEEE Trans. Inform. Theory, vol. IT21, pp. 638651, NOV. 1975. D. Haccoun, Variabilite de calcul et dkbordements de decodeurs sequentiels a pile, Traitement du signal, Paris, France, vol. 3, no. 3, pp, 127143, Dec. 1986. R. Johannesson, Some long rate onehalf binary convolutional codes with an optimal distance profile, IEEE Trans. Inform. Theory, vol. IT22, pp. 629631, Sept. 1976. J. P. Odenwalder, Optimal decoding of convolutional codes, Ph.D. dissertation, Dept. Elect. Eng., U.C.L.A., Los Angeles, 1970. P. Montreuil, Algorithmes de determination de spectres des codes convolutionnels, M.Sc.A. thesis, Dep. Elect. Eng., Ecole Polytechnique de Montreal, 1987. D. Haccoun and P. Montreuil, Weight spectrum determination of convolutional codes, to be submitted to IEEE Trans. Commun. , Algorithms for the spectrum determination of convolutional codes, Book of Abstracts, 1988 Int. Symp. Inform. Theory, Kobe, Japan, June 1988, pp. 4950. D. Haccoun and G. Begin, High rate punctured convolutional codes, Rapport Technique EPMIRT88/1, Ecole Polytechnique de Montreal, Jan. 1988, 63 pages. K. Larsen, Short convolutional codes with maximal free distance for rates 1/2, 113, and 1/4, IEEE Trans. Inform. Theory, vol. IT19, pp. 37172, May 1973. G. Begin and D. Haccoun, Simulation results of sequential decoding of high rate punctured convolutional codes, to be submitted to IEEE
Trans. Commun.
.kDavid Haccoun (S62M67S69M74SM84) received the Engineer and B.A.Sc. degrees in engineering physics from h o l e Polytechnique de Montreal, P.Q., Canada, in 1965, the S.M. degree in electrical engineering from the Massachusetts Institute of Technology, Cambridge, in 1966, and the Ph.D. degree in electrical engineering from McGill University, MontrCal, P.Q., Canada, in 1974 Since 1966 he has been with the Department of Electrical Engineering, Ecole Polytechnique de Montreal, where he is Professor of Electrical Engineering since 1980. He was Head of the Department of Communication and Computer from 1980 to 1982, Guy Begin (S86) was born in Jonquieres, P.Q , Canada, in 1960. He received the B.Eng. degree in unified engineering from Universite du Quebec a Chicoutim in 1983. He is currently working towards the Ph.D. degree in electrical engineering at Ecole Polytechnique de Montreal, P.Q., Canada. Since 1985, he has been a recipient of a postgraduate scholarship from Fonds FCAR of Qutbec. His research interests include theory and applications of errorcontrol coding, VLSI implementations of codecs and cryptology.