VIRGINIA POLYTECHNIC INSTITUTEAND STATE UNIVERSITY
TechVirginia
1 8 7 2
VIRGINIA POLYTECHNIC INSTITUTE & STATE UNIVERSITY
MOBILE & PORTABLE RADIO RESEARCH GROUP
MPRG
Iterative Detection and Decoding for Wireless Communications
Prelim Report September 29, 1998
Matthew Valenti Mobile and Portable Radio Research Group Bradley Department of Electrical and Computer EngineeringVirginia TechBlacksburg, Virginia
5/29/98
Outline Part I: Turbo codes
Overview of turbo codes Turbo codes for multimedia channels Turbo codes for unknown fading channels
Part II: Other applications of iterative processing Combined equalization and error correction coding Combined multiuser detection and error correction coding Spatial diversity applications
Proposal for future research
5/29/98
Intr
oduc
tion
Error Correction Coding Channel coding adds structured redundancy to a
transmission.
The input message m is composed of K symbols. The output code word x is composed of N symbols. Since N > K there is redundancy in the output. The code rate is r = K/N.
ChannelEncoder
m x
5/29/98
Intr
oduc
tion
Channel Capacity for the AWGN Channel
Channel Coding Theorem Shannon, 1948 Channel capacity is the
highest data rate that can be supported by a particular channel with arbitrarily low error rate.
Plot of Eb/No vs. r Additive White Gaussian
Noise channel. Eb is the energy per bit. No is the one-sided noise
spectral density. Shows minimum SNR
required for reliable transmission.
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-2
-1
0
1
2
3
4
5
code rate r
Eb/
No
in d
B
Antipodal input pdfGaussian input pdf
5/29/98
Turb
o Co
des
Part I: Turbo Codes Backgound
Turbo codes were proposed by Berrou and Glavieux in the 1993 International Conference in Communications.
Performance within 0.5 dB of the channel capacity limit was demonstrated.
Features of turbo codes Concatenated coding Pseudo-random interleaving Iterative decoding
5/29/98
Turb
o Co
des
Motivation: Performance of Turbo Codes.
Comparison: Rate 1/2 Codes. K=5 turbo code. K=15 convolutional code.
Plot is from: L. Perez, “Turbo Codes”, chapter 8 of Trellis Coding by C. Schlegel. IEEE Press, 1997.
Gain of almost 2 dB!
Theoretical Limit!
5/29/98
Turb
o Co
des
Concatenated Coding A single error correction code does not always provide
enough error protection with reasonable complexity. Solution: Concatenate two (or more) codes
This creates a much more powerful code. Serial Concatenation (Forney, 1966)
OuterEncoder
BlockInterleaver
InnerEncoder
OuterDecoder
De-interleaver
InnerDecoder
Channel
5/29/98
Turb
o Co
des
Parallel Concatenated Codes Instead of concatenating in serial, codes can also be
concatenated in parallel. The original turbo code is a parallel concatenation of two
recursive systematic convolutional (RSC) codes. systematic: one of the outputs is the input.
Encoder#1
Encoder#2In
terle
aver MUX
Input
ParityOutput
Systematic Output
5/29/98
Turb
o Co
des
Pseudo-random Interleaving The coding dilemma:
Shannon showed that large block-length random codes achieve channel capacity.
However, codes must have structure that permits decoding with reasonable complexity.
Codes with structure don’t perform as well as random codes. “Almost all codes are good, except those that we can think of.”
Solution: Make the code appear random, while maintaining enough
structure to permit decoding. This is the purpose of the pseudo-random interleaver. Turbo codes possess random-like properties. However, since the interleaving pattern is known, decoding is
possible.
5/29/98
Turb
o Co
des
Iterative Decoding
There is one decoder for each elementary encoder. Each decoder estimates the a posteriori probability (APP) of
each data bit. The APP’s are used as a priori information by the other
decoder. Decoding continues for a set number of iterations.
Performance generally improves from iteration to iteration, but follows a law of diminishing returns.
Decoder#1
Decoder#2
DeMUX
Interleaver
Interleaver
Deinterleaver
systematic data
paritydata
APP
APP
hard bitdecisions
5/29/98
Turb
o Co
des
The Turbo-Principle Turbo codes get their name because the decoder uses
feedback, like a turbo engine.
5/29/98
Dec
odin
g Al
gori
thm
s
Classes of Decoding Algorithms Requirements:
Accept Soft-Inputs in the form of a priori probabilities or log-likelihood ratios.
Produce APP estimates of the data.
“Soft-Input Soft-Output” MAP
(symbol-by-symbol) Maximum A Posteriori
SOVA Soft Output Viterbi Algorithm
Trellis-Based Estimation Algorithms
ViterbiAlgorithm
SOVA
ImprovedSOVA
MAPAlgorithm
max-log-MAP
log-MAP
Sequence Estimation
Symbol-by-symbol Estimation
5/29/98
Dec
odin
g Al
gori
thm
s
S0
S3
S2
S1
0/00
1/11
0/01
1/10
1/100/01
0/001/11i = 0 i = 6i = 3i = 2i = 1 i = 4 i = 5
)( 1 ii ss )( 1is)( is
The MAP algorithm
Jacobian Logarithm:
)1ln(),max()ln( || xyyx eyxee
5/29/98
Turb
o Co
des
Performance Factors and Tradeoffs
Complexity vs. performance Decoding algorithm. Number of iterations. Encoder constraint length
Latency vs. performance Frame size.
Spectral efficiency vs. performance Overall code rate
Other factors Interleaver design. Puncture pattern. Trellis termination.
5/29/98
Mul
tim
edia
App
licat
ions
Turbo Codes for Multimedia Services
Multimedia systems require varying quality of service. QoS Latency
Low latency for voice, teleconferencing Bit/frame error rate (BER, FER)
Low BER for data transmission. The tradeoffs inherent in turbo codes match with the
tradeoffs required by multimedia systems. Data: use large frame sizes
Low BER, but long latency Voice: use small frame sizes
Short latency, but higher BER
Results for AWGN Channel
r = 1/2
r = 1/3
Constraint Length 3. Improved SOVA. 8 iterations.
Voice
VideoConferencing
ReplayedVideo
Data
5/29/98
Fadi
ng C
hann
els
Turbo Codes for Fading Channels
The turbo decoding algorithm requires accurate estimates of channel parameters: Branch metric:
Average signal-to-noise ratio (SNR). Fading amplitude. Phase.
Because turbo codes operate at low SNR, conventional methods for channel estimation often fail. Therefore channel estimation and tracking is a critical issue
with turbo codes.
pi
pi
si
siiii xzxzmPss ][ln)( 1
*2
* 24iii
o
sii arr
NEaz
5/29/98
Fadi
ng C
hann
els
Fading Channel Model Antipodal modulation:
Gaussian Noise:
Complex Fading:
is a constant. =0 for Rayleigh Fading >0 for Rician Fading
X and Y are Gaussian random processes with autocorrelation:
TurboEncoder
ChannelInterleaver
BPSKModulator
TurboDecoder
De-interleaver
BPSKDemod
}1,1{ ks
ks
kn
ka
s
on E
NP2
kkk jYXa )(
)2()( kTfJkR sdo
0 1 2 3 4 5 6 7 8 9 1010
-6
10-5
10-4
10-3
10-2
10-1
100
101
Eb/No in dB
BE
R
fdTs = .0025, no interleaving fdTs = .005, no interleaving fdTs = .01, no interleaving fdTs = .0025, block interleavingfdTs = .005, block interleaving fdTs = .01, block interleaving fully interleaved
The Role of the Channel Interleaver
Simulation parameters: Rayleigh flat-fading. r=1/2, K=3. 1,024 bit random interleaver. 8 iterations of improved SOVA
decoding. Best performance occurs when
fading is independent from symbol to symbol. “fully-interleaved”:
Performance is poor in correlated fading.
Solution is to use a channel interleaver to reorder the transmission so that the fades are effectively uncorrelated.
)()( kkR
5/29/98
Fadi
ng C
hann
els
Pilot Symbol Assisted Modulation Pilot symbols:
Known values that are periodically inserted into the transmitted code stream. Used to assist the operation of a channel estimator at the receiver. Allow for coherent detection over channels that are unknown and time varying.
segment #1
symbol#1
symbol#Mp
symbol#1
symbol#Mp
pilot symbol
segment #2
symbol#1
symbol#Mp
symbol#1
symbol#Mp
pilot symbol
pilot symbols added here
5/29/98
Fadi
ng C
hann
els
Pilot Symbol AssistedTurbo Decoding
Desired statistic:
Initial estimates are found using pilot symbols only.
Estimates for later iterations also use data decoded with high reliability.
“Decision directed”
TurboEncoder
Insert Pilot
Symbols
ChannelInterleaver
ChannelInterleaver
Insert Pilot
Symbols
Compareto
Threshold
Filter
jd
)(ˆ qjd
ix ixka
kn
)(ˆ qka
ks
kr
)(qi
TurboDecoder
ChannelDeinterleaver
Remove Pilot
Symbols
)(qiy
)(qiy
)(ˆ qks
Delay
)(ˆ qix )(ˆ q
ix
2
2
Re
*
2
2Re kkar
Performance of Pilot Symbol Assisted Decoding
0 1 2 3 4 5 6 7 8 9 1010
-5
10-4
10-3
10-2
10-1
100
101
Eb /No in dB
BE
R
DPSK with differential detection BPSK with estimation prior to decodingBPSK with refined estimation BPSK with perfect channel estimates
Simulation parameters: Rayleigh flat-fading. r=1/2, K=3 1,024 bit random interleaver. 8 iterations of log-MAP. fdTs = .005 Mp = 16
Estimation prior to decoding degrades performance by 2.5 dB.
Estimation during decoding only degrades performance by 1.5 dB.
Noncoherent reception degrades performance by 5 dB.
5/29/98
Part II: Other Applications of Turbo Decoding
The turbo-principle is more general than merely its application to the decoding of turbo codes.
The “Turbo Principle” can be described as: “Never discard information prematurely that may be useful in
making a decision until all decisions related to that information have been completed.”
-Andrew Viterbi “It is a capital mistake to theorize before you have all the
evidence. It biases the judgement.”-Sir Arthur Conan Doyle
Can be used to improve the interface in systems that employ multiple trellis-based algorithms.
5/29/98
Turb
o Pr
inci
ple
Applications of the Turbo Principle
Other applications of the turbo principle include: Decoding serially concatenated codes. Combined equalization and error correction decoding. Combined multiuser detection and error correction
decoding. (Spatial) diversity combining for coded systems in the
presence of MAI or ISI.
5/29/98
Turb
o Pr
inci
ple
Serial Concatenated Codes The turbo decoder can also be used to decode serially
concatenated codes. Typically two convolutional codes.
OuterConvolutional
EncoderData
n(t)AWGN
InnerDecoder
OuterDecoder
EstimatedData
TurboDecoder
interleaver
deinterleaver
interleaver
InnerConvolutional
Encoder
APP
5/29/98
Turb
o Eq
ualiz
atio
n
Turbo Equalization The “inner code” of a serial concatenation could be an
Intersymbol Interference (ISI) channel. ISI channel can be interpreted as a rate 1 code defined
over the field of real numbers.
(Outer)Convolutional
EncoderData
n(t)AWGN
SISOEqualizer
(Outer)SISO
DecoderEstimated
Data
TurboEqualizer
interleaver
deinterleaver
interleaver
ISIChannel
APP
5/29/98
Turb
o M
UD
Turbo Multiuser Detection The “inner code” of a serial concatenation could be a
multiple-access interference (MAI) channel. MAI channel describes the interaction between K
nonorthogonal users sharing the same channel. MAI channel can be thought of as a time varying ISI
channel. MAI channel is a rate 1 code with time-varying coefficients
over the field of real numbers. The input to the MAI channel consists of the encoded and
interleaved sequences of all K users in the system.
5/29/98
Turb
o M
UD
System Diagram
ConvolutionalEncoder
#K
n(t)AWGN
SISOMUD
Bank ofK SISO
DecodersEstimated
Data
TurboMUD
interleaver #K
multiuserdeinterleaver
multiuserinterleaver
MAIChannel
APP
ConvolutionalEncoder
#1interleaver #1
Parallelto
Serial
“multiuser interleaver”
1b
b
y)(qΨ )'(qΨ
)'(qΛ)(qΛ
1d
KbKd
)(ˆ qd
5/29/98
Turb
o M
UD
Received Signal:
Where: ak is the signature waveform of user k. k is a random delay (i.e. asynchronous) of user k. Pk[i] is received power of user k’s ith bit (fading ampltiude).
Matched Filter Output:
MAI Channel Model
K
kk tntstr
1
)( )()(
L
i
jkkkkk
keiTtaibiPts1
)(][][)(
dteiTtatriy kjkkk
)()(][
5/29/98
Turb
o M
UD
Optimal Multiuser Detection Algorithm: Setup
Place y and b into vectors:
Place the fading amplitudes into a vector:
Compute cross-correlation matrix:
][,],[,],1[,],1[
][,],[,],1[,],1[
11
11
LbLbbbLyLyyy
KK
KK
by
KjidttataT
KjidtTtataTG
jjKjiKjijKji
jjjijijji
ij
if ,)()()cos(1
if ,)()()cos(1
][,,][,,]1[,,]1[ 11 LPLPPP Kk c
5/29/98
Turb
o M
UD
Optimal MUD: Execution Run SOVA or MAP algorithm with branch metric:
where
The p(b) term is supplied by the channel decoder. Initially assume equiprobable data.
The algorithm produces reliability estimates of the code symbols. The reliability estimates are fed into the channel decoder.
1
1)(,22)(ln)(
K
jijKdijiiiiiii
obi Gcbcbycbbp
NEn
b
0)mod( if0)mod( ifmod
)(KiKKiKi
i
5/29/98
Turb
o M
UD
Types of Multiple-Access Systems
The nature of the signature sequence depends on the type of multiple-access.
Direct-Sequence Code-Division Multiple-Access DS-CDMA. Each user is assigned a distinct signature waveform. The signature waveform is a sequence of N “chips” per bit. N is called the “processing gain” or “spreading factor”. Cochannel interference is from the same cell.
Time-Division Multiple-Access TDMA Users are not assigned distinct signature waveforms. Cochannel interference is from nearby cells.
0 1 2 3 4 5 6 710
-6
10-5
10-4
10-3
10-2
10-1
100
101
Eb/No in dB
BE
R
Matched Filter Turbo-MUD: iter 1Turbo-MUD: iter 2Turbo-MUD: iter 3Single User Bound
Simulation Results: DS-CDMA in AWGN Channel
DS-CDMA uplink. K=5 mobiles. N=7 spreading factor. Random spreading codes.
Convolutionally coded. Constraint length 3. Code rate 1/2.
Log-MAP algorithm. MUD. Channel decoder.
Iterative processing. LLR from decoder fed back to
MUD’s.
0 2 4 6 8 10 12 14 1610
-6
10-5
10-4
10-3
10-2
10-1
100
101
Eb/No in dB
BE
R
Matched Filter Turbo-MUD: iter 1Turbo-MUD: iter 2Turbo-MUD: iter 3Single User Bound
Simulation Results:Rayleigh Flat-Fading Channel
Fully-interleaved Rayleigh flat-fading. i.e. fades are independent
from symbol to symbol. Same parameters as AWGN
case.
5/29/98
TDM
A Sy
stem
s
Multiuser Detectionfor TDMA systems
Multiuser detection can also be used for TDMA systems. A TDMA systems can be thought of as a DS-CDMA
system with where all cochannel users have the same signature waveform.
With CDMA systems, the interferers are in the same cell as the desired user.
With TDMA systems the interferers come from other cells.
5/29/98
TDM
A Sy
stem
s
Macrodiversity Combining for TDMA Systems
In TDMA systems, the cochannel interference comes from adjacent cells.
Interferers to one cell are desired signals to another cell.
Performance could be improved if the base stations were allowed to share information.
If the outputs of the multiuser detectors are log-likelihood ratios, then adding the outputs improves performance.
BS 1
BS 2
BS 3MS 3
MS 1
MS 2
0 5 10 15 20 25 3010
-4
10-3
10-2
10-1
100
Eb/No in dB
Bit
Error
Rat
e
Conventional Receiver MUD without diversity MUD with equal gain combining
Performance for Constant C/I TDMA cellular system.
Uplink. mobile to base station.
120 sectorized antennas. K=3 users.
mobiles M=3 receivers.
base stations Log-MAP MUD. Uncoded. C/I = 7 dB
C/I ratio of desired signal to interference.
Performance for Constant Eb/No
TDMA uplink. K=3 mobiles. M=3 base stations.
Performance as a function of C/I. Eb/No = 20 dB.
For conventional receiver, performance is worse as C/I gets smaller.
For macrodiversity combining, performance improves as C/I gets smaller.
0 2 4 6 8 10 12 14 16 18 2010
-5
10-4
10-3
10-2
10-1
100
C/I in dB
Bit
Err
or R
ate
Conventional Receiver MUD without diversity MUD with equal gain combining
5/29/98
TDM
A Sy
stem
s
Macrodiversity Combining for Coded TDMA Systems
Each base station has a multiuser detector. Sum the LLR outputs of each MUD. Pass through a bank of channel decoder. Feed back LLR outputs of the decoders.
MultiuserEstimator
#1
MultiuserEstimator
#M
Bank ofK SISOChannelDecoders
1y
My
)(qMΨ
)(1qΨ
) (qz )(qΛ
)(ˆ qd)(qΩ
0 2 4 6 8 10 12 1410
-6
10-5
10-4
10-3
10-2
10-1
100
Eb/No in dB
Bit
Err
or R
ate
(BE
R)
Conventional Reception Macrodiverity using Conventional Receivers joint MUD/FEC with macrodiversity, 1st iterationjoint MUD/FEC with macrodiversity, 2nd iterationjoint MUD/FEC with macrodiversity, 3rd iterationjoint MUD/FEC with macrodiversity, 4th iteration
TDMA uplink. K=3 mobiles. M=3 base stations.
C/I = 7 dB Convolutionally coded.
Constraint length 3. Code rate 1/2.
Log-MAP algorithm. MUD. Channel decoder.
Iterative processing. LLR from decoder fed back
to MUD’s.
Performance for Constant C/I
0 2 4 6 8 10 12 14 16 18 2010
-6
10-5
10-4
10-3
10-2
10-1
100
C/I in dB
Bit
Err
or R
ate
(BE
R)
Conventional Reception Macrodiverity using Conventional Receivers joint MUD/FEC with macrodiversity, 1st iterationjoint MUD/FEC with macrodiversity, 2nd iterationjoint MUD/FEC with macrodiversity, 3rd iterationjoint MUD/FEC with macrodiversity, 4th iteration
TDMA uplink. K=3 mobiles. M=3 base stations. Convolutionally coded.
Performance as a function of C/I, with Eb/No = 20 dB.
For conventional receiver, performance is worse as C/I gets smaller.
For macrodiversity combining, performance improves as C/I gets smaller.
Performance for Constant Eb/No
5/29/98
Conclusion Turbo codes can achieve remarkable power efficiency in
AWGN and flat-fading channels. The tradeoffs inherent to turbo codes make them
attractive for multimedia systems. Because turbo codes operate at very low SNR, channel
estimation and tracking is a critical issue. The principle of iterative or “turbo” processing can be
applied to other problem. Turbo-multiuser detection can improve performance of
both CDMA and TDMA systems. In TDMA systems with multiuser detection, significant
performance gains can be achieved by combining the outputs of the multiuser detectors.
5/29/98
Proposal of Future Work Turbo codes for unknown fading channels.
More extensive study of pilot symbol assisted turbo decoding with refined estimation.
Study the impact of choice of Mp, the pilot symbol spacing. Turbo-multiuser detection for DS-CDMA systems.
Study the performance as a function of K, the number of users in the system.
Macrodiversity combining for coded TDMA. Performance using other convolutional codes (Kc=5,6). Study the impact of varying N, the number of cells per cell.
Smaller N means more spectrally efficient system. Performnace as a function K, number of users. Channel estimation effects.
5/29/98
List of Publications Journal Papers:
“Refined channel estimation for coherent detection of turbo codes over flat-fading channels,” IEE Electronics Letters, Aug. 20, 1998.
Conference Papers: “Combined multiuser detection and channel decoding with base station
diversity,” GLOBECOM Comm. Theory Mini-Conf. Nov. 1998. “Multiuser detection with base station diversity,” ICUPC, Oct. 1998. “Iterative multiuser detection for convolutionally coded asynchronous
DS-CDMA,” PIMRC, Sept. 1998. “Performance of turbo codes in interleaved flat fading channels with
estimated channel state information,” VTC, May 1998. “Combined multiuser reception and channel decoding for TDMA
cellular systems,” VTC, May 1998. “Variable latency Turbo codes for wireless multimedia applications,”
Int. Symp. Turbo Codes & Related Topics , Sept. 1997.
5/29/98
Intr
oduc
tion
Contributions Study of turbo codes for multimedia systems. Study of the performance of turbo codes in correlated
Rayleigh and Rician flat-fading. Developed several channel estimation algorithms
suitable for turbo codes. Developed a method to combine multiuser detection and
error correction coding for multiple-access systems. Developed a method to incorporate macrodiversity
combining for TDMA systems with multiuser detection. Showed significant performance improvement in TDMA
sytems by using the combination of multiuser detection, macrodiversity combining, error correction coding, and iterative-processing.
5/29/98
Appe
ndix
IBlock Codes vs.
Convolutional Codes Advantages of block codes
Large minimum distance High error correction capability, in theory.
Disadvantage of block codes Most block codes are algebraically decoded. 2.5 dB loss from using hard-decisions rather than soft.
Advantages of convolutional codes Trellis structure enable soft-decision decoding. Stream oriented nature allows for continuous decoding. No need for frame synchronization
Disadvantages of convolutional codes Small free distance Performance worse than hard-decision decoded block codes at
high SNR and low BER.
5/29/98
Appe
ndix
I
Comparison of Convolutional and Block Codes
Convolutional codes. Hard decision
decoding. For K=5
dmin = 12 Pb=10-5 at 7.3 dB Pb=10-10 at 10.2 dB
5/29/98
Appe
ndix
I
Comparison of Convolutional and Block Codes
BCH codes. Hard decision
decoding. For (255,131)
dmin = 37 Pb=10-5 at 5.7 dB Pb=10-10 at 7.1dB
5/29/98
Appe
ndix
I
Comparison of Convolutional and Block Codes
Convolutional codes. Soft-decision
decoding. For K=5
dfree = 12 Pb=10-5 at 4.9 dB Pb=10-10 at 7.8 dB
5/29/98
Appe
ndix
II
Performance Bounds for Linear Block Codes
Union bound for soft-decision decoding:
For convolutional and turbo codes this becomes:
The free-distance asymptote is the first term of the sum:
For convolutional codes N is unbounded and:
N
i o
si
ib N
EdQNwP
2
1 2
)(
2
~Nmn
dd o
sddb
freeNEdQ
NwNP
o
sfree
freefreeb N
EdQNwN
P2
~
o
sfreedb N
EdQWP2
0
5/29/98
Appe
ndix
II
Free-distance Asymptotes For convolutional code:
dfree = 18 Wd
o = 137
For turbo code dfree = 6 Nfree = 3 wfree = 2
o
bb N
EQP25.018137
o
bb N
EQP25.06
6553623
