Channel Coding I + IIChannel Coding I + II
Dirk WübbenDepartment of Communications Engineering
Room: N 2380, Phone: 0421/218-2545Email: [email protected]
LectureTuesday, 08:30 – 10:00 in N2420
Wednesday, 14:00 – 16:00 in N2250Dates for exercises will be announced
during lectures.
http://www.ant.uni-bremen.de/teaching/kc
TutorRonald BöhnkeRoom: N2380
Phone [email protected]
Dirk Wübben Channel Coding I 2UniversitätBremen
PreliminariesPreliminaries
� Master students: � Channel Coding I and Channel Coding II are elective courses� Written examination at the end of each semester
� Diplomstudenten� Kanalcodierung als (Wahl)pflichtfach, mündliche Prüfung� Wahlweise dreistündig (Prüfung nach 1 Semester)
oder sechsstündig (Prüfung nach 2 Semestern)
� Documents� Script Kanalcodierung I/II by Volker Kühn (in German), these slides and tasks for
exercises are available in the internet http://www.ant.uni-bremen.de/teaching/kc
� Exercises� Take place on Wednesday, 14:00-16:00 in Room N2250� Dates will be arranged in the lesson and announced by mailling list
[email protected]� Contain theoretical analysis and tasks to be solved in Matlab
Dirk Wübben Channel Coding I 3UniversitätBremen
LiteratureLiterature
� Books on Channel Coding� B. Friedrichs, Kanalcodierung, Springer Verlag, 1996 � M. Bossert, Kanalcodierung, B.G. Teubner Verlag, 1998, � M. Bossert, Channel Coding for Telecommunications, Wiley 1999� R. E. Blahut, Algebraic Codes for Data Transmission, Cambridge University Press, 2003 � S. Lin, D. J. Costello, Jr., Error Control Coding: Fundamentals and Applications, Prentice- Hall, 2004 � W.C. Huffman, V. Pless, Fundamentals of Error-Correcting Codes, Cambridge University Press, 2003 � S. B. Wicker, Error Control Systems for Digital Communications and Storage, Prentice-Hall, 1995 � J. Huber, Trelliscodierung, Springer Verlag, 1992
� Books on Information Theory� T. M. Cover, J. A. Thomas, Information Theory, Wiley, 1991� R. G. Gallager, Information Theory and Reliable Communication, Wiley, 1968 � R. Johannesson, Informationstheorie - Grundlagen der (Tele-)Kommunikation, Addison-Wesley, 1992
� General Books on Digital Communication� K. D. Kammeyer, Nachrichtenübertragung, B.G. Teubner, 2004 � K.D. Kammeyer, V. Kühn, MATLAB in der Nachrichtentechnik, Schlembach, 2001� J. Proakis, Digital Communications, McGraw-Hill, 2001 � B. Sklar, Digital Communications, Fundamentals and Applications, Prentice-Hall, 2003
� Internet Resources
Dirk Wübben Channel Coding I 4UniversitätBremen
Outline Channel Coding IOutline Channel Coding I
� Introduction� Declarations and definitions, general principle of channel coding� Structure of digital communication systems
� One Lesson of Information Theory� Probabilities, measure of information� SHANNON‘s channel capacity for different channels
� Linear block codes� Properties of block codes and general decoding principles� Bounds on error rate performance� Representation of block codes with generator and parity check matrices� Cyclic block codes (Reed-Solomon and BCH codes)
� Convolutional Codes� Structure, algebraic and graphical presentation� Distance properties and error rate performance� Optimal decoding with Viterbi algorithm
Dirk Wübben Channel Coding I 5UniversitätBremen
Outline Channel Coding IIOutline Channel Coding II
� Trelliscoded Modulation (TCM)� Motivation by information theory� TCM of Ungerböck, pragmatic approach by Viterbi, Multilevel codes� Distance properties and error rate performance� Applications (data transmission via modems)
� Concatenated Codes� Serial Concatenation� Parallel Concatenation (Turbo Codes)� Iterative Decoding with Soft-In/Soft-Out decoding algorithms
� Adaptive Error Control� Automatic Repeat Request (ARQ)� Performance for perfect and disturbed feedback channel� Hybrid FEC/ARQ schemes
Dirk Wübben Channel Coding I 6UniversitätBremen
IntroductionIntroduction
� Basics about Channel Coding� General declarations� Basic principles� Applications of Channel Coding� Declarations and definitions, general principle of channel coding
� Structure of digital communication systems
� Discrete Channel � Statistical description� AWGN and fading channel� Discrete Memoryless Channel (DMC)� Binary Symmetric Channel (BSC and BSEC)
Dirk Wübben Channel Coding I 7UniversitätBremen
General DeclarationsGeneral Declarations
� Important terms:� Message Amount of transmitted data or symbols by the source� Information Part of message, which is new for the sink� Redundancy Difference of message and information, which is
unknown to the sinkMessage = Information + Redundancy
� Irrelevance Information, which is not of importance to the sink� Equivocation Information, not stemming from sink of interest
� Message is also transmitted in a distinct amount of time� Messageflow Amount of message per time� Informationflow Amount of information per time� Transinformation Amount of error-free information per time transmitted
from the source to the sink
Dirk Wübben Channel Coding I 8UniversitätBremen
Three Main Areas of CodingThree Main Areas of Coding
� Source coding (entropy coding)� Compression of the information stream so that no significant information is lost,
enabling a perfect reconstruction of the information. � Thus, by eliminating superfluous and uncontrolled redundancy the load on the
transmission system is reduced. � Entropy defines the minimum amount of necessary information
� Channel coding (error-control coding)� Encoder adds redundancy (additional bits) to information bits in order to detect or
even correct transmission errors
� Cryptography� The information is encrypted to make it unreadable to unauthorized persons or to
avoid falsification or deception during transmission.� Decryption is only possible by knowing the encryption key
� Claude E. Shannon (1948): A Mathematical Theory of Communication
Dirk Wübben Channel Coding I 9UniversitätBremen
Basic PrincipleBasic Principless of Channel Codingof Channel Coding
� Forward Error Correction (FEC)� Added redundancy is used to correct transmission errors at the receiver� Channel condition affects the quality of data transmission � errors after
decoding occur if the error-correction capability of the code is passed� No feedback channel is required� varying reliability, constant bit throughput
� Automatic Repeat Request (ARQ)� Small amount of redundancy is added to detect transmission errors
� retransmission of data in case of a detected error� feedback channel is required
� Channel condition affects the throughput� constant reliability, but varying throughput
� Hybrid FEC/ARQ
Dirk Wübben Channel Coding I 10UniversitätBremen
Basic Idea of Channel CodingBasic Idea of Channel Coding
� Add redundancy to the information data in order to detect and/orcorrect transmission errors at the receiver
� Channel Coding � Create a vector x of length n>k for an information vector u of length k� In case of a binary alphabet 2n different vectors x exist.
Due to the bijective mapping from u to x only 2k<2n vectors are used.� Find a k-dimensional subset out of an n-dimensional space so that
• Minimum distance between elements within the subset is maximized� effects probability of detecting errors
• Cardinality of subset is as large as possible� Code rate equals the rate of length of the uncoded and the
coded sequence and describes the required expansion of the signalbandwidth
c
kR
n=
• Code is a subset
• Coder is the mapper
Dirk Wübben Channel Coding I 11UniversitätBremen
Visualizing Distance Properties with Code CubeVisualizing Distance Properties with Code Cube
� Code rate Rc = 1
� No error correction
� No error detection
� Code rate Rc = 2/3
� No error correction
� Detection of single error
� Code rate Rc = 1/3
� Correction of single error
� Detection of 2 errors
010 011
100
110 111
101
001000
dmin=1
010 011
100
110 111
101
001000
dmin=2
010 011
100
110 111
101
001000
dmin=3
Dirk Wübben Channel Coding I 12UniversitätBremen
Applications of Channel CodingApplications of Channel Coding
� Importance of channel coding increased with digital communications� First use for deep space communications:
� AWGN channel, no bandwidth restrictions, only few receivers (costs negligible)
� Examples:Viking (Mars), Voyager (Jupiter, Saturn), Galileo (Jupiter), ...
� Mass storage � compact disc, digital versatile disc, magnetic tapes
� Digital wireless communications: � GSM, UMTS, WLAN (Hiperlan, IEEE 802.11), ...
� Digital wired communications� Modem transmission (V.90, ...)
� Digital broadcasting � DAB, DVB
Dirk Wübben Channel Coding I 13UniversitätBremen
digital source
• Source transmits signal d(t) (e.g. analog speech signal)
• Source coding samples, quantizes and compresses analog signal
• Digital Source: comprises analog source and source coding, delivers
digital data vector u = (u0 u1 … uk-1) of length k
Structure of Digital Transmission SystemStructure of Digital Transmission System
uanalogsourceanalogsource
sourceencodersource
encoder
Dirk Wübben Channel Coding I 14UniversitätBremen
• Channel encoder adds redundancy to u so that errors in
x = (x0 x1 … xn-1) can be detected or even corrected
• Channel encoder may consist of several constituent codes
• Code rate: Rc = k / n
Structure of Digital Transmission SystemStructure of Digital Transmission System
uanalogsourceanalogsource
sourceencodersource
encoder
digital source
xchannelencoderchannelencoder
Dirk Wübben Channel Coding I 15UniversitätBremen
discrete channel
• Modulator maps discrete vector x onto analog waveform and moves it into the transmission band
• Physical channel represents transmission medium– Multipath propagation → intersymbol interference (ISI)– Time varying fading, i.e. deep fades in complex
envelope– Additive noise
• Demodulator: Moves signal back into baseband and performs lowpass filtering, sampling, quantization
Structure of Digital Transmission SystemStructure of Digital Transmission System
• Discrete channel: comprises analog part of modulator, physical channel and analog part of demodulator
uanalogsourceanalogsource
sourceencodersource
encoder
digital source
xchannelencoderchannelencoder
y
modulatormodulator
physicalchannelphysicalchannel
demodulatordemodulator
Dirk Wübben Channel Coding I 16UniversitätBremen
y
discrete channelChannel decoder: • Estimation of u on the basis of received vector y• y need not to consist of hard quantized values (0,1)• Since encoder may consist of several parts,decoder may also consist of several modules
Structure of Digital Transmission SystemStructure of Digital Transmission System
uanalogsourceanalogsource
sourceencodersource
encoder
digital source
xchannelencoderchannelencoder
uchanneldecoderchanneldecoder
modulatormodulator
physicalchannelphysicalchannel
demodulatordemodulator
Dirk Wübben Channel Coding I 17UniversitätBremen
y
discrete channel
Structure of Digital Transmission SystemStructure of Digital Transmission System
feedback channel
uanalogsourceanalogsource
sourceencodersource
encoder
digital source
xchannelencoderchannelencoder
sinksink sourcedecodersource
decoder
uchanneldecoderchanneldecoder
modulatormodulator
physicalchannelphysicalchannel
demodulatordemodulator
Citation of Massey: “The purpose of the modulation system is to create a good discrete channel from the modulator input to the demodulator output, and the purpose of the coding system is to transmit the information bits reliably through this discrete channel at the highest practicable rate.”
Dirk Wübben Channel Coding I 18UniversitätBremen
Discrete ChannelDiscrete Channel
� Discrete channel comprises analog parts of modulator and demodulator as well as physical transmission medium
� Discrete input alphabet Ain={X0, ..., X|Ain|-1}
� Discrete output alphabet Aout={Y0, ..., Y|Aout|-1}
� Probabilities: Pr{Xν}, Pr{Yµ}
� Joint probability of event Xν,Yµ: Pr{Xν,Yµ}
� Transition probabilities: Pr{Yµ | Xν}
discretechannel
xi∈Ain yi∈Aout
{ } { } { }{ } { }
Pr , Pr Pr
Pr Pr
X Y Y X X
X Y Y
ν µ µ ν ν
ν µ µ
=
=
Dirk Wübben Channel Coding I 19UniversitätBremen
Discrete ChannelDiscrete Channel
� Description of discrete channel by transition diagram
� Restrictions in this lecture� Binary input alphabet (BPSK): Ain={-1,+1}
� Output alphabet depends on quantization• No quantization Aout=R
• Hard decision (1-bit) Aout=Ain
Y1X1
X|Ain|-1
Pr{Y0 | X0} Y0X0
Y|Aout|-1
Pr{Y1 | X0}
Dirk Wübben Channel Coding I 20UniversitätBremen
Discrete ChannelDiscrete Channel
� A-posteriori probabilities:
� For statistical independent elements
� General probability relations
{ } { }Pr Pr 1in outX Y
X Yν µ
ν µ∈ ∈
= =∑ ∑A A
{ }Pr , 1in outX Y
X Yν µ
ν µ∈ ∈
=∑ ∑A A
{ } { } { } { }Pr Pr , Pr Pr ,in outX Y
Y X Y X X Yν µ
µ ν µ ν ν µ∈ ∈
= =∑ ∑A A
{ } { } { }Pr , Pr PrX Y X Y= ⋅ν µ ν µ
{ } { }Pr Pr , jj
a a b=∑{ }Pr 1i
i
a =∑
{ }Pr , 1i ji j
a b =∑∑
{ } { }{ }
Pr ,Pr
Pr
X YX Y
Y
ν µν µ
µ
=
{ } { }{ }
{ } { }{ } { }
Pr , Pr PrPr Pr
Pr Pr
X Y X YX Y X
Y Y
⋅= = =ν µ ν µ
ν µ νµ µ
Dirk Wübben Channel Coding I 21UniversitätBremen
BayesBayes RuleRule
� Bayes Rule
� Attention
but
{ } { }{ } { } { }
{ }Pr , Pr
Pr PrPr Pr
a b bb a a b
a a= = ⋅ { } { } { }
{ }Pr
Pr PrPr
YY X X Y
X= ⋅ µ
µ ν ν µν
{ } { }{ }
Pr ,Pr 1
Prout outY Y
X YX Y
Yµ µ
ν µν µ
µ∈ ∈
= ≠∑ ∑A A
{ } { }{ }
{ }{ }
Pr , PrPr 1
Pr Prin inX X
Y X YX Y
Y Yν ν
µ ν µν µ
µ µ∈ ∈
= = =∑ ∑A A
{ }Pr 1outY
Y Xµ
µ ν∈
=∑A
and
�Example
Dirk Wübben Channel Coding I 22UniversitätBremen
BasebandBaseband TransmissionTransmission
� Time-continuous, band-limited signal x(t)∈R
� Average Power per period
� Average Energy of each (real) transmit symbol
� Noise with spectral density (of real noise)� Power
� Signal to Noise Ratio (after matched filtering)
ix
( )n t
iy( )Tg t
( )x t
( ) ( )i si
x t x g t iT= ⋅ −∑
{ }2
s sE E TX µ= ⋅
{ }22 2 /X s s sBE E T E X µσ = = =
( )Rg t
f
0 / 2N
2B
sE 2Xσ,XX NNΦ Φ
channelchannel
2Nσ
( )20 0 2N sN B N Tσ = =
0 / 2NN NΦ =
siT
2
20
/2
sX
N
ES N
N
σσ
= =
1 2A sf T B= =Sampling
Dirk Wübben Channel Coding I 23UniversitätBremen
BandpassBandpass TransmissionTransmission
� Transmit real part of complex signal shifted to a carrier frequency f0
� With x(t)∈C the complex envelope of the data signal
� Re{} results in two spetra around -f0 and f0� Received signal� Transformation to basis band� Analytical signal achieved by Hilbert transformation (suppress f<0)
� Low pass signal
( )y t+
ix
( )n tɶ
iy( )Tg t
( )x tBP
1√2 · e
−jω0t√2 · ejω0t
{ }Re i
( )x t
( ) ( ){ } ( ) ( ) ( ) ( )0BP 0 02 Re 2 ' cos '' sinj tx t ex t x t t x t tω ω ω= = −
channelchannel
{ }j iH
( )Rg t
( ) ( ) ( ){ }BP BP BPy t y t j y t+ = + ⋅H
( ) ( ) 01TP BP2
j ty t y t e ω−+=
BP BP BP BP( ) ( ) ( ) ( )y t x t h t n t= ∗ +
xBP(t)∈R
doubles spectrum for f >0
( )y tBP
Dirk Wübben Channel Coding I 24UniversitätBremen
Equivalent Equivalent BasebandBaseband RepresentationRepresentation
� Equivalent received signal
� In this semester, only real signals are investigated (BPSK)� Signal is only given in the real part� Only the real part of noise is of interest
( ) ( ) ( ) ( ) ( ) ( ) ( )0BP BP BP
1
2j ty t e x t h t n tx t h t n t ω+ −= = ∗ + ∗ +
f
0 2N
B
2rs sE E=
212 Xσ
,BP BP BP BPX X N NΦ Φ
212 Nσ
B
-f0 f0
2
20 0
2 2/
2 2s sX
N
B E ES N
B N N
σσ
= = =
f
0 / 2N
B
sE 2Xσ,XX NNΦ Φ
2Nσ
2
20
/2
sX
N
ES N
N
σσ
= =
Dirk Wübben Channel Coding I 25UniversitätBremen
Probability Density Functions for AWGN ChannelProbability Density Functions for AWGN Channel
-4 -2 0 2 40
0.2
0.4
0.6
0.8
ddd
signal-to-noise-ratio Es/N0 = 2 dB
( )1|| −=ξϑxyp ( )1|| +=ξϑxyp
ϑ
( )ϑyp
-4 -2 0 2 40
0.2
0.4
0.6
0.8
ddd
signal-to-noise-ratio Es/N0 = 6 dB
ϑ
( )1|| −=ξϑxyp ( )1|| +=ξϑxyp
( )ϑyp
2
| 22
( )1( | ) exp
22y x
NN
y Xp y X ν
ν σπσ −= ⋅ − xi
ni
yi
Dirk Wübben Channel Coding I 26UniversitätBremen
Error Probability of AWGN Channel Error Probability of AWGN Channel
� Signal-to-Noise-Ratio S/N
� Error Probability for antipodal modulation
{ } ( ) ( )
{ } ( ) ( )
( )
|
0 0
0 0
|
2
220
Pr error 1 |
Pr error 1 |
1exp
22
e i y x s s n s s
i y x s s n s s
s s
NN
P x p E T d p E T d
x p E T d p E T d
E Td
∞ ∞
−∞ −∞
∞
= = − = = − = +
= = + = = = −
+ = ⋅ −
∫ ∫∫ ∫
∫
ϑ ξ ϑ η η
ϑ ξ ϑ η η
ηη
σπσ
0 02 2s s s
s
E T ES
N N T N= =
Es : symbol energyN0/2: noise density
( ),s s s sE T E T+ −
Dirk Wübben Channel Coding I 27UniversitätBremen
Error Probability of AWGN ChannelError Probability of AWGN Channel
� With the error probability becomes
� Using the substitution
or
( )2
000
1exp
s s
ess
E TP d
N TN T
ηη
π
∞ + = ⋅ − ∫
( ) 0s s sE T N Tξ η= +
20 2N sN N Tσ= =
2
00
1 1e erfc
2s
se
E N
EP d
N
∞−
= ⋅ = ∫ ξ ξπ
with error function complement
( ) 0 2s s sE T N Tξ η= +2
0
2
02
21e Q
2s
se
E N
EP d
N
ξ
ξπ
∞−
= ⋅ = ∫ ( )2
21Q
2e d
η
α
α ηπ
∞−
= ∫
( ) ( ) η
α
α α ηπ
∞−= − = ∫ 22
erfc 1 erf e d
with
Dirk Wübben Channel Coding I 28UniversitätBremen
Probability Density Functions for Frequency Probability Density Functions for Frequency Nonselective Rayleigh Fading ChannelNonselective Rayleigh Fading Channel
xi
ni
ri yi
αi
( ) ( )2 2 22 exp for 0
0 else
s spα
ξ σ ξ σ ξξ ⋅ ≥=
ir-4 -2 0 2 40
0.02
0.04
0.06
0.08
0.1
iα0 2 4
0
0.02
0.04
0.06
0.08
0.1
iy-4 -2 0 2 40
0.02
0.04
0.06
0.08
0.1
Dirk Wübben Channel Coding I 29UniversitätBremen
Bit Error Rates for AWGN and Flat Rayleigh Bit Error Rates for AWGN and Flat Rayleigh Channel (BPSK modulation)Channel (BPSK modulation)
0 10 20 3010
-6
10-4
10-2
100
17 dB
AWGN Rayleigh
Pb
� AWGN channel:
� Flat Rayleigh fading channel:
� Channel coding for fading channels essential(time diversity)
( )( )
0
0
1erfc /
2
2 /
b s
s
P E N
Q E N
= ⋅
=
0
0
/11
2 1 /s
bs
E NP
E N
= ⋅ − +
Es / N0 in dB
Dirk Wübben Channel Coding I 30UniversitätBremen
� If line-of-sight connection exist for α, its real part is non-central Gaussian distributed� Rice factor K determines power ratio between line-of-sight path and scattered paths� K = 0 : Rayleigh fading channel� K → ∞ : AWGN channel
(no fading)
� Coefficient α[k] has Rayleigh distributed magnitude with average power 1� Relation between total average power and variance:
� Magnitude is Rician distributed
Rice Fading ChannelRice Fading Channel
2 2/02 2
| |
22 0
( )
0
K Ke I
pα−ξ σ −
α α α
ξ ⋅ ⋅ ξ ξ ≥ ξ = σ σ
for
else
1
1 1'[ ]K
i K Ka k
+ += + ⋅α
1
K
K+
'α
in
1
1 K+
iyix
2(1 ) HP K= + ⋅ σ
Dirk Wübben Channel Coding I 31UniversitätBremen
Discrete Memoryless Channel (DMC)Discrete Memoryless Channel (DMC)
� Transition probabilities of a discrete memoryless channel (DMC)
� The probability, that exactly m errors occur at distinct positions in a sequence of n is given by
� Probability, that in a sequence of length n exactly m errors occur (at any place)
With giving the number of possibilities to choose melements out of n different elements, withoutregarding the succession (combinations)
{ } { } { }1
0 1 1 0 1 10
Pr Pr , , , , , , Prn
n n i ii
y y y x x x y x−
− −=
= … … = ∏y x
{ } ( )Pr bit of incorrect 1n mm
e em n P P−= ⋅ −
{ } ( )Pr error in a sequence of length 1n mm
e e
nm n P P
m−
= ⋅ ⋅ −
( )!
! !
n n
m m n m
= ⋅ −
Dirk Wübben Channel Coding I 32UniversitätBremen
Binary Symmetric Channel (BSC and BSEC)Binary Symmetric Channel (BSC and BSEC)
� Binary Symmetric Channel (BSC)
� Binary Symmetric Erasure Channel (BSEC)
{ } { }0 1 1 00
1Pr Pr erfc
2s
e
EY X Y X P
N
= = =
{ } { }
{ } ( )0 0 1 1 1 1
1
0
Pr Pr , , ,
Pr 1
n n
nn
i i ei
x y x y x y
x y P
− −
−
=
= = = = … =
= = −∏
x y
1-Pe
1-Pe
Pe
Pe
Y0
Y1
X0
X1
1-Pe-Pq
1-Pe-Pq
Pe
Pe
Y0
Y1
X0
X1
Y2
Pq
Pq
{ }1
Pr ?e q
q
e
P P y x
x y P y
P y x
− − == = ≠
{ } { } ( )Pr 1 Pr 1 1n
e eP n P≠ = − = = − − ≈x y x y