Outline• Transmitters (Chapters 3 and 4, Source Coding and
Modulation) (week 1 and 2)• Receivers (Chapter 5) (week 3 and 4) • Received Signal Synchronization
(Chapter 6) (week 5)• Channel Capacity (Chapter 7) (week 6)• Error Correction Codes (Chapter 8) (week 7 and 8)• Equalization (Bandwidth Constrained Channels) (Chapter
10) (week 9)• Adaptive Equalization (Chapter 11) (week 10 and 11)• Spread Spectrum (Chapter 13) (week 12)• Fading and multi path (Chapter 14) (week 12)
Channel Capacity (Chapter 7) (week 6)
• Discrete Memoryless Channels• Random Codes• Block Codes• Trellis Codes
Channel Models
• Discrete Memoryless Channel– Discrete-discrete
• Binary channel, M-ary channel– Discrete-continuous
• M-ary channel with soft-decision (analog)– Continuous-continuous
• Modulated waveform channels (QAM)
Discrete Memoryless Channel
• Discrete-discrete– Binary channel, M-ary channel
11
11
........)|(.....)|(
jiji
p
pxyPxXyYP
P
Probability transition matrix
Discrete Memoryless Channel
• Discrete-continuous– M-ary channel with soft-decision (analog)
output x0
x1
x2
.
.
.xq-1
y
)|(
)|( 1
kxXyp
xXyp
P
22 2/)(
21)|(
kxy
k exyp
AWGN
Discrete Memoryless Channel
• Continuous-continuous– Modulated waveform channels (QAM)– Assume Band limited waveforms, bandwidth = W
• Sampling at Nyquist = 2W sample/s– Then over interval of N = 2WT samples use an
orthogonal function expansion:
)(
)(
1
tfx
txN
iii
)(
)(
1
tfn
tnN
iii
N
iii tfy
ty
1
)(
)(
Discrete Memoryless Channel
• Continuous-continuous– Using orthogonal function expansion:
)(
)(
1
tfx
txN
iii
)(
)(
1
tfn
tnN
iii
N
iiii
N
ii
T
i
N
ii
T
i
N
iii
tfnx
tfdttftntx
tfdttfty
tfy
ty
1
10
*
10
*
1
)(
)()()()(
)()()(
)(
)(
Discrete Memoryless Channel
• Continuous-continuous– Using orthogonal function expansion get an
equivalent discrete time channel:
Nx
x
.
.
.
.1
Ny
yy
.
.
.2
1
22 2/)(
21)|( iii xy
iii exyp
111 nxy Gaussian noise
Capacity of binary symmetric channel
• BSC
pppp
11
P
}1,0{ }1,0{ YX
0 0
11
X Yp1
p1
p p
Capacity of binary symmetric channel
• Average Mutual Information 0 0
11
X Yp1
p1
p p
)1()1()0(1log)1)(1(
)1()0()1(log)1(
)1()1()0(log)0(
)1()0()1(1log)1)(0(
)1()1|1(log)1|1()1(
)0()1|0(log)1|0()1(
)1()0|1(log)0|1()0(
)0()0|0(log)0|0()0();(
XPpXpPppXP
XpPXPpppXP
XPpXpPppXP
XpPXPpppXP
YPXYPXYPXP
YPXYPXYPXP
YPXYPXYPXP
YPXYPXYPXPYXI
Capacity of binary symmetric channel
• Channel Capacity is Maximum Information– earlier showed:
0 0
11
X Yp1
p1
p p
pppp
XPpXpPppXP
XpPXPpppXP
XPpXpPppXP
XpPXPpppXPYXI
XPXP
2log)1(2log)1(
)1()1()0(1log)1)(1(
)1()0()1(log)1(
)1()1()0(log)0(
)1()0()1(1log)1)(0());(max(C
21)0()1(
21)0()1());(max( XPXPYXI
Capacity of binary symmetric channel• Channel Capacity
– When p=1 bits are inverted but information is perfect if invert them back!
0 0
11
X Yp1
p1
p p
pppp 2log)1(2log)1(C 22
Capacity of binary symmetric channel• Effect of SNR on Capacity
– Binary PAM signal (digital signal amplitude 2A)
0 0
11
X Yp1
p1
p p
)(tg
)(tg)(tg )(tg
)(tgA
A
02
02
/)(
02
/)(
01
1)|(
1)|(
Nr
Nr
b
b
eN
srp
eN
srp
AGWN
Capacity of binary symmetric channel• Effect of SNR
– Binary PAM signal (digital signal amplitude 2A)
0 0
11
X Yp1
p1
p p
)(tg
)(tg)(tg )(tg
)(tgA
A
)|(2
1)|(
20
0 /)(
01
02
sePN
Q
dreN
seP
b
Nr b
0
1s
Capacity of binary symmetric channel• Effect of SNR
– Binary PAM signal (digital signal amplitude 2A)
0 0
11
X Yp1
p1
p p
0
0
221
121
212
22
2
)|()|(
NAQ
QSNRQ
NQ
sePsePP
bb
b
b
Capacity of binary symmetric channel• Effect of SNR
– Binary PAM signal (digital signal amplitude 2A)
0 0
11
X Yp1
p1
p p
noise rms22
422
212
2 noise rms
21
21
0
0
0
AQAQ
NAQ
NAQP
N
b
Not sure about thisDoes it depend on bandwidth?
Capacity of binary symmetric channel• Effect of SNR
– Binary PAM signal (digital signal amplitude 2A)
0 0
11
X Yp1
p1
p p
noise rms 2Amplitudeerfc
21
noise rmsAmplitude
21
21QpPb
pppp 2log)1(2log)1(C
Capacity of binary symmetric channel• Effect of SNR
– Binary PAM signal (digital signal amplitude 2A)
0 0
11
X Yp1
p1
p p
SNR Pb1 0.3085382 0.1586553 0.0668074 0.022755 0.006216 0.001357 0.0002338 3.17E-059 3.4E-06
10 2.87E-0711 1.9E-0812 9.87E-1013 4.02E-1114 1.28E-1215 3.19E-1416 6.11E-16
Pb (BER) vs SNR for binary channel
1.40E+01, 1.28E-12
1.20E+01, 9.87E-10
1E-16
1E-15
1E-14
1E-13
1E-12
1E-11
1E-10
1E-09
1E-08
1E-07
1E-06
1E-05
0.0001
0.001
0.01
0.1
1
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
SNR = A/rms noise
BER
noise rms 2Amplitudeerfc
21
21
bP
Capacity of binary symmetric channel• Effect of SNR
– Binary PAM signal (digital signal amplitude 2A)
0 0
11
X Yp1
p1
p p
pppp 2log)1(2log)1(C 22 SNR Pb C
0 0.5 01 0.308538 0.1085222 0.158655 0.3689173 0.066807 0.6461064 0.02275 0.8433855 0.00621 0.9455446 0.00135 0.9851857 0.000233 0.9968578 3.17E-05 0.9994819 3.4E-06 0.999933
10 2.87E-07 0.99999311 1.9E-08 0.99999912 9.87E-10 113 4.02E-11 114 1.28E-12 115 3.19E-14 116 6.11E-16 1
Capacity © and BER vs SNR for binary channel
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
SNR = A/rms noise
BER
noise rms 2Amplitudeerfc
21
21
bP
At capacity SNR = 7, so waste lots of SNR to get low BER!!!
Capacity of binary symmetric channel• Effect of SNRb
– Binary PAM signal (digital signal amplitude 2A)
0 0
11
X Yp1
p1
p p
pppp 2log)1(2log)1(C 22 SNRb (dB)Pb C
-20 0.443769 0.009143-18 0.429346 0.014452-16 0.411325 0.022809-14 0.388906 0.03591-12 0.361207 0.056319-10 0.32736 0.087793-8 0.286715 0.135561-6 0.239229 0.206245-4 0.186114 0.306729-2 0.130645 0.4407970 0.07865 0.6025972 0.037506 0.7692614 0.012501 0.903056 0.002388 0.9757578 0.000191 0.997366
10 3.87E-06 0.99992512 9.01E-09 114 6.81E-13 1
Capacity C and BER vs SNR for binary channel
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
-20 -12 -4 4 12
SNR per bit (dB)
Cap
acity
C a
nd B
ER
b
bb QpP
erfc21
2
Channel Capacity of Discrete Memoryless Channel
• Discrete-discrete– Binary channel, M-ary channel
11
11
........)|(.....)|(
jiji
p
pxyPxXyYP
P
Probability transition matrix
Channel Capacity of Discrete Memoryless Channel
Average Mutual Information
1
0
1
1
1
0
1
1
)()|(
log)|()(
)()|(
log)|()();(
q
j
Q
i j
jijij
q
j
Q
i j
jijij
yPxyP
xyPxP
yYPxXyYP
xXyYPxXPYXI
Channel Capacity of Discrete Memoryless Channel
Channel Capacity is Maximum InformationOccurs for only if Otherwise must work out max
1)( ,0)(
1
0
1
1)(
1
0
1
1)()(
1
0
)()|(
log)|()(max
)()|(
log)|()(max);(max
q
jjj
j
jj
xPxP
q
j
Q
i j
jijijxP
q
j
Q
i j
jijijxPxP
yPxyP
xyPxP
yYPxXyYP
xXyYPxXPYXIC
jpxP j allfor ,)( symmetric P
)( jxP
Channel Capacity Discrete Memoryless Channel
• Discrete-continuous• Channel Capacity
x0
x1
x2
.
.
.xq-1
y
)|(
)|( 1
kxXyp
xXyp
P
1
0
1
0)()(
)|()()(
where)(
)|(log)|()(max);(max
q
iii
q
i
iiixPxP
xXyYpxXPyYp
dyyYp
xXyYpxXyYpxXPYXICii
Channel Capacity Discrete Memoryless Channel
• Discrete-continuous• Channel Capacity with AWGN
x0
x1
x2
.
.
.xq-1
y
22 2/)(
21)|(
kxy
k exyp
1
01
0
2/)(
2/)(
2/)(
)( 22
22
22
21)(
21
log21)(max
q
iq
i
xyi
xy
xyixP
dyexXP
eexXPC
i
i
i
i
Channel Capacity Discrete Memoryless Channel
• Binary Symmetric PAM-continuous• Maximum Information when:
x0
x1
x2
.
.
.xq-1
y
21)()( AXPAXP
dyeee
e
dyeee
eC
yAA
A
yAA
A
22
2222
22
22
2222
22
2/2/2/
2/2
2/2/2/
2/2
21
2log
2log21
Channel Capacity Discrete Memoryless Channel
• Binary Symmetric PAM-continuous• Maximum Information when:
dyeee
e
dyeee
eC
yAA
A
yAA
A
22
2222
22
22
2222
22
2/2/2/
2/2
2/2/2/
2/2
21
2log
2log21
Channel Capacity Discrete Memoryless Channel
• Binary Symmetric PAM-continuous• Versus Binary Symmetric discrete SNRb (dB)Pb C
-20 0.443769 0.009143-18 0.429346 0.014452-16 0.411325 0.022809-14 0.388906 0.03591-12 0.361207 0.056319-10 0.32736 0.087793-8 0.286715 0.135561-6 0.239229 0.206245-4 0.186114 0.306729-2 0.130645 0.4407970 0.07865 0.6025972 0.037506 0.7692614 0.012501 0.903056 0.002388 0.9757578 0.000191 0.997366
10 3.87E-06 0.99992512 9.01E-09 114 6.81E-13 1
Capacity C and BER vs SNR for binary channel
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
-20 -12 -4 4 12
SNR per bit (dB)
Cap
acity
C a
nd B
ER
Discrete Memoryless Channel
• Continuous-continuous– Modulated waveform channels (QAM)– Assume Band limited waveforms, bandwidth = W
• Sampling at Nyquist = 2W sample/s– Then over interval of N = 2WT samples use an
orthogonal function expansion:
)(
)(
1
tfx
txN
iii
)(
)(
1
tfn
tnN
iii
N
iii tfy
ty
1
)(
)(
Discrete Memoryless Channel
• Continuous-continuous– Using orthogonal function expansion get an
equivalent discrete time channel:
Nx
x
.
.
.
.1
Ny
yy
.
.
.2
1
22 2/)(
21)|( iii xy
iii exyp
111 nxy Gaussian noise
Discrete Memoryless Channel
• Continuous-continuous• Capacity is (Shannon)
)(
)(
1
tfx
txM
iii
)(
)(
1
tfn
tnM
iii
M
iii tfy
ty
1
)(
)(
);(1maxlim)(
YXIT
CxpT
ii
N
i i
iijii
NNN
NNNNNNN
dxydyp
xypxpxyp
ddp
pppI
WTN
NN
1 )()|(log)()|(
)()|(log)()|();(
2
XY
yxyxyxxyYX
22 2/)(
21)|( iii xy
iii exyp
Discrete Memoryless Channel
• Continuous-continuous• Maximum Information when:
0
2
0
2
1 0
2
21
)(
21log
21log21
21log);(max
NWT
NN
NI
x
x
N
i
xNNxp
YX
22 2/
21)( xix
xi exp
Statistically independent
zero mean Gaussian inputs
then
Discrete Memoryless Channel
• Continuous-continuous• Constrain average power in x(t):
22
1
2
0
2
2
)(21
)]([1
xx
N
ii
T
av
WT
N
xE
dttxET
P
Discrete Memoryless Channel
• Continuous-continuous• Thus Capacity is:
)(
)(
1
tfx
txM
iii
)(
)(
1
tfn
tnM
iii
M
iii tfy
ty
1
)(
)(
0
0
2
)(
1log
21loglim
);(1maxlim
WNPW
NW
IT
C
av
x
T
NNxpT
YX
22 2/)(
21)|( iii xy
iii exyp
Discrete Memoryless Channel
• Continuous-continuous• Thus Normalized Capacity is:
)(
)(
1
tfx
txM
iii
)(
)(
1
tfn
tnM
iii
M
iii tfy
ty
1
)(
)(
WCN
WNC
CPWNP
WC
WCb
b
bavav
/12
1log
but ,1log
/
0
02
02
22 2/)(
21)|( iii xy
iii exyp
etab/No (dB)C/W-1.44036 0.1-1.36402 0.15-1.24869 0.225-1.07386 0.3375-0.8075 0.50625
-0.39875 0.7593750.234937 1.1390631.230848 1.7085942.822545 2.5628915.41099 3.844336
9.669259 5.76650416.65749 8.64975627.92605 12.9746345.69444 19.4619573.22669 29.19293115.4055 43.78939179.5542 65.68408 0.1
1
10
-10 0 10 20 30