COMM 604
Channel Coding
Lecture 3
Linear Block Codes
Dr. Engy Aly Maher
Spring 2020
Last Lecture
What are Linear Block Codes?
Linear Systematic Block Codes
Generator Matrix and Encoding Process
Last Lecture
Parity Check Matrix
Syndrome & Error Detection Process
consequently, we obtain the following relation between the
syndrome and the error pattern :
Syndrome and Error Detection
If the parity-check matrix H is expressed in the systematic
form multiplying out e • HT yield the following linear relationship
between the syndrome digits and the error digits :
Syndrome and Error Detection
• The syndrome digits are linear combinations of the error digits
• Because the n – k linear equations do not have a unique solution
but have 2k solutions
• i.e. there are 2k error pattern that result in the same syndrome,
and the true error pattern e is one of them
• To minimize the probability of a decoding error, the most probable
error pattern that satisfies the equations is chosen as the true error
vector
Syndrome and Error Detection
Example
Example
Example
Hamming Weight and Hamming Distance
Hamming Distance d(v,w):
The number of places where v and w differ
Hamming Weight w(v):
The number of nonzero components of v
Hamming Weight and Hamming Distance
The Minimum Distance of a Block Code
The Hamming distance between two code vectors in C is equal to the
Hamming weight of a third code vector in C
minimum weight of the linear code C
The Minimum Distance of a Block Code
Let C be an (n,k) linear block code with parity check matrix
H. For each code word of Hamming weight l, there exists l
columns of H such that the vector sum of these l columns is
equal to the zero vector.
The Minimum Distance of a Block Code
The Minimum Distance of a Block Code
Conversely, if there exists l columns of H whose vector sum is
the zero vector, there exists a codeword of Hamming weight l
in C.
Corollaries
Given C is a linear block code with parity check matrix H. If
no d-1 or fewer columns of H add to 0, the code has
minimum weight of at least d.
Given C is a linear block code with parity check matrix H.
The minimum distance of C is equal to the smallest number
of columns of H that sum to 0.
Example
(7,4) Linear Block Code
1110100
0111010
1101001
=H
No all 0 column. No two columns are identical:
dmin>2
Columns 0, 1, 3 sum to zero vector
dmin=3
Error Detection
Error Detection Capability means:
HOW MANY ERRORS ARE GUARANTEED TO BE DETECTED AT THE RECEIVER SIDE
Channelv
e
r
v: transmitted codeworde: error pattern caused by channelr: received pattern
If error pattern includes l errors: d(v,r)=l
TWO Possibilities:
You could be absolutely surethat r is not a valid codeword.
Number of errors (no. of 1s in e) is smaller than dmin
Number of errors (no. of 1s in e) is greater than dmin
THE ERROR IS GUARANTEED TO BE DETECTED
There is a chance that r is a valid codeword (if e is a valid code word)
THE ERROR PATTERN MIGHT NOT BE DETECTED
Error Detection
The “Error Detection Capability” of a code defines the
number of errors that are GUARANTEED to be
detected.
For a code with minimum Hamming distance dmin:
The error detection capability is dmin-1
Number of Detectable Error Patterns
An error pattern is defined as any nonzero n-tuple
that could affect a transmitted code word.
For an (n,k) code:
There are 2n-1 error patterns. Note that (0 0 0 . . . 0) is not an
error pattern
There are 2k-1 undetectable error patterns. Note that
(0 0 0 . . . 0) is selected to be a valid codeword
An (n,k) code has the ability to detect 2n-2k error patterns
Example: A (7,4) code is able to detect:
128-16=112 error patterns
Weight Distribution of a Block Code
For an (n.k) block code:
Let Ai be the number of codewords of weight i in C. The numbers A0, A1, …, An are called the
Weight Distribution
111111115
010111014
000110113
101110012
100101111
001101010
01110019
11010008
00101117
10001106
11001015
01101004
01000113
11100102
10100011
00000000
Parity BitsInformationCode Word
111111115
010111014
000110113
101110012
100101111
001101010
01110019
11010008
00101117
10001106
11001015
01101004
01000113
11100102
10100011
00000000
Parity BitsInformationCode Word
Example:
For the (7,4) code shown,
A0 = A7 = 1,
A1 = A2 = A5 = A6= 0
A3 = A4 = 7
Note that:
∑Ai=16=24 (Number of valid
code words)
Probability of Detecting an Error Pattern
For an (n,k) linear code:
Given that the bit error probability of the physical channel is p.
The probability that an error pattern of weight j occurs is
pj(1-p)n-j
In Total, there are nCj error patterns that have j erroneous bits.
ONLY Aj of those are NOT DETECTABLE because they
represent valid codewords
Probability of not detecting an error pattern (Pu(E)) is:
n
n jj
u jj 1
P E A p 1 p
Example
Error Correction
The “error correction capability” defines the number
of errors that are GUARANTEED to be corrected.
For a code with minimum Hamming distance dmin:
The error correction capability is (dmin-1)/2
x means floor(x)
Error Correction (Proof)
Assume w is also a valid codeword in CTriangle Inequality:
i.e., if an error pattern of t or
fewer errors occur, the
received vector r MUST BE
closer to v than to any other
codeword in C
Channelv
e
r
v: transmitted codeworde: error pattern caused by channelr: received pattern
Assume a Block Code with dmin
Define a Positive Integer t such that:
min2t 1 d 2t 2
d d d v,r w,r v,w
Assume an error pattern with t’ errors d v,r t'
Given that v and w are codewords d mindv, w
d d
dmind 2 t 1
2 t 1 t'
v,r w,r
w,r
If t’≤t d mind 1t, t
2w,r
Error Correction & Detection
For a code with minimum Hamming distance dmin:
If the code could correct λ and detect up to l then
dmin≥ λ+l+1
Error Correction & Detection
Example:If it is required for a code to correct 3 errors AND detect up to 6 errors then dmin must satisfy: dmin≥10
Notes: In the example above,
• If the number of errors are 3 or less: you can provide a GUARNTEEto correct all of them.
• If the number of errors are from 4 to 6: you can provide a GUARNTEE to detect that the number of errors are between 4 to 6 without being able to correct ANY of them.
• If the number of errors are greater than 6: there is NO GUARANTEE that you would be able to detect or correct the errors
Quantization and Analogy to Error Correction and
Detection Operation
v0=0 v1=1 v2=2 v3=3 v*0=0 v*
1=1 v*2=2 v*
3=3
Transmitter Side Receiver Side
Correct Reception:
The value received is identical to what has been transmitted
Channel
Quantization and Analogy to Error Correction and
Detection Operation
Error Detection:
The value received IS NOT EQUAL to any of the valid representation levels
ARQ: Request retransmission
v0=0 v1=1 v2=2 v3=3 v*0=0 v*
1=1 v*2=2 v*
3=3
Transmitter Side Receiver Side
Channel
Quantization and Analogy to Error Correction and
Detection Operation
Error Detection:
The value received IS NOT EQUAL to any of the valid representation levels
ARQ: Request retransmission
v0=0 v1=1 v2=2 v3=3 v*0=0 v*
1=1 v*2=2 v*
3=3
Transmitter Side Receiver Side
Channel
Quantization and Analogy to Error Correction and
Detection Operation
Un-Detected Errors:
The value received IS a valid representation level. However,
it is NOT what has been transmitted
v0=0 v1=1 v2=2 v3=3 v*0=0 v*
1=1 v*2=2 v*
3=3
Transmitter Side Receiver Side
Channel
Quantization and Analogy to Error Correction and
Detection Operation
Error Correction:
Approximate the received value to the closest valid representation level.
FEC: The receiver defines decision zones and maps ANY received value to a valid representation level. ARQ cannot be applied
v0=0 v1=1 v2=2 v3=3 v*0=0 v*
1=1 v*2=2 v*
3=3
Transmitter Side Receiver Side
Channel
D0 D1 D2 D3
Quantization and Analogy to Error Correction and
Detection Operation
Error Correction:
Approximate the received value to the closest valid representation level.
FEC: The receiver defines decision zones and maps ANY received value to a valid representation level. ARQ cannot be applied
v0=0 v1=1 v2=2 v3=3 v*0=0 v*
1=1 v*2=2 v*
3=3
Transmitter Side Receiver Side
Channel
D0 D1 D2 D3
Quantization and Analogy to Error Correction and
Detection Operation
False Correction:
The error pushes the received value outside the decision zone of the representation level that has been transmitted.
v0=0 v1=1 v2=2 v3=3 v*0=0 v*
1=1 v*2=2 v*
3=3
Transmitter Side Receiver Side
Channel
D0 D1 D2 D3
Quantization and Analogy to Error Correction and
Detection Operation
False Correction:
The error pushes the received value outside the decision zone of the representation level that has been transmitted.
v0=0 v1=1 v2=2 v3=3 v*0=0 v*
1=1 v*2=2 v*
3=3
Transmitter Side Receiver Side
Channel
D0 D1 D2 D3
Quantization and Analogy to Error Correction and
Detection Operation
Error Correction and Error Detection:
Defines two types of decision zone such that both FEC and ARQ could be used dependent on the received value
v0=0 v1=1 v2=2 v3=3
Transmitter Side Receiver Side
Channel
D0 D1 D2 D3
v*0=0 v*
1=1 v*2=2 v*
3=3
Quantization and Analogy to Error Correction and
Detection Operation
v0=0 v1=1 v2=2 v3=3
Transmitter Side Receiver Side
Channel
D0 D1 D2 D3
v*0=0 v*
1=1 v*2=2 v*
3=3
Error Correction and Error Detection:
Defines two types of decision zone such that both FEC and ARQ could be used dependent on the received value
Error Correction
Quantization and Analogy to Error Correction and
Detection Operation
v0=0 v1=1 v2=2 v3=3
Transmitter Side Receiver Side
Channel
D0 D1 D2 D3
v*0=0 v*
1=1 v*2=2 v*
3=3
Error Correction and Error Detection:
Defines two types of decision zone such that both FEC and ARQ could be used dependent on the received value
Quantization and Analogy to Error Correction and
Detection Operation
v0=0 v1=1 v2=2 v3=3
Transmitter Side Receiver Side
Channel
D0 D1 D2 D3
v*0=0 v*
1=1 v*2=2 v*
3=3
Error Correction and Error Detection:
Defines two types of decision zone such that both FEC and ARQ could be used dependent on the received value
False Error Correction
Quantization and Analogy to Error Correction and
Detection Operation
v0=0 v1=1 v2=2 v3=3
Transmitter Side Receiver Side
Channel
D0 D1 D2 D3
v*0=0 v*
1=1 v*2=2 v*
3=3
Error Correction and Error Detection:
Defines two types of decision zone such that both FEC and ARQ could be used dependent on the received value
Error Detection and ARQ could be used