Date post: | 14-Apr-2018 |
Category: |
Documents |
Upload: | rajesh-nandi |
View: | 223 times |
Download: | 0 times |
of 16
7/30/2019 04 Hamming Codes and Performance
1/16
EE4-07 Introduction to Coding Theory
Dr. Wei DaiImperial College London (IC)
Autumn 2011
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 1
http://find/7/30/2019 04 Hamming Codes and Performance
2/16
Section 4 Hamming Codes and Performance
The Hamming code: definition and properties
The dual of the Hamming codeSeveral bounds in coding theory
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 2
http://find/7/30/2019 04 Hamming Codes and Performance
3/16
Binary Hamming Codes
The H [7, 4, 3] Binary Hamming code is a linear code with
H= 0 0 0 1 1 1 10 1 1 0 0 1 1
1 0 1 0 1 0 1
1 2 3 4 5 6 7
(contains all the nonzero columns of length 3)
Let r 2. The parity-check matrix of the binary Hamming codeH [2r 1, 2r 1 r, 3] consists of all the nonzero vectors (columns) oflength r. (Also denoted by Hr.)
d = 3 because every 2 columns are linearly independent andthere are 3 columns that are linearly dependent.
Correct exactly 1 error.
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 3
http://find/7/30/2019 04 Hamming Codes and Performance
4/16
Decoding a Binary Hamming Code
Syndrome decoding:
Arrange columns of H in the alphabetically increasing order1 Compute the syndrome s = yHT.
2 The syndrome s gives the location of error.
3 Flip the bit located at s. x1 = y1, ,xs = ys + 1, ,xn = yn.
Easy to decode.
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 4
http://find/7/30/2019 04 Hamming Codes and Performance
5/16
Coding Bounds
Given n and d, what is the maximum r or k or M?
Given n and k, what is the maximum d?
Sphere packing (Hamming) boundSphere covering (Gilbert-Varshamov) bound
Improved Gilbert-Varshamov bound for linear codes
Plotkin bound
Singleton bound
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 5
http://find/http://goback/7/30/2019 04 Hamming Codes and Performance
6/16
Hamming Bound and Perfect Codes
Spheres in Fnq : B (x, t) = y Fnq : d (x,y) t
.
Volume: Vol (B (x, t)) =t
i=0ni
(q 1)i
.Remark: Volume does not depend on the center. Denote it by V (t)
Hamming bound (sphere-packing bound): For a code of length n anddistance d, the number of codewords satisfies
M qn/V
d12
= qn/
d12 i=0
ni
(q 1)i
.
Proof: Let t =d12
. Let C = {c1, , cM}. Clearly, B (ci, t)s are
disjoint, 1 i M. Hence, M V (B (e)) qn.
Perfect codes: the codes that attain the Hamming bound.
Hamming code is a perfect code.
Perfect codes are rare (Hamming codes & Golay codes).
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 6
http://find/http://goback/7/30/2019 04 Hamming Codes and Performance
7/16
Gilbert-Varshamov Bound
Gilbert-Varshamov (sphere covering) bound: For given code length nand distance d, there exists a code with number of codewords
qn/V (d 1) M.Proof: Its proved by construction. Let M0 = q
n/V (d 1) > 1.Take an arbitrary c1 F
nq . Since F
nq B (c1, d 1) = , take arbitrary
c2 Fn
q B (c1, d 1). Clearly, d (c1, c2) d.Inductively, suppose that we have obtained codewords c1, , cM01in this way.
Since VolM01
i=1 B (ci, d 1)
(M0 1) V (d 1) < qn,
Fn
q
M01i=1
B (ci, d 1) = . Take arbitrary
cM0 Fnq
M01i=1 B (ci, d 1) = . Clearly, d (cM0 , ci) d as
cM0 / B (ci, d 1) for all 1 i M0 1.
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 7
http://find/7/30/2019 04 Hamming Codes and Performance
8/16
Illustration for Sphere Packing and Covering
Sphere Packing Sphere Covering
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 8
http://find/7/30/2019 04 Hamming Codes and Performance
9/16
Improved Gilbert-Varshamov Bound for Linear Codes
Improved G-V bound for linear codes: For given n,d,k such that
d2i=0
n1i
(q 1)i
< qnk
, there exists an [n, k] linear code over Fqwith minimum distance at least d.Proof: We shall show that an (n k) n matrix H such that everyd 1 columns of H are linearly independent. We construct H asfollows.
Let h1be any nonzero vector in Fnkq . Let h2 be any vector not inspan (c1). Let h3 be any vector not in span ([c1, c2]). .Inductively, suppose that we have constructed H of size
(n k) (n 1) such that every d 1 columns of H are linearlyindependent. Note that the number of vectors in the linear span of
d 2 or fewer columns of H is given by
d2i=0
n1i
(q 1)i. Sinced2
i=0
n1i
(q 1)i < qnk, h Fnkq such that every d 1 columns
of the resulting H= [H,h] are linearly independent.
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 9
http://find/7/30/2019 04 Hamming Codes and Performance
10/16
Plotkin Bound
Plotkin bound: For a code of length n. Suppose that rn < d where
r = 1 q1. Then the number of codewords M
ddrn
.
(M
2d2dn
when q = 2.)
Proof: 1) Let T =
cC
cC
d (c, c). Since d d (c, c) for c = c, wehave M(M 1) d T.
2) Let A be the M n matrix whose rows are the codewords. Let ni,adenote the number of entries in the ith column of A that are equal to a.Then
aF ni,a = M for all i. We have T =
ni=1 (
c
c d (ci, c
i)) =
ni=1
aF ni,a (M ni,a) = M
2n
ni=1
aF n
2i,a.
3) By Cauchy-Schwarz inequality,
aF ni,a2
q
aF n2i,a. Hence,
T M2n n
i=1 q1
aF ni,a2
= rM2n. Therefore,M2 (d rn) M d.
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 10
http://find/7/30/2019 04 Hamming Codes and Performance
11/16
Cauchy-Schwarz Inequality and Proof
Cauchy-Schwarz Inequality: For real numbers x1, , xn andy1, , yn,
(
mi=1 xiyi)
2 (
mi=1 xi)
2 (
mi=1 yi)
2.
Proof: It hold if either x or y is 0.
Assume that both x and y are nonzero. Define z = rx + y.Then 0 z22 = r
2 x22 + 2r x,y + y22 for all r R.
This implies b2 4ac = 4 x,y2 4 x22 y22 0, i.e.,
x,y2 x22 y22.
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 11
http://find/7/30/2019 04 Hamming Codes and Performance
12/16
Dual of Binary Hamming Codes
(Hr) = Sr
2r 1, r, (n + 1) /2 = 2r1
is called simplex code.
A very low rate code with very large distance.
Proof of d (Sr) = 2r1: by induction.
Simplex codes attain Plotkin bound: M = 2r; 2d2dn =2r
2r(2r1) = 2r.
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 12
http://goforward/http://find/http://goback/7/30/2019 04 Hamming Codes and Performance
13/16
Singleton Bound and MDS
Theorem (Singleton Bound):
This distance of any code C Fnq with |C| = M satisfies
M qnd+1.In particular, if the code is linear and M = qk, then
d n k + 1.
Proof: For a code with length n and distance d, take arbitrary ci = cj .Let ci,1:nd+1 be the first n d + 1 entries of ci and ci,nd+2:n be thelast d 1 entries of ci. Thend dH (ci, cj) = dH (ci,1:nd+1, cj,1:nd+1) + dH (ci,nd+2:n, cj,nd+2:n) .But dH (ci,nd+2:n, cj,nd+2:n) d 1. Hence,
dH (ci,1:nd+1, cj,1:nd+1) 1, i.e., the first n d + 1 coordinates of allcodewords are distinct. Hence, M qnd+1.
Codes meet singleton bound are maximum distance separable (MDS).
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 13
http://find/7/30/2019 04 Hamming Codes and Performance
14/16
Properties of MDS codes
Properties of MDS codes:Let C[n,k,d] be a linear code over Fq. Let H
and G be the corresponding parity-check and generator matrices.Then the following statements are equivalent
1 C is an MDS code.
2 Every set of n k columns of H is linear independent.
3
Every set of k of G is linear independent.4 C is an MDS code.
Proof: It is clear that 12 and 34 as G is the parity-check matrix ofC. We prove that 14. Once we proved that, it is clear that 41.
To show C is MDS, it suffices to show that the minimum distance d isk + 1.
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 14
http://find/7/30/2019 04 Hamming Codes and Performance
15/16
Proof of Properties of MDS codes
Suppose that d k. Then a nonzero codeword c C with at most
k nonzero entries and at least n k zeros. Since permuting thecoordinates reserves the codeword weights (i.e., the distance), w.l.o.g.,
assume that the last n k coordinates of c are 0.
Write H= [A, H] where A F(nk)kq and H F
(nk)(nk)q . From
assumption 1 (and hence 2), the columns of H are linear
independent. Hence, H is invertible. The only way to obtain 0 in all
the last n k coordinates of c = sH is s = 0. It contradicts with theassumption c = 0. Hence, d k + 1. Together with the Singleton
bound, it follows that d
= k + 1. Hence, 14.
Hamming codes are not MDS in general.
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 15
S
http://find/7/30/2019 04 Hamming Codes and Performance
16/16
Summary
Examples of linear codes
Hamming codes Simplex codes
Coding Bounds
Sphere packing and covering bounds Plotkin bound Singleton bound and MDS
Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 16
http://goforward/http://find/http://goback/