A VC-D IMENSION-BASED OUTER BOUND ON
THE ZERO-ERRORCAPACITY OF THE BINARY
ADDER CHANNEL
Or OrdentlichJoint work with Ofer Shayevitz
ISIT 2015, Hong Kong
June 19, 2015
THE BINARY ADDER CHANNEL
X1 ∈ {0, 1}
X2 ∈ {0, 1}
+ X1 +X2 ∈ {0, 1, 2}
1 / 16
THE BINARY ADDER CHANNEL
M1 ∈[
2nR1]
:
M2 ∈[
2nR2]
:
X1 ∈ {0, 1}
X2 ∈ {0, 1}
+ X1 +X2 ∈ {0, 1, 2}
1 / 16
THE BINARY ADDER CHANNEL
M1 ∈[
2nR1]
:
M2 ∈[
2nR2]
:
X1 ∈ {0, 1}
X2 ∈ {0, 1}
+ X1 +X2 ∈ {0, 1, 2}
CodebooksC1, C2 ⊆ {0, 1}n with |C1| = 2nR1 and|C2| = 2nR2
1 / 16
THE BINARY ADDER CHANNEL
M1 ∈[
2nR1]
:
M2 ∈[
2nR2]
:
X1 ∈ {0, 1}
X2 ∈ {0, 1}
+ X1 +X2 ∈ {0, 1, 2}
CodebooksC1, C2 ⊆ {0, 1}n with |C1| = 2nR1 and|C2| = 2nR2
Error ⇔ c1 + c2 = c′1 + c
′2 for c1, c′1 ∈ C1, c2, c
′2 ∈ C2
1 / 16
THE BINARY ADDER CHANNEL
M1 ∈[
2nR1]
:
M2 ∈[
2nR2]
:
X1 ∈ {0, 1}
X2 ∈ {0, 1}
+ X1 +X2 ∈ {0, 1, 2}
CodebooksC1, C2 ⊆ {0, 1}n with |C1| = 2nR1 and|C2| = 2nR2
Error ⇔ c1 + c2 = c′1 + c
′2 for c1, c′1 ∈ C1, c2, c
′2 ∈ C2
Shannon Capacity Region (vanishing error)
Simple MAC channel... the region is
R1 ≤ 1
R2 ≤ 1
R1 +R2 ≤ 1.5
1 / 16
THE BINARY ADDER CHANNEL
M1 ∈[
2nR1]
:
M2 ∈[
2nR2]
:
X1 ∈ {0, 1}
X2 ∈ {0, 1}
+ X1 +X2 ∈ {0, 1, 2}
CodebooksC1, C2 ⊆ {0, 1}n with |C1| = 2nR1 and|C2| = 2nR2
Error ⇔ c1 + c2 = c′1 + c
′2 for c1, c′1 ∈ C1, c2, c
′2 ∈ C2
Zero-Error Capacity Region
No errors allowed (no collisions)
All sums are different
1 / 16
THE BINARY ADDER CHANNEL
M1 ∈[
2nR1]
:
M2 ∈[
2nR2]
:
X1 ∈ {0, 1}
X2 ∈ {0, 1}
+ X1 +X2 ∈ {0, 1, 2}
CodebooksC1, C2 ⊆ {0, 1}n with |C1| = 2nR1 and|C2| = 2nR2
Error ⇔ c1 + c2 = c′1 + c
′2 for c1, c′1 ∈ C1, c2, c
′2 ∈ C2
Zero-Error Capacity Region
No errors allowed (no collisions)
All sums are different
Capacity region unknown
Large gap between best inner and outer bounds
1 / 16
THE BINARY ADDER CHANNEL
M1 ∈[
2nR1]
:
M2 ∈[
2nR2]
:
X1 ∈ {0, 1}
X2 ∈ {0, 1}
+ X1 +X2 ∈ {0, 1, 2}
CodebooksC1, C2 ⊆ {0, 1}n with |C1| = 2nR1 and|C2| = 2nR2
Error ⇔ c1 + c2 = c′1 + c
′2 for c1, c′1 ∈ C1, c2, c
′2 ∈ C2
Zero-Error Capacity Region
No errors allowed (no collisions)
All sums are different
Capacity region unknown
Large gap between best inner and outer bounds
In this talk: Outer bound improved
1 / 16
KNOWN BOUNDS
R2
0 0.2 0.4 0.6 0.8 1
R1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Best inner bound
[Lindström ’69], [Kasami & Lin ’76], [Weldon ’78], [Peterson & Costello ’79], [Khachatrian
’82], [van Tilborg ’83], [Kasami et al ’83], [van der Braak & van Tilborg ’85], [Guo &
Watanabe ’91, ’92], [Blake ’94], [Ahlswede & Balakirsky ’99], [Mattas & Ostergard ’05]
2 / 16
KNOWN BOUNDS
R2
0 0.2 0.4 0.6 0.8 1
R1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Best inner boundShannon capacity region
2 / 16
KNOWN BOUNDS
R2
0 0.2 0.4 0.6 0.8 1
R1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Best inner boundShannon capacity regionUrbanke and Li
[Urbanke & Li 1998]
ForR1 = 1: 0.25 ≤ R2 < 0.49216 [Kasami et al ’83], [Urbanke & Li ’98]
2 / 16
KNOWN BOUNDS
R2
0 0.2 0.4 0.6 0.8 1
R1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Best inner boundShannon capacity regionUrbanke and Li UBNew UB
ForR1 = 1: 0.25 ≤ R2 <✭✭✭✭0.49216 0.4798 (this talk)
2 / 16
KNOWN BOUNDS
R2
0.48 0.485 0.49 0.495 0.5 0.505 0.51 0.515 0.52
R1
0.98
0.985
0.99
0.995
1
1.005
Best inner boundShannon capacity regionUrbanke and Li UBNew UB
ForR1 = 1: 0.25 ≤ R2 <✭✭✭✭0.49216 0.4798 (this talk)
2 / 16
BACKGROUND & M OTIVATION
Projection and Shattering
Let c ∈ {0, 1}n
Let S ⊆ [n]def= {1, . . . , n} be some subset of coordinates
c(S) ∈ {0, 1}|S| is theprojectionof c ontoS
3 / 16
BACKGROUND & M OTIVATION
Projection and Shattering
Let c ∈ {0, 1}n
Let S ⊆ [n]def= {1, . . . , n} be some subset of coordinates
c(S) ∈ {0, 1}|S| is theprojectionof c ontoS
c = 011101, S = {2, 3, 5, 6}
3 / 16
BACKGROUND & M OTIVATION
Projection and Shattering
Let c ∈ {0, 1}n
Let S ⊆ [n]def= {1, . . . , n} be some subset of coordinates
c(S) ∈ {0, 1}|S| is theprojectionof c ontoS
c = 011101, S = {2, 3, 5, 6} ⇒ c(S) = 1101
3 / 16
BACKGROUND & M OTIVATION
Projection and Shattering
Let c ∈ {0, 1}n
Let S ⊆ [n]def= {1, . . . , n} be some subset of coordinates
c(S) ∈ {0, 1}|S| is theprojectionof c ontoS
The projection of a codebookC ⊆ {0, 1}n ontoS is themultiset
C(S)def= {c(S) ∈ {0, 1}|S| : c ∈ C} with multiplicities
3 / 16
BACKGROUND & M OTIVATION
Projection and Shattering
Let c ∈ {0, 1}n
Let S ⊆ [n]def= {1, . . . , n} be some subset of coordinates
c(S) ∈ {0, 1}|S| is theprojectionof c ontoS
The projection of a codebookC ⊆ {0, 1}n ontoS is themultiset
C(S)def= {c(S) ∈ {0, 1}|S| : c ∈ C} with multiplicities
S is shattered byC if C(S) contains all2|S| possible vectors
3 / 16
BACKGROUND & M OTIVATION
Projection and Shattering
Let c ∈ {0, 1}n
Let S ⊆ [n]def= {1, . . . , n} be some subset of coordinates
c(S) ∈ {0, 1}|S| is theprojectionof c ontoS
The projection of a codebookC ⊆ {0, 1}n ontoS is themultiset
C(S)def= {c(S) ∈ {0, 1}|S| : c ∈ C} with multiplicities
S is shattered byC if C(S) contains all2|S| possible vectors
C = {101100, 011101, 100110, 000111}
3 / 16
BACKGROUND & M OTIVATION
Projection and Shattering
Let c ∈ {0, 1}n
Let S ⊆ [n]def= {1, . . . , n} be some subset of coordinates
c(S) ∈ {0, 1}|S| is theprojectionof c ontoS
The projection of a codebookC ⊆ {0, 1}n ontoS is themultiset
C(S)def= {c(S) ∈ {0, 1}|S| : c ∈ C} with multiplicities
S is shattered byC if C(S) contains all2|S| possible vectors
C = {101100, 011101, 100110, 000111} ⇒ S = {5, 6} shattered
3 / 16
BACKGROUND & M OTIVATION
Projection and Shattering
Let c ∈ {0, 1}n
Let S ⊆ [n]def= {1, . . . , n} be some subset of coordinates
c(S) ∈ {0, 1}|S| is theprojectionof c ontoS
The projection of a codebookC ⊆ {0, 1}n ontoS is themultiset
C(S)def= {c(S) ∈ {0, 1}|S| : c ∈ C} with multiplicities
S is shattered byC if C(S) contains all2|S| possible vectors
C = {101100, 011101, 100110, 000111}
3 / 16
BACKGROUND & M OTIVATION
Projection and Shattering
Let c ∈ {0, 1}n
Let S ⊆ [n]def= {1, . . . , n} be some subset of coordinates
c(S) ∈ {0, 1}|S| is theprojectionof c ontoS
The projection of a codebookC ⊆ {0, 1}n ontoS is themultiset
C(S)def= {c(S) ∈ {0, 1}|S| : c ∈ C} with multiplicities
S is shattered byC if C(S) contains all2|S| possible vectors
C = {101100, 011101, 100110, 000111} ⇒ S = {1, 2} not shattered
3 / 16
BACKGROUND & M OTIVATION
Theorem[Weldon 78]
Let (C1, C2) be a pair of zero-error codebooks for the BAC with cardi-nalities(2nR1 , 2nR2). If C1 shatters a set of coordinatesS with cardi-nality |S| = nα, then
R2 ≤ (1− α) log 3
4 / 16
BACKGROUND & M OTIVATION
Proof:
Choose anyc2 ∈ C2
By shatterdness, there existsc1 ∈ C1 such thatc1(S) = c2(S), i.e.,
(c1 + c2)(S) = (1 . . . , 1)
|C2| = 2nR2 =⇒ 2nR2 suchS-complement pairsexist (at least)
4 / 16
BACKGROUND & M OTIVATION
Proof:
Choose anyc2 ∈ C2
By shatterdness, there existsc1 ∈ C1 such thatc1(S) = c2(S), i.e.,
(c1 + c2)(S) = (1 . . . , 1)
|C2| = 2nR2 =⇒ 2nR2 suchS-complement pairsexist (at least)
Their sums should be all distinct on̄Sdef= [n] \ S, hence
2nR2 ≤ 3|S̄|
4 / 16
BACKGROUND & M OTIVATION
Proof:
Choose anyc2 ∈ C2
By shatterdness, there existsc1 ∈ C1 such thatc1(S) = c2(S), i.e.,
(c1 + c2)(S) = (1 . . . , 1)
|C2| = 2nR2 =⇒ 2nR2 suchS-complement pairsexist (at least)
Their sums should be all distinct on̄Sdef= [n] \ S, hence
2nR2 ≤ 3n(1−α)
4 / 16
BACKGROUND & M OTIVATION
Proof:
Choose anyc2 ∈ C2
By shatterdness, there existsc1 ∈ C1 such thatc1(S) = c2(S), i.e.,
(c1 + c2)(S) = (1 . . . , 1)
|C2| = 2nR2 =⇒ 2nR2 suchS-complement pairsexist (at least)
Their sums should be all distinct on̄Sdef= [n] \ S, hence
2nR2 ≤ 3n(1−α) ⇒ R2 ≤ (1− α) log 3
4 / 16
BACKGROUND & M OTIVATION
Proof:
Choose anyc2 ∈ C2
By shatterdness, there existsc1 ∈ C1 such thatc1(S) = c2(S), i.e.,
(c1 + c2)(S) = (1 . . . , 1)
|C2| = 2nR2 =⇒ 2nR2 suchS-complement pairsexist (at least)
Their sums should be all distinct on̄Sdef= [n] \ S, hence
2nR2 ≤ 3n(1−α) ⇒ R2 ≤ (1− α) log 3
Special case: IfC1 is systematic(e.g., linear) it shatters a setS ofcardinalitynR1. Thus in this caseR2 < (1−R1) log 3
4 / 16
BACKGROUND & M OTIVATION
Proof:
Choose anyc2 ∈ C2
By shatterdness, there existsc1 ∈ C1 such thatc1(S) = c2(S), i.e.,
(c1 + c2)(S) = (1 . . . , 1)
|C2| = 2nR2 =⇒ 2nR2 suchS-complement pairsexist (at least)
Their sums should be all distinct on̄Sdef= [n] \ S, hence
2nR2 ≤ 3n(1−α) ⇒ R2 ≤ (1− α) log 3
Special case: IfC1 is systematic(e.g., linear) it shatters a setS ofcardinalitynR1. Thus in this caseR2 < (1−R1) log 3
Weldon stopped here, but what can be said in general?
4 / 16
BACKGROUND & M OTIVATION
VC Dimension[Vapnik-Chervonenkis 1971]
TheVC dimensionof a codebookC ⊆ {0, 1}n is the cardinality of thelargest set shattered byC
5 / 16
BACKGROUND & M OTIVATION
VC Dimension[Vapnik-Chervonenkis 1971]
TheVC dimensionof a codebookC ⊆ {0, 1}n is the cardinality of thelargest set shattered byC
Lemma[Sauer-Perles-Shelah 1972]
If the VC dimension ofC is d, then
|C| ≤d
∑
k=0
(
n
k
)
5 / 16
BACKGROUND & M OTIVATION
VC Dimension[Vapnik-Chervonenkis 1971]
TheVC dimensionof a codebookC ⊆ {0, 1}n is the cardinality of thelargest set shattered byC
Lemma[Sauer-Perles-Shelah 1972]
If the VC dimension ofC is d, then
|C| ≤d
∑
k=0
(
n
k
)
Remark: The Lemma is tight for a Hamming Ball of radiusd
5 / 16
BACKGROUND & M OTIVATION
VC Dimension[Vapnik-Chervonenkis 1971]
TheVC dimensionof a codebookC ⊆ {0, 1}n is the cardinality of thelargest set shattered byC
Lemma[Sauer-Perles-Shelah 1972]
If the VC dimension ofC is d, then
|C| ≤d
∑
k=0
(
n
k
)
≈ 2nh(dn)
Remark: The Lemma is tight for a Hamming Ball of radiusd
h(p) = −p log p− (1− p) log (1− p)
5 / 16
BACKGROUND & M OTIVATION
VC Dimension[Vapnik-Chervonenkis 1971]
TheVC dimensionof a codebookC ⊆ {0, 1}n is the cardinality of thelargest set shattered byC
Lemma[Sauer-Perles-Shelah 1972]
If the VC dimension ofC is d, then
|C| ≤d
∑
k=0
(
n
k
)
≈ 2nh(dn)
Remark: The Lemma is tight for a Hamming Ball of radiusd
h(p) = −p log p− (1− p) log (1− p)
Corollary
If |C| = 2n(R+ε) then there is a shattered setS with |S| ≥ nh−1(R).
5 / 16
BACKGROUND & M OTIVATION
Plugging this into Weldon’s theorem gives
6 / 16
BACKGROUND & M OTIVATION
Plugging this into Weldon’s theorem gives
Corollary
If (R1, R2) is achievable then
R2 ≤ (1− h−1(R1)) log 3
6 / 16
BACKGROUND & M OTIVATION
Plugging this into Weldon’s theorem gives
Corollary
If (R1, R2) is achievable then
R2 ≤ (1− h−1(R1)) log 3
Unfortunately, for anyR1 ∈ [0, 1]
R1 + (1− h−1(R1)) log 3 > 1.5
6 / 16
BACKGROUND & M OTIVATION
Plugging this into Weldon’s theorem gives
Corollary
If (R1, R2) is achievable then
R2 ≤ (1− h−1(R1)) log 3
Unfortunately, for anyR1 ∈ [0, 1]
R1 + (1− h−1(R1)) log 3 > 1.5
Weaknesses:We assumed only oneS-complement for eachc2 ∈ C2Weak lower bound on# of S-complement pairs (2nR2)We disregarded the sumset structure outsideSWeak upper bound on# of S-complement pairs (3|S̄|)
6 / 16
HIGHER ORDER VC DIMENSION
k-shattering
S ⊆ [n] isk-shattered byC ⊆ {0, 1}n, if C(S) contains all2|S| possiblevectors, each with multiplicity at leastk
7 / 16
HIGHER ORDER VC DIMENSION
k-shattering
S ⊆ [n] isk-shattered byC ⊆ {0, 1}n, if C(S) contains all2|S| possiblevectors, each with multiplicity at leastk
C = {101100, 011101, 100110, 000111}
7 / 16
HIGHER ORDER VC DIMENSION
k-shattering
S ⊆ [n] isk-shattered byC ⊆ {0, 1}n, if C(S) contains all2|S| possiblevectors, each with multiplicity at leastk
C = {101100, 011101, 100110, 000111} , S = {6} 2-shattered
7 / 16
HIGHER ORDER VC DIMENSION
k-shattering
S ⊆ [n] isk-shattered byC ⊆ {0, 1}n, if C(S) contains all2|S| possiblevectors, each with multiplicity at leastk
C = {101100, 011101, 100110, 000111}
7 / 16
HIGHER ORDER VC DIMENSION
k-shattering
S ⊆ [n] isk-shattered byC ⊆ {0, 1}n, if C(S) contains all2|S| possiblevectors, each with multiplicity at leastk
C = {101100, 011101, 100110, 000111} , S = {2} 1-shattered
7 / 16
HIGHER ORDER VC DIMENSION
k-shattering
S ⊆ [n] isk-shattered byC ⊆ {0, 1}n, if C(S) contains all2|S| possiblevectors, each with multiplicity at leastk
C = {101100, 011101, 100110, 000111} , S = {2} 1-shattered
kth-order VC dimension
Thekth-orderVC dimensionof a codebookC ⊆ {0, 1}n is the cardi-nality of the largest subsetk-shattered byC
7 / 16
HIGHER ORDER VC DIMENSION
Lemma (“soft” Sauer-Perles-Shelah lemma)
If the kth-order VC dimension ofC ⊆ {0, 1}n is d− 1, then
|C| ≤t∗∑
t=1
(
n
t
)
+
(
n
t∗
) n∑
t=t∗+1
(
t∗
d
)
(
td
)
wheret∗ is the smallest integert satisfying(
n−dt−d
)
≥ k if such an inte-ger exists, andt∗ = n otherwise.
8 / 16
HIGHER ORDER VC DIMENSION
Lemma (“soft” Sauer-Perles-Shelah lemma)
If the kth-order VC dimension ofC ⊆ {0, 1}n is d− 1, then
|C| ≤t∗∑
t=1
(
n
t
)
+
(
n
t∗
) n∑
t=t∗+1
(
t∗
d
)
(
td
)
wheret∗ is the smallest integert satisfying(
n−dt−d
)
≥ k if such an inte-ger exists, andt∗ = n otherwise.
O(n/d)-tight for a Hamming Ball of radiust∗
8 / 16
HIGHER ORDER VC DIMENSION
Lemma (“soft” Sauer-Perles-Shelah lemma)
If the kth-order VC dimension ofC ⊆ {0, 1}n is d− 1, then
|C| ≤t∗∑
t=1
(
n
t
)
+
(
n
t∗
) n∑
t=t∗+1
(
t∗
d
)
(
td
)
wheret∗ is the smallest integert satisfying(
n−dt−d
)
≥ k if such an inte-ger exists, andt∗ = n otherwise.
O(n/d)-tight for a Hamming Ball of radiust∗
Corollary
If |C| = 2n(R+ε) then for any0 ≤ α ≤ h−1(R) there existsS with|S| ≥ nα that is2nβ-shattered byC, where
β = (1− α) · h
(
h−1(R)− α
1− α
)
8 / 16
NUMBER OFS-COMPLEMENT PAIRS
Assume(C1, C2) are zero-error
9 / 16
NUMBER OFS-COMPLEMENT PAIRS
Assume(C1, C2) are zero-error
By the soft lemma, there exists a subsetS with cardinality|S| = nα that is2nβ-shattered byC1
9 / 16
NUMBER OFS-COMPLEMENT PAIRS
Assume(C1, C2) are zero-error
By the soft lemma, there exists a subsetS with cardinality|S| = nα that is2nβ-shattered byC12n(R2+β)≤ # of S-complement pairs
9 / 16
NUMBER OFS-COMPLEMENT PAIRS
Assume(C1, C2) are zero-error
By the soft lemma, there exists a subsetS with cardinality|S| = nα that is2nβ-shattered byC12n(R2+β)≤ # of S-complement pairs≤ 3n(1−α)
9 / 16
NUMBER OFS-COMPLEMENT PAIRS
Assume(C1, C2) are zero-error
By the soft lemma, there exists a subsetS with cardinality|S| = nα that is2nβ-shattered byC12n(R2+β)≤ # of S-complement pairs≤ 3n(1−α)
R2 < (1− α) log 3− β
9 / 16
NUMBER OFS-COMPLEMENT PAIRS
Assume(C1, C2) are zero-error
By the soft lemma, there exists a subsetS with cardinality|S| = nα that is2nβ-shattered byC12n(R2+β)≤ # of S-complement pairs≤ ?
9 / 16
NUMBER OFS-COMPLEMENT PAIRS
Assume(C1, C2) are zero-error
By the soft lemma, there exists a subsetS with cardinality|S| = nα that is2nβ-shattered byC12n(R2+β)≤ # of S-complement pairs≤ ?
Partition (C1, C2) to disjoint pairs{C1,i, C2,i}2nα
i=1 of subcodes
9 / 16
NUMBER OFS-COMPLEMENT PAIRS
Assume(C1, C2) are zero-error
By the soft lemma, there exists a subsetS with cardinality|S| = nα that is2nβ-shattered byC12n(R2+β)≤ # of S-complement pairs≤ ?
Partition (C1, C2) to disjoint pairs{C1,i, C2,i}2nα
i=1 of subcodes
Example:S = {1, 2}
C1:0010110111100111010110110111000010110100100111000
...
C2:1000110011110110010101111101010011010111010000000
...9 / 16
NUMBER OFS-COMPLEMENT PAIRS
Assume(C1, C2) are zero-error
By the soft lemma, there exists a subsetS with cardinality|S| = nα that is2nβ-shattered byC12n(R2+β)≤ # of S-complement pairs≤ ?
Partition (C1, C2) to disjoint pairs{C1,i, C2,i}2nα
i=1 of subcodes
Example:S = {1, 2}
C1:0010110111100111010110110111000010110100100111000
...
C1,a
C2:1000110011110110010101111101010011010111010000000
...
C2,a
9 / 16
NUMBER OFS-COMPLEMENT PAIRS
Assume(C1, C2) are zero-error
By the soft lemma, there exists a subsetS with cardinality|S| = nα that is2nβ-shattered byC12n(R2+β)≤ # of S-complement pairs≤ ?
Partition (C1, C2) to disjoint pairs{C1,i, C2,i}2nα
i=1 of subcodes
Example:S = {1, 2}
C1:0010110111100111010110110111000010110100100111000
...
C1,a
C2:1000110011110110010101111101010011010111010000000
...
C2,a
∀c1,a ∈ C1,a, c2,a ∈ C2,a:
00*****+11*****11*****
In addition: |C1,a| ≥ 2nβ
9 / 16
NUMBER OFS-COMPLEMENT PAIRS
Assume(C1, C2) are zero-error
By the soft lemma, there exists a subsetS with cardinality|S| = nα that is2nβ-shattered byC12n(R2+β)≤ # of S-complement pairs≤ ?
Partition (C1, C2) to disjoint pairs{C1,i, C2,i}2nα
i=1 of subcodes
Example:S = {1, 2}
C1:0010110111100111010110110111000010110100100111000
...
C1,b
C2:1000110011110110010101111101010011010111010000000
...
C2,b
9 / 16
NUMBER OFS-COMPLEMENT PAIRS
Assume(C1, C2) are zero-error
By the soft lemma, there exists a subsetS with cardinality|S| = nα that is2nβ-shattered byC12n(R2+β)≤ # of S-complement pairs≤ ?
Partition (C1, C2) to disjoint pairs{C1,i, C2,i}2nα
i=1 of subcodes
Example:S = {1, 2}
C1:0010110111100111010110110111000010110100100111000
...
C1,b
C2:1000110011110110010101111101010011010111010000000
...
C2,b
∀c1,b ∈ C1,b, c2,b ∈ C2,b:
01*****+10*****11*****
In addition: |C1,b| ≥ 2nβ
9 / 16
NUMBER OFS-COMPLEMENT PAIRS
Assume(C1, C2) are zero-error
By the soft lemma, there exists a subsetS with cardinality|S| = nα that is2nβ-shattered byC12n(R2+β)≤ # of S-complement pairs≤ ?
Partition (C1, C2) to disjoint pairs{C1,i, C2,i}2nα
i=1 of subcodes
Example:S = {1, 2}
C1:0010110111100111010110110111000010110100100111000
...
C1,c
C2:1000110011110110010101111101010011010111010000000
...
C2,c
9 / 16
NUMBER OFS-COMPLEMENT PAIRS
Assume(C1, C2) are zero-error
By the soft lemma, there exists a subsetS with cardinality|S| = nα that is2nβ-shattered byC12n(R2+β)≤ # of S-complement pairs≤ ?
Partition (C1, C2) to disjoint pairs{C1,i, C2,i}2nα
i=1 of subcodes
Example:S = {1, 2}
C1:0010110111100111010110110111000010110100100111000
...
C1,c
C2:1000110011110110010101111101010011010111010000000
...
C2,b
∀c1,c ∈ C1,c, c2,c ∈ C2,c:
11*****+00*****11*****
In addition: |C1,c| ≥ 2nβ
9 / 16
NUMBER OFS-COMPLEMENT PAIRS
We get{C1,i, C2,i}2nα
i=1 with |C1,i| = 2nβ and|C2,i| ≈ 2n(R2−α) foreveryi
10 / 16
NUMBER OFS-COMPLEMENT PAIRS
We get{C1,i, C2,i}2nα
i=1 with |C1,i| = 2nβ and|C2,i| ≈ 2n(R2−α) foreveryiInduces a zero-error scheme for BAC with common message
M1 ∈ [2nr1 ]
M2 ∈ [2nr2 ]
M0 ∈ [2nr0 ]
X1 ∈ {0, 1}
X2 ∈ {0, 1}
+ X1 +X2 ∈ {0, 1, 2}
SendC1,M0(M1) andC2,M0(M2)
10 / 16
NUMBER OFS-COMPLEMENT PAIRS
We get{C1,i, C2,i}2nα
i=1 with |C1,i| = 2nβ and|C2,i| ≈ 2n(R2−α) foreveryiInduces a zero-error scheme for BAC with common message
M1 ∈ [2nr1 ]
M2 ∈ [2nr2 ]
M0 ∈ [2nr0 ]
X1 ∈ {0, 1}
X2 ∈ {0, 1}
+ X1 +X2 ∈ {0, 1, 2}
SendC1,M0(M1) andC2,M0(M2)
r0 = α r1 = β r2 = R2 − α
10 / 16
NUMBER OFS-COMPLEMENT PAIRS
We get{C1,i, C2,i}2nα
i=1 with |C1,i| = 2nβ and|C2,i| ≈ 2n(R2−α) foreveryiInduces a zero-error scheme for BAC with common message
M1 ∈ [2nr1 ]
M2 ∈ [2nr2 ]
M0 ∈ [2nr0 ]
X1 ∈ {0, 1}
X2 ∈ {0, 1}
+ X1 +X2 ∈ {0, 1, 2}
SendC1,M0(M1) andC2,M0(M2)
r0 = α r1 = β r2 = R2 − α
For eachc1,i ∈ C1,i, c2,i ∈ C2,i we have
(c1,i + c2,i)(S) = (1, . . . , 1)
Thenα coordinates inS can be discarded!10 / 16
NUMBER OFS-COMPLEMENT PAIRS
We get{C1,i, C2,i}2nα
i=1 with |C1,i| = 2nβ and|C2,i| ≈ 2n(R2−α) foreveryiInduces a zero-error scheme for BAC with common message
M1 ∈ [2nr1 ]
M2 ∈ [2nr2 ]
M0 ∈ [2nr0 ]
X1 ∈ {0, 1}
X2 ∈ {0, 1}
+ X1 +X2 ∈ {0, 1, 2}
SendC1,M0(M1) andC2,M0(M2)
r0 =α
1− αr1 =
β
1− αr2 =
R2 − α
1− α
For eachc1,i ∈ C1,i, c2,i ∈ C2,i we have
(c1,i + c2,i)(S) = (1, . . . , 1)
Thenα coordinates inS can be discarded!10 / 16
THE BAC WITH A COMMON MESSAGE
A reduction lemma
If (R1, R2) are in the BAC zero-error capacity region, then for any0 < α < h−1(R1)
r0 =α
1− α, r1 =
β(α,R1)
1− α, r2 =
R2 − α
1− α
is in the zero-error capacity region of the BAC with common message.
11 / 16
THE BAC WITH A COMMON MESSAGE
A reduction lemma
If (R1, R2) are in the BAC zero-error capacity region, then for any0 < α < h−1(R1)
r0 =α
1− α, r1 =
β(α,R1)
1− α, rΣ =
R2 + β(α,R1)
1− α
is in the zero-error capacity region of the BAC with common message.
11 / 16
THE BAC WITH A COMMON MESSAGE
A reduction lemma
If (R1, R2) are in the BAC zero-error capacity region, then for any0 < α < h−1(R1)
r0 =α
1− α, r1 =
β(α,R1)
1− α, rΣ =
R2 + β(α,R1)
1− α
is in the zero-error capacity region of the BAC with common message.
R2 < (1− α)rΣ − β(α,R1)
11 / 16
THE BAC WITH A COMMON MESSAGE
A reduction lemma
If (R1, R2) are in the BAC zero-error capacity region, then for any0 < α < h−1(R1)
r0 =α
1− α, r1 =
β(α,R1)
1− α, rΣ =
R2 + β(α,R1)
1− α
is in the zero-error capacity region of the BAC with common message.
R2 < (1− α)rΣ − β(α,R1)
New goal: Upper boundrΣ under the above constraints onr0, r1
11 / 16
THE BAC WITH A COMMON MESSAGE
A reduction lemma
If (R1, R2) are in the BAC zero-error capacity region, then for any0 < α < h−1(R1)
r0 =α
1− α, r1 =
β(α,R1)
1− α, rΣ =
R2 + β(α,R1)
1− α
is in the zero-error capacity region of the BAC with common message.
R2 < (1− α)rΣ − β(α,R1)
New goal: Upper boundrΣ under the above constraints onr0, r1Trivial bound:rΣ ≤ log 3
11 / 16
THE BAC WITH A COMMON MESSAGE
A reduction lemma
If (R1, R2) are in the BAC zero-error capacity region, then for any0 < α < h−1(R1)
r0 =α
1− α, r1 =
β(α,R1)
1− α, rΣ =
R2 + β(α,R1)
1− α
is in the zero-error capacity region of the BAC with common message.
R2 < (1− α) log 3− β(α,R1)
New goal: Upper boundrΣ under the above constraints onr0, r1Trivial bound:rΣ ≤ log 3
11 / 16
THE BAC WITH A COMMON MESSAGE
A reduction lemma
If (R1, R2) are in the BAC zero-error capacity region, then for any0 < α < h−1(R1)
r0 =α
1− α, r1 =
β(α,R1)
1− α, rΣ =
R2 + β(α,R1)
1− α
is in the✭✭✭✭✭zero-error capacity region of the BAC with common message.
R2 < (1− α)rΣ − β(α,R1)
New goal: Upper boundrΣ under the above constraints onr0, r1Trivial bound:rΣ ≤ log 3
11 / 16
THE BINARY ADDER WITH A COMMON MESSAGE
Theorem[Slepian-Wolf 1973], [Willems 1982]
The Shannon capacity region of the BAC with a common message,isthe closure of the union of all rate triplets satisfying
r1 ≤ H(X1|U)
r2 ≤ H(X2|U)
r1 + r2 ≤ H(X1 +X2|U)
rΣ = r0 + r1 + r2 ≤ H(X1 +X2)
for somePUPX1|UPX2|U , where|U| ≤ 4.
12 / 16
THE BINARY ADDER WITH A COMMON MESSAGE
Theorem[Slepian-Wolf 1973], [Willems 1982]
The Shannon capacity region of the BAC with a common message,isthe closure of the union of all rate triplets satisfying
r1 ≤ H(X1|U)
✭✭✭✭✭✭✭
r2 ≤ H(X2|U)
r1 + r2 ≤ H(X1 +X2|U)
rΣ = r0 + r1 + r2 ≤ H(X1 +X2)
for somePUPX1|UPX2|U , where|U| ≤ 3.
12 / 16
THE BINARY ADDER WITH A COMMON MESSAGE
Theorem[Slepian-Wolf 1973], [Willems 1982]
The Shannon capacity region of the BAC with a common message,isthe closure of the union of all rate triplets satisfying
r1 ≤ H(X1|U)
✭✭✭✭✭✭✭
r2 ≤ H(X2|U)
r1 + r2 ≤ H(X1 +X2|U)
rΣ = r0 + r1 + r2 ≤ H(X1 +X2)
for somePUPX1|UPX2|U , where|U| ≤ 3.
Still difficult - 7 parameters to optimize...
Need to upper boundrΣ(r0, r1) analytically!
12 / 16
THE BINARY ADDER WITH A COMMON MESSAGE
Lemma (Sum-Rate Bound)
If (r0, r1, r2) is achievable then there is someη ∈ [0, 12 ] s.t.
rΣ ≤ maxh−1(r1)≤η≤ 1
2
min(
L(η), J(h−1(r1), η) + r0)
where
L(η)def= h(η) + 1− η
J(p, η)def=
2h(
12
(
1−√1− 2η
))
− η η ≥ p ⋆ p
2h
(
12
(
1− 1−η−p⋆p√1−2(p⋆p)
))
− 12
(
1− (1−η−p⋆p)2
1−2(p⋆p)
)
η < p ⋆ p
13 / 16
THE BINARY ADDER WITH A COMMON MESSAGE
r0
0 0.1 0.2 0.3 0.4 0.5
r Σ
1.5
1.51
1.52
1.53
1.54
1.55
1.56
1.57
1.58
1.59
r1= 0.8
r1= 0.95
r1= 0.99
14 / 16
TYING THE LOOSEENDS
Theorem (Outer bound for the Zero-error Capacity Region)
Let
rΣ(r0, r1) , maxh−1(r1)≤η≤ 1
2
min{L(η), J(h−1(r1), η) + r0}
Then any zero-error achievable rate pair(R1, R2) satisfies
R2 < min0≤α≤h−1(R1)
(1− α)
(
rΣ
(
α
1− α, Γ(R1, α)
)
− Γ(R1, α)
)
where
Γ(R1, α) , h
(
h−1(R1)− α
1− α
)
15 / 16
TYING THE LOOSEENDS
R2
0.48 0.485 0.49 0.495 0.5 0.505 0.51 0.515 0.52
R1
0.98
0.985
0.99
0.995
1
1.005
Best inner boundShannon capacity regionUrbanke and Li UBNew UB
15 / 16
SUMMARY AND DISCUSSION
New outer bound on BAC zero-error capacity region
We introduced the notion ofkth order VC-dimension and provedan analog of Sauer’s Lemma
Our bounding technique combined this combinatorial notionwithnetwork information theoretic arguments
Weaknesses of our boundThe lower bound on# of S-complement pairs is valid for any pair(C1, C2) (not just zero-error pairs)We lower bounded the number ofS-complement pairs foranyk-shattered set inC1, but there aremany such sets.
Our technique may be applicable to other zero-error problems
16 / 16