Enumerative Source Coding and Shaping
Frans M.J. Willems
ICT-Lab,Department of Electrical EngineeringEindhoven University of Technology
Munich, DoktorandenSeminar, July 19-20, 2018
LABT
IC
Where innovation starts1/55 July 21, 2018
LABT
IC
OutlinePIET SCHALKWIJK
FeedbackSource CodingTwo-Way Channel
ENUMERATIVE SOURCE CODINGBinary IID SourceLexicographical OrderingPascal-∆ Algorithm
VARIATIONSSch-Antonio-PetrySch-Petry CodeShell Mapping (D & C)Rounding
SHAPING CODESGap to CapacityProbablistic Shaping, Distribution Matching
ENUMERATIVE SHAPINGShaping for 802.11
REMARK/REFERENCES
2/55 July 21, 2018
LABT
IC
PIET SCHALKWIJK: Algorithms, Algorithms, Algorithms, ...
1963 - 1965: PhD Stanford (supervisor Thomas Kailath)1968 - 1972: Assistant Professor University California San Diego.1972 - 1996: Professor “Telecommunicatie” Technische Hogeschool Eindhoven
3/55 July 21, 2018
LABT
IC
Feedback, AWGN Channel, Wideband Case [1966]
1967 Information Theory Society Paper Award
4/55 July 21, 2018
LABT
IC
Feedback, AWGN Channel, Band-Limited Case [1966]
5/55 July 21, 2018
LABT
IC
Feedback, Binary Symmetric Channel [1971]
6/55 July 21, 2018
LABT
IC
Source Coding [1972]
7/55 July 21, 2018
LABT
IC
AND - Channel [1982]
8/55 July 21, 2018
LABT
IC
Binary Sources, Sequences, IID
Binary Sourcex1x2 · · ·xN
The binary source produces a sequence xN1 = x1x2 · · ·xN with components ∈ {0, 1} withprobability P (xN1 ).
Definition (Binary IID Source)For an independent identically distributed (i.i.d.) source with parameter θ, for 0 ≤ θ ≤ 1,
P (xN1 ) =N∏n=1
P (xn), where P (1) = θ, and P (0) = 1− θ.
A sequence xN1 containing N − w zeros and w ones has probability
P (xN1 ) = (1− θ)N−wθw.
Entropy IID Source
The ENTROPY of this source is h(θ) ∆= (1− θ) log21
1−θ + θ log21θ
(bits).9/55 July 21, 2018
LABT
IC
Lexicographical OrderingIDEA:Sequences having the same composition (and probability) only need to be INDEXED. Thebinary representation of the index can be taken as codeword.
Definition (Lexicographical Ordering)In a lexicographical ordering (0 < 1) we say that xN1 < yN1 if xn < yn for the smallestindex n such that xn 6= yn.Consider a subset S of the set {0, 1}N . Let iS(xN1 ) be the lexicographical index ofxN1 ∈ S, i.e. the number of sequences yN1 < xN1 for yN1 ∈ S.
ExampleLet N = 5 and S = {xN1 : w(xN1 ) = 2} where w(xN1 ) is the weight of xN1 . Then|S| =
(52
)= 10 and:
iS(11000) = 9 iS(01100) = 4iS(10100) = 8 iS(01010) = 3iS(10010) = 7 iS(01001) = 2iS(10001) = 6 iS(00110) = 1iS(01100) = 5 iS(00011) = 0
10/55 July 21, 2018
LABT
IC
Pascal-∆ Algorithm
IDEA:Index sequences of fixed weight. Use a Huffman code (or a fixed-length code) to describethe weights.
Example (Schalkwijk (1972))Let N = 5 and S = {xN1 :
∑xn = 2}. Then |S| =
(52
)= 10.
10
4
6
1
3
3
1
2
1
1
1
1
1
1
1
1
1
1
1
10
0
0
0
0
0
0
0
0
Index from Sequence:
i(10100) = 6 + 2 = 8.
Sequence from Index:Index i = 8, nowa) 8 ≥ 6 hence x1 = 1,b) i < 6 + 3 hence x2 = 0,c) i ≥ 6 + 2 hence x3 = 1,d) x4 = x5 = 0.
11/55 July 21, 2018
LABT
IC
Sequential Enumeration
Theorem (Cover, 1973)From the sequence xN1 ∈ S we can compute index
iS(xN1 ) =∑
n=1,N :xn=1
#S(x1, x2, · · · , xn−1, 0),
where #S(x1, x2, · · · , xk) denotes the number of sequences in S having prefixx1, · · · , xk.Moreover from the index iS(xN1 ) the sequence xN1 can be computed if numbers#S(x1, x2, · · · , xn−1, 0) for n = 1, N are available.
The index of a sequence can be represented by a codeword of fixed length dlog2 |S|e.
ExampleIndex iS(10100) = 8 hence, since |S| = 10 the corresponding codeword is 1000b.
12/55 July 21, 2018
LABT
IC
Analysis Pascal-∆ Method
First note that
Nh(θ) = H(XN1 ) = H(XN
1 , w(XN1 )) = H(W ) +H(XN
1 |W ).
If we use enumerative coding for XN1 given weight w, since all sequences with a fixed
weight have equal probability
E[L(XN1 |W )] =
∑w=0,1,N
P (w) log2d(Nw
)e
<∑
w=0,1,N
P (w) log2(Nw
)+ 1 = H(XN
1 |W ) + 1.
If W is encoded using a Huffman code we obtain
E[L(XN1 )] = E[L(W )] + E[L(XN
1 |W )]≤ H(W ) + 1 +H(XN
1 |W ) + 1= H(XN
1 ) + 2.
Worse than Huffman, but no large code-table needed.
13/55 July 21, 2018
LABT
IC
Pascal-∆ Method (again): Weight = 3, Length = 8
,
56
21
35
6
15
20
1
5
10
10
.
1
4
6
4
.
.
1
3
3
1
.
.
.
1
2
1
.
.
.
.
.
1
1
.
.
.
.
.
.
.
1
.
.
.
00
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
001
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
11
14/55 July 21, 2018
LABT
IC
Schalkwijk Antonio Petry: Weight ≤ 3, Length = 8
93
29
64
7
22
42
1
6
16
26
.
1
5
11
15
.
.
1
4
7
8
.
.
.
1
3
4
4
.
.
.
.
1
2
2
2
.
.
.
.
.
1
1
1
1
00
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
011
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
J.P.M. Schalkwijk, F. Antonio, and R. Petry,Hawaii Int. Conf. Syst. Sciences [1972]
15/55 July 21, 2018
LABT
IC
Schalkwijk-Petry Code: Variable-to-Fixed Length Code
55
21
34
8
13
21
3
5
8
13
1
2
3
5
8
.
1
1
2
3
5
.
.
.
1
1
2
3
.
.
.
.
.
1
1
2
.
.
.
.
.
.
.
1
1
00
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
11
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
21 3
J.P.M. Schalkwijk, Benelux Symp. Inform. Theory [1981]
16/55 July 21, 2018
LABT
IC
Schalkwijk-Petry Code: AnalysisIDEA:Modify the Tunstall segment sets such that the segments can be indexed.
Again let 0 < θ ≤ 1/2. It can be shown that a proper-and-complete segment set is aTunstall set (maximal E[N(X∗)] given the number of segments) if and only if for all nodesn and all leaves l
P (n) ≥ P (l).
ConsequenceIf the segments x∗ in a proper-and-complete segment set satisfy
P (x∗−1) > γ ≥ P (x∗),
this segment set is a Tunstall set. Constant γ determines the size of the set.
SinceP (x∗) = (1− θ)n0(x∗)θn1(x∗),
where n0(x∗) is the number or zeros in x∗ and n1(x∗) the number of ones in x∗, this isequivalent to
An0(x∗−1) +Bn1(x∗−1) < C ≤ An0(x∗) +Bn1(x∗)
for A = − logb(1− θ), B = − logb θ, C = − logb γ, and some log-base b.17/55 July 21, 2018
LABT
IC
Schalkwijk-Petry Code: Analysis
Note that log-base b has to satisfy
1 = (1− θ) + θ = b−A + b−B .
For special values of θ, A and B are integers. E.g. for θ = (1− θ)2 we obtain that A = 1and B = 2 for b = (1 +
√5)/2. Now C can also assumed to be integer. The corresponding
codes are called Petry codes.
Definition (Petry (Schalkwijk), 1982)Fix integers A and B. The segments x∗ in a proper-and-complete Petry segment set satisfy
An0(x∗−1) +Bn1(x∗−1) < C ≤ An0(x∗) +Bn1(x∗).
Integer C can be chosen to control the size of the set.
Linear ArrayPetry codes can be implemented using a linear array.
18/55 July 21, 2018
LABT
IC
Schalkwijk-Petry Code: Analysis
ExampleConsider A = 1, B = 2.For given C, let S(C) denote the resulting segment set and σ(C) its cardinality. LetS(−1) = S(0) = ∅, then S(1) = {0, 1}, S(2) = {00, 01, 1}, etc. Moreover nowσ(−1) = σ(0) = 1, σ(1) = 2 and σ(2) = 3, etc. It is easy to see that
σ(C) = σ(C − 1) + σ(C − 2),
and therefore σ(3) = 5, σ(4) = 8, σ(5) = 13, σ(6) = 21, σ(7) = 34, and σ(8) = 55.Now take C = 8. Note that 010010 ∈ S(8).We can now determine the index i(010010) using Cover’s formula:
i(010010) = #(00) + #(01000) = σ(6) + σ(2) = 21 + 3 = 24.
55 34 21 13 8 5 3 2 1 10
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
19/55 July 21, 2018
LABT
IC
Schalwijk-Petry Code: Analysis
Theorem (Tjalkens & W. (1987))A Petry code with parameters A < B, and C is a Tunstall code for parameter q whereq = b−B when b is the solution of b−A + b−B = 1.For arbitrary θ the rate
log2 σ(C)E[N(X∗)]
≤C + (B − 1)
C(h(θ) + d(θ||q)).
ExampleIn the table q for several values of A and B:
A B 2 3 4 51 0.382 0.318 0.276 0.2452 0.4302 0.382 0.3463 0.450 0.412
20/55 July 21, 2018
LABT
IC
Shell Mapping (Divide & Conquer)
56
21
35
6
15
20
1
5
10
10
.
1
4
6
4
.
.
1
3
3
1
.
.
.
1
2
1
.
.
.
.
.
1
1
.
.
.
.
.
.
.
1
.
.
.
00
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
001
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
11
20
101
i(0100010) = 20 + 10 + 1 = 31
21/55 July 21, 2018
LABT
IC
Shell Mapping (Divide & Conquer)
1
1
1
1
2
1
1
3
3
1
.
4×1
6×4
4×6
1×4
.
.
1
3
3
1
.
.
.
1
2
1
.
.
.
.
.
1
1
.
.
.
.
.
.
.
1
.
.
.
00
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
001
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
11
i(0100010) = 4 + 24 + · · ·
22/55 July 21, 2018
LABT
IC
Shell Mapping (Divide & Conquer)
6
3
3
1
2
1
.
1
1
.
.
.
1
.
.
.
.
4
.
.
.
.
1
3
.
.
.
.
.
1
2
.
.
.
.
.
.
1
1
.
.
.
.
.
.
.
1
.
.
.
00
00
00
00
0
0
001
1
1
1
11
11
11
Lang and Longstaff [1989]...
Laroia, Farvardin, and Tretter [1994]
i(01100010) = 4 + 24 + (1 + 1) ∗ 4 + 1 = 37 = 100101b23/55 July 21, 2018
LABT
IC
Pascal-∆ Method (again): Weight = 3, Length = 8
56
21
35
6
15
20
1
5
10
10
.
1
4
6
4
.
.
1
3
3
1
.
.
.
1
2
1
.
.
.
.
.
1
1
.
.
.
.
.
.
.
1
.
.
.
00
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
001
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
11
20
101
i(0100010) = 20 + 10 + 1 = 31 = 011111b24/55 July 21, 2018
LABT
IC
Rounding (2-bit mantissa), Encoding
96
32
48
8
24
24
1
6
12
12
.
1
4
6
4
.
.
1
3
3
1
.
.
.
1
2
1
.
.
.
.
.
1
1
.
.
.
.
.
.
.
1
.
.
.
00
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
001
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
11
24
121
Folklore in arithmetic coding (Pasco [1976], Rissanen [1976]),and in enumerative coding (Tjalkens [1987]).
i(0100010) = 24 + 12 + 1 = 37 = 0100101b25/55 July 21, 2018
LABT
IC
Rounding (2-bit mantissa), Decoding
96
32
48
8
24
24
1
6
12
12
.
1
4
6
4
.
.
1
3
3
1
.
.
.
1
2
1
.
.
.
.
1
1
.
.
.
.
.
.
1
.
.
00
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
001
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
11
24
121
37 < 48→ 037 ≥ 24→ 1
37 ≥ 24 + 12→ 137 < 24 + 12 + 4→ 0
37 < 24 + 12 + 3→ 037 < 24 + 12 + 2→ 0
37 ≥ 24 + 12 + 1→ 1forced → 0
26/55 July 21, 2018
LABT
IC
SHAPING: Gap to Channel CapacityFor signal-to-noise ratio SNR horizontally in dB the AGN capacity CAGN in bits/channeluse is depicted in black in the figure below.
-5 0 5 10 15 20 25 30 35
SNR (dB)
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5C
agn
, C
2, C
4, C
8, C
16, C
32 (
bit)
The curves for uniform 2-PAM, 4-PAM, 8-PAM, 16-PAM and 32-PAM are depicted in blue.A gap to AGN-capacity appears since the PAM inputs are not Gaussian, but Uniform.
27/55 July 21, 2018
LABT
IC
SHAPING: Gap in bit and in SNR lossAssumptions: (a) M -PAM where M →∞ and(b) SNR→∞ or equivalently that σ2
n → 0.
-5 -4 -3 -2 -1 0 1 2 3 4 5
x
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
pg(x
), p
u(x
)
Consider difference of the capacity I(Xg ;Yg), where Xg is the Gaussian channelinput and Yg the corresponding output, and the mutual information I(Xu;Yu), whereXu is a uniform channel input and Yu the corresponding output:
I(Xg ;Yg)− I(Xu;Yu)= h(Yg)− h(Yg |Xg)− h(Yu) + h(Yu|Xu)= h(Yg)− h(Yu)
=12
log2(2πeσ2x)− log2 12σ2
x =12
log2πe
6= 0.2546 bit,
or equivalently 1.53 dB in SNR.28/55 July 21, 2018
LABT
IC
Probabilistic Shaping: Equidistant Signal Points
Eight equidistant signals x ∈ {−7γ,−5γ, · · · ,+7γ} for γ = 1.089.Non-uniform probability distribution P (x): {0.0521, 0.0989, 0.1562, 0.1927, 0.1927,0.1562, 0.0989, 0.0521}. In the plot:
p(x, y) = P (x)p(y|x) for x ∈ {−7γ,−5γ, · · · ,+7γ}
p(y) =∑x
P (x)p(y|x) where p(y|x) =1√
2πexp(−
(y − x)2
2).
Moreover I(X;Y ) = 2.001 bit at SNR = 11.96 dB.29/55 July 21, 2018
LABT
IC
Probabilistic Shaping, Distribution Matching
Q: How can we generate sequences with a given composition?
We can use Distribution Matching (Boecherer [2014], Schulte and Boecherer [2016]).
Distribution matching is inspired by arithmetic data compression techniques (e.g.Langdon and Rissanen [1979], Witten, Neal, and Cleary [1987]). In their methodssequences are represented by intervals.
30/55 July 21, 2018
LABT
IC
Sequences and Intervals, ExampleWe want to generate binary sequences of length 5 containing 2 ones. There are(5
2
)= 10 such sequences.
Each of these sequences corresponds to a subinterval of length 1/10 of the [0, 1) interval.This interval can be computed sequentially from the sequence. The first digit of thesequence splits the interval in fractions 3/5 and 2/5. After a first 0 the interval [0, 3/5) issplit according to 2/4 and 2/4, etc.Note that the sequences and their intervals are now in a lexicographical order.
0.0
1.0
0
1
0.6
0
1
0
1
0.3
0.9
0
1
0
1
0
1
0
0.1
0.5
0.8
1
0
1
0
1
0
0
1
0
0
0.2
0.4
0.7
1
1
0
1
0
0
1
0
0
0
31/55 July 21, 2018
LABT
IC
Sequences and Intervals, ExampleWe want to generate binary sequences of length 5 containing 2 ones. There are(5
2
)= 10 such sequences.
Each of these sequences corresponds to a subinterval of length 1/10 of the [0, 1) interval.This interval can be computed sequentially from the sequence. The first digit of thesequence splits the interval in fractions 3/5 and 2/5. After a first 0 the interval [0, 3/5) issplit according to 2/4 and 2/4, etc.Note that the sequences and their intervals are now in a lexicographical order.
0.0
1.0
0
1
0.6
0
1
0
1
0.3
0.9
0
1
0
1
0
1
0
0.1
0.5
0.8
1
0
1
0
1
0
0
1
0
0
0.2
0.4
0.7
1
1
0
1
0
0
1
0
0
0
31/55 July 21, 2018
LABT
IC
Sequences and Intervals, ExampleWe want to generate binary sequences of length 5 containing 2 ones. There are(5
2
)= 10 such sequences.
Each of these sequences corresponds to a subinterval of length 1/10 of the [0, 1) interval.This interval can be computed sequentially from the sequence. The first digit of thesequence splits the interval in fractions 3/5 and 2/5. After a first 0 the interval [0, 3/5) issplit according to 2/4 and 2/4, etc.Note that the sequences and their intervals are now in a lexicographical order.
0.0
1.0
0
1
0.6
0
1
0
1
0.3
0.9
0
1
0
1
0
1
0
0.1
0.5
0.8
1
0
1
0
1
0
0
1
0
0
0.2
0.4
0.7
1
1
0
1
0
0
1
0
0
0
31/55 July 21, 2018
LABT
IC
Sequences and Intervals, ExampleWe want to generate binary sequences of length 5 containing 2 ones. There are(5
2
)= 10 such sequences.
Each of these sequences corresponds to a subinterval of length 1/10 of the [0, 1) interval.This interval can be computed sequentially from the sequence. The first digit of thesequence splits the interval in fractions 3/5 and 2/5. After a first 0 the interval [0, 3/5) issplit according to 2/4 and 2/4, etc.Note that the sequences and their intervals are now in a lexicographical order.
0.0
1.0
0
1
0.6
0
1
0
1
0.3
0.9
0
1
0
1
0
1
0
0.1
0.5
0.8
1
0
1
0
1
0
0
1
0
0
0.2
0.4
0.7
1
1
0
1
0
0
1
0
0
0
31/55 July 21, 2018
LABT
IC
Sequences and Intervals, ExampleWe want to generate binary sequences of length 5 containing 2 ones. There are(5
2
)= 10 such sequences.
Each of these sequences corresponds to a subinterval of length 1/10 of the [0, 1) interval.This interval can be computed sequentially from the sequence. The first digit of thesequence splits the interval in fractions 3/5 and 2/5. After a first 0 the interval [0, 3/5) issplit according to 2/4 and 2/4, etc.Note that the sequences and their intervals are now in a lexicographical order.
0.0
1.0
0
1
0.6
0
1
0
1
0.3
0.9
0
1
0
1
0
1
0
0.1
0.5
0.8
1
0
1
0
1
0
0
1
0
0
0.2
0.4
0.7
1
1
0
1
0
0
1
0
0
0
31/55 July 21, 2018
LABT
IC
Constant Composition Sequence Intervals, IndicesConsider 3-digit indices 000, 001, · · · , 111. Index b1b2b3 connects to a constantcomposition sequence if b12−1 + b22−2 + b32−3 is in the interval corresponding to theconstant composition sequence.
0.0
1.0
0
1
0.6
0
1
0
1
0.3
0.9
0
1
0
1
0
1
0
0.1
0.5
0.8
1
0
1
0
1
0
0
1
0
0
0.2
0.4
0.7
1
1
0
1
0
0
1
0
0
0
0.000(000)
0.500(100)
0.250(010)
0.750(110)
0.125(001)
0.375(011)
0.625(101)
0.875(111)
(a) Only one index can connect to a sequence since 2−3 ≥ 1/10.(b) Observe that for not all constant composition sequences there is an index.(c) For all indices there is a sequence however.
32/55 July 21, 2018
LABT
IC
Constant Composition Sequence Intervals, IndicesConsider 3-digit indices 000, 001, · · · , 111. Index b1b2b3 connects to a constantcomposition sequence if b12−1 + b22−2 + b32−3 is in the interval corresponding to theconstant composition sequence.
0.0
1.0
0
1
0.6
0
1
0
1
0.3
0.9
0
1
0
1
0
1
0
0.1
0.5
0.8
1
0
1
0
1
0
0
1
0
0
0.2
0.4
0.7
1
1
0
1
0
0
1
0
0
0
0.000(000)
0.500(100)
0.250(010)
0.750(110)
0.125(001)
0.375(011)
0.625(101)
0.875(111)
(a) Only one index can connect to a sequence since 2−3 ≥ 1/10.(b) Observe that for not all constant composition sequences there is an index.(c) For all indices there is a sequence however.
32/55 July 21, 2018
LABT
IC
Combining Shaping with Coding
Schematic:
m distrib.matcher
a Graydemapper
b3
b2systematic
rate 23
coder b3
b2
b1
Graymapper
x
The distribution matcher converts message m into amplitude sequence a of thedesired composition.The Gray demapper (sign bit missing!) converts the amplitude sequence a into thetwo amplitude bitstreams b2 and b3 both of length N .
a b2 b31 1 03 1 15 0 17 0 0
Now parity is generated from b2 and b3, using a systematic code of rate 2/3. Thisparity is used as bitstream b1. This bitstream represents the sign bits.
33/55 July 21, 2018
LABT
IC
Combining Shaping with Coding
The three bitstreams are combined into an 8-PAM symbol x stream, using Graymapping.
b1 b2 b3 x0 0 0 −70 0 1 −50 1 1 −30 1 0 −11 1 0 +11 1 1 +31 0 1 +51 0 0 +7
We have described a Bit-Interleaved Coded Modulation construction, where onlysequences with constant amplitude composition are generated.Log-Likelihood Ratio calculation now includes a priori symbol information.
LLRi =
∑b1b2b3:bi=0 p(b1b2b3)p(y|x(b1, b2, b3))∑b1b2b3:bi=1 p(b1b2b3)p(y|x(b1, b2, b3))
34/55 July 21, 2018
LABT
IC
Boecherer Simulations
FER (frame error rate) = 10−3, LDPC codes from DVB-S2. Boecherer, Schulte, andSteiner [2016].
35/55 July 21, 2018
LABT
IC
Alternative to Arithmetic-Coding Approach
Instead of arithmetic-coding approach we can use Schalkwijk’s Pascal-∆ method.
Non-binary case: Use binary splits. Example:
n1357
n13
n57
n1
n3
n5
n7 n1357!n1!n3!n5!n7! =
(n1357n13
)(n13n1
)(n57n5
)Use rounding to reduce complexity.
36/55 July 21, 2018
LABT
IC
Enumerative Shap., from a Partially-Filled Surface to a SphereExample by Y. Gultekin [TU/e, 2017]: In 802.11, in each OFDM symbol 96 realdimensions are used. Now (37, 30, 19, 10) is the Maxwell- Boltzmann composition over 96dimensions with rate 1.75. Composition (37, 30, 19, 10) leads to 2168.72 sequences (rate1.75) and sequence energy 1272.
QUESTION: Can we obtain more sequences such that the average sequence energy doesnot exceed 1272?
Note first that there are more com-positions with energy equal to 1272.Add the corresponding sequences.This leads to 2172.75 sequences.Add all the sequences with an en-ergy smaller than 1272. Now weobtain 2175.04 sequences. More-over the average energy drops to1242.4.
√1272
If we are interested in rate 1.75 we can decrease the radius to√
1120 Now we find 2168.03
sequences with average sequence energy 1096.9. Gain = 12721096.9 = 0.6431 dB.
37/55 July 21, 2018
LABT
IC
Enumerative Shap., from a Partially-Filled Surface to a SphereExample by Y. Gultekin [TU/e, 2017]: In 802.11, in each OFDM symbol 96 realdimensions are used. Now (37, 30, 19, 10) is the Maxwell- Boltzmann composition over 96dimensions with rate 1.75. Composition (37, 30, 19, 10) leads to 2168.72 sequences (rate1.75) and sequence energy 1272.
QUESTION: Can we obtain more sequences such that the average sequence energy doesnot exceed 1272?
Note first that there are more com-positions with energy equal to 1272.Add the corresponding sequences.This leads to 2172.75 sequences.Add all the sequences with an en-ergy smaller than 1272. Now weobtain 2175.04 sequences. More-over the average energy drops to1242.4.
√1272
If we are interested in rate 1.75 we can decrease the radius to√
1120 Now we find 2168.03
sequences with average sequence energy 1096.9. Gain = 12721096.9 = 0.6431 dB.
37/55 July 21, 2018
LABT
IC
Enumerative Shap., from a Partially-Filled Surface to a SphereExample by Y. Gultekin [TU/e, 2017]: In 802.11, in each OFDM symbol 96 realdimensions are used. Now (37, 30, 19, 10) is the Maxwell- Boltzmann composition over 96dimensions with rate 1.75. Composition (37, 30, 19, 10) leads to 2168.72 sequences (rate1.75) and sequence energy 1272.
QUESTION: Can we obtain more sequences such that the average sequence energy doesnot exceed 1272?
Note first that there are more com-positions with energy equal to 1272.Add the corresponding sequences.This leads to 2172.75 sequences.Add all the sequences with an en-ergy smaller than 1272. Now weobtain 2175.04 sequences. More-over the average energy drops to1242.4.
√1272
If we are interested in rate 1.75 we can decrease the radius to√
1120 Now we find 2168.03
sequences with average sequence energy 1096.9. Gain = 12721096.9 = 0.6431 dB.
37/55 July 21, 2018
LABT
IC
Enumerative Sphere Shaping: Bounded-Energy TrellisWuijts, internship TU/e [1991], MSc report TU/e, [1993],W. and Wuijts, IEEE Symp. Comm. Vehic. Techn. Benelux [1993].N = 4, amplitude alphabet is {1, 3, 5, · · · }, Emax = 28 i.e. sphere radius
√28.
0/19 1/11
9/7
17
25/1
2/6
10/4
18/3
26/1
3/3
11/2
19/2
27/1
4/1
12/1
20/1
28/1
1 1 1 1
3 3 3 3
5 5 5 5
1 1 13 3 3
1 13 3
1 1 1
38/55 July 21, 2018
LABT
IC
Lexicographical Ordering. Index of sequence 3131 is 13.
19 11
7
1
6
4
3
1
3
2
2
1
1
1
1
1
1 1 1 1
3 3 3 3
5 5 5 5
1 1 13 3 3
1 13 3
1 1 1
39/55 July 21, 2018
LABT
IC
Lexicographical ordering. Sequence with index 8 is 1331.
19 11
7
1
6
4
3
1
3
2
2
1
1
1
1
1
1 1 1 1
3 3 3 3
5 5 5 5
1 1 13 3 3
1 13 3
1 1 1
40/55 July 21, 2018
LABT
IC
N = 6, Emax = 70, Trellis
517 237
177
87
16
100
82
58
44
32
11
5
38
35
26
20
17
11
7
4
1
13
13
11
8
8
6
4
3
1
4
4
4
3
3
3
2
2
1
1
1
1
1
1
1
1
1
1
41/55 July 21, 2018
LABT
IC
N = 6, Emax = 70, Analysis
Sequences 731111, 731113, 731131, 731311, 733111 are not used if 0 ≤ index < 512.Adding signs leads to 64 · 512 = 215 sequences.Total rate
Rtot =log2 215
6= 2.500 bits/symbol.
Average energy per symbol Eav/N = 9.2422.Gain
G =22Rtot−1
3Eav/N
= 0.4847 dB.
42/55 July 21, 2018
LABT
IC
N = 6, Emax = 70, Rounded Trellis, 6-bit Mantissa
512 236
176
86
16
100
82
58
44
32
11
5
38
35
26
20
17
11
7
4
1
13
13
11
8
8
6
4
3
1
4
4
4
3
3
3
2
2
1
1
1
1
1
1
1
1
1
1
43/55 July 21, 2018
LABT
IC
N = 6, Emax = 70, Analysis
Sequences 173311, 373111, 553311, 731311, 733111 are not used in rounded trellis.Adding signs leads to 64 · 512 = 215 sequences.Total rate
Rtot =log2 215
6= 2.500 bits/symbol.
Average energy per symbol Eav/N = 9.2422.Gain
G =22Rtot−1
3Eav/N
= 0.4847 dB.
44/55 July 21, 2018
LABT
IC
Combining Enumerative Sphere Shaping with Coding
Schematic:
m enumer.shaper
a Graydemapper
b3
b2systematic
rate 23
coder b3
b2
b1
Graymapper
x
The enumerative shaper converts a message m into a bounded energy amplitudesequence a.No difference with Boecherer’s approach.
45/55 July 21, 2018
LABT
IC
Probabilistic Shaping versus Enumerative Shaping
Consider a hypersphere in N (even) dimensions with radius ρ, then
h(X) =h(X1) + h(X2) + · · ·+ h(XN )
N
≥h(X1, X2, · · · , XN )
N=
1N
logπN/2
(N/2)!ρN
≥1N
logπN/2
(N2e )N/2√
e2N2
ρN =12
log 2πeρ2
N−
12N
loge2N
2.
h(X) =h(X1) + h(X2) + · · ·+ h(XN )
N
≤log 2πeE[X2
1 ] + log 2πeE[X22 ] + · · ·+ log 2πeE[X2
N ]2N
≤12
log 2πeE[X2
1 ] + E[X22 ] + · · ·+ E[X2
N ]N
=12
log 2πeρ2
N.
HENCE enumerative shaping for large N results in Gaussians components!Wuijts [1993], continuous case.
46/55 July 21, 2018
LABT
IC
Probabilistic Shaping versus Enumerative Sphere Shaping
Both probabilistic shaping and enumerative sphere shaping lead to Gaussian inputdistributions, which is required to achieve capacity.In a coding environment the bounded energy constraint could be more elementarythan the Gaussian input constraint.For short blocklengths there is something to gain with enumerative shaping.For large blocklengths the gain is negligible (sphere hardening argument).Complexity of enumerative sphere shaping is larger than that of probabilistic shaping(same blocklength).
47/55 July 21, 2018
LABT
IC
Enumerative Sphere Shaping for 802.11
Gultekin, van Houtum, Serbetli, and W., IEEE PIMRC, Montreal [2017]:
Note that shaping does no effect the Peak-Average-Power-Ratio in combination withOFDM. In pulse transmission the PAPR increases.Shaping in combination with the mother convolutional code in 802.11.Amplitudes A7, A13, A25, · · · are prescribed. Input bits can be toggled to hit theamplitude bits. Toggling a bit results in a complemented output pair. Sign bits andPunctured bits follow.
48/55 July 21, 2018
LABT
IC
Enumerative Sphere Shaping for 802.11Puncture-then-Shape Performance:
49/55 July 21, 2018
LABT
IC
Enumerative Sphere Shaping for 802.11Gultekin, van Houtum, and W., Symp. Inform. Th. and Sign. Proc. in the Benelux,Enschede [2018]:
(Enumerative) Sphere Shaping results in Maxwell-Boltzmann Distribution forincreasing blocklengths.Rate-loss (ESS vs. CCDM):
50/55 July 21, 2018
LABT
IC
Enumerative Sphere Shaping for 802.11Enumerative Sphere Shaping in combination with the LDPC codes for 802.11. Nr. ofreal dimensions 162 and 486. Results:
51/55 July 21, 2018
LABT
IC
Approximate Enumerative Sphere Shaping
Gultekin, W., van Houtum, and Serbetli, IEEE ISIT, Vail, Colorado [2018]:
Rounding for Enumerative Sphere Shaping:
Rounding for Shell-Mapping (D & C) Shaping:
Proof that Rounded Trellis Still Gives Unique Decodability.
52/55 July 21, 2018
LABT
IC
Approximate Enumerative Sphere Shaping
Rate-Loss due to rounding:
53/55 July 21, 2018
LABT
IC
REMARK: Shaping Combined with Coded Modulation (Set Partitioning)
W. and Wuijts, IEEE Symp. Comm. Vehic. Techn. Benelux [1993]:
Enumerative Sphere Shaping is combined with the coded modulation techniques asdescribed in “Pragmatic Coded Modulation”, by Viterbi, Wolf, Zehavi, and Padovani[1989].
54/55 July 21, 2018
LABT
IC
REFERENCES (BASIC)
Forney, Gallager, Lang, Longstaff, and Qureshi [1984]: constellation shaping, codingand shaping can be separated.Kschischang and Pasupathy [1990]: variable-rate shaping, geometric shaping.Calderbank and Ozarow [1991]: shaping on regions.Forney [1992]: trellis shaping (more codewords for same data, choose lowest energycodeword), sign-bit shaping, constellation expansion, peak-to-average power ratioexpansion.Sun and van Tilborg [1993]: geometrical shaping.Laroia, Farvardin, and Tretter [1994]: enumerative shaping (two schemes).V34 Modem standard [1994]: shell mapping, enumerative shaping.Fischer [2002]: overview.
55/55 July 21, 2018