Finite-blocklength schemes in information theory

Finite-blocklength schemes in information theory

Li Cheuk Ting

Department of Information Engineering, The Chinese University of Hong Kong

[email protected]

Part of this presentation is based on my lecture notes for Special Topics in Information Theory

Overview

• In this talk, we study an unconventional approach to code construction• An alternative to conventional random coding

• Gives tight one-shot/finite-blocklength/asymptotic results

• Very simple (proof of Marton’s inner bound for broadcast channel can be written in one slide!)

• Apply to channel coding, channel with state, broadcast channel, multiple access channel, lossy source coding (with side information), etc

How to measure information?

How to measure information?

• How many bits are needed to store a piece of information?• E.g. We can use one bit to represent whether it will rain

tomorrow

• In general, to represent 𝑘 possibilities, need ⌈log2 𝑘⌉ bits

• How much information does “it will rain tomorrow” really contain?• For a place that always rains, this contains no

information

• The less likely it will rain, the more information (“surprisal”) it contains

Self-information

• For probability mass function 𝑝𝑋 of random variable 𝑋, the self-information of the value 𝑥 is

𝜄𝑋 𝑥 = log1

𝑝𝑋(𝑥)

• We use log to base 2 (unit is bit)

• For joint pmf 𝑝𝑋,𝑌 of a random variables 𝑋, 𝑌,

𝜄𝑋,𝑌 𝑥, 𝑦 = log1

𝑝𝑋,𝑌(𝑥, 𝑦)

Self-information

• E.g. in English text, the most frequent letter is “e” (13%), and the least frequent letter is “z” (0.074%) (according to https://en.wikipedia.org/wiki/Letter_frequency)

• Let 𝑋 ∈ {a,… , z} be a random letter

• Have

𝜄𝑋 e = log1

0.13≈ 2.94 bits

𝜄𝑋 z = log1

0.00074≈ 10.40 bits

Self-information - Properties

• 𝜄𝑋 𝑥 ≥ 0

• If 𝑝𝑋 is the uniform distribution over [1. . 𝑘],𝜄𝑋 𝑥 = log 𝑘 for 𝑥 ∈ [1. . 𝑘]

• (Invariant under relabeling) If 𝑓 is an injective function, then 𝜄𝑓(𝑋) 𝑓(𝑥) = 𝜄𝑋 𝑥

• (Additive) If 𝑋, 𝑌 are independent,𝜄𝑋,𝑌 𝑥, 𝑦 = 𝜄𝑋 𝑥 + 𝜄𝑌 𝑦

Information spectrum

• If 𝑋 is a random variable, 𝜄𝑋 𝑋 is random as well

• Some values of 𝑋 may contain more information than others

• The distribution of 𝜄𝑋 𝑋 (or its cumulative distribution function) is called the information spectrum

• 𝜄𝑋 𝑋 is a constant if and only if 𝑋 follows a uniform distribution

• Information spectrum is a probability distribution, which can be unwieldy

• We sometimes want a single number to summarize the amount of information of 𝑋

Entropy

• The Shannon entropy

𝐻 𝑋 = 𝐻(𝑝𝑋) = 𝐄 𝜄𝑋 𝑋 =𝑥𝑝𝑋(𝑥) log

1

𝑝𝑋(𝑥)is the average of the self-information• A number (not random) that roughly corresponds to

the amount of information in 𝑋

• Treat 0 log(1/0) = 0

• Similarly the joint entropy of 𝑋 and 𝑌 is 𝐻 𝑋, 𝑌 = 𝐄 𝜄𝑋,𝑌 𝑋, 𝑌

Entropy - Properties

• 𝐻(𝑋) ≥ 0, and 𝐻 𝑋 = 0 iff 𝑋 is (almost surely) a constant

• If 𝑋 ∈ [1. . 𝑘], then 𝐻 𝑋 ≤ log 𝑘• Equality iff 𝑋 is uniform over [1. . 𝑘]• Proof: Jensen’s ineq. on concave function 𝑧 ↦ 𝑧 log(1/𝑧)

• If 𝑓 is a function, then 𝐻 𝑓(𝑋) ≤ 𝐻(𝑋)• If 𝑓 is injective, equality holds (invariant under relabeling)

• Consequences: 𝐻 𝑋, 𝑌 ≥ 𝐻(𝑋), 𝐻 𝑋, 𝑓 𝑋 = 𝐻(𝑋)

• (Subadditive) 𝐻 𝑋, 𝑌 ≤ 𝐻 𝑋 + 𝐻(𝑌)• Equality holds iff 𝑋, 𝑌 independent (additive)

• 𝐻 𝑋 is concave in 𝑝𝑋

A random English letter

• Self-information ranges from 𝜄𝑋 e ≈ 2.94 to𝜄𝑋 z ≈ 10.40

• 𝐻 𝑋 ≈ 4.18

𝑝𝑋(𝑥)

𝜄𝑋(𝑥)

(according to https://en.wikipedia.org/wiki/Letter_frequency)

Why is entropy a reasonable measure of information?

• Axiomatic characterization:𝐻 𝑋 is the only measure that satisfies • Subadditivity. 𝐻 𝑋, 𝑌 ≤ 𝐻 𝑋 + 𝐻 𝑌

• Additivity. 𝐻 𝑋,𝑌 = 𝐻 𝑋 + 𝐻 𝑌 if 𝑋,𝑌 independent

• Invariant under relabeling and adding a zero mass

• 𝐻 𝑋 is continuous in 𝑝𝑋• 𝐻 𝑋 = 1 when 𝑋~Unif{0,1}

[Aczél, J., Forte, B., & Ng, C. T. (1974). Why the Shannon and Hartley entropies are 'natural’]

• Operational characterizations:• 𝐻 𝑋 is approximately the number of coin flips needed to

generate 𝑋 [D. E. Knuth & A. C. Yao. (1976). The complexity of nonuniform random number generation]

• 𝐻 𝑋 is approximately the number of bits needed to compress 𝑋

Information density

• The information density between two random variables 𝑋, 𝑌 is

𝜄𝑋;𝑌 𝑥; 𝑦 = 𝜄𝑌 𝑦 − 𝜄𝑌|𝑋 𝑦 𝑥

= log𝑝𝑋,𝑌 𝑥, 𝑦

𝑝𝑋 𝑥 𝑝𝑌 𝑦= log

𝑝𝑌|𝑋(𝑦|𝑥)

𝑝𝑌(𝑦)• 𝜄𝑌 𝑦 is the info of 𝑌 = 𝑦 without knowing 𝑋 = 𝑥

• 𝜄𝑌|𝑋 𝑦 𝑥 is the info of 𝑌 = 𝑦 after knowing 𝑋 = 𝑥

• 𝜄𝑋;𝑌 𝑥; 𝑦 measures how much knowing 𝑋 = 𝑥 reduces the info of 𝑌 = 𝑦

• Can be positive/negative/zero

• Zero if 𝑋, 𝑌 independent

Information density

• 𝜄𝑋;𝑌 𝑥; 𝑦 = 𝜄𝑌 𝑦 − 𝜄𝑌|𝑋 𝑦 𝑥

= log𝑝𝑋,𝑌 𝑥,𝑦

𝑝𝑋 𝑥 𝑝𝑌 𝑦= log

𝑝𝑌|𝑋(𝑦|𝑥)

𝑝𝑌(𝑦)

• E.g. 𝑋, 𝑌 are the indicators of whether it rains today/tomorrow resp., with the following prob. matrix

• 𝜄𝑋;𝑌 1; 1 = log0.2

0.3⋅0.3≈ 1.15

• Knowing it rains today decreases the info of “tomorrow will rain”

• 𝜄𝑋;𝑌 1; 0 = log0.1

0.3⋅0.7≈ −1.07

• Knowing it rains today increases the info of “tomorrow will not rain”

𝑌 = 0 𝑌 = 1

𝑋 = 0 0.6 0.1

𝑋 = 1 0.1 0.2

Mutual information

• The mutual information between two random variables 𝑋, 𝑌 is

𝐼 𝑋; 𝑌 = 𝐄 𝜄𝑋;𝑌 𝑋; 𝑌

= 𝐄 log𝑝𝑋,𝑌 𝑋, 𝑌

𝑝𝑋 𝑋 𝑝𝑌 𝑌= 𝐻 𝑌 − 𝐻 𝑌 𝑋= 𝐻 𝑋 + 𝐻 𝑌 − 𝐻(𝑋, 𝑌)

• Always nonnegative since 𝐻 𝑌 ≥ 𝐻 𝑌 𝑋

• Measures the dependency between 𝑋, 𝑌• Zero iff 𝑋, 𝑌 independent

Source coding & channel coding

• Source coding: compressing a source 𝑋~𝑝𝑋

• Channel coding: transmitting a message 𝑀 through a noisy channel

Enc𝑀~Unif{1,… , 𝑘}𝑋

Dec 𝑀Channel𝑝𝑌|𝑋

𝑌

Enc𝑀 ∈ {1,… , 𝑘}

𝑋~𝑝𝑋 Dec 𝑋

One-shot channel coding

• Message 𝑀~Unif{1,… , 𝑘}

• Encoder maps message to channel input 𝑋 = 𝑓(𝑀)• The set 𝒞 = 𝑓 𝑚 :𝑚 ∈ 1,… , 𝑘 is the codebook

• Its elements 𝑓 𝑚 are called codewords

• Channel output 𝑌 follows conditional distribution 𝑝𝑌|𝑋

• Decoder maps 𝑌 to decoded message 𝑀 = 𝑔(𝑌)

• Goal: error prob 𝐏( 𝑀 ≠ 𝑀) is small



𝑌


• Want 𝐏 𝑀 ≠ 𝑀 ≤ 𝜖

Thm [Yassaee et al. 2013]. Fix any 𝑝𝑋. There exists code with

𝐏 𝑀 ≠ 𝑀 ≤ 1 − 𝐄1

1 + 𝑘2−𝜄𝑋;𝑌 𝑋;𝑌

≤ 𝐄 min{𝑘2−𝜄𝑋;𝑌 𝑋;𝑌 , 1}where 𝑋, 𝑌 ~𝑝𝑋𝑝𝑌|𝑋

[Yassaee, Aref, and Gohari, "A technique for deriving one-shot achievability results in network information theory," ISIT 2013.]



𝑌


• Random codebook generation: generate𝑓 𝑚 ~𝑝𝑋 i.i.d. for 𝑚 ∈ {1,… , 𝑘}

Given 𝑌, the decoder:

• (Maximum likelihood decoder) Find ෝ𝑚 that maximizes 𝑝𝑌|𝑋(𝑌|𝑓 ෝ𝑚 )• Optimal – attains the lowest error prob. for a fixed 𝑓

• (Stochastic likelihood decoder) Chooses ෝ𝑚 with prob.

𝐏 ෝ𝑚 𝑌 =𝑝𝑌|𝑋(𝑌|𝑓 ෝ𝑚 )

σ𝑚′ 𝑝𝑌|𝑋(𝑌|𝑓 𝑚′ )=

2𝜄𝑋;𝑌(𝑓 ෝ𝑚 ;𝑌)

σ𝑚′ 2𝜄𝑋;𝑌(𝑓 𝑚′ ;𝑌)

[Yassaee-Aref-Gohari 2013]

• 𝐏 ෝ𝑚 𝑌 =2𝜄𝑋;𝑌(𝑓 ෞ𝑚 ;𝑌)

σ𝑚′ 2

𝜄𝑋;𝑌(𝑓 𝑚′ ;𝑌)[Yassaee-Aref-Gohari 2013]

𝐏 𝑀 = 𝑀

= 𝐄𝒞1

𝑘σ𝑚,𝑦𝑝𝑌|𝑋(𝑦|𝑓(𝑚))

2𝜄𝑋;𝑌(𝑓 𝑚 ;𝑦)

σ𝑚′ 2

𝜄𝑋;𝑌(𝑓 𝑚′ ;𝑦)

= 𝐄𝒞 σ𝑦𝑝𝑌|𝑋(𝑦|𝑓(1))2𝜄𝑋;𝑌(𝑓 1 ;𝑦)

σ𝑚′ 2

𝜄𝑋;𝑌(𝑓 𝑚′ ;𝑦)(Symmetry)

= σ𝑦𝐄𝑓(1)𝐄𝑓 2 ,…,𝑓(𝑘) 𝑝𝑌|𝑋(𝑦|𝑓(1))2𝜄𝑋;𝑌(𝑓 1 ;𝑦)

2𝜄𝑋;𝑌(𝑓 1 ;𝑦)+σ𝑚′≠1

2𝜄𝑋;𝑌(𝑓 𝑚′ ;𝑦)

≥ σ𝑦 𝐄𝑓(1) 𝑝𝑌|𝑋(𝑦|𝑓(1))2𝜄𝑋;𝑌(𝑓 1 ;𝑦)

2𝜄𝑋;𝑌(𝑓 1 ;𝑦)+𝑘−1(Jensen)

≥ σ𝑦 𝐄𝑓(1) 𝑝𝑌|𝑋(𝑦|𝑓(1))1

1+𝑘2−𝜄𝑋;𝑌(𝑓 1 ;𝑦)

= σ𝑦σ𝑥 𝑝𝑋(𝑥) 𝑝𝑌|𝑋(𝑦|𝑥)1

1+𝑘2−𝜄𝑋;𝑌(𝑥;𝑦)

= 𝐄1

1+𝑘2−𝜄𝑋;𝑌 𝑋;𝑌

Asymptotic channel coding

• Memoryless: 𝑝𝑌𝑛|𝑋𝑛 𝑦𝑛 𝑥𝑛 = ς𝑖=1𝑛 𝑝𝑌|𝑋(𝑦𝑖|𝑥𝑖)

• Applying one-shot:

𝑃𝑒 = 𝐏 𝑀 ≠ 𝑀 ≤ 𝐄 min{2𝑛𝑅−σ𝑖=1𝑛 𝜄𝑋;𝑌 𝑋𝑖;𝑌𝑖 , 1} ,

where 𝑋𝑖 , 𝑌𝑖 ~𝑝𝑋𝑝𝑌|𝑋 i.i.d. for 𝑖 = 1, … , 𝑛

• Asymptotic (𝑛 → ∞): haveσ𝑖=1𝑛 𝜄𝑋;𝑌 𝑋𝑖; 𝑌𝑖 ≈ 𝑛𝐼(𝑋; 𝑌) by law of large numbers,

so 𝑃𝑒 → 0 if 𝑅 < 𝐼 𝑋; 𝑌

• Recovers (achievability part of) Shannon’s channel coding theorem: Channel capacity is

𝐶 = max𝑝𝑋

𝐼(𝑋; 𝑌)

Enc𝑀~Unif{1,… , 2𝑛𝑅}𝑋𝑛


𝑌𝑛

Codebook as a black box

• Random codebook: 𝒞 = {𝑓 𝑚 }~𝑝𝑋 i.i.d. for 𝑚 ∈ {1,… , 𝑘}

• Decoder: Find ෝ𝑚 = argmax 𝑝𝑋|𝑌 𝑓 ෝ𝑚 𝑌 /𝑝𝑋(𝑓 ෝ𝑚 )

• Treat codebook 𝒞 as a box:• Operation 1: Query 𝑀, get 𝑋~𝑝𝑋• Operation 2: Query posterior distribution 𝑝𝑋|𝑌, get 𝑀



𝑌

Box

𝑀 𝑋~𝑝𝑋 𝑝𝑋|𝑌 𝑀

A general black box

• Consider random variable 𝑈

• Only one operation: Query distribution 𝑄, get 𝑈~𝑄

• Want box to have “memory”• If we query the same 𝑄 twice, should get the same 𝑈

• If we query similar 𝑄1, 𝑄2, then 𝑈1, 𝑈2 are equal with high probability

Magic box!𝑄 𝑈~𝑄

Using the general black box

• Let 𝑈 = (𝑋,𝑀)

• Encoding: Query 𝑄 = 𝑃𝑋 × 𝛿𝑚 (𝛿𝑚 is degenerate distribution 𝐏 𝑀 = 𝑚 = 1), get (𝑋,𝑚)

• Decoding: Query 𝑄 = 𝑃𝑋|𝑌 × 𝑃𝑀 (𝑃𝑀 is Unif{1,… , 𝑘}), get ( 𝑋, ෝ𝑚)

• Input partial knowledge into box, get full knowledge



𝑌

Magic box about (𝑋,𝑀)

𝑄 = 𝑃𝑋 × 𝛿𝑚 𝑋~𝑝𝑋 𝑄 = 𝑝𝑋|𝑌 × 𝑃𝑀 𝑀

How to build the box

• Operation: Query distribution 𝑄, get 𝑈~𝑄• Memory: If we query similar 𝑄1, 𝑄2, then 𝑈1, 𝑈2 are equal

with high probability

• Attempt 1: Generate 𝑈~𝑄 afresh for each query?• Does not have memory!

• Attempt 2: Generate random seed 𝑍 at the beginning, then use the same seed to generate all 𝑈~𝑄 ?• Only guarantees to give the same 𝑈 for the same 𝑄• No guarantee for similar but different 𝑄1, 𝑄2

• Need a way to generate 𝑈 that is not sensitive to small changes to 𝑄


How to build the box• Generate random seed 𝑍 at the beginning, then use

the same seed to generate all 𝑈~𝑄 ?

• Exponential distribution with rate 𝜆Exp(𝜆) has prob. density function

𝑓 𝑧; 𝜆 = 𝜆𝑒−𝜆𝑧 for 𝑧 ≥ 0• If 𝑍~Exp(𝜆), then 𝑎𝑍~Exp(𝜆/𝑎)

• For 𝑍𝑖~Exp(𝜆𝑖) indep. for 𝑖 = 1, … , 𝑙, have

𝐏 argmin𝑖𝑍𝑖 = 𝑗 =𝜆𝑗

𝜆1 +⋯+ 𝜆𝑙

• Let 𝑍 = 𝑍1, … , 𝑍𝑙 be the seed, 𝑍𝑢~Exp(1) i.i.d.

• Query 𝑄, output 𝑈 = argmin𝑢𝑍𝑢

𝑄(𝑢)


C. T. Li and A. El Gamal. Strong functional representation lemma and applications to coding theorems. IEEE Trans. Inf. Theory, 64(11):6967–6978, 2018.

C. T. Li and V. Anantharam, "A Unified Framework for One-Shot Achievability via the Poisson Matching Lemma," IEEE Trans. Inf. Theory, vol. 67, no. 5,

pp. 2624-2651, 2021.


• Let 𝑍 = 𝑍1, … , 𝑍𝑙 be the seed, 𝑍𝑖~Exp(1) i.i.d.


𝑄(𝑢)

• 𝐏 𝑈 = 𝑢 =𝑄(𝑢)

𝑄(1)+⋯+𝑄(𝑙)= 𝑄(𝑢) OK!

• Give same 𝑈 for same 𝑄 since 𝑈 is a function of 𝑄 and 𝑍 (fixed at the beginning) OK!

• Small changes to 𝑄 is unlikely to affect

argmin𝑢𝑍𝑢

𝑄(𝑢)OK!





𝑄(𝑢)

• If 𝑙 = 2, then 𝑈 = 1 iff𝑍1

𝑄(1)<

𝑍2

𝑄(2)⇔

𝑍1

𝑍1+𝑍2< 𝑄(1)


𝑍1𝑍1 + 𝑍2

0

1

𝑄 1 = 𝐏𝑋~𝑄(𝑋 = 1)

𝑈 = 1

1

𝑈 = 2

Poisson matching lemma


• Query 𝑄, output 𝑈𝑄 = argmin𝑢𝑍𝑢

𝑄(𝑢)

• Poisson matching lemma [Li-Anantharam 2018]:If we query 𝑃, 𝑄 to get 𝑈𝑃, 𝑈𝑄 respectively, then

𝐏 𝑈𝑄 ≠ 𝑈𝑃 𝑈𝑃 ≤𝑃(𝑈𝑃)

𝑄(𝑈𝑃)

C. T. Li and A. El Gamal. Strong functional representation lemma and applications to coding theorems. IEEE Trans. Inf. Theory, 64(11):6967–6978, 2018.

C. T. Li and V. Anantharam, "A Unified Framework for One-Shot Achievability via the Poisson Matching Lemma," IEEE Trans. Inf. Theory, vol. 67, no. 5,

pp. 2624-2651, 2021.

A general black box

• Operation: Query distribution 𝑄, get 𝑈~𝑄

• Guarantee: If we query 𝑃, 𝑄 to get 𝑈𝑃, 𝑈𝑄respectively, then

𝐏 𝑈𝑄 ≠ 𝑈𝑃 𝑈𝑃 ≤𝑃(𝑈𝑃)

𝑄(𝑈𝑃)

• We can use this box alone to prove many tight one-shot/finite-blocklength/asymptotic coding results


• Let 𝑈 = (𝑋,𝑀)

• Encoding: Query 𝑄 = 𝑃𝑋 × 𝛿𝑀, get (𝑋,𝑀)

• Decoding: Query 𝑄 = 𝑃𝑋|𝑌 × 𝑃𝑀, get ( 𝑋, 𝑀)

• Poisson matching lemma:𝐏 𝑀 ≠ 𝑀 ≤ 𝐄 𝐏 𝑀 ≠ 𝑀 𝑀,𝑋, 𝑌

≤ 𝐄 min(𝑃𝑋×𝛿𝑀)(𝑋,𝑀)

(𝑃𝑋|𝑌×𝑃𝑀)(𝑋,𝑀), 1

= 𝐄 min𝑃𝑋(𝑋)

𝑃𝑋|𝑌(𝑋|𝑌)/𝑘, 1

= 𝐄 min 𝑘2−𝜄𝑋;𝑌(𝑋;𝑌), 1



𝑌

Magic box about (𝑋,𝑀)

𝑄 = 𝑃𝑋 × 𝛿𝑀 𝑋~𝑝𝑋 𝑄 = 𝑃𝑋|𝑌 × 𝑃𝑀 𝑀

C. T. Li and V. Anantharam, "A Unified Framework for

One-Shot Achievability via the Poisson Matching Lemma,"

IEEE Trans. Inf. Theory, vol. 67, no. 5, pp. 2624-2651, 2021.

Channel coding – removing the box

• The box contains a random seed in it

• In reality, encoder and decoder cannot share common randomness

• 𝑃𝑒 ≤ 𝐄 min 𝑘2−𝜄𝑋;𝑌(𝑋;𝑌), 1 averaged over choices of seed

• There exists fixed seed s.t. 𝑃𝑒 ≤ 𝐄 min 𝑘2−𝜄𝑋;𝑌(𝑋;𝑌), 1



𝑌

Fixed box about (𝑋,𝑀)

𝑄 = 𝑃𝑋 × 𝛿𝑀 𝑋~𝑝𝑋 𝑄 = 𝑃𝑋|𝑌 × 𝑃𝑀 𝑀

Second-order asymptotics

• 𝑃𝑒 ≤ 𝐄 min 2𝐿−σ𝑖=1𝑛 𝜄𝑋;𝑌 𝑋𝑖;𝑌𝑖 , 1 , 𝑋𝑖 , 𝑌𝑖 ~𝑝𝑋𝑝𝑌|𝑋 i.i.d.

• 𝑃𝑒 ≈ 0 if 𝐿 ≪ σ𝑖=1𝑛 𝜄 𝑋𝑖; 𝑌𝑖 , 𝑃𝑒 ≈ 1 if 𝐿 ≫ σ𝑖=1

𝑛 𝜄 𝑋𝑖; 𝑌𝑖• First-order: optimal 𝐿 ≈ 𝑛𝐼(𝑋; 𝑌)

• Central limit theorem:σ𝑖=1𝑛 𝜄 𝑋𝑖; 𝑌𝑖 approximately follows 𝑁(𝑛𝐼 𝑋; 𝑌 , 𝑛𝑉), where

𝑉 = Var[𝜄 𝑋; 𝑌 ]

• For a fixed 𝑃𝑒 = 𝜖, optimal 𝐿 ≈ 𝑛𝐼 𝑋; 𝑌 − 𝑛𝑉𝑄−1 𝜖where 𝑄−1 𝜖 is the inverse of the Q-function(𝑄 𝛾 = 1 − Φ(𝛾), Φ is the cdf of 𝑁(0,1))

• The 𝑉 when 𝑝𝑋 is the capacity-achieving distribution (that maximizes 𝐼(𝑋; 𝑌)) is called the channel dispersion

Enc𝑀~Unif{1,… , 2𝐿}𝑋𝑛


𝑌𝑛

Y. Polyanskiy, H. V. Poor, and S. Verdú, “Channel coding rate in the finite blocklength regime,” IEEE Transactions on Information Theory, vol. 56, no.

5, pp. 2307–2359, 2010.

σ𝑖=1𝑛 𝜄 𝑋𝑖; 𝑌𝑖

𝑛𝐼(𝑋; 𝑌)

𝛾 𝑛𝑉

Fixed error prob.

cutoff point

(second order)

𝑠𝑑 =

𝑛𝑉

Error prob. ≈𝐏(σ𝑖=1

𝑛 𝜄 𝑋𝑖; 𝑌𝑖 ≤ 𝐿)

Date post:	17-Jan-2022
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Finite-blocklength schemes in information theory

Documents