of 17July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 1

Communication Complexity of Randomness Manipulation

Madhu SudanHarvard University

Joint works with Mitali Bafna (Harvard), Badih Ghazi (Google), and Noah Golowich (Harvard)

of 17

Randomness Manipulation

Classical problem in CS Theory Von Neumann ‘50s: Given coin of unknown

bias, simulate unbiased coin! Blum ’70s: Markovian coins. Santha-Vazirani:

Pr 𝑋𝑋𝑛𝑛 = 1 | 𝑋𝑋<𝑛𝑛 = 𝑎𝑎 = 𝑝𝑝𝑎𝑎 ; 𝛿𝛿 ≤ 𝑝𝑝𝑎𝑎 ≤ 1 − 𝛿𝛿 Nisan-Zuckerman: Unknown source with lower

bound on “min”-entropy. Untold #works: Exceptionally successful in converting all

the above to pure, independent randomness with little loss and explicit efficient algorithms.

Emphasis: Single unknown source.

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 2

1. Toss two coins2. If 𝐻𝐻𝐻𝐻 output 𝐻𝐻3. If 𝐻𝐻𝐻𝐻 output 𝐻𝐻4. Else go back to 1.

of 17

2-player setting:

Is 𝛿𝛿 = 0 possible? If not minimize 𝛿𝛿! Etc. Alice + Bob can use private randomness Zero communication version: “Non-Interactive

Simulation” (NIS).

Distributed Randomness Processing

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 3

Alice 𝑈𝑈

Bob𝑌𝑌 𝑉𝑉

≤ 𝐶𝐶 bits, ≤ 𝑟𝑟 Rounds

𝑋𝑋𝑋𝑋1,𝑋𝑋2, …

𝑌𝑌1,𝑌𝑌2, …

𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃 Goal: 𝑈𝑈,𝑉𝑉 ≈𝛿𝛿 𝑄𝑄

Infinite sequence

of 17

A classical example

Zero communication 𝑋𝑋,𝑌𝑌 ∼ Unif 0,0 , 0,1 , 1,0 𝑈𝑈,𝑉𝑉 ∼ 𝜌𝜌-correlated bits.

𝑈𝑈,𝑉𝑉 ∼ Unif 0,1 , Pr 𝑈𝑈 = 𝑉𝑉 = 1+𝜌𝜌2

[Witsenhausen ‘70s]: Can’t achieve 𝜌𝜌 = 1. Not even 𝜌𝜌 = .51

(Naïve strategy achieves 𝜌𝜌 = 14)

Can achieve 𝜌𝜌 = 13

by a general method.

Best 𝜌𝜌? Open!!

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 4

of 17

Witsenhausen …

Intermediate Problem: Output (𝐴𝐴,𝐵𝐵) w. 𝐴𝐴,𝐵𝐵 ∼ 𝑁𝑁 0,1with maximum correlation.

Definition: Max-Corr 𝜌𝜌 𝑋𝑋,𝑌𝑌 ≜ max𝑓𝑓,𝑔𝑔

𝔼𝔼𝑋𝑋,𝑌𝑌 𝑓𝑓 𝑋𝑋 𝑔𝑔(𝑌𝑌)

Where 𝑓𝑓,𝑔𝑔:Ω → ℝ,𝔼𝔼𝑋𝑋 𝑓𝑓 𝑋𝑋 = 𝔼𝔼𝑌𝑌 𝑔𝑔 𝑌𝑌 = 0𝔼𝔼𝑋𝑋 𝑓𝑓 𝑋𝑋 2 = 𝔼𝔼𝑌𝑌 𝑔𝑔 𝑌𝑌 2 = 1

Thm 1: For any 𝑈𝑈,𝑉𝑉 output, 𝜌𝜌 𝑈𝑈,𝑉𝑉 ≤ 𝜌𝜌(𝑋𝑋,𝑌𝑌) Thm 2: For Gaussian output (𝐴𝐴,𝐵𝐵), can achieve

𝜌𝜌 𝐴𝐴,𝐵𝐵 = 𝜌𝜌(𝑋𝑋,𝑌𝑌) Thm 3: For binary 𝑈𝑈,𝑉𝑉 , can achieve

1 −2 ⋅ cos−1 𝜌𝜌 𝑋𝑋,𝑌𝑌

𝜋𝜋≤ 𝜌𝜌 𝑈𝑈,𝑉𝑉 ≤ 𝜌𝜌 𝑋𝑋,𝑌𝑌

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 5

For Bernoulli/GaussiansMax-Correlation = Correlation

of 17

“Tensorization”

Amortized Problems: Given (1) structure 𝑆𝑆 (graph, distribution, game) (2) product operation 𝑆𝑆⊗𝑡𝑡, (3) amortized measure 𝑀𝑀, compute lim

𝑡𝑡→∞𝑀𝑀 𝑆𝑆⊗𝑡𝑡

Shannon capacity, Compression length, Channel capacity, parallel repetition value of 2-prover game, Direct sum complexity, NIS

“Tensorizing bound”: Typically: 𝑀𝑀 𝑆𝑆⊗𝑡𝑡 ≥ 𝑀𝑀(𝑆𝑆). Find: ℳ(⋅) s.t. 𝑀𝑀 𝑆𝑆 ≤ ℳ(𝑆𝑆) and ℳ 𝑆𝑆⊗𝑡𝑡 = ℳ(𝑆𝑆)

Max-Correlation “Tensorizes” If 𝑋𝑋,𝑌𝑌 ∼ 𝜇𝜇 & 𝑋𝑋𝑡𝑡,𝑌𝑌𝑡𝑡 ∼ 𝜇𝜇⊗𝑡𝑡 then 𝜌𝜌 𝑋𝑋𝑡𝑡 ,𝑌𝑌𝑡𝑡 = 𝜌𝜌(𝑋𝑋,𝑌𝑌) Proof idea: 𝜇𝜇 described by matrix 𝑃𝑃; 𝜌𝜌 related to its

singular values, 𝜇𝜇⊗𝑡𝑡 described by 𝑃𝑃⊗𝑡𝑡

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 6

of 17

Problem Single-shot Amortized

Compressibility P P

Channel capacity NP P

Problem Single-shot Amortized

Compressibility P P

Channel capacity NP P

Independent set/Shannon capacity

NP [0,∞]

Value of 2-prover game NP [NP,∞]

Problem Single-shot Amortized

Compressibility P P

Channel capacity NP P

Independent set/Shannon capacity

NP [0,∞]

Value of 2-prover game NP [NP,∞]

Non-interactive-simulation (“correlation”)

NP [0,𝐶𝐶𝐴𝐴]

Communication Complexity [NP,Exp?] 0,𝐶𝐶𝐴𝐴

Problem Single-shot Amortized

Compressibility P P

Channel capacity NP P

Independent set/Shannon capacity

NP [0,∞]

Value of 2-prover game NP [NP,∞]

Non-interactive-simulation (“correlation”)

NP [0,𝐶𝐶𝐴𝐴]

Communication Complexity [NP,Exp?] 0,𝐶𝐶𝐴𝐴

? P [NP,∞]

Amortized Embarassment

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 7

Glossary: 0 ≤ P ≤ 𝑁𝑁𝑃𝑃 ≤ EXP ≤ Computable≤ CA Computably Approximable ≤ ∞

of 17

Complexity of NIS

“Computably approximable” [Ghazi,Kamath,S.], [De,Mossel,Neeman]2,

[Ghazi,Kamath,Raghavendra] There exists a finite time algorithm determining if we can

get 𝜖𝜖-close to (𝑈𝑈,𝑉𝑉) from i.i.d. samples of (𝑋𝑋,𝑌𝑌)

Idea: “Invariance Priniciple” [Mossel] Either there is a method using few samples of (𝑋𝑋,𝑌𝑌) or

many samples 𝑋𝑋1,𝑌𝑌1 , … , 𝑋𝑋𝑛𝑛,𝑌𝑌𝑛𝑛 each with low influence on 𝑈𝑈,𝑉𝑉 .

If latter, can replace 𝑋𝑋,𝑌𝑌 by finite # Gaussians. Gives computable upper bound on #samples needed to

get close to optimal strategy.

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 8

of 17

Communication?

Restrict to “Common Randomness Generation” (𝑘𝑘-CRG): Output (𝑈𝑈 = 𝑉𝑉), with 𝑈𝑈 ∼ Unif( 0,1 𝑘𝑘)

First considered ~70 [Ahlswede-Csizar]: “Characterization”: Can communicate 𝑅𝑅 ⋅ 𝑘𝑘

+ 𝑜𝑜 𝑘𝑘 bits, where 𝑅𝑅 = minΠ

ICint(Π)ICext Π

Computable? Computably approximable? For one-way communication: 𝑅𝑅 = 1 − 𝜌𝜌 𝑋𝑋,𝑌𝑌 2

[Zhao-Chia] (see also [Guruswami-Radhakrishnan, Ghazi-Jayram, S.-Tyagi-Watanabe])

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 9

Alice 𝑈𝑈

Bob 𝑉𝑉

≤ 𝐶𝐶 bits, ≤ 𝑟𝑟 Rounds

𝑋𝑋1,𝑋𝑋2, …

𝑌𝑌1,𝑌𝑌2, …

𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃 Goal: 𝑈𝑈,𝑉𝑉 ≈𝛿𝛿 𝑄𝑄

of 17

This talk: Rounds in CRG

Does increasing #rounds ease CRG? (For some (𝑋𝑋,𝑌𝑌)?) … Is the following true?

Question: ∀𝜖𝜖 > 0 , 𝑟𝑟, ∃(𝑋𝑋,𝑌𝑌) s.t. ∀𝑘𝑘 ∃ 𝑟𝑟, 𝜖𝜖𝑘𝑘 -protocol for 𝑘𝑘-CRG 𝑋𝑋,𝑌𝑌 No 𝑟𝑟 − 1, 1 − 𝜖𝜖 .𝑘𝑘 -protocol for 𝑘𝑘-CRG(𝑋𝑋,𝑌𝑌)

Still don’t know. Partial progress. Thm. [BGGS’19] : ∀𝑛𝑛,𝑘𝑘, 𝑟𝑟 there exists 𝑋𝑋,𝑌𝑌 s.t.

∃ 𝑟𝑟,𝑂𝑂(𝑟𝑟 log 𝑛𝑛 -protocol for 𝑘𝑘-CRG(𝑋𝑋,𝑌𝑌).

No 𝑟𝑟2− 3, min 𝑘𝑘, 𝑛𝑛

polylog 𝑛𝑛-protocol for 𝑘𝑘-CRG(𝑋𝑋,𝑌𝑌)

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 10

of 17

Our Source:

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 11

𝑖𝑖0

𝜋𝜋1

𝜋𝜋2

𝜋𝜋3

𝜋𝜋𝑟𝑟

⋮

𝐴𝐴1 𝐴𝐴2 𝐴𝐴3 𝐴𝐴𝑛𝑛 𝐵𝐵1 𝐵𝐵2 𝐵𝐵3 𝐵𝐵𝑛𝑛… …

Alice’s Inputs𝑋𝑋 = (𝜋𝜋1,𝜋𝜋3, … ;𝐴𝐴1, … ,𝐴𝐴𝑛𝑛)

Bob’s Inputs𝑌𝑌 = (𝑖𝑖0;𝜋𝜋2,𝜋𝜋4, … ;𝐵𝐵1, … ,𝐵𝐵𝑛𝑛)

𝑖𝑖0 ∼ [𝑛𝑛]

𝜋𝜋1, … ,𝜋𝜋𝑛𝑛 ∼ 𝑆𝑆𝑛𝑛𝐴𝐴1, … ,𝐴𝐴𝑛𝑛 ∼ 0,1 𝑘𝑘

𝐵𝐵1, … ,𝐵𝐵𝑛𝑛 ∼ 0,1 𝑘𝑘

𝐴𝐴𝑖𝑖𝑟𝑟 = 𝐵𝐵𝑖𝑖𝑟𝑟where 𝑖𝑖𝑗𝑗 = 𝜋𝜋𝑗𝑗 𝑖𝑖𝑗𝑗−1

“Pointer Chasing Source” (PCS)

of 17

CRG from Pointer Chasing Source

Easy direction easy: (𝑟𝑟, 𝑟𝑟 log𝑛𝑛)-protocol for 𝑘𝑘-CRG Hardness?

Pointer chasing is hard: [Duris-Galil-Schnitger, Nisan-Wigderson,…].

CRG from PCS requires pointer chasing? No! Can solve problem without pointers! Small non-deterministic complexity! Hardness needs to use hardness of

disjointness? And of pointer-chasing? How to combine modularly?

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 12

of 17

Our solution: Pointer Verification Problem

PVP: Distinguish 𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃𝑉𝑉YES from 𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃𝑉𝑉No 𝑃𝑃𝑉𝑉YES:𝑋𝑋 = 𝜋𝜋1,𝜋𝜋3, …𝜋𝜋𝑟𝑟−1 ; 𝑌𝑌 = 𝑖𝑖0, 𝑖𝑖𝑟𝑟; 𝜋𝜋2,𝜋𝜋4, …𝜋𝜋𝑟𝑟 𝑃𝑃𝑉𝑉NO:𝑋𝑋 = 𝜋𝜋1,𝜋𝜋3, …𝜋𝜋𝑟𝑟−1 ; 𝑌𝑌 = 𝑖𝑖0, 𝑖𝑖𝑟𝑟′ ; 𝜋𝜋2,𝜋𝜋4, …𝜋𝜋𝑟𝑟

YES instance satisfy: 𝑖𝑖𝑗𝑗 = 𝜋𝜋𝑗𝑗 𝑖𝑖𝑗𝑗−1 NO instance: 𝑖𝑖𝑟𝑟′ random

Claims: Hardness of (Unique) Set Disjointness ⇒

Protocol for 𝑘𝑘-CRG(PCS) solves PVP.

No 𝑟𝑟2− 𝑂𝑂 1 , 𝑛𝑛

polylog 𝑛𝑛-protocol for PVP.

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 13

of 17

PVP ≤ CRG

Let 𝜇𝜇𝑋𝑋𝑌𝑌 denote dist. of PCS source. 𝜇𝜇𝑋𝑋, 𝜇𝜇𝑌𝑌 marginals

CRG Hard ⇐ 𝜇𝜇𝑋𝑋𝑌𝑌 indist. from 𝜇𝜇𝑋𝑋 × 𝜇𝜇𝑌𝑌 (to 𝑟𝑟2

,𝐶𝐶 -protocols)

Intermediate distribution: 𝜇𝜇mid 𝑋𝑋 = (𝜋𝜋odd,𝐴𝐴1, … ,𝐴𝐴𝑛𝑛); 𝑌𝑌 = 𝑖𝑖0,𝜋𝜋even,𝐵𝐵1, … ,𝐵𝐵𝑛𝑛 ,

with 𝐴𝐴𝑗𝑗 = 𝐵𝐵𝑗𝑗 for random 𝑗𝑗. (Correlation exists but pointers don’t point to it.)

Claims: 𝜇𝜇𝑋𝑋𝑌𝑌 indist. from 𝜇𝜇mid ⇐ PVP is hard 𝜇𝜇mid indist. from 𝜇𝜇𝑋𝑋 × 𝜇𝜇𝑌𝑌 ⇐ (Unique) disjointness

hard.

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 14

𝑖𝑖

⋮

of 17

Bulk of paper: PVP is hard

Main idea: “Round elimination” a la NW ’93 No black-box use No simple variant

Induction on #rounds Many invariants … roughly

𝐻𝐻 𝜋𝜋odd,𝜋𝜋even transcript) ≈ 𝑀𝑀𝑎𝑎𝑀𝑀 − 𝑂𝑂 𝐶𝐶 𝐻𝐻 𝑖𝑖𝑡𝑡 transcript) ≈ log𝑛𝑛 − 𝑜𝑜 1 𝑋𝑋 ⫫ 𝑌𝑌 | path, transcript

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 15

of 17

Conclusions

Many interesting problems in distributed randomness manipulation.

Complexity of the “Single-letter characterizations”.

Tight characterization of round complexity of CRG.

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 16

of 17

Thank You!

July 25, 2019 CMSA: Comm. Comp. Randomness Manipulation 17