Lower Bounds on the Communication of Distributed Graph Algorithms:
Progress and Obstacles
Rotem OshmanADGA 2013
Overview: Network ModelsCONGESTED CLIQUE
ASYNC MESSAGE-PASSING
LOCAL
CONGEST / general network
X
Talk Overview
I. Lower bound techniquesa. CONGEST general networks: reductions from 2-
party communication complexityb. Asynchronous message passing: reductions from
multi-party communication complexityII. Obstacles on proving lower bounds for the
congested clique
Communication Complexity
𝑋 𝑌
= ?
Example: DISJOINTNESS
𝑋⊆ {1 ,…,𝑛} 𝑌⊆ {1 ,…,𝑛}
𝑋∩𝑌=∅ ?
bitsneeded
[Kalyanasundaram and Schnitger, Razborov ’92]
DISJ :
Applying 2-Party Communication Complexity Lower Bounds
Textbook reduction:Given algorithm for solving task …
Solution for answer for DISJOINTNESS
bits
𝑌𝑋
Based on
Based on
Simulate
Example: Spanning Trees
• Setting: directed, strongly-connected network• Communication by local broadcast with
bandwidth • UIDs • Diameter 2• Question: how many rounds to find a rooted
spanning tree?
New Problem: PARTITION
• Inputs: , with the promise that
• Goal: Alice outputs ,Bob outputs such that partition .
𝑋 𝑌
The PARTITION Problem
• Trivial algorithm:– Alice sends her input to Bob– Alice outputs all tasks in her input– Bob outputs all remaining tasks
• Communication complexity: bits• Lower bound?
Reduction from DISJ to PARTITION
• Given input for DISJ :– Notice: iff – To test whether :• Try to solve PARTITION on • Ensure • Check if is a partition of : Alice sends Bob hash(), Bob
compares it to hash()
4 65
From PARTITION to Spanning Tree
a b
1 2 3
𝑋={1,2,3 } 𝑌={2,4,5,6 }
Given a spanning tree algorithm …
4 65
From PARTITION to Spanning Tree
a b
1 2 3
𝑋={1,2,3 } 𝑌={2,4,5,6 }
Simulating one round of :
Node a’s message
Node b’s message
4 65
From PARTITION to Spanning Tree
a b
1 2 3
𝑋={1,2,3 } 𝑌={2,4,5,6 }
When outputs a spanning tree:
From PARTITION to Spanning Tree
• If runs for rounds, we use bits
• One detail: randomness– Solution: Alice and Bob use public randomness
When Two Players Just Aren’t Enough
• No bottlenecks in the network
When Two Players Just Aren’t Enough
• Too much information revealed
Multi-Player Communication Complexity
• Communication by shared blackboard• Number-on-forehead• Number-in-hand
??
The Message-Passing Model
• players• Private channels• Private -bit inputs • Private randomness
• Goal: compute • Cost: total communication
The Coordinator Model
• players, one coordinator• The coordinator has no input
Message-Passing vs. Coordinator
≈
Prior Work on Message-Passing
• For players with -bit inputs…• Phillips, Verbin, Zhang ’12:– for bitwise problems (AND/OR, MAJ, …)
• Woodruff, Zhang ‘12, ‘13:– for threshold and graph problems
• Braverman, Ellen, O., Pitassi, Vaikuntanathan ‘13: for
Set Disjointness
Disj𝑛 ,𝑘 =¿ 𝑖=1¿𝑛¿ 𝑗=1¿𝑘 𝑋 𝑖𝑗¿
?
𝑋 1𝑋 2
𝑋 3
𝑋 4𝑋 5
Notation
• : randomized protocol– Also, the protocol’s transcript– : player ’s view of the transcript
• worst-case communication of
in the worst case
Entropy and Mutual Information
• Entropy:
• A lossless encoding of requires bits• Conditional entropy:
Entropy and Mutual Information
• Mutual information:
• Conditional mutual information:
Information Cost for Two Players[Chakrabarti, Shi, Wirth, Yao ’01], [Bar-Yossef, Jayram, Kumar, Sivakumar ‘04], [Braverman, Rao ‘10], …
Fix a distribution , • External information cost:
• Internal information cost:
Extension to the coordinator model:
Why is Info Complexity Nice?
• Formalizes a natural notion– Analogous to causality/knowledge
• Admits direct sum theorem:
“The cost of solving independent copies of problem is times the cost of
solving ”
Example
Example (Work in Progress)
• Triangle detection in general congested graphs• “Is there a triangle” =
”is a triangle”
Application of DISJ Lower Bound
• Open problem from Woodruff & Zhang ‘13:– Hardness of computing the diameter of a graph
• We can show: bits to distinguish diameter 3 from diameter
• Reduction from DISJ : given ,– Notice: disjoint iff
Application of DISJ Lower Bound
• Diameter • Diameter
𝑝1 𝑝3
𝑝2
𝑝4
1
32 4
5
6
𝑋 3
Part II: The Power of the Congested Clique
CONGESTED CLIQUE
Conversion from Boolean Circuit
• Suppose we have a Boolean circuit – Any type of gate, inputs– Fan-in – Depth = , #gates and wires =
• Step 1: reduce the fan-out to – Convert large fan-out gates to “copying tree”– Blowup: depth, size
• Step 2: convert to a layered circuit
Conversion from Boolean Circuit
• Now we have a layered circuit of depth and size = – With fan-in and fan-out
• Design a CONGEST protocol:– Fix partition of inputs of size each– Assign each gate to a random CONGEST node– Simulate the circuit layer-by-layer
Simulating a Layer
• If node “owns” gate on layer , it sends ’s output to the nodes that need it on layer
• Size of layer size of layer • What is the load on edge ?– For each wire from layer to layer ,
– At most wires in total– By Chernoff, w.h.p. the load is
Conversion from Boolean Circuit
• A union-bound finishes the proof• Corollary: explicit lower bounds in the
congested clique imply explicit lower bounds on Boolean circuits with polylogarithmic depth and nearly-linear size.
• Even worse:– Reasons to believe even bound hard
ConclusionCONGESTED CLIQUE
ASYNC MESSAGE-PASSING
LOCAL
CONGEST / general network
X