Date post: | 21-Dec-2015 |
Category: |
Documents |
View: | 214 times |
Download: | 0 times |
1
Time-space tradeoff lower bounds for
non-uniform computation
Paul BeameUniversity of Washington
4 July 2000
2
Why study time-space tradeoffs?
To understand relationships between the two most critical measures of computation unified comparison of algorithms with
varying time and space requirements.non-trivial tradeoffs arise frequently in
practice avoid storing intermediate results by re-
computing them
3
e.g. Sorting n integers from [1,n2]
Merge sort S = O(n log n), T = O(n log n)
Radix sort S = O(n log n), T = O(n)
Selection sort only need - smallest value output so far
- index of current element
S = O(log n) , T = O(n2)
4
Complexity theory
Hard problems prove L P prove non-trivial time lower bounds for
natural decision problems in PFirst step
Prove a space lower bound, e.g. S=(log n), given an upper bound on time T, e.g. T=O(n) for a natural problem in P
5
An annoyance
Time hierarchy theorems imply unnatural problems in P not solvable in
time O(n)
Makes ‘first step’ vacuous for unnatural problems
6
Non-uniform computation
Non-trivial time lower bounds still open for problems in P First step still very interesting even without
the restriction to natural problems
Can yield bounds with precise constants
But proving lower bounds may be harder
7
Talk outlineThe right non-uniform model (for now)
branching programs
Early success multi-output functions, e.g. sorting
Progress on problems in P Crawling
restricted branching programs
That breakthrough first step (and more)true time-space tradeoffs
The path ahead
8
Branching programs
x1
x4
x2
x3
x5x5
x3
x7
x1
x2 x8x7
1
0
10
9
Branching programs
x1
x4
x2
x3
x5x5
x3
x7
x1
x2 x8x7
1
0
10
To computef:{0,1} n {0,1}on input (x1,…,xn)follow path fromsource to sink
x=(0,0,1,0,...)
10
Branching program properties
Length = length of longest pathSize = # of nodes
Simulate TM’s node = configuration with input bits erased time T= Length space S=log2Size =TM space +log2n (head)
= space on an index TM polysize = non-uniform L
11
TM space complexity
x1 x2 x3 x4 … xn read-only input
working storage
output
Space = # of bitsof working storage
12
Branching program properties
Simulate random-access machines (RAMs) not just sequential access
Generalizations Multi-way version for xi in arbitrary domain D
good for modeling RAM input registers Outputs on the edges
good for modeling output tape for multi-output functions such as sorting
BPs can be leveled w.l.o.g. like adding a clock to a TM
13
Talk outlineThe right non-uniform model (for now)
branching programs
Early success multi-output functions, e.g. sorting
Progress on problems in P Crawling
restricted branching programs
That breakthrough first step (and more)true time-space tradeoffs
The path ahead
14
Success for multi-output problems
Sorting T S = (n2/log n) [Borodin-Cook 82] T S =(n2) [Beame 89]
Matrix-vector product T S = (n3) [Abrahamson 89]
Many others including Matrix multiplication Pattern matching
15
Proof ideas: layers and treesm outputs on input xat least m/r outputs in
some tree Tv
Only 2S trees Tv
Typical Claim if T/r = n, each tree Tv
outputs p correct answers on only a c-p fraction of inputs
Correct for all x implies 2Sc-
m/r is at least 1S=(m/r)=(mn/T)
v0
vr-1
v
v1
vr10
T
T/r
T/r
16
Limitation of the technique
Never more than T S = (nm) where m is number of outputs
“It is unfortunately crucial to our proof that sorting requires many output bits, and it remains an interesting open question whether a similar lower bound can be made to apply to a set recognition problem, such as recognizing whether all n input numbers are distinct.” [Cook: Turing Award Lecture, 1983]
17
Talk outlineThe right non-uniform model (for now)
branching programs
Early success multi-output functions, e.g. sorting
Problems in P Crawling
restricted branching programs
That breakthrough first step (and more)true time-space tradeoffs
The path ahead
18
Restricted branching programs
Constant-width - only a constant number of nodes per level
[Chandra-Furst-Lipton 83]
Read-once - every variable read at most once per path
[Wegener 84], [Simon-Szegedy 89], etc.
Oblivious - same variable queried per level[Babai-Pudlak-Rodl-Szemeredi 87],
[Alon-Maass 87], [Babai-Nisan-Szegedy 89]
BDD = Oblivious read-once
19
BDDs and best-partition communication complexity
Given f:{0,1}8->{0,1}Two-player game
Player A has {x1,x3,x6,x7}
Player B has {x2,x4,x5,x8} Goal: communicate fewest bits
possible to compute fPossible protocol: Player A sends
the name of node.BDD space # of bits sent for
best partition into A and B
10
x7
x1
x6
x3
x2
x5
x4
x8
A
B
20
Communication complexity ideas
Each conversation for f:{0,1}Ax{0,1}B {0,1} corresponds to a rectangle YAxYB of inputs YA {0,1}A YB {0,1}B
BDD lower bounds size min(A,B) # of rectangles in tiling of inputs
by f-constant rectangles with partition (A,B)
Read-once bounds same tiling as BDD bounds but each rectangle in
tiling may have a different partition
21
Restricted branching programs
Read-k - no variable queried > k times onany path - syntactic read-k[Borodin-Razborov-Smolensky 89],
[Okol’nishnikova 89], etc.
any consistent path - semantic read-kmany years of no results
nothing for general branching programs either
22
Uniform tradeoffs
SAT is not solvable using O(n1-) space if time is n1+o(1). [Fortnow 97] uses diagonalization works for co-nondeterministic TM’s
Extensions for SAT S=logO(1) n implies T= (n1.4142..-) deterministic
[Lipton-Viglas 99]
with up to no(1) advice [Tourlakis 00]
S= O(n1-) implies T=(n 1.618..-). [Fortnow-van Melkebeek 00]
23
Non-uniform computation [Beame-Saks-Thathachar FOCS 98]
Syntactic read-k branching programs exponentially weaker than semantic read-twice.
f(x) = “xTMx=0 (mod q)” x GF(q)n
nloglog n time (n log1-n) space for q~n
f(x) = “xTMx=0 (mod 3)” x {0,1}n
1.017n time implies (n) spacefirst Boolean result above time n for general branching
programs
24
Non-uniform computation
[Ajtai STOC 99] 0.5log n Hamming distance for x
[1,n2]n kn time implies (n log n) spacefollows from [Beame-Saks-Thathachar 98]improved to (nlog n) time by [Pagter-00]
element distinctness for x [1,n2]n
kn time implies (n) spacerequires significant extension of techniques
25
That breakthrough first step!
[Ajtai FOCS 99] f(x,y) = “xTMyx (mod 2)”
kn time implies (n) space
First result for non-uniform Boolean computation showing time O(n) space (log n)
x {0,1}n
y {0,1}2n-1
26
Ajtai’s Boolean function
0
y1
y2n-1y8y7y6yn
y4
y3
y2
My
f(x,y)= xTMyx (mod 2)
My is a modified Hankel matrix
27
Superlinear lower bounds
[Beame-Saks-Sun-Vee FOCS 00] Extension to -error randomized
non-uniform algorithms Better time-space tradeoffs
Apply to both element distinctness and f(x,y) = “xTMyx (mod 2)”
)(n/S)log/loglog(nT
28
(m,)-rectangles
An (m,)-rectangle R DX is a subset defined by
disjoint sets A,B X, DAUB SA DA, SB DB such that
R = { z | zAUB = , zA SA, zB SB }|A|,|B| m|SA|/|DA|, |SB|/|DB|
29
An (m,)-rectangle
SA
A B
SB
x1 xnmm
In general A and B may be interleaved in [1,n]
SA and SB each have density at least
DA
DB
SA
SB
30
Key lemma [BST 98]
Let program P use time T = kn space S accept fraction of its inputs in Dn
then P accepts all inputs in some (m,)-rectangle where m = n is at least 2-4(k+1) m - (S+1) r
-1 ~ 2k and r ~ k2 2k
31
Improved key lemma [Ajtai 99 s]
Let program P use time T = kn space S accept fraction of its inputs in Dn
then P accepts all inputs in some (m,)-rectangle where m = n is at least -1 and r are constants depending on k
Srm1/50k
2
32
Proving lower bounds using the key lemmas
Show that the desired function f evaluates to 1 a large fraction of the time
i.e., is large
evaluates to 0 on some input in any large (m,)-rectanglewhere large is given by the lemma bounds
or ... do the same for f
33
Our new key lemmaLet program P use time T = kn space S and
accept fraction of its inputs in Dn
Almost all inputs P accepts are in (m,)-rectangles accepted by P where m = n is at least
-1 and r are
no input is in more than O(k) rectangles
Srm2
1/8k2
)O(k2
k
34
Proving randomized lower bounds from our key lemma
Show that the desired function f evaluates to 1 a large fraction of the time
i.e, is large
evaluates to 0 on a fraction of inputs in any large-enough (m,)-rectangle
or ... do the same for f
Gives space lower bound for O(/k)-error randomized algorithms running in time kn
35
Proof ideas: layers and trees
v0
vr-1
v2
v1
vr10
kn
kn/r
kn/r
(v1,…,vr-1)
f = (v1,…,vr-1)f
# of (v1,…,vr-1) is 2S(r-1)
(v1,…,vr-1)f =
i=1
r
vi-1vif
vi-1vif can be computed inkn/r height
36
(r,)-decision forest
The conjunction of r decision trees (BP’s that are trees) of height nEach is a computed by a
(r,k/r)-decision forestOnly 2S(r-1) of themThe various accept disjoint sets of inputs
(v1,…,vr-1)f
(v1,…,vr-1)f
37
Decision forest
Assume wlog all variables read on every inputFix an input x accepted by the forestEach tree reads only a small fraction of the
variables on input xFix two disjoint subsets of trees, F and G
kn/r
T1 T2 T3 TrT4
38
Core variables
Can split the set of variables into core(x,F)=variables read only in F (=not read outside F)
core(x,G)=variables read only in G (=not read outside G)
remaining variablesstem(x,F,G)=assignment to remaining variables
General idea: use core(x,F), core(x,G), and stem(x,F,G) to define (m,)-rectangles
kn/r
T1 T2 T3 TtT4
39
A partition of accepted inputs
Fix F, G,x accepted by P Rx,F,G={ y | core(y,F)=core(x,F),
core(y,G)=core(x,G), stem(y,F,G)=stem(x,F,G),
and P accepts y}
For each F, G the Rx,F,G partition the accepted inputs into equivalence classes
Claim: the Rx,F,G are (m,)-rectangles
40
Classes are rectangles
Let A=core(x,F), B=core(x,G), =stem(x,F,G) SA={yA| y in Rx,F,G }, SB={zB| z in Rx,F,G }
Let w=(,yA,zB) w agrees with y in all trees outside G
core(w,G)=core(y,G)=core(x,G) w agrees with z in all trees outside F
core(w,F)=core(z,F)=core(x,F) stem(w,F,G)==stem(x,F,G) P accepts w since it accepts y and z
So... w is in Rx,F,G
41
Few partitions suffice
Only 4k pairs F,G suffice to cover almost all inputs accepted by P by large (m,)-rectangles Rx,F,G
Choose F,G uniformly at random of suitable size, depending on access pattern of inputprobability that F,G isn’t good is tinyone such pair will work for almost all inputs with
the given access pattern
Only 4k sizes needed.
42
Special case: oblivious BPs
core(x,F), core(x,G) don’t depend on xChoose Ti in F with prob q
G with prob q neither with prob 1-2q
43
xTMyx on an (m,)-rectangle
My
A Bx
A
B
x
For every on AUB, f(xAUB,,y)
= xAT MAB xB
+ g(xA,y) + h(xB,y)
44
Rectangles, rank, & rigidity
largest rectangle on which xATMxB is
constant has 2-rank(M)
[Borodin-Razborov-Smolensky 89]
Lemma [Ajtai 99] Can fix y s.t. every nxn minor MAB of My has rank(MAB) cn/log2(1/) improvement of bounds of
[Beame-Saks-Thathachar 98] & [Borodin-Razborov-Smolensky 89] for Sylvester matrices
45
High rank implies balance
For any rectangle SAxSB {0,1}Ax{0,1}B with (SAxSB) |A||B|23-rank(M)
Pr[ xATMxB= 1 | xA SA, xB SB] 1/32
Pr[ xATMxB= 0 | xA SA, xB SB] 1/32
derived from result for inner product in r dimensions
So rigidity also implies balance for all large rectangles and so
Also follows for element distinctness [Babai-Frankl-Simon 86]
)(n/S)log/loglog(nT
46
Talk outlineThe right non-uniform model (for now)
branching programs
Early success multi-output functions, e.g. sorting
Progress on problems in P Crawling
restricted branching programs
That breakthrough first step (and more)true time-space tradeoffs
The path ahead
47
Improving the bounds
What is the limit? T=(nlog(n/S)) ? T=(n2/S) ?
Current bounds for general BPs are almost equal to best current bounds for oblivious BPs ! T=(nlog(n/S)) using 2-party CC [AM]
T=(nlog2(n/S)) using multi-party CC [BNS]
48
Improving the bounds
(m,a)-rectangles a 2-party CC idea insight: generalizing to non-oblivious BPs yields same bound as [AM] for oblivious BPs
Generalize to multi-party CC ideas to get better bounds for general BPs? similar framework yields same bound as [BNS]
for oblivious BPs
Improve oblivious BP lower bounds? ideas other than communication complexity?
49
Extension to other problems
Problem should be hard for (best-partition) 2-party communication complexity (after most variables fixed). try oblivious BPs first
Prime candidate: (directed) st-connectivity Many non-uniform lower bounds in structured
JAG models [Cook-Rackoff], [BBRRT], [Edmonds], [Barnes-Edmonds], [Achlioptas-Edmonds-Poon]
Best-partition communication complexity bounds known
50
Limitations of current method
Need n>T/r = decision tree height else all functions trivial so r > T/n
A decision forest works on a 2-Sr fraction of the accepted inputs
•only place space bound is usedSo need Sr<n else d.f. need only work on
one inputimplies ST/n < n, i.e. T < n2/S