Post on 19-Jan-2018
description
transcript
Fall 2013 CMU CS 15-855Computational Complexity
Lecture 2
Diagonalization,
9/12/2013
9/12/2013 2
Languages• Let ∑ be any finite alphabet. • L µ ∑*
• L is a language.
• The decision problem corresponding to L is the function:
• f:∑* ! {yes, no} deciding membership in L, i.e., {x | f(x)=‘yes}
9/12/2013 3
Model for this class;Multi-tape Turing Machines
• Read-only Input tape; write only output . . . finite
controla b a b
a a
b b c d
. . .
. . .
k tapes
δ:Q x ∑k ! Q x ∑k x {L,R,-}k
(input tape)
9/12/2013 4
Turing Machines• Q finite set of states• ∑ alphabet including blank: “_”• qstart, qaccept, qreject in Q• δ:Q x ∑k ! Q x ∑k x {L,R,-}k transition fn.• input written on tape, head on 1st square,
state qstart
• sequence of steps specified by δ• if reach qaccept or qreject then halt
9/12/2013 5
TIME and SPACETIME(f(n)) = languages decidable by a multi-tape TM in at most f(n) steps, where n is the input length, and f :N ! N
SPACE(f(n)) = languages decidable by a multi-tape TM that touches at most f(n) squares of its work tapes, where n is the input length, and f :N ! N
9/12/2013 6
Time and Space Classes
L = SPACE(log n)PSPACE = k SPACE(nk)
P = k TIME(nk)
EXP = k TIME(2nk)
9/12/2013 7
Outline
• Why big-oh? Linear Speedup Theorem
• Hierarchy Theorems
• Robust Time and Space Classes
• Relationships between classes
• Some complete problems
9/12/2013 8
Linear SpeedupTheorem: Suppose TM M decides language L in
time f(n). Then for any > 0, there exists TM M’ that decides L in time
f(n) + n + 2.• Proof:
– simple idea: increase “word length”– M’ will have
• one more tape than M• m-tuples of symbols of M
∑new = ∑old ∑oldm
• many more states
9/12/2013 9
Linear Speedup
• part 1: compress input onto fresh tape
. . . a b a b b a a a
. . . aba bba aa_
9/12/2013 10
Linear Speedup
• part 2: simulate M, m steps at a timeb b a a b a b a a a b
. . . abb aab aba aab aba
. . . . . .
. . . m m
– 4 (L,R,R,L) steps to read relevant symbols, “remember” in state
– 2 (L,R or R,L) to make M’s changes
9/12/2013 11
Linear Speedup• accounting:
– part 1 (copying): n + 2 steps– part 2 (simulation): 6 (f(n)/m)– set m = 6/– total: f(n) + n + 2
Theorem: Suppose TM M decides language L in space f(n). Then for any > 0, there exists TM M’ that decides L in space f(n) + 2.
• Proof: same.
9/12/2013 12
Time and Space• Moral: big-oh notation necessary given our
model of computation– Recall: f(n) = O(g(n)) if there exists c such that f(n) ≤ c
g(n) for all sufficiently large n.– TM model incapable of making distinctions between
time and space usage that differs by a constant.• In general: interested in course distinctions not
affected by model– e.g. simulation of k-string TM running in time f(n) by
single-string TM running in time O(f(n)2)
9/12/2013 13
Hierarchy Theorems
• Does genuinely more time permit us to decide new languages?
• how can we construct a language L that is not in TIME(f(n))…
• idea: same as “HALT undecidable” diagonalization and simulation
9/12/2013 14
Recall proof for Halting Problem
Turing Machines
inputs
Yn
Yn
nY
n
Y n Y Y nn YH’ :
box (M, x): does M halt on x?
The existence of H which tells us yes/no for each box allows us to construct a TM H’ that cannot be in the table.
9/12/2013 15
Time Hierarchy Theorem
Turing Machines
inputs
Yn
Yn
nY
n
Y n Y Y nn YD :
box (M, x): does M accept x in time f(n)?
• TM SIM tells us yes/no for each box in time g(n)
• rows include all of TIME(f(n))
• construct TM D running in time g(2n) that is not in table
9/12/2013 16
Time Hierarchy Theorem
Theorem (Time Hierarchy Theorem): For every proper complexity function f(n) ≥ n:
TIME(f(n)) ( TIME(f(2n)3).
• more on “proper complexity functions” later…
9/12/2013 17
Proof of Time Hierarchy Theorem
• Proof: – SIM is TM deciding language
{ <M, x> : M accepts x in ≤ f(|x|) steps }– Claim: SIM runs in time g(n) = f(n)3.– define new TM D: on input <M>
• if SIM accepts <M, M>, reject• if SIM rejects <M, M>, accept
– D runs in time g(2n)
9/12/2013 18
Proof of Time Hierarchy Theorem
• Proof (continued):– suppose M in TIME(f(n)) decides L(D)
• M(<M>) = SIM(<M, M>) ≠ D(<M>)• but M(<M>) = D(<M>)
– contradiction.
9/12/2013 19
Proof of Time Hierarchy Theorem• Claim: there is a TM SIM that decides
{<M, x> : M accepts x in ≤ f(|x|) steps}and runs in time g(n) = f(n)3.
• Proof sketch: SIM has 4 work tapes• contents and “virtual head” positions for M’s
tapes • M’s transition function and state• f(|x|) “+”s used as a clock• scratch space
9/12/2013 20
Proof of Time Hierarchy Theorem
• contents and “virtual head” positions for M’s tapes • M’s transition function and state• f(|x|) “+”s used as a clock• scratch space
– initialize tapes– simulate step of M, advance head on tape 3;
repeat.– can check running time is as claimed.
• Important detail: need to initialize tape 3 in time O(f(n))
9/12/2013 21
Proper Complexity Functions
• Definition: f is a proper complexity function if– f(n) ≥ f(n-1) for all n– there exists a TM M that outputs exactly f(n)
symbols on input 1n, and runs in time O(f(n) + n) and space O(f(n)).
9/12/2013 22
Proper Complexity Functions• includes all reasonable functions we will
work with– log n, √n, n2, 2n, n!, …– if f and g are proper then f + g, fg, f(g), fg, 2g
are all proper• can mostly ignore, but be aware it is a
genuine concern:• Theorem: 9 non-proper f such that
TIME(f(n)) = TIME(2f(n)).
9/12/2013 23
Hierarchy Theorems
• Does genuinely more space permit us to decide new languages?
Theorem (Space Hierarchy Theorem): For every proper complexity function f(n) ≥ log n:
SPACE(f(n)) ( SPACE(f(n) log f(n)).• Proof: same ideas.
9/12/2013 24
Best Hierarchy Theorems
Theorem (Time Hierarchy Theorem): For every proper complexity function f(n) ≥ n:
TIME(f(n)) ( TIME( (f(n)log(f(n))).
Theorem (Space Hierarchy Theorem): For every proper complexity function f(n) ≥ log n:
SPACE(f(n)) ( SPACE( f(n)).
9/12/2013 25
Time and Space Classes
L = SPACE(log n)PSPACE = k SPACE(nk)
P = k TIME(nk)
EXP = k TIME(2nk)
9/12/2013 26
Hierarchy Separations
• Time Hierarchy Theorem impliesP ( EXP
– P µ TIME(2n) ( TIME(2(2n)3) µ EXP
• Space Hierarchy Theorem impliesL ( PSPACE
– L = SPACE(log n) ( SPACE(log2 n) µ PSPACE
9/12/2013 27
TIME(f(n)) SPACE(f(n))
• Any time step can only affect constant number of tape cells
: P µ PSPACE
9/12/2013 28
SPACE(f(n)) TIME(2 O(f(n)) )
• T in TIME(f(n)) says yes or no – never loops
• So T can’t repeat a configuration.• log(n) + klog(f(n))+ c + O(f(n) bits
• = O(f(n)) bits if f(n)>=log(n)28
input-tape head position work-tape head
position
state work-tape contents
SPACE(f(n)) TIME(2 O(f(n)) )
• L µ P
• PSPACE µ EXP
• So far:L µ P µ PSPACE µ EXP
• Open: L = P ??? P = PSPACE ??9/12/2013 29
9/12/2013 30
Reductions• reductions are the main tool for relating
problems to each other • given two languages L1 and L2 we say “L1
reduces to L2” and we write “L1 ≤ L2” to mean:– there exists an efficient (logspace) algorithm
that computes a function f s.t.• x L1 implies f(x) L2
• x L1 implies f(x) L2
9/12/2013 31
Reductions
• positive use: given new problem L1 reduce it to L2 that we know to be in P. Conclude L1 in P (how?)– e.g. bipartite matching ≤ max flow – formalizes “L1 as easy as L2”
yes
no
yes
noL1 L2
f
f
9/12/2013 32
Reductions
• negative use: given new problem L2 reduce L1 (that we believe not to be in P) to it. Conclude L2 not in P if L1 not in P (how?)– e.g. satisfiability ≤ graph 3-coloring– formalizes “L2 as hard as L1”
yes
no
yes
noL1 L2
f
f
9/12/2013 33
Reductions
• Example reduction:– 3SAT = { φ : φ is a 3-CNF Boolean formula
that has a satisfying assignment }(3-CNF = AND of OR of ≤ 3 literals)
– IS = { (G, k) | G is a graph with an independent set V’ µ V of size ≥ k }
(ind. set = set of vertices no two of which are connected by an edge)
9/12/2013 34
Ind. Set is as easy as 3-SAT
The reduction f: givenφ = (x y z) (x w z) … (…)
we produce graph Gφ:
x
y z
x
w z• one triangle for each of m clauses• edge between every pair of contradictory literals• set k = m
…
9/12/2013 35
Reductionsφ = (x y z) (x w z) … (…)
• Claim: φ has a satisfying assignment if and only if G has an independent set of size at least k– proof?
• Conclude that 3SAT ≤ IS.
x
y z
x
w z…
9/12/2013 36
Hardness
• L is C-hard if– every language in C reduces to L
9/12/2013 37
Completeness
• complexity class C• language L is C-complete if
– L is in C– L is C-hard