Games and Complexity, Guangzhou 2009. Peter van Emde Boas

KNOW YOUR NUMBERS !The Impact of Complexity

a short intro in Complexity theory

Peter van Emde Boas

ILLC-FNWI-UvA

Guangzhou 2009

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

Moore’s Law in action

© Scientifi American 274, 199601, p45

Computation speeddoubles every 1.5 years

Corresponding rate ofincrease ofMemory capacity

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

1982 1985

4 MHZ, 64K, 1.2M (FD) ? MHZ, 256K, 10/20 M (HD)

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

1986 1988

8 MHZ, 512K, 20/30M 16 MHZ, 1 M, 20/80M

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

1990 1992

12/33 MHZ, 1/4 M, 40/150M 25/40 MHZ, 4/32 M, 125/210M

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

1994 1996

25/60 MHZ, 4/8 M, 212/527M 166 MHZ, 8/32 M, 1/4 G

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

1998 2000

533/733 MHZ, 64/1024 M, 9/10 G166/300 MHZ, 32/64 M, 2.1/4 G

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

2003 2006

2/3 GHZ, 512 M, 40/80 G 1.8/2.6 GHZ, 512/1024 M, 80/320 G

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

2008

? GHZ, 2 G, ? G

2009

2.5 GHZ, 4 G, 250 G

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

Is it really so nice ?

sum := 0for i = 1 to N do

sum := sum + a[i] ;print ( sum )

Back in 1977 this would work in less than a second for N = 10 000Can we deal with N = 400 000 000 today within a second ?

sum := 0for i = 1 to N do

j:= index[i] ; sum := sum + a[j] ;print ( sum )

And what about this one ?

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

Information must be moved!

Source: Ph.d. Thesis Stefan ManegoldUnderstanding, Modeling, and Improving

Main-Memory Database PerformanceUvA Dec 16, 2002

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

Information must be moved!

Source: Ph.d. Thesis Peter A. BonczMonet; A next-Generation

DBMS Kernel for Query-IntensiveApplications

UvA Mei 31, 2002

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

Know your numbers !# Instructions Time

1 000 0.00001 sec1 000 000 0.01 sec Invisible10 000 000 0.1 sec Perceptive100 000 000 1 sec1 000 000 000 10 sec10 000 000 000 100 sec Coffee100 000 000 000 16.7 min Chat1 000 000 000 000 2 hr 47 min WHAARGH!10 000 000 000 000 27 hr the Limit!!100 000 000 000 000 11.25 days Hopeless1 000 000 000 000 000 4 months10 000 000 000 000 000 3.33 years Unrealistic100 000 000 000 000 000 33 years1 000 000 000 000 000 000 3.3 century

Processing times on a 100 MIPS system

1 day = 86400 sec 1 000 000 sec ≈ 11 days1 year ≈ 31 536 000 sec 1 000 000 000 sec ≈ 33 years

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

Notation for asymptotic order of growth

f(x) = O( g(x) ) means f(x) / g(x) is bounded for x -->

f(x) = o( g(x) ) means f(x) / g(x) --> 0 for x -->

f(x) = ( g(x) ) means f(x) / g(x) > C > 0 for x -->

f(x) = ( g(x) ) means f(x) / g(x) is > C > 0 and bounded for x -->

f(x) = ( g(x) ) means f(x) / g(x) is unbounded for x -->

All functions here are non-negativeSimilar for integer argument n -->

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

WHAT IS COMPLEXITY ?

Relation between size of the input instance and the time required for solving this instance.

Traversing a list Linear O(n)Sorting O(n.logn)Multiplication Quadratic O(n2)Matrix multiplication Cubic O(n3)Truth table checking Exponential O(2n)Naive TSP O(n!)

What can be done in principle can become undoable in practice

Order of growth is more relevant than the implicit constants

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

The Order of growth

10 20 50 100 200 1000

n .00001 .00002 .00005 .0001 .0002 .001

n.logn .00003 .00008 .00027 .0006 .0016 .01

n2 .0001 .0004 .0025 .01 .04 1

n3 .001 .008 .125 1 8 16 min

2n .001 1 31 y ?? !!?*

Time on a 1 MIPS system in seconds (unless stated otherwise)

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

Impact of a 1000 times faster machine

n 1000 times as large

n.logn 500 - 1000 times as large

n2 31 times as large

n3 10 times as large

2n add 10 to input size

That’s why Edmonds stated his thesis:Effective ==> Polynomial Time Complexity

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

A Fallacy ??

If computation speed doubles every 2 yearswe can perform (2N - 1) M operations in 2Nyears, where M is the number of operationsperformed in the first 2 years.

So why worry about exponential timealgorithms ??

What is wrong with this argument ?

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

The Fundamental Complexity Classes

A program Mj belongs to the Complexity Class of programs F(t)iff its run-time TMj (x) < t( |x| ) for all sufficiently large inputs x(here |x| denotes the length of x, and t is a function from N to N)

A function (set) belongs to the Complexity Class C(t) iffit is computed (recognized) by some program in the class F(t)

This explains classes like F( 60.n ) or C(n.log(n))

Similarly one can define Space bounded Complexity ClassesH(t) and S(t) .

NF(t) , NC(t) , NH(t) , and NS(t) also allow Nondeterministic Programs

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

DetailsThe fundamental distinction between a run-time TMj (x) and just an arbitrary computable function f(x) is that for a run-time the predicate:

TMj (x) < k is decidable,

whereas for general computable f(x) thisis not the case (simple reduction from the Halting Problem).(Second Blum Axiom in Abstract Complexity Theory)

Another complication results from the fact that runtimes are boundedin terms of the input length rather than the input itself:TMj (x) < t( |x| ) , rather than TMj (x) < t( x ) There are many inputs of the same length; the complexity isdetermined by the worst case input. If one input leads to divergencethe no finite bound applies.

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

The Fundamental Complexity ClassesLOGSPACE = S(k.log(n)) for k N

NLOG = NS(k.log(n)) for k N

P = C( nk ) for k N

NP = NC( nk ) for k N

PSPACE = S ( nk ) for k N

NPSPACE = NS( nk ) for k N

E = C( 2kn ) for k N

NE = NC( 2kn ) for k N

EXP = C( 2^ nk ) for k Netc....

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

The Fundamental Complexity Classes

The following Inclusions follow from these definitions:

LOGSPACE NLOG P NP PSPACE NPSPACE EXP

It is known that PSPACE = NPSPACE (Savitch Theorem)For all remaining inclusions it is unknown whether the inclusionis strict or not; E.G., the Notorious P = NP problem(still good for US $ 1 000 000 if you solve this question)

Some of the inclusions are known to be strict since LOGSPACE PSPACE and P E EXPAll inclusions are conjectured to be strict.

Problem: To what extent are these classes Machine Model Dependent ?

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

The Invariance Thesis

Reasonable Sequential MachinesSimulate Each Other with

Polynomial overhead in Time andConstant Factor Overhead in Space

M’ simulates M : …. ; no definition given

TM’( x ) ≤ k.TM( x )k

SM’( x ) ≤ k.SM( x )

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

ConsequenceThe classes in the fundamental ComplexityHierarchy:

LOGSPACE NLOGSPACE P NP PSPACE EXPTIME NEXPTIME

are machine independent.(Classes like LIN and NLIN are not)LIN = C( k.n ) for k N

NLIN = NC( k.n ) for k N

Games and Complexity, Guangzhou 2009. Peter van Emde Boas

Parallel Computation Thesis

// PTIME = // NPTIME = PSPACE

The symbol // refers to machines involvingParallellismTrue for Computational Models which combineExponential Growth potential withUniform Behavior.

The Second Machine Class