Limits of Computation

Hard vs Impossible

Intractable Problems – trillions of years to solve: practically unfeasible

Unsolvable Problems – factually impossible, as opposed to hard - .eg mutilated chessboard problem

Hard vs Impossible

Intractable Problems – trillions of years to solve: practically unfeasible

Unsolvable Problems – factually impossible, as opposed to hard - .eg mutilated chessboard problem

Complexity Class

how fast #ops grow with size of input

Problem Types

Optimization Problems: min/max. Must examine all data

Decision Problems: boolean -eg <, >. stop when we find what we are looking for.

Polynomial Problems: (P) – feasible, tractable #ops

Non-Polynomial Problems (NP): n^2.5+, 2^n, n!

Eg Encryption Key asymmetry: multiplication of 2 primes is n^2 , factoring is NP

Easy Problems (P)

Eg arithmetic, searching, sorting Brute-Force vs algorithm ‘tricks’ - eg binary-

search (split up, recurse) Appropriate algorithm may depend on type

of input

Graph Theory

Eg 6 degrees of separation

Max #edges that one must traverse to connect any 2 vertices

7 bridges of Konigsberg

Easy Graph problems

Euler Cycle problem Q: is there a path that starts & finishes at same vertex that passes every edge exactly once ?

A: if every vertex has an even number of edges.

Easy Graph problems

Euler Path problem Q: is there a path that passes every vertex exactly once ?

A: if Max 2 vertices have odd #edges & rest have even

Hard Problems (NP)

Traveling Salesman Problem

travel shortest route to visit each city once. (Weighted graph)

Brute-force: n! ops Eg n=100. For each

route, need to add distance between 100 cities and & compare sum to shortest distance already found

Traveling Salesman Problem

100! = 9.332622e+157 Given a computer that can check 1 million

routes per sec → 3 x 10^144 years. There are 10^80 atoms in the universe – if

each were a computer, this would take 10^62 centuries. And this is for 100!

Hamiltonian Cycle Problem

Is there a path that passes every vertex exactly once that starts & finishes at same vertex ?

Brute force: n! Similar to Euler Cycle - but unfortunately

no trick discovered (& probably wont be) !!!

Hard Set Problems

Set Partition Problem split set of natural numbers into 2 groups

that both sum to the same number first evaluate solvability: is the total sum

Odd ? brute force: 2^nSubset Sum Problem for a given set of natural numbers, and a

natural number C (capacity), is there a subset that sums to C ? eg fill a room

The Satisfiability Problem deals with logical statements ^ (and), v (or), ~ (not) , →

(implies) eg (p v ~q) → ~ (p ^ q) – is this true if p=true and q=false ? Generalize: can we assign boolean values to variables to make

the statement true ? Logic Rules – eg Modus Ponens: ((p → q) ^ p) → q If today is

Tuesday, then John will go to work. Today is Tuesday. Therefore, John will go to work.

Brute-force: truth-table lookup: 2^n

NP Problems are Inherently Hard

Intractability is not because we lack the technology.Faster computers – if 10,000 times faster, TS (100) problem still takes 3 x 10^140 yearsParallelization – Similarly, perform 10,000 simultaneous ops – same storyQuantum Computing – superposition of search states: examine multiple possibilities at once. For searching list of size n: Sqrt(n) ops. But when searching through all n! possible solutions to an NP problem: Sqrt(n!) ops – still intractable. More speed cant solve the unsolvable.Better Algorithms - most researchers believe there arn’t

Reduction

Transform a problem – reduction of a problem x to y. if we can solve y then we can definitely solve xeg x is as hard as or easier than y: x <=p y climbing K2 is reducible to climbing mount

Everest Set Partition Problem is reducible to

Subset Sum Problem Hamiltonian Cycle Problem is reducible to

Traveling Salesman Problem

Reduction

Cook-Levin Theorem: because a computer works on logic gates, ALL NP problems are reducible to Satisfiability Problem (even undiscovered ones)

NP-Complete and P != NP

NP-Complete problems – the hardest NP problems. Every NP problem is reducible to NP-Complete. If we could find an algorithm to solve it in P …

Unsolved problem: If NP is a subset of P, then P = NP. Most researchers believe P is a subset of NP.

Approximation Algorithms

if NP – best we can do is Heuristics. Better than waiting 400 trillion centuries.

Traveling Salesman Problem approximation: use a Greedy Algorithm: OSPF, Nearest Neighbor

Set Partition Problem approximation: Extreme Pairs – repeat, alternating partitions: pick min & max from source set & place in partition x

Even Harder problems

Super-exponential – eg 2^(2^n) (if n=10, need 2^1024 ops), (n!)!

PSPACE (a superset of NP). Demand a polynomial amount of space

eg is there a winning strategy for n x n tic-tac-toe ?

Chess, Checkers, Nim, Go

Computing Impossibilities

Q: Can computers recognize art, make moral decisions ? A: Quantification of Aesthetics, Ethics – subjective.

Halting Problem Incompleteness Theorem

Some Definitions

Science – language we use to describe & predict the physical & measurable universeApplied Maths – the language of scienceLogic – the language of reason Epistemology – philosophy of human knowledge & its limits

Logic

Use symbols to reason about structure of proofs – set up axiom systems to put mathematics on a firm foundationPeano Arithmetic – Axiom system for natural numberssymbolization / arithmetization – conversion between logical symbolic systems and numeric systems

Logical Paradoxes

input: axioms + assumptions; processing: logical reasoning laws; output: logically unacceptable falsehoods, self-contradictions, nonsensical false facts → jettison assumptionsoften counterintuitive (eg “space is continuous”)reductio ad absurdum (proof by contradiction)

Logical Paradoxes

self-referential paradoxes - eg Liar Paradox: “this sentence is false”reduction – infer limitations from other problemsreality, science, maths, logic cannot permit contradictions – but our minds & language can: vagueness, jokes

Zeno’s Paradoxes

The Halting Problem

Alan Turing, 1936: proof by contradiction (Liar Paradox) that the Halting Problem is undecidableA limitation of mechanized processes, its not a hard problem, its an impossible problema computer (or brain ?) cannot determine if a black box program given input x will terminate (with a return code) or enter an infinite loop. Processing ….https://www.youtube.com/watch?v=92WHN-pAFCs

Oracle Machines & Turtles

imagine a non-mechanical mystic Oracle Machine – the Halt Oracle can solve the Halting Problem – could solve many other unsolved math problems eg Goldbach Conjecture: every positive even number greater than 2 is a sum of 2 primes. (we know this holds up to 10^17)the Halt Oracle searches for a counter-example – if it exists, it can tell if the program will halt.Turing: Halting Problem for Halt Oracle Programs - Halt Oracle Programs cannot solve all unsolvable problems, but a Halt` Oracle can solve this

Computers vs Minds

Q: Can a human solve the Halting Problem ?A: Kurt Godel , Sir Roger Penrose, Douglas Hofstadfer - consciousness arises from ability to self-reference. Self-Reference in Computers brings paradoxical limitations, while in humans it causes consciousness ???

Incompleteness

Kurt Godel – the man who broke Logic

Godel Sentence: “This Logical Statement is Unprovable” → contradiction

1st Incompleteness Theorem – There are logical statements that are true but not provable.

2nd Incompleteness Theorem - A self-referencing system cannot prove its own consistency.

Incompleteness

If Peano Arithmetic is consistent then the Godel sentence is unprovable and true –> limitation of basic arithmetic: cannot determine when its own statements are true / consistent.

There are stronger axiom systems that do prove Peano Logic – eg ZFC, But, ZFC cannot prove its own consistency.

There will always be larger systems (less self-evident though) – relative limitations.

Incompleteness

consistency of logical systems at basis of mathematics and science is beyond the bounds of reason!

Note: majority of maths work does not require working with basic axiomatic proofs