Post on 22-Feb-2016
description
transcript
CS 448 Foundation of CS 448 Foundation of ComputingComputing
Soundararajan EzekielSoundararajan Ezekiel
Department of Computer ScienceOhio Northern University
ADA, Ohio 45810e-mail: s-ezekiel@onu.edu
http://www.onu.edu/user/FS/sezekiel
04/22/23 2
Today’s PlanToday’s Plan What are the fundamental capabilities What are the fundamental capabilities
and limitation of computersand limitation of computers Three major topics: What is a computer? - Automata Theory What can (cannot) be computed? - Computability Theory What can (cannot) be computed efficiently?--
Complexity Theory
04/22/23 3
Why Study Theory?
Pragmatic reasonsPragmatic reasons• apply efficient algorithms to tractable problemsapply efficient algorithms to tractable problems• avoid intractable or impossible problemsavoid intractable or impossible problems• learn to tell the differencelearn to tell the differenceMathematical EducationMathematical Education• automata theoryautomata theory• complexity classescomplexity classes• decidability theorydecidability theoryPhilosophyPhilosophy• what is computation?what is computation?• what is provable?what is provable?
04/22/23 4
Topics Automata Theory: What is a computer? (Review) Computability Theory: What can computers do?(Semi-review) Complexity Theory: What makes some problems computationally hard
and others easy? Evading intractability: Approximation Randomization Parallelism ... Lower bounds: Circuit complexity.
04/22/23 5
Complexity Theory Key notion: Key notion: tractable tractable vs. vs. intractable intractable problems.problems. A A problem problem is a general question:is a general question:
– description of parametersdescription of parameters– description of solutiondescription of solution
An An algorithm algorithm is a step-by-step procedureis a step-by-step procedure– a recipea recipe– a computer programa computer program– a mathematical objecta mathematical object
We want the most We want the most efficient efficient algorithmalgorithm– fastest (mostly)fastest (mostly)– most economical with memory (sometimes)most economical with memory (sometimes)– expressed as a function of problem sizeexpressed as a function of problem size
Example: Traveling Salesman Problem
a
b
c
d
10
5
6
9
3
9
(not drawn to scale)
ONU
CS dept
Park
Dave’s market
Parameters:• set of places• set of inter- places distances
Example: Traveling Salesman Problem
a
b
c
d
10
5
6
9
3
9
(not drawn to scale)Solution: Want the shortest tour through the places Example: a; b; d; c; a has length 27.
ONU
CS dept
Park
Dave’s market
04/22/23 8
Problem Size What is appropriate measure of problem size?
– m nodes?– m(m+1)/2distances?
Use an encoding of the problem– alphabet of symbols– strings: abcd//10/5/9//6/9//3.
Measures– Problem Size: length of encoding– Time Complexity: how long an algorithm takes, as function of problem
size.
04/22/23 9
Time Complexity
What is tractable? A function f(n) is O(g(n)) whenever|f(n)| c*|
g(n)| for n >some constant. A polynomial-time algorithm is one whose time
complexity is O(p(n)) for some polynomial p(n). An exponential-time algorithm is one whose time
complexity cannot be bounded by a polynomial (e.g., n log n).
04/22/23 10
TractabilityBasic distinction:• polynomial time = tractable exponential time = intractable
Execution time 10 20 30 40 50 60n .00001 .00002 .00003 .00004 .00005 .00006 second second second second second secondn2 .00001 .00004 .00009 .00016 .00025 .00036 second second second second second secondn3 .00001 .00008 .027 .064 .125 .216 second second second second second secondn5 .1 3.2 24.3 1.7 5.2 13.0 second seconds seconds minute minutes minutes2n .001 1.0 17.9 12.7 35.7 366 second second minutes days years centuries3n .059 58 6.5 3855 2*10^8 1.3*10^13 second minutes years centuries centuries centuries
11 04/22/23
Effect of Speed-UpsLet's wait for faster hardware!Consider maximum problem size you can solve in an hour. present 100 times faster 1000 times faster
n N1 100 N1 1000 N1
n2 N2 10 N2 31.6 N2
n3 N3 4.64 N3 10 N3
n5 N4 2.5 N4 3.98 N4
2n N5 N5+6.64 N5+9.97
3n N6 N6 +4.19 N6+6.29
04/22/23 12
Your boss says:Your boss says:Get me an efficient traveling-salesmanGet me an efficient traveling-salesmanalgorithm, or else.”algorithm, or else.”What are you going to do?What are you going to do?
Response“Yes Ma'am, expect it this afternoon!"Problem is• All known algorithms (essentially) check all possible paths.• Exhaustive checking is exponential.• Good luck!
04/22/23 13
AnswerAnswer “Hah!, I will prove that no such algorithm is possible" Problem is, proving intractability is very hard. Many important problems have no known tractable
algorithms -- no known proof of intractability. “I can't find an efficient algorithm, I guess I'm just a
pathetic loser. " Bad for job security. “The problem is NP-complete. I can't find an efficient
algorithm, but neither can any of these famous people . . ." Advantage is: The problem is “just as hard" as other
problems ---smart people can't solve efficiently.----So it would do no good to fire you and hire a Dartmouth graduate.
04/22/23 14
Further ReadingsFurther Readings http://www.claymath.org/prizeproblems/milliondollarminesweeper.htm http://www.claymath.org/prizeproblems/pvsnp.htm
Response “Would you settle for a pretty good algorithm?" Intractability isn't everything.
– find an approximate solution– use randomization (“probably pretty good")– parallelism can also help
Approximation, randomization, etc. are (arguably) the hottest areas in complexity theory today.
04/22/23 15
Computability In the first half of the 20th century, mathematicians such as In the first half of the 20th century, mathematicians such as
Kurt Goedel, Alan Turing, and Alonzo Church discovered Kurt Goedel, Alan Turing, and Alonzo Church discovered that some problems do not have a computable solution.that some problems do not have a computable solution.
Example: is a mathematical statement true or false?Example: is a mathematical statement true or false? What could be more amenable to automation?What could be more amenable to automation? Results needed theoretical models for computers.Results needed theoretical models for computers. These theoretical models helped lead to the construction of These theoretical models helped lead to the construction of
real computersreal computers..
04/22/23 16
Computability Hilbert's tenth problem (1900): Find an finite procedure
(algorithm) that decides whether a polynomial has an integer root.
A polynomial is a sum of terms, each term is a product of variables and a constant (called coefficient).
A root of a polynomial is an assignment of values to the variables such that the value of the polynomial is 0.
x = 2; y = 1is a root of the polynomial 5x +15y= 25 There is no such algorithm (1970)!
04/22/23 17
Some Other Undecidable Some Other Undecidable ProblemsProblems Does a program run forever?Does a program run forever? Is a program correct?Is a program correct? Are two programs equivalent?Are two programs equivalent? Is a program optimal?Is a program optimal? Is a finitely-presented group Is a finitely-presented group
trivial?trivial?
04/22/23 18
Sets---order doesn't matter finite:{2; 4; 6} infinite:{2, 4,6,...},{x/ x is even} subset: UV , UV union: UV--- intersection: UV Sequences--order matters (1, 3, 5, ...) finite sequence called a tuple-2 elements is pair k elements is k-tuple Powerset of U is set of all subsets Cartesian product of U x V
Mathematical ToolsMathematical Tools
04/22/23 19
Strings and Languages an alphabet is a finite set of symbols a string over an alphabet is a finite sequence of symbols from that
alphabet. the length of a string is the number of symbols the empty string reverse: abcd is dcba. substring: xyz in xyzzy. concatenation of xyz and zy is xyzzy. xk and x, ... x, k times. a language is a set of strings.
04/22/23 20
ProofsProofs We will use the following basic kinds of proofs. by constructionby construction by contradictionby contradiction by inductionby induction by reductionby reduction
Sometimes we mix them.Sometimes we mix them.
04/22/23 21
Proof by Construction Graph:- G = (V;E), where V is set of nodes or vertices,
and E is set of edges --degree of a vertex is number of edges A graph is k-regular if every node has degree k. Theorem: for every even n >2,there exists a 3-regular graph with
n nodes
Note: picture is helpful, but not a proof!
04/22/23 22
Theorem: 2 is irrational. Proof: Suppose not. Then 2 = m/n ,where m and n are relatively relatively
primeprime. n 2 = m 2n2 = m2
So m2 is even, and so is m = 2k. 2n2 = (2k)2= 4k2
n2 = 2k2 So n2 is even, and so is n. Thus both m and n are even, Thus both m and n are even, not relatively prime!not relatively prime!
Proof by ContradictionProof by Contradiction
04/22/23 23
Induction Prove properties of elements of an infinite set.N={1,
2, 3,....} To prove that holds for each element, show: base step: show that (1) is true. induction step: show that if (i) is true (the
induction hypothesis), then so is (i +1).
04/22/23 24
Theorem: All cows are the same color. Base step A single-cow set is definitely the same color. Induction Step: Assume all sets of ii cows are the same
color. Divide the set{1..., i+1} into U ={1,…, i}, and V {2,…, i+1}.
all cows in U are the same color by the induction hypothesis. all cows in V are the same color by the induction hypothesis. all cows in UV are the same color by the induction hypothesis.
Ergo, all cows are the same color.
Induction Example
04/22/23 25
Back to Language alphabet example ={x} L1={ x^n/ n=1,2,..} L2={x^(2n+1)/ n=0,1,2...} ={ a b} L=PALINDROME={, set of all string x/
reverse(x)=x}={ a b aa bb aaa ....} KLEENE CLOSURE (Kleene star/ Kleene
operator) Stpehern Kleene http://burks.brighton.ac.uk/burks/foldoc/55/109.htm
given an alphabet -- define language = + any string of letter from and denoted by *-- language called closure
04/22/23 26
Example ={x} then *={ x^n, n=1,2..} ={0 1} then *={ 0 1 00 01 ..} extend it to set of words-- S= set of words then S*={ plus
set of all finite strings formed by concatenating words} example: S={ a ab} then S*={ b a aa bb ... abaab.. .} Factoring:- abaab= (ab)(a)(ab) these are in S Factoring is not unique--- example S={xx xxx} S*={x^n,
n=0,2,3,..}-- x^6= x^3 x^3 or x^2 x^2 x^2
04/22/23 27
Note parentheses () are not letters --- in case if you use () are letters
of alphabet -- ={x ( )} length(xxxxx)=5 but length( (xx) (xxxx))=10 Proof by constructive algorithm closure( closure )= closure ---- (S*)*= S**= S* Proof: ==> S*S** Every word in S** is made up of factors from
S*. Every factors from S* is made up of factors from S--- every word in S** is made up from S. therefore every word from S** is also in S*
<=== S** S* we know that S S* apply closure both side we have S* S**
Together S*=S**
04/22/23 28
Homework Problems
Problem # 5, 7, 9. 14,