1

Computer Language Computer Language

TheoryTheory

Chapter 4: DecidabilityChapter 4: Decidability

2

Limitations of Algorithmic SolvabilityLimitations of Algorithmic Solvability

In this Chapter we investigate the power of algorithms In this Chapter we investigate the power of algorithms

to solve problemsto solve problems

Some can be solved algorithmically and some Some can be solved algorithmically and some cannotcannot

Why we study unsolvabilityWhy we study unsolvability

Useful because then can realize that searching for an Useful because then can realize that searching for an

algorithmic solution is a waste of timealgorithmic solution is a waste of time

Perhaps the problem can be simplifiedPerhaps the problem can be simplified

Gain an perspective on computability and its limitsGain an perspective on computability and its limits

In my view also related to complexity (Chapter 7)In my view also related to complexity (Chapter 7)

First we study whether there is an algorithmic solution and then we First we study whether there is an algorithmic solution and then we

study whether there is an “efficient” (polynomialstudy whether there is an “efficient” (polynomial--time) onetime) one

3

Chapter 4.1Chapter 4.1

Decidable LanguagesDecidable Languages

4

Decidable LanguagesDecidable Languages

We start with problems that are decidableWe start with problems that are decidable

We first look at problems concerning regular We first look at problems concerning regular

languages and then those for contextlanguages and then those for context--free languagesfree languages

5

Decidable Problems for Regular LanguagesDecidable Problems for Regular Languages

We give algorithms for testing whether a finite We give algorithms for testing whether a finite automaton accepts a string, whether the language of a automaton accepts a string, whether the language of a finite automaton is empty, and whether two finite finite automaton is empty, and whether two finite automata are equivalentautomata are equivalent

We represent the problems by languages not finite We represent the problems by languages not finite automataautomata

Let ALet ADFADFA={(B, w)|B is a DFA that accepts string w}={(B, w)|B is a DFA that accepts string w}

The problem of testing whether a DFA B accepts an input w is The problem of testing whether a DFA B accepts an input w is the same as testing whether (B,w) is a member of the language the same as testing whether (B,w) is a member of the language AADFADFA..

Showing that the language is decidable is the same thing as Showing that the language is decidable is the same thing as showing that the computational problem is decidableshowing that the computational problem is decidable

6

AADFADFA is a Decidable Languageis a Decidable Language

Theorem: ATheorem: ADFADFA is a decidable languageis a decidable language

Proof Idea: Present a TM M that decides AProof Idea: Present a TM M that decides ADFADFA

M = On input (B,w), where B is a DFA and w is a string:M = On input (B,w), where B is a DFA and w is a string:

1.1. Simulate B on input wSimulate B on input w

2.2. If the simulation ends in an accept state, then accept; else reject.If the simulation ends in an accept state, then accept; else reject.

Background for actual proof:Background for actual proof:

Books description too simpleBooks description too simple–– leads to wrong understandingleads to wrong understanding

The TM M cannot “be” the DFA BThe TM M cannot “be” the DFA B

If it could, then it would be simple. Both would have essentially the If it could, then it would be simple. Both would have essentially the

same transition functions (TM just needs to move right over w as same transition functions (TM just needs to move right over w as

each symbol read).each symbol read).

7

Outline of ProofOutline of Proof

Must take B as input, described as a string, and then simulate itMust take B as input, described as a string, and then simulate it

This means the algorithm for simulating any DFA must be embodied in This means the algorithm for simulating any DFA must be embodied in the TM’s state transitionsthe TM’s state transitions

Think about this. Given a current state and input symbol, scan the tape Think about this. Given a current state and input symbol, scan the tape for the encoded transition function and then use that info to determine for the encoded transition function and then use that info to determine new statenew state

The actual proof would describe how a TM simulates a DFAThe actual proof would describe how a TM simulates a DFA Can assume B is represented by its 5 components and then we have wCan assume B is represented by its 5 components and then we have w

Note that the TM must be able to handle Note that the TM must be able to handle anyany DFA, not just this oneDFA, not just this one

Keep track of current state and position in w by writing on the tapeKeep track of current state and position in w by writing on the tape

Initially current state is q0 and current position is leftmost symbol of wInitially current state is q0 and current position is leftmost symbol of w

The states and position are updated using the transition function The states and position are updated using the transition function δδ

TM M’s TM M’s δδ not the same as DFA B’s not the same as DFA B’s δδ

When M finishes processing, accept if in an accept state; else reject. The When M finishes processing, accept if in an accept state; else reject. The implementation will make it clear that will complete in finite time.implementation will make it clear that will complete in finite time.

8

AANFANFA is a Decidable Languageis a Decidable Language

Proof Idea:Proof Idea:

Because we have proven decidability for DFAs, all we need Because we have proven decidability for DFAs, all we need to do is convert the NFA to a DFA.to do is convert the NFA to a DFA. N = On input (B,w) where B is an NFA and w is a stringN = On input (B,w) where B is an NFA and w is a string

1.1. Convert NFA B to an equivalent DFA C, using the procedure for Convert NFA B to an equivalent DFA C, using the procedure for conversion given in Theorem 1.39conversion given in Theorem 1.39

2.2. Run TM M on input (C,w) using the theorem we just provedRun TM M on input (C,w) using the theorem we just proved

3.3. If M accepts, then accept; else rejectIf M accepts, then accept; else reject

Running TM M in step 2 means incorporating M into the Running TM M in step 2 means incorporating M into the design of N as a subproceduredesign of N as a subprocedure

Note that these proofs allow the TM to be described at the Note that these proofs allow the TM to be described at the highest of the 3 levels we discussed in Chapter 3 (and even highest of the 3 levels we discussed in Chapter 3 (and even then, without most of the details!). then, without most of the details!).

9

Computing whether a DFA accepts any StringComputing whether a DFA accepts any String

EEDFADFA is a decidable languageis a decidable language

Proof:Proof: A DFA accepts some string iff it is possible to reach the accept state from A DFA accepts some string iff it is possible to reach the accept state from

the start state. How can we check this?the start state. How can we check this?

We can use a marking algorithm similar to the one used in Chapter 3.We can use a marking algorithm similar to the one used in Chapter 3.

T = On input (A) where A is a DFA:T = On input (A) where A is a DFA:

1.1. Mark the start state of AMark the start state of A

2.2. Repeat until no new states get marked:Repeat until no new states get marked:

3.3. Mark any state that has a transition coming into it from any state already markedMark any state that has a transition coming into it from any state already marked

4.4. If no accept state is marked, accept; otherwise rejectIf no accept state is marked, accept; otherwise reject

In my opinion this proof is clearer than most of the previous ones because In my opinion this proof is clearer than most of the previous ones because the pseudothe pseudo--code above specifies enough details to make it clear to the code above specifies enough details to make it clear to the reader how to implement itreader how to implement it

One assumption is that all transitions include valid alphabet symbols, but One assumption is that all transitions include valid alphabet symbols, but since this is part of the definition of a DFA, no need to worry about this. since this is part of the definition of a DFA, no need to worry about this.

10

EQEQDFADFA is a Decidable Languageis a Decidable Language

EQEQDFADFA={(A,B)|A and B are DFAs and L(A)=L(B)}={(A,B)|A and B are DFAs and L(A)=L(B)}

Proof ideaProof idea

Construct a new DFA C from A and B, where C accepts only Construct a new DFA C from A and B, where C accepts only

those strings accepted by either A or B but not both.those strings accepted by either A or B but not both.

If A and B accept the same language then C will accept nothing and If A and B accept the same language then C will accept nothing and

we can use the previous proof to check for this.we can use the previous proof to check for this.

So, the proof is:So, the proof is:

F = On input (A,B) where A and B are DFAs:F = On input (A,B) where A and B are DFAs:

1.1. Construct DFA C (more on this in a minute …)Construct DFA C (more on this in a minute …)

2.2. Run TM T from the proof from last slide on input (C)Run TM T from the proof from last slide on input (C)

3.3. If T accepts, then accept. If T rejects, then rejectIf T accepts, then accept. If T rejects, then reject

11

Constructing CConstructing C

L(C) = (L(A) ∩ L(B)’) L(C) = (L(A) ∩ L(B)’) (L(A)’ (L(A)’ ∩ L(B))∩ L(B))

We used proofs by construction that regular We used proofs by construction that regular

languages are closed under languages are closed under , ∩ , and complement, ∩ , and complement

We can use those constructions to construct a FA We can use those constructions to construct a FA

that accepts L(C)that accepts L(C)

L(A) L(B)

L(C) Complement symbol

12

AACFGCFG is a Decidable Languageis a Decidable Language

Proof Idea:Proof Idea: For CFG G and string w want to determine whether G For CFG G and string w want to determine whether G

generates w. One idea is to use G to go through all generates w. One idea is to use G to go through all derivations. This will not work, why?derivations. This will not work, why? Because this method will yield a TM that is a recognizer, not a Because this method will yield a TM that is a recognizer, not a

decider. That is, it could infinite loop and never stop (e.g., imagine decider. That is, it could infinite loop and never stop (e.g., imagine applying a rule like A →xA repeatedlyapplying a rule like A →xA repeatedly

But since we know the length of w, we can exploit this. How?But since we know the length of w, we can exploit this. How?

A string w of length n will have a derivation that uses 2nA string w of length n will have a derivation that uses 2n--1 steps if the 1 steps if the CFG is in ChomskyCFG is in Chomsky--Normal Form.Normal Form. So first convert to ChomskySo first convert to Chomsky--Normal FormNormal Form

Then list all derivations of length 2nThen list all derivations of length 2n--1 steps. If any generates w, then 1 steps. If any generates w, then accept, else reject.accept, else reject.

This is a variant of breadth first search, but instead of extended the This is a variant of breadth first search, but instead of extended the depth 1 at a time we allow it to go 2ndepth 1 at a time we allow it to go 2n--1 at a time. As long as finite depth 1 at a time. As long as finite depth extension, we are okayextension, we are okay

13

EECFGCFG is a Decidable Languageis a Decidable Language

How can you do this? What is the brute force How can you do this? What is the brute force

approach?approach?

Try all possible strings w. Will this work?Try all possible strings w. Will this work?

The number is not bounded, so this would not be decidable. The number is not bounded, so this would not be decidable.

It would be TMIt would be TM--recognizable as long as you work breadth recognizable as long as you work breadth

firstfirst

Instead, think of this as a graph problem where you want to Instead, think of this as a graph problem where you want to

know if you can reach a string of terminals from the start know if you can reach a string of terminals from the start

statestate

Do you think it is easier to work forward or backwards?Do you think it is easier to work forward or backwards?

Answer: backwardsAnswer: backwards

14

EECFGCFG is a Decidable Language (cont)is a Decidable Language (cont)

Proof Idea: Proof Idea:

Can the start variable generate a string of terminals?Can the start variable generate a string of terminals?

Determine for each variable if it can generate any Determine for each variable if it can generate any string of terminals and if so, mark itstring of terminals and if so, mark it

Keep working backwards so that if the rightKeep working backwards so that if the right--side of side of any rule has only marked items, then mark the LHSany rule has only marked items, then mark the LHS

For example, if XFor example, if X YZ and Y and Z are marked, then YZ and Y and Z are marked, then mark Xmark X

If you mark S, then done; if nothing else to mark and S If you mark S, then done; if nothing else to mark and S not marked, then rejectnot marked, then reject

15

EQEQCFGCFG is not a Decidable Languageis not a Decidable Language

We cannot reuse the reasoning to show that We cannot reuse the reasoning to show that

EQEQDFADFA is a decidable language since CFGs are is a decidable language since CFGs are

not closed under complement and intersectionnot closed under complement and intersection

As it turns out, EQAs it turns out, EQCFGCFG is not decidableis not decidable

We will learn in Chapter 5 how to prove things We will learn in Chapter 5 how to prove things

undecidableundecidable

16

Every ContextEvery Context--Free Language is DecidableFree Language is Decidable

Note that a few slides back we showed that a ContextNote that a few slides back we showed that a Context--Free Grammar (CFG) is decidable, not a CFLFree Grammar (CFG) is decidable, not a CFL

But since we already know that CFG’s define CFLs this is not But since we already know that CFG’s define CFLs this is not an issue (i.e., we can ignore PDAs if it is easier to prove with an issue (i.e., we can ignore PDAs if it is easier to prove with a CFG than a PDA).a CFG than a PDA).

The proof in the book is so trivial it may confuse youThe proof in the book is so trivial it may confuse you

Essentially it is the same proof we saw beforeEssentially it is the same proof we saw before

That is a CFL is decidable because we can build a TM to That is a CFL is decidable because we can build a TM to decide membership of any stringdecide membership of any string

This leads us to the following picture of the hierarchy This leads us to the following picture of the hierarchy of languagesof languages

17

Hierarchy of Classes of LanguagesHierarchy of Classes of Languages

Regular

Context-Free

Decidable

Turing-recognizable

We proved Regular Context-free since

we can convert a FA into a CFG

We just proved that every Context-free

language is decidable

From the definitions in Chapter 3 it is

clear that every Decidable language is

trivially Turing-recognizable. We hinted

that not every Turing-recognizable

language is Decidable. Next we prove

that!

18

Chapter 4.2Chapter 4.2

The Halting ProblemThe Halting Problem

19

The Halting ProblemThe Halting Problem One of the most philosophically important theorems in One of the most philosophically important theorems in

the theory of computationthe theory of computation There is a specific problem that is algorithmically unsolvable.There is a specific problem that is algorithmically unsolvable.

In fact, ordinary/practical problems may be unsolvableIn fact, ordinary/practical problems may be unsolvable Software verificationSoftware verification

Given a computer program and a precise specification of what the Given a computer program and a precise specification of what the program is supposed to do (e.g., sort a list of numbers)program is supposed to do (e.g., sort a list of numbers)

Come up with an algorithm to prove the program works as requiredCome up with an algorithm to prove the program works as required

This cannot be done!This cannot be done!

But wait, can’t we prove a sorting algorithm works?But wait, can’t we prove a sorting algorithm works?

Note: the input has two parts: specification and task. The proof is Note: the input has two parts: specification and task. The proof is not only to prove it works for a specific task, like sorting numbers.not only to prove it works for a specific task, like sorting numbers.

Our first undecidable problem:Our first undecidable problem: Does a TM accept a given input string?Does a TM accept a given input string?

Note: we have shown that a CFL is decidable and a CFG can be Note: we have shown that a CFL is decidable and a CFG can be simulated by a TM. This does not yield a contradiction. TMs are more simulated by a TM. This does not yield a contradiction. TMs are more expressive than CFGs.expressive than CFGs.

20

Halting Problem IIHalting Problem II

AATMTM = {(M,w)|M is a TM and M accepts w}= {(M,w)|M is a TM and M accepts w}

AATMTM is undecidableis undecidable

It can only be undecidable due to a loop of M on w.It can only be undecidable due to a loop of M on w.

If we could determine if it will loop forever, then could reject. If we could determine if it will loop forever, then could reject.

Hence AHence ATMTM is often called the halting problem.is often called the halting problem.

As we will show, it is impossible to determine if a TM will always halt As we will show, it is impossible to determine if a TM will always halt

(i.e., on every possible input).(i.e., on every possible input).

Note that this is Turing recognizable:Note that this is Turing recognizable:

Simulate M on input w and if it accept, then accept; if it ever rejects, Simulate M on input w and if it accept, then accept; if it ever rejects,

then rejectthen reject

We start with the diagonalization methodWe start with the diagonalization method

21

Diagonalization MethodDiagonalization Method

In 1873 mathematician Cantor was concerned with the In 1873 mathematician Cantor was concerned with the

problem of measuring the sizes of infinite sets.problem of measuring the sizes of infinite sets.

How can we tell if one infinite set is bigger than another or if How can we tell if one infinite set is bigger than another or if

they are the same size?they are the same size?

We cannot use the counting method that we would use for finite sets. We cannot use the counting method that we would use for finite sets.

Example: how many even integers are there?Example: how many even integers are there?

What is larger: the set of even integers or the set of all strings over What is larger: the set of even integers or the set of all strings over

{0,1} (which is the set of all integers){0,1} (which is the set of all integers)

Cantor observed that two finite sets have the same size if each Cantor observed that two finite sets have the same size if each

element in one set can be paired with the element in the otherelement in one set can be paired with the element in the other

This can work for infinite setsThis can work for infinite sets

22

Function Property DefinitionsFunction Property Definitions

From basic discrete math (e.g., CS 1100)From basic discrete math (e.g., CS 1100)

Given a set A and B and a function Given a set A and B and a function ff from A to Bfrom A to B

ff is oneis one--toto--one if it never maps two elements in A to the one if it never maps two elements in A to the same element in Bsame element in B The function The function addadd--twotwo is oneis one--toto--one whereas one whereas absoluteabsolute--valuevalue is notis not

ff is onto if every item in B is reached from some value in a is onto if every item in B is reached from some value in a (i.e., f(a) = b for every b (i.e., f(a) = b for every b B).B). For example, if A and B are the set of integers, then For example, if A and B are the set of integers, then addadd--twotwo is onto is onto

but if A and B are the positive integers, then it is not onto since b but if A and B are the positive integers, then it is not onto since b = 1 is never hit.= 1 is never hit.

A function that is oneA function that is one--toto--one and onto has a (oneone and onto has a (one--toto--one) one) correspondencecorrespondence This allows all items in each set to be pairedThis allows all items in each set to be paired

23

An Example of Pairing Set ItemsAn Example of Pairing Set Items

Let N be the set of natural numbers {1, 2, 3, …} and Let N be the set of natural numbers {1, 2, 3, …} and let E be the set of even natural numbers {2, 4, 6, …}.let E be the set of even natural numbers {2, 4, 6, …}.

Using Cantor’s definition of size we can see that N and Using Cantor’s definition of size we can see that N and E have the same size.E have the same size.

The correspondence f from N to E is f(n) = 2n.The correspondence f from N to E is f(n) = 2n.

This may seem bizarre since E is a proper subset of N, This may seem bizarre since E is a proper subset of N, but it is possible to pair all items, since f(n) is a 1:1 but it is possible to pair all items, since f(n) is a 1:1 correspondence, so we say they are the same size.correspondence, so we say they are the same size.

Definition:Definition:

A set is A set is countablecountable if either it is finite or it has the same size as if either it is finite or it has the same size as N, the set of natural numbersN, the set of natural numbers

24

Example: Rational NumbersExample: Rational Numbers

Let Q = {m/n: m,n Let Q = {m/n: m,n N}, the set of positive N}, the set of positive

Rational NumbersRational Numbers

Q seems much larger than N, but according to Q seems much larger than N, but according to

our definition, they are the same size.our definition, they are the same size.

Here is the 1:1 correspondence between Q and NHere is the 1:1 correspondence between Q and N

We need to list all of the elements of Q and then label We need to list all of the elements of Q and then label

the first with 1, the second with 2, etc.the first with 1, the second with 2, etc.

We need to make sure each element in Q is listed only onceWe need to make sure each element in Q is listed only once

25

Correspondence between N and QCorrespondence between N and Q To get our list, we make an infinite matrix containing all the To get our list, we make an infinite matrix containing all the

positive rational numbers.positive rational numbers. Bad way is to make the list by going rowBad way is to make the list by going row--toto--row. Since 1row. Since 1stst row is row is

infinite, would never get to the second rowinfinite, would never get to the second row

Instead use the diagonals, not adding the values that are equivalentInstead use the diagonals, not adding the values that are equivalent So the order is 1/1, 2/1, ½, 3/1, 1/3, …So the order is 1/1, 2/1, ½, 3/1, 1/3, …

This yields a correspondence between Q and NThis yields a correspondence between Q and N That is, N=1 corresponds to 1/1, N=2 corresponds to 2/1, N=3 That is, N=1 corresponds to 1/1, N=2 corresponds to 2/1, N=3

corresponds to ½ etc.corresponds to ½ etc.

1/11/1 1/21/2 1/31/3 1/41/4 1/51/5

2/12/1 2/22/2 2/32/3 2/42/4 2/52/5

3/13/1 3/23/2 3/33/3 3/43/4 3/53/5

4/14/1 4/24/2 4/34/3 4/44/4 4/54/5

5/15/1 5/25/2 5/35/3 5/45/4 5/55/5

26

Theorem: R is UncountableTheorem: R is Uncountable

A real number is one that has a decimal A real number is one that has a decimal

representation and R is set of Real Numbersrepresentation and R is set of Real Numbers

Includes those that cannot be represented with a Includes those that cannot be represented with a

finite number of digits, like Pi and square root of 2finite number of digits, like Pi and square root of 2

Will show that there can be no pairing of Will show that there can be no pairing of

elements between R and Nelements between R and N

Will find some x that is always not in the pairings Will find some x that is always not in the pairings

and thus a proof by contradictionand thus a proof by contradiction

27

Finding a New Value xFinding a New Value x To the right is an example mappingTo the right is an example mapping

Assume that it is completeAssume that it is complete

I now describe a method that will be guaranteed to I now describe a method that will be guaranteed to generate a value x not already in the infinite listgenerate a value x not already in the infinite list

Generate x to be a real number between 0 and 1 as Generate x to be a real number between 0 and 1 as followsfollows

To ensure that x ≠ f(1), pick a digit not equal to the To ensure that x ≠ f(1), pick a digit not equal to the first digit after the decimal point. Any value not equal first digit after the decimal point. Any value not equal to 1 will work. Pick 4 so we have .4to 1 will work. Pick 4 so we have .4

To x ≠ f(2), pick a digit not equal to the second digit. To x ≠ f(2), pick a digit not equal to the second digit. Any value not equal to 5 will work. Pick 6. We have .46Any value not equal to 5 will work. Pick 6. We have .46

Continue, choosing values along the “diagonal” of Continue, choosing values along the “diagonal” of digits (i.e., if we took the f(n) column and put one digit digits (i.e., if we took the f(n) column and put one digit in each column of a new table). in each column of a new table).

When done, we are guaranteed to have a value x not When done, we are guaranteed to have a value x not already in the list since it differs in at least one already in the list since it differs in at least one position with every other number in the list.position with every other number in the list.

nn f(n)f(n)

11 3.3.114159…4159…

22 55.555.55555…55…

33 0.120.123345…45…

44 0.5000.500000000

.. ..

28

ImplicationsImplications

The theorem we just proved about R being The theorem we just proved about R being uncountable has an important application in the uncountable has an important application in the theory of computationtheory of computation

It shows that some languages are not decidable or It shows that some languages are not decidable or even Turingeven Turing--recognizable, because there are recognizable, because there are uncountably many languages yet only countably uncountably many languages yet only countably many Turing Machines.many Turing Machines.

Because each Turing machine can recognize a single Because each Turing machine can recognize a single language and there are more languages than Turing language and there are more languages than Turing machines, some languages are not recognized by any machines, some languages are not recognized by any Turing machine.Turing machine. Corollary: some languages are not TuringCorollary: some languages are not Turing--recognizablerecognizable

29

Some Languages are Not TuringSome Languages are Not Turing--recognizablerecognizable

Proof:Proof: All strings All strings ∑* is countable∑* is countable

With only a finite number of strings of each length, we may form a list of ∑* With only a finite number of strings of each length, we may form a list of ∑* by writing down all strings of length 0, length 1, length 2, etc.by writing down all strings of length 0, length 1, length 2, etc.

The set of all Turing Machines M is countable since each TM M has an The set of all Turing Machines M is countable since each TM M has an encoding into a string <M>encoding into a string <M> If we simply omit all strings that do not represent valid TM’s, we obtain a list If we simply omit all strings that do not represent valid TM’s, we obtain a list

of all Turing Machinesof all Turing Machines

The set of all languages L over ∑ is uncountableThe set of all languages L over ∑ is uncountable the set of all possible binary sequences B is uncountablethe set of all possible binary sequences B is uncountable

The same diagonalization proof we used to prove R is uncountableThe same diagonalization proof we used to prove R is uncountable

L is uncountable because it has a correspondence with BL is uncountable because it has a correspondence with B See book for details, but assuming ∑ = {sSee book for details, but assuming ∑ = {s11, s, s22, …} we can encode any s, …} we can encode any sii as a as a

binary sequence. Thus, there is a 1:1 mapping.binary sequence. Thus, there is a 1:1 mapping.

Since B is uncountable and L and B are of equal size, L is uncountableSince B is uncountable and L and B are of equal size, L is uncountable

Since the set of TMs is countable and the set of languages is not, we Since the set of TMs is countable and the set of languages is not, we cannot put the set of languages into a correspondence with the set of cannot put the set of languages into a correspondence with the set of Turing Machines. Thus there exists some languages without a Turing Machines. Thus there exists some languages without a corresponding Turing machinecorresponding Turing machine

30

Halting Problem is UndecidableHalting Problem is Undecidable

Prove that halting problem is undecidableProve that halting problem is undecidable

We started this a while ago …We started this a while ago …

Let ALet ATMTM = {<M,w>| M is a TM and accepts w}= {<M,w>| M is a TM and accepts w}

Proof Technique:Proof Technique:

Assume AAssume ATMTM is decidable and obtain a contradictionis decidable and obtain a contradiction

A diagonalization proofA diagonalization proof

31

Proof: Halting Problem is UndecidableProof: Halting Problem is Undecidable

Assume AAssume ATMTM is decidableis decidable

Let H be a decider for ALet H be a decider for ATMTM On input <M,w>, where M is a TM and w is a string, H halts and accepts if M On input <M,w>, where M is a TM and w is a string, H halts and accepts if M

accepts w; otherwise it rejectsaccepts w; otherwise it rejects

Construct a TM D using H as a subroutineConstruct a TM D using H as a subroutine D calls H to determine what M does when the input string is its own D calls H to determine what M does when the input string is its own

description <M>. description <M>.

D then outputs the opposite of H’s answerD then outputs the opposite of H’s answer

In summary:In summary:

D(<M>) accepts if M does not accept <M> and rejects if M accepts <M>D(<M>) accepts if M does not accept <M> and rejects if M accepts <M>

Now run D on its own descriptionNow run D on its own description D(<D>) = accept if D does not accept <D> and reject if D accepts DD(<D>) = accept if D does not accept <D> and reject if D accepts D

A contradiction so H cannot be a decider for AA contradiction so H cannot be a decider for ATMTM

32

The Diagonalization ProofThe Diagonalization Proof

<M1><M1>

<M2><M2> <M3><M3> <M4><M4> …… <D><D>

M1M1 AcceptAccept RejectReject AcceptAccept RejectReject …… AcceptAccept

M2M2 AcceptAccept AcceptAccept AcceptAccept AcceptAccept …… AcceptAccept

M3M3 RejectReject RejectReject RejectReject RejectReject …… RejectReject

M4M4 AcceptAccept AcceptAccept RejectReject RejectReject …… AcceptAccept

..

DD RejectReject RejectReject AcceptAccept AcceptAccept …… ??

..

33

Slightly more concrete versionSlightly more concrete version

You write a program, halts(P, X) in C that takes as input You write a program, halts(P, X) in C that takes as input any C program, P, and the input to that program, Xany C program, P, and the input to that program, X Your program halts(P, X) analyzes P and returns “yes” if P will Your program halts(P, X) analyzes P and returns “yes” if P will

halt on X and “no” if P will not halt on Xhalt on X and “no” if P will not halt on X

You now write a short procedure foo(X):You now write a short procedure foo(X): foo(X) {a: if halts(X,X) then goto a else halt}foo(X) {a: if halts(X,X) then goto a else halt}

This program halts if P does not halt on X; otherwise it does haltThis program halts if P does not halt on X; otherwise it does halt

Does foo(foo) halt?Does foo(foo) halt? It halts if and only if halts(foo,foo) returns noIt halts if and only if halts(foo,foo) returns no

It halts if and only if it does not halt. Contradiction.It halts if and only if it does not halt. Contradiction.

Thus we have proven that you cannot write a program to Thus we have proven that you cannot write a program to determine if an arbitrary program will halt or loopdetermine if an arbitrary program will halt or loop

34

What does this mean?What does this mean?

Recall what was said earlierRecall what was said earlier

The halting problem is not some contrived problemThe halting problem is not some contrived problem

The halting problem asks whether we can tell if some TM M The halting problem asks whether we can tell if some TM M will accept an input stringwill accept an input string

We are asking if the language below is decidableWe are asking if the language below is decidable AATMTM = {(M,w)|M is a TM and M accepts w}= {(M,w)|M is a TM and M accepts w}

It is It is notnot decidabledecidable But as I keep emphasizing, M is a input variable too!But as I keep emphasizing, M is a input variable too!

Of course some algorithms are decidable, like sorting algorithmsOf course some algorithms are decidable, like sorting algorithms

It is TuringIt is Turing--recognizable (we covered this earlier)recognizable (we covered this earlier) Simulate the TM on w and if it accepts/rejects, then accept/reject.Simulate the TM on w and if it accepts/rejects, then accept/reject.

Actually the halting problem is special because it gets at the Actually the halting problem is special because it gets at the heart of the matterheart of the matter

35

CoCo--Turing RecognizableTuring Recognizable

A language is coA language is co--Turing recognizable if it is the Turing recognizable if it is the

complement of a Turingcomplement of a Turing--recognizable languagerecognizable language

Theorem: A language is decidable if and only if it is Theorem: A language is decidable if and only if it is

TuringTuring--recognizable and corecognizable and co--TuringTuring--recognizablerecognizable

Why? To be TuringWhy? To be Turing--recognizable, we must accept in finite recognizable, we must accept in finite

time. If we don’t accept, we may reject or loop (it which case time. If we don’t accept, we may reject or loop (it which case

it is not decidable).it is not decidable).

Since we can invert any “question” by taking the complement, taking Since we can invert any “question” by taking the complement, taking

the complement flips the accept and reject answers. Thus, if we invert the complement flips the accept and reject answers. Thus, if we invert

the question and it is Turingthe question and it is Turing--recognizable, then that means that we recognizable, then that means that we

would get the answer to the original reject question in finite time.would get the answer to the original reject question in finite time.

36

More Formal ProofMore Formal Proof

Theorem: A language is decidable iff it is TuringTheorem: A language is decidable iff it is Turing--recognizable recognizable and coand co--TuringTuring--recognizablerecognizable

Proof (2 directions)Proof (2 directions) Forward direction easy. If it is decidable, then both it and its complement Forward direction easy. If it is decidable, then both it and its complement

are Turingare Turing--recognizablerecognizable

Other direction: Other direction: Assume A and A’ are TuringAssume A and A’ are Turing--recognizable and let M1 recognize A and M2 recognizable and let M1 recognize A and M2

recognize A’recognize A’

The following TM will decide AThe following TM will decide A

M = On input wM = On input w 1.1. Run both M1 and M2 on input w in parallelRun both M1 and M2 on input w in parallel

2.2. If M1 accepts, accept; if M2 accepts, then rejectIf M1 accepts, accept; if M2 accepts, then reject

Every string is in either A or A’ so every string w must be accepted by either Every string is in either A or A’ so every string w must be accepted by either M1 or M2. Because M halts whenever M1 or M2 accepts, M always halts and M1 or M2. Because M halts whenever M1 or M2 accepts, M always halts and so is a decider.so is a decider.

Furthermore, it accepts all strings in A and rejects all not in A, so M is also a Furthermore, it accepts all strings in A and rejects all not in A, so M is also a decider for A and thus A is decidabledecider for A and thus A is decidable

37

ImplicationImplication

Thus for any undecidable language, either the Thus for any undecidable language, either the

language or its complement is not Turinglanguage or its complement is not Turing--

recognizablerecognizable

38

Complement of AComplement of ATMTM is not Turingis not Turing--

recognizablerecognizable

AATMTM’ is not Turing’ is not Turing--recognizablerecognizable

Proof:Proof:

We know that AWe know that ATMTM is Turingis Turing--recognizable but not recognizable but not

decidabledecidable

If AIf ATMTM’ were also Turing’ were also Turing--recognizable, then Arecognizable, then ATMTM would would

be decidable, which it is notbe decidable, which it is not

Thus AThus ATMTM’ is not Turing’ is not Turing--recognizablerecognizable

39

Computer Language Computer Language

TheoryTheory

Chapter 5: ReducibilityChapter 5: Reducibility

Due to time constraints we are only going to cover the first

3 pages of this chapter. However, we cover the notion of

reducibility in depth when we cover Chapter 7.

40

What is Reducibility?What is Reducibility?

A reduction is a way of converting one problem A reduction is a way of converting one problem to another such that the solution to the second to another such that the solution to the second can be used to solve the firstcan be used to solve the first

We say that problem A is reducible to problem BWe say that problem A is reducible to problem B

Example: finding your way around NY City is Example: finding your way around NY City is reducible to the problem of finding and reading a mapreducible to the problem of finding and reading a map

If A reduces to B, what can we say about the relative If A reduces to B, what can we say about the relative difficulty of problem A and B?difficulty of problem A and B?

A can be no harder than B since the solution to B solves AA can be no harder than B since the solution to B solves A

A could be easier (the reduction is “inefficient” in a sense)A could be easier (the reduction is “inefficient” in a sense)

In example above, A is easier than B since B can solve any In example above, A is easier than B since B can solve any routing problemrouting problem

41

Practice on ReducibilityPractice on Reducibility

In our previous class work, did we reduce NFAs In our previous class work, did we reduce NFAs to DFAs or DFAs to NFAs?to DFAs or DFAs to NFAs?

We reduced NFAs to DFAsWe reduced NFAs to DFAs

We showed that an NFA can be reduced (i.e., converted) We showed that an NFA can be reduced (i.e., converted) to a DFA via a set of simple stepsto a DFA via a set of simple steps

That means that NFA can not be any more powerful than That means that NFA can not be any more powerful than a DFAa DFA

Based only on the reduction, the NFA could be less Based only on the reduction, the NFA could be less powerfulpowerful

But since we know this is not possible, since an DFA is a But since we know this is not possible, since an DFA is a degenerate form of an NFA, we showed they have the degenerate form of an NFA, we showed they have the same expressive powersame expressive power

42

How Reducibility is used to Prove How Reducibility is used to Prove

Languages Undecidable Languages Undecidable

If A is reducible to B and B is decidable then what can If A is reducible to B and B is decidable then what can we say?we say? A is decidable (since A can only be “easier”)A is decidable (since A can only be “easier”)

If A is reducible to B and A is decidable then what can If A is reducible to B and A is decidable then what can we say?we say? Nothing (so this is not useful for us)Nothing (so this is not useful for us)

If A is undecidable and reducible to B, then what can If A is undecidable and reducible to B, then what can we say about B?we say about B? B must be undecidable (B can only be harder than A)B must be undecidable (B can only be harder than A)

This is the most useful part for Chapter 5, since this is how This is the most useful part for Chapter 5, since this is how we can prove a language undecidablewe can prove a language undecidable We can leverage past proofs and not start from scratchWe can leverage past proofs and not start from scratch

43

Example: Prove HALTExample: Prove HALTTMTM is Undecidableis Undecidable

Need to reduce ANeed to reduce ATM TM to HALTto HALTTMTM, where A, where ATM TM already already

proven to be proven to be undecidableundecidable

Can use HALTCan use HALTTM TM to solve Ato solve ATMTM

Proof by contradictionProof by contradiction

Assume HALTAssume HALTTMTM is decidable and show this implies Ais decidable and show this implies ATMTM is is

decidabledecidable

Assume TM R that decides HALTAssume TM R that decides HALTTMTM

Use R to construct S a TM that decides AUse R to construct S a TM that decides ATMTM

Pretend you are S and need to decide APretend you are S and need to decide ATMTM so if given input <M, w> must so if given input <M, w> must

output accept if M accepts w and reject if M loops on w or rejects w.output accept if M accepts w and reject if M loops on w or rejects w.

First try: simulate M on w and if it accepts then accept and if rejects then First try: simulate M on w and if it accepts then accept and if rejects then

reject. But in trouble if it loops.reject. But in trouble if it loops.

This is bad because we need to be a decider and This is bad because we need to be a decider and

44

Example: Prove HALTExample: Prove HALTTMTM is Undecidableis Undecidable

Instead, use assumption that have TM R that Instead, use assumption that have TM R that

decides HALTdecides HALTTMTM

Now can test if M halts on wNow can test if M halts on w

If R indicates that M does halt on w, you can use the If R indicates that M does halt on w, you can use the

simulation and output the same answersimulation and output the same answer

If R indicates that M does not halt, then reject since If R indicates that M does not halt, then reject since

infinite looping on w means it will never be infinite looping on w means it will never be

accepted.accepted.

The formal solution on next slideThe formal solution on next slide

45

Solution: Solution: HALTHALTTMTM is Undecidableis Undecidable

Assume TM R decides HALTAssume TM R decides HALTTMTM

Construct TM S to decide AConstruct TM S to decide ATM as followsas follows

S = “On input <M, w>, an encoding of a TM M S = “On input <M, w>, an encoding of a TM M

and a string w:and a string w:

1.1. Run TM R on input <M, w>Run TM R on input <M, w>

2.2. If R rejects, If R rejects, rejectreject

3.3. If R accepts, simulate M on w until it haltsIf R accepts, simulate M on w until it halts

4.4. If M has accepted, If M has accepted, acceptaccept; If M has rejected, ; If M has rejected, rejectreject””