CSC 3130: Automata theory and formal languages

Undecidable Languages

MELJUN P. CORTES, MBA,MPA,BSCS,ACSMELJUN P. CORTES, MBA,MPA,BSCS,ACS

MELJUN CORTESMELJUN CORTES

Are Turing Machines computers?

A Turing Machine does not take the programas part of its input.

computerinput data

instructions(program)

output data

... but it can!

The Universal Turing Machine

Uinput x

program 〈 M 〉

M on input x

The universal TM U takes as inputs a program Mand a string x and simulates M on x

The program M itself is specified as a TM!

Turing Machines as strings

• This Turing Machine can be described by the string

〈 M 〉 = ((q0, qacc, qrej), ((q0, q0, / R☐ ☐ ),

(q0, qacc, 0/0R), (q0, qrej, 1/1R)))

M

q0

qacc

qrej

☐/☐R0/0R

1/1R

The universal Turing Machine

U ((q0, qacc, qrej), ((q0, q0, / R☐ ☐ ),

001 input x

program 〈 M 〉

q0 state

U on input ( 〈 M 〉 , x) will simulate M on input x

Languages about automata

• A TM can take another TM (or DFA, CFG) as input

ADFA = {( 〈 D 〉 , w): D is a DFA that accepts input w}

APDA = {( 〈 P 〉 , w): P is a PDA that accepts w}

ATM = {( 〈 M 〉 , w): M is a TM that accepts w}

Which of these is decidable?

decidable

decidable

Languages about automata

• Idea: Simulate M on input w

• Simulation is correct: – If M accepts w, we accept– If M rejects w, we reject– If M loops on w, we loop

• This TM recognizes but does not decide ATM

ATM = {( 〈 M 〉 , w): M is a TM that accepts w}

Recognizing versus deciding

– This kind of TM is called a decider

• The textbook uses different terminology:

The language recognized by a TM is the set ofall inputs that make it reach qacc A TM decides language L if it recognizes L anddoes not loop on any input

recognizable = recursively enumerable (r. e.)

decidable = recursive

Undecidability

• Turing’s Theorem:

• Before we prove this, let’s observe one thing:

The language ATM is undecidable.

A Turing Machine M can be given its owndescription 〈 M 〉 as an input!

Proof of undecidability

• Proof by contradiction:

Suppose ATM is decidable.

Then there is a TM H that decides ATM:

H〈 M 〉, w

accept if M accepts wreject if M rejects w

or M loops on w

〈 M 〉, 〈 M 〉

accept if M accepts 〈 M 〉reject if M rejects 〈 M 〉

or M loops on 〈 M 〉What happens when w = 〈 M 〉 ?

Proof of undecidability

H〈 M 〉 , 〈 M 〉

accept if M accepts 〈 M 〉reject if M rejects 〈 M 〉

or M loops on 〈 M 〉Let H’ be a TM that does the opposite of H

Hacc

rej

acc

rej

H’

Proof of undecidability

H〈 M 〉 , 〈 M 〉

accept if M accepts 〈 M 〉reject if M rejects 〈 M 〉

or M loops on 〈 M 〉

H’〈 M 〉 , 〈 M 〉

reject if M accepts 〈 M 〉

if M rejects 〈 M 〉 or M loops on 〈 M 〉

accept

Proof of undecidability

H’〈 M 〉 , 〈 M 〉

reject if M accepts 〈 M 〉

if M rejects 〈 M 〉 or M loops on 〈 M 〉

accept

Let D be the following TM:

copy〈 M 〉 〈 M 〉 ,

〈 M 〉 H’

Proof of undecidability

D〈M〉 reject if M accepts

〈 M 〉

if M rejects 〈 M 〉 or M loops on 〈 M 〉

accept

What happens when M = D?

If D accepts 〈 D 〉then D rejects 〈 D 〉

If D rejects 〈 D 〉then D accepts 〈 D 〉so D does not exist!

Proof of undecidability: conclusion

• Proof by contradiction– We assumed ATM was decidable

– Then we had Turing Machines H, H’, D– But D does not exist!

• Conclusion

The language ATM is undecidable.

What happened?

M1

M2

M3

…

0 1 00 …

acc rej rej acc

rej acc loop rej

rej loop rej rej

…

…

Tu

rin

g M

ach

ine

input w

We can write an infinite table for every pair (M, w)

What happened?

M1

M2

M3

…

〈 M1 〉〈 M2 〉 〈 M4 〉…

acc loop rej acc

rej acc rej acc

loop rej loop rej

…

…

Now let’s look only at those w that describe a TM M

〈 M3 〉

What happened?

M1

M2…

〈 M1 〉〈 M2 〉 〈 M4 〉…

acc loop rej acc

rej acc rej acc …

…

If ATM is decidable, then TM D is in the table

〈 M3 〉

D rej rej acc rej

… …

What happened?

M1

M2…

〈 M1 〉〈 M2 〉 〈 M4 〉…

acc loop rej acc

rej acc rej acc …

…D does the opposite of the diagonal entries

〈 M3 〉

D rej rej acc rej

D〈M〉

reject if M accepts 〈 M 〉

accept if M rejects or loops on 〈 M 〉

What happened?

M1

M2

…〈 M1 〉〈 M2 〉 〈 M4 〉…

acc loop rej acc

rej acc rej acc …

…

We run into trouble when we look at (D, 〈 D 〉 )!

〈 M3 〉

D rej rej acc rej

〈 D 〉

loop

acc

?…

Unrecognizable languages

• How about languages that are not recognizable?

The language ATM is recognizable but not decidable.

ATM = {( 〈 M 〉 , w): M is a TM that does not accept w}

= {( 〈 M 〉 , w): M rejects or loops on input w}

The language ATM is not recognizable.

Unrecognizable languages

• General Theorem

• We know ATM is recognizable, so if ATM were also,then ATM would be decidable

• Impossible by Turing’s Theorem

If L and L are both recognizable, then L isdecidable.

Unrecognizable languages

• Proof idea

If L and L are both recognizable, then L isdecidable.

M rej/loop if w ∉ L

accept if w ∈ Lw

M’ rej/loop if w ∈ L

accept if w ∉ Lw

w

accept

reject

Unrecognizable languages

• Proof attempt

Let M = TM for L, M’ = TM for L

On input w,

Simulate M on input w. If it accepts, accept.

Simulate M’ on input w. If it accepts, reject.

If L and L are both recognizable, then L isdecidable.

Problem: If M loops on w, we will never get to step 2!

①

②

Bounded simulation

• Proof

Let M = TM for L, M’ = TM for L

On input w,

Simulate M on input w for k steps. If it accepts, accept.

Simulate M’ on input w for k steps. If it accepts, reject.

If L and L are both recognizable, then L isdecidable.

For k = 0 to infinity

Another undecidable language

• To prove this, we will show that

… so by Turing’s Theorem it is undecidable.

HALTTM = {( 〈 M 〉 , w): M is a TM that halts on input w}

If HALTTM can be decided, so can ATM.

HALTTM is an undecidable language

Undecidability of halting

• Suppose H decides HALTTM

H reject if M loops on w

accept if M halts on w〈 M 〉 , w

If HALTTM can be decided, so can ATM.

• We want to use H to design D for ATM

? reject if M rejects or loops on

w

accept if M accepts w〈 M 〉 , w

Undecidability of halting

H reject if M loops on w

accept if M halts on w〈 M 〉 , w

D

reject if M rejects or loops on

w

accept if M accepts w〈 M 〉 , w

Hrej

acc simulate

M on wrej

acc

More undecidable problems

L1 = { 〈 M 〉 : M is a TM that accepts input }

L2 = { 〈 M 〉 : M is a TM that accepts some input}

L4 = { 〈 M, M’ 〉 : M and M’ accept the same inputs}

Which of these are recognizable?

L3 = { 〈 M 〉 : M is a TM that accepts all inputs}

Gödel’s incompleteness theorem

• Proof due to Turing:

Some mathematical statementsare true but not provable.

If every true statement is provable, then ATM can be decided.

Proof sketch of Gödel’s theorem

• Suppose every true statement has a proof

• For every TM M and input w, one of the statements

is true, so it must have a proof

M accepts w M does not accept wor

Proof sketch of Gödel’s theorem

• Consider this algorithm for ATM:

On input 〈 M 〉 , w:For k = 0 to infinity For all possible proofs p of length k If p is a proof of “M accepts w”, accept. If p is a proof of “M does not accept w”, reject.

One of them is true, so it has a proof pEventually, the algorithm will find this proof

Proof sketch of Godel’s theorem

• Consider this algorithm for ATM:

On input 〈 M 〉 , w:For k = 0 to infinity For all possible proofs p of length k If p is a proof of “M accepts w”, accept. If p is a proof of “M does not accept w”, reject.

Recall that TMs can check if a proof p is correct