Computational Models - Lecture...

Computational Models - Lecture 3 1

Handout Mode

Iftach Haitner and Yishay Mansour.

Tel Aviv University.

March 13/18, 2013

1Based on frames by Benny Chor, Tel Aviv University, modifying frames by MauriceHerlihy, Brown University.

Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 1 / 43

Computational Models - Lecture 3

Non Regular Languages: Two Approaches

(1) The Pumping Lemma

(2) Myhill-Nerode Theorem (not in Sipser’s book)

Closure properties

Algorithmic questions for NFAs

Sipser’s book, 1.4, 2.1, 2.2

Hopcroft and Ullman, 3.4


Proved Last Time

Theorem 1A language, L, is described by a regular expression, R, if and only if Lis regular.

=⇒ construct an NFA accepting R.

⇐= Given a regular language, L, construct anequivalent regular expression

We have made a lot of progress understanding what finite automatacan do. But what can’t they do?


Negative Results

Is there a DFA that accepts

B = {0n1n : n ≥ 0}

C = {w : w has an equal number of 0’s and 1’s}

D = {w : w has an equal number of occurrencesof 01 and 10 substrings}

Consider B:

DFA must “remember” how many 0’s it has seen

impossible with finite state.

The others are exactly the same...

Question: Is this a proof?Answer: No, D is regular!???


Part I

Pumping Lemma


Pumping Lemma

We will show that all regular languages have a special property.

Suppose L is regular.

If a string in L is longer than a certain critical length ` (thepumping length),

then it can be “pumped” to a longer string by repeating an internalsubstring any number of times.

The longer string must be in L too.

This is a powerful technique for showing that a language is notregular.


Pumping Lemma

Lemma 2If L is a regular language, then there is an ` > 0 (the pumping length),where if s is any string in L of length |s| ≥ `, then s may be divided intothree pieces s = xyz such that

1 for every i ≥ 0, xy iz ∈ L,2 |y | > 0, and3 |xy | ≤ `.

Remarks: Without the second condition, the theorem would be trivial.The third condition is technical and sometimes useful.


Pumping Lemma – Proof

Let M = (Q,Σ, δ, q1,F ) be a DFA that accepts L.

Let ` be |Q|, the number of states of M.

If s ∈ L has length at least `, consider the sequence of states M goesthrough as it reads s:

s1 s2 s3 s4 s5 s6 . . . sn↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑

q1 q20 q9 q17 q12 q13 q9 q2 q5 ∈ F

Since the sequence of states is of length |s|+ 1 > `,and there are only ` different states in Q,at least one state is repeated (by the pigeonhole principle).


Pumping Lemma – Proof (cont.)

Write down s = xyz

q1

q9

q5x

y

z

By inspection, M accepts xykz for every k ≥ 0.

|y | > 0 because the state (q9 in figure) is repeated.

To ensure that |xy | ≤ `, pick first state repetition, which must occurno later than `+ 1 states in sequence.


Application # 1

Corollary 3

The language B = {0n1n : n > 0} is not regular.

Proof: By contradiction. Suppose B is regular. Let 2` be the pumpinglength.

Consider the string s = 0`1`.

By pumping lemma s = xyz, where xykz ∈ B for every k ≥ 0.

If y is all 0, then xykz has too many 0’s, for k ≥ 2.

If y is all 1, then xykz has too many 1’s, for k ≥ 2.

If y is mixed, then xykz is not of right form. ♣


Using the Pumping Lemma

How to prove that a language is not regular

Given a language L

An adversary sets the parameter `.

Select a word s ∈ L.

The word s would (usually) depend on `.

The adversary selects a partition s = xyz, such that |y | ≥ 1 and|xy | ≤ `.

Show an index k , such that xykz 6∈ L.

Need to prove for any parameter ` and any partition xyz.


Application # 2

Corollary 4

The language C = {w : #1(w) = #0(w)}is not regular.

Proof: By contradiction. Suppose C is regular. Let ` be the pumpinglength.

Consider the string s = 0`1`.

By pumping lemma s = xyz, where xykz ∈ C for every k ≥ 0.

Since |xy | ≤ ` then y is all 0, and xykz has too many 0’s. ♣


Reflections

What about using (01)` ∈ C?

What about D ={w : w has an equal number of occurrences of 01 and 10 substrings}?


Application # 3

Corollary 5

The language E = {0i1j : i > j} is not regular.

Proof: By contradiction. Suppose E is regular. Let ` be its pumpinglength.

Consider the string s = 0`1`−1.

By pumping lemma s = xyz, where xykz ∈ E for every k ≥ 0,|y | > 0 and |xy | ≤ `

But for k = 0 we have xz /∈ E , contradiction. ♣


Application # 4

Corollary 6

The language Primes ⊂ {1}∗, which contains all strings whose lengthis a prime number, is not regular.

Proof: By contradiction. Suppose Primes is regular, accepted by DFAM. Let ` be the pumping length.

Let s = 1p ∈ Primes, where p ≥ ` is a prime.

By pumping lemma s = xyz, where xykz ∈ Primes for every k .

For k = p + 1 we have xykz = 1p+mp, where |y | = m.

since p(m + 1) is not prime, we have a contradiction. ♣


Another Example

Consider the language L = {aibncn : n ≥ 0, i ≥ 1} ∪ {bncm : n,m ≥ 0},

For any word s ∈ L we can apply the pumping lemma:

If s = aibncn, then set x = ε and y = a.

If s = bncm, then set x = ε and y = b.

If s = cm, then set x = ε and y = c.

Is L regular?!

How can we prove it?!


Part II

Characterization of Regular Languages


The Equivalence Relation ∼L

Let L ⊆ Σ∗ be a language.

Define an equivalence relation ∼L on pairs of strings:

Let x , y ∈ Σ∗. We say that x∼L y if for every string z ∈ Σ∗, xz ∈ L ifand only if yz ∈ L.

It is easy to see that ∼L is indeed an equivalence relation (reflexive,symmetric, transitive) on Σ∗.

In addition, if x∼L y then for every string z ∈ Σ∗, xz∼L yz as well (thisis called right invariance).


The Equivalence Relation ∼L cont.

Like every equivalence relation, ∼L partitions Σ∗ to (disjoint)equivalence classes. For every string x , let [x ] ⊆ Σ∗ denote itsequivalence class w.r.t. ∼L (if x ∼L y then [x ] = [y ] – equality of sets).

Question is, how many equivalence classes does ∼L induce?

∼L finite or infinite?

Well, it could be either finite or infinite. This depends on the languageL.


Three Examples

Let L1 ⊂ {0, 1}∗ contain all strings where the number of 1s isdivisible by 4. Then ∼L1 has finitely many equivalence classes.

Let L2 ⊂ {0, 1}∗ contain all strings of the form 0n1n. Then ∼L2 hasinfinitely many equivalence classes.

Let L3 = {aibncn : n ≥ 0, i ≥ 1} ∪ {bncm : n,m ≥ 0}. Then ∼L3

has infinitely many equivalence classes.

(Proof on the board)


Myhill-Nerode Theorem

Theorem 7Let L ⊆ Σ∗ be a language. Then L is regular⇐⇒ ∼L has finitely manyequivalence classes.

Three specific consequences:

L1 ⊂ {0, 1}∗ contains all strings where the number of 1s isdivisible by 4. Then L1 is regular.

L2 ⊂ {0, 1}∗ contains all strings of the form 0n1n. Then L2 is notregular.

Let L3 = {aibncn : n ≥ 0, i ≥ 1} ∪ {bncm : n,m ≥ 0}. Then L3 isnot regular.


Myhill-Nerode Theorem: Proof

=⇒

Suppose L is regular. Let M = (Q,Σ, δ, q0,F ) be a DFA accepting it.The relation ∼M on pairs of strings is defined as follows: x ∼M y if

δ(q0, x) = δ(q0, y).Clearly, ∼M is an equivalence relation.

Furthermore, if x ∼M y , then xz ∼M yz for every z ∈ Σ∗.Therefore, xz ∈ L if and only if yz ∈ L.

This means that x ∼M y =⇒ x ∼L y (i.e., ∼M is a refinement of ∼L ).


Myhill-Nerode Theorem: Proof cont.

=⇒

The equivalence relation ∼M has finitely many equivalenceclasses (at most the number of states in M).

We saw that x ∼M y =⇒ x ∼L y , so the number of equivalenceclasses of ∼L is less or equal than the number of equivalenceclasses of ∼M .

Therefore, ∼L has finitely many equivalence classes. ♠



⇐=

Suppose ∼L has finitely many equivalence classes. We’llconstruct a DFA M = (Q,Σ, δ, q0,F ) that accepts L.

Let x1, . . . , xn ∈ Σ∗ be representatives for the finitely manyequivalence classes of ∼L.

Q = {[x1], . . . , [xn]}.

δ([xi ], a) = [xia] for all a ∈ Σ.

q0 = [ε].

F = {[xi ] : xi ∈ L}.



⇐=

δ([ε], x) = [x ]

proof: Assume δ([ε], x) = [x ] = [xi ]. By right invarianceδ([ε], xa) = δ([xi ], a) = [xia] = [xa]

Therefore, M accepts x iff x ∈ L

So L is accepted by DFA, hence L is regular. ♠


Example

Construct DFA (via the above method) for L1 ⊂ {0, 1}∗ contains allstrings where the number of 1s is divisible by 5.


Applications of the Proof

Let L be a regular language and let M be a DFA accepting it

The number of equivalence classes of ∼L lowerbounds thenumber of equivalence classes of ∼M , which equals the number ofstates in M.

The equivalence relation ∼M , is a refinement of ∼L (eachequivalence class of ∼L correspond to a union of states).

There is an automata whose number of states equals the numberof equivalence classes of ∼L


Minimizing Automata

Input: An automata MOutput: An automata M ′, such that L(M) = L(M ′) and M ′ has aminimal number of states.

Let S1 = F and S2 = Q − F . Set S = {S1,S2}.

While exists equivalence class Si ∈ S, q1, q2 ∈ Si and σ ∈ Σ suchthat,

δ(q1, σ) ∈ Sj1 and δ(q2, σ) ∈ Sj2 , j1 6= j2, then

let Si,1 = {q ∈ Si : δ(q, σ) ∈ Sj1}, j1 6= i .

S = S − Si ∪ Si,1 ∪ (Si − Si,1)


Minimizing Automata

Output M ′ = (Q′, δ′, q′

0,F′): where

Q′ = S,

q′

0 = S0 ∈ S such that q0 ∈ S0

F ′ = {S1, . . . ,Sk} ⊂ S, such that Si ⊂ F .

δ′(Si , σ) = Sj if for q ∈ Si then δ(q, σ) ∈ Sj .

Claim 8The algorithm terminates, and outputs a (in fact, the) minimalautomata.


Example

q1

q2

a b

a

a b

b

a

b

b

a

r1

s

r2


Part III

Closure Properties of Regular Languages


Simple Closure Properties

Regular languages are closed under complement.

Let M = (Q,Σ, δ, q0,F ) be a DFA that accepts L.

Then M ′ = (Q,Σ, δ, q0,Q − F ) is a DFA that accepts L̄ = Σ∗ \ L.

NFA ?!

Regular languages are closed under intersection.

L1 ∩ L2 = L1 ∪ L2.

Proof with automata ?!


Division

L1/L2 = {x : ∃y ∈ L2, xy ∈ L1}

Examples:

L1 = (01 ∪ 1)∗ and L2 = 00, L1/L2 =?

L1/L2 = ∅.

L3 = a∗b∗c∗ and L4 = b, L3/L4 =?.

L3/L4 = a∗b∗.


Division cont.

Theorem 9Regular languages are closed under division with any language.

Proof:

L1 is a regular language, so it has a DFA M = (Q,Σ, δ, q0,F ).

L2 is an arbitrary language.

For L1/L2 we build M ′ = (Q,Σ, δ, q0,F ′).

F ′ = {q : ∃y ∈ L2, δ(q, y) ∈ F}.

F ′ is well defined, but might be hard to compute – “nonconstructive proof". ♣


Homomorphism

Definition 10Homomorphism: an assignment that replaces each letter with a word,i.e., h : ∆→ Σ∗. For a word w = w1, . . . ,wn, thenh(w) = h(w1) · · · h(wn), ad for a language L, thenh(L) = {h(x) : x ∈ L}.

Example: h(1) = aba, h(0) = aah(010) = aa aba aaL1 = (01)∗, h(L1) = (aaaba)∗.

h(0) = a, h(1) = aL2 = {0n1n|n ≥ 0}, h(L2) = {a2n|n ≥ 0}.

Theorem 11Regular languages are closed under homomorphism.

Proof:Using regular expressions.Using Automata


Inverse Homomorphism

Definition 12

Inverse homomorphism: h−1(w) = {x : h(x) = w},h−1(L) = {x : h(x) ∈ L}

Example: h(1) = aba, h(0) = aaL2 = (ab ∪ ba)∗a, h−1(L2) = {1}.

Claim 13

h(h−1(L))⊆L⊆h−1(h(L)).


Inverse Homomorphism cont.

Theorem 14Regular languages are closed under inverse homomorphism.

Proof idea:

Let M = (Q,Σ, δ, q0,F ) the automata for L, and h : ∆→ Σ∗.

for each letter a ∈ ∆ we advance in M using h(a).

Formally, we define M ′ = (Q,∆, δ′, q0,F ), whereδ′(q, a) = δ(q, h(a)).

Hence, δ′(q,w) = δ(q, h(w))

w ∈ L(M ′)←→ h(w) ∈ L(M) ♣


Using Homomorphism

We know that L1 = {0n1n : n ≥ 1} is not regular.

Show that L2 = {anban : n ≥ 1} is not regular

We will prove using homomorphism and inverse homomorphism

h1(a) = a, h1(b) = b, h1(c) = a.

h2(a) = 0, h2(b) = ε, h2(c) = 1.

h2(h−11 (L2) ∩ a∗b∗c∗) = L1

h−11 (L2) = (a ∪ c)kb(a ∪ c)k

h−11 (L2) ∩ a∗b∗c∗ = {anbcn : n ≥ 1}

h2(h−11 (L2) ∩ a∗b∗c∗) = {0n1n : n ≥ 1}


Part IV

Algorithmic Questions for NFAs


Algorithmic Questions for NFAs

Q.: Given an NFA, N, and a string w , is w ∈ L(N)?

Answer: Construct the DFA equivalent to N and run it on w .

Q.: Is L(N) = ∅?Answer 1: Build a minimal size DFA. (Might be exponential time!)Answer 2: This is a reachability question in graphs: Is there a path inthe states’ graph of N from the start state to some accepting state.There are simple, efficient algorithms for this task.


More Questions

Q.: Is L(N) = Σ∗?

Answer: Check if L(N) = ∅.

Q.: Given N1 and N2, is L(N1) ⊆ L(N2)?

Answer: Check if L(N2) ∩ L(N1) = ∅.

Q.: Given N1 and N2, is L(N1) = L(N2)?

Answer: Check if L(N1) ⊆ L(N2) and L(N2) ⊆ L(N1).

In the future, we will see that for stronger models of computations,many of these problems cannot be solved by any algorithm.


Part V

Summary — Regular Languages


Summary - Regular Languages

So far we saw

finite automata,

regular languages,

regular expressions,

Myhill-Nerode theorem and pumping lemma for regular languages.

Next class we introduce stronger machines andlanguages with more expressive power:

pushdown automata,

context-free languages,

context-free grammars


Date post:	14-Jul-2020
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

Computational Models - Lecture...

Documents