· Relation and Partial/Total Order Relation re exive: (x;x) 2 R. symmetric: (x;y) 2 R ! (y;x) 2...

I237 Formal Language and Automata

Satoshi TojoJAIST

2018

Satoshi Tojo JAIST I237 Formal Language and Automata 2018 1 / 88

1 Mathematical FoundationLogicSetFunctionGraph Theory

2 Regular Grammar and Finite Automata

3 Context-Free Grammar and Pushdown Automata

4 Turing Machine


Logic

Propositional logic

propositional variable: p, q, · · · , true or false (T/F, 1/0).logical connectives: ∧ (and), ∨ (or), → (implies), ¬(negation).

p q ¬p p ∨ q p ∧ q p → q p ↔ q

T T F T T T TT F F T F F FF T T T F T FF F T F F T T

tautology: such formula that is always true.

First-order Logic

universal quantifier: ∀x p(x), For all x , p(x).existential quantifier: ∃x p(x), There exists an x , p(x).bounded variable: bounded by quantifiers.free variable: not bounded by quantifiers.


Set

extensional definition: {a1, a2, ..., an}intensional definition: {x | P(x)} i.e., y ∈ {x | P(x)} ⇐⇒ P(y).{x ∈ A | P(x)} = {x | x ∈ A ∧ P(x)}.subset: A ⊂ B ⇐⇒ ∀x(x ∈ A → x ∈ B). A = B ⇐⇒ A ⊂ B andB ⊂ A. (cf. A ⊆ B)

empty set: ∅ = { }.set of numbers

▶ R : the set of real number.▶ Q : the set of rational number.▶ Z : the set of integers.▶ N : the set of natural number. {x | x ∈ Z ∧ 0 ≤ x} (including 0).

set operations▶ Union: A ∪ B = {x | x ∈ A ∨ x ∈ B}▶ Intersection: A ∩ B = {x | x ∈ A ∧ x ∈ B}▶ Difference: A \ B = {x | x ∈ A ∧ x /∈ B)}▶ Product: A× B = {(x , y) | x ∈ A ∧ y ∈ B}, A2 = A× A.▶ Power set: P(A) = 2A = {C | C ⊂ A}▶ Complement: Ac = {x ∈ U | x /∈ A}


Relation

Let A and B be sets. Then, a relation from A to B is R ⊂ A× B. WhenA = B, R ⊂ A× A is called a relation on A. For relation R,

composition: For R ⊂ A× B and S ⊂ B × C , S ◦ R ⊂ A× C is

S ◦ R = {(x , z) : ∃y ∈ B((x , y) ∈ R ∧ (y , z) ∈ S)}.

identity: iA = {(x , x) : x ∈ A}.iteration: for R ⊂ A× A, Rn = R ◦ Rn−1 (n > 0) and R0 = iA;Rm ◦ Rn = Rm+n.

inverse: R−1 = {(y , x) ∈ B × A : (x , y) ∈ R}.


Relation and Partial/Total Order

Relation▶ reflexive: (x , x) ∈ R.▶ symmetric: (x , y) ∈ R → (y , x) ∈ R.▶ transitive: (x , y) ∈ R ∧ (y , z) ∈ R → (x , z) ∈ R.▶ asymmetric: (x , y) ∈ R → (y , x) /∈ R.▶ anti-symmetric: (x , y) ∈ R ∧ x = y → (y , x) /∈ R.▶ irreflexive: (x , x) /∈ R.

% asymmetric ↔ anti-symmetric ∧ irreflexive. irreflexive ∧ transitive→ asymmetric. asymmetric ∧ transitive → irreflexive.

Equivalence Relation▶ reflexive, symmetric, and transitive.

Order▶ preorder: reflexive and transitive.▶ partial order (⪯): reflexive, transitive, and anti-symmetric▶ strict partial order (≺): irreflexive and transitive.▶ total order (linear order): connected, irreflexive, and transitive.

% connected: either (x , y) ∈ R, (y , x) ∈ R, or x = y .


Mapping and Function

Mapping For a relation R ⊂ A× B,

for any x ∈ A,there is a unique y ∈ B.

Function A mapping between sets of numbers

f : A → B (A : domain,B : range).

f (x) = y or f : x 7→ y for x ∈ A and y ∈ B.

surjection: for any y ∈ B there is such x ∈ A that f (x) = y .

injection: for any x , y ∈ A (x = y) implies f (x) = f (y).

bijection: if f is a surjection and an injection.

If f : A → B is a bijection, there is an inverse function f −1 : B → A

f −1(y) = x where f (x) = y .

% Though there is no f −1, there is f −1(C ) (inverse image) for any C ⊂ B.Satoshi Tojo JAIST I237 Formal Language and Automata 2018 7 / 88

Directed/ Undirected Graph

A graph is a tuple (V ,E ) for a set of verteices (nodes) V and a set ofedges E ,

Undirected Graph E ⊂ {{x , y} | x , y ∈ V }.Directed Graph E ⊂ {(x , y) | x , y ∈ V } = V 2.

Let v1v2 ∈ E be abbreviation for either graph.

Isomorphism For G1 = (V1,E1) and G2 = (V2,E2), if there is suchbijection f : V1 → V2 that

v1v2 ∈ E1 ↔ f (v1)f (v2) ∈ E2,

G1 and G2 are isomorphic and f is an isomorphism.


Terminology

adjacent: for ab ∈ E , i.e., a, b are incident to a common edge.

loop: for vertex a, aa.

degree of v : the number of edges to which v is incident.

isolated point: a vertex with degree 0.

end-vertex: a vertex with degree 1.

Handshaking Lemma: The sum of degree in an undirected graphbecomes an even number.

subgraph: for G = (V ,E ), G ′ = (V ′,E ′) where V ′ ⊂ V and E ′ ⊂ Eis a subgraph of G .

complete graph: every two vertices is connected by an edge.

empty graph: graph with no edges.

regular graph: every vertex has the same degree.

bipartie graph: a graph whose vertices can be divided into twodisjoint and independent sets U and V , such that every edgeconnects a vertex in U to one in V .


Walk, Trail, Path, and Cycle

walk: a sequence of vertices w = a1....an, where any aiai+1 ∈ E .

connected: if there is a walk for any two vertices.

strongly connected: for a directed graph if there is a walk from anyvertex to any other vertices.

bridge: such an edge without which the graph becomes unconnected.

trail: a walk with all different edges.

path: a walk with all different vertices except the start and the goal.

closed: if the start and the goal are identical.

cycle: a closed path.

Ex.

v→w→x→w→y→x is a walk.

v→w→x→y→z→x is a trail.

v→w→x→y→z→x→v is a closed trail.

v→w→x→y→v is a cycle.Satoshi Tojo JAIST I237 Formal Language and Automata 2018 10 / 88

Eulerian/ Hamilton Graph

Eulerian Graph When connected graph G has a closed trail with all theedges, i.e., the trail passes all the edges just once, G is calledan Eulerian graph.

Hamilton Graph When connected graph G has a cycle which visits all thevertices just once, G is called a Hamilton graph and the pathis called Hamilton cycle. To decide if a given graph isHamilton or not, is NP-complete.

Weighted Graph A graph where each edge is weighted, is called aweighted graph. For a given graph, finding such a Hamilton cycle thatminimizes the sum of the weights of visited edges, is called the travelingsalesman problem and is known as NP-hard.


Tree

A graph with no cycle is called a forest; a connected forest is called a tree.The followings are equivalent.

T is a tree.

T has no cycle, and has n − 1 edges.

T is connected, and has n − 1 edges.

T is connected and all the edges are bridges.

T has only one walk between any two vertices.

T has no closed trail, but any additional edge makes a closed trail.

Corollary When forest G has n vertices with k trees, G has n − k edges.


1 Mathematical Foundation

2 Regular Grammar and Finite AutomataFinite StringsDeterministic Finite AutomataNondeterministic Finite AutomataSubset Constructionϵ-transitionRegular ExpressionEquivalence between RE and FAClosure Property and DecidabilityIsomorphismPumping LemmaMinimizationMyhill-Nerode Relation


4 Turing MachineSatoshi Tojo JAIST I237 Formal Language and Automata 2018 13 / 88

Finite Strings

For a non-empty set Σ,

symbol: member of Σ.

length: for a string x of members of Σ, the number of symbols,written as |x |.empty string: the string of length 0, written as ϵ.

For u ∈ Σ, let u0 = ϵ and un+1 = unu.

Let all the strings consists of the members of Σ be Σ∗. ϵ ∈ Σ∗.

Ex. for Σ = {a, b},Σ∗ = {ϵ, a, b, aa, ab, ba, bb, ...}|abb| = 3.|ϵ| = 0.ab = ba, aab = ab.a3 = aaa.


Concatenation

For two strings x and y , xy means their concatenation.

Ex.

For x = aab and y = ba, xy = aabba.

x = ϵx = xϵ.

x0 = ϵ, xn+1 = xnx

Ex. For x = ab, x5 = ababababab.Note that for x , y , z , (xy)z = x(yz).


Set Operations

For A ⊂ Σ∗ and B ⊂ Σ∗,

A ∪ B = {x : x ∈ A ∨ x ∈ B}A ∩ B = {x : x ∈ A ∧ x ∈ B}Ac =∼ A = {x : x ∈ Σ∗ ∧ x /∈ A}AB = {xy : x ∈ A ∧ y ∈ B}A0 = {ϵ}, An+1 = {x : x ∈ AnA}A∗ =

∪n≥0 A

n, A+ =∪

n≥1 An

Ex. For A = {ab} and B = {ba, aa},AB = {abba, abaa}An = {(ab)n}


Deterministic Finite Automata

Deterministic Finite Automaton, DFA M is given by (Q,Σ, δ, s,F ), where

Q: a finite set of statesΣ: a finite set of input alphabetδ : Q × Σ → Q: transition functionδ(current state，one character) = next states ∈ Q: initial (start) statesF ⊂ Q: accept (final) states


Ex. Q = {q0, q1, q2, q3}, Σ = {0, 1}, s = q0, F = {q0},δ(q0, 0) = q0, δ(q0, 1) = q1, δ(q1, 0) = q1, δ(q1, 1) = q2, δ(q2, 0) =q2, δ(q2, 1) = q3, δ(q3, 0) = q3, δ(q3, 1) = q0.

0 1→ q0F q0 q1

q1 q1 q2q2 q2 q3q3 q3 q0

q0 q1 q2 q3

0 0 0 0

1 1 1

1(double circle means final state)


Acceptance

Given DFA M = (Q,Σ, δ, s,F ) and string x = x0 . . . xn ∈ Σ∗, transitionfunction for a string δ is defined by the transition function recursively asfollows.

δ(q, ϵ) = q, δ(q, xa) = δ(δ(q, x), a) where x ∈ Σ∗, a ∈ Σ.

When δ(s, x) ∈ F , x is accepted; otherwise, rejected.

L(M) = {x ∈ Σ∗ : δ(s, x) = q, q ∈ F}: language accepted by M. A set ofstrings accepted by a finite automaton A ⊂ Σ∗ is called a regular set.

Ex. {xaaay : x , y ∈ {a, b}∗} ⊂ {a, b}∗ is a regular set.


Nondeterministic Finite Automata

Nondeterministic Finite Automaton, NFA) M is given by (Q,Σ,∆,S ,F )where

Q: a finite set of states

Σ a finite set of input alphabets

∆ : Q × Σ → 2Q : transition rules (note 2Q = P(Q)).

S ⊂ Q: initial states, start states

F ⊂ Q: accept (final) states

Ex. Q = {q0, q1, q2}, Σ = {a, b}, s = {q0, q1}, F = {q1, q2} で,∆(q0, a) = {q0, q1},∆(q1, a) = {q0, q2},∆(q2, a) = ∅,∆(q, b) = {q}(q ∈Q).

a b→ q0 {q0, q1} {q0}→ q1F {q0, q2} {q1}

q2F ∅ {q2}q0 q1 q2

a, b b b

a aa


Acceptance by NFA

String x is accepted by NFA M = (Q,Σ,∆,S ,F ) when NFA has apossibility to reach a final state after reading the whole x . Ex.

q0 q1 q2

0,1

1 0,1

010 and 111 are accepted.In general, when the penult symbol is 1 the string is accepted;otherwise, rejected.


String x is accepted by NFA M = (Q,Σ,∆,S ,F ) if ∆ : 2Q × Σ∗ → 2Q

and ∆(S , x) ∩ F = ∅; otherwise rejected.

∆(A, ϵ) = A ∆(A, xa) =∪

q∈∆(A,x)

∆(q, a).

A

∆(A, x)

x

x a

∆(A, xa)

x aa


Ex. Automata that accept x , the length of which is a multiple of 2 or 3.

NFA

q0 q10,1

q2 q3 q40,1 0,1

0,1

0,1

FA

q0 q1 q20,1 0,1

q5 q4 q30,1 0,1

0,10,1


Subset Construction

Theorem For language L(M) ⊂ Σ∗ accepted by NFA M, there exists aDFA M that accepts the same L(M).(Proof) We employ subset construction. For NFA M = (Q,Σ,∆,S ,F ), wedefine a new DFA M ′ = (Q ′,Σ, δ, s,F ′) as follows.

Q ′ = 2Q δ(A, a) = ∆(A, a) =∪p∈A

∆(p, a)

s = S F ′ = {A ∈ 2Q : A ∩ F = ∅}

Then, M ′ accepts L(M).


Ex.

1 NFA0 1

→ q0 {q0} {q0, q1}q1 {q2} {q2}q2F ∅ ∅

q0 q1 q2

0,1

1 0,1

2 DFA0 1

→ q{0} q{0} q{0,1}q{0,1} q{0,2} q{0,1,2}q{0,2}F q{0} q{0,1}q{0,1,2}F q{0,2} q{0,1,2}

q{0} q{0,2} q{0,1,2}

0

0 0

q{0,1}

01

11

1


ϵ-transition

ϵ-NFA: NFA which includes ϵ-transition.

Ex.

ϵ-NFA that accpets b, bb, bbb.

q0 q1 q2

q3 q4 q5

b b b

ϵ ϵ

ϵ ϵ

ϵ-NFA that accepts |x | = 2, 3.

q0 q10,1

q2 q3 q40,1 0,1

0,1

0,1

q6ϵ

ϵ


ϵ-NFA

ϵ-closure: a set of states Cϵ(X ) reachable from a set of states X , via zeroor more ϵ moves.

Cϵ(X ) = ∆(X , ϵ∗) =∪p∈X

∆(p, ϵ∗).

For ϵ-NFA (Q,Σ ∪ {ϵ},∆, S ,F ) and X ⊂ Q, define ∆ as follows.

∆(X , ϵ) = Cϵ(X )

∆(X , xa) = Cϵ(∪

r∈∆(X ,x)∆(r , a))

Then, ∆(Q, x) ∩ F = ∅ iff x ∈ Σ∗ is accepted.


ϵ-closure

∆(X , ba) = Cϵ(∪

q∈∆(X ,b)

∆(q, a))

= Cϵ(∪

q∈Cϵ(∪

r∈∆(X ,ϵ) ∆(r ,b))

∆(q, a))

= Cϵ(∪

q∈Cϵ(∪

r∈Cϵ(X ) ∆(r ,b))

∆(q, a))

X

∆(Cϵ(X ), x)

x

a

∆(Cϵ(X ), xa)

x aa

ϵϵϵ

ϵ

Cϵ(X )


Get rid of ϵ-transition

Theorem For an ϵ-NFA M, there exists such NFA N without ϵ-transitionthat L(M) = L(N).(Proof) When M = (Q,Σ ∪ {ϵ},∆, S ,F ), let new NFA N be(Q,Σ,∆′,S ,F ′) where

∆′(q, a) =∪

r∈Cϵ({q})

∆(r , a), F ′ = {q : Cϵ({q}) ∩ F = ∅}.

Cϵ({q})a

∆′(q, a)

aa

ϵq

Note that the set of states Q has not been changed.


Ex.

1 NFA with ϵ-transitionϵ a b

→ q0 {q1} {q0} ∅q1F ∅ ∅ {q1}

q1

b

ϵq0

a

2 NFA without ϵ-transitiona b

→ q0F {q0} {q1}q1F ∅ {q1} q1

b

bq0

a


Regular Expression

Regular expression (RE) on Σ is such α that

α ::= ∅ | ϵ | a | α1α2 | α1 + α2 | α∗1

where a ∈ Σ.

Regular expression α defines language L(α) as follows.

L(∅) = ∅L(ϵ) = {ϵ}L(a) = {a}L(α1α2) = L(α1)L(α2)

L(α1 + α2) = L(α1) ∪ L(α2)

L(α∗) = (L(α))∗


Regular SetThe language defined by a regular expression is a regular set. L(α) = L(β)iff α ≡ β.

∅+ α ≡ α ≡ α+∅,

∅α ≡ ∅ ≡ α∅,

α(β + γ) ≡ αβ + αγ,

α+ α ≡ α,

ϵα ≡ αϵ,

(α+ β)γ ≡ αγ + βγ,

α+ β ≡ β + α,

α(βγ) ≡ (αβ)γ,

αα∗ ≡ α∗α

α+ (β + γ) ≡ (α+ β) + γ,

ϵ+ αα∗ ≡ α∗


RE ⇒ FA

1 For α1α2, compose such a part thatα1 α2ϵ

2 For α∗, compose such a part that

α

ϵ

ϵ

3 For α1 + α2, compose such a part that

α2

α1

Continue this process until whole the regular expression is decomposed.


FA ⇒ RELet DFA (and thus no ϵ-transition) be M = (Q,Σ, δ, s,F ).

α∅pq =

{+{a ∈ Σ | q = δ(p, a)} if p = q

+{a ∈ Σ | q = δ(p, a)}+ ϵ if p = q

αXpq = α

X\{r}pq + α

X\{r}pr (α

X\{r}rr )∗α

X\{r}rq

Then, αXpq represents such x that q = δ(p, x), where the passable states

are restricted to X except p and q. As a result, L(∑

f ∈F αQsf ) = L(M).

Ex. When Q = {s, p, q, f }, F = {f }, δ(s, a) = f , δ(s, b) = p, δ(p, c) =q, δ(q, d) = p, δ(q, a) = f，

α{s,p,q,f }sf = α

{s,p,f }sf + α

{s,p,f }sq (α

{s,p,f }qq )∗α

{s,p,f }qf = a+ bc(dc)∗a.

α{s,p,f }sf = a, α

{s,p,f }qq = dc

α{s,p,f }sq = α

{s,f }sq + α

{s,f }sp (α

{s,f }pp )∗α

{s,f }pq = ∅+ bϵc = bc

α{s,p,f }qf = α

{s,f }qf + α

{s,f }qp (α

{s,f }pp )∗α

{s,f }pf = a+ dϵ∅ = a


Closure Property and Decidability

If A ⊂ Σ∗ is a regular set, Ac is a regular set. Let L = Σ∗ \ L, then Lis accepted by (Q,Σ, δ, s,Q \ F ).If A,B ⊂ Σ∗ is a regular set, A ∩ B is a regular set. Let a new statebe the pair of states from existing automata.

If A,B ⊂ Σ∗ is a regular set, A ∪ B is a regular set.

As a result,

The difference of two regular languages is regular.

The reversal (all strings are written backwards) of a regular languageis regular.

The closure (star) of a regular language is regular.

The concatenation of regular languages is regular.

Regular set is decidable, that is, there exists a finite automaton to decideif a given string x belongs to the set or not.


Homomorphism

Let Σ1,Σ2 be two sets of alphabet. The following h : Σ∗1 → Σ∗

2 is calledhomomorphism.

∀x , y ∈ Σ∗1, h(xy) = h(x)h(y) ∈ Σ∗

2

When h is homomorphism, |h(ϵϵ)| = |h(ϵ)h(ϵ)| = |h(ϵ)|+ |h(ϵ)| and thus|h(ϵ)| = 0, i.e., h(ϵ) = ϵ.

Ex.

1 For Σ1 = Σ2 = {0, 1}, h(0) = 0, h(1) = 11 is a homomorphism.2 For Σ1 = Σ2 = {0, 1}, h(x) = x2 is not a homomorphism.

A homomorphic bijection is called isomorphism.


Homomorphism and Regular Set

Theorem For homomorphism h : Σ∗1 → Σ∗

2, let A ⊂ Σ∗1 and B ⊂ Σ∗

2.

1 If A is a regular set, h(A) is a regular set.(Proof) For a regular expression α, we define function h : A → Σ∗

2 asfollows, to show L(h(α)) = h(L(α)).

h(∅) = ∅, h(ϵ) = ϵ, h(a) = h(a),

h(α+ β) = h(α) + h(β), h(αβ) = h(α)h(β), h(α∗) = (h(α))∗.

2 If B is a regular set, h−1(B) is a regular set.(Proof) For such DFA M = (Q,Σ2, δ, s,F ) that L(M) = B, we definea new DFA N = (Q,Σ1, δ

′, s,F ) as follows.

δ′(q, a) = δ(q, h(a)).

Note that we use δ (for string) since h(a) may not be a single letter.


Pumping LemmaFor the number of states n of FA, we consider a string of length m (> n).When δ(q0, a1a2 · · · ai ) = qi , q1 ∼ qm cannot be all different and thusthere must be some identical states.

q0qj = qk

qm

Pumping Lemma Given A ⊂ Σ∗, there exists such n that for all strings|w | ≥ n (w ∈ A) there are strings x , y , z ∈ Σ∗ satisfying the followingconditions.

w = xyz , y = ϵ, |xy | ≤ n, and xy iz ∈ L (i ≥ 0).

That is,

∃n ∀w (|w | ≥ n) ∃y (w = xyz) ∀i ≥ 0 [w ∈ L → xy iz ∈ L].

(Proof) Let |Q| = n, and y be the looping part.Satoshi Tojo JAIST I237 Formal Language and Automata 2018 38 / 88

Games with Demon

We would like to prove L is not a regular language.

1 Demon says number n randomly.

2 You can provide a sentence w (∈ L) where |w | ≥ n.

3 Even though the demon divides w into xyz arbitrarily,

4 You can show xy iz ∈ L.

∀n ∃w (|w | ≥ n) ∀y (w = xyz) ∃i ≥ 0 [w ∈ L ∧ xy iz ∈ L].

Ex. We prove ambm is not a regular language. When the demon suggestsa random n, you can provide w = anbn. The demon tries to divide w intow = xyz .

When the demon makes y inside of an, xy iz = an+ibn ∈ L.

When the demon makes y including the boundary of a and b, xy izbecomes · · · ababab · · · ∈ L.

When the demon makes y inside of bn, xy iz = anbn+i ∈ L.


Minimization of the Number of States

We may regard the two states are the same if we can reach the final statefrom them by the same strings.

f

q1

q2

x

xf

q{1,2}

x

Even when q1 → f and q2 → f can accept the same string x , if there is apath in q1 → q2 (excluding ϵ) we cannot regard q1 and q2 as the same

f

q1

q2

x

x

since q1 → f and q2 → f may accept different strings.


Equivalence on QLet ≈ be the equivalence relaton for Q.

p ≈ qdef⇐⇒ ∀x ∈ Σ∗(δ(p, x) ∈ F ↔ δ(q, x) ∈ F )

Then, let M≈ = (Q ′,Σ, δ′, s ′,F ′) be

Q ′ = Q/ ≈ δ′([q], a) = [δ(q, a)]

s ′ = [s] F ′ = {[q] : q ∈ F}

where [p] ≡ {q | q ≈ p}, p ≈ q ⇐⇒ [p] = [q].

Then,

p ∈ F ↔ [p] ∈ F ′.For all x ∈ Σ∗, [δ(q, x)] = δ′([q], x).L(M) = L(M≈)M≈ = (M≈)≈.M≈ is the minimal DFA to accept A.


Minimization Algorithm

To minimize M = (Q,Σ, δ, s,F ),

1 Write a table of pair {p, q} in Q.

2 Check such a pair that p ∈ F and q /∈ F where the check means ≈.(Note that when Q = F all the states becomes the same if thetransition function δ is total.)

3 When pair {δ(p, a), δ(q, a)} is already checked for some a ∈ Σ, alsocheck {p, q}.

4 Repeat the above checking operation exhaustively, and remaining pair{p, q} becomes p ≈ q.


Ex. Minimize DFA accepting {x ∈ {a, b}∗ : |x | = 1 ∨ |x | ≥ 3}.

6

2 a, b4a

1

3 5

a

ab

a, b

a, b

b

b

1 2 3 4 5 6

1 - - - - - -2 ✓ - - - - -3 ✓ - - - -4 ✓ ✓ ✓ - - -5 ✓ ✓ ✓ - -6 ✓ ✓ ✓ ✓ ✓ -

{1}{2, 3} {4, 5}

{6}

a,b

a,b a,b a,b


Ex. Minimize the following DFA.

6

2 a, b4a

1

3 5

a

ab

a, b

a, b

b

b

1 2 3 4 5 6

1 - - - - - -2 ✓ - - - - -3 ✓ - - - -4 ✓ ✓ ✓ - - -5 ✓ ✓ ✓ - -6 ✓ ✓ ✓ -

q0 = [1] = {1},q1 = [2] = [3] = {2, 3},q2 = [4] = [5] = [6] = {4, 5, 6}

q0 q1 q2

a, b

a, b a, b


Myhill-Nerode RelationGiven R ⊂ Σ∗, regular or not, we define an equivalence relation ≡R , by

x ≡R ydef⇐⇒ ∀z ∈ Σ∗ (xz ∈ R ↔ yz ∈ R).

When the number of equivalence classes is finite, Myhill-Nerode Relation.

If there is an MNR, R becomes regular. Let finite states

[x ]def⇐⇒ {y | y ≡R x}

and construct DFA (Q,Σ, δ, s,F ) as follows.

Q = {[x ] : x ∈ Σ∗}, s = [ϵ], F = {[x ] : x ∈ R}, δ([x ], a) = [xa].

If R is regular, there is an MNR. Let M = (Q,Σ, δ, s,F ) be the FA toaccept regular set R. Then the following ≡M satisfies the condition.

x ≡M ydef⇐⇒ δ(s, x) = δ(s, y)

where {x ∈ Σ∗ | δ(s, x) = q ∈ Q} are finite, since |Q| is finite.If we define by ∀a ∈ Σ(x ≡ y ⇒ xa ≡ ya), MNR becomes finer-grained.The coarsest MNR corresponds to the minimal DFA.


aq0 q1 q3

a, bbb

aq2

b

a

Ex. A = {x ∈ {a, b}∗ | x = anbn} is not a regular set. If k = m, thoughambm ∈ A, akbm ∈ A. Thus, we cannot classify ak and am to the sameequivalence class. Since there is a class for ak for any k ≥ 0, the numberof classes by ≡A is not finite.




3 Context-Free Grammar and Pushdown AutomataContext-Free Grammar/ LanguageChomsky Normal FormPumping LemmaGreibach Normal FormPushdown AutomataEquivalence of AcceptanceEquivalence between CFG and NPDAChomsky Hierarchy

4 Turing Machine


Context-Free Grammar

Context-Free Grammar (CFG) G is given by (N,Σ,P,S).

N: a finite set of non-terminal symbols (variables).

Σ: a finite set of terminal symbols (constants).

P: a finite subset of N × (N ∪ Σ)∗, called production rules.

S ∈ N: start symbol.

Let G = (N,Σ,P,S). (A, α) is written by A → α, and A → α1, A → α2,and A → α3 are collectively written as A → α1|α2|α3.

Ex. Let G = (N,Σ,P,S).

N = {S}Σ = {a, b}P = {S → aSb | ϵ}


Derivation

Let G = (N,Σ,P,S).

For all α, α0→G

α.

α1→G

β: β when one of the non-terminal symbols in α is replaced.

When αn→G

γ and γ1→G

β, αn+1→G

β.

When there exists n such that αn→G

β, α∗→G

β and is called β is

derivable from α.L(G ) = {x ∈ Σ∗ : S

∗→G

x} is the context-free language (CFL),

generated by G .

Ex. G = (N,Σ,P,S) = ({S}, {a, b}, {S → aSb | ϵ})

S1→G

aSb1→G

aaSbb1→G

aaaSbbb1→G

aaabbb


Parse Tree

Ex. The parse tree of aabb from CFG S → aSb | ϵ.

When the derivation from CFG is not unique, the grammar is calledambiguous.


Getting rid of ϵ-Production/ Unit Production

Lemma For any CFG G = (N,Σ,P,S), there exists such CFG G ′ that

No unit production: A → B (A,B ∈ N).

No ϵ-production: A → ϵ, and thus L(G ′) = L(G ) \ {ϵ}.(Proof) Let P be the minimal expansion of P such that

1 If A → αBβ and B → ϵ ∈ P, let A → αβ ∈ P.

2 If A → B and B → γ ∈ P, let A → γ ∈ P.

Then,

in G = (N,Σ, P, S), P is finite.

(S∗→G

β) ↔ (S∗→G

β).

For any x (= ϵ) ∈ L(G ), such rules as A → ϵ or A → B are not used.Then, let G ′ = (N,Σ,P ′,S) where P ′ = P \ {A → ϵ,A → B | A,B ∈ N}.


Chomsky Normal FormChomsky Normal Form (CNF): every grammar rule is in the form{

A → BC

A → a

where A,B,C ∈ N, a ∈ Σ, and is ϵ-free.

Theorem For any CFG G = (N,Σ,P, S), there exists CNFG ′ = (N ′,Σ,P ′, S) that L(G ′) = L(G ) \ {ϵ}.(Proof) Repeat the following procedure until saturated.

1 For any a ∈ Σ, add a new non-terminal symbol Aa as N ′ = N ∪ {Aa},and P ′ = P ∪ {Aa → a}.

2 Replace all b ∈ Σ in P ′, except those in the form of A → a, with Ab.Thus, the rules in P ′ are restricted to the following two kinds: eitherA → a or A → B1 · · ·Bk (B1, ...,Bk ∈ N, n ≥ 2).

3 For A → B1 · · ·Bk (k ≥ 3), add a new symbol C to N ′ and replace itwith {A → B1C , C → B2 · · ·Bk}.Satoshi Tojo JAIST I237 Formal Language and Automata 2018 52 / 88

CYK-algorithm

Let CNF of a CFG be S → AB | BC ,

A → BA | a,B → CC | b,C → AB | a.

Given an input baaba, we can fill out the following triangular table, withpossible non-terminal symbols.

{S ,A,B}∅ {S ,A,C}∅ {B} {B}

{S ,A} {B} {S ,C} {S ,A}{B} {A,C} {A,C} {B} {A,C}b a a b a


Pumping Lemma of CFL

Theorem For any CFL A there exists k ≥ 0, and for any z ∈ A (|z | ≥ k)there exists such u, v ,w , x , y that

z = uvwxy , vx = ϵ, |vwx | ≤ k, ∀i (≥ 0) uv iwx iy ∈ A.

(Proof) Let G = (N,Σ,P,S) be in CNF.

1 For |N| = n, let k = 2n+1

2 For z ∈ L(G ) such that |z | ≥ k , since there must be the longest(deepest) path more than n in the parse tree, the same A ∈ Nappears more than twice.

3 Replace the second (deeper) A by the subtree from the first(shallower) A.

4 Let w be the sequence of leaves in the above subtree. Then, letw ′ = vwx , taking v and x .

5 Let z = uvwxy , taking u and y .


If A → u A v , then an embedding procedure results in A → u(u A v)v．This embedding can be repetitive, and thus S → x ui y v i z .


Games with Demon

We prove L is not CFL.

1 The demon tells you a random number k．2 You can provide a sentence z (|z | ≥ k) ∈ L.

3 Even though the demon divide z into uvwxy in any way,

4 You can show that when |vwx | ≤ k , uv iwx iy ∈ L．

Ex. We show A = {anbnanbn | n > 0} is not CFL．For any k，providez = akbkakbk . For any division z = uvwxy，i.e.,

when either v or x includes the boundary of a and b,

when both of v and x are in ak , or both in bk ,

when v is chosen from the inside of bk , and x is so from ak (or vfrom ak and x from bk), （|vxw | ≤ k implies that ak and bk must beadjacent. ）

uv2wx2y ∈ A.


Greibach Normal Form

Greibach Normal Form (GNF): Every generation rule is in the form:{A → aB1B2 · · ·Bk

A → a

where a ∈ Σ, {A,Bi} ∈ N, k ≥ 0.

Lemma The following replacement produces the same language.

1 Remove A → Bγ, and from B → β1 | β2 add A → β1γ | β2γ.Ex. {A → BA︸︷︷︸

remove

,B → a | bA} ∪ {A → aA | bAA}.

2 Remove A → Aγ, and from other A-rules: A → β1 | β2 and with anew non-terminal symbol C , add A → β1C | β2C and C → γ | γC .Ex. {A → Ab︸︷︷︸

remove

,A → c | aB} ∪ {A → cC | aBC ,C → b | bC}.

Theorem For any CFG G , there is GNFG ′ such that L(G ′) = L(G ) \ {ϵ}.


Conversion to GNF

Let CFG be CNF, and Ak (1 ≤ k ≤ m) be all the non-terminal symbols.

1 For i = 1 to m, apply the first lemma for such Ai → Ajγ that i > j . Ifi = j apply the second lemma. Then, all the productions will be either

Ai → Ajγ (i < j), Ai → aγ, Ci → γ.

2 Now, Am → amγ (am ∈ T ) when m is the largest index. Then, for anyk downward from m, apply Ak → akγ to Ak−1 → Akγ. Accordingly,every Ai has a production starting from a terminal symbol.

3 Finally, convert each Ci applying the two lemmas.

Note that all symbols after the first in any production body arenon-terminal symbols, if we start from CNF.

Ex. {A1 → A2A2 | a,A2 → A1A1 | b} becomes{A1 → a | bA2 | aA1A2 | aA1CA2 | bCA2, A2 → b | aA1C | bC | aA1,C →bA1 | aA1CA1 | bCA1 | aA1A1 | bA1C | aA1A1C | aA1CA1C | bCA1C}.


Pushdown Automata

Non-deterministic Pushdown Automaton (NPDA) M is the followingtuple: (Q,Σ, Γ, δ, s,⊥,F ).

Q: a finite set of states

Σ: a finite set of input alphabet

Γ: a finite set of stack alphabet

δ ⊂ (Q × (Σ ∪ {ϵ})× Γ)× (Q × Γ∗): a transition function

s ∈ Q: start state

⊥ ∈ Γ: initial stack

F ⊂ Q: a finite set of final states


Configuration and Transition

configuration: (p, abaabba︸︷︷︸remaining string

, A︸︷︷︸stack top

)

transition: ((p, a,A), (q,B1B2)) ∈ δ.

When M is reading an input letter a at state p with stack top A,▶ the head moves to the next input letter,▶ the state shifts from p to q, and▶ M pops A out from the stack and pushes B1B2 (B1 is the stack top

and the B2 is to the bottom).


Acceptance

Given PDA M = (Q,Σ, Γ, δ, s,⊥,F ),

For ((p, a,A), (q, γ)) ∈ δ, y ∈ Σ∗, β ∈ Γ∗, (p, ay ,Aβ)1→M

(q, y , γβ).

For ((p, ϵ,A), (q, γ)) ∈ δ, y ∈ Σ∗, β ∈ Γ∗, (p, y ,Aβ)1→M

(q, y , γβ).

For C ,D ∈ Q × Σ∗ × Γ∗,

▶ C0→M

Ddef↔ C = D

▶ Cn+1→M

Ddef↔ ∃E (C n→

ME ∧ E

1→M

D)

▶ C∗→M

Ddef↔ ∃n(C n→

MD)

Acceptance: M accepts x ∈ Σ∗

By final state: for q ∈ F and γ ∈ Γ∗, (s, x ,⊥)∗→M

(q, ϵ, γ).

By vacant stack: for q ∈ Q, (s, x ,⊥)∗→M

(q, ϵ, ϵ).


Ex.

PDA ({s, q}, {a, b}, {⊥,A}, s, δ,⊥, ∅) that accepts anbn by emptystack.

((s, a,⊥), (s,A)), ((s, a,A), (s,AA)),

((s, b,A), (q, ϵ)), ((q, b,A), (q, ϵ)).

PDA ({s, q}, {a, b, c}, {⊥,A,B}, s, δ,⊥, ∅) that accepts palindromeswcwR (w ∈ {a, b}∗) by empty stack.

((s, a,⊥), (s,A)), ((s, b,⊥), (s,B)), ((s, c ,⊥), (q, ϵ)),

((s, a,A), (s,AA)), ((s, a,B), (s,AB)), ((q, a,A), (q, ϵ)),

((s, b,A), (s,BA)), ((s, b,B), (s,BB)), ((q, b,B), (q, ϵ)),

((s, c ,A), (q,A)), ((s, c,B), (q,B)).


Equivalence of Acceptance

Let M = (Q,Σ, Γ, δ, s,⊥,F ) be NPDA that accepts by empty stack or byfinal state. Then, we can construct an equivalent NPDA

M ′ = (Q ∪ {u, t},Σ, Γ ∪ {⊤}, δ′, u,⊤, {t})

by empty stack at a single final state t.

We put another bottom ⊤ at u (new start)

to ascertain that even though the stack in M accidentally becomesempty it is not yet empty in M ′.

to see if the stack is empty and to define transitions from the emptystack. % Note that δ( , , ϵ) is not defined.


Final ⇒ Empty

Rename those former final states q ∈ F to non-final states, and transitfrom them to the new final state t to dump the stack contents Γ ∪ {⊤}.

δ′ = δ ∪ {((u, ϵ,⊤), (s,⊥⊤))}∪ {((q, ϵ,A), (t,A)) | q ∈ F ,A ∈ Γ ∪ {⊤}}∪ {((t, ϵ,A), (t, ϵ)) | A ∈ Γ ∪ {⊤}}.


Empty ⇒ Final

From any state q ∈ Q where the stack in M is empty when no morereadable letter (ϵ), transit to the new final state t to release ⊤.

δ′ = δ ∪ {((u, ϵ,⊤), (s,⊥⊤))}∪ {((q, ϵ,⊤), (t, ϵ)) | q ∈ Q}.


CFG⇒NPDA

Theorem For CFG, there exists NDPA with one state.

(Proof) Construct M = ({q},Σ,N, δ, q, S , ∅) where the set of stacksymbols is N, the set of non-terminal symbols of CFG. Then, for a rule inGNF

A → cB1 · · ·Bk , (c ∈ Σ ∪ {ϵ}, k ≥ 0),

let δ(⋆, c ,A) = (⋆,B1 · · ·Bk) where ⋆ is a unique anonymous state.


NPDA⇒CFG

Lemma For any NPDA, there is a corresponding NPDA with one state.

(Proof) For M = (Q,Σ, Γ, δ, s,⊥, {t}), let Γ′ = Q × Γ× Q and

M ′ = ({⋆},Σ, Γ′, δ′, ⋆, ⟨s⊥t⟩, ∅).

For each ((p, c ,A), (q0,B1B2 · · ·Bk)) ∈ δ, find all the possible transitionsq0q1 · · · qk until Bk is popped, and let

((⋆, c , ⟨pAqk⟩), (⋆, ⟨q0B1q1⟩⟨q1B2q2⟩ · · · ⟨qk−1Bkqk⟩)) ∈ δ′.

When k = 0, if ((p, c ,A), (q0, ϵ)) ∈ δ then ((⋆, c , ⟨pAq0⟩), (⋆, ϵ)) ∈ δ′.

% Note that only a pair of ⟨q0,B1⟩ is not enough since after B1 is poppedout we need to know the new state for B2.

Theorem There exists a CFG corresponding to NPDA with one state.(Proof) For ((⋆, c ,A), (⋆,B1 · · ·Bk)) ∈ δ, give CFG rule A → cB1 · · ·Bk .


Ex. For ((p, c ,A), (q0,B1B2)) ∈ δ, (even though p and q0 are fixed)there may be multiple possible states when A is released (the substringcorresponding to A is consumed), as below.

((q0, d ,B1), (q1, ϵ)), (q0, e,B1), (q3, ϵ)),

((q1, f ,B2), (q2, ϵ)), ((q1, g ,B2), (q4, ϵ)),

((q3, h,B2), (q4, ϵ))

((⋆, c, ⟨pAq2⟩), (⋆, ⟨q0B1q1⟩⟨q1B2q2⟩))((⋆, c, ⟨pAq4⟩), (⋆, ⟨q0B1q1⟩⟨q1B2q4⟩))((⋆, c, ⟨pAq4⟩), (⋆, ⟨q0B1q3⟩⟨q3B2q4⟩))

When the stack top is ⟨pAq4⟩ after reading c in the new automaton, thenew stack top is either ⟨q0B1q1⟩ or ⟨q0B1q3⟩.


Ex. NPDA accepting palindromes {wwR | w ∈ (a+b)∗} with empty stack

({s, q}, {a, b}, {⊥,A,B}, s, δ,⊥, ∅)(s: pushing state/ q: popping state)

((s, a,⊥), (s,A)), ((s, b,⊥), (s,B)),

((s, a,A), {(s,AA), (q, ϵ)}), ((s, a,B), (s,AB)),

((s, b,A), (s,BA)), ((s, b,B), {(s,BB), (q, ϵ)}),((q, a,A), (q, ϵ)), ((q, b,B), (q, ϵ)).

One state: ({s}, {a, b}, {⊥,A,B,A′,B ′}, s, δ,⊥, ∅)(A′ = ⟨sAq⟩,B ′ = ⟨sBq⟩: pushing/ A = ⟨qAq⟩,B = ⟨qBq⟩: popping)

((s, a,⊥), (s,A′)), ((s, b,⊥), (s,B ′)),

((s, a,A′), {(s,A′A), (s, ϵ)}), ((s, a,B ′), (s,A′B)),

((s, b,A′), (s,B ′A)), ((s, b,B ′), {(s,B ′B), (s, ϵ)}),((s, a,A), (s, ϵ)), ((s, b,B), (s, ϵ)).


Deterministic Pushdown Automata

The full-stop symbol (right end marker) needs to be added at the tailof input string. Otherwise, the input string may happen to be wronglyaccepted halfway by empty stack or by final state. In case of NPDA,every possibility should be exhaustively searched, but withdeterministic PDA we have no such luxury.

DCFL (Deterministic CFL) ⊊ CFL.


Closure Property

When A,B ⊂ Σ∗ are CFL,

A ∪ B is a CFL. Merge two production rules, and add a new startsymbol S → S1 | S2.AB is a CFL. Merge two production rules, and S → S1S2.

A∗ is a CFL. With a new start symbol S → S1S | ϵ.AR is a CFL. Reverse every production rule in CNF.

A ∩ B is not necessarily CFL. In general, there is no way to simulatetwo independent stacks by one stack.

Ac is not necessarily CFL. {ww | w ∈ Σ∗} is not CFL, but itscomplement is CFL.

% DCFL is closed under complement, but not closed under union.


Regular Language and Context-Free Language

Regular Grammar (RG) CFG, but each production rule is restricted to

A → xB or A → x (right-linear).A → aB or A → ϵ (strongly right-linear).A → Bx or A → x (left-linear).A → Ba or A → ϵ (strongly left-linear).

where x ∈ Σ∗, a ∈ Σ,A,B ∈ N in CFG (N,Σ,P,S).

Regular Language (RL) A set of sentences produced by a RG.

A RL is equivalent to a regular set. Since RG can be converted to bestrongly right-linear, let a non-terminal symbol be a state of the automaton

A → aB ⇐⇒ δ(A, a) = B (A → ϵ ⇐⇒ A ∈ F ).

% RL ⊂ CFL (cf. anbn), but sentences accepted by a CFL ⊂ that by a RL(cf. {anbn} ⊂ {a∗b∗}). For A ∈ CFL and R ∈ RL, A ∩ R ∈ CFL.


Chomsky Hierarchy

Type-0 Grammar Phrase Structure Grammar ⇐⇒ Turing machine

Type-1 Grammar Context-Sensitive Grammar ⇐⇒ Non-deterministicLinear Bounded AutomatonXαY → XβY where |α| ≤ |β|.

Type-2 Grammar Context-Free Grammar ⇐⇒ Non-deterministicPushdown Automaton{

A → aB1B2

A → a

Type-3 Grammar Regular Grammar ⇐⇒ Finite Automaton{A → aB (GNF with one production variable)A → a





4 Turing MachineDeterministic Turing MachineDecidabilityUniversal Turing Machine and Halting ProblemComputational ComplexityNondeterministic Turing MachineP and NP


Turing Machine (TM)

The head is movable both to the left- and right-hand side.

The head can rewrite an input character.

TM = (Q,Σ, Γ, δ, qs ,�,⊗) where

Q: a finite set of states.

Σ; input alphabet.

Γ: tape alphabet, Σ ⊂ Γ and [ (blank) ∈ Γ(/∈ Σ).

δ : Q × Γ → Q × Γ× {L,R}: a transition function.

qs ∈ Q: the initial state.� ∈ Q: the accept state.

⊗ ∈ Q: the reject state. (� = ⊗).

configuration (instantaneous description) e.g., 11q0101 where the headpoints the left-most 0 at state q in string 110101.

transition e.g., 11q0101 ⊢ 1q′11101 where δ(q, 0) = (q′, 1, L).


Semi-infinite Tape and Multiple Tapes

Theorem A TM can be simulated by a semi-infinite TM, where the headnever moves left of its initial position.

Theorem A multiple-tape machine can be simulated by a one-tapemachine.

Ex. pushdown automaton: The second tape simulates the pushdownstack. The left-end is the stack bottom, and only the right-end (the stacktop) is rewritable.


Decidability

A language L is decidable if there exists a Turing machine and it halts(either acceptance/rejection) on any input.

P is decidable ⇐⇒ {x | P(x)} is recursive.

A is recursive. ⇐⇒ ‘x ∈ A’ is decidable.

P is semi-decidable ⇐⇒ {x | P(x)} is recursively enumerable.

A is recursively enumerable ⇐⇒ ‘x ∈ A’ is semi-decidable.

Ex. To find a Hamilton circuit in a given graph is harder than to decidewhether there is one, since if we can find a circuit we can easily answerthat there is a circuit. (cf. NP-hard problems)


Universal Turing MachineUniversal Turing Machine is a program-loadable machine, that is, given adescription of a program (that is another TM) M with an input string w ,the machine either halts by accept/reject, or does not halt.

In the above figure, the upper tape is the description of M and the bottomtape is the input w to M.

Godel Numbering For a TM M,

⟨M⟩ = ⟨· · · , δ(p, a) = (q, b, L), δ(q, b) = (p, a, L), · · · ⟩= ⟨main(){int n,x,y; while{for(x=1;x<=n;x++)...;...;}n++;}⟩= 20F3B3215C7F66A890717D · · · 0940B5023A09346C1A8E

= 1001101010101001011 · · · 0101100110100010010011001011110100011010010


Halting Problem

1 Remember, in general, a TM given an input, either▶ halts at �,▶ halts at ⊗, or▶ does not halt (infinitely loops).

2 Let H0 be a halt tester, fed a program M and an input w ,▶ always halts (H0 itself never loops),▶ decides either � (M(w) halts) or ⊗ (M(w) loops),▶ without executing M(w).

3 Let H1 be H0 with the following revision;▶ replace � with an artificial infinite loop.▶ replace ⊗ with �.

4 Let H2 be a machine with one input M. Inside H2, it translates M toa numeric expression ⟨M⟩, and feeds (M, ⟨M⟩) to H1.

5 Now, what happens if H2 is fed to H2?


DiagonalizationThere does not exist a one-to-one correspondence between naturalnumbers N and its power set 2N (Cantor). For f : N → 2N, themembership (1/0) of f (n) ⊂ N is written in the following table.

0 1 2 3 4 5 6 7 · · ·f (0) = {0, 4, 5, · · · } 1 0 0 0 1 1 0 0 · · ·f (1) = {2, 3, 4, 7, · · · } 0 0 1 1 1 0 0 1 · · ·f (2) = {2, 3, 4, 6, · · · } 0 0 1 1 1 0 1 0 · · ·f (3) = {1, 2, 4, · · · } 0 1 1 0 1 0 0 0 · · ·f (4) = {0, 3, 4, 7, · · · } 1 0 0 1 1 0 0 1 · · ·f (5) = {0, 2, 6, 7, · · · } 1 0 1 0 0 0 1 1 · · ·f (6) = {2, 4, 5, · · · } 0 0 1 0 1 1 0 0 · · ·f (7) = {0, 3, 5, 7 · · · } 1 0 0 1 0 1 0 1 · · ·

......

......

......

......

...

But we can construct a new set, taking the Boolean complement for thediagonal elements. For example, 10101001 · · · meaning {0, 2, 4, 7, · · · } in theabove table becomes 01010110 · · · meaning {1, 3, 5, 6, · · · }. This new set mustbe in 2N so if f is one-to-one it must appear somewhere in the table, butnever (!) since the set is different from any (k , k)-element in the table.


Align all the Turing machines in the lexicographic order, and let ⟨Mi ⟩ bethe numeric expression of Mi . The following table shows whetherMi (⟨Mj⟩) halts (1) or not (0).

⟨M1⟩ ⟨M2⟩ ⟨M3⟩ ⟨M4⟩ ⟨M5⟩ ⟨M6⟩ ⟨M7⟩ · · ·M1 0 1 1 1 0 0 1 · · ·M2 0 1 1 1 0 1 0 · · ·M3 0 1 0 1 0 0 0 · · ·M4 1 0 1 1 0 1 0 · · ·M5 0 1 0 0 0 1 1 · · ·M6 0 0 1 1 1 0 0 · · ·M7 1 0 0 1 1 0 0 · · ·...

......

......

......

...

If there is a Turing machine halt, there exists h ∈ N somewhere andMh(⟨x⟩) ≡ halt(x , ⟨x⟩). As the program that returns the reverse value ofMh is another Turing machine, the program appears at some f ∈ N as Mf .What is the value for Mf (⟨Mf ⟩)?


Reduction

To prove a problem Q to be undecidable, we reduce a known undecidableproblem P to Q. If we assume Q were decidable, P would be decidable;contradiction.

Ex. if there were Q to decide membership in general, we could use Q as asubroutine to decide halting.

1 Suppose there were Qϵ to decide if a TM M accepts ϵ (empty string),i.e., Qϵ(M) always results in � or ⊗.

2 Let Tr be a translator, given a TM M and a string w , outputs such anew TM Mw , in which w is copied on its internal tape, that simulatesM with w given ϵ. This translation is a finite procedure.

3 Mw (ϵ) halts only when M(w) halts.

4 The connected machine TrQϵ, given a pair of M and w , outputs theresult of Qϵ(Mw ), which is either � or ⊗ dependent on if M(w) haltsor not; that is the halt tester.


Computational Complexity

Given the input size n, when the problem solving requires f (n) steps, itscomputational complexity is written as O(f (n)).

to decide membership of CFL is O(n3).

quick sort, heap sort: O(n log n)

bubble sort: O(n2)

reachability: if there is a walk from a vertex to another vertex in agiven graph; O(n2)

Eulerian Graph: if there is an Eulerian circuit in a given graph; O(n2)

Hamilton Graph: given a candidate (witness), the correctness isverified in O(n3), but creating all the candidates takes O(n!).


Nondeterministic Turing Machine (NTM)

The transition function of NTM returns multiple configurations, and thuscomposes a computation tree.

⃝: configuration

↙ ↘ : transition

⊗: reject�: accept

path: a sequence oftransitions from qs .

Computational time for NTM O(f (n)) is defined by the depth in thecomputation tree from the root to the shallowest accept state. Thatmeans an exhaustive parallel search, disregarding the reject states.


P and NP

For any NTM, there exists an equivalent deterministic TM (DTM).(Proof) Number all possible transitions from each configuration; thus a path

is in {1, · · · ,m}∗ (m = maxq∈Q,a∈Γ

|δ(q, a)|). Then, let DTM follow each path in

NTM, selecting the paths in the lexicographic order (the shorter path first).

An NTM can guess an exponential number of possible solutions andcheck each one in polynomial time in parallel, but the equivalentDTM can do so in polynomial time?

A class of decidability problem is▶ P; when solvable in polynomial time by DTM.▶ NP; when solvable in polynomial time by NTM.

To prove a problem cannot be solved in polynomial time, we reduceanother known non-polynomial problem to the problem in apolynomial translation time. But, can we reduce an NP problem inpolynomial time to a P problem?


Deterministic Verifier

Another definition of class NP: given a problem description together withits witness, possibly a candidate of solution, a DTM called verifier decidesaccept/reject in polynomial time.

Ex. Given a graph description on one tape and a candidate of Hamiltoncircuit on the other.

When the number of vertices is n, for each edge in the candidate, theverifier refers to the adjacent matrix on the first tape of the size O(n2);thus, the verifier halts in O(n3) and answers yes/no.


Equivalence of NP

V ⇒ N Suppose there is a verifier V , given an input |w | = n with awitness, that decides accept/reject within O(nk).

1 Let NTM N be such that outputs a random sequence d ,given w .

2 Each time N outputs |d | = nk , call V and test if Vaccepts ⟨w , d⟩.

Define N to accept w only when V accepts ⟨w , d⟩.N ⇒ V Suppose there is an NTM N which accepts sentence w in

O(nk).

1 Let each transition in N be numbered.2 Let d be a sequence of transitions, i.e., a path in the

computation tree of N.

When N accepts w after following d , define V to accept⟨w , d⟩; otherwise to reject it.


NP-hard and NP-completeA problme is

NP-hard The problem may not be a decision problem, but has thecomplexity at least as much as NP.Ex. Traveling salesman problem, Finding a Hamilton circuit.

NP-complete If the problem is reduced to P, then other NP problems canbe reduced also to P.Ex. SAT, Existence of Hamilton circuit

Other important classes:

co-NP The class of the deciding problems for the negation of thesentences, i.e., the counter-membership. P ⊆ NP andP ⊆ co-NP, but no relation between NP and co-NP.

PSPACE The class that the capacity of tape is restricted to nk .

EXPTIME The class of O(kn).

P ⊆ NP ⊆ PSPACE ⊆ EXPTIME.


Date post:	17-Mar-2020
Category:	Documents
Upload:	others
View:	10 times
Download:	0 times

· Relation and Partial/Total Order Relation re exive: (x;x) 2 R. symmetric: (x;y) 2 R ! (y;x) 2...

Documents