1
Chapter 1Chapter 1
Introduction to the Theory of Introduction to the Theory of ComputationComputation
2
Theory of ComputationTheory of Computation
Can we mathematically model the computation?Can we mathematically model the computation? Automata Theory
Regular languages: DFA, NFA, Regular expressions Context-free languages: Context-free grammar,
pushdown automata Turing machines: the most powerful model of
computation Can the computer solve all problems?Can the computer solve all problems?
Computability Theory Halting problem
How efficient the solution is?How efficient the solution is? Complexity Theory
Time and space complexity (Big O notation)
3
1.11.1 Mathematical Preliminaries Mathematical Preliminaries
Sets (1)Sets (1) A A setset is a collection of objects is a collection of objects
Order and repetition is irrelevant X = {1, 2, 3} X = {n | 1 n 3, n N } Y = { } = Y is an empty set
The The power setpower set of X is the set of all subsets of X of X is the set of all subsets of X (X) = 2X = { Y | Y X} |(X)| = 2|X|
Example X = {1, 2, 3}Example X = {1, 2, 3} 2X = (X)={{},{1},{2},{3},{1,2},{1,3},{2,3}, {1,2,3}} {1,2} X is a proper subset of X {1,2,3} X is NOT a proper subset of X {1,2,3} is a superset of {1, 2}
4
Sets (2)Sets (2)
The The symbol signifies membership (belong to): symbol signifies membership (belong to): x X indicates that x is a member or element
of the set X x X indicates that x is not a member or
element of the set X Assume X = {1, 2, 3}, then 1 X, but 4 X
Two sets are equal if they contain the same Two sets are equal if they contain the same members (elements)members (elements) If X = {1, 2, 3}, Y = {1, 2, 3}, and Z = {2, 3, 4} Then X = Y, but X Z, and so Y Z
5
Sets (3)Sets (3)
Assume X = {1,2,3} and Y = {2,4}Assume X = {1,2,3} and Y = {2,4}
UnionUnion X Y = {z | z X or z Y} = {1,2,3,4}
IntersectionIntersection X Y = {z | z X and z Y} = {2}
DifferenceDifference X - Y = {z | z X and z Y} = {1,3}
6
Sets (4)Sets (4)
Complement Complement Let X be a subset of Y The complement of X ( X ) with respect to Y is
the set of elements in Y but not in X X = { z | z Y and z X } = Y - X
Assume X = {1,2,3} is a subset of Y = {1,2,3,4}Assume X = {1,2,3} is a subset of Y = {1,2,3,4} X = {4}
DeMorgan’s LawsDeMorgan’s Laws
(X Y) = X Y
(X Y) = X Y
7
Inductive Proofs
Prove a statement S(X) about a family of objects X (e.g., integers, trees) in three parts: Basis Step:
Prove for one or several small values of X directly Assumption step :
Assume S(Y) for Y smaller than X Prove S(X) using that assumption
8
Inductive Proofs (EX1)
Prove by induction thatProve by induction that S: 1 + 2 + 3 + ……+ n = n (n + 1) / 2
Basis step:Basis step: n = 1
1 ( 1 + 1) / 2 = 1 Assumption step:Assumption step:
Assume that S is true for all series of length < n Prove that S is true for a series of length n?Prove that S is true for a series of length n?
The series of length n is 1+2+3+…+n-1+n By the assumption 1+2+…+n-1 = (n-1)((n-1)+1)/2 Add n to the summation proves S.
9
Inductive Proofs (EX2)
Example A complete binary tree with n leaves has 2n - 1
nodes
10
Inductive Proofs (EX2)
Basis: If T has 1 leaf, it is a one-node tree. 1 = 2*1 - 1, so OK
Assumption:Assumption: Assume that the statement is true for all trees that have leaves
less than n Proof: Proof: Proof that the statement is true for trees that have n leavesProof that the statement is true for trees that have n leaves
T has two subtrees, U and V U has u leaves and V has v leaves According to the assumption: U has (2u – 1) of nodes and V
has (2v – 1) of nodes T must be a root plus two subtrees U and V. The number of
leaves in T equal number of leaves in U + number of leaves in V = u + v
The number of nodes in T = (2u – 1) + (2v – 1) + 1 = 2 ( u + v) - 1
11
If-And-Only-If Proofs
Often, a statement we need to prove is of the form “X if and only if Y”
We are then required to do two things: Prove the if-part: Assume Y and prove X Prove the only-if-part: Assume X, prove Y
Remember: The if and only-if parts are converses of each
other
One part, say “if X then Y” says nothing about whether Y is true when X is false.
An equivalent form to “if X then Y” is “if not Y then not X”; the latter is the contra-positive of the former.
12
Equivalence of Sets
Many important facts in language theory are of the form that two sets of strings, described in two different ways, are really the same set
To prove sets S and T are the same, prove: x is in S if and only if x is in T. That is:
Assume x is in S; prove x is in T . Assume x is in T ; prove x is in S.
13
FunctionsFunctions
Let Let ff be a function defined on a set be a function defined on a set AA and taking values to and taking values to a set a set BB. Set . Set AA is called the is called the domaindomain of of ff; set ; set BB is called the is called the rangerange of of ff..
The The arityarity of the functionsof the functions The number and type of input arguments the number
and type of output Examples:
The arity of the Multiplication: Num × Num Num The arity of the SquareRoot : Num Num
14
1.2:1.2: Strings and Languages (1)Strings and Languages (1)
AlphabetAlphabet: Finite, nonempty set of symbols: Finite, nonempty set of symbols Examples:
= {0, 1}: binary alphabet = {a, b, c, …, z}: the set of all lower case letters The set of all ASCII characters
StringString: Finite sequence of symbols from an : Finite sequence of symbols from an alphabet alphabet Examples:
01101 where = {0, 1} abracadabra where = {a, b, c, …, z}
15
1.2:1.2: Strings and Languages (2)Strings and Languages (2)
Empty StringEmpty String: The string with : The string with zerozero occurrences of occurrences of symbols from symbols from and is denoted and is denoted or or
Length of StringLength of String: Number of symbols in the string: Number of symbols in the string The length of a string w is usually written |w| |1010| = 4 || = 0 |uv| = |u| + |v|
ReverseReverse : : wwRR
If w = abc, wR = cba
16
1.2:1.2: Strings and Languages (3)Strings and Languages (3)
ConcatenationConcatenation: if : if xx and and yy are strings, then are strings, then xyxy is the is the string obtained by placing a copy of string obtained by placing a copy of yy immediately immediately after a copy of after a copy of xx x = a1a2 …ai, y = b1b2 …bj xy = a1a2 …aib1b2 …bj Example: x = 01101, y = 110, xy = 01101110 x = x = x
17
1.2:1.2: Strings and Languages (4)Strings and Languages (4)
Power of an AlphabetPower of an Alphabet: : kk = the set of strings of length k = the set of strings of length k with symbols from with symbols from Example: = {0, 1} 1 = = {0, 1} 2 = {00, 01, 10, 11} 0 =
QuestionQuestion: How many strings are there in : How many strings are there in 33??
The set of all strings over The set of all strings over is denoted is denoted ** * = 0 1 2 3 …
Also Also + = 1 2 3 … * = + + = * -
18
1.2:1.2: Strings and Languages (5)Strings and Languages (5)
SubstringSubstring: any string of consecutive characters in : any string of consecutive characters in some string some string ww If w = abc , a, ab, abc are substrings of w
PrefixPrefix and and suffixsuffix:: if w = vu v is a prefix of w u is a suffix of w Example
If w = abc a, ab , abc are prefixes of w c, bc, abc are suffixes of w
19
1.2:1.2: Strings and Languages (6)Strings and Languages (6)
Suppose: S is the string banana Prefix : ban, banana Suffix : ana, banana Substring : nan, ban, ana, banana
20
1.2:1.2: Strings and Languages (8)Strings and Languages (8)
LanguageLanguage: set of strings chosen from some alphabet: set of strings chosen from some alphabet A language is a subset of A language is a subset of **
Example of languages:The set of valid Arabic wordsThe set of strings consisting of n 0’s followed by n 1’s
– {, 01, 0011, 000111, …} The set of strings with equal number of 0’s and 1’s
– {, 01, 10, 0011, 0101, 1010, 1001, 1100, …} Empty languageEmpty language: : = { } = { } The language The language consisting of the empty string consisting of the empty string Note: Note:
21
1.2:1.2: Strings and Languages (9)Strings and Languages (9)
Can concatenate languagesCan concatenate languages L1L2 = {xy | x L1, y L2} Ln = L concatenated with itself n times L0 = {}; L1 = L
Star-closureStar-closure L* = L0 L1 L2 L+ = L* - L0
Languages can be finite or infiniteLanguages can be finite or infinite L = {a, aba, bba} L = {an | n > 0}
22
1.2:1.2: Strings and Languages (10)Strings and Languages (10)
OPERATION DEFINITIONunion of L and M written L Mconcatenation of L and M written LMKleene closure of L written L*
positive closure of L written L+
L M = {s | s is in L or s is in M}
LM = {st | s is in L and t is in M}
L+ = Li, i=1,..,
L* denotes “zero or more concatenations of “ L
L* = Li , i=0,..,
L+ denotes “one or more concatenations of “ L
L+ = LL*
23
1.2:1.2: Strings and Languages (11)Strings and Languages (11)
L = {A, B, …, Z, a, b, …z} D = {1, 2, …, 9}L D = the set of letters and digits
LD = all strings consisting of a letter followed by a digit
L2 = the set of all two-letter strings
L4 = L2 L2 = the set of all four-letter strings
L* = { All possible strings of L plus }, L+ = L* -
D+ = set of strings of one or more digits
L (L D ) = set of all strings consisting of a letter followed by a a letter or a digit
L (L D )* = set of all strings consisting of letters and digits beginning with a letter
24
1.2:1.2: Strings and Languages (12)Strings and Languages (12)
The language The language LL consists of strings over consists of strings over {a,b}{a,b} in in which which eacheach string begins with an string begins with an aa should have an should have an eveneven length length
aa, ab L aaaa,aaab,aaba,aabb,abaa,abab,abba,abbb L
baa L a L
25
1.2:1.2: Strings and Languages (13)Strings and Languages (13)
The language The language LL consists of strings over consists of strings over {a,b}{a,b} in in which each occurring of which each occurring of bb is is immediately immediately precededpreceded by an by an aa
L a L abaab L
bb L bab L abb L
26
1.2:1.2: Strings and Languages (14)Strings and Languages (14)
Let X = {a,b,c} and Y = {abb, ba}. ThenLet X = {a,b,c} and Y = {abb, ba}. Then XY = {aabb, babb, cabb, aba, bba, cba} X0 = {} X1 = X = {a,b,c} X2 = XX = {aa,ab,ac,ba,bb,bc,ca,cb,cc} X3 = XXX =
{aaa,aab,aac,aba,abb,abc,aca,acb,acc,baa,bab,bac,bba,bbb,bbc,bca,bcb,bcc,caa,cab,cac,cba,cbb,cbc,cca,ccb,ccc}
27
1.2:1.2: Strings and Languages (15)Strings and Languages (15)
The language L = {a,b}The language L = {a,b}**{bb}{a,b}{bb}{a,b}** = = **{bb}{bb}**
consists of the strings over {a,b} that contain the substring bb
bb L abb L bbb L aabb L bbaaa L bbabba L abab L bab L b L
28
1.2:1.2: Strings and Languages (16)Strings and Languages (16)
Let Let LL be the language that consists of all strings that be the language that consists of all strings that beginbegin with with aaaa or or endend with with bbbb L1 = {aa}{a,b}*
L2 = {a,b}*{bb} L = L1 L2 = {aa}{a,b}* {a,b}*{bb} bb L abb L bbb L aabb L bbaaa L bbabba L abab L bab L ba L
29
1.2:1.2: Strings and Languages (16)Strings and Languages (16)
Let Let LL11 = {bb} and = {bb} and LL22 = { = {, bb, bbbb} over , bb, bbbb} over bb
The languages LThe languages L11**
and L and L22**
both contain precisely both contain precisely the strings consisting of an the strings consisting of an eveneven number of number of bb’s.’s.
, with length zero, is an element of L, with length zero, is an element of L11**
and L and L22**
30
1.2:1.2: Strings and Languages (17)Strings and Languages (17)
What is the language of all What is the language of all even-lengtheven-length strings over strings over {a,b}{a,b} L = {aa, bb, ab, ba}* = (aa|bb|ab|ba)*
What is the language of all What is the language of all odd-lengthodd-length strings over strings over {a,b}{a,b} L = {a,b}* - {aa, bb, ab, ba}* or L = {a,b}{aa, bb, ab, ba}* or L = {aa, bb, ab, ba}* {a,b}