Theory of Computation

Mathematical Preliminaries

Vladimir KulyukinDepartment of Computer Science

Utah State University

www.vkedco.blogspot.com

Outline

● Useful Texts● What is Theory of Computation?● Alphabets, Strings, Languages● Numbers, Sets, Set Formers● Proofs

Useful Texts

● M. Davis, R. Sigal, E. Weyuker. Computability, Complexity, and Languages

● A. Brooks Weber.Formal Language: A Practical Introduction● M. Genesereth, N. Nilsson. Logical Foundations of AI

What is Theory of Computation?

Basic Methodology● Abstraction of hardware details: they are important

for implementation but not theoretical investigations

● Focus on what can be solved, not on how it can be solved: if we can solve something, someone will find an algorithm

● Problem analysis in terms of devices (aka automata) and inputs (aka strings, languages)

Basic Methodology

● Abstraction of hardware details● Focus on what can be solved, not on how it can

be solved● Problem analysis in terms of devices (aka

automata) and inputs (aka strings, languages)

Abstraction of Hardware Details

Junun Mark III Robot

SmartphonePersonal Computer

While these devices have very different hardware, they have the same computational model

Raspberry PI

Focus on What, Not How

● Problem: sorting a sequence of numbers from smallest to highest

● Algorithmic answer: merge sort, heap sort, quick sort, etc

● Computability answer: sorting is a primitive recursive function, hence, it is computable

Automata & Languages● Finite State Automata/Regular Expressions – Regular

Languages (Automata w/o memory)● Push Down Automata/Stack Machines – Context-Free

Languages (Automata with memory)● Linear Bounded Automata – Context Sensitive

Languages (Aumata with memory & restricted input)● Turing Machines/Universal Programs – Recursively

Enumerable Languages (Automata with memory & unrestricted input)

Automata & Languages

Alphabets, Languages, Strings

Alphabets

● An alphabet is a finite set of symbols● The Greek letter Σ is typically used to denote an

alphabet● Examples: Σ1 = {a, b}, Σ2 = {0, 1, 2, 3, 4, 5, 6,

7, 8, 9}.● The symbols in the alphabet do not have any

meaning in and of themselves: their meanings are assigned by someone else

Strings● A string is a finite sequence of symbols● ε is the empty string; ε is not a symbol in any

alphabet; it denotes the string with zero symbols● In the book “Computability, Complexity, &

Languages” by Davis et al., the empty string is denoted as 0

● In formal language theory, strings are typically written without quotation marks: aab and 010001 instead of “aab” and “010001”

Strings● The length of a string is the number of

symbols/characters in it● The length of a string is denoted with a pair of

matching vertical lines around it● Examples:

if x = aab, then |x| = 3; if x = ε, then |x| = 0

String Concatenation

● The concatenation of two strings x and y is the strings containing the symbols of x followed by the symbols of y

● Examples: if x = ab and y = 100, then xy = ab100; if x = ε and y = abc, then xy = abc

● For any string, xε = εx = x

Power Notation in String Concatenation● When a natural number n is used as an exponent

on a string, it denotes the concatenation of that string with itself n times

● Examples: x0 = ε x1 = x x2 = xx (ab)3 = ababab

Languages

● A language is a set strings over a alphabet● Note that we can define multiple languages over

the same alphabet● Example: Σ = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}. L1 =

the set of all strings over Σ that end in 0 (e.g., 0, 110, 213450, etc.); L2 = the set of all strings over Σ that end in 1 (e.g., 1, 01, 0001, 91, etc.)

Kleene Closures of Alphabets

● The Kleene closure of an alphabet is the set of all strings over it

● If Σ is an alphabet, its Kleene closure is written as Σ*

● Example: Σ = {a, b}. Σ* = the set of all strings consisting of a's and b's, including the empty string

Numbers, Sets, Set Formers

Natural Numbers

● N is a set of natural numbers.● N includes 0, 1, 2, 3, …● Some texts exclude 0 from the set of natural

numbers, but we will keep it in● In the texts, the word number refers to natural

number

Sets

sets.empty are or

Set Former Notation

.or by followed is or

wherestrings ofset theis ,,,|

s.' ofnumber the

toequal is s' ofnumber theand s' precede s'that

such ,over stringsempty -non ofset theis 1|

3.or 2, 1, 0, islength whose

,over strings all ofset theis 3|, *

ccaaba

ccaaybaxxy

b

aba

banba

baxbax

nn

Subsets

.

:setevery ofsubset a isset empty The

. and if)only and (if iff

R

RSSRSR

Proper Subsets

. and iff SRSRSR

Set-Theoretic Equalities

s.complement theofunion theis

onintersecti theof complement thei.e. ,

s.complement theofon intersecti the

isunion theof complement thei.e. ,

).(

. and ofon intersecti -

SRSR

SRSR

SRRSR

SRSR

Sets and N-Tuples

.,,,,,,

:matter does sequences ain elements oforder The

}.,,{},,{},,{

:matternot doesset ain elements oforder The

set. a is },...,,{ 21

bacacbcba

cabbcacba

aaa n

Sets and N-tuples

}.,...,,|),...,,{(

...

:follows as defined isset theseofproduct

Cartesian Then the sets. are ,...,,Let

221121

21

21

nnn

n

n

SaSaSaaaa

SSS

SSS

Predicates

.0)(or 1)(or )(or )(either

each for such that on function

valued-Boolean totala is predicate A

aPaPFaPTaP

SaS

P

Predicates

R. offunction sticcharacteri a is )(

}1)(|{

Then

if 0

if 1)(

set. a be Let

xP

xPxR

Rx

RxxP

R

Proofs

Proof Methods

● In CS, there are, broadly speaking, two methods of proving things: formal and empirical

● Formal methods are used in theory of computation, algorithms, operations research, etc.

● Empirical methods are used in many applied branches of CS; most of us are painfully familiar with buggy software that must be tested out

● Many R&D projects combine formal and empirical methods

Mathematical Proofs

● The corner stone of the formal method is the mathematical proof

● Many online and printed CS materials contain proofs● It is of vital importance for a CS practitioner to read at

least some proofs● The good news for practically inclined CS majors is that

reading proofs is significantly easier than doing them

Proof Techniques● Proof techniques are independent of their subject matter: valid proofs

in calculus use the same proof techniques as valid proofs in algorithms or theory of computation

● Common proof techniques can be identified● Learning to identify common proof techniques will enable you to study

many areas of CS independently● The ability to identify proof techniques is based on your ability to

understand how the technique works and when it is likely to be applicable

● In CS, a prominent proof technique is induction

Learning to Love the P-Word● General advice: Do not be afraid of proofs; one can be a

mediocre theorem prover but a very good proof reader● The first step in mastering the art of mathematical proof

is to read and do proofs of known facts; do not think of it as a waste of time

● When you read some CS material, do not shy away from it, if it contains proofs