Computation Engineeringcs6110/GaneshBookExcerpts.pdf · · 2011-01-14Ganesh Lalitha...

Ganesh Lalitha Gopalakrishnan

Computation Engineering

applied automata theory and logic

SpringerBerlin Heidelberg NewYorkHongKong LondonMilan Paris Tokyo

Whatever you do will be insignificant,but it is very important that you do it.

– Mohandas K. Gandhi

Contents

Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .XXV

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .XXVII

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Computation Science and Computation Engineering . . . . . . . . 1What is ‘Computation?’ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2A Minimalist Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3How to Measure the Power of Computers? . . . . . . . . . . . . . . . . . 5Complexity Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Automata Theory and Computing . . . . . . . . . . . . . . . . . . . . . . . . 6Why “Mix-up” Automata and Mathematical Logic? . . . . . . . . 7Why Verify? Aren’t Computers “Mostly Okay?”. . . . . . . . . . . . 7Verifying Computing Systems Using Automaton Models . . . . . 9Automaton/Logic Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Avoid Attempting the Impossible . . . . . . . . . . . . . . . . . . . . . . . . . 11Solving One Implies Solving All . . . . . . . . . . . . . . . . . . . . . . . . . . 11Automata Theory Demands a Lot From You! . . . . . . . . . . . . . . 12A Brief Foray Into History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Disappearing Formal Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2 Mathematical Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . 152.1 Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.2 Boolean Concepts, Propositions, and Quantifiers . . . . . . . 152.3 Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.3.1 Defining sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.3.2 Avoid contradictions . . . . . . . . . . . . . . . . . . . . . . . . . . 17

VIII Contents

2.3.3 Ensuring uniqueness of definitions . . . . . . . . . . . . . . 182.4 Cartesian Product and Powerset . . . . . . . . . . . . . . . . . . . . . 21

2.4.1 Powersets and characteristic sequences . . . . . . . . . . 222.5 Functions and Signature . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.6 The ! Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.7 Total, Partial, 1-1, and Onto Functions . . . . . . . . . . . . . . . 252.8 Computable Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.9 Algorithm versus Procedure . . . . . . . . . . . . . . . . . . . . . . . . . 282.10 Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.11 Functions as Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.11.1 More ! syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3 Cardinalities and Diagonalization . . . . . . . . . . . . . . . . . . . . 373.1 Cardinality Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.1.1 Countable sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.1.2 Cardinal numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.1.3 Cardinality “trap” . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.2 The Diagonalization Method . . . . . . . . . . . . . . . . . . . . . . . . . 393.2.1 Simplify the set in question . . . . . . . . . . . . . . . . . . . . 403.2.2 Avoid dual representations for numbers . . . . . . . . . . 403.2.3 Claiming a bijection, and refuting it . . . . . . . . . . . . 413.2.4 ‘Fixing’ the proof a little bit . . . . . . . . . . . . . . . . . . . 413.2.5 Cardinality of 2Nat and Nat ! Bool . . . . . . . . . . . . 43

3.3 The Schroder-Bernstein Theorem . . . . . . . . . . . . . . . . . . . . . 433.3.1 Application: cardinality of all C Programs . . . . . . . 443.3.2 Application: functions in Nat ! Bool . . . . . . . . . . . 443.3.3 Proof of the Schroder-Bernstein Theorem . . . . . . . . 46

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

4 Binary Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534.1 Binary Relation Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

4.1.1 Types of binary relations . . . . . . . . . . . . . . . . . . . . . . 544.1.2 Preorder (reflexive plus transitive) . . . . . . . . . . . . . . 574.1.3 Partial order (preorder plus antisymmetric) . . . . . . 574.1.4 Total order, and related notions . . . . . . . . . . . . . . . . 58

4.2 Equivalence (Preorder plus Symmetry) . . . . . . . . . . . . . . . . 584.2.1 Intersecting a preorder and its inverse . . . . . . . . . . . 594.2.2 Identity relation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594.2.3 Universal relation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Contents IX

4.2.4 Equivalence class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604.2.5 Reflexive and transitive closure . . . . . . . . . . . . . . . . . 60

4.3 The Power Relation between Machines . . . . . . . . . . . . . . . . 614.3.1 The equivalence relation over machine types . . . . . 62

4.4 Lattice of All Binary Relations over S . . . . . . . . . . . . . . . . 644.5 Equality, Equivalence, and Congruence . . . . . . . . . . . . . . . . 64

4.5.1 Congruence relation . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

5 Mathematical Logic, Induction, Proofs . . . . . . . . . . . . . . . 735.1 To Prove or Not to Prove! . . . . . . . . . . . . . . . . . . . . . . . . . . . 735.2 Proof Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

5.2.1 The implication operation . . . . . . . . . . . . . . . . . . . . . 755.2.2 ‘If,’ or ‘Definitional Equality’ . . . . . . . . . . . . . . . . . . . 765.2.3 Proof by contradiction . . . . . . . . . . . . . . . . . . . . . . . . 775.2.4 Quantification operators " and # . . . . . . . . . . . . . . . 785.2.5 Generalized DeMorgan’s Law Relating " And # . . 805.2.6 Inductive definitions of sets and functions . . . . . . . . 80

5.3 Induction Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 825.3.1 Induction over natural numbers . . . . . . . . . . . . . . . . 825.3.2 Noetherian induction . . . . . . . . . . . . . . . . . . . . . . . . . . 845.3.3 Structural . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

5.4 Putting it All Together: the Pigeon-hole Principle . . . . . . 85

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

6 Dealing with Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 936.1 Recursive Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

6.1.1 Recursion viewed as solving for a function . . . . . . . 946.1.2 Fixed-point equations . . . . . . . . . . . . . . . . . . . . . . . . . 956.1.3 The Y operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 966.1.4 Illustration of reduction . . . . . . . . . . . . . . . . . . . . . . . 97

6.2 Recursive Definitions as Solutions of Equations . . . . . . . . 976.2.1 The least fixed-point . . . . . . . . . . . . . . . . . . . . . . . . . . 100

6.3 Fixed-points in Automata Theory . . . . . . . . . . . . . . . . . . . . 101

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

X Contents

7 Strings and Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1057.1 Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

7.1.1 The empty string " . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1067.1.2 Length, character at index, and substring of a

string . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1077.1.3 Concatenation of strings . . . . . . . . . . . . . . . . . . . . . . . 107

7.2 Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1077.2.1 How many languages are there? . . . . . . . . . . . . . . . . 1087.2.2 Orders for Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1087.2.3 Operations on languages . . . . . . . . . . . . . . . . . . . . . . . 1107.2.4 Concatenation and exponentiation . . . . . . . . . . . . . . 1107.2.5 Kleene Star, ‘$’ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1127.2.6 Complementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1127.2.7 Reversal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1137.2.8 Homomorphism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1137.2.9 Ensuring homomorphisms . . . . . . . . . . . . . . . . . . . . . 1137.2.10 Inverse homomorphism . . . . . . . . . . . . . . . . . . . . . . . . 1147.2.11 An Illustration of homomorphisms . . . . . . . . . . . . . . 1157.2.12 Prefix-closure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

8 Machines, Languages, DFA . . . . . . . . . . . . . . . . . . . . . . . . . . 1198.1 Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

8.1.1 The DFA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1218.1.2 The “power” of DFAs . . . . . . . . . . . . . . . . . . . . . . . . . 1268.1.3 Limitations of DFAs . . . . . . . . . . . . . . . . . . . . . . . . . . 1268.1.4 Machine types that accept non-regular languages . 1278.1.5 Drawing DFAs neatly . . . . . . . . . . . . . . . . . . . . . . . . . 129

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

9 NFA and Regular Expressions . . . . . . . . . . . . . . . . . . . . . . . 1339.1 What is Nondeterminism? . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

9.1.1 How nondeterminism a!ects automaton operations 1369.1.2 How nondeterminism a!ects the power of machines137

9.2 Regular Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1379.3 Nondeterministic Finite Automata . . . . . . . . . . . . . . . . . . . 141

9.3.1 Nondeterministic Behavior Without " . . . . . . . . . . . 1419.3.2 Nondeterministic behavior with " . . . . . . . . . . . . . . . 1439.3.3 Eclosure (also known as "-closure) . . . . . . . . . . . . . 1459.3.4 Language of an NFA . . . . . . . . . . . . . . . . . . . . . . . . . . 147

Contents XI

9.3.5 A detailed example: telephone numbers . . . . . . . . . . 1499.3.6 Tool-assisted study of NFAs, DFAs, and REs . . . . . 151

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

10 Operations on Regular Machinery . . . . . . . . . . . . . . . . . . . 15910.1 NFA to DFA Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15910.2 Operations on Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

10.2.1 Union . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16210.2.2 Intersection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16510.2.3 Complementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16510.2.4 Concatenation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16510.2.5 Star . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16610.2.6 Reversal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16710.2.7 Homomorphism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16810.2.8 Inverse Homomorphism. . . . . . . . . . . . . . . . . . . . . . . . 16810.2.9 Prefix-closure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

10.3 More Conversions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16910.3.1 RE to NFA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16910.3.2 NFA to RE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17010.3.3 Minimizing DFA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

10.4 Error-correcting DFAs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17610.4.1 DFA constructed using error strata . . . . . . . . . . . . . 17710.4.2 DFA constructed through regular expressions . . . . 177

10.5 Ultimate Periodicity and DFAs . . . . . . . . . . . . . . . . . . . . . . 179

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

11 The Automaton/Logic Connection, SymbolicTechniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18511.1 The Automaton/Logic Connection . . . . . . . . . . . . . . . . . . . 186

11.1.1 DFA can ‘scan’ and also ‘do logic’ . . . . . . . . . . . . . . 18611.2 Binary Decision Diagrams (BDDs) . . . . . . . . . . . . . . . . . . . 18711.3 Basic Operations on BDDs . . . . . . . . . . . . . . . . . . . . . . . . . . 191

11.3.1 Representing state transition systems . . . . . . . . . . . 19211.3.2 Forward reachability . . . . . . . . . . . . . . . . . . . . . . . . . . 19311.3.3 Fixed-point iteration to compute the least

fixed-point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19411.3.4 An example with multiple fixed-points . . . . . . . . . . 19711.3.5 Playing tic-tac-toe using BDDs . . . . . . . . . . . . . . . . . 198

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

XII Contents

12 The ‘Pumping’ Lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20512.1 Pumping Lemmas for Regular Languages . . . . . . . . . . . . . . 205

12.1.1 A stronger incomplete Pumping Lemma . . . . . . . . . 20912.2 An adversarial argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21012.3 Closure Properties Ameliorate Pumping . . . . . . . . . . . . . . . 21112.4 Complete Pumping Lemmas . . . . . . . . . . . . . . . . . . . . . . . . . 212

12.4.1 Ja!e’s complete Pumping Lemma . . . . . . . . . . . . . . . 21212.4.2 Stanat and Weiss’ complete Pumping Lemma . . . . 213

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

13 Context-free Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21713.1 The Language of a CFG . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21813.2 Consistency, Completeness, and Redundancy . . . . . . . . . . 220

13.2.1 More consistency proofs . . . . . . . . . . . . . . . . . . . . . . . 22313.2.2 Fixed-points again! . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

13.3 Ambiguous Grammars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22513.3.1 If-then-else ambiguity . . . . . . . . . . . . . . . . . . . . . . . . . 22613.3.2 Ambiguity, inherent ambiguity . . . . . . . . . . . . . . . . . 227

13.4 A Taxonomy of Formal Languages and Machines . . . . . . . 22813.4.1 Non-closure of CFLs under complementation . . . . . 23013.4.2 Simplifying CFGs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

13.5 Push-down Automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23413.5.1 DPDA versus NPDA . . . . . . . . . . . . . . . . . . . . . . . . . . 23513.5.2 Deterministic context-free languages (DCFL) . . . . . 23513.5.3 Some Factoids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

13.6 Right- and Left-Linear CFGs . . . . . . . . . . . . . . . . . . . . . . . . 23713.7 Developing CFGs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23913.8 A Pumping Lemma for CFLs . . . . . . . . . . . . . . . . . . . . . . . . 239

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

14 Push-down Automata and Context-free Grammars . . . 24514.1 Push-down Automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

14.1.1 Conventions for describing PDAs . . . . . . . . . . . . . . . 24714.1.2 Acceptance by final state . . . . . . . . . . . . . . . . . . . . . . 24714.1.3 Acceptance by empty stack . . . . . . . . . . . . . . . . . . . . 24914.1.4 Conversion of P1 to P2 ensuring L(P1) = N(P2) . . 25114.1.5 Conversion of P1 to P2 ensuring N(P1) = L(P2) . . 251

14.2 Proving PDAs Correct Using Floyd’s Inductive Assertions25314.3 Direct Conversion of CFGs to PDAs . . . . . . . . . . . . . . . . . . 25414.4 Direct Conversion of PDAs to CFGs . . . . . . . . . . . . . . . . . . 257

Contents XIII

14.4.1 Name non-terminals to match stack-emptyingpossibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

14.4.2 Let start symbol S set up all stack-draining options25814.4.3 Capture how each PDA transition helps drain

the stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25914.4.4 Final result from Figure 14.6 . . . . . . . . . . . . . . . . . . . 260

14.5 The Chomsky Normal Form . . . . . . . . . . . . . . . . . . . . . . . . . 26214.5.1 Cocke-Kasami-Younger (CKY) parsing algorithm . 262

14.6 Closure and Decidability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26414.7 Some Important Points Visited . . . . . . . . . . . . . . . . . . . . . . 264

14.7.1 Chapter Summary – Lost Venus Probe . . . . . . . . . . 267

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268

15 Turing Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27115.1 Computation: Church/Turing Thesis . . . . . . . . . . . . . . . . . . 272

15.1.1 “Turing machines” according to Turing . . . . . . . . . . 27315.2 Formal Definition of a Turing machine . . . . . . . . . . . . . . . . 273

15.2.1 Singly- or doubly-infinite tape? . . . . . . . . . . . . . . . . . 27515.2.2 Two stacks+control = Turing machine . . . . . . . . . . 27615.2.3 Linear bounded automata . . . . . . . . . . . . . . . . . . . . . 27715.2.4 Tape vs. random access memory . . . . . . . . . . . . . . . . 278

15.3 Acceptance, Halting, Rejection . . . . . . . . . . . . . . . . . . . . . . . 27815.3.1 “Acceptance” of a TM closely examined . . . . . . . . . 27815.3.2 Instantaneous descriptions . . . . . . . . . . . . . . . . . . . . . 279

15.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27915.4.1 Examples illustrating TM concepts and conventions28015.4.2 A DTM for w#w . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

15.5 NDTMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28215.5.1 Guess and check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28315.5.2 An NDTM for ww . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

15.6 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28615.6.1 Multi-tape vs. single-tape Turing machines . . . . . . . 28615.6.2 Nondeterministic Turing machines . . . . . . . . . . . . . . 28615.6.3 The Simulation itself . . . . . . . . . . . . . . . . . . . . . . . . . . 287

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288

16 Basic Undecidability Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . 29116.1 Some Decidable and Undecidable Problems . . . . . . . . . . . . 292

16.1.1 An assortment of decidable problems . . . . . . . . . . . . 29216.1.2 Assorted undecidable problems . . . . . . . . . . . . . . . . . 294

XIV Contents

16.2 Undecidability Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29516.2.1 Turing recognizable (or recursively enumerable)

sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29516.2.2 Recursive (or decidable) languages . . . . . . . . . . . . . . 29716.2.3 Acceptance (ATM ) is undecidable (important!) . . . 29816.2.4 Halting (HaltTM ) is undecidable (important!) . . . . 29916.2.5 Mapping reductions . . . . . . . . . . . . . . . . . . . . . . . . . . . 30116.2.6 Undecidable problems are “ATM in disguise” . . . . . 305

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306

17 Advanced Undecidability Proofs . . . . . . . . . . . . . . . . . . . . . 30917.1 Rice’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30917.2 Failing proof attempt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

17.2.1 Corrected proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31117.2.2 Greibach’s Theorem. . . . . . . . . . . . . . . . . . . . . . . . . . . 312

17.3 The Computation History Method . . . . . . . . . . . . . . . . . . . 31217.3.1 Decidability of LBA acceptance . . . . . . . . . . . . . . . . 31317.3.2 Undecidability of LBA language emptiness . . . . . . . 31317.3.3 Undecidability of PDA language universality . . . . . 31317.3.4 Post’s correspondence problem (PCP) . . . . . . . . . . . 31517.3.5 PCP is undecidable . . . . . . . . . . . . . . . . . . . . . . . . . . . 31617.3.6 Proof sketch of the undecidability of PCP . . . . . . . 317

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320

18 Basic Notions in Logic including SAT . . . . . . . . . . . . . . . . 32318.1 Axiomatization of Propositional Logic . . . . . . . . . . . . . . . . 32418.2 First-order Logic (FOL) and Validity . . . . . . . . . . . . . . . . . 326

18.2.1 A warm-up exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . 32618.2.2 Examples of interpretations . . . . . . . . . . . . . . . . . . . . 32718.2.3 Validity of first-order logic is undecidable . . . . . . . . 32918.2.4 Valid FOL formulas are enumerable . . . . . . . . . . . . . 331

18.3 Properties of Boolean Formulas . . . . . . . . . . . . . . . . . . . . . . 33118.3.1 Boolean satisfiability: an overview . . . . . . . . . . . . . . 33118.3.2 Normal forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33218.3.3 Overview of direct DNF to CNF conversion . . . . . . 33318.3.4 CNF-conversion using gates . . . . . . . . . . . . . . . . . . . . 33618.3.5 DIMACS file encoding . . . . . . . . . . . . . . . . . . . . . . . . 33718.3.6 Unsatisfiable CNF instances . . . . . . . . . . . . . . . . . . . 33918.3.7 3-CNF, %=-satisfiability, and general CNF . . . . . . . . 34018.3.8 2-CNF satisfiability . . . . . . . . . . . . . . . . . . . . . . . . . . . 341

Contents XV

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342

19 Complexity Theory and NP-Completeness . . . . . . . . . . . 34519.1 Examples and Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346

19.1.1 The traveling salesperson problem . . . . . . . . . . . . . . 34619.1.2 P-time deciders, robustness, and 2 vs. 3 . . . . . . . . . 34619.1.3 A note on complexity measurement . . . . . . . . . . . . . 34719.1.4 The robustness of the Turing machine model . . . . . 34819.1.5 Going from “2 to 3” changes complexity . . . . . . . . . 348

19.2 Formal Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34819.2.1 NP viewed in terms of verifiers . . . . . . . . . . . . . . . . . 34819.2.2 Some problems are outside NP . . . . . . . . . . . . . . . . . 34919.2.3 NP-complete and NP-hard . . . . . . . . . . . . . . . . . . . . . 35019.2.4 NP viewed in terms of deciders . . . . . . . . . . . . . . . . . 35019.2.5 An example of an NP decider . . . . . . . . . . . . . . . . . . 35119.2.6 Minimal input encodings . . . . . . . . . . . . . . . . . . . . . . 353

19.3 NPC Theorems and proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . 35319.3.1 NP-Completeness of 3-SAT . . . . . . . . . . . . . . . . . . . . 35419.3.2 Practical approaches to show NPC . . . . . . . . . . . . . . 358

19.4 NP-Hard Problems can be Undecidable (Pitfall) . . . . . . . . 36019.4.1 Proof that Diophantine Equations are NPH . . . . . . 36019.4.2 “Certificates” of Diophantine Equations . . . . . . . . . 36119.4.3 What other complexity measures exist? . . . . . . . . . . 362

19.5 NP, CoNP, etc. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365

20 DFA for Presburger Arithmetic . . . . . . . . . . . . . . . . . . . . . . 36920.1 Presburger Formulas and DFAs . . . . . . . . . . . . . . . . . . . . . . 371

20.1.1 Presburger formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . 37120.1.2 Encoding conventions . . . . . . . . . . . . . . . . . . . . . . . . . 37320.1.3 Example 1 — representing x & 2 . . . . . . . . . . . . . . . 37320.1.4 Example 2 — "x.#y.(x + y) = 1 . . . . . . . . . . . . . . . . 37520.1.5 Conversion algorithm: Presburger formulas to

automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37620.2 Pitfalls to Avoid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378

20.2.1 The restriction of equal bit-vector lengths . . . . . . . . 379

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380

XVI Contents

21 Model Checking: Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38121.1 An Introduction to Model Checking . . . . . . . . . . . . . . . . . . 38121.2 What Are Reactive Computing Systems? . . . . . . . . . . . . . . 384

21.2.1 Why model checking? . . . . . . . . . . . . . . . . . . . . . . . . . 38521.2.2 Model checking vs. testing . . . . . . . . . . . . . . . . . . . . . 387

21.3 Buchi automata, and Verifying Safety and Liveness . . . . . 38921.4 Example: Dining Philosophers . . . . . . . . . . . . . . . . . . . . . . . 390

21.4.1 Model (proctype) and property (never) automata . 394

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396

22 Model Checking: Temporal Logics . . . . . . . . . . . . . . . . . . . 39922.1 Temporal Logics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399

22.1.1 Kripke structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39922.1.2 Computations vs. computation trees . . . . . . . . . . . . 40122.1.3 Temporal formulas are Kripke structure classifiers! 40322.1.4 LTL vs. CTL through an example . . . . . . . . . . . . . . 40522.1.5 LTL syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40622.1.6 LTL semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40622.1.7 CTL syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40722.1.8 CTL semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414

23 Model Checking: Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 41923.1 Enumerative CTL Model Checking . . . . . . . . . . . . . . . . . . . 41923.2 Symbolic Model Checking for CTL . . . . . . . . . . . . . . . . . . . 421

23.2.1 EG p through fixed-point iteration . . . . . . . . . . . . . . 42123.2.2 Calculating EX and AX . . . . . . . . . . . . . . . . . . . . . . . 42423.2.3 LFP and GFP for ‘Until’ . . . . . . . . . . . . . . . . . . . . . . 42523.2.4 LFP for ‘Until’ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42623.2.5 GFP for Until . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427

23.3 Buchi Automata and LTL Model Checking . . . . . . . . . . . . 42823.3.1 Comparing expressiveness . . . . . . . . . . . . . . . . . . . . . 42823.3.2 Operations on Buchi automata . . . . . . . . . . . . . . . . . 43023.3.3 Nondeterminism in Buchi automata . . . . . . . . . . . . . 431

23.4 Enumerative Model Checking for LTL . . . . . . . . . . . . . . . . 43223.4.1 Reducing verification to Buchi automaton

emptiness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436

Contents XVII

24 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439

Book web site and tool information . . . . . . . . . . . . . . . . . . . . . . 443A.1 Web site and e-mail address . . . . . . . . . . . . . . . . . . . . . . . . . 443A.2 Software tool usage per chapter . . . . . . . . . . . . . . . . . . . . . . 443A.3 Possible Syllabi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444

BED Solution to the tic-tac-toe problem . . . . . . . . . . . . . . . . . 447

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461

List of Figures

1.1 The power of various machines, and how to realize them 4

3.1 Proof of the Schroder-Bernstein Theorem . . . . . . . . . . . . . . 463.2 Proof approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.1 Some example binary relations . . . . . . . . . . . . . . . . . . . . . . . 544.2 The binary relation Power is shown. The dotted edges

are some of the edges implied by transitivity. Undottedand dotted means the same in this diagram. Therefore,Power actually contains: (i) the pairs corresponding tothe solid edges, (ii) the pairs indicated by the dottededges, (iii) and those pairs indicated by those dottedtransitive edges not shown. . . . . . . . . . . . . . . . . . . . . . . . . . . 61

4.3 The equivalence relation Power' Power!1 . . . . . . . . . . . . . 634.4 The identity relation over MT . . . . . . . . . . . . . . . . . . . . . . . 64

8.1 A DFA that recognizes strings over {0, 1} ending with 1 . 1218.2 Pseudocode for the DFA of Figure 8.1 . . . . . . . . . . . . . . . . . 1228.3 Multiple symbols labeling a transition in lieu of multiple

transitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1228.4 A DFA with unreachable (redundant) states . . . . . . . . . . . 1228.5 Another example DFA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1238.6 Drawing DFAs using dot . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1298.7 Processing .ps files in Latex . . . . . . . . . . . . . . . . . . . . . . . . . . 129

9.1 Minimal DFAs for Lk for k = 2, 3, 4, 5, with 2k+1 states . 1409.2 An NFA for (0 + 1)" 0 (0 + 1)k, for k = 2 . . . . . . . . . . . . . . 1419.3 An NFA for ((0 + 1)" 0 (0 + 1)k)+, for k = 2 . . . . . . . . . . . 1439.4 An example NFA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

XX List of Figures

9.5 DFA obtained using grail . . . . . . . . . . . . . . . . . . . . . . . . . . 1459.6 Legal and illegal phone numbers in our example . . . . . . . . 1499.7 An NFA formally specifying allowed telephone numbers,

and the RE describing it . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1509.8 A minimal DFA for the NFA of Figure 9.7 . . . . . . . . . . . . . 1519.9 Skeleton of an encoding of the telephone number syntax

in flex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

10.1 NFA to DFA conversion illustrated on the NFA ofFigure 9.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

10.2 DFA(a) ( DFA(b) = DFA(c); DFA(a) ' DFA(b) =

DFA(d); DFA(a) = DFA(e) . . . . . . . . . . . . . . . . . . . . . . . . . . . 16310.3 NFA(c) = NFA(a) ( NFA(b); NFA(d) = NFA(a) ) NFA(b) . 16410.4 The result of Starring Figure 10.3(c) . . . . . . . . . . . . . . . . . . 16610.5 The result of reversing the NFA of Figure 10.4 . . . . . . . . . 16710.6 An example NFA to be converted to a regular expression 17110.7 Result of the preprocessing step . . . . . . . . . . . . . . . . . . . . . . 17110.8 Result of eliminating state A1. . . . . . . . . . . . . . . . . . . . . . . . 17210.9 Result of Eliminating B0 and A2 . . . . . . . . . . . . . . . . . . . . . 17210.10Result of Eliminating FA . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17310.11(i) Example for DFA minimization, (ii) initial table for

DFA minimization, (iii) steps in DFA minimization,and (iv) the final minimized DFA. . . . . . . . . . . . . . . . . . . . . 175

10.12A DFA that has two error strata implementing allstrings that are a Hamming distance of 2 away from thelanguage (0101)+ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

10.13Minimal DFAs for the same language are isomorphic . . . . 17810.14A regular expression for all 0-, 1-, and 2-bit errors . . . . . . 179

11.1 Minimal DFAs for d* (a+ b+ c) for (a) variable orderingabcd, and (b) dabc. The edges show transitions forinputs arriving according to this order. . . . . . . . . . . . . . . . . 188

11.2 Minimal DFAs where the variable ordering matters . . . . . 18911.3 BDD for a = b + c = d + e = f for variable order acebdf . 19011.4 Simple state transition system (example SimpleTR) . . . . 19211.5 BED commands for reachability analysis on SimpleTR,

and the fixed-point iteration leading up to the leastfixed-point that denotes the set of reachable statesstarting from I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

List of Figures XXI

11.6 Example where multiple fixed-points exist. This figureshows attainment of a fixed-point a* b which is betweenthe least fixed-point of a+ b and the greatest fixed-pointof 1. The figure shows the initial approximant P0 andthe next approximant P1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

13.1 (a) The parse tree for string 110 with respect to CFGG2; (b) and (c) are parse trees for 110 with respect to G4.219

13.2 A string that does not cause zero-crossings. Thenumbers below the string indicate the runningdi!erence between the number of a’s and the number ofb’s at any point along the string . . . . . . . . . . . . . . . . . . . . . . 221

13.3 A string that causes zero-crossings . . . . . . . . . . . . . . . . . . . . 22113.4 Parse tree for 4 + 5 $ 6 with respect to G8. . . . . . . . . . . . . . 22613.5 The Chomsky hierarchy and allied notions . . . . . . . . . . . . . 22813.6 A PDA for the language L0 = {wwR | w , {0, 1}"} . . . . 23513.7 Depiction of a parse tree for the CFL Pumping Lemma.

The upper drawing shows a very long path thatrepeats a non-terminal, with the lowest two repetitionsoccurring at V 2 and V 1 (similar to Occurrence-1and Occurrence-2 as in the text). With respect to thisdrawing: (i) the middle drawing indicates what happensif the derivation for V 2 is applied in lieu of that ofV 1, and (ii) the bottom drawing depicts what happensif the derivation for V 2 is replaced by that for V 1,which, in turn, contains a derivation for V 2 . . . . . . . . . . . 241

14.1 Transition table and transition graph of a PDA forthe language L0 = {wwR | w , {0, 1}"}, and anillustration of the - relation on input 001100 . . . . . . . . . . . 248

14.2 The PDA of Figure 13.6 converted to one that acceptsby empty stack. There are some redundancies in thisPDA owing to our following a standard constructionprocedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

14.3 A PDA for Lambnck of Illustration 13.2.1 . . . . . . . . . . . . . . . 25214.4 A PDA whose language is being proved correct using

Floyd’s method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25314.5 CFG to PDA conversion for the CFG of Illustration 13.2.125614.6 PDA to CFG conversion. Note that e means the same

as ". . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25714.7 Steps of the CKY parsing algorithm on input 001 . . . . . . 26314.8 Steps of the CKY parsing algorithm on input aabbab . . . 266

XXII List of Figures

15.1 A TM for {anbncn | n . 0}, with start state q0, finalstate q6, and moves occurring from the row-states tocolumn-states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

15.2 A Turing machine for w#w . . . . . . . . . . . . . . . . . . . . . . . . . . 28115.3 A Nondeterministic Turing machine for ww. Letter ‘S’

means that the head stays where it is . . . . . . . . . . . . . . . . . 285

16.1 ATM to HaltTM reduction. Notice that if we assumethat the inner components – namely the OR-gate, theability to run M on w, and DHaltTM

exist, then DATM

can be constructed; and hence, DHaltTMcannot exist! . . . 299

16.2 Illustration of mapping reduction A &M B . . . . . . . . . . . . . 30116.3 How the mapping reduction from ATM to HaltTM works 30216.4 Mapping reduction from ATM to ETM . . . . . . . . . . . . . . . . 30316.5 Mapping reduction from ATM to RegularTM . . . . . . . . . . 303

17.1 Machine M!in the proof of Rice’s Theorem . . . . . . . . . . . . 310

17.2 An accepting computation history and its encoding asper the PCP encoding rules . . . . . . . . . . . . . . . . . . . . . . . . . . 318

18.1 Min/Max terms for nand, whose DNF form ism0 + m1 + m2 and CNF form is M3 (the only max termwhere the function is 0) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332

18.2 A CNF generator, Part-1 (continued in Figure 18.3) . . . . 33418.3 A CNF generator, Part-2 (continued from Figure 18.2) . . 33518.4 An unsat CNF instance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

19.1 Venn diagram of the language families P, NP, and NPC;these set inclusions are proper if P %= NP — which isan open question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346

19.2 Diagram illustrating how NPC proofs are accomplished.Definition 19.2 is illustrated by the reduction shown onthe left while Definition 19.7 is illustrated on the right. . . 354

19.3 Proof of the Cook-Levin Theorem . . . . . . . . . . . . . . . . . . . . 35519.4 The Proof that Clique is NPH using an example

formula # = (x1 * x1 * x2) + (x1 * x1 * ¬x2) + (¬x1 *¬x1 * x2) + (¬x1 * ¬x1 * ¬x2) . . . . . . . . . . . . . . . . . . . . . . . 359

19.5 The language families P, NP, and NPC. All these setinclusions are likely to be proper . . . . . . . . . . . . . . . . . . . . . 362

20.1 Prenexing code in Ocaml . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37220.2 Presburger Formula x & 2 and its DFA . . . . . . . . . . . . . . . . 373

List of Figures XXIII

20.3 DFA for x + y = 1. The transition names are referredto in Section 20.1.5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374

20.4 NFA for #y.x + y = 1 and its equivalent DFA . . . . . . . . . . 37520.5 Obtaining the DFA for "x.#y.(x + y) = 1 from the

DFA in Figure 20.4 through complementation, hiding,determinization, and complementation . . . . . . . . . . . . . . . . 377

20.6 Study of "x.#y. x / y = 1 versus "y.#x. x / y = 1 . . . . . . 379

21.1 Three dining philosophers . . . . . . . . . . . . . . . . . . . . . . . . . . . 39021.2 Promela model for three dining philosophers . . . . . . . . . . . 39121.3 Message sequence chart showing liveness violation bug . . 395

22.1 Two Kripke structures and some of their computations.In the Kripke structure on the left, the assertion‘Henceforth (a * b)’ is true. . . . . . . . . . . . . . . . . . . . . . . . . . . 400

22.2 The basic motivation for computation trees . . . . . . . . . . . . 40222.3 Computation trees for the two Kripke structures

of Figure 22.2, as well as the left-hand side Kripkestructure of Figure 22.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402

22.4 AG (EF x) is true, yet there is a computation where xis permanently false . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405

22.5 Illustration of basic notions relating to computation trees 40922.6 Nine diagrams, some of which are Kripke Structures . . . . 415

23.1 A Kripke structure for critical sections . . . . . . . . . . . . . . . . 42023.2 Formula and its Subformulas . . . . . . . . . . . . . . . . . . . . . . . . . 42123.3 Algorithm used in CTL model checking illustrated on

the “AF” operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42223.4 Table for our CTL model checking example . . . . . . . . . . . . 42323.5 Kripke structures that help understand CTL . . . . . . . . . . . 42323.6 BDD for EX (a != b) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42423.7 The Strong Until (U) iteration that reaches a fixed-point 42623.8 Transition graphs read as an NFA or as an NBA . . . . . . . 42823.9 An NBA accepting finitely many a’s (left) and a DBA

accepting infinitely many a’s (right) . . . . . . . . . . . . . . . . . . 43123.10A Kripke structure (left), its corresponding Buchi

automata (middle), and a property automatonexpressing GFx (right) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432

23.11System automaton (left), complemented propertyautomaton (middle), and product automaton (right) . . . . 432

XXIV List of Figures

23.12Searching of a product graph for cycles: Exampleproduct graph (left), group of stacks for the outer DFS(middle), and group of stacks for the inner DFS (right) . . 434

Foreword

It takes more e!ort to verify that digital system designs are correctthan it does to design them, and as systems get more complex theproportion of cost spent on verification is increasing (one estimate isthat verification complexity rises as the square of design complexity).

Although this verification crisis was predicted decades ago, it isonly recently that powerful methods based on mathematical logic andautomata theory have come to the designers’ rescue. The first suchmethod was equivalence checking, which automates Boolean algebracalculations. Next came model checking, which can automatically verifythat designs have – or don’t have – behaviours of interest specified intemporal logic. Both these methods are available today in tools sold byall the major design automation vendors.

It is an amazing fact that ideas like Boolean algebra and modallogic, originating from mathematicians and philosophers before moderncomputers were invented, have come to underlie computer aided toolsfor creating hardware designs.

The recent success of ’formal’ approaches to hardware verificationhas lead to the creation of a new methodology: assertion based design,in which formal properties are incorporated into designs and are thenvalidated by a combination of dynamic simulation and static modelchecking. Two industrial strength property languages based on tempo-ral logic are undergoing IEEE standardisation.

It is not only hardware design and verification that is changing: newmathematical approaches to software verification are starting to be de-ployed. Microsoft provides windows driver developers with verificationtools based on symbolic methods.

Discrete mathematics, logic, automata, and the theory of com-putability are the basis for these new formal approaches. Although they

XXVI Foreword

have long been standard topics in computer science, the uses made ofthem in modern verification are quite di!erent to their traditional roles,and need di!erent mathematical techniques. The way they are taughtoften puts more emphasis on cultivating ‘paper and pencil’ proof skills,and less on their practical applications and implementation in tools.Topics in logic are often omitted, or taught without emphasizing con-nections with automata, and without explaining the algorithms (e.g.,fixed-point computation) used in verification.

This classroom-tested undergraduate textbook is unique in present-ing logic and automata theory as a single subject. Public domain soft-ware is used to animate the theory, and to provide a hands-on taste ofthe algorithms underlying commercial tools. It is clearly written andcharmingly illustrated. The author is a distinguished contributor toboth theory and to new tool implementation methods.

I highly recommend this book to you as the best route I know intothe concepts underlying modern industrial formal verification.

Dr. Michael J.C. Gordon FRSProfessor of Computer Assisted Reasoning

The University of Cambridge Computer Laboratory

Preface

Computation Engineering, Applied Automata Theory and Logic:

With the rapidly evolving nature of Computing, and with multiple newtopics vying for slots in today’s undergraduate and graduate curricula,“classical” topics such as Automata Theory, Computability, Logic, andFormal Specification cannot fill the vast expanses they used to fill inboth the undergraduate and the graduate syllabi. This move is alsonecessary considering the fact that many of today’s students preferlearning theory as a tool rather than theory for theory’s sake. This bookkeeps the above facts in mind and takes the following fresh approach:

• approaches automata theory and logic as the underlying engineeringmathematics for Computation Engineering,

• attempts to restore the Automaton-Logic connection missing inmost undergraduate books on automata theory,

• employs many interactive tools to illustrate key concepts,• employs humor and directly appeals to intuitions to drive points

home,• covers classical topics such as the Rice’s Theorem, as well as modern

topics such as Model Checking, Buchi Automata, and TemporalLogics.

We now elaborate a bit further on these points, and then provide achapter-by-chapter description of the book.

Teach Automata Theory and Logic as if they were Engineering Math:

The computer hardware and software industry is committed to usingmathematically based (formal) methods, and realizes that the biggestimpediment to the large scale adoption of these methods would be

XXVIII Preface

the lack of adequately trained manpower. In this context, it is cru-cial that students who go through automata theory and logic coursesretain what they have learned, and know how to use their knowledge.Today’s undergraduate textbooks in the area of this book typically em-phasize automata theory, and not logic. This runs the risk of impartinga skewed perspective, as both these perspectives are needed in practice.Also most of today’s books do not include tool-based experiments. Ex-perience shows that tool based experimentation can greatly enhanceone’s retention.

Restoring the Missing Automaton/Logic Connection:

Automata theory and logic evolved hand-in-hand - and yet, this connec-tion was severed in the 1970’s when the luxury of separate automatatheory and logic courses became possible. Now, the crowded syllabithat once again forces these topics to co-exist may actually be doinga huge favor: providing the opportunity to bring these topics back to-gether! For example, Binary Decision Diagrams (BDD) - central datastructures for representing Boolean functions - can be viewed as min-imized DFA. One can introduce this connection and then show howfinite state machines can be represented and manipulated either usingexplicit state graphs or implicitly using BDD based Boolean transitionrelations. Another example I’ve employed with great success is thatof the formulation and solution of the logical validity of simple sen-tences from Presburger arithmetic – such as “"xyz : x + y = z” –using DFA. Here, the use of automaton operations (such as intersec-tion, complementation, and projection) and corresponding operationsin logic (such as conjunction, negation, and existential quantification)in the same setting helps illustrate the interplay between two intimatelyrelated areas, and additionally helps build strong intuitions.

Teaching Through Interactive Tools:

To the best of my knowledge, none of the present-day undergraduatebooks in automata theory employ interactive tools in any significantmanner. This approach tends to give the false impression to studentsthat these are topics largely of theoretical interest, with the only prac-tical examples coming from the area of compiler parsers. We encouragetool usage in the following ways:

• we illustrate the use of the Grail tools, originally from the Universityof Western Ontario, to illustrate the application of operations onautomata in an interactive manner,

Preface XXIX

• we illustrate the use of the JFLAP tool kit written by ProfessorSusan Rodger’s group at Duke when discussing nondeterministicTuring machines,

• we employ the BED Binary Decision Diagram package of Anderssonand Hulgaard in illustrating several aspects of BDD manipulationand fixed-point computation,

• we illustrate the use of Boolean satisfiability tools such as Zcha!and Minisat in leading up to the concept of NP-completeness,

• we illustrate DNF to CNF conversion and obtaining the PrenexNormal Form using simple programs written in the functional lan-guage Ocaml, and finally

• we present simple examples using the model checking tool SPIN.

On page 443, we provides the address of the book website where tool-specific instructions will be maintained.

Letting Intuitions be the Guide:

I have found that introducing diagonalization proofs early on can helpstudents tie together many later ideas more e!ectively. I employ gentleintuitive introductions (e.g., “the US map has more points than hairon an infinite-sized dog”).

Many topics become quite clear if demonstrated in multiple do-mains. For example, I illustrate fixed-point theory by discussing howcontext-free grammars are recursive language equations. I also intro-duce fixed-points by pointing out that if one repeatedly photocopies1

a document, it will often tend towards one of the fixed points of theimage transformation function of the photocopy machine(!).

My example to introduce the fact that there are a countably infinitenumber of C programs consists of pointing out that the following are le-gal C programs: main(){}, main(){{}}, main(){{{}}}, ... and then I usethe Schroder-Bernstein theorem relating this sequence to the sequenceof even numbers . 8 (which can then be bijectively mapped to Natu-ral numbers). In another example, I show that the set of C programsare not regular by applying the Pumping Lemma to main(){{. . .}} andgetting a syntax error when the Pumping Lemma mangles the brack-eting structure of such C programs! In my opinion, these examples donot diminish rigor, and may succeed in grabbing the attention of manystudents.

1 Xerox is still a trademark of Xerox Inc.

XXX Preface

Model Checking, Buchi Automata, and Temporal Logics:

From a practical point of view, automata theory and logic play a cen-tral role in modeling and verifying concurrent reactive systems. Thehardware and software industry employs model checking tools to finddeep seated bugs in hardware and software. This book closes o! withan introduction to the important topic of model checking.

A Tour Through the Book:

Chapter 1 motivates the material in the book, presenting in great detailwhy the topics it covers are important both in terms of theory andpractice.

Chapter 2 begins with the quintessentially important topics of sets,functions, and relations. After going through details such as expressingfunction signatures, the di!erence between partial and total functions,we briefly examine the topic of computable functions - functions thatcomputers can hope to realize within them. We point out importantdi!erences between the terms procedure and algorithm, briefly touchingon the 3x + 1 problem - a four-line program that confounds scientistsdespite decades of intense research. In order to permit you to discussfunctions concretely, we introduce the lambda notation. A side benefitof our introduction to Lambda calculus is that you will be able to studyanother formal model of computation besides Turing machines (that weshall study later).

Chapter 3 goes through the concept of cardinality of sets, which initself is extremely mentally rewarding, and also reinforces the techniqueof proof by contradiction. It also sets the stage for defining fine distinc-tions such as ‘all languages,’ of which there are “uncountably many”members, and ‘all Turing recognizable languages,’ of which there areonly “countably many” members.

Chapter 4 discusses important classes of binary relations, such asreflexive, transitive, preorder, symmetric, anti-symmetric, partial order,equivalence, identity, universal, equivalence, and congruence (modulooperators).

Chapter 5 provides an intuitive introduction to mathematical logic.We have written this chapter with an eye towards helping you readdefinitions involving the operators if, if and only if (i!), and quanti-fiers for all, and there exists. The full import of definitions laden withifs and if and only ifs, as well as quantifiers, is by no means readilyapparent - and so it is essential to cultivate su"cient practice. You willsee proof by contradiction discussed at a ‘gut level’ - we encourage you

Preface XXXI

to play the game of Mastermind, where you can apply this techniquequite e!ectively to win pretty much every time. Nested quantificationis reinforced here, as well as while discussing the pumping lemma inChapter 12. Our message is: write proofs clearly and lucidly - that way,in case you go wrong, others can spot the mistake and help you.

Chapter 6 studies the topic of recursion at some depth. Given thata book on automata theory is ‘ridden with definitions,’ one has to un-derstand carefully when these definitions make sense, and when theyend up being ‘total nonsense,’ owing, say, to being circular or notuniquely defining something. Lambda calculus provides basic notationwith which to reason about recursion. We present to you the “friendli-est foray into fixed-point theory’ that we can muster.” We will usefixed-points as a ‘one-stop shopping’ conceptual tool for understandinga diverse array of topics, including context-free productions and thereachable states of finite-state machines.

In Chapter 7, we begin discussing the notions of strings and lan-guages. Please, however, pay special attention2 to “the five most con-fused objects of automata theory,” namely ", 0, {"}, {0}, and theequation 0" = {"}. We discuss the notion of concatenation, exponentia-tion, ‘starring,’ complementation, reversal, homomorphism, and prefix-closure applied to languages.

In Chapter 8, we discuss machines, languages, and deterministic fi-nite automata. We construct Deterministic Finite Automata (DFA) forseveral example languages. One problem asks you to build a DFA thatscans a number presented in any number-base b, either most signifi-cant digit-first, or least significant digit-first, and shine its green lightexactly at those moments when the number scanned so far equals 0in some modulus k. The latter would be equivalent to division carriedout least significant digit-first, with only the modulus to be retained.We will have more occasions to examine the true “power” of DFAsin Chapters 12 and 20. Chapter 8 closes o! with a brief study of thelimitations of DFA.

Chapter 9 continues these discussions, now examining the cruciallyimportant concept of non-determinism. It also discusses regular ex-pressions - a syntactic means for describing regular languages. The im-portant topic of Non-deterministic Finite Automaton (NFA) to DFAconversion is also examined.

Automata theory is a readily usable branch of computer science the-ory. While it is important to obtain a firm grasp of its basic principles

2 Once you understand the fine distinctions between these objects, you will detecta faint glow around your head.

XXXII Preface

using paper, pencil, and the human brain, it is quite important thatone use automated tools, especially while designing and debugging au-tomata. To paraphrase Professor Dana Scott, computer-assisted toolsare most eminently used as the telescopes and microscopes of learning- to see farther, to see closer, and to see clearer than the human mindalone can discern. We demonstrate the use of grail tools to generateand verify automata.

In Chapter 10, we discuss operations that combine regular lan-guages, most often yielding new regular languages as a result. We dis-cuss the conversion of NFA to DFA - an important algorithm boththeoretically and in practice. We also discuss the notion of ultimate pe-riodicity which crystallizes the true power of DFAs in terms of languageacceptance, and also provide a tool-based demonstration of this idea.

In Chapter 11, we will begin discussing binary decision diagrams(BDD), which are nothing but minimized DFA for the language ofsatisfying assignments (viewed as sequences, as we will show) for givenBoolean expressions. The nice thing about studying BDDs is that ithelps reinforce not only automata-theoretic concepts but also conceptsfrom the area of formal logic. It teaches you a technique widely used inindustrial practice, and also paves the way to your later study of thetheory of NP-completeness.

In Chapter 12, we discuss the Pumping Lemma. We define thelemma in first-order logic, so that the reader can avoid common confu-sions, and grasp how the lemma is employed. We also discuss completePumping Lemmas (regular if and only if certain conditions are met).

In Chapter 13, we present the idea of context-free languages. Context-free languages are generated by a context-free grammar that consists ofproduction rules. By reading production rules as recursive equations,we can actually solve for the context-free language being defined. Wewill also have occasion to prove, via induction, that context-free pro-ductions are sound and complete - that they do not generate a stringoutside of the language, but do generate all strings within the language.The important notions of ambiguity and inherent ambiguity will alsobe studied.

Chapter 14 will introduce our first infinite state automaton variety- the push-down automaton (PDA) - a device that has a finite-controland exactly one unbounded stack. We will study a method to convertPDA to CFG and vice versa. We will also show how to prove PDAscorrect using Floyd’s Inductive Assertions method.

In Chapter 15, we study Turing machines (TM). Important notions,such as instantaneous descriptions (ID) are introduced in this chapter.

Preface XXXIII

Several Turing machine simulators are available on the web - you areencouraged to download and experiment with them. In this chapter,we will also introduce linear bounded automata (LBA) which are TMswhere one may deposit new values only in that region of the tape wherethe original input was presented.

Chapter 16 discusses some of the most profound topics in computerscience: the halting problem, the notion of semi-decision procedures,and the notion of algorithms. With the introduction of unboundedstore, many decision problems will become formally undecidable. Ina large number of cases, it will no longer be possible to predict whatthe machine will do. For example, it will become harder or impossibleto tell whether a machine will halt when started on a certain input,whether two machines are equivalent, etc. In this chapter, we will for-mally state and prove many of these undecidability results. We willpresent three proof methods: (i) through contradiction, (ii) through re-ductions from languages not known to be decidable, and (iii) throughmapping reductions.

Chapter 17 continues with the notion of undecidability, discussingtwo additional proof techniques: (i) through the computational historymethod, and (ii) by employing Rice’s theorem. These are advancedproof methods of undecidability that may be skipped during the initialpass through the textbook material. One can proceed to Chapter 18after finishing Chapter 16 without much loss of continuity.

Chapter 18 sets the stage to discuss the theory of NP completeness.It touches on a number of topics in mathematical logic that will helpyou better appreciate all the nuances. We briefly discuss a “Hilbertstyle” axiomatization of propositional logic, once again touching onsoundness and completeness. We discuss basic definitions including sat-isfiability, validity, tautology, and contradiction. We discuss the various“orders” of logic including zeroth-order, first-order, and higher-order.We briefly illustrate that the validity problem of first-order logic isonly semi-decidable by reduction from the Post Correspondence prob-lem. We illustrate how to experiment with modern SAT tools. We alsocover related ideas such as %=-sat, 2-sat, and satisfiability-preservingtransformations.

Chapter 19 discusses the notion of polynomial-time algorithms, com-putational complexity, and the notion of NP-completeness. We reiteratethe importance of showing that the problem belongs to NP , in additionto demonstrating NP-hardness.

In Chapter 20, we introduce a small subset of first-order logic calledPresburger arithmetic that enjoys the following remarkable property:

XXXIV Preface

given a formula in this logic, we can build a finite automaton suchthat the automaton has an accepting run if and only if the formula issatisfiable. The neat thing about this technique is that it reinforces theautomaton/logic connection introduced in the previous chapter.

Chapters 21 through 23 introduce temporal logic and model check-ing. These topics are ripe for introduction to undergraduates and grad-uates in all areas of computer science, but without all the generalityfound in specialized books. To this end, in Chapter 21, we providea history of model checking and also a detailed example. Chapter 22introduces linear-time temporal logic and Computational Tree Logic,contrasting their expressive power, and exactly why computation treesmatter. Finally, Chapter 23 presents an enumerative as well as a sym-bolic algorithm for CTL model checking. We also present an enumer-ative algorithm for LTL model checking through an example. We in-troduce Buchi automata, discussing how Boolean operations on Buchiautomata are performed, and that non-deterministic Buchi automataare not equivalent to deterministic Buchi automata.

Chapter 24 reviews the material presented in the book. Chapter Alists the book website, software tool related information, and possiblesyllabi based on this book.

Acknowledgements:

This book was written over three years, with numerous interruptions.I have classroom-tested parts of this book in an undergraduate classentitled Models of Computation (CPSC 3100, Spring 2002), a graduateclass entitled Foundations of Computing (CPSC 6100, Spring 2005),and a graduate class entitled Model Checking (CPSC 6964, Fall 2005).Many thanks to students who took these classes (with due apologiesfor having subjected them to half-done material). It was also o!ered asone of the references by Professor Konrad Slind when he taught CPSC3100 during Fall 2005, for which I am grateful.

I am very grateful for many friends, colleagues, and students whohave proof-read significant parts of this book and provided very valu-able comments. As best as I recall, here is a list of people who deservespecial mention: Steven Johnson, John Harrison, Rajeev Alur, MichaelJ.C. Gordon, Mark Aagaard, Paliath Narendran, Ching-tsun Chou,Panagiotis (Pete) Manolios, Konrad Slind, Joe Hurd, Kathi Fisler,Salman Pervez, Xiaofang Chen, Michael Freedman, Geo! Draper, VijayDurairaj, Shrish Jain, Vamshi Kadaru, Seungkeol Choe, and SonjongHwang (and apologies to those I am forgetting). I would also like to

Preface XXXV

thank the authors of the computer-aided design tools I have used inthis book (listed on page 443).

Thanks also to the support team at Springer, especially FrankHolzwarth and Deborah Doherty, for their timely help. Many thanksto Michael Hackett, Editor, for helping me launch this book project.Special thanks to Carl W. Harris, Senior Editor, Springer, for patientlyleading me through all the major stages of this book, and for verypatiently working with the reviewers of this book. Thanks, Carl!

I especially thank Geo! Draper, a student in my CPSC 6100 class(Spring 2005) who provided extensive comments, drew the cartoonsyou see in this book, as well as kindly gave me permission to employhis cartoons in this book. A reader expecting to read “the usual theorybook” is in for a surprise: they will find me joking about at variousjunctures, as I normally do when I lecture. I keep books such as TheCartoon Guide to Genetics by Larry Gonick and Mark Wheelis in mindwhen I engage in these diversions. Such books are immensely importantcontributions, in that they make an otherwise dull and dry subject comealive, especially for newcomers. In my humble opinion, many more suchbooks are badly needed to describe computer science theory in a plainand accessible manner to students who might otherwise swallow theoryas if it were a bitter pill. I have attempted to slant this book in thatdirection, without compromising the seriousness of the topic. If I havefailed in any way to do so, I o!er my advance apologies.3

I have benefited immensely by reading many contemporary books -notably one by Michael Sipser and one by Dexter Kozen. I have alsodrawn from Zohar Manna’s book Mathematical Theory of Computation.I also like to thank the creators of all the computer based tools employedin this book.

Last but not least, my sincere thanks to my wife Kalpana and daugh-ters Kamala and Kajal for all their love, caring, and cheer. Withouttheir understanding and accommodation, I could not have worked onthis book. I also thank my parents, whose names are my middle andlast names, for their venerable position of bringing me into this worldand seeing to it that I lacked nothing.

Salt Lake City, Utah, USA Ganesh GopalakrishnanApril 2006

3 Even Patch Adams might occasionally have overdone his antics!

1

Introduction

Welcome to Computation Engineering - the discipline of applying engi-neering principles to model, analyze, and construct computing systems.Human society is, more than ever, reliant on computing systems operat-ing correctly within automobile control systems, medical electronic de-vices, telephone networks, mobile robots, farms, nuclear power plants,etc. With so much entrusted to computers, how can we ensure that allof these computers are being built and operated in a manner responsi-ble to all flora and fauna? How do we avoid potential disasters - suchas mass attacks due to malevolent “viruses,” or an avionic computercrashing and rebooting mid-flight? On a deeper, more philosophicalnote, what exactly is computation? Can we mathematically character-ize those tasks that computers are capable of performing, and thosethey are incapable of performing? The subject-matter of this book isabout seeking answers to these deep questions using automata theoryand mathematical logic.

Computation Science and Computation Engineering

We distinguish a computer scientist from a computation engineer. Wedefine those people who seek an in-depth understanding of the phe-nomenon of computation to be a computer scientist. We define thosepeople who seek to e"ciently apply computer systems to solve real-world problems to be computation engineers.1 The distinction is, in asense, similar to that between a chemist and a chemical engineer. This

1 We prefer calling the latter activity computation engineering as opposed to com-

puter engineering because the term “computer engineer” is, unfortunately, appliednowadays to people with a hardware bias in their outlook. In this book, we carryno such bias.

2 1 Introduction

book will expose you to computer science through automata theory andlogic. It will show you how computation engineers use automata theoryand logic as the engineering mathematics, much like ‘traditional engi-neers’ apply di!erential and integral calculus to build better-engineeredproducts such as bridges, automobiles, and airplanes.

What is ‘Computation?’

By the way of popular analogy,2 we can o!er one possible feeble answer:“if it involves a computer, a program running on a computer, and num-bers going in and out, then computation is likely happening.3” Suchan answer invites a barrage of additional questions such as “what is acomputer?”; “what is a program?”, etc. There are other tricky situa-tions to deal with as well. Consider another place where computationseems to be happening: within our very body cells. Thanks to modernadvances in genetics, we are now able to understand the mind-bogglingamount of “string processing” that occurs within our cells - in the pro-cess of transcribing the genetic code (which resembles assembly codein a strange programming language), doing all of that wonderful stringmatching and error correction, and resulting in the synthesis of pro-teins. Is this also computation?

The short answer is that we cannot have either a comprehensive or apermanent definition of what ‘computation’ means. Unless we employthe precise language o!ered by mathematics, philosophical or emotion-ally charged discussions are bound to lead nowhere. One must buildformal models that crystallize the properties observed in real-worldcomputing systems, study these models, and then answer questionsabout computing and computation in terms of the models. The ab-straction must also be at the right level. Otherwise, we will end upmodeling a computer as a mindless electronic oscillator that hauls bitsaround.4

Given all this, it is indeed remarkable that computer science hasbeen able to capture the essence of computing in terms of a single for-mal device: the so called Turing machine. A Turing machine is a simpledevice that has finite-state control that interacts with an unboundedstorage tape (or, equivalently, a finite-state control that interacts withtwo unbounded stacks, as we shall show very soon). In fact, several

2 “If it walks like a duck and quacks like a duck, then it is a duck.”3 I’ve “laced” this book with several footnotes, hoping to ‘break the ice,’ and make

you believe that you are not reading a theory book.4 A similar end-result should we abstract the music of Mozart as a sound pressure

waveform.

1 Introduction 3

other formal devices - such as the Lambda calculus, Thue systems, etc.- were proposed around the same time as Turing machines. All thesedevices were also formally shown to be equivalent to Turing machines.This caused Alonzo Church to put forth his (by now famous) thesis:“All e!ectively computable functions can be understood in terms ofone of these models.” In Chapter 15 we shall study Turing machinesin great detail; Chapter 6 gives a glimpse of how the Lambda calculus,essentially through the fixed-point theory, provides a formal model forcomputer programs.

A Minimalist Approach

In a minimalist approach, models are created with the smallest possibleset of mechanisms. In the case of computational models, it is a bit ironicthat the first model proposed - namely Turing machines- was also themost powerful. However, with the increasing usage of computers,5 twoother models born out of practical necessity were proposed, roughlytwo decades after Turing machines were proposed: finite automata inthe late 1950’s, and push-down automata shortly thereafter. Rearrang-ing computer history a bit, we will discuss finite automata first, push-down automata next, and finally Turing machines (see Figure 1.1). Allthese types of machines are meant to carry out computational proce-dures (“procedures” for short,) consisting of instructions. They di!erprimarily in the manner in which they record data (“state”). A proce-dure always begins at an initial state which is highlighted by an arrowimpinging from nowhere, as in Figure 1.1. The “data input” to a proce-dure, if any, is provided through the data storage device of the machine.Each instruction, when executed, helps transform the current (data andcontrol) state into the next state. An instruction may also read an in-put symbol (some view these inputs coming from a read-only tape).Also, at every state, one or more instructions may become eligible toexecute. A deterministic machine is one that has at most one eligibleinstruction to execute at any time, while a nondeterministic machinecan have more than one eligible instruction.

A procedure halts when it encounters one of the predetermined finalstates. It is possible for a procedure to never encounter one of its finalstates; it may loop forever. If a procedure is guaranteed to halt on allinputs, it is called an algorithm. Unfortunately, it is impossible to tell

5 Thomas J. Watson, Chairman of IBM in 1943, is said to have remarked, “I thinkthere is a world market for maybe five computers.” Well, there are more than fivecomputers in a typical car today. Some cars carry hundreds, in fact!

4 1 Introduction

whether a given procedure is an algorithm - a topic that we shall revisitmany times in this book.

One of the central points made in this book is that there are essen-tially three ways to organize the data (state) recording apparatus of amachine: (i) have none at all, (ii) employ one stack to record data, and(iii) employ two stacks to record data (in other words, employ zero,one, or two stacks)! A finite-state control device by itself (i.e., withoutany additional history recording device) is called a finite automaton- either a deterministic finite automaton (DFA) or a nondeterminsticfinite automaton (NFA). A finite automaton is surprisingly versatile.However, it is not as powerful as a machine with one stack, which,by the way, is called a push-down automaton (PDA). Again there areNPDA and DPDA - a distinction we shall study in Chapters 13 and14.

A PDA is more powerful than a finite automaton. By employingan unbounded stack, a PDA is able to store an arbitrary amount ofinformation in its state, and hence, is able to refer to data items stackedarbitrarily prior. However, a PDA is not as powerful as a machine withtwo stacks. This is because a PDA is not permitted to “peek” insideits stack to look at some state s held deep inside the stack, unless it isalso willing to pop away all the items stacked since s was stacked. Sincethere could be arbitrarily many such stacked items, a PDA cannot hopeto preserve all these items being popped and restore them later.

Two stacksTMFA

No stackPDAOne stack

Data states

Instructions

controlstates

no data state

Fig. 1.1. The power of various machines, and how to realize them

1 Introduction 5

The moment a finite-state control device has access to two un-bounded stacks, however, it will have the ability to pop items fromone stack and push them into the other stack. This gives these two-stack machines the ability to peek inside the stacks; in e!ect, a finite-state control device with two stacks becomes equivalent to a Turingmachine. We can show that adding further stacks does not increase itspower! (These two-stack devices were historically first introduced in theform of Turing machines, as we shall see in more detail in Chapter 15,Section 15.2.2.) All computers, starting from the humble low-end com-puter of a Furbee doll or a digital wrist-watch, through all varieties ofdesktop and laptop computers, all the way to ‘monstrously powerful’computers, (Say, the IBM Blue Gene/L [4] computer) can be modeledin terms of Turing machines.

There is an important point of confusion we wish to avoid early. Realcomputers only have a finite amount of memory. However, models ofcomputers employ an unbounded amount of memory. This allows us tostudy the outcomes possible in any finite memory device, regardless ofhow much memory it has. So long as the finite memory device does nothit its storage limit, it can pretend that it is working with an infiniteamount of memory.

How to Measure the Power of Computers?

In comparing computational machines (“computers,” loosely), we can-not go by any subjective measure such as their absolute speed, as such“speed records” tend to become obsolete with the passage of time. Forinstance, the “supercomputers” of the 1950s did not even possess thecomputational power of many modern hand-held computers and calcu-lators. Instead, we go by the problem solving ability of computationaldevices: we deem two machines M1 and M2 to be equivalent if they cansolve the same class of problems, ignoring the actual amount of timetaken. Problem solving, in turn, can be modeled in terms of algorith-mically deciding membership in languages. Alas, these topics will haveto be discussed in far greater detail than is possible now, since we havenot set up any of the necessary formal definitions.

Complexity Theory

You may have already guessed this: in studying and comparing com-puting devices, we cannot ignore time or resources entirely. They domatter! But then one will ask, “how do we measure quantities suchas time, space, and energy?” It is very easy to see that using metrics

6 1 Introduction

such as minutes and hours is unsatisfactory, as they are non-robustmeasures, being tied to factors with ephemeral significance, such as theclock speed of computers, the pipeline depth, etc. With advances intechnology, computations that took hours a few years ago can nowa-days be performed in seconds. Again, Turing machines come to ourrescue! We define a unit of time to be one step of a Turing machinethat is running the particular program or computation. We then definetime in terms of the asymptotic worst-case complexity notation “O()”employed in any book on algorithm analysis (see for instance [29] or[6]). However, such a characterization is often not possible:

There are many problems (called NP-complete problems) whosebest known solutions are exponential time algorithms. It is unknownwhether these problems have polynomial time algorithms.There are problems for which algorithms are known not to exist; forothers, it is not known whether algorithms exist.

However, it is true that researchers engaged in studying even theseproblems employ Turing machines as one of their important formalmodels. This is because any result obtained for Turing machines canbe translated into corresponding results for real computers.

Automata Theory and Computing

In this book, we approach the above ideas through automata theory.What is automata theory? Should its meaning be cast in concrete,or should it evolve with advances in computer science and computerapplications? In this book, we take a much broader meaning for theterm ‘automata theory.’ We will use it to refer to a comprehensive listof closely related topics, including:

• finite and infinite automata (together called “automata”),• mathematical logic,• computability and complexity (TR, co-TR, NP-complete, etc.),• formal proofs,• the use of automata to decide the truth of formal logic statements,• automata on infinite words to model the behavior of reactive sys-

tems, and last but not least,• applications of the above topics in formal verification of systems.

Automata theory is a ‘living and breathing’ branch of theory - not ‘fos-silized knowledge.’ It finds day-to-day applications in numerous walksof life, in the analysis of computing system behavior. In this book,we present a number of tools to help understand automata theoretic

1 Introduction 7

concepts and also to illustrate how automata theory can be appliedin practice. We illustrate the use of automata to describe many real-life finite-state systems, including: games, puzzles, mathematical logicstatements, programming language syntax, error-correcting codes, andcombinational and sequential digital circuits. We also illustrate toolsto describe, compose, simplify, and transform automata.

Why “Mix-up” Automata and Mathematical Logic?

We believe that teaching automata theory hand-in-hand with mathe-matical logic allows us to not only cover concepts pertaining to formallanguages and machines, but also illustrate the deep connections thatthese topics have to the process of formalized reasoning – proving prop-erties about computing systems. Formalized reasoning about comput-ing systems is escalating in importance because of the increasing use ofcomputers in safety critical and life critical systems. The software andhardware used in life and resource critical applications of computers isbecoming so incredibly complex that testing these devices for correctoperation has become a major challenge. For instance, while perform-ing 100 trillion tests sounds more than adequate for many systems, itis simply inadequate for most hardware/software systems to cover allpossible behaviors.

Why Verify? Aren’t Computers “Mostly Okay?”

Human society is crucially dependent on software for carrying out anincreasing number of day-to-day activities. The presence of bugs insoftware is hardly noticed until, say, one’s hand-held computer hangs,when one pokes its reset button and moves on, with some bewilderment.The same is the case with many other computers that we use; in general,the mantra seems to be, “reset and move on!” However, this surelycannot be a general design paradigm (imagine a machine getting stuckin an infinite reboot loop if the problem does not clear following areset). In the modern context, one truly has to worry about the logicalcorrectness of software and hardware because there have been many“close calls,” some real disasters, and countless dollars have been wastedin verifying products before they are released. In 2004, the Mars SpiritRover’s computer malfunctioned when the number of allowed open filesin flash memory were exceeded. This caused a shadow of uncertaintyto hang over the project for a few days, with scientists wasting theirtime finding a cure for the problem. Recently, car companies have hadrecalls due to software bugs in their computerized engine control; the

8 1 Introduction

societal costs of such bugs (“software defects”) are estimated to be veryhigh [116, 103]. In 1996, Ariane-5, a rocket costing 2B, self-destructedfollowing an arithmetic overflow error that initially caused the rocketnozzles to be pointed incorrectly [84].

A very important human lesson is contained in many software fail-ures. In 1987, a radiation therapy machine called Therac-25 actuallycaused the death of several patients who came to get radiation therapyfor cancer. At the time of the incident, the machine had recently beenredesigned and its software was considered so reliable that many ofits safety interlock mechanisms had been dispensed with. One of theseinterlocks was for monitoring the amount of radiation received by pa-tients. To cut a long story short, the software was, after all, buggy, andended up administering massive radiation overdoses to patients [76].Every engineer shoulders the societal responsibility to adopt simpleand reliable safety measures in the systems they design and deploy toavoid such catastrophes. The approach taken in each project must be:“when in doubt, play it safe, keeping the well being of lives and naturein mind.”

The root cause of a large majority of bugs is the ambiguous and/orinconsistent specification of digital system components and subsystems.Software built based on such specifications is very likely to be flawed,hard to debug, and impossible to systematically test. As an examplefrom recent practice, the specification document of a widely used com-puter interconnect bus called the ‘PCI’ [95] was shown to be internallyinconsistent [28]. Unfortunately, the design community had moved toofar along to take advantage of these findings. Many of the disasters ofcomputer science are not directly in the form of crashes of rockets orchips (although such disasters have happened6). In 2001, the UnitedStates Federal Bureau of Investigation (FBI) launched a project tooverhaul their software to coordinate terrorist databases. After nearlyfour years and over 300 million dollars spent, the project has been de-clared to have an unrealizable set of software requirements, and henceabandoned.

The ability to write precise, as well as, unambiguous specifications iscentral to using them correctly and testing them reliably. Such formalmethods for hardware and software have already been widely adopted.Organizations such as Intel, AMD, IBM, Microsoft, Sun, HP, JPL,NASA, and NSA employ hundreds of formal verification specialists.

6 The Ariane rocket, worth 2B, was lost because of incorrect version of softwarerunning. Intel lost .5B due to a floating-point bug in their Pentium II micropro-cessor.

1 Introduction 9

Unfortunately, there are many areas where the power of formal methodshas not been demonstrated. This book hopes to expose students to thebasics of formal methods, so that they will be able to participate in, aswell as contribute to, the formal methods revolution that is happening.

Verifying Computing Systems Using Automaton Models

It turns out that in order to prove that even the simplest of comput-ing systems operates correctly, one has to examine so many inputs andtheir corresponding outputs for correctness. In many cases, it will takeseveral thousands of centuries to finish complete testing. For example,if such a computer has one megabit of internal storage, one has to checkfor correctness over 2106

states. This is a number of unfathomable mag-nitude. Chapters 21 through 23 present how automata theory, alongwith the use of finite-state abstractions and symbolic state represen-tations, helps debug concurrent systems through a powerful approachknown as model checking.

How does automata theory help ensure that safety critical systemsare correct? First of all, it helps create abstract models for systems.Many systems are so complex that each vector consists of thousands, ifnot millions, of variables. If one considers going through all of the 21000

assignments for the bits in the vector, the process will last thousandsof millennia. If one runs such tests for several days, even on the fastestavailable computer, and employs a good randomization strategy in vec-tor selection, one would still have covered only some arbitrary sets ofbehaviors of the system. Vast expanses of its state-space would be leftunexamined. A much more e!ective way to approach this problem inpractice is to judiciously leave out most of the bits from vectors, andexamine the system behavior exhaustively over all the remaining be-haviors. (In many cases, designers can decide which bits to leave out;in some cases, computer-based tools can perform this activity.) That is,by leaving out the right set of bits, we end up fully covering an abstractmodel. Experience has repeatedly shown that verifying systems at thelevel of abstract models can often find serious bugs quicker. The simu-lation of non-abstracted models is also a necessary part of verificationin practice.

It turns out that abstracted models of most systems are finite au-tomata. There are a number of techniques being developed that canrepresent, as well as manipulate, very large finite automata. These tech-niques help minimize the degree to which systems have to be abstractedbefore they can be exhaustively verified. This, in turn, means that the

10 1 Introduction

risk of accidentally missing an error due to overly heavy abstractionsis also reduced.

Automaton/Logic Connection

Various branches of mathematical logic are employed to precisely andunambiguously describe both system requirements and system behav-iors. In modern verification systems, automata theoretic techniques areoften employed to process these logical specifications to check whetherthey are true or false. It is therefore important for students to see thesetopics treated in a cohesive manner.

The importance of precision and clarity in system descriptions can-not be overstated. Many system description documents rival Los Ange-les telephone directories in their size, containing a very large number ofsubtle assertions that tax the human mind. Each person reading sucha document comes up with a di!erent understanding. While engineersare incredibly smart and are able to correct their misunderstandingsmore often than not, they still waste their time poring over lengthyprose fragments that are ambiguous, and simply too complex to trustone’s mind with. It has been widely shown that formal statements inmathematical logic can serve as very valuable augments to text-baseddocuments, supplying the missing precision in the text.

One of the most serious deficiencies of an exclusively natural-language system description is that engineers cannot mechanically cal-culate their consequences. In other words, they cannot ask “what if”(putative) queries about scenarios that are not explicitly discussed inthe document. An example from the I/O system world is, “what if I/Otransaction x is allowed to overtake transaction y? Does it cause wronganswers to be returned? Does it cause system deadlocks?” The numberof potentially interesting “what if” questions pertaining to any real sys-tem is extremely large. It is impossible or even counterproductive fora specification document to list all these questions and their answers.7

On the other hand, an approach where one is able to state the speci-fication of complex systems (such as I/O buses) in a precise languagebased on mathematical logic, and is able to pose putative or challengequeries (also expressed in mathematical logic) is highly conducive togaining a proper understanding of complex systems. Ideally, systems ofthis kind must either try to show that the posed conjecture or queryis true, or provide a clear explanation of why it is false. Model check-ing based verification methods (discussed Chapter 21 onwards) provide

7 Doing so would result in not just a telephone directory, but an entire city library!

1 Introduction 11

this ability in most formal verification based design approaches underdevelopment or in actual use today.

The ability to decide - provide an answer, without looping, for allpossible putative queries - is, of course, a luxury enjoyed when we em-ploy simpler mathematical logics. Unfortunately, simpler mathematicallogics are often not as expressive, and we will then have to deduce, us-ing partly manual steps and partly automatic (decidable) steps, theanswer. In Chapter 18, this book provides a formal proof as to whysome of these logics are undecidable.

Avoid Attempting the Impossible

Automata theory and logic often help avoid pursuing the impossible. Ifone can prove that there cannot be a decider for some task, then thereis no point wasting everyone’s time in pursuit of an algorithm for thattask. On the other hand, if one does prove that an algorithm exists,finding an e"cient algorithm becomes a worthwhile pursuit.

As an example, the next time your boss asks you to produce a C-grammar equivalence checker that checks the equivalence between anytwo arbitrary C-grammar files (say, written in the language Yacc orBison) and takes no more than “a second per grammar-file line,” don’twaste your time coding - simply prove that this task is impossible!8

Solving One Implies Solving All

There is another sense in which automata theory helps avoid work.In many of these cases, researchers have found that while we cannotactually solve a given problem, we can gang up or “club together”thousands of problems such that the ability to solve any one of theseproblems gives us the ability to solve all of these problems. Often thesolvability question will be whether it is tractable to solve the problems- i.e., solve in polynomial time. In many cases, we are simply interestedin solvability, without worrying about the amount of time. In thesecases also, repeated work is avoided by grouping a collection of prob-lems into an equivalence class and looking for a solution to only oneof these problems; this solution can be easily modified to solve thou-sands of practical problems. This is the motivation behind studyingNP-completeness and related notions. We will also study the famousHalting problem and problems related to it by taking this approach

8 You may wish to state that the di!erence between Bison and Yacc is that onecannot wash one’s face in a Yacc!

12 1 Introduction

of clubbing problems together through the powerful idea of mappingreductions.

Automata Theory Demands a Lot From You!

The study of automata theory is challenging. It exposes students to a va-riety of mathematical models and helps build confidence in them, thusencouraging them to be creative, to take chances, and to tread newground while designing computing systems. Unfortunately, the notionof formally characterizing designs is not emphasized in traditional sys-tems classes (architecture, operating systems, etc.), where the goal has,historically, been high performance and not high reliability. Therefore,it takes an extra bit of e!ort to put formal specification and verificationinto practice. Fortunately, the computer industry has embraced formalmethods, and sees it as the main hope for managing the complexityand ensuring the reliability of future designs.

It is impossible to learn automata theory “in a hurry.” While thesubject is quite simple and intuitive in hindsight, to get to that stagetakes patience. You must allow enough time for the problems to gestatein your minds. After repeatedly trying and failing, you will be able tocarry the problems in your minds. You may end up solving some ofthem in the shower.9

A Brief Foray Into History

Let us take a historical perspective, to look back into how this subjectwas viewed by two of the originators of this subject - Michael Rabinand Dana Scott - in their 1959 paper (that, incidentally, is cited in theirACM Turing award citation [100]):

Turing machines are widely considered to be the abstract pro-totype of digital computers; workers in the field, however, havefelt more and more that the notion of a Turing machine is toogeneral to serve as an accurate model of actual computers. Itis well known that even for simple calculations, it is impossibleto give a prior upper bound on the amount of tape a Turingmachine will need for any given computation. It is precisely thisfeature that renders Turing’s concept unrealistic.In the last few years, the idea of a finite automaton has appearedin the literature. These are machines having only a finite number

9 You may wish to warn your family members that you may one day bolt out ofthe shower, all soaking wet, shouting “Eureka!”

1 Introduction 13

of internal states that can be used for memory and computation.The restriction of finiteness appears to give a better approxima-tion to the idea of a physical machine. [...]. Many equivalentforms of the idea of finite automata have been published. Oneof the first of these was the definition of “nerve-nets” given byMcCulloch and Pitts. [...]

In short, Rabin and Scott observe that the theory of Turing machines,while all encompassing, is “too heavy-weight” for day-to-day studies ofcomputations. They argue that perhaps finite automata are the rightmodel of most real-world computations. From a historical perspective,the more complex machine form (Turing machine) was proposed muchbefore the much simpler machine form (finite automata). All this isclearly not meant to say that Turing machines are unimportant—farfrom it, in fact! Rather, the message is that a more balanced view ofthe topics studied under the heading of automata will help one betterappreciate how this area came about, and how the priorities will shiftover time.

Disappearing Formal Methods

In the article “disappearing formal methods [104],” John Rushby pointsout how throughout the history of engineering, various new technolo-gies had been [over] advertised, until they became widely accepted andtaken for granted. For instance, many olden-day radios had a digit (‘6’or ‘8’) boldly written on their aluminum cases, advertising the factthat they actually employed 6 or 8 (as the case may be) transistors.10

Centuries ago, di!erential and integral calculus were widely advertisedas the “magic” behind studying planetary motions, as well as buildingeveryday objects, such as bridges. Today, nobody hangs a sign-boardon a bridge proclaiming, “this bridge has been designed using di!eren-tial and integral calculus!” In the same vein, the term formal methodsis nowadays used as a “slogan” to call attention to the fact that weare really using ideas based on logic and automata theory to designproducts, as opposed to previously, when we used no such disciplinedapproach. While the explicit advertisement of the technology behindmodern developments is inevitable, in the long run when such applica-tions are well understood, we would no longer be pointing out explicitly

10 Bob Colwell recently told me the story of a radio he opened in his younger days,only to find that it had two transistors connected to nowhere - apparently servingthe only purpose of jacking up the number advertised outside!

14 1 Introduction

that we are actually designing, say, floating-point units, using higherorder logic.

Said another way, in a future book on automata theory, perhaps onecould dispense with all these motivational remarks that have preventedus from heading towards the core of this book. Luckily, that is preciselywhere we are headed now.

Exercises

1.1. Read the 1972 Turing Award Lecture by Edsger W. Dijkstra en-titled “The Humble Programmer” [36]. This, and a significant numberof other articles referred to in this book, is available through the ACMDigital Library, and often through the Google search engine.

1.2. Read and summarize the article “The Emperor’s Old Clothes” byC. A. R. Hoare [57].

1.3. Read and summarize the article “Computational Thinking” byJeanette M. Wing [123]. Computational thinking represents a univer-sally applicable attitude and skill set everyone, not just computer sci-entists, would be eager to learn and use.

2

Mathematical Preliminaries

In this chapter, we introduce many of the fundamental mathemati-cal ideas used throughout the book. We first discuss sets, which helporganize “things” into meaningful unordered collections. We then dis-cuss functions which help map things to other things. Next, we dis-cuss relations that relate things. We provide a concrete syntax, thatof Lambda expressions, for writing down function definitions. We thenpresent ways to “count” infinite sets through a measure known as car-dinality.

2.1 Numbers

We will refer to various classes of numbers. The set Nat, or natu-ral numbers, refers to whole numbers greater than or equal to zero,i.e., 0, 1, 2, . . .. The set Int, or integers, refers to whole numbers, i.e.,0, 1,/1, 2,/2, 3,/3 . . .. The set Real, or real numbers, refers to bothrational as well as irrational numbers, including 0.123,

12, $, 1, and

/2.

2.2 Boolean Concepts, Propositions, and Quantifiers

We assume that the reader is familiar with basic concepts from Booleanalgebra, such as the use of the Boolean connectives and (+), or (*), andnot (¬). We will be employing the two quantifiers “for all” (") and “forsome” or equivalently “there exists” (#) in many definitions. Here weprovide preliminary discussions about these operators; more details areprovided in Section 5.2.4.

16 2 Mathematical Preliminaries

In a nutshell, the quantifiers " and # are iteration schemes for +, *,and ¬, much like % (summation) and & (product) are iteration schemesfor addition (+) and multiplication (2). The universal quantificationoperator " is used to make an assertion about all the objects in thedomain of discourse. (In mathematical logic, these domains are assumedto be non-empty). Hence,

"x : Nat : P (x)

is equivalent to an infinite conjunction

P (0) + P (1) + P (2) . . .

or equivalently +x#Nat : P (x).

2.3 Sets

A set is a collection of things. For example, A = {1, 2, 3} is a set contain-ing three natural numbers. The order in which we list the contents of aset does not matter. For example, A = {3, 1, 2} is the same set as above.A set cannot have duplicate elements. For example, B = {3, 1, 2, 1} isnot a set.1

A set containing no elements at all is called the empty set, written{}, or equivalently, 0. A set may also consist of a collection of othersets, as in

P = {{}, {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, {1, 2, 3}}.

P has a special status; it contains every subset of set A. P is in factthe powerset of A. We will have more to say about powersets soon.

2.3.1 Defining sets

Sets are specified using the set comprehension notation

S = {x , D | p(x)}.

Here, S includes all x from some universe D such that p(x) is true. p(x)is a Boolean formula called characteristic formula. p by itself is calledthe characteristic predicate. We can leave out D if it is clear from thecontext.

1 An unordered collection with duplicates, such as B, is called a multi-set or bag.

2.3 Sets 17

Examples:

• Set A, described earlier, can be written as

A = {x | x = 1 * x = 2 * x = 3}.

• For any set D,{x , D | true} = D.

Notice from this example that the characteristic formula can simplybe true, or for that matter false.

• For any set D,{x , D | false} = 0.

The next two sections illustrate that care must be exercised in writ-ing set definitions. The brief message is that by writing down a collec-tion of mathematical symbols, one does not necessarily obtain some-thing that is well defined. Sometimes, we end up defining more thanone thing without realizing it (the definitions admit multiple solutions),and in other cases we may end up creating contradictions.

2.3.2 Avoid contradictions

Our first example illustrates the famous Russell’s Paradox. This para-dox stems from allowing expressions such as x , x and x /, x insidecharacteristic formulas. Consider the possibility of defining a set S suchthat x is in S exactly when x is not a member of x. If S were a set, we cannow ask, “is S a member of S?”

• If S is a member of S, it cannot be in S, because S cannot containsets that contain themselves.

• However, if S is not a member of S, then S must contain S!

Contradictions are required to be complete, i.e., apply to all possible cases.For example, if S /, S does not result in a contradiction, that, then, be-comes a consistent solution. In this example, we fortunately obtain a con-tradiction in all the cases. The proposition S , S must produce a definiteanswer - true or false. However, both answers lead to a contradiction. Wecan better understand this contradiction as follows. For Boolean quan-tities a and b, let a 3 b stand for “a implies b” or “if a then b;” in otherwords, 3 is the implication operator. Suppose S , S. This allows usto conclude that S /, S. In other words, (S , S) 3 (S /, S) is true. Inother words, ¬(S , S) * (S /, S), or (S /, S) * (S /, S), or (S /, S) istrue. Likewise, (S , S) 3 (S /, S). This allows us to prove (S , S)true. Since we have proved S , S as well as S /, S, we have proved


their conjunction, which is false! With false proved, anything else canbe proved (since false 3 anything is ¬(false) * anything, or true).Therefore, it is essential to avoid contradictions in mathematics.

Russell’s Paradox is used to conclude that a “truly universal set”– a set that contains everything – cannot exist. Here is how such aconclusion is drawn. Consider a set S defined as follows: S

S = {x , D | x /, x}

where D is a completely arbitrary set. If we now assume that S is in D, wederive a contradiction in the same manner as illustrated earlier. Specifically,assuming S , D, S , S will lead to the conclusion S /, S, and vice versa.Now, if a universal set U existed, we could use it in place of D (since D isarbitrary). But then, U can’t be universal after all, because if it is, it willcontain S, leading to the above contradiction!

This is the reason why the notion of a universal set is not an abso-lute notion. Rather, a universal set specific to the domain of discourseis defined each time. This is illustrated below in the section devotedto universal sets. In practice, we disallow sets such as S by banningexpressions of the form x , x. In general, such restrictions are handledusing type theory [48].

2.3.3 Ensuring uniqueness of definitions

When a set is defined, it must be uniquely defined. In other words, wecannot have a definition that does not pin down the exact set beingtalked about. To illustrate this, consider the “definition” of a set

S = {x , D | x , S},

where D is some domain of elements. In this example, the set beingdefined depends on itself. The circularity, in this case, leads to S notbeing uniquely defined. For example, if we select D = Nat, and plugin S = {1, 2} on both sides of the equation, the equation is satisfied.However, it is also satisfied for S = 0, S = {3, 4, 5}. Hence, in the abovecircular definitions, we cannot pin down exactly what S is.

The message here is that one must avoid using purely circular defini-tions. However, sets are allowed to be defined through recursion which,at first glance, is “a sensible way to write down circular definitions.”Chapter 7 explains how recursion is understood, and how sets can beuniquely defined even though “recursion seems like circular definition.”

2.3 Sets 19

Operations on Sets

Sets support the usual operations such as membership, union, intersec-tion, subset, powerset, Cartesian product, and complementation. x , Ameans x is a member of A. The union of two sets A and B, writtenA(B, is a set such that x , (A(B) if and only if x , A or x , B. Inother words, x , (A ( B) implies that x , A or x , B. Also, x , A orx , B implies that x , (A ( B). similarly, the intersection of two setsA and B, written A ' B, is a set such that x , (A ' B) if and only ifx , A and x , B.

A proper subset A of B, written A 4 B, is a subset of B di!erentfrom B. A 5 B, read ‘A is a subset of B’, means that A 4 B or A = B.Note that the empty set has no proper subset.

Subtraction, Universe, Complementation, SymmetricDi!erence

Given two sets A and B, set subtraction, ‘\’, is defined as follows:

A \ B = {x | x , A + x /, B}.

Set subtraction basically removes all the elements in A that are in B.For example, {1, 2} \ {2, 3} is the set {1}. 1 survives set subtractionbecause it is not present in the second set. The fact that 3 is presentin the second set is immaterial, as it is not present in the first set.

For each type of set, there is a set that contains all the elements ofthat type. Such a set is called the universal set.

The reference to ‘universal sets’ here should be taken as ‘the leastset closed under the operations’. For example, when we say “set ofall strings over some alphabet,” we mean the set generated throughthe operation of concatenation of the symbols in the alphabet,starting with the empty string epsilon.

For example, consider the set of all strings over some alphabet, such as{a, b}. This is universal set, as far as sets of strings are concerned. Wecan write this set as

SigmaStar = {", a, b, aa, ab, ba, bb, aaa, aab, . . .}.

(The reason why we name the above set SigmaStar will be explainedin Chapter 7.) Here, " is the empty string, commonly written as "", aand b are strings of length 1, aa, ab, ba, and bb are strings of length2, and so on. While discussing natural numbers, we can regard Nat ={0, 1, 2, . . .} as the universe.


The symbol " is known to confuse many students. Think ofit as the “zero” element of strings, or simply read it as theempty string "". By way of analogy, the analog of the arithmeticexpression 0+1, which simplifies to 1, is " concatenated with a,which simplifies to a. (We express string concatenation throughjuxtaposition). Similarly, 0+2+0 = 2 is to numbers as " aa " =aa is to strings. More discussions are provided in Section 7.2.4.

Universal sets help define the notion of complement of a set. Considerthe universal set (or “universe”) SigmaStar of strings over some alpha-bet. The complement of a set of strings such as {a, ba} isSigmaStar \ {a, ba}. If we now change the alphabet to, say, {a}, theuniversal set of strings over this alphabet is

SigmaStar1 = {", a, aa, aaa, aaaa, . . .}.

Taking the complement of a set such as {a, aaa} with respect toSigmaStar1 yields a set that contains strings of a’s such that the num-ber of occurrences of a’s is neither 1 nor 3.

Given two sets A and B, their symmetric di!erence is defined to be

(A \ B) ( (B \ A).

For example, if A = {1, 2, 3} and B = {2, 3, 4, 5}, their symmetricdi!erence is the set {1, 4, 5}. The symmetric di!erence of two sets pro-duces, in e!ect, the XOR (exclusive-OR) of the sets.

For any alphabet % and its corresponding universal set SigmaStar,the complement of the empty set 0 is SigmaStar. One can think of 0 asthe empty set with respect to every alphabet.

Types versus Sets

The word type will be used to denote a set together with its associatedoperations. For example, the type natural number, or Nat, is associatedwith the set {0, 1, 2, . . .} and operations such as successor, +, etc. 0is an overloaded symbol, denoting the empty set of every type. Whenwe use the word “type,” most commonly we will be referring to theunderlying set, although strictly speaking, types are “sets plus theiroperations.”

2.4 Cartesian Product and Powerset 21

Numbers as Sets

In mathematics, it is customary to regard natural numbers themselvesas sets. Each natural number essentially denotes the set of naturalnumbers below it. For example, 0 is represented by {}, or 0, as thereare no natural numbers below 0. 1 is represented by {0}, or (moregraphically) {{}}, the only natural number below 1. Similarly, 2 is theset {0, 1}, 3 is the set {0, 1, 2}, and so on. This convention comes inquite handy in making formulas more readable, by avoiding usages suchas

"i : 0 & i & N / 1 : ..something..

and replacing them with

"i , N : ..something..

Notice that this convention of viewing sets as natural numbers isexactly similar to how numbers are defined in set theory textbooks,e.g., [50]. We are using this convention simply as a labor-saving devicewhile writing down definitions. We do not have to fear that we aresuddenly allowing sets that contain other sets.

As an interesting diversion, let us turn our attention back to thediscussion on Russell’s Paradox discussed in Section 2.3.2. Let us takeD to be the set of natural numbers. Now, the assertion x /, x evaluatesto true for every x , D. This is because no natural number (viewedas a set) contains itself - it only contains all natural numbers strictlybelow it in value. Hence, no contradiction results, and S ends up beingequal to Nat.

2.4 Cartesian Product and Powerset

The Cartesian product operation ‘2’ helps form sets of tuples of el-ements over various types. The terminology here goes as follows:‘pairs’ are ‘two-tuples,’ ‘triples’ are ‘three-tuples,’ ‘quadruples’ are‘four-tuples,’ and so on. After 5 or so, you are allowed to say ‘n-ple’ -for instance, ‘37-ple’ and so on. For example, the set Int2Int denotesthe sets of pairs of all integers. Mathematically, the former set is

Int 2 Int = {6x, y7 | x , Int + y , Int}.

Given two sets A and B, the Cartesian product of A and B,written A 2 B, is defined to be the set


{6a, b7 | a , A + b , B}.

We can take Cartesian product of multiple sets also. In general, theCartesian product of n sets Ai, i , n results in a set of “n-tuples”

A0 2 A1 2 . . . An!1 = {6a0, a1, . . . , an!17 | ai , Ai for every i}.

If one of these sets, Ai, is 0, the Cartesian product results in 0 becauseit is impossible to “draw” any element out of Ai in forming the n-ples.Here are some examples of Cartesian products:

• {2, 4, 8} 2 {1, 3} 2 {100} ={62, 1, 1007, 62, 3, 1007, 64, 1, 1007, 64, 3, 1007, 68, 1, 1007, 68, 3, 1007}.

• SigmaStar12SigmaStar1 = {6x, y7 | x and y are strings over {a}}.

In taking the Cartesian product of n sets Ai, i , n, it is clear thatif n = 1, we get the set A0 back. For example, if A0 = {1, 2, 3}, the1-ary Cartesian product of A0 is itself. Note that A0 = {1, 2, 3} canalso be written as A0 = {617, 627, 637} because, in classical set theory[50], 1-tuples such as 607 are the same as the item without the tuplesign (in this case 0).

It is quite common to take the Cartesian product of di!erent typesof sets. For example, the set Int 2 Bool denotes the sets of pairs ofintegers and Booleans. An element of the above set is 622, true7, whichis a pair consisting of one integer and one Boolean.

2.4.1 Powersets and characteristic sequences

The powerset of a set S is the set of all its subsets. As is traditional,we write 2S to denote the powerset of S. In symbols,

2S = {x | x 5 S}.

This “exponential” notation suggests that the size of the powerset is2 raised to the size of S. We can argue this to be the case using thenotion of characteristic sequences. Take S = {1, 2, 3} for example. Eachsubset of S is defined by a bit vector of length three. For instance, 000represents 0 (include none of the elements of S), 001 represents {3}, 101represents {1, 3}, and 111 represents S. These “bit vectors” are calledcharacteristic sequences. All characteristic sequences for a set S are ofthe same length, equal to the size of the set, |S|. Hence, the number ofcharacteristic sequences for a finite set S is exponential in |S|.

2.6 The ! Notation 23

2.5 Functions and Signature

A function is a mathematical object that expresses how items called“inputs” can be turned into other items called “outputs.” A functionmaps its domain to its range; and hence, the inputs of a function belongto its domain and the outputs belong to its range. The domain andrange of a function are always assumed to be non-empty. Theexpression “f : TD ! TR” is called the signature of f , denoting that fmaps the domain of type TD to the range of type TR. Writing signaturesdown for functions makes it very clear as to what the function “inputs”and what it “outputs.” Hence, this is a highly recommended practice.As a simple example, + : Int 2 Int ! Int denotes the signature ofinteger addition.

Function signatures must attempt to capture their domains andranges as tightly as possible. Suppose we have a function g that acceptssubsets of {1, 2, 3}, outputs 4 if given {1, 2}, and outputs 5 given any-thing else. How do we write the signature for g? Theoretically speaking,it is correct to write the signature as 2Nat ! Nat; however, in order toprovide maximum insight to the reader, one must write the signatureas

2{1,2,3} ! {4, 5}.

If you are unsure of the exact domain and range, try to get as tight aspossible. Remember, you must help the reader.The image of a function is the set of range points that a functionactually maps onto. For function f : TD ! TR,

image(f) = {y , TR | #x , TD : y = f(x)}.

2.6 The ! Notation

The Lambda calculus was invented by Alonzo Church2 as a formal rep-resentation of computations. Church’s thesis tells us that the lambda-based evaluation machinery, Turing machines, as well as other formalmodels of computation (Post systems, Thue systems, . . .) are all for-mally equivalent. Formal equivalences between these systems have allbeen worked out by the 1950s.

2 When once asked how he chose ! as the delimiter, Church replied, “Eenie meeniemynie mo!”


More immediately for the task at hand, the lambda notation pro-vides a literal syntax for naming functions. First, let us see in the con-text of numbers how we name them. The sequence of numerals (in pro-gramming parlance, the literal) ‘1’ ‘9’ ‘8’ ‘4’ names the number 1984.We do not need to give alternate names, say ‘Fred’, to such numbers!A numeral sequence such as 1984 su"ces. In contrast, during program-ming one ends up giving such alternate names, typically derived fromthe domain of discourse. For example,

function Fred(x) {return 2;}.Using Lambda expressions, one can write such function definitionswithout using alternate names. Specifically, the Lambda expression!x.2 captures the same information as in the above function definition.We think of strings such as !x.2 as a literal (or name) that describesfunctions. In the same vein, using Lambda calculus, one name for thesuccessor function is !x.(x+1), another name is !x.(x+2/ 1), and soon.3 We think of Lambda expressions as irredundant names for functions(irredundant because redundant strings such as ‘Fred’ are not stuck in-side them). We have, in e!ect, “de-Freded” the definition of functionFred. In Chapter 6, we show that this idea of employing irredundantnames works even in the context of recursive functions. While it mayappear that performing such ‘de-Fredings’ on recursive definitions ap-pears nearly impossible, Chapter 6 will introduce a trick to do so usingthe so-called Y operator. Here are the only two rules pertaining to itthat you need to know:

• The Alpha rule, or “the name of the variable does not matter.”For example, !x.(x + 1) is the same as !y.(y + 1). This process ofrenaming variables is called alpha conversion. Plainly spoken, theAlpha rule simply says that, in theory,4 the formal parameters of afunction can be named however one likes.

• The Beta rule, or “here is how to perform a function call.” A func-tion is applied to its argument by writing the function name andthe argument name in juxtaposition. For example, (!x.(x + 1)) 2says “feed” 2 in place of x. The result is obtained by substituting 2for x in the body (x + 1). In this example, 2 + 1, or 3 results. Thisprocess of simplification is called beta reduction.

3 You may be ba"ed that I suddenly use “23” and “+” as if they were Lambdaterms. As advanced books on Lambda calculus show [48], such quantities canalso be encoded as Lambda expressions. Hence, anything that is e!ectively

computable—computable by a machine—can be formally defined using only theLambda calculus.

4 In practice, one chooses mnemonic names.

2.7 Total, Partial, 1-1, and Onto Functions 25

The formal arguments of Lambda expressions associate to the right. Forexample, as an abbreviation, we allow cascaded formal arguments ofthe form (!xy.(x+y)), as opposed to writing it in a fully parenthesizedmanner as in (!x.(!y.(x+y))). In addition, the arguments to a Lambdaexpression associate to the left. Given these conventions, we can nowillustrate the simplification of Lambda expressions. In particular,

(!zy.(!x.(z + x))) 2 3 4

can be simplified as follows (we show the bindings introduced duringreduction explicitly):

= (!zy.(!x.(z + x))) 2 3 4= (using the Beta rule) (!z = 2 y = 3.(!x.(z + x)))4.= (!x.(2 + x))4= (using the Beta rule) (!x = 4.(2 + x))= 2 + 4= 6

The following additional examples shed further light on Lambda cal-culus:

• (!x.x) 2 says apply the identity function to argument 2, yielding 2.• (!x.x) (!x.x) says “feed the identity function to itself.” Before per-

forming beta reductions here, we are well-advised to perform alphaconversions to avoid confusion. Therefore, we turn (!x.x) (!x.x)into (!x.x) (!y.y) and then apply beta reduction to obtain (!y.y),or the identity function back.

• As the Lambda calculus seen so far does not enforce any “typechecking,” one can even feed (!x.(x + 1)) to itself, obtaining (afteran alpha conversion) (!x.(x+1))+1. Usually such evaluations thenget “stuck,” as we cannot add a number to a function.

2.7 Total, Partial, 1-1, and Onto Functions

Functions that are defined over their entire domain are total. An ex-ample of a total function is !x.2x, where x , Nat. A partial functionis one undefined for some domain points. For example, !x.(2/x) is apartial function, as it is undefined for x = 0.

The most common use of partial functions in computer science is tomodel programs that may go into infinite loops for some of their inputvalues. For example, the recursive program over Nat,

f(x) = if (x = 0) then 1 else f(x)


terminates only for x = 0, and loops for all other values of x. Viewedas a function, it maps the domain point 0 to the range point 1, andis undefined everywhere else on its domain. Hence, function f can benaturally modeled using a partial function. In the Lambda notation,we can write f as !x.if (x = 0) then 1. Notice that we use an “if-then”which leaves the “else” case undefined.

One-to-one (1-1) functions f are those for which every point y ,image(f) is associated with exactly one point x in TD. A functionthat is not 1-1 is ‘many-to-one.’ One-to-one functions are also knownas injections. An example of an injection is the predecessor functionpred : Nat ! Nat, defined as follows:

!x.if (x > 0) then (x / 1).

We have pred(1) = 0, pred(2) = 1, and so on. This function is partialbecause it is undefined for 0.

Onto functions f are those for which image(f) = TR. Onto functionsare also known as surjections. While talking about the type of the range,we say function f maps into its range-type. Hence, onto is a specialcase of into when the entire range is covered. One-to-one, onto, andtotal functions are known as bijections. Bijections are also knownas correspondences.Examples: We now provide some examples of various types of func-tions. In all these discussions, assume that f : Nat ! Nat.

• An example of a one-to-one (1-1) function (“injection”) is f =!x.2x. (Note: Into functions are 1-1, and not necessarily onto. )

• An example of a many-to-one function is f = !x.(x mod 4).• An example of an onto function is f = !x.x.• An example of a partial function is f = !x.if even(x) then (x/2).• An example of a bijection is f = !x.if even(x) then (x+1) else (x/

1). All bijections f : TD ! TR where TD = TR = T are the sametype, are permutations over T .

• Into means not necessarily onto. A special case of into is onto.• Partial means not necessarily total. A special case of partial is total.

For any given one-to-one function f , we can define its inverse tobe f!1. This function f!1 is defined at all its image points. Therefore,whenever f is defined at x,

f!1(f(x)) = x.

For f : TD ! TR, we have f!1 : TR ! TD. Consequently, if f isonto, then f!1 is total—defined everywhere over TR. To illustrate this,

2.8 Computable Functions 27

consider the predecessor function, pred. The image of this function isNat. Hence, pred is onto. Hence, while pred : Nat ! Nat is not total,pred!1 : Nat ! Nat is total, and turns out to be the successor functionsucc.

Given a bijection f with signature TD ! TR, for any x , TD,f!1(f(x)) = x, and for any y , TR, f(f!1(y)) = y. This shows that iff is a bijection from TD to TR, f!1 is a bijection from TR to TD. Forthis reason, we tend to call f a bijection between TD and TR - giventhe forward mapping f , the existence of the backward mapping f!1 isimmediately guaranteed.

Composition of functions

The composition of two functions f : A ! B and g : B ! C, writteng ) f , is the function !x . g(f(x)).

2.8 Computable Functions

Computational processes map their inputs to their outputs, and there-fore are naturally modeled using functions. For instance, given twomatrices, a computational process for matrix multiplication yields theproduct matrix. All those functions whose mappings may be obtainedthrough a mechanical process are called computable functions, e!ec-tively computable functions, or algorithmically computable functions.For practical purposes, another equivalent definition of a computablefunction is one whose definition can be expressed in a general-purposeprogramming language. By ‘mechanical process,’ we mean a sequenceof elementary steps, such as bit manipulations, that can be carried outon a machine. Such a process must be finitary, in the sense that forany input for which the function is defined, the computational processproducing the mapping must be able to read the input in finite timeand yield the output in a finite amount of time. Chapter 3 discusses thenotion of a ‘machine’ a bit more in detail; for now, think of computerswhen we refer to a machine.

Non-computable functions are well-defined mathematical concepts.These are genuine mathematical functions, albeit those whose map-pings cannot be obtained using a machine. In Section 3.1, based oncardinality arguments, we shall show that non-computable functionsdo exist. We hope that the intuitions we have provided above will al-low you to answer the following problems intuitively. The main pointwe are making in this section is that just because a function “makes


sense” mathematically doesn’t necessarily mean that we can code it upas a computer program!

2.9 Algorithm versus Procedure

An algorithm is an e!ective procedure, where the word ‘e!ective’ means‘can be broken down into elementary steps that can be carried outon a computer.’ The term algorithm is reserved to those proceduresthat come with a guarantee of termination on every input. If such aguarantee is not provided, we must not use the word ‘algorithm,’ butinstead use the word procedure. While this is a simple criterion onecan often apply, sometimes it may not be possible to tell whether tocall something an algorithm or a procedure. Consider the celebrated“3x + 1 problem,” also known as “Collatz’s problem,” captured by thefollowing program:

function three_x_plus_one(x){ if (x==1) then return 1;if even(x) then three_x_plus_one(x/2);else three_x_plus_one(3x+1); }

For example, given 3, the three_x_plus_one function obtains 10, 5,16, 8, 4, 2, 1, and halts. Will this function halt for all x? Nobodyknows! It is still open whether this function will halt for all x in Nat[24]! Consequently, if someone were to claim that the above program istheir actual implementation of an algorithm (not merely a procedure) torealize the constant function !x.1, not even the best mathematicians orcomputer scientists living today would know how to either confirm orto refute the claim! That is, nobody today is able to prove or disprovethat the above program will halt for all x, yielding 1 as the answer.5

2.10 Relations

Let S be a set of k-tuples. Then a k-ary relation R over S is defined to bea subset of S. It is also quite common to assume that the word ‘relation’means ‘binary relation’ (k = 2); we will not follow this convention,and shall be explicit about the arity of relations. For example, givenS = Nat 2 Nat, we define the binary relation < over S to be

< = {6x, y7 | x, y , Nat and x < y}.

5 Our inability to deal with “even a three-line program” perhaps best illustratesDijkstra’s advice on the need to be humble programmers [36].

2.10 Relations 29

It is common to overload symbols such as <, which can be used to de-note binary relations over Int (the set of positive and negative numbers)or Real. For the sake of uniformity, we permit the arity of a relationto be 1. Such relations are called unary relations, or properties. Forexample, odd can be viewed as a unary relation

odd = {x | x , Nat and x is odd},

or, equivalently,

odd = {x | x , Nat and #y , Nat : x = 2y + 1}.

Much like we defined binary relations, we can define ternary rela-tions (or 3-ary relations), 4-ary relations, etc. An example of a 3-aryrelation over Nat is between, defined as follows:

between = {6x, y, z7 | x, y, z , Nat + (x & y) + (y & z)}.

Given a binary relation R over set S, define the domain of R to be

domain(R) = {x | #y : 6x, y7 , R},

and the co-domain of R to be

codomain(R) = {y | #x : 6x, y7 , R}.

Also, the inverse of R, written R!1 is

R!1 = {6y, x7 | 6x, y7 , R}.

As an example, the inverse of the ‘less than’ relation, ‘<,’ is the greaterthan relation, namely ‘>.’ Similarly, the inverse of ‘>’ is ‘<.’ Pleasenote that the notion of inverse is defined only for binary relations - andnot for ternary relations, for instance. Also, inverse is di!erent fromcomplement. The complement of < is . and the complement of ‘>’ is‘&,’ where the complementations are being done with respect to theuniverse Nat 2 Nat.

For a binary relation R, let elements(R) = domain(R)(codomain(R).The restriction of R on a subset X 5 elements(R) is written

R |X= {6x, y7 | 6x, y7 5 R + x, y , X}.

Restriction can be used to specialize a relation to a “narrower” domain.For instance, consider the binary relation < defined over Real. Therestriction < |Nat restricts the relation to natural numbers.

Putting these ideas together, the symmetric di!erence of ‘<’ and‘>’ is the ‘%=’ (not equal-to) relation. You will learn a great dealby proving this fact, so please try it!


2.11 Functions as Relations

Mathematicians seek conceptual economy. In the context of functionsand relations, it is possible to express all functions as relations; hence,mathematicians often view functions as special cases of relations. Letus see how they do this.

For k > 0, a k-ary relation R 5 T 1D2T 2

D2. . .2T kD is said to be single-

valued if for any 6x1, . . . , xk!17 , T 1D 2T 2

D2 . . .2T k!1D , there is at most

one xk such that 6x1, . . . , xk!1, xk7 , R. Any single-valued relation Rcan be viewed as a k/1-ary function with domain T 1

D2T 2D2 . . .2T k!1

Dand range T k

D. We also call single-valued relations functional relations.As an example, the ternary relation

{6x, y, z7 | x, y, z , Nat + (x + y = z)}

is a functional relation. However, the ternary relation between definedearlier is not a functional relation.

How do partial and total functions “show up” in the world of re-lations? Consider a k / 1-ary function f . If a xk exists for any input6x1, . . . , xk!17 , T 1

D 2T 2D 2 . . .2T k!1

D , the function is total; otherwise,the function is partial.

To summarize, given a single-valued k-ary relation R, R can beviewed as a function fR such that the “inputs” of this function arethe first k / 1 components of the relation and the output is the lastcomponent. Also, given a k-ary function f , the k + 1-ary single-valuedrelation corresponding to it is denoted Rf .

2.11.1 More ! syntax

There are two di!erent ways of expressing two-ary functions in theLambda calculus. One is to assume that 2-ary functions take a pairof arguments and return a result. The other is to assume that 2-aryfunctions are 1-ary functions that take an argument and return a result,where the result is another 1-ary function.6 To illustrate these ideas,let us define function RMS which stands for root mean squared in boththese styles, calling them rms1 and rms2 respectively:

rms1 : !6x, y7.!

x2 + y2

rms2 : !x.!y.!

x2 + y2

6 The latter style is known as the Curried form, in honor of Haskell B. Curry. Itwas also a notation proposed by Schoenfinkel; perhaps one could have named itthe ‘Schoenfinkeled’ form, as well.

2.11 Functions as Relations 31

Now, rms162, 47 would yield1

20. On the other hand, we apply rms2

to its arguments in succession. First, rms2(2) yields !y.!

22 + y2, i.e.,

!y.!

4 + y2, and this function when applied to 4 yields1

20. Usually, weuse parentheses instead of angle brackets, as in programming languages;for example, we write rms1(2, 4) and !(x, y).

!

x2 + y2.The above notations can help us write characteristic predicates quite

conveniently. The characteristic predicate

(!(z, y).(odd(z) + (4 & z & 7) + ¬y))

denotes (or, ‘defines’) the relation = {65, false7, 67, false7}. This isdi!erent from the characteristic predicate

(!(x, z, y).(odd(z) + (4 & z & 7) + ¬y))

which, for x of type Bool, represents the relation

R! 5 Bool 2 Nat 2 Bool

equal to

{6false, 5, false7, 6true, 5, false7, 6false, 7, false7}, 6true, 7, false7}.

Variable x is not used (it is a “don’t care”) in this formula.

Chapter Summary

This chapter provided a quick tour through sets, numbers, functions,relations, and the lambda notation. The following exercises are designedto give you su"cient practice with these notions.

Exercises

2.1. Given the characteristic predicate p = !x. (x > 0 + x < 10),describe the unary relation defined by p as a set of natural numbers.

2.2. Given the characteristic formula f = (x > 0 + x < 10), describethe unary relation defined by f as a set of natural numbers.

2.3. Given the characteristic predicate

r = !(x, y, z). (x 5 y + y 5 z + x 5 {1, 2} + y 5 {1, 2} + z 5 {1, 2})

write out the relation described by r as a set of triples.

2.4. Repeat Exercise 2.3 with the conjunct x 5 y removed.


2.5. What is the set defined by P = {x , Nat | 55 < 44}?

2.6. The powerset, P , introduced earlier can also be written as

P = {x | x 5 {1, 2, 3}}

What set is defined by replacing 5 by 4 above?

2.7.1. What is the set described by the expression

{1, 2, 3} ' {1, 2} ( {2, 4, 5}.

Here, ' has higher precedence than (.2. What is the symmetric di!erence between {1, 2, 3, 9} and {2, 4, 5,/1}?3. How many elements are there in the following set:

{0} ( 0( {{2}, 0}? 0 denotes the empty set. It is assumed that setsmay contain other sets.

2.8. Formally define the set S of divisors of 64. Either show the setexplicitly or define it using comprehension.

2.9. Formally define the set S of divisors of 67,108,864. Either showthe set explicitly (!) or define it using comprehension.

2.10. What is the set defined by {x | x . 0 + prime(x) + x & 10}?

2.11. What is the set defined by

{x | x , 13 + composite(x) + x . 1}?

A composite number is one that is not prime.

2.12. What is the set defined by

{x | x , 25 + square(x)}

square(x) means x is the square of a natural number.

2.13. What is the set S = {x | x 4 Nat + 23 = 24}?

2.14. Take S = Nat, which contains an infinite number of elements.How many elements are there in the powerset of S? Clearly it alsocontains an infinite number of elements; but is it the “same kind ofinfinity?” Think for five minutes and write down your thoughts in aboutfour sentences (we shall revisit this issue in Chapter 3).


2.15. The set Odd of odd numbers is a proper subset of Nat. It is truethat Odd “appears to be smaller” than Nat - yet, both sets containan infinite number of elements. How can this be? Is the ‘infinity’ thatmeasures the size of Odd a ‘smaller infinity’ than that which measuresthe size of Nat? Again, express your thoughts in about four sentences.

2.16. Let E be the set of Even natural numbers. Express the set E22E

using set comprehension.

2.17. An undirected graph G is a pair 6V,E7, where V is the set ofvertices and E is the set of edges. For example, a triangular graph overV = {0, 1, 2} is

6{0, 1, 2}, {60, 17, 61, 27, 60, 27}7.We follow the convention of not listing symmetric variants of edges -such as 61, 07 for 60, 17.

Now, this question is about cliques. A triangle is a 3-clique. A cliqueis a graph where every pair of nodes has an edge between them. Weshowed you, above, how to present a 3-clique using set-theoretic nota-tion.

Present the following n-cliques over the nodes i , n in the sameset-theoretic notation. Also draw a picture of each resulting graph:

1. 1-clique, or a point.2. 2-clique, or a straight-line.3. 4-clique.4. 5-clique.

2.18. Write a function signature for the sin and tan functions thataccept inputs in degrees.

2.19. Decipher the signature given below by writing down four distinctmembers of the domain and the same number from the range of thisfunction. Here, X stands for “don’t care,” which we add to Bool. Foryour examples, choose as wide a variety of domain and range elementsas possible to reveal your detailed understanding of the signature:

(Int ( {/1}) 2 2Int 2 Bool ! 22Bool"{X} 2 Int.

2.20. Write a function signature for the function 1/(1/x) for x , Nat.

2.21. Express the successor function over Nat using the Lambda no-tation.

2.22. Express the function that sums 1 through N using the Lambdanotation.


2.23. Simplify (!zy.(!x.(z + ((!v.(v + x))5)))) 2 3 4

2.24. A half-wave rectifier receives a waveform at its input and pro-duces output voltage as follows. When fed a positive voltage on theinput, it does not conduct below 0.7 volts (e!ectively producing 0volts). When fed a positive voltage above 0.7 volts, it conducts, butdiminishes the output by 0.7 volts. When fed a negative voltage, itproduces 0 volts, except when fed a voltage below -100 volts, when itblows up in a cloud of smoke (causing the output to be undefined).View the functionality of this rectifier as a function that maps inputvoltages to output voltages. Describe this function using the Lambdanotation. You can assume that ifthenelse and numbers are primitivesin Lambda calculus.

2.25. Provide one example of a bijection from Nat to Int.

2.26. Point out which of the following functions can exist and whichcannot. Provide reasons for functions that cannot exist, and examplesfor functions that can exist.

1. A bijection from 0 to 0.2. A bijection from {"} to {0}.3. A partial 1-1 and onto function from Nat to Nat.4. A partial 1-1 and onto function from Int to Nat.5. A 1-1 into function from Nat to Nat.6. A 1-1 into, but not onto, function from Nat to Nat.7. A bijection from Int to Real.8. A bijection from a set to its powerset. (Recall that we cannot have

0 as either the domain or range of a function.)9. A many-to-one function from the powerset of a set to the set.

10. An into map from a set to its powerset.

2.27. Describe a bijection from the set {"} to the set {0}. Here, " isthe empty string.

2.28. Think about the following question, writing your thoughts in afew sentences, in case you cannot definitely answer the question (thesewill be addressed in Chapter 3).

Can there be a bijection between Int and 2Int?How about a finite subset, F , of Int, and 2F ?How about an infinite subset, I, of Int, and 2I?Which other kinds of functions than bijections may exist?


2.29. Do you see any problems calling a function g “computable” if gwere to accept a subset of Nat, output 4 if given {1, 2}, and output 5given any other subset of Nat? How about a variant of this problemwith “any other subset” replaced by “any other proper subset?”

2.30. Which of the following functions are computable?:

1. A function that inverts every bit of an infinite string of bits.2. A function that inverts every bit of a finite (but arbitrarily long)

string of bits.3. A function that outputs a 1 when given $, and 0 when given

any other Real number. (Recall that $ is not 22/7, 3.14, or even3.1415926. In fact, $ is not a Rational number.)

2.31. Does there exist a procedure that, given a C program P and itsinput x, answers whether P halts on x? Does there exist an algorithmfor this purpose?

2.32. Can there be an algorithm that, given two C programs, checksthat they have identical functionality (over all their inputs)?

2.33. Can there be an algorithm that, given two Yacc grammar files(capturing context-free grammar productions) checks whether the gram-mars encode the same language or not? (Yacc is a tool to generateparsers). Write your ‘best guess’ answer for now; this problem will beformally addressed in Chapter 17.

2.34. What is the symmetric di!erence between ‘&’ and ‘.’? Howabout the symmetric di!erence between ‘<’ and ‘&’?

2.35. Consider the binary relation relprime over Nat 2 Nat such thatrelprime(x, y) exactly when x and y are relatively prime (the greatestcommon divisor of x and y is 1). Is relprime a functional relation?What is its inverse? What is its complement?

2.36. Consider the 3-ary relation “unequal3,” which consists of triples6a, b, c7 such that a %= b, b %= c, and a %= c. Is this relation a functionalrelation? Provide reasons.

2.37. How many functions with signature Boolk ! Bool exist as afunction of k? Here Boolk is Bool2Bool2 . . .2Bool (k times). Thinkcarefully about all possible distinct functions that can have this sig-nature. Each k-ary Boolean function can be presented using a k + 1-column truth table with each row corresponding to one input and itscorresponding output.


2.38. Repeat Exercise 2.37 for partial functions with signature Boolk

! Bool. View each partial function as a table with the output fieldbeing either a Boolean or the special symbol 8, standing for undefined.

4

Binary Relations

Binary relations help impart structure to sets of related elements. Theyhelp form various meaningful orders as well as equivalences, and henceare central to mathematical reasoning. Our definitions in this chapterfollow [50, 77, 63] closely.

4.1 Binary Relation Basics

A binary relation R on S is a subset of S 2 S. It is a relation that canbe expressed by a 2-place predicate. Examples: (i) x loves y, (ii) x > y.

Set S is the domain of the relation. Theoretically, it is possible thatthe domain S is empty (in which case R will be empty). In all instancesthat we consider, the domain S will be non-empty. However, it is quitepossible that S is non-empty and R is empty.

We now proceed to examine various types of binary relations. In allthese definitions, we assume that the binary relation R in question ison S, i.e., a subset of S2S. For a relation R, two standard prefixes areemployed: irr- and non-. Their usages will be clarified in the sequel.

Relations can be depicted as graphs. Here are conventions attributedto Andrew Hodges in [63]. The domain is represented by a closed curve(e.g., circle, square, etc) and the individuals in the domain by dotslabeled, perhaps, a, b, c, and so on. The fact that 6a, b7 , R will bedepicted by drawing a single arrow (or equivalently one-way arrow)from dot a to dot b. We represent the fact that both 6a, b7 , R and6b, a7 , R by drawing a double arrow between a and b. We representthe fact that 6a, a7 , R by drawing a double arrow from a back to itself(this is called a loop). We shall present examples of these drawings inthe sequel.

54 4 Binary Relations

4.1.1 Types of binary relations

R1

R4 R5 R6

1

2 3

1

2 3

1

2 3

R2

1

2 3

1

2 3

R3

1

2 3

Fig. 4.1. Some example binary relations

We shall use the following examples. Let S = {1, 2, 3}, R1 = {6x, x7 |x , S}, R2 = S 2 S, and

R3 = {61, 17, 62, 27, 63, 37, 61, 27, 62, 17, 62, 37, 63, 27}.

All these (and three more) relations are depicted in Figure 4.1.

Reflexive, and Related Notions

R is reflexive, if for all x , S, 6x, x7 , R. Equivalently,

In R’s graph, there is no dot without a loop.

Informally, “every element is related to itself.”

A relation R is irreflexive if there are no reflexive elements; i.e., forno x , S is it the case that 6x, x7 , R. Equivalently,

In R’s graph, no dot has a loop.

4.1 Binary Relation Basics 55

Note that irreflexive is not the negation (complement) of reflexive. Thisis because the logical negation of the definition of reflexive would be,“there exists x , S such that 6x, x7 /, R. This is not the same as ir-reflexive because all such pairs must be absent in an irreflexive relation.

A relation R is non-reflexive if it is neither reflexive nor irreflexive.Equivalently,

In R’s graph, at least one dot has a loop and at least one dotdoes not.

Examples:

R1, R2, R3 are all reflexive.R = 0 is reflexive and irreflexive. It is not non-reflexive.For x, y , Nat, x = y2 is non-reflexive (true for x = y = 1, false forx = y = 2).

Symmetric, and Related Notions

R is symmetric if for all x, y , S, 6x, y7 , R 3 6y, x7 , R. Here, xand y need not be distinct. Equivalently,

In R’s graph, there are no single arrows. If the relation holdsone way, it also holds the other way.

Examples: R1, R2, and R3 are symmetric relations. Also note that 0 isa symmetric relation.

R is asymmetric if there exists no two x, y , S, not necessarily distinct,such that 6x, y7 , R + 6y, x7 , R. In other words, if 6x, y7 , R, then6y, x7 /, R. Example: “elder brother” is an asymmetric relation, and sois < over Nat. Equivalently,

There are no double arrows in its graph; if the relation holdsone way, it does not hold the other.

Again, note that asymmetric is not the same as the negation of (thedefinition of) symmetric. The negation of the definition of symmetricwould be that there exists distinct x and y such that 6x, y7 , R, but6y, x7 /, R.R is non-symmetric if it is neither symmetric nor asymmetric (there isat least one single arrow and at least one double arrow).Example: 0 is symmetric and asymmetric, but not non-symmetric.

R is antisymmetric if for all x, y , S, 6x, y7 , R + 6y, x7 , R 3x = y (they are the same element). Equivalently,


There is no double arrow unless it is a loop.

Antisymmetry is a powerful notion that, unfortunately, is too strongfor many purposes. Consider the elements of 2S , the powerset of S, asan example. If, for any two elements x and y in S, we have x 5 y andy 5 x, then we can conclude that x = y. Therefore, the set containmentrelation 5 is antisymmetric; and hence, antisymmetry is appropriatefor comparing two sets in the “less than or equals” sense.

Consider, on the other hand, two basketball players, A and B.Suppose the coach of their team defines the relation 9BB as follows:A 9BB B if and only if B has more abilities or has the same abili-ties as A. Now, if we have two players x and y such that x 9BB yand y 9BB x, we can conclude that they have identical abilities - theydon’t end up becoming the very same person, however! Hence, 9BB

must not be antisymmetric. Therefore, depending on what we are com-paring, antisymmetry may or may not be appropriate.

Transitive, and Related Notions

To define transitivity in terms of graphs, we need the notions of abroken journey and a short cut. There is a broken journey from dot xto dot z via dot y, if there is an arrow from x to y and an arrow fromy to z. Note that dot x might be the same as dot y, and dot y mightbe the same as dot z. Therefore if 6a, a7 , R and 6a, b7 , R, there is abroken journey from a to b via a. Example: there is a broken journeyfrom Utah to Nevada via Arizona. There is also a broken journey fromUtah to Nevada via Utah.

There is a short cut just if there is an arrow direct from x to z. So if6a, b7 , R and 6b, c7 , R and also 6a, c7 , R, we have a broken journeyfrom a to c via b, together with a short cut. Also if 6a, a7 , R and6a, b7 , R, there is a broken journey from a to b via a, together with ashort cut.Example: There is a broken journey from Utah to Nevada via Arizona,and a short cut from Utah to Nevada.

R is transitive if for all x, y, z , S, 6x, y7 , R + 6y, z7 , R 36x, z7 , R. Equivalently,

There is no broken journey without a short cut.

R is intransitive if, for all x, y, z , S, 6x, y7 , R + 6y, z7 , R 36x, z7 /, R. Equivalently,

There is no broken journey with a short cut.

4.1 Binary Relation Basics 57

R is non-transitive if and only if it is neither transitive nor intransi-tive. Equivalently,

There is at least one broken journey with a short cut and atleast one with.

Examples:

Relations R1 and R2 above are transitive.R3 is non-transitive, since it is lacking the pair 61, 37.Another non-transitive relation is %= over Nat, because from a %= band b %= c, we cannot always conclude that a %= c.R4 is irreflexive, transitive, and asymmetric.R5 is still irreflexive. It is not transitive, as there is no loop at 1. Itis not intransitive because there is a broken journey (2 to 3 via 1)with a short cut (2 to 1). It is non-transitive because there is onebroken journey without a short cut and one without.R5 is not symmetric because there are single arrows.R5 is not asymmetric because there are double arrows.From the above, it follows that R5 is non-symmetric.R5 is not antisymmetric because there is a double arrow that is nota loop.

4.1.2 Preorder (reflexive plus transitive)

If R is reflexive and transitive, then it is known as a preorder. Contin-uing with the example of basketball players, let the 9BB relation forthree members A, B, and C of the team be

{6A,A7, 6A,B7, 6B,A7, 6B,B7, 6A,C7, 6B,C7, 6C,C7}.

This relation is a preorder because it is reflexive and transitive. It helpscompare three players A, B, and C, treating A and B to be equivalentin abilities, and C to be superior in abilities to both.In Section 4.3, we present a more elaborate example of a preorder.

4.1.3 Partial order (preorder plus antisymmetric)

If R is reflexive, antisymmetric, and transitive, then it is known as apartial order. As shown in Section 4.1.1 under the heading of antisym-metry, the subset or equals relation 5 is a partial order.


4.1.4 Total order, and related notions

A total order is a special case of a partial order. R is a total order if forall x, y , S, either 6x, y7 , R or 6y, x7 , R. Here, x and y need not bedistinct (this is consistent with the fact that total orders are reflexive).

The & relation on Nat is a total order. Note that ‘<’ is not a totalorder, because it is not reflexive.1 However, ‘<’ is transitive. Curiously,‘<’ is antisymmetric.A relation R is said to be total if for all x , S, there exists y , S suchthat 6x, y7 , R. In other words, a “total” relation is one in which everyelement x is related to at least one other element y. If we consider yto be the image (mapping) of x under R, this definition is akin to thedefinition of a total function.

Note again that R being a total order is not the same as R being apartial order and a total relation. For example, consider the followingrelation R over set S = {a, b, c, d}:

R = {6a, a7, 6b, b7, 6c, c7, 6d, d7, 6a, b7, 6c, d7}

R is a partial order. R is also a total relation. However, R is not a totalorder, because there is no relationship between b and c (neither 6b, c7nor 6c, b7 is in R).

4.2 Equivalence (Preorder plus Symmetry)

An equivalence relation is reflexive, symmetric, and transitive. Considerthe 9BB relation for three basketball players A, B, and C. Now, con-sider a “specialization” of this relation obtained by leaving out certainedges:

:BB = {6A,A7, 6A,B7, 6B,A7, 6B,B7, 6C,C7}.

This relation is an equivalence relation, as can be easily verified.Note that :BB = 9BB ' 9!1

BB . In other words, this equivalencerelation is obtained by taking the preorder 9BB and intersecting itwith its inverse. The fact that 9BB ' 9!1

BB is an equivalence relationis not an accident. The following section demonstrates a general resultin this regard.

1 Some authors are known to abuse these definitions, and consider < to be a totalorder. It is better referred to as strict total order or irreflexive total order.

4.2 Equivalence (Preorder plus Symmetry) 59

4.2.1 Intersecting a preorder and its inverse

Theorem 4.1. The relation obtained by intersecting a preorder r withits inverse r!1 is an equivalence relation.

Proof: First we show that R'R!1 is reflexive. Let R be a preorder overset S. Therefore, R is reflexive, i.e., it contains 6x, x7 pairs for everyx , S. From the definition of R!1, it also contains these pairs. Hence,the intersection contains these pairs. Therefore, R ' R!1 is reflexive.

Next we show that R ' R!1 is symmetric. That is, to show thatfor every x, y , S, 6x, y7 , R ' R!1 implies 6y, x7 , R ' R!1. If theantecedent, i.e., 6x, y7 , R ' R!1 is false, the assertion is vacuouslytrue. Consider when the antecedent is true for a certain 6x, y7. Thesex and y must be such that 6x, y7 , R and 6x, y7 , R!1. The formerimplies that 6y, x7 , R!1. The latter implies that 6y, x7 , R. Hence,6y, x7 , R ' R!1. Hence, R ' R!1 is symmetric.

Next we prove that R'R!1 is transitive. Since R is a preorder, it istransitive. We now argue that the inverse of any transitive relation istransitive. From the definition of transitivity, for every x, y, z , S, fromthe antecedents 6x, y7 , R and 6y, z7 , R, the consequent 6x, z7 , Rfollows. From these antecedents, we have that 6y, x7 , R!1 and 6z, y7 ,R!1 respectively. From the above conclusion 6x, z7 , R, we can inferthat 6z, x7 , R!1. Hence, R!1 is transitive and so is the conjunction ofR and R!1.

4.2.2 Identity relation

Given a set S, the identity relation R over S is {6x, x7 | x , S}. Anidentity relation is one extreme (special case) of an equivalence relation.This relation is commonly denoted by the equality symbol, =, andrelates equals with equals. Please note the contrast with Theorem 4.1.

4.2.3 Universal relation

The universal relation R over S is S 2 S, and represents the otherextreme of relating everything to everything else. This is often an un-interesting binary relation.2

2 This is sort of what would happen if one were to give everyone in a theory classan ‘A’ grade.


4.2.4 Equivalence class

An equivalence relation R over S partitions elements(R) = domain(R)(codomain(R) into equivalence classes. Intuitively, the equivalence classesEi are those subsets of elements(R) such that every pair of elementsin Ei is related by R, and Eis are the maximal such subsets. Moreformally, given an equivalence relation R, there are two cases:

1. R is a universal relation. In this case, there is a single equivalenceclass E1 associated with R, which is elements(R) itself.

2. R is not a universal equivalence relation. In this case, an equivalenceclass Ei is a maximal proper subset of elements(R) such that therestriction of R on Ei, R |Ei

, is universal (meaning that every pairof elements inside each of the Eis is related by R).

Putting it all together, the set of all equivalence classes of an equiv-alence relation R is written “elements(R)/R.” It can be read, “theelements of R partitioned according to R.” In general, we will writeS/ :, meaning “set S partitioned according to the equivalence relation:.”in Section 4.3.1, we will demonstrate Theorem 4.1 on the Power relationthat relates machine types.

4.2.5 Reflexive and transitive closure

The reflexive closure of R, denoted by R0, is

R0 = R ( {6x, x7 | x , S}.

This results in a relation that is reflexive.The transitive closure of R, denoted by R+, is

R+ = R ( {6x, z7 | #y , S : 6x, y7 , R + 6y, z7 , R+}.

R+ is the least such set. The use of ‘+’ highlights the fact that transitiveclosure relates items that are “one or more steps away.”The reflexive and transitive closure of a relation R, denoted by by R",is

R" = R0 ( R+.

The use of ‘"’ highlights the fact that reflexive and transitive closurerelates items that are “zero or more steps away.”Example: Consider a directed graph G with nodes a, b, c, d, e, and f .Suppose it is necessary to define the reachability relation among the

4.3 The Power Relation between Machines 61

nodes of G. Oftentimes, it is much easier to instead define the one-stepreachability relation

Reach = {6a, b7, 6b, c7, 6c, d7, 6e, f7}

and let the users3 perform the reflexive and transitive closure of Reach.Doing so results in ReachRTclosed, that has all the missing reflexive andtransitive pairs of nodes in it:

ReachRTclosed = {6a, b7, 6b, c7, 6c, d7, 6e, f7, 6a, a7, 6b, b7, 6c, c7, 6d, d7,6e, e7, 6f, f7, 6a, c7, 6a, d7, 6b, d7}.

4.3 The Power Relation between Machines

DFA

NFA NDTM

DTM

DPDA NPDA LBA

Fig. 4.2. The binary relation Power is shown. The dotted edges are someof the edges implied by transitivity. Undotted and dotted means the same inthis diagram. Therefore, Power actually contains: (i) the pairs correspondingto the solid edges, (ii) the pairs indicated by the dotted edges, (iii) and thosepairs indicated by those dotted transitive edges not shown.

An example that nicely demonstrates the versatility of preorders is theone that defines the “power” of computing machines. Let

MT = {dfa, nfa, dpda, npda, lba, dtm, ndtm}

represent the set of machine types studied in this book. These acronymsstand for deterministic finite automata, nondeterministic finite au-tomata, deterministic push-down automata, nondeterministic push-down automata, linear bounded automata, deterministic Turing ma-chines, and nondeterministic Turing machines, respectively. A binary

3 “Why sweat? Let the end users do all the work.”


relation called Power that situates various machine types into a domi-nance relation is shown in Figure 4.2. Each ordered pair in the relationshows up as an arrow (!). We draw an arrow from machine type m1

to m2 if for every task that a machine of type m1 can perform, we canfind a machine of type m2 to do the same task. Spelled out as a set,the relation Power is

Power={6dfa, dfa7, 6nfa, nfa7,6dpda, dpda7, 6npda, npda7,6lba, lba7,6dtm, dtm7, 6ndtm,ndtm7,

6dfa, nfa7, 6nfa, dfa7, 6dtm, ndtm7,6ndtm, dtm7, 6dfa, dpda7, 6nfa, dpda7,6dfa, lba7, 6nfa, lba7,6dpda, npda7, 6npda, dtm7, 6npda, ndtm7,6dpda, lba7, 6npda, lba7,6lba, dtm7, 6lba, ndtm7,

6dfa, npda7, 6nfa, npda7,6dpda, dtm7, 6dpda, ndtm7,

6dfa, dtm7, 6dfa, ndtm7,6nfa, dtm7, 6nfa, ndtm7

}.

We will now study this dominance relation step by step.Any machine is as powerful as itself; hence, Power is reflexive, as

shown by the ‘self-loops.’ Power is transitive relation because for everym1,m2,m3 , MT , if 6m1,m27 ,Power and 6m2,m37 ,Power, thencertainly 6m1,m37 ,Power. (We do not show the transitive edges in thedrawing). Power is not antisymmetric because even though dfa and nfadominate each other, they have distinct existence in the set of machinetypes MT . Power is a preorder, and in our minds captures exactly howthe space of machines must be subdivided. It is not a partial order.

4.3.1 The equivalence relation over machine types

Applying Theorem 4.1 to Power, we obtain the equivalence relation:MT over machine types:

Power ' Power!1 ={6dfa, dfa7, 6nfa, nfa7,6dpda, dpda7, 6npda, npda7,

4.3 The Power Relation between Machines 63

DFA

NFA NDTM

DTM

DPDA NPDA LBA

Fig. 4.3. The equivalence relation Power! Power!1

6dtm, dtm7, 6ndtm,ndtm7,6lba, lba7,

6dfa, nfa7, 6nfa, dfa7, 6dtm, ndtm7, 6ndtm, dtm7}.

Figure 4.3 illustrates :MT . We can see that :MT subdivides MTinto five equivalence classes: {dfa, nfa}, {dpda}, {npda}, {lba}, and{dtm, ndtm}. Let us now look at this equivalence relation formally. Aspointed out earlier, an equivalence relation partitions the underlyingset into equivalence classes. The set of equivalence classes is denotedby elements(R)/R below. The mathematical definition below elaborateson equivalence relations and equivalence classes:

elements(R)/R = { ' | ' 5 elements(R)+ ' 2 ' 5 R+¬#Y : ' 4 Y + Y 5 elements(R)

+ Y 2 Y 5 R }

This definition says the following:

• Each equivalence class Ei is a subset of the elements of the under-lying set

• For each Ei, Ei 2 Ei is a subset of the equivalence relation.• Each such set Ei is maximal (no bigger Y containing each Ei exists)

Coming to our specific example, there are five equivalence classesover Power ' Power!1: {dfa, nfa}, {dpda}, {npda}, {lba}, and {dtm, ndtm}.As can be seen from Figure 4.3, they are five maximal universal re-lations. Given an equivalence relation R, the equivalence class of anelement x , elements(R) is denoted by [x].


4.4 Lattice of All Binary Relations over S

A lattice (Q,9) is a partially ordered set such that for any two elementsx, y , Q, the greatest lower-bound glb(x, y) and the least upper boundlub(x, y) exist in Q, and are unique. An element x

!is lower than x if

x$ 9 x. An element x!is a lower-bound of x and y if x

!is lower than

x and y. glb(x, y) is then a lower-bound that is below no other lowerbound. Similarly, we can define lub(x, y) to be an upper-bound that isabove no other upper-bound. As an example, the powerset of a set S,2S , forms a lattice under the partial order 5. In this lattice, the glbamong all elements of S is 0, and the lub among all elements is S itself.

As another illustration, the set of all equivalence relations over Sforms a lattice under the normal set inclusion operator. The lub of thislattice is the universal relation S 2 S, and the glb of this lattice is theidentity relation {6x, x7 | x , S}.

4.5 Equality, Equivalence, and Congruence

DFA

NFA NDTM

DTM

DPDA NPDA LBA

Fig. 4.4. The identity relation over MT

By now, we have seen the notion of the identity relation, which iswhat the = symbol really corresponds to. Our use of the term “equality”will, without further qualifications, refer to =. Figure 4.4 illustratesidentity. Our use of the word equivalence, denoted by :, has alreadybeen explained.

4.5.1 Congruence relation

A congruence relation captures the notion of substitutability. For ex-ample, not only are 2 and (3/ 1) equivalent, but in most contexts, onecan use 2 instead of (3 / 1). As discussed in Section 4.1.2, a resistor

4.5 Equality, Equivalence, and Congruence 65

may be substituted with another having the same Ohmic value. Letus take a generic example. Consider something “big”, say B, and let xbe a “small part of it.” In symbols, we denote B as B[x], or “B thatcontains x.” Let x : y be true under some equivalence relation :. If :is a congruence relation (or simply “congruence”), then B[x] : B[y].As an example of big, think of a radio receiver, with x being a 10(resistor inside the receiver, and y being two 5( resistors in series. Mostradio receivers will continue to behave the same if one were to pull outa 10( resistor and replace it by two 5( resistors in series. This abil-ity to replace “equals for equals” in various contexts is really the ideabehind congruence.

A context (or “hole”) inside a “big” expression is elegantly capturedby a lambda expression. For example, !z.z+1 gives the context “.+1.”Now, 2 and 200/ 198 are congruent modulo operator !z.z + 1 because(!z.z + 1)2 = (!z.z + 1)(200 / 198). Such congruence can be lost insystems where addition is implemented in a resource-bounded fashion,causing overflows or loss of precision. Therefore, while arithmetic sub-stitutability is a congruence, in real hardware implementations, thissubstitutability may be lost! In other words, congruence respects theoperators in the underlying context. In the examples we discussed,these operators are addition (+) and resistor parallel composition.

Formally, an equivalence relation R modulo operator

op : 2n elements(R) ! elements(R)

for some n . 0 is called a congruence relation, denoted R/%, if

op(6x1, . . . , xn7) = op(6y1, . . . , yn7), for all i , n, and"i : yi , [xi] (here, yi in the equivalence class of xi).

Illustration 4.5.1 (Binary relations:) Let a relation ; over intervalsof Reals be defined as follows. Let [a, b] for a, b , Real be an interval.For intervals [a, b] and [c, d], [a, b] ; [c, d] i! a & c * d & b. Find outthe exact nature of ;: i.e., is it reflexive, symmetric, antisymmetric,transitive, etc.?Solution: We will show the following. First of all, ; is reflexive. It isnot symmetric. If it were symmetric, then we must have

((a & c) * (d & b)) 3 ((c & a) * (b & d))

This would mean that

((c > a) + (b > d)) 3 ((a > c) + (d > b))


But this is not so; consider (a, b) = (0, 10) and (c, d) = (2, 9).It is not transitive. If it were transitive, it would mean that

((a & c) * (d & b) + (c & e) * (f & d)) 3 ((a & e) * (f & b))

But this is not so; consider (a, b) = (2, 10), (c, d) = (3, 15) and (e, f) =(1, 11).

Illustration 4.5.2 We will need this result in Illustration 4.5.3. Letus prove that there are exactly 22N

Boolean functions over N Booleanvariables. Any N -ary Boolean function can be portrayed as a truthtable (see Table 18.1 if you have never seen a truth table) of 2N rows,giving a value of the function for each of the 2N input combinations. Foreach way of filling these output value columns, we obtain one distinctfunction (e.g., the 2-input nand function in Table 18.1 has outputs 1,

1, 1, and 0). Since there are 22Nways of filling the value column, there

are that many functions of N inputs. For example, there are 16 2-inputBoolean functions.

Illustration 4.5.3 (Equivalence over Boolean formulas): Let V ={x, y, z} be a set of Boolean variables. Let F be the set of all Booleanformulas formed over V using +,*,¬. Let 3 stand for the implicationsymbol. Define the relation R to be the set of all formula pairs 6f1, f27such that f1 3 f2 is true, where f1 and f2 come from F . It is easyto see that R is a preorder. Also, R ' R!1 is an equivalence relationwhere each equivalence class contains formula pairs that are logicallyequivalent. Since there are only 3 Boolean variables used, there are 223

equivalence classes (see Illustration 4.5.2). Each equivalence class is aset with cardinality <0 (e.g., f , ¬¬f , ¬¬¬¬f , etc., are in the equiva-lence class of formula f). Also, the set of equivalence classes of F (eachdenoting a Boolean function as in Illustration 4.5.2) with the operator 3forms a lattice, with f1*f2 being the least upper-bound and f1+f2 beingthe greatest lower-bound of two Boolean functions f1 and f2.

Chapter Summary

This chapter provided a thorough self-contained introduction to binaryrelations. It went through many familiar definitions, such as reflexive,symmetric, transitive, etc. It also introduced the “irr-” and “non-”variants of most relations. For instance, it is clearly shown why anirreflexive relation is not the negation (complement) of a reflexive rela-tion. Preorders—very important in comparing machines with more or


equivalent behaviors—are introduced. It was shown that given a pre-order, one can define an equivalence relation by intersecting it with itsinverse. This was illustrated by taking the “power” of various machineswe are going to study in this book into account. We then introduceduniversal as well as identity relations, defined congruence, and brieflylooked at the “power” of machines.

Exercises

4.1. Argue that 0 is both symmetric and asymmetric.

4.2. Find a, b, and c for which transitivity holds, and those for whichtransitivity does not hold.

4.3. Referring to Figure 4.1,

1. Which relations are preorders?2. Which are partial orders?3. Which are equivalence relations?4. Which are universal equivalence relations?5. Which are identity equivalence relations?

4.4. Characterize these human relationships in as many ways as youcan (ascribe as many attributes as you can think of):

1. x is the spouse of y2. x is a sibling of y3. x is an ancestor of y4. x likes y5. x does not know y

4.5. Explain the notions of a total relation and a total order in termsof the required graph properties (in terms of dots and arrows).

4.6.1. Give two examples (each) of a binary relation and a ternary relation

over Nat (choose practical examples of these relations).2. Draw a directed graph G with nodes being the subsets of set {1, 2},

and with an edge from node ni to node nj if either ni 5 nj or|ni| = |nj|. |S| stands for the size or cardinality of set S. Now, viewthe above graph G as a binary relation RG. Write down the pairsin this relation. A relation is written as

{6a0, b07, 6a1, b17, . . .}

where 6ai, bi7 are the pairs in the relation.


3. Is RG a functional relation? Why, or why not? A functional relationis one for which for one ‘input’, at most one ‘output’ is produced.As far as an n-tuple 6a1, a2, . . . , an7 goes, the last position an isregarded as the ‘output’ and the remaining n / 1 positions areregarded as the ‘input.’ Therefore, for a functional binary relation,if 6ai, bj7 and 6ai, bk7 are in the relation, then bj = bk.

4.7. Consider the Power relation defined in Section 4.3. Depict a sim-plified power relation (“Spower”) that retains only DFA, NFA, DTM,and NDTM. Argue that Spower is a preorder. Compute the equivalencerelation obtained by intersecting Spower and its inverse.

4.8. Consider electrical resistors. Let the set of resistors be definedas follows. If x is a Real number, then the term res(x) is a resistor.If r1 and r2 are in resistor, then series(r1, r2) is a resistor. (Again,series(r1, r2) is a term). Some of the elements in set Resistor are:

• res(1), res(2), res(3), series(res(1), res(2)),• series(res(1), series(res(2), res(3))), and• series(series(res(2), res(3)), res(1)).

In any4 circuit, we must be able to substitute res(3) by series(res(1), res(2)),as series connected resistors add up the resistivity. However, we cannotregard res(3) and series(res(1), res(2)) as identical because they havedistinct representations in the set. To compare two resistors “properly,”we define the relation Resistor leq:

Resistor leq = {6x, y7 | sumR(x) & sumR(y)}

where sumR is defined as follows:

sumR(res(i)) = i,sumR(series(x, y)) = sumR(x) + sumR(y).

Given the above, show that Resistor leq is reflexive and transitive, andhence a preorder. Also show that Resistor leq is not antisymmetric.

4.9. Show that the ‘less than’ relation, ‘<’, defined over Nat, is anti-symmetric.

4.10. Prove that the intersection of a partial order R over set S andits inverse R!1 is the identity relation over S.

4 We are ignoring aspects such as size, weight, tolerance, etc.


4.11. Define the following relation ; over intervals of Reals as follows.Let [a, b] for a, b , Real be an interval. For intervals [a, b] and [c, d],[a, b] ; [c, d] i! a & c + d & b. Now, is ; a preorder? Is it a partialorder? Show that ; ' ;!1 is an identity relation.

4.12. Continuing with the above question, let’s change ; as follows.For intervals [a, b] and [c, d], define

[a, b] ; [c, d] = #x : (a & x & b) + (c & x & d).

Now, is ; a preorder or a partial order? Is ; ' ;!1 an identity equiv-alence relation or simply an equivalence relation?

4.13. Find out which are true and which are false assertions:

1. The intersection of two preorders is a preorder.2. The intersection of a preorder order and its inverse can never be

the identity relation.3. The universal relation is a preorder.4. The empty relation is a preorder.

4.14. Choose all correct answers. The intersection of a partial orderand its inverse is:

1. The universal relation.2. The identity relation.3. An equivalence relation.4. A preorder.

4.15. Choose all correct answers. The intersection of a preorder and itsinverse is:

1. a reflexive relation.2. a congruence relation, modulo any given function f .3. a transitive relation.4. an antisymmetric relation.

4.16. Find a relation that is symmetric, asymmetric, antisymmetric,irreflexive, and transitive (all at the same time).

4.17. List some of the equivalence classes of the equivalence relation; ' ;!1 of Exercise 4.12.

4.18. Compare the equivalence classes defined by ; ' ;!1 as definedby Exercise 4.12 and Exercise 4.11. It is the case that each equivalenceclass of one of these examples contains many equivalence classes of the


other example. Explain this statement. Specifically, consider the inter-val [x, x] for some Real number x. Consider its equivalence class [[x, x]].In which example does this equivalence class contain more members be-sides [x, x]?

4.19. Define the following family of equivalence relations over Nat:equivalence relation Ei, for 2 & i & 4, contains all pairs 6x, y7 such that(x = y) mod i. For example, for i = 2, one equivalence class is Oddand another equivalence class is Even. Sketch the equivalence classesgenerated by i = 2, i = 3, and i = 4. Show the inclusion relationshipsbetween these equivalence classes. Also show the inclusion relationshipsbetween these equivalence classes and the universal and the identityrelations over Nat.

4.20. The two DFAs in Figure 10.11 are language-equivalent determin-istic finite-state automaton (DFA) which will be discussed in Chapter 8.

A string s is in the language of a DFA if there is an s-labeled pathfrom the start state (state labeled by the arrow) to one of the finalstates. Given two DFAs D1 and D2, define D1 &DFA D2 exactly whenfor every string in D1’s language there exists the same string in D2’slanguage.

1. Show that &DFA is a preorder.2. Describe the equivalence relation &DFA ' &!1

DFA.3. Show that the DFAs of Figure 10.11 (say DFA1 and DFA2) are

such that DFA1 &DFA DFA2 and DFA2 &DFA DFA1.

4.21. Electrical engineers can realize a resistor of a certain Ohmic valueby connecting resistors in series, parallel, or a combination of both. Tworesistors of Ohmic value R1 and R2 in series are equivalent to a singleresistor of Ohmic value R1 + R2, while two such resistors in parallelare equivalent to a resistor of R1R2

R1+R2Ohms. This equivalence is a con-

gruence relation modulo the series connection and parallel connectionoperators, as far as resistivity goes.

Some attributes of circuits may not be preserved by this substitu-tion.

• Derive a formula for the tolerance of a parallel connection, andcheck whether tolerance is preserved by parallel composition.For example, if each resistor in a collection has a tolerance in Ohmicvalue of 5% (meaning that the actual Ohmic value may be +

!5%)regardless of its Ohmic value, then do two 10( resistors from thiscollection that are connected in parallel have the same overall tol-erance as a single 5( resistor, also drawn from this collection?


• Is tolerance preserved by any assembly? (In other words, supposewe realize r Ohms through two di!erent circuits c1 and c2 withdi!erent serial/parallel structures; do these circuits have the sametolerance overall?)• Is the planarity of the circuit layout preserved by serial/parallelresistive combinations? How about the wattage rating?• Think of three other attributes that are not preserved.

This exercise drives home the point that not everything of interest maybe preserved in a substitution.

5

Mathematical Logic, Induction, Proofs

In this chapter, we present a sampling of ideas from mathematical logic.We consider various operators such as if and if and only if that arecrucial to the proper understanding of formal statements made whilediscussing automata theory. We then describe methods to write downformal definitions and formal proofs, including inductive definitions,and proofs by contradiction.

5.1 To Prove or Not to Prove!

Writing proofs is tedious work. Therefore, it goes without saying thatthis activity is rewarding only if the proofs that one embarks on suc-ceed in establishing the proof goal. Unfortunately, there are no guar-anteed safe methods that one can follow to ensure success in proving(or disproving) things. Here are some familiar (and some unfamiliar)situations in slanted fonts, and the safety net that the human societyrelies on to avoid them:

• One makes mistakes in the deductive steps, and nobody else whoproof-reads our proofs is able to spot mistakes in the proof. Whilerelatively infrequent, this situation is avoided in practice by oneof two means: (i) having multiple people proof-check, perhapsspanning several years or even decades; (ii) having modern proof-checkers (“theorem provers”) check the steps. The Flyspeck projectof Thomas Hales [69] attempts to find such machine-checked proofsfor several mathematical conjectures for which the problem state-ment is easy, but the proofs are so immensely hard that it requiresexperts plus reliable computer programs working in concert. Several

74 5 Mathematical Logic, Induction, Proofs

impressive results have already been obtained, including a solutionto the Kepler conjecture.1

• We end-up proving something like 6 = 6, and then realize that wehave perhaps applied a circular chain of reasoning steps that ledus to a tautology. We crumple and discard our worksheets and tryanother attack.This could happen in a reasoning tool that loops due to the presenceof an unfortunate circular chain of reasoning steps. The solution isto build proof tools that avoid such direct circularities from evermanifesting. The ACL2 theorem prover [3] is an example of a toolthat incorporates such precautions.

• We attempt to prove a famous open conjecture, such as P %= NP ,and end up deducing, after a few hours, that 5 = 5. This allowsus to conclude nothing! (Had we ended up proving 5 = 6 througha sound chain of reasoning steps, we would have ended up provingthat P = NP ). In Chapter 12 on Pumping lemmas, we will revisitthis issue.

• We are asked to prove that 5 = 6. By negating the proof goal, wetry to prove that 5 %= 6 and get nowhere. In this particular case,we might stop after a few hours and try instead to prove 5 %= 6.We then begin by assuming the negation of 5 %= 6, which is indeed5 = 6, and easily finish the proof by obtaining a contradiction (endup proving false). At this point, we would have proved 5 %= 6 (seeSection 5.2.3).This situation shows that one has to know what proof goal is likelyto be true before wasting much time.

• We are asked to prove that 267 / 1 is a prime number. We mightattempt to search for all factors in a certain range, but find suchbrute-force techniques to be utterly infeasible. We sit utterly hope-less until someone gives us one of the factors (761, 838, 257, 287),and then we easily find the other factor.This situation demonstrates that some proofs are hard because ofthe sheer computational complexity involved. In fact, finding theprime factors of an arbitrary natural number is a very hard prob-lem. Cryptography systems rely on this fact for the secure transmis-sion of data. Rivest, Shamir, and Adleman invented an algorithmthat goes by the acronym RSA, and depends on this computationalcomplexity [102].

1 The Kepler conjecture is about the most compact 3-dimensional arrangement forcannon balls – or oranges in a fruit-stand for that matter. For details, see [69].

5.2 Proof Methods 75

In summary, before embarking on any proof, one must attempt to as-sess the likelihood of the proof goal being true as well as the inherentdi"culty of finding the proof. Unfortunately, as discussed in Chap-ters 17 and 18, these are, in general, impossible to do. Fortunately, inpractice, such proofs are routinely carried out in computer companiessuch as AMD and Intel (e.g., [52]), with respect to systems such asfloating-point hardware.

5.2 Proof Methods

A proof is a sequence of steps justifying why something is true. Moreformally, a proof of a statement sn in formal logic is a sequences0, s1, . . . , sn, such that each si is either an axiom or follows from someof the earlier sj (j < i) through a rule of inference. In Chapter 18, weshall introduce the formal logical machinery necessary to define whata step can be, and what true means. In this section, we list a few stylesof proofs and their uses throughout this book. We begin with a reviewof some of the basic operators.

5.2.1 The implication operation

Implication, or 3, stands for “if then.” It is used to assert the truth ofits consequent conditional on the truth of its antecedent. For example,

if (x > 2 + even(x)) then composite(x)

is a true assertion. Written using 3, the above statement reads

(x > 2 + even(x)) 3 composite(x).

This assertion is true because every even number is a composite num-ber, and for odd numbers, the above assertion is vacuously true (itsantecedent is false). What about the statement

(x > 2 + x < 2) 3 (x = x + 1)?

The antecedent that asserts that x is both > 2 and < 2 is false. In thiscase also, the assertion is vacuously true. Note that we are not drawingthe conclusion x = x + 1 - the moment we find that the antecedent isfalse, we leave the consequent alone. Another sentence of this form is,“if the moon is made of blue cheese, then horses can fly,” in that it isa true assertion.2 The assertion2 Notice that we are not concluding that horses can fly.


(x > 2 + even(x)) 3 prime(x)

is, on the other hand, neither true nor false (it is both falsifiable andsatisfiable). Note that a 3 b is equivalent in meaning to ¬a * b.

An exercise in reading i!

We now turn to the i! operator. Basically, a i! b, also written a = bor a = b, means a 3 b and b 3 a. In other words, a has the same truthvalue as b; if a is true, so is b; if a is false, so is b.

Definitions involving i! are sometimes tricky to read. Consider thefollowing definition that talks about ultimately periodic sets (see alsoSection 10.5):

Definition 5.1. (Ultimately Periodic) A set S 5 N is said to be ul-timately periodic (UP) if there exists a number n . 0 and anothernumber (called ‘period’) p > 0 such that for all m . n, m , S i!m + p , S.

See if you understand the above definition by doing Exercise 5.4 onPage 86. The fact that the set in Question 3 is UP is clear, becausen = 26 and p = 15 works. Think how you get the other two answers.

5.2.2 ‘If,’ or ‘Definitional Equality’

When one presents mathematical ideas, one often adopts a certain sys-tematic style consisting of first introducing a bunch of definitions, fol-lowed by the statement and proof of various theorems. While introduc-ing definitions, you may have noticed that people use various kinds ofequality or arrow symbols, some of the typically used symbols being.=,

&=, or sometimes even >. Also while defining Boolean quantities,

we might even employ i! or = instead of these other variants. We willprefer

.= over all these various forms.

This is known as definitional equality. Definitional equalities areused to introduce new definitions into the domain of discourse. Notethat such a definition, by itself, causes no contradictions. For example,let there be a definition

foo(x).= odd(x) + prime(x)

in some domain of discourse, where foo has never before been intro-duced, while odd and prime have already been defined. However, if onecontinues defining functions ‘willy nilly,’ and later introduces the def-inition foo(x)

.= even(x), one would have introduced a contradiction


that asserts that even(x) = odd(x) + prime(x)! In many frameworks,a second definition of something that is already defined is flagged asan error. Consequently, some protection (through checking) is availablewhile using definitional equalities.

Considering all this, a real conundrum seems to arise when one usesa definitional equality symbol to introduce a new symbol which alsoappears on the right-hand side of the definition! How can we be intro-ducing something “afresh” when we are also using it on the right-handside, as if it were already existing? This is, however, how recursive def-initions are! Do such recursive definitions always ‘make sense?’ Canthey cause contradictions of the kind alluded to above? How do weunderstand recursive definitions that, on one hand, are so very conve-nient, but on the other hand prone to circularity? Chapter 6 examinesthese questions in greater detail, and helps set the stage for using re-cursive definitions with more assurance. If you are planning to skipChapter 6, try to at least study the following recursive definitions tosee which ones define a function (any function at all), which ones definea function uniquely. Also, try to understand carefully why uniquenessis achieved.

LHS and RHS: We define two abbreviations that will oftenbe used: LHS stands for left-hand side and RHS for right-handside. These abbreviations may refer to equations, productionrules, etc., all of which have two “sides” to them.

The as Notation

Sometimes, when we define a function f , we might be interested notonly in the entire argument fed to f but also the substructure of thearguments. In this case, the use of the ‘as’ notation comes in handy.Example: f(x as 6y, z7, w) = if w > 0 then f(x,w / 2) else z.

In this function, if w > 0 then we recurse, passing x “as is” in thefirst argument. If w = 0, we return the second component of x, whichis z. The as notation allows both x and its components to be referredto on the right-hand side.

5.2.3 Proof by contradiction

Proof by contradiction, or reductio ad absurdum, is a reasoning principleone uses frequently in day-to-day life (for example in playing games, indetecting contradictions in arguments, etc.). This principle is capturedby the contrapositive rule


(A 3 B) = (¬B 3 ¬A),

as can easily be seen by converting A 3 B to ¬A*B and also similarlyconverting (¬B 3 ¬A).3 If we assume something (“A”) and derive acontradiction, we have essentially proved A 3 false, which, by thecontrapositive, allows us to conclude true 3 ¬A, or ¬A.

We have already seen how proof by diagonalization uses proof bycontradiction. We shall also use proof by contradiction in the contextof the Pumping lemma in Chapter 12, and in the context of mappingreducibility arguments in Chapter 17.

5.2.4 Quantification operators ! and "

We now present some facts about the " and # operators. These opera-tors allow you to perform iteration over a certain range, similar to what% (summation) and & (product) do in traditional mathematics. Theuniversal quantification operator " is used to make an assertion aboutall the objects in the domain of discourse. (In mathematical logic, thesedomains are assumed to be non-empty). For example,

"x : Nat : P (x)

is equivalent to an infinite conjunction

P (0) + P (1) + P (2) . . .

or equivalently +x#Nat : P (x). Note that we are saying “x : Nat.” Thisis simply saying “x belongs to type Nat.” It is entirely equivalent tox , Nat—we show both uses here, as both usages may be found in theliterature; therefore, it pays to be familiar with both usages.Here is an example of how " works for a finite range:

"x , {0, 1, 2} : P (x)

The above formula is equivalent to P (0) + P (1) + P (2).The above definitions allow us to establish numerous identities, forinstance:

(" x : Nat : P (x))+ (" x : Nat : Q(x)) : (" x : Nat : (P (x)+Q(x))),

as the explicit + gets absorbed into the iterated conjunction denotedby ".

3 The above theorem can also be written (A ! B) " (¬B ! ¬A), as the equalitysymbol " takes the meaning of # for Booleans.


Similarly,#x : Nat : P (x)

is equivalent to an infinite disjunction

P (0) * P (1) * P (2) . . .

or equivalently *x#Nat : P (x).In [75], Lamport introduces a notational style for writing long math-

ematical formulas in a readable manner. His insight is that by usingsuitable indentations, and by employing the operators + and * in amanner similar to bullets, one can avoid many parentheses that wouldotherwise be needed. For instance, the above formula would be writtenas

* P (0)* P (1)* P (2)* . . .

We will often employ this style in the following illustration.

Illustration 5.2.1 Fully expand out the " and # below, and simplifythe resulting Boolean formula. “and” and “or” are equivalent to + and*, except they are written in the prefix syntax (not infix). Your finalanswer should be T or F (true or false), with justifications.

("x , Bool : "y , Bool : (or(x, y) 3 #z , Bool : and(y, z)))

Solution: The given formula is

"x , Bool : "y , Bool :or(x, y) 3 #z , Bool : and(y, z)

Expanding x, we get+ "y : or(0, y) 3 #z : and(y, z)+ "y : or(1, y) 3 #z : and(y, z)

Now expand y:+ + or(0, 0) 3 #z : and(0, z)

+ or(0, 1) 3 #z : and(1, z)+ + or(1, 0) 3 #z : and(0, z)

+ or(1, 1) 3 #z : and(1, z)

Now, expand z:


+ + or(0, 0) 3 * and(0, 0)* and(0, 1)

+ or(0, 1) 3 * and(1, 0)* and(1, 1)

+ + or(1, 0) 3 * and(0, 0)* and(0, 1)

+ or(1, 1) 3 *and (1, 0)*and (1, 1)

This simplifies to 0, or false.

In first-order logic, quantification is allowed over the domain of indi-viduals (numbers, strings, etc.) while in the second-order logic, quan-tification over functions and relations is allowed. Further facts aboutquantification will be introduced as necessary.

5.2.5 Generalized DeMorgan’s Law Relating ! And "

DeMorgan’s law relates + and *. For two Boolean variables a and b,the DeMorgan’s laws are:

¬(a + b) = (¬a * ¬b), and ¬(a * b) = (¬a + ¬b).

Extending the above to infinite conjunctions (") and infinite disjunc-tions (*), we have the following generalized DeMorgan’s laws, otherwiseknown as the duality between " and #:

" x : Nat : P (x) : ¬# x : Nat : ¬P (x)

and# x : Nat : P (x) : ¬" x : Nat : ¬P (x).

5.2.6 Inductive definitions of sets and functions

The term inductive definition finds widespread use throughout com-puter science, and tends to mean a whole bunch of (related) things.We introduce two specific usages of this term now.

Inductively Defined Sets and Functions

A set that is inductively defined is constructed in a principled way, byfirst introducing the basis elements, and then “pumping” new elementsout of existing elements through constructors. For example, the set ofall lists over N , denoted list[N ], is defined as follows:


nil , list[N ]if l , list[N ] and n , N , then cons(n, l) , list[N ].

The intent is to say that list[N ] is the smallest set closed under theabove two rules. In a general setting, the process of inductively defininga set S consists of first including the basis elements B = {bi | i , m}into S. Thereafter, for every element x in the set S, one of a (usuallyfinite) set of constructors C = {ci | i , n} are applied to obtainelements of the form ci(x). The set S being defined is the least setthat includes the basis-case elements and is closed under constructorapplication. Said more formally:

Rule 1: For every i , m, bi , S.Rule 2: If x , S, and ci is a constructor for i , n, then ci(x) , S.Rule 3: S is the least such set.

There are actually two other (and somewhat non-obvious) ways to ex-press the intent of Rule 3. They are explained below, as they tend tooccur in other books as well as papers.

1. S is the intersection of all such sets.2. Express S directly as follows:

S = {x | (x , B) * (#ci , C : x = ci(y) + y , S)}.

The constructors of an inductively defined set S have the status ofprimitive functions that operate over the set. In most cases, a wholelist of other inductively (or, equivalently, recursively) defined functionsare also defined over an inductively defined set. One can then establishproperties of these functions by structural induction; for instance, if wedefine the length of a list, and the append function on lists, inductivelyas follows:

length(nil) = 0length(cons(n,L)) = 1 + length(L)

append(nil, L) = Lappend(cons(x,L1), L2) = cons(x, append(L1, L2)).

One can then easily prove properties such as

length(append(L1, L2)) = length(L1) + length(L2)

by inducting on the number of applications of the cons operator(more detailed discussions of structural induction are provided in Sec-tion 5.3.3). We shall use proof by structural induction when we dealwith data structures such as trees and lists.


Free inductive definitions: The inductive definition of a set S iscalled a free inductive definition if, for every object x in S, either x , B,or x = ci(y) for exactly one constructor ci and exactly one elementy , S. Only one of these conditions must hold. Such sets are calledfreely generated, from a set of basis elements B and a set of constructorsC.

To understand the notion of freely generated sets better (and to seewhy it matters), suppose we have a set S such that for some elementy in it, y , B and y = ci(z). Or suppose we have a set S such thatfor some element y in it, y = ci(z1) and also y = cj(z2) for z1 %= z2

or ci %= cj . Then S is not freely generated. For example, in list[N ] ifsome element y = nil, as well as y = cons(x1, cons(x2, cons(x3, nil)))for some arbitrary x1, x2, and x3, what can go wrong? (Exercise 5.18.)With freely generated sets, we can be assured that functions that areinductively defined over them exist and are unique. With non-freelygenerated sets, such functions may not have a unique definition.

5.3 Induction Principles

We now provide a brief tour through some of the induction princi-ples that we shall use repeatedly. We begin with a recap of the twofamiliar principles of induction over N , namely arithmetic inductionand complete induction. We shall demonstrate that these principles areequivalent.

5.3.1 Induction over natural numbers

Generally, the induction principle over natural numbers is stated intwo forms: arithmetic induction and complete induction. We now statethese forms and prove them equivalent.Arithmetic: The principle of arithmetic induction states that for allP , if we can show P (0) and for all x we can show that P (x/1) impliesP (x), then for all x, P (x). In symbols:

"P : + P (0)+ "x > 0 : P (x / 1) 3 P (x)3 ("x : P (x)).

Complete: The principle of complete induction states that for all P ,if we can show that for all y, for all x < y, P (x) implies P (y), then forall x, P (x). In symbols:

5.3 Induction Principles 83

"P : "y : ("x < y : P (x)) 3 P (y)3 ("x : P (x)).

Proof of equivalence: To prove that these induction principles areequivalent, we proceed as follows. First pick an arbitrary predicate Pfor which the proof is being done. Then, these induction principles areof the form A 3 B and C 3 B (A standing for arithmetic and C forcomplete). Therefore, we merely need to prove A = C, i.e., that theantecedents of these induction theorems are equivalent.

1. We need to prove A = C, where

A = P (0) + "x > 0 : P (x / 1) 3 P (x)

andC = "y : ("x < y : P (x)) 3 P (y).

2. (A = C) : (A 3 C) + (C 3 A).3. Let’s prove (C 3 A). I’ll leave (A 3 C) as an exercise for you

(Exercise 5.21).4. Here there are two approaches possible: one is to prove (C 3 A)

itself, and the other is to prove the contrapositive form (¬A 3 ¬C).We now detail the proof using the contrapositive form (the otherform is quite similar).

5. ¬A = ¬P (0) * ¬("x > 0 : P (x / 1) 3 P (x)).6. Let’s assume ¬P (0).7. Now, ¬C = #y : ("x < y : P (x)) + ¬P (y).8. Consider y = 0. For this y, there is no x < y. Hence, the ("x < y :

P (x)) part is vacuously true.9. Therefore, by the “infinite disjunction” property of #,

#y : ("x < y : P (x)) + ¬P (y)

reduces to¬P (0) * . . . ,

which obviously follows from the assumption ¬P (0).10. This finishes one of the cases of ¬A. Let us move on to the other

case, by now assuming

¬("x > 0 : P (x / 1) 3 P (x)).

This means (#x > 0 : P (x / 1) + ¬P (x)).11. (I know how to “pull” a magic assumption here, but won’t do it,

as I want to illustrate this proof getting stuck and having to comeback to discover the magic).


12. Since (#x > 0 : P (x/ 1)+¬P (x)), assume a witness for x.4 Call itx0.

13. Therefore, we have P (x0 / 1) + ¬P (x0) as our assumption.14. We can instantiate the y in C to x0, to get ("x < x0 : P (x)) 3

P (x0).15. Here, we are stuck. We cannot instantiate the x in the above for-

mula, as it lies under an “implicit negation.” (Anything belongingto the antecedent of an implication is under an implicit negation).Said another way, if ", an infinite conjunction, implies something,we cannot say that one of the conjuncts implies the same thing).

16. Revisit Step 12. In there, assert that the witness x0 is the first suchx such that P (x / 1) + ¬P (x).

17. Now, consider ("x < x0 : P (x)) 3 P (x0) again. Clearly, becauseof the way we selected x0, for all x < x0 we do have P (x) holdingtrue. So we are left with P (x0). Since this is false, we again have¬C.

18. Consequently, for both cases resulting from ¬A we have ¬C, or inother words, ¬A 3 ¬C, or C 3 A.

5.3.2 Noetherian induction

Both arithmetic and complete induction are special cases of the noethe-rian induction rule stated in terms of well-founded partial orders. Awell-founded partial order 9 is one that has no infinite descendingchains of the form . . . 9 a!3 9 a!2 9 a!1 9 a0. The total order (N,&)is a special case of a well-founded partial order. Other well-founded par-tial orders include the lexicographic ordering of words, subset ordering,etc. In a general setting, well-founded partial orders can have multipleminimal elements (giving rise to multiple basis cases). For example,consider the ordering between closed intervals over natural numbers:

9= {6[a, b], [A,B]7 | a . A + b & B}.

We have [2, 2] 9 [1, 3], but neither [2, 2] 9 [3, 3] nor [3, 3] 9 [2, 2]. Inthis case, we have an infinite number of minimal elements.The principle of noetherian induction: Suppose (W,9) is a well-founded partial order. Suppose property P holds for all minimal ele-ments in W . Also, for all non-minimal elements x, if for all y 9 x P (y)implies P (x), then "x.P (x). In symbols:

4 A witness is a value that, when substituted in place of a variable that is existen-tially quantified in a formula, makes the formula true. For example, a witness fory in $y.y < 5 is 4.

5.4 Putting it All Together: the Pigeon-hole Principle 85

"P : +i#min(W ) P (i)+ "x /, min(W ) : ("y 9 x : P (y)) 3 P (x)3 ("x : P (x)).

5.3.3 Structural

The principle of structural induction can easily be seen to be a specialcase of the principle of noetherian induction by selecting 9 to keep trackof the number of “layers” of application of constructors. Specifically,min(W ) = B = {bi | i , m}, and

preceq = {6x, y7 | x , B * x /! y},

where x /! y means there exists a sequence of constructors that canbe applied to x to obtain y.

5.4 Putting it All Together: the Pigeon-hole Principle

The colloquial version of the pigeon-hole principle states that if thereare n pigeons and n / 1 holes, then surely there is a hole with morethan one pigeon. Stated in general, we can say that there is no totalinjection from a finite set S into any of its proper subsets. This is nottrue of infinite sets, as Exercise 5.25 shows.

Illustration 5.4.1 We can prove the pigeon-hole principle using in-duction, as well as proof by contradiction, as follows (adapted from[106]). Consider finite sets Bi from some universe, where i is the sizeof the set. We denote the size of a set (e.g., S) by the | | operator (e.g.,|S|). Now, in order to show that for all k, there is no total injection ffrom any Bk set to one of its proper subsets, we proceed as follows:

Basis case: Consider sets B2 with two elements. There is no totalinjection from B2 into any of its proper subsets.Induction hypothesis: Now assume by induction hypothesis that for2 < i & (k / 1), there is no total injection from any of the Bis toany of their proper subsets. Suppose a particular set B

!

k of size khas a total bijection into its proper subset AB

!k. But now, we can

remove any element x from B!

k and its image under f , namely f(x),from AB

!k, and get a total injection from a set in Bk!1 to one of its

proper subsets. This is a contradiction. Therefore, we conclude thatfor any k, there does not exist a total injection f from any set Bk

to one of its proper subsets.


Chapter Summary

This chapter began with some cautionary remarks pertaining to con-ducting formal proofs: how one has to spend some time assessingwhether the proof goal is achievable, and then how to carry out theproof in a manner that is easily verifiable. It then discussed the op-erators ‘if’ (3) and ‘i!’ (=). Proof by contradiction was introducedthrough the game of Mastermind. After discussing quantifications, in-ductively defined sets and functions, and induction principles, a proofof equivalence between arithmetic and complete induction was given.Various other induction principles were also discussed. Many of theseconcepts were illustrated by the pigeon-hole principle.

Exercises

5.1. Read about the following man/machine produced proofs that havemade headlines (all references obtainable fromhttp://en.wikipedia.org/wiki or other sources):

1. The four-color theorem including the classic proof of Appel andHaken and the recent proof of Georges Gonthier.

2. Andrew Wiles’s proof of Fermat’s Last Theorem.

5.2. With respect to the discussions concerning 267 / 1 on Page 74,determine the other factor of 267 / 1 by long division.

5.3. Indicate which of the following implicational formulas are true,and which are false:

1. (1 = 2) 3 (2 = 3).2. (gcd(x, y) = z) 3 (gcd(x + y, y) = z). Here, gcd stands for the

greatest common divisor of its arguments.3. (k > 1) + hasclique(G, k) 3 hasclique(G, k / 1). Here,

hasclique(G, k) means that graph G has a clique of size k.4. (k > 1) + hasclique(G, k) 3 hasclique(G, k + 1).5. ((a 3 (b 3 c)) 3 ((a 3 b) 3 (a 3 c))).

5.4.1. Justify that 0 is ultimately periodic (UP).2. Justify that {0, 1, 27} is UP.3. Justify that {0, 1, 5, 23, 24, 26, 39, 41, 54, 56, 69, 71, 84, 86, 99, 101, . . .}

is UP.

5.5. Study the following “recursive definitions:” Assume that x rangesover Int. The operator |x| takes the absolute value of x:


1. f(x).= f(x) + 1

2. f(x).= f(x + 1)

3. f(x).= f(x)

4. f(x).= if (x = 0) then 1 else f(x + 1)

5. f(x).= if (x = 0) then 1 else f(x / 1)

6. f(x).= if (x = 0) then 1 else f(|x|/ 1)

7. f(x).= if (x = 0) then 1 else x + f(|x|/ 1)

Answer the following questions:

(a) Which definitions are malformed (are not definitions but arecontradictions)?(b) Which definitions are well-formed but end up defining an every-where undefined function?(c) For all other cases, describe the mapping e!ected by the function.

5.6. Prove that1

2 is irrational.

5.7. Consider the game of Mastermind played using three colors (tosimplify the exposition), y, g, and b, standing for yellow, green, andblue.

CODE y g b b Move number-------------------------

rrr g g b b 3www b b y y 2rw y y g g 1

The game is played between two players A and B. A selects a secret codehidden from view of the opponent B. In the example below, the codeis y g b b. Player B must try and guess the code, improving his/herguesses based on scores assigned by A. In our example, B’s moves areshown below, numbered as shown. B’s first move is y y g g, whichfetches a r (for red) and w (for white). A red means that one of thepegs is in the right place and of the right color, while a white meansthat there is a peg of the right color, but in the wrong place. In ourexample first move, one y peg fetches a red, and one of the g pegsfetches a white. The next two moves of B are as shown. With the gamepoised at this point, show that B can, by using the principle of proof bycontradiction, finish the game in two more moves. (Hint: B views move2 with interest. This move reveals that if there are two ys in the code,there must only be one b. Taking this assumption, to move 1 leads toan immediate contradiction. Continue to argue through the details).


5.8. How does one prove the statement, “There exist horses with awhite tail?” (For this, and the following ‘horse proofs,’ conserve yourhorse power - simply o!er proofs, in a few sentences, of the form: “gofind a horse satisfying criterion x, then look for criterion y in them;hence, proved/disproved”).

5.9. How does one disprove the statement, “There exist horses with awhite tail?”

5.10. How does one prove the statement, “For all horses that haveblack ears there are some that have a white tail?”

5.11. How does one disprove the statement, “For all horses that haveblack ears there are some that have a white tail?”

5.12. Propose a quantified statement in first-order logic that expressesthe fact that set S 5 Nat is infinite. Construct a statement that goeslike, “for all elements of S, . . ..”

5.13. Propose a quantified statement in first-order logic that expressesthe fact that set S 5 Nat is finite. Construct a statement that goeslike, “there exists. . ..”

5.14. Eliminate the negation operator entirely from the following for-mula:

¬("x : #y : "z : (p(x, y) 3 ¬q(y, z))).

5.15.1. Prove by induction that the sum of 1 through N is N(N + 1)/2.2. Obtain an expression for the total number of nodes in a balanced

k-ary tree. Prove this result by induction.3. Suppose we have the following kind of “almost binary” finite trees.

All nodes of such a tree have a branching factor of 2, except for thefather node of a leaf node, which has a branching factor of only 1(these nodes “father” exactly one child).

Such trees arise in the study of context-free grammars whoseproductions are of the A->BC or A->a. We shall study thesegrammars, called the Chomsky Normal Form grammars, inChapter 15.

Obtain an expression for the length of the frontier of such a treeas a function of the number of interior nodes. Verify your answerthrough an inductive proof.


5.16. Inductively define the set of all binary trees with nodes labeledby elements from Nat. Employ the constructor Node(n) to introducea tree of height zero containing just a node n , Nat. Employ theconstructor Tree(n, T1, T2) to introduce a tree rooted at node n withleft sub-tree and right sub-tree being T1 and T2, respectively. Writedown the mathematical equations that achieve such a definition.

5.17. Given a tree such as defined in Exercise 5.16, inductively definea function that sums the values labeling all the nodes.

5.18. Pertaining to the definition of freely generated sets on Page 5.2.6,explain what can go wrong with a non-freely generated set. Hint: Con-sider length(y). Can we get two possible answers when length is appliedto the same y?

5.19. Find the smallest N such that using only 4-cent postage stampsand 7-cent postage stamps, it is possible to make postage for any de-nomination k . N cents. Then prove your result by induction.

5.20. Prove using induction that using a 6-liter water jug and a 3-literwater jug, one cannot measure out exactly 4 liters of water. Here arefurther instructions.

1. What is wrong if you directly attack this induction proof, takingthe given property as the proof-goal (“cannot measure out”) (onesentence)?

2. How would you choose the new proof goal (one sentence)?3. Show the new proof goal by induction.

5.21. Prove the remaining case of A 3 C in the derivation on Page 84.

5.22. Provide an example of a well-founded partial order with morethan one minimal element.

5.23. If you have not already, then first read Section 5.3.2. Now con-sider the relation [c1, c2] 9c [d1, d2] where c1, c2, d1, d2 are characters(belong to {a . . . z}) and d1 & c1 and c2 & d2, with the character ordi-nal positions compared by & in the usual way.

1. Show that &c is a well-founded partial order.2. Show that any subset S of &c is also a well-founded partial order.3. How many minimal elements does &c have?


4. Define the height of any non-minimal element [c1, c2] of &c to beone more than the maximum number of elements that could properlyexist between [c1, c2] and one of the minimal elements of &c. Theheight of minimal elements are defined to be 0. For example, theheight of [b, d] is 2, as either [b, c] properly lies between [b, d] and[b, b], or [c, d] properly lies between [b, d] and [d, d]. Write down ageneral formula for the height of an element of &c and prove it byNoetherian induction.

5.24. Develop a formula for the number of leaf nodes in a k-ary tree.Prove the formula by structural induction.

5.25. Show that there exist infinite sets S such that there is a totalinjection from S into one of its proper subsets.

5.26. How many di!erent ways can n identical pigeons occupy n /1 pigeon holes? (by “identical,” we mean that only the count of thepigeons in various holes matters; also, each hole can accommodate anynumber of pigeons).

5.27. I heard this one on NPR radio in the “Car Talk” show. A manand his wife went to a party where a total of 52 people attended.The party went well, with everyone mingling and shaking hands (theexact number of such handshakes, nor who shook whose hands is notknown; also, a person shaking his/her own hands does not count as ahandshake). However, while returning from the party, the man told hiswife, “Gee, what a nice party!” The wife replied, “Yes! And, I knowfor a fact that there were at least two people (let’s call them A and B)who shook the same number of hands.” Prove this statement for 52,and then prove it for any N , stating all steps clearly.

5.28. Consider the set of characteristic formulas, CF (x, y), over twovariables x and y. This set can be inductively defined by the followingrules:

1. true , CF (x, y) and false , CF (x, y).2. If f1 , CF (x, y) and f2 , CF (x, y), then

a) f1 + f2 , CF (x, y)b) f1 * f2 , CF (x, y)c) ¬f1 , CF (x, y)d) (f1) , CF (x, y).

3. CF (x, y) is the intersection of all such sets.


While we allow the use of parentheses ( and ), they may be left outwith the convention that ¬ has the highest precedence, followed by +and finally *.

Examples of formulas in CF (x, y) are (x * y), x * y, false, (true),(x+¬x), (x+ y * x+ y *¬x), etc. Clearly many of these characteristicformulas are equivalent, in the sense that they denote the same relation.

1. Show that (CF,3) forms a preorder. 3 is the implication opera-tor. 6x, y7 ,3 precisely when the formula x 3 y is valid (or ‘is atautology,’ or ‘is true for all x, y’).

2. Show that := (3 ' 3!1)x is an equivalence relation. Recall that3!1 can be written as >.

3. How many equivalence classes does the equivalence relation (CF,:)have? Show by going through the definition of equivalence classesgiven earlier (specifically, elements(R)/R).

4. Argue that two formulas, f1 : f2, denote the same relation, andhence [f1] = [f2].

5. Arrange the relations denoted by the equivalence classes of CF (x, y)into a lattice, clearly pointing out the glb and lub.

Two Puzzles

The following two puzzles are due to Lewis Carroll (the author of Alicein Wonderland and a mathematician at Christ Church, Oxford).

5.29. From the premises

1. Babies are illogical;2. Nobody is despised who can manage a crocodile;3. Illogical persons are despised.

Conclude that Babies cannot manage crocodiles.

5.30. From the premises

1. All who neither dance on tight ropes nor eat penny-buns are old.2. Pigs, that are liable to giddiness, are treated with respect.3. A wise balloonist takes an umbrella with him.4. No one ought to lunch in public who looks ridiculous and eats

penny-buns.5. Young creatures, who go up in balloons, are liable to giddiness.6. Fat creatures, who look ridiculous, may lunch in public, provided

that they do not dance on tight ropes.7. No wise creatures dance on tight ropes, if liable to giddiness.


8. A pig looks ridiculous carrying an umbrella.9. All who do not dance on tight ropes and who are treated with

respect are fat.

Show that no wise young pigs go up in balloons.

6

Dealing with Recursion

Recursion is a topic central to computer science. Lambda calculus o!ersus a very elegant (and fundamental) way to model and study recursion.In a book on computability and automata, such a study also servesanother purpose; to concretely demonstrate that Lambda calculus pro-vides a universal mechanism to model computations, similar to therole played by Turing machines. While this chapter can be skimmed,or even skipped, we have taken su"cient pains to make this chapter as“friendly” and intuitive as possible, permitting it to be covered withoutspending too much time. Covering this chapter will permit fixed-pointtheory to be used as a conceptual “glue” in covering much importantmaterial, including studying context-free grammars, state-space reach-ability methods, etc.

6.1 Recursive Definitions

Let’s get back to the discussion of function ‘Fred’ introduced in Chap-ter 2, Section 2.6. Now consider a recursively defined function, alsocalled ‘Fred.’ We fear that this will make it impossible to ‘de-Fred:’

function Fred x = if (x=0) then 0 else x + Fred(x-1)

It is quite tempting to arrive at the following de-Freded form:

function(x) = if (x=0) then 0 else x + self(x-1)

However, it would really be nice if we can avoid using such ad hocconventions, and stay within the confines of the Lambda notation asintroduced earlier. We shall soon demonstrate how to achieve this; itis, however, instructive to examine one more recursive definition, nowinvolving a function called Bob:

function Bob(x) = Bob(x+1).

94 6 Dealing with Recursion

One would take very little time to conclude that this recursive definitionis “plain nonsense,” from the point of view of recursive programming.For example, a function call Bob(3) would never terminate. However,unless we can pin down exactly why the above definition is “nonsense,”we will have a very ad hoc and non-automatable method for detectingnonsensical recursive definitions - relying on human visual inspectionalone. Certainly we do not expect humans to be poring over millionsof lines of code manually detecting nonsensical recursion. Here, then,is how we will proceed to “de-Fred” recursive definitions (i.e., createirredundant names for them):

We will come up with a ‘clever’ Lambda expression, called Y, thatwill allow recursion to be expressed purely within the framework ofLambda calculus. No arbitrary strings such as “Fred” will be stuckinside the definition.Then we will be able to understand recursion in terms of solving anequation involving a function variable F .We will then demonstrate that recursive definitions ‘make sense’when we can demonstrate that such equations have a unique solu-tion. Or if there are multiple solutions, we can select (using a soundcriterion) which of these solutions ‘makes sense’ as far as computersgo.

6.1.1 Recursion viewed as solving for a function

Let us write our Fred function as an equation:

Fred=lambda x. if (x=0) then 0 else x + Fred(x-1).

The above equation can be rewritten as

Fred=(lambda Fred’ .lambda x. if(x=0) then 0 else x + Fred’(x-1)) Fred

The fact that this equation is equivalent to the previous one caneasily be verified. We can apply the Beta rule, plugging in Fred inplace of Fred’, to get back the original form.

We can now apply the Alpha rule, and change Fred’ to y, to obtainthe following:

Fred = (lambda y . lambda x . if(x=0) then 0 else x + y(x-1)) Fred.

Well, we are almost done eliminating the redundant name “Fred.”What we have achieved is that we have expressed Fred using an equa-tion of the form

Fred = H Fred

where H is the Lambda expression(lambda y . lambda x . if(x=0) then 0 else y(x-1)). Note thatH contains no trace of Fred.

6.1 Recursive Definitions 95

6.1.2 Fixed-point equations

We now make a crucial observation about the nature of the equationFred = H Fred. Recall that juxtaposing H and Fred is, in Lambdacalculus, equivalent to applying H to Fred. If we wish, we can rewrite theabove equation as Fred = H(Fred) to express it in the more familiarsyntax of “f(x)”—for f applied to x. Note that there is somethingpeculiar going on; when we feed Fred to H, it spits out Fred. Theapplication of H seems to be “stuck” at Fred.

In mathematics, an equation of the form x = f(x) is called a fixed-point equation. Think of a fixed-point as a fixed point, i.e., an “immov-able point.” It appears that things are “stuck” at x; even if we applyf to x, the result is stuck at x.

Can we find such “stuck function applications?” Surely!

• Take a calculator, and clear its display to get 0 on the display. Thenhit the cos (cosine) key to compute cos(0) = 1.

• Hit cos again, to get 0.9998477.• Hit cos again, to get 0.99984774.• Hit cos again, to get 0.99984774 - we are now “stuck” at the fixed-

point of cos. (The number of steps taken to achieve the fixed-pointwill, of course, depend on the precision of your calculator).

• Now (assuming you have the factorial function on your calculator),compute 0, 0!, 0!!, etc., then 2, 2!, 2!!, etc., and finally 3, 3!, 3!!, etc.How many fixed-points did you discover for factorial?

Here is another way to get a “fixed-point out of a photocopyingmachine.” More specifically, we can get the fixed-point of the imagetransformation function of a photocopying machine. Go photocopy yourface; then photocopy the photocopy, photocopy the photocopy of thephotocopy, etc. In most photocopying machines, the image stabilizes,by turning all gray regions to black or (sometimes) white. Such a stableimage is then one fixed-point of the image transformation function ofthe photocopying machine.

Coming back to our example, we want a fixed-point, namely a func-tion Fred, that solves the fixed-point equation Fred = H(Fred). Ide-ally, we would like this fixed-point to be unique, but if that’s not possi-ble, we would like some way to pick out the fixed-point that correspondsto what computers would compute should they handle the recursiveprogram.

The beauty of Lambda calculus is that it does not fundamentallydistinguish between functions and “values,” and so the principles ofobtaining fixed-points remain the same, independent of whether the


fixed-points in question are functions, numbers, or images. For the ex-ample at hand, if Fred is a fixed-point of H, we must

either ensure that there is only one fixed-point, thus giving a uniquesolution to Fred, orin case multiple fixed-points exist, find a canonical way of pickingthe desired fixed-point (solution).

We shall resolve these issues in what follows.

6.1.3 The Y operator

There exists a class of (almost “magical”) functions, the most popularone being function Y below, that can find the unique fixed-points offunctions such as H!

Y = (lambda x. (lambda h. x(h h)) (lambda h. x(h h)))

In other words, we claim that (Y f) is the fixed-point of any arbi-trary function f. Things seem almost too good to be true: if we desirethe fixed-point of any function f expressed in Lambda calculus, simply“send Y after it.” In other words, Y (f) would be the fixed-point. Forthe above to be true, the following fixed-point equation must hold:

(Y f) = f(Y f).

Here is how the proof goes:

Y f = (lambda x. (lambda h. x(h h)) (lambda h. x(h h))) f

= (lambda h. f(h h)) (lambda h. f(h h)) <-- look --|||

= f( (lambda h. f(h h)) (lambda h. f(h h)) ) -------

= f( Y f ).

In the above derivation steps, in the penultimate step, we have f appliedto a big, long Lambda form that was obtained in the second step asan expansion of Y f. Therefore, in the last step, we can obtain thesimplification indicated. Okay, now, finally, we have successfully “de-Freded” our original recursive definition for Fred. We can write Fredas

Fred = Y (lambda y . lambda x . if(x=0) then 0 else x + y(x-1))

where the right-hand side contains no trace of Fred. The right-handside is now an irredundant name defining what was originally cast asan explicit recursion.

6.2 Recursive Definitions as Solutions of Equations 97

6.1.4 Illustration of reduction

While we have shown how to turn the equation Fred = H Fred intoFred = (Y H), does the new irredundant name really capture the re-duction semantics of functional evaluation? Let us demonstrate thatindeed it does, by applying Fred to an argument, say 5:

Fred 5= (Y H) 5= H (Y H) 5= (lambda y . lambda x . if(x=0) then 0 else x + y(x-1)) (Y H) 5= (lambda x . if(x=0) then 0 else x + (Y H)(x-1)) 5= (if(5=0) then 0 else 5 + (Y H)(5-1))= 5 + (Y H) 4= ...= 5 + 4 + (Y H) 3= ...= 5 + 4 + 3 + 2 + 1 + 0= 15

From the above, one can observe that the main characteristic of Y isthat it has the ability to “self-replicate” a Lambda expression. Noticehow a copy of (Y H) is “stashed away” “just in case” there would beanother recursive call. Self-replication, unfortunately, is also the basison which many malicious programs such as computer viruses operate.In advanced computability theory, the deep connections between “self-replication” and computability are captured in the so-called recursiontheorem. The interested reader is encouraged to read up on this topic,including Ken Thompson’s widely cited article “Reflections on trustingtrust,” to be fully informed of the true potentials, as well as societalconsequences, of computers. On one hand, computers are mankind’smost impressive invention to date; on the other hand, they are proneto abuse, stemming either from innocent oversight or malicious intent -in both cases demanding the eternal vigil of the computing communityto guard against, detect outbreaks, and restore normal operations ifbad things do happen.

6.2 Recursive Definitions as Solutions of Equations

The reduction behavior using Y indeed tracks the normal function eval-uation method.

Fred 5 = (lambda x . if(x=0) then 0 else x + Fred(x-1)) 5= if(5=0) then 0 else 5 + Fred(5-1)= 5 + (Fred 4)


= (lambda x . if(x=0) then 0 else x + Fred(x-1)) 4= 5 + (if(4=0) then 0 else 4 + Fred(4-1))= 5 + 4 + (Fred 3)= ...= 5 + 4 + 3 + 2 + 1 + 0= 15.

What really is the advantage of a fixed-point formulation? To betterunderstand this, let us study the connection between recursion andsolving for functions more deeply.

The fact that we have equated Fred to a Lambda term (lambda x.if (x=0) then 0 else x + Fred(x-1)) containing Fred suggests thatFred is a solution for f in an equation of the form

f = (!x.if (x = 0) then 0 else x + f(x / 1)).

How many solutions exist for f? In other words, how many di!erentfunctions can be substituted in place of f and satisfy the equation?Also, if there are multiple possible solutions, then which of these solu-tions did (Y H) correspond to? Might the function f0 below, which isundefined over its entire domain N , for instance, be a solution?

f0 = (!x. 8)

Here, 8 stands for “undefined” or “bottom” value.Substituting f0 for f , the right-hand simplifies to the function

(!x.if (x = 0) then 0 else x+ 8),

or(!x.if (x = 0) then 0 else 8).

This function is di!erent from f0 in that it is defined for one input,namely 0. Hence the above equation is not satisfied. Calling this func-tion f1, let us see whether it would serve as a solution. Substituting f1

on the right-hand side, we get

(!x.if (x = 0) then 0 else x + f1(x / 1)).

Simplifying f1(x / 1), we get

(!x.if (x = 0) then 0 else 8)(x / 1)= if ((x / 1) = 0) then 0 else 8= if (x = 1) then 0 else 8.

The right-hand side now becomes

(!x.if (x = 0) then 0 else x + (if (x = 1) then 0 else 8))

6.2 Recursive Definitions as Solutions of Equations 99

= (!x.if (x = 0) then 0 else if (x = 1) then x + 0 else x+ 8)= (!x.if (x = 0) then 0 else if (x = 1) then 1 else 8)

which (calling it f2) is yet another function that defined for one moreinput value. In summary, substituting f0 for f , one gets f1, and substi-tuting f1, one gets f2. Continuing this way, each fi turns into a functionfi+1 that is defined for one more input value. While none of these func-tions satisfies the equation, in the limit of these functions is a totalfunction that satisfies the equation, and hence is a fixed-point (com-pared with earlier examples, such as the cos function, which seemedto “stabilize” to a fixed-point in a few steps on a finite-precision cal-culator; in case of the fi series, we achieve the fixed-point only in thelimit). This limit element happens to be the least fixed-point (in a senseprecisely defined in the next section), and is written

µf.(if (x = 0) then 0 else x + f(x / 1)).

Let this least fixed-point function be called h. It is easy to see that his the following function:

h(n) = %ni=0 i.

It is reassuring to see that the least fixed-point is indeed the functionthat computes the same “answers” as function Fred would computeif compiled and run on a machine. It turns out that in recursive pro-gramming, the “desired solution” is always the least fixed-point, whilein other contexts (e.g., in reachability analysis of finite-state machinesdemonstrated in Chapter 9), that need not be true.

The ‘solution’ point of view for recursion also explains recursivedefinitions of the form

function Bob(x) = Bob(x+1).

The only solution for function Bob is the everywhere undefined function!x. 8. To see this more vividly, one can try to de-Bob the recursion toget

Bob = Y (lambda y . lambda x . y(x+1)).

Suppose H = (lambda y . lambda x . y(x+1)). Now, supplying avalue such as 5 to Bob, and continuing as with function Fred, oneobtains the following reduction sequence:

Bob 5 = (Y H) 5= H (Y H) 5= (lambda y . lambda x . y(x+1)) (Y H) 5= (lambda x . (Y H)(x+1)) 5= (Y H)(5+1)


= (Y H) 6= ...= (Y H) 7= ...= (non-convergent).

In other words, Bob turns out to be the totally undefined function, or“bottom function.”

6.2.1 The least fixed-point

In the above example, we obtained a fixed-point h which we assertedto be the “least” in a sense that will now be made clear. In general,given a recursive program, if there are multiple fixed-points,

The desired meaning of recursive programs corresponds to the leastfixed-point, andThe fixed-point finding combinator Y is guaranteed to compute theleast fixed-point [114].

To better understand these assertions, consider the following fourdefinitions of functions of type Z 2 Z ! Z, where Z is the integers(this example comes from [78]):

f1 = !(x, y) . if x = y then y + 1 else x + 1f2 = !(x, y) . if x . y then x + 1 else y / 1f3 = !(x, y) . if x . y and x / y is even then x + 1 else 8Now consider the recursive definition:

F (x, y) = if x = y then y + 1 else F (x, F (x / 1, y + 1)).

We can substitute f1, f2, or f3 in place of F and get a true equation!Exercise 6.8 asks you to demonstrate this. However, of these functions,f3 is the least defined function in the sense that

whenever f3(x) is defined, fi(x) is defined for i = 1, 2, but not viceversa, andthere is no other function (say f4) that is less defined than f3 andalso serves as a solution to F .

To visualize concepts such as “less defined,” imagine as if we were plot-ting the graphs of these functions. Now, when a function is undefinedfor an x value, we introduce a “gap” in the graph. In this sense, thegraph of the least fixed-point function is the “gappiest” of all; it is asolution, and is the most undefined of all solutions – it has the mostnumber of “gaps.” These notions can be captured precisely in terms of

6.3 Fixed-points in Automata Theory 101

least upper-bounds of pointed complete partial orders [107]. In our ex-ample, it can be shown that f3 is the least fixed-point for the recursion.

6.3 Fixed-points in Automata Theory

We will have many occasions to appeal to the least fixed-point idea inthis book. Here are some examples:

Cross-coupled combinational gates attain stable states defined byfixed-points. For instance, two inverters in a loop become a flip-flopthat stores a 0 or a 1. These are the two solutions of the recursivecircuit-node equation x = not(not(x)).1

Three inverters in a loop form a ring oscillator. The reason, to afirst approximation, is that they are all trying, in vain, to solve therecursive node equation x = not(not(not(x))); in addition, there isa 3-inverter delay around the loop.Context-free grammar productions are recursive definitions of lan-guages. The language defined by such recursive definitions can bedetermined through a least fixed-point iteration similar to what wedid by starting with 8; for context-free grammars, the empty lan-guage 0 serves as the “seed” from which to begin such iterations.In the section pertaining to Binary Decision Diagrams, we will for-mulate a process of computing the set of reachable states as a fixed-point computation.In the section on nondeterministic automata, we will write recursivedefinitions for Eclosure and justify that such recursive definitionsare well-defined (i.e., they are not nonsensical, similar to the recur-sion for function Bob, for which no solution exists).

It is also worth noting that while least fixed-points are most often ofinterest, in many domains such as temporal logic, the greatest fixed-points are also of interest.

The functional (higher order function) H from Section 6.1.1, namely

H = (!y.!x.if (x = 0) then 0 else x + y(x / 1))

has an interesting property: it works as a “bottom refiner!” In otherwords, when fed the “everywhere undefined function” f0 (called the

1 Of course, this is a simplified explanation, as a golden wedding ring can also bethought of as setting up a recursive node equation x = x or as x = not(not(x)),since the two nots cancel. The crucial di!erences are of course that there is noamplification around the wedding ring, and there is far less loop delay in a ringthan in case of a flip-flop loop.


“bottom function”), it returns the next better approximation of theleast fixed-point h in the form of f1. When fed f1, it returns the nextbetter approximation f2, and so on. It may sound strange that themeaning of recursive programs is determined starting from the mostuninteresting of functions, namely the ‘bottom’ or 8 function. However,this is the most natural approach to be taking, because

8 contains no information at all,going through the fixed-point iteration starting from 8, pulls intosubsequent approximants f1, f2, etc., only relevant informationstemming from the structure of the recursion.

Later when we study state-space reachability analysis techniques inSection 11.3.2, and context-free grammars in Section 13.2.2, we willappeal to this “bottom-refining” property. We will be identifying asuitable “bottom” object in each such domain.

Chapter Summary

Fixed-point theory is often a dreaded topic. We attempted to presentthe “‘friendliest foray into fixed-point theory” that we could muster. Westart with simple recursive function definitions and motivate the needto have irredundant forms of these definitions. We arrive at such irre-dundant forms using Lambda calculus, and the Y operator. We studyfixed-point equations and how to solve them using fixed-point itera-tion, starting from the totally undefined function ‘bottom’ (8). Weshall employ fixed-points in numerous chapters of this book, especiallyin two contexts: (i) viewing context-free grammars as recursive equa-tions (Section 13.2.2), and (ii) computing the reachable set of states instate transition systems (Section 11.3.2).

6.3 Fixed-points in Automata Theory 103

Exercises

6.1. Present a recursive algorithm to mirror a binary tree “along they axis.” Mirroring an empty tree gives back an empty tree. Mirroringa non-empty tree rooted at node n with left sub-tree L and right sub-tree R results in a tree rooted at n with left sub-tree . . . (complete thisdefinition). Write the pseudocode.

6.2.1. Write a recursive program (pseudocode will do) to traverse a binary

tree in postorder.2. Write a recursive program to solve the Towers of Hanoi problem.3. Describe a recursive descent parser for arithmetic expressions.4. Describe a recursive descent parser for Reverse Polish expressions.

6.3. Define the Fibonacci function fib that determines the nth Fi-bonacci number using tree recursion (non-linear recursion) using therecipe “the first and second Fibonacci numbers are both 1, and thenth Fibonacci number for n > 2 is obtained by adding the previoustwo Fibonacci numbers.” Then compute fib(3) by first obtaining thefunction H and using the Y combinator – as illustrated in Section 6.1.4for function Fred. Show all the reduction steps.

6.4. With respect to the function f3 defined in Section 6.2.1, try toarrive at di!erent solutions by returning something non-! instead of! in f3, and see if such variants serve as solutions. If they do, explainwhy; also explain if they don’t.

6.5. Perform other fixed-point experiments (similar to the ones de-scribed in Section 6.1.2) using a calculator. Try to find a fixed-point ineach case. In each case, answer whether the fixed-point is unique (arethere multiple fixed-points for any of the functions you choose?). Tryto find out at least one function that has multiple fixed-points.

6.6. This is Exercise 63, page 89 of [48], where it is attributed to Baren-dregt: show that

(!xy.y(xxy))(!xy.y(xxy))

is a fixed-point operator.

6.7. What is one fixed point of Y itself? (Hint: Straightforward fromdefinition of Y as Y f = f(Y f).)

6.8. Verify the assertion on page 100 (that f1, f2, and f3 are solutionsto F ).


7

Strings and Languages

Now we embark on studying strings of symbols over a given alphabet,and languages, which are sets of strings. An alphabet is a finite col-lection of symbols. Usually, an alphabet is denoted by %. Alphabetsare always non-empty. In most cases, alphabets are finite, althoughthat is strictly not necessary.

We said that an alphabet % is a finite collection of symbols. Whatcan symbols be? They can be letters, such as {a, b, c}, bits {0, 1}, mu-sical notes, Morse code characters, a collection of smoke-signal pu!s,hieroglyphics, kicks you might administer to an errant vending ma-chine, or just about anything we wish to regard as an indivisible unit ofcommunication or activity. We do not worry about the inner structureof a symbol, even if it has one. We emphasize this point early on, lestyou be totally surprised when we start regarding such objects as regularexpressions (expressions that denote languages - they are discussed inChapter 8) as symbols.

Strings are zero or more symbols in juxtaposition. Each symbol froman alphabet can also be regarded as a string of length one. Languagesare sets of strings. We will also use the term sequence instead of“string,” and employ operations on strings on sequences also.

Each string or sequence can model sequences of key strokes youmight type on a computer, a piano, a telegraph machine, etc. We calleach such sequence or string a computation, a run of a machine, etc.A vending machine has a computation consisting of coin plinks, candyejects, and kicks. A formal machine such as a Turing machine has com-putations depending on what program is loaded into it, and what in-put is submitted on its tape. In most chapters, we consider only finitestrings. Beginning with Chapter 21, we will consider automata on in-finite strings. Infinite strings are useful to model certain properties of

106 7 Strings and Languages

reactive systems - systems, such as operating systems that are supposedto run forever (at least at a conceptual mathematical level). Such be-haviors cannot be adequately modeled at a high level using only finitestrings.1

Strings and languages are fundamental to computer science theory.By measuring run times or storage requirements of programs in termsof the lengths of their input strings, we are able to study time andspace complexity. By studying patterns that sets of strings possess, weare able to taxonomize degrees of problem solving abilities (computa-tional “power”). In fact, life seems to be full of strings (pun intended2).We now proceed to systematically examine this central role played bystrings and languages in computer science theory.

7.1 Strings

7.1.1 The empty string "

A string may also be empty, meaning it has zero symbols in it. Wedenote empty strings by ". When the available type fonts (mostly inmechanically-generated illustrations) do not permit, we also write "as e, or epsilon. Empty strings are the identity element for con-catenation, meaning the concatenation of " at the beginning or at theend of a string s yields s itself. Many programming languages denoteempty strings by "". Many books on automata employ ! instead of "(we totally avoid this usage, as for us ! is reserved for use in Lambdaexpressions). Examples of non-empty strings from programming lan-guages are "abcd", "aaa", etc. In automata theory, we shall model thesestrings using abcd and aaa, or in ordinary fonts sometimes quoted—asin ‘abcd’.

To avoid some confusion early on, we stress upon a few basic con-ventions. In a context where we are talking about strings, ‘123’ is astring of length three, consisting of the symbols 1, 2, and 3, and not asingle integer. As an example, when asked to design a DFA that has

1 Of course, by taking timing into account, one can model everything in termsof computations occurring over a finite number of steps—or finite strings. Thepurpose of employing infinite strings is to reason about propositions such as “eventa happens, but following that event b never happens” with due precision. Forthis, the concept of ‘never’ must be modeled in a manner that does not dependon absolute timings. The only way to achieve this is to allow for an infinite stringafter a wherein b is never found to occur.

2 DNA sequences are strings over the symbol-set A, C, T , and G – see “CartoonGuide to Genetics,” by Larry Gonick and Mark Wheelis.

7.2 Languages 107

alphabet {0, . . . , 9} and accepts all numeral sequences divisible with-out remainder by 3, the DFA reads the symbols one by one, and checksfor the familiar condition for divisibility by 3 (the digits add up to anumber divisible by 3). Also, when given the input 126, it reads threesymbols—not the entire number 126 in one fell swoop. These conven-tions mirror the reality that physical machines may never consume anunbounded amount of information in a finite amount of time. The no-tion of a symbol connotes this reality.

7.1.2 Length, character at index, and substring of a string

Given string s, its length is denoted by length(s). We have, for everystring s, length(s) . 0. We of course have length(") = 0.

For each position i , length(s) (i.e., 0 & i & (length(s) / 1)) of astring s, s[i] denotes the character at position i. Notice the following:since length(") = 0, the s[.] notation is undefined, as there are no isuch that i , length(s), if s were to be ".

If i and j are positions within a string s, and j . i, substr(s, i, j)is the substring of s beginning at i and ending at j, both inclusive.If j < i, substr(s, i, j) = ". For instance, substr(apple, 0, 0) = a,substr(apple, 1, 1) = substr(apple, 2, 2) = p, substr(apple, 0, 2) = app,substr(apple, 2, 4) = ple, and substr(apple, 4, 3) = ". For every strings, s = substr(s, 0, length(s) / 1).

7.1.3 Concatenation of strings

Given two strings s1 and s2, their concatenation, written as s1s2, yieldsa string of length | s1 | + | s2 | in the obvious way. Sometimes we writestring concatenation as s1 ) s2 to enhance readability. Examples: appleworm results in appleworm, as does apple ) worm.

7.2 Languages

A language is a set of strings. Like strings, languages are a central no-tion in computer science theory. Questions pertaining to undecidabilityas well as complexity can be studied in terms of languages. The small-est language is 0, since |0| = 0. The next larger language has one stringin it. Of course there are many such languages containing exactly onestring. One example is the language {"}, which contains one string thathappens to be empty. Given the alphabet % = {0, 1}, a few languagesof size 1 are {1}, {0}, {0101}; a few languages of size 2 are {0, 00},


{0, 0101}, etc. Many (most?) of the languages we study will be infinitein size.

7.2.1 How many languages are there?

Are there countably many languages, or uncountably many languages?The answer is that there are uncountably many languages over any al-phabet. In other words, there are as many languages over % = {0, 1} asthere are Real numbers. This is because each language L can be repre-sented by an infinite characteristic sequence which tells which (amongthe infinite strings over % = {0, 1}) are present in L and which areabsent. Since each such characteristic sequence can be read as a realnumber in the range [0, 1], the set of all such characteristic sequencesis uncountable, as explained in Section 3.2.5.

Why does it matter that there are uncountably many languages? Toanswer this question, let us briefly consider how we shall model compu-tations. We shall study computations in terms of languages accepted byTuring machines. Each Turing machine M is a very simple computerprogram written for a very simple model of a computer. M is startedwith an input string x on its tape, and “allowed to run.” Each such runhas three outcomes: M halts in the accept state, it halts in the rejectstate, or it loops (much like a computer program can go into an infiniteloop). The set of all strings x that cause M to halt in the accept stateconstitute the language of M . The language of a Turing machine is, ina sense, the essence or meaning of a Turing machine. Now, each Tur-ing machine M itself can be represented by a single finite string over% = {0, 1} (this is akin to viewing an entire a.out file obtained by com-piling a C program as a single string). Since the set of all finite stringsis countable, there can only be countably many Turing machines, andhence countably many Turing machine languages. Hence, there are lan-guages that are the language of no Turing machine at all. In a sense,these languages (called non-Turing recognizable) carry “patterns” thatare “beyond the reach of” (or “defy analysis by”) any man-made com-putational device. Their existence is confirmed by the above cardinalityarguments.

7.2.2 Orders for Strings

There are two systematic methods available for listing the contents ofsets of words. The first is known as the lexicographic order, or “orderas in a dictionary” while the second is called the numeric order whichlists strings by length-groups: all strings of a lower length are listed

7.2 Languages 109

before a string of a higher length is listed. Within a length-group, thestrings are listed in lexicographic order. We now present some detailsas well as examples.

Lexicographic order for strings

In this order, given two strings x and y, we compare the characterscomprising x and y position by position. We do this for each positioni , min(length(x), length(y)) (here, min finds the minimum of twonumbers). For example, if we compare apple and pig, we compare foreach position i , 3, as pig is the shorter of the two and has length 3.While comparing characters at a position, we compare with respect tothe ASCII code of the character at that position.3

Given all this, for strings x and y such that x %= y, x said to bestrictly before y in lexicographic order, written x <lex y, under one oftwo circumstances:

1. If there exists a position j , min(length(x), length(y)) such thatx[j] < y[j], and for all i , j (meaning, for positions 0 . . . (j /1)), x[i] = y[i]. For example, aaabb <lex aaaca, aaron <lex abate,apple <lex pig, and pig <lex putter.

2. For all positions j , min(length(x), length(y)), we have x[j] = y[j],and length(x) < length(y).

Definition: Define string identity, x =lex y, to hold for two identicalstrings x and y. Also define &lex to be <lex ( =lex, and >lex to be thecomplement of &lex.Definition: Given two strings x and y, x is before y in lexicographicorder i! x &lex y.Let us now go through some examples of lexicographic order:

• " <lex aa follows from condition 2 above. This is because thereare no positions within a length of 0, and so the only condition tobe satisfied is length(x) < length(y), which is true for x %= y andx = ". Since " <lex aa, we also have " &lex aa.

• a <lex aa <lex aaa <lex aaaa . . .. Hence, these are also in the &lex

relation.

3 The ASCII code totally orders all the characters available on modern keyboards.Refer to the web or a book on hardware design to know what this total order is.We are taking this more pragmatic route of relying on ASCII codes to keep ourdefinitions concrete.


Numeric order for strings

The idea of numeric order is of importance while systematically enu-merating sets. To motivate this idea, consider a lexicographic orderingof all strings over alphabet {a, b}. Such a listing will go as follows: ",a, aaa, aaaa, . . .. In other words, not a single string containing a b willget listed. To avoid this problem, we can define the notion of a numericorder as follows. For two strings x and y:

1. If length(x) = length(y), then x <numeric y exactly when x <lex y.2. Otherwise, if length(x) < length(y), then x <numeric y.3. Otherwise (length(x) > length(y)) x >numeric y.

As before, we define <numeric to be strictly before in numeric order, and

&numeric = <numeric ( =lex

to be before in numeric order, or simply “in numeric order.”The numeric order of listing the strings over alphabet {a, b} will yieldthe sequence

", a, b, aa, ab, ba, bb, aaa, aab, aba, abb, baa, bab, bba, bbb, aaaa, . . . .

7.2.3 Operations on languages

Given languages L,L1, and L2, new languages may be defined using var-ious operations. First, consider the set operations of union, intersection,set di!erence, and symmetric di!erence. These familiar set operationshelp define new languages as defined below:

Union: L1 ( L2 = {x | x , L1 * x , L2}intersection: L1 ' L2 = {x | x , L1 + x , L2}Set di!erence: L1 \ L2 = {x | x , L1 + x /, L2}Symmetric di!erence: (L1 \ L2) ( (L2 \ L1)

7.2.4 Concatenation and exponentiation

The concatenation operation on two languages L1 and L2 performsthe concatenation (juxtaposition) of strings from these languages. Ex-ponentiation is n-ary concatenation. These ideas are basically quitesimple. Suppose we have a finite-state machine M1 that accepts stringscontaining either an odd number of a’s or an even number of b’s. Sup-pose we have another finite-state machine M2 that accepts strings con-taining either an even number of a’s or an odd number of b’s. We can

7.2 Languages 111

now build a new machine M whose runs are of the form xy such that xis accepted by M1 and y by M2. We can also build a new machine M "

1whose runs are of the form xx . . . x, where x is repeated some k . 0times, and x is a run of M1. The set of runs of these new machines canbe captured using concatenation and exponentiation.We now formally define the concatenation and exponentiation opera-tions on languages:Concatenation: L1 ) L2 = {xy | x , L1 + y , L2}.Note: Often, we will omit the ‘)’ operator and write L1L2 instead ofL1 ) L2.Exponentiation: First of all, for any language L, define L0 = {"}.That is, the zero-th exponent of any language L is {"}. Note that thismeans that even for the empty language 0, 0" = {"}. (The reason forthis convention will be clear momentarily).

Exponentiation may now be defined in one of two equivalent ways:

Definition-1 of Exponentiation: For all n . 1,

Ln = {x1 . . . xn | xi , L}.

Definition-2 of Exponentiation:

For n > 0, Ln = {xy | x , L + y , Ln!1}.

In the second definition, note that we should define the basis case forthe recursion, which is L0. We must put into L0 anything that servesas the identity element for string concatenation, which is ". Hence, wedefine L0 = {"} for any language, L.Consider a simple example:

{a, aba} ) {ca, da} = {aca, ada, abaca, abada},

while{ca, da}3 = {ca, da} ) {ca, da} ) {ca, da},

which is the set

{cacaca, cacada, cadaca, cadada, dacaca, dacada, dadaca, dadada}.

Consider another example where L1 (which models the runs of M1) isthe set

L1 = {x | x is a sequence of odd number of a$s or even number of b$s}

and L2 (which models the runs of M2) is the set


L2 = {x | x is a sequence of even number of a$s or odd number of b$s}.

Then, L1 ) L2 is a language in which each string consists of

• an odd number of a’s,• an odd number of b’s,• an odd number of a’s followed by an odd number of b’s, or• an even number of b’s followed by an even number of a’s.

In addition, Lk1 is a language in which each string can be obtained as

a k-fold repetition of a string from L1.

7.2.5 Kleene Star, ‘#’

Star performs the union of repeated exponentiations:

L" = {x | #k , N : x , Lk}.

This definition can also be written as

L" = (k'0 Lk,

or even asL" = (k#Nat Lk.

Notice that according to these definitions, 0" = {"} This is because forany k, 0k is defined to be {"}. In turn, this is so because we must havea basis element in the recursive definition of concatenation. In our case," is the identity for string concatenation.Examples:

• {a, b, c}" is all possible strings over a, b, and c, including the emptystring ".

• {a, bc}" is all possible strings formed by concatenating a and bc insome order some number of times. In other words, all the strings in{a, bc}k , for all k, are present in this set.

In a programming sense, star is iteration that is unable to keep trackof the iteration count. The ability to “forget” the number of iterationsis crucial to obtaining simpler languages, and ultimately, decidability.

7.2.6 Complementation

Using the above-defined notion of star, we can now specify a universe%" of strings, and define complementation with respect to that uni-verse:

L = {x | x , %" \ L}.

7.2 Languages 113

7.2.7 Reversal

The notion of reversing a string is pretty straightforward - we simplyput the first character last, the second character penultimate, and soon. The definition of reverse(s) for s being a string in some alphabet,is as follows:

reverse(") = "reverse(s as ax) = reverse(x) ) a.

We use the as construct to elaborate a quantity. ‘s as ax’ means‘view string s as a concatenation of symbol a and string x.’ The term‘reverse(x) ) a’ says ‘reverse x and append a to its end.’

For language L, reverse(L) is then {reverse(x) | x , L}.

7.2.8 Homomorphism

A homomorphism is a function that maps strings to strings and thatrespects string concatenation. In other words, if h : %" ! ) " is ahomomorphism that takes strings from some alphabet % to stringsin an alphabet ) (not necessarily distinct), then it must satisfy twoconditions:

• h(") = "• For string s = xy in %", h(xy) = h(x)h(y). In other words, the

result of applying h to s is the same as the result of concatenatingthe application of h to its pieces x and y.

To obtain a better appreciation for the fact that a homomorphism“respects string concatenation,” let us consider something that is nota homomorphism—say g. Let % = {a, b, c, d} and ) = {0, 1, 2}.

• g(abcd) = 0• g(ab) = 1• g(cd) = 1• g(s) = 2, for all other strings.

g is not a homomorphism because g(abcd) = 0 while g(ab)g(cd) is 11.

7.2.9 Ensuring homomorphisms

How do we ensure that a given function on strings is a homomorphism?The most commonly used approach to ensure that something is a ho-momorphism is to specify a mapping that goes from symbols to strings,


and then to “lift” it up to map from strings to strings. Let h!

be thesymbol to string map, with signature

h!: % ( {"} ! ) ",

and define h using h!. Here is an example. Let h

!be as follows:

• h!(") = "

• h!(a) = 0

• h!(b) = 1

• h!(c) = h

!(d) = 2

Formally, h(s) is defined in terms of h!as follows:

• If length(s) > 1, then let x %= " and y %= " be such that s = xy.Then, h(s) = h(x)h(y).

• If length(s) = 1, then h(s) = h!(s).

Now, by definition, h respects string concatenation and all works outwell!Homomorphisms can also be defined over languages in a straightfor-ward manner. Given L,

h(L) = {y | h(x) = y for some x in L}.

7.2.10 Inverse homomorphism

Given a homomorphism h, an inverse homomorphism h!1 maps a stringy to the maximal set of strings that y could have come from; formally:

h!1(y) = {x | h(x) = y}.

Example: Consider h to be

h!(") = "

h!(a) = 0

h!(b) = 1

h!(c) = h

!(d) = 2

Then, h!1(012) = {abc, abd} because 2 could have come either from cor from d. Inverse homomorphisms can also be defined over languagesin a straightforward manner. Given L,

h!1(L) = (y#L h!1(y).

In other words, for each y , L, take h!1(y), and then take the unionof all these sets.

7.2 Languages 115

7.2.11 An Illustration of homomorphisms

In parsing a programming language such as C, the parser appeals to atokenizer that first recognizes the structure of tokens (sometimes calledlexemes) such as integers, floating-point numbers, variable declarations,keywords, and such. The tokenizer pattern-matches according to reg-ular expressions, while the parser analyzes the structure of the tokenstream using a context free grammar (context free grammars are dis-cussed in Chapters 13 and 14; basically, they capture the essential lexi-cal structure of most programming languages, such as nested brackets,begin/end blocks, etc.). Suppose one wants to get rid of the tokenizerand write a context free grammar up to the level of individual char-acters. Such a grammar can be obtained by substituting the characterstream corresponding to each token in place of each token in a modularfashion, according to a homomorphism. For example, if begin were tobe a keyword in the language, instead of treating it as a token keyword,one would introduce additional productions of the form

begin -> b e g i n

Thanks to the modular nature of the substitutions, it can be shownthat the resulting grammar would also be context-free.

7.2.12 Prefix-closure

A language L is said to be prefix-closed if for every string x , L, everyprefix of x is also in L. If we are interested in every run of a machine,then its language will be prefix-closed. This is because a physically real-izable string processing machine must encounter substrings of a stringbefore it encounters the whole string. Prefix closure is an operation that‘closes’ a set by putting in it all the prefixes of the strings in the set.For instance, given the set of strings {a, aab, abb}, its prefix closure is{", a, aa, ab, aab, abb}.

Chapter Summary

This chapter introduced the motivations for studying computations interms of strings and languages. It put our previous discussions aboutcardinality in Chapter 3 to very good use by quickly showing the factthat there are uncountably many languages. After defining various or-derings between strings, we discuss operations on languages: how tomake new languages given existing languages. These discussions willset the stage for virtually all of what the rest of this book involves.


Exercises

7.1. How many languages over an alphabet of size 1 exist? Expressyour answer using the < notation.

7.2. Using the Schroder-Bernstein Theorem, argue that there are asmany languages over the alphabet {0} as there are over the alphabet{0, 1}.

7.3. Name the main di!erence between the proof of the uncountabilityof languages versus the uncountability of the set of Reals.

7.4. Choose among the sets given below, those with cardinality <2 (re-call that Nat has cardinality <0):

1. The set of all languages over alphabet % = {0, 1}.2. The powerset of the set of all languages over alphabet % = {0, 1}.3. The set of all languages over alphabet % = {0}.4. The powerset of the set of all languages over alphabet % = {0}.

7.5. Prove that &lex is a total order, while <lex is not.

7.6. Viewing C programs as ASCII strings, obtain a good estimate ofthe number C programs of less than n bytes, for n & 12? Obtain atleast five such C programs; compile them and demonstrate that theydo not cause any compilation errors.

Hint: main(){;} is one such program, with byte count 9. View eachC program as a string (so you could write "main(){;}" for clarity, ifyou wish). In later discussions of strings, we will omit the quotes (")which serve as string delimiters.

7.7. What is the set consisting of the first ten words in English oc-curring in lexicographic order? (Follow any dictionary you may haveaccess to. Treat the upper and lower cases the same).

7.8. What is the set consisting of the first ten words in English oc-curring in numeric order? (Follow any dictionary you may have accessto).

7.9. Suppose that in a strange language, ‘Longspeak’, typeset in char-acters a through z, there are words of all lengths beginning with anyletter. Consider a lexicographic order listing of all words in Longspeak.At what rank (ordinal position) does a word beginning with letter dappear? How about for a numeric order listing?

Considering the answers to these questions, suppose you are chargedwith the task of producing a dictionary for Longspeak. Fortunately, you

7.2 Languages 117

are told that you don’t have to include words of lengths beyond 30.Would you produce your dictionary in lexicographic order or numericorder? Explain your reasoning.

7.10. Define languages L1 and L2 over alphabet % = {0, 1} as follows:

L1 = {x | x is a string over % of odd length}

andL2 = L1 ' {x | x is a string of 0$s}.

What is the symmetric di!erence of languages L1 and L2?

7.11. Among the following languages, choose the ones for which LL =L:

• L = 0.• L = {"}.• L = {x | x is an odd length string over{0, 1}}.• L = {x | x is an even length string over{0, 1}}.• L = {x | #0(x) = #1(x)}; here, #0 determines the number of 0sin a given string, and likewise for #1.• L = {x | #0(x) %= #1(x)}.

7.12. Repeat Exercise 7.11 to meet the condition L" = L.

7.13. Repeat Exercise 7.11 to meet the condition L" = LL = L.

7.14. Complement each of the languages defined in Exercise 7.11 andexpress your answers using the set comprehension notation.

7.15. Consider the languages

• L1 = {x | x is an odd length string over{0, 1}}, and• L2 = {x | x is an even length string over{0, 1}}.

1. Argue that the following map is a homomorphism (call it h):h(") = "h(0) = 0h(1) = 00.

2. Determine h(L1) ' h(L2).

7.16. In Exercise 7.11, find those languages in this list such that L =pref(L).

8

Machines, Languages, DFA

This chapter begins our study of machine models. We will first brieflyexamine the general characteristics of various abstract machines thatcan serve as language recognizers for the languages depicted in Fig-ure 4.2 of Section 4.3. We will refer to this diagram as the “Power”diagram. We will then study the Deterministic Finite-state Automaton(DFA) model in the rest of the chapter.

8.1 Machines

All the machines discussed in the Power diagram, including DFAs dis-cussed in this chapter, are primarily string processors. They are startedin their initial state and are fed an input string. The inputs of inter-est are finitely long (“finite”) strings. The main question of interest iswhether, at the end of the input, the machine is in an accepting state ornot. The Turing machine has, in addition to a set of accepting states,a set of rejecting states.

There are several axes along which machines can be distinguished.Some of these distinctions are now pointed out. We will remind thereader of these details when specific machine types are discussed. Thepurpose of the current discussion is to portray the overall nature of thevariations that exist among machines.

All machine types in the Power diagram that are of lower power thanthe LBA (namely the DFA, NFA, DPDA, and NPDA) must read theirinput strings completely before it is deemed that they accept the inputstring. Furthermore, they must read their input string in the process ofa single left-to-right scan (meaning, they cannot go back and reread apreviously read symbol). In each step of computation, these machines

120 8 Machines, Languages, DFA

are allowed to read atmost one input symbol (they may read none, aswell).

The remaining machine types in the Power diagram, namely theLBA, DTM, and the NDTM, are not required to read the whole inputon their input “tape.” In other words, it is possible for these machinesto accept an input without having read all of it—or for that matter,1

without having read any of it! Furthermore, they are allowed to rereada previously read input. This rereading may happen in any order (i.e.,not confined to the order in which inputs are laid out on the tape).

All the machines in the Power diagram possess a finite-state controlflow graph describing their behavior. The NFA, NPDA, and NDTMallow nondeterminism in their control flow graph (nondeterminism isdescribed in the next section). While the DFA and NFA are totallydescribed in terms of this finite-state control, the remaining machinetypes carry an additional data structure in the form of a stack or one ormore tapes. The DPDA and NPDA have access to a single unboundedstack from which they pop a single symbol at every step. Additionally,in each step, zero or more symbols may be pushed back onto the stack.

The LBA, the DTM, as well as the NDTM have, in addition to acontrol flow graph, a single read/write tape. There is the notion of acurrent position on this input tape, usually specified by the position ofa read/write “head.” Also, there is the notion of presenting the inputstring over a contiguous sequence of cells on the input tape. The LBAis allowed to write a tape cell only if this cell belongs to the contiguoussequence of cells over which the initial tape input was presented. ATuring machine has no such restriction.

Note that we avoid depicting the Buchi automaton in the Powerdiagram of Chapter 3, as this machine type is not comparable withthese other machine types. For example, one can simulate an LBAon a Turing machine, or a DFA on a push-down automaton; no suchsimulation of the Buchi automata on any of the machines in the Powerdiagram is possible. In a sense, the presentation of Buchi automata isa way of making the reader aware of the existence of these machinesand the practical uses that these machines have in modeling time inan abstract manner, as well as broaden the reader’s perspective earlyon. Details on Buchi automata appear in Chapters 22 and 23. It is alsoimportant to point out that due to space/time limitations, we have leftout many machine types on finite strings that could easily have been

1 While this sounds odd, this freedom is required when modeling machines withpower comparable to general-purpose computers, which may, as we all know, oftenignore reading their input entirely.

8.1 Machines 121

depicted on the Power diagram, such as two-way automata [108]. Manyof these machine have been proved equivalent to machines in the Powerdiagram, as far as the class of languages they recognize is concerned.

With this general introduction, we proceed to examine the variousmachine types beginning with the DFA.

8.1.1 The DFA

A deterministic finite-state automaton (DFA) is specified through atransition graph such as in Figure 8.1. It starts its operation in an initialstate (in this case I). When a DFA is situated in one of its reachablestates s and is fed an input string w, it advances to a unique state as perits transition graph. Formally, a deterministic finite-state automatonD is described by five quantities presented as a tuple, (Q,%, *, q0, F ),where:

Q, a finite nonempty set of states;%, a finite nonempty alphabet;* : Q 2 % ! Q, a total transition function;q0 , Q, an initial state; andF 5 Q, a finite, possibly empty set of final (or accepting) states.

I

0

F10

1

Fig. 8.1. A DFA that recognizes strings over {0, 1} ending with 1

For example, in Figure 8.1, Q = {I, F}, % = {0, 1}, q0 = I, F = {F},and * = {6I, 0, I7, 6I, 1, F 7, 6F, 0, I7, 6F, 1, F 7}. A few useful conventionspertaining to drawing DFAs as well as naming their states are nowdescribed (these conventions will be followed in many of our initialdrawings, and possibly later also):

• The initial state will be called I. In drawings, it will be the state towhich an arrow without a source node points to.

• If the initial state is also a final state, it will be called IF. It willthen be a double-circled node.

• All final state names (other than the state named as IF, as explainedabove) will begin with letter F, and will be double-circled nodes.


/* Encoding of a DFA that accepts the language (0+1)*1 */main(){ bool state=0;char ch;while(1){ ch = getch();switch (state){ case I: switch-off the green light;

switch (ch){ case 0 : break;case 1 : state=F;

}case F: switch-on the green light;

switch (ch){ case 0 : state=I;case 1 : break;

}}

}}

Fig. 8.2. Pseudocode for the DFA of Figure 8.1

I

0

F1

0,1

Fig. 8.3. Multiple symbols labeling a transition in lieu of multiple transitions

I

0F

1

X 0,1

0

1Y 0,1

Fig. 8.4. A DFA with unreachable (redundant) states

• All other state names begin with letters other than I and F, and arebest chosen with mnemonic significance. They are drawn as singlecircles.

8.1 Machines 123

• Generally one would expect a DFA to decode every move out ofevery state. If, for expediency, we leave out some of these moves, werecommend that the following comment accompany such diagrams:“we assume that all moves not shown lead the DFA to the black

hole state.” We recommend that the state name BH (for black hole)be used for that state from which no escape is possible.2

• One could have a DFA with multiple symbols labeling a single tran-sition in lieu of separate transitions that bear these symbols, asshown in Figure 8.3.

• One could have a DFA with truly unreachable states, as shown inFigure 8.4. These states (states X and Y in this example) may beremoved without any loss of meaning.

IF

FA0

FB

1

0

A11

1

B00

0

1

1

0

Fig. 8.5. Another example DFA

One can see the conventions explained above more clearly in Figure 8.5.The DFA in Figure 8.1 is really encoding programs similar to the

one in Figure 8.2. All “DFA programs” are while(1) loop controlledswitch structures. While in a certain state, a DFA decodes the currentinput symbol and decides to either update its state or keep its currentstate. The next input symbol is decoded in this (possibly) updatedstate. Whenever any final state (including the IF state) is entered, theDFA turns on its green light. The green light stays on so long as theDFA stays in one of the final state, and is turned o! when it exits it. ADFA program is not allowed to use any other variable than the singlevariable called state. It cannot produce any other output than shinethe green light. It appears to be programming at its simplest - yet, suchhumble DFAs are enormously powerful and versatile!

2 This is similar to gravitationally hyper-dense stars from which nothing - includinglight - escapes. We did vote down the alternate name RM (for ‘Roach Motel’)proposed by some students.


The * function of a DFA is also commonly presented as a table, withrows being states, columns being inputs, with the final state(s) starred.For our example, we have

input-----

state| 0 1------------I | I F*F | I F

A DFA is said to accept a string s if its green-light is ‘on’ after pro-cessing that string; all other strings are rejected by the DFA. However,note that a DFA “never stops working”; after processing any string s, itwill be in a position to process any one of the input symbols a, and may(depending on the DFA) accept or reject that string, sa. DFAs can bethought of as string classifiers, assigning a status (accepted/rejected)for every string within %". The set of strings accepted by a DFA issaid to be its language. A DFA is said to recognize its language (notaccept its language - the term ‘accept’ is used for individual strings).

Formally speaking, a DFA is said to accept an input x , %" if xtakes it from q0 to one of its final states. Define a function * that mapsa state and a string to a state, as follows:

For a , % and x , %", *(q, ax) = *(*(q, a), x)*(q, ") = q.

Then, a DFA accepts x exactly when *(q0, x) , F .An alternate definition of the language of a DFA is based on the

notion of instantaneous descriptions (IDs). An ID is a “snapshot” ofa DFA machine in operation. For a DFA, this snapshot contains thecurrent state and the unconsumed input string. Knowing these, it ispossible to predict the entire future course of a DFA.

Formally speaking, we define the type of the ID of a DFA to beTID DFA = Q 2 %". In other words, we take the view that the type ofthe ID, TID DFA, includes all possible 6state, string7 pairs, i.e., Q2%".We now define two relations, - and -"; the latter being the reflexiveand transitive closure of the former. - is a one step move function thatcaptures how a DFA moves from one ID to another.

Suppose id1 is one snapshot, (q1,+1), and id2 is another snapshot,(q2,+2), such that +1 = a+2. Now, if *(q1, a) = q2, we know that theDFA can go from snapshot (q1,+1) to (q2,+2). In other words, definethe relation - 5 TID DFA 2 TID DFA. as follows:

- (id1 as (q1,+1), id2 as (q2,+2)) =

8.1 Machines 125

#a , % : +1 = a+2 + *(q1, a) = q2.

In the infix syntax,

(q1,+1) - (q2,+2) i! #a , % : +1 = a+2 + *(q1, a) = q2.

As said before, define -" is defined to be the reflexive transitive closureof -. In other words, -" is the smallest relation 5 TID DFA 2 TID DFA

closed under the following rules:

6x, x7 , -" for x , TID DFA;In other words, x -" x for x , TID DFA,6x, y7 , - + 6y, z7 , -" 3 6x, z7 , -";In other words, x - y + y -" z 3 x -" z,

Now, for DFA d, its language

L(d) = {x | #q , F : (q0, x) -" (q, ")}.

The above definition says that the language of a DFA consists of allstrings x such that -" can take the ID (q0, x) to the ID (q, "), whereq , F . Notice that the second ID contains ", indicating that the DFAhas consumed the entire string x.Example: Consider the DFA in Figure 8.3. What is in the - relationwith respect to snapshots that involve I and F? Since this is an infiniteset, we show a few members below (we use 6 and 7 as well as ( and )to delimit tuples, to enhance readability):

{6(I, 1), (F, ")7, 6(I, 111), (F, 11)7, . . . 6(I, 1010100110), (F, 010100110)7, }.

Now we show a few members of -":

{6(I, 1111), (F, ")7, . . . 6(I, 1010100110), (F, 0110)7,6(I, 1010100110), (F, 010)7, 6(I, 1010100110), (F, 01)7,6(I, 1010100110), (F, ")7, . . .}.

Note that -" can “consume” multiple symbols.Some definitions, tips, and conventions:

A language L is defined to be regular if there is a DFA D such thatL = L(D).While it is helpful to leave out those transitions of a DFA thatlead to the black hole - BH - state, please remember to reinstate theblack hole state and all the transitions going to this state before youperform DFA operations, such as complementation.3

3 Otherwise, you will find yourselves in a ‘DH’ - deep hole.


8.1.2 The “power” of DFAs

Despite their simple appearance, DFAs are incredibly powerful andversatile! Here are some examples of their power:

The most touted applications of DFAs are in the area of compilerscanner generation, where a scanner is a program that recognizes thepatterns hidden within strings representing keywords, floating-pointnumbers, etc.Exercises 8.16 and 8.17 demonstrate that using DFAs, one can per-form division either going MSB-first (the ‘high-school’ method) oreven LSB-first. Only a finite amount of information needs to beremembered across digits (essentially, the remainder after division;fully explained in these exercises).DFAs can be used to compactly represent Boolean functions (Chap-ter 11) or even help determine the validity of formulas in certainbranches of mathematical logic, known as Presburger arithmetic(Chapter 20).

Most questions about DFAs are algorithmically decidable. In particu-lar, the following questions about DFAs are decidable:

Does a given DFA accept a given input?Does a given DFA have an empty language?Is the language of a given DFA all possible strings over %?Are two given DFAs equivalent?

DFAs have many desirable properties, some of which are the following:

Given any DFA d, one can algorithmically obtain another DFA d!

whose language is the reverse of the language of d.Given any DFA d, one can algorithmically obtain another DFA d

!

whose language is the complement of the language of d.

8.1.3 Limitations of DFAs

DFAs have two main limitations. First, despite being adequately ex-pressive, they may require “too many” (an exponential number) states,which does adversely a!ect the space/time requirements of DFA-basedalgorithms. Second, for classifying strings based on many frequentlyoccurring patterns, DFAs are simply inadequate. For instance, it isimpossible to build a DFA that accepts all and only those strings con-taining an equal number of 0s and 1s. See Exercises 8.19, 8.20, and 8.21.We shall soon see that using the concept of nondeterminism, we canobtain exponentially more succinct finite-state representations, giving

8.1 Machines 127

rise to the next machine type we shall study, namely, nondeterministicfinite automata (NFA).

Considering Exercise 8.20, one can easily prove that there can existno DFA. A proof sketch goes as follows:

Suppose there exists an M -state DFA for this language. Considerthe string w = (M )M .w causes M + 1 states to be visited even while processing (M , astring of M left parentheses.Since only M of these states can be distinct states, one state repeatsduring the (M traversal.Therefore, there also exists a shorter, non state-repeating path lead-ing to the same final state. However, taking this path causes theDFA to omit one or more (, thus causing it to accept (k)M for somek < M – a contradiction with the fact that this is the DFA for thelanguage L.

It is obvious that to recognize a language such as L above, all we needto do is add a single stack to the DFA D. Doing so, we obtain a machineknown as deterministic push-down automaton (DPDA). In general, byadding di!erent kinds of data structures to the finite-state control, onecan handle languages whose strings have more complicated structures.In Section 8.1.4, we shall now present a few examples that illustratethis point.

8.1.4 Machine types that accept non-regular languages

Consider the language

L1 = {aibjck | i, j, k . 0 + if i = 1 then j = k}.

In all the strings in this language, the characters a, b, and c appear inthat order, with b’s and c’s being equal in number if the number of a’sis 1. This language can obviously be handled using a DPDA, using itsstack. In fact,

reverse(L1)

can also be handled using a DPDA, as the finite-state control can re-member whether the number of b’s and c’s were equal by the time thea (or a’s) appear.

Now consider

L2 = {aibjckdm | i, j, k,m . 0 + if i = 1 then j = k else k = m}.


This language can still be processed using only one stack, as the match-ing between b’s and c’s or c’s and d’s (whichever is to occur) can bedecided by first seeing the a’s. No such luck awaits

reverse(L2),

which has to have two decisions, (whether the number of b’s and c’smatch, or whether the number of c’s and d’s match), in hand by the timethe a (or a’s) arrive. Clearly, these two decisions cannot be generatedusing a single stack, thus showing that reverse(L2) cannot be processedby a DPDA. The same can be concluded about

L3 = {ww | w , {0, 1}"}

also, as the second w’s head must be matched against the first w’s headwhich, unfortunately, is at the bottom of the stack. It turns out (as weshall demonstrate later) that

L4 = L3

indeed can be processed using a single stack push-down automaton.One obvious solution to handling L3, as well as reverse(L2), would

be to employ two stacks instead of one. Unfortunately this gives morepower than necessary (the machine becomes as powerful as a Tur-ing machine). Another machine type, the linear bounded automaton(LBA), can be used. An LBA has finite control as well as a tape suchthat it can read/write only the region of the tape in which the inputinitially appeared. In addition, an LBA has a finite tape alphabet thatmay (and typically does) contain %, the input alphabet, plus (typi-cally) many additional symbols. Exploiting the ability to repeatedlyscan the input, an LBA can decide membership in all the languageslisted above. By the same token, an LBA can decide membership inthe language

{anbncn | n . 0}

as well as{anbn2 | n . 0}.

We later present languages that cannot be decided using LBAs. To han-dle the full generality of languages, we remove the restriction on LBAsthat they can write only where the input is presented, thus obtaininga Turing machine.

8.1 Machines 129

digraph G {/* Defaults */fontsize = 12;ratio = compress;rankdir=LR;

/* Bounding box */size = "4,4";

/* Node definitions */I [shape=circle, peripheries=1];F [shape=circle, peripheries=2];"" [shape=plaintext];

/* Orientation */orientation = landscape;

/* The graph itself */"" -> I;I -> I [label="0"];I -> F [label="1"];F -> F [label="1"];F -> I [label="0"];

/* Unix command: dot -Tps exdfa.dot >! exdfa.ps *//* For further details, see the ‘dot’ manual */}

Fig. 8.6. Drawing DFAs using dot

% Include everything till %--- in a latex document, and run latex\begin{figure}{\hfill {\psfig{file=exdfa.ps,height=4.5cm,width=2cm,angle=-90}\hfill}}\caption{Whatever caption you desire}\label{fig:label-for-cross-referencing}\end{figure}%---

Fig. 8.7. Processing .ps files in Latex

8.1.5 Drawing DFAs neatly

We close this chapter o! with some pragmatic tips for DFA drawing. Al-most all drawings in this book are created using the dot package whichis freely downloadable as part of the graphviz tools. After downloadingand installing this package, you will see an executable file dot. You maythen present your drawings in a file with extension .dot, and processthe drawing into a postscript file for inclusion into your documents, in-cluding Latex documents, where you may further process the drawing.For example, Figures 8.6 and 8.7 provide all the commands needed togenerate Figure 8.1.


Chapter Summary

This chapter began with a brief overview of various machine types aswell as some key di!erences between them. It then introduced determin-istic finite state automata (DFA). Various notations for representingDFA were discussed: tables, diagrams, and mathematical structures.After introducing the notion of acceptance of strings by DFA, the no-tion of a language L being regular was defined; there is at least oneDFA that accepts all and only those strings in L. The chapter closedwith a description of some of the limitations of DFAs in terms of be-ing able to serve as recognizers for certain (non-regular) languages. Wealso pointed out how having one stack allows us to handle languageswith more interesting structure such as balanced parentheses. We alsopresented a result that we shall see again much later; having two stacksgives a machine based on finite-state control as much power as a Turingmachine.

ExercisesYou are encouraged to use grail and/or JFLAP to check your resultsin these exercises.

8.1. Present the pseudocode of an algorithm to reverse a directed graphrepresented by an adjacency matrix. A directed graph is reversed byreversing all its edges.

8.2. What is the language of the DFA in Figure 8.1 if:

the only accept state is the starting state?there are no accepting states (no double-circled states)?we reverse every arc, rename state I as state F, and rename state Fas state I, making it the initial state?

8.3. What is the language recognized by the DFA in Figure 8.5? Listten strings in its language and ten not in its language in lexicographicorder.

8.4. Draw a DFA that is di!erent from (non-isomorphic to) the DFAshown in Figure 8.5, however with the same language.

8.5. What happens to the language of the DFA in Figure 8.5 if A1jumps to B0 upon receiving a ‘1’, and B0 jumps to A1 upon receivinga ‘0’?

8.6. Draw a DFA having language equal to the union of the languagesof the DFAs in Figure 8.1 and Figure 8.5.

8.1 Machines 131

8.7. Draw a DFA having language equal to the symmetric di!erence ofthe languages of the DFAs in Figure 8.1 and Figure 8.5.

8.8. Apply the following homomorphism to the language of the DFAof Figure 8.1 and draw a DFA for the resulting language. Repeat forthe DFA of Figure 8.5.

h(") = "h(0) = 11h(1) = 00

8.9. Obtain a DFA for the prefix-closure of the language of the DFA ofFigure 8.1. Repeat for the DFA of Figure 8.5.

8.10. Draw a DFA having language equal to the concatenation of thelanguages of the DFAs in Figure 8.1 and Figure 8.5.

8.11. Draw a DFA having language equal to the star of the language ofthe DFA in Figure 8.1. Repeat this exercise for the DFA of Figure 8.5.

8.12. Draw a DFA having language equal to the reverse of the languageof the DFA in Figure 8.1. Repeat this exercise for the DFA of Figure 8.5.

8.13. Draw a DFA having language equal to the complement of thelanguage of the DFA in Figure 8.1. Repeat this exercise for the DFA ofFigure 8.5.

8.14. What happens if, in the DFA in Figure 8.1, every transition car-ries label ‘1’? Think of what the corresponding DFA program will do.

8.15. Give a DFA accepting the set of strings over % = {0, 1} suchthat each block of five consecutive symbols contains at least two 0s.

8.16. Develop a DFA that recognizes the following language:

L = {x | x , {0, 1}" and (||x|| mod 5) = 0}.

Here ||x|| stands for the magnitude of x viewed in unsigned binary.Example: || 0101 ||= 5, with the leftmost 0 coming first. Hint: Noticethat the bits of x are being presented MSB-first. Act on each newbit and recalculate the remainder and remember it in the DFA states.Think of what happens to a number, say N , when it gets shifted leftand a least significant bit b gets inserted. Basically N becomes 22N+b.


8.17. With respect to the language L of Exercise 8.16, design a DFA forrev(L), the set of strings which are reverses of strings in L (this is tan-tamount to processing the bit stream LSB-first, as opposed to L whichprocesses it MSB-first). Example: || 0101 ||= 5, with the rightmost 1coming first. Hint: N becomes b2 2m +N where m is the position intowhich the new MSB b walks in. N mod 5 becomes (b2 2m +N) mod 5.We need a recurrence that keeps track of 2m for various m. Each DFAstate remembers a pair(“powers of 2 mod 5”, “the mod of the number mod 5”).A few steps are:(1,0) - 1 -> (2,1) - 1 -> (4,3) - 1 -> (3,2).

In general, the evolution of the state goes as follows:

State N mod 5 /!(b 2 (Weight mod 5) + N mod 5) mod 5.

Now, Weight mod 5 is maintained as follows:

2m mod 5 /! (2 2 (2m mod 5)) mod 5.

A few of the state transitions are given below. Complete the rest.

(1,0) - 0 -> (2,0) (2,0) - 0 -> (4,0) (2,1) - 0 -> (4,1)(1,0) - 1 -> (2,1) (2,0) - 1 -> (4,2) (2,1) - 1 -> (4,3) ...

8.18. Repeat Exercises 8.16 and 8.17 for k instead of 5 and for arbitrarynumber base b (with % suitably adjusted) instead of binary. Obtain thestate transition relation in both these cases.

8.19. Design a DFA for the following language, for various n (up to thelimit of your time availability; do it at least up to n = 4):

{x0z | x , {0, 1}" and z , {0, 1}n}.

Prove that the required DFA has exponentially many states (exponen-tial in n).

8.20. Either design a DFA for the following language

L = {(n )n | n . 0},

or prove that no such DFA can exist! Note: Since this problem’s proofwas sketched on page 127, please come up with another proof thatconsiders taking the state-repeating path (perhaps more than once).

8.21. Write clear pseudocode for a stack-based algorithm to recognizestrings in L above.

9

NFA and Regular Expressions

In the last chapter, we have seen many examples of the versatility of reg-ular languages. We have also seen several examples of regular languagesfor which the DFAs were both simple and intuitive. In Chapters 11 and20, we will further elaborate on the power of DFAs by showing thatthey can be used to encode as well as reason about statements fromcertain decidable branches of mathematical logic. In Section 8.1.3, wealso discussed

some of the limitations of DFAs. Their main limitation is, of course,that they cannot serve as recognizers for non-regular languages such asL = {(n )n | n . 0}, and these languages are very important in com-puter science. In fact, they are part of the syntax of most computer pro-gramming languages. In later chapters, we will develop rigorous prooftechniques for proving that certain languages are not regular. We willthen study machines that are strictly more powerful than DFAs.

In this chapter, we continue the study of regular languages, and inthat context, the main limitation of DFAs is that they can be unnatu-ral,1 exponentially large or both. To overcome these limitations, in thischapter we will introduce a new machine type called nondeterministicfinite automata, or NFA. NFAs subsume DFAs; in fact, every DFA isan NFA (more precisely said, every DFA can, with a small adjustmentof mathematical definitions, be regarded as an NFA). We also wish topoint out that nondeterminism is one of the central ideas in theoreti-cal computer science, going well beyond its role as a ‘description sizecompressor.’ In this book, we will repeatedly be revisiting the conceptof nondeterminism in the context of various machine types.

1 DFAs can end up being unnatural with respect to highlighting the essential struc-ture of a regular language, as will be illustrated in Illustration 9.2.2.

134 9 NFA and Regular Expressions

NFA are specified much like DFA are: through pictures of directedgraphs (for human consumption), or through a description of the * func-tion, say through a tabular function description (for machine consump-tion). However, often a textual syntax is preferred over these styles.This is precisely what regular expressions are. In other words, regularexpressions allow us to write down mathematical expressions that de-note regular languages, and each such regular expression has a verystraightforward interpretation as an NFA. All regular languages overan alphabet % can be specified using just the primitive regular expres-sions 0, ", and a , %, and the regular expression building operators(, ) and $.To sum up, the key results we are aiming to establish by the end ofChapter 12 are as follows:

• Regular languages are those recognized by a deterministic finiteautomaton.

• For every DFA, there is an equivalent NFA; for every NFA there isan equivalent RE; and for every RE there is an equivalent DFA.

• For any regular language, the minimal DFA—the DFA with thesmallest number of states that serves as a recognizer for the language—is unique.

• Regular languages are closed under many operations, including:/ union, intersection, complementation, star/ homomorphism, inverse homomorphism/ reversal, prefix-closure.

In this chapter, we will introduce NFAs and REs through several exam-ples. We explain how NFA process strings by moving from sets of statesto other sets of states either by reading a symbol from an alphabet orwithout reading any symbols (through " moves). We explain how todetermine the language of an NFA using the notion of Eclosure.

We strive to provide an experimental flavor to this chapter by em-ploying the grail tool suite for illustrations. In Section 9.3.6, we illus-trate through a case study how to use the grail tool suite to design andformally debug regular expressions. Virtually all the diagrams in thisbook are generated using grail and a graph drawing package calleddot. We also try to relate the material to the real world by demon-strating how a simple lexical analyzer can be built using the flex tool.

9.1 What is Nondeterminism? 135

9.1 What is Nondeterminism?

Nondeterminism has many uses in computer science. It can help de-signers describe the systems they are about to build even at stagesof the design where they have not determined all the details. In somecases, while these details may be known to designers, they may stillchoose not to include them to avoid inundating their audience withexcess information.2

To understand nondeterminism in a real-world context, consider theact of summoning an elevator car in a busy building that has multipleelevators. After one presses the call button, it is not entirely predictablewhat will happen. One may, if lucky, get a car headed in the same di-rection as they intend to travel. If unlucky, one would get a car goingin the opposite direction, or a car that is full, etc. If one had perfectknowledge about the entire building and its occupants, they could pre-dict the outcome with certainty. However, most people3 do not want tokeep track of all such information, instead preferring to live accordingto a nondeterministic protocol that goes as follows: “if I am fated notto get an empty car, I should try again.” In short, by employing nonde-terminism, one can write system descriptions at a high level, withoutworrying about pinning down details too early. It has been said thatpinning down details too early (“premature optimization”) is at theroot of all that is evil in software design.

Nondeterministic descriptions have another property: they tendto over-approximate the system being described. Over-approximationhelps ignore special cases in the behavior of a system (it is akin topackaging a delicate, but odd-shaped, electronic gadget by insertingit between a pair of molded Styrofoam carriers, thus smoothening theoverall appearance). By adding behaviors, over-approximation oftenhelps “round” things up, hence simplifying the whole system.

To better understand the ramifications of over-approximation, con-sider a building where all north side elevators are designed not to beinterrupted during their upward journey during early mornings.4 Inother words, each elevator control algorithm has an if-then-else in itthat tests whether it is running inside a north side elevator, checkswhat time of the day it is, and prevents interruption if the tests con-firm ‘north’ and ‘morning.’ Now, if one were to hire a formal verifi-

2 In the modern society that su!ers from ‘information pollution,’ nondeterminismcan be the breath of oxygen that saves us from asphyxiation.

3 Except perhaps “control freaks.”4 Perhaps to ensure the speedy progression of the janitorial sta! and their accou-

terments to top floors.


cation specialist to mathematically verify that all the elevator controlalgorithms are working correctly, here is how they could employ non-determinism to simplify their activities. They could simply replace theif-then-else with a nondeterministic jump to both cases. Such a mod-ified control algorithm has more behaviors than the original, in whichevery elevator could nondeterministically decide whether to ignore theuser’s interrupt or to heed to it. Now, if the verification specialist ishanded the property to verify; “If I ring for a car, it will eventuallycome to my floor,” that property would pass on the nondeterministi-cally over-approximated system. The specialist would end up havingverified this property more easily, by ignoring the if-then-else. On theother hand, if they are handed another property, namely “if the caris below my floor, is headed to a floor above my floor and I press the‘up’ button, it will stop for me,” the verification will fail for the over-approximated model because every elevator can exercise the ‘ignoreinterrupt’ option. The verification specialist will realize that this is afalse positive—a false alarm—and then add the missing detail, which isthe if-then-else statement. During system design and verification, onecan then add just enough information to prove each properties of in-terest. The alternative approach of revealing all internal information,5

both taxes the mind and adds to verification time. Computer science’sessential mission is complexity management, and nondeterminism playsan essential role in this regard.

In this book, we now return to the use of nondeterminism for de-scribing regular languages. We now begin discussing the topic, “whatelse does nondeterminism a!ect?” We provide the answers under sep-arate headings.

9.1.1 How nondeterminism a!ects automaton operations

The presence of nondeterminism a!ects the ease with which automa-ton operations (such as union, concatenation, and star) can be carriedout. In the context of finite automata, while the operations of union,concatenation, and star become easier with the use of nondetermin-ism, the operations of complementation, intersection, and equivalencebecome harder to perform. We will see the details when we introducethese algorithms.

5 Popular with many legal departments, who do so in fine print.

9.2 Regular Expressions 137

9.1.2 How nondeterminism a!ects the power of machines

Nondeterminism also fundamentally a!ects the power of various ma-chine types, as follows (see also Section 4.3):

• Nondeterministic finite automata and deterministic finite automataare equivalent in power.

• Nondeterministic push-down automata are strictly more powerfulthan deterministic push-down automata.

• The equivalence between deterministic PDAs is algorithmically de-cidable. This is actually a recent result. It was obtained 30 yearsafter it was first conjectured to be true.

• The equivalence between nondeterministic and deterministic PDAs(and hence between two nondeterministic PDAs) is undecidable.

• It is still an open problem whether nondeterministic and determin-istic linear bounded automata are equivalent in power.

• The power of Turing machines does not change (with respect todecidability properties) through the use of nondeterminism. How-ever, with respect to complexity, certain problems (for instance,NP-complete problems which are discussed later) take exponentialtime on deterministic machines, but can be solved in polynomialtime on nondeterministic Turing machines. Also, Turing machineequivalence is undecidable.

With this general introduction, we look at regular expressions and NFAin greater detail. We begin with regular expressions because we believethat they will already be familiar to users of computers - if not knownby that name.

9.2 Regular Expressions

The idea of using regular expressions to denote whole classes of stringsis quite widespread. In many card games, the Joker card is called the“wild-card,” in that it can be used in lieu of any other card. In mostoperating systems, the wild-card is $ that matches all file names—as inrm *.* or del *.*. Such expressions are practical examples of regularexpressions in day-to-day use. Formally, we define regular expressionsas follows. We use the inductive definition style in which a basis caseand a list of inductive rules are provided, and define regular expressionsto be the least set satisfying these rules:

• 0 is an RE denoting the empty language 0.


Most computer-assisted tools have trouble accepting " directly,and so their syntax for " varies. It is written as {} in grail.Check with each tool to see how it encodes ". A command manregexp or man egrep issued on most Unix systems usually revealshow Unix encodes REs.

• " is an RE denoting the language {"}.Again, most computer-assisted tools encode " variously—as ""by grail, for example. In our drawings, we sometimes use e orepsilon in lieu of ". Unix has several flavors of "—those occurringwithin words, at the beginning, at the end, etc.6

• a , % is an RE denoting the language {a}.Most tools directly encode this regular expression in ASCII - forexample, a.

• For REs r, r1 and r2, r1r2, r1 + r2, r", and (r) are regular ex-pressions. These regular expressions denote the following languages,with the indicated encodings in grail (we write L(x) to mean ‘lan-guage of’):RE r1r2 denotes L(r1) ) L(r2) encoded in grail as r1 r2RE r1 + r2 denotes L(r1) ( L(r2) encoded in grail as r1+r2RE r" denotes (L(r))" encoded in grail as r*RE (r) denotes L(r) encoded in grail as (r)

The above regular operators form a basis set in the sense that we donot need any other operators such as complementation, reversal, inter-section RE, etc., to build regular languages. All regular languages canbe specified using only the above regular expression syntax. In practice,however, there are many languages that are very di"cult to specifywithout the use of these additional operators, especially complementa-tion (also known as negation).7 The reason for this di"culty is thatthe regular expression syntax, which only has $, ), and +, allows us tobuild up the language from simpler languages, while often it would bemuch more natural to build down a language, say, by specifying twodi!erent (but larger) languages and intersecting them. If you attemptExercises 9.6 and 9.8, you will confront the extreme di"culty of speci-fying two example languages directly in the RE syntax of the basis set- i.e., without the use of negation.

6 According to the Unix Manual pages for grep(1), the symbol \b matches theempty string at the edge of a word, and \B matches the empty string providedit’s not at the edge of a word.

7 Recall that for two languages L1 and L2, the complement of their union,(L1 % L2), is equivalent to the intersection of their complements, L1 & L2, byDeMorgan’s law. Hence, if we have complementation and union, we can obtainthe e!ect of intersection.

9.2 Regular Expressions 139

Illustration 9.2.1 In previous chapters we have seen how to writeout languages using set-theoretic syntax. In particular, we can expresslanguage

Labck = {xabc | x , {0, 1}k + a, b, c , {0, 1} + (a = 0 * b = 1)}

for various k as follows:

• k = 0 : {"} ) {00, 01, 11} ) {0, 1},• k = 1 : {0, 1} ) {00, 01, 11} ) {0, 1},• k = 2 : {0, 1}2 ) {00, 01, 11} ) {0, 1},• . . .

We can encode these languages using regular expressions as follows:

• k = 0 : " ) (00 + 01 + 11) ) (0 + 1),• k = 1 : (0 + 1) ) (00 + 01 + 11) ) (0 + 1),• k = 2 : (0 + 1)(0 + 1) ) (00 + 01 + 11) ) (0 + 1),• . . .

In the grail syntax, these regular expressions can be encoded as:

• k = 0 : ""(00+01+11)(0+1),• k = 1 : (0+1)(00+01+11)(0+1),• k = 2 : (0+1)(0+1)(00+01+11)(0+1),• . . .

Illustration 9.2.2 Consider the language

Lk = {x0z | x , {0, 1}" and z , {0, 1}k}.

For various k, we can express this language using regular expressions(in grail syntax) as follows:

• k = 0: (0+1)*0,• k = 1: (0+1)*0(0+1),• k = 2: (0+1)*0(0+1)(0+1),• k = 3: (0+1)*0(0+1)(0+1)(0+1),• . . .

We see that the size of the regular expression grows linearly with k—aproperty also shared by nondeterministic automata for this language,as we shall illustrate in Figure 9.2. In contrast, as shown in Figure 9.1,minimal DFAs for this language will grow exponentially with k. Inparticular, the minimal DFA for each k has 2k+1 states. The intuitivereason for this situation is that DFAs have no ability to “postponedecisions,” whereas NFAs have this ability “by keeping multiple optionsopen.”


1 1

5

0

7

0

3

1

0

1

0

6

0

2

1

4

0

1

1

0

0

1

1

0

1 1

9

0

0

1

0

5

1

11

0

2

1

8

0

1

0

3

1

0

4

1

13

0 1

10

0

1

0

6 0

14

1

1

12

0

7

0

1

0

15

10

1

1

0

1

0

30 1

0

0

1

0

2

1

3

0

4

1

5

0

6

1

7

0

8

1

9

0

10

1

11

0

12

1

13

0

14

1

15

0

16

1

17

0

18

1

19

0

20

1

21

0

22

1

23

0

31

1

24

0

25

1

26

0

27

1

28

0

29

1

0

1

0

1

0

1 0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

1

0

62 1

0

0

1

0

2

1

3

0

4

1

5

0

6

1

7

0

8

1

9

0

10

1

11

0

12

1

13

0

14

1

15

0

16

1

17

0

18

1

19

0

20

1

21

0

22

1

23

0

24

1

25

0

26

1

27

0

28

1

29

0

30

1

31

0

32

1

33

0

34

1

35

0

36

1

37

0

38

1

39

0

40

1

41

0

42

1

63

0

43

1

44

0

45

1

46

0

47

1

48

0

49

1

50

0

51

1

52

0

53

1

54

0

55

1

56

0

57

1

58

0

59

1

60

0

61

1

0

1

0

1

0

1 0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

1

0

Fig. 9.1. Minimal DFAs for Lk for k = 2, 3, 4, 5, with 2k+1 states

9.3 Nondeterministic Finite Automata 141

I

0,1

A0 B0,1 F0,1

Fig. 9.2. An NFA for (0 + 1)" 0 (0 + 1)k, for k = 2

9.3 Nondeterministic Finite Automata

We now formally introduce nondeterministic finite automata. Let %!

stand for (% ( {"}). A nondeterministic finite-state automaton N is astructure (Q,%, *, q0, F ), where:

Q, a finite non-empty set of states;%, a finite non-empty alphabet;* : Q 2 %! ! 2Q, a total transition function (note, however, thatthe range includes 0; therefore, if for some state q, *(q, x) = 0, themove from state q on input x is essentially to a black hole8);q0 , Q, an initial state; andF 5 Q, a finite, possibly empty set of final states.

The use of nondeterminism allows much more natural-looking as wellas compact descriptions of many automata. Contrast Figure 9.2 withthe equivalent minimal DFAs in Figure 9.1. Notice that these minimalDFAs are minimized as well as unique; in other words, it is impossible toavoid the exponential growth based on k. The NFA, on the other hand,simply accommodates all these figures by progressively elongating itstail. Figure 9.2 for k = 2 has a tail labeled with 0, 1 of length k =2, andthis sequence can be made three, four, and five steps long to correspondto the machines in Figure 9.1.

We now discuss various aspects of nondeterministic automata step-by-step, and present a comprehensive, practical example in Section 9.3.5.

9.3.1 Nondeterministic Behavior Without "

We provide two examples illustrating NFA. The first, provided in Fig-ure 9.2, demonstrates one type of nondeterminism where, from a singlestate, an NFA can advance to multiple next states upon reading aninput symbol—in this case, from state I upon reading 0 to states Iand A. The various components of this NFA are as follows:8 Unlike in recent theories on real black holes by Stephen Hawking where he proves

that information can escape, nothing escapes an automaton blackhole!


• Q ={I,A,B,F}• % ={0,1}• q0 =I• F =F• * : {I,A,B,F} 2 {0, 1, "} ! 2{I,A,B,F} is given by the following

table:

Input 0 1 e---------------------

States |I | {I,A} { I } { }

|A | { B } { B } { }

|B | { F } { F } { }

|F | { } { } { }

Notice that this NFA is assumed to move to the empty set of states, 0(written {}) in the absence of an explicitly given move in this table. Totrace out the behavior of this NFA on an input string such as 10101,we place a token in state I and subject it to moves according to its* function. The use of instantaneous descriptions (ID) as employed inSection 8.1.1 suggests itself as a way to keep track of ‘where the NFAis in its diagram’ and ‘what the unconsumed input string is,’ except weknow that an NFA can be in a set of states after a move. What typeof IDs must we employ to keep track of all this? Of the many choicespossible, we employ the following type:

TID NFA = 2Q 2%".

This ID keeps a set of states and the unconsumed string specific tothat set of states. We now use the - notation to indicate ‘one step’ offorward progress of the NFA, where a step may be made through amember of % or through ".

$ relation for an NFA

We define - 5 TID NFA 2 TID NFA as follows:

(Q1,+1) - (Q2,+2).= #a , %! : +1 = a+2 + Q2 = (q1#Q1

*(q1, a).

In other words, - causes a move from an ID (Q1,+1) to another ID(Q2,+2) by moving every state q1 , Q1 via the * function upon a , %!,and taking a union of the resulting sets of states. Notice that we allowthe choice of " for a, meaning that +1 can be the same as +2, and the


NFA can simply move through ". An example that illustrates the useof - is as follows:

• Initial ID of the NFA is ({I}, 10101).• ({I}, 10101) - ({I}, 0101); after reading 1, the NFA is still in the

set of states {I}, as given by the * function of the NFA.• ({I}, 0101) - ({I,A}, 101); after reading 0, the NFA is simultane-

ously in two states, {I,A}, as given by the * function.• ({I,A}, 101) - ({I,B}, 01); after reading 1, the token in state I

goes back to state I itself, while the token in state A advances tostate B. For example, the NFA is now simultaneously in two states,{I,B}. We obtain {I,B} by taking the union of *(I, 1) and *(A, 1).Continuing in this manner, we go through the IDs shown below.

• ({I,B}, 01) - ({I,A, F}, 1); after reading 0, state I goes to theset of states {I,A} whereas state B goes to state F . Now the DFAhas seen a substring that it accepts, namely 1010. If 1010 were theentire string, one token would have reached F , and consequentlythe NFA would have accepted. However, since the entire string is10101, we continue as below.

• ({I,A, F}, 1) - ({I,B}, "); after reading the final 1, F “falls out ofthe diagram” (goes to the empty set of states, as shown by *(F, 1)),while the other states, I and A, move to I and B. Since neither ofthese are final states, the NFA does not accept 10101.

9.3.2 Nondeterministic behavior with "

I

0,1A0 B0,1

F0,1

e

Fig. 9.3. An NFA for ((0 + 1)" 0 (0 + 1)k)+, for k = 2

We now illustrate an NFA’s ability to advance through " moves withthe aid of Figure 9.3, which is a slight modification of Figure 9.2. InFigure 9.3, the NFA can move from state F to state I via ". In short,this machine accepts one or more repetitions of a string in the languageof Figure 9.2. Let us see how. In terms of instantaneous descriptions,the sequence of ID changes, when running with input 10101010, are asfollows: As before,


• ({I}, 10101010) - ({I}, 0101010)• ({I}, 0101010) - ({I,A}, 101010)• ({I,A}, 101010) - ({I,B}, 01010)• ({I,B}, 01010) - ({I,A, F}, 1010).• One way to proceed is to treat the ID ({I,A, F}, 1010) as

({I,A, F}, "1010) and allow the " transition to fire. This sends Iand A to the empty set of states, while sending F to {I}:

• ({I,A, F}, "1010) - ({I}, 1010).• We now can perform ({I}, 1010) - ({I}, 010).• Let us now accelerate our presentation by using -".• ({I}, 010) -" ({I,A, F}, "). The string is accepted since one of the

tokens has reached F .

Another way in which the NFA could proceed from ({I,A, F}, 1010)is by moving states I, A, and F on 1 (without inserting the "):

• ({I,A, F}, 1010) - ({I,B}, 010), in which move I proceeds to {I},A proceeds to {B}, and F proceeds to 0.

• ({I,B}, 010) - ({I,A, F}, 10).• ({I,A, F}, 10) - ({I,B}, 0).• ({I,B}, 0) - ({I,A, F}, "), also resulting in the string being ac-

cepted.

To sum up, using -, we have to be prepared to read a string such as0100 as x0x1x0x0x, where each x is any number of "s in sequence. Sec-tion 9.3.3 introduces a better alternative for determining the behaviorof NFAs using the concept of Eclosure or "-closure.

Eclosure, in e!ect, considers all9 "-laden interpretations ofstrings in “one fell swoop.”

The language of an NFA

The language of an NFA consists of all those sequences of symbols thatcan be encountered while tracing a path from the start state to somefinal state. We eliminate all occurrences of " from such sequences unlessthe entire sequence consists of "s, in which case, we turn the sequenceinto a single10 ". Formally, given an NFA N = (Q,%, *, q0, F ) with the- relation defined as above, its language

9 While discussing nondeterministic push-down automata (NPDA) in later chap-ters, we will see that a construct similar to Eclosure cannot be defined for NPDAs,as the stack state also has to be kept around.

10 Please note that " is not part of the alphabet of the NFA.


L(n) = {w | ({q0}, w) -" (Q, ") + Q ' F %= 0}.

In other words, if the ID ({q0}, w) can evolve through zero or more -steps to an ID (Q, ") where Q contains a final state, then w is in thelanguage of the NFA.

9.3.3 Eclosure (also known as "-closure)

IF

eFA

0

FB1

A11 B0

e

0

1

ee

0

e

1

0

Fig. 9.4. An example NFA

1

2

0

3

1

04

1

0 1

1

5

0

0

1

Fig. 9.5. DFA obtained using grail

Eclosure(q) obtains, starting from a state q, the set of states reachedby an NFA traversing zero or more " labeled transitions. The best way


to see which states are included in Eclosure(q), for any q, is to imaginethe following:

Apply a high-voltage to state q; imagine that every " edge isa diode that conducts in the direction of the arrow; now seewhich are the states that would be fatal to touch due to thehigh-voltage;11 all those are in Eclosure(q).

Example: Let us obtain the Eclosure of various states in Figure 9.4.

• The Eclosure of state IF is IF itself. The high-voltage spreads fromIF to itself.

• What is Eclosure of FA? Applying high-voltage to this state, itspreads to state FA, to B0, to A1, and finally, to FB. For example,Eclosure(FA) = {FA,B0, A1, FB}.

Unfortunately, an intuitive definition in terms of voltages isn’t rigorousenough! Hence, we set up an alternate definition of Eclosure(q) asfollows:

• First, we need a way to compute all states which can be reachedfrom a given state by traversing " edges.Let !5 Q 2 Q be an arbitrary relation over Q. Then, given ! 5Q 2 Q, we will define a postfix usage of this operator, namely q !, to be the image of q under !:

q ! = {x | 6q, x7 , !}.

• Now define the relation"! which is a subset of Q 2 Q:

"! = {6q1, q27 | q1, q2 , Q + q2 , *(q1, ")}.• Next, define the reflexive and transitive closure of

"! in the usual

way. Call it"!".

Given all this, we define

Eclosure(q) = q"!".

Here, we use the"!" as a postfix operator. The reflexive part above

is very important to ensure that q gets included within Eclosure(q).

The overall e!ect of employing"!" as a postfix operator is to force all

"-only paths to be considered.Example: Redoing our example with respect to Figure 9.4,

Eclosure(FA) = FA"!".

11 An ideal non-leaky diode that does not break down.


• This “grabs” state FA itself (the reflexive part of"!" does this).

• Next, B0 enters this set, as it is one step away. In fact, B0 is in

FA"!1.

• Next, A1 enters this set, as it is two steps away. In fact, A1 is in

FA"!2.

• Finally, FB enters this set, as it is three steps away. In fact, FB is

in FA"!3.

• No more states enter Eclosure(FA).

As said earlier, Eclosure helps define the behavior of an NFA, aswell as its language more directly. It also helps us define the NFA toDFA conversion algorithm (Chapter 12) very directly.

9.3.4 Language of an NFA

Having defined Eclosure, we can now formally define the language ofan NFA. For a string x , %", define *(q0, x) of an NFA to be theset of states reached starting from Eclosure(q0) and traversing all thesymbols in x, taking an Eclosure after every step. We are, in e!ect,taking the image of q0 under string x, except that we are allowing anarbitrary number of "s to be arbitrarily inserted into x. Also, we willoverload Eclosure to work over sets of states in the obvious manner,as follows:

Eclosure(S) = {x | #s , S : x , Eclosure(s)}.

This definition can also be written as

Eclosure(S) = (s#S Eclosure(s).

Likewise, we overload * to work over sets of states:

*(S, a) = {x | #s , S : x , *(s, a)}.

Now we define *(q, x), the ‘string transfer function,’ for state q andstring x inductively as follows:

*(q, ") = Eclosure(q).For a , % and x , %",*(q, ax) = {y | #s , Eclosure(*(Eclosure(q), a)) : y , *(s, x)}.


In other words, for every symbol a in ax, we Eclose state q, “run” afrom each one of these states, and Eclose the resulting states. Fromeach state s that results, we recursively run x. Notice that we applyEclosure before as well as afterward. While this is strictly redundant,it leads to definitions that are simpler and more general, and henceeasier to reason about.12 Specifically, in the definition of *(q, ax), weneed not assume that q is an already Eclosed state, even though in thecurrent context of its usage, we will be inductively guaranteeing thatto be the case because: (i) we begin with *(q, ") = Eclosure(q), and(ii) in *(q, ax), we restore “Eclosedness.”Finally, the language of an NFA N is

L(N) = {w | *(q0, w) ' F %= 0}.

In other words, after running the NFA from state q0 with input w, wesee whether any ‘token’ has reached a final state.A good way to intuitively understand the above definitions pertainingto NFAs is through the following ‘token game:’

• Place a token in state q0. Spread one copy of the token to each statein Eclosure(q0).

• For each symbol a from % that is entered, advance each token toits set of a successors.

• Eclose the tokens and continue.• If and when one of the tokens reaches some final state, the string

seen so far is accepted.

Illustration 9.3.1 The NFA in Figure 9.4 has string 0001 in its lan-guage. First, Eclosure(IF ) = {IF}. After the first 0, the NFA is instate FA, whose Eclosure is {FA,B0, A1, FB}. After the second 0,A1 goes to FA, and the Eclosure results in {FA,B0, A1, FB}. Thesame happens after the third 0. After the 1, FA and A1 go to A1, B0goes to FB, and the token in FB goes to 0, resulting in {A1, FB},which is also its own Eclosure. This matches the definition

*(q, ax) = {y | #s , Eclosure(*(Eclosure(q), a)) : y , *(s, x)}

as argued below:

• Eclosure(*(Eclosure(IF ), 0)) is {FA,B0, A1, FB}.• We recursively process the remaining input 001 from these statesto reach {A1, FB}.

12 Another way to set up the definitions would have been to start the NFA in theEclosure of its start state, and at each stage perform a # step followed by oneEclosure step.


9.3.5 A detailed example: telephone numbers

-------------- --------------Legal Illegal-------------- --------------[201]221-1221 [201]2211221201-221-1221 201-22112212012211221 201221-1221221-1221 2211-2212211221-------------- --------------

Fig. 9.6. Legal and illegal phone numbers in our example

We now point out that in some cases we prefer NFAs (or regularexpressions) not because they are smaller, but simply because they aremuch easier to specify. Also, the results would be much more convincingthan if we were to draw a DFA directly or simply write a naturallanguage description of the language in question. Our example alsoconcretely illustrates the ability of " transitions to make descriptionsclearer.

Illustration 9.3.2 Suppose we want to develop a lexical analyzer13

for telephone numbers. We want to be maximally flexible, and allowmultiple syntaxes for phone numbers for users’ convenience. First, letus assume a world where we have not invented the terminology of non-deterministic automata or regular expressions. We will then be forcedto describe the set of legal telephone numbers in natural language prose(in our case, English). Here is how an initial attempt might look:

A telephone number consists of three parts: an area code (3 digits),a middle part (3 digits), and the extension (4 digits), written in thatorder. We consider only digits in the range of 0 to 2, to keep thingssimple. All three parts may be written in juxtaposition or may beseparated by exactly one dash, ‘-’. If the area code is separated fromthe middle part by a dash, the middle part must also be separated fromthe extension by a dash. The area code may also be surrounded bybrackets,14 ‘[’ and ‘].’ In this case, the middle part must be separated

13 Also known as a “scanner” - a tool that digests input at the character level andconverts them into meaningful entities called tokens.

14 We use brackets [ and ] in lieu of the more traditional parentheses ‘(’ and ‘)’because the grail tool we use assigns special significance to parentheses - and wewanted to avoid work-arounds to keep things simple.


0

1

e

3

e

5

e

61

e

62

e

128

e

129

e

131

e

133

e

20

0

22

1

24

2

2

0

4

1

6

2

85

0

87

1

89

2

63

[

150

0

152

1

154

2

130

0

132

1

134

2

26

0

28

1

30

2 0 120 12

8

0

10

1

12

2

14

0

16

1

18

20 1 20 1 2

0 1 20 1 2

0 12 0 12 0 12

32

0

34

1

36

2 0 120 12

38

0

40

1

42

2 0 120 12

44

0

46

1

48

2 0 120 12

50

0

52

1

54

2 0 120 12

56

0

58

1

60

2 01 201 2

91

0

93

1

95

2 0 120 12

65

0

67

1

69

2

71

0

73

1

75

20 1 20 1 2

77

0

79

1

81

2 012 012

83

]]]

0 12

97

0

99

1

101

2 0 120 12

103

- --

105

0

107

1

109

2

111

0

113

1

115

2 01 201 2

117

0

119

1

121

20 1 20 1 2

123

0

125

1

127

20 12 0 12

156

0

158

1

160

20 12 0 12

136

0

138

1

140

2

142

0

144

1

146

20 120 12

01 2 01 2

148

- --

01 2

162

0

164

1

166

2 0 120 12

168

- --

170

0

172

1

174

2

176

0

178

1

180

2 01 201 2

182

0

184

1

186

20 1 20 1 2

188

0

190

1

192

20 12 0 12

(""+(0+1+2)(0+1+2)(0+1+2))(0+1+2)(0+1+2)(0+1+2)(0+1+2)(0+1+2)(0+1+2)(0+1+2)+

(""+[(0+1+2)(0+1+2)(0+1+2)])(0+1+2)(0+1+2)(0+1+2)-(0+1+2)(0+1+2)(0+1+2)(0+1+2)+

(""+(0+1+2)(0+1+2)(0+1+2)-)(0+1+2)(0+1+2)(0+1+2)-(0+1+2)(0+1+2)(0+1+2)(0+1+2)

Fig. 9.7. An NFA formally specifying allowed telephone numbers, and theRE describing it

from the extension by a dash. The area code is optional; if not present,neither the brackets surrounding the area code nor the dash separatingthe area code from the middle part must be employed.


0

5

01

2

1

[

6

01

27

01

2 9

01

2

2

1220

1

1401

2

3 15

01

2

18

01

2

4 20

01

2

21

01

2

8

01

2

-

24

01

2

10

01

2

1101

2

]

1301

2 1601

2170

12

01

2

-

23

01

2

19

01

2

22

01

2

-

01

2

Fig. 9.8. A minimal DFA for the NFA of Figure 9.7

Example telephone numbers that are legal and illegal according to theseconventions are listed in Figure 9.6 (please verify that these indeedfollow the rules stated in English).

Unfortunately, as is well-known, natural language descriptions areoften prone to misinterpretation. Let us therefore provide a formal spec-ification of legal and illegal telephone numbers. We do this by writinga regular expression also shown in Figure 9.7. This RE can be turnedinto an NFA as shown in Figure 9.7. This NFA exhibits three classesof strings in the form of “three lobes” of state clusters. The corre-spondence between the RE and the NFA must be quite apparent. Theminimal DFA for this NFA is given in Figure 9.8. We can then use amodern lexical analyzer generator (flex in our case) that generates arecognizer for this language, as sketched in Figure 9.9. The commandsto generate the lexical analyzer and a few interactions with the gen-erated analyzer (including an error scenario) are also included. Theconcept behind scanner programs was illustrated in Figure 8.2; mod-ern scanners are, however, far more e"cient than the simple-mindedprogram used in Figure 8.2.

Writing regular expressions is a highly error-prone activity. The nextsection illustrates how to formally verify regular expressions throughputative queries (“challenge queries”). The main idea we stress is thatoften the complement of the language we are interested in is far easier tocharacterize—at least to a large approximation. We can then intersectthe complement of the language with the language of interest to see ifthe intersection is empty.

9.3.6 Tool-assisted study of NFAs, DFAs, and REs

In this section, we illustrate the use of the Grail tools to help understandthe material so far. Basically, grail and the support scripts provide usthe following commands that can be invoked from Unix:


/*----------- Definitions space -----------*/D [0-2] /* Digit */TD {D}{D}{D} /* Three Digits */FD {D}{TD} /* Four Digits */BAC "["{TD}"]" /* Bracketed Area Code */DAC {TD}"-" /* Dashed Area Code */BACOpt {BAC}|"" /* Optional Bracketed Area Code */DACOpt {DAC}|"" /* Optional Dashed Area Code */ACOpt {TD}|"" /* Optional Area Code */

Tele {ACOpt}{TD}{FD}|{BACOpt}{TD}"-"{FD}|{DACOpt}{TD}"-"{FD}/* (""+(0+1+2)(0+1+2)(0+1+2)) (0+1+2)(0+1+2)(0+1+2)(0+1+2)(0+1+2)(0+1+2)(0+1+2)+

(""+[(0+1+2)(0+1+2)(0+1+2)])(0+1+2)(0+1+2)(0+1+2)-(0+1+2)(0+1+2)(0+1+2)(0+1+2)+(""+(0+1+2)(0+1+2)(0+1+2)-)(0+1+2)(0+1+2)(0+1+2)-(0+1+2)(0+1+2)(0+1+2)(0+1+2) */

/*----------- Rules space -----------*/%%{Tele} { printf("A number:%s\n",yytext); }

[\t\n]+ /* eat up whitespace */

. printf("Unrecognized character:%s\n",yytext);/*----------- User code space -----------*/%%int yywrap();main(){ yylex();}

int yywrap(){ return 1;}/*----------- End -----------*/> flex telephone.l> cc lex.yy.c> a.out> 0120120120A number:0120120120> 000-0000A number:000-0000> [000]111-2222A number:[000]111-2222> 000-000-0000A number:000-000-0000> 0001111A number:0001111> [122] 111-1111Unrecognized character:[Unrecognized character:1Unrecognized character:2Unrecognized character:2Unrecognized character:]Unrecognized character:A number:111-1111

Fig. 9.9. Skeleton of an encoding of the telephone number syntax in flex

fa2grail.perl, a Perl scriptgrail2ps.perl, a Perl scriptdot, a graphics packageghostview or gv, a postscript viewerOne of several grail-specific commands. We include some of thesecommands below (for details, kindly see the grail user manual):


– retofm for converting regular expressions into grail-specific in-ternal “fm” format,

– fmdeterm for determinizing fm-formatted NFAs,– fmmin for minimizing DFAs,– fmcment for complementing DFAs, and– fmcross for obtaining the intersection of two DFAs.

A quick tool check

In order to check that all the tools are available in one’s search path,type

> echo ’(a+b)*’ | retofm | fmdeterm | fmmin \| perl grail2ps.perl - | gv -

Here > stands for the operating system prompt (grail currently runsunder Linux). This command has the following e!ect. The regular ex-pression (ab)*+ would be turned into an NFA, this NFA determinizedand minimized, and finally picked up by the script grail2ps.perlwhich, with the help of the dot tool, produces postscript and drawsthe DFA for you! Note how we quote the command using ‘ and ’ toprevent Unix from assigning a special interpretation to *. Also notethe use of the Unix pipe |, and the character - that denotes “standardinput.”

Tool-assisted RE debugging

While regular expressions are often easy to write, they can also prove tobe error-prone to write. We highly recommend that one always appliestools similar to grail for debugging proposed REs. In this section, wepresent a systematic method of debugging REs using grail.

Proposed RE

Someone proposes the following RE to express the language L, where“L = the set of strings of 0s and 1s with at most one pair of consecutive1s.”

(0+ + ") (10+)" (11 + 1) (0+1)" (0+ + ")

We assume that the reader did not find anything wrong about this RE,but nevertheless wanted to make sure. Let’s formally debug by applyingputative queries to study the expected properties of such an RE. We


know that the language of this machine must not have any 11(0+1)*11or any 111 in it. One can often identify and specify whole classes ofsuch “bad strings” much more reliably than one can do with the RE ofinterest. Capitalizing these facts, we proceed to debug as follows:

1. First, obtain a minimal DFA corresponding to the given RE:

> echo ’(00*+"")(100*)*(11+1)(00*1)*(00*+"")’ | retofm| fmdeterm | fmmin >! a1.txt

> cat a1.txt | perl grail2ps.perl - >! a1.ps1

0

41

3

0

0

11 0

201

0

2. Second, get “the machine of all bad strings” built. (Please makesure you understand that this machine contains all the bad strings;it contains two patterns, namely (i) at least two pairs of consecutive11s, and (ii) the 111 pattern):

> echo ’(0+1)*11(0+1)*11(0+1)*+(0+1)*111(0+1)*’ | retofm| fmdeterm | fmmin >! a2.txt

> cat a2.txt | perl grail2ps.perl - >! a2.ps

10

310 21

4

0

0

11 0

10

3. Now we take a look at the complement of the machine of bad strings:

> echo ’(0+1)*11(0+1)*11(0+1)*+(0+1)*111(0+1)*’ | retofm| fmdeterm | fmcment | fmmin >! a2c.txt

> cat a2c.txt | perl grail2ps.perl - >! a2c.ps

20

3

10

0

1

101

0


We find that the complement of the “bad machine” is not the sameas “our machine.” This is clearly indicative of something beingwrong with our original RE (assuming that the complement ma-chine was correct). Since the description of the complement machinewas so direct and simple, we have every room to suspect the former.In the following sequel, we will proceed to formally determine whereexactly the di!erence lies. This is the gist of the concept known asformal verification, where we have every room to trust the “prop-erty” (namely a description that is extremely easy to trust beingcorrect) and formally compare the system against it.

4. Let’s at least make sure that the original RE doesn’t contain anybad strings.

> fmcross a1.txt a2.txt | perl grail2ps.perl - >! cross.ps> gv cross.ps

6 0

19

1

0

2

01

0

5

7

0

0

15

1

10

22

01

0

0

20

0

1 0

4

1

11

21

0

1 0

14

1

16

0

1

12

1

0

17

0

1

3 0

1

8 0

1

13

23

0

1

0

1

18

0

1

10

9

1 01 0 1 0

24

10

Here we see the unminimized DFA. Apparently none of its statesare reachable. We now proceed to minimize and view this machine:

> fmcross a1.txt a2.txt | fmmin | perl grail2ps.perl - >!crossmin.ps

> gv crossmin.ps

The last command (gv crossmin.ps) displays virtually nothing onthe screen—indicative of an empty automaton. This experimentindicates that the original RE, indeed, does not include any badstrings. Therefore, its error must lie in the fact that it is leaving outsome of the good strings. The next quest is to find out which theseare.

5. So what is in the complement of the bad machine that’s not in theoriginal RE (i.e. what are we leaving out)?

> cat a1.txt | fmcment | fmmin >! a1c.txt> fmcross a2c.txt a1c.txt | fmmin >! a1ca2ccrossmin.txt> cat a1ca2ccrossmin.txt | perl grail2ps.perl - >! a1ca2ccrossmin.ps


Viola! We found the omitted set of strings!

0 0

Clearly we left out the all zeros case! Let’s fix that omission in ouroriginal RE, and view that RE after this fix (notice that the fixlies in using (11+1+"") now, instead of (11+1), which is what ouroriginal RE contained):

> echo ’(00*+"")(100*)*(11+1+"")(00*1)*(00*+"")’ | retofm| fmdeterm | fmmin >! a1fixed.txt

> cat a1fixed.txt | perl grail2ps.perl - >! a1fixed.ps

Now, this exactly matches the complement of the bad machine.

20

310

01 101

0

Since these perspectives agree, we consider our original RE as hav-ing been formally verified against the bad machine RE that wassimpler to fathom.

Inputting DFAs into the Grail pipeline

We have created a script to intuitively encode DFAs in an ASCII syntaxand input into grail. We illustrate this through an example. Let fileexpt1 contain the following:

I - 0 -> AI - 1 -> DeadA - 0 -> DeadA - 1 -> FbfiFbfi - 0 -> FfiaFbfi - 1 -> DeadFfia - 1 -> FbfiFfia - 0 -> ADead - 0,1 -> Dead

Now, if we type


> fa2grail.perl expt1 | fmdeterm | perl grail2ps.perl - | gv -

we can obtain the DFA in Figure 9.5. We will introduce further featuresof the grail pipeline in subsequent chapters.

Chapter Summary

This chapter began with a discussion of why nondeterminism is such apowerful idea in computer science. In addition to succinct descriptionsthat are made possible (a dramatic illustration of which is providedin Figure 9.1), the use of nondeterminism may make certain machinetypes strictly more powerful, as in the case of push-down automata.We also introduce the notion of regular expressions, hand in hand withour discussion of nondeterminism. We introduce the notion of " movesin NFA: basically, " allows machines to move without reading theirinput. The notion of " / closure was also introduced. This operatorallows NFAs to be converted into language equivalent DFAs. SeveralNFAs were illustrated. In particular, Section 9.3.6 discussed how NFAsfor given regular expressions may be obtained, as well as verified, in atool-assisted manner.

ExercisesIn all problems involving verification using grail, clearly explain yourverification method. If you checked for properties such as the pres-ence/absence of sub-languages (as was shown in Section 9.3.6), pleaseelaborate on those details. You are also encouraged to use JFLAP toperform these verification steps.

9.1. It seems as if when adding the " transition from F to I, we didn’tchange the language of the NFA in Figure 9.3. Argue that this is true.

9.2. For the NFA in Figure 9.4, step through its IDs starting with input101101. Also, answer whether this string is accepted or not.

9.3. This exercise may be attempted if you read Chapter 7, “Dealingwith Recursion.” An alternate definition of Eclosure is via recursion:

For q , Q, Eclosure(q) = {q} ( {y | y , *(q, ")* #z , *(q, ") + y , Eclosure(z)}

Notice that the use of recursion here is “well defined.” In particular,we can compute the least fixed-point of the Eclosure function. We startwith Eclosure(q) returning the empty set 0 for every state. Plug thisdefinition of Eclosure on the right-hand side and “pump” the recursionone step, obtaining the Eclosure function for the left-hand side for


every state. This adds q itself to Eclosure, and also adds those statesthat are directly reachable from q via " edges. Next plug in this versionof Eclosure on the right-hand side and pump up again until no morestates are added to any Eclosure. This process comes to a halt sincewe have a finite number of states. Argue that this least fixed-pointbased definition of Eclosure is equivalent to the transitive-closure baseddefinition.

9.4. What is the language of the NFA of Figure 9.4? List all strings oflength six or less in the language of this NFA. For each string, provideall possible reasons for that string to be in the language.

9.5. Give an NFA accepting the set of strings over % = {0, 1} such thatevery block of five consecutive symbols contains at least two 0s.

9.6. Attempt to write a regular expression directly for the language over% = {0, 1} in which every block of five consecutive symbols containsat least two 0s. Spend some time directly encoding the said conditionusing regular expressions, and verify your results using grail. If youfail, you may try again later by drawing a DFA and converting thatDFA to an RE.

9.7. Design an NFA to recognize the language of strings over {0, 1},beginning with a 0, ending with a 1, and having an occurrence of 0101somewhere in every string.

9.8. Attempt to directly write an RE for the language over % = {0, 1}in which no string has a 0101 occurring in it. Spend some time directlyencoding the ‘no 0101’ condition using regular expressions, and verifyyour results using grail. If you fail, you may try again later by drawinga DFA and converting that DFA to an RE.

9.9. Draw an NFA to recognize the star of the language of the NFA inFigure 9.4.

9.10. Design an NFA to recognize the reverse of the language of theNFA in Figure 9.4.

9.11. Design an NFA to capture the syntax of floating-point numbersover alphabet {0, 1,+,/, E, .} (for the sake of simplicity). A floating-point number begins with an optional sign character (one of + or /),followed by a mantissa, the letter E, an optional sign, and an exponent.While the exponent is an integer, the mantissa is a decimal fraction ofthe form whole.frac where whole or frac (but not both) may be empty.

11

The Automaton/Logic Connection, SymbolicTechniques

Most believe that computer science is a very young subject. In a sense,that is true - there was the theory of relativity, vacuum tubes, radio,and Tupperware well before there were computers. However, from an-other perspective, computer science is at least 150 years old! CharlesBabbage1 started building his Analytic Engine in 1834 which remainedunfinished till his death in 1871. His less programmable Di!erence En-gine No. 2 was designed between 1847 and 1849, and built to his spec-ifications in 1991 by a team at London’s Science Museum. As for the‘theory’ or ‘science’ behind computer science, George Boole publishedhis book on Boolean Algebra2 in 1853.

Throughout the entire 150 years (or so) history of computer science,one can see an attempt on part of researchers to understand reasoningas well as computation in a unified setting. This direction of thinkingis best captured by Hilbert in one of his famous speeches made in theearly 1900s in which he challenged the mathematical community with23 open problems. Many of these problems are still open, and somewere solved only decades after Hilbert’s speech. One of the conjecturesof Hilbert was that the entire body of mathematics could perhaps be“formalized.” What this meant is basically that mathematicians hadno more creative work to carry out; if they wanted to discover a newresult in mathematics, all they had to do was to program a computer tosystematically crank out all possible proofs, and check to see whetherthe theorem whose proof they are interested in appears in one of theseproofs!

1 Apparently, Babbage is also credited with the invention of the ‘cow-catcher’ thatyou see in front of locomotive engines!

2 Laws of thought. (You might add: to prevent loss of thought through loosethought).

186 11 The Automaton/Logic Connection, Symbolic Techniques

In 1931, Kurt Godel dropped his ‘bomb-shell.3 He formally statedand proved the result, “Such a device as Hilbert proposed is impossi-ble!” By this time, Turing, Church, and others demonstrated the truelimits of computing through concrete computational devices such asTuring machines and the Lambda calculus. The rest “is history!”

11.1 The Automaton/Logic Connection

Scientists now have a firm understanding of how computation and logicare inexorably linked together. The work in the mid 1960s, notably thatof J.R. Buchi, furthered these connections by relating branches of math-ematics known as Presburger arithmetic and branches of logic knownas WS1S4 with deterministic finite automata. Work in the late 1970s,notably by Pnueli, resulted in the adoption of temporal logic as a for-mal logic to reason about concurrency. Temporal logic was popularizedby Manna and Pnueli through several textbooks and papers. Work inthe 1980s, notably by Emerson, Clarke, Kurshan, Sistla, Sifakis, Vardi,and Wolper established deep connections between temporal logic andautomata on infinite words (in particular Buchi automata). Work inthe late 1980s, notably by Bryant, brought back yet another thread ofconnection between logic and automata by the proposal of using binarydecision diagrams, essentially minimized deterministic finite automatafor the finite language of satisfying instances of a Boolean formula, asa data structure for Boolean functions. The symbolic model checkingalgorithm proposed by McMillan in the late 1980s hastened the adop-tion of BDDs in verification, thus providing means to tackle the cor-rectness problem in computer science. Also, spanning several decades,several scientists, including McCarthy, Wos, Constable, Boyer, Moore,Gordon, and Rushby, led e!orts on the development of mechanicaltheorem-proving tools that provide another means to tackle the cor-rectness problem in computer science.

11.1.1 DFA can ‘scan’ and also ‘do logic’

In terms of practical applications, the most touted application do-main for the theory of finite automata is in string processing – patternmatching, recognizing tokens in input streams, scanner construction,etc. However, the theory of finite automata is much more fundamental

3 Some mathematicians view the result as their salvation.4 WS1S stands for the weak monadic second-order logic of one successor.

11.2 Binary Decision Diagrams (BDDs) 187

to computing. Most in-depth studies about computing in areas such asconcurrency theory, trace theory, process algebras, Petri nets, and tem-poral logics rest on the student having a solid foundation on classicalautomata, such as we have studied so far. This chapter introduces someof the less touted, but nevertheless equally important, ramifications ofthe theory of finite automata in computing. It shows how the theoryof DFA helps arrive at an important method for representing Booleanfunctions known as binary decision diagrams. The e"cient represen-tation as well as manipulation of Boolean functions is central to au-tomated reasoning in several areas of computing, including computer-aided design and formal verification. In Chapter 21, we demonstratehow exploiting the “full power” of DFAs, one can represent logics withmore power than propositional logic. In Chapter 22, we demonstratehow automata on infinite words can help reason about finite as well asinfinite computations generated by finite-state devices. In this context,we briefly sketch the connections between automata on infinite wordsas well as temporal logics. We now turn to binary decision diagrams,the subject of this chapter.Note: We use * and + interchangeably, depending on what looks morereadable in a given context; they both mean the same (the or function).

11.2 Binary Decision Diagrams (BDDs)

Binary Decision Diagrams (BDDs) are bit-serial DFA for satisfyinginstances of Boolean formulas.5 To better understand this characteri-zation, consider the finite language

L1 = {abcd | d * (a + b + c)}.

Since all finite languages (a finite number of finite strings in this case)are regular, a regular expression describing this language can be ob-tained by spelling out all the satisfying instances of d* (a+ b+ c). Thisfinite regular language is denoted by the following regular expression:

(1110+1111+0001+0011+0101+0111+1001+1011+1101)

By putting this regular expression in a file called a.b.c+d, we can usethe following grail command sequence to obtain a minimal DFA forit, as shown in Figure 11.1(a):

5 BDDs may also be viewed as optimized decision trees. We view BDDs as DFAfollowing the emphasis of this book. Also note that strictly speaking, we mustsay Reduced Ordered Binary Decision Diagrams or ROBDDs. We use “BDD” asa shorthand for ROBDD.


(a)

1

6

1

7

0

4

1

5

0 1 0

3

1

2

0 0 1

0

1 0 1

(b)

7

1

0

6

1

5

1

4

0 1

3

1

2

0 1

0

1 0 1

Fig. 11.1. Minimal DFAs for d # (a $ b $ c) for (a) variable ordering abcd,and (b) dabc. The edges show transitions for inputs arriving according to thisorder.

cat a.b.c+d | retofm | fmdeterm | fmmin | perl grail2ps.perl -> a.b.c+d.ps

Now consider the language that merely changes the bit-serial order inwhich the variables are examined from abcd to dabc:

L2 = {dabc | d * (a + b + c)}.

Using the regular expression

(0111+1000+1001+1010+1011+1100+1101+1110+1111)

as before, we obtain the minimal DFA shown in Figure 11.1(b). Thetwo minimal DFAs seem to be of the same size. Should we expect thisin general? The minimal DFAs in Figure 11.1 and Figure 11.2, aresuboptimal as far as their role in decoding the binary decision goes,as they contain redundant decodings. For instance, in Figure 11.1(a),

11.2 Binary Decision Diagrams (BDDs) 189

(a)

5

4

0

1

1

9

0 1

7

0

8

1

6

0 1

2

0

3

1

0

0 1

(b)

0

1

0

2

1

3

0

4

1

5

0

6

1

7

0

8

1

9

0

10

1

11

0

12

1

13

0

14

1

15

0

16

0

21

0

17

01 1 1 1

18

0

19

01 1

20

0 1

Fig. 11.2. Minimal DFAs where the variable ordering matters

after abc = 111 has been seen, there is no need to decode d; however,this diagram redundantly considers 0 and 1 both going to the acceptingstate 0. In Figure 11.1(b), we can make node 6 point directly to node0. Eliminating such redundant decodings, Figures 11.1(a) and (b) will,essentially, become BDDs; the only di!erence from a BDD at that pointwould be that BDDs explicitly include a 0 node to which all falsifyingassignments lead to.

Let us now experiment with the following two languages where weshall discuss these issues even more, and actually present the drawingof a BDD.

Linterleaved = {abcdef | a = b + c = d + e = f}

has a regular expression of satisfying assignments

(000000+001100+000011+110000+001111+110011+111100+111111)

andLnoninterleaved = {acebdf | a = b + c = d + e = f}


01

14: f

27: d

28: b

10: f

29: d

30: b

31: e

32: d

33: b

34: d

35: b

36: e

37: c

38: b 39: b

40: e

41: b 42: b

43: e

44: c

exp4: a

Fig. 11.3. BDD for a = b $ c = d $ e = f for variable order acebdf

has

(000000+010010+001001+100100+011011+101101+110110+111111).

When converted to minimized DFAs, these regular expressions yieldFigures 11.2(a) and (b), where the size di!erence due to the variableorderings is very apparent. The BDD for Figure 11.2(b) created usingthe BED tool appears in Figure 11.3. The commands used to createthis BDD were:

bed> var a c e b d f % declares six variablesbed> let exp4 = (a=b) and (c=d) and (e=f) % defines the desired expn.bed> upall exp4 % builds the BDD -bed> view exp4 % displays the BDD

By comparing Figure 11.3 against Figure 11.2(b), one can see how, ingeneral, BDDs eliminate redundant decodings.6

6 The numbers inside the BDD nodes—such as the “14:” and “10:” in the nodes forvariable f—may be ignored. They represent internal numberings chosen by theBED tool.

11.3 Basic Operations on BDDs 191

BDDs are e"cient data structures for representing Boolean func-tions and computing the reachable states of state transition systems. Inthese applications, they are very often ‘robust,’ i.e., their sizes remainmodest as the computation advances. As many of these state transi-tion systems have well over 2150 states (just to pick a large number!),this task cannot be accomplished in practice by explicitly enumeratingthe states. However, BDDs can often very easily represent such largestate-spaces by capitalizing on an implicit representation of states asdescribed in Section 11.3. However, BDDs can deliver this ‘magic’ onlyif a “good” variable ordering is chosen.

One also has to be aware of the following realities when it comes tousing BDDs:

The problem of determining an optimal variable ordering is NP-complete (see Chapter 20 for a definition of NP-completeness). [42];this means that the best known algorithms for this task run in ex-ponential worst-case time.In many problems, as the computation proceeds and new BDDsare built, variable orderings must be recomputed through dynamicvariable re ordering algorithms, which are never ideal and add tothe overhead.For certain functions (e.g., the middle bits of the result of multiply-ing two N -bit numbers), the BDD is provably exponentially sized,no matter which variable ordering is chosen.

Even so, BDDs find extensive application in representing as well as ma-nipulating state transition systems realized in hardware and software.We now proceed to discuss how BDDs can be used to represent statetransition relations and also how to perform reachability analysis.

11.3 Basic Operations on BDDs

BDDs are capable of e"ciently representing transition relations offinite-state machines. In some cases, transition relations of finite-statemachines that have of the order of 2100 states have been represented us-ing BDDs. For example, a BDD that represents the transition relationfor a 100-bit digital ripple-counter can be built using about 200 BDDnodes.7 Such compression is, of course, achieved by implicitly represent-ing the state space; an explicit representation (e.g., using pointer based

7 Basically, each bit of such a counter toggles when all the lower order bits are a1, and thus all the BDD basically represents is an and function involving all thebits.


data structures) of a state-space of this magnitude is practically impos-sible. Given a transition relation, one can perform forward or backwardreachability analysis. ‘Forward reachability analysis’ is the term usedto describe the act of computing reachable states by computing theforward image (“image”) of the current set of states (starting from theinitial states). Backward reachability analysis computes the pre-imageof the current set of states. One typically starts from the current set ofstates violating the desired property, and attempts to find a path backto the initial state. If such a path exists, it indicates the possibility ofa computation that violates the desired property.

Each step in reachability analysis takes the current set of statesrepresented by a BDD and computes the next set of states, also rep-resented by a BDD. It essentially performs a breadth-first traversal,generating each breadth-first frontier in one step from the currentlyreached set of states. The commonly used formulation of traversal is interms of computing the least fixed-point as explained in Section 11.3.2.When the least fixed-point is reached, one can query it to determine

the overall properties of the system. One can also check whether desiredsystem invariants hold in an incremental fashion (without waiting forthe fixed-point to be attained) by testing the invariant after each newbreadth-first frontier has been generated. Here, an invariant refers toa property that is true at every reachable state.

We will now take up these three topics in turn, first illustrating howwe are going to represent state transition systems.

11.3.1 Representing state transition systems

!b b

Fig. 11.4. Simple state transition system (example SimpleTR)

We capture transition systems by specifying a binary state transi-tion relation between the current and next states, and also specifyinga predicate capturing the initial states. If inputs and outputs are tobe modeled, they are made part of the state vector. Depending on theproblem being modeled, we may not care to highlight which parts ofthe state vector are inputs and which are outputs. In some cases, theentire state of the system will be captured by the states of inputs and


outputs. Figure 11.4 presents an extremely simple state transition sys-tem, called SimpleTR. Initially, the state is 0. Whenever the state is0, it can become 1. When it is 1, it can either stay 1 or become 0.These requirements can be captured using a single Boolean variable brepresenting the current state, another Boolean variable b

!represent-

ing the next state,8 and an initial state predicate and a state transitionrelation involving these variables, as follows:

The initial state predicate for SimpleTR is !b.¬b, since the initialstate is 0. Often, instead of using the lambda syntax, initial statepredicates are introduced by explicitly introducing a named initialstate predicate I and defining it by an equation such as I(b) = ¬b.For brevity,9 we shall often say “input state represented by ¬b.”The state transition relation for SimpleTR is !(b, b

!).¬bb

!+ bb

!+

b¬b!, where each product term represents one of the transitions.

The values of b and b!

for which this relation is satisfied representthe present and next states in our example. In other words,– a move where b is false now and true in the next state is repre-

sented by ¬bb!.

– a move where b is true in the present and next states is repre-sented by bb

!.

– finally, a move where b is true in the present state and false inthe next state is represented by b¬b

!.

This expression can be simplified to !(b, b!).(b + b

!). The above re-

lation can also written in terms of a transition relation T defined asT (b, b

!) = b + b

!. We shall hereafter say “transition relation b + b

!.”

Notice that this transition relation is false for b = 0 and b!

= 0,meaning there is no move from state 0 to itself (all other moves arepresent).

11.3.2 Forward reachability

The set of reachable states in SimpleTR starting from the initial state¬b can be determined as follows:

Compute the set of states in the initial set of states.Compute the set of states reachable from the initial states in n steps,for n = 1, 2, . . ..

8 The ‘primed variable’ notation was first used by Alan Turing in one of the veryfirst program proofs published by him in [89].

9 Syntactic sugar can cause cancer of the semi-colon – Perlis


In other words, we can introduce a predicate P such that a state x is inP if and only if it is reachable from the initial state I through a finitenumber of steps, as dictated by the transition relation T . The aboverecursive recipe is encoded as

P (s) = (I(s) * #x.(P (x) + T (x, s))).

This formula says that s is in P if it is in I, or there exists a state xsuch that x is in P , and the transition relation takes x to s.Rewriting the above definition, we have

P = !s.(I(s) * #x.(P (x) + T (x, s)))).

Rewriting again, we have

P = (!G.(!s.(I(s) * #x.(G(x) + T (x, s))))) P.

In other words, P is a fixed-point of

!G.(!s.(I(s) * #x.(G(x) + T (x, s)))).

Let us call this Lambda expression H:

H = !G.(!s.(I(s) * #x.(G(x) + T (x, s)))).

In general, H can have multiple fixed-points. Of these, the least fixed-point represents exactly the reachable set of states, as next explainedin Section 11.3.3.

11.3.3 Fixed-point iteration to compute the least fixed-point

As shown in Section 6.1, the least fixed-point can be obtained by “bot-tom refinement” using the functional obtained from the recursive defi-nition. In the same manner, we will determine P , the least fixed-pointof H, by computing its approximants that, in the limit, become P . Letus denote the approximants P0, P1, P2, . . .. We have P0 = !x.false, the“everywhere false” predicate. The next approximation to P is obtainedby feeding P0 to the “bottom refiner” (as illustrated in Section 6.1):

P1 = !G.(!s.(I(s) * #x.(G(x) + T (x, s))))P0

which becomes !s.I(s). This approximant says that P is a predicatetrue of s whenever I(s). While this is not true (P must represent thereachable state set and not the initial state alone), it is certainly a bet-ter answer than what P0 denotes, which is that there are no states inthe reachable state set! We now illustrate all the steps of this computa-tion, taking SimpleTR for illustration. We use the abbreviation of notshowing the lambda abstracted variables in each step.


I = !b.¬b.T = !(b, b

!). (b + b

!).

P0 = !s.false, which encodes the fact that “we’ve reached nowhereyet!”P1 = !G.(!s.(I(s) * #x.(G(x) + T (x, s))))P0.This simplifies to P1 = I, which is, in e!ect, an assertion that we’ve“just reached” the initial state, starting from P0.Let’s see the derivation of P1 in detail. Expanding T and P0, wehaveP1 = !G.(!s.(I(s) * #x.(G(x) + (x + s) ))) (!x.false).The above simplifies to ¬b.By this token, we are expecting P2 to be all states that are zero orone step away from the start state. Let’s see whether we obtain thisresult.P2 = !G.(!s.(I(s) * #x.(G(x) + T (x, s))))P1.= !s.(¬s * #x.(¬x + (x + s))).= !s.1.This shows that the set of states reached by all the breadth-firstfrontiers (combined) that are zero and one step away from thestart state, includes every state. Another iteration would not changethings; the10 least fixed-point has been reached.

BED Commands for SimpleTR:

The BED commands given in Figure 11.5 compute the reachable set ofstates using forward reachability in our example. We can see that P2,the least fixed-point, is indeed true — namely, the characteristic predi-cate for the set of all states. (Note: In BED, the primed variables mustbe declared immediately after the unprimed counterparts). In additionto the explicit commands to calculate the least fixed-point, BED alsoprovides a single command called reach. Using that, one can calculatethe least fixed-point in one step. In our present example, RS and P2 endup denoting the BDD for true.

let RS = reach(I,T)upall RSview RS

Section 11.3.4 discusses another example where the details of the fixed-point iteration using BED are discussed.

10 We do not discuss many of the theoretical topics associated with computing fixed-points in the domain of state transition systems — such as why least fixed-pointsare unique, etc. For details, please see [20].


var b bp % Declare b and b’let I = !b % Declare init statelet t1 = !b and bp % 0 --> 1upall t1 % Build BDD for itview t1 % View itlet t2 = b and bp % 1 --> 1let t3 = b and !bp % 1 --> 0let T = t1 or t2 or t3 % All three edgesupall T % Build and view the BDDview T %

let P0 = falseupall P0view P0

let P1 = I or ((exists b. (P0 and T))[bp:=b])upall P1view P1

let P2 = I or ((exists b. (P1 and T))[bp:=b])upall P2view P2

0 1

P0: b

P0

01

P0: b

P1: a

P1

1

P2, the least fixed-point

Fig. 11.5. BED commands for reachability analysis on SimpleTR, and thefixed-point iteration leading up to the least fixed-point that denotes the setof reachable states starting from I

Why Stabilization at a Fixed-Point is Guaranteed

In every finite-state system modeled using a finite-state Boolean transi-tion system, the least fixed-point is always reached in a finite number ofsteps. Let us try to argue this fact first using only a simple observation.The observation is that all the Boolean expressions generated duringthe course of fixed-point computation are over the same set of vari-


ables. Since there are exactly 22NBoolean functions over N Boolean

variables (see Illustration 4.5.2), eventually two of the approximants inthe fixed-point computation process will have to be the same Booleanfunction. However, this argument does not address whether it is pos-sible to have “evasive” or “oscillatory” approximants Pi, Pi+1, . . . , Pj

such that i %= j and Pj = Pi. If this were possible, it would be possibleto cycle through Pi, . . . , Pj without ever stabilizing on a fixed-point.Fortunately, this is not possible! Each approximant Pi+1 is more de-fined than the previous approximant Pi, in the sense defined by theimplication lattice defined in Illustration 4.5.3. With this requirement,the number of these ascending approximants is finite, and one of thesewould be the least fixed-point. See Andersson’s paper [7] for additionalexamples of forward reachability. The book by Clarke et.al. [20] givesfurther theoretical insights.

11.3.4 An example with multiple fixed-points

Consider the state transition system in Figure 11.6 with initial state s0(called MultiFP). The set of its reachable states is simply {s0} (and ischaracterized by the formula a+ b), as there is no reachable node froms0. Now, a fixed-point iteration beginning with the initial approximantfor the reachable states set to P0 = false will converge to the fixed-point a + b. What are the other fixed-points one can attain in thissystem? Here they are:

With the initial approximant set to {s0,s1}, which is characterizedby b, the iteration would reach the fixed-point of a * b, which char-acterizes {s0,s1,s2}.Finally, we may iterate starting from the initial approximant be-ing 1, corresponding to {s0,s1,s2,s3}. The fixed-point attained inthis case is 1, which happens to be the greatest fixed-point of therecursive equation characterizing reachable states.

Hence, in this example, there are three distinct fixed-points for therecursive formula defining reachable states. Of these, the least fixed-point is a+b, and truly characterizes the set of reachable states; a*b isthe intermediate fixed-point, and 1 is the greatest fixed-point. It is clearthat (a+ b) 3 (a* b) and (a* b) 3 1, which justifies these fixed-pointorderings. Figure 11.6 also describes the BED commands to producethis intermediate fixed-point.


var a ap b bp

let T = (a and b and ap and bp) or /* S0 -> S0 */(!a and b and !ap and bp) or /* S1 -> S1 */(a and !b and ap and !bp) or /* S2 -> S2 */(!a and !b and !ap and !bp) or /* S3 -> S3 */(!a and b and ap and !bp) or /* S1 -> S2 */(a and !b and !ap and bp) or /* S2 -> S1 */(!a and b and ap and bp) or /* S1 -> S0 */(a and !b and ap and bp) /* S2 -> S0 */

upall Tview T /* Produces BDD for TREL ’T’ */

let I = a and blet P0 = blet P1 = I or ((exists a. (exists b. (P0 and T)))[ap:=a][bp:=b])

upall P1view P1

{b}s1

{a,b}

{a}

s2

s0

s3Transition System MultiFP

0 1

P0: b

P0

01

P0: b

P1: a

P1

Fig. 11.6. Example where multiple fixed-points exist. This figure shows at-tainment of a fixed-point a # b which is between the least fixed-point of a $ band the greatest fixed-point of 1. The figure shows the initial approximant P0and the next approximant P1

11.3.5 Playing tic-tac-toe using BDDs

What good are state-space traversal techniques using BDDs? How doesone obtain various interesting answers from real-world problems? Whilewe cannot answer these questions in detail, we hope to leave this chapterwith a discussion of how one may model a game such as tic-tac-toe and,say, compute the set of all draws in one fell swoop. Following throughthis example, the reader would obtain a good idea of how to employ


mathematical logic to specify a transition system through constraints,and reason about it. We assume the reader knows the game of tic-tac-toe (briefly explained in the passing).

Modeling the players and the board:

We model two players, A and B. The state of the game board is modeledusing a pair of variables ai,j, bi,j (we omit the pairing symbols 67 forbrevity) for each square i, j where i , 3 and j , 3. We assume thatplayer A marks square i, j with an o, by setting ai,j and resettingbi,j, while player B marks square i, j with an x, by resetting ai,j andsetting bi,j. We use variable turn to model whose turn it is to play(with turn = 0 meaning it is A’s turn). The state transition relationfor each square will be specified using the four variables ai,j, ai,jp, bi,j,and bi,jp. We model the conditions for a row or column remaining thesame, using predicates samerowi and samecoli. We define nine possiblemoves for both A and for B. For example, M00 model’s A’s move intocell 0, 0; Similarly, we employ N00 to model B’s move into cell 0, 0, andso on for the remaining cells. The transition relation is now defined as adisjunction of the Mi,j and Ni,j moves. We now capture the constraintatmostone that says that, at most one player can play into any square.We then enumerate the gameboard for all possible wins and draws. Inthe world of BDDs, these computations are achieved through “symbolicbreadth first” traversals. We compute the reachable set of states, firstquerying it to make sure that only the correct states are generated.Then we compute the set of states defining draw configurations. Thecomplete BED definitions are given in Appendix B.

Chapter Summary

This chapter briefly reviewed the history of mathematical logic andpointed out the fact that in the early days of automata theory, math-ematical logic and automata were discussed in a unified setting. Thisapproach has immense pedagogical value which this book tries to re-store to some extent. A practitioner who works on advanced hard-ware/software debugging method needs to know both of these topicswell. For instance, automata theory has, traditionally, been consideredan essential prerequisite for an advanced class on compilation. How-ever, recent publications in systems/compilers (e.g., [121]) indicate thecentral role played by BDDs (see below) and related notions in math-ematical logic.

We then discuss how Boolean formulas can be represented in acanonical fashion using the so-called ‘reduced ordered binary decision


diagrams,’ or “BDDs” for short. We then present how finite-state ma-chines can be represented and manipulated using BDDs. We show howreachable states starting from a set of start states can be computedusing forward reachability, by using the notion of fixed-points intro-duced in Chapter 6. We finish the chapter with an illustration of howthe game of tic-tac-toe may be modeled using BDDs, and how a toolcalled BED may be used to compute interesting configurations, such asall the draw positions, all possible win positions, etc.

BDDs are far richer in scope and application than we have room toelaborate here. The reader is referred to [14, 13] for an exposition of howBDDs are used in hardware and software design, how BDDs may becombined using Boolean operations through the apply operator, etc. Analternate proof of canonicity of BDDs appears in [14]. Our presentationof BDDs as automata draws from [22], and to some extent from [111].

Exercises

11.1. Similar to Figure 11.3, draw a BDD for all 16 Boolean func-tions over variables x and y. (Some of these functions are !(x, y).true,!(x, y).false, !(x, y).x, !(x, y).y, etc. Down this list, you have more“familiar” functions such as !(x, y).nand(x, y), and so on. Express thesefunctions without the “lambda” part in BED syntax, and generate theBDDs using BED.)

11.2.1. Obtain an un-minimized DFA (in the form of a binary tree) for the

languageL = {abc | a 3 b + c}

picking the best variable ordering (in case two variable orderingsare equal, pick the one that is in lexicographic order). Show theblack-hole state also.

2. Minimize this DFA, and then show the additional steps that castthe minimized DFA into a BDD.

11.3. Consider the examples given in Figure 11.2. Construct similarexamples for the addition operation. More specifically, consider the bi-nary addition of two unsigned two-bit numbers a1a0 and b1b0, resultingin answer c2c1c0. Generate a BDD for the carry output bit, c2. Choosea variable ordering that minimizes the size of the resulting BDD andexperimentally confirm using BED.

11.4. Repeat Exercise 11.3 to find out the variable ordering that max-imizes the BDD size.


11.5. Represent the behavior of a nand gate, under the inertial delaymodel, as a state transition system. Encode this transition system usinga BDD. Here are some general details on how to approach this problem.

The behavior of an inverter can be modeled using a pair of bitsrepresenting its input and output. (For a nand gate, we will need toemploy three bits.) In the transport delay model, every input change,however short, is faithfully copied to the output, but after a small delay.There is another delay model called the inertial delay model in which“short” pulses may not make it to the output.

The behavior of an inverter under these delay models are shown infigures (a) and (b) below.

00 01

1110

Initial inverter state

Input changes to a 1Output has not changed

quiescent.inverter is and the transitionsThe output

Inputchangesand the inverter isin a transientstate again

The outputtransitionsand the inverter is quiescent.

(a)

Input changes. Inverter is in a transient state.However the inputcan be withdrawnnullifying thescheduled outputchange.

Input changes. Inverter is in a transient state.However the inputcan be withdrawnnullifying thescheduled outputchange.

00 01

1110

Initial inverter state

quiescent.inverter is and the transitionsThe output

The outputtransitionsand the inverter is quiescent.

(b)

11.6. Draw a BDD for the transition relation of a two-bit binarycounter with output bits a1a0 for initial state 00, counting in the usual0, 1, 2, 3 order. Repeat for a two-bit gray-code counter that counts 00,01, 11, 10, and back to 00.

11.7.1. With respect to the state transition relation of Figure 11.6(a), iden-

tify all the fixed-points of the recursive equation for reachability.2. Given a state transition system (say, as a graph, as in Figure 11.6(a)),

what is a general algorithm to determine the number of fixed-pointsof its recursive equation for reachability?

11.8. Consider a three-bit shift register based counter with the indi-cated next-state relation for its three bits:


next(a) = b ; next(b) = c ; next(c) = not(and(a,b,c))

c b a

1. Represent the next-state relation of this counter using a singleROBDD. Choose a variable ordering that minimizes the size of yourROBDD and justify your choice.

2. Compute the set of all reachable states using forward reachabilityanalysis, using the reach command, starting at state 000.

3. Justify the correctness of the answer you obtain. The answer youobtain must be a Boolean formula over a, b, c. Show that this for-mula is satisfied exactly for those states reachable by the system.

11.9. A three-bit Johnson counter11 consists of a three-bit shift registerwhere the final Q output is connected to the first D input. Startingfrom a reset state of 000, this counter will go through the sequence100, 110, 111, 011, 001, and back to 000. For this counter, repeat whatExercise 11.8 asks.

11.10. Using BED, determine the shortest number of steps to win inTic-Tac-Toe. Appendix B has a full description of the problem encod-ing.

11.11. Check two conjectures concerning Tic-Tac-Toe, using BED:(i) if a player starts by marking the top-left corner, he/she may lose;(ii) if a player starts by marking the middle square, he/she may win.

11.12. Construct an example with four distinct fixed-points under for-ward reachability, and verify your construction similar to that explainedin Figure 11.6.

11.13. Encode the Man-Wolf-Goat-Cabbage problem using BDDs. Inthis problem, a man has to carry a wolf, goat, and cabbage across ariver. The man has to navigate the boat in each direction. He maycarry no more than one animal/object on the boat (besides him) at atime. He must not leave the wolf and goat unattended on either bank,

11 Named after Emeritus Prof. Bob Johnson, University of Utah.


nor must he leave the goat and cabbage unattended on either bank.The number of moves taken is to be minimal. Use appropriate Booleanvariables to model the problem.

18

Basic Notions in Logic including SAT

This chapter is on propositional logic, first-order logic, and modernBoolean satisfiability methods. In particular, we will prove the unde-cidability of the validity of first-order logic sentences. Boolean satisfi-ability is discussed in su"cient detail to ensure that the reader is wellmotivated to study the theory of NP-completeness in Chapter 19.

Mathematical logic is central to formalizing the notion of assertionsand proofs. Classical logics include propositional (zeroth-order), first-order, and higher-order logics. Most logics separate the notion ofproof and truth. A proof is a sequence of theorems where each theoremis either an axiom or is obtained from previous theorems by applyinga rule of inference.

Informally speaking, it is intended that all theorems are “true.” Thisis formalized through the notion of a meaning function or interpretationwhich maps a well-formed formula (a syntactically correct formula,often abbreviated w!) to its truth value. If a w! is true under allinterpretations, it is called a tautology (or equivalently, it is called valid).Therefore, when we later mention the term, “the validity problem of

first-order logic,” we refer to the problem of deciding whether a givenfirst-order logic formula is valid (this problem is later shown to be onlysemi-decidable, or equivalently TR or RE).

A logical system consisting of a collection of axioms1 and rules ofinference is sound if every theorem is a tautology. A logical system iscomplete if every tautology is a theorem. A logical system has indepen-dent rules of inference if omitting any rule prevents certain theoremsfrom being derived.

1 Note that axioms can be regarded as rules of inference that have an emptypremise.

324 18 Basic Notions in Logic including SAT

One uses the terminology of “order” when talking about logics.Propositional (zeroth-order) logic includes Boolean variables that rangeover true and false (or 1 and 0), Boolean connectives, and Boolean rulesof inference. Predicate (first-order) logic additionally allows the use ofindividuals such as integers and strings that constitute infinite sets,variables that are quantified over such sets, as well as predicate andfunction constants. Second (and higher) order logics allow quantifica-tions to occur over function and predicate spaces also. For example,in Chapter 5, Section 5.3, we presented two principles of induction,namely arithmetic and complete; the formal statement of both theseinduction principles constitutes examples of higher order logic. Thereare also special classes of logics that vary according to the kinds ofmodels (interpretations) possible: these include temporal logic, which isdiscussed in Chapters 21 and 22. In these logics, notions such as va-lidity tend to acquire specialized connotations, such as validity undera certain model. In this chapter, however, we will stick to the classicalview of these notions.

Section 18.1 discusses a Hilbert style axiomatization of propositionallogic due to Church. Section 18.2 begins with an example involving“quacks and doctors,” presents examples of interpretations for formu-las, and closes o! with a proof for the fact that the validity problemof first-order logic is undecidable, but semi-decidable (or equivalently,RE or TR).

We then turn our attention to the topic of satisfiability in the settingof propositional logic in Section 18.3. We approach the subject basedon notions (and notation) popular in hardware verification - follow-ing how “Boolean logic” is treated in those settings (a model theoreticapproach). We first examine two normal forms, namely the conjunc-tive and the disjunctive normal forms, and discuss converting one tothe other. We then examine related topics such as %=-satisfiability, 2-satisfiability, and satisfiability-preserving transformations.

18.1 Axiomatization of Propositional Logic

We now present one axiomatization of propositional calculus followingthe Hilbert style. Let the syntax of well-formed formulas (w!) be asfollows:

Fmla ::= Pvar | ¬Fmla | Fmla 3 Fmla | (Fmla).

In other words, a formula is a propositional variable, the negation of apropositional formula, an implication formula, or a parenthesized for-

18.1 Axiomatization of Propositional Logic 325

mula. This grammar defines a complete set of formulas in the sensethat all Boolean propositions can be expressed in it (prove this asser-tion). Following Church [17], a Hilbert calculus for propositional logiccan be set up based on three axioms (A1-A3) and two rules of inference(R1-R2) shown below. Here, p and q stand for propositional formulas.

A1: p 3 (q 3 p)A2: (s 3 (p 3 q)) 3 (s 3 p) 3 (s 3 q)A3: (¬q 3 ¬p) 3 (p 3 q)

R1 (Modus Ponens): If P is a theorem and P 3 Q is a theorem, con-clude that Q is a theorem.

Example: From p and p 3 (p 3 q), infer p 3 q.R2 (Substitution): The substitution of w!s for propositional variables

in a theorem results in a theorem. A substitution is a “parallelassignment” in the sense that the newly introduced formulas them-selves are not a!ected by the substitution (as would happen if, forinstance, the substitutions are made serially).

Example: Substituting (p 3 q) for p and (r 3 p) for q in formulap 3 q, results in (p 3 q) 3 (r 3 p). It is as if p and q are replacedby fresh and distinct variables first, which, in turn, are replaced by(p 3 q) and (r 3 p) respectively.We do not perform the substitution of r 3 p for q first, and thena!ect the p introduced in this process by the substitution of (p 3 q)for p.

Given all this, a proof for a simple theorem such as p 3 p can becarried out – but it can be quite involved:

P1: From A1, through substitution of p 3 p for q, we obtain

p 3 ((p 3 p) 3 p).

P2: From A2, substituting p for s, p 3 p for p, and p for q, we obtain

(p 3 ((p 3 p) 3 p)) 3 (p 3 (p 3 p)) 3 (p 3 p).

P3: Modus ponens between P1 and P2 yields

(p 3 (p 3 p)) 3 (p 3 p).

P4: From A1, substituting p for q, we obtain

p 3 (p 3 p).

Modus ponens between P4 and P3 results in (p 3 p). !


It is straightforward to verify that the above axiomatization issound. This is because the axioms are true, and every rule of infer-ence is truth-preserving. The axiomatization is also complete, as canbe shown via a proof by induction. The take away message from thesediscussions is that it pays to hone the axiomatization of a formal logicinto something that is parsimonious as well as enjoys the attributes ofbeing sound, complete, and independent, as explained earlier.

18.2 First-order Logic (FOL) and Validity

Below, we will introduce many notions of first-order logic intuitively,through examples. We refer the reader to one of many excellent booksin first-order logic for details. One step called skolemization merits someexplanation. Basically, skolemization finds a witness to model the ex-istential variable. In general, from a formula of the form #X.P (X), wecan infer P (c) where c is a constant in the domain. Likewise, from P (c),we can infer #X.P (X); in other words, for an unspecified constant c inthe domain,

#X.P (X) = P (c).

We will use this equivalence in the following proofs. There is anotheruse of skolemization illustrated by the following theorem (which wewon’t have occasion to use):

"X.#Y.P (X,Y ) = P (X, f(X)).

Here, f is an unspecified (but fixed) function. This equivalence is validbecause of two reasons:

The right-hand side leaves X as a free variable, achieving the samee!ect as "X goes, as far as validity goes (must be true for all X).The right-hand side employs f(X) to model the fact that the selec-tion of #Y may depend on X.

18.2.1 A warm-up exercise

Suppose we are given the following proof challenge:

Some patients like every doctor. No patient likes a quack. There-fore, prove that no doctor is a quack.

18.2 First-order Logic (FOL) and Validity 327

It is best to approach problems such as this using predicates to modelthe various classes of individuals2 instead of employing ‘arbitrary con-stants’ such as p and d to denote patients, doctors, etc. Introduce pred-icate p to carve out a subset of people (P ) to be patients, and similarlyd for doctors. Let l (for “likes”) be a binary relation over P . Our proofwill consist of instantiation of formulas, followed by Modus ponens, andfinally proof by contradiction.

A1: The statement, “Some patients like every doctor:”

#x , P : (p(x) + "y : (d(y) 3 l(x, y))).

A2: The statement, “No patient likes a quack:”

"x, y , P : (p(x) + q(y) 3 ¬l(x, y)).

Proof goal: “No doctor is a quack.”

"x , P : (d(x) 3 ¬q(x)).

Negate the proof goal: #x , P : (d(x) + q(x)).Skolemize negated proof goal: (x0 , P + d(x0) + q(x0).Skolemize A1: c , P + (p(c) + "y : (d(y) 3 l(c, y))) (we will suppress

domain membership assertions such as c , P from now on).Specialize: From A1, we specialize y to x0 to get:p(c) + (d(x0) 3 l(c, x0)).But since d(x0) is true in the negated proof goal, we get:p(c) + l(c, x0) (more irrelevant facts suppressed).Since q(x0) is true in the negated proof goal, we also get:p(c) + l(c, x0) + q(x0).Use A2 and specialize x to c and y to x0 to get ¬l(c, x0).

Contradiction: Since we have l(c, x0) and ¬l(c, x0), we get a contradic-tion.

18.2.2 Examples of interpretations

Manna’s book [78] provides many insightful examples, some of whichare summarized below. We focus on the notion of interpretations,which, in case of first-order logic: (i) chooses domains for constants,predicates, and function symbols to range over, and (ii) assigns to them.Examples below will clarify.

2 Pun intended; in FOL, members of domains other than Booleans are called indi-viduals.


Example 1

Consider the formula

Fmla1 = #F.F (a) = b+("x).[p(x) 3 F (x) = g(x, F (f(x)))]

We will now provide three distinct interpretations for it.Interpretation 1.

D = Nata = 0b = 1f = !x.(x = 0 ! 0, x / 1)g = *p = !x.x > 0

Interpretation 2.

D = %"

a = "b = "f = !x.(tail(x))g(x,y) = concat(y,head(x))p = !x.x %= "

Interpretation 3.

D = Nata = 0b = 1f = !x.xg(x,y) = y+1p = !x.x > 0

It is clear that under Interpretation 1, Fmla1 is true, because thereindeed exists a function F , namely the factorial function, that makesthe assertion true. It is also true under Interpretation 2, while it isfalse under Interpretation 3 (Exercise 18.4 asks for proofs). Hence, thisformula is not valid — because it is not true under all interpretations.

Example 2

Consider the formula

18.2 First-order Logic (FOL) and Validity 329

Fmla2 = "P.P (a)+("x).[(x %= a) + P (f(x)) 3 P (x)]

3 ("x.P (x))

We can interpret this formula suitably to obtain the principle of induc-tion over many domains: for example, Nat, strings, etc.

18.2.3 Validity of first-order logic is undecidable

Valid formulas are those that are true under all interpretations. Forexample,

"x.f(x) = g(x) 3 #a.f(a) = g(a).

Validity stems from the innate structure of the formula, as it mustremain true under every conceivable interpretation. We will now sum-marize Floyd’s proof (given in [78]) that the validity problem for first-order logic is undecidable. First, an abbreviation: for +i , {0, 1}, usethe abbreviation

f"1,"2,...,"n(a) = f"n(f"n#1(. . . f"1

(a)) . . .).

The proof proceeds by building a FOL formula for a given “Post sys-tem” (an instance of Post’s correspondence problem).Given a Post system S = {(,1,-1), (,2,-2), . . . , (,n,-n), n . 1 over% = {0, 1}, construct the w! WS (we will refer to the two antecedentsof WS as A1 and A2, and its consequent as C1):

+ni=1 p(f#i

(a), f$i(a)) (A1)

+ "x "y [p(x, y) 3 +ni=1 p(f#i

(x), f$i(y))] (A2)

3 #z p(z, z) (C1)

We now prove that S has a solution i! WS is valid.Part 1. (WS valid) 3 (S has a solution).

If valid, it is true for all interpretations. Pick the following interpre-tation:

a = "f0(x) = x0 (string ‘x’ and string ‘0’ concatenated)f1(x) = x1 (similar to the above)p(x, y) = There exists a non-empty sequence i1i2 . . . im such that

x = ,i1,i2 . . .,im and y = -i1-i2 . . . -im

Under this interpretation, parts A1 and A2 of WS are true. Here iswhy:


• Under the above interpretation, f#i(a) = ",i = ,i and similarly

f$i(a) = -i.

• Thus A1 becomes +ni=1 p(,i,-i). Each conjunct in this formula is

true by p’s interpretation; hence A1 is true.• The part [p(x, y) 3 +n

i=1 p(f#i(x), f$i

(y))] reduces to the followingclaim: p(x, y) is true means that x and y can be written in theform x = ,i1,i2 . . .,im and y = -i1-i2 . . . -im ; the consequent ofthis implication then says that we can append some ,i and thecorresponding -i to x and y, respectively. The consequent is alsotrue by p’s interpretation. Thus A2 is also true.

• Since WS is valid (true), C1 must also be true. C1 asserts thatthe Post system S has a solution, namely some string z that lendsitself to being interpreted as some sequence ,i1,i2 . . .,im as well as-i1-i2 . . . -im . That is,

,i1,i2 . . . ,im = z = -i1-i2 . . . -im .

Part 2. (WS valid) > (S has a solution).If S has a solution, let it be the sequence i1i2 . . . im. In other words,

,i1,i2 . . .,im = -i1-i2 . . . -im = Soln. Now, in order to show that WS

is valid, we must show that for every interpretation it is true. We ap-proach this goal by showing that under every interpretation where theantecedents of WS , namely A1 and A2, are true, the consequent, namelyC1, is also true (if any antecedent is false, WS is true, so this case isnot considered).

From A1, we conclude that

p(f#i1(a), f$i1

(a))

is true. Now using A2 as a rule of inference, we can conclude throughModus ponens, that

p(f#i2(f#i1

(a)), f$i2(f$i1

(a))

is true. In other words,

p(f#i1#i2

(a), f$i1$i2

(a))

is true. We continue this way, applying the functions in the order dic-tated by the assumed solution for S; in other words, we arrive at theassertion that the following is true (notice that the subscripts of f de-scribe the order in which the solution to S considers the ,’s and -’s):

p(f#i1#i2

...#im(a), f$i1

$i2...$im

(a)).

18.3 Properties of Boolean Formulas 331

However, since

,i1,i2 . . .,im = -i1-i2 . . . -im = Soln,

we have essentially shown that

p(fSoln(a), fSoln(a)).

Now, p(fSoln(a), fSoln(a)) means that there exists a z such that p(z, z),namely z = fSoln(a). !

18.2.4 Valid FOL formulas are enumerable

It was proved by Godel in his dissertation that first-order predicatecalculus is complete. In other words, there are axiomatizations of first-order logic in which every valid formula has a proof. Hence, by enu-merating proofs, one can enumerate all valid first-order logic formulas.Thus the set of valid FOL formulas is recursively enumerable (or isTuring recognizable).

18.3 Properties of Boolean Formulas

18.3.1 Boolean satisfiability: an overview

Research on Boolean satisfiability methods (SAT) is one of the “hottest”areas in formal methods, owing to the fact that BDDs are often knownto become exponentially sized, as already hinted in Chapter 11. In [11],the idea of model checking without BDDs was introduced. This workalso coincided with the arrival on the scene, of e"cient Boolean sat-isfiability methods. These modern SAT solvers (e.g., [81, 90]) followedthe basic “DPLL” procedure presented in [33, 32], but made consid-erable improvements, including intelligent methods for backtrackingsearch over the space of satisfying assignments, and improving the ef-ficiency of computer memory system (e.g., cache) utilization in carry-ing out these algorithms. Easily modifiable versions of these tools arenow freely available (e.g., [87]). Boolean SAT methods are now ableto often handle extremely large system models, establish correctnessproperties, and provide explanations when the properties fail. Ander-sson [7] presents many of the basic complexity results, including theinevitability of exponential blow up, even in the domain of SAT. Wediscuss a summary of these issues, and o!er a glimpse at the underlyingideas behind modern SAT tools. Unfortunately, space does not permitour detailed presentation of the use of SAT techniques or some of theirunusual applications in system verification (e.g., [46]). The reader mayfind many tutorials on the Internet or web sites such as [105].


18.3.2 Normal forms

We now discuss several properties of Boolean (or propositional) formu-las to set the stage for discussions about the theory of NP-completeness.There are two commonly used normal forms for Boolean formulas:the conjunctive normal form (CNF) and the disjunctive normal form(DNF).3 Given a Boolean expression (function) over n variablesx1 . . . xn, a sum-of-products (SOP) or disjunctive normal form (DNF)expression for it is one where products of literals are disjoined (OR-ed)– a literal being a variable or its negation. For example, nand(x, y) isexpressed in SOP (DNF) as

nand(x, y) = ¬x¬y + ¬xy + x¬y.

A POS (CNF) expression for nand would be (¬x + ¬y). A systematicway to obtain DNF and CNF expressions from truth tables is illustratedbelow: basically, the DNF form for a Boolean function is obtained bydisjoining (OR-ing) the min terms at which the function is a 1, whilethe CNF form for a Boolean function is obtained by conjoining (AND-ing) the max terms at which the function is a 0. All these are illustratedin Figure 18.1, and details may be found in any standard digital systemdesign text book (e.g., [12]).

Row x y nand(x,y) Minterm Maxterm0 0 0 1 m0 = ¬x ¬y M0 = (x + y)1 0 1 1 m1 = ¬x y M1 = (x + ¬y)2 1 0 1 m2 = x ¬y M2 = (¬x + y)3 1 1 0 m3 = x y M3 = (¬x + ¬y)

Fig. 18.1. Min/Max terms for nand, whose DNF form is m0 + m1 + m2 andCNF form is M3 (the only max term where the function is 0)

Here are important points pertaining to these normal forms:

Conversion between the CNF and the DNF forms incurs an exponen-tial cost. This is best illustrated by a conversion program included inFigures 18.2 and Figures 18.3.4 You are encouraged to run this code

3 Strictly speaking, we should call the normal forms being discussed here the canon-

ical sum-of-products form and the canonical product-of-sums form, since eachproduct term or sum term consists of n literals.

4 These programs do not prove that the conversion is exponential, but providestrong intuitions as to why.


under an Ocaml system. You will see that an exponential growthin formula size can occur, as demonstrated in the tests at the endof Figure 18.3. While these tests indicate the results on DNF toCNF conversion, the same complexity growth exists even for CNFto DNF conversion (Exercise 18.8 asks you to modify this programto one that obtains the DNF form of a given formula. The modifi-cations are based on the duality between * and +, and hence thecomplexity follows).CNF satisfiability will be shown to be NP-complete in the next chap-ter, whereas DNF satisfiability is linear-time (see exercise below).However, due to the exponential size blow up, one cannot “win”against NP-completeness by converting CNF satisfiability to DNFsatisfiability.

18.3.3 Overview of direct DNF to CNF conversion

Consider Figure 18.3 in which some terms representing DNF formulasare given. In particular, note that formula f5 is a 4-DNF formula witheight product terms. The expression List.length ( gencnf( f5 ) 1[] ).cnf converts this formula to CNF, and measures its length, whichis found to be 65536, showing the exponential growth in length. Hereis a brief explanation of this code.5

In Figure 18.2, the types for literals, clauses, and CNF formulas aregiven. The Ocaml List of lists [[1; 3; -2]; [1; -1]] stands for theCNF formula (x1 * x3 * ¬x2) + (x1 * ¬x1). The syntax of formulas,fmla, declares what formula terms are permissible. According to theseconventions, true is represented by [], i.e., an empty list of clauses,while false is represented by [[]], i.e., one empty clause.

The program proceeds driven by pattern matching. Suppose fmlamatches true (Tt); this results in an Ocaml record structure as shownin the following line:

Tt -> {cnf=[ ]; nvgen=0; algen=[]}

Note that [] is how true is represented. This is a list of lists with theouter list empty. Since the outer list is a list of conjunctions, an emptyouter list corresponds to the basis case of a list of conjunctions, namelytrue. Likewise, [[]] represents false because the inner list is empty, andits basis case (for a list of disjunctions) is false (Ff). We also record thenumber of new variables generated in nvgen, and an association list of

5 This code can convert arbitrary Boolean expressions to CNF - we choose toillustrate it on DNF.


type literal = int (* 1, 3, -2, etc *)type clause = literal list (* [1; 3; -2] -> [ ] means TRUE and [ [ ] ] means FALSE *)type cnffmla = clause list (* [[1; 3; -2]; [1; -1]] -> (x1 \/ x3 \/ ~x2)/\(x1 \/ ~x1)*)

type fmla =Ff | Tt | Pv of string | And of fmla * fmla | Or of fmla * fmla

| Eq of fmla * fmla | Imp of fmla * fmla | Not of fmla

type cnf_str = {cnf:cnffmla; (* CNF formula as a list *)nvgen:int; (* Variable allocation index *)algen: (string * int) list} (* Association list between variable-names

and integers representing them *)let rec gencnf(fmla)(next_var_int)(var_al) =match fmla with

Tt -> {cnf=[ ]; nvgen=0; algen=[]}| Ff -> {cnf=[[]]; nvgen=0; algen=[]}| Pv(s) ->

if (List.mem_assoc s var_al)then {cnf = [ [ (List.assoc s var_al) ] ];

nvgen = 0; algen = []}else {cnf = [ [ next_var_int ] ]; nvgen = 1; (* 1 new var generated *)

algen = [(s,next_var_int)]}

| And(q1,q2) ->let {cnf=cnf1; nvgen=nvgen1; algen=algen1}= gencnf(q1)(next_var_int)(var_al) in

let {cnf=cnf2; nvgen=nvgen2; algen=algen2}= gencnf(q2)(next_var_int + nvgen1)(algen1 @ var_al) in

{cnf=doAnd(cnf1)(cnf2); nvgen=nvgen1+nvgen2;algen=algen1 @ algen2}

| Or(q1,q2) ->let {cnf=cnf1; nvgen=nvgen1; algen=algen1}= gencnf(q1)(next_var_int)(var_al) in

let {cnf=cnf2; nvgen=nvgen2; algen=algen2}= gencnf(q2)(next_var_int + nvgen1)(algen1 @ var_al) in

{cnf=doOr(cnf1)(cnf2); nvgen=nvgen1+nvgen2;algen=algen1 @ algen2}

| Imp(q1,q2) -> gencnf(Or(Not(q1),q2))(next_var_int)(var_al)| Eq(q1,q2) -> gencnf(And(Or(Not(q1),q2), Or(Not(q2),q1)))(next_var_int)(var_al)| Not(Pv(s)) ->

if (List.mem_assoc s var_al)then {cnf = [ [ (0-(List.assoc s var_al)) ] ];

nvgen = 0; algen = []}else {cnf = [ [ (0-next_var_int) ] ]; nvgen = 1; (* 1 new var generated *)

algen = [(s,next_var_int)]}| Not(q1) ->

let {cnf=cnf1; nvgen=nvgen1; algen=algen1}= gencnf(q1)(next_var_int)(var_al) in{cnf=doNot(cnf1); nvgen=nvgen1;algen=algen1}

anddoAnd(cnf1)(cnf2) =(match (cnf1,cnf2) with

| ([],cnf2’) -> cnf2’| (cnf1’,[]) -> cnf1’| ([[]],_) -> [[]]| (_,[[]]) -> [[]]| _ -> List.append(cnf1)(cnf2) )

(* See PART-2 ... *)

Fig. 18.2. A CNF generator, Part-1 (continued in Figure 18.3)


anddoOr(cnf1)(cnf2) =

(match (cnf1,cnf2) with| ([],_) -> []| (_,[]) -> []| ([[]],cnf2’) -> cnf2’| (cnf1’,[[]]) -> cnf1’| ([cl1],[cl2]) -> [List.append(cl1)(cl2)]| (cnf1’,[cl2]) -> doOr([cl2])(cnf1’)| (cnf1’, (cl2::cls)) ->

let door1 = doOr(cnf1’)([cl2]) inlet door2 = doOr(cnf1’)(cls) in

doAnd(door1)(door2) )anddoNot(cnf1) =(match cnf1 with

| [] -> [[]]| [[]] -> []| (cl1::cls) ->

let compclause = comp_clause(cl1) inlet donot’ = doNot(cls) in

doOr(compclause)(donot’) )andcomp_clause(clause) =(match clause with

| [] -> []| (lit::lits) ->

let cl = 0-lit in (* complement literal *)let cl_cnf = [[cl]] in (* turn literal into CNF *)let rest = comp_clause(lits) in

doAnd(cl_cnf)(rest) );;(* To run tests, load these definitions, and type, e.g., gencnf(f5) 1 [];;let f1 = And(And(Pv "x", Pv "y"), And(Pv "z", Pv "w"));;let f2 = And(And(Pv "p", Pv "q"), And(Pv "r", Pv "s"));;let f3 = Or(f1,f2);;let f4 = Or(f3,f3);;let f5 = Or(f4,f4);;

f5 is really (xyzw + pqrs + xyzw + pqrs + xyzw + pqrs + xyzw + pqrs)

List.length ( gencnf( f5 ) 1 [] ).cnf;;- : int = 65536

Fig. 18.3. A CNF generator, Part-2 (continued from Figure 18.2)

variable names to their integer values in algen. These are respectively0 and [] for the case of Tt. Formula Ff is handled similarly.

A propositional variable is converted by the code under Pv(s). Welook up s in the association list var_al, and if found, generate its CNFwith respect to the looked-up result. Else, we generate a CNF clausecontaining next_var_int, the next variable to be allocated, and alsoreturn nvgen=1 and algen suitably records the association between sand next_var_int.

The remaining cases with propositional connectives are handled asshown: for And, we recursively convert its arguments, and call doAnd(Figure 18.3). The first four cases of doAnd deal with one of the argu-


ments being false or true. The last case simply appends cnf1 and cnf2,as they are already in CNF form. In case of doOr, we perform a moreelaborate case analysis. The case

| ([cl1],[cl2]) -> [List.append(cl1)(cl2)]

corresponds to two CNF clauses; here, we simply append the list ofdisjuncts to form a longer list of disjuncts. Given the case

| (cnf1’,[cl2]) -> doOr([cl2])(cnf1’)

This helps bring the first argument towards a single clause, at whichtime we can apply the append rule described earlier.

Let us discuss the last case:

| (cnf1’, (cl2::cls)) ->let door1 = doOr(cnf1’)([cl2]) inlet door2 = doOr(cnf1’)(cls) indoAnd(door1)(door2) )

We recursively OR cnf1’ with the head of the second list, namely cl2,and the tail of the list, namely cls, and call doAnd on the result.

These are the places which cause an exponential size growth offormulas.

18.3.4 CNF-conversion using gates

To contain the size explosion during the conversion of Boolean formu-las to CNF or DNF, one can resort to a circuit-based representationof clauses. This keeps the sizes of formulas linear, but introduces alinear number of new intermediate variables. Theoretically, therefore,the exponential cost remains hidden. This is because during Booleansatisfiability, an exponential number of assignments can be sought forthese new variables. In practice, however, we can often avoid su!eringthis cost.We now illustrate the idea of CNF conversion using gates through an ex-ample. Consider converting the DNF expression xyzw + pqrs to CNF.We build a circuit net-list as follows, by introducing temporary nodest1 through t7:

t1 = and(x,y) ; t2 = and(z,w) ; t3 = and(p,q) ; t4 = and(r,s)t5 = and(t1,t2) ; t6 = and(t3,t4) ; t7 = or(t5,t6)

Now, we convert each gate to its own CNF representation. We nowillustrate one conversion in detail:

t1 = x + y.


Treat it as t1 3 (x + y) and (x + y) 3 t1.The former is equivalent to (¬t1) * (x + y) which is equivalent to(¬t1 * x) + (¬t1 * y).The latter is equivalent to (¬x * ¬y * t1).Hence, t1 = x + y can be represented through three clauses.

The following ML functions accomplish the conversion of AND gatesand OR gates into clausal form (t1 = x + y was an example of theformer):

let gen_andG_clauses({gatetype = Andg; inputs = (i1,i2); output = output} as andG)= [ [-i1; -i2; output]; [-output; i1]; [-output; i2] ]

let gen_orG_clauses ({gatetype = Org; inputs = (i1,i2); output = output} as orG)= [ [i1; i2; -output]; [output; -i1]; [output; -i2] ]

Using these ideas, the final result for the example xyzw + pqrs is thefollowing 15 variables (8 variables were in the original expression, andseven more – namely, t1 through t7 – were introduced) and 22 clauses(7 gates, each represented using 3 clauses each; the last clause is theunit clause t7 — encoded as variable number 15).

x=1, y=2, z=3, w=4, p=5, q=6, r=7, s=8, t1=9, t2=10,t3=11, t4=12, t5=13, t6=14, t7=15

[ [-1; -2; 9] ; [-9; 1] ; [-9; 2] ] t1 = and(x,y)[ [-3; -4; 10] ; [-10; 3] ; [-10; 4] ] t2 = and(z,w)[ [-5; -6; 11] ; [-11; 5] ; [-11; 6] ] t3 = and(p,q)[ [-7; -8; 12] ; [-12; 7] ; [-12; 8] ] t4 = and(r,s)[ [-9; -10; 13] ; [-13; 9] ; [-13; 10] ] t5 = and(t1,t2)[ [-11; -12; 14]; [-14; 11]; [-14; 12] ] t6 = and(t3,t4)[ [13; 14; -15] ; [15; -13]; [15; -14] ] t7 = or(t5,t6)[ 15 ] t7

18.3.5 DIMACS file encoding

At this point, having generated the gate net-list and their clauses, wenow need to generate a file format representing the conjunction of theseclauses. The standard format employed is the so called DIMACS format,where the literals in each clause are listed on separate lines, terminatedby 0.

Given such a DIMACS file, one can feed the file to a Boolean sat-isfiability solver. Three examples of SAT solvers are [87, 81, 90], withreference [105] listing some of the latest news in the area of Booleansatisfiability.


p cnf 15 22 # Problem CNF format : Nvars and Nclauses.-1 -2 9 0 # Each line lists the literals of each clause.-9 1 0 # All lines end with a 0.-9 2 0-3 -4 10 0-10 3 0-10 4 0-5 -6 11 0-11 5 0-11 6 0-7 -8 12 0-12 7 0-12 8 0-9 -10 13 0-13 9 0-13 10 0-11 -12 14 0-14 11 0-14 12 013 14 -15 015 -13 015 -14 015 0

Type command ‘‘zchaff CNF-File’’ - here the file is "cnf"

The result of feeding the above file to zchaff is shown below:

[ganesh@localhost CH19]$ zchaff cnfZ-Chaff Version: ZChaff 2003.11.04Solving cnf ......22 Clauses are true, Verify Solution successful.Instance satisfiable

1 2 3 4 -5 -6 -7 -8 9 10 -11 -12 13 -14 15Max Decision Level 7Num. of Decisions 8Num. of Variables 15Original Num Clauses 22Original Num Literals 50Added Conflict Clauses 0Added Conflict Literals 0Deleted Unrelevant clause 0Deleted Unrelevant literals 0Number of Implication 15Total Run Time 0

SAT

The SAT solver Zchaff picked the assignment given by the line


1 2 3 4 -5 -6 -7 -8 9 10 -11 -12 13 -14 15

which stands for the Boolean assignment

x=1, y=1, z=1, w=1, p=0, q=0, r=0, s=0, t1=1, t2=1,t3=0, t4=0, t5=1, t6=0, t7=1

Suppose we force the assignment of x=0 by adding the clause ¬x: theassignment then becomes the one shown below, where p,q,r,s are setto 1:

-1 -2 -3 -4 5 6 7 8 -9 -10 11 12 -13 14 15

18.3.6 Unsatisfiable CNF instances

Consider the unsatisfiable CNF formula represented by the DIMACSfile in Figure 18.4: An important theorem pertaining to unsatisfiable

p cnf 4 161 2 3 4 01 2 3 -4 01 2 -3 4 01 2 -3 -4 01 -2 3 4 01 -2 3 -4 01 -2 -3 4 01 -2 -3 -4 0-1 2 3 4 0-1 2 3 -4 0-1 2 -3 4 0-1 2 -3 -4 0-1 -2 3 4 0-1 -2 3 -4 0-1 -2 -3 4 0-1 -2 -3 -4 0

Fig. 18.4. An unsat CNF instance

CNF formulas is the following:

Theorem 18.1. In any unsatisfiable CNF formula, for any assignment,there will be one clause with all its variables false and another with allits variables true.

Proof: We can prove this fact by contradiction, as follows. SupposeFmla is an unsatisfiable CNF formula and there is an assignment +under which one of the following cases arise (both violating the ‘one


clause with all its variables false and another with all its variables true’requirement):

There are no clauses with all its literals assuming value true. Inthis case, we can complement the assignment +, thus achieving asituation where no clause will have all false values.There are no clauses with all its literals assuming value false. In thiscase, Fmla is satisfiable.

If we feed the file in Figure 18.4 to a SAT solver, it will concludethat the formula is indeed unsatisfiable. In modern SAT solvers, it ispossible to find out the root cause for unsatisfiability. In particular, ifthe clauses of Figure 18.4 were embedded amidst many other clauses,upon finding the system to be unsatisfiable, a modern SAT solvingframework is capable of extracting the clauses of Figure 18.4 as theroot cause. This capability is being used during formal verification toprovide explanations for failed proofs (e.g., [91, 109]).

18.3.7 3-CNF, %=-satisfiability, and general CNF

A 3-CNF formula is a CNF formula in which every clause has threeliterals. For example,

(a * a * ¬b) + (c * ¬d * e)

is a 3-CNF formula. As can be seen, there is no requirement on whatvariables the clauses might involve. Many proofs are rendered easier bystandardizing on the 3-CNF form.

In many proofs, it will be handy to restrict the satisfying assign-ments allowed. One useful form of this restriction is the %=-satisfiabilityrestriction. Given a 3-CNF formula, a %=-assignment is one under whichevery clause has two literals with unequal truth values. Given a set ofN clauses where the ith clause is ci = yi1 * yi2 * yi3, suppose we trans-form each such clause into two clauses by introducing a new variablezi per original clause, and a single variable b for the whole translation:

ci1 = (yi1 * yi2 * zi) and ci2 = (¬zi * yi3 * b).

We can show that for any given formula Fmla = +i#Nci, the abovedescribed transformation is a polynomial-time mapping reduction &P

from 3-SAT to %=-sat.Finally, a general CNF formula is one where the number of literals

in each clause can vary (a special case where this number is & 2 isdiscussed in Section 18.3.8). A conversion from general CNF to 3-CNF


that does not increase the length of the formula beyond a polynomialfactor and takes no more than polynomial-time is now described (foreach clause with less than 3 literals, replicating some literal can increaseits size to 3; e.g., (a* b) = (a* b* b)). Consider a clause with l literals:

a1 * a2 * . . . * al.

This clause can be rewritten into one with l / 2 clauses:

(a1 * a2 * z1) + (¬z1 * a3 * z2) + (¬z2 * a4 * z3) . . . (¬zl!3 * al!1 * al).

Applying these transformations to a general CNF formula results inone that is equivalent as far as satisfiability goes. This is known asequi-satisfiable.

18.3.8 2-CNF satisfiability

A 2-CNF formula is one where each clause has two literals. One canshow that the satisfiability of 2-CNF is polynomial-time, as follows:

Consider one of the clauses l1 * l2. This can be written as two im-plications, (¬l1 3 l2) and (¬l2 3 l1).The complete algorithm is to process each clause of a 2-CNF formulainto such a pair of implications. Now, viewing each such implicationas a graph edge, we can connect its source to its destination; forexample, in (¬l1 3 l2), connect node ¬l1 to node l2. In any situation,if a double negation is introduced, as in ¬¬a, we must simplify it toa before proceeding.Finally, we connect pairs of edges such that the destination vertex ofone edge is the source vertex of the other. For example, if (¬l1 3 l2)is an edge, and (l2 3 l3) is another edge, connect these edges at thecommon point l2.

Now, the following results can be shown:

If the graph resulting from the above construction is cyclic and acycle includes p and ¬p for some literal p, the formula is unsatisfi-able.If this situation does not arise, we can perform a value assignmentto the variables (find a satisfying assignment) as follows:– If a literal x (for variable x) is followed by literal ¬x in the graph,

assign x false.– If a literal ¬x (for variable x) is followed by x in the graph, assign

x true.– Else assign x arbitrarily.


Chapter Summary

This chapter considered many topics in propositional and first orderlogic. The highlights were: (i) the undecidability of the validity of firstorder logic sentences, by reduction from PCP, and (ii) Boolean sat-isfiability from a theoretical and practical point of view. Armed withthis background, we will next study the very important problem ofNP-completeness.

Exercises

18.1. Using the operators used in the Hilbert style axiomatization ofSection 18.1, describe how to implement the familiar operators +, *,and ? (exclusive-OR). Also implement the constants true and false.

18.2.1. Describe what the following terms mean:

a) axiomb) rule of inferencec) theoremd) satisfiablee) prooff) tautology

2. Show that the following is a tautology:

(¬x 3 ¬y) = (y 3 x)

Recall that x = y means (x 3 y) + (y 3 x).

18.3. Prove the following using a similar approach as illustrated above(Sperschneider’s book, p. 107): Every barber shaves everyone who doesnot shave himself. No barber shaves someone who shaves himself. Provethat there exists no barber.

18.4. Show that Fmla2 is true under Interpretation 2 of page 328,describing the witness function (“#F”). Also show that it is false underInterpretation 3.

18.5. One often falls into the following trap of claiming that FOL for-mulas are decidable: the negation of every FOL formula is an FOLformula; thus one can enumerate proofs, and see whether F or ¬F getsenumerated in a proof, and hence decide the validity of F . What is thefallacy in this argument? (Hint: Are the cases listed, namely F beingvalid and ¬F being unsatisfiable, exhaustive)?


18.6. Argue that the set of all non-valid but satisfiable FOL formulasis not recursively enumerable.

18.7.1. Minimize the Boolean expression (¬a+ab)c+ac using a Karnaugh

map.2. How many Boolean functions can you build with a 4-to-1 multi-

plexer that has two select inputs?

18.8. Modify the program in Figures 18.2 and 18.3 to obtain a CNF toDNF converter. Test your resulting program.

18.9. Argue that DNF satisfiability and CNF validity have linear-timealgorithms.

18.10. Solve Exercise 5.29 through proof by contradiction. Encode theproblem as a CNF formula, and use a SAT tool.

18.11. Solve Exercise 5.30 similar to Exercise 18.10.

18.12. Write down the CNF formula represented by the DIMACS filegiven in Figure 18.4.

18.13. Prove that this transformation is indeed a polynomial-timemapping reduction &P from 3-SAT to %=-sat. In other words, provethat (i) it is a mapping reduction, and (ii) the runtime of the functioninvolved is a polynomial with respect to the input size.

18.14. Why does the conversion from general CNF to 3-CNF describedon page 341 preserve only satisfiability? (i.e., why do the formulas notemerge to be logically equivalent)

18.15. Prove that the 2-CNF satisfiability algorithm sketched in Sec-tion 18.3.8 is correct (meaning that it finds a satisfying assignmentwhenever one exists).

18.16. Apply the 2-CNF satisfiability algorithm of Section 18.3.8 tothe following 2-CNF formulas, showing the steps as well as your finalconclusion as to the satisfiability of these formulas:

(a * b) + (a * ¬b) + (¬a * b)(a * b) + (a * ¬b) + (¬a * b) + (¬a * ¬b)


21

Model Checking: Basics

21.1 An Introduction to Model Checking

The development of model checking methods is one of the toweringachievements of the late 20th century in terms of debugging concur-rent systems. This development came about in the face of the pressingneed faced by the computing community in the late 1970’s for e!ectiveprogram debugging methods. The correctness of programs is the cen-tral problem in computing, because there are often very designs thatare correct, and vastly more that are incorrect. In some sense, definingwhat ‘correct’ means is half the problem - proving correctness beingthe other half of the problem.

The quest for high performance and flexibility in terms of usage(e.g., in mobile computing applications) require systems (software orhardware) to be designed using multiple computers, processes, threads,and/or function units, thus quickly making system behavior highly con-current and non-intuitive. With the rapid progress in computing, es-pecially with the availability of inexpensive microprocessors, the com-puter user community found itself – in the late 1970’s – in a positionwhere it had plenty of inexpensive hardware but close to no practicaldebugging methods for concurrent systems! We will now examine thechain of events that led to the introduction of model checking in thissetting.

The enterprise of sequential program verification pioneered, amongothers, by Floyd [40], Hoare [55], and Dijkstra [37] was soon followed bythe quest for parallel program correctness, pioneered, among others, byOwicki and Gries [93], and Lamport [73]. The di"culty of these meth-ods when applied to real-world programs led to alternate proposals,such as relying on social processes [35]. Unfortunately, the arbitrari-

382 21 Model Checking: Basics

ness hidden in such proposals makes it di"cult to situate computationengineering in the same plane as other engineering disciplines where de-sign verification against rigorous specifications is the rule rather thanthe exception. Contrast the following guarantees: where the former islikely to be o!ered by a

“If I press the eject button, I am guaranteed to be safely ejectedfrom a burning airplane in less than 5 seconds.”

versus

“If I am lucky to be in a plane that was debugged by an expertreader of a program who happened to spot a bug, then I mightget ejected in a reasonable amount of time.”

In one thread of work that was evolving in the late 1970’s, somescientists, notably Pnueli [97], had the vision of focusing on concur-rency. In a nutshell, by focusing on control and not data, it becomespossible to model a system in terms of finite-state machines, and thenemploy decision procedures to check for its reactive properties. Even af-ter such simplifications, system control tends to be highly non-intuitive,and hence simply not amenable to any reasonable social processes. Au-tomated analysis of finite-state models can, on the other hand, au-tomatically hunt bugs and report them back. Pnueli’s vision lead toManna, Pnueli, and many others developing temporal logic proof sys-tems [79, 80].

We must admit that in this historical “sampler” that we are pre-senting, it is entirely possible that we have overlooked some key ref-erences, despite our best e!orts to prevent any such omissions. Onesuch unfortunately omitted citation from many recent works on modelchecking pertains to Carl Adam Petri’s [96] seminal work. Petri notonly proposed many basic ideas in concurrency and work-flow as earlyas 1963, but also saw the importance of focusing on control flow andsynchronization. Similarly, Hoare [56] and Milner [85] pioneered muchof the understanding of concurrency in terms of process algebras. How-ever, none of the early works had emphasized algorithmic approachessimilar to model checking, which is our focus in this chapter.

The breakthrough towards algorithmic methods for reasoning aboutconcurrent systems (as opposed to the initial proof theoretic methods)was introduced in the work of Clarke and Emerson [18], Queille andSifakis [99], and Clarke, Emerson, and Sistla [19]. This line of workalso received multiple fundamental contributions, notably from Vardiand Wolper who introduced an automata theoretic approach to auto-matic program verification [120], and a team of researchers at AT&T

21.1 An Introduction to Model Checking 383

Bell Laboratories, notably by Holzmann, Peled, Yannakakis, and Kur-shan [58, 51, 72, 59], who developed various ways to build finite-statemachine models and formally analyze them. Known as model check-ing, these methods relied on (i) creating a finite state model of theconcurrent system being verified, and (ii) showing that this model pos-sesses desired temporal properties (expressed in temporal logic). Graphtraversal algorithms were employed in lieu of deductive methods, thusturning the whole exercise of verification largely into one of buildingsystem models as graphs, and performing traversals on these graphswithout encountering state explosion.

State explosion—having to deal with an exponential number ofstates—is an unfortunate reality of model checking methods becausefinite-state models of concurrent systems tend to interleave in an ex-ponential number of ways with respect to the number of componentsin the system. E!ective methods to combat state explosion became thehot topic of research – but meanwhile model checking methods werebeing applied to a number of real systems, with success, finding deep-seated bugs in them! In [14, 13], Bryant published many seminal resultspertaining to binary decision diagrams (BDD), and following his popu-larization of BDDs in the area of hardware verification, McMillan [83]wrote his very influential dissertation on symbolic model checking. Thisis one line of work that truly made model checking feasible for certain“well structured,” very large state spaces, found in hardware modeling.The industry now employs BDDs in symbolic trajectory evaluationmethods (e.g., [2]).

Model checking has truly caught on in the area of hardware veri-fication, and promises to make inroads into software verification—thearea of “software model checking” being very actively researched at thetime of writing this very sentence. In particular, Boolean satisfiability(SAT) methods are being widely researched, as already discussed inSection 18.3. In modern reasoning systems, SAT and BDD methodsare being used in conjunction with first-order (e.g., [92, 110]) reasoningsystems, for example in tools such as BLAST [53]. In addition, higher-order logic (e.g., [3, 47, 94]) based reasoning systems also employ BDD,SAT, and even model checking methods as automated proof assistantswithin them. As examples of concrete outcomes, we can mention twosuccess stories:Model checking in the design of modern microprocessors: Allmodern microprocessors are designed to be able to communicate withother microprocessors through shared memory.1 Unfortunately since

1 Often these other microprocessors are situated on the same silicon chip


only one processor can be writing at any memory location at a giventime, and since “shared memory” exists in the form of multiple levelsof caches, with further levels of caches being far slower to access thannearly levels of caches, extremely complex protocols are employed toallow processors to share memory. Even one bug in one of these pro-tocols can render a microprocessor useless, requiring a redesign thatcan cost several 100s of millions of dollars. No modern microproces-sor is sold today without its cache coherence protocol being debuggedthrough model checking.Model checking in the design of device drivers: Drivers forcomputer input/output devices such as Floppy Disk Controllers, USBDrivers, and Blue-tooth Drivers are extremely complex. Traditional de-bugging is unable to weed out hidden bugs unless massive amounts ofdebugging time are expended. Latent bugs can crash computers and/orbecome security holes. Projects such as the Microsoft Research SLAMproject [9] have technology transitioned model checking into the realworld by making the Static Driver Verifier [8] part of the WindowsDriver Foundation [122]. With this, and other similar developments,device-driver writers now have the opportunity to model-check theirprotocols and find deep-seated bugs that have often escaped, and/orhave taken huge amounts of time to locate using traditional debuggingcycles.Has the enterprise of model checking succeeded? What about socialprocesses? We o!er two quotes:Model checking has recently rescued the reputation of formal methods[64].Don’t rely on social processes for verification [38].In this chapter and Chapter 22, we o!er a tiny glimpse of model check-ing basics as well as tools. The rest of this chapter is organized asfollows. Section 21.2 examines reactive systems. Section 21.3 discussesthe verification of safety and liveness properties. Section 21.4 illustratesthese ideas on one example—namely the Dining Philosophers problemof Dijkstra [37]—using the Promela language and SPIN model checker.

21.2 What Are Reactive Computing Systems?

Reactive computing systems are hardware/software ensembles thatmaintain an ongoing interaction with their environment, coordinatingas well as facilitating these interactions. They are widely used in all

21.2 What Are Reactive Computing Systems? 385

walks of life—often in areas that directly a!ect life. Examples of reac-tive systems include device drivers that control the operation of com-puter peripherals such as disks and network cards, embedded controlsystems used in spacecraft or automobiles, cache coherency controllersthat maintain memory consistency in multiprocessor machines, andeven certain cardiac pacemakers that measure body functions (suchas body electric fields and exercise/sleep patterns) to keep a defectiveheart beating properly. Model checking has already been employed inmost of these areas, with its use imminent in critical areas such as med-ical systems. Clearly, knowing a little bit about the inner workings ofa model checker can go a long way towards their proper usage.

21.2.1 Why model checking?

The design of most reactive systems is an involved as well as exactingtask. Hundreds of engineers are involved in planning, analyzing, as wellas building and testing the various hardware and software componentsthat go into these systems. Despite all this exacting work, at least twovexing problems remain:

• Reactive systems often exhibit nasty bugs only when field-tested.Unfortunately, at such late stages of product development, identi-fying the root cause of bugs as well as finding solutions or work-arounds takes valuable product engineering time. A manufacturercaught in this situation can very easily lose their competitive ad-vantage, as these late life-cycle bug fixes can cost them dearly—especially if they miss critical market windows.

• The risk of undetected bugs in products is very high,2 in the formof law-suits and recalls. Since software testing methods are seldomexhaustive, product managers have a very di"cult time decidingwhen to begin selling products.

Formal methods based on model checking are geared towards eliminat-ing the above di"culties associated with late cycle debugging. Whilemodel checking is not a panacea, it has established a track record offinding many deep bugs missed by weeks or months of testing. Specifi-cally,

• model checking is best used when a reactive protocol is in its earlyconceptual design stages. This is also the most cost-e!ective pointat which to eliminate deep conceptual flaws.

2 Software is often like a bridge that does not fail when subject to 100 tons or 101tons of weight, but suddenly collapse when 101.1 tons of weight are applied.


• model checking can return answers — either successful verificationoutcomes or high level counterexamples — often in a matter of a fewminutes to a few hours. In contrast, testing can wastefully explorevast expanses of the state-space over weeks or months of testing.Error location can also become nightmarishly hard during testing,as the state-space sizes are large, and because an astronomicallylarge number of computation steps may be executed from whenthe actual erroneous steps were carried out until when the systemcrashes or other symptoms of “ill health” are manifested.

In reality, the success of model checking can be attributed to severalfacts. Many of these facts are just pure “common sense,” not exclusiveto model checking in any way. Yet, it has been observed that modelchecking facilitates the use of such common sense! We now list the“virtues of model checking,” starting from the most pragmatic andgoing towards the more mathematical reasons.

Successive refinement:

Human thought seldom advances3 without the benefit of symbolicthought or successive refinement. Once one erects symbols and defines“rules of the game,” one can begin playing. In the same sense, ratherthan remain frozen with indecision in the face of full design complex-ity, designers adopting model checking methods have at their disposal avehicle for testing early prototypes and evaluating design alternatives.The models that a designer builds for reactive systems are largely finite-state machines. These finite-state machines can either be standard onesor embellished with extra information pertaining to communication orcomputation.

Exhaustively verify simplified models:

Even though a manually created design model can be defective, creatingone actually allows the designer to unload a piece of their mountingmental burden, and test the integrity of their thoughts through modelchecking experiments. The modern tendency in this area is to supplantmanual model construction with abstraction refinement techniques thatcan help one gradually discover, through tool assistance, what a suitableabstract model is. This approach promises to considerably reduce thelevel of e!ort needed in creating abstract models.

3 Children are known to have di#culties with symbolic thought, and have beenobserved to try inserting their feet into pictures of shoes.

21.2 What Are Reactive Computing Systems? 387

State-space analysis tools are the ‘heart’ of any model checker. Thesetools help exhaustively analyze finite-state machine models through acombination of techniques that help reduce memory requirements aswell as overall computation time. As Rusbhy points out [104], expe-rience has shown that the exhaustive analysis of finite-state modelsof reactive systems can often lead to the discovery of unexpected in-teractions (“corner cases”) and bugs much more readily than testingmethods can, for the same amount of resources (human and computertime) spent.

21.2.2 Model checking vs. testing

Model checking is a technique for verifying finite-state models of re-active computing systems for proper behavior. The intuitive notion of‘proper behavior’ is, in turn, formalized through the use of either tem-poral logic (e.g., linear-time temporal logic) or automata (e.g., Buchiautomata) which are used to express the desired properties of thesesystems. In contrast, testing is a well-established and expansive areacentral to product integrity, and employs both formal and informal cri-teria for coverage. Neither approach excludes the other; in fact, someof the most promising recent results are in combining model checkingand testing ideas, for instance as in [10].

Traditional testing-based debugging methods are known to missbugs due to many reasons. Typical systems being tested contain anastronomical number of states: 2109

states, for instance, in a memoryof capacity 1MB! While engineers are known to hand-simplify designsbefore testing them, in model checking, such simplifications are oftendone much more aggressively, to the point of turning control branchesinto nondeterministic selections. Such simplifications help abstract (or“smoothen”) the state-space modeled. BDDs (Chapter 12), and manyother symbolic representation methods for state spaces, have the (cu-rious) property that by adding more information (which helps overap-proximate state spaces, perhaps by adding some infeasible states) theactual BDD sizes are dramatically reduced. Hence, even though thenumber of states modeled may increase, the memory for representingthe states diminishes.

Two additional important benefits due to the use of nondetermin-ism are: (i) failure possibilities are introduced without increasing modelor state-space sizes, and (ii) the e!ect of testing for all values of cer-tain critical system parameters is obtained through nondeterminism.Of course these benefits come at a cost, and a designer who under-stands this cost/benefit ratio can often swing the overall balance in


favor of benefit. To understand these discussions, consider the processof debugging of a concurrent system that uses a ‘send’ communicationprimitive in a library. Assume that send uses a FIFO bu!er, and wouldblock if the bu!er were to be full. Further, suppose that such blockedsends are considered errors. If one were to allocate a certain numberof bu!er locations and test in the traditional sense, one would have towait for the right combinations of global states to arise in which thesend bu!er in question becomes full, and then only be able to observethe error. Furthermore, the input/state combinations under which thiserror can be observed during testing may di!er from those in the realsystem because, clearly, the designer would have downscaled all systemparameters before testing begins. To make matters worse, it is easy toeliminate errors while downscaling systems in an ad hoc manner.

In contrast to the testing approach described above, a model check-ing based approach to solving the same problem would consist of: (i) notmodeling bu!er capacities, and (ii) nondeterministically triggering abu!er full condition at every possible point in the state-space (this iscalled over-approximation of the reachable state-space). The advantageof this approach is that all possible bu!er capacities are being simu-lated in one fell swoop. The obvious disadvantage of this approach isthat a bu!er full may be simulated by the nondeterministic selectionmechanism when not warranted (e.g., at the very first send), and thetest engineer is forced to study and overrule many false alarms reportedby the model checker. This is often a small price to be paid for the thor-oughness of coverage o!ered by a model checker, especially given thattesting may not be feasible at all. We are really talking about model-ing several thousands of lines of actual code (which may be impossibleto test in any real sense) by a less than 100-line model checker inputdescription. In this setting, repeated running of model checking andoverruling errors is actually feasible.

It is possible that, despite the best precautions, over-approximationbased model checking can inundate the engineer with false alarms.There are many recently emerging solutions to this problem, the mostimportant of which is an approach based on abstraction refinement.This approach promises to enhance the power of model checking, whilehelping us handle larger models and at the same time avoiding in-accuracies arising due to hand abstractions created by the designer.In the counterexample guided abstraction refinement (CEGAR, [21])approach for abstraction/refinement, the initial abstraction is deliber-ately chosen to be too coarse. This abstraction is successively refinedby heuristically discovering aspects of the system which need to be

21.3 Buchi automata, and Verifying Safety and Liveness 389

captured in detail. The end result is often that systems are verifiedwhile keeping much of their behavior highly abstract, thus reducingthe overall complexity of verification.

21.3 Buchi automata, and Verifying Safety and Liveness

Buchi automata are automata whose languages contain only infinitestrings. The ability to model infinite strings is important because ofthe fact that all bugs can be described in the context of infinite exe-cutions. We now elaborate on these potentially unusual sounding, butrather simple, ideas. Broadly speaking, all errors (bugs) in reactive sys-tems can be classified into two classes: safety (property) violations andliveness (property) violations.

• Safety violations are bugs that can be presented and explained tosomeone in the form of finite executions (finite sequence of states)ending in erroneous states. Some examples of systems that exhibitsafety violations are the following:/ two people who, following a faulty protocol, walk opposite in a

narrow dark corridor and collide;/ an elevator which, when requested to go to the 13th floor, pro-

ceeds to do so with its doors wide open;/ a process P which acquires a lock L and dies, thus permanently

blocking another process, say Q, from acquiring L.All finite executions of the form s1 . . . sk can be modeled as infiniteexecutions that infinitely repeat the last state, namely s1s2 . . . (sk)%.Modeling finite executions as infinite executions allows one to em-ploy Buchi automata.

• Liveness violations are bugs that can be presented and explained tosomeone only in the form of an infinite execution in which a desiredstate never occurs. In practice, liveness violations are those thatend in a bad “lasso” shaped cyclic execution path which does notcontain the desired state. Examples of liveness violations are:/ two people who, following a faulty protocol, engage in a per-

petual ‘dance,’ trying to pass each other in a narrow well-litcorridor;

/ an elevator that permanently oscillates between the 12th and14th floors when requested to go to the 13th floor;

/ A process P which acquires a lock L precisely before anotherprocess Q tries to acquire it, and releases the lock preciselyafter Q has decided to back o! and retry; this sequence repeatsinfinitely.


All infinite executions in finite state systems can be cast into theform s1s2 . . . (sj . . . sk)% where the reachable bad cycle (sj . . . sk)%

is called the “lasso.” Liveness verification of finite state systemsreduces to finding one of these reachable lassos.

Model checking methods based on the use of Buchi automata helpdetect safety and liveness violations through language containment ofBuchi automata as is described in Section 21.4.1.

Outline of the rest of the chapter

In Section 21.4, we present the example of three dining philosophers—aslight variant of the example presented by Dijkstra in [37] to illustratethe principles of concurrent programming and synchronization. We donot study the whole gamut of solutions that have been discussed for thisproblem for nearly four decades. Instead we simply use this example forthe sake of illustration, since it is such a well-known example. In fact,we deliberately present a buggy solution and demonstrate how modelchecking can be employed to detect this bug. Section 21.4.1 informallypresents how to express models as well as properties as automata in thesyntax of Promela and find a liveness bug in the philosophers example.

21.4 Example: Dining Philosophers

0

12

C2

C4

C3

C1

C5 C0

F2

F1F0

Fig. 21.1. Three dining philosophers

Imagine three philosophers numbered P0, P1, and P2 seated arounda table with a bowl of spaghetti in the middle (Figures 21.1 and 21.2).They decide to eat out of the same spaghetti bowl as well as share forksas per the following rules:4 P0 would eat with F0 and F1 in his right4 Foregoing basic hygiene...

21.4 Example: Dining Philosophers 391

mtype = {are_you_free, yes, no, release}byte progress; /* SPIN initializes all variables to 0 */proctype phil(chan lf, rf; int philno){ do:: do

:: lf!are_you_free ->if:: lf?yes -> break:: lf?nofi

od;do:: rf!are_you_free ->

if:: rf?yes -> progress = 1 -> progress = 0

-> lf!release -> rf!release -> break:: rf?no -> lf!release -> breakfi

odod

}proctype fork(chan lp, rp){ do:: rp?are_you_free -> rp!yes ->

do:: lp?are_you_free -> lp!no:: rp?release -> breakod

:: lp?are_you_free -> lp!yes ->do:: rp?are_you_free -> rp!no:: lp?release -> breakod

od}init {

chan c0 = [0] of { mtype }; chan c1 = [0] of { mtype };chan c2 = [0] of { mtype }; chan c3 = [0] of { mtype };chan c4 = [0] of { mtype }; chan c5 = [0] of { mtype };atomic {

run phil(c5, c0, 0); run fork(c0, c1);run phil(c1, c2, 1); run fork(c2, c3);run phil(c3, c4, 2); run fork(c4, c5); }

}never { /* Negation of []<> progress */do:: skip:: (!progress) -> goto accept;od;accept: (!progress) -> goto accept;}

Fig. 21.2. Promela model for three dining philosophers


and left hands (respectively), while P1 would eat with F1 and F2, andP2 with F2 and F0. Clearly, when one philosopher ‘chows,’ the othertwo have to lick their lips, patiently waiting their turn. It is assumedthat despite their advanced state of learning, they are disallowed fromtrying to eat with one fork or reach across the table for the disallowedfork. In order to pick up a fork, a philosopher sends a message to thefork through the appropriate channel. For example, to acquire F0 or F1,respectively, P0 would send an are_you_free request message throughchannel C5 or C0, respectively. Such a request obtains a “yes” or a “no”reply. If the reply is “yes,” the philosopher acquires the fork; otherwisehe retries (if for the left fork) or gives up the already acquired left forkand starts all over (if for the right fork). Consequently, the deadlockdue to all the philosophers picking up their left forks and waiting fortheir right forks does not arise in our implementation. After finishingeating, a philosopher puts down the two forks he holds by sequentiallyissuing two release messages, each directed at the respective forksthrough the appropriate channels.

Figure 21.2 describes how the above protocol is described in thePromela modeling language. Consider the init section where we cre-ate channels c0 through c5. Each channel is of zero size (indicated by[0]) and carries mtype messages. Communication through a channel ofsize 0 occurs through rendezvous wherein the sender and the receiverboth have to be ready in order for the communication to take place.As one example, consider when proctype phil executes its statementlf!are_you_free. We trace channel lf and find that it connects toa philosopher’s left-hand side fork. This channel happens to be thesame as the rp channel as far as fork processes are concerned (rpstands for the ‘right-hand philosopher’ for a fork). These connectionsare specified in the init section. Hence, proctype fork must reachrp?are_you_free statement at which time both lf!are_you_free andrp?are_you_free execute jointly. As a result of this rendezvous, proc-types phil as well as fork advance past these statements in their re-spective codes.

Continuing with the init section, after “wiring up” the phil andfork processes, we run three copies of phil and three copies of fork inparallel (the atomic construct ensures that all the proctype instancesstart their execution at the same time). The never section specifies theproperty of interest, and will be described in Sections 21.4.1 and 22.1.Now we turn our attention to the proctypes phil and fork.

Proctype phil consists of one endless outer do/od loop, inside whichare two sequential do/od loops. The first of these loops has as its first


statement lf!are_you_free. This statement acts as a guard, in thesense that the execution of this statement allows the subsequent state-ment to be engaged in — in our example, this happens to be a nestedif/fi statement (sequencing is indicated by ->; in Promela, one mayuse -> and ; interchangeably to denote sequencing). An if/fi o!ersa selection of guards, and nondeterministically selects one of the armsthat can execute (a do/od is like an if/fi with a goto leading back tothe beginning). Hence, if lf?yes rendezvous with rp!yes, the first in-ner do/od loop breaks, and the second do/od loop is engaged. In this,the guard is rf!are_you_free. If the reply obtained is rf?yes, thephilosopher in question can begin eating. To indicate that a philoso-pher has successfully started eating, we set a progress flag to 1 andthen to 0. The attainment of progress=1 will be monitored by a neverautomaton. The forks are released and the execution continues with an-other iteration of the outer do/od loop. Notice that if the second arm ofthe if/fi statement is chosen (through lf?no), the lf!are_you_freeis retried. Therefore, phil repeatedly requests its left fork, and if/whensuccessful, requests its right fork. If a rf?no is obtained when request-ing the right fork, notice that the already acquired left fork is releasedvia lf!release and the whole process is restarted.

The process of acquiring both forks in the manner described aboveis based on two-phase locking—an algorithm known to be able to avoiddeadlocks. The idea is (i) to acquire all the required resources in somesequential order, and (ii) when some resource is found unavailable, in-stead of holding on to all the resources acquired thus far and waitingfor the unavailable resource (which can cause deadlocks), we releaseall the resources acquired thus far and restart the acquisition process.It is easy to see that while this approach avoids deadlocks, it can in-troduce livelocks. To detect the livelock in our example, we employ aproperty automaton expressed in the never section of a Promela model.Our property automaton (or “never” automaton) is designed to checkwhether the progress bit is set to true infinitely often. Even if one philproctype violates this assertion—meaning that it only sets progresstrue a finite number of times—we want the bug to be reported. Wedescribe this property automaton as well as how it finds the livenessviolation in the next section. In Chapter 22, we explain the relation-ship between property automata as well as linear-time temporal logicassertions.


21.4.1 Model (proctype) and property (never) automata

A typical Promela model consists of multiple proctypes and one neverautomaton. The proctypes are concurrent processes whose atomicstatements interleave in all possible ways. The collection of proctypesrepresents the “system” or “model.” Formally, one can say that theasynchronous or interleaving product of the model automata representsthe set of all behaviors of interest. A never or property automaton ob-serves the infinite runs of this interleaving product (all model runs).Conceptually, after every move of the model (a move of one of the con-stituent proctypes), one move of the property automaton is performedto check whether the model automaton move left the system in a goodstate. Formally, one can say that the synchronous product of the modeland property automata are performed. In Figure 21.2, the property au-tomaton represents the complement5 of the desired property—hence,the keyword never. One of the desired properties in our example isthat progress is infinitely often set true. This means that infinitelyoften, every philosopher gets to eat. If there is even one philosopherwho, after a finite amount of eatings, is never allowed to eat again, wecertainly would like to know that. The never automaton accepts a run(or a computation) if the run visits one of the final states infinitely. Thenever automaton defined in Figure 21.2 can nondeterministically loopin its initial do/od loop. In fact, it can even stay in its initial state ifprogress is false. However, it may also choose to go to the final statelabeled accept when progress is false. Here, it can continue to visitaccept so long as progress is false. Hence, this never automaton ac-cepts only an infinite sequence of progress being false—precisely whenour desired liveness property is violated.

When we ran the description in Figure 21.2 through the SPIN modelchecker (which is a model checker for descriptions written in Promela),it quickly found a liveness violation which indicates that, indeed, thereexists at least one philosopher who will starve forever. SPIN producesa message sequence chart (MSC) corresponding to this error; this isshown in Figure 21.3.

An MSC displays the “lifelines” (time lines of behaviors) of all theprocesses participating in the protocol. Ignoring the never and initlifelines that are at the left extremity, the main lifelines shown are verti-cal lines tracing the behaviors of various process instances, specificallyphi1:1, fork:2, phi1:3, fork:4, phi1:5, and fork:6. Reading thisMSC, we find out how the bug occurs:5 The complement of the property automaton is expected to be given by the user

as complemented automata are, in the worst case, exponentially bigger.


phil:59

fork:410

4!are_you_free

12

134!yes

15

fork:616

5!are_you_free

18

195!yes

phil:121

226!are_you_free

24

256!no

27

286!are_you_free

Cycle/Waiting30

316!no

33

346!are_you_free

3535

35phil:335

fork:235

35:init::0

35:never::!1

35

Fig. 21.3. Message sequence chart showing liveness violation bug

• phil:5 acquires fork:4 by sending it an are_you_free messageand obtaining a yes reply (the channel names have been convertedinto internal channel numbers in this MSC). It also acquires fork:6in the same manner.

• phil:1 attempts to acquire fork:6 by sending an are_you_freeand obtaining a no repeatedly. This is a liveness violation becausephil:1 is starving.

Chapter Summary

In this chapter, we examined the basic di!erences between informaland formal approaches to debugging concurrent reactive systems. Wepresented an overview of the Promela modeling language and the SPINverifier through a simple example. In the next chapter, we delve slightlydeeper into how temporal specifications are written, and some of thealgorithms used in enumerative model checking approaches that explic-itly enumerate states.


Exercises

21.1. Write a Promela model for a two-process mutual exclusion al-gorithm that is based on Dekker’s solution (many operating systemsbooks contain a description of this algorithm). State one safety andone liveness property, express these using never automata, and verifythese properties in turn.

21.2. Repeat Exercise 21.1, but following Peterson’s mutual exclusionalgorithm, again described in most books on operating systems.

21.3. Modify the dining philosophers protocol to eliminate the livenessbug described in Section 21.4.1. Also make sure that your solution isdeadlock free.

21.4. Even sequential programs are easily gotten wrong. This exerciseshows that even though transformational programs are most often re-quired to be analyzed through Floyd-Hoare-Dijkstra’s logic [40, 55, 37],they can sometimes be easily verified through finite-state methods aswell.

In an old textbook on Algorithms (name withheld), the followingBubble sort program is o!ered, accompanied by the assertion, “It takesa moment’s reflection to convince oneself first that this works at all,second that the running time is quadratic.”

procedure bubblesort;var j, t: integer;beginrepeatt:=a[1];for j:=2 to N do

if a[j-1]>a[j] thenbegin t:=a[j-1]; a[j-1]:=a[j]; a[j]:=t end

until t=a[1];end;

Examine the above program and find a bug such that the programcan exit with an unsorted array(!)6

Next, run the following Promela model encoding this Bubblesortprogram and find the bug in it.

#define Size 5#define aMinIndx 1#define aMaxIndx (Size-1)

6 It may be that the author meant to write a few side conditions, but going byexactly what he wrote, I assert that there is a bug in the program.


/* Gonna "waste" a[0] because Sedgewick uses 1-based arrays */active proctype bubsort(){ byte j, t; /* Init to 0 by SPIN */bit a[Size]; /* Use 1-bit abstraction */

/* Nondeterministic array initialization */do ::break ::a[1]=1 ::a[2]=1 ::a[3]=1 ::a[4]=1 od;

t=a[aMinIndx];j=aMinIndx+1;

do /* First ‘‘repeat’’ iteration */:: (j >(aMaxIndx)) -> break /*-- For-loop exits --*/:: (j<=(aMaxIndx)) ->

if:: (a[j-1] > a[j]) -> t=a[j-1]; a[j-1]=a[j]; a[j]=t:: (a[j-1] <= a[j])fi;j++

od;

do /* Subsequent ‘‘repeat’’ iterations */:: t!=a[1] ->

t=a[aMinIndx];j=aMinIndx+1;do:: (j >(aMaxIndx)) -> break /*-- For-loop exits --*/:: (j<=(aMaxIndx)) ->

if:: (a[j-1] > a[j]) -> t=a[j-1]; a[j-1]=a[j]; a[j]=t:: (a[j-1] <= a[j])fi;j++ /*-- for-index increments --*/

od /*-- end of for-loop --*/

:: t==a[1] -> breakod;

t=1; /*-- Comb from location-1 to look for sortedness --*/do:: t < aMaxIndx-1 -> t++:: t > aMinIndx -> t--:: a[t] > a[t+1] -> assert(0) /*- announce there is a bug! -*/od

}

21.5. Modify the Bubblesort algorithm of Exercise 21.4, recode inPromela, and prove that it is now correct.


22

Model Checking: Temporal Logics

This chapter presents two temporal logics, namely computational treelogic (CTL) and linear-time temporal logic (LTL). Section 22.1 presentsKripke structure, 22.1.2 presents computations versus computationtrees, and 22.1.3 show how LTL and CTL assertions serve as Kripkestructure classifiers. Section 22.1.4 presents key di!erences betweenCTL and LTL through some examples. Chapter 23 will present Buchiautomata as well as model checking algorithms.

22.1 Temporal Logics

22.1.1 Kripke structures

Kripke structures1 are finite-state machines used to model concurrentsystems. A Kripke structure is a four-tuple 6S, s0, R,L7 where S is afinite set of states, s0 is an initial state, R 5 S 2 S is a total relationknown as the reachability relation, and L : S ! 2P is a labeling func-tion, with P being a set of atomic propositions. It is standard practiceto require that R be total, as Kripke structure are used to model infi-nite computations (in modeling finite computations, terminal states areequipped with transitions to themselves, thus in e!ect making R total).The fact that we choose one initial state is a matter of convenience.

Figure 22.1 (bottom) presents two Kripke structures. In the Kripkestructure on the left, S = {s0, s1, s2}, s0 = s0, R is shown by thedirected edges, and L by the subsets of {a, b} that label the states. Inits behavior over time, this Kripke structure asserts a and b true in

1 Named after Saul Kripke, who is famous for his contributions to philosophy andmodal logic. Temporal logic can be viewed as an o!shoot of modal logic wherethe possible “worlds” are various points in time.

400 22 Model Checking: Temporal Logics

a=1

a=0

b=1

b=0

(a\/b)=1

(a\/b)=0Time

Logic Values

{a,b} {b}{a}{b} {a}

Time

Logic Values

{a,b} {b} {a} {b}{}

... ...

Waveform a\/b

Waveform a

Waveform b

s1

s2

s0 s1

s2 s3

s0bb

ba

a,b a,b

Fig. 22.1. Two Kripke structures and some of their computations. In theKripke structure on the left, the assertion ‘Henceforth (a # b)’ is true.

the first time step, b true in the second time step, and a in the thirdtime step. In the fourth time step, it may assert either a or b. One ofthese sequences is shown in the middle – namely, choosing to assert ain the fourth time step, and b in the fifth. When a label is omitted (e.g.,there is no b in state s2), we are e!ectively asserting ¬b. Therefore, theassertion in state s3 is ¬a+¬b. The corresponding “waveforms2” (as ifviewed on an oscilloscope) are shown at the top of this figure.

From these discussions it is clear that we model the behavior of sys-tems in terms of the truth values of a set of Boolean variables. Thisis the most common approach taken when modeling systems as finite-state machines. We therefore let P be the set of these Boolean variables.There are of course other ways to model systems. For instance, if one

2 Notice that we ‘connect the dots’ only for visual clarity. The values of the signalsshown are actually undefined in between various discrete time points. Dependingon the actual context, intermediate values may or may not be defined. For exam-ple, if a signal is the output of a clocked flip-flop, it may be valid even betweenthe points. Even in this case, it is only sampled at discrete points.

22.1 Temporal Logics 401

were to employ a set of Boolean variables B and a set of integer vari-ables I to model a system, P would consist of B as well as propositionsover integers, such as i > 2 or i = j +1. In any case, knowing the truthof the atomic propositions P , it is possible to calculate the truth of anyBoolean proposition built over P .

Each state s , S is an equivalence class of time points, while Rmodels how time advances. Whenever the system is at a state s , S, afixed set of Boolean propositions determined by L are true. Since S isfinite, we can immediately see that the temporal behavior of systemsmodeled using Kripke structures recur over time, thus basically gener-ating a regular language of truth values of the propositions in P . Forexample, Figure 22.1 shows how the truth of the Boolean propositiona* b varies with time. Since either a or b is always true, a* b is alwaystrue for the Kripke structure on the left; that is not the case for theKripke structure on the right, as we can enter state s3 where both aand b are false.

This chapter is concerned with how we can study such Kripke struc-tures through formal analysis. We will conduct this analysis by evaluat-ing temporal logic properties with respect to Kripke structures. We willstudy both linear sequences of runs through Kripke structures, such ascaptured by waveforms, as well as branching behaviors (to be discussedunder the heading of computation trees).

22.1.2 Computations vs. computation trees

In general, it is not su"cient to view the behavior of a system in terms ofcomputational sequences, such as captured by waveforms. To see why,consider Figure 22.2, where two scenarios of a user interacting witha machine (say, a vending machine) are shown in the top and bottomhalf. In each scenario, the behavior shown to the left of the @ sign is thatof the vending machine, while the behavior shown to the right of the @sign is that of the human customer. The upper vending machine acceptsa coin-drop (shown as a), and decides to allow the user to bang open itstrap door (b). Since there are two (nondeterministic) ways to open thetrap door, the user may be faced with a candy (c) or a donut (d) – neverboth at the same time! Since the user is always insistent on collectinga candy (c) after a b event, the system will deadlock if the machinechooses d instead of a c. Unfortunately, these “internal choices” madeby the machine cannot3 often be controlled from outside, and so suchmachines do exist in real life. The machine in the bottom half is not

3 Perhaps depending on the “mood” of the machine or weather conditions...


a

a a

a

b b

c d

b

c d

b

c

b

c

this...

...versus this

Fig. 22.2. The basic motivation for computation trees

a

b b

c d

b

d

a

c d

a

b

a

b

c d

a

b

c d

.... .... ........

a

a

b

....

b

....a

....

b

a

a,b

a

b b

c d

b

c

.... .... .... ....

Fig. 22.3. Computation trees for the two Kripke structures of Figure 22.2,as well as the left-hand side Kripke structure of Figure 22.1

prone to this deadlock, as it always leaves c and d enabled after a b.How does one model this subtle, but important di!erence between thetwo machines? Modeling the finite behaviors of the machines throughregular expressions, one obtains (ab(c + d))$ as the regular expression


for both machines — and so the use of regular expressions does not helpdistinguish the machines.4

Let us therefore take another approach to see how we can distin-guish these machines. In this approach, we unwind the machines intotheir infinite computation trees, as shown in Figure 22.3 for the Kripkestructures of Figure 22.2 and the left-hand side Kripke structure ofFigure 22.1. With these computation trees at hand, one can then askquestions such as:

“At all times, after seeing a b event being o!ered, does thereexist a path leading into a future where the c event is guaranteedto be o!ered?”

It is clear that the Kripke structures of Figure 22.2 are distinguishedby this query: the bottom machine provides this guarantee by alwaysallowing the user to pull out c, while the top machine might some-times precommit to o!ering only a d. Computational tree logic (CTL)considers not only the truths of atomic propositions along paths aslinear-time temporal logic (LTL) does, but also has the ability to con-sider: (i) whether there exists (E) paths at a state that satisfy a givencriterion, or whether for all (A) paths at a state, some criterion holds.However, this is not to say that CTL is superior to LTL. There are manypath-centered properties such as fairness that are expressible only inLTL and not in CTL. We will compare these logics in Section 22.1.4 aswell as in Section 23.3.1. We now come to the main idea behind LTLand CTL that allows them to be studied on an equitable basis.

22.1.3 Temporal formulas are Kripke structure classifiers!

CTL formulas can be viewed as classifiers of Kripke structures or asclassifiers of computation trees. Both these views are equivalent (know-ing the Kripke structure, we can obtain the computation tree, and viceversa). LTL formulas can also be viewed in the same manner. We nowexplain these ideas, beginning with CTL first.

4 Since infinite behaviors of reactive systems are of interest, LTL would view thevending machines as having the set of behaviors described by (ab(c + d))! (seeSection 2.8 for an explanation of $) while CTL would view them in terms ofinfinite computation trees.


CTL formulas are Kripke structure classifiers

Given a CTL formula #, all possible computation trees fall into twobins—models and non-models.5 The computation trees in the model(‘good’) bin are those that satisfy # while those in the non-model (‘bad’)bin obviously falsify #.Consider the CTL formula AG (EF (EG a)) as an example. Here,

• ‘A’ is a path quantifier and stands for all paths at a state• ‘G’ is a state quantifier and stands for everywhere along the path• ‘E’ is a path quantifier and stands for exists a path• ‘F’ is a state quantifier and stands for find (or future) along a path• ‘X’ is a state quantifier and stands for next along a path

The truth of the formula AG (EF (EG a)) can be calculated as follows:

• In all paths, everywhere along those paths, EF (EG a) is true• The truth of EF (EG a) can be calculated as follows:

/ There exists a path where we will find that EG a is true./ The truth of EG a can be calculated as follows:

$ There exists a path where a is globally true.

In CTL, one is required to use path quantifiers (A and E) and statequantifiers (G, F, X, and U) in combinations such as AG, AF, AX,AU, EG, EF, EX, and EU. More details are provided in Section 22.1.7.In other temporal logics such as CTL*, these operators may be usedseparately; see references such as [20] for details.

Coming back to the examples in Figure 22.1, AG (EF (EG a)) —which means,

Starting from any state s of the system (s0, s1, or s2 in ourcase), one can find a future reachable state t such that startingfrom t, there is an infinite sequence of states along which a istrue

is true of the Kripke structure on the left, but not the one on the right.This is because wherever we are in the “machine” on the left, we canbe permanently stuck in state s2 that emits a. In the machine on theright, a can never be permanent. As another example, with respect tothe Kripke structures of Figure 22.2, the assertion AG(b 3 EX c)(“wherever we are in the computation, if b is true now, that means

5 It is understood that for any formula % involving variables vars(%), the Kripkestructures that are considered to be models or non-models include all of thevariables vars(%) in their set of variables. In other words, these Kripke structuresmust assign all these variables; they may assign more variables, but not less.


that there exists at least one next state where c is true”) is true of thebottom Kripke structure but not the top Kripke structure.

LTL formulas are Kripke structure classifiers

Turning back to LTL, at its core it is a logic of infinite computationsor truth-value sequences (“waveforms”). For example, the LTL formula!(a * b) (also written as G (a * b) or “henceforth a * b”) is true withrespect to the computation (waveform) shown on the left-hand side ofFigure 22.1 against (a * b) = 1, while it is false with respect to thewaveform shown on the right.

It is customary to view LTL also as a Kripke structure (or compu-tation tree) classifier. This is really a simple extension of the basic ideabehind LTL. Under this view, an LTL formula # is true of a computa-tion tree if and only if it is true of every infinite path in the tree. Asan example, no LTL formula can distinguish the two Kripke structuresgiven in Figure 22.2, as they both have the same set of infinite paths.

22.1.4 LTL vs. CTL through an example

The di!erences between LTL and CTL are actually quite subtle.Consider Figure 22.4, and the CTL formula AG (EFx). In order for

s1s0

{x}{}

Fig. 22.4. AG (EF x) is true, yet there is a computation where x is perma-nently false

this formula to be true over this Kripke structure, in all paths start-ing from s0, everywhere along those paths, EFx must be true. This, inturn, means that there must exist a path where x is eventually found.Starting either from s0 or s1, we see that there exists a path on whichx is found true eventually. Hence, AG (EFx) is true of this Kripkestructure.

Let us try to pretend that the CTL formula AG (EF x) is equiva-lent to the LTL formula G (F x). An LTL formula is true of a Kripkestructure if and only if it is true of every infinite path in the Kripkestructure considered separately! Now consider the infinite path of spin-ning in state s0. For this infinite path, ¬x is permanently true. There-fore, we will violate Fx, and hence G (Fx). In short, this one path


serves as the “death knell” for G (F x) with respect to this Kripkestructure—whatever other paths might exist in this Kripke structure.Section 23.3.1 in Chapter 23 will have more to say about LTL versusCTL. We now introduce these logics formally.

22.1.5 LTL syntax

LTL formulas # are inductively defined as follows, through a context-free grammar:

# ! x, a propositional variable| ¬# negation of an LTL formula| (#) parenthesization| #1 * #2 disjunction| G# henceforth #| F# eventually # (“future”)| X# next #| (#1U #2) #1 until #2

| (#1W #2) #1 weak-until #2

Note that G is sometimes denoted by ! while F is denoted by ". Wealso introduce the X operator into the syntax above to capture thenotion of next in a time sense. It is clear that in many real systems—for example, in globally clocked digital sequential circuits—the notionof next time makes perfect sense. However, in reasoning about theexecutions of one sequential process Pi among a collection of parallelprocesses P1, . . . , Pn, the notion of next time does not have a uniquemeaning. As far as Pi is concerned, it has a list of candidate “next”statements to be considered; however, these candidate statements maybe selected only after an arbitrary amount of interleavings from otherprocesses. Hence, what is “next” in a local sense (from the point of viewof Pi alone) becomes “eventually” in a global sense. While conductingverification, we will most likely not be proving properties involving X.However, X as a temporal operator can help expand other operatorssuch as G through recursion. With this overview, we now proceed toexamine the semantics of LTL.

22.1.6 LTL semantics

Recall that the semantics of LTL are defined over (infinite) computa-tional sequences. LTL semantics can be defined over computation treesby conjoining the truth of an LTL formula over every computationalsequence in the computational tree. Let + = +0 = s0, s1, . . ., where the


superscript 0 in +0 emphasizes that the computational sequence beginsat state s0. By the same token, let +i = si, si+1, . . ., namely the infinitesequence beginning at si. By + |= # we mean # is true with respect tocomputation +; + %|= # means # is false with respect to computation +.Here is the inductive definition for the semantics of LTL:

+ |= x i! x is true at s0 (written s0(x))+ |= ¬# i! + %|= #+ |= (#) i! + |= #+ |= #1 * #2 i! + |= #1 * + |= #2

+ |= G# i! +i |= # for every i . 0+ |= F# i! +i |= # for some i . 0+ |= X# i! +1 |= #+ |= (#1U #2) i! +k |= #2 for some k . 0 and +j |= #1 for all j < k+ |= (#1W #2) i! + |= G#1 * + |= (#1U #2)

LTL example

Consider formula GF x (a common abbreviation for G(F x)). Its se-mantics are calculated as follows:

+ |= GFx i! +i |= Fx, for all i . 0+i |= Fx i! +j |= x, for some j . i

Putting it all together, we obtain the meaning as:

x is true infinitely often—meaning, beginning at no point in timeis it permanently false.

22.1.7 CTL syntax

CTL formulas . are inductively defined as follows:

. ! x a propositional variable| ¬. negation of .| (.) parenthesization of .| .1 * .2 disjunction| AG . on all paths, everywhere along each path| AF . on all paths, somewhere on each path| AX . on all paths, next time on each path| EG . on some path, everywhere on that path| EF . on some path, somewhere on that path| EX . on some path, next time on that path| A[.1 U .2] on all paths, .1 until .2


| E[.1 U .2] on some path, .1 until .2

| A[.1 W .2] on all paths, .1 weak-until .2

| E[.1 W .2] on some path, .1 weak-until .2

22.1.8 CTL semantics

The semantics of CTL formulas are defined over computation trees.Our approach to defining this semantics recurses over the structureof computation trees. We will also employ fixed-points to capture thissemantics precisely. Other approaches may be found in references suchas [20] and [83]. Some of the reasons for taking a di!erent approach6

are the following:

Some approaches (e.g., [20]) introduce a more general temporal logic(namely CTL") and then introduce CTL and LTL as special cases.We wanted to keep our discussions at a more elementary level.Our definitions make an explicit connection with the standard fixed-point semantics of CTL (discussed in Chapter 23).The view of LTL and CTL being Kripke structure classifiers (seeSection 22.1.3) is more readily apparent from our definitions.

Let a computation tree be denoted by / . Here are the notations weshall use (see Figure 22.5):

• / = / !. We are going to “exponentiate” / with tree paths as de-scribed in Section 15.6.2, with " denoting the empty sequence.

• The state at / ! is s(/ !).• / ! has -(/ !) + 1 branches numbered 0 through -(/ !). In e!ect, -

specifies the arity at every node of the computation tree. Since eachKripke structure has a total state transition relation R, -(/) . 0for any computation tree / .

• Branch 0 & j & -(/ !) leads to computation tree / j (for example,/0 is the computation tree rooted at the first child of / !, /1 is thecomputation tree rooted at the second child of / !, etc).

• Generalizing this notation, for a sequence of natural numbers $, / &

denotes the computation tree arrived at by traversing the branchesidentified in $. We call these the (computation) trees yielded by $(for example, / 1,0,1,1,2 is a computation tree obtained by traversingthe tree path described by 1, 0, 1, 1, 2, starting from / !).

• For an arbitrary sequence of natural numbers $, / & is not definedif any of the natural numbers in the sequence $ exceeds the corre-sponding - value, i.e., the arity of the tree at a certain level. We

6 Our definitions appear to resemble co-inductive definitions [86].


! !

!!

!!

!!

!!!

""

"""

""""..... !!

##

##

###

##

$$

$$

$$

$$$.....

!

! ! ! ! !

!

!!

!!

!!

!!!

%%

%%

%%

%%%

###

###

###

"""

"""

"""

$$

$$

$$

$$$

%%

%%

%%

%%%

.....

! !

! 0 !"(# !)

! 0,0 ! 0,1 ! 0,"(# 0) !"(# !),0 !"(# !),"(#$('!))

Fig. 22.5. Illustration of basic notions relating to computation trees

will make sure not to exceed the arity. Hence, s(/ &) is the state thatthe computation tree /& is rooted at, and -(/&) + 1 is the numberof children it has.

Basically, there are as many computation trees as states. Each stateuniquely defines a tree. Each tree is represented by a state at its root.Given that Kripke structures only have a finite number of states, it iseasy to see that with respect to any computation tree / , only finitelymany distinct computation trees may be yielded. Therefore, along everypath, the computation trees tend to recur. In the Kripke structure onthe left-hand side of Figure 22.1, / ! is the computation tree rooted at s0.Taking the self-loop at state s2 as its 0-th child, / 0,0,0 is the computationtree rooted at s2. This is the same computation tree as / 0,0,0,0, or as/0,0,0,1,0. Given all this, we will end up defining the semantics of CTLthrough mutual recursion over a finite number of computation trees.This recursion can then be solved by appealing to the least fixed-pointor the greatest fixed-point.More notation: For a computation tree / = / ! associated with a Kripkestructure (S, s0, R,L), define &(/) to be a set of tree paths such that forevery state s , S, there is exactly one path in &(/) that corresponds


to the path taken by standard depth first search (DFS) applied to theKripke structure to reach s. For example, for the Kripke structure ofFigure 22.1, &(/) = {", 0, 00}.Note: We really do not care about the exact search method; we merelywant &(/) to be a unique set. We use this definition of & towards theend of this page.We first define the semantics of the propositional logic subset of CTLformulas:

/& |= x i! x is true at s(/&) (written s(/&)(x))/& |= ¬. i! /& %|= ./& |= (.) i! /& |= ./& |= .1 * .2 i! /& |= .1 or /& |= .2

We now define the CTL semantics for the remaining CTL formulas,beginning with the AG operator.Notation: For sequences of natural numbers , and -, we write ,.- todenote their concatenation. For example, if , = 1, 2 and - = 3, then, . - = 1, 2, 3. Similarly, 1, 2 . 3, 4 = 1, 2, 3, 4.

• AG :

/& |= AG. i! /& |= . and /&.j |= AG . for all 0 & j & -(/&).

The first question that probably pops into one’s mind is: does thisrecursion define a unique semantics? To answer this question, noticethat the quantity being defined, generically written as the configuration“/& |= Fmla”, will recur since there are only a finite number of compu-tation trees. In e!ect, we have a standard finitary mutual recursion forwhich we can provide a fixed-point semantics. Illustration 22.1.1 willclarify these ideas through an example.

The intended semantics for ‘AG’ is that of the greatest fixed-point(the greatest fixed-point defines a fixed-point that is higher than anyother fixed-point in the implication order defined in Illustration 4.5.3).To cut a long story short, when a configuration “/ |= Fmla” recurs, onemust substitute true for the second occurrence, unravel the formula upto and including the second occurrence, and thus obtain a closed-formsolution.7 One can see that AG indeed computes the finitary conjunc-tion computed by the following equivalent formulation:

7 We refer the reader to [20] and/or [83] for an explanation of why greatest fixed-point computations begin with the “bottom” or “seed” element being true. Seealso the discussion on this topic in Chapter 6. Likewise, for the least fixed-pointcomputation, we use the seed of false.


/& |= AG. i! /&." |= . for all + , &(/&).

Therefore, AG. evaluates . at every state of the computation tree andtakes the conjunction of the results.We now proceed to define the remaining CTL operators through recur-sion, pointing out in each case which fixed-point is to be taken.

• AF :

/& |= AF. i! /& |= . or /&.j |= AF. for all 0 & j & -(/&).

Here, the least fixed-point is what is intended. This means that whena configuration “/& |= Fmla” recurs, we must substitute false forall such repeated occurrences, thus obtaining a closed-form solution.Again, Illustration 22.1.1 provides a simple example. Notice that theAF obligation is required of every path going forwards. In other words,in all paths, . holds somewhere.

• EG :

/& |= EG. i! /& |= . and /&.j |= EG. for some 0 & j & -(/&).

Here, the greatest fixed-point is what is intended. In other words, whena configuration “/ |= Fmla” recurs, we must substitute true for allsuch repeated occurrences, thus obtaining a finitary conjunction alongsome path.

• EF :

/& |= EF. i! /& |= . or /&.j |= EF. for some 0 & j & -(/&).

Here, the least fixed-point is what is intended, thus obtaining a finitarydisjunction along some path.

• AU :

/& |= A[.1 U .2] i! /& |= .2 or /& |= .1 and /&.j |= A[.1U.2]for all 0 & j & -(/&).

Here, the least fixed-point is what is intended. The least fixed-pointensures that the recursion will “bottom out” with .2 holding somewherealong all paths.

• EU :

/& |= E[.1 U .2] i! /& |= .2 or /& |= .1 and /&.j |= E[.1U.2]for some 0 & j & -(/&).

Here, the least fixed-point is what is intended. The least fixed-pointensures that the recursion will “bottom out” with .2 holding somewherealong some path.


• AW :

/& |= A[.1 W .2] i! /& |= AG.1 or /& |= A[.1 U .2].

We define AW by permitting the possibility of .2 not happening. Theidea behind EW is also similar. In e!ect, these are weakenings of AUand EU. Section 23.2.4 shows that this subtle variation is caused merelyby changing which fixed-point we choose: least fixed-point for AU, andgreatest fixed-point for AW. This is the reason why ‘U’ is termed thestrong until while ‘W’ is termed the weak until.

• EW :

/& |= E[.1 W .2] i! /& |= EG.1 or /& |= E[.1 U .2].

Illustration 22.1.1 Consider a Kripke structure with states {s0, s1, s2}and reachability relation s0 ! s1, s0 ! s2, s1 ! s0, and s2 ! s0.The labeling over one variable x is L(s2) = {x}, L(s1) = L(s0) = {}.For example, s2(x) = true, s1(x) = false, s0(x) = false.

s0

s1

s2{}{}

{x}

We show how to evaluate AG (EF x):

s0 |= AG (EF x) = + s0 |= (EFx)+ s1 |= AG(EFx)+ s2 |= AG(EFx)

s1 |= AG (EF x) = + s1 |= (EFx)+ s0 |= AG(EFx)

s2 |= AG (EF x) = + s2 |= (EFx)+ s0 |= AG(EFx)

Let

R = 6 s0 |= AG(EFx), s1 |= AG(EFx), s2 |= AG(EFx) 7

Also let #1, #2, and #3 represent tuple component selectors. Then wehave the following recursion in terms of R which represents the above“tuple of solutions:”


R = 6 s0 |= (EFx) + #1(R) + #2(R),s1 |= (EFx) + #2(R),s2 |= (EFx) + #0(R)

7.

The conjunctions can be solved using the greatest fixed-point (GFP)iteration, starting from “seed” 6T, T, T 7, standing for a tuple of threetrues. The GFP iteration proceeds as follows, with the R approximantsnamed R0, R1, R2, etc:

R0 = 6 T, T, T 7.

R1 = 6 s0 |= (EFx), s1 |= (EFx), s2 |= (EFx) 7.

R2 = 6 s0 |= (EFx) + s1 |= (EFx) + s2 |= (EFx),s1 |= (EFx) + s0 |= (EFx),s2 |= (EFx) + s0 |= (EFx)

7.R3 = 6 s0 |= (EFx) + s1 |= (EFx) + s2 |= (EFx),

s0 |= (EFx) + s1 |= (EFx) + s2 |= (EFx),s0 |= (EFx) + s1 |= (EFx) + s2 |= (EFx)

7.

The GFP has been attained.Now, we need to solve for EFx. Proceeding as before, define

S = 6 s0 |= (EFx), s1 |= (EFx), s2 |= (EFx) 7.

The recursive equation for S is

S = 6 s0 |= x * #1(S) * #2(S),s1 |= x * #0(S),s2 |= x #0(S)

7.

The LFP iteration proceeds as follows:

S0 = 6 F, F, F 7

S1 = 6 s0 |= x * F * F, s1 |= x * F, s1 |= x * F 7.i.e.,

S1 = 6 s0 |= x, s1 |= x, s2 |= x 7.


S2 = 6 s0 |= x * s1 |= x * s2 |= x,s1 |= x * s0 |= x,s2 |= x * s0 |= x

7.

S3 = 6 s0 |= x * s1 |= x * s2 |= x,s0 |= x * s1 |= x * s2 |= x,s0 |= x * s1 |= x * s2 |= x

7.

The LFP has been attained. So now, coming back to R, the item ofinterest, and substituting for S in it, and simplifying, we get:

R = 6 s0 |= x * s1 |= x * s2 |= x,s0 |= x * s1 |= x * s2 |= x,s0 |= x * s1 |= x * s2 |= x

7.

and so, s0 |= AG(EFx) is (s0 |= x * s1 |= x * s2 |= x). !

CTL example

Consider formula AG (EFx). Its semantics is calculated as follows:

/ ! |= AG EFx i! s(/ !)(EFx) and/ !.j |= AG EFx for all 0 & j & -(/ !).

This simply means that EFx is true at every state of the computationtree. This assertion is satisfied by the computation tree generated bythe Kripke structure shown in Figure 22.4, as was discussed in Sec-tion 22.1.4.

Chapter Summary

This chapter presented the syntax and semantics of LTL and CTL,both formally and informally. The next chapter will further concretizethis body of knowledge by presenting three model checking algorithms:an enumerative algorithm for CTL, a symbolic algorithm for CTL, andan enumerative algorithm for LTL.

Exercises

22.1. Based on the definitions in Section 22.1.1, argue that all compu-tational trees are infinite. Also argue that the number of children ofany node of a computation tree is finite (“finitary branching”).


22.2. In both LTL and CTL, show that the “until” operator ‘U’ canhelp realize the ‘G’ and ‘F’ operators.

22.3. Explain the semantics of all sixteen combinations of [.1 U .2]where .1 and .2 range over {true, false, .,¬.}. Explain which famil-iar temporal logic operator (if any) is modeled by each of the casesconsidered. Explain the overall semantics in all cases.

{b}

s0 s1

{a,b}

{a}

s2

{b}

s0 s1

{a,b}

{a}

s2

{b}

s0 s1

{a,b}

{a}

s2

{b}

s0 s1

{a,b}

{a}

s2

{a,b} {b}

{a}

s0 s1

s2 s3

{b}

s0 s1

{a,b}

{a}

s2

{b}

s0 s1

{a,b}

{a}

s2

{b}

s0 s1

{a,b}

{a}

s2

(a) {b}

s0 s1

{a,b}

{a}

s2 s3

(b) (c)

(d) (e) (f)

(g) (h) (i)

Fig. 22.6. Nine diagrams, some of which are Kripke Structures

22.4. Consider the nine state machines (a) through (i) in Figure 22.6.They are all drawn with respect to two variables, a and b. Which ofthem are Kripke structures and which are not?

22.5. We now discuss how to construct an LTL formula such that it istrue only of structure (b) of Figure 22.6, and not of structure (a). Weinclude an encoding of these Kripke structures in Promela as well asan LTL query that demonstrates this assertion.

/* Declare states that’ll be used in Kripke structures */mtype = {s0, s1, s2, s3}

/* Declare sa,... standing for kripke structure a, ... */mtype = {sa, sb, sd, se, sf, sg, sh}


byte state=s0; /* Init state to s0 */bit a=1; /* Initial values of a and b */bit b=1;

proctype generic(mtype structure){ if:: structure==sa ->

do:: d_step{state==s0;a=1;b=1} -> d_step{state=s1;a=0;b=1}:: d_step{state==s0;a=1;b=1} -> d_step{state=s2;a=1;b=0}

:: d_step{state==s1;a=0;b=1} -> d_step{state=s2;a=1;b=0}

:: d_step{state==s2;a=1;b=0} -> d_step{state=s2;a=1;b=0}:: d_step{state==s2;a=1;b=0} -> d_step{state=s1;a=0;b=1}od

:: structure==sb ->do:: d_step{state==s0;a=1;b=1} -> d_step{state=s0;a=1;b=1}:: d_step{state==s0;a=1;b=1} -> d_step{state=s1;a=0;b=1}:: d_step{state==s0;a=1;b=1} -> d_step{state=s2;a=1;b=0}

:: d_step{state==s1;a=0;b=1} -> d_step{state=s2;a=1;b=0}

:: d_step{state==s2;a=1;b=0} -> d_step{state=s2;a=1;b=0}:: d_step{state==s2;a=1;b=0} -> d_step{state=s3;a=0;b=0}

:: d_step{state==s3;a=0;b=0} -> d_step{state=s1;a=0;b=1}od

fi/*:: structure==sd -> ... similar ... */

}

init{ run generic(sb) }

/* sb satisfies this, and not sa *//*Type ‘spin -f "formula"’ and cut&paste the resulting never aut. *//*----------------------------------------------------------------*/never {/* !( !(<>([](a && b))) -> ((<>([]a)) || (<> (!a && !b)))) */T0_init:

if:: (! ((a)) && (b)) -> goto accept_S373:: ((((a)) || ((b)))) -> goto T0_initfi;

accept_S373:if:: ((((a)) || ((b)))) -> goto T0_init


fi;}/*--- in contrast, both sa and sb satisfy this --->never { /* !( !(<>([](a && b))) -> ( (<>([]a)) || (<> (b)) ) ) * /}<---*/

Understand the Promela code for the Kripke structures. In partic-ular note that the never automaton specifies a property true of sband not sa. This never automaton was synthesized using SPIN’sLTL translator.There is also another property given in the comments such that theproperty is true both of sa and sb. Translate this property into aBuchi automaton using SPIN’s LTL translator and check the resultson sa and sb.Come up with assertions that pairwise distinguish the remainingKripke structures and test your understanding using Promela andSPIN.

22.6. Consider the Kripke structures (d) and (f) of Figure 22.6.

Obtain two LTL formulas Pd and Pf that are true of (d) and false of(f); these formulas must capture as many of the features of structure(d) as possible. Demonstrate using SPIN that Pd and Pf are bothtrue of (d) and are both false of (f).Now obtain Qd and Qf that are both false of (d) and both true of(f), and demonstrate as above using SPIN.Now write down Rd and Rf which are two CTL formulas that areboth true of (d) and both false of (f) and check using NuSMV.Now write down Sd and Sf which are two CTL formulas that areboth false of (d) and both true of (f) and check using NuSMV.

Here is a hint of how one might code things in NuSMV (please find outwhich Kripke structure is modeled by this):

MODULE mainVAR state: {s0, s1, s2};

ASSIGN init(state) := s0;

next(state) := case state = s0 : {s1, s2};state = s1 : s2;state = s2 : {s2, s1};

esac;

DEFINE a := (state = s0) | (state = s2);b := (state = s0) | (state = s1);

SPEC AG (EF (a & b))

23

Model Checking: Algorithms

This chapter closes o! our presentation of model checking by present-ing three di!erent model checking algorithms at an intuitive level. Sec-tion 23.1 presents an enumerative algorithm for CTL model checking.Section 23.2 presents a symbolic algorithm for CTL model checking.Section 23.3 introduces Buchi automata, discusses how Boolean oper-ations on Buchi automata are performed, and that nondeterministicBuchi automata are not equivalent to deterministic Buchi automata.It also presents an enumerative algorithm for LTL model checking. Allalgorithms are presented through intuitive examples; however, we dopresent salient details such as the exact fixed-point computation process(for symbolic CTL model checking) or the nested depth-first search pro-cess (for enumerative LTL model checking). For further details, pleasesee references such as [20, 83].

23.1 Enumerative CTL Model Checking

We now present the basic ideas behind an enumerative (or explicitstate) method for CTL model checking through one example, reinforc-ing ideas presented in Chapter 22. In the enumerative approach, explicitgraph-based algorithms are employed, with states typically stored inhash tables. This is as opposed to representing and manipulating statesas well as state transition systems using BDDs. Our presentation of enu-merative CTL model checking contains excerpts from the paper [19] byClarke, Emerson, and Sistla.

Let us consider the problem of verifying the formula below at thestarting state of the Kripke structure presented in Figure 23.1:

(AG (OR (NOT T1) (AF C1)))

420 23 Model Checking: Algorithms

N1,N20

1 2

3 4 5 6

7 8

T1,N2

T1,T2 T1,T2

N1,T2

C1,N2 N1,C2

C1,T2 T1,C2

Fig. 23.1. A Kripke structure for critical sections

The algorithm proposed by [19] accomplishes this task, and in the pro-cess actually labels all states of this Kripke structure with all formulas,as well as subformulas of this formula, thanks to a recursive depth-firsttraversal over the formula structure. The algorithm runs in polynomialtime with respect to the size of the Kripke structure and the CTLformula to be checked. However, please bear in mind that the size ofthe Kripke structure may be exponential in the number of constituentprocesses, and so our polynomial is actually over an exponential.

The Kripke structure of Figure 23.1 represents the execution of twoparallel processes with alternating priorities, taking N to stand for “innon-critical region,” T for “trying to enter,” C for “in critical region,”and the numbers adjoining N, T, and C representing process IDs.

The first step of the model checking algorithm is to number theformula and its subformulas as shown in Figure 23.2. Basically, eachsubformula acquires a higher number in this scheme than all its includ-ing formulas:

The next step of the model checking algorithm consists of executingthe for loop shown in Figure 23.3 that considers the subformulas of thegiven formula bottom-up, and labels each state of the FSM with thosesubformulas that are true in that state (a ‘*’ in the table indicatesthat the formula in the column heading is true in the state shown onthe left).

In our example, we would invoke label graph on formulas C1, (AFC1), T1, (NOT T1), (OR ...), and (AG ...), in that order. While doing

23.2 Symbolic Model Checking for CTL 421

Formulanumber Formula

SubformulaList

1 (AG (OR (NOT T1)(AF C1))) (2)2 (OR (NOT T1)(AF C1)) (3 5)3 (NOT T1) (4)4 T1 nil5 (AF C1) (6)6 C1 nil

Fig. 23.2. Formula and its Subformulas

so, we generate and fill the table given in Figure 23.4. Each columnunder a subformula indicates whether the subformula holds in variousstates. Because of the order in which the subformulas are considered,the table can be filled using a recursive depth-first traversal. We presentthe pseudocode executed by label graph(AF C1) in Figure 23.3. Forfurther details, please see [19]. Figure 23.4 summarizes the results ofmodel checking on this example. We see that the property of interestis true at state 0, and so is true of the given Kripke structure (in fact,this property is true of every state).

23.2 Symbolic Model Checking for CTL

We will now present symbolic model checking algorithms for CTLthrough examples. Our first example is to evaluate EG p on the Kripkestructure of Figure 23.5(a), where p will be a ? b.

23.2.1 EG p through fixed-point iteration

The steps below show how to use the BED tool as a “calculator” inthis process.• First we obtain a fixed-point formulation for EG p.

EG p = p + (EX (EG p))

• We then realize that this can be solved through greatest fixed-pointiteration (iterating with “seed” equal to true). Iterating with a seedof “false” yields a solution for EG p equal to false—clearly not itsintended semantics!• We realize that we will need the TREL of this Kripke structure. Weobtain it in the same manner as already explained in Section 11.3.1 ofPage 192. The BED tool session is below:


for i := length(f) step -1 until 1do label_graph(fi)

procedure label_graph(f)var b : boolean; % dummy var{ %-- Do a case analysis based on the principal operator

CASE: principal operator of "f" is AF :beginST := empty_stack;for all s in S do marked(s) := false; -- S’ are the set of statesfor all s in S do -- of the Kripke structure.if not marked(s) then AF(f, s, b);

%-- Other cases are not shown...}

procedure AF(f, s, var b) -- var parameters can return values{if marked(s) then{if labeled(s,f) then

{ b := true; return }b := false; return}marked(s) := true;if labeled(s, f) then{ add_label(s, f); b := true; return }

push(s, ST);for all s1 in successors(s) do{ AF(f, s1, b1);if not(b1) then{ pop(ST); b := false; return }

}pop(ST); b := true; add_labels(s, f); return}%-- procedure AF

Fig. 23.3. Algorithm used in CTL model checking illustrated on the “AF”operator

bed> var a a1 b b1var a a1 b b1bed> let TREL =

(not(a) and b and a1 and not(b1)) or (a and not(a1) and b1) or(a and not(b) and b1) or (a and not(b) and a1)

bed> upall TRELUpall( TREL ) -> 53bed> view TREL ... (displays the BDD)


Statenumber

AtomicPropositions C1 (AF C1) T1 (NOT T1)

(OR(NOT T1)(AF C1))

(AG(OR(NOT T1)(AF C1)))

0 N1,N2 * * * *1 T1,N2 * * * *2 N1,T2 * * * *3 C1,N2 * * * * *4 T1,T2 * * * *5 T1,T2 * * * *6 N1,C2 * * * *7 C1,T2 * * * * *8 T1,C2 * * * *

Fig. 23.4. Table for our CTL model checking example

{b}

s0 s1

{a,b}

{a}

s2

(a)

s2s3

s1

{p}

{p} {q}

{p,q}s0(b)

Fig. 23.5. Kripke structures that help understand CTL

• In the BED syntax, a ? b is written a != b. Now we perform thefixed-point iteration assisted by BED. We construct variable namesthat mnemonically capture what we are achieving at each step:

EG_a_xor_b_0 = true -- first approximant

EG_a_xor_b_1 = (a != b) and (EX true) -- second approximant

This simplifies to (a != b), as (EX true) is true.Now, in order to determine EG_a_xor_b_2, we continue the fixed-

point iteration process, and write

EG_a_xor_b_2 = (a != b) and EX (a != b)

At this juncture, we realize that we need to calculate EX (a != b).This can be calculated using BED as follows:

bed> let EX_a_xor_b = exists a1. exists b1. (TREL and (a1 != b1))bed> upall EX_a_xor_bbed> view EX_a_xor_b


01

4: b

EX_a_xor_b: a

Fig. 23.6. BDD for EX (a != b)

Section 23.2.2 explains the details of why this represents EX_a_xor_b.The BDD in Figure 23.6 confirms that EX (a != b) is true at thosestates where a or b are true — i.e., the states {s0, s1, s2}. Now wecarry on with the fixed-point iteration:

bed> let EG_a_xor_b_2 = (a != b) and EX_a_xor_bbed> upall EG_a_xor_b_2bed> view EG_a_xor_b_2

We see that EG_a_xor_b_2 becomes equal to (a != b), and hence westabilize at this fixed-point.

The set of states in which EGa?b is true is a?b, which includesstates s1 and s2. Notice that state s0 is avoided because a? b isfalse there.

23.2.2 Calculating EX and AX

Calculating EX for a simple example

Let’s take a simpler example to begin with. Suppose for some machineM, whose state is described using two bits a and b, the TREL is as below,where a1 and b1 describe the next states:

TREL = (!a and !b and a1 and b1) or (!a and b and !a1 and b1)

Basically, machine M has the following moves:

00 -> 11 and 01 -> 01.

Suppose we want to find out all starting states from which M has atleast one move that ends up in a state which satisfies (a != b) i.e., EX(a != b). Here is a portrayal of the situation:

Present State Next State Next States Satisfying (a != b)00 11 This next state does not satisfy01 01 This next state does satisfy


So we basically perform

STARTSTATES = exists a1. exists b1. ( TREL and (a1 != b1) ).

This simplifies to !a and b. Indeed, these are the states that have amove to a state satisfying (a1 != b1).

If we wanted to calculate those states that have at least one nextstate that satisfies !a1 and !b1, we would have obtained STARTSTATES= false (try it using BED). This is of course correct because there areno “next states” where !a1 and !b1.

Calculating AX

If we have to calculate AX p, we would employ duality and write it as

!(EX !p)

This approach will be used in the rest of this book.

Calculating EX for the example of Section 23.2.1

As said earlier on Page 424, the following formula represents EX_a_xor_b:

EX_a_xor_b = exists a1. exists b1. ( TREL and (a1 != b1) ).

TREL captures constraints between a, b, a1, and b1, saying which a,bpairs can move to which other a,b pairs. Of course, the next values ofa and b are modeled using a1 and b1. Now, in the resulting new states,a and b must satisfy the formula (a != b). Since it is the next statethat must satisfy the XOR constraint, we write a1 != b1. However, itis the present state we must return. So, we constrain the next state andthen quantify away a1 and b1.

23.2.3 LFP and GFP for ‘Until’

We now will perform the least fixed-point iteration sequence for cal-culating A[p U q] with respect to Kripke structure (b) of Figure 23.5.This will be followed by the greatest fixed-point iteration sequence forA[p U q] with respect to the same Kripke structure. As Section 22.1.8observed, these respectively obtain the strong Until and weak Untilsemantics.

The recursion over which these semantics are calculated is the fol-lowing:

A[pUq] = q * (p + AX (A[pUq]))

We will calculate AX using duality, as explained in Section 23.2.2.


23.2.4 LFP for ‘Until’

0 1

45: q1

58: q

59: p1

9: q1 41: q

66: p1

TREL: p

0 0 1

A_p_U_q_1: q

0 1

A_p_U_q_1: q 41: q

EX_not_q: p

0 1

41: q A_p_U_q_1: q

AX_q: p

0 1

A_p_U_q_1: q

Fig. 23.7. The Strong Until (U) iteration that reaches a fixed-point

bed> var p p1 q q1bed> let TREL = (p and not(q) and p1 and not(q1))

or (p and not(q) and p1 and q1)or (p and q and not(p1) and q1)or (not(p) and q and p1 and not(q1))or (p and not(q) and not(p1) and q1)or (p and not(q) and p1 and not(q1))

bed> upall TRELUpall( TREL ) -> 67bed> view TRELbed> let A_p_U_q_0 = falsebed> let AX_A_p_U_q_0 = falsebed> let A_p_U_q_1 = (q or (p and AX_A_p_U_q_0))


bed> upall A_p_U_q_1Upall( A_p_U_q_1 ) -> 3bed> view A_p_U_q_1bed> let EX_not_q = exists p1. exists q1. (TREL and !q1)bed> upall EX_not_qUpall( EX_not_q ) -> 80bed> view EX_not_qbed> let AX_q = !EX_not_qbed> upall AX_qUpall( AX_q ) -> 82bed> view AX_qbed> let A_p_U_q_2 = (q or (p and AX_q))bed> upall A_p_U_q_2Upall( A_p_U_q_2 ) -> 3bed> view A_p_U_q_2 --> gives ‘‘q’’, hence denotes {S1,S2} -- LFP

The results of all the view commands are pooled in Figure 23.7. Thefinal fixed-point reached is q, which means that states s1 and s2 areincluded in it. Clearly, these are the only states from which it is thecase that all paths satisfy A[pUq]. Therefore, we have the following:

Starting from either s0 or s3, there is the possibility of neverencountering a q, and hence these states are eliminated by thestrong until semantics of AU.

23.2.5 GFP for Until

Taking the greatest fixed-point, we obtain the ‘W’ (weak Until) se-mantics. We give the BED scripts, leaving it to the reader to try them.We indeed reach the greatest fixed-point of q * p, which covers states{S0,S1,S2,S3}. We also see that from these states, we either have pholding forever, or p holding until q holds.

bed> let A_p_U_q_0 = truebed> let AX_A_p_U_q_0 = truebed> let A_p_U_q_1 = (q or (p and AX_A_p_U_q_0))bed> upall A_p_U_q_1Upall( A_p_U_q_1 ) -> 72view A_p_U_q_1bed> let EX_not_p_or_q = exists p1. exists q1. (TREL and !(p1 or q1))bed> upall EX_not_p_or_qUpall( EX_not_p_or_q ) -> 0bed> let AX_p_or_q = !EX_not_p_or_qbed> upall AX_p_or_qUpall( AX_p_or_q ) -> 1bed> view A_p_U_q_1bed> let A_p_U_q_2 = (q or (p and AX_p_or_q)) --> reached

Fixed-point (q or p) which denotes {S0,S1,S2,S3}


23.3 Buchi Automata and LTL Model Checking

We now revert to a more detailed look at LTL. In particular, we presentmaterial leading up to a study of enumerative LTL model checkingusing Buchi automata.

Buchi automata come in the deterministic and nondeterministic va-rieties; the latter are strictly more expressive than the former. There-fore, by the term “Buchi automata,” we will mean nondeterministicBuchi automata (NBA). NBA are structures (Q,%, *, q0, F ) where Qis a finite non-empty set of states, % is a finite non-empty alphabet,* : Q2%! ! 2Q is a total transition function, q0 , Q is an initial state,and F 5 Q is a finite, possibly empty set of final states. An infiniterun (or computation) + = s0, s1, . . . is in the language of a Buchi au-tomaton if there is an infinite number of occurrences of states from Fin +. Notice that the graphs in Figure 23.8, when viewed as an NFA,

a

b

a

b

Fig. 23.8. Transition graphs read as an NFA or as an NBA

have di!erent languages while they have the same language (ab)% whenviewed as an NBA. In Section 23.3.2, we define the notion of 0-regularsets which are similar to regular sets, except, in addition to a Kleenestar operator, we also have an infinite iterate operator, ()%. This willhelp us define the notion of infinite runs.

23.3.1 Comparing expressiveness

From the discussions in Sections 22.1.3 and 22.1.4, it is clear that whenLTL and CTL formulas are viewed as Kripke structure classifiers (eachformula partitions all Kripke structures into two bins “models” and“non-models”), LTL tends to forget the branching structure, focusingonly on the “straight-line runs” while CTL considers the branchingstructures. For example, an important fact about the example in Fig-ure 22.4 is that in whichever state the computation is situated, thereexists a path leading to a state where x is true.

We now state some facts pertaining to the expressiveness of LTL,CTL, and Buchi automata. Consider all possible Kripke structures

23.3 Buchi Automata and LTL Model Checking 429

to be the set K. Let KV 5 K be Kripke structures that involveall the variables of V (assign all the variables in V , and possiblymore). Let ka, ka1

, ka2, . . . , kb, kb1 , kb2 , . . . refer to various subsets of

KV (with a, b, a1, b1, . . . ranging over a suitable index set), and letka, ka1

, ka2, . . . , kb, kb1 , kb2 , . . . be the complements of these sets relative

to KV . Given all this, we state a few facts pertaining to the expres-siveness of various temporal logics and automata; for details, see [59]as well as [112].

• There exist sets ka, ka1, . . . such that for any one of these sets, say

kaj, there exists an LTL formula #LTL

ajthat regards all of kaj

as its

models and all of kajas its non-models exactly when there exists

a CTL formula #CTLaj

that also regards kajas its models and all of

kajas its non-models. In other words, partitions such as kaj

capturetemporal patterns that are equally expressible in both these logics./ An example of a temporal fact that corresponds to such a par-

titioning would be “henceforth a is true.”• There are partitionings, say kbj

, such that there exist only LTL

formulas #LTLbj

that regard all of kbjas its models and all of kbj

asits non-models. In other words, the partitioning expressed by kbj

captures temporal patterns expressible only in LTL./ An example of a temporal fact that corresponds to such a par-

titioning would be “infinitely often a.” Any CTL formula oneattempts to write down to e!ect such a classification would endup “lying”—it would either put a Kripke structure that is cor-rect with respect to this behavior into the non-model bin or puta Kripke structure that is incorrect with respect to this behaviorinto the model bin.

• There are partitioning kcjfor which only CTL formulas exist and

no LTL formulas exist.

/ A temporal fact that corresponds to such a partitioning is theAG (EFx) example considered earlier.

• There exist CTL* formulas in all the above cases. In fact, using alogical connective, one can combine an LTL formula that has noequivalent CTL formula and a CTL formula that has no equivalentLTL formula into a CTL* formula that can now express somethingthat neither LTL nor CTL can express.

• The never claim language of Promela allows (negated) Buchi au-tomata to be specified. All LTL specifications have correspondingBuchi automata specifications, but not vice versa. An example of a


temporal pattern expressible using Buchi automata but not linear-time temporal logic is

p can hold after an even number of execution steps, but neverholds after an odd number of steps.

• A logic called modal µ-calculus properly subsumes all of the abovelogics in terms of expressiveness.

23.3.2 Operations on Buchi automata

The language of a Buchi automaton is a 0-regular set of strings over agiven set %. Following [115], we now inductively define 0-regular sets;this definition involves the notion of a regular set over % as defined inChapter 10. For completeness we repeat that definition also here (albeitin terms of context-free productions); we employ two non-terminals reand ore standing for regular expressions and 0-regular.

Please note: ab%c does not make sense as an infinite string,as there is “no sensible way” in which we can a"x c after aninfinite run of b’s. In other words, infinite sequences will be “tailinfinite.”

re ! 0 | " | a , % | (re) | re1 + re2 | re1re2 | re"

ore ! re% | re ore | ore1 + ore2

With respect to the above grammar, we can now define operations onBuchi automata.

Union of Buchi Automata: We can stitch the two Buchi automata inquestion using ", much like we perform the corresponding operationwith NFAs.

Concatenation: We may stitch an NFA and a Buchi automata in thatorder, as specified in the grammar for 0-regular sets.

Complementation: Buchi automata are closed under complementation,although the resulting Buchi automata are exponentially bigger,and the algorithm for complementation is extremely complex (ref-erences such as [43, 49] provide some details). For this reason, usersare often required to manually come up with negated Buchi au-tomata, as is the case with never automata in SPIN.In Section 23.3.3 we prove that DBAs are not closed under comple-mentation.

Intersection: The standard intersection algorithm no longer works, asone has to keep track of when a single infinite run is in the languageof the constituent Buchi automata, visiting final states infinitely in

23.3 Buchi Automata and LTL Model Checking 431

both of them. This algorithm is given in many references (e.g., see[20]).

23.3.3 Nondeterminism in Buchi automata

q0 q1

a,b

b

b

b

b

q0 q1

a

aFig. 23.9. An NBA accepting finitely many a’s (left) and a DBA acceptinginfinitely many a’s (right)

We now prove that there is at least one NBA—in particular, the oneon the left-hand side of Figure 23.9—that has no corresponding DBA.The language of this NBA is (a + b)"b%. Suppose there is a DBA Dwith this language. Clearly, then, b% is in L(D). Hence, there exists anumber n1 , Nat such that after encountering bn1 , the DBA will bein some final state – say x1 – of D for the first time. Note that if Dnever visits a final state, it cannot have any string in its language, letalone b%. Moreover, any given string takes D from a given state s to aunique state s

!.

Now, consider string bn1ab%: it is also in L(D). Hence there existsn2 such that bn1abn2 takes D to the first final state after x1 – say x2 –in D. Continuing this way, bn1abn2a . . . bnk first takes D to some finalstate xk in D. But all these states xi cannot be distinct, as the numberof states is finite. Assume that state xi is the first one to repeat, andthe first repeated occurrence is identified by x

!

i. This means that thereis an accepting cycle q0 . . . xi . . . x

!

i where the loop xi . . . x!

i includes ana. In turn, this means that there is some string with an infinite numberof a’s in L(D). This is a direct contradiction with respect to what thelanguage L(D) is supposed to be. Therefore, a DBA D does not existfor this language.

Also, note that DBAs are not closed under complementation. Themachines in Figure 23.9 are complements, and yet the one on the right-hand side is a DBA whereas we just now proved that the machine onthe left has no equivalent DBA.


23.4 Enumerative Model Checking for LTL

{}

{x}

{x}{}

{}

B

A P

Q

x

x x

x

Fig. 23.10. A Kripke structure (left), its corresponding Buchi automata (mid-dle), and a property automaton expressing GFx (right)

We now provide a glimpse of the enumerative model checking al-gorithm used by model checkers such as SPIN. Consider the Kripkestructure given in Figure 23.10 on the left-hand side. In its initial state,x is false. In one of the computations of this Kripke structure, x wouldremain false forever; in all others (an infinite number of them exist), xwould become infinitely often true. This Kripke structure can be viewedas a Buchi automata as shown in the middle. Notice how we go fromstate labels to edge labels, as is our convention when diagramming thestate transition relation of automata. We are, in e!ect, declaring thealphabet of these Buchi automata to be powersets of atomic propo-sitions. Also, by virtue of the fact that all states are final states, weare also asserting that all infinite runs of such Buchi automata are ofinterest to observe.

{x}{}

{}

B

A P

Q

x

x

1 1

x

x1x x

x

BP

AP BQ

AQ

Fig. 23.11. System automaton (left), complemented property automaton(middle), and product automaton (right)

23.4 Enumerative Model Checking for LTL 433

When it comes to formal verification, we would like to state proper-ties of interest in either temporal logic or in terms of Buchi automata.From the discussions in Section 23.3.1, we can gather that anythingthat can be stated in LTL has a corresponding Buchi automata ver-sion. An example of this appears in the right-hand side of Figure 23.10.Here, we present a Buchi automata that expresses GFx. In other words,the language of this Buchi automata is exactly those computations thatsatisfy the LTL formula GFx.

23.4.1 Reducing verification to Buchi automaton emptiness

Notice that the “property automaton” (P ) on the right-hand side ofFigure 23.10 includes all runs that satisfy GFx. Therefore, to determinewhether a given Kripke structure (“system” S) satisfies a property P ,we can check whether

L(S) 5 L(P ).

This check is equivalent to

L(S) ' L(P ) = 0.

Figure 23.11 shows the system automaton (left), the complementedproperty automaton (middle), and the product automaton (right) real-izing the intersection (') operator, above. Since the “system automa-ton” has all its states being final states, this intersection is obtainedthrough the standard NFA intersection algorithm. In particular, a stateof the product machine is final if and only if the constituent states ofthe component machines are both final states.

We notice that the intersection gives a Buchi automata whose lan-guage is not empty. In particular, it has an accepting run where theproduct state AP leads to product state AQ which repeats infinitely.Hence,

L(S) ' L(P ) %= 0.

When this condition is true, a bug has been found (i.e., the propertyhas been violated). This accepting run can be displayed as an error-trace or a MSC. Consequently, the debugging of concurrent systemscan be reduced to emptiness checking of Buchi automata.

Explicit enumeration algorithms, such as used in tools like SPIN,find property violations and depict them in an intuitive fashion, suchas shown in the MSC of Figure 21.3. In summary, here are the stepsthey employ (for details, see references such as [20]):


S5

S4

S3 S1

S2

S0

2 2

1 1S 0

S 1

S 3

4SS 2

S 0

S 1

S 3

4S

S 0

S 1

S 2

S 1

4SS 2

S 1

S 2

4SS 2

S 1

S 3

4S

Fig. 23.12. Searching of a product graph for cycles: Example product graph(left), group of stacks for the outer DFS (middle), and group of stacks for theinner DFS (right)

• Obtain the system automaton by performing an interleaving (asyn-chronous) product of the system proctypes. In e!ect, we pick oneof the enabled processes nondeterministically, and perform one if itsenabled moves nondeterministically. This is called taking the “sys-tem step,” and corresponds to the interleaving product mentionedin Section 21.4.1.

• Next, we take a “property step” by taking the never automaton(negated property automaton) and move it one step. In e!ect, thecondition of the property automaton move must be enabled in thestate resulting from the earlier system step, in order for the prop-erty automaton to be taking a step. This corresponds to taking thesynchronous product as mentioned in Section 21.4.1.

• Generate the entire reachable graph by alternating the system- andproperty- steps using depth-first search (DFS). A hash table is em-ployed to record states already visited, thus cutting o! search atthese re-visitations. This depth-first search can be thought of asa mechanism for enumerating the final states of the synchronousproduct machine in postorder.

• Whenever the DFS backs o! a final state, a nested DFS is spawned.The task of this “inner” DFS is to look for Buchi automata “lassos”(accepting cycles). In [30, 20], it has been shown that if the productBuchi automata has at least one accepting cycle, then one suchcycle will be found by this procedure.

In a sense, this is an on-the-fly cycle detection approach that oftenworks much more e"ciently than an o#ine cycle detection proceduresuch as Tarjan’s strongly connected components (SCC) algorithm. Weillustrate this nested depth-first search on a simple example in Fig-ure 23.12. We supply all details below, including details glossed overabove. Please pay attention to “outer DFS hash table”—ODHT, and


“inner DFS hash table”—IDHT, below. Basically, we enter states intothe same hash table, but tag it with an additional bit saying whetherthe state was inserted during an outer or an inner DFS:

• Suppose the synchronous product generates the graph shown on theleft-hand side of Figure 23.12.

• The “outer” DFS will perform many traversals, as shown by thestack histories shown in the middle of Figure 23.12. These traversalsare as follows (we assume that when at state S1, the first edge tobe searched is the one indicated with a “1” on it, and the secondedge has a “2”; likewise for state S3 as well):/ Search S0, S1, take outgoing edge “1” and continue with S2,

and back to S1. The stack snapshot at this point is shown inthe left-hand stack of the middle group of stacks in Figure 23.12.This closes a cycle, and so DFS unwinds the stack up to stateS1. ODHT now has states S0, S1, S2 in it.

/ Search forward from S1 through outgoing edge marked “2”, go-ing up to state S3, then taking “1” to go to S4, and finally to S2.The stack snapshot at this point is shown in the middle stackof the middle group of stacks in Figure 23.12. Since S2 is in theODHT, the DFS is cut o!, and the search wants to unwind tostate S3.

/ However, we notice that we are backing o! from S4, which is afinal state. Hence, we spawn an inner DFS.

• The inner DFS proceeds in the manner shown in the right-handgroup of two stacks shown in Figure 23.12. In particular,/ The inner DFS proceeds forward from S4, producing the stack

history S4, S2, S1, and taking the “1” edge, back to S2. Allthese states are inserted into the IDHT, and that is why we didnot cut o! at S2 after S4. This did not find an accepting cycle“lasso.”

/ The inner DFS backs o! to state S1 and continues via the “2”edge to S3, and then to S4.

/ At this point, the inner DFS has reached a state that is alreadyin the stack, and so it “knows” that a cycle has been closed!

/ Also since the inner DFS was triggered from a final state, weknow that we have found a “lasso” that includes a final state.

/ Last but not least, since the outer DFS had performed a pos-torder enumeration of the final states, a reachable lasso has beenfound.

/ Therefore, we conclude that

L(S) ' L(P ) %= 0


and so the property in question has been violated./ An error-trace is generated and printed, following the DFS stack

history, in the form of a MSC.

Chapter Summary

We presented an enumerative model checking algorithm for CTL, and asymbolic model checking algorithm for CTL. The interactive sessions inthe BED tool present intermediate BDDs created during the fixed-pointiteration. Then we presented LTL and Buchi automata, and presentedan enumerative algorithm for LTL model checking.

Exercises

23.1. Write the pseudo-code of an algorithm (similar to Figure 23.3)for procedure EG that handles an EG formula. Then show the detailsof checking(EG (OR N1 (OR T1 C1))) by filling a table similar to Figure 23.2.

23.2. Consider the “release” (R) operator whose definition is as follows.A[p R q] means that q holds along the path up to and including thefirst state where p holds. However, p is not required to hold eventually.Express the semantics of ‘R’ more rigorously. Now, find a fixed-pointformulation for ‘R’, and illustrate it on the same example as illustratedin Section 23.2.3. If there are two natural interpretations for the LFPand GFP, consider both, and then explain what their semantics are.Present all the BED sessions you employed in arriving at your answers.

23.3. Write three LTL properties that try to capture “p can hold afteran even number of execution steps, but never holds after an odd num-ber of steps,” and point out where they go wrong. Now draw a Buchiautomata that specifies this pattern.

23.4. Assume that you are given a digital circuit to verify and youhave access to only a linear-time temporal logic model checker. Youare, however, required to state and prove a CTL correctness assertionwith respect to some of the input/output ports of this circuit. ThisCTL assertion is known to not have any equivalent LTL assertions.Can you proceed to check these assertions, somehow, using the givenLTL model checker? Assume that you have access to the internal nodesof the circuit, and so you are allowed to state LTL assertions not onlywith respect to the input/output ports, but also with respect to theinternal nodes of the circuit.


If it is theoretically possible to check the CTL assertions in termsof a (perhaps larger) set of LTL assertions, think of a simple practicalexample where such a situation can be faced (employing simple reset-table counters and logic gates), and describe, in terms of this example,what your solution will be.

23.5. Do a literature search and find out whether the property ex-pressed in Exercise 23.3 is expressible in (a) CTL, (b) CTL*, (c) µ-calculus.

23.6. Argue that if one of the Buchi automata being subject to theintersection operation has only final states (it has no non-final states),then the standard intersection algorithm for NFA can be used to inter-sect two Buchi automata. Provide an example illustrating your answer.

23.7. Change the property to be verified in Figure 23.10 from GFx toFGx, and rework all the steps of that example, including details of thenested DFS algorithm. Verify your answer by inspection (see if FGx issupposed to be true, and compare with what you got when you appliedthe nested DFS algorithm).

24

Conclusions

This book is about “automata” in the broad sense of the word intro-duced in Chapter 1. We viewed automata and logic as two sides ofthe same coin. Connections between automata and logic were empha-sized multiple times. We often took an informal approach to drive themain point across as intuitively and in as friendly a fashion as pos-sible. We believe that the material presented here will help someonewith virtually no formal computation engineering background to pickup a considerable amount of that topic and apply it towards debug-ging sequential and concurrent hardware and software systems throughautomated finite-state methods known as model checking.A journey of a 1000 miles begins with one step (ancient Chineseproverb); however, such a journey must somehow end! We close o!,rather unceremoniously, picking four important (but random) topics toreiterate.

Beginner’s mistakes

Experience has shown that a large majority of proofs written by begin-ners are flawed. For instance, it is easy to simply forget to prove oneof the directions of an i! proof. Other common mistakes include usingunproven facts, using totally circular definitions, etc. While there is nopoint fretting over one’s past mistakes, good ways to avoid fundamentalmistakes are to proofread one’s proof presentations, write it out neatly(preferably in a bulleted fashion), and presenting one’s proofs to others.Other ways to avoid these mistakes include a careful observation of how“common situations” are handled, and remember them by heart. Forinstance, while negating A 3 B, since A is already “under a negation”(because A 3 B : ¬A*B), it does not get negated in the final result,which is A + ¬B.

440 24 Conclusions

Mistakes Due to Inattention to Detail

After passing the initial hurdles, the second category of “mistakes”are simply due to inattention to detail, or to unfamiliarity with theparticular domain. Here are some examples of common mistakes (thedetails are unimportant now; only the pattern of the mistakes needs tobe noted):

1. Misusing the unidirectional Pumping Lemma: In Chapters 12 and13, we studied several Pumping Lemmas of the form A 3 B as wellas A = B. An example of a Pumping Lemma of the form A 3 B isRegular(L) 3 C where C is a big complicated formula. The recom-mended usage is to assume for some language L that Reg(L) holds,and to find that ¬C (i.e., a contradiction), thus proving ¬Reg(L).Many students abuse this argument to try and prove Reg(L).

2. Forgetting to do full case analysis: Even after grasping the PumpingLemma, many students forget to do full case analysis to derive con-tradictions in all cases. Similar mistakes are often repeated in doingproofs by reducibility, where also contradictions must be obtainedin all possible cases.

3. Forgotten cases of definitions: Here, my favorite is relating to NP-complete (NPC) proofs, where the full definition of NPC is not keptin mind. Many students forget to show that the given problem is inthe class NP, thus leaving open the question of whether the prob-lem is even decidable (i.e., it is not enough to establish polynomialreducibility).

4. Abuse of “hard” theorems: Theorems such as the Rice’s Theoremare often not well understood, and hence abused. They often treatit as if it were a “magic” theorem or “panacea,” which it isn’t.For instance, many students are known to apply Rice’s Theorem tocontext-free languages and even regular languages in a loose man-ner. Rice’s Theorem basically states the impossibility of buildingan algorithmic classifier for Turing machines based on the languagerecognized by these Turing machines, if the classification attemptedis anything but trivial. They do not remember that it is the Tur-ing machine codes (“programs”) that we need to classify, based onthe language of these Turing machines, should we run these Turingmachines; we are not classifying DFA or PDA.

Here, there is no quick answer except to gain experience, as well as beformal about definitions.

24 Conclusions 441

Experts’ mistakes

Humans are fallible - expert or not! Lamport [74] observes that abouta third of the proofs published in mathematical journals are flawed.In fact, everyone knows that, and the attitude at this level is not to“denigrate” or “get even” with the person, but to help fix the proofs.Therefore, the best advice one can pass on is

“How do I present my proofs so that it helps others easily spotmy mistakes and correct them?”

Proofs do have value even when flawed. Proofs are, after all, pro-grams, albeit in a mathematical language, and hence prone to ‘pro-gramming errors.’ While mathematical notation helps you avoid nastyerrors due to pointers overflowing their ranges, etc., a poorly writtenformal description is no better than a program written at an insidiouslylow level and hence fraught with many dangers.

Whether to trust machine checked proofs

Mechanical theorem proving has been developed to be able to machinecheck proofs. Other tools such as model checkers and finite-state rea-soning tools have also been developed to verify correctness. In a well-developed mechanical reasoning system, all the critical reasoning stepsare carried out inside a logic kernel that is very compact, and hencesubject to much scrutiny. In Chapter 11, as well as Chapters 21 through23, we obtained a glimpse of the inner workings of many mechanicalcorrectness checking tools. While these tools are no panacea, they haveoften helped find very subtle flaws that, for all practical purposes, can-not be detected through casual inspection or repeated simulation usingad hoc test cases. Continued development of these tools and techniquesis indeed a sign of maturation of computation engineering. For addi-tional details, please refer to [41].

The End. Fini. Thank you for your company in this rather long jour-ney.1 If you spot any mistakes,2 or have suggestions, kindly drop mea line, won’t you? My email address and the book website appear onpage 443. Many thanks in advance!

1 Unless you are peeking at this page without reading most others!2 The safest place for a fly to sit is on the fly-swatter handle! – Quoted from “The

Logic of Bugs” by Gerard Holzmann, Foundations of Software Engineering, 2002.

A

Book web site and tool information

A.1 Web site and e-mail address

The author maintains a web site for this book to post errata, updates,and copies of some of the tools and scripts. It is

www.cs.utah.edu/ganesh-comp-engg-book.The author welcomes your comments and suggestions. Please send

them to [email protected].

A.2 Software tool usage per chapter

8 through 11:dot and graphviz tools mentioned in these chapters are down-loadable from www.graphviz.org.grail, originally developed at the University of Western Ontarioby Darrel Raymond and Derick Wood(http://www.csd.uwo.ca/Research/grail/index.html).A copycourtesy of Andrei Paun and Shenghua Ni will be placed on thewebsite of this book also.Two scripts, fa2grail.perl and grail2ps.perl, kept at thewebsite of this book.

11: BED, a BDD manipulation package by Henrik Reif Anderssonand Henrik Hulgaard, is available athttp://www.itu.dk/research/bed/. It will be kept on thewebsite of this book also.

15: JFLAP, developed by Susan Rodger’s team, is downloadablefrom http://www.jflap.org/.

18 and 19:

444 A Book web site and tool information

Zcha!, developed by Sharad Malik’s group, is downloadable fromhttp://www.princeton.edu/~chaff/. There are many other SATsolvers that serve the same purpose, such as MiniSAT(http://www.cs.chalmers.se/Cs/Research/FormalMethods/MiniSat/).Ocaml is used in a few examples, and may be downloaded fromcaml.inria.fr.

21: SPIN is used in this chapter, and may be downloaded fromwww.spinroot.com

22: This chapter may also involve the use of BED.

A.3 Possible Syllabi

This book includes more material than can be comfortably taught in atypical academic semester. Subsets of this book may be used to meetthe needs of the following types of courses:

An undergraduate course on basic discrete mathematics, finite au-tomata, and logic may be taught along the following lines:– Week 1: Mathematical Preliminaries (Chapter 2).– Week 2: Main topics from Binary Relations (Chapter 4), includ-

ing Pre- and Partial Orders, Reflexive and Transitive Closure.– Week 3: Proof Methods (Chapter 5), skipping Induction Princi-

ples.– Week 4-5: Strings and Languages (Chapter 7), perhaps skipping

Homomorphisms.– Week 6: Machines, Languages, and DFA (Chapter 8).– Week 7: NFA and Regular Expressions (Chapter 9).– Week 8: Operations on Regular Machinery (Chapter 10), includ-

ing illustrations using Grail.– Week 9: The Pumping Lemma (Chapter 12).– Week 10: Context-free Languages (Chapter 13).– Week 11: Push-down Automata and Context-free Grammars

(Chapter 14).– Week 12: Turing Machines (Chapter 15).– Week 13: Selected topics from Basic Notions in Logic (Chap-

ter 18).– Week 14: Complexity Theory and NP-Completeness (Chapter 19).An undergraduate course on automata, languages, Turing machines,mathematical logic, and applied Boolean methods may be taughtalong the following lines:

A.3 Possible Syllabi 445

– Week 1: Assign exercises from Mathematical Preliminaries (Chap-ter 2). Then convey the main ideas from Cardinalities and Di-agonalization (Chapter 3), including a proof of the Schroder-Bernstein Theorem.

– Week 2: Binary Relations (Chapter 4). Also dive straight intoSection 5.3.1 and work out a proof of the equivalence betweenthe two induction principles (Arithmetic and Complete).

– Week 3: Strings and Languages (Chapter 7).– Week 4: Machines, Languages, and DFA (Chapter 8).– Week 5: NFA and Regular Expressions (Chapter 9).– Week 6: Operations on Regular Machinery (Chapter 10), includ-

ing illustrations using Grail.– Week 7: The Automaton/Logic Connection, Symbolic Tech-

niques (Chapter 11), including illustration using BED.– Week 8: The Pumping Lemma (Chapter 12).– Week 9: Context-free Languages (Chapter 13), including Consis-

tency and Completeness proofs.– Week 10: Push-down Automata and Context-free Grammars

(Chapter 14).– Week 11: Turing Machines (Chapter 15). Basic Undecidability

Proofs (Chapter 16).– Week 12: Basic Notions in Logic (Chapter 18).– Week 13: Complexity Theory and NP-Completeness (Chapter 19).– Week 14: Model Checking: Basics (Chapter 21).A graduate course on automata, relationships with Boolean meth-ods, undecidability, NP-completeness, and modern formal verifica-tion methods may be taught along the following lines, taking advan-tage of graduate students’ relative independence, as well as keepingtheir impatience with rote work in mind:

– Week 1-3: Quickly go through Chapters 2 through 6.– Week 4-6: Review concepts from NFA, DFA, and RE (Chapters 7

through 10 and Chapter 12), suggesting that they use Grail toquickly note various results.

– Week 7: Chapters 11, Automaton/Logic Connection. Reachabil-ity using BDDs. It is possible to cover Chapter 20 (Presburgerarithmetic) at this juncture.

– Week 8: Chapter 13, 14.– Week 9: Chapter 15.– Week 10: Chapter 16.– Week 11: Chapter 17.– Week 12: Chapter 18.

446 A Book web site and tool information

– Week 13: Chapter 19.– Week 14: Chapter 21 through 23.

B

BED Solution to the tic-tac-toe problem

The reader may enter the following script into the BED tool and watchthe display of all draws.

var turn turnpa00 a00p b00 b00pa01 a01p b01 b01pa02 a02p b02 b02pa10 a10p b10 b10pa11 a11p b11 b11pa12 a12p b12 b12pa20 a20p b20 b20pa21 a21p b21 b21pa22 a22p b22 b22p ;

let init=!a00 and !b00 and !a01 and !b01 and!a02 and !b02 and !a10 and !b10 and!a11 and !b11 and !a12 and !b12 and

!a20 and !b20 and !a21 and !b21 and!a22 and !b22 and !turn ;

let initp=!a00p and !b00p and !a01p and !b01p and

!a02p and !b02p and !a10p and !b10p and!a11p and !b11p and !a12p and !b12p and

!a20p and !b20p and !a21p and !b21p and!a22p and !b22p and !turnp ;

let samerow0 = (a00=a00p) and (b00=b00p) and (a01=a01p) and (b01=b01p)and (a02=a02p) and (b02=b02p);let samerow1 = (a10=a10p) and (b10=b10p) and (a11=a11p) and (b11=b11p)and (a12=a12p) and (b12=b12p);let samerow2 = (a20=a20p) and (b20=b20p) and (a21=a21p) and (b21=b21p)

448 B BED Solution to the tic-tac-toe problem

and (a22=a22p) and (b22=b22p);let samecol0 = (a00=a00p) and (b00=b00p) and (a10=a10p) and (b10=b10p)and (a20=a20p) and (b20=b20p);let samecol1 = (a01=a01p) and (b01=b01p) and (a11=a11p) and (b11=b11p)and (a21=a21p) and (b21=b21p);let samecol2 = (a02=a02p) and (b02=b02p) and (a12=a12p) and (b12=b12p)and (a22=a22p) and (b22=b22p);

let M00 = !a00 and !b00 and a00p and !b00p and !turn and turnpand samerow1 and samerow2 and samecol1 and samecol2;









let N00 = !a00 and !b00 and !a00p and b00p and turn and !turnpand samerow1 and samerow2 and samecol1 and samecol2;





B BED Solution to the tic-tac-toe problem 449





let T = M00 or M01 or M02 orM10 or M11 or M12 orM20 or M21 or M22 or

N00 or N01 or N02 orN10 or N11 or N12 orN20 or N21 or N22 ;

let atmostone =!(a00 and b00) and !(a01 and b01) and !(a02 and b02)and!(a10 and b10) and !(a11 and b11) and !(a12 and b12)and!(a20 and b20) and !(a21 and b21) and !(a22 and b22) ;

let wina1 = atmostone and(a00 and !b00) and (a01 and !b01) and (a02 and !b02) ;







450 B BED Solution to the tic-tac-toe problem


let winb1 = atmostone and(!a00 and b00) and (!a01 and b01) and (!a02 and b02) ;








let allmoved = (a00 = !b00) and (a01 = !b01) and (a02 = !b02)and

(a10 = !b10) and (a11 = !b11) and (a12 = !b12)and

(a20 = !b20) and (a21 = !b21) and (a22 = !b22) ;

let draw = allmoved and!wina1 and !wina2 and !wina3 and !wina4 and!wina5 and !wina6 and !wina7 and !wina8 and!winb1 and !winb2 and !winb3 and !winb4 and!winb5 and !winb6 and !winb7 and !winb8 ;

upall init; upall samerow0; upall samerow1; upall samerow2;upall samecol0; upall samecol1; upall samecol2; upall M00;

upall M01; upall M02; upall M10; upall M11; upall M12;upall M20; upall M21; upall M22;

upall N00; upall N01; upall N02; upall N10; upall N11;upall N12; upall N20; upall N21; upall N22; upall T;

upall atmostone;

B BED Solution to the tic-tac-toe problem 451

upall wina1; upall wina2; upall wina3; upall wina4; upall wina5;upall wina6; upall wina7; upall wina8;

upall winb1; upall winb2; upall winb3; upall winb4; upall winb5;upall winb6; upall winb7; upall winb8;

upall allmoved; upall draw; view draw;

Here, be prepared to obtain a six-page BDD describing all possibledraws in one fell swoop.

References

1. Agrawal, M., Kayal, N., and Saxena, N. “PRIMES is in P”, August2002. http://www.cse.iitk.ac.in/primality.pdf.

2. Aagaard, M. D., Jones, R. B., and Seger, C. -J. “Combining TheoremProving and Trajectory Evaluation in an Industrial Environment”. InAlan Hu, editor, Design Automation Conference (DAC), 1998.

3. ACL2 Version 2.9. http://www.cs.utexas.edu/users/moore/acl2/.4. Adiga, N. R., et al. “An Overview of the BlueGene/L Supercom-

puter”. In Conference on High Performance Networking and Computing:SC2002, page 60, 2002.

5. Aho, A., Hopcroft, J.E., and Ullman, J.D. The Design and Analysis ofComputer Algorithms. Addison-Wesley, 1974.

6. Aho, Alfred V., Hopcroft, John E., and Ullman, Je!rey D. Data Struc-tures and Algorithms. Addison-Wesley, 1983.

7. Andersen, Henrik Reif. “An Introduction to Binary Decision Diagrams”(tutorial associated with the bed tool), April 1998.

8. Ball, T., Cook, B., Levin, V., and Rajamani, S. K. “SLAM andStatic Driver Verifier: Technology Transfer of Formal Methods insideMicrosoft”. In IFM 04: Integrated Formal Methods, pages 1–20. Springer-Verlag, April 2004.

9. Ball, T. and Rajamani, S.K. “The SLAM Toolkit”. In Proc. Computer-Aided Verif., pages 260–264, 2001. LNCS 2102.

10. Barringer, H., Goldberg, A., Havelund, K., and Sen, K. “Program Mon-itoring with LTL in EAGLE”. In International Conference on Paralleland Distributed Processing Systems (IPDPS), 2004.

11. Biere, A., Cimatti, A., Clarke, E.M., and Zhu, Y. “Symbolic ModelChecking without BDDs”. In Tools and Algorithms for the Analysis andConstruction of Systems (TACAS’99), 1999. LNCS 1579.

12. Brown, S. and Vranesic, Z. Fundamentals of Digital Logic. McGraw-Hill,2003.

454 References

13. Bryant, Randal E. “Symbolic Boolean Manipulation with Ordered Bi-nary Decision Diagrams”. ACM Computing Surveys, 24(3):293–318,1992.

14. Bryant, R.E. “Graph-Based Algorithms for Boolean Function Manip-ulation”. IEEE Transactions on Computers, C-35(8):677–691, August1986.

15. Cantin, J. F., Lipasti, M. H., and Smith, J. E. “The Complexity ofVerifying Memory Coherence”. In Proceedings of the fifteenth annualACM symposium on Parallel algorithms and architectures (SPAA), pages254 – 255, San Diego, 2003.

16. Cantin, Jason. The Complexity of Verifying Memory Coherence.www.jfred.org/patpub.html.

17. Church, Alonzo. Mathematical Logic. Princeton University Press, 1956.18. Clarke, E. M. and Emerson, E. A. “Synthesis of Synchronization Skele-

tons for Branching Time Temporal Logic”. In Logic of Programs, LNCS131, pages 52–71. Springer-Verlag, 1981.

19. Clarke, E. M., Emerson, E. A., and Sistla, A.P. “Automatic Verifica-tion of Finite-state Concurrent Systems using Temporal Logic Specifi-cations. ACM Transactions on Programming Languages and Systems(TOPLAS), 8(2):244–263, 1986.

20. Clarke, E. M., Grumberg, O., and Peled, D. Model Checking. MIT Press,December 1999.

21. Clarke, E.M., Grumberg, O., Jha, S., Lu, Y., and Veith, H.“Counterexample-guided Abstraction Refinement for Symbolic ModelChecking”. Journal of the ACM (JACM), 50:752 – 794, September 2003.

22. Clarke, E.M., Kimura, S., Long, D.E., Michaylov, S., Schwab, S. A.,and Vidal, J.-P. “Parallel Symbolic Computation on Shared MemoryMultiprocessor”. In International Symposium on Shared Memory Multi-processors, Tokyo, Japan, April 1991. (Also CMU TR CMU-CS-90-182).

23. Clay Mathematical Institute. http://www.claymath.org/millennium/P vs NP/O"cial Problem Description.pdf.

24. Collatz Problem. http://mathworld.wolfram.com/CollatzProblem.html.25. Comon, H., Marche, C., and Treinen, R. Constraints in Computational

Logics, Theory and Applications. Springer, 2001.26. Continuum Hypothesis.

http://mathworld.wolfram.com/ContinuumHypothesis.html.27. Cook, Stephen A. “Computer Science Lectures, NP-Completeness”.

www.cs.toronto.edu/~sacook/.28. Corella, F., Shaw, R., and Zhang, C. “A formal proof of absence of

deadlock for any acyclic network of PCI buses”. In Hardware DescriptionLanguages and their Applications, pages 134–156. Chapman Hall, 1997.

29. Cormen, Thomas H., Leiserson, Charles E., and Rivest, Ronald L. In-troduction to Algorithms. McGraw-Hill Company, 1990.

References 455

30. Courcoubetis, C., Vardi, M.Y., Wolper P., and Yannakakis, M. “MemoryE"cient Algorithms for the Verification of Temporal Properties”. InFormal Methods in System Design, volume 1, pages 275–288, 1992.

31. Curry, H.B. and Feys, R. Combinatory Logic (Vol. 1). North Holland,Amsterdam, 1958.

32. Davis, M., Logemann, G., and Loveland, D. “A Machine Program forTheorem Proving”. Communications of the ACM, 5(7):394–397, 1962.

33. Davis, M. and Putnam, H. “A Computing Procedure for QuantificationTheory”. Journal of the ACM, 7(1):201–215, 1960.

34. Demaine, E.D., Hohenberger, S., and Liben-Nowell, D. “Tetris is Hard,Even to Approximate”. In Computing and Combinatorics (COCOON),pages 351–363, 2003.

35. DeMillo, R. A., Lipton, R.J., and Perlis, A.J. “Social Processes andProofs of Theorems and Programs”. Communications of the ACM,22(5):271–280, 1979.

36. Dijkstra, Edsger W. “The Humble Programmer”. Communications ofthe ACM, 15(10):859–866, 1972.

37. Dijkstra, Edsger W. A Discipline of Programming. Prentice-Hall, 1976.38. Dill D. Comment made in keynote address o!ered at the Principles of

Programming Languages (POPL) Conference, 1999.39. Du, D.-Z. and Ko, K.I. Problem Solving in Automata, Languages, and

Complexity. John Wiley & Sons, 2001.40. Floyd, R. W. “Assigning Meanings to Programs”. In J. T. Schwartz,

editor, Mathematical Aspects of Computer Science, pages 19–32, Provi-dence, RI, 1967. American Mathematical Society.

41. Formal Methods. Jonathan Bowen’s Formal Methods Resource Page athttp://www.afm.sbu.ac.uk.

42. Friedman, S. J. and Supowit, K. J. “Finding the Optimal VariableOrdering for Binary Decision Diagrams”. In Proceedings of the 24thACM/IEEE conference on Design automation, pages 348–356, 1987.

43. Fritz, Carsten. “Constructing Buchi Automata from Linear TemporalLogic Using Simulation Relations for Alternating Buchi Automata”. InCIAA. Springer-Verlag, 2003. An excellent LTL to Buchi Automatontool is available from http://www.ti.informatik.uni-kiel.de/fritz/ABA-Simulation/ltl.cgi.

44. Garey, M. R. and Johnson, D.S. Computers and Intractability, A Guideto the Theory of NP-Completeness, chapter Appendix A5, pages 236–237. W.H. Freeman and Company, 1978.

45. Ginsburg, Seymour. The Mathematical Theory of Context Free Lan-guages. McGraw-Hill, 1966. QA 267.5 S4 G5.

46. Gopalakrishnan, G.L., Yang, Y., and Sivaraj, H. “QB or Not QB: An Ef-ficient Execution Verification Tool for Memory Orderings”. In ComputerAided Verification, pages 401–413, 2004.

456 References

47. Gordon, Michael. “Why Higher-order Logic is a Good Formalism forSpecifying and Verifying Hardware”. In Formal aspects of VLSI design,1986.

48. Gordon, Michael. Programming Language Theory and Implementation.Prentice-Hall, 1993.

49. Gurumurthy, S., Kupferman, O., Somenzi, F., and Vardi, M.Y. “OnComplementing Nondeterministic Buchi Automata”. In 12th AdvancedResearch Working Conference on Correct Hardware Design and Veri-fication Methods, Lecture Notes in Computer Science. Springer-Verlag,2003.

50. Halmos, Paul R. Naıve Set Theory. Van Nostrand, 1968.51. Har’El, Z. and Kurshan, R.P. “Software for Analysis of Coordination”.

In Proc. Int’l Conference on System Science, 1988.52. Harrison, John. “Formal Verification of Square Root Algorithms”. For-

mal Methods in System Design, 22(2):143–153, 2003.53. Henzinger, T. A., Jhala, R., Majumdar, R., and Sutre, G. “Lazy

Abstraction”. In Principles of Programming Languages, pages 58–70.ACM, January 2002.

54. Hilbert’s Problems. http://mathworld.wolfram.com/HilbertsProblems.html.55. Hoare, C. A. R. “An Axiomatic Basis of Computer Programming”.

Communications of the ACM, 12(10):576–580, October 1969.56. Hoare, C. A. R. Communicating Sequential Processes. CACM,

21(8):666–677, 1978.57. Hoare, C.A.R. “The Emperor’s Old Clothes”. Communications of the

ACM, 24(2):75–83, 1981.58. Holzmann, Gerard, J. Design and Validation of Computer Protocols.

Prentice Hall, 1991.59. Holzmann, Gerard J. The SPIN Model Checker: Primer and Reference

Manual. Addison Wesley, 2004.60. Hopcroft, J.E., Motwani, R., and Ullman, J.D. Introduction to Automata

Theory, Languages, and Computation. Addison-Wesley, 2001.61. Hopcroft, J.E. and Ullman, J.D. Introduction to Automata Theory, Lan-

guages, and Computation. Addison-Wesley, 1979.62. Joe Hurd. “Computer Science Lectures, Schroder Bernstein Theorem”.

http://www.cl.cam.ac.uk/~jeh1004/compsci/lectures/.63. Introduction to Logic. http://logic.philosophy.ox.ac.uk/.64. Jackson, D. In tutorial o!ered at the Third IEEE International Sympo-

sium on Requirements Engineering, Annapolis, Maryland, USA, January6-10, 1997.

65. Ja!e, J. “A Necessary and Su"cient Pumping Lemma for Regular Lan-guages”. In ACM SIGACT News archive, volume 10, pages 48–49. ACMPress, 1978. Issue 2, Summer 1978, ISSN:0163-5700.

66. JFLAP. http://www.jflap.org/.67. Jones, Neil D. Computability and Complexity from a Programming Per-

spective. MIT Press, 1997.

References 457

68. Kelley, D. Automata and Formal Languages: an introduction. Prentice-Hall, 1995.

69. Kepler Conjecture. http://mathworld.wolfram.com/KeplerConjecture.html.70. Klarlund, N., Møller, A., and Schwartzbach, M. I. “MONA Implemen-

tation Secrets”. International Journal of Foundations of Computer Sci-ence, 13(4):571–586, 2002.

71. Kozen, Dexter C. Automata and Computability. Springer, 1997.72. Kurshan, R.P. Computer-aided Verification of Coordinating Processes.

Princeton University Press, 1994.73. Lamport, Leslie. “Proving the Correctness of Multiprocess Programs”.

IEEE Transactions on Software Engineering, SE-3(2):125–143, 1977.74. Lamport, Leslie. “How to Write a Proof”. American Mathematical

Monthly, 102(7):600–608, August-September 1993.75. Lamport, Leslie. “How to Write a Long Formula”. Formal Aspects of

Computing, 3:1–000, 1994.76. Nancy G. Leveson. SAFEWARE: System Safety and Computers.

Addison-Wesley, 1995.77. Loeckx, J.and Sieber, K. The Foundations of Program Verification: Sec-

ond Edition. John Wiley & Sons, 1987.78. Manna, Z. Mathematical Theory of Computation. McGraw-Hill, 1974.79. Manna, Z. and Pnueli, A. “The Modal Logic of Programs”. In ICALP,

pages 385–409, 1979.80. Manna, Z. and Pnueli, A. The Temporal Logic of Reactive and Concur-

rent Systems. Springer Verlag, 1992.81. Marques-Silva, Joao P. and Sakallah, Karem A. “GRASP - a New Search

Algorithm for Satisfiability”. In International Conference on Computer-Aided Design, pages 220–227, 1996.

82. Maurer, H. “A Direct Proof of the Inherent Ambiguity of a SimpleContext-Free Language”. JACM, 16(2):256 – 260, April 1969.

83. McMillan, Kenneth L. Symbolic Model Checking. Kluwer AcademicPress, 1993.

84. Meyer, Bertrand. “Design by Contract: The Lessons of Ariane”. IEEEComputer, 30(2):129–130, January 1997.

85. Milner, R. A Calculus of Communicating Systems, volume 92 of LectureNotes in Computer Science. Springer-Verlag, 1980.

86. Milner, R. and Tofte, M. “Co-induction in Relational Semantics”. The-oretical Computer Science, 87(1):209–220, 1990.

87. MiniSAT. http://www.cs.chalmers.se/Cs/Research/FormalMethods/MiniSat/.

88. Moret, B.M.E. and Shapiro, H.D. Algorithms from P to NP. BenjaminCummings, 1991.

89. Morris, F.L. and Jones, C.B. “An Early Program Proof by Alan Tur-ing”. IEEE Annals of the History of Computing, 6(2):139–143, April-June 1984.

458 References

90. Moskewicz, M.W., Madigan, C.F., Zhao, Y., Zhang, L., and Malik, S.“Cha!: Engineering an E"cient SAT Solver”. In Proceedings of the 38thDesign Automation Conference (DAC’01), 2001.

91. The MPEC Tool Release for Verifying Multiprocessor Executions.http://www.cs.utah.edu/formal verification/.

92. Nelson, G. and Oppen, D.C. “Simplification by Cooperating DecisionProcedures”. ACM Transactions on Programming Languages and Sys-tems, 1(2):245–257, 1979.

93. Owicki, S. S. and Gries, D. “Verifying Properties of Parallel Programs:An Axiomatic Approach”. Commun. ACM, 19(5):279–285, 1976.

94. Owre, S., Shankar, N., and Rushby, J. “PVS: A Prototype VerificationSystem”. In 11th International Conference on Automated Deduction(CADE), Saratoga Springs, NY, pages 748–752, June 1992.

95. PCI Special Interest Group–PCI Local Bus Specification, Revision 2.1,June 1995.

96. C. A. Petri. Fundamentals of a Theory of Asynchronous InformationFlow. In Proc. of IFIP Congress 62., pages 386–390. North HollandPubl. Comp., 1963.

97. Pnueli, Amir. “The Temporal Logic of Programs”. In Proceedings of the18th IEEE Symposium Foundations of Computer Science (FOCS 1977),pages 46–57, 1977.

98. Pratt, Vaughan. “Every Prime Has a Succinct Certificate”. SIAM Jour-nal on Computing, (4):214–220, 1975.

99. Queille, J. and Sifakis, J. “Specification and Verification of ConcurrentSystems in CESAR”. In M. Dezani-Ciancaglini and U. Montanari, edi-tors, Fifth International Symposium on Programming, Lecture Notes inComputer Science 137, pages 337–351. Springer-Verlag, 1981.

100. Rabin, Michael O. and Scott, Dana S. “Finite Automata and their De-cision Problems”. IBM J. Res. Dev., (3):114–125, 1959.

101. Ramalingam, G. “The Undecidability of Aliasing”. ACM Transactionson Programming Languages and Systems, 16(5):1476–1471, September1994.

102. RSA Algorithm: 2002 A. M. Turing Award Winners WebPage. http://www.acm.org/awards/turing citations/rivest-shamir-adleman.html.

103. RTI. “The Economic Impacts of Inadequate Infrastructure for SoftwareTesting,” Final Report, National Institute of Standards and Technology,May 2002. Research Triangle Institute (RTI) Project Number 7007.001.Accessed at http://spinroot.com/spin/Doc/course/NISTreport02-3.pdf(February 2006).

104. Rushby, John, M. “Disappearing Formal Methods”. In High-AssuranceSystems Engineering Symposium, pages 95–96, Albuquerque, NM, nov2000. Association for Computing Machinery.

105. “SAT Live”. www.satlive.org.

References 459

106. Savage, John E. Models of Computation: Exploring the Power of Com-puting. Addison-Wesley, 1998.

107. Schmidt, David. Denotational Semantics: A Methodology for Lan-guage Development. 1979. Out of print; copies of book available athttp://www.cis.ksu.edu/~schmidt/text/densem.html.

108. Shepherdson, J. C. “The Reduction of Two-Way Automata To One-WayAutomata”. IBM Journal of Research and Development, 3(2):198–200.http://www.research.ibm.com/journal/rd/032/ibmrd0302P.pdf.

109. Shlyakhter, I., Seater, R., Jackson, D., Sridharan, M., and Taghdiri,M. “Debugging Overconstrained Declarative Models Using Unsatisfi-able Cores”. In Proceedings of the 18th International Conference onAutomated Software Engineering (ASE’03), 2003.

110. Shostak, Robert. “A Practical Decision Procedure for Arithmetic withFunction Symbols”. Journal of the ACM, 26(2):351–360, April 1979.

111. Sipser, Michael. Introduction to the Theory of Computation. PWS Pub-lishing Company, 1997.

112. Spec Patterns. Discussion of LTL Patterns athttp://patterns.projects.cis.ksu.edu.

113. Stanat, D.F. and Weiss, S.F. “A Pumping Theorem for Regular Lan-guages”. In ACM SIGACT News archive, volume 14, pages 36–37. ACMPress, 1982. Issue 1, Winter 1982, ISSN:0163-5700.

114. Stoy, Joseph E. Denotational Semantics: The Scott-Strachey Approachto Programming Language Theory. MIT Press, 1981.

115. Tauriainen, Heikki. “Automated Testing of Buchi Automata Transla-tors for Linear Temporal Logic”. Master’s thesis, Helsinki University ofTechnology, 2000.

116. The Changing World of Software.http://www.sei.cmu.edu/publications/articles/watts-humphrey/changing-world-sw.html.

117. Turing, Alan M. “On Computable Numbers: With an Application tothe Entscheidungsproblem”. In Proceedings of the London MathematicalSociety, volume Series 2, November 1936. Number 42.

118. The Alan Turing Home Page. www.turing.org.uk/turing.119. The Turing Machine and Computability.

http://plato.stanford.edu/entries/turing/#2.120. Vardi, M. Y. and Wolper, P. “An Automata-theoretic Approach to Au-

tomatic Program Verification”. In Proc. 1st Symp. on Logic in ComputerScience, pages 332–344, Cambridge, June 1986.

121. Whaley, J. and Lam, M.S. “Cloning-Based Context-Sensitive PointerAlias Analysis Using Binary Decision Diagrams”. In Proceedings ofthe Conference on Programming Language Design and Implementation,pages 131–144. ACM Press, June 2004.

122. Windows Driver Foundation.http://www.microsoft.com/whdc/driver/wdf/ default.mspx.

460 References

123. Wing, Jeanette M. “Computational Thinking”. Communications of theACM, 49(3):33–35, 2006.

124. Wise, David S. “A Strong Pumping Lemma for Context-Free Lan-guages”. Theoretical Comp. Sci., (3):359–369, 1976.

125. Zhao, L. “Solving and Creating Di"cult Instances of Post’s Correspon-dence Problem”. Master’s thesis, University of Alberta, 2002. PCPSolver is available from http://web.cs.ualberta.ca/zhao/PCP/intro.htm.

(All web links were accessed on February 26, 2006 and found working.)

Index

<lex, 109=lex, 109>lex, 109ATM , 298CP , the set of all legal C programs,

44ETM , 304Eclosure function, 157HaltTM , 299Ibs, infinite bit-sequences, 44R", given R, 60R+, given R, 60R0, given R, 60RegularTM , 305S/ %, 60TID DFA, 124&, 76', 76(, 75#, 105#! = # ) {"}, 141$, for a DFA, 121$, for an NFA, 141", 107$, 124, 147*lex, 109|, Unix pipe, 153+=-CNF, 340Clique, 349,, 98-, 64

., 146.=, 76#=, 76", 106"-closure, 145/, 124/", 124as notation, 77, 113q ., 146q0, 121’, quoting, 153"., 146".", 146‘0,’ Kleene star, 112‘0,’ Star, 112‘-’, Read standard input, 153graphviz, 1292-CNF, 3412-CNF satisfiability, 3412-partition, 3643-CNF, 3403-SAT, 354

NPC, 3543-partition, 3643x+1 problem, 28

Acceptance of strings, 124Acceptance problem, 299Accepting state, 119Agrawal, Kayal, Saxena, 363

462 Index

Aleph, 37Algorithm, 3, 27Algorithmically computable, 27Algorithmically decidable, 126Alphabet, 105Ambiguity, 253

if-then-else , 226of CFGs , 225undecidability, 227

Ambiguity , 225Ambiguity and nondeterminism, 253Ambiguous grammar, 225Approximant, 194

for fixed-points, 194Ariane, 8AU, 425

GFP, 427LFP, 426recursion, 425

Automata theory, 6Automaton/logic connection, 186Avoid wasted work, 11Axiomatization, 324

Buchi Automata, 389Buchi automata, 428

emptiness, 433expressiveness, 428

Bag, 16Barter, 38Basketball, 56BDD, 186, 199

as minimized DFA, 187backward reachability, 192BED tool, 190Boolean functions, 191forward reachability, 192reachability analysis, 192redundant decoding, 189variable ordering, 190

BDD vs ROBDD, 187BED

reachability example, 197BED tool, 190Binary decision diagrams, 187

Black hole state, 123Boolean functions, 191Boolean satisfiability, 331Bottom, 98

function, 100value, 98

Bryant, 186

Cantor, 39Cardinal numbers, 49Cardinality, 3710, 3711, 3710.5??, 43trap, 39

Cartesian product, 21examples of, 21

Certificate, 348Diophantine, 361

CFG, 217fixed-points , 224left-linear , 237right-linear , 237simplification, 232

CFG to NFA, 238CFG to PDA, 254CFL

deterministic, 235pumping lemma, 239

CFLs and decidability, 264Characteristic sequence, 22

infinite, 108Chomsky hierarchy , 228Chomsky normal form, 262Church-turing thesis, 272CKY algorithm, 262Clay institute, 345Clique, 349Closure, 60, 124

prefix, 115reflexive, 60reflexive and transitive, 60, 124transitive, 60

Closure of CFLs, 264Closure properties of CFLs , 230

Index 463

CNF, 332CNF conversion, 336Cocke-Kasami-Younger Algorithm,

262Code, 292

angle bracket, 292Coiled, 179Collatz’s problem, 28Complete, 323Completeness, 220

of a CFG, 220undecidable , 220

Complexity theory, 5Composite, 350, 363

selling, 350Computation, 105Computation engineering, 1Computation history, 312Computation tree, 401Computational procedure, 3Computational process, 27Computational tree logic, 403Computer engineering, 1Congruence

radio receiver, 65Congruence relation, 64Conjunctive normal form, 332coNP, 362coNPC, 362Consistency

of a CFG, 220Consistency , 220Constructor, 80Context-free, 229Context-free grammar, 217

fixed-points, 224Continuum Hypothesis, 43Contradiction

proof by, 207Contrapositive, 207Contrapositive rule, 77Conversion

NFA to DFA, 159Cryptography, 74

RSA, 74

CTL, 403enumerative model checking, 419expressiveness, 428fixed-point iteration, 421fixed-point semantics, 410illustration using BED, 424LFP and GFP, 425semantics, 408symbolic model checking, 421syntax, 407

Curried form, 30

DBA, 430DCFL, 235De-fred, 24, 93, 94, 96Deadlock, 392Decidable, 126Decider, 351Decision problem, 273Definitional equality, 76Demorgan’s law, 80

generalized, 80Derivation, 219

leftmost , 219parse tree shape, 219rightmost , 219

Derivation sequence , 218Derivation step , 218Determinism vs ambiguity , 236Deterministic BA, 430Deterministic CFLs, 235Deterministic finite automaton, 4DFA, 4

complementation, 165Hamming distance, 176intersection, 165intersection with PDAs, 293Kleene star, 166minimization, 174Myhill-Nerode Theorem, 174relationship with BDDs, 174string classifier, 124ultimate periodicity, 179union, 162

DFA Transition function, 121

464 Index

Diagonalization, 39proof, 41

DIMACS format, 337Diophantine, 360Disambiguation , 226Disjunctive normal form, 332DNF, 332DNF to CNF, 333Dog, 37

Fifi, 37Howard, 37

Dominance relation, 62DPDA and NPDA , 235Duality

between 2 and 3, 80

Eclosure, 144E!ective computability, 272E!ective procedure, 27E!ectively computable, 27EG, 421

recursion, 421Elementary productions, 233Engineering mathematics, 2Enumerative model checking, 419,

432Equivalence, 58

class, 60Equivalence class, 60Error correcting DFA, 176Exists

as infinite disjunction, 79Exponential DFA, 139Expressiveness, 428

Buchi automata, 428CTL, 428LTL, 428

FBI, 8Final state, 121Finite regular language, 187First-order logic, 79, 323First-order validity, 326Fixed-point, 95, 192

and automata, 101

equation, 95immovable point, 95least, 100multiple, 100of a Photocopying machine, 95on a calculator, 95

Fixed-point iteration, 194Fixed-points

greatest, 101FOL

enumerable, 331semi decidable, 331

Forallas infinite conjunction, 16, 78

Formal methods, 8disappearing, 13

Formal model, 2Formal verification, 136, 155Forward reachability, 193Fred, 23, 93Frontier , 218Full contradiction, 209Function, 22

1-1, 25bijection, 26computable, 27correspondences, 26domain, 22gappiest, 100injection, 25into, 26many to one, 25non computable, 27non-termination, 25one to one, 25onto, 26partial, 25range, 22signature, 22surjection, 26total, 25

Function constant, 324

Godel, 185Game

Index 465

mastermind, 78General CNF, 340Generalized NFA, 170GNFA method, 170Grail, 151|, 153dot, 153fa2grail.perl, 153fmcment, 153fmcross, 153fmdeterm, 153fmmin, 153ghostview, 153gv, 153retofm, 153

Grail tool, 151Greibach’s Theorem, 312

Halting Problemdiagonalization, 50

Halting problem, 301Hamming distance, 176Hampath, 358Higher-order logic, 323Hilbert, 185

23 problems, 185first problem, 42problems, 42

Hilbert style, 324Hilbert’s problems, 42History, 12

computation theory, 12Homomorphism, 113

inverse, 114

ID, 124Identity relation, 59I!, 76Image, 23, 192, 193Implication, 17

antecedent, 75consequent, 75

Independent, 323Individual, 324Individuals, 79

Induction, 81arithmetic, 82complete, 82noetherian, 84structural, 81, 85

Inductive, 80basis elements, 80closed under, 81constructor, 80definition, 80free, 81least set, 81set, 80

Inductive assertions, 247Infinite loop, 25Infinitely often, 394Inherently ambiguous, 227Input encoding, 353

strong NPC, 364unary, 353

Instantaneous description, 124Interior node , 218Interleaving, 383Interpretation, 323Intractable, 345Invariant, 192

checking, 192Inverse homomorphism, 114, 168Irredundant name, 24, 94Isomorphism, 174Ivory soap, 40

Kripke structure, 399

Lambda calculus, 3, 23alpha conversion, 25alpha rule, 24associativity, 24beta reduction, 24beta rule, 24bindings, 24function application, 24irredundant names, 24syntactic conventions, 30Y, 94

Language, 5, 105, 107, 125

466 Index

cardinality, 108concatenation, 110, 111context-free , 217exponentiation, 111homomorphism, 113, 114intersection, 110of a CFG, 218prefix closure, 115recursive definition, 101regular, 125reversal, 113setminus, 110star, 112symmetric di!erence, 110uncountably many, 108union, 110universality, 294

languageempty, 107

Language , 217Lasso, 180Lasso shape, 179Lattice, 64-, 64all equivalences, 64glb, 64greatest lower bound, 64illustration on 2S , 64least upper-bound, 64lub, 64

LBA, 128, 277acceptance decidable, 313undecidable emptiness, 313

Least and greatest FP, 425Least fixed-point, 192Left-end marker, 276Left-to-right scan, 119Lexical analyzer, 149Lexicographic, 109

strictly before, 109Lexicographic order, 109LHS, 77Linear bounded automata, 277Linear bounded automaton, 128Linear CFGs, 237

Linear-time temporal logic, 405Livelock, 393Liveness, 389Logic3, 782, 78if-then, 75axiom, 75axiomatization, 324complete, 323first-order, 323FOL validity, 326higher-order, 323Hilbert style, 324if, 75implication, 75independent, 323individual, 324interpretation, 323modus ponens, 325predicate, 324proof, 75, 323propositional, 323quantification, 78rule of inference, 75sound, 323substitution, 325theorem, 323vacuous truth, 75validity, 323well-formed formula, 323w!, 323zeroth-order, 323

logic(, 75

Logic/automaton connection, 186Loop invariant, 253LTL, 405

enumerative model checking, 432expressiveness, 428semantics, 406syntax, 406

LTL vs. CTL, 405, 428

Machines

Index 467

with one stack, 4with two stacks, 4with zero stacks, 4

Map of USA, 39Mapping reduction, 301*m, 301

Matrix, 372Mechanical process, 27Message sequence chart, 394Millennium problem, 345Minimalist approach, 3Mixed left/right linear, 238Model checking, 381

vs. testing, 387BDD, 383disappearing, 387history, 381

Modular, 113homomorphism, 113substitution, 113

Modus ponens, 325MSC, 394Multiple fixed-points, 197Myhill-Nerode Theorem, 174

Natural numberas set, 20

NBA, 428, 431versus DBA, 431

Nested DFS, 434NFA, 141

$, 142., 146" moves, 143"-closure, 145/, 142/", 142concatenation, 165generalized, 170homomorphism, 168ID, 142instantaneous description, 142inverse homomorphism, 168Kleene-star, 166language, 147

prefix-closure, 169reversal, 167to DFA, 159to Regular Expressions, 170token game, 148union, 162

NFA transition function, 141NLBA, 277Non-terminals , 217Non-trivial property, 312Nondeterminism, 135, 136, 387

abstraction, 387over-approximation, 387power of machines, 137

Nondeterministic Buchi automata,431

Nondeterministic BA, 428Nondeterministic machines, 137NP, 345, 348, 350NP decider, 350NP verifier, 348NP-completeness, 345NP-hard, 350

Diophantine, 360NPC, 345

2-partition, 3643-SAT, 3543-partition, 364decider, 351funnel diagram, 354strong, 364tetris, 364verifier, 348

Number10.5??, 43cardinal, 37, 38integer, 15natural, 15real, 15

Numericstrictly before, 110

Numeric order, 110

Omega-regular language, 430syntax, 430

468 Index

One-stop shopping, 224Order

partial, 57pre, 57

Over-approximation, 135

P, 345P versus NP, 288P vs. NP, 345Parse tree , 218Parser, 217PCP, 315

computation history, 318dominoes, 315solution, 315tiles, 315undecidable, 316

PDA, 4, 245, 253/, 246ID, 246instantaneous description, 246intersection with DFAs, 293proving their correctness, 247undecidable universality, 313

PDA acceptance, 247by empty stack, 249by final state, 247

PDA to CFG, 257Periodic

ultimate, 76Philosophers, 390Photocopying machine

fixed-point, 95image transformation, 95

Pigeon, 85Pigeon-hole principle, 85POS, 332Post’s correspondence, 315Power of computers, 5Power of machines, 3, 62, 137Pre-image, 192Predicate constant, 324Predicate logic, 324Prefix, 372Prefix closure, 115

Prenexing, 371Presburger, 371

atomic, 371conversion to automata, 376encoding, 373interpretation, 373pitfall to avoid, 378quantification, 376sentences, 371term, 371

Presburger arithmetic, 126, 370Primed variable, 193Primes, 363Procedure, 3Product of sums, 332Production, 217

elementary, 219Promela, 384

accept label, 394never automaton, 394proctype, 394progress label, 394

Proof, 75, 323reductio ad absurdum, 77axiom, 75by contradiction, 77machine-checked, 441mistakes, 439model-checker, 441of a statement, 75reliability, 439rule of inference, 75theorem prover, 441

Proof by contradiction, 207Property, 28Propositional logic, 323Proving PDAs, 247Pumping

case analysis, 207full contradiction, 209stronger, incomplete, 209

Pumping Lemma, 205complete, 205, 212incomplete, 205Ja!e, 212

Index 469

one-way, 205quantifier alternation, 206Stanat and Weiss, 213

Purely left-linear , 238Purely right-linear , 238Push-down automata, 245Push-down automata , 234Push-down automaton, 4Putative queries, 10

Quantifier alternation, 206

RE, 134, 137, 295closure, 134Complementation, 138DeMorgan’s Law, 138

Reachabilityin graphs, 60multiple fixed-points, 197

Recognize, 124Recursion, 93

nonsensical, 94solution of equations, 97solving an equation, 94

Recursive definition, 77Recursively enumerable, 295Reflexive and transitive closure, 124Reflexive transitive closure, 60Regular, 125Regular Expression

to NFA, 169Regular expression, 137Regular expressions, 134Regular language

closure properties, 211Regular set

closure properties, 211Regular sets, 211Regularity, 211

preserving operations, 211Rejecting state, 119Relation, 28

irr, 53non, 53antisymmetric, 55

asymmetric, 55binary, 28, 53broken journey, 56co-domain, 29complement, 29domain, 29, 53equivalence, 58functional, 30identity, 59intransitive, 56inverse, 29irreflexive, 54non-reflexive, 55non-symmetric, 55non-transitive, 57partial functions as, 30partial order, 57preorder, 57reflexive, 54restriction, 29, 60short cut, 56single-valued, 30symmetric, 55ternary, 29total, 58total functions as, 30total order, 58transitive, 56unary, 28universal, 59

Resistor, 68Respect, 65, 113

concatenation, 113operator, 65

Reverse, 238of a CFG, 238of a CFL, 238

RHS, 77Rice’s Theorem

corrected proof, 311failing proof, 310

Rice’s theorem, 309partition, 309

Robustness of TMs, 276Run, 105

470 Index

Russell’s paradox, 17, 21

Safety, 389SAT solver, 338Satisfiability, 331

2-CNF, 341Scanner, 149

telephone number, 149Schoenfinkeled form, 30Schroder-Bernstein Theorem, 43

Nat . Bool, 44all C programs, 44

Second-order logic, 79Sentence , 218Sentential form , 218Sequence, 105Set, 16

Real versus Nat, 43cardinality, 37complement, 20comprehension, 16countable, 38empty, 16intersection, 19numbers as, 20powerset, 16, 22proper subset, 19recursively defined, 18subtraction, 19symmetric di!erence, 20union, 19unique definition, 18universal, 18, 19

Skolem constant, 326Skolem function, 326Skolemization, 326Solving one implies all, 11SOP, 332Sound, 323SPIN, 384

interleaving product, 394message sequence chart, 394MSC, 394property automaton, 394

Stack, 128

single, 128two, 128

Start symbol , 217State

accepting, 121black hole, 123final, 121

State explosion, 383State transition systems, 192String, 105

", 106length, 107substr, 107concatenation, 107empty, 106in living beings, 106lexicographic order, 109numeric order, 110reversal, 113

String classifier, 124Strong NP-completeness, 364Strongly NP-complete, 364Structural induction

proof by, 81Substitution, 325Sum of products, 332Symbol, 105, 107

bounded information, 107Symbolic model checking, 421

Tape, 275doubly infinite, 275singly infinite, 275

Telephone number NFA, 149Temporal logic, 382Term, 68Terminals , 217Testing computers, 9Tetris, 364Theorem, 323Theorem prover, 73

ACL2, 74floating point arithmetic, 75Flyspec project, 74

Therac-25, 8

Index 471

Tic-tac-toe, 198TM, 274

deterministic, 274nondeterministic, 274robust, 276

TR, 108, 295Transition function, 124

$, 121$, 124for strings, 124total, 121

Transition systems, 192Triple, 21Trivial partition, 309Tuple, 21Turing machine, 2, 128Turing recognizable, 108, 295Turing-Church thesis, 272Two stacks + control = TM, 276Twocolor, 347Type, 20

versus sets, 20

Ultimate periodicity, 179, 181Ultimately periodic, 76Unambiguous, 10Unique definition, 18

function, 77, 94functions, 82sets, 18

Uniqueness, 77Universal relation, 59Unsatisfiability core, 340Unsatisfiable CNF, 340Until

GFP, 427LFP, 426recursion, 425

Vacuously true, 75Valid, 323Validity, 326

undecidable, 329Variable ordering, 190Venus probe, 267Verifier, 348

Weird but legal C program, 44Well-formed formula, 323Well-founded partial order, 84Well-parenthesized, 224well-parenthesized, 215WFF, 323What if queries, 10Wild-card, 137

Y function, 96

Zcha!, 338Zeroth-order logic, 323

Date post:	24-May-2018
Category:	Documents
Upload:	truonghanh
View:	244 times
Download:	4 times

Computation Engineeringcs6110/GaneshBookExcerpts.pdf · · 2011-01-14Ganesh Lalitha...

Documents