Reading and Writing Mathematical Proofs

Spring 2015

Lecture 4: Beyond Basic Induction

Previously on Reading and Writing

Mathematical Proofs

Proving Correctness of Algorithms

Hoare Logic

Hoare Logic Formal system for logical reasoning about computer programs

Hoare triple

{P} C {Q}

Hoare logic contains rules to determine if Hoare triple is correct If P holds, then after running C, Q holds

Pre- and postcondition are statements about variables

precondition postcondition

command(s)

Hoare Triples

Maximum(A, n)// Algorithm that computes sum of integers in A[1..n]1. {A contains n integers} 2. r = 03. {r = 0}4. for i = 1 to n5. do {r = sum of elements in A[1..i-1]}6. r = r + A[i]7. {r = sum of elements in A[1..i]}8. {r = sum of elements in A[1..n]}9. return r

Goal is to prove Hoare triple {P} C {Q} where C is whole program We have inference rules for single commands Must “break down” Hoare triple into components

loop invariant

While Rule

While Rule

P ⇒ S , {S ⋀ B} C {S} , S ⋀ ¬B ⇒ Q{P} while B do C {Q}

We already know how to prove loops S is the invariant P ⇒ S is the initialization {S ⋀ B} C {S} is the maintenance S ⋀ ¬B ⇒ Q is the termination

It is hard to come up with a good invariant Therefore you must always prove it in Data Structures!

Summary

Hoare logic Formal system for proving algorithms Basically defines the “rules of the game”

Proofs in Data Structures No Hoare logic! (only in the background) Assignments: generally without proof If-statements: prove using case distinction Loops: prove using loop invariant

Always make the distinction between “what the code does” and “what it is supposed to do”! The goal is to prove that these two things are the same

Proving Steps

Steps of Proving

1. Figure out what needs to be shown When is a proof complete? What are the proof requirements?

2. Come up with the proof Proving techniques: induction, contradiction, etc. How to choose/combine techniques

3. Write down the proof As clearly as possible With the reader in mind

Hoare logic

No Hoare logic

Nested Loops

How to prove nested loops?

1. {P1}

2. for i = 1 to n

3. do {S1}

4. …..

5. {P2}

6. for j = i+1 to n

7. do {S2}

8. ….

9. {Q2}

10. {Q1}

{P1} for i = 1 to n do OLB {Q1}

P1 ⋀ i=1 ⇒ S1 S1 ⋀ i=n+1 ⇒ Q1

{S1 ⋀ i≤n} OLB {S1[i+1/i]}

{S1 ⋀ i≤n} … {P2}

{P2} for j = i+1 to n do … {Q2}

Q2 ⇒ S1[i+1/i]

P2 ⋀ j=i+1 ⇒ S2 S2 ⋀ j=n+1 ⇒ Q2

{S2 ⋀ j≤n} … {S2[j+1/j]}

Beyond Basic Induction

Today…

Recursion

But first…

Function Calls

How to prove a function call?

1. {P}

2. EpicFunction(x)

3. {Q}

Function should have specification: pre- and postcondition If precondition is met, then postcondition will follow Function is proved separately

What if the function call is recursive?

Then we need a different proving technique…

Recursion

EpicFunc(x)

1. {P}

2. if x = 1

3. then return 1

4. else r = EpicFunc(x-1)

5. r = r + EpicFunc(⌊x/2⌋)6. {Q}7. return r

Cannot prove what EpicFunc does using what EpicFunc does Or can we…?

Recursive calls must have “smaller” input We can use strong induction!

No idea what this does!Let’s try something else

Example

BinarySearch(A, i, j, x)

// Returns true iff A[i…j] contains x

1. {A is sorted, i + 1 ≤ j, and A[i] ≤ x < A[j]}2. if i + 1 = j

3. then return (A[i] = x)

4. h = (i + j)/2

5. if A[h] ≤ x6. then return BinarySearch(A, h, j, x)7. else return BinarySearch(A, i, h, x)

Strong induction Base case(s): when no recursive calls are made Induction step: the rest… Induction on what?

Must be “smaller” input!

ExampleTheorem

If A is sorted, i + 1 ≤ j, and A[i] ≤ x < A[j], then BinarySearch(A, i, j, x) returns true iff A[i…j] contains x

Proof

We use strong induction on |j – i|.

Base case (|j – i| = 1):Then, since A[j] > x, only A[i] can contain x. This is correctly checked by the algorithm.

Step (|j – i| ≥ 2):First note that i < h < j, so 1 ≤ |j – h| < |j – i| and 1 ≤ |h – i| < |j – i|. We consider two cases:

Case (1): A[h] ≤ x

Since A is sorted, x cannot be in A[i…h-1] and must be in A[h…j]. This is checked by the recursive call. As required, A[h] ≤ x.

Case (2): A[h] > x

Since A is sorted, x cannot be in A[h+1…j] and must be in A[i…h]….

Recursion

Notes on recursion

Always make sure recursive calls are valid Input should satisfy requirements Input should be “smaller” than original input Must eventually reach a base case

What does “smaller” input mean? You get to define it! But must satisfy certain requirements…

Induction

A more general approach…

Induction

Different types of induction

Mathematical induction P(1) P(n) ⇒ P(n+1)

Strong induction P(1) P(1) ⋀ … ⋀ P(n) ⇒ P(n+1)

Structural induction ?

Well-founded induction ?

For natural numbers

For other sets

Induction

Mathematical Induction

Prove something for all positive integers: ∀n[n ϵ ℕ: P(n)]

What if we want to use a different set: ∀x[x ϵ S: P(x)] Cannot use standard induction: P(x) ⇒ P(x+1) What to do?

Examples Prove for all strings that … Prove for all rooted binary trees that … Prove for all graphs that … Prove for all polygons that …

Set definitions

Defining Infinite Sets

Using properties: Set of rationals = {x | ∃p,q ϵ ℤ[q ≠ 0 and qx = p]}

Set of primes = {p | ¬∃d ϵ ℤ[1 < d < p and p is multiple of d]}

Set of squares = {x2 | x ϵ ℤ}

Inductive (or recursive) definition: Natural numbers ℕ:

1) 1 ϵ ℕ2) If n ϵ ℕ, then n + 1 ϵ ℕ

Full binary trees T:1) ϵ T

2) If x ϵ T and y ϵ T, then ϵ Tx y

Useful for induction!

Examples

1. Set of positive even numbers E 2 ϵ E If n ϵ E, then n + 2 ϵ E

2. Set of (non-empty) binary strings B 0 ϵ B, 1 ϵ B If X ϵ B, then 0X ϵ B and 1X ϵ B

3. Set of powers of 3: Q 1 ϵ Q If p ϵ Q, then 3p ϵ Q

4. Set of arithmetic expressions A n ϵ A for all n ϵ ℕ If e1, e2 ϵ A, then –e1, (e1), e1 + e2, e1 – e2, e1 * e2, e1 / e2 ϵ A

5. Set of prime numbers P Don’t know…

Structural Induction

Full binary trees T

1) ϵ T

2) If x ϵ T and y ϵ T, then ϵ T

Structural Induction on T Base case

Prove property for a single node Induction step

Prove property for

Can use induction hypothesis on x and y

x y

x y

Example

Theorem

A full binary tree with n nodes has (n+1)/2 leaves

Proof

We use structural induction on the set of full binary trees with the inductive rule on the previous slide.

Base case (rule (1)):For a single node n = 1 and there is (1+1)/2 = 1 leaf.

Step (rule (2)):Suppose that the subtrees x and y, with a and b nodes, have (a+1)/2 and (b+1)/2 leaves, respectively (IH). We need to show that the tree T with n nodes formed by adding a root above x and y has (n+1)/2 leaves.

A leaf of T is either a leaf of x or a leaf of y. By IH, the number of leaves of T is then (a+1)/2 + (b+1)/2 = (a+b+2)/2. Since n = a + b + 1, we get that (n+1)/2 = (a+b+2)/2, as required.

Example

Theorem

A full binary tree with n nodes has (n+1)/2 leaves

Proof

We use structural induction on the set of full binary trees with the inductive rule on the previous slide.

Base case (rule (1)):For a single node n = 1 and there is (1+1)/2 = 1 leaf.

Step (rule (2)):Consider a full binary tree T with subtrees x and y.

Let x and y have a and b nodes, respectively, such that n = a + b + 1.

A leaf of T is either a leaf of x or a leaf of y. By the IH, we get that T has (a+1)/2 + (b+1)/2 = (a+b+2)/2 = (n+1)/2 leaves.

Isn’t this the same as strong induction?

Practice

Fancy Sequences S

1) x ϵ S for all x ϵ ℕ2) If Y, Z ϵ S and x ϵ ℕ such that x ∉ Y and x ∉ Z, then YxZ

ϵ S

What does this mean? S contains sequences of integers Any sequence of 1 integer is in S If x ϵ ℕ is not in sequences Y, Z ϵ S, then YxZ is also in S

Examples 12345 ϵ S 121423 ϵ S 12334 ∉ S 12121 ∉ S

Practice

Fancy Sequences S

1) x ϵ S for all x ϵ ℕ2) If Y, Z ϵ S and x ϵ ℕ such that x ∉ Y and x ∉ Z, then YxZ

ϵ S

TheoremEvery non-empty contiguous subsequence of a fancy sequence contains at least one natural number uniquely

PracticeFancy Sequences S

1) x ϵ S for all x ϵ ℕ2) If Y, Z ϵ S and x ϵ ℕ such that x ∉ Y and x ∉ Z, then YxZ ϵ S

TheoremEvery non-empty contiguous subsequence of a fancy sequence contains at least one natural number uniquely

ProofWe use structural induction on fancy sequences with the above rule.Base case (rule (1)):

The (sub)sequence contains only one number, so it must be unique.

Step (rule (2)):We perform a case distinction based on the subsequence:Case 1 (subsequence contains x): Since x is not in Y or Z, x is unique.Case 2 (subsequence contained in Y): By the IH on Y, the result holds.Case 3 (subsequence contained in Z): By the IH on Z, the result holds.

Induction

Different types of induction

Mathematical induction P(1) P(n) ⇒ P(n+1)

Strong induction P(1) P(1) ⋀ … ⋀ P(n) ⇒ P(n+1)

Structural induction Induction using inductive definition of set

Well-founded induction ?

Binary Search (again)

BinarySearch(A, i, j, x)

// Returns true iff A[i…j] contains x

1. {A is sorted, i + 1 ≤ j, and A[i] ≤ x < A[j]}2. if i + 1 = j

3. then return (A[i] = x)

4. h = (i + j)/2

5. if A[h] ≤ x6. then return BinarySearch(A, h, j, x)7. else return BinarySearch(A, i, h, x)

Smaller input Size of input defined as |j – i| ⇒ strong induction on |j –

i| In general: Order on set of possible inputs (A, i, j, x) ≺ (A’, i’, j’, x’) if |j – i| < |j’ – i’|

Must be “smaller” input!

Partial order

Strict partial order relation ≺ on set S Binary relation: x ≺ y for certain pairs x, y ϵ S Anti-reflexive: x ⊀ x Anti-symmetric: if x ≺ y, then y ⊀ x Transitive: If x ≺ y and y ≺ z, then x ≺ z

Examples For x, y ϵ ℤ: x ≺ y iff x < y For sets X, Y: X ≺ Y iff X ⊂ Y ({1,3} ≺ {1,2,3,4}) For strings S1, S2: S1 ≺ S2 iff S1 is substring of S2 (“ab”

≺ “cab”) For trees T1, T2: T1 ≺ T2 iff T1 is substring of T2

Partial order

Does (strong) induction work for any partial order ≺ ? Not exactly…

Theorem

For all x ϵ ℤ it holds that x = x + 1Proof

By induction on x:

We apply IH to x – 1, so that x – 1 = x. (Note that x – 1 < x)

By adding 1 to both sides we obtain that x = x + 1.

We need base cases! The partial order ≺ must have minimal elements Minimal elements are base cases

Well-founded Relation

Well-founded relation ≺ on S Every non-empty subset X ⊆ S must have a minimal

element Minimal element m ϵ X: for all x ϵ X it holds that x ⊀ m S contains no infinite descending chains: a ≻ b ≻ c ≻ ….

Well-founded induction on S First need well-founded (partial) order ≺ on S Base case(s): Minimal elements of S Induction step: If P(x) for all x ≺ y, then P(y) (for all y ϵ S)

Practice

Which of these partial orders are well-founded?

For x, y ϵ ℤ: x ≺ y iff x < y

For x, y ϵ ℤ: x ≺ y iff |x| < |y|

For x, y ϵ ℕ: x ≺ y iff y is a multiple of x and x ≠ y

For rational numbers x, y ϵ ℚ: x ≺ y iff x < y

For strings S1, S2: S1 ≺ S2 iff S1 lexicographically before S2

½ > ⅓ > ¼ > ⅕ > …

“b” ≻ “ab” ≻ “aab” ≻ “aaab” ≻ …

Practice

Ackermann(m, n)

1. if m = 0

2. then return n+1

3. else if n = 0

4. then return Ackermann(m – 1, 1)

5. else return Ackermann(m – 1, Ackermann(m, n – 1))

A recursive function terminates on all input if and only if there exists a well-founded order ≺ on the set of inputs such that:

“input of recursive call” ≺ “original input”

Does the Ackermann function terminate on all inputs?

(m, n) ≺ (m’, n’) iff m < m’ or m = m’ and n < n’

Practice

T(1, n) = 1

T(m, 1) = 1

T(m, n) = T(m/2, n) + T(m, n/2) – T(m/2, n/2) + 1 m, n > 1

Theorem

T(m, n) = log(m) log(n) + 1

Practice

Theorem

T(m, n) = log(m) log(n) + 1

Proof

We use induction on (m, n), where (m, n) ≺ (m’, n’) iff m < m’ or n < n’.Base case (m = 1):

T(1, n) = 1 = log(1) log(n) + 1.

Base case (n = 1):T(m, 1) = 1 = log(m) log(1) + 1.

Step (m, n > 1):T(m, n) = T(m/2, n) + T(m, n/2) – T(m/2, n/2) + 1{definition}T(m, n) = log(m/2) log(n) + log(m) log(n/2) – log(m/2) log(n/2) + 2 {IH}T(m, n) = log(m/2) (log(n) – log(n/2)) + log(m) log(n/2) + 2T(m, n) = log(m/2) + 1 + log(m) (log(n) – 1) + 1T(m, n) = log(m) log(n) + 1

Well-founded Induction

Well-founded induction Very general type of (strong) induction … but also very abstract

Main lesson If you can order the elements of a set, you can do induction Induction hypothesis may always be applied to “smaller” elements

Recursion First argue that it terminates Then you can use IH on recursive call to argue correctness

Induction

Different types of induction

Mathematical induction P(1) P(n) ⇒ P(n+1)

Strong induction P(1) P(1) ⋀ … ⋀ P(n) ⇒ P(n+1)

Structural induction Induction using inductive definition of set

Well-founded induction Strong induction for any set Just needs well-founded order…

Summary

Steps of Proving

1. Figure out what needs to be shown When is a proof complete? What are the proof requirements?

2. Come up with the proof Proving techniques: induction, contradiction, etc. How to choose/combine techniques

3. Write down the proof As clearly as possible With the reader in mind

“Rules” of proving

“Rules” of proving

“Tools” of proving

hard, creative, fun(?) part