An Example in the Design and Analysis of an Algorithm

• Demonstrates:

-- Recurrence Relations

-- Design Techniques

-- The fact that analysis provide useful information 

• Problem to be solved: BUNNY PROBLEM

When a pair of rabbits is less than one month old, it cannot reproduce, but during the second and each succeeding month of life, each pair of rabbits gives rise to one new pair of rabbits. Given that you (Fibonacci) just purchased a pair of new born rabbits, how many pairs will you have n months from now?

Illustration

Observation Month Pairs of newborn

rabbits at month’s end (S(n))

Pairs of mature rabbits at month’s

end (B(n))

Total pairs of rabbits at month’s

end (f(n))

01234567…

10112358…

0112358

13…

112358

1321…

The Recurrence Relation• i 0 1 2 3 4 5 6 7 8 9 ...

f(i) 1 1 2 3 5 8 13 21 34 55 ... 

• f(i) satisfies the following recurrence relation

f(i) = f(i-1) + f(i-2) for I ≥ 2, f(0) = f(1) = 1

 

B(n+1) = B(n) + S(n) = f(n)

S(n+1) = B(n) = f(n-1)

f(n+1) = B(n+1) + S(n+1) = f(n) + f(n-1)

Solution #11. Function f(n: integer): integer;

2. begin

3. if n ≤ 1 then return(1)

4. else return(f(n-1)+f(n-2))

5. end.

 • Analysis: Two factors -- # of function calls and # of

additions.

Let FC(n) = # of function calls needed to evaluate f(n)

ADD(n) = # of additions needed

• Consider a tree of function calls (showing the value of n at each node).

Analysis

• Thus, we have the recurrence relations

FC(n) = FC(n-1) + FC(n-2) + 1, if n≥2

FC(0) = FC(1) = 1

ADD(n) = ADD(n-1) + ADD(n-2) + 1, if n≥2

ADD(0) = ADD(1) = 0• Later, we will learn how to solve such equations. For

now

f(n) =

FC(n) = 2f(n)-1, ADD(n) = f(n)-1

Derivation

1

2

51

5

11

2

51

5

1

nn

• Since  

• Hence, f(n) =

=

ADD(n) and FC(n) = O(1.618n) Exponential!!

Asymptotic Performance

01

2

51,1

2

51

n

1

2

51

5

11

2

51

5

1

nn

)618.1(1

2

51

5

1 nO

nO

Avoid Redundant Computations• We can use the dynamic programming technique.

Idea: Solving the problem (calculating f(n)) consists of solving two smaller problems (calculating f(n-1) and f(n-2)) and "merging" (i.e., adding) the two smaller solutions to obtain a solution to the original problem. Also, we will have to solve the same problem more than once. When a problem has these characteristics, build up a TABLE of SOLUTIONS to the smaller problems, starting with the smallest, and proceed upwards to the largest (original) problem.

Solution #2

1. Program FIB

2. var f: array[0..n] of integer; i: integer;

3. begin

4. f[0] := 1;

5. f[1] := 1;

6. for i := 2 to n do f[i] := f[i-1] + f[i-2];

7. end.

• The answer is stored in f[n].

Analysis• Clearly n-1 operations (additions) are required -- time is

O(n). Space required is also O(n). (Space can be reduce to O(1).)

• This method usually trades space for time (though not in this case).

LINEAR!! a vast improvement.

Modified Solution #21. Program FIB

2. var a, b, i: integer;

3. begin

4. a := 1;

5. b := 1;

6. for i := 2 to n do

7. begin b:= a + b; a := b – a end;

8. end.

• The answer is stored in the variable “b”.

Interlude•  This still doesn’t seem very good. Do we need to calcula

te all the f(i)? Can't we skip over some?• Try substituting recurrence into itself.

 

f(n) = f(n-1) + f(n-2) = f(n-2) + f(n-3) + f(n-2)

= 2f(n-2) + f(n-3)

= 3f(n-3) + 2f(n-4)

= 5f(n-4) + 3f(n-5)

= 8f(n-5) + 5f(n-6) (these #s look familiar )

Conjecture:

f(n) = f(a)f(n-a) + f(a-1)f(n-a-1) where n≥2; n-1 ≥ a ≥1.

1. Function f(x : integer) : integer;

2. var i, j: integer;

3. begin

4. if x ≤1 then return(1)

5. else begin

6. j := x/2 ; 7. i := x/2;

8. return(f(i)*f(j)+f(i-1)*f(j-1));

9. end;

10. end.

Solution #3

Analysis• Why did we choose i and j x/2?

Another design heuristic -- BALANCING: divide problem sizes as evenly as possible.

"Divide and Conquer".

• Let n = 100

• Look at "tree" of function calls

(notice that FC(n) = 4FC(n/2)+1, FC(1)=1).

Analysis

What Happened?

• But obviously the above can be improved. f(100) calls f(50) twice!

 • Another important technique -- (subtree isomorphism)

Eliminate REDUNDANT Computation

A Possible Improvement• Consider 2 cases:

– x is even then i = j we want to return f(i)2+f(i-1)2.– x is odd then i-1 = j we want to return f(i-1)*(f(i)+f(i-2)).

• What does the tree look like now?– An even node has 1 even son and 1 odd son.– Some odd nodes have 2 odd and 1 even sons (5, 9,

13, ..., i.e. 1 mod 4).– The rest have 1 odd and 2 even sons ( 3 mod 4).

• The tree has between O(n1.271) and O(n1.385) nodes.• Using the relation f(i) = f(i-1) + f(i-2), we have

f(i-2) = f(i) - f(i-1), the "odd" return value can be further simplified: f(i-1)(f(i)+f(i-2)) = f(i-1)(2f(i)-f(i-1))

Solution #4 1. Function f(x : integer) : integer;2. var i, FI, FIM1: integer;3. begin4. if x ≤1 then return(1)5. else begin6. i := x/2;7. FI := f(i);8. FIM1 := f(i-1);9. if x is even 10. then return(FI**2+FIM1**2)11. else return(FIM1*(2*FI-FIM1))12. end;13.end.

Analysis• Tree of calls is now completely binary. Take n = 32,

• Height of tree is about log n, hence it has about n nodes, i.e., FC(n) = O(n).

Are You Satisfied?• Obviously there still is some redundant computation. Can

all the calls at one level be done only once? Then we would have:

Solution #5 Procedure calc(x : integer; var fx, fxm1: integer);

var i, fi, fim1, fim2: integer;begin

if x ≤ 1 then begin fx := 1; fxm1 := 1 end else begin i := x/2; calc(i, fi, fim1) if x is even then begin

fx := fi**2+fim1**2; fxm1 := fim1*(2*fi-fim1)) end

else begin fim2 := fi-fim1; fx := fim1*(2*fi-fim1)); fxm1 := fim1**2+fim2**2 end;

end;end.

Analysis: Tree obviously has (log n) nodes!

Solution #6

Repeated Squaring

• X(n) = AX(n-1) = A (A X(n-2)) = … = An-1 X(1).

• Use this relation, we can design a (log n) algorithm for calculating Fibonacci numbers.

• Hint: Calculate A, A2, A4, …, A2log n, then compute

An-1 using these items.

Questions: Is there a better way for calculating An-1?

Exercise• Implement the above solutions 1, 2 and 6, and compare

their running time for calculating f(50), f(100), and f(150).

• Can your programs calculate f(500)?

• How about f(1000)?

• Now, you should realize that real implementation sometimes take a lot of efforts even you understand the underlying algorithm.