36
1 Recursive Algorithm Analysis Dr. Ying Lu [email protected] RAIK 283: Data Structures & RAIK 283: Data Structures & Algorithms Algorithms September 13, 2012
| Date post: | 16-Dec-2015 |
| Category: |
Documents |
| Upload: | margaretmargaret-wright |
| View: | 224 times |
| Download: | 0 times |
1
Recursive Algorithm Analysis
Dr. Ying [email protected]
RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms
September 13, 2012
2
Giving credit where credit is due:Giving credit where credit is due:• Most of the lecture notes are based on the slides Most of the lecture notes are based on the slides
from the Textbook’s companion websitefrom the Textbook’s companion website
http://www.aw-bc.com/info/levitinhttp://www.aw-bc.com/info/levitin
• Several slides are from Hsu Wen Jing of the Several slides are from Hsu Wen Jing of the National University of SingaporeNational University of Singapore
• I have modified them and added new slidesI have modified them and added new slides
RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms
3
Example: a recursive algorithmExample: a recursive algorithm
Algorithm:Algorithm:if if nn=0 then =0 then FF((nn) := 1) := 1
else else FF((nn) := ) := FF((nn-1) * -1) * nnreturn return FF((nn) )
4
Algorithm F(n)
// Compute the nth Fibonacci number recursively
//Input: A nonnegative integer n
//Output: the nth Fibonacci number
if n 1 return n
else return F(n-1) + F(n-2)
Example: another recursive Example: another recursive algorithmalgorithm
5
Recurrence RelationRecurrence Relation
Recurrence RelationRecurrence Relation
6
Recurrence RelationRecurrence Relation
Recurrence Relation:Recurrence Relation: an equation or inequality that an equation or inequality that describes a function in terms of its value on smaller describes a function in terms of its value on smaller inputsinputs
7
Example: a recursive algorithmExample: a recursive algorithm
Algorithm:Algorithm:if if nn=0 then =0 then FF((nn) := 1) := 1
else else FF((nn) := ) := FF((nn-1) * -1) * nnreturn return FF((nn) )
What does this algorithm compute?What does this algorithm compute?
8
Example: a recursive algorithmExample: a recursive algorithm
Algorithm:Algorithm:if if nn=0 then =0 then FF((nn) := 1) := 1
else else FF((nn) := ) := FF((nn-1) * -1) * nnreturn return FF((nn) )
What does this algorithm compute?What does this algorithm compute?
What’s the basic operation of this algorithm?What’s the basic operation of this algorithm?
9
Example: recursive evaluation Example: recursive evaluation of of nn ! !
Recursive definition of Recursive definition of nn!:!: Algorithm:Algorithm:
if if nn=0 then =0 then FF((nn) := 1) := 1else else FF((nn) := ) := FF((nn-1) * -1) * nn
return return FF((nn) )
M(n): number of multiplications to compute M(n): number of multiplications to compute nn! ! with this recursive algorithmwith this recursive algorithm
10
Example: recursive evaluation Example: recursive evaluation of of nn ! !
Recursive definition of Recursive definition of nn!:!: Algorithm:Algorithm:
if if nn=0 then =0 then FF((nn) := 1) := 1else else FF((nn) := ) := FF((nn-1) * -1) * nn
return return FF((nn) )
M(n): number of multiplications to compute M(n): number of multiplications to compute nn! ! with this recursive algorithmwith this recursive algorithm
Could we establish a Could we establish a recurrence relationrecurrence relation for for deriving M(n)? deriving M(n)?
11
Example: recursive evaluation Example: recursive evaluation of of nn ! !
Definition:Definition: n n ! = 1*2! = 1*2*…*(n-*…*(n-1)*1)*nn Recursive definition of Recursive definition of nn!:!: Algorithm:Algorithm:
if if nn=0 then =0 then FF((nn) := 1) := 1else else FF((nn) := ) := FF((nn-1) * -1) * nn
return return FF((nn) )
M(n) = M(n-1) + 1M(n) = M(n-1) + 1 Initial Condition: M(0) = ?Initial Condition: M(0) = ?
12
Example: recursive evaluation Example: recursive evaluation of of nn ! !
Recursive definition of Recursive definition of nn!:!: Algorithm:Algorithm:
if if nn=0 then =0 then FF((nn) := 1) := 1
else else FF((nn) := ) := FF((nn-1) * -1) * nn
return return FF((nn) )
M(n) = M(n-1) + 1M(n) = M(n-1) + 1 Initial condition: M(0) = 0Initial condition: M(0) = 0 Explicit formula for M(n) in terms of n only? Explicit formula for M(n) in terms of n only?
13
Time efficiency of recursive Time efficiency of recursive algorithmsalgorithms
Steps in analysis of recursive algorithms: Steps in analysis of recursive algorithms: Decide on parameter Decide on parameter nn indicating indicating input sizeinput size Identify algorithm’s Identify algorithm’s basic operationbasic operation Determine Determine worstworst, , averageaverage, and , and bestbest case for inputs of size case for inputs of size nn
Set up a recurrence relation and initial condition(s) for Set up a recurrence relation and initial condition(s) for CC((nn)-the number of times the basic operation will be )-the number of times the basic operation will be executed for an input of size executed for an input of size nn
Solve the recurrence to obtain a closed form or Solve the recurrence to obtain a closed form or determine the order of growth of the solution (see determine the order of growth of the solution (see Appendix B)Appendix B)
14
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
Problem:Problem:• Given three pegs (A, B, C) and n disks of different sizes Given three pegs (A, B, C) and n disks of different sizes • Initially, all the disks are on peg A in order of size, the largest Initially, all the disks are on peg A in order of size, the largest
on the bottom and the smallest on top on the bottom and the smallest on top • The goal is to move all the disks to peg C using peg B as an The goal is to move all the disks to peg C using peg B as an
auxiliaryauxiliary• Only 1 disk can be moved at a time, and a larger disk cannot Only 1 disk can be moved at a time, and a larger disk cannot
be placed on top of a smaller onebe placed on top of a smaller one
A C
B
n disks
15
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
Design a recursive algorithm to solve this problem:Design a recursive algorithm to solve this problem:• Given three pegs (A, B, C) and n disks of different sizes Given three pegs (A, B, C) and n disks of different sizes • Initially, all the disks are on peg A in order of size, the largest on Initially, all the disks are on peg A in order of size, the largest on
the bottom and the smallest on top the bottom and the smallest on top • The goal is to move all the disks to peg C using peg B as an The goal is to move all the disks to peg C using peg B as an
auxiliaryauxiliary• Only 1 disk can be moved at a time, and a larger disk cannot be Only 1 disk can be moved at a time, and a larger disk cannot be
placed on top of a smaller oneplaced on top of a smaller one
A C
B
n disks
16
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
Step 1: Solve simple case when n<=1?Step 1: Solve simple case when n<=1?Just trivialJust trivial
A C
B
A C
B
Move(A, C)
17
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
Step 2: Assume that a smaller instance can be Step 2: Assume that a smaller instance can be solved, i.e. can move n-1 disks. Then?solved, i.e. can move n-1 disks. Then?
A C
B
A C
B
A C
B
18
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
A C
BA C
B
A C
B
A C
B
19
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
A C
BA C
B
A C
B
A C
B
TOWER(n, A, B, C)
20
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
A C
BA C
B
A C
B
A C
B
TOWER(n, A, B, C)
TOWER(n-1, A, C, B)
Move(A, C)
TOWER(n-1, B, A, C)
21
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
TOWER(n-1, A, C, B); Move(A, C);
TOWER(n-1, B, A, C)}
if n<1 return;
22
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
TOWER(n-1, A, C, B); Move(A, C);
TOWER(n-1, B, A, C)}
if n<1 return;
Algorithm analysis: Algorithm analysis: Input size? Basic operation?Input size? Basic operation?
23
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
TOWER(n-1, A, C, B); Move(A, C);
TOWER(n-1, B, A, C)}
if n<1 return;
Algorithm analysis: Algorithm analysis: Do we need to differentiate best case, worst Do we need to differentiate best case, worst
case & average case for inputs of size n?case & average case for inputs of size n?
24
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
TOWER(n-1, A, C, B); Move(A, C);
TOWER(n-1, B, A, C)}
if n<1 return;
Algorithm analysis: Algorithm analysis: Set up a recurrence relation and initial Set up a recurrence relation and initial
condition(s) for C(n)condition(s) for C(n)
25
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
TOWER(n-1, A, C, B); Move(A, C);
TOWER(n-1, B, A, C)}
if n<1 return;
Algorithm analysis: Algorithm analysis: C(n) = 2C(n-1)+1C(n) = 2C(n-1)+1
26
In-Class ExerciseIn-Class Exercise
P. 76 Problem 2.4.1 (c): solve this recurrence relation: P. 76 Problem 2.4.1 (c): solve this recurrence relation:
x(n) = x(n-1) + n for n>0, x(0)=0x(n) = x(n-1) + n for n>0, x(0)=0 P. 77 Problem 2.4.4: consider the following recursive algorithm: P. 77 Problem 2.4.4: consider the following recursive algorithm:
• Algorithm Q(n)Algorithm Q(n)// Input: A positive integer n// Input: A positive integer n
If n = 1 return 1If n = 1 return 1
else return Q(n-1) + 2 * n – 1else return Q(n-1) + 2 * n – 1
• A. Set up a recurrence relation for this function’s values and solve A. Set up a recurrence relation for this function’s values and solve it to determine what this algorithm computesit to determine what this algorithm computes
• B. Set up a recurrence relation for the number of multiplications B. Set up a recurrence relation for the number of multiplications made by this algorithm and solve it. made by this algorithm and solve it.
• C. Set up a recurrence relation for the number of C. Set up a recurrence relation for the number of additions/subtractions made by this algorithm and solve it. additions/subtractions made by this algorithm and solve it.
27
Example: BinRec(n) Example: BinRec(n)
Algorithm BinRec(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary representation
if n = 1 return 1
else return BinRec( n/2 ) + 1
28
Smoothness ruleSmoothness rule
If T(n) If T(n) (f(n)) (f(n))
for values of n that are powers of b, where b for values of n that are powers of b, where b 2, 2,
then then
T(n) T(n) (f(n))(f(n))
29
Example: BinRec(n) Example: BinRec(n)
Algorithm BinRec(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary representation
if n = 1 return 1
else return BinRec( n/2 ) + 1
If C(n) If C(n) (f(n)) (f(n)) for values of n that are powers of b, where b for values of n that are powers of b, where b 2, 2, then then C(n) C(n) (f(n))(f(n))
30
Fibonacci numbersFibonacci numbers
The Fibonacci sequence:The Fibonacci sequence:0, 1, 1, 2, 3, 5, 8, 13, 21, … 0, 1, 1, 2, 3, 5, 8, 13, 21, …
Fibonacci recurrence:Fibonacci recurrence:F(F(nn) = F() = F(nn-1) + F(-1) + F(nn-2) -2)
F(0) = 0 F(0) = 0
F(1) = 1F(1) = 1
2nd 2nd order linear homogeneous order linear homogeneous recurrence relation recurrence relation
with constant coefficientswith constant coefficients
31
Solving linear homogeneous Solving linear homogeneous recurrence relations with constant recurrence relations with constant coefficientscoefficients
Easy first: 1Easy first: 1stst order LHRRCCs: order LHRRCCs:CC((nn) = ) = a Ca C((nn -1) -1) CC(0) = (0) = tt … Solution: … Solution: CC((nn) =) = t a t ann
Extrapolate to 2Extrapolate to 2ndnd order orderLL((nn) = ) = aa LL((nn-1) + -1) + bb LL((nn-2) … A solution?: -2) … A solution?: LL((nn) = ) = ??
Characteristic equation (quadratic)Characteristic equation (quadratic) Solve to obtain roots Solve to obtain roots rr11 and and rr2 2 ((quadratic formula))
General solution to RR: linear combination of General solution to RR: linear combination of rr11nn
and and rr22nn
Particular solution: use initial conditionsParticular solution: use initial conditions
32
Solving linear homogeneous Solving linear homogeneous recurrence relations with constant recurrence relations with constant coefficientscoefficients
Easy first: 1Easy first: 1stst order LHRRCCs: order LHRRCCs:CC((nn) = ) = a Ca C((nn -1) -1) CC(0) = (0) = tt … Solution: … Solution: CC((nn) =) = t a t ann
Extrapolate to 2Extrapolate to 2ndnd order orderLL((nn) = ) = aa LL((nn-1) + -1) + bb LL((nn-2) … A solution?: -2) … A solution?: LL((nn) = ) = ??
Characteristic equation (quadratic)Characteristic equation (quadratic) Solve to obtain roots Solve to obtain roots rr11 and and rr2 2 ((quadratic formula))
General solution to RR: linear combination of General solution to RR: linear combination of rr11nn
and and rr22nn
Particular solution: use initial conditionsParticular solution: use initial conditions
Explicit Formula for Explicit Formula for Fibonacci Number: F(n) = F(n-1) +F(n-2)Fibonacci Number: F(n) = F(n-1) +F(n-2)
33
1. Definition based recursive algorithm1. Definition based recursive algorithm
Computing Fibonacci numbersComputing Fibonacci numbers
Algorithm F(n)
// Compute the nth Fibonacci number recursively
//Input: A nonnegative integer n
//Output: the nth Fibonacci number
if n 1 return n
else return F(n-1) + F(n-2)
34
2. Nonrecursive brute-force algorithm2. Nonrecursive brute-force algorithm
Computing Fibonacci numbersComputing Fibonacci numbers
Algorithm Fib(n)
// Compute the nth Fibonacci number iteratively
//Input: A nonnegative integer n
//Output: the nth Fibonacci number
F[0] 0; F[1] 1
for i 2 to n do
F[i] F[i-1] + F[i-2]
return F[n]
35
Computing Fibonacci numbersComputing Fibonacci numbers
3. Explicit formula algorithm3. Explicit formula algorithm
integernearest the torounded )2
51(
51
)( nnF
Special care in its implementation: Special care in its implementation:
Intermediate results are irrational numbersIntermediate results are irrational numbersTheir approximations in the computer are accurate enoughTheir approximations in the computer are accurate enough
Final round-off yields a correct result Final round-off yields a correct result
36
In-Class Exercises In-Class Exercises
What is the explicit formula for A(n)?What is the explicit formula for A(n)?A(A(nn) = 3A() = 3A(nn-1) – 2A(-1) – 2A(nn-2) A(0) = 1 A(1) = 3-2) A(0) = 1 A(1) = 3
P.83 2.5.3. Climbing stairs: Find the number of P.83 2.5.3. Climbing stairs: Find the number of different ways to climb an n-stair stair-case if each different ways to climb an n-stair stair-case if each step is either one or two stairs. (For example, a 3-step is either one or two stairs. (For example, a 3-stair staircase can be climbed three ways: 1-1-1, 1-stair staircase can be climbed three ways: 1-1-1, 1-2, and 2-1.)2, and 2-1.)
