Date post: | 26-Mar-2015 |
Category: |
Documents |
Upload: | tyler-burgess |
View: | 222 times |
Download: | 0 times |
한양대학교 정보보호 및 알고리즘 연구실
2008. 2. 12이재준
담당교수님 : 박희진 교수님1
Recurrences
Contents of Table
Recurrence solution methodsThe substitution method
The recursion-tree method
2
Relationship between asymptotic notations
A. Growth of function
B. Recurrence
3
3. Relationship between this notations
• Analogy
f(n) = Θ(g(n)) ≈ f(n) = g(n).f(n) = O(g(n)) ≈ f(n) ≤ g(n).f(n) = Ω(g(n)) ≈ f(n) ≥ g(n).f(n) = o(g(n)) ≈ f(n) < g(n). f(n) = ω(g(n)) ≈ f(n) > g(n).
4
3. Relationship between this notations
• Transitivity
• Transpose symmetry
f(n) = Θ(g(n)) and g(n) = Θ(h(n)) imply f(n) = Θ(h(n)) ,f(n) = O(g(n)) and g(n) = O(h(n)) imply f(n) = O(h(n)) , f(n) = Ω(g(n)) and g(n) = Ω(h(n)) imply f(n) = Ω(h(n)) ,f(n) = o(g(n)) and g(n) = o(h(n)) imply f(n) = o(h(n)) ,f(n) = ω(g(n)) and g(n) = ω(h(n)) imply f(n) = ω(h(n)).
f(n) = O(g(n)) if and only if g(n) = Ω(f(n)),f(n) = o(g(n)) if and only if g(n) = ω(f(n)).
5
3. Relationship between this notations
• Reflexivity
f(n) = Θ(f(n))f(n) = O(f(n))f(n) = Ω(f(n))
• Symmetry
f(n) = Θ(g(n)) if and only if g(n) = Θ(f(n)).
Recurrences
6
• Recurrence
When should we use the recurrence?
When algorithm contains a recursive call to itself, Its running time can often be described by a recurrence.
Recurrences
7
• Recurrence - Definition Equation or inequality that describes a function in terms
of its value on smaller inputs.
Recurrences
8
• Recurrence Example
Worst-case running time of T(n) could be described by the recurrence
Merge-Sort procedure
Show that the solution of T(n) = Θ(n log n)
Merge-Sort procedure
Show that the solution of T(n) = Θ(n log n)
)()2/(2
)1()(
nnTnT
if n=1
if n>1
Recurrences
9
• Recurrence Solution method to example
1) The substitution method
guess a bound and prove our guess correct.
2) The recursion-tree method
convert the recurrence into a tree
whose nodes represent the costs incurred at various level of recursion
3) The master method
determining asymptotic bounds
for many simple recurrences of the form)()/()( nfbnaTnT ( a ≥ 1, b ≥ 1, and f(n) is given function )
1. Technicalities for abbreviation
• Technicalities 1) assumption of integer
The running time T(n) of algorithm is only defined when
n is an integer 2) floors / ceilings
We often omit floors and ceilings3) boundary condition
The running time of an algorithm on a constant-sized input is
a constant that we typically ignore.
1. Technicalities for abbreviation
• Technicalities ExampleThe recurrence describing the worst-case running time of Merge-Sort
assumption of integer
Omit boundary condition
)()2/(2
)1()(
nnTnT
if n=1
if n>1
if n=1
if n>1
)()2/()2/(
)1()(
nnTnTnT
)()2/(2)( nnTnT
)()2/(2
)1()(
nnTnT
if n=1
if n>1
Omit floors and ceilings
2. Recurrence solution methods
12
• Substitution Method - Definition
- It can be used to establish either upper or lower bounds on a recurrence.
1. Guess the form of the solution
2. Use mathematical induction to find the constant and show that the solution works.
1. Guess the form of the solution
2. Use mathematical induction to find the constant and show that the solution works.
2. Recurrence solution methods
13
• Substitution Method Example
Let us determine an upper bound on the recurrence
Let us determine an upper bound on the recurrence
nnTnT )2/(2)(
2. Recurrence solution methods
14
• Substitution Method Solution to example
1. guess that the solution is 2. prove that ( c > 0 )
1. guess that the solution is 2. prove that ( c > 0 )
)lg()( nnnT
ncnnT lg)(
We need to find constant c and n0 to meet the requirement about assumption T(n) ≤ cn lg n ( c>0 )
We using the Mathematical induction to show that our assumption holds for the boundary condition.
2. Recurrence solution methods
15
• Substitution Method Solution to example
Mathematical induction We can prove something stronger for a given value by assuming
something stronger for smaller values.
k means ⌊n/2⌋
k+1 means n
nnTnT )2/(2)(
→ T(1), T(2), T(3), … ….. T(⌊n/2⌋ ), T(n)
k k+1
2. Recurrence solution methods
16
• Substitution Method Solution to example
Recurrence :
Guess : T(n)= O(n)Prove : T(n) ≤ cn
nnTnT )2/(2)(
k means k+1 means
n
-> 2/n
Substitution
T (⌊n/2 ) ≤ ⌋ c ⌊n/2 lg (⌋ ⌊n/2 )⌋
T(n) ≤ 2(c ⌊n/2 lg(⌋ ⌊n/2 )) + ⌋ n ≤ cn lg(n/2) + n ( because, ⌊n/2 < ⌋ n/2 )
= cn lg n - cn lg 2 + n = cn lg n -(c+1)n ( It’s our assumed fact )
2. Recurrence solution methods
17
• Substitution Method Solution to example
Assumed Fact : cn lg n –(c+1)nProve : T(n) ≤ cn
T(n) ≤ cn lg n – (c+1)n ≤ cn lg n (as long as c ≥
1)
Assumed Fact ≤ Prove
2. Recurrence solution methods
18
• Substitution Method Solution to example
- Find the constant n0 - we can take advantage of asymptotic notation,
we can replace base case
T(1) ≤ cn lg n T(1) = 1 but, c1 lg1 = 0
- So, we only have to prove T (n) = cn lg n for n ≥ n0 for n (n0 =2)
Replace T(1) by T(2) and T(3) as the base cases by letting n0 =2.
Replace T(1) by T(2) and T(3) as the base cases by letting n0 =2.
nnTnT )2/(2)(
2. Recurrence solution methods
19
• Substitution Method Solution to example
- Find the constant c - choosing c large enough
T (2) = 4 and T (3) = 5.T (2) = 4 ≤ c ( 2 lg 2 ) T (3) = 5 ≤ c ( 3 lg 3 ).
Any choice of c ≥ 2 suffices for base case of n=2 and n =3 Any choice of c ≥ 2 suffices for base case of n=2 and n =3
nnTnT )2/(2)(
2. Recurrence solution methods
20
• Substitution Method Skill
- SubtletiesWe can prove something stronger for given value by assuming something stronger for a smaller values.
Overcome difficulty by subtracting a lower-order term from our previous guess
2. Recurrence solution methods
21
• Substitution Method Skill - Subtleties example
Recurrence :
Guess : T(n)= O(n)Prove : T(n) ≤ cn
1)2/()2/()( nTnTnT
Substitution
12/2/)( ncncnT
1cn
k means
k+1 means n
2/)2/( ncnT
2/)2/( ncnT
→ 2/n
2/n →
So, we now prove T(n) ≤ cn-bSo, we now prove T(n) ≤ cn-b
2. Recurrence solution methods
22
• Substitution Method Skill - Subtleties example
12/2/)( ncncnT
1cn
cn ( It’s Impossible. Then T(n)= O(n) is wrong? )
Subtracting a lower-order term from our previous guess T(n) ≤ cn
2. Recurrence solution methods
23
• Substitution Method Skill - Subtleties example
bcn
bcn
12
( b≥1, c must be chosen large enough )
1)2/()2/( bncbncT(n)
Recurrence :
Guess : T(n)= O(n)Prove : T(n) ≤ cn-b
1)2/()2/()( nTnTnT
2. Recurrence solution methods
24
• Substitution Method Skill
- Avoid pitfalls We need to prove the exact form of the inductive
hypothesis
T(n) ≤ cn
T(n) ≤ (c+1)n
2. Recurrence solution methods
25
• Substitution Method Skill - Avoid pitfalls example
nncnT )2/(2)(
)(
)1(2
2
nO
nc
nn
c
Recurrence :
Guess : T(n)= O(n)Prove : T(n) ≤ cn
nnTnT )2/(2)(
( Wrong !!! exact form :T(n) ≤ cn)
2. Recurrence solution methods
26
• Substitution Method Skill- Simplify this recurrence by changing variable
nnTnT lg)(2)(
mTT mm )2(2)2( 2/
mmSmS )2/(2)(
Renaming m = lg n
S(m) = T(2m)
This new recurrence has same solution : S(m) = O(m log m)
2. Recurrence solution methods
27
• Substitution Method Skill- Simplify this recurrence by changing variable
mmSmS )2/(2)(
This new recurrence has same solution : S(m) = O(m log m) Changing back from S(m) to T(n)
T(n) = T(2m) = S(m)= O(m lg m)= O( lg n lg(lg n) )
2. Recurrence solution methods
28
• Substitution Method Skill - Making a good guess
prove loose upper and lower bounds on the recurrence
and then reduce the range of uncertainty.
- Guessing a solution takes : experience, occasionally, creativity
We can make a good guess by using recursion tree !!We can make a good guess by using recursion tree !!
2. Recurrence solution methods• Recursion-tree - Definition
For come up with good guess,convert the recurrence into a tree that each node represents the cost of a single
subproblem somewhere in the set of recursive function
invocations.
We sum all the per-level costs to determine the total cost of all levels of the recursion
29
2. Recurrence solution methods• Recursion-tree Example
30
The recurrence isT (n) = 3T (⌊n/4 ) + Θ(⌋ n2)
= 3T (n/4) + cn2
Use the recursion tree to making a good guess for upper bound for the solution.
The recurrence isT (n) = 3T (⌊n/4 ) + Θ(⌋ n2)
= 3T (n/4) + cn2
Use the recursion tree to making a good guess for upper bound for the solution.
2. Recurrence solution methods• Recursion-tree Solution to example Draw recursion tree
31
cn2
)4
(n
T
T(n)
)4
(n
T )4
(n
T
)16
(n
T )16
(n
T )16
(n
T
2)4
(n
c
)16
(n
T )16
(n
T )16
(n
T
2)4
(n
c
)16
(n
T )16
(n
T )16
(n
T
2)4
(n
c
cn2
3T (n/4) + cn2
2. Recurrence solution methods• Recursion-tree Solution to example
32
cn2
c(n/4)2 c(n/4)2 c(n/4)2
2)16
(n
c 2)16
(n
c 2)16
(n
c 2)16
(n
c 2)16
(n
c 2)16
(n
c 2)16
(n
c 2)16
(n
c 2)16
(n
c
T(1)T(1)T(1)T(1)T(1)T(1)T(1)T(1) T(1)T(1)T(1)T(1)…3log4n
log4n
When depth i then subprogram size is n/4i = 1 so, i = log4 n and it means tree’s height and tree has log4 n + 1 levels. The number of nodes at depth i is 3i and finally
When depth i then subprogram size is n/4i = 1 so, i = log4 n and it means tree’s height and tree has log4 n + 1 levels. The number of nodes at depth i is 3i and finally 3loglog 443 nn
2. Recurrence solution methods
33
• Recursion-tree Solution to exampleTotal Cost
)(1)16/3(
1)16/3(
)()16
3(
)()16
3(...)
16
3(
16
3)(
3log2log
3log21log
0
3log21log2222
4
4
4
4
44
ncn
ncn
ncncncncnnT
n
in
i
n
Sum of cost for root to i-1 depth
Sum of cost for i depth
)(
)(13
16
)()16/3(1
1
)()16
3(
)()16
3()(
2
3log2
3log2
3log2
0
3log21log
0
4
4
4
4
4
nO
ncn
ncn
ncn
ncnnT
i
i
in
i
xx
k
i
1
1
0
2. Recurrence solution methods
34
• Recursion-tree Solution to exampleTotal Cost is…
( When 0<x<1 then, )
2. Recurrence solution methods
35
• Recursion-tree Solution to exampleProve T (n) = O(n2) by the substitution method
Recurrence :
Guess : T (n) = O(n2) Prove : T (n) ≤ dn2 (for some d > 0 and for the same c > 0)
T (n) = 3T (⌊n/4 ) + Θ(⌋ n2) = 3T(⌊n/4 ) + ⌋ cn2
2. Recurrence solution methods
36
Recursion-tree Solution to exampleFind constant d
T(n) ≤ 3T(⌊n/4 ) + ⌋ cn2
≤ 3d⌊n/4⌋2 + cn2
≤ 3d(n/4)2 + cn2 = 3/16 dn2 + cn2
≤ dn2 (where the last step holds as long as d ≥ (16/13)c )
T (n) = 3T(⌊n/4 ) + ⌋ cn2
2. Recurrence solution methods• Recursion-tree Example
37
The recurrenceT (n) = 3T (n/3)+T(2n/3)+O(n)
Use the recursion tree to making a good guess for upper
bound for the solution.
The recurrenceT (n) = 3T (n/3)+T(2n/3)+O(n)
Use the recursion tree to making a good guess for upper
bound for the solution.
2. Recurrence solution methods• Recursion-tree Solution to example Draw recursion tree
38
)9
(n
c )9
2(
nc
)3
(n
c
)9
2(
nc )
9
4(
nc
)3
2(
nc
cn
log3/2n
cn
cn
cn
Total : O(n log n)
T (n) = 3T (n/3)+T(2n/3)+ cn
This recursion tree is not a complete binary tree. Not all leaves contribute a cost of exactly cn
2. Recurrence solution methods• Recursion-tree Solution to example Draw recursion tree
39
The total cost = Cost of each level x Height = cn x log3/2 n
= O(cnlog3/2n) = O(n lg n)
The longest path from root to a leaf is n→(2/3)n → (2/3)2n → … → 1. Since, (2/3)kn = 1 when k=log3/2 n and it means tree’s height is log3/2 n .
The longest path from root to a leaf is n→(2/3)n → (2/3)2n → … → 1. Since, (2/3)kn = 1 when k=log3/2 n and it means tree’s height is log3/2 n .
T (n) = 3T (n/3)+T(2n/3)+ cn
)9
(n
c )9
2(
nc
)3
(n
c
)9
2(
nc )
9
4(
nc
)3
2(
nc
cn
log3/2n
cn
cn
cn
2. Recurrence solution methods• Recursion-tree Solution to example Prove T (n) = O(n lg n) by the substitution
method
40
Recurrence :Guess : T (n) = O(n lg n)Prove : T (n) ≤ d n lg n (for some d > 0 and for the same c > 0)
T(n/3) + T(2n/3) + cn
2. Recurrence solution methods
41
Recursion-tree Solution to exampleFind constant d T(n) ≤ T(n/3) + T(2n/3) + cn
≤ d(n/3)lg(n/3) + d(2n/3)lg(2n/3) + cn = (d(n/3)lgn - d(n/3)lg 3) + (d(2n/3)lgn – d(2n/3)lg(3/2)) + cn = dnlgn - d((n/3)lg3 + (2n/3)lg(3/2)) + cn = dnlgn - d((n/3)lg3 + (2n/3)lg(3/2)) + cn = dnlgn - d((n/3)lg3 + (2n/3)lg3 - (2n/3)lg2) + cn = dnlgn - dn(lg3 - 2/3) + cn
≤ dnlgn (as long as d ≥ c/(lg 3 - (2/3)) )
T(n)=T(n/3) + T(2n/3) + cn
Content of next week
Recurrence solution methodsThe substitution method
The recursion-tree method
The master method
42
Ch4. Recurrence