Date post: | 19-Jan-2016 |
Category: |
Documents |
Upload: | ethel-booker |
View: | 241 times |
Download: | 0 times |
Recurrence Relations
OutlineRecurrence relations
Solving recurrence relations
Recurrence and Divide-and-conquer algorithms
Generating functions
2301233 Recurrence Relation 2
RECURRENCE RELATIONS
2301233 Recurrence Relation 3
What is a recurrence relation?• A recurrence relation for the sequence {an} is an
equation that expresses an in terms of one or more of the previous terms of the sequence, namely a0, a1,…, an-1, for all integers n n0, where n0 is a non-negative integer.
• A sequence is called a solution of a recurrence relation if its terms satisfy the recurrence relation.
• Example: Given the recurrence relation an=2an-1 -an-2.• 3, 5, 7, 9, 11, … satisfies the recurrence relation.• 2, 3, 4, 5, 6, … also satisfies the recurrence relation.
2301233 Recurrence Relation 4
Compound interest : ExampleLet Pn be the amount of money in the account after n years. If you initially deposit P0=1000 in your account and the bank gives k % per year with interest compounded annually.
Pn = (1+k/100) Pn-1
P1 = (1+k/100) P0 = (1+k/100) P0
P2 = (1+k/100) P1 = (1+k/100)2 P0
P3 = (1+k/100) P2 = (1+k/100)3 P0
Pn = (1+k/100) Pn-1 = (1+k/100)n P0
2301233 Recurrence Relation 5
Binary strings with no “00” : Example
2301233 Recurrence Relation 6
Strings with length n
Starting with 0
Starting with 00 Starting with 01
Starting with 1
00000 00001 00010 00011 00100 00101 00110 00111
01000 01001 01010 01011 01100 01101 01110 01111
10000 10001 10010 10011 10100 10101 10110 10111
11000 11001 11010 11011 11100 11101 11110 11111
All strings of length n-1 with no 00
All strings of length n-2 with no 00none
Binary strings with no “00” : ExampleLet an be the number of n-bit binary strings containing no “00”.
an = an-1 + an-2
a0 = 1, a1 = 2, a2 = 32301233 Recurrence Relation 7
Strings with length n
Starting with 0
Starting with 00 Starting with 01
Starting with 1
All strings of length n-1 with no 00
All strings of length n-2 with no 00none
Rabbit breeding & Fibonacci Numbers • A pair of rabbits is placed on an island. After 2 months
old, a pair of rabbits produces another pair each month. How many rabbits are there after n months?
month 1-month rabbits >=2-month rabbits Total (fn)
1 1 0 12 0 1 13 1 1 24 1 2 35 2 3 56 3 5 87 5 8 138 8 13 21
2301233 Recurrence Relation 8
fn =fn-1 + fn-2
Tower of Hanoi
2301233 Recurrence Relation 9
Let H(n) be the number of move to solve tower of Hanoi with n discs.
H(n) = 2H(n-1) + 1 H(1) = 1
SOLVING RECURRENCE RELATIONS
2301233 Recurrence Relation 10
Verify solutions of recurrence relations
2301233 Recurrence Relation 11
an = (-4)n
an = 1
an = 3n
an = 2n + 1an = 4an-2 – 3 an-1
(-4)n = 4(-4)n-2 – 3(-4)n-1 ?
4(-4)n-2+12(-4)n-2 = 16(-4)n-2
4(-4)n-2+12(-4)n-2 = (-4)n
3n = 4(3(n-2)) – 3 (3(n-1)) ?
12n – 24 – 9n +9 = 3n – 15
3n
1= 4 (1) – 3(1) ?
1= 1
2n+1=4(2(n-2)+1) –3(2(n-1)+1)?
8n – 12 – 6n +3 = 2n – 9
2n +1
If no initial condition is given, there can be more than one
solution of a recurrence relation.
Types of recurrence relations
2301233 Recurrence Relation 12
k, the number of terms referred back
• Order or degree
Function g
• linear or non-linear• The coefficients of terms are constants or variables
Function F
• some constant (Homogeneous)• some function of n (Non-homogeneous)
an= g(an-1 , an-2 ,…,an-k ) + F(n)
Example: Recurrence relations
• linear, homogeneous, degree 1, with constant coefficient
an = 1.2 an-1
• linear, homogeneous, degree 3, with constant coefficient
an = 3an-3
• Non-linear, homogeneous, degree 2, with constant coefficient
an = a2n-1 + an-2
• Linear, homogeneous, degree 2, with variable coefficient
an = nan-1 - 2an-2
2301233 Recurrence Relation 13
• Linear, non-homogeneous, degree 1, with constant coefficient
an = an-1 + 2n
• Linear, homogeneous, degree 1, with constant coefficient
hn = 2hn-1 + 1
• Non-linear, non-homogeneous, degree 1, constant coefficient
an = a2n-1 + 2n
• Linear, non-homogeneous, degree 1, variable coefficient
an = n2 an-1 + n
How to solve recurrence relations
Guess and verify
Unfold
Variable transformation
Using formula
Generating function
2301233 Recurrence Relation 14
Guess and verifyFind the closed form formula of an = 2an-1 +1 for n >0, a0=0
< a0 , a1, a2, a3 , a4, a5,…> = <0, 1, 3, 7, 15, 31, …>
Guess: an = 2n -1, for n 0.
Verify: • a0 = 20 -1 = 0
• an = 2an-1 +1 = 2(2n-1 -1) +1 = 2n - 2 +1 = 2n – 1
So, our guess is correct. • an = 2n -1, for n 0. 2301233 Recurrence Relation 15
Difficult to make a correct guess
Guess and VerifyFind the closed form formula of an = an-1 +n for n >0, a0=1
a0 = 1, a1= 1+1, a2 = 1+1+2, a3 =1+1+2+3, a4 = 1+1+2+3+4, a5 = 1+1+2+3+4+5
Guess: an = 1+ 0in
i , for n 0.
an = 1+n(n+1)/2 = (n2+n+2)/2Verify: a0 = (02+0+2)/2 = 1
an = an-1+n =[(n-1)2+(n-1)+2]/2 +n = (n2+n+2)/2
So, our guess is correct. • an = (n2+n+2)/2 , for n 0. 2301233 Recurrence Relation 16
UnfoldFind the closed form formula of an = an-1 +n for n >0, a0=1
an = an-1 +n
= (an-2 + n-1)+n
= ((an-3 + n-2) + n-1)+n
…
= ((…(a0+1)… + n-2) + n-1)+n
= 1+1… + n-2 + n-1+n
=1+n(n+1)/2
= (n2+n+2)/22301233 Recurrence Relation 17
Recurrence in Tower of HanoiH(1) = 1
H(n) = 2H(n-1) + 1
= 2(2H(n-2) +1) +1
= 4H(n-2) + 2 + 1
= 4(2(H(n-3) + 1) + 2 + 1
= 8H(n-3) + 4 + 2 + 1
= 8(2H(n-4) +1) + 4 + 2 + 1
= 16H(n-4) + 8 + 4 + 2 + 1
= 2n-1 + 2n-2 + 2n-3 + … + 1
= 2n - 1
Let x = 2n-1 + 2n-2 + … + 4 + 2 + 1
x + 1=(2n-1 + 2n-2 + … + 4 + 2 + 1)+1
x + 1 = 2n-1 + 2n-2 + … + 4 + 2 + 2
x + 1 = 2n-1 + 2n-2 + … + 4 + 4
x + 1 = 2n-1 + 2n-2 + … + 8
x + 1 = 2n
x = 2n - 1
2301233 Recurrence Relation 18
Variable TransformationFind the closed form formula of a(n) = a(n/2) +1 for n =2k, k 0 a1=1
Let b(k) = a(2k) .From a(1)=1 a(n) = a(n/2) +1
We get b(0)=1 b(k) = b(k-1) +1
The closed form formula of b(k) is 1+ k.
b(k) = 1+k = 1+ log2 n= a(n).
Thus, a(n) = 1+ log2 n.2301233 Recurrence Relation 19
Variable TransformationFind the closed form formula of an = nan-1 +n! for n >0, a0=2.
Divide the recurrence relation by n!, and we have an/n! = an-1/(n-1)! + 1.
Let bn = an/n!.
Then, b0=2 bn = bn-1 +1.
The closed form formula of bn is 2+ n.
bn = an/n! = 2+n .
Thus, an = n!(2 + n).2301233 Recurrence Relation 20
Variable TransformationFind the closed form formula of an = (an-1)2/an-2 for n >1, a0=1, a1=2.
Divide the recurrence relation by an-1, and we have an/an-1 = an-1/an-2.
Let bn = an/an-1.
Then, bn = bn-1 b1 = a1/a0 =2.
The closed form formula of bn is 2.
bn = an/an-1 = 2.
Thus, an = 2an-1, and an = 2n.2301233 Recurrence Relation 21
Using formulaLinear homogeneous recurrence relation
• The sequence {an} is a solution of the recurrence relation an = c1 an-1 + c2 an-2 iff an = 1 rn
1 + 2 rn2 ,
where :– r1 and r2 are two distinct roots of r2 - c1r - c2 = 0,
– a0 = 1 + 2 and a1 = 1 r1 + 2 r2
(or
– 1 = (a1 – a0 r2)/(r1 - r2 ) and 2 = (a0 r1 –a1)/(r1 - r2 ) )
2301233 Recurrence Relation 22
Where the formula comes from
Let r1 and r2 be two distinct roots of r2 – c1r – c2= 0.
Let and be constants.
Since r1 and r2 are roots of r2 - c1r - c2= 0,
r12 - c1r1 - c2 = 0 (r1
2 = c1r1 + c2)
and r22 - c1r2 - c2= 0 (r2
2 = c1r2 + c2).
c1an-1 + c2an-2 = c1(1r1n-1+2r2
n-1) + c2(1r1n-2+2r2
n-2)
= c11r1n-1+ c1 2r2
n-1 + c21r1n-2+ c2 2r2
n-2
= c11r1n-1+ c21r1
n-2 +c1 2r2n-1 + c2 2r2
n-2
= 1r1n-2 (c1r1 + c2) + 2r2
n-2 (c1r2 + c2)
= 1r1n-2 r1
2 + 2r2n-2 r2
2 = 1 r1n + 2 r2
n = an2301233 Recurrence Relation 23
Using formulaLinear homogeneous recurrence relation
The sequence {an} is a solution of the recurrence relation an = c1an-1 + c2an-2 +…+ ckan-k iff an = 1rn
1 + 2rn
2 +…+ k rnk , where :
– r1, r2 ,…, rk are k distinct roots of rk -c1rk-1 - c2rk-2 -…- ck= 0,
– a0 = 1 + 2 +…+ k
– a1 = 1 r1 + 2 r2 +…+ k rk
– a2 = 1 r21 + 2 r2
2 +…+ k r2k
– …
2301233 Recurrence Relation 24
ExampleFind the solution of an = an-1 + an-2 with a0=0 a1=1 ?
We have c1 = 1 and c2 = 1.
The characteristic equation is r2 - r - 1 = 0.
The roots r1= (1+5)/2 and r2= (1-5)/2 .
From a0 = 1 + 2 and a1 = 1 r1 + 2 r2
0 = 1 + 2 and 1 = 1 r1 + 2 r2
1 = 1 r1 - 1 r2 = 1(r1 - r2) = 15 1 = 1/5
2= - 1 = -1/5 2= -1/5
an = 1 rn1 + 2 rn
2 = ((1+5)/2)n /5 - ((1-5)/2)n /5
an = ( (1+5)n - (1-5)n )/(52n )2301233 Recurrence Relation 25
ExampleWhat is the solution of an = an-1 + 2an-2
with a0=2, a1=7 ?
We have c1 = 1 and c2 = 2.
The characteristic equation is r2 - r - 2 = 0.
The roots r1= 2 and r2= -1 .
1 = (a1 – a0r2 )/(r1 - r2 ) = (7+2)/(2+1) = 3
2 = (a0r1 – a1)/(r1- r2 ) = (4-7)/(2+1) = -1
an = 1 rn1 + 2 rn
2 = 32n - (-1)n
2301233 Recurrence Relation 26
ExampleWhat is the solution of an=6an-1-11an-2+6an-3
with a0=2, a1=5, a2=15 ?
The characteristic equation is r3 - 6r2 + 11r - 6 = 0, with roots r1= 1, r2= 2 and r3= 3 .
a0 = 1 + 2 + 3
a1 = 1 r1 + 2 r2 + 3 r3 = 1 + 22 + 33
a2 = 1 r12 + 2 r2
2 + 3 r32 = 1 + 42 + 93
1 = 1, 2 = -1, 3 = 2
an = 1 – 2n + 23n
2301233 Recurrence Relation 27
Using formulaLinear homogeneous recurrence relation
The sequence {an} is a solution of the recurrence relation an = c1an-1 + c2an-2 +…+ ckan-k iff
an = (1,0 + 1,1n + …+ 1,m1nm1-1
) rn1 +
(2,0 + 2,1n + …+ 2,m2nm2-1
) rn2 +…+
(t,0 + t,1n + …+ t,mtnmt-1
) rnt , where :
r1, r2 ,…, rt are distinct roots of rk -c1rk-1 - c2rk-2 -…- ck= 0, with multiplicities m1, m2 ,…, mt
2301233 Recurrence Relation 28
ExampleWhat is the solution of an= 6an-1 -9an-2
with a0=1, a1=6 ?
We have c1 = 6 and c2 = -9.
The characteristic equation is r2 - 6r + 9 = 0.
The root r0= 3.
1 = a0 = 1
2 = a1/r0 – a0 = 6/3-1 = 1
an = 1rn0 + 2nrn
0 = 3n + n 3n = 3n (1+ n).
2301233 Recurrence Relation 29
Using formulaLinear non-homogeneous recurrence relation
If the sequence {an(p)} is a particular solution of
the recurrence relation an = c1 an-1 + c2 an-2 +…+ ck an-k + F(n) then every solution is of the form
{an(p) + an
(h)} , where {an(h)} is a solution of the
associated homogeneous recurrence relation an = c1 an-1 + c2 an-2 +…+ ck an-k.
2301233 Recurrence Relation 30
ExampleWhat are the solutions of an= 3an-1+ 2n with a1=3 ?
We have c1 = 3.
The associated characteristic eqn is r-3=0, with root r = 3.
Then, an(h) = 3n .
Let an(p) = cn +d.
Then, from an= 3an-1+ 2n,
cn +d = 3(c(n-1)+d ) + 2n.
= (3cn + 2n) - 3c + 3d
= (3c + 2)n - 3c + 3d
c = 3c + 2, d = -3c + 3d .
Thus, c = -1 and d = -3/2. That is, an
(p) = -n -3/2
an= an(h) + an
(p) = 3n -n -3/2.
From a1= 3 = 3 -1 - 3/2,
= 11/6.
Solution:an= (11/6)3n -n -3/2.
2301233 Recurrence Relation 31
TheoremLet {an} be a solution of the linear non-homogeneous recurrence relation
an = c1an-1 + c2an-2 +…+ ckan-k + F(n),
where c1, c2, …, ck are real numbers, and
F(n) = (btnt + bt-1nt -1+…+ b1n + b0) sn,
where b0, b1,…, bt and s are real numbers.
When s is not a root of the characteristic equation of the associated linear homogeneous recurrence relation, there is a particular solution of the form
(ptnt + pt-1nt -1+…+ p1n + p0) sn.2301233 Recurrence Relation 32
TheoremLet {an} be a solution of the linear non-homogeneous recurrence relation
an = c1an-1 + c2an-2 +…+ ckan-k + F(n),
where c1, c2, …, ck are real numbers, and
F(n) = (btnt + bt-1nt -1+…+ b1n + b0) sn,
where b0, b1,…, bt and s are real numbers.
When s is a root of the characteristic equation with multiplicity m, there is a particular solution of the form
nm (ptnt + pt-1nt -1+…+ p1n + p0) sn.
2301233 Recurrence Relation 33
Example: SummationWhat are the solutions of an= an-1+ n with a1=1 ?
We have c1 = 1.
The associated characteristic eqn is r-1=0, with root r = 1.
Then, an(h) = 1n = .
Let an(p) = n(cn +d) = cn2 +dn.
Then, from an= an-1+ n, cn2 +dn = c(n-1)2+d (n-1) + n.
Thus, cn2 +dn = cn2-2cn+c +d n - d + n , c-d + n(1-2c)=0.
c-d =0 and 1-2c=0. That is, c=d=1/2.
an= an(h) + an
(p) = + n(n+1)/2. From a1= 1 = +1, = 0.
The solution is an= n(n+1)/2.2301233 Recurrence Relation 34
DIVIDE-AND-CONQUER ALGORITHMSRECURRENCE AND
2301233 Recurrence Relation 35
Divide-and-conquer and Recurrence
Let f(n) be the number of operations required to solve a problem of size n.
f (n) = a f (n/b) + g(n)
2301233 Recurrence Relation 36
XX1 X2 X3 Xa...n/b n/b n/b n/b
n elements
Binary Search• To search for an element a from a sorted list X of
size n elements.
– If a=m, then stop– If a<m, then search for a from X1.
– If a>m, then search for a from X2.
Let f (n) be the number of comparisons in binary search within n elements.
f (n) = f ( n/2 ) + 2.2301233 Recurrence Relation 37
XX1 X2m
Master MethodLet f be an eventually non-decreasing function that satisfies the recurrence relation
f (n) = a f (n/b) + cnd
whenever n n0, a 1, b > 1, c > 0, d 0, n=n0bk for any integer k > 0.
Let n=n0bk . Then, f (n0bk) = a f (n0bk-1) + cn0dbkd.
Let hi = f (n0bi ) . Then, hi = a hi-1 + c (bd)i, where c =cn0d.
hi( h) = ai
When abd : hi( p) = p bdi
When a=bd : hi( p) = pi ai
2301233 Recurrence Relation 38
Master MethodWhen abd : hi
( p) = p bdi
p bdi = a p bd(i-1) + c (bd)i
p = a p/bdi + c
p (1 – a/bd ) = c p = c /(1 – a/bd )
hi = hi( h) + hi
( p)
= ai + c /(1 – a/bd ) bdi
f (n) = f (n0bi ) = hi (i=logb n/n0)
= ai + c /(1 – a/bd ) bdi
(ai = (n/n0)logb a , bdi = (n/n0)d )
= (n/n0)logb a+c /(1–a/bd ) (n/n0)d
= c1 nlogb a + (c/(1–a/bd )) nd
When a=bd : hi( p) = pi ai
pi ai = a p(i-1) a(i-1) + c ai
p ai = c ai
p = c = cn0d.
hi = hi( h) + hi
( p)
= ai + c i ai
f (n) = f (n0bi ) = hi (i=logb n/n0)
= ( + c i)ai
(ai = bd logb n/n0 = (n/n0)d)
= ( + c i )(n/n0)d
= c1 nd + cnd logb (n/n0) 2301233 Recurrence Relation 39
Master MethodLet f be an eventually non-decreasing function that satisfies the recurrence relation
f (n) = a f (n/b) + cnd
whenever n n0, a 1, b > 1, c > 0, d 0, n = n0bk for any positive integer k.
When a=bd, f(n) c1nd logbn.
When a>bd, f(n) c2nlogba
When a<bd, f(n) c3nd.2301233 Recurrence Relation 40
Binary Search• If a=m, then stop• If a<m, then search for a from X1.
• If a>m, then search for a from X2.
Let f(n) be the # of comparison in binary search within n elements.
f(n) = f(n/2) + 2.
Because a=1, f(n) k log n.
2301233 Recurrence Relation 41
Merge sortprocedure msort(L: list of a1, a2, …, an)if n>1 then { m := n/2 L1 := list of a1, a2, …, am
L2 := list of am+1, am+2, …, an
merge(msort(L1), msort(L2))}
f(n) = 2f(n/2) + n. a = 2, b = 2, c = 1, d = 1
Since a=bd, f(n) kn log n.
procedure merge(A, B: list of n
elements)i := 1 j := 1 k := 1while i<n & j<n{ if A[i]<B[j] then C[k++] := A[i++]
else C[k++] := B[j++]
}
2301233 Recurrence Relation 42
Fast Multiplication for Large Integers a = 2n A1 + A0. b = 2n B1 + B0.
A1 A0 B1 B0
a b = (22n + 2n) A1B1 + 2n(A1-A0)(B0 – B1) + (2n+1) A0 B0.
Let f (n) be the number of operations needed to multiply two n-bit integers.
f (2n) = 3f(n) + Cn.a = 3, b = 2, c = C, d = 1. Because a>bd, f(n) k nlog23.
2301233 Recurrence Relation 43
10010 00011 10000 10011
GENERATING FUNCTION
2301233 Recurrence Relation 44
Generating function: Definition• The generating function for the sequence
a0, a1, …, ak, … of real numbers is the infinite series
G(x) = a0 + a1x + … + akxk + … =
k =0 ak xk
Examples:• The generating function for 1, 2, 4, ,…2kxk, … is 1
+ 2x + 4x2 + … + 2kxk + …• The generating function for 1, -1, 1, ,…(-1)k xk, …
is 1 - x + x2 - … + x2k - x2k+1 +…2301233 Recurrence Relation 45
Power Series
Let f (x) =
k =0 ak xk and g(x) =
k =0 bk xk.
Then, f (x) + g(x) =
k =0 (ak +bk)xk and
f (x) g(x) =
k =0(k
j=0 aj bk-j) xk .
2301233 Recurrence Relation 46
Extended Binomial TheoremLet u be a real number and k be a nonnegative integer.
The extended binomial coefficient C(u, k) or ( u k ) is
u(u-1)…(u-k+1)/k! if k > 0. 1 if k = 0.
Let x be a real number with | x |<1 and u be a real number.
Then, (1+x)u = xn( u
k ) xk.
2301233 Recurrence Relation 47
Useful Generating Functions
G(x) ak
(1 + x)n = 1 + C(n,1)x + C(n,2)x2 + …+ xn . C(n, k)
(1 + ax)n = 1 + C(n,1)ax + C(n,2)a2x2 + …+ anxn C(n, k) ak
(1 + xr)n = 1 + C(n,1)xr + C(n,2)x2r + …+ xnr C(n, k/r) if r | k0 otherwise
1/(1-x) =1 + x + x2 + … . 1
1/(1-ax) =1 + ax + a2x2 + … . ak
(1-xn+1)/(1-x) =1 + x + x2 + …+ xn . 1 if k n0 otherwise
2301233 Recurrence Relation 48
ExampleSolve the recurrence relation ak= 3ak-1 with a0=2.
Let G(x) = a0 + a1x + a2x2 + a3x3 + … be the generating function for the sequence {ak}.
G(x) = a0 + a1x + a2x2 + a3x3 + …
3x G(x) = 3a0x + 3a1x2 + 3a2x3 + …
(1-3x)G(x) = a0 +(a1-3a0)x+(a2-3a1)x2+(a3-3a2) x3+ ..
(1-3x)G(x) = a0 = 2, and G(x) = 2/(1-3x)
From 1/(1-ax) =1 + ax + a2x2 +…, G(x)/2 = 1/(1-3x).
Thus, a = 3, and ak= 2(3k)-1 2301233 Recurrence Relation 49
ExampleSolve the recurrence relation an= 8an-1 + 10n-1 with a0=1.
Multiply the recurrence relation by xn , and get an xn = 8an-1 xn
+ 10n-1 xn
Let G(x) =
n=0 anxn be the generating function for
the sequence {an}.
G(x) =
n=0 anxn = a0 +
n=1 anxn
G(x) - 1 = 8
n=1 an-1 xn
+
n=1 10n-1 xn2301233 Recurrence Relation 50
Example
G(x) - 1 = 8
n=1 an-1 xn
+
n=1 10n-1 xn
G(x) - 1 = 8x
n=1 an-1 xn-1
+ x
n=1 10n-1 xn-1
G(x) - 1 = 8x
n=0 an xn
+ x
n=0 10n xn
G(x) - 1 = 8x G(x) + x /(1 - 10x)
(1 - 8x) G(x) = 1+ x /(1 - 10x)
(1 - 8x) G(x) = (1 - 9x) /(1 - 10x)
G(x) = (1 - 9x) /((1 - 10x) (1 - 8x))2301233 Recurrence Relation 51
ExampleG(x) = (1 - 9x) /((1 - 10x) (1 - 8x))
Using partial fraction, we get
G(x) = ½(1 /(1 - 10x) + 1/(1 - 8x))
From the known power series,
G(x) = ½(
n=0 10n xn +
n=0 8n xn )
G(x) =
n=0 ½(10n + 8n) xn
Thus, we know that an = ½(10n + 8n) 2301233 Recurrence Relation 52
2301233 Recurrence Relation 53
Aren’t you glad it’s done?