8/6/2019 Lecture 2 Rec Rel

1/23

MS 101: Algorithms

Instructor

Neelima Guptangupta@cs.du.ac.in

8/6/2019 Lecture 2 Rec Rel

2/23

Table Of Contents

Solving Recurrences

The MasterTheorem

8/6/2019 Lecture 2 Rec Rel

3/23

Recurrence Relations

8/6/2019 Lecture 2 Rec Rel

4/23

Recurrences

The expression:

is a recurrence.

Recurrence: an equation that describes a function in

terms of its value on smaller functions

"

!

!

12

2

1

)(

ncnn

T

nc

nT

8/6/2019 Lecture 2 Rec Rel

5/23

Recurrence Examples

"

!

!

0

0

)1(

0)(

n

n

nscns

"

!!

0)1(

00)(

nnsn

nns

1

22

1

)(

ncn

nc

n

1

1

)(

ncnb

na

ncn

8/6/2019 Lecture 2 Rec Rel

6/23

Solving Recurrences

Substitution method

Iteration method

Master method

8/6/2019 Lecture 2 Rec Rel

7/23

Solving Recurrences

The substitution method (CLR 4.1)

Making a good guess method

Guess the form of the answer, then useinduction to find the constants and show that

solution works

Examples:

T(n) = 2T(n/2) + 5(n) T(n) = 5(n lg n) T(n) = 2T( n/2) + n ???

8/6/2019 Lecture 2 Rec Rel

8/23

Solving Recurrences

The substitution method (CLR 4.1)

Making a good guess method

Guess the form of the answer, then use

induction to find the constants and show thatsolution works

Examples: T(n) = 2T(n/2) + 5(n) T(n) = 5(n lg n)

T(n) = 2T( n/2) + n T(n) = 5(n lg n)

T(n) = 2T( n/2 )+ 17) + n ???

8/6/2019 Lecture 2 Rec Rel

9/23

Substitution method

Guess the form of the solution .

Use mathematical induction to find the constants and

show that the solution works .

The substitution method can beused to establish eitherupperor lower bounds on a recurrence.

8/6/2019 Lecture 2 Rec Rel

10/23

An example (Substitution method )

T(n) = 2T(floor(n/2) ) +n

We guess that the solution is T(n)=0(n lg n).

i.e. to show that T(n) cn lg n , for some constant c> 0 and n m.

Assume that this bound holds for [n/2]. So , we getT(n) 2(c floor (n/2) lg(floor(n/2))) + n

cn lg(n/2) + n

= cn lg n cn lg 2 + n

= cn lg n cn + n

cn lg n

where , the last step holds as long as c 1.

8/6/2019 Lecture 2 Rec Rel

11/23

Boundary conditions :

Suppose , T(1)=1 is the sole boundary condition of therecurrence .

then , for n=1 , the bound T(n) c n lg n yields

T(1) c lg1=0 , which is at odds with T(1)=1.

Thus ,the base case of our inductiveproof fails to hold.

To overcome this difficulty , we can takeadvantage of theasymptotic notation which only requires us to prove

T(n)c n lg n for n m.

The idea is to remove the difficult boundary condition T(1)= 1

from consideration.Thus , we can replace T(1) by T(2) as the base cases in theinductiveproof , letting m=2.

8/6/2019 Lecture 2 Rec Rel

12/23

Contd....

From the recurrence , with T(1) = 1, we get

T(2)=4

We require T(2) c 2 lg 2

It is clear that , any choice of c2 suffices for

the base cases

8/6/2019 Lecture 2 Rec Rel

13/23

Are we done?

Have weproved the case for n =3.

Have weproved that T(3) c 3 lg 3.

No. Since floor(3/2) = 1 and for n =1, it does not hold. Soinduction does notapply on n= 3.

From the recurrence, with T(1) = 1, we get T(3) = 5.

The inductiveproof that T(n) c n lg n for some constant c1

can now be completed by choosing c largeenough that

T(3)c 3 lg 3 also holds.

It is clear that , any choice of c 2 is sufficient for this to hold.

Thus we can conclude that T(n) c n lg n forany c 2 and n 2.

8/6/2019 Lecture 2 Rec Rel

14/23

What if we have extra lower order

terms?

8/6/2019 Lecture 2 Rec Rel

15/23

Wrong Application of induction

8/6/2019 Lecture 2 Rec Rel

16/23

Solving Recurrences

Another option is iteration method

Expand the recurrence

W

ork some algebra to express as asummation

Evaluate the summation

We will show some examples

8/6/2019 Lecture 2 Rec Rel

17/23

Solve the following recurrence:

T(n) = T(n) + T(n) + n,

where 0 < < 1

Assume suitable initial conditions.

Assignment 4

8/6/2019 Lecture 2 Rec Rel

18/23

The MasterTheorem

Given: a divide andconqueralgorithm

An algorithm that divides the problem of size

n into a subproblems, each of size n/b

Let the cost of each stage (i.e., the work to

divide the problem + combine solved

subproblems) be described by the function

f(n) Then, the MasterTheorem gives us a

cookbook for the algorithms running time:

8/6/2019 Lecture 2 Rec Rel

19/23

The MasterTheorem

if T(n) = aT(n/b) + f(n) then

"

;!

5!

!

5

5

5

!

1

0

largefor)()/(

AND)(

)(

)(

)(

log)(

log

log

log

log

log

c

nncfbnaf

nnf

nnf

nOnf

nf

nn

n

nT

a

a

a

a

a

b

b

b

b

b

I

I

I

8/6/2019 Lecture 2 Rec Rel

20/23

Using The Master Method

T(n) = 9T(n/3) + n

a=9, b=3, f(n) = n

n

logb a

= n

log3 9

=5

(n

2

) Since f(n) = O(nlog3 9 - I), where I=1, case 1

applies:

Thus the solution is T(n) = 5(n2)

I

!5!

aa bb

nOnfnnloglog

)(hen)(

8/6/2019 Lecture 2 Rec Rel

21/23

More Examples of Masters

Theorem T(n) = 3T(n/5) + n

T(n) = 2T(n/2) + n

T(n) = 2T(n/2) + 1 T(n) = T(n/2) + n

T(n) = T(n/2) + 1

8/6/2019 Lecture 2 Rec Rel

22/23

When Masters Theorem cannot

be applied T(n) = 2T(n/2) + n logn T(n) = 2T(n/2) + n/ logn

8/6/2019 Lecture 2 Rec Rel

23/23

Up Next

Proving the correctness of Algorithms