CS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer...

1

CS 4407

Algorithms

Lecture 2:

Iterative and Divide and Conquer

Algorithms

Prof. Gregory Provan

Department of Computer Science

University College Cork

CS 4407, Algorithms University College Cork,

Gregory M. Provan

Lecture Outline

Growth Functions Mathematical specification of growth functions

Iterative Algorithms Divide-and-Conquer Algorithms


Gregory M. Provan

Today’s Learning Objectives

Describe mathematical principles for specifying the

growth of run-time of an algorithm

– Classify the growth functions

• , O, , o,

Iterative Algorithm Analysis

Divide-and-Conquer Algorithm Analysis


Gregory M. Provan

Analyzing Algorithms

Assumptions – generic one-processor, random access machine

– running time (others: memory, communication, etc)

Worst Case Running Time: the longest time for any input

of size n – upper bound on the running time for any input

– in some cases like searching, this is close

Average Case Behavior: the expected performance

averaged over all possible inputs – it is generally better than worst case behavior

– sometimes it’s roughly as bad as worst case


Gregory M. Provan

Notation: Growth Functions

Theta f(n) = θ(g(n)) f(n) ≈ c g(n)

BigOh f(n) = O(g(n)) f(n) ≤ c g(n)

Omega f(n) = Ω(g(n)) f(n) ≥ c g(n)

Little Oh f(n) = o(g(n)) f(n) > c g(n)


Gregory M. Provan

-notation

(g(n)) = {f(n): there exist

positive constants c1, c2 and

n0 such that

0 c1g(n) f(n) c2g(n),

for all n n0 }

For a given function g(n),

we denote by (g(n)) the

set of functions

We say g(n) is an asymptotically tight bound for f(n)


Gregory M. Provan

O-notation

For a given function g(n), we

denote by O(g(n)) the set of

functions

O(g(n)) = {f(n): there exist

positive constants c and n0

such that

0 f(n) cg(n),

for all n n0 }

We say g(n) is an asymptotic upper bound for f(n)


Gregory M. Provan

-notation

For a given function g(n),

we denote by (g(n))

the set of functions

(g(n)) = {f(n): there

exist positive constants

c and n0 such that

0 cg(n) f(n)

for all n n0 }

We say g(n) is an asymptotic lower bound for f(n)


Gregory M. Provan

Relations Between , , O

For any two functions g(n) and f(n),

f(n) = (g(n)) if and only if f(n) = O(g(n)) and

f(n) = (g(n)).

i.e., (g(n)) = O(g(n)) (g(n)).


Gregory M. Provan

Complexity of “Simple” Algorithms

Simple Interation

Dealing with Loops

– Loop invariant method

Divide and Conquer Algorithms


Gregory M. Provan

Iterative Algorithm Running Time

1. n = read input from user

2. sum = 0

3. i = 0

4. while i < n

5. number = read input from user

6. sum = sum + number

7. i = i + 1

8. mean = sum / n

T(n), or the running time of a particular algorithm on input of size n, is taken to be the number of

times the instructions in the algorithm are executed. Pseudo code algorithm illustrates the

calculation of the mean (average) of a set of n numbers:

Statement Number of times executed 1 1

2 1

3 1

4 n+1

5 n

6 n

7 n

8 1

The computing time for this algorithm in terms on input size n is: T(n) = 4n + 5.


Gregory M. Provan

Analysis of Simple Programs (no recursion)

Sum the “costs” of the lines of the program

Compute the bounding function

– e.g., O(*)


Gregory M. Provan

Analysing Loops

Use Loop Invariants

Intuitive notion and example


Gregory M. Provan

Designing an Algorithm Define Problem Define Loop

Invariants

Define Measure of

Progress

Define Step Define Exit Condition Maintain Loop Inv

Make Progress Initial Conditions Ending

km

79 km

to school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit


Gregory M. Provan

Typical Loop Invariant

If the input consists of an array of objects

I have a solution for the first i objects.

Extend the solution into a solution for the first

i+1.

i objects

Exit

79 km 75 km Exit i to i+1

Done when

solution for n


Gregory M. Provan

Typical Loop Invariant

If the output consists of an array of objects

I have an output produced for the first i objects.

Produce the i+1-st output object.

i objects

Exit 79 km 75 km

Exit i to i+1 Done when

output n objects.


Gregory M. Provan

Example of Approach

Binary search

– Standard algorithm

Input

– Sorted list, and key

Goal: find key in list


Gregory M. Provan

Define Problem: Binary Search

PreConditions

– Key 25

– Sorted List

PostConditions

– Find key in list (if there).

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95


Gregory M. Provan

Define Loop Invariant

Maintain a sublist

Such that

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95


Gregory M. Provan

Define Loop Invariant

Maintain a sublist.

If the key is contained in the original list, then

the key is contained in the sublist.

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95


Gregory M. Provan

Define Step

Make Progress

Maintain Loop Invariant

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95


Gregory M. Provan

Define Step

Cut sublist in half.

Determine which half key would be in.

Keep that half.

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95


Gregory M. Provan

Define Step

Cut sublist in half.

Determine which half the key would be in.

Keep that half.

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

If key ≤ mid,

then key is in

left half.

If key > mid,

then key is in

right half.


Gregory M. Provan

Define Step

It is faster not to check if the middle element

is the key.

Simply continue. key 43

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

If key ≤ mid,

then key is in

left half.

If key > mid,

then key is in

right half.


Gregory M. Provan

Make Progress

The size of the list becomes smaller.

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

79 km

75 km

79 km 75 km

Exit


Gregory M. Provan

Initial Conditions km

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

If the key is contained in

the original list,

then the key is contained

in the sublist.

• The sublist is the entire original list.

n km


Gregory M. Provan

Ending Algorithm

If the key is contained in the original list,

then the key is contained in the sublist.

Sublist contains one element.

Exit

Exit

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

0 km

• If the key is contained in the original list,

then the key is at this location.

key 25


Gregory M. Provan

If key not in original list

If the key is contained in

the original list,

then the key is contained

in the sublist.

• Loop invariant true, even if the key is not in the list.

• If the key is contained in the original list,

then the key is at this location.

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

key 24

• Conclusion still solves the problem.

Simply check this one location for the key.


Gregory M. Provan

Running Time

The sublist is of size n, n/2, n/4,

n/8,…,1

Each step (1) time.

Total = (log n)

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

If key ≤ mid,

then key is in

left half.

If key > mid,

then key is in

right half.


Gregory M. Provan

Code


Gregory M. Provan

Divide and Conquer Algorithms

Well-known class of algorithm

Requires special approach for complexity

analysis


Gregory M. Provan

Divide-and-Conquer Analysis

• The analysis of divide and conquer algorithms require us to solve a recurrence.

• Recurrences are a major tool for analysis of algorithms


Gregory M. Provan

MergeSort

T(n/2) T(n/2)

cn

A L G O R I T H M S

divide A L G O R I T H M S


Gregory M. Provan

MergeSort

T(n/2) T(n/2)

cn

A L G O R I T H M S

Divide #1 A L G O R I T H M S

Divide #2 A L G O R I T H M S

T(n/4) T(n/4) T(n/4) T(n/4)


Gregory M. Provan

MergeSort

12

2

1

)(

ncnn

T

nc

nT

(n/2) +c (n/2) +c

cn

Solve T(n) = T(n/2) + T(n/2) + cn

T(n/4) T(n/4) T(n/4) T(n/4)

Recurrence


Gregory M. Provan

Recursion-tree method

• A recursion tree models the costs (time) of a recursive execution of an algorithm.

• The recursion tree method is good for generating guesses for the substitution method.

• The recursion-tree method can be unreliable, just like any method that uses ellipses (…).

• The recursion-tree method promotes intuition, however.


Gregory M. Provan

Example of recursion tree

Solve T(n) = T(n/4) + T(n/2) + n2:


Gregory M. Provan


T(n)

Solve T(n) = T(n/4) + T(n/2) + n2:


Gregory M. Provan


T(n/4) T(n/2)

n2

Solve T(n) = T(n/4) + T(n/2) + n2:


Gregory M. Provan


Solve T(n) = T(n/4) + T(n/2) + n2:

n2

(n/4)2 (n/2)2

T(n/16) T(n/8) T(n/8) T(n/4)


Gregory M. Provan


(n/16)2 (n/8)2 (n/8)2 (n/4)2

(n/4)2 (n/2)2

(1)

Solve T(n) = T(n/4) + T(n/2) + n2:

n2


Gregory M. Provan


Solve T(n) = T(n/4) + T(n/2) + n2:

(n/16)2 (n/8)2 (n/8)2 (n/4)2

(n/4)2 (n/2)2

(1)

2nn2


Gregory M. Provan


Solve T(n) = T(n/4) + T(n/2) + n2:

(n/16)2 (n/8)2 (n/8)2 (n/4)2

(n/4)2 (n/2)2

(1)

2

165 n

2nn2


Gregory M. Provan


Solve T(n) = T(n/4) + T(n/2) + n2:

(n/16)2 (n/8)2 (n/8)2 (n/4)2

(n/4)2

(1)

2

165 n

2n

2

25625 n

n2

(n/2)2

…


Gregory M. Provan


Solve T(n) = T(n/4) + T(n/2) + n2:

(n/16)2 (n/8)2 (n/8)2 (n/4)2

(n/4)2

(1)

2

165 n

2n

2

25625 n

13

1652

165

1652 n

…

Total =

= (n2)

n2

(n/2)2

geometric series


Gregory M. Provan

Solution: geometric series

1

11 2

xxx for |x| < 1

1

11

12

x

xxxx

nn for x 1


Gregory M. Provan

The master method

The master method applies to recurrences of the form

T(n) = a T(n/b) + f (n) ,

where a 1, b > 1, and f is asymptotically positive.


Gregory M. Provan

Idea of master theorem

f (n/b) f (n/b) f (n/b)

(1)

Recursion tree:

…

f (n) a

f (n/b2) f (n/b2) f (n/b2) …

a h = logbn

f (n)

a f (n/b)

a2 f (n/b2)

…

#leaves = ah

= alogbn

= nlogba nlogba (1)


Gregory M. Provan

Three common cases

Compare f (n) with nlogba:

1. f (n) = O(nlogba – ) for some constant > 0.

• f (n) grows polynomially slower than nlogba (by an n factor).

Solution: T(n) = (nlogba) .

# leaves in

recursion tree


Gregory M. Provan


f (n/b) f (n/b) f (n/b)

(1)

Recursion tree:

…

f (n) a

f (n/b2) f (n/b2) f (n/b2) …

a h = logbn

f (n)

a f (n/b)

a2 f (n/b2)

…

nlogba (1) CASE 1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight. (nlogba)


Gregory M. Provan

Three common cases


2. f (n) = (nlogba lgkn) for some constant k 0.

• f (n) and nlogba grow at similar rates.

Solution: T(n) = (nlogba lgk+1n) .


Gregory M. Provan


f (n/b) f (n/b) f (n/b)

(1)

Recursion tree:

…

f (n) a

f (n/b2) f (n/b2) f (n/b2) …

a h = logbn

f (n)

a f (n/b)

a2 f (n/b2)

…

nlogba (1) CASE 2: (k = 0) The weight is approximately the same on each of the logbn levels. (nlogbalg n)


Gregory M. Provan

Three common cases (cont.)


3. f (n) = (nlogba + ) for some constant > 0.

• f (n) grows polynomially faster than nlogba (by an n factor),

and f (n) satisfies the regularity condition that a f (n/b) c f (n) for some constant c < 1.

Solution: T(n) = ( f (n) ) .


Gregory M. Provan


f (n/b) f (n/b) f (n/b)

(1)

Recursion tree:

…

f (n) a

f (n/b2) f (n/b2) f (n/b2) …

a h = logbn

f (n)

a f (n/b)

a2 f (n/b2)

…

nlogba (1) CASE 3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight. ( f (n))


Gregory M. Provan

Examples

Ex. T(n) = 4T(n/2) + n a = 4, b = 2 nlogba = n2; f (n) = n. CASE 1: f (n) = O(n2 – ) for = 1. T(n) = (n2).

Ex. T(n) = 4T(n/2) + n2 a = 4, b = 2 nlogba = n2; f (n) = n2. CASE 2: f (n) = (n2lg0n), that is, k = 0. T(n) = (n2lg n).


Gregory M. Provan

Examples

Ex. T(n) = 4T(n/2) + n3 a = 4, b = 2 nlogba = n2; f (n) = n3. CASE 3: f (n) = (n2 + ) for = 1 and 4(cn/2)3 cn3 (reg. cond.) for c = 1/2. T(n) = (n3).

Ex. T(n) = 4T(n/2) + n2/lg n a = 4, b = 2 nlogba = n2; f (n) = n2/lg n. Master method does not apply. In particular, for every

constant > 0, we have n (lg n).


Gregory M. Provan

Growth Function Summary

Defined the mathematical principles for specifying the run-

time of an algorithm

Classified the growth functions

– , O, , o,

Defined several relationships among the growth functions

Theta f(n) = θ(g(n)) f(n) ≈ c g(n)

BigOh f(n) = O(g(n)) f(n) ≤ c g(n)

Omega f(n) = Ω(g(n)) f(n) ≥ c g(n)

Little Oh f(n) = o(g(n)) f(n) > c g(n)


Gregory M. Provan

Appendix: Review of Growth Functions

Detailed review of different growth

functions


Gregory M. Provan

A Simple Example

INPUT: a sequence of n numbers

OUTPUT: the smallest number among them

1. x T[1]

2. for i 2 to n do

3. if T[i] < x then x T[i]

* Performance of this algorithm is a function of n.


Gregory M. Provan

Runtime Analysis

Elementary operation: an operation whose execution time can be bounded above by a constant depending on implementation used.

Assume all elementary operations can be executed at unit cost. This is not true, but they are within some constant factor of each other. We are mainly concerned about the order of growth.


Gregory M. Provan

Order of Growth

For very large input size, it is the asymptotic rate of

growth (or order of growth) that matters.

We can ignore the lower-order terms, since they are

relatively insignificant for very large n. We can also

ignore leading term’s constant coefficients, since they

are not as important for the rate of growth in

computational efficiency for very large n.

Higher order functions of n are normally considered less

efficient.


Gregory M. Provan

Comparisons of Algorithms

Multiplication – classical technique: O(nm)

– divide-and-conquer: O(nmln1.5) ~ O(nm0.59)

For operands of size 1000, takes 40 & 15 seconds

respectively on a Cyber 835.

Sorting – insertion sort: (n2)

– merge sort: (n lg n)

For 106 numbers, it took 5.56 hrs on a supercomputer

using machine language and 16.67 min on a PC using

C/C++.


Gregory M. Provan

Why Order of Growth Matters

Computer speeds double every two years, so

why worry about algorithm speed?

When speed doubles, what happens to the

amount of work you can do?


Gregory M. Provan

Effect of Faster Machines

• Higher gain with faster hardware for more efficient algorithm.

• Results are more dramatic for higher speeds: for the n lg n

algorithm, doubling the speed almost doubles the number of items

that can be sorted.

No. of items sorted

Ο(n2)

H/W Speed

Comp. of Alg. 1 M* 2 M Gain

1000 1414 1.414

O(n lgn) 62700 118600 1.9

* Million operations

per second.


Gregory M. Provan

Classifying Functions

Functions

Con

stant

Lo

garith

mic

Po

ly L

ogarith

mic

Poly

no

mial

Exp

on

ential

Exp

Dou

ble E

xp

(log n)5 n5 25n 5 log n 5 2 n5 2

5n

2


Gregory M. Provan

Ordering Functions

Functions

Con

stant

Lo

garith

mic

Po

ly L

ogarith

mic

Poly

no

mial

Exp

on

ential

Exp

Dou

ble E

xp

(log n)5 n5 25n 5 log n 5 2 n5 2

5n

2


Gregory M. Provan

Classifying Functions

Functions

Con

stant

Lo

garith

mic

Po

ly L

ogarith

mic

Poly

no

mial

Exp

on

ential

Ex

p

Dou

ble E

xp

(log n)θ(1) nθ(1) 2θ(n) θ(log n) θ(1) 2 nθ(1) 2

θ(n)

2


Gregory M. Provan

Asymptotic Notation

, O, , o,

Used to describe the running times of algorithms

Instead of exact running time, say (n2)

Defined for functions whose domain is the set of

natural numbers, N

Determine sets of functions, in practice used to

compare two functions


Gregory M. Provan

Example

3n2 + 7n + 8= (n2)

What constants for n0, c1, and c2 will work?

Make c1 a little smaller than leading coefficient,

and c2 a little bigger

To compare orders of growth, look at the

leading term


Gregory M. Provan

Definition of Theta

f(n) is sandwiched between c1g(n) and c2g(n)

for some sufficiently small c1 (= 0.0001)

for some sufficiently large c2 (= 1000)

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

f(n) = θ(g(n))


Gregory M. Provan

Definition of Theta

For all sufficiently large n

For some definition of “sufficiently large”

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

f(n) = θ(g(n))

n0


Gregory M. Provan

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)


Gregory M. Provan

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

c1·n2 3n2 + 7n + 8 c2·n

2 ? ?


Gregory M. Provan

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·n2 3n2 + 7n + 8 4·n2 3 4


Gregory M. Provan

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·12 3·12 + 7·1 + 8 4·12 1

False


Gregory M. Provan

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·72 3·72 + 7·7 + 8 4·72 7

False


Gregory M. Provan

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·82 3·82 + 7·8 + 8 4·82 8

True


Gregory M. Provan

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·n2 3n2 + 7n + 8 4·n2 n 8 8


Gregory M. Provan

Running Times

“The running time is O(f(n))”

Worst case is O(f(n))

“Running time is (f(n))” Best case is (f(n))

Can still say “Worst case running time is (f(n))”

– Means worst case running time is given by some

unknown function g(n) (f(n))


Gregory M. Provan

Example

Insertion sort takes (n2) worst case time, so

sorting (as a problem) is O(n2)

Any sort algorithm must look at each item, so

sorting is (n)

In fact, using (e.g.) merge sort, sorting is (n lg n)

in the worst case


Gregory M. Provan

Asymptotic Notation in Equations

Asymptotic notation is used to replace expressions containing lower-order terms.

For example,

4n3 + 3n2 + 2n + 1 = 4n3 + 3n2 + (n)

= 4n3 + (n2) = (n3)

In equations, (f(n)) always stands for an anonymous function g(n) (f(n)).

– In the example above, (n2) stands for 3n2 + 2n + 1.


Gregory M. Provan

o-notation

For a given function g(n), we denote by o(g(n)) the set of functions:

o(g(n)) = {f(n): for any positive constant c > 0, there exists a constant n0 > 0 such that 0 f(n) < cg(n) for all n n0 }

f(n) becomes insignificant relative to g(n) as n approaches infinity: lim [f(n) / g(n)] = 0

n

We say g(n) is an upper bound for f(n) that is not

asymptotically tight.


Gregory M. Provan

O(*) versus o(*)

O(g(n)) = {f(n): there exist positive constants c and n0 such that 0 f(n) cg(n), for all n n0 }.

o(g(n)) = {f(n): for any positive constant c > 0, there exists a constant n0 > 0 such that 0 f(n) < cg(n) for all n n0 }.

Thus o(f(n)) is a weakened O(f(n)).

For example: n2 = O(n2)

n2 o(n2)

n2 = O(n3)

n2 = o(n3)


Gregory M. Provan

-notation

For a given function g(n), we denote by (g(n)) the set of

functions

(g(n)) = {f(n): for any positive constant c > 0, there

exists a constant n0 > 0 such that 0 cg(n) < f(n) for

all n n0 }

f(n) becomes arbitrarily large relative to g(n) as n

approaches infinity: lim [f(n) / g(n)] =

n

We say g(n) is a lower bound for f(n) that is not

asymptotically tight.


Gregory M. Provan

Limits

lim [f(n) / g(n)] = 0 f(n) (g(n))

n

lim [f(n) / g(n)] < f(n) (g(n))

n

0 < lim [f(n) / g(n)] < f(n) (g(n))

n

0 < lim [f(n) / g(n)] f(n) (g(n))

n

lim [f(n) / g(n)] = f(n) (g(n))

n

lim [f(n) / g(n)] undefined can’t say

n


Gregory M. Provan


Gregory M. Provan

Additional Useful Relations

The following slides provide many useful

relations for orders of growth

These relations may prove helpful in

homeworks and exams


Gregory M. Provan

Comparison of Functions

f g a b

f (n) = O(g(n)) a b

f (n) = (g(n)) a b

f (n) = (g(n)) a = b

f (n) = o(g(n)) a < b

f (n) = (g(n)) a > b


Gregory M. Provan

Properties

Transitivity f(n) = (g(n)) & g(n) = (h(n)) f(n) = (h(n))

f(n) = O(g(n)) & g(n) = O(h(n)) f(n) = O(h(n))

f(n) = (g(n)) & g(n) = (h(n)) f(n) = (h(n))

f(n) = o (g(n)) & g(n) = o (h(n)) f(n) = o (h(n))

f(n) = (g(n)) & g(n) = (h(n)) f(n) = (h(n))

Symmetry

f(n) = (g(n)) if and only if g(n) = (f(n))

Transpose Symmetry f(n) = O(g(n)) if and only if g(n) = (f(n))

f(n) = o(g(n)) if and only if g(n) = ((f(n))


Gregory M. Provan

Useful Facts

For a 0, b > 0, lim n ( lga n / nb ) = 0, so lga n =

o(nb), and nb = (lga n )

– Prove using L’Hopital’s rule and induction

lg(n!) = (n lg n)


Gregory M. Provan

Examples

A B

5n2 + 100n 3n2 + 2

log3(n2) log2(n

3)

nlg4 3lg n

lg2n n1/2


Gregory M. Provan

Examples

A B

5n2 + 100n 3n2 + 2 A (B) A (n2), n2 (B) A (B)

log3(n2) log2(n

3)

nlg4 3lg n

lg2n n1/2


Gregory M. Provan

Examples

A B

5n2 + 100n 3n2 + 2 A (B) A (n2), n2 (B) A (B)

log3(n2) log2(n

3) A (B) logba = logca / logcb; A = 2lgn / lg3, B = 3lgn, A/B =2/(3lg3)

nlg4 3lg n

lg2n n1/2


Gregory M. Provan

Examples

A B

5n2 + 100n 3n2 + 2 A (B) A (n2), n2 (B) A (B)

log3(n2) log2(n


nlg4 3lg n A (B) alog b = blog a; B =3lg n=nlg 3; A/B =nlg(4/3) as n

lg2n n1/2


Gregory M. Provan

Examples

A B

5n2 + 100n 3n2 + 2 A (B) A (n2), n2 (B) A (B)

log3(n2) log2(n


nlg4 3lg n A (B) alog b = blog a; B =3lg n=nlg 3; A/B =nlg(4/3) as n

lg2n n1/2 A (B) lim ( lga n / nb ) = 0 (here a = 2 and b = 1/2) A (B)

n


Gregory M. Provan

Review on Summation

Why do we need summation formulas?

We need them for computing the running times of some

algorithms. (CLRS/Chapter 3)

Example: Maximum Subvector

Given an array a[1…n] of numeric values (can be positive,

zero and negative) determine the subvector a[i…j] (1 i

j n) whose sum of elements is maximum over all

subvectors.

1 -2 2 2


Gregory M. Provan

Reviews on Summation

MaxSubvector(a, n)

maxsum 0;

for i 1 to n

for j = i to n

sum 0;

for k i to j

sum += a[k]

maxsum max(sum, maxsum);

return maxsum;

n n j T(n) = 1 i=1 j=i k=i

NOTE: This is not a simplified solution. What is the final answer?


Gregory M. Provan


Constant Series: For n 0,

b

1 = b - a + 1 i=a

Linear Series: For n 0,

n

i = 1 + 2 + … + n = n(n+1) / 2 i=1


Gregory M. Provan


Quadratic Series: For n 0,

n

i2 = 12 + 22 + … + n2 = (2n3 + 3n2 + n) / 6 i=1

Linear-Geometric Series: For n 0,

n

ici = c + 2c2 + … + ncn = [(n-1)c(n+1) - ncn + c] / (c-1)2 i=1


Gregory M. Provan

Divide-and-Conquer Examples


Gregory M. Provan

Integer Multiplication

Let X = A B and Y = C D where A,B,C and D

are n/2 bit integers

Simple Method: XY = (2n/2A+B)(2n/2C+D)

Running Time Recurrence

T(n) < 4T(n/2) + 100n

How do we solve it?


Gregory M. Provan

Substitution method

1. Guess the form of the solution.

2. Verify by induction.

3. Solve for constants.

The most general method:

Example: T(n) = 4T(n/2) + 100n

• [Assume that T(1) = (1).]

• Guess O(n3) . (Prove O and separately.)

• Assume that T(k) ck3 for k < n .

• Prove T(n) cn3 by induction.

Date post:	07-Feb-2021
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

CS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer...

Documents