7/30/2019 C analysis

1/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 1

Analysis and Complexity:2

P. P. Chakrabarti

7/30/2019 C analysis

2/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 2

Analysis of Algorithms

? Quantifying the resources required.? Measures of resource utilization (efficiency):

? Execution time ? time complexity

? Memory space ? space complexity

? Observation :

? The larger the input data the more the resource

requirement: Complexities are functions of the

amount of input data (input size).

Refer to first few chapters of the book Introduction

To Algorithms by Cormen, Leiserson, Rivest and Stein

7/30/2019 C analysis

3/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 3

Yardstick?

?The same algorithm will run at different speedsand will require different amounts of space whenrun on different computers, differentprogramming languages, different compilers.

?Algorithms usually consume resources in somefashion that depends on the size of the problemthey solve.

?Need a machine independent complexity

measure that is a function of input size, n. (Alsowe are interested at asymptotic behaviour, thatis, when n becomes large.

7/30/2019 C analysis

4/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 4

Analyzing SelSort

? Computer 1:f1(n) = 0.0007772 n

2 + 0.00305 n +0.001

? Computer 2:

f2(n) = 0.0001724 n2 + 0.00040 n +

0.100Note: Both are quadratic functions

of n

? The shape of the curve thatexpresses the running time as a

function of the problem sizestays the same.

We say that Selection Sort is ofcomplexity Order n2 or O(n2)

int max_loc(int x[], int k, int size)

{ int j, pos;pos = k;

for (j=k+1; j x[pos])

pos = j;

return pos;}

int selsort (int x[], int size) {

int k, m, temp;

for (k=0; k

7/30/2019 C analysis

5/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 5

Complexity classes

? The running time for different algorithmsfall into different complexity classes.? Each complexity class is characterized by a

different family of curves.

?All curves in a given complexity class sharethe same basic shape. (In the Asymptotic

sense)

? The O-notationis used for talking aboutthe complexity classes of algorithms.

7/30/2019 C analysis

6/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 6

Order of Complexity, O- notation

? For the quadratic functionf(n) = an2 + bn + c

we will say that f(n) is O(n2).

? We focus on the dominant term, and ignore the lesserterms; then throw away the coefficient. [AsymptoticAnalysis]

? Since constants are finally not important we mayassume that each machine instruction takes one unit of

time when we analyze the complexity of an algorithm.? We may sometimes abstract this to unit time operators

like add, sub, multiply, etc and do an operator count.

? We can perform worst case or average case analysis.

7/30/2019 C analysis

7/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 7

How execution time is affected by various

complexity measures:

Assume speed S is 107 instructions per second.

size 10 20 30 50 100 1000 10000

n .001

ms

.002

ms

.003

ms

.005

ms

.01

ms

.1 ms 1 ms

nlogn

.003ms

.008ms

.015ms

.03ms

.07ms

1 ms 13 ms

n2 .01

ms.04ms

.09ms

.25ms

1 ms 100ms

10 s

n

3

.1

ms

.8

ms

2.7

ms

12.5

ms

100

ms

100 s 28 h

2n .1

ms.1 s 100 s 3 y 3x

1013

c

inf inf

C

O

MP

L

E

X

I

T

Y

7/30/2019 C analysis

8/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 8

Maximum size solvable within 1 hour

speedcomplexity

S 100 S 1000 S

n N1 =3.6x10

10

100 N1 1000 N1

n log n N2 =

1.2x109

85 N2 750 N2

n2 N3 =

2x105

10 N3 30 N3

2n N4 = 35 N4+7 N4+10

7/30/2019 C analysis

9/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 9

Formal Definition

? T(N) = O(f(N)) if there are positive constants c and n0 suchthat T(N) ? c f(N) when N ? n0.

Meaning : As N increases, T(N) grows no faster

than f(N).

The function T is eventually bounded by some multiple off(N). f(N) gives an upper bound in the behavior of T(N).

logen = O(n)

n2 = O(n2).

n2 = O(n3).nlogn = O(n2).

3 n2 + 5n + 1 = O(n2)

7/30/2019 C analysis

10/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 10

Some Worst Case Complexities:

? Linear Search in an array: ?

? Binary Search: ?

? Selection Sort: ?

? Mergesort: ?

? Quicksort: ?

? Stack ADT: Push and Pop are ?

? Queue using a singly linked list: ?? (What about a

doubly linked list??)

? Multiplying 2 square matrices:?

? The primes that we wrote in class:?

7/30/2019 C analysis

11/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 11

Some Worst Case Complexities:

? Linear Search in an array: O(n)

? Binary Search: O(log n)

? Selection Sort: O(n2)

?Mergesort:? T(n) = 2T(n/2) + cn

? Solves to O(n log n)

? T(n) = 2(2T(n/4) + cn/2) + cn

= 4T(n/4) + 2cn= 2kT(n/2k) + kcn [Note that T(2) = 1]

= O(nk) = O(n log n)

7/30/2019 C analysis

12/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 12

Some Worst Case Complexities:

?

Quicksort: O(n2

)[This occurs when every partition breaks

the array of n elements into 1 and n-1]

?

Stack ADT: Push and Pop are O(1)?Queue using a singly linked list:

? Insert (Enqueue): O(1)

?Delete (Dequeue): O(n)?Queue using a doubly linked list: All O(1)

7/30/2019 C analysis

13/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 13

Primes:

? For an integer of value k we loop at most sqrt(k)

times. So is the complexity O(sqrt(n))?

? Remember that the input for value n is in log2 k

bits.

? So the input size is n = log2 n

? The complexity in terms of input size for this

algorithm is therefore O(2n)

? The latest proof of Dr Manindra Agarwal et alproves that prime detection can be done in O(nk)

time for some constant k

7/30/2019 C analysis

14/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 14

More definitions

? T(N) = ? (g(N)) if there are positive constants c and n0 such that

T(N) ? c f(N) when N ? n0.

Meaning : As N increases, T(N) grows no slower than g(N) ;

T(N) grows at least as fast as g(N).

? T(N) = ?(h(N)) if and only if T(N) = O (h(N)) and T(N) = ? (h(N))Meaning : As N increases, T(N) grows as fast as h(N).

? T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) ? ?(p(N))

Meaning : As N increases, T(N) grows slower than p(N).

lim n? ? T(N)/p(N) = 0.

logen = O(n)

n10 = o(2n)

3 n2 + 5n + 1 = ?(n2)

7/30/2019 C analysis

15/15

10-04-03 P.P.Chakrabarti, IIT Kharagpur 15

Space Complexity

? Measures the work space or space required in addition toinput storage

? Selection Sort: O(1)

? Binary Search:

?Recursive algorithm : O(log n) {Recursion Stack}

? Iterative algorithm: O(1)

? Mergesort: O(n) {Why: Additional array. Here recursion

stack is of depth log n}

? Quicksort: O(n) {Why: recursion stack in the worst case}