CSE 326Asymptotic Analysis

David Kaplan

Dept of Computer Science & EngineeringAutumn 2001

Asymptotic Analysis

CSE 326 Autumn 20012

Housekeeping Homework 1 status Join cse326@cs Poll:

Slide format(s)? Office hours today?

Asymptotic Analysis

CSE 326 Autumn 20013

Analyzing AlgorithmsAnalyze algorithms to gauge:

Time complexity (running time) Space complexity (memory use)

Input size is indicated by a number n sometimes have multiple inputs, e.g. m

and n

Running time is a function of nn, n2, n log n, 18 + 3n(log n2) + 5n3

Asymptotic Analysis

CSE 326 Autumn 20014

RAM ModelRAM (random access machine)

Ideal single-processor machine (serialized operations)

“Standard” instruction set (load, add, store, etc.)

All operations take 1 time unit (including, for our purposes, each C++ statement)

Asymptotic Analysis

CSE 326 Autumn 20015

Order Notation (aka Big-O)

Big-O

T(n) = O( f(n) )Exist positive constants c, n0 such that

T(n) cf(n) for all n n0

Upper bound

Omega

T(n) = ( f(n) )Exist positive constants c, n0 such that

T(n) cf(n) for all n n0

Lower bound

ThetaT(n) = θ( f(n) )T(n) = O(f(n)) AND T(n) = (f(n))

Tight bound

little-oT(n) = o( f(n) )T(n) = O(f(n)) AND T(n) != θ(f(n))

Strict upper bound

Asymptotic Analysis

CSE 326 Autumn 20016

Simplifying with Big-OBy definition, Big-O allows us to:

Eliminate low order terms 4n + 5 4n 0.5 n log n - 2n + 7 0.5 n log n

Eliminate constant coefficients 4n n 0.5 n log n n log n log n2 = 2 log n log n log3 n = (log3 2) log n log n

But when might constants or low-order terms matter?

Asymptotic Analysis

CSE 326 Autumn 20017

Big-O Examplesn2 + 100 n = O(n2)

follows from … ( n2 + 100 n ) 2 n2 for n 10

n2 + 100 n = (n2)follows from …( n2 + 100 n ) 1 n2 for n 0

n2 + 100 n = (n2)by definition

n log n = O(n2)n log n = (n log n)n log n = (n)

Asymptotic Analysis

CSE 326 Autumn 20018

Big-O UsageOrder notation is not symmetric:

we can say 2n2 + 4n = O(n2) … but never O(n2) = 2n2 + 4nRight-hand side is a crudification of the left

Order expressions on left can produce unusual-looking, but true, statements:

O(n2) = O(n3)(n3) = (n2)

Asymptotic Analysis

CSE 326 Autumn 20019

Big-O Comparisons

Function A

n3 + 2n2

n0.1

n + 100n0.1

5n5

n-152n/100

82log n

Function #2

100n2 + 1000

log n

2n + 10 log n

n!

1000n15

3n7 + 7n

vs.

Asymptotic Analysis

CSE 326 Autumn 200110

Race 1100n2 + 1000vs.n3 + 2n2

Asymptotic Analysis

CSE 326 Autumn 200111

Race 2n0.1 log nvs.

In this one, crossover point is very late! So, which algorithm is really better???

Asymptotic Analysis

CSE 326 Autumn 200112

Race Cn + 100n0.1 2n + 10 log nvs.

Is the “better” algorithm asymptotically better???

Asymptotic Analysis

CSE 326 Autumn 200113

Race 45n5 n!vs.

Asymptotic Analysis

CSE 326 Autumn 200114

Race 5n-152n/100 1000n15vs.

Asymptotic Analysis

CSE 326 Autumn 200115

Race VI82log(n) 3n7 + 7nvs.

Asymptotic Analysis

CSE 326 Autumn 200116

Big-O Winners (i.e. losers)

Function A

n3 + 2n2

n0.1

n + 100n0.1

5n5

n-152n/100

82log n

Function #2

100n2 + 1000

log n

2n + 10 log n

n!

1000n15

3n7 + 7n

vs.

Winner

O(n2)

O(log n)

O(n) TIE

O(n5)

O(n15)

O(n6) why???

Asymptotic Analysis

CSE 326 Autumn 200117

Big-O Common Namesconstant: O(1)logarithmic: O(log n) linear: O(n)log-linear: O(n log n)superlinear: O(n1+c) (c is a constant

> 0)quadratic: O(n2)polynomial: O(nk) (k is a constant)exponential: O(cn) (c is a constant

> 1)

Asymptotic Analysis

CSE 326 Autumn 200118

Kinds of AnalysisRunning time may depend on actual input,

not just length of input

Distinguish Worst case

Your worst enemy is choosing input Average case

Assume probability distribution of inputs Amortized

Average time over many runs Best case (not too useful)

Asymptotic Analysis

CSE 326 Autumn 200119

Analyzing CodeC++ operations

Consecutive stmts

ConditionalsLoops

Function callsRecursive functions

constant timesum of timeslarger branch plus

testsum of iterationscost of function bodysolve recursive

equation

Asymptotic Analysis

CSE 326 Autumn 200120

Nested Loops

2

11

1

1 nnn

i

n

j

n

i

for i = 1 to n do

for j = 1 to n do

sum = sum + 1

Asymptotic Analysis

CSE 326 Autumn 200121

Dependent Nested Loops

for i = 1 to n do

for j = i to n do

sum = sum + 1

ininn

i

n

i

n

i

n

ij

n

i 1111

)1()1(1

2

2

)1(

2

)1()1( n

nnnnnn

Asymptotic Analysis

CSE 326 Autumn 200122

Recursion A recursive procedure can often be

analyzed by solving a recursive equation

Basic form:T(n) =

base case: some constantrecursive case: T(subproblems) + T(combine)

Result depends upon how many subproblems how much smaller are subproblems how costly to combine solutions

(coefficients)

Asymptotic Analysis

CSE 326 Autumn 200123

Sum of QueueSumQueue(Q) if (Q.length == 0 ) return 0 else return Q.dequeue() + SumQueue(Q)

One subproblem

Linear reduction in size (decrease by 1)

Combining: constant (cost of 1 add)

T(0) b

T(n) c + T(n – 1) for n>0

Asymptotic Analysis

CSE 326 Autumn 200124

Sum of Queue Solution

T(n) c + c + T(n-2) c + c + c + T(n-3) kc + T(n-k) for all k nc + T(0) for k=n cn + b = O(n)

T(0) b

T(n) c + T(n – 1) for n>0

Equation:

Solution:

Asymptotic Analysis

CSE 326 Autumn 200125

Binary SearchBinarySearch(A, x) Search A, a sorted array, for item x

One subproblem, half as large

7 12 30 35 75 83 87 90 97 99

T(1) b

T(n) T(n/2) + c for n>1

Equation:

Asymptotic Analysis

CSE 326 Autumn 200126

Binary Search: Solution

T(n) T(n/2) + c T(n/4) + c + c T(n/8) + c + c + c T(n/2k) + kc T(1) + c log n where k = log n b + c log n = O(log n)

T(1) b

T(n) T(n/2) + c for n>1

Equation:

Solution: