DR. Gatot F. Hertono, MSc.

Design and Analysis of ALGORITHM(Session 2)

Our Machine Model: Assumptions

Generic Random Access Machine (RAM) Executes operations sequentially Set of basic operations:

Arithmetic. Logical, Comparisons, Function calls

Simplifying assumption: all ops cost 1 unit

Eliminates dependence on the speed of our computer, otherwise impossible to verify and to compare

Notes:

Running time

• The running time depends on the input. Example: an already sorted sequence is easier to

sort.• Major Simplifying Convention: Parameterize the

running time by the size of the input. TA(n) = time of A on length n inputs

• Generally, we seek upper bounds on the running time, to have a guarantee of performance.

Examples of Basic Operations

Algorithm Input Types Basic Operations

List Searching List with n elements Comparation

List Sorting List with n elements Comparation

Matrix Product n x n matrices Scalar Products

Prime Factorisation n digit numbers Scalar Division

Polynomial Evaluation n degree polynomial Scalar Products

Tree Traversal Tree with n nodes Visiting a node

Notes: The running time of an algorithm is determined by its input size n

Time Complexity

The complexity of an algorithm is determined by the number of basic operations and how many time the algorithm computes those basic operations.

Notes: The complexity analysis is machine independent.

Time complexity of an algorithm will determine the running time depends on its input size, i.e. the time complexity is a function of input size.

Time Complexity maps “input size”

to “time” T(n) executed.

Purpose

To estimate how long a program will run. To estimate the largest input that can reasonably be

given to the program. To compare the efficiency of different algorithms. To help focus on the parts of code that are executed

the largest number of times. To choose an algorithm for an application.

Time Complexity: an example

Best, Worst and Average Case

Sometimes, given two different inputs with a same size, an algorithm can have different running time.

Example: Suppose a sorting algorithm has some inputs with a same size but different order:

-Input 1: 10, 5, 23, 45, 1, 100

-Input 2: 1,5,10, 23,45, 100

-Input 3: 100, 45, 23, 10, 5, 1

Do those inputs give the same running time?

In ascending order

Average case

Best case

Worst case

Best, Worst and Average Case (cont.)

Best, Worst and Average Cases (cont.)

Best-case Complexity: is a function B(n) B(n) = min{ (i) i In }

Let In denote a set of all input with size n of an algorithm and (i) denote the number of primitive operations of the corresponding algorithm when given input i.

Worst-case Complexity: is a function W(n) W(n) = max{ (i) i In }

Average-case Complexity: is a function A(n) A(n) = nIi

ipi )().(

where p(i) is the probability of i occurs as an input of an algorithm.

Mathematically, we can define:

Example of insertion sort8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6

2 4 8 9 3 6

2 3 4 8 9 6

2 3 4 6 8 9 done

Running Time of Insertion Sort

n

j jt2

1Insertion_Sort(A)

for j 2 to length(A)Costc1

Timesn

2 key A(j) c2 n-1

3 i j-1 c3 n-1

4 while i > 0 and A(i) > key

c4

5 A(i+1) A(i) c5

6 i i – 1 c6

7 A(i+1) key c7 n-1

12

n

j jt

12

n

j jt

n = length(A)

tj = number of while loop execution for a certain value j

Insertion Sort: an analysis

n

j j

n

j

n

j jj

nctc

tctcncncncnT

2 76

2 254321

)1()1(

)1()1()1()(

)()()( 743274321 ccccncccccnT Best case: in an ordered list (i.e tj = 1, for j = 2, …, n)

Worst case: in a reverse ordered list (i.e tj = j, for j = 2, …, n)

)(

)222

()222

()(

7432

7654

3212654

cccc

ncccc

cccnccc

nT

Time Complexity: a comparison

Machine-independent time

What is insertion sort’s worst-case time?BIG IDEAS:

• Ignore machine dependent constants, otherwise impossible to verify and to compare algorithms

• Look at growth of T(n) as n → ∞ .

“Asymptotic Analysis”“Asymptotic Analysis”

Analysis

Simplifications Ignore actual and abstract statement

costs Order of growth is the interesting

measure: Highest-order term is what counts

Remember, we are doing asymptotic analysis As the input size grows larger it is the high

order term that dominates

Assignment 1

In order to show that an algorithm is not unique, design two different algorithms of a specific problem.

Design an algorithm and show its time complexity to compute a product of two n x n matrices (how is the time complexity in the best and worst cases?)

Performance & Speed