2. Getting started

Hsu, Lih-Hsing

Chapter 2 P.2

Computer Theory Lab.

2.1 Insertion sort Example: Sorting problem

Input: A sequence of n numbers Output: A permutation of

the input sequence such that .

The number that we wish to sort are known as the keys.

a a an1 2, ,...,

a a an1 2' ' ', ,...,

2

1 ... naaa

Chapter 2 P.3

Computer Theory Lab.

Pseudocode    Insertion sort

Insertion-sort(A)1 for j 2 to length[A]2 do keyA[j]3 Insert A[j] into the sorted sequence A[1..j-1]4 i j - 15 while i>0 and A[i]>key6 do A[i+1] A[i]7 i i - 18 A[i +1] key

Chapter 2 P.4

Computer Theory Lab.

The operation of Insertion-Sort

Chapter 2 P.5

Computer Theory Lab.

Sorted in place : The numbers are rearranged within the array A, wi

th at most a constant number of them sorted outside the array at any time.

Loop invariant : At the start of each iteration of the for loop of line

1-8, the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order.

Initialization Maintenance Termination

Chapter 2 P.6

Computer Theory Lab.

2.2 Analyzing algorithms Analyzing an algorithm has come to mean pr

edicting the resources that the algorithm requires.

Resources: memory, communication, bandwidth, logic gate, time.

Assumption:one processor, RAM(random-access machine)

(We shall have occasion to investigate models for parallel computers and digital hardware.)

Chapter 2 P.7

Computer Theory Lab.

2.2 Analyzing algorithms The best notion for input size depends

on the problem being studied.. The running time of an algorithm on a

particular input is the number of primitive operations or “steps” executed. It is convenient to define the notion of step so that it is as machine-independent as possible

Chapter 2 P.8

Computer Theory Lab.

Insertion-sort(A) cost times1 for j 2 to length[A]2 do keyA[j]3 Insert A[j] into the sorted sequence A[1..j -1]4 i j -15 while i>0 and A[i]>key6 do A[i +1] A[i]7 i i -18 A[i +1] key : the number of times the while loop test in

line 5 is executed for the value of j.

c1 nc2 n 1

0c4 n 1

c5 t jj

n

2

c6 ( )t jj

n

2

1

c7 ( )t jj

n

2

1

c8 n 1t j

Chapter 2 P.9

Computer Theory Lab.

T n c n c n c n c t

c t c t c n

jj

n

jj

nj

j

n

( ) ( ) ( )

( ) ( ) ( )

1 2 4 52

62

72

8

1 1

1 1 1

for j = 2,3,…,n : Linear function on n

t j 1

T n c n c n c n c n c n

c c c c c n c c c c

( ) ( ) ( ) ( ) ( )

( ) ( )

1 2 4 5 8

1 2 4 5 8 2 4 5 8

1 1 1 1

the running time

Chapter 2 P.10

Computer Theory Lab.

the running time for j = 2,3,…,n : quadratic function on

n. jt j

)(

)2

()2

(

)1()2

)1(()

2)1(

(

)12

)1(()1()1()(

8542

8765

421

2765

876

5421

cccc

ncccc

cccnccc

ncnn

cnn

c

nncncncncnT

Chapter 2 P.11

Computer Theory Lab.

Usually, we concentrate on finding only on the worst-case running time

Reason:  It is an upper bound on the running time  The worst case occurs fair often The average case is often as bad as the

worst case. For example, the insertion sort. Again, quadratic function.

Worst-case and average-case analysis

Chapter 2 P.12

Computer Theory Lab.

Order of growth In some particular cases, we shall be i

nterested in average-case, or expect running time of an algorithm.

It is the rate of growth, or order of growth, of the running time that really interests us.

Chapter 2 P.13

Computer Theory Lab.

2.3 Designing algorithms There are many ways to design

algorithms:

Incremental approach: insertion sort Divide-and-conquer: merge sort

recursive: divide conquer combine

Chapter 2 P.14

Computer Theory Lab.

1 n1 q – p + 1

2 n2 r – q

3 create array L[ 1 .. n1 + 1 ] and R[ 1 ..n2 + 1 ]

4 for i 1 to n1

5 do L[ i ] A[ p + i – 1 ]

6 for j 1 to n2

7 do R[ i ] A[ q + j ]

8 L[n1 + 1]

9 R[n2 + 1]

   Merge(A,p,q,r)

Chapter 2 P.15

Computer Theory Lab.

10 i 1

11 j 1

12 for k p to r

13 do if L[ i ] R[ j ]

14 then A[ k ] L[ i ]

15 i i + 1

16 else A[ k ] R[ j ]

17 j j + 1

   Merge(A,p,q,r)

Chapter 2 P.16

Computer Theory Lab.

Example of Merge Sort

Chapter 2 P.18

Computer Theory Lab.

MERGE-SORT(A,p,r)

1 if p < r2 then q(p+r)/2 3 MERGE-SORT(A,p,q)4 MERGE-SORT(A,q+1,r)5 MERGE(A,p,q,r)

Chapter 2 P.19

Computer Theory Lab.

Chapter 2 P.20

Computer Theory Lab.

Analyzing divide-and-conquer algorithms   See Chapter 4 Analysis of merge sort   

T nif n c

aT n b D n c n otherwise( )

( )

( / ) ( ) ( )

1

T nif n

T n n if n( )

( )

( / ) ( )

1 1

2 2 1

T n n n( ) ( log )

Analysis of merge sort

Chapter 2 P.21

Computer Theory Lab.

Analysis of merge sort

where the constant c represents the time require to solve problems of size 1 as well as the time per array element of the divide and combine steps.

1)2/(2

1)(

nifcnnT

nifcnT

Chapter 2 P.22

Computer Theory Lab.

Chapter 2 P.23

Computer Theory Lab.

Outperforms insertion sort!