8.Sorting in linear time

Chapter 8 P.2

Computer Theory Lab.

8.1 Lower bound for sortingThe decision tree model

a1:a2

a2:a3 a1:a3

<1,2,3> a1:a3

<3,1,2><1,3,2>

a2:a3<2,1,3>

<3.2,1><2,3,1>

Chapter 8 P.3

Computer Theory Lab.

Theorem 9.1. Any decision tree that sorts n elements has

height ( log )n n .

Proof:

).lg()!log(

,2!

nnnh

ln h

Corollary 9.2 Heapsort and merge sort are asymptotically

optimal comparisons.

Chapter 8 P.4

Computer Theory Lab.

8.2 Counting sortAAAssssssuuummmeee ttthhhaaattt eeeaaaccchhh ooofff ttthhheee nnn iiinnnpppuuuttt eeellleeemmmeeennntttsss iiisss aaannn

iiinnnttteeegggeeerrr iiinnn ttthhheee rrraaannngggeee 111 tttooo kkk fffooorrr sssooommmeee iiinnnttteeegggeeerrr kkk...

COUNTING_SORT(A,B,k)

1 for i 1 to k

2 do c i[ ] 0

3 for j 1 to length[A]

4 do c A j c A j[ [ ]] [ [ ]] 1

5 ► c[i] now contains the number

of elements equal to i

6 for i 2 to k

7 do c i c i c i[ ] [ ] [ ] 1

8 ► c[i] now contains the number of

elements less than or equal to i

9 for j length A [ ] downto 1

10 do B c A j A j[ [ [ ]]] [ ]

11 c A j c A j[ [ ]] [ [ ]] 1

Chapter 8 P.5

Computer Theory Lab.

A n a l y s i s : O k n( )

S p e c i a l c a s e : O n( ) w h e n k O n ( ) .

P r o p e r t y : sss ttt aaa bbb lll eee ( n u m b e r w i t h t h e s a m e v a l u e a p p e a r i n t h e

o u t p u t a r r a y i n t h e s a m e o r d e r a s t h e y d o i n t h e i n p u t a r r a y . )

Chapter 8 P.6

Computer Theory Lab.

The operation of Counting-sort on an input array A[1..8]

Chapter 8 P.7

Computer Theory Lab.

8.3 Radix sort Used by the card-sorting machines you can now

find only in computer museum.

RADIX_SORT(A,d)

1 for i 1 to d

2 do use a stable sort to sort array A on digit i

Chapter 8 P.8

Computer Theory Lab.

Analysis: O d n k O dn dk( ( )) ( )

NOTE: not in-place (in-place: only constant number of elements

of the input array are ever stored outside the array.)

Lemma 8.3

Given n d-digit numbers in which each digit can take on up to k

possible values, RADIX-SORT correctly sorts these number in

(d(n + k)) time.

Chapter 8 P.9

Computer Theory Lab.

Lemma 8.4

Given n b-bit numbers and any positive integer r ≤ b, RADIX- SORT

correctly sorts these numbers in ((b/r)(n+2r)) time.

Proof : Choose d = b/r .

IF b < lg n , choose r = b.

((b/r)(n+2r)) = (bn/logn)

Chapter 8 P.10

Computer Theory Lab.

8.4 Bucket sort

B U C K E T _ S O R T ( A )

1 n l e n g t h A [ ]

2 f o r i 1 t o n

3 d o i n s e r t A i[ ] i n t o

l i s t B n A i[ ]

4 f o r i 1 t o n - 1

5 d o s o r t l i s t B i[ ] w i t h i n s e r t i o n s o r t

6 c o n c a t e n a t e B B B n[ ] , [ ] , . . . , [ ]0 1 1 t o g e t h e r i n

o r d e r

Chapter 8 P.11

Computer Theory Lab.

AnalysisThe running time of bucket sort is

.)()()(1

0

2

n

iinOnnT

taking expectations of both sides and using linearity of expectation, we have

1

0

2

1

0

2

1

0

2

)]([)(

)]([)(

)()()]([

n

ii

n

ii

n

ii

nEOn

nOEn

nOnEnTE

Chapter 8 P.12

Computer Theory Lab.

We claim that

nnE i /12][ 2

We define indicator random variablesXij = I {A[j] falls in bucket i}

for i = 0, 1, …, n-1 and j = 1, 2,…,n. thus,

.1

n

jiji Xn

Chapter 8 P.13

Computer Theory Lab.

,][][

][

1 11

2

1 11

2

1 1

2

1

2

njjk

nkikij

n

jij

njjk

nkikij

n

jij

n

jik

n

kij

n

jiji

XXEXE

XXXE

XXE

XEnE

Chapter 8 P.14

Computer Theory Lab.

Indicator random variable Xij is 1 with probability 1/n and 0

otherwise, and therefore

n

nnXE ij

1

110

11][ 2

When k j, the variables Xij and Xik are independent, and hence

.1

11

][][][

2n

nn

XEXEXXE ikijikij

Chapter 8 P.15

Computer Theory Lab.

n

nn

nnn

nn

nnnE

njjk

nk

n

ji

12

11

1)1(

1

11][

2

1 12

1

2

We can conclude that the expected time for bucket sort is (n)+n·O(2-1/n)= (n).

Chapter 8 P.16

Computer Theory Lab.

Another analysis:

O E n O E n O O nii

ni

i

n

i

n( [ ]) ( [ ]) ( ( )) ( )2

0

1 2

0

1

0

11

Because

E n Var n E n

E n np p=n

Var n np pn

E nn n

i i i

i

i

i

( ) [ ] [ ]

[ ]

[ ] ( )

( ) ( )

2 2

2 2

11

1 11

111 2

11

where

(Basic Probability Theory)