CS420 lecture fivePriority Queues, Sorting

wim bohm cs csu

Heaps• Heap: array representation of a

complete binary tree– every level is completely filled except the bottom level: filled from left

to right

• Can compute the index of parent and children– parent(i) = floor(i/2) left(i)= 2i index(i)=2i+1 (for 1 based arrays)

• Heap property: A[parent(i)] >= A[i]

16

10

9 3

142

78

14

16 14 10 8 7 9 3 2 4 1

HeapifyTo create a heap at i, assuming left(i) and right(i) are heaps, bubble A[i] down: swap with max child until heap property holds

heapify(A,i){ L=left(i); R=right(i); if L<=N and A[L] > A[i] max=L else max = i; if R<=N and A[R]>A[max] max =R; if max != i { swap(A,i,max); heapify(A.max) } }

Building a heap

• heapify performs at most lg n swaps• building a heap out of an array:– The leaves are all heaps– heapify backwards starting at last internal node

buildheap(A){ for i = floor(n/2) downto 1 heapify(A,i)}

Complexity buildheap

• Suggestions? ...

Complexity buildheap

• initial thought O(nlgn), but

half of the heaps are height 1 quarter are height 2 only one is height log n

It turns out that O(nlgn) is not tight!

complexity buildheapheight

0

1

2

3

max #swaps

0 = 21-2

1 = 22-3

2*1+2 = 4 = 23-4

2*4+3 = 11 = 24-5

complexity buildheapheight

0

1

2

3

max #swaps

0 = 21-2

1 = 22-3

2*1+2 = 4 = 23-4

2*4+3 = 11 = 24-5

Conjecture: height = h max #swaps = 2h+1-(h+2)

Proof: induction height = (h+1) max #swaps: 2*(2h+1-(h+2))+(h+1) = 2h+2-2h-4+h+1 = 2h+2-(h+3) = 2(h+1)+1-((h+1)+2) n nodes O(n) swaps

Cormen et.al. complexity buildheap

€

N

2hh=0

lgN

∑ h = Nh

2hh=0

lgN

∑

h

2hh=0

∞

∑ =1/2

(1−1/2)2 = 2

Nh

2hh=0

lgN

∑ =O(n)

Differentiation trick (Cormen et.al. Appendix A)

€

x h

h=0

∞

∑ =1

1− x Infinite geometrics series | x |<1

differentiate both sides and multiply by x :

hx h

h=0

∞

∑ =x

(1− x)2

take x =1/2

Heapsort

heapsort(A){ buildheap(A); for i = n downto 2{ swap(A,1,i); n=n-1; heapify(A,1); }}

Complexity heapsort

• buildheap: O(n)• swap/heapify loop: O(nlgn)

• space: in place: n– less space than merge sort

142

78

14

1412

74

8

1482

14

7

1487

12

4

1487

41

2

1487

42

1

Priority Queues

• heaps are used in priority queues– each value associated with a key– max priority queue S (as in heapsort) has

operations that maintain the heap property of S• insert(S,x)• max(S) returning max element• Extract-max(S) extracting and returning max element• increase key(S,x,k) increasing the key value of x

Extract max: O(log n)

Extract-max(S){ // pre:N>0 max=S[1]; S[1]=S[N]; N=N-1; heapify(S) }

O(log N)

Increase key: O(log n)

Increase-key(S,i,k){ //pre: k>S[i] A[i]=k; // bubble up while(i>1 and S[parent(i)]<S[i]){ swap(S,i,parent(i)); i = parent(i) }}

Insert O(log n)

• Insert(S,x)– put x at end of S– bubble x up like in Increase-key

Decrease-key

• How would decrease key work?

• What would be its complexity?

Quicksort

• Quicksort has worst case complexity?

Quicksort

• Quicksort has worst case complexity O(n2)

• So why do we care about Quicksort?

Quicksort

• Quicksort has worst case complexity O(n2)

• So why do we care about Quicksort?– Because it is very fast in practice• Average complexity: O(nlgn)• Low multiplicative constants• Sequential array access• In place • Often faster than MergeSort and HeapSort

Partition: O(n)Partition(A,p,r){ // partition A[p..r] in-place in two sub arrays: low and hi // all elements in low < all elements in hi // return index of last element of low x=A[p]; i=p-1; j=r+1; while true repeat j=j-1 until A[j]<=x repeat i=i+1 until A[i]>x if i<j swap(A,i,j) else return j}

QuickSort

Quicksort(A,p,r){ if p<r { q=Partition(A,p,r) Quicksort(A,p,q) Quicksort(A,q+1,r) } }

Worst case complexity: when one partition is size 1: T(n)=T(n-1)+n=T(n-2)+(n-1)+n=T(n-3)+(n-2)+(n-1)+n= = O(n2)

Quicksort average case complexity

• Assume a uniform distribution: each partition index has equal probability 1/n. Thus

€

T(n) = (n +1) +1

nT(q) +T(n −q)

q=1

n−1

∑

T(0) = T(1) = 0

Quicksort average case complexity

We are summing all Ts up (T(q)) and down (T(N-q)), both are the same sums so

€

T(n) = (n +1) +1

nT(q) +T(n −q)

q=1

n−1

∑

T(0) = T(1) = 0

€

T(n) = (n +1) +2

nT(q)

q=1

n−1

∑

Quicksort average case complexity

• multiply both sides by n

• substitute n-1 for n€

T(n) = (n +1) +2

nT(q)

q=1

n−1

∑

€

nT(n) = n(n +1) + 2 T(q)q=1

n−1

∑ (2)

€

(n −1)T(n −1) = n(n −1) + 2 T(q)q=1

n−2

∑ (3)

Quicksort average case complexity

• subtract (2)-(3)€

nT(n) = n(n +1) + 2 T(q)q=1

n−1

∑ (2)

€

(n −1)T(n −1) = n(n −1) + 2 T(q)q=1

n−2

∑ (3)

€

nT(n) − (n −1)T(n −1) = 2n + 2T(n −1)

nT(n) = 2n + 2T(n −1) + (n −1)T(n −1)

nT(n) = 2n + (n +1)T(n −1)

Quicksort average case complexity

• Which technique that we have learned can help us here?

€

nT(n) = 2n + (n +1)T(n −1)

a non − linear recurrence relation :

T(n) = 2 +n +1

nT(n −1)

Quicksort average case complexity

• repeated substitution!!

€

T(n) = 2 +n +1

nT(n −1)

T(n −1) = 2 +n

n −1T(n − 2)

T(n − 2) = 2 +n −1

n − 2T(n − 3)

etc.

T(n) = 2 + 2n +1

n+ 2

(n +1)n

n(n −1)+ 2

(n +1)n(n −1)

n(n −1)(n − 2)+ ...

Quicksort average case complexity

€

T(n) = 2 + 2n +1

n+ 2

(n +1)n

n(n −1)+ 2

(n +1)n(n −1)

n(n −1)(n − 2)+ ...

simplify

T(n) = 2 + 2(n +1)1

n+ 2(n +1)

1

(n −1)+ 2(n +1)

1

(n − 2)+ ...

reorder

T(n) = 2 + 2(n +1)1

kk=1

n

∑ = 2(n +1)1

kk=1

n

∑

Harmonic series from previous lecture : 1

kk=1

n

∑ = ln(n) +O(1)

T(n) =O(n lgn)