04/11/23 1
Analysis of Algorithms
Lecture
Analysis of quick sort
04/11/23 2
Quick sort
• Another divide and conquer sorting algorithm – like merge sort
• Anyone remember the basic idea?
• The worst-case and average-case running time?
• Learn some new algorithm analysis tricks
04/11/23 3
Quick sort
Quicksort an n-element array:
1. Divide: Partition the array into two subarrays around a pivot x such that elements in lower subarray x elements in upper subarray.
2. Conquer: Recursively sort the two subarrays.
3. Combine: Trivial.
x x xx ≥ x≥ x
Key: Linear-time partitioning subroutine.
04/11/23 4
Partition
• All the action takes place in the partition() function– Rearranges the subarray in place– End result: two subarrays
• All values in first subarray all values in second
– Returns the index of the “pivot” element separating the two subarrays
x x xx ≥ x≥ xp rq
04/11/23 5
Pseudocode for quicksort
QUICKSORT(A, p, r)if p < r
then q PARTITION(A, p, r) QUICKSORT(A, p, q–1)QUICKSORT(A, q+1, r)
Initial call: QUICKSORT(A, 1, n)
04/11/23 6
Idea of partition
• If we are allowed to use a second array, it would be easy
66 1010 55 88 1313 33 22 1111
66 55 33 22 1111 1313 88 1010
22 55 33 66 1111 1313 88 1010
04/11/23 7
Another idea
• Keep two iterators: one from head, one from tail
66 1010 55 88 1313 33 22 1111
66 22 55 33 1313 88 1010 1111
33 22 55 66 1313 88 1010 1111
04/11/23 8
In-place Partition
66 1010 55 88 1313 33 22 11112 3 8 103 6
04/11/23 9
Partition In Words
• Partition(A, p, r):– Select an element to act as the “pivot” (which?)
– Grow two regions, A[p..i] and A[j..r]• All elements in A[p..i] <= pivot
• All elements in A[j..r] >= pivot
– Increment i until A[i] > pivot
– Decrement j until A[j] < pivot
– Swap A[i] and A[j]
– Repeat until i >= j
– Swap A[j] and A[p]
– Return j
Note: different from book’s partition(), which uses two iterators that both move forward.
04/11/23 10
Partition CodePartition(A, p, r) x = A[p]; // pivot is the first element i = p; j = r + 1; while (TRUE) {
repeat i++; until A[i] > x or i >= j; repeat j--; until A[j] < x or j < i; if (i < j) Swap (A[i], A[j]); else break;
} swap (A[p], A[j]); return j;
What is the running time of partition()?
partition() runs in (n) time
04/11/23 11
i j66 1010 55 88 1313 33 22 1111x = 6
p r
i j66 1010 55 88 1313 33 22 1111
i j66 22 55 88 1313 33 1010 1111
i j66 22 55 88 1313 33 1010 1111
i j66 22 55 33 1313 88 1010 1111
ij66 22 55 33 1313 88 1010 1111
33 22 55 66 1313 88 1010 1111qp r
scan
scan
scan
swap
swap
final swap
Partition example
04/11/23 12
66 1010 55 88 1111 33 22 1313
33 22 55 66 1111 88 1010 1313
22 33 55 66 88 1010 1111 1313
22 33 55 66 1010 88 1111 1313
22 33 55 66 88 1010 1111 1313
Quick sortexample
04/11/23 13
Analysis of quicksort
• Assume all input elements are distinct.
• In practice, there are better partitioning algorithms for when duplicate input elements may exist.
• Let T(n) = worst-case running time on an array of n elements.
04/11/23 14
Worst-case of quicksort
• Input sorted or reverse sorted.• Partition around min or max element.• One side of partition always has no elements.
)(
)()1(
)()1()1(
)()1()0()(
2n
nnT
nnT
nnTTnT
(arithmetic series)
04/11/23 15
Worst-case recursion treeT(n) = T(0) + T(n–1) + n
04/11/23 16
Worst-case recursion treeT(n) = T(0) + T(n–1) + n
T(n)
04/11/23 17
n
T(0) T(n–1)
Worst-case recursion treeT(n) = T(0) + T(n–1) + n
04/11/23 18
n
T(0) (n–1)
Worst-case recursion treeT(n) = T(0) + T(n–1) + n
T(0) T(n–2)
04/11/23 19
n
T(0) (n–1)
Worst-case recursion treeT(n) = T(0) + T(n–1) + n
T(0) (n–2)
T(0)
T(0)
04/11/23 20
n
T(0) (n–1)
Worst-case recursion treeT(n) = T(0) + T(n–1) + n
T(0) (n–2)
T(0)
T(0)
2
1
nkk
height
height = n
04/11/23 21
n
T(0) (n–1)
Worst-case recursion treeT(n) = T(0) + T(n–1) + n
T(0) (n–2)
T(0)
T(0)
2
1
nkk
n
height = n
04/11/23 22
n
(n–1)
Worst-case recursion treeT(n) = T(0) + T(n–1) + n
(n–2)
(1)
2
1
nkk
n
height = n
(1)
(1)
(1)T(n) = (n) + (n2)
= (n2)
04/11/23 23
Best-case analysis(For intuition only!)
If we’re lucky, PARTITION splits the array evenly:
T(n) = 2T(n/2) + (n)= (n log n) (same as merge sort)
What if the split is always 109
101 : ?
)()(109
101 nnTnTnT
What is the solution to this recurrence?
04/11/23 24
Analysis of “almost-best” case
)(nT
04/11/23 25
Analysis of “almost-best” case
nT101 nT
109
n
04/11/23 26
Analysis of “almost-best” case
n101 n10
9
nT100
1 nT100
9 nT100
9 nT10081
n
04/11/23 27
Analysis of “almost-best” case
n1001
(1)
(1)
… …log10/9n
n
n
n
…O(n) leavesO(n) leaves
n101 n10
9
n1009 n100
9 n10081
n
04/11/23 28
log10n
Analysis of “almost-best” case
n
(1)
(1)
… …log10/9n
n
n
n
T(n) n log10/9n + (n)
…
n log10n
O(n) leavesO(n) leaves
(n log n)
n1001
n101 n10
9
n1009 n100
9 n10081
04/11/23 29
Quicksort Runtimes
• Best-case runtime Tbest(n) (n log n)
• Worst-case runtime Tworst(n) (n2)
• Worse than mergesort? Why is it called quicksort then?
• Its average runtime Tavg(n) (n log n )
• Better even, the expected runtime of randomized quicksort is (n log n)
04/11/23 30
Randomized quicksort
• Randomly choose an element as pivot– Every time need to do a partition, throw a die to
decide which element to use as the pivot– Each element has 1/n probability to be selected
Rand-Partition(A, p, r) d = random(); // a random number between 0 and 1 index = p + floor((r-p+1) * d); // p<=index<=r swap(A[p], A[index]); Partition(A, p, r); // now do partition using A[p] as pivot
04/11/23 31
Running time of randomized quicksort
T(n) =
T(0) + T(n–1) + dn if 0 : n–1 split,T(1) + T(n–2) + dn if 1 : n–2 split,T(n–1) + T(0) + dn if n–1 : 0 split,