1 TCSS 342, Winter 2005 Lecture Notes Sorting Weiss Ch. 8, pp. 283-321.

1

TCSS 342, Winter 2005Lecture Notes

Sorting

Weiss Ch. 8, pp. 283-321

2

Sorting

Sorting = putting objects in order in array/list

one of the fundamental problems in computer science

comparison-based sorting: must determine order through comparison operations on the input data:

<, >, compareTo, …

15 2 8 1 17 10 12 5

0 1 2 3 4 5 6 7

3

1. Bogo sort

Bogo sort orders a list of values by repetitively shuffling them and checking if they are sorted

More specifically: scan the list, seeing if it is sorted if not, shuffle the values in the list and repeat

This sorting algorithm has terrible performance! Can we deduce its runtime?

4

Bogo sort codepublic static void bogoSort(int[] a) { while (!isSorted(a)) shuffle(a);}

// Returns true if array a's elements// are in sorted order.public static boolean isSorted(int[] a) { for (int i = 0; i < a.length - 1; i++) if (a[i] > a[i+1]) return false; return true;}

5

Bogo sort code, helpers// Shuffles an array of ints by randomly swapping each// element with an element ahead of it in the array.public static void shuffle(int[] a) { for (int i = 0; i < a.length - 1; i++) { // pick random number in [i+1, a.length-1] inclusive int range = a.length-1 - (i+1) + 1; int j = (int)(Math.random()*range + (i+1)); swap(a, i, j); }}

// Swaps a[i] with a[j].private static void swap(int[] a, int i, int j) { if (i == j) return;

int temp = a[i]; a[i] = a[j]; a[j] = temp;}

6

Bogo sort runtime How long should we expect bogo sort to take?

related to probability of shuffling into sorted order assuming shuffling code is fair, probability equals

1 / (number of permutations of n elements)

bogo sort takes roughly factorial time to run note that if array is initially sorted, bogo finishes

quickly! it should be clear that this is not satisfactory...

!nPnn

7

2. Bubble sort

Bubble sort orders a list of values by repetitively comparing neighboring elements and swapping their positions if necessary

More specifically: scan the list, exchanging adjacent elements if

they are not in relative order; this bubbles the highest value to the top

scan the list again, bubbling up the second highest value

repeat until all elements have been placed in their proper order

8

Traverse a collection of elements Move from the front to the end "Bubble" the largest value to the end using

pair-wise comparisons and swapping

512354277 101

1 2 3 4 5 6

Swap42 77

"Bubbling" largest element

9




512357742 101

1 2 3 4 5 6

Swap35 77

10




512773542 101

1 2 3 4 5 6

Swap12 77

11




577123542 101

1 2 3 4 5 6

No need to swap

12




577123542 101

1 2 3 4 5 6

Swap5 101

13




77123542 5

1 2 3 4 5 6

101

Largest value correctly placed

14

Bubble sort codepublic static void bubbleSort(int[] a) { for (int i = 0; i < a.length; i++) for (int j = 1; j < a.length - i; j++) if (a[j-1] > a[j]) swap(a, j-1, j);}

private static void swap(int[] a, int i, int j) { if (i == j) return;

int temp = a[i]; a[i] = a[j]; a[j] = temp;}

15

Bubble sort runtime Running time (# comparisons) for input size

n:

number of actual swaps performed depends on the data; out-of-order data performs many swaps

)(O

2

)1(

)(1

2

1

0

1

0

1

0 1

n

nn

i

in

n

i

n

i

n

i

in

j

16

3. Selection sort

Selection sort orders a list of values by repetitively putting a particular value into its final position

More specifically: find the smallest value in the list switch it with the value in the first position find the next smallest value in the list switch it with the value in the second position repeat until all values are in their proper

places

17

Selection sort example

18

Index 0 1 2 3 4 5 6 7

Value 27 63 1 72 64 58 14 9

1st pass 1 63 27 72 64 58 14 9

2nd pass 1 9 27 72 64 58 14 63

3rd pass 1 9 14 72 64 58 27 63

…

Selection sort example 2

19

Selection sort codepublic static void selectionSort(int[] a) { for (int i = 0; i < a.length; i++) { // find index of smallest element int min = i; for (int j = i + 1; j < a.length; j++) if (a[j] < a[min]) min = j; // O(1)

// swap smallest element with a[i] swap(a, i, min); // O(1) }}

20

Selection sort runtime

Running time for input size n: in practice, a bit faster than bubble sort.

Why?

)(O

2

)1(

)(

)1)1((1

2

1

0

1

0

1

0

1

0 1

n

nn

i

in

in

n

i

n

i

n

i

n

i

n

ij

21

4. Insertion sort Insertion sort orders a list of values by

repetitively inserting a particular value into a sorted subset of the list

More specifically: consider the first item to be a sorted sublist

of length 1 insert the second item into the sorted sublist,

shifting the first item if needed insert the third item into the sorted sublist,

shifting the other items as needed repeat until all values have been inserted into

their proper positions

22

Insertion sort

Simple sorting algorithm. n-1 passes over the array At the end of pass i, the elements that

occupied A[0]…A[i] originally are still in those spots and in sorted order.

2 8 15 1 17 10 12 5

0 1 2 3 4 5 6 7

1 2 8 15 17 10 12 5

0 1 2 3 4 5 6 7

afterpass 2

afterpass 3

23

Insertion sort example

24

Names: Fred, Alice, David, Bill, and Carol

Insertion sort example 2

25

Insertion sort example 2, cont.

26

Insertion sort example 2, cont.

27

Worst-case for insertion sort

28

Insertion sort codepublic static void insertionSort(int[] a) { for (int i = 1; i < a.length; i++) { int temp = a[i];

// slide elements down to make room for a[i] int j; for (j = i; j > 0 && a[j - 1] > temp; j--) a[j] = a[j - 1];

a[j] = temp; }}

29

Insertion sort runtime worst case: reverse-ordered elements in array.

best case: array is in sorted ascending order.

average case: each element is about halfway in order.

)(

2

)1()1(...321

2

1

1

nO

nnni

n

i

)(111

1

nOnn

i

)(

4

)1())1(...321(

2

1

22

1

1

nO

nnn

in

i

30

Comparing sorts We've seen "simple" sorting algos. so far, such

as: selection sort insertion sort

They all use nested loops and perform approximately n2 comparisons

They are relatively inefficient

comparisons swaps

selection n2/2 n

insertionworst: n2/2best: n

worst: n2/2best: n

31

Average case analysis

Given an array A of elements, an inversion is an ordered pair (i, j) such that i < j, but A[i] > A[j]. (out of order elements)

Assume no duplicate elements. Theorem: The average number of

inversions in an array of n distinct elements is n (n - 1) / 4.

Corollary: Any algorithm that sorts by exchanging adjacent elements requires O(n2) time on average.

32

Merge sort Merge sort orders a list of values by

recursively dividing the list in half until each sub-list has one element, then recombining

More specifically: divide the list into two roughly equal parts recursively divide each part in half, continuing

until a part contains only one element merge the two parts into one sorted list continue to merge parts as the recursion unfolds

a "divide and conquer" algorithm

33

Merge sort idea: Divide the array into two halves. Recursively sort the two halves (using merge sort). Use merge to combine the two arrays.

sort sortmerge(0, n/2, n-1)

mergeSort(0, n/2-1) mergeSort(n/2, n-1)

Merge sort

34

674523 14 6 3398 42

35

674523 14 6 3398 42

674523 14 6 3398 42

36

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

37

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398

38

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398

Merge

39

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398

23

Merge

40

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398

23 98

Merge

41

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

23 98

42

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

Merge

23 98

43

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

14

Merge

23 98

44

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

45

Merge

23 98 14

45

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

Merge

98 451423

46

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

Merge

98 14

14

23 45

47

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

Merge

23 14

14 23

98 45

48

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

Merge

23 98 4514

14 23 45

49

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

Merge

23 98 4514

14 23 45 98

50

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

23 98 4514

14 23 45 98

51

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676

23 98 4514

14 23 45 98

52

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676

Merge

23 98 4514

14 23 45 98

53

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676

6

Merge

23 98 4514

14 23 45 98

54

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676

67

Merge

23 98 4514 6

14 23 45 98

55

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

23 98 4514 676

14 23 45 98

56

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 676

14 23 45 98

57

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

3323 98 4514 676

14 23 45 98

58

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

4223 98 4514 676 33

14 23 45 98

59

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 676 4233

14 23 45 98

60

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 6 4233

14 23 45 98 6

67

61

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 6 33

14 23 45 98 6 33

67 42

62

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 6 4233

14 23 45 98 6 33 42

67

63

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 676 4233

14 23 45 98 6 33 42 67

64

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 676 4233

23 45 98 33 42 6714 6

65

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 676 4233

23 45 98 6 42 67

6

14 33

66

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 676 4233

14 45 98 6 42 67

6 14

23 33

67

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 676 4233

14 23 98 6 42 67

6 14 23

45 33

68

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 676 4233

14 23 98 6 33 67

6 14 23 33

45 42

69

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 676 4233

14 23 98 6 33 42

6 14 23 33 42

45 67

70

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 676 4233

14 23 45 6 33 42

6 14 23 33 42 45

98 67

71

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 676 4233

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67

72

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

Merge

23 98 4514 676 4233

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67 98

73

674523 14 6 3398 42

674523 14 6 3398 42

4523 1498

2398 45 14

676 33 42

676 33 42

23 98 4514 676 4233

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67 98

74

674523 14 6 3398 42

6 14 23 33 42 45 67 98

75

13 6 21 18 9 4 8 20

0 7

4 6 8 9 13 18 20 21

0 7

13 6 21 18

0 3

9 4 8 20

4 7

6 13 18 21

0 3

4 8 9 20

4 7

13 6

0 1

21 18

2 3

9 4

4 5

8 20

6 7

6 13

0 1

18 21

2 3

4 9

4 5

8 20

6 7

13

0

6

1

21

2

18

3

9

4

4

5

8

6

20

7

Merge sort illustrated

76

merge operation: Given two sorted arrays, merge operation produces

a sorted array with all the elements of the two arrays

A 6 13 18 21 B 4 8 9 20

C 4 6 8 9 13 18 20 21

Running time of merge: O(n), where n is the number of elements in the merged array.

Merging two sorted arrays

77

Merge sort codepublic static void mergeSort(int[] a) { int[] temp = new int[a.length]; mergeSort(a, temp, 0, a.length - 1);}

private static void mergeSort(int[] a, int[] tmp, int left, int right) { if (left >= right) return;

// sort the two halves int mid = (left + right) / 2; mergeSort(a, tmp, left, mid); mergeSort(a, tmp, mid + 1, right);

// merge the sorted halves into a sorted whole merge(a, tmp, left, mid, right);}

78

Merge codeprivate static void merge(int[] a, int[] temp, int left, int leftEnd, int rightEnd) { int count = rightEnd - left + 1; int right = leftEnd + 1; // main loop to copy the halves into the temp array for (int i = 0, l = left, r = right; i < count; i++) if (r > rightEnd) temp[i] = a[l++]; else if (l > leftEnd) temp[i] = a[r++]; else if (a[l] < a[r]) temp[i] = a[l++]; else temp[i] = a[r++];

// copy sorted temp array back into main array for (int i = 0; i < count; i++) a[left + i] = temp[i];}

79

Merge sort runtime

let T(n) be runtime of merge sort on n items T(0) = 1 T(1) = 2*T(0) + 1 T(2) = 2*T(1) + 2 T(4) = 2*T(2) + 4 T(8) = 2*T(4) + 8 ... T(n/2) = 2*T(n/4) + n/2 T(n) = 2*T(n/2) + n

80

Merge sort runtime T(n) = 2*T(n/2) + n T(n/2) = 2*T(n/4) + n/2

T(n) = 2*(2*T(n/4) + n/2) + n T(n) = 4*T(n/4) + 2n T(n) = 8*T(n/8) + 3n ... T(n) = 2k T(n/2k) + kn

To get to a base case, let's set k = log2 n. T(n) = 2log n T(n/2log n) + (log n) n T(n) = n * T(n/n) + n log n T(n) = n * T(1) + n log n T(n) = n * 1 + n log n T(n) = n + n log n T(n) = O(n log n)

81

Quick sort

Quick sort orders a list of values by partitioning the list around one element called a pivot, then sorting each partition

More specifically: choose one element in the list to be the pivot

(= partition element) organize the elements so that all elements

less than the pivot are to the left and all greater are to the right

apply the quick sort algorithm (recursively) to both partitions

82

Quick sort, continued

For correctness, can choose any pivot. For efficiency, one of following is best

case, the other worst case: pivot partitions the list roughly in half pivot is greatest or least element in list

Which case above is best? We will divide the work into two methods:

quickSort – performs the recursive algorithm partition – rearranges the elements into

two partitions

83

Quick sort pseudo-code

Let S be the input set.1. If |S| = 0 or |S| = 1, then return.2. Pick an element v in S. Call v the pivot.3. Partition S – {v} into two disjoint groups:

• S1 = {x S – {v} | x v}

• S2 = {x S – {v} | x v}

4. Return { quicksort(S1), v, quicksort(S2) }

84

40

10

1832

235

37

176

12

pick a pivot

610 12 2

1718

40 3732 35

partition

quicksort quicksort

1810 12 1762 40373532

combine

1810 12 1762 40373532

Quick sort illustrated

85

How to choose a pivot first element

bad if input is sorted or in reverse sorted order bad if input is nearly sorted variation: particular element (e.g. middle

element)

random element even a malicious agent cannot arrange a bad

input

median of three elements choose the median of the left, right, and center

elements

86

Partitioning algorithm

Basic idea:1. Move the pivot to the rightmost position.2. Starting from the left, find an element

pivot. Call the position i.3. Starting from the right, find an element

pivot. Call the position j.4. Swap S[i] and S[j].

8 1 4 9 0 3 5 2 7 6

0 9

87

Quick sort codeprivate static void quickSort(int[] a, int min, int max) { if (min >= max) // base case; no need to sort return;

// choose pivot (we'll use the middle element) int mid = (min + max) / 2; int pivot = a[mid]; swap(a, mid, max); // move pivot to end

// partition the two sides of the array int i = partition(a, min, max - 1, pivot);

// restore the pivot to its proper location swap(a, max, i);

// recursively sort the left and right partitions quickSort(a, min, i - 1); quickSort(a, i + 1, max);}

88

Quick sort code, cont'd.// partitions a with elements < pivot on left and// elements > pivot on rightprivate static int partition(int[] a, int i, int j, int pivot) {

i--; j++; // kludge because the loops pre-increment while (true) { // move index markers i,j toward center // until we find a pair of mis-partitioned elements do { i++; } while (i < j && a[i] < pivot); do { j--; } while (i < j && a[j] > pivot);

if (i >= j) break;

swap(a, i, j); }

return i;}

89

9 17 3 12 8 7 21 1

0 7

1 17 3 12 8 7 21 9

0 7

1 7 3 12 8 17 21 9

0 7

1 7 3 8 12 17 21 9

0 7

1 7 3 8 9 17 21 12

0 7

pick pivot

i

i

j

j

j i swap S[i]with S[right]

"Median of three" pivot

90

Special cases

What happens when the array contains many duplicate elements?

What happens when the array is already sorted (or nearly sorted) to begin with?

Small arrays Quicksort is slower than insertion sort when is

N is small (say, N 20). Optimization: Make |A| 20 the base case

and call insertion sort.

91

Quick sort runtime Worst case: pivot is the smallest (or largest)

element all the time.T(n) = T(n-1) + cnT(n-1) = T(n-2) + c(n-1)T(n-2) = T(n-3) + c(n-2)…T(2) = T(1) + 2c

Best case: pivot is the medianT(n) = 2 T(n/2) + cn

T(n) = cn log n + n = O(n log n)

)O(N i c T(1)T(N) 2N

2i

92

Quick sort runtime, cont'd.

Assume each of the sizes for S1 are equally likely. 0 |S1| N-1.

22N

0i

21N

0i

1N

0i

1N

0i

1)-c(N T(i)21)T(N 1)(N

cN T(i)2 T(N) N

cNT(i)N

2

cN 1)]-i-T(N T(i)[N

1 T(N)

93

2cN 1)T(N 1)(NT(N) N

1N

2c

N

1)T(N

1N

T(N)

N

2c

1-N

2)T(N

N

1)-T(N

1N

2c

2-N

3)T(N

1N

2)-T(N

…

3

2c

2

)1T(

3

T(2)

1N

3i i

12c

2

T(1)

1N

T(N)

N) log O(NT(N) loge (N+1) – 3/2

divideequationby N(N+1)

Quick sort runtime, cont'd.

94

Quick sort runtime summary

O(n log n) on average. O(n2) worst case.

comparisons

merge O(n log n)

quickaverage: O(n log n)worst: O(n2)

95

Selection problem, quickSelect

Recall: the Selection problem Find the kth smallest element in an array a

quickSelect(a, k):1. If a.length = 1, then k=1 and return the element.2. Pick a pivot v a.3. Partition a – {v} into a1 (left side) and a2 (right side).

• if k a1.length, then the kth smallest element must be in a1. So return quickSelect(a1, k).

• else if k = 1 + a1.length, return the pivot v.

• Otherwise, the kth smallest element is in a2. Return quickSelect(a2, k - a1.length - 1).

96

Quick select runtime

1N

0i

cNT(i)N

1T(N)

c- 2cN 1)T(N NT(N) N

2c 1)T(N T(N)

2c 2)T(N 1)-T(N

2c 3)T(N 2)-T(N

…2c T(1) T(2)

2c 1)-(N T(1) T(N) = O(N)

average case:

worst case: same as Quick sort, O(N2)

Date post:	20-Dec-2015
Category:	Documents
View:	216 times
Download:	2 times

1 TCSS 342, Winter 2005 Lecture Notes Sorting Weiss Ch. 8, pp. 283-321.

Documents