Been-Chian Chien, Wei-Pang Yang, and Wen-Yang Lin 7-1 Chapter 7 Sorting Introduction to Data...

7-1Been-Chian Chien, Wei-Pang Yang, and Wen-Yang Li

n

Chapter 7 Sorting

Introduction to Data Structure

CHAPTER 7

SORTING

7.1 Searching and List Verification7.2 Definitions7.3 Insertion Sort7.4 Quick Sort7.5 Optimal Sorting Time7.6 Merge Sort7.7 Heap Sort7.8 Radix Sort


n

Chapter 7 Sorting

Chapter 1 Basic Concepts

Chapter 2 Arrays

Chapter 3 Stacks and Queues

Chapter 4 Linked Lists

Chapter 5 Trees

Chapter 6 Graph

Chapter 7 Sorting

Chapter 8 Hashing

Chapter 9 Heap Structures

Chapter 10 Search Structures

Contents


n

Chapter 7 Sorting

7.1 Searching and List Verification Motivation of Sorting

The term list here is a collection of records. Each record has one or more fields. Each record has a key to distinguish one record with

another. For example, the phone directory is a list. Name, phone

number, and even address can be the key, depending on the application or need.

Key (Student_ID)

Field (Dept.)


n

Chapter 7 Sorting

Sequential Searching Two ways to store a collection of records

Sequential Non-sequential

Assume a sequential list f. To retrieve a record with key f[i].key from such a list, we can do search in the following order:

f[n].key, f[n-1].key, …, f[1].key => sequential search


n

Chapter 7 Sorting

Sequential Searching (cont.)

int SeqSearch (int list[], int searchnum, int n)

{

int i;

list[n] = searchnum;

for (i = 0; list[i] != searchnum; i++);

return (i < n) ? i: -1);

}

Program 7.1, p. 321

The average number of comparisons for a successful search is

ni

nnin1

2/)1(/)1(


n

Chapter 7 Sorting

Basic conceptCompare searchnum and list[middle].key searchnum < list[middle].key

search list[0] ~ list[middle-1] searchnum = list[middle].key

return TRUE searchnum > list[middle].key

search list[middle+1] ~ list[n-1]

Program 7.2, p. 322

Complexity A binary search only takes O(log n) time to search a sequential

list with n records.

Binary Search


n

Chapter 7 Sorting

List Verification

Definition Given two lists, verification if both are the same.

list1 = list2?

Example: Tax verification The IRS gets salary reports from employers. IRS also gets tax filing reports from employees about their sala

ry. Need to verify the two numbers match for each individual empl

oyee.

Two methods Method 1: Random verification Method 2: Ordered verficiation


n

Chapter 7 Sorting

Random Verification: Unsorted Lists

void verify1(element list1[], element list2[], int n, int m)/* Compare two unordered lists list1 and list2 */{

int i, j; int marked [MAX_SIZE];

for (i = 0; i < m; i++) marked[i] = FALSE;for (i = 0; i< n; i++) if ((j == seqsearch(list2, m, list1[i].key()) < 0)

printf(“%d is not in list 2\n”, list1[i].key);

else /* check each of the other fields from list1[i] and

list2[j], and print out any discrepancies */ marked[j] = TRUE; for (i = 1; i < m; i++) if (!marked[i])

printf(“%d is not in list 1\n”, list2[i].key);}

Complexity: O(mn)

Why?


n

Chapter 7 Sorting

Sorted Verifying: Sorted Lists

void verify2(element list1[], element list2[], int n, int m){

int i, j; sort(list1, n);

sort(list2, m);i = j = 0;while (i < n && j < m) if (list1[i].key < list2[j].key) {

printf(“%d is not in list 2\n”, list1[i].key); i++;

} else if (list1[i].key == list2[j].key) { /* compare list1[i] and list2[j] on each of the other fields and report any discrepancies */ i++; j++;

} else { printf(“%d is not in list 1\n”, list2[j].key);

j++; }

for (; i < n; i++) printf(“%d is not in list 2\n”, list1[i].key; for (; j < m; i++) printf(“%d is not in list 2\n”, list1[i].key;

}

Complexity: O(max{n log n, m log m})


n

Chapter 7 Sorting

7.2 Definition

Formal definition Given a list of records (R0, R1, …, Rn-1), each with a key Ki.

The sorting problem is to find permutation, σ, such that

Kσ(i) ≤ Kσ(i+1) , 1 ≤ i ≤ n – 1.

The desired ordering is (Rσ(1), Rσ(2), Rσ(n)).

If a list has several key values that are identical, the permutation, σs, is not unique.

Let σs be the permutation of the following properties:

(1) [sorted] Kσ(i) ≤ Kσ(i+1) , 1 ≤ i ≤ n – 1

(2) [stable] If i < j and Ki == Kj in the input list, then Ri precedes Rj in the

sorted list. The above sorting method that generates σs is stable.


n

Chapter 7 Sorting

Stable Sorting

Example

Stable permutation

Unstable permutation

3 2 41 7 42 5

0 1 2 3 4 5Key

a b c d e fRecord

0 1 2 3 4 5

1 0 2 5 3 4

2 3 41 42 5 7

0 1 2 3 4 5Key

b a c e f dRecord

2 3 42 41 5 7

0 1 2 3 4 5Key

b a e c f dRecord

0 1 2 3 4 5

1 0 3 5 2 4


n

Chapter 7 Sorting

Category of Sorting Methods

Internal method Methods to be used when the list to be sorted is small enough so

that the entire sort list can be carried out in the main memory. Example

• Insertion sort

• Quick sort

• Merge sort

• Heap sort

• Radix sort

External method Methods to be used on larger lists


n

Chapter 7 Sorting

Basic concept

7.3 Insertion Sort

a1 a2 a3 … ak ai

sorted

insert

a1 a2 ai a3 … ak

sorted

1 3 5 7 9 4

1 3 4 5 7 9


n

Chapter 7 Sorting

Insertion Sort: Program

Program 7.5, p 327

void insertion_sort(element list[], int n)

{

int i, j;

element next;

for (i = 1; i < n; i++) {

next = list[i];

for (j = i – 1; j >= 0 && next.key < list[j].key; j--)

list[j+1] = list[j];

list[j+1] = next;

}

}

1

0

2 )()(

:Complexityn

i

nOiO


n

Chapter 7 Sorting

Insertion Sort: Example

Record Ri is left out of order (LOO) iff Ri < Example 1

Assume n = 5 and the input key sequence is 5, 4, 3, 2, 1

}{max1

jij

R

j [1] [2] [3] [4] [5]

- 5 4 3 2 1

2 4 5 3 2 1

3 3 4 5 2 1

4 2 3 4 5 1

5 1 2 3 4 5


n

Chapter 7 Sorting

Insertion Sort: Example

Example 2 Assume n = 5 and the input key sequence is 2, 3, 4, 5, 1

j [1] [2] [3] [4] [5]

- 2 3 4 5 1

2 2 3 4 5 1

3 2 3 4 5 1

4 2 3 4 5 1

5 1 2 3 4 5

O(1)

O(1)

O(1)

O(n)


n

Chapter 7 Sorting

Insertion Sort

Analysis If there are k LOO records in a list, the computing ti

me for sorting the list via insertion sort is

O((k+1)n) = O(kn)

Therefore, if k << n, then insertion sort might be a good sorting choice.


n

Chapter 7 Sorting

7.4 Quick Sort

Quick sort is developed by C. A. R. Hoare. Quick sort has the best average behavior among the sorting

methods. Basic concept

Based on the divide and conquer paradigm Choose an element (pivot) p, i.e., p = K0

Place p into a proper position j such that

• K0 ~ Kj1 p

• Kj1 ~ Kn1 > p

K0 K1 … Kj-1 Kj = p Kj+1 Kj+2 … Kn-1


n

Chapter 7 Sorting

Quick Sort (cont.)

Example

25 57 48 37 12 92 86 33

(12) 25 (57 48 37 92 86 33)

12 25 (48 37 33) 57 (92 86)

12 25 (37 33) 48 57 (92 86)

12 25 33 37 48 57 (92 86)

12 25 33 37 48 57 86 92


n

Chapter 7 Sorting

Quick Sort (cont.)

Key mechanism: a partition method Algorithm

quicksort(list, left, right)

{

if (left < right) {partition(list, left, right, j);quicksort(list, left, j1);quicksort(list, j+1, right);

}

}


n

Chapter 7 Sorting

Quick Sort: Partition

Concept of partition method Let pivot = list[left] be the pivot Use two pointers, i and j

• i until list[i] pivot;

• j until list[j] pivot; list[i] list[j] if i < j


n

Chapter 7 Sorting

Quick Sort: Partition (cont.)

Example of partition method 25 57 48 37 12 92 86 33

25 12 48 37 57 92 86 33

(12) 25 (48 37 57 92 86 33)

i j

i j

ij


n

Chapter 7 Sorting

Quick Sort: Program Codes

void quicksort(element list[], int left, int right){ int pivot, i, j; element temp; if (left < right) {

i = left; j = right + 1; pivot = list[left].key; do {

do i++; while (list[i].key < pivot); do j--; while (list[j].key > pivot); if (i < j) SWAP(list[i], list[j], temp);

} while (i < j); SWAP(list[left], list[j], temp); quicksort(list, left, j–1); quicksort(list, j+1, right);

}}

partition


n

Chapter 7 Sorting

Quick Sort: Example

Example Input list: 10 records with keys (26, 5, 37, 1, 61, 11, 59, 15, 48, 19).

K0 K1 K2 K3 K4 K5 K6 K7 K8 K9 Left Right

[26 5 37 1 61 11 59 15 48 19 1 10

[11 5 19 1 15] 26 [59 61 48 37] 1 5

[1 5] 11 [19 15] 26 [59 61 48 37] 1 2

1 5 11 [19 15] 26 [59 61 48 37] 4 5

1 5 11 15 19 26 [59 61 48 37] 7 10

1 5 11 15 19 26 [48 37] 59 [61] 7 8

1 5 11 15 19 26 37 48 59 [61] 10 10

1 5 11 15 19 26 37 48 59 61


n

Chapter 7 Sorting

Quick Sort: Analysis

Analysis of QuickSort Worse case: O(n2) Average case:

• Assume each time a record is correctly positioned

|left sublist| = |right sublist|

• Let T(n) be the time taken to sort a list of size n

T(n) ≤ cn + 2T(n/2), for some constant c ≤ cn + 2(cn/2 +2T(n/4))

≤ 2cn + 4T(n/4)::

≤ cn log2n + T(1) = O(n logn)


n

Chapter 7 Sorting

Quick Sort Variant

Quick sort using a median of three Pick the median of the first, middle, and last keys in the

current sublist as the pivot. Thus,

pivot = median{Kleft, K(left+right)/2, Kright}.


n

Chapter 7 Sorting

7.5 Optimal Sorting Time

Question

How quickly can we sort a list of n objects? Answer

If only operations permitted on keys are comparisons and interchanges, then O(n logn) is the best possible time.

Method This is done by using a tree called decision tree that de

scribes the sorting process. Each vertex of the tree represents a key comparison, an

d the branches indicate the result.


n

Chapter 7 Sorting

Decision Tree for Insertion Sort

K1 ≤ K2

stop K1 ≤ K3

K2 ≤ K3 K1 ≤ K3

stop K2 ≤ K2

stop stop stop stop

Yes No

Yes

Yes

Yes

Yes

No

No No

No

I

II III

IV

V VI

[0, 1, 2]

[1, 0, 2]

[1 , 2, 0]

[1 , 2, 0] [2, 1 , 0]

[0, 1, 2]

[0, 1, 2] [0, 2, 1]

[2, 0, 1][0, 2, 1]

[1, 0, 2]


n

Chapter 7 Sorting

Decision Tree (cont.)

Theorem 7.1: Any decision tree that sorts n distinct elements has a height of at least log2(n!) + 1

Corollary: Any algorithm that sorts only by comparisons must have a worst-case computing time of Ω(n log n)


n

Chapter 7 Sorting

7.6 Merge Sort

Kernel operation of Merge Sort: Merging

Given two sorted list, merge them into a single sorted list

Example

25 37 48 57

12 33 86 92

=> 12 25 33 37 48 57 86 92

Methods for merging

Simple merge

O(1) space merge


n

Chapter 7 Sorting

Simple Merge

void merge(element list[], element sorted[], int i, int m, int n)/* merge list[i],…,list[m], and list[m+1],…,list[n] */{ int j, k, t; j = m+1; k = i; while (i <= m && j <= n) { if (list[i].key <= list[j].key)

sorted[k++] = list[i++]; else

sorted[k++] = list[j++]; } if (i > m) for (t = j; t <= n; t++) sorted[k+t-j] = list[t]; else for (t = i; t <= m; t++) sorted[k+t-i] = list[t];}

Time & space complexity:O(n i + 1)


n

Chapter 7 Sorting

O(1) Space Merge

A merge algorithm only requires O(1) additional space

Assumption

The total number of records n is a perfect square

The numbers of records in the left sublist and the right sublist are multiple of n


n

Chapter 7 Sorting

O(1) Space Merge

AlgorithmStep 1: Identify the records with largest keys. This is done by followi

ng right to left along the two lists to be merged.

Step 2: Exchange records of the second list identified in Step 1 with those just to the left of those identified from the first list.

Step 3: Swap the block of largest with the leftmost block (unless it is already the leftmost block). Sort the rightmost block.

Step 4: Reorder the blocks, excluding the block of largest records, into nondecreasing order of the last key in the blocks.

Step 5: Perform as many merge substeps as needed to merge the blocks, other than the block with the largest keys.

Step 6: Sort the block with the largest keys.

n

n


n

Chapter 7 Sorting

O(1) Space Merge: Example

0 2 4 6 8 a c e g i j k l m n t w z|1 3 5 7 9 b d f h o p q r s u v x y

0 2 4 6 8 a c e g i j k l m n t w z 1 3 5 7 9 b d f h o p q r s u v x y

0 2 4 6 8 a|c e g i j k|u v x y w z|1 3 5 7 9 b|d f h o p q|r s l m n t

u v x y w z|c e g i j k|0 2 4 6 8 a|1 3 5 7 9 b|d f h o p q|l m n r s t

u v x y w z 0 2 4 6 8 a|1 3 5 7 9 b|c e g i j k|d f h o p q|l m n r s t

0 v x y w z u 2 4 6 8 a|1 3 5 7 9 b|c e g i j k|d f h o p q|l m n r s t

0 1 x y w z u 2 4 6 8 a|v 3 5 7 9 b|c e g i j k|d f h o p q|l m n r s t

0 1 2 y w z u x 4 6 8 a|v 3 5 7 9 b|c e g i j k|d f h o p q|l m n r s t


n

Chapter 7 Sorting

O(1) Space Merge: Example (cont.)

0 1 2 3 4 5 u x w 6 8 a|v y z 7 9 b|c e g i j k|d f h o p q|l m n r s t

0 1 2 3 4 5 6 7 8 u w a|v y z x 9 b|c e g i j k|d f h o p q|l m n r s t

0 1 2 3 4 5 6 7 8 9 a w|v y z x u b|c e g i j k|d f h o p q|l m n r s t

0 1 2 3 4 5 6 7 8 9 a w v y z x u b c e g i j k|d f h o p q|l m n r s t

0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k v z u|y x w o p q|l m n r s t

0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k v z u y x w o p q|l m n r s t

0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q y x w|v z u r s t

0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t|v z u y x w


n

Chapter 7 Sorting

O(1) Space Merge: Analysis

Steps 1 and 2 O( ) time and O(1) space

Step 3 Swapping: O( ) time and O(1) space Sorting: O(n) time and O(1) space (via insertion sort)

Step 4 O(n) time and O(1) space (via selection sort)

• Selection sort sorts m records using O(m2) key comparisons and O(m) record moves.

O(n1.5) time and O(1) space (via insertion sort)

• Insertion sort needs O(m2) record moves ( records per block * n record moves).

n

n

n


n

Chapter 7 Sorting

O(1) Space Merge: Analysis (cont.)

Step 5 Merge substeps: The total number of is at most . T

he total time for is O(n).

Step 6 The sort of can be done in O(n) by using either a selection sort or

an insertion sort.

In total O(n) time and O(1) space

1n


n

Chapter 7 Sorting

Iterative Merge Sort

Concept Treat the input as n sorted lists, each of length 1. Lists are merged by pairs to obtain n/2 lists, each of

size 2 (if n is odd, the one list is of length 1). The n/2 lists are then merged by pairs, and so on until

we are left with only one list.


n

Chapter 7 Sorting

Iterative Merge Sort: Example

26 5 77 1 61 11 59 15 48 19

5 26 1 77 11 61 15 59 19 48

1 5 26 77 11 15 59 61 19 48

1 5 11 15 26 59 61 77 19 48

1 5 11 15 19 26 48 59 61 77


n

Chapter 7 Sorting

Iterative Merge Sort: Analysis

Program code Program 7.9 and 7.10

Time complexity Total of passes are made over the data Each pass of merge sort takes O(n) time The total of computing time is O(n log n)

n2log


n

Chapter 7 Sorting

Recursive Merge Sort

Concept Divide the list to be sorted into two roughly equal parts:

• left sublist [left : (left+right)/2]

• right sublist [(left+right)/2 +1 : right]

Sort each sublist recursively, and merge the sorted sublists

To avoid copying, the use of a linked list (integer instead of real link) for sublist is desirable.

Program code Program 7.11 and 7.12, complexity: O(n log n)


n

Chapter 7 Sorting

Recursive Merge Sort: Example

26 5 77 1 61 11 59 15 48 19

5 26 11 59 19 48

5 26 77 11 15 59 19 48

1 5 26 61 77 11 15 19 48 59

1 5 11 15 19 26 48 59 61 77

1 61


n

Chapter 7 Sorting

Natural Merge Sort

Concept It takes advantage of the prevailing order within the lis

t before performing merge sort

It runs an initial pass over the data to determine the sublists of records that are in order

Then it uses the sublists for the merge sort


n

Chapter 7 Sorting

Natural Merge Sort: Example

26 5 77 1 61 11 59 15 48 19

1 11 59 61 15 19 485 26 77

1 5 11 26 59 61 77 15 19 48

1 5 11 15 19 26 48 59 61 77


n

Chapter 7 Sorting

7.7 Heap Sort

Preliminary Merge sort needs O(n) additional storage space, even though its c

omputing time is O(n log n)

Merge sort using O(1) merge only needs O(1) space but the sorting algorithm is much slower

Heap sort

• only requires a fixed amount of additional storage

• achieves worst-case and average computing time O(n log n)


n

Chapter 7 Sorting

Heap Sort (cont.)

Concept Adopt the max-heap structure Consists of two phases Phase 1: create the heap

• Insert the n records into an empty heap Phase 2: adjust the heap

• Exchange the max element with the current last element and perform adjustment


n

Chapter 7 Sorting

Heap Sort: Program Code

void heapsort (element list[], int n){ int i, j; element temp;

for (i = n/2; i > 0; i--) /* Phase 1 */ adjust(list, i, n); for (i = n-1; i > 0; i--) { /* Phase 2 */ SWAP(list[1], list[i+1], temp); adjust(list, 1, i); }}

Complexity:

O(n log n)


n

Chapter 7 Sorting

Heap Sort: Program Code (cont.)

void adjust (element list[], int root, int n){ int child, rootkey; element temp = list[root]; rootkey = list[root].key; child = 2*root; while (child <= n) { if (child < n) && (list[child].key < list[child+1].key)) child++; if (rootkey > list[child].key) break; else { list[child/2] = list[child]; child *= 2; } } list[child/2] = temp;}


n

Chapter 7 Sorting

Heap Sort: Example

26

15 48 19

5

1 61

77

11 59

77

15 1 5

61

48 19

59

11 26

[1]

[2]

[3]

[4]

[5]

[6] [7]

[8] [9] [10]

[2]

[3]

[4]

[5]

[6] [7]

[8] [9] [10]

(a) Input array

(b) Initial heap

[1]


n

Chapter 7 Sorting

Heap Sort: Example (cont.)

61

5 1

48

15 19

59

11 26

[2]

[3]

[4]

[5]

[6] [7]

[8] [9]

[1]

Heap size = 9, Sorted = [77]

59

5

48

15 19

26

11 1

[2]

[3]

[5]

[6] [7]

[8]

[1]

Heap size = 8, Sorted = [61, 77]


n

Chapter 7 Sorting

Heap Sort: Example (cont.)

61

5 1

48

15 19

59

11 26

[2]

[3]

[4]

[5]

[6] [7]

[8] [9]

[1]

Heap size = 9, Sorted = [77]

59

5

48

15 19

26

11 1

[2]

[3]

[5]

[6] [7]

[8]

[1]

Heap size = 8, Sorted = [61, 77]


n

Chapter 7 Sorting

7.8 Radix Sort

Sorting on multiple keys A list of records are said to be sorted with respect to the keys K0, K2, …,

Kr-1 iff

• for every pair of records i and j, i < j and

• (K0i, K1

i, …, Kr-1i) ≤ (K0

j, K1j, …, Kr-1

j).

(x0, x2, …, xr-1) ≤ (y0, y2, …, yr-1) iff

• either xi = yi, 0 ≤ i ≤ j, and xj+1 < yj+1 for some j < r-1

• or xi = yi , 0 ≤ i < r

Example, sorting a deck of cards: suite and face value.

• K0 [suit]: ♣ < ♦ < ♥ < ♠

• K1 [Face value]: 2 < 3 < 4 … < 10 < J < Q < K < A

• A possible ordering

2♣, … A♣, 2♦, …, A♦, 2♥, …, A♥, 2♠, …, A♠

Most significant


n

Chapter 7 Sorting

Radix Sort (cont.)

Two popular ways to sort on multiple keys MSD: sort on the most significant key into multiple piles LSD: sort on the least significant digit first

LSD and MSD only defines the order in which the keys are to be sorted

LSD and MSD can be used even when there is only one key

E.g., if the keys are numeric, then each decimal digit may be regarded as a subkey

=> Radix sort


n

Chapter 7 Sorting

Radix Sort: Example

179 208 306 93 859 984 55 9 271 33

list[0]

list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9]

e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9]

f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9]

179

859

9

2083065598493

33

271

271 93 33 984 55 306 208 179 859 9

list[0]



n

Chapter 7 Sorting

Radix Sort: Example (cont.)

e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9]

f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9]

179859

9

208

306 55 984 9333 271

271 93 33 984 55 306 208 179 859 9

list[0]


306 208 9 33 55 859 271 179 984 93

list[0]



n

Chapter 7 Sorting

Radix Sort: Example (cont.)

e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9]

f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9]

179

55

33

9 859 984306

271

9 33 55 93 179 208 271 306 859 948

list[0] list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9]

306 208 9 33 55 859 271 179 984 93

list[0]


93

208


n

Chapter 7 Sorting

Radix Sort: Analysis

Let d: the number of digits r: radix size n: number of records

Time complexity O(d(n+r)) Usually r << n, so O(dn)

Date post:	26-Dec-2015
Category:	Documents
Upload:	rachel-pearson
View:	213 times
Download:	0 times

Been-Chian Chien, Wei-Pang Yang, and Wen-Yang Lin 7-1 Chapter 7 Sorting Introduction to Data...

Documents