Searching and Sorting - Tiziana Ligorio · Searching and Sorting Tiziana Ligorio...

Searching and SortingTiziana Ligorio

[email protected]

!1

mailto:[email protected]

Today’s Plan

Recap

Searching algorithms and their analysis

Sorting algorithms and their analysis

!2

Announcements

Questions?

!3

Searching

!4

Looking for something!In this discussion we will assume

searching for an element in an array

Linear searchMost intuitiveStart at first position and keep looking until you find it

int linearSearch(int a[], int size, int value){

for (int i = 0; i < size; i++) { if (a[i] == value) { return i; } } return-1;}

!5

How long does linear search take?

If you assume value is in the array and probability of finding it at any location is uniform, on average n/2

If value is not in the array (worst case) n

Either way it’s O(n)

!6

What if you know array is sorted? Can you do better than linear search?

!7

Lecture Activity

You are given a sorted array of integers.

How would you search for 115? ( try to do it in fewer than n steps: don’t search sequentially)

You can write pseudocode or succinctly explain your algorithm

!8

We have done this before! When?

!9

!10

Look in ?

Binary Search

!11

14 43 76 100 108 158 195 200 274 523 543 5993

Binary Search

!12

14 43 76 100 108 158 195 200 274 523 543 5993

Binary Search

!13

14 43 76 100 108 158 195 200 274 523 543 5993

Binary Search

!14

14 43 76 100 108 158 195 200 274 523 543 5993

Binary Search

!15

14 43 76 100 108 158 195 200 274 523 543 5993

Binary Search

!16

14 43 76 100 108 158 195 200 274 523 543 5993

Binary Search

!17

14 43 76 100 108 158 195 200 274 523 543 5993

Binary Search

What is happening here?

!18

Binary Search


Size of search is cut in half at each step

!19

Binary Search



Let T(n) be the running time and assume n = 2kT(n) = T(n/2) + 1

!20

Simplification: assume n is a power of 2 so it can be

evenly divided in two partsThe running time

One comparison

Search lower OR upper half

Binary Search



Let T(n) be the running time and assume n = 2kT(n) = T(n/2) + 1 T(n/2) = T(n/4) +1

!21

One comparison

Search lower OR upper half of n/2

Binary Search



Let T(n) be the running time and assume n = 2kT(n) = T(n/2) + 1 T(n/2) = T(n/4) +1 T(n) = T(n/4) + 1 + 1

!22

Binary Search



Let T(n) be the running time and assume n = 2kT(n) = T(n/2) + 1 T(n) = T(n/4) + 2 . . .

!23

22 2

211

Binary Search



Let T(n) be the running time and assume n = 2kT(n) = T(n/2) + 1 T(n) = T(n/4) + 2 . . . T(n) = T(n/2k) + k

!24

Binary Search



Let T(n) be the running time and assume n = 2kT(n) = T(n/2) + 1 T(n) = T(n/4) + 2 . . . T(n) = T(n/2k) + k T(n) = T(1) + log2(n)

!25n/n = 1

The number to which I need to raise 2 to get n

And we said n = 2k

Binary Search



Let T(n) be the running time and assume n = 2kT(n) = T(n/2) + 1 T(n) = T(n/4) + 2 . . . T(n) = T(n/2k) + k T(n) = T(1) + log2(n)

!26

Binary search is O(log(n))

Sorting

!27

Rearranging a sequence into increasing (decreasing) order!

Several approaches

Can do it in may ways

What is the best way?

Let’s find out using Big-O

!28

Lecture Activity

Write pseudocode to sort an array.

!29

144376 100108158 195200 274523543 5993

There are many approaches to sorting We will look at some comparison-

based approaches here

!30

Selection Sort

!31

Selection Sort

!32

Find smallest element and move it at lowest position

Unsorted

Sorted

1st Pass

Selection Sort

!33


Unsorted

Sorted

Swap

1st Pass

Selection Sort

!34


Unsorted

Sorted

1st Pass

Selection Sort

!35


Unsorted

Sorted

Unsorted

2nd Pass

Selection Sort

!36


Unsorted

Sorted

Swap

2nd Pass

Selection Sort

!37


Unsorted

Sorted

2nd Pass

Selection Sort

!38


Unsorted

Sorted

3rd Pass

Selection Sort

!39


Unsorted

Sorted

3rd Pass

Selection Sort

!40


Unsorted

Sorted

4th Pass

Selection Sort

!41


Unsorted

Sorted

4th Pass

Selection Sort

!42


Unsorted

Sorted

5th Pass

Selection Sort

!43


Unsorted

Sorted

Swap

5th Pass

Selection Sort

!44


Unsorted

Sorted

5th Pass

Selection Sort

!45


Unsorted

Sorted

6th Pass

Selection Sort

!46


Unsorted

Sorted

Selection Sort

Find the smallest item and move it at position 1

Find the next-smallest item and move it at position 2

. . .

!47

Selection Sort Analysis

How much work?

Find smallest: look at n elements

!48


How much work?


Find second smallest: look at n-1 elements

!49


How much work?



Find third smallest: look at n-2 elements. . .

!50


How much work?



Find third smallest: look at n-2 elements. . .

Total work: n + (n-1) + (n-2) + . . . +1

!51

!52

n + (n-1) + ... + 2 + 1

n

n + 1

= n(n+1) / 2


T(n) = n(n+1) / 2 comparisons + n data moves = O( )?

!53



T(n) = (n2+n) / 2 + n = O( )?

!54



T(n) = (n2+n) / 2 + n = O( )?

!55

Ignore constant

Ignore non-dominant terms



T(n) = (n2+n) / 2 + n = O( n2)

!56

Ignore constant

Ignore non-dominant terms



T(n) = (n2+n) / 2 + n = O( n2)

Selection Sort run time is O( n2)

!57

template<class T>void selectionSort(T the_array[], int n) { // last = index of the last item in the subarray of items yet // to be sorted; // largest = index of the largest item found for (int last = n - 1; last >= 1; last--) { // At this point, the_array[last+1..n-1] is sorted, and its // entries are greater than those in the_array[0..last]. // Select the largest entry in the_array[0..last] int largest = findIndexOfLargest(the_array, last+1); // Swap the largest entry, the_array[largest], with // the_array[last] std::swap(the_array[largest], the_array[last]); } // end for } // end selectionSort

!58

template<class T>void selectionSort(T the_array[], int n) { // last = index of the last item in the subarray of items yet // to be sorted; // largest = index of the largest item found for (int last = n - 1; last >= 1; last--) { // At this point, the_array[last+1..n-1] is sorted, and its // entries are greater than those in the_array[0..last]. // Select the largest entry in the_array[0..last] int largest = findIndexOfLargest(the_array, last+1); // Swap the largest entry, the_array[largest], with // the_array[last] std::swap(the_array[largest], the_array[last]); } // end for } // end selectionSort

!59

PassO(n)

O(n)

O( n2)

Stability

A sorting algorithm is Stable if elements that are equal remain is same order relative to each other after sorting

!60

Selection Sort

!61


Unsorted

Sorted

Selection Sort

!62


Unsorted

Sorted

Selection Sort

!63


Unsorted

Sorted

Swap

Selection Sort

!64


Unsorted

Sorted

Unstable


Execution time DOES NOT depend on initial arrangement of data => ALWAYS O( n2)

O( n2) comparisons

Good choice for small n and/or data moves are costly ( O( n ) data moves )

Unstable

!65

Understanding O(n2)

!66

14 43100 200 2743

T(n)

Understanding O(n2)

!67

14 43

76

100

108 158195

200 274

523 599

3

14 43100 200 2743

T(2n) ≈ 4T(n)

T(n)

(2n)2 = 4n2

Understanding O(n2)

!68

14 43

76

100

108 158195

200 274

523 599

3

14 43100 200 2743 11260 5642 932

T(n)

T(3n) ≈ 9T(n)

(3n)2 = 9n2

Understanding O(n2) on large input

If size of input increases by factor of 100Execution time increases by factor of 10,000 T(100n) = 10,000T(n)

!69



Assume n = 100,000 and T(n) = 17 seconds Sorting 10,000,000 takes 10,000 longer

!70



Assume n = 100,000 and T(n) = 17 seconds Sorting 10,000,000 takes 10,000 longer

Sorting 10,000,000 entries takes ≈ 2 days

Multiplying input by 100 to go from 17sec to 2 days!!!

!71

Raise your hand if you had Selection Sort

!72

Bubble Sort

!73

Bubble Sort

!74

Compare adjacent elements and if necessary swap them

Unsorted

Sorted

Bubble Sort

!75

Compare adjacent elements and if necessary swap them 1st Pass

Unsorted

Sorted

Bubble Sort

!76


Unsorted

Sorted

Swap

Bubble Sort

!77


Unsorted

Sorted

Bubble Sort

!78


Unsorted

Sorted

Swap

Bubble Sort

!79


Unsorted

Sorted

Bubble Sort

!80


Unsorted

Sorted

Swap

Bubble Sort

!81


Unsorted

Sorted

Bubble Sort

!82


Unsorted

Sorted

Bubble Sort

!83


Unsorted

Sorted

Swap

Bubble Sort

!84


End of1st Pass: Not sorted, but largest has “bubbled up” to its proper

position

Bubble Sort

!85


2nd Pass: Sort n-1

Bubble Sort

!86

Compare adjacent elements and if necessary swap them 2nd Pass

Unsorted

Sorted

Bubble Sort

!87


Unsorted

Sorted

Bubble Sort

!88


Swap

2nd Pass

Unsorted

Sorted

Bubble Sort

!89


Unsorted

Sorted

Bubble Sort

!90


Unsorted

Sorted

Bubble Sort

!91


3rd Pass: Sort n-2

Bubble Sort

!92

Compare adjacent elements and if necessary swap them 3rd Pass

Unsorted

Sorted

Bubble Sort

!93


SwapArray is sorted

But our algorithm doesn’t know It keeps on going

Unsorted

Sorted

Bubble Sort

!94


Unsorted

Sorted

Bubble Sort

!95


Unsorted

Sorted

Bubble Sort

!96


4th Pass: Sort n-3

Bubble Sort

!97

Compare adjacent elements and if necessary swap them 4th Pass

Unsorted

Sorted

Bubble Sort

!98


Unsorted

Sorted

Bubble Sort

!99


5th Pass: Sort n-4

Bubble Sort

!100


Unsorted

Sorted

Bubble Sort

!101

Compare adjacent elements and if necessary swap them Done!

Unsorted

Sorted

Bubble Sort Analysis

How much work?

First pass: n-1 comparisons and at most n-1 swaps

Second pass: n-2 comparisons and at most n-2 swaps

Third pass: n-3 comparisons and at most n-3 swaps. . .

Total work: (n-1) + (n-2) + . . . +1

!102

!103

n + (n-1) + ... + 2 + 1

n

n + 1

= n(n+1) / 2 (n-1) + (n-2) + . . . + 2 + 1 = n(n-1)/2

(n-1)

n


T(n) = n(n-1) / 2 comparisons + n(n-1) / 2 swaps = O( )?

A swap is usually more than one operation but this simplification does not change the analysis

T(n) = 2( n(n-1) / 2 )= O( )?

!104




T(n) = 2( n(n-1) / 2 )= O( )?

T(n) = 2( (n2-n) / 2 )= O( )?

!105




T(n) = 2( n(n-1) / 2 )= O( )?

T(n) = 2( (n2-n) / 2 )= O( )?

T(n) = n2-n = O( )?

!106Ignore non-dominant terms




T(n) = 2( n(n-1) / 2 )= O( )?

T(n) = 2( (n2-n) / 2 )= O( )?

T(n) = n2-n = O( n2)

Bubble Sort run time is O(n2)!107

Optimize!

Easy to check: if there are no swaps in any given pass stop because it is sorted

!108

Bubble Sort

!109


Bubble Sort

!110


Bubble Sort

!111


Bubble Sort

!112


Bubble Sort

!113


Bubble Sort

!114


Bubble Sort

!115


template<class T> void bubbleSort(T the_array[], int n) { bool sorted = false; // False when swaps occur int pass = 1; while (!sorted && (pass < n)) { // At this point, the_array[n+1-pass..n-1] is sorted // and all of its entries are > the entries in the_array[0..n-pass] sorted = true; // Assume sorted for (int index = 0; index < n - pass; index++) { // At this point, all entries in the_array[0..index-1] // are <= the_array[index] int nextIndex = index + 1; if (the_array[index] > the_array[nextIndex]) { // Exchange entries std::swap(the_array[index], the_array[nextIndex]); sorted = false; // Signal exchange } // end if } // end for // Assertion: the_array[0..n-pass-1] < the_array[n-pass] pass++; } // end while } // end bubbleSort

!116

template<class T> void bubbleSort(T the_array[], int n) { bool sorted = false; // False when swaps occur int pass = 1; while (!sorted && (pass < n)) { // At this point, the_array[n+1-pass..n-1] is sorted // and all of its entries are > the entries in the_array[0..n-pass] sorted = true; // Assume sorted for (int index = 0; index < n - pass; index++) { // At this point, all entries in the_array[0..index-1] // are <= the_array[index] int nextIndex = index + 1; if (the_array[index] > the_array[nextIndex]) { // Exchange entries std::swap(the_array[index], the_array[nextIndex]); sorted = false; // Signal exchange } // end if } // end for // Assertion: the_array[0..n-pass-1] < the_array[n-pass] pass++; } // end while } // end bubbleSort

!117

PassO(n)

O(n)

O( n2)


Execution time DOES depend on initial arrangement of data

Worst case: O(n2) comparisons and data moves

Best case: O(n) comparisons and data moves

Stable

If array is already sorted bubble sort will stop after first pass and no swaps => good choice for small n and data likely somewhat sorted

!118

Raise your hand if you had Bubble Sort

!119

https://www.youtube.com/watch?v=lyZQPjUT5B4

!120

Insertion Sort

!121

Insertion Sort

!122

Pick first element in unsorted region and put it in right place

in sorted region

Unsorted

Sorted

1st Pass

Insertion Sort

!123


in sorted region

Unsorted

Sorted

1st Pass

Insertion Sort

!124


in sorted region

Unsorted

Sorted

Swap

1st Pass

Insertion Sort

!125


in sorted region

Unsorted

Sorted

1st Pass

Insertion Sort

!126


in sorted region

Unsorted

Sorted

Insertion Sort

!127


in sorted region

Unsorted

Sorted

2nd Pass

Insertion Sort

!128


in sorted region

Unsorted

Sorted

Swap

2nd Pass

Insertion Sort

!129


in sorted region

Unsorted

Sorted

2nd Pass

Insertion Sort

!130


in sorted region

Unsorted

Sorted

2nd Pass

Insertion Sort

!131


in sorted region

Unsorted

Sorted

Insertion Sort

!132


in sorted region

Unsorted

Sorted

3rd Pass

Insertion Sort

!133


in sorted region

Unsorted

Sorted

Swap

3rd Pass

Insertion Sort

!134


in sorted region

Unsorted

Sorted

3rd Pass

Insertion Sort

!135


in sorted region

Unsorted

Sorted

Swap

3rd Pass

Insertion Sort

!136


in sorted region

Unsorted

Sorted

3rd Pass

Insertion Sort

!137


in sorted region

Unsorted

Sorted

Swap

3rd Pass

Insertion Sort

!138


in sorted region

Unsorted

Sorted

3rd Pass

Insertion Sort

!139


in sorted region

Unsorted

Sorted

Insertion Sort

!140


in sorted region

Unsorted

Sorted

4th Pass

Insertion Sort

!141


in sorted region

Unsorted

Sorted

4th Pass

Insertion Sort

!142


in sorted region

Unsorted

Sorted

Insertion Sort

!143


in sorted region

Unsorted

Sorted

5th Pass

Insertion Sort

!144


in sorted region

Unsorted

Sorted

Swap

5th Pass

Insertion Sort

!145


in sorted region

Unsorted

Sorted

5th Pass

Insertion Sort

!146


in sorted region

Unsorted

Sorted

5th Pass

Insertion Sort

!147


in sorted region

Unsorted

Sorted

Insertion Sort Analysis

How much work?

First pass: 1 comparison and at most 1 swap

Second pass: at most 2 comparisons and at most 2 swaps

Third pass: at most 3 comparisons and at most 3 swaps. . .

Total work: 1 + 2 + 3 + . . . + (n-1)

!148

!149

n + (n-1) + ... + 2 + 1

n

n + 1

= n(n+1) / 2 1 + 2 + . . . (n-2) + (n-1) = n(n-1)/2

(n-1)

n



T(n) = 2( (n2-n) / 2 )= O( )?

T(n) = n2-n = O( n2)

Insertion Sort run time is O(n2)

!150

Insertion Sort

!151


in sorted region

Unsorted

Sorted

Insertion Sort

!152


in sorted region

Unsorted

Sorted

Insertion Sort

!153


in sorted region

Unsorted

Sorted

Insertion Sort

!154


in sorted region

Unsorted

Sorted

Insertion Sort

!155


in sorted region

Unsorted

Sorted

Insertion Sort

!156


in sorted region

Unsorted

Sorted

Insertion Sort

!157


in sorted region

Unsorted

Sorted

Insertion Sort

!158


in sorted region

Unsorted

Sorted

Insertion Sort

!159


in sorted region

Unsorted

Sorted

Insertion Sort

!160


in sorted region

Unsorted

Sorted

Insertion Sort

!161


in sorted region

Unsorted

Sorted


Execution time DOES depend on initial arrangement of data

Worst case: O( n2) comparisons and data moves

Best case: O(n) comparisons and data moves

Stable

If array is already sorted Insertion sort will do only n comparisons and no swaps => good choice for small n and data likely somewhat sorted

!162

template<class T> void insertionSort(T the_array[], int n) { // unsorted = first index of the unsorted region, // loc = index of insertion in the sorted region, // next_item = next item in the unsorted region. // Initially, sorted region is the_array[0], // unsorted region is the_array[1..n-1]. // In general, sorted region is the_array[0..unsorted-1], // unsorted region the_array[unsorted..n-1] for (int unsorted = 1; unsorted < n; unsorted++) { // At this point, the_array[0..unsorted-1] is sorted. // Find the right position (loc) in the_array[0..unsorted] // for the_array[unsorted], which is the first entry in the // unsorted region; shift, if necessary, to make room T next_item = the_array[unsorted]; int loc = unsorted; while ((loc > 0) && (the_array[loc - 1] > next_item)) { // Shift the_array[loc - 1] to the right the_array[loc] = the_array[loc - 1]; loc--; } // end while // At this point, the_array[loc] is where next_item belongs the_array[loc] = next_item; // Insert next_item into sorted region } // end for } // end insertionSort

!163

template<class T> void insertionSort(T the_array[], int n) { // unsorted = first index of the unsorted region, // loc = index of insertion in the sorted region, // next_item = next item in the unsorted region. // Initially, sorted region is the_array[0], // unsorted region is the_array[1..n-1]. // In general, sorted region is the_array[0..unsorted-1], // unsorted region the_array[unsorted..n-1] for (int unsorted = 1; unsorted < n; unsorted++) { // At this point, the_array[0..unsorted-1] is sorted. // Find the right position (loc) in the_array[0..unsorted] // for the_array[unsorted], which is the first entry in the // unsorted region; shift, if necessary, to make room T next_item = the_array[unsorted]; int loc = unsorted; while ((loc > 0) && (the_array[loc - 1] > next_item)) { // Shift the_array[loc - 1] to the right the_array[loc] = the_array[loc - 1]; loc--; } // end while // At this point, the_array[loc] is where next_item belongs the_array[loc] = next_item; // Insert next_item into sorted region } // end for } // end insertionSort

!164

PassO(n)

O(n)

O( n2)

Raise your hand if you had Insertion Sort

!165

What we have so far

!166

Worst Case Best Case

Selection Sort O( n2 ) O( n2 )

Bubble Sort O( n2 ) O( n )

Insertion Sort O( n2 ) O( n )

Lecture Activity

Sort the array using Insertion SortShow the entire array after each comparison/swap operation and at each step mark clearly the division between the sorted and unsorted portions of the array

!167

5 8 3 4 9

5 8 3 4 9

Pick first element in unsorted region and put it in right place in sorted region

2 7

2 7

!168

5 8 3 4 9

5 8 3 4 9

2 7

2 7

5 3 8 4 9

3 5 8 4 9

2 7

2 7

3 5 4 8 9 2 7

3 4 5 8 9 2 7

3 4 5 8 9 2 7

3 4 5 8 2 9 7

3 4 5 2 8 9 7

3 4 2 5 8 9 7

3 2 4 5 8 9 7

2 3 4 5 8 9 7

2 3 4 5 8 7 9

2 3 4 5 7 8 9

!169

https://www.toptal.com/developers/sorting-algorithms

Can we do better?

!170

Can we do better?

!171

Divide and Conquer!!!

Merge Sort

!172

Understanding O(n2)

!173

76108 158195523 59914 43100 200 2743 11260 5642 932

T(n)

Understanding O(n2)

!174

76108 158195523 59914 43100 200 2743 11260 5642 932

T(n)

10852314 43100 200 2743 76 158195 599 11260 5642 932

Understanding O(n2)

!175

76108 158195523 59914 43100 200 2743 11260 5642 932

T(n)

10852314 43100 200 2743 76 158195 599 11260 5642 932

T(1/2n) T(1/2n)

Understanding O(n2)

!176

76108 158195523 59914 43100 200 2743 11260 5642 932

T(n)

10852314 43100 200 2743 76 158195 599 11260 5642 932

T(1/2n) T(1/2n)

(n/2)2 = n2/4

Understanding O(n2)

!177

76108 158195523 59914 43100 200 2743 11260 5642 932

T(n)

10852314 43100 200 2743 76 158195 599 11260 5642 932

(n/2)2 = n2/4

T(1/2n) ≈ 1/4 T(n) T(1/2n) ≈ 1/4 T(n)

Understanding O(n2)

!178

76108 158195523 59914 43100 200 2743 11260 5642 932

T(n)

108 52314 43 100 200 2743 76 158 195 59911 2605 642 932

(n/2)2 = n2/4

T(1/2n) ≈ 1/4 T(n) T(1/2n) ≈ 1/4 T(n)

Key Insight: Merge is linear

!179

2 1 34 57 68 910


!180

2 1 34 57 68 910


!181

2 34 57 68 910

1


!182

2 34 57 68 910

1


!183

34 57 68 910

21


!184

34 57 68 910

21


!185

4 57 68 910

21 3


!186

4 57 68 910

21 3


!187

57 68 910

21 3 4


!188

57 68 910

21 3 4


!189

7 68 910

21 3 4 5


!190

7 68 910

21 3 4 5


!191

7 8 910

21 3 4 5 6


!192

7 8 910

21 3 4 5 6


!193

8 910

21 3 4 5 76


!194

8 910

21 3 4 5 76


!195

910

21 3 4 5 76 8


!196

910

21 3 4 5 76 8


!197

10

21 3 4 5 76 8 9


!198

10

21 3 4 5 76 8 9


!199

21 3 4 5 76 8 9 10


!200

21 3 4 5 76 8 9 10

Each step makes one comparison and reduces the number of elements to

be merged by 1. If there are n total elements to be

merged, merging is O(n)

Divide and Conquer

!201

76108 158195523 59914 43100 200 2743 11260 5642 932

T(n)

108 52314 43 100 200 27476 158 195 59911 26064 932

T(1/2n) ≈ 1/4 T(n) T(1/2n) ≈ 1/4 T(n)

Divide and Conquer

!202

76108 158195523 59914 43100 200 2743 11260 5642 932

T(n)

108 52314 43 100 200 2743 76 158 195 59911 2605 642 932

Speed up insertion sort by a factor of two by splitting in half, sorting separately and merging results!

T(1/2n) ≈ 1/4 T(n) T(1/2n) ≈ 1/4 T(n)

108 52314 43 100 200 27476 158 195 59911 26064 932

T(n) ≈ 1/2 T(n) + n

Divide and Conquer

Splitting in two gives 2x improvement.

!203

Divide and Conquer


Splitting in four gives 4x improvement.

!204

Divide and Conquer



Splitting in eight gives 8x improvement.

!205

Divide and Conquer



Splitting in eight gives 8x improvement.

What if we never stop splitting?

!206

Merge Sort

!207

76108 158195523 59914 43 200 2743 11260 642 932

7610852314 43 200 2743 158195 599 11260 642 932

14 43 2003 76108523274 158195 599 2 11260 64 932

14 3 43 200 523274 76108 195 599 158 2 11260 64 932

14 3 43 200 523274 76108 195 599 158 2 11260 64 932

Merge Sort

!208

76108 158195523 59914 43 200 2743 11260 642 932

7610852314 43 200 2743 158195 599 11260 642 932

14 43 2003 76108523274 158195 599 2 11260 64 932

14 3 43 200 523274 76108 195 599 158 2 11260 64 932

14 3 43 200 523274 76108 195 599 158 2 11260 64 932

Merge Sort

!209

76108 158195523 59914 43 200 2743 11260 642 932

7610852314 43 200 2743 158195 599 11260 642 932

14 43 2003 76108523274 158195 599 2 11260 64 932

3 14 43 200 523274 10876 195 599 2 158 2611 64 932

14 3 43 200 523274 76108 195 599 158 2 11260 64 932

Merge Sort

!210

76108 158195523 59914 43 200 2743 11260 642 932

7610852314 43 200 2743 158195 599 11260 642 932

3 43 20014 52327410876 1952 158 599 2611 64 932

3 14 43 200 523274 10876 195 599 2 158 2611 64 932

14 3 43 200 523274 76108 195 599 158 2 11260 64 932

Merge Sort

!211

76108 158195523 59914 43 200 2743 11260 642 932

5232742003 43 76 10814 262 11 195158 59964 932

3 43 20014 52327410876 1952 158 599 2611 64 932

3 14 43 200 523274 10876 195 599 2 158 2611 64 932

14 3 43 200 523274 76108 195 599 158 2 11260 64 932

Merge Sort

!212

7664 19510843 1582 11 14 263 523274 599200 932

5232742003 43 76 10814 262 11 195158 59964 932

3 43 20014 52327410876 1952 158 599 2611 64 932

3 14 43 200 523274 10876 195 599 2 158 2611 64 932

14 3 43 200 523274 76108 195 599 158 2 11260 64 932

Merge Sort

!213

7664 19510843 1582 11 14 263 523274 599200 932

5232742003 43 76 10814 262 11 195158 59964 932

3 43 20014 52327410876 1952 158 599 2611 64 932

3 14 43 200 523274 10876 195 599 2 158 2611 64 932

14 3 43 200 523274 76108 195 599 158 2 11260 64 932

Merge Sort Analysis

!214

7664 19510843 1582 11 14 263 523274 599200 932

5232742003 43 76 10814 262 11 195158 59964 932

3 43 20014 52327410876 1952 158 599 2611 64 932

3 14 43 200 523274 10876 195 599 2 158 2611 64 932

14 3 43 200 523274 76108 195 599 158 2 11260 64 932

O(n)

O(n)

O(n)

O(n)

O(n)

Merge Sort Analysis

!215

7664 19510843 1582 11 14 263 523274 599200 932

5232742003 43 76 10814 262 11 195158 59964 932

3 43 20014 52327410876 1952 158 599 2611 64 932

3 14 43 200 523274 10876 195 599 2 158 2611 64 932

14 3 43 200 523274 76108 195 599 158 2 11260 64 932

Merge how many times?

n

n/2

n/4

. . .

n/2k

Merge Sort Analysis

!216

7664 19510843 1582 11 14 263 523274 599200 932

5232742003 43 76 10814 262 11 195158 59964 932

3 43 20014 52327410876 1952 158 599 2611 64 932

3 14 43 200 523274 10876 195 599 2 158 2611 64 932

14 3 43 200 523274 76108 195 599 158 2 11260 64 932

Merge how may times? n/2k = 1 n = 2k

log2 n = k

n

n/2

n/4

. . .

n/2k

Merge Sort Analysis

!217

7664 19510843 1582 11 14 263 523274 599200 932

5232742003 43 76 10814 262 11 195158 59964 932

3 43 20014 52327410876 1952 158 599 2611 64 932

3 14 43 200 523274 10876 195 599 2 158 2611 64 932

14 3 43 200 523274 76108 195 599 158 2 11260 64 932

n

n/2

n/4

. . .

n/2k

Merge n elements log2 n times

Merge Sort Analysis

!218

7664 19510843 1582 11 14 263 523274 599200 932

5232742003 43 76 10814 262 11 195158 59964 932

3 43 20014 52327410876 1952 158 599 2611 64 932

3 14 43 200 523274 10876 195 599 2 158 2611 64 932

14 3 43 200 523274 76108 195 599 158 2 11260 64 932

O(n log n )

O(n)

O(n)

O(n)

O(n)

O(n)

Merge Sort

How would you code this?

!219

Merge Sort

How would you code this?

Hint: Divide and Conquer!!!

!220

Merge Sort

void mergeSort(array){ if array size <= 1 return //base case split array into left_array and right_array mergeSort(left_array) mergeSort(right_array) merge(left_array, right_array, sorted_array)}

!221

Merge Sort Analysis

Execution time does NOT depend on initial arrangement of data

Worst Case: O( n log n) comparisons and data moves

Best Case: O( n log n) comparisons and data moves

Stable

Best we can do with comparison-based sorting that does not rely on a data structure in the worst case => can’t beat O( n log n)

Space overhead: auxiliary array at each merge step

!222

What we have so far

!223





Merge Sort O( n log n ) O( n log n )

Quick Sort

!224

Quick Sort

!225

Select a pivot. Arrange other entries s.t. entries in left partition are ≤ pivot

and entries in right partition are > pivot

<= pivot

> pivot

Partition

Quick Sort

!226

pivot

<= pivot

> pivot

Partition

Quick Sort

!227

pivot



<= pivot

> pivot

Partition

Quick Sort

!228

pivot



<= pivot

> pivot

Partition

Quick Sort

!229

pivot



<= pivot

> pivot

swap

Partition

Quick Sort

!230

pivot



<= pivot

> pivot

Partition

Quick Sort

!231

pivot



<= pivot

> pivot

Partition

Quick Sort

!232

pivot



<= pivot

> pivot

Partition

Quick Sort

!233

pivot



<= pivot

> pivot

swap

Partition

Quick Sort

!234

pivot



<= pivot

> pivot

Partition

Quick Sort

!235

pivot



<= pivot

> pivot

Partition

Quick Sort

!236

pivot



<= pivot

> pivot

Partition

Quick Sort

!237

pivot



<= pivot

> pivot

Partition

Quick Sort

!238

pivot



<= pivot

> pivot

swap

Partition

Quick Sort

!239

pivot



<= pivot

> pivot

Partition

Quick Sort

!240



<= pivot

> pivot

≤ pivot > pivotquickSort( ) quickSort( )

Partition

Quick Sort Analysis

Divide and Conquer

n comparisons for each partition

How many subproblems? => Depends on pivot selection

Ideally partition divides problem into two n/2 subproblems for logn recursive calls (Best case)

Possibly (though unlikely) each partition has 1 empty subarray for n recursive calls (Worst case)

!241

template<class T> void quickSort(T the_array[], int first, int last) { if (last - first + 1 < MIN_SIZE) { insertionSort(the_array, first, last); } else { // Create the partition: S1 | Pivot | S2 int pivot_index = partition(the_array, first, last); // Sort subarrays S1 and S2 quickSort(the_array, first, pivot_index - 1); quickSort(the_array, pivotIndex + 1, last); } // end if } // end quickSort

!242

How to select pivot?

!243


Ideally median Need to sort array to find median

Other ideas?

!244



Other ideas? Pick first, middle, last position and order them making middle the pivot

!245

695 13



Other ideas? Pick first, middle, last position and order them making middle the pivot

!246

136 95

pivot

Quick Sort AnalysisExecution time DOES depend on initial arrangement of data AND on PIVOT SELECTION (luck?) => on random data can be faster than Merge Sort

Optimization (e.g. smart pivot selection, speed up base case, iterative instead of recursive implementation) can improve actual runtime -> fastest comparison-based sorting algorithm on average

Worst Case: O( n2) comparisons and data moves

Best Case: O( n log n) comparisons and data moves

Unstable

!247

!248





Merge Sort O( n log n ) O( n log n )

Quick Sort O( n2 ) O( n log n )

!249

https://www.toptal.com/developers/sorting-algorithms

!250

https://www.youtube.com/watch?v=kPRA0W1kECg

https://www.youtube.com/watch?v=kPRA0W1kECg

Date post:	13-Jun-2020
Category:	Documents
Upload:	others
View:	8 times
Download:	0 times

Searching and Sorting - Tiziana Ligorio · Searching and Sorting Tiziana Ligorio...

Documents