1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity...

transcript

Sorting Algorithms (Basic)Search Algorithms

BinaryInterpolation

Big-O NotationComplexity

Sorting, Searching, Recursion Intro to Algorithms

Selection Sortfor int’sfor string’s

Templated Functions

Recursion

Sorting Elementary sorting algorithms

– Bubble– Insertion*– Selection

Bubble Sort

const size_t N = <constant>;int A[N];// ‘A’ then is populated // Sort followsfor (size_t i = N - 1; i >= 1; --i) for (size_t j = 0; j < i; ++j) if (A[j] > A[j + 1]) std::swap (A[j], A[j + 1]);

Complexity?

Insertion Sort

for (i = 1; i < N; ++i){ // Invariant: A[0..i) sorted v = A[i]; // Element to place j = i; // Index to place ‘v’ while (j >= 1 && v < A[j–1]) { A[j] = A[j–1]; --j; } // Invariant: v >= A[j-1] or j == 0

A[j] = v;}

Selection Sort

// Find N-1 smallest and placefor (i = 0; i < N - 1; ++i) { min = i; for (j = i+1; j < N; ++j) if (A[j] < A[min])

min = j; //if (i != min) swap (A[i], A[min]);}

Searching Sequential

– Examine 1st element, 2nd, …– Assume int A[N];– int* pos = std::find (A, A+N, val);

Binary– Requires sequence be sorted– More efficient – quantify– int* pos = lower_bound (A, A+N, val);

// *pos >= val or pos @ end– bool found = binary_search (A, A+N, val);

Interpolation– Sorted sequence– O(lg lg (N)) average

Binary Search If data are sorted can use Binary Search Look at middle element

– If match, return location– If smaller, search left sub-array– If larger, search right sub-array

Binary Search

Case 1: Matchif (target == midValue)

return mid;

m idfirs t

target

C as e 1: target = m id valueS earc h is d o ne

las t-1 las t

Binary Search

Case 2: Target smaller than middle value// Search the left sublist

if (target < midValue)

<reposition last to mid><search

sublist arr[first]…arr[mid-1]

las t-1firs t

target

C as e 2: target < m id valueS earc h lo w er s ub lis t

m id -1 las t

Binary Search

Case 3: Target larger than middle value// Search upper sublist if (target > midValue)

<reposition first to mid+1><search sublist arr[mid+1]…arr[last-

C as e 3: target > m id valueS earc h up p er s ub lis t

las t-1new firs t = m id + 1

firs t

target

Illustrating the Binary Search- Successful Search

1. Search for target = 23Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4.

Since target = 23 > midvalue = 12, step 2 searches the upper sublist with first = 5 and last = 9.

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

Step 2:Indices first = 5, last = 9, mid = (5+9)/2 = 7.

Since target = 23 < midvalue = 33, step 3 searches the lower sublist with first = 5 and last = 7.

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

Step 3: Indices first = 5, last = 7, mid = (5+7)/2 = 6.

Since target = midvalue = 23, a match is found at index mid = 6.

-7 3 5 8 12 16arr0 1 2 3 4 5

23 33 55

6 7 8 9

Illustrating the Binary Search- Unsuccessful Search

Search for target = 4. Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4.

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

Since target = 4 < midvalue = 12, step 2 searches the lower sublist with first = 0 and last = 4.

Since target = 4 < midvalue = 5, step 3 searches the lower sublist with first = 0 and last 2.

-7 3 5 8 12 16arr0 1 2 3 4 5

23 33 556 7 8

Since target = 4 > midvalue = 3, step 4 should search the upper sublist with first = 2 and last =2. However, since first >= last, the target is not in the list and we return index -1.

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

Binary Search Algorithm

int binSearch (const int A[],

int first, int last,

int target)

{int midIndex;int midValue;

// Test for nonempty sublistwhile (first < last)

{midIndex = (first+last)/2;midValue = A[mid];if (target == midValue)

return midIndex;// Determine which sublist to

// search

else if (target < midValue)last = midIndex;

elsefirst = midIndex + 1;

}return -1;

}// Note half-open range convention

Binary Search (Observation) midIndex = first + 1 * (last – first) / 2; Implicit assumption? Can we do better?

Interpolation Searchint iSearch (const vector <int>& A, int v) {

int loc, s = 0, e = A.size () - 1;

if (v < A[s]) return (-1);

if (v >= A[e]) s = e;

while (s < e) {

loc = s + (e - s) * (v – A[s]) / (A[e] – A[s]);

if (v > A[loc]) { s = loc + 1; }

else if (v < A[loc]) { e = loc - 1; }

else return (loc);

if (v != A[s]) s = -1;

return (s);

Algorithm Complexity Big-O notation – machine-independent means

for expressing efficiency of an algorithm Count key operations or stmts, and relate this

count to problem size using growth fn. f(N) = O(g(N)) if there are positive constants c

and n0 such that f(N) <= c * g(N) when N >= n0

Express op count as function of input size– f1(N) = 3N2 + 50N + 120– f2(N) = 1000 lg(N) + 50– f3(N) = 5

f1(N) = O(N2); n0 = 1, c = 200 3N2 + 50N + 120 <= 200N2, for N >= n0

f3(N) = O(1); n0 = 1, c = 6

Big-O Notation

Constant Time Algorithms

An algorithm is O(1) (constant time) when its running time is independent of input size

push_back on vector involves a simple assignment statement complexity O(1) (if capacity sufficient)

fro nt rear

D irec t In se r t a t R ear

Linear Time AlgorithmsAn algorithm is O(N) when its running time is proportional to input size

S equential S earch fo r the Minim um E lem ent in an A rray

32 46 8 12 3

m in im u m elem en t fou n d in th e list a fter n com p a rison s

n = 51 2 3 4 5

Polynomial AlgorithmsAlgorithms with running time O(N2) are

quadratic– practical only for relatively small values of N;

whenever N triples, the running time increases by ?

Algorithms with running time O(N3) are cubic– efficiency is generally poor; tripling N

increases the running time by ?

Algorithm Complexity Chart

n log2n n log2n n2 n3 2n 2 1 2 4 8 4 4 2 8 16 64 16 8 3 24 64 512 256 16 4 64 256 4096 65536 32 5 160 1024 32768 4294967296 128 7 896 16384 2097152 3.4 x 1038

1024 10 10240 1048576 1073741824 1.8 x 10308

65536 16 1048576 4294967296 2.8 x 1014 Forget it!

Logarithmic Time AlgorithmsThe logarithm of N, base 2, is commonly used when analyzing computer algorithmsEx. log2(2) = lg (2) = 1

log2(75) = lg (75) ~= 6.2288

When compared to functions N and N2, the function lg N grows very slowly

lo g 2n

Function Templates Same logic is appropriate for different types

– Templatize function Example: selection sort logic identical for

floats, ints, strings, etc.

Selection Sort AlgorithmInteger Version

void selectionSort (int A[], int n) {

. . .int temp;

// int temp used for the exchangefor (pass = 0; pass < n-1; ++pass)

{. . .

if (A[j] < A[smallIndex])

// Compare integer elements . . .

Selection Sort AlgorithmString Version

void selectionSort (string A[], int n) {

. . .string temp;

// string temp used for the exchange

for (pass = 0; pass < n-1; ++pass) {

. . .if (A[j] <

A[smallIndex])// compare string elements

Template Syntax Include keyword template followed list of formal

types enclosed in angle brackets In the argument list, each type is preceded by

the keyword typename (or class)

// Argument list with multiple template

// types template <typename T, typename U,

typename V, ...>

Template Syntax Example

template <typename T>void selectionSort (T A[], int n){ int smallIndex;

// index of smallest element in the // sublist

int pass, j;T temp;

// pass has the range 0 to n-2for (pass = 0; pass < n-1; +

+pass){

// scan the sublist starting at // index pass

smallIndex = pass;

// j traverses the sublist // A[pass+1] to A[n-1]

for (j = pass+1; j < n; ++j) // update if smaller element found

if (A[j] < A[smallIndex])

smallIndex = j;// if smallIndex and

pass are not // the same location, exchange the

// smallest item in the sublist with // A[pass]

if (smallIndex != pass)

swap (A[pass],

A[smallIndex]);}

Recursion Must ensure

– Recursive calls work on smaller problem– Eventually reach base case

Many problems have naturally recursive solutions (difficult to express iteratively)

Recursion Examples Power function Binary search Towers of Hanoi Fibonacci numbers (*)

Recursive Definition of the Power Function

Recursive definition– exponent n = 0 (base case)– n 1 which assumes we already know xn-1

Compute successive powers of x by multiplying the previous value by x

0,11 nxx

Implementing the Recursive Power Function

Recursive power:double power (double x, int n)

// n is a non-negative integer{

if (n == 0)return 1.0; // base case

elsereturn x * power (x, n-1);

// recursive step} // runtime? how to improve?

Solving Recurrence T(N) = T(N-1) + 1; T(0) = 0 Solve:

– T(N) = T(N-1) + 1– T(N-1) = T(N-2) + 1– T(N-2) = T(N-3) + 1– …– T(1) = T(0) + 1– T(0) = 0– :: T(N) = N = O(N)– (not good enough)

Binary Search// Search A [s, e)int bs (int A[], int s, int e, int v){ if (s >= e) return -1; int m = (s + e) / 2; if (v == A[m]) return m; if (v < A[m]) return bs (A, s, m, v); else return bs (A, m+1, e, v);}

Improvement

Reformulate definition to cut problem in half each time (?)

Suppose N = 2k for k >= 0 T(N) = T(N/2) + 1; T(0) = 0 Solve:

– T(N) = T(N/2) + 1– T(N/2) = T(N/4) + 1– T(N/4) = T(N/8) + 1– …– T(1) = T(0) + 1– T(0) = 0– :: T(N) = lg(N) + N = O(lg(N))

Solving the Tower of Hanoi Puzzle using Recursion

N eed le A

. . . . . . . .

N eed le CN eed le B N eed le C

. . . . . . . .

N eed le BN eed le A

Move N-1 discs from A to B (using B as temp)Move disc N from A to CMove N-1 discs from B to C (using A as temp)

Recurrence?

Fibonacci Numbers using Iteration

int fibIter (int n){ // Don’t use recursion!

if (n == 0 || n == 1)return n;

// Store previous two Fib #s int oneBack = 0, twoBack = 1,

current;

Fibonacci Numbers using Iteration

// Compute successive terms

for (int i = 2; i <= n; ++i) {

current = oneBack + twoBack;twoBack = oneBack;

oneBack = current; }

return current;}// Counting adds: N >= 2// T(N) = N-1 = O(N)

1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity...

Documents