Algorithms

Friday

7th Week

Algorithms

• What is an Algorithm?– A series of precise steps, known to

stop eventually, to solve a problem– NOT necessarily tied to computers– There can be many algorithms to

solve the same problem

Characteristics of an Algorithm

1. Precise steps

2. Effective steps

3. Has an output

4. Terminates eventually

Trivial Algorithm

• Computing an average:1. Add up all of the values

2. Divide by the number of values

Greatest Common Divisor

• Computing the Greatest Common Divisor (gcd) of two numbers

• Examples:– gcd(12, 2) = 2– gcd(100, 95) = 5– gcd(100, 75) = 25– gcd(39, 26) = 13– gcd(17, 8) = 1

Possible Algorithm #1

• Assumption: A > B >= 0– If A is a multiple of B, then gcd(A, B) = B.– Otherwise, return an error.

• Works for gcd(12,2) = 2

• But what about gcd(100, 95)???

Possible Algorithm #2

– Start with 1 and go up to B.– If a number if a common divisor of both A

and B, remember it.– When we get to B, stop. The last number

we remembered is the gcd.

• Works, but is there a better way?

• Think about gcd(100, 95)

Euclid’s Algorithm

• Make use of the fact that:

• gcd(A, B) = gcd(B, A rem B)– Note: A rem B refers to the remainder left

when A is divided by B.– Examples:

• 12 rem 2 = 0• 100 rem 95 = 5

Euclid’s Algorithm

– If B = 0, then gcd(A, B) = A– Otherwise, gcd(A, B) = gcd (B, A rem B)

• Note – this algorithm is recursive.

• Examples:– gcd(12, 2) = gcd(2, 0) = 2– gcd(100, 95) = gcd(95, 5) = gcd(5, 0) = 5

Program vs. Algorithm

• Program: “A set of computer instructions written in a programming language”

• We write Programs (in JavaScript) that implement various Algorithms

Program vs. Algorithm

• Problem: Sort a list of numbers• Algorithms:

– Quicksort– Insertion Sort – Merge Sort

• Programs: – Using JavaScript to implement Insertion Sort– Using Pascal to implement Quicksort

Algorithms to Solve Problems

• Problems can be categorized into two categories– Computable problems can be solved using

an algorithm.– Non-computable problems have no

algorithm to solve them.

• We don’t necessariliy know which category a problem is in. (this is non-computable)

Computable Problems

• Tractable– There is an efficient algorithm to solve the

problem.

• Intractable– There is an algorithm to solve the problem

but the best known algorithm is not efficient.

Examples

• Sorting: tractable

• Chess Playing: intractable

• Halting Problem: non-computable

How do we measure efficiency?

• To do this, we count the number of steps an algorithm takes.

• For example, consider the following algorithm which adds up the numbers in an array:

Add up the numbers in an arraysum = 0;for (var i=0; i<numArray.length; i++){

sum += numArray[i];

}• The statement inside the for loop gets executed

numArray.length times – it depends on what the length is.

• If the length is n, we say this algorithm is “on the order of n”, or, this algorithm is O(n).

Definition of O(g(n))• f(n)=O(g(n)) means that for all n larger than

some threshold, f(n) <= C*g(n) for some constant C,

Or, the ratio f(n)/g(n) <= C.• This means that f grows no faster than g, or f

is proportional to g.• CONSTANTS DON’T MATTER• f could represent time an algorithm takes, or

the number of steps executed.

Big O Analysis

• What is the worst case running time on input of size N, discounting constants

Comparison of Functions

• Look at graphs of f(x)=x, g(x)=x2, h(x)=x3, j(x)=logx k(x) = 2x

• How do they grow? How do they compare with each other?

Tractable vs. Intractable

• Polynomial time algorithms are considered tractable.

• Exponential time algorithms are considered intractable.

Sorting (Insertion Sort)

function insertionSort (numArray) { for ( var i=1; i < numArray.length; i++) { var temp = numArray[i]; for (var j = i; (j > 0) && ( numArray[j-1] > temp); j--) { numArray[j] = numArray[j-1]; } numArray[j] = temp; }}

Sorting (Merge Sort)

• Given a list, split it into 2 equal piles.

• Then split each of these piles in half. Continue to do this until you are left with 1 element in each pile.

• Now merge piles back together in order.

(Merge Sort, cont)

• An example of how the merge works:Suppose the first half and second half of an array are sorted:5 9 10 12 17 1 8 11 20 32

• Merge these by taking a new element from either the first or second subarray, choosing the smallest of the remaining elements.

Big O Can Be Deceiving

• Big O analysis is concerned with worst case behavior

• Sometimes we know that we are not dealing with the worst case

Search the Array

for (var i=0; i<numArray.length; i++)

{if (numArray[i] == “what we want”)

stop and jump for joy;

}

Binary Search

• If we are searching in a sorted list, we choose the half that the value would be in.

• We continue to cut the area we are searching in half until we find the value we want or there are no more values to check.

Algorithms Exercise