COMP 355 Advanced Algorithms - Rhodescs.rhodes.edu/welshc/COMP355_S14/Lecture2.pdf · 2014. 1....

COMP 355 Advanced Algorithms

1/10/2014 COMP 355: Advanced Algorithms

Spring 2014 1

Algorithm Design Review: Mathematical Background

Polynomial Time


Spring 2014 2

Brute force. For many non-trivial problems, there is a natural brute force search algorithm that checks every possible solution. –Typically takes 2N time or worse for inputs of size N. –Unacceptable in practice.

Desirable scaling property. When the input size doubles, the

algorithm should only slow down by some constant factor C. Def. An algorithm is poly-time if the above scaling property

holds.

There exists constants c > 0 and d > 0 such that on every

input of size N, its running time is bounded by c Nd steps.

n ! for stable matching with n men and n women

choose C = 2d

Worst-Case Analysis


Spring 2014 3

Worst case running time. Obtain bound on largest possible running time of algorithm on input of a given size N. –Generally captures efficiency in practice. –Draconian view, but hard to find effective alternative.

Average case running time. Obtain bound on running

time of algorithm on random input as a function of input size N. –Hard (or impossible) to accurately model real instances

by random distributions. –Algorithm tuned for a certain distribution may perform

poorly on other inputs.

Worst-Case Polynomial Time


Spring 2014 4

An algorithm is efficient if its running time is polynomial. Justification: It really works in practice!

– Although 6.02 1023 N20 is technically poly-time, it would be useless in practice.

– In practice, the poly-time algorithms that people develop almost always have low constants and low exponents.

– Breaking through the exponential barrier of brute force typically exposes some crucial structure of the problem.

Exceptions. – Some poly-time algorithms do have high constants and/or

exponents, and are useless in practice. – Some exponential-time (or worse) algorithms are widely used

because the worst-case instances seem to be rare.

simplex method

Unix grep

Big-O Notation


Spring 2014 5

• Asymptotic O-notation (“big-O”) provides a way to simplify the messy functions that often arise in analyzing the running times of algorithms

• Allows us to ignore less important elements (constants)

• Focus on important issues (growth rate for large values of n)

Formal Definition Big-O


Spring 2014 6

• Formally, f(n) is O(g(n)) if there exist constants c > 0 and n0 ≥ 0 such that, f(n) ≤ c · g(n), for all n ≥ n0.

• Thus, big-O notation can be thought of as a way of expressing a sort of fuzzy “≤” relation between functions, where by fuzzy, we mean that constant factors are ignored and we are only interested in what happens as n tends to infinity.

Intuitive Form of Big-O


Spring 2014 7

𝑓 𝑛 𝑖𝑠 𝑂 𝑔 𝑛 𝑖𝑓 lim𝑛→ ∞

𝑓 𝑛

𝑔 𝑛≥ 𝑐, 𝑓𝑜𝑟 𝑠𝑜𝑚𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑐 ≥ 0.

For example, if f(n) = 15n2 + 7n log3 n and g(n) = n2, we have f(n) is O(g(n)) because

In the last step of the derivation, we have used the important fact that log n raised to any positive power grows asymptotically more slowly than n raised to any positive power.

Useful facts about limits


Spring 2014 8

9

Survey of Common Running Times

• Linear Time - O(n)

• Linearithmic Time - O(n log n)

• Quadratic Time – O(n2)

• Cubic Time – O(n3)

• Polynomial Time: O(nk)

• Exponential Time – 𝑂(2𝑛𝑘)


Spring 2014

10

Linear Time: O(n)

• Linear time. Running time is at most a constant factor times the size of the input.

• Computing the maximum. Compute maximum of n numbers a1, …, an.

max a1 for i = 2 to n {

if (ai > max)

max ai }


Spring 2014

Linear Time: O(n) Merge. Combine two sorted lists A = a1,a2,…,an with B = b1,b2,…,bn into sorted whole.

Claim. Merging two lists of size n takes O(n) time.

Pf. After each comparison, the length of output list increases by 1.

i = 1, j = 1

while (both lists are nonempty) {

if (ai bj) append ai to output list and increment i

else(ai bj)append bj to output list and increment j }

append remainder of nonempty list to output list


Spring 2014 11

O(n log n) Time

• O(n log n) time. Arises in divide-and-conquer algorithms.

• Sorting. Mergesort and heapsort are sorting algorithms that perform O(n log n) comparisons.

• Largest empty interval. Given n time-stamps x1, …, xn on which copies of a file arrive at a server, what is largest interval of time when no copies of the file arrive?

• O(n log n) solution. Sort the time-stamps. Scan the sorted list in order, identifying the maximum gap between successive time-stamps.


Spring 2014 12

also referred to as linearithmic time

Quadratic Time: O(n2)

• Quadratic time. Enumerate all pairs of elements.

• Closest pair of points. Given a list of n points in the plane (x1, y1), …, (xn, yn), find the pair that is closest.

• O(n2) solution. Try all pairs of points.

min (x1 - x2)2 + (y1 - y2)

2

for i = 1 to n {

for j = i+1 to n {

d (xi - xj)2 + (yi - yj)

2

if (d < min)

min d

}

}


Spring 2014 13

don't need to take square roots

Cubic Time: O(n3) • Cubic time. Enumerate all triples of elements.

• Set disjointness. Given n sets S1, …, Sn each of which is a subset of 1, 2, …, n, is there some pair of these which are disjoint?

• O(n3) solution. For each pairs of sets, determine if they are disjoint.

foreach set Si {

foreach other set Sj {

foreach element p of Si {

determine whether p also belongs to Sj

}

if (no element of Si belongs to Sj)

report that Si and Sj are disjoint

}

}


Spring 2014 14

Polynomial Time: O(nk) Time • Independent set of size k. Given a graph, are there k nodes

such that no two are joined by an edge?

• O(nk) solution. Enumerate all subsets of k nodes.

– Check whether S is an independent set = O(k2).

– Number of k element subsets =

– O(k2 nk / k!) = O(nk).

foreach subset S of k nodes {

check whether S in an independent set

if (S is an independent set)

report S is an independent set

}

}

n

k

n (n1) (n 2) (n k 1)

k (k 1) (k 2) (2) (1)

nk

k!

poly-time for k=17, but not practical

k is a constant

1/10/2014 15

Exponential Time

• Independent set. Given a graph, what is maximum size of an independent set?

• O(n2 2n) solution. Enumerate all subsets.

S* ϕ

foreach subset S of nodes {

check whether S in an independent set

if (S is largest independent set seen so far)

update S* S

}

}


Spring 2014 16

Comparison of Running Times


Spring 2014 17

Other Asymptotic Forms


Spring 2014 18

• Big-O has a number of relatives, which are useful for expressing other sorts of relations.

• More on these next time!

Summations


Spring 2014 19

• Naturally arise in analysis of iterative algorithms

• More complex forms of analysis, such as recurrences, are often solved by reducing to summations

• Solving a summation means reducing it to a closed-form formula – No summations, recurrences, integrals, or other

complex operators

• Often don’t need to solve a summation exactly to find the asymptotic approximation

Constant Series


Spring 2014 20

Arithmetic Series


Spring 2014 21

Geometric Series


Spring 2014 22

Quadratic Series


Spring 2014 23

Linear-geometric Series


Spring 2014 24

Harmonic Series


Spring 2014 25

Summations With General Bounds


Spring 2014 26


Spring 2014 27

Linearity of Summation


Spring 2014 28

Approximate Using Integrals


Spring 2014 29

Example: Previous Larger Element Given a sequence of numeric values, <a1, a2, . . . , an>. For each element ai, for 1 ≤ i ≤ n, we want to know the index of the rightmost element of the sequence <a1, a2, . . . , ai−1> whose value is strictly larger than ai. If no element of this subsequence is larger than ai then, by convention, the index will be 0. (Or, if you like, you may imagine that there is a fictitious sentinel value a0 = ∞.) More formally, for 1 ≤ i ≤ n, define pi to be pi = max{j | 0 ≤ j < i and aj > ai}, where a0 = ∞ (see Fig. 2).


Spring 2014 30

Naive Algorithm For Previous Larger Element


Spring 2014 31

Recurrences

Arise naturally in analysis of divide-and-conquer algorithms

• Divide: Divide the problem into two or more sub-problems (ideally of roughly equal sizes)

• Conquer: Solve each sub-problem recursively

• Combine: Combine the solutions to the sub-problems into a single global solution.


Spring 2014 32

Recurrences

• To analyze recursive procedures such as divide-and-conquer, we need to set up a recurrence.

• Example: Suppose we break a problem into two sub-problems, each of size roughly n/2.

• Additional overhead of splitting and merging the solutions is O(n).

• When sub-problems are reduced to size 1, we can solve them in O(1) time.

• Ignoring constants and writing O(n) as n, we get: T(n) = 1 if n = 1,

T(n) = 2T(n/2) + n if n > 1


Spring 2014 33

Example Problem

• Use mathematical induction to show that when n is an exact power of 2, the solution of the recurrence

𝑇 𝑛 = 2, 𝑖𝑓 𝑛 = 2,

2𝑇𝑛

2+ 𝑛, 𝑖𝑓 𝑛 = 2𝑘 , 𝑓𝑜𝑟 𝑘 > 1

is 𝑇 𝑛 = 𝑛 lg 𝑛

Next Time

• Other Asymptotic Forms

• Read Chapter 3


Spring 2014 34

Date post:	01-Nov-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

COMP 355 Advanced Algorithms - Rhodescs.rhodes.edu/welshc/COMP355_S14/Lecture2.pdf · 2014. 1....

Documents