Welcome to Cpt S 450Design and Analysis of AlgorithmsSyllabus and Introduction

CptS 450Design and Analysis of Algorithms

Spring 2015Instructor: John Miller, West 134E

jhmiller@tricity.wsu.eduClass web page can be found at

http://users.tricities.wsu.edu/~jhmiller

Objectives: To present the mathematical basis for quantitative analysis of

algorithms.To introduce students to various types of algorithms and analysis

techniques

Textbook: “Introduction to Algorithms” 3th Edition by Corman, et al

Course content:Chapters 1-2: IntroductionChapter 3: Growth of functionsChapter 4: Solving recurrencesChapter 5: Probabilistic analysisSelected topics from the following:Chapters 6-9: Sorting algorithmsChapters 15-17: Advanced design and analysis techniquesChapters 22-26: Graph algorithmsFinal exam

Required assignments: Prior approval is required for late submission homework assignments.No partial credit on homework. Full credit for corrected homework.

Criteria for student evaluation: homework 34%, quizzes 33%, final exam 33%.

Academic integrity:Academic integrity will be strongly enforced in this course. Any student caught cheating on any assignment will be given an F grade for the course and will be reported to the Office Student Standards and Accountability. Cheating is defined in the Standards for Student Conduct WAC 504-26-010 (3). It is strongly suggested that you read and understand these definitions:http://www.studentmediagroup.com/planners/palouse2012/#/18/

I encourage you to work with classmates on assignments. However, each student must turn in original work. No copying will be accepted. Students who violate WSU’s Standards of Conduct for Students will receive an F as a final grade in this course, will not have the option to withdraw from the course and will be reported to the Office Student Standards and Accountability. Cheating is defined in the Standards for Student Conduct WAC 504-26-010 (3). It is strongly suggested that you read and understand these definitions: http://www.studentmediagroup.com/planners/palouse2012/#/18/

Academic integrity is the cornerstone of the university. Any student who attempts to gain an unfair advantage over other students by cheating, will fail the assignment and be reported to the Office Student Standards and Accountability. Cheating is defined in the Standards for Student Conduct WAC 504-26-010 (3). http://www.studentmediagroup.com/planners/palouse2012/#/18/

Campus Safety StatementWSU Tri-Cities is committed to maintaining a safe environment for its faculty, staff and students. The Campus Safety Plan can be found at http://www.tricity.wsu.edu/safetyplan/See also the WSU Office of Emergency Management site at http://oem.wsu.edu/emergenciesUp-to-date WSU emergency alerts are available at http://alert.wsu.edu/

Disability Services Reasonable Accommodations Statement: Reasonable accommodations are available for students who have a documented disability. Classroom accommodation forms are available through the Disability Services Office. If you have a documented disability, even if it’s temporary, make an appointment as soon as possible with the Disability Services Coordinator, Cherish Tijerina Pearson, West Building, Room 269J, at 372-7352 or ctijerina@tricity.wsu.edu.You will need to provide your instructor with the appropriate classroom accommodation form. The form should be completed and submitted during the first week of class. Late notification can delay your accommodations or cause them to be unavailable. All accommodations for disabilities must be approved through the Disability Services Coordinator.

Algorithm is sequence of operations that transforms input to output

Example: Sort N itemsinput sequence (a1, a2, … aN) output some permutations that is sorted

Most often items to be sorted do not have a numerical valueexample: medical recordsdefine keys that point to satellite data

Algorithms: the big picture

Example: Insertion sortIn partially sorted intermediates, find where next item belongsCreate storage allocation and insert the itemRepeat until all items are sorted

Important questions about this idea:will it work?how much storage is required?are there problems with numerical precision?how fast does it run?

To get answers to these questions, we need a pseudocode.

Algorithm begins with an idea

Operation of insertion sort on 6 items

To write a pseudocode, a picture may be helpful

Starting with the second item, we copy it to a new variable.This creates a storage allocation that we fill with items in the partially sorted intermediate if their value is larger than the value of the item being sorted.

Note: this algorithm “sorts in place”

Note: indents show the structure of pseudocode

Worst case tj = j

Our primary interest is“How does runtime change with increasing input size?”

Linear: T(N) = a + bN Quadratic : T(N) = a + bN + cN2

Logarithmic: T(N) = aN(log(bN))

The coefficients in these expressions are hard to calculate and probably depend on properties of the input other than size.

To avoid this difficulty, we use “order of growth”Linear: T(N) = order(N) Quadratic : T(N) = order(N2)Logarithmic: T(N) = order(N(log(N)))

Order of growth concept is expanded in Chapter 3

In this class we focus on runtime

Line-by-line accounting of operations

In worst-case must compare aj to every element in A[1…j-1] With tj = j sums can be evaluated

Worst case analysis: tj = j

Proof that

2

)1(k

n

1k

nn

example of induction on integersusingif S(n-1) then S(n)

With tj = j sums can be evaluatedNon of the sums needed are exactly the arithmetic sum

Worst case analysis: tj = j

Use the arithmetic sum to evaluate the sums in the analysis of insertion sort runtime

We find the polynomial dependence of runtime on input size in worst case

Line-by-line analysis of insertion sort: worst case

Collect terms: T(n) = a + bn + cn2

This is an upper bound that has the possibility of being equal to the runtime In notation of Chapter 3, T(n) = O(n2)

Worst case tj = j

T(n) = a + bn + cn2

All quadratic terms come from analysis of “while” statement

Was algebra really necessary?

Loop invariant and correctness

Loop invariant = statement about iterative pseudo-code

To prove that the statement is true by induction we need

Initialization: true before 1st iteration

Maintenance: if true on ith iteration, also true on i+1

Termination: truth shows that the algorithm is correct

Application of loop invariant to insertion sort

Merge sort: A new idea about sorting

Divide-and-conquer phase: Recursively divide the problem into smaller pieces until get a solution by default

Execution phase: Given default solution, assemble the full solution from successively larger pieces

For efficiency analysis, consider execution phase only

Example of execution phase for input (5, 2, 4, 7, 1, 3, 2, 6)

Recursive sorting algorithm

For recursive algorithm, analysis of runtime begins with a recurrence relation that describes divide and conquer

T(N) = mT(N/k) + “overhead”

Overhead includes everything not involved in solving m problems of size N/k

Example: merge sort with even number of itemsT(2k) = 2T(2k-1) + cost of merging

To evaluate “cost of merging” need a pseudocode for Merge(A,p,q,r), where subarrays A(p) to A(q) and A(q+1) to A(r) are merged in sorted order

Use of “sentinel” cards

No nested loops.Cost is linear in total number of items to be merged = r - p

Merge does not use the fact that files to be merged are sortedSentinel cards make algorithm simpler but maybe slower

Although a faster Merge algorithm may exist, a linear Merge is sufficient to make Merge-Sort asymptotically optimal.

Recurrence for runtime becomes T(2k) = 2T(2k-1) + c2k for an even number of items.

In general, T(N)=2T(| N/2 |) + cN

In chapter 4 show that this recurrence has order of growth Nlog(N)

In chapter 8 show that order(Nlog(N)) is an asymptotic lower bound runtime of sorting algorithm based on comparison of values.

CptS 450 Spring 2015 Homework Assignment 1: due 1/21/15

Prove by induction on integers that recurrence T(2k) = 2 if k=1T(2k) = 2T(2k-1) + 2k if k>1 Note: assuming c = 1has solution T(2k) = k2k

Hint: use if S(n-1) then S(n)

Analysis of T(n)=2T(n/2)+cn by trees (n is even)

n/2istop = 1

All 3 trees are equally valid but not equally useful

Smallest tree just graphicalrepresentation of recurrence

As tree expands, overhead appears at each branch point

At each level get the total overhead

Calculate number of levels of D&C

Sum cost of levels

Express as order of growth

ex.2.3-4 p39 Sort A[1…n] by recursively sorting A[1…n-1] then insert A[n] (as in Insertion-Sort).

Write recurrence for worst-case run time.

What is the solution of this recurrence?