CSC 8301 – Design and Analysis of Algorithms:...

Villanova University Department of Computing Sciences

CSC 8301 – Design and Analysis of Algorithms:

Introduction

Professor Henry CarterFall 2016


Greatest Common Divisor

• Problem: Find gcd(m,n), the greatest common divisor of two nonnegative, not both zero integers m and n

• Ex.: gcd(60,24) = gcd(10,50) = gcd(47,0) =


Euclid’s Algorithm

• Euclid’s algorithm is based on repeated application of this equality

• gcd(m, n) = gcd(n, m mod n)

• Ex.: gcd(60,24) =

• while n ≠ 0 do r ← m mod nm← n n ← r return m


What is an algorithm?

• An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.

• Muhammad ibn Musa al-Khwarizmi – 9th century Islamic mathematician


What is an algorithm?

problem

“computer” input

algorithm

output


This course ...

• This course is a theory course covering basic algorithm analysis and design principles. We will investigate a series of specific algorithms designed to solve common problems and consider their broader applicability. Topics will include:

‣ analysis of recursive and nonrecursive algorithms; design techniques: brute force, greedy, divide-and-conquer, dynamic programming, backtracking, branch-and-bound, space and time tradeoffs; NP-completeness; approximation algorithms; computational problems: sorting, searching, string processing, graphs, arithmetic, linear algebra.

6


Who cares about algorithms?

• Computer Scientists

‣ Foundational theory for the field

• Computer Engineers/Developers

‣ Significant impact on application performance and correctness

• Logistical planning in general

‣ Scheduling? Sorting? Routing? Puzzles and games?


You need to understand ...

• Basic programming and data structure concepts

• Discrete math and logic

• Logical proofing techniques

8


Should you take this course?

• Students without the necessary background.

• Students who are not willing to put forth significant effort.

• Students who are not interested in learning.

9


Goals

• My goal: to provide you with the tools to analyze algorithms and use common techniques to build programs.

‣ Speed/Space/Correctness/Optimality analysis

‣ Common techniques for search, sort, etc.•This is going to be a hard course. The key to success is

sustained effort. Failure to keep up with readings and homework will likely result in poor grades, and ultimately little understanding of the course material.

•Pay-off: A strong knowledge of algorithm design is among the most desirable abilities for anycomputer scientists

•Technical interviews test precisely this!!!

10


Course Materials

• Website - I am maintaining the course website at

‣ http://www.csc.villanova.edu/~carterh/Courses/csc8301/f16/

• Course assignments, slides, and other artifacts will be made available on the course website.

• Course textbook

‣ Anany Levitin, Design and Analysis of Algorithms, 3rd edition

11


Course Calendar

• The course calendar as all the relevant readings, assignments and test dates

• The calendar page contains electronic links to assignments and online papers assigned for course readings.

• Please check the website frequently for announcements and changes to the schedule. Students are responsible for any change on the schedule (I will try to make announcements in class).

12

6/20/2016 CSC 8301 - Design and Analysis of Algorithms - Professor Henry Carter

file:///Users/carterh/Sites/csc8301/f16/schedule.html 1/1

COURSE CALENDAR

Below is the calendar for this semester course. This is the preliminary schedule, which may need to be alteredas the semester progresses. It is the responsibility of the students to frequently check this webpage forschedule, readings, and assignment changes. As the professor, I will attempt to announce any change to theclass, but this webpage should be viewed as authoritative. If you have any questions, please contact me(contact information is available at the course homepage).

Date Topic Assignments Due Readings/Discussions (do readings before class) Slides

08/25/16 Introduction Course syllabus (link)

Textbook, chapter 1

Assignment #0 Introductions

08/30/16 Algorithm Analysis Assignment #0 Textbook, Chapter 2.12.2

09/01/16 Nonrecursive Analysis Textbook, Chapter 2.3

09/06/16 Recursive Analysis Textbook, Chapter 2.42.5

09/08/16 Bruteforce Textbook, Chapter 3.13.2

09/13/16 Exhaustive Search Textbook, Chapter 3.4

09/15/16 Tree Searching Textbook, Chapter 3.5

09/20/16 Insertion and Topological Sort Textbook, Chapter 4.14.2

09/22/16 Binary Search Textbook, Chapter 4.4

09/27/16 Variable Decrease Algorithms Textbook, Chapter 4.5

09/29/16 Merge and Quick Sort Textbook, Chapter 5.15.2

10/04/16 Further Divideandconquer Textbook, Chapter 5.35.5

10/06/16 Midterm Exam10/11/16 Fall Break No Class10/13/16 Fall Break No Class10/18/16 Presorting and Gaussian Elimination Textbook, Chapter 6.16.2

10/20/16 Balanced Search Trees Textbook, Chapter 6.3

10/25/16 Representation Change Textbook, Chapter 6.46.5

10/27/16 Problem Reduction Textbook, Chapter 6.6

11/01/16 String Matching Textbook, Chapter 7.17.2

11/03/16 Dynamic Programming Textbook, Chapter 8.18.2

11/08/16 Dynamic Programming Textbook, Chapter 8.38.4

11/10/16 Prim's and Kruskal's Algorithms Textbook, Chapter 9.19.2

11/15/16 Dijkstra's and Huffman's Algorithm Textbook, Chapter 9.39.4

11/17/16 Iterative Improvement Textbook, Chapter 10.110.2

11/22/16 Iterative Improvement Textbook, Chapter 10.310.4

11/24/16 Thanksgiving No Class11/29/16 Lower Bound Arguments and Decision Trees Textbook, Chapter 11.111.2

12/01/16 P, NP, NPComplete and Numerical Algorithms Textbook, Chapter 11.311.4

12/06/16 Backtracking and Branch and Bound Textbook, Chapter 12.112.2

12/08/16 Approximating NPHard Problems Textbook, Chapter 12.3

12/17/16 Final Exam 8:00am 10:50am


Grading

• Grading in this class will be distributed as follows:30% Mid-term Exam 40% Final Exam25% Quizzes5% Class Participation

• You need to average above 50% across both exams to pass

• You get the grade that you earn, so be surethat you earn a grade you like.

13


Homework

• Homeworks are designed to be difficult

‣ Expect to sweat

‣ Preparation for the exams

• Collaboration is allowed

‣ Talk to your fellow students and me!

14


Readings

• There are a large amount of readings in this course covering various topics. These assignments are intended to:

‣ Support the lectures in the course (provide clarity)

‣ Augment the lectures and provide a broader exposure to security topics.

• Students are required to do the reading!

• The material covered in class is not enough to fully understand the complex concepts of algorithm design and analysis. The reading is necessary to gain a complete understanding of the material and to do well on the homework and exams!

15


Quizzes

• 10 minute quizzes will take place every Friday

• Your 10 best quiz grades will determine 25% of your final grade

• These quizzes will be quick, so practice beforehand is critical to success


A Note on Academic Integrity

• As scientists and engineers, we must trust each other to make progress.

• Numerous examples exist to show the consequences of this breakdown.

‣ Jan Hendrik Schon...

• Academic dishonesty, whether from cheating, copying, or through any other dishonest practice will not be tolerated.

‣ I take this very personally - you should too.

17


Assignment 0 - Introduction

• Your first assignment is to write a one-page summary about yourself. Details can be found on the course webpage.

• A PDF of your write up is due on Blackboard on Monday.

18


Analyzing an algorithm

• Efficiency

• Correctness

‣ Precision

‣ Accuracy

• Optimality


Efficiency

• Speed efficiency

‣ Clock time

‣ Processor cycles

‣ Operation count

• Space (storage) efficiency

‣ Bytes of memory


Example: sequential search

• Input: a list of items and a keyOutput: the item matching the specified keyThink: Names and grade records

• Algorithm: Check each name in order through the entire list of names. Return either the matching record or “not present”

• What impacts the speed most?


Example: Euclid’s algorithm

• Euclid’s algorithm is based on repeated application of equality

• gcd(m, n) = gcd(n, m mod n)

• Ex.: gcd(60,24) =

• while n ≠ 0 do r ← m mod nm← n n ← r return m


Input size

• Does measuring strictly input size tell us anything?

‣ How does this map to speed?

‣ How does this compare across algorithms?

• Input size impacts speed, doesn’t directly determinespeed


Counting Operations

• Basic operation count determines speed

• Exact formula:

‣ e.g., C(n) = n(n-1)/2

• Formula indicating order of growth with specific multiplicative constant:

‣ e.g., C(n) ≈ 0.5 n2

• Formula indicating order of growth with unknown multiplicative constant:

‣ e.g., C(n) ≈ cn2


Additional Examples

Problem/algorithm Input size measure Basic operation

Search sequentially for a key in a list of n items Number of items n Key comparison

Multiply two n× nmatrices of real numbers

Matrix order n ornumber of elements 2n2

Multiplication of two real numbers

Primality checking of given integer n

#bits in n’s binary repres.b = log2(n) 1=

log2(n+1)Integer division

Graph traversal #vertices (and edges) Visiting a vertex


Orders of growth

• The relation of basic operations to size of input determines how efficient an algorithm performs as n grows

• Basically: does increasing the input size increase the operation count a LOT or a little?


Common orders of growth

1 – constant

log n – logarithmic

n – linear

n logn – “n-log-n” (linearithmic)

n2 – quadratic

n3 – cubic

2n – exponential

n! – factorial


Example values

n log2n n n log2n n² n³ 2ⁿ n!

10 3.3 101 3.3⋅10 102 103 103 3.6⋅106

102 6.6 102 6.6⋅102 104 106 1.3⋅1030 9.3⋅1015

103 10 103 10⋅103 106 109

104 13 104 13⋅104 108 1012

105 17 105 17⋅105 1010 1015

106 20 106 20⋅106 1012 1018


Basic Analysis Framework

• Efficiency based on input size and “basic operation count”

• This framework applies to both speed and storage analysis

• One assumption we are missing…



• Input: a list of items and a keyOutput: the item matching the specified keyThink: Names and grade records

• Algorithm: Check each name in order through the entire list of names. Return either the matching record or “not present”

• What operation do we count?

• What conditions impact the speed?


Case-wise efficiency

• Best case: what is the absolute fastest efficiency for a good input size n?

• Average case: out of several executions, what is the average efficiency?

• Worst case: what is the absolute slowest efficiency for a bad input of size n?

• Amortized efficiency: does repeating the algorithm many times reduce the per execution speed?



• Best case: 1

• Average case: n/2

‣ With one caveat

• Worst case: n


Recap

• Time and storage are most critical

• We measure efficiency in basic operation counts with respect to the input size

• Always consider: how fast does the speed/storage requirement grow (i.e. order of growth)?

• Different “qualities” of inputs affect the efficiency


Next Time...

• Levitin Chapter 2.1-2.2

‣ Remember, you need to read it BEFORE you come to class!

• Homework:

‣ 1.1: 6, 12

‣ 1.2: 1, 2

‣ 1.3: 4, 5

‣ 2.1: 5, 6, 8

34

Date post:	10-May-2018
Category:	Documents
Upload:	truongdieu
View:	224 times
Download:	2 times

CSC 8301 – Design and Analysis of Algorithms:...

Documents