242-535 ADA: 0. Preliminaries1 Objective o to give some background on the course Algorithm Design...

242-535 ADA: 0. Preliminaries

1

• Objectiveo to give some background on the course

Algorithm Design and Analysis

(ADA)242-535, Semester 1 2014-2015

0. Preliminaries

Who I am:Andrew DavisonWiG [email protected]

Please askquestions


2

1. What is an Algorithm?2. Meeting Times / Locations3. Workload4. Exercises5. Course Materials6. Books7. Videos8. Web Sites

Overview


3

• An algorithm is a finite set of unambiguous instructions for solving a problem. o An algorithm is correct if on all legitimate inputs, it

outputs the right answer in a finite amount of time

• Can be expressed as o pseudocodeo flow chartso text in a natural language (e.g. English)o computer code

1. What is a Algorithm?


4

The theoretical study of how to solve computational problems

• sorting a list of numbers• finding a shortest route on a map• scheduling when to work on homework• answering web search queries• and so on...

Algorithm Design


5

• Their impact is broad and far-reaching.o Internet. Web search, packet routing, distributed file

sharing, ... o Biology. Human genome project, protein folding, ...o Computers. Circuit layout, file system, compilers, ...o Computer graphics. Movies, video games, virtual

reality, ...o Security. Cell phones, e-commerce, voting machines, ...o Multimedia. MP3, JPG, DivX, HDTV, face recognition, ...o Social networks. Recommendations, news feeds,

advertisements, ...o Physics. N-body simulation, particle collision simulation,

...

The Importance of Algorithms


6

• Ten algorithms having "the greatest influence on the development and practice of science and engineering in the 20th century".o Dongarra and Sullivan

Top Ten Algorithms of the CenturyComputing in Science and EngineeringJanuary/February 2000

o Barry CipraThe Best of the 20th Century: Editors Name Top 10 AlgorithmsSIAM NewsVolume 33, Number 4, May 2000• http://www.siam.org/pdf/news/637.pdf

The Top 10 Algorithms of the 20th

Century


7

• 1946: The Metropolis (Monte Carlo) Algorithm. Uses random processes to find answers to problems that are too complicated to solve exactly.

• 1947: Simplex Method for Linear Programming. A fast technique for maximizing or minimizing a linear function of several variables, applicable to planning and decision-making.

• 1950: Krylov Subspace Iteration Method. A technique for rapidly solving the linear equations that are common in scientific computation.

What are the Top 10?


8

• 1951: The Decompositional Approach to Matrix Computations. A collection of techniques for numerical linear algebra.

• 1957: The Fortran Optimizing Compiler.

• 1959: QR Algorithm for Computing Eigenvalues. A crucial matrix operation made swift and practical. Application areas include computer vision, vibration analysis, data analysis.

• 1962: Quicksort Algorithm. We will look at this.


9

• 1965: Fast Fourier Transform (FFT). It breaks down waveforms (like sound) into periodic components. Used in many different areas (e.g. digital signal processing , solving partial differential equations, fast multiplication of large integers.)

• 1977: Integer Relation Detection. A fast method for finding simple equations that explain collections of data.

• 1987: Fast Multipole Method. Deals with the complexity of n-body calculations. It is applied in problems ranging from celestial mechanics to protein folding.


10

• Simple recursive algorithms• Divide and conquer• Backtracking• Dynamic programming• Greedy algorithms• Brute force• Randomized algorithms

Some Algorithm Types


11

• A simple recursive algorithm:o Non-recursive base caseo Recurs with a simpler subproblem

• Examples:o Count the number of occurrence of an element in a treeo Test if a value occurs in a listo Fibonnaci number calculation

• Several of the other algorithm types use recursion in more complex ways

Simple Recursive

11


12

• The Fibonacci sequence:o 1, 1, 2, 3, 5, 8, 13, 21, 33, ...

• Find the nth Fibonacci number:fib(int n) if (n <= 1) return 1; else return fib(n-1) + fib(n-2);

Fibonnaci numbers


13

• Execution of fib(5):lots of repeated work,that only gets worsefor bigger n


14

• A divide and conquer algorithm :o Divide the problem into smaller subproblems o Combine the solutions to the subproblems into a

solution to the original problem

• Examples:o quicksorto merge sorto binary search

Divide and Conquer


15

• Backtracking algorithms use a depth-first recursive searcho involves choice (non-determinism)o backtracking means "go back to where you came from"

• Examples:o search a mazeo color a map with no

more than four colors

Backtracking


16

• A dynamic programming algorithm remembers past results and uses them to find new results.o multiple solutions exist; find the “best” one (the optimal

one)

o the problen contains overlapping (repeated) subproblems• solutions are stored and reused

Dynamic Programming


17

• Finding the nth Fibonacci number involves lots of repeated work (overlapping subproblems) that can be avoided using memorization:

Fibonacci numbers Again

17


18

• An optimization problem: find the best solution

• A “greedy algorithm” sometimes works well for optimization problems, and is easy to code

• At each step:o use the best solution you can get right now, without regard

for future stepso You hope that by choosing a local optimum at each step,

you will end up with a globally optimum final solution

Greedy algorithms

18


19

• Count out some money using the fewest possible notes and coins

• A greedy algorithm will take the largest possible note or coin at each step

• Example: count out $6.39 using:• a $5 bill• a $1 bill // to make $6• a 25¢ coin // to make $6.25• a 10¢ coin // to make $6.35• four 1¢ coins // to make $6.39

Example: Counting Money


20

• “Krons” money come in 1, 7, and 10 coins

• Count out 15 krons:o A 10 kron pieceo Five 1 kron pieces, for a total of 15 krons

• This requires 6 coins, but a better solution is two 7 kron pieces and one 1 kron piece (3 coins)

• The greedy algorithm 'fails' because its final solution is not the best (not globally optimal)

Greedy Algorithms Can 'Fail'


21

• A brute force algorithm tries all possibilities until a satisfactory solution is found.

• Often, brute force algorithms require exponential running time (very large time, so very slow)

• Various heuristics and optimizations can be usedo heuristic means “a rule of thumb”o look for sub-optimal solutions (not the best), which can

be calculated more quickly than the best one

Brute Force


22

• A randomized algorithm uses a random number to make a choice during the computation o faster than calculating a choiceo the choice may be just as good

• Examples: o Quicksort randomly chooses a pivot

o Factor a large number by choosing random numbers as possible divisors

Randomized Algorithms


23

The theoretical study of algorithm performance and resource usage.

Performance isn't the only important things for code:

• modularity • user-friendliness• correctness • programmer time• maintainability • simplicity• functionality • extensibility• robustness • reliability

Analysis of Algorithms

This subject isn't aboutthese things.


24

• It help us to understand algorithm scalability.

• Performance often draws the line between what is feasible and what is impossible.

• Algorithmic mathematics provides a precise way to talk about program behavior.

• The lessons of program performance generalize to other computing resources.

Why study Algorithm Analysis?


25

• Mathematical induction, running time of programs; growth of functions

• Divide-and-conquer; comparison and linear sorts

• Dynamic programming• Greedy algorithms• Elementary graph algorithms, minimum

spanning trees, shortest path problems, maximum flow

• String matching• Computational geometry• NP completeness; approximation algorithms

Course Structure


26

• Wednesday 9:00 – 10:30 R301-2Thursday 10:30 – 12:00 R301-2

• I want to change these times to be 3 classes/week, each of 1 hour.

• Tell me your preferences.

2. Meeting Times / Locations


27

• Mid-term exam: 35% (2 hours)o week 8

• Final exam: 45% (3 hours)o weeks 17-18

• Two exercises: 20% (2*10)• weeks 7-8 and weeks 15-16

3. Workload


28

• I may take registration at the start of a class.

• If someone is not there, they lose 1% (unless they have a good excuse).

• A maximum of 10% can be losto deducted from your final mark

Non-Attendence Penalty


29

• The two exercises are worth a total of 20% (each worth 10%).

• They will be maths problems and/or algorithms to design/write.

4. Exercises

continued


30

• Planned exercise times (which may change):o ex. 1 in weeks 7-8o ex. 2 in weeks 15-16

• Cheating will result in 0 marks.o YOU HAVE BEEN WARNED!!


31

• All the handouts (and other materials) will be placed on-line at

http://fivedots.coe.psu.ac.th/ Software.coe/242-535_ADA/

• Print 6 slides-per-page, grayscale, and bring to class.

5. Course Materials


32

• Introduction to AlgorithmsThomas Cormen, Charles Leiserson, Ronald Rivest, Clifford SteinMcGraw Hill, 2003, 2nd editiono mathematical, advanced, the standard texto now up to version 3 (MIT)

o lots of resources online; see video section

6. Books

continued


33

• AlgorithmsRobert Sedgewick, Kevin WayneAddison-Wesley, 2011, 4th ed.o implementation (Java) and theoryo intermediate level

• Data Structures and Algorithms in JavaRobert LaforeSams Publishing, 2002, 2nd ed.o Java examples; oldo basic level; not much analysis


34

• Algorithms UnlockedThomas H. CormenMIT Press, March 2013

• Nine Algorithms that Changed the FutureJohn MacCormickPrinceton University Press, 2011o http://users.dickinson.edu/~jmac/

9algorithms/• search engine indexing, pagerank, public key

cryptography, error-correcting codes, pattern recognition, data compression, databases, digital signatures, computablity

Fun Overviews


35

• Algorithmic PuzzlesAnany Levitin, Maria LevitinOxford University Press, , 2011

• Algorithmics: The Spirit of ComputingDavid Harel, Yishai FeldmanAddison-Wesley; 3 ed., 2004(and Springer, 2012)o http://

www.wisdom.weizmann.ac.il/~harel/algorithmics.html


36

• The New Turing Omnibus: Sixty-Six Excursions in Computer ScienceA. K. DewdneyHolt, 1993o 66 short article; e.g. detecting primes, noncomputable

functions, self-replicating computers, fractals, genetic algorithms, Newton-Raphson Method, viruses


37

• MIT 6.046J / 18.410J Intro. to Algorithms, Fall 2005o http://ocw.mit.edu/6-046JF05

• original course website for Cormen book

o http://videolectures.net/mit6046jf05_introduction_algorithms/• video and slides side-by-side

o http://www.catonmat.net/category/introduction-to-algorithms• notes taken while

watching the videos

7. Videos


38

• Hi-tech TrekRoyal Institution Christmas Lectures 2008o A hi-tech trek through the world of computer science by

Professor Chris Bishop; aimed at school kids.• Lecture 1: Breaking the speed limit• Lecture 2: Chips with everything• Lecture 3: Ghost in the machine• Lecture 4: Untangling the web• Lecture 5: Digital intelligence

o http://richannel.org/christmas-lectures/2008/

o I have copies on a DVD


39

• Algorithm Tutorialso http://community.topcoder.com/tc?module=Static&

d1=tutorials&d2=alg_index

• Algorithmyo http://en.algoritmy.net/

• brief explanations and code

• Algorithmisto http://algorithmist.com/index.php/Main_Page

• explanations and code

• Wikipaedia page for an algorithmo e.g. http://en.wikipedia.org/wiki/Quicksort

8. Web Sites

Date post:	23-Dec-2015
Category:	Documents
Upload:	silas-henderson
View:	225 times
Download:	0 times

242-535 ADA: 0. Preliminaries1 Objective o to give some background on the course Algorithm Design...

Documents