Date post: | 01-Jan-2016 |
Category: |
Documents |
Upload: | howard-nelson |
View: | 216 times |
Download: | 2 times |
ECOE 556, 2
This Week’s Outline Introduction and basics
Fundamental concepts, formal definitions, pseudocode Motivation for studying algorithms Analyzing algorithms
Growth of functions Asymptotic notation
Recurrence equations Solving recurrences to compute asymptotic complexity
ECOE 556, 3
Algorithms Al-gorithm: Named after the 9th century Arab mathematician
al Harezmi (not Al Gore) An algorithm: A tool for solving a well-specified computational
problem. Problem statement:
Inputs, outputs The desired input/output relationship
Algorithm describes a specific computational procedure for producing the required output
Example: Sorting Input: A sequence of n numbers <a1,a2,…,an> Output: A reordering <a’1,a’2,…,a’n> of the sequence such that
a’1 a’2 … a’n Given the input <6, 3, 1, 7>, the algorithm should produce
<1, 3, 6, 7> Called an instance of the problem
ECOE 556, 4
When do we need algorithmic solutions? Almost every engineering application The Human Genome Project
Bioinformatics in general
The Internet Routing algorithms Searching, indexing
E-commerce Cryptography Authentication
Scheduling, optimization of industrial processes Numerical algorithms
e.g. matrix multiplication, finite-element simulation
ECOE 556, 5
When do we need non-trivial data structures?
Almost any industrial strength software tool needs them Need to represent
Sets Relations Discrete functions Sequences Queues …
Have varied requirements for what kind of operations to perform on them. Insert, delete, lookup Next, previous Minimum, maximum …
ECOE 556, 6
Why worry about efficiency? Hardware and memory are fast and cheap. Computing
getting cheaper and cheaper. Intelligent manpower is expensive Why worry about efficient algorithms and data
structures?
ECOE 556, 7
Why worry about algorithms and data structures?
Algorithms are as important a technology as other advanced technologies, such as Hardware architecture Graphical user interfaces Programming technologies Networks
A stupid approach uses up computing power faster than you might think. Examples: Sorting a million numbers
Interactive graphics: Algorithms must terminate in 1/30 of a sec.
O(n2) algorithm
2n2
instructions109
inst/second2000 sec.
O(n lg n) algorithm
50 n lg ninstructions
107
inst/second100 sec.
ECOE 556, 8
Movie: Sorting Algorithms http://www.iti.fh-flensburg.de/lang/algorithmen/sortieren/sortcontest/sortcontest.htm
ECOE 556, 9
Computationally hard problems Polynomial complexity:
Requires resources O(P(n)) for some polynomial Exponential complexity Why divide this way?
Most polynomial problems have low-degree polynomial complexity
Any exponential is asymptotically bigger than any polynomial Grey area in between
No known polynomial algorithm No proof that one doesn’t exist
Interesting class of problems: NP-complete problems Come up very often in practical applications All computationally equivalent
If an efficient algorithm exists for one, all NP-complete problemsare polynomially solvable
ECOE 556, 10
Analyzing algorithms Is the algorithm correct?
Does it terminate on all inputs? Does it produce the required output?
What amount of resources does the algorithm use up? Memory Communication bandwidth Logic gates (if implemented in hardware), speed of logic circuit Running time
Machine model: Single processor, random-access machine One instruction at a time, no concurrent processing
Memory access also counts as one instruction Idealized, but adequate for characterizing general behavior of
algorithms
ECOE 556, 11
Example: Insertion Sort Takes array A[1..n] containing a sequence of length n to
be sorted Sorts the array in place
Numbers rearranged inside A with at most a constant number of them stored outside.
ECOE 556, 16
Correctness of INSERTION-SORT
Loop invariant: At the start of each iteration, the subarray A[1..j-1] contains the
elements originally in A[1..j-1] but in sorted (increasing) order Prove initialization, maintenance, termination Invariant must imply interesting property about algorithm
ECOE 556, 17
Analyzing running time Depends on input size: How to quantify?
Number of items in input Number of bits required to encode input
Running time = Number of primitive steps executed in computation
Each line of pseudocode takes constant time
ECOE 556, 20
Running time of INSERTION-SORT Depends on the input instance
Best case: Worst case:
We usually care about the worst and average case Why worst case?
Upper bound May occur often. e.g. search, database look-up Average case often as bad as worst case
Many exceptions Don’t need to assume particular input probability distribution
Simplifying assumption: We care about rate of growth of running time only. Called “asymptotic complexity”.
ECOE 556, 22
How to design algorithms? Many styles
We’ll see examples throughout the semester
Insertion sort was an example of an “incremental algorithm”
Another common paradigm: Divide and conquer Steps
Divide into simpler/smaller subproblems Solve subproblems recursively Combine results of subproblems
Example: Merge-sort
ECOE 556, 23
Merge Sort Divide into two subsequences of half size Call yourself recursively on the halves Combine results
How do you prove MergeSort correct?
ECOE 556, 24
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
ECOE 556, 25
Recurrence equation for MERGE-SORT
Express T(n) in terms of subproblems and cost of division into subproblems.
T(n/2)
O(n)
ECOE 556, 26
Recurrence equation for MERGE-SORT
Express T(n) in terms of subproblems and cost of division into subproblems.
T(n/2)
O(n)
T(n) = (n lg n) Insertion sort was (n2)
ECOE 556, 27
Solving recurrences The substitution method The recursion tree method
A graphical method for coming up with a good guess Guess needs to be verified using substitution method
The master method
Technicalities Integer arguments. Floors, ceilings Boundary conditions: T(n) constant for some small enough n. Powers of k
ECOE 556, 28
Substitution method Guess the form of the solution Use mathematical induction to verify correctness
Example: T(n) = 2T(n/2) + n
Guess T(n) = O(n lg n) Prove that T(n) cn lg n for appropriate choice of c
using strong induction
How about T(n) = O(n)?