29
Computer Sciences Department 1
Computer Sciences Department 1
Greedy Algorithms Chapter 16
(16.1, 16.2, and 16.3)
3 Computer Sciences Department
Problem Formulation
Examples
The Basic Problem
Principle of optimality
Important techniques:
dynamic programming (Chapter 15),
greedy algorithms (Chapter 16)
Backtracking (study - slides)
4
Objectives
Computer Sciences Department
A greedy algorithm always makes the choice that looks best at the moment.
Greedy algorithms do not always yield optimal solutions, but for many problems they do.
The greedy method is quite powerful and works well for a wide range of problems.
5
Introduction
Computer Sciences Department
6
16.1 An activity-selection problem (self study)
Computer Sciences Department
We shall then observe that we need only consider one choice—the greedy choice—and that when we make the greedy choice, one of the subproblems is guaranteed to be empty, so that only one nonempty subproblem remains. Based on these observations, we shall develop a recursive greedy algorithm to solve the activity-scheduling problem. We shall complete the process of developing a greedy solution by converting the recursive algorithm to an iterative one.
7
Observation
Computer Sciences Department
8
The optimal substructure of the activity-selection problem
Computer Sciences Department
9
A recursive solution
Computer Sciences Department
10
Converting a dynamic-programming solution to a greedy solution
Computer Sciences Department
11
A recursive greedy algorithm
Computer Sciences Department
12
A recursive greedy algorithm
Computer Sciences Department
13
A recursive greedy algorithm
Computer Sciences Department
14
Elements of the greedy strategy
Computer Sciences Department
15
Greedy-choice property
Computer Sciences Department
16
Optimal substructure
Computer Sciences Department
17
16.2 Greedy versus dynamic programming
Computer Sciences Department
Algorithm structure Iterative execute action in loop
Recursive reapply action to subproblem(s)
Problem type Satisfying find any satisfactory solution
Optimization find best solutions
Suppose you have to make a series of decisions, among various choices, where: You don’t have enough information to know what to choose
Each decision leads to a new set of choices
Some sequence of choices (possibly more than one) may be a solution to your problem
Computer Sciences Department 18
Backtracking
Backtracking is a form of recursion
The usual scenario is that you are faced with a number of options, and you must choose one of these. After you make your choice you will get a new set of options; just what set of options you get depends on what choice you made. This procedure is repeated over and over until you reach a final state. If you made a good sequence of choices, your final state is a goal state; if you didn't, it isn't.
Conceptually, you start at the root of a tree; the tree probably has some good leaves and some bad leaves, though it may be that the leaves are all good or all bad. You want to get to a good leaf. At each node, beginning with the root, you choose one of its children to move to, and you keep this up until you get to a leaf.
Suppose you get to a bad leaf. You can backtrack to continue the search for a good leaf by revoking your most recent choice, and trying out the next option in that set of options. If you run out of options, revoke the choice that got you here, and try another choice at that node. If you end up at the root with no options left, there are no good leaves to be found.
Computer Sciences Department 19
Backtracking
Starting at Root, your options are A and B. You choose A.
At A, your options are C and D. You choose C.
C is bad. Go back to A.
At A, you have already tried C, and it failed. Try D.
D is bad. Go back to A.
At A, you have no options left to try. Go back to Root.
At Root, you have already tried A. Try B.
At B, your options are E and F. Try E.
E is good. Well done!
http://www.cis.upenn.edu/~matuszek/cit594-2012/Pages/backtracking.html
Computer Sciences Department 20
Strategy
21
0-1 knapsack problem
Computer Sciences Department
22 Computer Sciences Department
23
16.3 Huffman codes
Prefix codes (no codeword)
Computer Sciences Department
24
We interpret the binary codeword for a character as the path from the root to that character, where 0 means “go to the left child” and 1 means “go to the right child.” Figure 16.4 shows the trees for the two codes of our example. Note that these are not binary search trees, since the leaves need not appear in sorted order and internal nodes do not contain character keys.
Computer Sciences Department
25
Constructing a Huffman code
Computer Sciences Department
26
Correctness of Huffman’s algorithm
Computer Sciences Department
Problem space consists of states (nodes) and actions (paths that lead to new states). When in a node can can only see paths to connected nodes
If a node only leads to failure go back to its "parent" node. Try other alternatives. If these all lead to failure then more backtracking may be necessary.
Computer Sciences Department 27
Start Success!
Success!
Failure
Example
Lemma 16.3
Theorem 16.4
28
Read only
Computer Sciences Department
A greedy algorithm always makes the choice that looks best at the moment. Greedy algorithms do not always yield optimal solutions, but for many problems they do.
Backtracking is a form of recursion.
Backtracking - A scheme for solving a series of sub-problems each of which may have multiple possible solutions and where the solution chosen for one sub-problem may affect the possible solutions of later sub-problems.
To solve the overall problem, we find a solution to the first sub-problem and then attempt to recursively solve the other sub-problems based on this first solution. If we cannot, or we want all possible solutions, we backtrack and try the next possible solution to the first sub-problem and so on. Backtracking terminates when there are no more solutions to the first sub-problem
29
Conclusion -2
Computer Sciences Department
