Post on 19-Jan-2016
transcript
Theory of Algorithms:Theory of Algorithms:Greedy TechniquesGreedy Techniques
Theory of Algorithms:Theory of Algorithms:Greedy TechniquesGreedy Techniques
James Gain and Edwin Blake{jgain | edwin} @cs.uct.ac.za
Department of Computer Science
University of Cape Town
August - October 2004
ObjectivesObjectives
To introduce the mind set of greedy techniques
To show some examples of greedy algorithms: Change Making
Huffman Coding
To discuss the strengths and weaknesses of greedy techniques
“Greed, for lack of a better word, is good! Greed is right! Greed works!” Gordon Grecko (Michael Douglas) in Wall Street
Greedy AlgorithmsGreedy Algorithms
Optimization problems solved through a sequence of choices that are:1. Feasible - satisfy problem constraints
2. Locally Optimal - best choice among all feasible options for that step
3. Irrevocable - no backing out
A Greedy grab of the best alternative, hoping that a sequence of locally optimal steps will lead to a globally optimal solution
Even if not optimal, sometimes an approximation is acceptable
Not all optimization problems can be approached in this manner!
Applications of the Greedy StrategyApplications of the Greedy Strategy
Optimal solutions: Change making Minimum Spanning Trees (MST) - Prim’s and Kruskal’s
Algorithms Single-source shortest paths - Dijkstra’s Algorithm Simple scheduling problems Huffman codes
Approximations: Traveling Salesman Problem (TSP) Knapsack problem Other combinatorial optimization problems
Change MakingChange Making
Problem: give change for a specific amount n, with the least number of coins of the denominations d1 > d2 > … > dm
Example: Smallest change for R2.54 using R5,
R2, R1, 50c, 20c, 10c, 5c, 2c, 1c R2 + 50c + 2c + 2c = R2.54
Algorithm: At any step choose the coin of the largest denomination
that doesn’t exceed the remaining total; Repeat
Note: Optimal for reasonable sets of coins
Minimum Spanning Tree ProblemMinimum Spanning Tree Problem
Problem: Given n points, connect them in the cheapest way so that there is a path between any pair of points
Spanning Tree (of a connected graph G ): A connected acyclic subgraph of G that includes all of G’s vertices.
Minimum Spanning Tree (of a weighted graph G ): A spanning tree of G with minimum total weight
Algorithms: Prim’s and Kruskal’s Algorithms Example: 3
42
14
26
1
3
3
42
1
2
1
3
Shortest Paths ProblemShortest Paths Problem
Single Source Shortest Paths Problem: Given a weighted graph G, find the shortest paths from
a source vertex s to each of the other vertices Solution:
Dijkstra’s Algorithm (a relative of Prim’s MST) for positive weights
3
42
14
26
1
3
2
1
2
42
1
2
3
31
4
Text CompressionText Compression
Variable length encoding: Compresses text by assigning codes of different length to characters
based on their probability of occurrence
Used by Samuel Morse in constructing telegraph codes
Prefix-free Codes: Codes are unique in that no code is a prefix for a code of another
character
Tree for Binary Codes: Leaves are characters
Left edge codes 0, right edge codes 1
The code of a character is a simple walk from root to leaf
Algorithm for Huffman CodingAlgorithm for Huffman Coding
Invented by David Huffman as part of a class assignment while he was an undergraduate
Algorithm: Construct a Huffman Tree that assigns shorter strings to higher
frequencies. This defines a Huffman Code
1. Initialize n one-node trees labeled with the characters of the alphabet. Record the frequency (weight) of each character in the root
2. REPEAT until a single tree is obtained:
Find two trees with the smallest weight
Make them the left and right sub-tree of a new tree and record the sum of their weights
in the root
Example: Huffman CodingExample: Huffman Coding
0.55C0.15
A0.4
B0.45
C0.15
A0.4
B0.45
0.55
C0.15
A0.4
B0.45
1.0
Char Code
A 01
B 1
C 00
Notes on Huffman CodingNotes on Huffman Coding
Compression Ratio: Standard measure of compression
CR = (x - y) / y * 100%, where x is compressed and y is uncompressed
Typically, 20-80% in the case of Huffman Coding
Yields optimal compression provided: The probabilities of character occurrences are
independent
Probabilities are known in advance
Strengths and Weaknesses of Strengths and Weaknesses of Greedy TechniquesGreedy Techniques
Strengths: Intuitively simple and appealing
Weaknesses: Only applicable to optimization problems
Doesn’t always produce an optimal solution
Summary To-DateSummary To-Date
Introduction 1.1-1.4
Fundamentals of the Analysis of Algorithm Efficiency
2.1-2.4, 2.6, 2.7
Brute Force 3.1-3.4
Divide-and-Conquer 4.1-4.3, 4.5, 4.6
Decrease-and-Conquer 5.1-5.6
Transform-and-Conquer 6.1, 6.5, 6.6
Space and Time Tradeoffs 7.1-7.2
Greedy Techniques 9.4