Greedy Algorithms

2

Change Making Problem

How to make 48 cents of change using coins of denominations of 25, 10, 5, and 1 so that the total number of coins is the smallest?The idea:

make the locally best choice at each step.

Is the solution optimal?

3

Greedy AlgorithmsA greedy algorithm makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution.The choice made at each step must be:

FeasibleSatisfy the problem’s constraints

locally optimalBe the best local choice among all feasible choices

IrrevocableOnce made, the choice can’t be changed on subsequent steps.

Do greedy algorithms always yield optimal solutions?Example: change making problem with a denomination set of 7, 5 and 1?

4

Applications of the Greedy StrategyOptimal solutions:

change makingMinimum Spanning Tree (MST)Single-source shortest paths Huffman codes

Approximations:Traveling Salesman Problem (TSP)Knapsack problemother optimization problems

5

Minimum Spanning Tree (MST)Spanning tree of a connected graph G: a connected acyclic subgraph (tree) of G that includes all of G’s vertices.

Minimum Spanning Tree of a weighted, connected graph G: a spanning tree of G of minimum total weight.

Example: 3

42

14

26

1

3

6

Prim’s MST algorithmStart with a tree , T0 ,consisting of one vertex

“Grow” tree one vertex/edge at a time Construct a series of expanding subtrees T1, T2, … Tn-1. .At each stage

construct Ti+1 from Ti by adding the minimum weight edge connecting a vertex in tree (Ti) to one not yet in tree

choose from “fringe” edges (this is the “greedy” step!)

Or (another way to understand it)expanding each tree (Ti) in a greedy manner by attaching to it the nearest vertex not in that tree. (a vertex not in the tree connected to a vertex in the tree by an edge of the smallest weight)

Algorithm stops when all vertices are included

Fringe edges: one vertex is in Ti and the other is not.Unseen edges: both vertices are not in Ti.

7

Examples3

42

14

26

1

3

a

edc

b1

5

24 6

3 7

8

The Key PointNotations

T: the expanding subtree.Q: the remaining vertices.At each stage, the key point of expanding the current subtree T is to

determine which vertex in Q is the nearest vertex. Q can be thought of as a priority queue:

The key(priority) of each vertex, key[v], means the minimum weight edge from v to a vertex in T. Key[v] is ∞ if v is not linked to any vertex in T.The major operation is to to find and delete the nearest vertex (v, for which key[v] is the smallest among all the vertices)

Remove the nearest vertex v from Q and add it and the corresponding edge to T.

With the occurrence of that action, the key of v’s neighbors will be changed.

9

ALGORITHM MST-PRIM( G, w, r ) //w: weight; r: root, the starting vertex

1. for each u ∈ V[G]2. do key[u] ← ∞3. π[u] ← NIL// π[u] : the parent of u

4. key[r] ← 05. Q ← V[G] //Now the priority queue, Q, has been built.

6. while Q ≠ ∅7. do u ← Extract-Min(Q) //remove the nearest vertex from Q

8. for each v ∈ Adj[u] // update the key for each of v’s adjacent nodes.

9. do if v ∈ Q and w(u,v) < key[v]10. then π[v] ← u11. Key[v] ← w(u,v)

10

Notes about Prim’s algorithmNeed priority queue for locating the nearest vertex

Use unordered array to store the priority queue:

Efficiency: Θ(n2)

use min-heap to store the priority queue

Efficiency: For graph with n vertices and m edges:(n + m) logn

O(m log n)

number of stages(min-heap deletions)

Key decreases/deletion from min-heapnumber of edges considered(min-heap key decreases)

11

Another Greedy Algorithm for MST: Kruskal

Edges are initially sorted by increasing weightStart with an empty forest“grow” MST one edge at a time

intermediate stages usually have forest of trees (not connected)

at each stage add minimum weight edge among those not yet used that does not create a cycle

at each stage the edge may:expand an existing treecombine two existing trees into a single treecreate a new tree

need efficient way of detecting/avoiding cyclesalgorithm stops when all vertices are included

12

Kruskal’s AlgorithmALGORITHM Kruscal(G)//Input: A weighted connected graph G = <V, E>//Output: ET, the set of edges composing a minimum spanning tree of G.

Sort E in nondecreasing order of the edge weights w(ei1) <= … <= w(ei|E|)

ET ∅; ecounter 0 //initialize the set of tree edges and its size

k 0 while encounter < |V| - 1 do

k k + 1if ET U {eik} is acyclic

ET ET U {eik} ; ecounter ecounter + 1return ET

13

Shortest Paths – Dijkstra’s Algorithm

Shortest Path ProblemsAll pair shortest paths (Floy’s algorithm)Single Source Shortest Paths Problem (Dijkstra’s algorithm): Given a weighted graph G, find the shortest paths from a source vertex s to each of the other vertices.

a

edc

b3

4

62 5

7 4

14

Prim’s and Dijkstra’s AlgorithmsGenerate different kinds of spanning trees

Prim’s: a minimum spanning tree.Dijkstra’s : a spanning tree rooted at a given source s, such that the distance from s to every other vertex is the shortest.

Different greedy strategiesPrims’: Always choose the closest (to the tree) vertex in the priority queue Q to add to the expanding tree VT.Dijkstra’s : Always choose the closest (to the source) vertex in the priority queue Q to add to the expanding tree VT.

Different labels for each vertexPrims’: parent vertex and the distance from the tree to the vertex..Dijkstra’s : parent vertex and the distance from the source to the vertex.

15

Shortest Paths-Dijkstra’s AlgorithmDijkstra’s algorithm: Similar to Prim’s MST algorithm, with the following difference:

Start with tree consisting of one vertex.

“Grow” tree one vertex/edge at a time.Construct a series of expanding subtrees T1, T2, …

Keep track of shortest path from source to each of the vertices in Ti

At each stage construct Ti+1 from Ti: add minimum weight edge connecting a vertex in tree (Ti) to one not yet in tree

choose from “fringe” edges (this is the “greedy” step!)

Algorithm stops when all vertices are included.

edge (edge (u*,uu*,u) with lowest ) with lowest dduu** + + ww((u*, uu*, u))

16

Dijkstra’s AlgorithmALGORITHM Dijkstra(G, s)//Input: A weighted connected graph G = <V, E> and a source vertex s//Output: The length dv of a shortest path from s to v and its penultimate vertex pv

for every vertex v in V

Initialize (Q) //initialize vertex priority in the priority queuefor every vertex v in V do

dv ∞ ; Pv null // Pv , the parent of vinsert(Q, v, dv) //initialize vertex priority in the priority queue

ds 0; Decrease(Q, s, ds) //update priority of s with ds, making ds, the minimumVT ∅

for i 0 to |V| - 1 do //produce |V| - 1 edges for the treeu* DeleteMin(Q) //delete the minimum priority elementVT VT U {u*} //expanding the tree, choosing the locally best vertexfor every vertex u in V – VT that is adjacent to u* do

if du* + w(u*, u) < dudu du + w(u*, u); pu u*Decrease(Q, u, du)

17

Notes on Dijkstra’s AlgorithmDoesn’t work with negative weights

Can you give a counter example?

Applicable to both undirected and directed graphsEfficiency

Use unordered array to store the priority queue: Θ(n2)

Use min-heap to store the priority queue: O(m log n)

18

Test 3, Apr. 13th, chapter 7,8,9Programming 2 (Apr. 21st)Homework 5 dueInstructor evaluation