David Luebke 1 04/19/23

CS 332: Algorithms

Topological Sort

Minimum Spanning Trees

David Luebke 2 04/19/23

Review: Breadth-First Search

BFS(G, s) {

initialize vertices;

Q = {s}; // Q is a queue (duh); initialize to s

while (Q not empty) {

u = RemoveTop(Q);

for each v u->adj { if (v->color == WHITE)

v->color = GREY;

v->d = u->d + 1;

v->p = u;

Enqueue(Q, v);

}

u->color = BLACK;

}

}

v->p represents parent in tree

v->d represents depth in tree

David Luebke 3 04/19/23

Review: DFS Code

DFS(G)

{

for each vertex u G->V {

u->color = WHITE;

}

time = 0;

for each vertex u G->V {

if (u->color == WHITE)

DFS_Visit(u);

}

}

DFS_Visit(u)

{

u->color = YELLOW;

time = time+1;

u->d = time;

for each v u->Adj[] {

if (v->color == WHITE)

DFS_Visit(v);

}

u->color = BLACK;

time = time+1;

u->f = time;

}

David Luebke 4 04/19/23

Review: DFS Example

sourcevertex

David Luebke 5 04/19/23

Review: DFS Example

1 | | |

| | |

| |

sourcevertex

d f

David Luebke 6 04/19/23

Review: DFS Example

1 | | |

| | |

2 | |

sourcevertex

d f

David Luebke 7 04/19/23

Review: DFS Example

1 | | |

| | 3 |

2 | |

sourcevertex

d f

David Luebke 8 04/19/23

Review: DFS Example

1 | | |

| | 3 | 4

2 | |

sourcevertex

d f

David Luebke 9 04/19/23

Review: DFS Example

1 | | |

| 5 | 3 | 4

2 | |

sourcevertex

d f

David Luebke 10 04/19/23

Review: DFS Example

1 | | |

| 5 | 63 | 4

2 | |

sourcevertex

d f

David Luebke 11 04/19/23

Review: DFS Example

1 | 8 | |

| 5 | 63 | 4

2 | 7 |

sourcevertex

d f

David Luebke 12 04/19/23

Review: DFS Example

1 | 8 | |

| 5 | 63 | 4

2 | 7 |

sourcevertex

d f

David Luebke 13 04/19/23

Review: DFS Example

1 | 8 | |

| 5 | 63 | 4

2 | 7 9 |

sourcevertex

d f

David Luebke 14 04/19/23

Review: DFS Example

1 | 8 | |

| 5 | 63 | 4

2 | 7 9 |10

sourcevertex

d f

David Luebke 15 04/19/23

Review: DFS Example

1 | 8 |11 |

| 5 | 63 | 4

2 | 7 9 |10

sourcevertex

d f

David Luebke 16 04/19/23

Review: DFS Example

1 |12 8 |11 |

| 5 | 63 | 4

2 | 7 9 |10

sourcevertex

d f

David Luebke 17 04/19/23

Review: DFS Example

1 |12 8 |11 13|

| 5 | 63 | 4

2 | 7 9 |10

sourcevertex

d f

David Luebke 18 04/19/23

Review: DFS Example

1 |12 8 |11 13|

14| 5 | 63 | 4

2 | 7 9 |10

sourcevertex

d f

David Luebke 19 04/19/23

Review: DFS Example

1 |12 8 |11 13|

14|155 | 63 | 4

2 | 7 9 |10

sourcevertex

d f

David Luebke 20 04/19/23

Review: DFS Example

1 |12 8 |11 13|16

14|155 | 63 | 4

2 | 7 9 |10

sourcevertex

d f

David Luebke 21 04/19/23

Review: Kinds Of Edges

● Thm: If G is undirected, a DFS produces only tree and back edges

● Thm: An undirected graph is acyclic iff a DFS yields no back edges

● Thus, can run DFS to find cycles

David Luebke 22 04/19/23

1 |12 8 |11 13|16

14|155 | 63 | 4

2 | 7 9 |10

sourcevertex

d f

Tree edges Back edges Forward edges Cross edges

Review: Kinds of Edges

David Luebke 23 04/19/23

DFS And Cycles

● Running time: O(V+E)● We can actually determine if cycles exist in

O(V) time:■ In an undirected acyclic forest, |E| |V| - 1 ■ So count the edges: if ever see |V| distinct edges,

must have seen a back edge along the way■ Why not just test if |E| <|V| and answer the

question in constant time?

David Luebke 24 04/19/23

Directed Acyclic Graphs

● A directed acyclic graph or DAG is a directed graph with no directed cycles:

David Luebke 25 04/19/23

DFS and DAGs

● Argue that a directed graph G is acyclic iff a DFS of G yields no back edges:■ Forward: if G is acyclic, will be no back edges

○ Trivial: a back edge implies a cycle

■ Backward: if no back edges, G is acyclic○ Argue contrapositive: G has a cycle a back edge

Let v be the vertex on the cycle first discovered, and u be the predecessor of v on the cycle

When v discovered, whole cycle is white Must visit everything reachable from v before returning from DFS-Visit() So path from uv is yellowyellow, thus (u, v) is a back edge

David Luebke 26 04/19/23

Topological Sort

● Topological sort of a DAG:■ Linear ordering of all vertices in graph G such that

vertex u comes before vertex v if edge (u, v) G

● Real-world example: getting dressed

David Luebke 27 04/19/23

Getting Dressed

Underwear Socks

ShoesPants

Belt

Shirt

Watch

Tie

Jacket

David Luebke 28 04/19/23

Getting Dressed

Underwear Socks

ShoesPants

Belt

Shirt

Watch

Tie

Jacket

Socks Underwear Pants Shoes Watch Shirt Belt Tie Jacket

David Luebke 29 04/19/23

Topological Sort Algorithm

Topological-Sort()

{

Run DFS

When a vertex is finished, output it

Vertices are output in reverse topological order

}

● Time: O(V+E)● Correctness: Want to prove that

(u,v) G uf > vf

David Luebke 30 04/19/23

Correctness of Topological Sort

● Claim: (u,v) G uf > vf■ When (u,v) is explored, u is yellow

○ v = yellow (u,v) is back edge. Contradiction (Why?)○ v = white v becomes descendent of u vf < uf

(since must finish v before backtracking and finishing u)

○ v = black v already finished vf < uf

David Luebke 31 04/19/23

Minimum Spanning Tree

● Problem: given a connected, undirected, weighted graph:

1410

3

6 4

5

2

9

15

8

David Luebke 32 04/19/23

Minimum Spanning Tree

● Problem: given a connected, undirected, weighted graph, find a spanning tree using edges that minimize the total weight

1410

3

6 4

5

2

9

15

8

David Luebke 33 04/19/23

Minimum Spanning Tree

● Which edges form the minimum spanning tree (MST) of the below graph?

H B C

G E D

F

A

1410

3

6 4

5

2

9

15

8

David Luebke 34 04/19/23

Minimum Spanning Tree

● Answer:

H B C

G E D

F

A

1410

3

6 4

5

2

9

15

8

David Luebke 35 04/19/23

Minimum Spanning Tree

● MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees■ Let T be an MST of G with an edge (u,v) in the middle

■ Removing (u,v) partitions T into two trees T1 and T2

■ Claim: T1 is an MST of G1 = (V1,E1), and T2 is an MST of G2 = (V2,E2) (Do V1 and V2 share vertices? Why?)

■ Proof: w(T) = w(u,v) + w(T1) + w(T2)(There can’t be a better tree than T1 or T2, or T would be suboptimal)

David Luebke 36 04/19/23

Minimum Spanning Tree

● Thm: ■ Let T be MST of G, and let A T be subtree of T■ Let (u,v) be min-weight edge connecting A to V-A■ Then (u,v) T

David Luebke 37 04/19/23

Minimum Spanning Tree

● Thm: ■ Let T be MST of G, and let A T be subtree of T■ Let (u,v) be min-weight edge connecting A to V-A■ Then (u,v) T

● Proof: in book (see Thm 24.1)

David Luebke 38 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

David Luebke 39 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

1410

3

6 45

2

9

15

8

Run on example graph

David Luebke 40 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

1410

3

6 45

2

9

15

8

Run on example graph

David Luebke 41 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

0

1410

3

6 45

2

9

15

8

Pick a start vertex r

r

David Luebke 42 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

0

1410

3

6 45

2

9

15

8

Red vertices have been removed from Q

u

David Luebke 43 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

0

3

1410

3

6 45

2

9

15

8

Red arrows indicate parent pointers

u

David Luebke 44 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

14

0

3

1410

3

6 45

2

9

15

8

u

David Luebke 45 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

14

0

3

1410

3

6 45

2

9

15

8u

David Luebke 46 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

14

0 8

3

1410

3

6 45

2

9

15

8u

David Luebke 47 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

10

0 8

3

1410

3

6 45

2

9

15

8u

David Luebke 48 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

10

0 8

3

1410

3

6 45

2

9

15

8u

David Luebke 49 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

10 2

0 8

3

1410

3

6 45

2

9

15

8u

David Luebke 50 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

10 2

0 8 15

3

1410

3

6 45

2

9

15

8u

David Luebke 51 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

10 2

0 8 15

3

1410

3

6 45

2

9

15

8

u

David Luebke 52 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

10 2 9

0 8 15

3

1410

3

6 45

2

9

15

8

u

David Luebke 53 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

10 2 9

0 8 15

3

4

1410

3

6 45

2

9

15

8

u

David Luebke 54 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

5 2 9

0 8 15

3

4

1410

3

6 45

2

9

15

8

u

David Luebke 55 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

5 2 9

0 8 15

3

4

1410

3

6 45

2

9

15

8

u

David Luebke 56 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

5 2 9

0 8 15

3

4

1410

3

6 45

2

9

15

8

u

David Luebke 57 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

5 2 9

0 8 15

3

4

1410

3

6 45

2

9

15

8

u

David Luebke 58 04/19/23

Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

5 2 9

0 8 15

3

4

1410

3

6 45

2

9

15

8

u

David Luebke 59 04/19/23

Review: Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

What is the hidden cost in this code?

David Luebke 60 04/19/23

Review: Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

DecreaseKey(v, w(u,v));

David Luebke 61 04/19/23

Review: Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

DecreaseKey(v, w(u,v));

How often is ExtractMin() called?How often is DecreaseKey() called?

David Luebke 62 04/19/23

Review: Prim’s Algorithm

MST-Prim(G, w, r)

Q = V[G];

for each u Q key[u] = ; key[r] = 0;

p[r] = NULL;

while (Q not empty)

u = ExtractMin(Q);

for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u;

key[v] = w(u,v);

What will be the running time?A: Depends on queue binary heap: O(E lg V) Fibonacci heap: O(V lg V + E)

David Luebke 63 04/19/23

Single-Source Shortest Path

● Problem: given a weighted directed graph G, find the minimum-weight path from a given source vertex s to another vertex v■ “Shortest-path” = minimum weight ■ Weight of path is sum of edges■ E.g., a road map: what is the shortest path from

Chapel Hill to Charlottesville?