CS 361 – Chapter 14

• Weighted graph– How would you represent?

• Shortest path algorithms from 1 vertex to any other:– General– Handling negative weight– Acyclic algorithm

• All pairs shortest paths• Minimum spanning tree

Dijkstra’s algorithm

• How do you find the shortest path in a weighted graph?• A greedy algorithm found by Edsger Dijkstra in 1959.• Can you find the shortest path in this example?

4 7

3

6 8

3

1 6

9

2

7 4

• Let’s say we want to go from “A” to “Z”.• The idea is to label each vertex with a

number – its best known distance from A. As we work, we may find a cheaper distance, until we “mark” or finalize the vertex.

1. Label A with 0, and mark A.

2. Label A’s neighbors with their distances from A.

3. Find the lowest unmarked vertex, and mark it. Let’s call this vertex “B”.

4. Recalculate distances for B’s neighbors via B. Some of these neighbors may now have a shorter known distance.

5. Repeat 3 and 4 until you mark Z.

4 7

3 4

2

A

C

Z

B

• First, we label A with 0. Mark A.

• The neighbors of A are B and C. Label B = 4 and C = 7.

• Now, the unmarked vertices are B=4 and C=7. The lowest is B.

• Mark B, and recalculate B’s neighbors via B. The neighbors are C and Z.– If we go to C via B, the total distance is

4 + 2 = 6. This is better than the old distance of 7. So, re-label C = 6.

– If we go to Z via B, the total distance is

4 + 3 = 7.

• Now, the unmarked vertices are C=6 and Z = 7. C is lowest.– Recalculate C’s neighbors…

• … Finally, Z becomes marked.

4 7

3 4

2

A

C

Z

B

• For clarification:

The idea is to label each vertex with a number: its best known distance from A. As we work, we may find a cheaper distance, until we “mark” or finalize the vertex.

• When you mark a vertex and look to recalculate distances to its neighbors:– We don’t need to recalculate a distance for a

vertex already marked. So, only consider unmarked neighbors.

– We only update (“relax”) a vertex’s distance if it is an improvement: if it’s shorter than what the distance already was.

• Practice with the 9 vertex example.

4 7

3 4

2

A

C

Z

B

Variants of problem

• Dijkstra’s algorithm works if graph also directed (one-way streets).

• But not if there are negative weights!– Greedy algorithm can’t predict future negative number to add.– Can’t just pad all edge values, because # of edges traversed

should not factor into total distance.– In this problem, need to make sure there is no “negative cycle”.

This would mean there is no solution.

• 2nd algorithm: Bellman-Ford.– Finds shortest path in directed weighted graph, allowing for

negative weights.– Tedious: for n-1 iterations, need to reconsider all edges in order

to “relax” the vertices.

Bellman-Ford algorithm

// One vertex identified s = source or origin.

Label s as 0 and other vertices as // representing "distance"

for i = 1 to n-1:

for each edge (u,v):

see if this edge can improve label(v)

// i.e. "relax" if necessary

for each edge (u,v):

if (label(v) > label(u) + weight(u,v))

return false!

return true

Example

• Practice Bellman-Ford algorithm on this simple example. Assume origin is A. We’re especially interested in distance to B.

• What is the label on each vertex after each iteration?• Is there a negative cycle?

A B C D

A 4 5

B 4

C –2

D 1 –3

DAG shortest paths

• Behind door #3 we have an elegant algorithm that finds shortest path as long as there is no directed cycle.

• Can handle negative weights.• Relies on a topological ordering of vertices.• Spreads the disease according to this ordering.

// Given s is the source or origin

topologically sort the graph

label(s) = 0 and label(all other vertices) = for u = each vertex in graph:

for v = each neighbor of u:

see if (u,v) can improve label(v)

Weighted graphs (2)

Weighted graphs, continued:• Single-source shortest paths √

– Dijkstra– Bellman-Ford (for negative weights)– DAG shortest path (relies on topological sort)

• All-pairs shortest paths– (Distance table on a map/atlas)– Can be done in O(n3) time.– Works for directed/undirected, negative weights too.

• Minimum spanning trees

Idea

• A variant of Floyd-Warshall we saw earlier for transitive closure• For each vertex in graph, we make successive updates to a

“distance” matrix

• D0 matrix uses initial distances info:

– D0 [ v, v ] = 0

– D0 [ u, v ] = adj [ u, v ] if there is an edge from u to v

– D0 [ u, v ] = if not

– The adjacency matrix values might already have these

• Computing Dk from Dk – 1

– We’re considering effect of vertex # k.

– Dk [ u, v ] = Dk – 1 [ u, v ] or D k – 1 [ u, k ] + D k – 1 [ k, v ] whichever is less.

Algorithm

// Given adj matrix of weighted graph with n vertices

// Assume that 0 entries are for vertex to itself, and // infinity means not adjacent

Arbitrarily number the vertices 1 to n.

for i = 1 to n:

for j = 1 to n:

D[0] [i][j] = adj[i][j]

for k = 1 to n:

for i = 1 to n:

for j = 1 to n:

D[k][i][j] = min(D[k-1][i][j],

D[k-1][i][k]+D[k-1][k][j])

Example

• Let’s observe what happens. It turns out we don’t need to consider all 125 cases.

• k = 1, i = 1, j = 1– To go from i=1 to j=1, do

we save time going thru k=1? No!

• k = 1, i = 1, j = 2– To go from i=1 to j=2, do

we save time going thru k=1?

– If i = k, this isn’t going to help because “going thru k” is same as “going thru i” which adds 0 to our time.

0 3 8 –4

0 1 7

4 0

2 –5 0

6 0

Example (2)

• k = 1, i = 2, j = 1– To go from i=2 to j=1, do we

save time going thru k=1?– If j = k, “going thru k” means

“going thru j”. But we’re already at j, so adding 0 won’t help.

• Another observation:– If i = j, then this distance is

already 0. If we improve upon this, then we’d have a negative cycle!

• Bottom line is to consider cases where i,j,k distinct.

• k = 1, i = 2, j = 3– Do we save time going from

2 1 3 ? No.

• k = 1, i = 2, j = 4– Do we save time going from

2 1 4 ? No.

0 3 8 –4

0 1 7

4 0

2 –5 0

6 0

Example (3)

• Another observation:– If k is not reachable from i,

then skip on to next value of i.

– In this case, for k = 1, we don’t need to consider i = 2 or i = 3.

• k = 1, i = 4, j = 5– Do we save time going

from 4 1 5? Yes!– So, D[1][4][5] equals this

improved distance, rather than merely copying D[0][4][5].

• Etc. thru k = 5.• Note: Be careful with

representing . We don’t want overflow with e.g.

+ 2 !

0 3 8 –4

0 1 7

4 0

2 –5 0

6 0

Final answer

• When algorithm terminates, we have 2-d array D[k], which gives shortest “distance” between any 2 vertices in graph.

• For example, D[5][3][5] = 3.• It does not tell us how to get there!

– What more should we compute in the algorithm?

0 1 –3 2 –4

3 0 –4 1 –1

7 4 0 5 3

2 –1 –5 0 –2

8 5 1 6 0

Weighted graphs (3)

Weighted graphs, continued:• Minimum spanning tree algorithms (greedy)

– Boruvka– Kruskal – Prim

• Minimum Steiner tree

Background

• MST also known as the shortest network problem– Want to connect all vertices with minimum total length of edges.

• Applications– Sources of oil need to be connected to pipelines. Want to

minimize total mileage.– Private telecom networks are billed according to total mileage of

the network. Client should not have to pay for phone company’s inefficiency.

• Some Algorithms– O. Boruvka (1926) – published Slovak paper in obscure journal

(first known solution)– J. B. Kruskal (1956) – AT&T Bell Labs– R. C. Prim (1957) – also from Bell Labs

Boruvka

Create a set (connected component) for each vertex in the graph.

repeat:

for each set S:

find the lightest edge (u,v) connecting a vertex u in S with a vertex v not in S.

Add (u,v) to the spanning tree.

Union v’s set into u’s set.

until we have chosen n-1 edges.

Kruskal

• Repeatedly add edges from low to high, until you have added n – 1 edges.

• How do we avoid a cycle?– Consider the graph to be a forest of trees, initially with 1 vertex per tree.

As we pick an edge, we union 2 trees together. When we’re done, there is just 1 tree.

– When examining an edge (u, v): check to see that u and v are in different vertex sets.

Create a set for each vertex in graph.

Sort edges in ascending order of weight.

for each edge (u, v):

if u and v are in different sets

add (u, v) to spanning tree

Union v’s set into u’s set.

Prim

• During algorithm, we have just 1 tree, with vertices either in or “not yet” in the tree.

• Spread disease from current vertices of tree: add lightest edge adjacent to some vertex not yet in tree.

// Let T = set of vertices accounted for

Select one vertex to initially add to T.

for i = 2 to n:

(u,v) = shortest edge connecting a vertex u in T with a vertex v not in T.

add (v,w) to spanning tree

add w to T

Complexity

• The algorithms are conceptually simple, but best performance requires attention to potential bottlenecks. Optimizations can be quite sophisticated!

• Kruskal says– for each edge, see if they are in different sets algorithm to determine disjoint sets in log time. (?)

• Optimizing Prim’s algorithm– Need to speed up the way of finding lightest edge connected to any

vertex in the growing tree.

– Can assign labels to vertices, and search for vertex with smallest label.

– Priority queue quick way to find smallest element

– “Fibonacci” heap has better performance than binomial heap

Steiner tree

• The “minimum Steiner tree” problem is similar to the minimum spanning tree.– Problem posed by Swiss mathematician Jacob Steiner.

• We can find an even shorter spanning tree if we are permitted to add more vertices!– In particular, a vertex in the middle or between existing ones.

• Ex. Consider 4 vertices of a square.– Minimum spanning tree has total length 3.– Minimum Steiner tree would put 2 vertices inside the square to reduce total

length by about 9%.

• No known general solution other than brute force!• Please listen/watch this video lecture by Ron Graham:

http://www.archive.org/download/RonaldLG1988/RonaldLG1988.mpeg

Background info

• Big numbers & infinity• Limits discovered by Gödel, Heisenberg, and

Turing• Some problems are:

– Impossible (i.e. undecidable).– Infeasible (NP-complete).– There are classifications even among these types of

problems!

• P vs. NP question