CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 1
Shortest Paths
Single-Source Shortest PathsDijkstra’s AlgorithmBellman-Ford AlgorithmDifference ConstraintsAll-Pairs Shortest PathsFloyd-Warshall AlgorithmJohnson’s Algorithm
Shortest-Paths
⊲ shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 2
Shortest path problems on weighted graphs(directed or undirected) have two main types:
Single-Source Shortest-Path: find the shortestpaths from source vertex s to all other vertices.
All-Pairs Shortest-Path: find the shortest pathsbetween all pairs of vertices.
Some notation:w(u, v) = weight of edge (u, v)w(p) = sum of weights on path p
Let δ(u, v) = shortest distance from u to v.
Let sopt❀ v stand for shortest path from s to v.
Example Weighted Graphs
shortest paths
⊲ example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 3
1
23
7
85
54 7
9
6
4
8
a
b c d
ei
h g f
Example Bad Graph
shortest paths
example graphs
⊲ bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 4
Graphs with negative-weight cycles can’t besolved.
Key Properties
shortest paths
example graphs
bad graph
⊲ properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 5
Triangle Inequality: If (u, v) is an edge, thenδ(s, v) ≤ δ(s, u) + w(u, v). If p is a path from u
to v, then δ(s, v) ≤ δ(s, u) + w(p).
Proof: δ(s, v) > δ(s, u) + w(u, v) contradicts δ’sdefinition. So does δ(s, v) > δ(s, u) + w(p).
Optimal Subpath Property: If u is on sopt❀ v, then
δ(s, v) = δ(s, u) + δ(u, v).
Proof: Let p be sopt❀ v, and let p1 and p2 be the
subpaths s ❀ u and u ❀ v. By definition,w(p) = w(p1)+w(p2) ≥ δ(s, u)+ δ(u, v). A lowerδ(s, u) or δ(s, v) would contradict p’s optimality.
Key Properties Continued
shortest paths
example graphs
bad graph
properties 1
⊲ properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 6
Convergence Property: If s ❀ u→ v is a shortestpath, then δ(s, v) = δ(s, u) + w(u, v).
Proof: Follows from the optimal subpath property.
Let v.d ≥ δ(s, v) for all v with s.d = 0 = δ(s, s).
Path-Relaxation Property: If v.d > δ(s, v), thensome edge (x, y) satisfies v.d = δ(s, x) andy.d > x.d+ w(x, y).
Proof: Some edge (x, y) on sopt❀ v must be the
first edge with x.d = δ(s, x) andy.d > x.d+ w(x, y) = δ(s, y).
Basic Procedures
shortest paths
example graphs
bad graph
properties 1
properties 2
⊲basic procedures
1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 7
These properties justify the following proceduresfor single-source shortest-paths.v.d = distance from s to v.v.π = previous vertex on s ❀ v path.
Initialize-Single-Source(G, s)for each vertex v in G
v.d ← ∞v.π ← nil
s.d ← 0
Initial ∞ ensures that d.v ≥ δ(s, v) for all v.s.d = 0 ensures that s.d = δ(s, s).
Basic Procedures Continued
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
⊲basic procedures
2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 8
Relax(u, v, w)if v.d > u.d+ w(u, v)v.d ← u.d+ w(u, v)v.π ← u
If u.d ≥ δ(s, u) and v.d ≥ δ(s, v) beforeRelax(u, v, w) then they remain true afterwards.
Also, if v.d > δ(s, v), then the Path RelaxationProperty implies that some call to Relax willimprove d somewhere.
Dijkstra’s Algorithm
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
⊲ dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 9
Dijkstra’s Algorithm assumes all weights arenonnegative.
Dijkstra(G,w, s)Initialize-Single-Source(G, s)S ← ∅Q ← G.V using d field for priority queuewhile Q 6= ∅
u ← Extract-Min(Q)S ← S ∪ {u}for each v ∈ G.Adj[u]
Relax(u, v, w)
Dijkstra Analysis
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
⊲ dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 10
Proof of Correctness:
Basis: s is assigned to u, s.d = 0 = δ(s, s)Assume: For all x ∈ S, x.d = δ(s, x)Show: u.d = δ(s, u) when u is extracted.Induction: Proof by contradiction.
Suppose δ(s, u) < u.d.
Some edge (x, y) in sopt❀ u goes from S to Q.
y.d = δ(s, y) because (x, y) has been relaxed,but u.d ≤ y.d = δ(s, y) ≤ δ(s, u) < u.d
is a contradiction.
Dijkstra is O(E lg V ) with binary heaps.
Bellman-Ford Algorithm
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
⊲ bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 11
This works with negative weights. false isreturned if a negative cycle is detected.
Bellman-Ford(G,w, s)Initialize-Single-Source(G, s)for i ← 1 to |V | − 1
for each edge (u, v) in G
Relax(u, v, w)for each edge (u, v) in G
if v.d > u.d+ w(u, v)return false
return true
Bellman-Ford Algorithm
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
⊲ bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 12
Proof of Correctness:
All finite shortest paths have ≤ V − 1 edges.
The ith iteration relaxes ith edge in the optimalpath.
If shortest path is finite, then after V − 1iterations, v.d = δ(s, v).
If there is an infinite shortest path,then v.d > u.d+ w(u, v) for some edge (u, v).
Bellman-Ford is O(V E).
Difference Constraints
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
⊲ diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 13
A difference constraint is xj − xi ≤ bk, where xiand xj are variables and bk is a constant.
Difference constraints can represent timerelationships between events.
If x2 must occur within 2 hours after x1, thenx2 − x1 ≤ 2.
If x2 must occur at least 2 hours after x1, thenx2 − x1 ≥ 2, which is equivalent to x1 − x2 ≤ −2.
Difference Constraints Continued
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
⊲ diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 14
The triangle inequality for shortest paths is adifference constraint. δ(s, v) ≤ δ(s, u) +w(u, v) isequivalent to δ(s, v)− δ(s, u) ≤ w(u, v).
A set of difference constraints xj − xi ≤ bk can bereduced to a weighted graph by w(vi, vj) = bk andw(s, vj) = w(s, vi) = 0.
There is a solution for single-source shortest pathsif and only if there is a solution to the differenceconstraints.
Difference Constraints Example
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
⊲ diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 15
x1 − x2 ≤ 0x1 − x5 ≤ −1x2 − x5 ≤ 1x3 − x1 ≤ 5x4 − x1 ≤ 4x4 − x3 ≤ −1x5 − x3 ≤ −3x5 − x4 ≤ −3
All-Pairs Shortest Paths
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
⊲ all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 16
The all-pairs shortest-paths problem is finding theshortest path for each pair of vertices. Algorithmsfor this problem can be adapted for transitiveclosure.
Let δ(i, j,m) be the shortest length from i to j
using ≤ m edges.
δ(i, j, 1) =
0 if i = j
w(i, j) if (i, j) is an edge∞ otherwise
δ(i, j, 2m) =n
mink=1
(δ(i, k,m) + δ(k, j,m))
All-Pairs Recursion
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
⊲ all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 17
This recursive definition is correct.
Basis: Equation for δ(i, j, 1) is correct.Assume: δ(i, j,m) is correct.Show: Equation for δ(i, j, 2m) is correct.Induction: Optimal path of ≤ 2m edges consists
of two optimal paths of ≤ m edges.
This leads to a O(V 3 lg V ) algorithm.
All-Pairs Algorithm
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
⊲ all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 18
All-Pairs(G,w)n ← |G.V |D ← an n× n matrix initialized to δ(i, j, 1)m ← 1while m < n− 1
Extend(D,n)m ← 2m
return D
Extend(D,n)for i ← 1 to n
for j ← 1 to n
for k ← 1 to n
D[i, j] ← min(D[i, j], D[i, k] +D[k, j])
All-Pairs Example
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
⊲ all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 19
Floyd-Warshall Algorithm
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
⊲ floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 20
Floyd-Warshall(G,w)n ← |G.V |D ← an n× n matrix initialized to δ(i, j, 1)for k ← 1 to n
for i ← 1 to n
for j ← 1 to n
D[i, j] ← min(D[i, j], D[i, k] +D[k, j])return D
Floyd-Warshall is O(V 3).
For transitive closure, use a 0/1 matrix and:D[i, j] ← D[i, j] ∨ (D[i, k] ∧D[k, j])
Floyd-Warshall Analysis
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
⊲ floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 21
Let λ(i, j, k) be the shortest distance from i to j
with all intermediate vertices ≤ k.
Basis: D[i, j] ≤ λ(i, j, 0) due to initialization.Assume: D[i, j] ≤ λ(i, j, k−1) before kth iteration.Show: D[i, j] ≤ λ(i, j, k) after kth iteration.Induction: Either λ(i, j, k) = λ(i, j, k − 1) or
λ(i, j, k) = λ(i, k, k−1) + λ(k, j, k−1)
Floyd-Warshall Example
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
⊲ floyd-warshall 3
johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 22
Johnson’s Algorithm for Sparse Graphs
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
⊲ johnson 1
johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 23
Idea of Johnson’s Algorithm:Run Dijkstra’s algorithm n times.Obtain O(V E lg V ) time using binary heaps.Better than O(V 3) if E is o(V 2)
Problem: Negative weights.
Solution: Clever way to transform the graph sothat all weights are positive
Johnson’s Algorithm
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
⊲ johnson 2
johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 24
Johnson(G,w, s)(G′, w′) ← (G,w) with new vertex s, w′(s, v) = 0if Bellman-Ford(G′, w′, s) = false
error negative-weight cyclefor (u, v) ∈ G
w(u, v) ← w(u, v) + δ′(s, u)− δ′(s, v)for each vertex u in G
Dijkstra(G, w, u)for each vertex v in G
D[u, v] ← δ(u, v)− δ′(s, u) + δ′(s, v)return D
Johnson’s Algorithm Analysis
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
⊲ johnson 3
johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 25
Triangle inequality, δ′(s, v) ≤ δ′(s, u) + w(u, v),implies 0 ≤ δ′(s, u) + w(u, v)− δ′(s, v) = w(u, v).Thus, w(u, v) ≥ 0 for all edges (u, v).
For any path p from u to v,w(p) = w(p) + δ′(s, u)− δ′(s, v) because of thetelescoping sum.
Johnson is O(V E lg V ) if binary heaps are usedfor Dijkstra.
Johnson Example
shortest paths
example graphs
bad graph
properties 1
properties 2
basic procedures 1
basic procedures 2
dijkstra 1
dijkstra 2
bellman-ford 1
bellman-ford 2
diff constraints 1
diff constraints 2
diff constraints 3
all pairs
all pairs 2
all pairs 3
all pairs 4
floyd-warshall 1
floyd-warshall 2
floyd-warshall 3
johnson 1
johnson 2
johnson 3
⊲ johnson 4
CS 5633 Analysis of Algorithms Chapter 24 and 25: Slide – 26