Minimum Spanning TreesSpanning Tree
◦A tree (i.e., connected, acyclic graph) which contains all the vertices of the graph
Minimum Spanning Tree◦Spanning tree with the minimum sum of weights
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
Prim’s Algorithm
Starts from an arbitrary “root”: VA = {a}
At each step:
◦ Find a light edge crossing (VA, V - VA)
◦ Add this edge to set A (The edges in set A always form a single
tree)
◦ Repeat until the tree spans all vertices
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
Example
0 Q = {a, b, c, d, e, f, g,
h, i} VA =
Extract-MIN(Q) a
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
key [b] = 4 [b] = akey [h] = 8 [h] = a
4 8
Q = {b, c, d, e, f, g, h, i} VA = {a}
Extract-MIN(Q) b
4
8
4
8
8
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
Q = {c, d, e, f, g, h, i} VA = {a, b}
key [c] = 8 [c] = bkey [h] = 8 [h] = a -
unchanged
8 8 Extract-MIN(Q) c
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
Q = {d, e, f, g, h, i} VA = {a, b, c}
key [d] = 7 [d] = ckey [f] = 4 [f] = ckey [i] = 2 [i] = c
7 4 8 2 Extract-MIN(Q) i
4
8
8
7
4
2
5
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
Q = {d, e, f, g, h} VA = {a, b, c, i}
key [h] = 7 [h] = ikey [g] = 6 [g] = i
7 4 6 7 Extract-MIN(Q) f
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
Q = {d, e, g, h} VA = {a, b, c, i, f}
key [g] = 2 [g] = fkey [d] = 7 [d] = c
unchanged
key [e] = 10 [e] = f 7 10 2 7
Extract-MIN(Q) g
4 7
8 4
8
2
7 64 7
7 6 4
8
2
2
10
6
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
Q = {d, e, h} VA = {a, b, c, i, f, g}
key [h] = 1 [h] = g 7 10 1 Extract-MIN(Q) h
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
Q = {d, e} VA = {a, b, c, i, f, g, h}
7 10 Extract-MIN(Q) d
4 7
10
7 2 4
8
2
1
4 7
10
1 2 4
8
2
7
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
Q = {e} VA = {a, b, c, i, f, g, h, d}
key [e] = 9 [e] = d
9
Extract-MIN(Q) e
Q = VA = {a, b, c, i, f, g, h, d, e}
4 7
10
1 2 4
8
2 9
PRIM(V, E, w, r) % r : starting vertex
1. Q ←
2. for each u V
3. do key[u] ← ∞
4. π[u] ← NIL
5. INSERT(Q, u)
6. DECREASE-KEY(Q, r, 0) % key[r] ← 0
7. while Q
8. do u ← EXTRACT-MIN(Q)
9. for each v Adj[u]
10. do if v Q and w(u, v) < key[v]
11. then π[v] ← u
12. DECREASE-KEY(Q, v, w(u, v))
O(V) if Q is implemented as a min-heap
Executed |V| times
Takes O(lgV)
Min-heap operations:O(VlgV)Executed O(E) times
totalConstant
Takes O(lgV) O(ElgV
)
Total time: O(VlgV + ElgV) = O(ElgV)
O(lgV)
9
Prim’s Algorithm
Total time: O(ElgV )
Prim’s algorithm is a “greedy” algorithm
◦Greedy algorithms find solutions based on a
sequence of choices which are “locally” optimal
at each step.
Nevertheless, Prim’s greedy strategy
produces a globally optimum solution!
10
We would addedge (c, f)
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
Kruskal’s Algorithm
Start with each vertex being its own component
Repeatedly merge two components into one by choosing the lightest edge that connects them
Which components to consider at each iteration?◦ Scan the set of edges in
monotonically increasing order by weight. Choose the smallest edge.
Example1. Add (h, g)
2. Add (c, i)
3. Add (g, f)
4. Add (a, b)
5. Add (c, f)
6. Ignore (i, g)
7. Add (c, d)
8. Ignore (i, h)
9. Add (a, h)
10. Ignore (b,
c)
11. Add (d, e)
12. Ignore (e, f)
13. Ignore (b, h)
14. Ignore (d, f)
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
1: (h, g)2: (c, i),
(g, f)4: (a, b),
(c, f)6: (i, g)7: (c, d),
(i, h)
8: (a, h), (b, c)
9: (d, e)10: (e, f)11: (b, h)14: (d, f)
{g, h}, {a}, {b}, {c},
{d},{e},{f},{i}
{g, h}, {c, i}, {a}, {b},
{d}, {e}, {f}
{g, h, f}, {c, i}, {a},
{b}, {d}, {e}
{g, h, f}, {c, i}, {a, b},
{d}, {e}
{g, h, f, c, i}, {a, b},
{d}, {e}
{g, h, f, c, i}, {a, b},
{d}, {e}
{g, h, f, c, i, d}, {a, b},
{e}
{g, h, f, c, i, d}, {a, b},
{e}
{g, h, f, c, i, d, a, b},
{e}
{g, h, f, c, i, d, a, b},
{e}
{g, h, f, c, i, d, a, b, e}
{g, h, f, c, i, d, a, b, e}
{g, h, f, c, i, d, a, b, e}
{g, h, f, c, i, d, a, b, e}
{a}, {b}, {c}, {d}, {e},
{f}, {g}, {h}, {i}
Operations on Disjoint Data Sets
Kruskal’s Alg. uses Disjoint Data Sets (UNION-FIND : Chapter 21) to determine whether an edge connects vertices in different components
MAKE-SET(u) – creates a new set whose only member is u
FIND-SET(u) – returns a representative element from the set that contains u. It returns the same value for any element in the set
UNION(u, v) – unites the sets that contain u and v, say Su and Sv
◦ E.g.: Su = {r, s, t, u}, Sv = {v, x, y}
UNION (u, v) = {r, s, t, u, v, x, y}
We had seen earlier that FIND-SET can be done in O(lgn) or O(1) time
and UNION operation can be done in O(1) (see Chapter 21)
1. A ← 2. for each vertex v V3. do MAKE-SET(v)
4. sort E into non-decreasing order by w
5. for each (u, v) taken from the sorted list
6. do if FIND-SET(u) FIND-SET(v)
7. then A ← A {(u, v)}
8. UNION(u, v)
9. return A
Running time: O(V+ElgE+ElgV) O(ElgE)
Implemented by using the disjoint-set data structure (UNION-FIND)
Kruskal’s algorithm is “greedy”It produces a globally optimum solution
KRUSKAL(V, E, w)
O(V)
O(ElgE)
O(E)
O(lgV)
Another Example for Prim’s Method
b
a
d
c
f
e
2
2
1
3
10
7
1
98
a b c d e f
a 0 2 7 1b 2 0 10 1 c 7 10 0 8 2
d 1 8 0 9
e 2 9 0 3f 1 3 0
a b c d e f
D[.] = 2 0 10 1
S
new D[i] = Min{ D[i], w(k, i) }
where k is the newly-selected nodeand w[.] is the distance between k and i
d
a b c d e f
a 0 2 7 1b 2 0 10 1 c 7 10 0 8 2
d 1 8 0 9
e 2 9 0 3f 1 3 0
a b c d e f
L [.] = 2 0 10 1 a b c d e
f new L [.] = 2 0 8 1 9
b
a
c
f
e
2
2
1
3
10
7
1
98
new D[i] = Min{ D[i], w(k, i) }
where k is the newly-selected nodeand w[.] is the distance between k and i
a b c d e f
a 0 2 7 1b 2 0 10 1 c 7 10 0 8 2
d 1 8 0 9
e 2 9 0 3f 1 3 0
a b c d e f
L [.] = 2 0 8 1 9 a b c d e
f new L [.] = 2 0 7 1 9
1
d
b
a
c
f
e
2
2
1
3
10
7
1
98
new D[i] = Min{ D[i], w(k, i) }
where k is the newly-selected nodeand w[.] is the distance between k and i
a b c d e f
a 0 2 7 1b 2 0 10 1 c 7 10 0 8 2
d 1 8 0 9
e 2 9 0 3f 1 3 0
a b c d e f
L [.] = 2 0 7 1 9 1 a b c d e
f new L [.] = 2 0 7 1 3
1
d
b
a
c
f
e
2
2
1
3
10
7
1
98
a b c d e f
a 0 2 7 1b 2 0 10 1 c 7 10 0 8 2
d 1 8 0 9
e 2 9 0 3f 1 3 0
a b c d e f
L [.] = 2 0 7 1 3 1 a b c d e
f new L [.] = 2 0 2 1 3
1
d
b
a
c
f
e
2
2
1
3
10
7
1
98
a b c d e f a 0 2 7 1b 2 0 10 1 c 7 10 0 8 2
d 1 8 0 9
e 2 9 0 3f 1 3 0
a b c d e f
L [.] = 2 0 2 1 3 1 a b c d e
f new L [.] = 2 0 2 1 3
1
Running time: (V2) (array representation)
(ElgV) (Min-
Heap+Adjacency List)Which one is better?
d
b
a
c
f
e
2
2
1
3
10
7
1
98
Greedy MST Methods
Prim’s method is fastest. O(n2) (worst case) O(E log n) if a Min Heap is used to keep track
of distances of vertices to partially built tree. If e=O(n2), MinHeap is not a good idea!
Kruskal’s uses union-find trees to run in O(E log n) time.
• P processors, n=|V| vertices
• Each processor is assigned n/p vertices (Pi gets the set Vi)
• Each PE holds the n/p columns of A and n/p elements of d[] array
. . . . . .d[.]
A
P0 P1 Pi Pp-1
| | || | || | || | || | || | || | || | |
| | || | || | || | || | || | || | || | |
| | || | || | || | || | || | || | || | |
| | || | || | || | || | || | || | || | |
…..
…..
…..
…..
…..
…..
n/p columns
Parallel MST Algorithm (Prim’s)
Parallel MST Algorithm (Prim’s)
1. Initialize: Vt := {r}; d[k] = for all k (except d[r] = 0)
2. P0 broadcasts selectedV = r using one-to-all broadcast.
3. The PE responsible for "selectedV" marks it as belonging to set Vt.
4. For v = 2 to n=|V| do
5. Each Pi updates d[k] = Min[d[k], w(selectedV, k)] for all k Vi
6. Each Pi computes MIN-di =(minimum d[] value among its unselected elements)
7. PEs perform a "global minimum" using MIN-di values and store the result in P0.
Call the winning vertex, selectedV.
8. P0 broadcasts "selectedV" using one-to-all broadcast.
9. The PE responsible for "selectedV" marks it as belonging to set Vt.
10. EndFor
TIME COMPLEXITY ANALYSIS:
E=O(n2) then Tseq = n2
(Hypercube)
Tpar = n*(n/p) + n*logp
computation + communication
(Mesh)
Tpar = n*(n/p) + n * Sqrt(p)
The algorithm is cost-optimal on a hypercube if plogp/n =O(1)
Parallel MST Algorithm (Prim’s)
Dijkstra’s SSSP Algorithm (adjacency matrix)
b
a
d
c
f
e
2
2
1
3
10
7
1
98
a b c d e f
a 0 2 7 1b 2 0 10 1 c 7 10 0 8 2
d 1 8 0 9
e 2 9 0 3f 1 3 0
a b c d e f
L [.] = 2 0 10 1 S
new L[i] = Min{ L[i], L[k] + W[k, i] }
where k is the newly-selected intermediate nodeand W[.] is the distance between k and i
d
a b c d e f
a 0 2 7 1b 2 0 10 1 c 7 10 0 8 2
d 1 8 0 9
e 2 9 0 3f 1 3 0
a b c d e f
L [.] = 2 0 10 1 a b c d e
f new L [.] = 2 0 9 1 10
SSSP cont.
new L[i] = Min{ L[i], L[k] + W[k, i] }
where k is the newly-selected intermediate nodeand W[.] is the distance between k and i
b
a
c
f
e
2
2
1
3
10
7
1
98
a b c d e f
a 0 2 7 1b 2 0 10 1 c 7 10 0 8 2
d 1 8 0 9
e 2 9 0 3f 1 3 0
a b c d e f
L [.] = 2 0 9 1 10 a b c d e
f new L [.] = 2 0 9 1 10
3
new L[i] = Min{ L[i], L[k] + W[k, i] }
where k is the newly-selected intermediate nodeand W[.] is the distance between k and i
d
b
a
c
f
e
2
2
1
3
10
7
1
98
a b c d e f
a 0 2 7 1b 2 0 10 1 c 7 10 0 8 2
d 1 8 0 9
e 2 9 0 3f 1 3 0
a b c d e f
L [.] = 2 0 9 1 10 3 a b c d e
f new L [.] = 2 0 9 1 6
3
d
b
a
c
f
e
2
2
1
3
10
7
1
98
a b c d e f
a 0 2 7 1b 2 0 10 1 c 7 10 0 8 2
d 1 8 0 9
e 2 9 0 3f 1 3 0
a b c d e f
L [.] = 2 0 9 1 6 3 a b c d e
f new L [.] = 2 0 8 1 6
3
d
b
a
c
f
e
2
2
1
3
10
7
1
98
a b c d e f a 0 2 7 1b 2 0 10 1 c 7 10 0 8 2
d 1 8 0 9
e 2 9 0 3f 1 3 0
a b c d e f
L [.] = 2 0 8 1 6 3 a b c d e
f new L [.] = 2 0 8 1 6
3
Running time: (V2) (array representation)
(ElgV) (Min-
Heap+Adjacency List)Which one is better?
d
b
a
c
f
e
2
2
1
3
10
7
1
98
Task Partitioning for Parallel SSSP Algorithm
• P processors, n=|V| vertices
• Each processor is assigned n/p vertices (Pi gets the set Vi)
• Each PE holds the n/p columns of A and n/p elements of L[] array as shown below
. . . . . .L[.]
A
P0 P1 Pi Pp-1
| | || | || | || | || | || | || | || | |
| | || | || | || | || | || | || | || | |
| | || | || | || | || | || | || | || | |
| | || | || | || | || | || | || | || | |
…..
…..
…..
…..
…..
…..
n/p columns
1. Initialize: Vt := {r}; L[k] = for all k except L[r] = 0;
2. P0 broadcasts selectedV = r using one-to-all broadcast.
3. The PE responsible for "selectedV" marks it as belonging to set Vt.
4. For v = 2 to n=|V| do
5. Each Pi updates L[k] = Min[ L[k], L(selectedV)+W(selectedV, k) ] for k Vi
6. Each Pi computes MIN-Li = (minimum L[.] value among its unselected elements)
7. PEs perform a "global minimum" using MIN-Li values and result is stored in P0.
Call the winning vertex, selectedV.
8. P0 broadcasts "selectedV" and L[selectedV] using one-to-all broadcast.
9. The PE responsible for "selectedV" marks it as belonging to set Vt.
10. EndFor
Parallel SSSP Algorithm (Dijkstra’s)
TIME COMPLEXITY ANALYSIS:
In the worst-case, Tseq = n2
(Hypercube)
Tpar = n*(n/p) + n*logp
computation + communication
(Mesh)
Tpar = n*(n/p) + n * Sqrt(p)
The algorithm is cost-optimal on a hypercube if plogp/n = O(1)
Parallel SSSP Algorithm (Dijkstra’s)
