Post on 31-Dec-2015
description
transcript
Minimum Spanning Tree Algorithms
Prim’s Algorithm
Kruskal’s Algorithm
Runtime Analysis
Correctness Proofs
Copyright 1999 by Cutler/Head
Greedy/Prims 2
What is A Spanning Tree?
u
v
b
a
c
d
e
f
• A spanning tree for an undirected graph G=(V,E) is a subgraph of G that is a tree and contains all the vertices of G
• Can a graph have more than one spanning tree?
• Can an unconnected graph have a spanning tree?
Copyright 1999 by Cutler/Head
Greedy/Prims 3
Minimal Spanning Tree.
4 4
3
2
9
15
8
1014
3
u
v
b
a
c
d
e
f
Mst T: w( T )= (u,v) T w(u,v ) is minimized
• The weight of a subgraph is the sum of the weights of it edges.
• A minimum spanning tree for a weighted graph is a spanning tree with minimum weight.
• Can a graph have more then one minimum spanning tree?
Copyright 1999 by Cutler/Head
Greedy/Prims 4
Example of a Problem that Translates into a MST
The Problem• Several pins of an electronic circuit must be
connected using the least amount of wire.
Modeling the Problem • The graph is a complete, undirected graph
G = ( V, E ,W ), where V is the set of pins, E is the set of all possible interconnections between the pairs of pins and w(e) is the length of the wire needed to connect the pair of vertices.
• Find a minimum spanning tree.
Copyright 1999 by Cutler/Head
Greedy/Prims 5
Greedy Choice
We will show two ways to build a minimum spanning tree.
• A MST can be grown from the current spanning tree by adding the nearest vertex and the edge connecting the nearest vertex to the MST. (Prim's algorithm)
• A MST can be grown from a forest of spanning trees by adding the smallest edge connecting two spanning trees. (Kruskal's algorithm)
Copyright 1999 by Cutler/Head
Greedy/Prims 6
Notation
• Tree-vertices: in the tree constructed so far• Non-tree vertices: rest of vertices
Prim’s Selection rule
• Select the minimum weight edge between a tree-node and a non-tree node and add to the tree
Copyright 1999 by Cutler/Head
Greedy/Prims 7
The Prim algorithm Main Idea
Select a vertex to be a tree-node
while (there are non-tree vertices) { if there is no edge connecting a tree node with a non-tree node
return “no spanning tree”
select an edge of minimum weight between a tree node and a non-tree node
add the selected edge and its new vertex to the tree}
return tree
6 4
5
2
158
1014
3
u
v
b
a
c
d
f
Copyright 1999 by Cutler/Head
Greedy/Prims 8
Implementation Issues
• How is the graph implemented?
– Assume that we just added node u to the tree.
– The distance of the nodes adjacent to u to the tree may now be decreased.
– There must be fast access to all the adjacent vertices.
– So using adjacency lists seems better
• How should the set of non-tree vertices be represented?
– The operations are:
• build set
• delete node closest to tree
• decrease the distance of a non-tree node from the tree
• check whether a node is a non- tree node
Copyright 1999 by Cutler/Head
Greedy/Prims 9
Implementation Issues
• How should the set of non-tree vertices be represented?
– A priority queue PQ may be used with the priority D[v] equal to the minimum distance of each non-tree vertex v to the tree.
– Each item in PQ contains: D[v], the vertex v, and the shortest distance edge (v, u) where u is a tree node
• This means:
– build a PQ of non-tree nodes with initial values -
• fast build heap O (V )
• building an unsorted list O(V)
• building a sorted list O(V) (special case)
Copyright 1999 by Cutler/Head
Greedy/Prims 10
Implementation Issues
– delete node closest to tree (extractMin)• O(lg V ) if heap and • O(V) if unsorted list • O(1) sorted list
– decrease the distance of a non-tree node to the tree
– We need to find the location i of node v in the priority queue and then execute (decreasePriorityValue(i, p)) where p is the new priority
– decreasePriorityValue(i, p)
• O(lg V) for heap,
• O(1) for unsorted list
• O(V ) for sorted list (too slow)
Copyright 1999 by Cutler/Head
Greedy/Prims 11
Implementation Issues
• What is the location i of node v in a priority queue?
– Find in Heaps, and sorted lists O(n)
– Unsorted – if the nodes are numbered 1 to n and we use an array where node v is the v item in the array O(1)
Extended heap
– We will use extended heaps that contain a “handle” to the location of each node in the heap.
– When a node is not in PQ the “handle” will indicate that this is the case
– This means that we can access a node in the extended heap in O(1), and check v PQ in O(1)
– Note that the “handle” must be updated whenever a heap operation is applied
Copyright 1999 by Cutler/Head
Greedy/Prims 12
Implementation Issues
2. Unsorted list
– Array implementation where node v can be accesses as PQ[v] in O(1), and the value of PQ[v] indicates when the node is not in PQ.
Copyright 1999 by Cutler/Head
Greedy/Prims 13
Lines 1-5 initialize the priority queue PQ to contain all Vertices. Ds for all vertices except r, are set to infinity.
r is the starting vertex of the TThe T so far is empty
Add closest vertex and edge to current T
Get all adjacent vertices v of u,update D of each non-tree vertex adjacent to u
Store the current minimum weight edge, and updated distance in the priority queue
Prim’s Algorithm
1. for each u V2. do D [u ] 3. D[ r ] 0
4. PQ make-heap(D,V, {})//No edges 5. T 6.7. while PQ do 8. (u,e ) PQ.extractMin() 9. add (u,e) to T10. for each v Adjacent (u )
// execute relaxation 11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue
( D[v], v, (u,v )) 14. return T // T is a mst.
Copyright 1999 by Cutler/Head
Greedy/Prims 14
Prim’s AlgorithmInitialization
Prim (G )
1. for each u V2. do D [u ] 3. D[ r ] 0
4. PQ make-heap(D,V, {})//No edges 5. T
Copyright 1999 by Cutler/Head
Greedy/Prims 15
Building the MST
// solution check
7. while PQ do//Selection and feasibility 8. (u,e ) PQ.extractMin()
// T contains the solution so far . 9. add (u,e) to T10. for each v Adjacent (u ) 11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue
(D[v], v, (u,v) ) 14. return T
Copyright 1999 by Cutler/Head
Greedy/Prims 16
Using Extended Heap implementation
Lines 1 -6 run in O (V )
Max Size of PQ is | V |
Count7 =O (V )
Count7(8) = O (V ) O( lg V )
Count7(10) = O(deg(u ) ) = O( E )
Count7(10(11)) = O(1)O( E )
Count7(10(11(12))) = O(1) O( E )
Count7(10(13)) = O( lg V) O( E ) Decrease- Key operation on the extended heap can be implemented in O( lg V)
So total time for Prim's Algorithm is O ( V lg V + E lg V )
What is O(E ) ?Sparse Graph, E =O(V) , O (E lg V)=O(V lg V )
Dense Graph, E=O(V2), O (E lg V)=O(V2 lg V)
Time Analysis1. for each u V2. do D [u ] 3. D[ r ] 0 4. PQ make-PQ(D,V, {})//No edges 5. T 6.7. while PQ do 8. (u,e ) PQ.extractMin() 9. add (u,e) to T10. for each v Adjacent (u ) 11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue
(D[v], v, (u,v)) 15. return T // T is a mst.Assume a node in PQ can be accessed in O(1)
** Decrease key for v requires O(lgV ) provided the node in heap with v’s data can be accessed in O(1)
Copyright 1999 by Cutler/Head
Greedy/Prims 17
Using unsorted PQ
Lines 1 - 6 run in O (V )
Max Size of PQ is | V |Count7 = O (V )
Count7(8) = O (V ) O(V )
Count7(10) = O(deg(u ) ) = O( E )
Count7(10(11)) = O(1)O( E )
Count7(10(11(12))) = O(1) O( E )
Count7(10(13)) =O( 1) O( E )
So total time for Prim's Algorithm is O (V + V2 + E ) = O (V2 )
For Sparse/Dense graph : O( V2 )Note growth rate unchanged for adjacency
matrix graph representation
Time Analysis
1. for each u V2. do D [u ] 3. D[ r ] 0
4. PQ make-PQ(D,V, {})//No edges 5. T 6.7. while PQ do 8. (u,e ) PQ.extractMin() 9. add (u,e) to T10. for each v Adjacent (u ) 11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue
(D[v], v, (u,v))
15. return T // T is a mst.
Copyright 1999 by Cutler/Head
Greedy/Prims 18
handle
ABC
123
Prim - extended HeapAfter Initialization
T PQ
0, (A, {})
, (B, {}) , (C, {})
1
2 3
A
B
C
2
56
G
Prim (G, r)1. for each u V2. do D [u ] 3. D[ r ] 0
4. PQ make-heap(D,V, { })5. T A
BC
G
A 2
A 6
B 2 C 6
C 5
B 5
Copyright 1999 by Cutler/Head
Greedy/Prims 19
Prim - extended Heap Build tree - after PQ.extractMin
handle
ABC
Null21
T(A, {})
PQ
, (C, {})
, (B, {})
1
2
A
C
2
56
G
7. while PQ do 8. (u,e) PQ.extractMin()9. add (u,e) to T
B
Copyright 1999 by Cutler/Head
Greedy/Prims 20
Update B adjacent to A
handle
ABC
Null12
T(A, {})
PQ
2, (B, {A, B})
, (C, {})
1
2
A
C
2
56
G
10. for each v Adjacent (u ) 11. // relaxation operation
// relaxation11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue
( D[v], v, (u,v))
B
Copyright 1999 by Cutler/Head
Greedy/Prims 21
Update C adjacent to A
handle
ABC
Null12
T(A, {})
PQ
2, (B, {A, B})
6, (C, {A, C})
1
2A
B
C
2
56
G
Copyright 1999 by Cutler/Head
Greedy/Prims 22
Build tree - after PQ.extractMin
handle
ABC
NullNull1
T(A, {})(B, {A, B})
PQ
6, (C, {A, C})1
A
B
C
2
56
G
7. while PQ do 8. (u,e) PQ.extractMin()9. add (u,e) to T
Copyright 1999 by Cutler/Head
Greedy/Prims 23
Update C adjacent to B
handle
ABC
NullNull1
T(A, {})
PQT(A, {})(B, {A, B})
5, (C, {B, C})1
10. for each v Adjacent (u ) 11. // relaxation operation
A
B2
56
GC
Copyright 1999 by Cutler/Head
Greedy/Prims 24
Build tree - after PQ.extractMin
handle
ABC
NullNullNull
T(A, {})
PQT(A, {})(B, {A, B}) (C, {B, C})
7. while PQ do 8. (u,e) PQ.extractMin()9. add (u,e) to T
A
B2
56
GC
Copyright 1999 by Cutler/Head
Greedy/Prims 25
Prim - unsorted listAfter Initialization
T PQ
A BA
B
C
12
54
G
0, (A, {}) , (B, {}) , (C, {})
C
Prim (G, r)1. for each u V2. do D [u ] 3. D[ r ] 0
4. PQ make-PQ(D,V, { })5. T
ABC
G
A 12
A 4
B 12 C 4
C 5
B 5
Copyright 1999 by Cutler/Head
Greedy/Prims 26
Build tree - after PQ.extractMin
T(A, {})
PQB
, (B, {}) , (C, {})
C
Null
A
B
C
12
54
G
A
7. while PQ do 8. (u,e) PQ.extractMin()9. add (u,e) to T
Copyright 1999 by Cutler/Head
Greedy/Prims 27
Update B, C adjacent to A
T(A, {})
PQB
12, (B, {A, B}) 4, (C, {A, C})
C
Null
A
B
C
12
54
G
A
10. for each v Adjacent (u ) 11. // relaxation operation
Copyright 1999 by Cutler/Head
Greedy/Prims 28
Build tree - after PQ.extractMin
T(A, {})(C, {A, C})
PQ
B
Null12, (B, {A, B})
C
Null
AB
C
12
54
G
A
7. while PQ do 8. (u,e) PQ.extractMin()9. add (u,e) to T
Copyright 1999 by Cutler/Head
Greedy/Prims 29
Update B adjacent to C
T(A, {})
PQT(A, {})(C, {A, C})
B
Null5, (B, {C, B})
C
Null
A
10. for each v Adjacent (u ) 11. // relaxation operation
B
C
12
54
G
A
Copyright 1999 by Cutler/Head
Greedy/Prims 30
Build tree - after PQ.extractMin
T(A, {})
PQT(A, {})(C, {A, C}) (B, {C, B})
B
Null Null
C
Null
A
7. while PQ do 8. (u,e) PQ.extractMin()9. add (u,e) to T
B
C
12
54
G
A
Copyright 1999 by Cutler/Head
Greedy/Prims 31
Prim (G)1. for each u V2. do D [u ] 3. D[ r ] 0
4. PQ make-heap(D,V, { })5. T
6 4
5
2
9
15
8
1014
3
e
f
b
a
c
d
g
h
r
PQ = {( 0,(a,)), (,(b,?)), ...(,(h,?))}
T = { } G =
D = [ 0, , …, ]
Copyright 1999 by Cutler/Head
Greedy/Prims 32
6 4
5
2
9
15
8
1014
3
e
f
b
a
c
d
g
h
r
PQ = {
T = {
7. while PQ do 8. (u,e) PQ.extractMin()9. add (u,e) to T10. for each v Adjacent (u ) 11. // relaxation operation15. return T
G =
// relaxation11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue
( D[v], v, (u,v))
D = [ 0,
Copyright 1999 by Cutler/Head
Greedy/Prims 33
Lemma 1
Let G = ( V, E) be a connected, weighted undirected graph. Let T be a promising subset of E. Let Y be the set of vertices connected by the edges in T.
If e is a minimum weight edge that connects a vertex in Y to a vertex
in V - Y, then T { e } is promising.
Note: A feasible set is promising if it can be extended to produce not only a solution , but an optimal solution.
In this algorithm: A feasible set of edges is promising if it is a subset of a Minimum Spanning Tree for the connected graph.
Copyright 1999 by Cutler/Head
Greedy/Prims 34
Outline of Proof of Correctness of Lemma 1
T is the promising subset, and e the minimum cost edge of Lemma 1
Let T ' be the MST such that T T '
We will show that if e T' then there must be another MST T" such that T {e} T".
Proof has 4 stages:
1. Adding e to T', closes a cycle in T' {e}.
2. Cycle contains another edge e' T' but e' T
3. T"=T' {e} - {e’ } is a spanning tree
4. T" is a MST
Copyright 1999 by Cutler/Head
Greedy/Prims 35
The Promising Set of Edges Selected by Prim
MST T' but e T'
X
X
XX
X
X
X
TX
X
XX
X
X
XeX
Y
V -Y
Lemma 1
Copyright 1999 by Cutler/Head
Greedy/Prims 36
Since T' is a spanning tree, it is connected. Adding the
edge, e, creates a cycle.
X
X
XX
X
X
XT' {e}
e
In T' there is a path from uY to v V -Y. Therefore the path must include another edge e' with one vertex in Y and the other in V-Y.
X
X
XX
X
X
X
X
Y
V -Y
e'
Lemma 1
u v Stage 1
Stage 2
Copyright 1999 by Cutler/Head
Greedy/Prims 37
• If we remove e’ from T’ { e } the cycle disappears.• T”=T’ { e } - {e’} is connected. T’ is connected. Every pair of vertices
connected by a path that does not include e’ is still connected in T”. Every pair of vertices connected by a path that included e’ , is still connected in T” because there is a path in T" =T’ { e } - { e’ } connecting the vertices of e’.
X
Y
V -Y
Lemma 1
X
X
XX
X
X
Xe
T "
w( e ) w( e' )By the way Prim picks the next edge
Stage 3
Copyright 1999 by Cutler/Head
Greedy/Prims 38
• w( e ) w( e' ) by the way Prim picks the next edge.• The weight of T", w(T") = w (T' ) + w( e ) - w( e' ) w(T').• But w (T' ) w(T") because T' is a MST.• So w (T' ) = w(T") and T" is a MST
X
Y
V -Y
Lemma 1
X
X
XX
X
X
Xe
T"
Conclusion T { e } is promising
Stage 4
Copyright 1999 by Cutler/Head
Greedy/Prims 39
Theorem : Prim's Algorithm always produces a minimum spanning tree.
Proof by induction on the set T of promising edges.Base case: Initially, T = is promising.
Induction hypothesis: The current set of edges T selected by Prim is promising.
Induction step: After Prim adds the edge e, T { e } is promising.
Proof: T { e } is promising by Lemma 1.
Conclusion: When G is connected, Prim terminates with |T | = |V | -1, and T is a MST.
Copyright 1999 by Cutler/Head
Greedy/Prims 41
solution = { } while ( more edges in E) do // Selection
select minimum weight edgeremove edge from E // Feasibility
if (edge closes a cycle with solution so far)then reject edgeelse add edge to solution
// Solution check
if |solution| = |V | - 1 return solution
return null // when does this happen?
Kruskal's Algorithm: Main Idea
6 4
5
2
158
1014
3
u
v
b
a
c
d
f
Copyright 1999 by Cutler/Head
Greedy/Prims 42
6 4
5
2
9
15
8
1014
3
e
f
b
a
c
d
g
h
C = { {a}, {b}, {c}, {d}, {e}, {f}, {g}, {h} }
C is a forest of trees.
Kruskal's Algorithm:
1. Sort the edges E in non-decreasing weight2. T 3. For each v V create a set. 4. repeat 5. Select next {u,v} E, in order 6. ucomp find (u) 7. vcomp find (v) 8. if ucomp vcomp then8. add edge (u,v) to T 9. union ( ucomp,vcomp )10.until T contains |V | - 1 edges11. return tree T
Copyright 1999 by Cutler/Head
Greedy/Prims 43
Kruskal - Disjoint setAfter Initialization
TA
B
C
2
56
G
1. Sort the edges E in non-decreasing weight2. T
3. For each v V create a set.
A B 2
Sorted edges
B C 5
A C 6
A B CDisjoint data set for G
D
B D 77
D
Copyright 1999 by Cutler/Head
Greedy/Prims 44
Kruskal - add minimum weight edge if feasible
A
C
2
56
G
B
5. for each {u,v} in ordered E 6. ucomp find (u) 7. vcomp find (v) 8. if ucomp vcomp then9. add edge (v,u) to T 10. union( ucomp,vcomp )
TSorted edges
A B CDisjoint data set for G
Find(A) Find(B)
AB
C
After merge(A, B)
(A, B) A B 2
B C 5
A C 6
B D 7
D
D
7
D
Copyright 1999 by Cutler/Head
Greedy/Prims 45
Kruskal - add minimum weight edge if feasible
A
C
2
56
G
B
5. for each {u,v} in ordered E 6. ucomp find (u) 7. vcomp find (v) 8. if ucomp vcomp then9. add edge (v,u) to T 10. union ( ucomp,vcomp )
T
A
B
C
Find(B) Find(C)
AB C
After merge(A, C)
(A, B)
(B, C)
Sorted edges A B 2
B C 5
A C 6
B D 7
D
D
7
D
Copyright 1999 by Cutler/Head
Greedy/Prims 46
Kruskal - add minimum weight edge if feasible
A
C
2
56
G
B
5. for each {u,v} in ordered E 6. ucomp find (u) 7. vcomp find (v) 8. if ucomp vcomp then9. add edge (v,u) to T 10. union ( ucomp,vcomp )
T
A
B C
Find(A) Find(C)
A and C in same set
(A, B)
(B, C)
Sorted edges A B 2
B C 5
A C 6
B D 7
D
7
D
Copyright 1999 by Cutler/Head
Greedy/Prims 47
Kruskal - add minimum weight edge if feasible
A
C
2
56
G
B
5. for each {u,v} in ordered E 6. ucomp find (u) 7. vcomp find (v) 8. if ucomp vcomp then9. add edge (v,u) to T 10. union ( ucomp,vcomp )
T
A
B C
Find(B) Find(D)
(A, B)
(B, C)
(B, D)
Sorted edges A B 2
B C 5
A C 6
B D 7
D
A
B C D
After merge
7
D
Copyright 1999 by Cutler/Head
Greedy/Prims 48
Kruskal ( G )1. Sort the edges E in non-decreasing weight2. T 3. For each v V create a set. 4. repeat 5. {u,v} E, in order 6. ucomp find (u) 7. vcomp find (v) 8. if ucomp vcomp then9. add edge (v,u) to T 10. union ( ucomp,vcomp )11.until T contains |V | - 1 edges12. return tree T
Count1 = ( E lg E )
Count2= (1)
Count3= ( V )
Count4 = O( E )
Using Disjoint set-height andpath compression
Count4(6+7+10)= O((E +V) (V))
Sorting dominates the runtime. We get T( E,V ) = ( E lg E), so for a sparse graph we get ( V lg V)for a dense graph we get( V2 lg V2) = ( V2 lg V)
Kruskal's Algorithm: Time Analysis