1 Minimum Spanning Tree Problem Topic 10 ITS033 – Programming & Algorithms JFK BOS MIA ORD LAX DFW SFO BWI PVD 867 2704 187 1258 849 144 740 1391 184 946 1090 1121 2342 1846 621 802 1464 1235 337 Asst. Prof. Dr. Bunyarit Uyyanonvara Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University http://www.siit.tu.ac.th/bunyarit [email protected]02 5013505 X 2005
Transcript
1
Minimum Spanning Tree Problem
Topic 10 ITS033 – Programming & Algorithms JFK
BOS
MIA
ORD
LAXDFW
SFO BWI
PVD
8672704
187
1258
849
144740
1391
184
946
1090
1121
2342
1846 621
802
1464
1235
337
Asst. Prof. Dr. Bunyarit UyyanonvaraAsst. Prof. Dr. Bunyarit UyyanonvaraIT Program, Image and Vision Computing Lab.
School of Information, Computer and Communication Technology (ICT)
Sirindhorn International Institute of Technology (SIIT)
ITS033Topic 01Topic 01 -- Problems & Algorithmic Problem SolvingProblems & Algorithmic Problem SolvingTopic 02Topic 02 – Algorithm Representation & Efficiency Analysis – Algorithm Representation & Efficiency AnalysisTopic 03Topic 03 - State Space of a problem - State Space of a problemTopic 04Topic 04 - Brute Force Algorithm - Brute Force AlgorithmTopic 05Topic 05 - Divide and Conquer - Divide and ConquerTopic 06Topic 06 -- Decrease and ConquerDecrease and ConquerTopic 07Topic 07 - Dynamics Programming - Dynamics ProgrammingTopic 08 - Transform and ConquerTopic 09Topic 09 - Graph Algorithms - Graph AlgorithmsTopic 10Topic 10 - Minimum Spanning Tree - Minimum Spanning TreeTopic 11Topic 11 - Shortest Path Problem - Shortest Path ProblemTopic 12Topic 12 - Coping with the Limitations of Algorithms Power - Coping with the Limitations of Algorithms Power
Imagine: 1. You wish to connect all the computers in an office building using the least amount of cable
Concrete exampleConcrete example
B
A
H
E
G
F
D
C
5 0 2 5
4 62 5
2 3
5 5
3 2
9 8
3 5
6 7
B
A
H
E
G
F
D
C
2 5
4 6
2 52 3
5 5
3 2
3 5
M in im u m am o u n t o f cab le = 2 4 1
M in im u m s p an n in g t reeN et w o rk o f H u b s
6
Imagine:
2. If you want to build a railway network to all provinces in Thailand with minimum cost, i.e., minimum total length of the rail way.
Concrete exampleConcrete example
7
8
Imagine:
3. Or a network of super-highway to connect all the cities of Bangkok with minimum cost
9
Imagine:
4. We want to connect electricity to all of the houses in the whole village with minimum total of cable length
Concrete exampleConcrete example
10
Each vertex in a graph G represents a computer, a province or a house
Each edge represents the amount of cable, length of a railway needed to connect all vertices
That‘s a weighted graph problem !! A problem of finding A Minimum Spanning Tree
Minimum Spanning Tree Minimum Spanning Tree ProblemProblem
11
• Given a connected, undirected graph, a spanning tree of that graph is a sub-graph which is a tree and connects all the vertices together. A single graph can have many different spanning trees.
• We can assign a weight to a spanning tree by computing the sum of the weights of the edges in that spanning tree.
• A minimum spanning tree or minimum weight spanning tree is then a spanning tree with weight less than or equal to the weight of every other spanning tree.
Minimum Spanning Tree Minimum Spanning Tree (MST)(MST)
ORD
PIT
ATL
STL
DEN
DFW
DCA
101
9
8
6
3
25
7
4
12
We are interested in:
Finding a tree T that contains all the vertices of a graph G (spanning tree)and has the least total weight over all.
such trees is minimum-spanning tree (MST)
Tuv
uvwTw),(
)),(()(
13
Minimum Spanning Tree
14
Possible multiplicityThere may be several minimum spanning trees of the same weight; in particular, if all weights are the same, every spanning tree is minimum.
PropertiesProperties
15
UniquenessIf each edge has a distinct weight then there will only be one, unique minimum spanning tree.
Minimum-cost subgraphIf the weights are non-negative, then a minimum spanning tree is in fact the minimum-cost subgraph connecting all vertices, since subgraphs containing cycles necessarily have more total weight.
Cycle propertyFor any cycle C in the graph, if the weight of an edge e of C is larger than the weights of other edges of C, then this edge cannot belong to a MST.
PropertiesProperties
16
Cycle Property
Cycle Property: Let T be a minimum
spanning tree of a weighted graph G
Let e be an edge of G that is not in T and C let be the cycle formed by e with T
For every edge f of C, weight(f) weight(e)
Proof: By contradiction If weight(f) weight(e) we
can get a spanning tree of smaller weight by replacing e with f
84
2 36
7
7
9
8e
C
f
84
2 36
7
7
9
8
C
e
f
Replacing f with e yieldsa better spanning tree
17
U V
Partition Property
Partition Property: Consider a partition of the vertices
of G into subsets U and V Let e be an edge of minimum weight
across the partition There is a minimum spanning tree
of G containing edge e
Proof: Let T be an MST of G If T does not contain e, consider the
cycle C formed by e with T and let f be an edge of C across the partition
By the cycle property,weight(f) weight(e)
Thus, weight(f) weight(e) We obtain another MST by
replacing f with e
74
2 85
7
3
9
8 e
f
74
2 85
7
3
9
8 e
f
Replacing f with e yieldsanother MST
U V
18
Topic 10.2 ITS033 – Programming & Algorithms JFK
BOS
MIA
ORD
LAXDFW
SFO BWI
PVD
8672704
187
1258
849
144740
1391
184
946
1090
1121
2342
1846 621
802
1464
1235
337
Asst. Prof. Dr. Bunyarit UyyanonvaraAsst. Prof. Dr. Bunyarit UyyanonvaraIT Program, Image and Vision Computing Lab.
School of Information, Computer and Communication Technology (ICT)
Sirindhorn International Institute of Technology (SIIT)
Try to use exhaustive search ? If we were to try an exhaustive-search approach to
constructing a minimum spanning tree, we would face two serious obstacles.
1) the number of spanning trees grows exponentially with the graph size (at least for dense graphs).
2) generating all spanning trees for a given graph is not easy;
Greedy Technique20
Greedy Approach The greedy approach suggests constructing a solution
through a sequence of steps, each expanding a partially constructed solution obtained so far, until a complete solution to the problem is reached. On each step—and this is the central point of this technique—the choice made must be
feasible, i.e., it has to satisfy the problem’s constraints. locally optimal, i.e., it has to be the best local choice
among all feasible choices available on that step. irrevocable, i.e., once made, it cannot be changed on
subsequent steps of the algorithm.
21
Prim’s Algorithm Prim’s algorithm constructs a minimum spanning
tree through a sequence of expanding subtrees.
The initial subtree in such a sequence consists of a single vertex selected arbitrarily from the set V of the graph’s vertices.
On each iteration, we expand the current tree in the greedy manner by simply attaching to it the nearest vertex not in that tree.
The algorithm stops after all the graph’s vertices have been included in the tree being constructed.
22
Prim’s Algorithm
23
Prim’s Algorithm The nature of Prim’s algorithm makes it necessary to
provide each vertex not in the current tree with the information about the shortest edge connecting the vertex to a tree vertex.
We can provide such information by attaching two labels to a vertex:
(1) The name of the nearest tree vertex and the length (the weight) of the corresponding edge.
(2) Vertices that are not adjacent to any of the tree vertices canbe given the ∞ label indicating their “infinite” distance to the tree vertices and a null label for the name of the nearest tree vertex.
24
Prim’s Algorithm We pick an arbitrary vertex s and we grow the MST as a
cloud of vertices, starting from s
We store with each vertex v a label d(v) = the smallest weight of an edge connecting v to a vertex in the cloud
At each step: We add to the cloud the vertex u outside the cloud with the
smallest distance label
We update the labels of the vertices adjacent to u
25
Prim‘s AlgorithmPrim‘s Algorithm
1. All vertices are marked as not visited
2. Any vertex v you like is chosen as starting vertex and is marked as visited (define a cluster C)
3. The smallest- weighted edge e = (v,u), which connects one vertex v inside the cluster C with another vertex u outside of C, is chosen and is added to the MST.
4. The process is repeated until a spanning tree is formed
Prim’s Algorithm Analysis Efficiency of Prim’s algorithm depends on the data
structures chosen for the graph and for the priority queue of the set V-VT whose vertex priorities are the distances to the nearest tree vertices.
For example, if a graph is represented by its weight matrix and the priority queue is implemented as an unordered array, the algorithm’s running time will be in θ(|V |2).
Greedy Technique42
Prim’s Algorithm A min-heap is a complete binary tree in which every element
is less than or equal to its children.
Deletion of the smallest element from and insertion of a new element into a min-heap of size n are O(log n) operations, and so is the operation of changing an element’s priority
If a graph is represented by its adjacency linked lists and the priority queue is implemented as a min-heap, the running time of the algorithm is in O(|E| log |V |).
This is because the algorithm performs |V| - 1 deletions of the smallest element and makes |E| verifications and, possibly, changes of an element’s priority in a min-heap of size not greater than |V |. Each of these operations, as noted earlier, is a O(log |V |) operation. Hence, the running time of this implementation of Prim’s algorithm is in
•Kruskal's algorithm is an algorithm in graph theory that finds a minimum spanning tree for a connected weighted graph.
•If the graph is not connected, then it finds a minimum spanning forest (a minimum spanning tree for each connected component).
•Kruskal's algorithm is also an example of a greedy algorithm.
Minimum Spanning Tree Minimum Spanning Tree ProblemProblem
46
It works as follows:
1.Each vertex is in its own cluster
2. Take the edge e with the smallest weight - if e connects two vertices in different clusters, then e is added to the MST and the two clusters, which are connected by e, are merged into a single cluster - if e connects two vertices, which are already in the same cluster, ignore it
3. Continue until n-1 edges were selected
Minimum Spanning Tree Minimum Spanning Tree ProblemProblem
Greedy Technique47
9.2 Kruskal’s Algorithm
Greedy Technique48
Kruskal’s Algorithm Analysis On each of its iterations, Kruskal’s algorithm has
to check whether the addition of the next edge to the edges already selected would create a cycle.
A new cycle is created if and only if the new edge connects two vertices already connected by a path, i.e., if and only if the two vertices belong to the same connected component
Each connected component of a subgraph generated by Kruskal’s algorithm is a tree because it has no cycles.
Greedy Technique49
Kruskal’s Algorithm Analysis
New edge connecting two vertices may (a) or may not (b) create a cycle
Kruskal’s Algorithm Analysis The check whether two vertices belong to the same
tree is crucial in determining running time of Kruskal’s algorithm
There is efficient algorithm that perform this check => union-find algorithm
With an efficient union- find algorithm, the running time of Kruskal’s algorithm will be dominated by the time needed for sorting the edge weights of a given graph.
Hence, with an efficient sorting algorithm, the time efficiency of Kruskal’s algorithm will be in O(|E| log |E|).
74
Data Structure for Kruskal Algortihm The algorithm maintains a forest of trees An edge is accepted it if connects distinct
trees We need a data structure that maintains a
partition, i.e., a collection of disjoint sets, with the operations:
75
Kruskal’s Minimum Spanning Tree
76
77
78
ITS033Topic 01Topic 01 -- Problems & Algorithmic Problem SolvingProblems & Algorithmic Problem SolvingTopic 02Topic 02 – Algorithm Representation & Efficiency Analysis – Algorithm Representation & Efficiency AnalysisTopic 03Topic 03 - State Space of a problem - State Space of a problemTopic 04Topic 04 - Brute Force Algorithm - Brute Force AlgorithmTopic 05Topic 05 - Divide and Conquer - Divide and ConquerTopic 06Topic 06 -- Decrease and ConquerDecrease and ConquerTopic 07Topic 07 - Dynamics Programming - Dynamics ProgrammingTopic 08 - Transform and ConquerTopic 09Topic 09 - Graph Algorithms - Graph AlgorithmsTopic 10Topic 10 - Minimum Spanning Tree - Minimum Spanning TreeTopic 11Topic 11 - Shortest Path Problem - Shortest Path ProblemTopic 12Topic 12 - Coping with the Limitations of Algorithms Power - Coping with the Limitations of Algorithms Power
