23/4/21 2004 SDU

Lecture 6-Minimum Spanning Tree

1. The Minimum Spanning Tree Problem

2. Greedy algorithms

3. A Generic Algorithm

4. Kruskal’s Algorithm

5. Prim’s Algorithm

2004 SDU 2

An Application

In the design of networkingGiven n computers, we want to connect them so that each pair of them can communicate with each other.Every foot of cable costs us $1We want the cheapest possible network.

In the design of electronic circuitryGiven n pins that should be electrically equivalent by wiring them togetherTo interconnect a set of n pins, n-1 wires can be used, each connecting two pinsWe want the least possible length of wire to save cost

2004 SDU 3

Minimum Spanning Tree--The Definition

Given a connected undirected graph G = (V, E) and a weight w(e) to each edge e of G, a minimum spanning tree T of G is a spanning tree with minimum total edge weight

Note: (1). A spanning tree of a graph is a tree that connects all vertices.(2). A connected undirected graph may have many different spanning trees(3). The minimum spanning tree may not be unique

2004 SDU 4

Minimum Spanning Tree Problem

5

4

3

3

6

1

6

98

3

7 1

27

5

4

3

3

6

1

6

98

3

7 1

27

The Problem:• Input: A connected undirected graph with weight (G, W).• Output: A minimum spanning tree T for G.

Page 5

Greedy Algorithms

A greedy algorithm always makes the choice that looks best at the moment.

It makes a local optimal choice in the hope that this choice will lead to a globally optimal solution.

Greedy algorithms yield optimal solutions for many (but not all) problems.

2004 SDU 5

Page 6

The 0-1 Knapsack problem

The 0-1 knapsack problem:N items, where the i-th item is worth vi dollars and weight wi pounds. 11 p 3 p 4p 58 p 8p 88p

vi and wi are integers.

3$ 6 $ 35$ 8$ 28$ 66$

The thief can carry at most W (integer) pounds.How to take as valuable a load as possible. An item cannot be divided into pieces.

The fractional knapsack problem: The same setting, but the thief can take fractions of items. W may not be integer.

W

2004 SDU 6

Page 7

Solve the fractional Knapsack problem: Greedy on the value per pound vi/wi. Each time, take the item with maximum vi/wi .

If exceeds W, take fractions of the item. Example: (1, 5$), (2, 9$), (3, 12$), (3, 11$) and w=4. vi/wi : 5 4.5 4.0 3.667 First: (1, 5$), Second: (2, 9$), Third: 1/3 (3, 12$) Total W: 1, 3, 4.

Can only take part of

item

2004 SDU 7

Page 8

Proof of correctness: (The hard part)Let X = i1, i2, …ik be the optimal items taken. • Consider the item j : (vj, wj) with the highest v /w. • if j is not used in X (the optimal solution), get rid of some items with total weight w j

(possibly fractional items) and add item j. (since fractional items are allowed, we can do it.)

Total value is increased. Why?

One more item selected by greedy is added to X Repeat the process, X is changed to contain all items selected by greedy WITHOUT decreasing the total value taken by the thief.

jj

jj

j

jpp

j

jp

j

jp

j

j

j

j

p

pp

p

pp

pp

vw

vw

w

vwqwww

w

vwq

w

vw

w

vw

w

vw

w

vwq

w

vw

w

vw

w

vw

vqvvv

)()()%...(

)(%)(...)()(

)(%)(...)()(

%...

121

121

1

11

2

22

1

11

121

2004 SDU 8

Page 9

The 0-1 knapsack problem cannot be solved optimally by greedy

Counter example: W=10 2 1.8Items found (6pounds, 12dollars), (5pounds, 9 dollar),

1.8 1 1 (5pounds, 9 dollars), (3pounds, 3 dollars), (3 pounds, 3 dollars)

If we first take (6, 12) according to greedy algorithm, then solution is (6, 12), (3, 3) (total value is 12+3=15).However, a better solution is (5, 9), (5, 9) with total value 18.

Note: To show that a statement does not hold, we only have to give an example.

2004 SDU 9

Greedy algorithm analysisIngredients of greedy algorithm for solving a problem: Greedy-choice property: a global optimal solution can be arrived at by

making a locally optimal(greedy) choice. (fractional knapsack yes, 0-1knapsack no )

Optimal substructure: an optimal solution to the problem contains within it optimal solutions to sub-problems.(both fractional and 0-1 knapsack yes)

Correctness proof Inductively prove that after each step of the greedy algorithm, its

solution is at least as good as any other algorithm’s. (Huffman code) Gradually transform any solution to the one found by the greedy

algorithm. (Fractional knapsack) Discover a simple “structural” bound asserting that any possible solution

must have a certain value (lower bound). Then show that the greedy algorithm always achieves that bound. (Interval partitioning)

2004 SDU 10

2004 SDU 11

Idea of the Generic Algorithm for MST

It grows the minimum spanning tree one edge at a time. The algorithm manages a set of edges A, maintaining the

following loop invariant:– Prior to each iteration, A is a subset of some minimum spanning

tree.

An edge (u, v) is a safe edge for A if A{(u, v)} is also a subset of a minimum spanning tree, that is, (u, v) can be correctly added to A without violating the

invariant

Genetic algorithmKruskal’s and Prim’s algorithms are implementations of the generic algorithm on how to maintain A and find the safe edge (u, v) for A.

At any time, edges of Kruskal’s form some trees, and (u,v) is the least edge between these trees

Edges of Prim’s form one tree, and (u,v) is the least connecting the tree and other vertices.

Safe edge is the greedy choice. At any time, the sub-problem is the graph obtained by contracting any component of A as a vertex.

The problem has both greedy-choice property and optimal-substructure property.

2004 SDU 12

2004 SDU 13

1. A cut (X, Y) of a graph G = (V, E) is a partition of the vertex set V into two sets X and Y = V \ X.

2. An edge (u, v) E is said to cross the cut (X, Y) if u ∈ X and v ∈ Y.3. A cut (X, Y) respects a set A of edges if no edge in A crosses the cut.4. An edge is a light edge crossing a cut if its weight is the minimum of

any edge crossing the cut.

Related Notions

• Set of red vertices X and set of green vertices Y form a cut (X, Y) of the graph.

• Blue edges cross the cut (X, Y)

• (X, Y) respects a set A of edges consisting onlyof red or green edges.

2004 SDU 14

Determine Safe Edges for A

Theorem 23.1 Let G = (V, E) be a connected, undirected graph with

real-valued weight function w defined on E. Let A be a subset of E that is included in some minimum spanning tree for G, let (X, Y) be any cut of G that respects A, and let (u, v) be a light edge crossing (X, Y). Then, edge (u, v) is safe for A.

2004 SDU 15

The Proof of Theorem 23.1

Proof: Let T be a MST including A, and assume that T does not contain the light edge (u, v). We shall construct another MST T’ that includes A{(u, v)}, showing that (u, v) is a safe edge for A.

The edge (u, v) forms a cycle with the edges on the path from u to v in T. Since u and v are on opposite sides of the cut (X,Y), there is at least one edge in T on the path p that also crosses the cut. Let (x, y) be any such edge. The edge (x, y) is not in A since the cut respects A. Since (x, y) is on the unique path from u to v in T, removing (x, y) breaks T into two components. Adding (u, v) reconnects them to form a new spanning tree T’ = T-{(x, y)} {(u, v)}.

2004 SDU 16

A Corollary

Corollary 23.2 Let G = (V, E) be a connected, undirected graph with

real-valued weight function w defined on E. Let A be a subset of E that is included in some minimum spanning tree for G, and let C = (VC, EC) be a connected component in the forest GA = (V, A). If (u, v) is a light edge connecting C to some other component in GA, then (u, v) is safe for A.

Note: Kruskal’s and Prim’s algorithms are based on the corollary.

2004 SDU 17

Idea of the Kruskal’s

1. Initialize the forest, each vertex as a tree, A .

2. Find the least weight edge (u, v) that connects any two trees in the forest.

3. Add (u, v) to A and make the two tree as one tree, number of trees in the forest decreases 1.

4. Repeat step 2 and 3 until A forms a spanning tree.

2004 SDU 18

Kruskal’s Algorithm

2004 SDU 19

2

1

1 12 8

53

9

4

3

25

1

6

ab

cd

e

f

gi

h

(a, d):1 (h, i):1 (c, e):1 (f, h):2 (g, h):2(b, c):3 (b, f):3 (b, e):4 (c, d):5 (f, g):5(e, i):6 (d, g):8 (a, b):9 (c, f):12

2

1

1 12 8

53

9

4

3

25

1

6

ab

cd

e

f

gi

h

(a, d):1(h, i):1 (c, e):1 (f, h):2 (g, h):2(b, c):3 (b, f):3 (b, e):4 (c, d):5 (f, g):5(e, i):6 (d, g):8 (a, b):9 (c, f):12

2004 SDU 20

2

1

1 12 8

53

9

4

3

25

1

6

ab

cd

e

f

gi

h

(a, d):1(h, i):1 (c, e):1 (f, h):2 (g, h):2(b, c):3 (b, f):3 (b, e):4 (c, d):5 (f, g):5(e, i):6 (d, g):8 (a, b):9 (c, f):12

2

1

1 12 8

53

9

4

3

25

1

6

ab

cd

e

f

gi

h

(a, d):1(h, i):1 (c, e):1 (f, h):2 (g, h):2(b, c):3 (b, f):3 (b, e):4 (c, d):5 (f, g):5(e, i):6 (d, g):8 (a, b):9 (c, f):12

2004 SDU 21

2

1

1 12 8

53

9

4

3

25

1

6

ab

cd

e

f

gi

h

(a, d):1(h, i):1 (c, e):1 (f, h):2 (g, h):2(b, c):3 (b, f):3 (b, e):4 (c, d):5 (f, g):5(e, i):6 (d, g):8 (a, b):9 (c, f):12

a

2

1

1 12 8

53

9

4

3

25

1

6

bc

d

e

f

gi

h

(a, d):1(h, i):1 (c, e):1 (f, h):2 (g, h):2(b, c):3 (b, f):3 (b, e):4 (c, d):5 (f, g):5(e, i):6 (d, g):8 (a, b):9 (c, f):12

2004 SDU 22

2

1

1 12 8

53

9

4

3

25

1

6

ab

cd

e

f

gi

h

(a, d):1(h, i):1 (c, e):1 (f, h):2 (g, h):2(b, c):3 (b, f):3 (b, e):4 (c, d):5 (f, g):5(e, i):6 (d, g):8 (a, b):9 (c, f):12

2

1

1 12 8

53

9

4

3

25

1

6

ab

cd

e

f

gi

h

(a, d):1(h, i):1 (c, e):1 (f, h):2 (g, h):2(b, c):3 (b, f):3 (b, e):4 (c, d):5 (f, g):5(e, i):6 (d, g):8 (a, b):9 (c, f):12

2004 SDU 23

2

1

1 12 8

53

9

4

3

25

1

6

ab

cd

e

f

gi

h

(a, d):1(h, i):1 (c, e):1 (f, h):2 (g, h):2(b, c):3 (b, f):3 (b, e):4 (c, d):5 (f, g):5(e, i):6 (d, g):8 (a, b):9 (c, f):12

2

1

112 8

53

9

4

3

25

1

6

ab

cd

e

f

gi

h

(a, d):1(h, i):1 (c, e):1 (f, h):2 (g, h):2(b, c):3 (b, f):3 (b, e):4 (c, d):5 (f, g):5(e, i):6 (d, g):8 (a, b):9 (c, f):12

2004 SDU 24

2

1

1 12 8

53

9

4

3

25

1

6

ab

cd

e

f

gi

h

(a, d):1(h, i):1 (c, e):1 (f, h):2 (g, h):2(b, c):3 (b, f):3 (b, e):4 (c, d):5 (f, g):5(e, i):6 (d, g):8 (a, b):9 (c, f):12

2

1

112 8

53

9

4

3

25

1

6

ab

cd

e

f

gi

h

(a, d):1(h, i):1 (c, e):1 (f, h):2 (g, h):2(b, c):3 (b, f):3 (b, e):4 (c, d):5 (f, g):5(e, i):6 (d, g):8 (a, b):9 (c, f):12

2004 SDU 25

Time Complexity Analysis

1. O(V2+ElgV) when disjoint-set data structure are not used.

• Keep two arrays: c[u]: The component name of vertex u

• v[c]: The vertices in component c, is a list

• FIND-SET(u): Checks c[u] //O(1), total (E)

• Union (u, v): Append list c[v] at the end of list c[u], and for each vertex in c[v], define c[] as c[u]. //O(V), total O(V2)

2. When use disjoint-set data structure with union-by-rank

• Keep two arrays: p[u]: parent of u in the min-tree

• rank[u]: height of tree rooted at u

• FIND-SET (u): if p[u] = u, return u; else FIND-SET (p[u]); // O(lgV)

• Union (u, v): if rank (FIND-SET(u)) > rank (FIND-SET(v)), p [FIND-SET(v)]=FIND-SET(u); if rank (FIND-SET(u)) < rank(FIND-SET(v)), p[FIND-SET(u)]=FIND-SET(v); else p[FIND-SET(u)]=FIND-SET(v), rank[FIND-SET(v)]++; //O(1), total (V)

So total time is: O(ElgV)u v

1. At any time, A is a subset of a MST2. In the end, A is a MST

Proof of 1. By induction on |A|.• Base case A0, of course true.• Suppose 1 is true for Ak-1. Suppose the kth edge added to A is ek=(u,v). W.L.O.G, suppose uC, C is a connectedcomponent of G(V, Ak-1). Since (u, v) is the smallest remaining edge, it must be a light edge connecting C to all other components of G(V, Ak-1), so, it is safe for Ak-1, and hence Ak is subset of a MST.

Correctness

ei

Proof of 2. By 1, in the end, A is a subset of a MST T. By contradiction suppose that A is not a MST. Then, T \ A . Suppose ei T\A. ei must be weightier than each edge in A, sinceotherwise ei should have been added to A. Then, after all the edges in A have been chosen, there must be a time that ei is considered, since eiA do not contain a cycle, ei should be added to A at that time. A contradiction.

Correctness (cont.)

2004 SDU 28

Idea of the Prim

1. Start from a vertex r, add r to a vertex set U which is initialized to empty.

2. Find the least weight edge (u, v), uU, vV-U, add (u, v) to A, and add v to U. A is the loop invariant.

3. Repeat 2 until A forms a spanning tree or U = V.

2004 SDU 29

Prim’s Algorithm

//((u), u) is added to Au is added to U

2004 SDU 30

2

1

112 8

53

9

4

3

25

1

6

rb

cd

e

f

gi

h 2

1

1 12 8

539

4

3

25

1

6

r

bc

d

e

f

gi

h

2

1

112 8

53

9

4

3

25

1

6

r

bc

d

e

f

gi

h2

1

112 8

53

9

4

3

25

1

6

r

bc

d

e

f

gi

h

2004 SDU 31

2

1

112 8

53

9

4

3

25

1

6

r

bc

d

e

f

gi

h

2

1

112 8

53

9

4

3

25

1

6

r

bc

d

e

f

gi

h

a

2

1

112 8

53

9

4

3

25

1

6

bc

d

e

f

gi

h

r

2

1

112 8

53

9

4

3

25

1

6

r

bc

d

e

f

gi

h

2004 SDU 32

2

1

112 8

53

9

4

3

25

1

6

r

bc

d

e

f

gi

h

r

2

1

112 8

53

9

4

3

25

1

6

bc

d

e

f

gi

h

2004 SDU 33

Time Complexity

Running time Using binary min-heap

– EXTRACT-MIN: return and delete root of the heap, put the last leaf as the new root, and search down to find its proper position. //O(lgV), total O(VlgV)

– key(v)w(u,v): check array p[v] to find v’s position in the heap, then change its key value, and search back to find its proper position in the heap. //O(lgV), total O(ElgV)

– Total: O(E lg V)

Using Fibonacci heap O(E+VlgV)

1: each time we add a safe edge (a light edge crossing (U, Q)).2: Notice that since G is connected, at any time there is at leastone vertex in Q whose key value is less than . Thus, from the second iteration, at each iteration, we always can add an edge into A. After V iterations, we get V-1 edges.

Correctness

2004 SDU 35