November 13, 2003

1

Algorithms and Data StructuresLecture XII

Simonas ŠaltenisAalborg Universitysimas@cs.auc.dk

November 13, 2003

2

This Lecture

Weighted Graphs Minimum Spanning Trees

Greedy Choice Theorem Kruskal’s Algorithm Prim’s Algorithm

November 13, 2003

3

Spanning Tree

A spanning tree of G is a subgraph which is a tree contains all vertices of G

How many edges are there in a spanning tree, if there are V vertices?

November 13, 2003

4

Minimum Spanning Trees

Undirected, connected graph G = (V,E)

Weight function W: E R (assigning cost or length or other values to edges)

( , )

( ) ( , )u v T

w T w u v

Spanning tree: tree that connects all the vertices

Optimization problem – Minimum spanning tree (MST): tree T that connects all the vertices and minimizes

November 13, 2003

5

Optimal Substructure

”Cut and paste” argument If G’ would have a cheaper ST T’, then we

would get a cheaper ST of G: T’ + (u, v)

MST(G) = T

uv

”u+v”

MST(G’) = T – (u,v)

November 13, 2003

6

Idea for an Algorithm

We have to make V – 1 choices (edges of the MST) to arrive at the optimization goal

After each choice we have a sub-problem one vertex smaller than the original Dynamic programming algorithm, at each

stage, would consider all possible choices (edges)

If only we could always guess the correct choice – an edge that definitely belongs to an MST

November 13, 2003

7

Greedy Choice

Greedy choice property: locally optimal (greedy) choice yields a globally optimal solution

Theorem Let G=(V, E), and let S V and let (u,v) be min-weight edge in G

connecting S to V – S : a light edge crossing a cut

Then (u,v) T – some MST of G

November 13, 2003

8

Greedy Choice (2) Proof

Suppose (u,v) is light, but (u,v) any MST look at path from u to v in some MST T Let (x, y) – the first edge on path from u to v in T

that crosses from S to V – S. Swap (x, y) with (u,v) in T.

this improves T – contradiction (T supposed to be an MST)

u v

xy

S V-S

November 13, 2003

9

Generic MST Algorithm

Generic-MST(G, w)1 A// Contains edges that belong to a MST2 while A does not form a spanning tree do3 Find an edge (u,v) that is safe for A 4 AA{(u,v)}5 return A

Generic-MST(G, w)1 A// Contains edges that belong to a MST2 while A does not form a spanning tree do3 Find an edge (u,v) that is safe for A 4 AA{(u,v)}5 return A

Safe edge – edge that does not destroy A’s property

MoreSpecific-MST(G, w)1 A// Contains edges that belong to a MST2 while A does not form a spanning tree do3.1 Make a cut (S, V-S) of G that respects A 3.2 Take the min-weight edge (u,v) connecting S to V-S

4 AA{(u,v)}5 return A

MoreSpecific-MST(G, w)1 A// Contains edges that belong to a MST2 while A does not form a spanning tree do3.1 Make a cut (S, V-S) of G that respects A 3.2 Take the min-weight edge (u,v) connecting S to V-S

4 AA{(u,v)}5 return A

November 13, 2003

10

Pseudocode assumptions Graph ADT with an operation

V():VertexSet E(): EdgeSet w(u:Vertex, v:Vertex):int – returns a weight of

(u,v) A looping construct “for each v V ”, where V

is of a type VertexSet, and v is of a type Vertex

Vertex ADT with operations: adjacent():VertexSet key():int and setkey(k:int) parent():Vertex and setparent(p:Vertex)

November 13, 2003

11

Prim-Jarnik Algorithm

Vertex based algorithm Grows one tree T, one vertex at a

time A set A covering the portion of T

already computed Label the vertices v outside of the set

A with v.key() – the minimum weigth of an edge connecting v to a vertex in A, v.key() = , if no such edge exists

November 13, 2003

12

MST-Prim(G,r)01 for each vertex u G.V()02 u.setkey(03 u.setparent(NIL)04 r.setkey(0)05 Q.init(G.V()) // Q is a priority queue ADT06 while not Q.isEmpty()07 u Q.extractMin() // making u part of T08 for each v u.adjacent() do09 if v Q and G.w(u,v) < v.key() then10 v.setkey(G.w(u,v))11 Q.modifyKey(v)12 v.setparent(u)

Prim-Jarnik Algorithm (2)

updating keys

November 13, 2003

13

Prim-Jarnik’s Example

Let’s do an example:

H

B C

I

A

D

G

F E

Let’s keep track: What is set A, which vertices are in Q,

what cuts are we making, what are the key values

1

68

4

12

8

3

6

5 3

4 13 9

10

November 13, 2003

14

MST-Prim(G,r)01 for each vertex u G.V()02 u.setkey(03 u.setparent(NIL)04 r.setkey(0)05 Q.init(G.V()) // Q is a priority queue ADT06 while not Q.isEmpty()07 u Q.extractMin() // making u part of T08 for each v u.adjacent() do09 if v Q and G.w(u,v) < v.key() then10 v.setkey(G.w(u,v))11 Q.modifyKey(v)12 v.setparent(u)

Implementation issues

November 13, 2003

15

Priority Queues A priority queue is an ADT for maintaining a

set S of elements, each with an associated value called key

We need PQ to support the following operations init(S:VertexSet) extractMin():Vertex modifyKey(v:Vertex)

To choose how to implement a PQ, we need to count how many times the operations are performed: init is performed just once and runs in O(n)

November 13, 2003

16

Prim-Jarnik’s Running Time

Time = |V|T(extractMin) + O(E)T(modifyKey) Binary heap implementation:

Time = O(V lgV + E lgV) = O(E lgV)

Q T(extractMin)

T(modifyKey) Total

array O(V) O(1) O(V 2)

binary heap

O(lg V) O(lg V) O(E lgV )

Fibonacci heap

O(lg V) O(1) amortized

O(V lgV +E )

November 13, 2003

17

Greedy Algorithms

One more algorithm design technique Greedy algorithms can be used to

solve optimization problems, if: There is an optimal substructure We can prove that a greedy choice at

each iteration leads to an optimal solution.

November 13, 2003

18

Kruskal's Algorithm

Edge based algorithm Add the edges one at a time, in

increasing weight order The algorithm maintains A – a forest

of trees. An edge is accepted it if connects vertices of distinct trees (the cut respects A)

November 13, 2003

19

Disjoint-Set ADT

We need a Disjoint-Set ADT that maintains a partition, i.e., a collection S of disjoint sets. Operations: makeSet(x:Vertex): SetID

S S {{x}} union(x:Vertex, y:Vertex) –

X S.findSet(x) Y S.findSet(y) S S – {X, Y} {X Y}

findSet(x:Vertex): SetID returns unique X S, where x X

November 13, 2003

20

Kruskal's Algorithm

The algorithm keeps adding the cheapest edge that connects two trees of the forestMST-Kruskal(G)01 A 02 S.init() // Init disjoint-set ADT03 for each vertex v G.V()04 S.makeSet(v)05 sort the edges of G.E() by non-decreasing G.w(u,v)06 for each edge (u,v) E, in sorted order do07 if S.findSet(u) S.findSet(v) then08 A A {(u,v)}09 S.union(u,v)10 return A

November 13, 2003

21

Kruskal’s Example

Let’s do an example:

H

B C

I

A

D

G

F E

Let’s keep track: What is set A, what is a collection S ,

what cuts are we making

1

68

4

12

8

3

6

5 3

4 13 9

10

November 13, 2003

22

Disjoint Sets as Lists

Each set – a list of elements identified by the first element, all elements in the list point to the first element

Union – add a smaller list to a larger one FindSet: O(1), Union(u,v): O(min{|C(u)|, |C(v)|})

1 2 3 A B C

4

1 2 3 A B C

4

November 13, 2003

23

Kruskal Running Time Initialization O(V) time Sorting the edges (E lg E) = (E lg V) (why?) O(E) calls to FindSet Union costs

Let t(v) be the number of times v is moved to a new cluster

Each time a vertex is moved to a new cluster the size of the cluster containing the vertex at least doubles: t(v) log V

Total time spent doing Union Total time: O(E lg V)

( ) logv V

t v V V

November 13, 2003

24

Next Lecture

Shortest Paths in Weighted Graphs