Princeton University • COS 423 • Theory of Algorithms • Spring 2002 • Kevin Wayne
Binary and Binomial Heaps
These lecture slides are adaptedfrom CLRS, Chapters 6, 19.
2
Priority Queues
Supports the following operations. Insert element x. Return min element. Return and delete minimum element. Decrease key of element x to k.
Applications. Dijkstra's shortest path algorithm. Prim's MST algorithm. Event-driven simulation. Huffman encoding. Heapsort. . . .
3
Priority Queues in Action
PQinit() for each v V key(v) PQinsert(v)
key(s) 0while (!PQisempty()) v = PQdelmin() for each w Q s.t (v,w) E if (w) > (v) + c(v,w) PQdecrease(w, (v) + c(v,w))
Dijkstra's Shortest Path Algorithm
4
Dijkstra/Prim 1 make-heap|V| insert|V| delete-min|E| decrease-key
Priority Queues
make-heap
Operation
insert
find-min
delete-min
union
decrease-key
delete
1
Binary
log N
1
log N
N
log N
log N
1
Binomial
log N
log N
log N
log N
log N
log N
1
Fibonacci *
1
1
log N
1
1
log N
1
Relaxed
1
1
log N
1
1
log N
1
Linked List
1
N
N
1
1
N
is-empty 1 1 1 11
Heaps
O(|E| + |V| log |V|)O(|E| log |V|)O(|V|2)
5
Binary Heap: Definition
Binary heap. Almost complete binary tree.
– filled on all levels, except last, where filled from left to right Min-heap ordered.
– every child greater than (or equal to) parent
06
14
78 18
81 7791
45
5347
6484 9983
6
Binary Heap: Properties
Properties. Min element is in root. Heap with N elements has height = log2 N.
06
14
78 18
81 7791
45
5347
6484 9983
N = 14Height = 3
7
Binary Heaps: Array Implementation
Implementing binary heaps. Use an array: no need for explicit parent or child pointers.
– Parent(i) = i/2 – Left(i) = 2i – Right(i) = 2i + 1
06
14
78 18
81 7791
45
5347
6484 9983
1
2 3
4 5 6 7
8 9 10 11 12 13 14
8
Binary Heap: Insertion
Insert element x into heap. Insert into next available slot. Bubble up until it's heap ordered.
– Peter principle: nodes rise to level of incompetence
06
14
78 18
81 7791
45
5347
6484 9983 42 next free slot
9
Binary Heap: Insertion
Insert element x into heap. Insert into next available slot. Bubble up until it's heap ordered.
– Peter principle: nodes rise to level of incompetence
06
14
78 18
81 7791
45
5347
6484 9983 4242
swap with parent
10
Binary Heap: Insertion
Insert element x into heap. Insert into next available slot. Bubble up until it's heap ordered.
– Peter principle: nodes rise to level of incompetence
06
14
78 18
81 7791
45
4247
6484 9983 4253
swap with parent
11
Binary Heap: Insertion
Insert element x into heap. Insert into next available slot. Bubble up until it's heap ordered.
– Peter principle: nodes rise to level of incompetence O(log N) operations.
06
14
78 18
81 7791
42
4547
6484 9983 53
stop: heap ordered
12
Binary Heap: Decrease Key
Decrease key of element x to k. Bubble up until it's heap ordered. O(log N) operations.
06
14
78 18
81 7791
42
4547
6484 9983 53
13
Binary Heap: Delete Min
Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered.
– power struggle principle: better subordinate is promoted
06
14
78 18
81 7791
42
4547
6484 9983 53
14
Binary Heap: Delete Min
Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered.
– power struggle principle: better subordinate is promoted
53
14
78 18
81 7791
42
4547
6484 9983 06
15
Binary Heap: Delete Min
Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered.
– power struggle principle: better subordinate is promoted
53
14
78 18
81 7791
42
4547
6484 9983
exchange with left child
16
Binary Heap: Delete Min
Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered.
– power struggle principle: better subordinate is promoted
14
53
78 18
81 7791
42
4547
6484 9983
exchange with right child
17
Binary Heap: Delete Min
Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered.
– power struggle principle: better subordinate is promoted O(log N) operations.
14
18
78 53
81 7791
42
4547
6484 9983
stop: heap ordered
18
Binary Heap: Heapsort
Heapsort. Insert N items into binary heap. Perform N delete-min operations. O(N log N) sort. No extra storage.
19
H1H2
14
78 18
81 7791
11
6253
6484 9941
Binary Heap: Union
Union. Combine two binary heaps H1 and H2 into a single heap.
No easy solution. (N) operations apparently required
Can support fast union with fancier heaps.
20
Priority Queues
make-heap
Operation
insert
find-min
delete-min
union
decrease-key
delete
1
Binary
log N
1
log N
N
log N
log N
1
Binomial
log N
log N
log N
log N
log N
log N
1
Fibonacci *
1
1
log N
1
1
log N
1
Relaxed
1
1
log N
1
1
log N
1
Linked List
1
N
N
1
1
N
is-empty 1 1 1 11
Heaps
21
Binomial Tree
Binomial tree. Recursive definition:
Bk-1
Bk-1
B0 Bk
B0 B1 B2 B3 B4
22
Binomial Tree
Useful properties of order k binomial tree Bk.
Number of nodes = 2k. Height = k. Degree of root = k. Deleting root yields binomial
trees Bk-1, … , B0.
Proof. By induction on k.
B0 B1 B2 B3 B4
B1
Bk
Bk+1
B2B0
23
Binomial Tree
A property useful for naming the data structure. Bk has nodes at depth i.
B4
i
k
62
4
depth 2
depth 3
depth 4
depth 0
depth 1
24
Binomial Heap
Binomial heap. Vuillemin, 1978. Sequence of binomial trees that satisfy binomial heap property.
– each tree is min-heap ordered– 0 or 1 binomial tree of order k
B4 B0B1
55
45 32
30
24
23 22
50
48 31 17
448 29 10
6
37
3 18
25
Binomial Heap: Implementation
Implementation. Represent trees using left-child, right sibling pointers.
– three links per node (parent, left, right) Roots of trees connected with singly linked list.
– degrees of trees strictly decreasing from left to right
50
48 31 17
4410
6
37
3 18
29
6
37
3 18
48
3150
10
4417
heap
29
Leftist Power-of-2 HeapBinomial Heap
Paren
t
Left
Right
26
Binomial Heap: Properties
Properties of N-node binomial heap. Min key contained in root of B0, B1, . . . , Bk.
Contains binomial tree Bi iff bi = 1 where bn b2b1b0 is binary
representation of N. At most log2 N + 1 binomial trees.
Height log2 N.
B4 B0B1
55
45 32
30
24
23 22
50
48 31 17
448 29 10
6
37
3 18
N = 19# trees = 3height = 4binary = 10011
27
Binomial Heap: Union
Create heap H that is union of heaps H' and H''. "Mergeable heaps." Easy if H' and H'' are each order k binomial trees.
– connect roots of H' and H''– choose smaller key to be root of H
H''55
45 32
30
24
23 22
50
48 31 17
448 29 10
6
H'
28
Binomial Heap: Union
001 1
100 1+
011 1
11
1
1
0
1
19 + 7 = 26
55
45 32
30
24
23 22
50
48 31 17
448 29 10
6
37
3 18
41
3328
15
25
7 12
+
29
Binomial Heap: Union
55
45 32
30
24
23 22
50
48 31 17
448 29 10
6
37
3 18
41
3328
15
25
7 12
+
30
Binomial Heap: Union
55
45 32
30
24
23 22
50
48 31 17
448 29 10
6
37
3
41
3328
15
25
7
+
12
18
18
12
31
55
45 32
30
24
23 22
50
48 31 17
448 29 10
6
37
3
41
3328
15
25
7
+
12
18
25
377
3
18
12
18
12
32
55
45 32
30
24
23 22
50
48 31 17
448 29 10
6
37
3
41
3328
15
25
7
12
+
18
25
377
3
41
28 33 25
3715 7
3
18
12
18
12
33
55
45 32
30
24
23 22
50
48 31 17
448 29 10
6
37
3
41
3328
15
25
7
+
18
12
41
28 33 25
3715 7
3
12
18
25
377
3
41
28 33 25
3715 7
3
18
12
34
55
45 32
30
24
23 22
50
48 31 17
448 29 10
6
37
3
41
3328
15
25
7
+
18
12
41
28 33 25
3715 7
3
12
18
25
377
3
41
28 33 25
3715 7
3
55
45 32
30
24
23 22
50
48 31 17
448 29 10
6
18
12
35
Binomial Heap: Union
Create heap H that is union of heaps H' and H''. Analogous to binary addition.
Running time. O(log N) Proportional to number of trees in root lists 2( log2 N + 1).
001 1
100 1+
011 1
11
1
1
0
1
19 + 7 = 26
36
3
37
6 18
55
45 32
30
24
23 22
50
48 31 17
448 29 10
H
Binomial Heap: Delete Min
Delete node with minimum key in binomial heap H. Find root x with min key in root list of H, and delete H' broken binomial trees H Union(H', H)
Running time. O(log N)
37
Binomial Heap: Delete Min
Delete node with minimum key in binomial heap H. Find root x with min key in root list of H, and delete H' broken binomial trees H Union(H', H)
Running time. O(log N)
55
45 32
30
24
23 22
50
48 31 17
37
6 18
448 29 10
H
H'
38
3
37
6 18
55
x 32
30
24
23 22
50
48 31 17
448 29 10
H
Binomial Heap: Decrease Key
Decrease key of node x in binomial heap H. Suppose x is in binomial tree Bk.
Bubble node x up the tree if x is too small.
Running time. O(log N) Proportional to depth of node x log2 N .
depth = 3
39
Binomial Heap: Delete
Delete node x in binomial heap H. Decrease key of x to -. Delete min.
Running time. O(log N)
40
Binomial Heap: Insert
Insert a new node x into binomial heap H. H' MakeHeap(x) H Union(H', H)
Running time. O(log N)
3
37
6 18
55
45 32
30
24
23 22
50
48 31 17
448 29 10
H
x
H'
41
Binomial Heap: Sequence of Inserts
Insert a new node x into binomial heap H. If N = .......0, then only 1 steps. If N = ......01, then only 2 steps. If N = .....011, then only 3 steps. If N = ....0111, then only 4 steps.
Inserting 1 item can take (log N) time. If N = 11...111, then log2 N steps.
But, inserting sequence of N items takes O(N) time! (N/2)(1) + (N/4)(2) + (N/8)(3) + . . . 2N Amortized analysis. Basis for getting most operations
down to constant time.
50
48 31 17
4429 10
3
37
6 x
221
22
2 11
NN
N
nn
Nn
42
Priority Queues
make-heap
Operation
insert
find-min
delete-min
union
decrease-key
delete
1
Binary
log N
1
log N
N
log N
log N
1
Binomial
log N
log N
log N
log N
log N
log N
1
Fibonacci *
1
1
log N
1
1
log N
1
Relaxed
1
1
log N
1
1
log N
1
Linked List
1
N
N
1
1
N
is-empty 1 1 1 11
Heaps
just did this
