Priority Queues, Heaps & Leftist Trees

CSE, POSTECH

2 2

Priority Queues

A priority queue is a collection of zero or more elements each element has a priority or value

Unlike the FIFO queues, the order of deletion from a priority queue (e.g., who gets served next) is determined by the element priority

Elements are deleted by increasing or decreasing order of priority rather than by the order in which they arrived in the queue

3 3

Priority Queues

Operations performed on priority queues1) Find an element, 2) insert a new element, 3) delete an

element, etc.

Two kinds of (Min, Max) priority queues exist In a Min priority queue, find/delete operation

finds/deletes the element with minimum priority In a Max priority queue, find/delete operation

finds/deletes the element with maximum priority Two or more elements can have the same priority

4 4

Priority Queues

See ADT 12.1 & Program 12.1 for max priority queue specification

What would be different for min priority queue specification?

Read Examples 12.1, 12.2 What are other examples in our daily lives that

utilize the priority queue concept?

5 5

Implementation of Priority Queues

Implemented using heaps and leftist trees

Heap is a complete binary tree that is efficiently stored using the array-based representation

Leftist tree is a linked data structure suitable for the implementation of a priority queue

6 6

Max (Min) Tree

A max tree (min tree) is a tree in which the value in each node is greater (less) than or equal to those in its children (if any)– See Figure 12.1, 12.2 for examples– Nodes of a max or min tree may have more than two

children (i.e., may not be binary tree)

7 7

Max Tree Example

8 8

Min Tree Example

9

A max heap (min heap) is a max (min) tree that is also a complete binary tree– Figure 12.1 (a) & (c) are max heap– Figure 12.2 (a) & (c) are min heap– Why aren’t Figure 12.1 (b) & 12.2 (b) max/min heap?– How can you change the Figure 12.1 (b) & 12.2 (b) so

that they are max/min heap?

9

Heaps - Definitions

30

25

14

12 7

10 8 6

9

6

5

10 10

Max Heap with 9 Nodes

11 11

Min Heap with 9 Nodes

12 12

Array Representation of Heap

A heap is efficiently represented as an array.

13 13

Heap Operations

When n is the number of elements (heap size), Insertion O(log2n)

Deletion O(log2n) Initialization O(n)

14 14

Insertion into a Max Heap

9

8

6

7

7 2 6

5 1 5• New element is 5• Are we finished?

15 15

9

8

6

7

7 2 6

5 1 20

• New element is 20• Are we finished?

Insertion into a Max Heap

16 16

9

8

6

7

2 6

5 1 7

20

• Exchange the positions with 7• Are we finished?

Insertion into a Max Heap

17 17

9

6

7

2 6

5 1 7

8

20

Insertion into a Max Heap

• Exchange the positions with 8• Are we finished?

18 18

6

7

2 6

5 1 7

8

9

20

Insertion into a Max Heap

• Exchange the positions with 9• Are we finished?

19 19

Complexity of Insertion

See also Figure 12.3 for another insertion example

At each level, we do (1) work Thus the time complexity is O(height) = O(log2n),

where n is the heap size

20 20

Deletion from a Max Heap

• Max element is in the root• What happens when we delete an element?

20

6

7

2 6

5 1 7

15

8

9

21 21

Deletion from a Max Heap

• After the max element is removed.• Are we finished?

6

7

2 6

5 1 7

15

8

9

22 22

Deletion from a Max Heap

• Heap with 10 nodes. • Reinsert 8 into the heap.

6

7

2 6

5 1 7

15

8

9

23 23

Deletion from a Max Heap

• Reinsert 8 into the heap.• Are we finished?

6

7

2 6

5 1 7

15

9

8

24 24

Deletion from a Max Heap

• Exchange the position with 15• Are we finished?

6

7

2 6

5 1 7

9

15

8

25 25

Deletion from a Max Heap

6

7

2 6

5 1 7

8

15

9

• Exchange the position with 9• Are we finished?

26 26

Complexity of Deletion

See also Figure 12.4 for another deletion example

The time complexity of deletion is the same as insertion

At each level, we do (1) work Thus the time complexity is O(height) = O(log2n),

where n is the heap size

27 27

Max Heap Initialization

• Heap initialization means to construct a heap by adjusting the tree if necessary• Example: input array = [-,1,2,3,4,5,6,7,8,9,10,11]

28 28

Max Heap Initialization

- Start at rightmost array position that has a child.- Index is floor(n/2).

29 29

Max Heap Initialization

30 30

Max Heap Initialization

31 31

Max Heap Initialization

32 32

Max Heap Initialization

33 33

Max Heap Initialization

•Are we finished?•Done!

34 34

Complexity of Initialization

• See Figure 12.5 for another initialization example

• Height of heap = h.• Number of nodes at level j is <= 2j-1.• Time for each node at level j is O(h-j+1).• Time for all nodes at level j is <= 2j-1(h-j+1) = t(j).• Total time is t(1) + t(2) + … + t(h) = O(2h) = O(n).

35 35

The Class MaxHeap

See Program 12.2 for Insertion into a MaxHeap See Program 12.3 for Deletion from a MaxHeap See Program 12.4 for Initializing a nonempty

MaxHeap

36

PUSH OPERATIONtemplate<class T>void maxHeap<T>::push(const T& theElement){// Add theElement to heap.

// increase array length if necessaryif (heapSize == arrayLength - 1) {// double array lengthchangeLength1D(heap, arrayLength, 2 * arrayLength); arrayLength *= 2; }

// find place for theElement // currentNode starts at new leaf and moves up tree int currentNode = ++heapSize; while (currentNode != 1 && heap[currentNode / 2] < theElement) { // cannot put theElement in heap[currentNode] heap[currentNode] = heap[currentNode / 2]; // move element down currentNode /= 2; // move to parent }

heap[currentNode] = theElement;}

36

37

POP OPERATIONtemplate<class T>void maxHeap<T>::pop(){// Remove max element. // if heap is empty return null if (heapSize == 0) // heap empty throw queueEmpty();

// Delete max element heap[1].~T();

// Remove last element and reheapify T lastElement = heap[heapSize--];

// find place for lastElement starting at root int currentNode = 1, child = 2; // child of currentNode while (child <= heapSize) { // heap[child] should be larger child of currentNode if (child < heapSize && heap[child] < heap[child + 1]) child++;

// can we put lastElement in heap[currentNode]? if (lastElement >= heap[child]) break; // yes

// no heap[currentNode] = heap[child]; // move child up currentNode = child; // move down a level child *= 2; } heap[currentNode] = lastElement;}3

7

38

INITIALIZEtemplate<class T>void maxHeap<T>::initialize(T *theHeap, int theSize){// Initialize max heap to element array theHeap[1:theSize]. delete [] heap; heap = theHeap; heapSize = theSize;

// heapify for (int root = heapSize / 2; root >= 1; root--) { T rootElement = heap[root];

// find place to put rootElement int child = 2 * root; // parent of child is target // location for rootElement while (child <= heapSize) { // heap[child] should be larger sibling if (child < heapSize && heap[child] < heap[child + 1]) child++;

// can we put rootElement in heap[child/2]? if (rootElement >= heap[child]) break; // yes

// no heap[child / 2] = heap[child]; // move child up child *= 2; // move down a level } heap[child / 2] = rootElement; }}

38

39 39

Exercise 12.7

Do Exercise 12.7 – theHeap = [-, 10, 2, 7, 6, 5, 9, 12, 35, 22, 15, 1, 3, 4]

40 40

Exercise 12.7 (a)

12.7 (a) – complete binary tree

41 41

Exercise 12.7 (b)

12.7 (b) – The heapified tree

42 42

Exercise 12.7 (c)

12.7 (c) – The heap after 15 is inserted is:

43 43

Exercise 12.7 (c)

12.7 (c) – The heap after 20 is inserted is:

44 44

Exercise 12.7 (c)

12.7 (c) – The heap after 45 is inserted is:

45 45

Exercise 12.7 (d) 12.7 (d) – The heap after the first remove max operation is:

46 46

Exercise 12.7 (d) 12.7 (d) – The heap after the second remove max operation is:

47 47

Exercise 12.7 (d) 12.7 (d) – The heap after the third remove max operation is:

48 48

Leftist Trees

Despite heap structure being both space and time efficient, it is NOT suitable for all applications of priority queues

Leftist tree structures are useful for applications– to meld (i.e., combine) pairs of priority queues– using multiple queues of varying size

Leftist tree is a linked data structure suitable for the implementation of a priority queue

A tree which tends to “lean” to the left.

49 49

Leftist Trees

External node – a special node that replaces each empty subtree

Internal node – a node with non-empty subtrees Extended binary tree – a binary tree with external

nodes added (see Figure 12.6)

50 50

Extended Binary Tree

Figure 12.6 s and w values

51 51

Height-Biased Leftist Tree (HBLT) Let s(x) be the length (height) of a shortest path from

node x to an external node in its subtree If x is an external node, s(x) = 0 If x is an internal node, s(x) = min {s(L), s(R)} + 1, where L

and R are left and right children of x A binary tree is a height-biased leftist tree (HBLT) iff at

every internal node, the s value of the left child is greater than or equal to the s value of the right child

Is Figure 12.6(a) an HBLT? If not, how can we change it to become an HBLT?

52 52

Max/Min HBLT

A max HBLT is an HBLT that is also a max tree– Are the trees of Figure 12.1 are also max HBLTs?– YES!

A min HBLT is an HBLT that is also a min tree– Are the trees of Figure 12.2 are also min HBLTs?– YES!

53 53

Weight-Biased Leftist Tree (WBLT)

Let the weight, w(x), of node x to be the number of internal nodes in the subtree with root x

If x is an external node, w(x) = 0 If x is an internal node, its weight is one more than the sum

of the weights of its children A binary tree is a weight-biased leftist tree (WBLT) iff at

every internal node, the w value of the left child is greater than or equal to the w value of the right child

54 54

Weight-Biased Leftist Tree (WBLT)

A max (min) WBLT is a max (min) tree that is also a WBLT

Is Figure 12.6(a) an WBLT? If not, how can we change it to become an WBLT?

55 55

Operations on a Max HBLT

Read Section 12.5.2 for Insertion into a Max HBLT Read Section 12.5.3 for Deletion from a Max HBLT Read Section 12.5.4 for Melding Two Max HBLTs See Figure 12.7 and read Example 12.3 for Melding max

HBLTs Read Section 12.5.5 for Initialization of a Max HBLT See Figure 12.8 for Initializing a max HBLT

56 56

Melding max HBLTs

Figure 12.7 Melding (combining) max HBLTs

57 57

The Class maxHBLT

See Program 12.5 for Melding of two leftist trees See Program 12.6 for meld, push and pop

methods See Program 12.7 for Initializing a max HBLT

Do Exercise 12.19

58 58

Exercise 12.19

(a) The first six calls to meld create the following six max leftist trees.

5

3

7

6

20

8

9

2

15

12

30

17

The next three calls to meld combine pairs of these trees to create the following three trees:

5

3

7

6 9

2

20

8 15

12

30

17

What would be next?

59 59

Applications of Heaps

Sort (heap sort) Machine scheduling Huffman codes

60 60

Heap Sort

use element key as priority

Algorithm put elements to be sorted into a priority queue

(i.e., initialize a heap) extract (delete) elements from the priority queue

– if a min priority queue is used, elements are extracted in non-decreasing order of priority

– if a max priority queue is used, elements are extracted in non-increasing order of priority

61 61

Heap Sort Example

After putting into a max priority queue

62 62

Sorting Example

After first remove max operation

63 63

Sorting Example

After second remove max operation

64 64

Sorting Example

After third remove max operation

65 65

Sorting Example

After fourth remove max operation

66 66

Sorting Example

After fifth remove max operation

67 67

Complexity Analysis of Heap Sort

See Program 12.8 for Heap Sort See Figure 12.9 for another Heap Sort example

Heap sort n elements.– Initialization operation takes O(n) time– Each deletion operation takes O(log n) time– Thus, the total time is O(n log n) - Why?

The heap has to be reinitialized (melded) after each delete operation

– compare with O(n2) for sort methods of Chapter 2

68 68

Machine Scheduling Problem

m identical machines n jobs to be performed The machine scheduling problem is to assign jobs

to machines so that the time at which the last job completes is minimum

69 69

Machine Scheduling Example

3 machines and 7 jobs job times are [6,2,3,5,10,7,14] What are some possible schedules? A possible schedule:

What algorithm did we use for the above scheduling? What are other scheduling algorithms

70 70

Machine Scheduling Example

What is the finish time (length) of the schedule?

21 Objective: Find schedules with minimum finish time Minimum finish time scheduling is NP-hard.

71 71

The class of problems for which no one has developed a polynomial time algorithm.

No algorithm whose complexity is O(nk ml) is known for any NP-hard problem (for any constants k and l)

NP stands for Nondeterministic Polynomial NP-hard problems are often solved by heuristics (or

approximation algorithms), which do not guarantee optimal solutions

Longest Processing Time (LPT) rule is a good heuristic for minimum finish time scheduling.

NP-hard Problems

72 72

LPT Schedule & Example

Longest Processing Time (LPT) first Jobs are scheduled in the descending order

14, 10, 7, 6, 5, 3, 2 Each job is scheduled on the machine

on which it finishes earliest

finish time is 16!

73 73

LPT Schedule & Example

What is the minimum finish time with thee machines for jobs (2, 14, 4, 16, 6, 5, 3)?

See Figure 12.10

74 74

LPT using a Min Heap

Min Heap has the finish times of the m machines. Initial finish times are all 0. To schedule a job, remove the machine with

minimum finish time from the heap. Update the finish time of the selected machine and

put the machine back into the min heap. See Program 12.9 for LPT scheduler

75 75

Complexity Analysis of LPT

When n m (i.e., more machines than jobs), LPT takes (1) time

When n m, (i.e., more jobs than machines), the heap sort takes O(n log n) time

Heap initialization takes O(m) time DeleteMin operation takes O(log m) time Insert operation takes O(log m) time n DeleteMin and n Insert takes O(n log m) time Thus, the total time is O(n log n + n log m) = O(n log n)

time (as n > m)

76 76

Huffman Codes For text compression, the LZW method relies on the

recurrence of substrings in a text Huffman codes is another text compression method,

which relies on the relative frequency (i.e., the number of occurrences of a symbol) with which different symbols appear in a text

Uses extended binary trees Variable-length codes that satisfy the property, where no

code is a prefix of another Huffman tree is a binary tree with minimum weighted

external path length for a given set of frequencies (weights)

77 77

Huffman Codes

READ Section 12.6.3

READ all of Chapter 12