Chapter 7

Chapter 7: Priority Queues

A priority queue is an ADT for storing a collection of prioritized elements that supports arbitrary element insertion but supports removal of elements in order of priority; that is, the element with first priority can be removed at any time.
Examples:

Standby passenger for a fully booked flight - priority here is measured in terms of the fare paid, frequent-flyer status, and check-in time.
In any operating system, each process is assumed a priority and the scheduler will always choose a process with higher priority over one of lower priority.

The priority is typically a numerical value - the smallest numerical value should have first (highest) priority.

7.1.1 Keys, Priorities, and Total Order Relations
We define a key to be an object that is assigned to an element as a specific attribute for that element and that can be used to identify, rank, or weight that element.

Comparing Keys with Total Orders

Keys in a priority queue can be arbitrary objects on which an order is defined
Two distinct items in a priority queue can have the same key
Mathematical concept of total order relation ≤ (a rule for comparing keys)

Reflexive property: x ≤ x
Antisymmetric property: x ≤ y ∧ y ≤ x ⇒ x = y
Transitive property: x ≤ y ∧ y ≤ z ⇒ x ≤ z

Linear ordering relationship among a set of keys
Smallest key ≤ any key in our set

A priority queue stores a collection of items (cannot use positions!)
An item is a pair (k,e)
Main methods of the Priority Queue ADT

insertItem(k,e) inserts an item with key k and element e
removeMin() removes the item with the smallest key

Additional methods

minKey() returns, but does not remove, the smallest key of an item
minElement() returns, but does not remove, the element of an item with smallest key
size(), isEmpty()

Applications:

Standby flyer's

Example 7.1: Suppose a certain flight is fully booked an hour prior to departure. Because of possibility of cancellations, the airline maintains a priority queue of standby passengers hoping to get a seat. The priority of each passenger is determined by the fare paid, the frequent-flyer status, and the time when the passenger is inserted into the priority queue. When a passenger requested to fly standby, the associated passenger object is inserted into the priority queue with an insertItem operation. Shortly before the flight departure, if seats become available (for example, due to last-minute cancellations), the airline repeatedly removes, from the priority queue, a standby passenger with first priority, using a combination of minElement() and removeMin() operations, and lets this person board.

7.1.2 Sorting with a Priority Queue

We can use a priority queue to sort a set of comparable elements (total order in the set of elements)

Insert the elements one by one with a series of insertItem(e,e) operations
Remove the elements in sorted order with a series of removeMin() operations

The running time of this sorting method depends on the priority queue implementation

Algorithm PriorityQueueSort(S, P):
Input: A sequence S storing n elements, on which a total order relation is
    defined, and a priority queue P, that compares keys using the same total
    order relation
Output: The sequence S sorted by the total order relation
while !S.isEmpty() do
   e <- S.removeFirst()   {remove an element e from S}
   P.insertItem(e,e)         {the key is the element itself}
while !P.isEmpty() do
   e <- P.minElement()    {get a smallest element from P}
   P.removeMin()               {remove this element from P}
   S.insertLast(e)             {add the element at the end of S}

The running time of the algorithm is determined by the running times of operations insertItem(e,e), minElement(), and removeMin(), which do depend how P is implemented.
Example 7.2:

Operation	Output	Priority Queue
insertItem(5,A)	-	{(5,A)}
insertItem(9,C)	-	{(5,A),(9,C)}
insertItem(3,B)	-	{(3,B),(5,A),(9,C)}
insertItem(7,D)	-	{(3,B),(5,A),(7,D),(9,C)}
minElement()	B	{(3,B),(5,A),(7,D),(9,C)}
minKey()	3	{(3,B),(5,A),(7,D),(9,C)}
removeMin()	-	{(5,A),(7,D),(9,C)}
size()	3	{(5,A),(7,D),(9,C)}
minElement()	A	{(5,A),(7,D),(9,C)}
removeMin()	-	{(7,D),(9,C)}
removeMin()	-	{(9,C)}
removeMin()	-	{}
removeMin()	ERROR	{}
isEmpty()	true	{}

There are still two important issues that we have left undetermined to this point:

How do we keep track of the associations between elements and their keys?
How do we compare keys so as to determine a smallest key?

7.1.4 Compositions and Comparators

The Composition Pattern

The composition pattern defines a single object that is a composition of other objects.
A pair (k,e) is the simplest composition, for it composes two objects, k and e, into a single pair object.

html-7.2 (Item)

The Comparator Pattern

A comparator encapsulates the action of comparing two objects according to a given total order relation
When the priority queue needs to compare two keys, it uses its comparator
A generic priority queue uses a comparator as a template argument, to define the comparison function (<, =, >)
Example: Lexicographical less than operator:

bool operator<(const Point& p1, const Point& p2)
{ if (p1.getX() == p2.getX()) return p1.getY() < p2.getY();
  else return p1.getX() < p1.getX();
}

The comparator is external to the keys being compared. Thus, the same objects can be sorted in different ways by using different comparators.
Assume that comp is a comparison class object. comp(a,b) returns integer i, such that

i < 0 if a < b,
i = 0 if a = b, and
i > 0 if a > b.

We use special comparator object, external to the keys, to supply the comparison rules.
The approach that we will use is to overload "()" operator (to create so called function object).

Example: Compare two points in the plane lexicographically.

class LexCompare {
public:
   int operator()(Point a, Point b) {
   if (a.x < b.x) return –1
   else if (a.x > b.x) return +1

   else if (a.y < b.y) return –1
   else if (a.y > b.y) return +1
   else return 0;
   }
};

html-7.3 (Compare)

Using Comparator Objects

To use the comparator, define an object of this type, and invoke it using its “()” operator:
Example of usage:

Point p(2.3, 4.5);
Point q(1.7, 7.3);
LexCompare lexCompare;
if (lexCompare(p, q) < 0) cout << “p less than q”;
else if (lexCompare(p, q) == 0) cout << “p equals q”;
else if (lexCompare(p, q) > 0) cout << “p greater than q”;

html-7.4 (Generic)

7.2 Implementing a Priority Queue with a Sequence ADT

7.2.1 Implementation with an Unsorted Sequence

Add the new pair object p = (k,e) at the end of sequence S, by executing function insertLast(p) on S - takes O(1) time, independent of whether the sequence is implemented using an array or a linked list.
To perform operations minElement(), minKey() or removeMin() on P, we must inspect all the elements of sequence S to find an element p = (k,e) of S with minimal k - takes O(n) time.

7.2.2 Implementation with a Sorted Sequence

We can implement functions minElement()or minKey() on P simply by accessing the first element of the sequence with the first() function of S. We can implement removeMin() function of P as S.remove(S.first()). O(1)
The function insertItem(k,e) of P requires that we scan through the sequence S to find the appropriate position to insert the element and key. O(n)

html-7.5 (SSPQ1)
html-7.6 (SSPQ2)

[priority.cpp]

Comparing the Two Implementations

Implementation with an unsorted sequence Store the items of the priority queue in a list-based sequence, in arbitrary order	Implementation with a sorted sequence Store the items of the priority queue in a sequence, sorted by key
Performance: insertItem takes O(1) time since we can insert the item at the beginning or end of the sequence removeMin, minKey and minElement take O(n) time since we have to traverse the entire sequence to find the smallest key	Performance: insertItem takes O(n) time since we have to find the place where to insert the item removeMin, minKey and minElement take O(1) time since the smallest key is at the beginning of the sequence

7.2.3 Selection-Sort and Insertion-Sort

Selection-Sort

Selection-sort is the variation of PriorityQueueSort where the priority queue is implemented with an unsorted sequence
Running time of Selection-sort:

Inserting the elements into the priority queue with n insertItem operations takes O(n) time
Removing the elements in sorted order from the priority queue with n removeMin operations takes time proportional to 1 + 2 + …+ n

Selection-sort runs in O(n²) time

	Sequence S	Priority Queue P (unsorted sequence)
Input	(7,4,8,2,5,3,9)	()
Phase 1 O(n)	(4,8,2,5,3,9) (8,2,5,3,9) (2,5,3,9) (5,3,9) (3,9) (9) ()	(7) (7,4) (7,4,8) (7,4,8,2) (7,4,8,2,5) (7,4,8,2,5,3) (7,4,8,2,5,3,9)
Phase 2 O(n^₂)	(2) (2,3) (2,3,4) (2,3,4,5) (2,3,4,5,7) (2,3,4,5,7,8) (2,3,4,5,7,8,9)	(7,4,8,5,3,9) (7,4,8,5,9) (7,8,5,9) (7,8,9) (8,9) (9) ()

Insertion-Sort

Insertion-sort is the variation of PriorityQueueSort where the priority queue is implemented with a sorted sequence
Running time of Insertion-sort:

Inserting the elements into the priority queue with n insertItem operations takes time proportional to
1 + 2 + …+ n
Removing the elements in sorted order from the priority queue with a series of n removeMin operations takes O(n) time

Insertion-sort runs in O(n²) time

	Sequence S	Priority Queue P (sorted sequence)
Input	(7,4,8,2,5,3,9)	()
Phase 1 O(n^₂)	(4,8,2,5,3,9) (8,2,5,3,9) (2,5,3,9) (5,3,9) (3,9) (9) ()	(7) (4,7) (4,7,8) (2,4,7,8) (2,4,5,7,8) (2,3,4,5,7,8) (2,3,4,5,7,8,9)
Phase 2 O(n)	(2) (2,3) (2,3,4) (2,3,4,5) (2,3,4,5,7) (2,3,4,5,7,8) (2,3,4,5,7,8,9)	(3,4,5,7,8,9) (4,5,7,8,9) (5,7,8,9) (7,8,9) (8,9) (9) ()

7.3 Heaps

An efficient realization of a priority queue uses a (nonlinear) data structure called a heap − performs both insertions and removals in logarithmic time.

7.3.1 The Heap Data Structure

A heap is a binary tree storing keys at its internal nodes and satisfying the following properties:

Heap-Order: for every internal node v other than the root, key(v) ≥ key(parent(v))
Complete Binary Tree: let h be the height of the heap

for i = 0, … , h − 2, there are 2ⁱ nodes of depth i
at depth h − 1, the internal nodes are to the left of the external nodes

The last node of a heap is the rightmost internal node of depth h − 1

Proposition 7.5: A heap T storing n keys has height h = [log(n + 1)].
Proof: (we apply the complete binary tree property)

Let h be the height of a heap storing n keys
Since there are 2ⁱ keys at depth i = 0, … , h −2 and at least one key at depth h −1, we have n ≥ 1 + 2 + 4 +… + 2^{h −2} + 1 = 2^{h −1}, thus, n ≥ 2^{h − 1} , i.e., h ≤ log n + 1
Since for complete binary tree when there are 2^{h − 1} keys at depth h −1, we have n ≤ 1 + 2 + 4 +… + 2^{h −1} = 2^h−1, thus, n ≤ 2^h −1, i.e., h ≤ log (n + 1)

7.3.2 Implementing a Priority Queue with a Heap

We can use a heap to implement a priority queue
We store a (key, element) item at each internal node
We keep track of the position of the last node

The Vector Representation of a Heap

We can represent a heap with n keys by means of a vector of length n + 1
For the node at rank i

the left child is at rank 2i
the right child is at rank 2i +1

Links between nodes are not explicitly stored
The leaves are not represented
The cell of at rank 0 is not used
Operation insertItem corresponds to inserting at rank n + 1
Operation removeMin corresponds to removing at rank n
Yields in-place heap-sort

Insertion

Method insertItem of the priority queue ADT corresponds to the insertion of a key k to the heap
The insertion algorithm consists of three steps

Find the insertion node z (the new last node)
Store k at z and expand z into an internal node
Restore the heap-order property (discussed next)

Up-Heap Bubbling after an Insertion

After the insertion of a new key k, the heap-order property may be violated
Algorithm upheap restores the heap-order property by swapping k along an upward path from the insertion node
Upheap terminates when the key k reaches the root or a node whose parent has a key smaller than or equal to k
Since a heap has height O(log n), upheap runs in O(log n) time

Removal

Method removeMin of the priority queue ADT corresponds to the removal of the root key from the heap
The removal algorithm consists of three steps

Replace the root key with the key of the last node w
Compress w and its children into a leaf
Restore the heap-order property (discussed next)

Down-Heap Bubbling after a Removal

After replacing the root key with the key k of the last node, the heap-order property may be violated
Algorithm downheap restores the heap-order property by swapping key k along a downward path from the root
Downheap terminates when key k reaches a leaf or a node whose children have keys greater than or equal to k
Since a heap has height O(log n), downheap runs in O(log n) time

Analysis

Function	Time
size(), isEmpty()	O(1)
minElement(), minKey()	O(1)
insertItem(k,e)	O(log n)
removeMin()	O(log n)

7.3.3 C++ Implementation

html-7.7 (HeapTree)
html-7.8 (HPQ1)
html-7.9 (HPQ2)

7.3.4 Heap-Sort

Consider a priority queue with n items implemented by means of a heap

the space used is O(n)
methods insertItem and removeMin take O(log n) time
methods size, isEmpty, minKey, and minElement take O(1) time

Using a heap-based priority queue, we can sort a sequence of n elements in O(n log n) time
The resulting algorithm is called heap-sort
Heap-sort is much faster than quadratic sorting algorithms, such as insertion-sort and selection-sort