Chapter 10

Chapter 10: Sorting, Sets, and Selection

The sorting problem is defined as follows: Let S be a sequence of n elements that can be compared to each other according to a total order relation. We want to rearrange S in such a way that the elements appear in increasing order.
The PriorityQueueSort algorithms:

sequence implementation: insertion-sort and selection-sort O(n²)
heap implementation: heap-sort O(n log n)

10.1 Merge-Sort

10.1.1 Divide-and-Conquer
MergeSort.pdf 3

Divide-and-conquer is a general algorithm design paradigm:

Divide: divide the input data S in two disjoint subsets S₁ and S₂
Recur: solve the subproblems associated with S₁ and S₂
Conquer: combine the solutions for S₁ and S₂ into a solution for S

The base case for the recursion are subproblems of size 0 or 1
Merge-sort is a sorting algorithm based on the divide-and-conquer paradigm
Like heap-sort

It uses a comparator
It has O(n log n) running time

Unlike heap-sort

It does not use an auxiliary priority queue
It accesses data in a sequential manner (suitable to sort data on a disk)

Using Divide-and-Conquer for Sorting
MergeSort.pdf 4

Merge-sort on an input sequence S with n elements consists of three steps:

Divide: partition S into two sequences S₁ and S₂ of about n/2 elements each
Recur: recursively sort S₁ and S₂
Conquer: merge S₁ and S₂ into a unique sorted sequence

MergeSort.pdf 7 (applet)
Merging two sorted sequences, each with n/2 elements and implemented by means of a doubly linked list, takes O(n) time
The running time of merge-sort is O(n log n)

10.1.2 A C++ Implementation of Merge-Sort
html-10.2 (MergeSort)

mergesort.cpp

10.2 The Set ADT

A set is a container of distinct objects (no duplicate elements).
Set operations - union, intersection and subtraction:

union(A,B) - return a set C, which is the union of A and B
intersect(A,B) - return a set C, which is the intersection of A and B
subtract(A,B) - return a set C, which is the difference of A and B

10.2.1 A Simple Set Implementation
Sets.pdf 3
Using a Sorted Sequence to Implement a Set

We represent a set by the sorted sequence of its elements.
In general, it should always be possible to impose an order on the elements.
The generic merge function iteratively examines and compares the current elements a and b of the sequences A and B, respectively, and finds out whether a < b, a = b or a > b.
Based on the outcome, it determines whether it should copy one or none of elements a and b to the end of the output sequence C - the result of set operations.
The total running time of genericMerge(A,B) is O(n_A + n_B), where n_A is the size of A and n_B is the size of B.

Generic Merging as a Template Method Pattern
html-10.4a (Merger1)
html-10.4b (Merger2)

Using Inheritance to Derive Set Operations
html-10.5a (UnionMerger)
html-10.5b (IntersectMerger)
html-10.5c (SubtractMerger)

10.3 Quick-Sort

High-Level Description of Quick-Sort
QuickSort.pdf 3-12

Quick-sort is a randomized sorting algorithm based on the divide-and-conquer paradigm:

Divide: pick a random element x (called pivot) and partition S into
L elements less than x
E elements equal x
G elements greater than x
Recur: sort L and G
Conquer: join L, E and G

We partition an input sequence as follows:

We remove, in turn, each element y from S and
We insert y into L, E or G, depending on the result of the comparison with the pivot x

Each insertion and removal is at the beginning or at the end of a sequence, and hence takes O(1) time
Thus, the partition step of quick-sort takes O(n) time
QuickSort.pdf 6 (applet)

Algorithm partition(S, p)
Input sequence S, position p of pivot
Output subsequences L, E, G of the elements of S less than, equal to, or greater than the pivot, resp.
L, E, G ← empty sequences
x ← S.remove(p)
while ¬S.isEmpty()
     y ← S.remove(S.first())
     if y < x
          L.insertLast(y)
     else if y = x
          E.insertLast(y)
     else { y > x }
         G.insertLast(y)
return L, E, G

10.3.1 In-Place Quick-Sort

Quick-sort can be implemented to run in-place
In the partition step, we use replace operations to rearrange the elements of the input sequence such that

the elements less than the pivot have rank less than h
the elements equal to the pivot have rank between h and k
the elements greater than the pivot have rank greater than k

The recursive calls consider

elements with rank less than h
elements with rank greater than k

Perform the partition using two indices to split S into L and (E union G).

Repeat until j and k cross:

Scan j to the right until finding an element > pivot.
Scan k to the left until finding an element < pivot.
Swap elements at indices j and k

Algorithm inPlaceQuickSort(S, l, r)
   Input: sequence S, ranks l and r
   Output: sequence S with the elements of rank between l and r rearranged in increasing order
   if l ≥ r
       return
   i ← a random integer between l and r
   x ← S.elemAtRank(i)
   (h, k) ← inPlacePartition(x)
   inPlaceQuickSort(S, l, h − 1)
   inPlaceQuickSort(S, k + 1, r)

html-10.7 (QuickSort)

qsort.cpp

Running Time of Quick-Sort
QuickSort.pdf 13-15

The worst case for quick-sort occurs when the pivot is the unique minimum or maximum element
One of L and G has size n − 1 and the other has size 0
The running time is proportional to the sum: n + (n − 1) + …+ 2 + 1
Thus, the worst-case running time of quick-sort is O(n²)

The best case for quick-sort on a sequence of distinct elements occur when subsequences L and G happen to have roughly the same size.
The tree T has height O(log n) and quick-sort runs in O(n log n) time.

10.3.2 Randomized Quick-Sort

Since the goal of the partition step of the quick-sort function is to divide the sequence S almost equally, let us pick a random element of the input sequence as the pivot.
The expected running time of randomized quick-sort on a sequence with n elements is O(n log n).
Consider a recursive call of quick-sort on a sequence of size n

Good call: the sizes of L and G are each less than 3n/4
Bad call: one of L and G has size greater than 3n/4

A call is good with probability 1/2 (1/2 of the possible pivots cause good calls):

10.4 A Lower Bound on Comparison-Based Sorting

SortingLowerBound.pdf 1-5

10.5 Bucket-Sort and Radix-Sort

10.5.1 Bucket-Sort
RadixSort.pdf 2

Stable Sorting

10.5.2 Radix-Sort
RadixSort.pdf 7

10.6 Comparison of Sorting Algorithms

QuickSort.pdf 18

Algorithm	Time	Notes
insertion-sort selection-sort bubble-sort	O(n²)	in-place slow (good for small inputs)
quick-sort	O(n²) worst case and O(n log n) in average	in-place, randomized fastest (good for large inputs)
heap-sort	O(n log n)	in-place fast (good for large inputs)
merge-sort	O(n log n)	sequential data access fast (good for huge inputs)
bucket-sort radix-sort	O(n)	the keys are integers in the range [0, N −1] fast (good for huge inputs)