Lecture 4: Interesting sorting algorithms

Shellsort is a refined insertion sort that attempts to improve upon the slowness of regular insertion sort, which spends considerable time shifting elements by one place.
It was invented by Donald Shell.
The core idea involves choosing a stride length (a number) to create subsequences within the array.
For example, with a stride length of 7, the subsequence a[0], a[7], a[14], and so on, would be insertion sorted.
This is repeated for a[1], a[8], a[15], and so on, and then for a[2], a[9], a[16], and so forth.
Sorting with a stride length of 7 is referred to as 7-sorting.
After 7-sorting, the process is repeated with a shorter stride length, like 3, eventually ending with 1-sorting, which is equivalent to regular insertion sort.

Starting with a stride length of 3:
- Original array: 3 6 2 1 5 4
- 3-sorted array: 1 5 2 3 6 4
Moving to stride 1 (regular insertion sort):
- 1 5 2 3 6 4
- 1 2 5 3 6 4
- 1 2 3 5 6 4
- 1 2 3 5 4 6
- 1 2 3 4 5 6

When sorting with a long stride length, the subarrays being sorted are small, making sorting faster.
When sorting with a short stride, the array is already almost sorted, which makes insertion sort faster.
If an array is $p$ -sorted and then $q$ -sorted, the array remains $p$ -sorted. This means each pass increases the sortedness of the array.

The complexity of Shellsort depends on the sequence of stride lengths used, known as the gap sequence.
Shell's original gap sequence used the numbers $⌊\frac{n}{2^k}⌋$ , halving every step, resulting in a worst-case complexity of $O(n^2)$ .
Many gap sequences give a worst-case complexity of $O(n^2)$ . For example, using numbers of the form $2^k − 1$ (that is: 1, 3, 7, 31, …) can achieve this.

Shellsort's correctness is based on the fact that the only operations performed are swaps, ensuring the array is always a permutation of the original.
The final pass, which is a regular insertion sort, guarantees that the array is fully sorted.

Fast algorithms can be obtained by a divide and conquer strategy: split the input data into pieces, work on those, then combine the results somehow.

Mergesort is a divide-and-conquer sorting algorithm.
It works by recursively splitting the input array into two halves, sorting the two halves using mergesort, and then merging the two sorted halves together while preserving the order.

Merging two arrays of total length $n$ takes $n$ steps.
The process involves comparing the first unmerged element in each input array, adding the smaller of the two to the output array, and repeating until all elements have been added to the output.

Given an array of length $n$ (where $n$ is a power of 2):
- If $n ≤ 1$ , stop.
- Otherwise, mergesort elements $0$ to $\frac{n}{2} - 1$ , and mergesort elements $\frac{n}{2}$ to $n − 1$ .
- Merge the two sorted halves.

If splitting the input array into two equal halves, the time complexity of mergesort is described by the recurrence:
- $T(1) = 0$
- $T(n) = 2T(\frac{n}{2}) + n$
Unrolling the second line once:
- $T(n) = n + (2 × T(\frac{n}{4}))$
- $T(n) = n + n + (4T(\frac{n}{4}))$
The recurrence stops when we reach 1, after $log<em>2 n$ unrollings, so $T(n) = n \log</em>2 n$ . This is better than $n^2$ or $n^{\frac{3}{2}}$ .

Consider a computer working at GHz rates (10^9 instructions per second), with 10^8 comparisons per second.
To sort 10^9 elements:
- An $n^2$ algorithm needs $10^{18}$ operations; $10^{10}$ seconds = 316 years.
- An $n \log n$ algorithm needs $3 × 10^{10}$ operations: 300 seconds = 5 minutes.

Algorithms covered: selection sort $Θ(n^2)$ , insertion sort $Θ(n^2)$ , Shellsort $O(n^{\frac{3}{2}})$ , and mergesort $Θ(n \log n)$ .
Some algorithms' run times depend on the input data; consider best-case, worst-case, and average-case complexity.
Reasoning about loops using invariants is an important theoretical idea.