3.6.9 · D5Sorting & Searching
Question bank — Lower bound for comparison sorts — Ω(n log n) proof via decision trees
Parent topic: Lower bound for comparison sorts.
True or false — justify
The bound applies to every sorting algorithm ever written.
False. It binds only comparison sorts. Counting Sort and Radix Sort read the key values (they index arrays with them), so the decision-tree model never applies and they run in .
A decision tree must have exactly leaves.
False. It needs at least leaves (one reachable leaf — a red square in the figure — per output permutation). A clumsy algorithm can have far more leaves — the same permutation can label many leaves reached by different comparison paths.
If a binary tree has height then it has exactly leaves.
False. It has at most leaves. Only a perfectly balanced full tree (like the fully-drawn bottom row in the figure) hits equality; the proof only ever needs the upper bound leaves .
The bound says every input of size costs comparisons.
False. It bounds the tree's height (the red arrow in the figure) = the worst-case path. A best-case input (e.g. already-sorted array for insertion sort) can finish in ; the claim is only that some input forces .
Merge sort being proves the lower bound.
False. An algorithm's running time is an upper bound; it can never prove that no faster algorithm exists. The lower bound is a separate argument about all possible comparison trees. Together they pin the complexity at .
Because , we could replace " leaves" by " leaves" and get the same result.
False. and differ by a factor of — a colossal gap. Confusing "locate one item" ( outcomes) with "determine the whole order" ( outcomes) is the single most common error.
The theorem is about average-case comparisons.
False (as stated). The tree-height argument is a worst-case bound. Average-case is also , but that needs a separate argument about the average leaf depth, not the maximum.
Using in "leaves " and in "leaves " is a logical slip.
False, it's exactly right. The chain gives . We want the big side () forced above ; the two inequalities point the correct way.
Stirling's Approximation is required to prove the bound is .
False. Stirling gives a tighter constant (), but the crude "keep the top factors, each " trick already yields with no Stirling.
A comparison sort may use , , , and ; adding more comparison types could break the bound.
False. In the standard model a single test still splits the flow into two branches, so the tree stays binary and still caps the leaves. Even a true multi-outcome primitive only changes the base (see the ternary edge-case below), never the class.
A randomized quicksort, because it flips coins, isn't captured by a decision tree so escapes the bound.
False. Fix the coin flips and you get one ordinary comparison decision tree; a randomized algorithm is a probability distribution over such trees. Its expected comparison count on the worst input is still — randomness can't extract more than 1 bit per comparison on average.
Spot the error
"There are items, so yes/no questions suffice, like Binary Search."
Binary Search locates one target among — that's outcomes. Sorting must pick one ordering among , so the count of outcomes is , giving , not .
"Height , so if leaves then height exactly."
Wrong direction and wrong equality. Height only for the number of leaves, and leaves , giving height — an inequality, never an equality.
"Since each comparison halves the possibilities, after comparisons we're always done."
A comparison halves possibilities only in the best case; an adversary can answer so one branch keeps most permutations. The bound guarantees you can't do better than , not that you always achieve it.
"Counting sort violates the theorem, so the theorem is false."
The theorem's premise is comparison-only. Counting sort does
count[x]++, using the value as an array index — it breaks the premise, so it simply lies outside the theorem's scope. No contradiction."A tree of height has nodes, and we need nodes."
The relevant count is leaves (red squares), not total nodes — leaves are where answers live. A height- binary tree has leaves (and up to nodes); the leaf bound is the one we combine with .
"The proof assumes all elements are distinct, so it says nothing about real data with duplicates."
Distinctness is a worst case chosen to make the argument clean; duplicates can only reduce the number of distinct required outputs. Since the hard instances (all-distinct) are legal inputs, the worst-case lower bound still holds for any correct sort.
" means the best algorithm needs that many comparisons for every input."
It means the best algorithm's tree height is , i.e. at least one input forces that many comparisons. Individual inputs may be far cheaper.
Why questions
Why does any comparison algorithm — with loops, recursion, swaps — reduce to a single fixed binary tree?
The only thing that changes control flow based on data is a comparison outcome. Fix , and the set of all execution paths is exactly one branching tree of comparisons (the figure); swaps and loops don't add branching on their own.
Why must we take (base 2) rather than natural log in ""?
Because each node has two outcomes, the leaf count grows as ; inverting gives . Base 2 is dictated by the branching factor, not a stylistic choice. Any base only changes the constant hidden in .
Why is "keep only the top half of the factors" a valid lower bound for ?
Dropping the small factors (all ) can only shrink a product of positive numbers. So , and every surviving factor is , giving — a legitimate underestimate.
Why does this bound make Merge Sort and Heapsort "asymptotically optimal"?
Their upper bound meets the lower bound, so no comparison sort can beat them by more than a constant factor — the comparison complexity of sorting is exactly .
Why can Radix Sort beat without contradiction?
It never asks "?"; it distributes keys into buckets by their digits, extracting information from the key's structure, not from comparisons. Outside the model, the bound doesn't bind it.
Why is this fundamentally an Information Theory & Entropy statement?
Identifying one outcome among equally-likely ones requires bits of information, and each comparison yields at most 1 bit. You cannot extract more information than the questions carry — that's the entire proof in one sentence.
Why does the argument talk about the tree's height rather than its average depth?
Height (the red arrow in the figure) = the longest root-to-leaf path = the worst-case comparison count, which is what a worst-case lower bound must control. Average depth would give the (separate) average-case bound.
Why does the bound still bite a randomized comparison sort on average?
A randomized sort is a distribution over decision trees; averaging over its coin flips, the expected leaf depth on the worst-case input is still (a Yao/entropy argument). Randomness reshuffles which input is hard, but can't lower the average information needed.
Edge cases
What does the bound say for the empty array, ?
leaf, : zero comparisons. An empty array is trivially sorted — the tree is a single leaf with no internal node, the same shape as .
What does the bound say for ?
leaf, : zero comparisons needed. A one-element array is already sorted — the tree is a single leaf, no internal node.
What about ?
leaves, so : exactly one comparison ("is ?") suffices and is necessary. The tree is one node with two leaves.
For , why is height 2 impossible?
outcomes but a height-2 binary tree has at most leaves . Four leaves can't label six distinct permutations, so height .
Does the bound break if the input contains all-equal elements?
No — that's a degenerate, easy input needing few comparisons. The lower bound is about the worst input; an all-equal case simply isn't the one forcing the height.
Is the statement meaningful for tiny like ?
is an asymptotic claim — it describes behaviour as . For fixed small you compute the exact (e.g. for ); the notation hides constants that only matter in the limit.
What if the machine has a true ternary primitive that returns , , or as three separate branches from one node?
Then that node has three children, the tree is ternary, and leaves , giving still. A genuine -way outcome node caps leaves at , so — the base changes, the asymptotic class does not, and a bounded number of outcomes buys only a constant-factor speedup.
Can a comparison sort ever have a leaf that is unreachable?
Yes — a poorly designed tree may contain leaves no input can reach. They don't help: the reachable leaves must still number , so unreachable leaves only inflate the tree without lowering the height bound.
Recall One-line self-test
Cover everything above. Can you state, in a single breath, why comparison sorting is ? Because sorting must distinguish orderings, each comparison gives bit, and bits are needed — so comparisons in the worst case.