Worked examples — Random forest algorithm

Everything here uses only three formulas from the parent, re-stated in plain words before use:

Each symbol will be pinned to a picture before we compute with it.

---

The scenario matrix

Before solving, let us list every case class a random-forest problem can be. Each worked example below is tagged with the cell it fills.

#	Case class	What is extreme / tricky	Example
A	$\rho$ = 0 (fully de-correlated trees)	best case, variance drops like $1/B$	Ex 1
B	$\rho$ = 1 (identical trees)	worst case, variance never drops	Ex 2
C	$0<\rho <1$ typical + effect of raising $B$	the realistic middle	Ex 3
D	$B\to \infty$ limiting behaviour	the irreducible floor	Ex 4
E	$B=1$ degenerate (one tree)	forest collapses to a single tree	Ex 5
F	OOB fraction for small vs large $n$	tiny data vs the $e^{-1}$ limit	Ex 6
G	Classification vote incl. a TIE	odd vs even $B$, tie-break	Ex 7
H	Regression average	continuous target	Ex 8
I	Real-world word problem	choosing $B$ to hit a variance target	Ex 9
J	Exam twist: "how much did feature-subsampling help?"	compare bagging ($\rho$ high) vs RF ($\rho$ low)	Ex 10

The variance law $\mathrm{Var}(\bar{f})=\rho {\sigma }^2+\frac{1-\rho }{B}{\sigma }^2$ is a straight line if we treat $\frac{1}{B}$ as the horizontal axis. Fixing that picture in your head kills cases A–E at once:

Look at the figure. The height where each line lands on the right edge (as $\frac{1}{B}\to 0$, i.e. infinite trees) is $\rho {\sigma }^2$ — the floor. The steepness of each line is $(1-\rho ){\sigma }^2$ — how fast more trees help. Keep referring back to this as we do the numbers.

---

Case A — fully de-correlated trees ($\rho =0$)

Step 1 — write the tool. Use variance law (1). Why this step? We are asked for the variance of the averaged forest, which is exactly what formula (1) computes.

Step 2 — substitute $\rho =0$. The first term $\rho {\sigma }^2=0$, so $\mathrm{Var}(\bar{f})=0+\frac{1-0}{3}\cdot 9=\frac{9}{3}=3.$ Why this step? With no correlation the floor collapses to zero — the picture's line ends at height $0$ on the right edge. All that survives is the $\frac{{\sigma }^2}{B}$ term, the classic "averaging $B$ independent things divides variance by $B$" result.

Verify: $\mathrm{Var}(\bar{f})=3$, which is ${\sigma }^2/B=9/3$. Averaging $3$ independent estimators divides variance by $3$ — matches the textbook independent case exactly. ✓

---

Case B — identical trees ($\rho =1$)

Step 1 — substitute $\rho =1$ into law (1). $\mathrm{Var}(\bar{f})=1\cdot 9+\frac{1-1}{100}\cdot 9=9+0=9.$ Why this step? When $\rho =1$ the second term's numerator $(1-\rho )=0$, so the number of trees is irrelevant. In the picture, this is the flat horizontal line — its slope $(1-\rho ){\sigma }^2=0$.

Step 2 — interpret. $\mathrm{Var}(\bar{f})=9={\sigma }^2$: the forest is exactly as jittery as one tree. Averaging identical clones does nothing. Why? Averaging cancels disagreements, and identical trees never disagree.

Verify: $\mathrm{Var}(\bar{f})=9={\sigma }^2$. This is the parent note's punchline made numeric — the variance floor is $\rho {\sigma }^2=1\times 9=9$, and here we already sit on it. ✓

---

Case C — realistic middle, and turning up $B$

Step 1 — case (a), $B=10$. Law (1): $\mathrm{Var}(\bar{f})=0.2\cdot 4+\frac{1-0.2}{10}\cdot 4=0.8+\frac{0.8}{10}\cdot 4.$ Careful: $\frac{0.8}{10}\cdot 4=\frac{3.2}{10}=0.32$. So $\mathrm{Var}(\bar{f})=0.8+0.32=1.12$. Why this step? We plug the given numbers straight in; the $0.8$ is the floor $\rho {\sigma }^2$, the $0.32$ is the shrinkable part.

Step 2 — case (b), $B=50$. $\mathrm{Var}(\bar{f})=0.8+\frac{0.8}{50}\cdot 4=0.8+0.064=\mathrm{0.864.}$

Step 3 — how much did we gain? From $1.12$ to $0.864$, a drop of $0.256$. But the floor is $0.8$, so we can never go below it no matter how many trees. Why this matters: in the picture we slid leftward toward the floor line but the gap to $0.8$ only shrank by $0.256$ — diminishing returns, exactly the "more trees can't beat the floor" lesson.

Verify: floor $=\rho {\sigma }^2=0.8$; both answers exceed it ($1.12>0.864>0.8$) and decrease with $B$. ✓

---

Case D — the limit $B\to \infty$

Step 1 — take the limit of the second term. As $B\to \infty$, $\frac{1-\rho }{B}{\sigma }^2\to 0$. Why this step? A fixed numerator divided by a number growing without bound goes to zero — that is what "limit as $B\to \infty$" means; in the picture $\frac{1}{B}\to 0$, so we read off the right-edge height.

Step 2 — what remains. ${\lim }_{B\to \infty }\mathrm{Var}(\bar{f})=\rho {\sigma }^2=0.2\cdot 4=\mathrm{0.8.}$

Verify: $0.8$ equals the floor we predicted in Ex 3, and Ex 3(b)'s answer $0.864$ is already close, sitting just $0.064$ above it. ✓ The sequence approaches $0.8$ from above and never crosses it.

---

Case E — degenerate forest, one tree ($B=1$)

Step 1 — substitute $B=1$. $\mathrm{Var}(\bar{f})=\rho {\sigma }^2+\frac{1-\rho }{1}{\sigma }^2=\rho {\sigma }^2+(1-\rho ){\sigma }^2.$ Why this step? $B=1$ makes the denominator $1$, so the whole second term stays.

Step 2 — simplify. The $\rho {\sigma }^2$ terms cancel: $\rho {\sigma }^2+(1-\rho ){\sigma }^2={\sigma }^2=4.$ Why this matters: with one tree there is nothing to average, so the "forest" variance must equal a single tree's variance. The formula passes this sanity test — a great way to remember it is right.

Verify: $\mathrm{Var}(\bar{f})=4={\sigma }^2$, correct regardless of $\rho$. A one-tree forest is a tree. ✓

---

Case F — OOB fraction: tiny data vs the limit

Step 1 — tool. Use OOB formula (2): $P(\text{left out})=(1-1/n{)}^n$. Why this tool and not another? Each bootstrap draw independently misses a fixed row with probability $\frac{n-1}{n}=1-\frac{1}{n}$; over $n$ independent draws we multiply, giving the $n$-th power.

Step 2 — (a) $n=5$. $(1-\frac{1}{5}{)}^5=(0.8{)}^5=0.32768\approx 32.8\%.$

Step 3 — (b) $n=100$. $(1-\frac{1}{100}{)}^{100}=(0.99{)}^{100}\approx 0.3660\approx 36.6\%.$

Step 4 — (c) the limit. $(1-1/n{)}^n\to e^{-1}\approx 0.3679\approx 36.8\%$. Why this step? This is the classic definition of $e^{-1}$; small $n$ gives a slightly smaller fraction, converging up to $36.8\%$.

Verify: $0.32768<0.3660<0.3679$ — the fraction rises monotonically toward $e^{-1}\approx 0.3679$, matching the parent's $36.8\%$ claim. ✓

---

Case G — classification vote, including a TIE

Picture six ballot boxes; count the 1's and 0's. This is the discrete cousin of "averaging" — the class with more votes wins.

Step 1 — (a) count. Votes for $1$: positions 1,2,4 → three ones. Votes for $0$: two. $3>2$, so predict class 1 (sick). Why this step? Majority vote = argmax of the class counts; with $B=5$ (odd) there is always a strict winner.

Step 2 — (b) the tie. Now $3$ ones and $3$ zeros. There is no majority — the definition breaks. Why this happens: with even $B$ a tie is possible. Standard tie-break rules: (i) predict the class with the higher summed probability, or (ii) default to the majority class in training, or (iii) sklearn picks the lowest class label (here $0$). Using rule (iii): predict class 0.

Verify: count of ones $=3$, zeros $=3$; $3=3$ confirms the tie in (b), while $3>2$ confirms class 1 in (a). ✓ Lesson: use an odd $B$ to avoid ties in binary classification.

---

Case H — regression, plain average

Step 1 — tool. Regression forests average (formula 3), not vote. Why this tool? The target is continuous, so "most common value" is meaningless; the mean minimizes squared error across the trees.

Step 2 — compute. $\bar{f}=\frac{210+250+190+270}{4}=\frac{920}{4}=230.$ So the forest predicts $230{,}000.

Verify: sum $=920$, divided by $4$ gives $230$; it lies between the min $190$ and max $270$, as any average must. ✓

---

Case I — real-world word problem: choosing $B$

Forecast: the floor is $\rho {\sigma }^2=3$. Since $3<3.5$, the target is reachable. Now find $B$.

Step 1 — set up the inequality. We need $3+\frac{1-0.3}{B}\cdot 10\le 3.5⟹\frac{0.7\cdot 10}{B}\le 0.5⟹\frac{7}{B}\le \mathrm{0.5.}$ Why this step? Subtract the floor $\rho {\sigma }^2=3$ from both sides; only the shrinkable term must fit under the remaining budget $0.5$.

Step 2 — solve for $B$. $B\ge \frac{7}{0.5}=14.$ So $B=14$ trees suffice (the smallest integer $\ge 14$ is $14$ itself).

Step 3 — sanity check the floor. If the SLA had demanded $\mathrm{Var}\le 2.5<3$, no $B$ would work, because you cannot dip below the floor $\rho {\sigma }^2=3$; you'd have to lower $\rho$ instead (more feature subsampling). Why this matters: this is the parent's "you're stuck at $\rho {\sigma }^2$" lesson turned into a design decision.

Verify: at $B=14$: $3+\frac{7}{14}=3+0.5=3.5\le 3.5$ ✓. At $B=13$: $3+7/13\approx 3.538>3.5$ ✗. So $14$ is exactly the smallest. ✓

---

Case J — exam twist: how much did feature-subsampling buy?

Step 1 — bagging variance. ${\mathrm{Var}}_{\text{bag}}=0.6\cdot 8+\frac{1-0.6}{100}\cdot 8=4.8+\frac{0.4\cdot 8}{100}=4.8+0.032=\mathrm{4.832.}$

Step 2 — random-forest variance. ${\mathrm{Var}}_{\text{rf}}=0.15\cdot 8+\frac{1-0.15}{100}\cdot 8=1.2+\frac{0.85\cdot 8}{100}=1.2+0.068=\mathrm{1.268.}$

Step 3 — attribute the improvement. Total drop $=4.832-1.268=3.564$.

• The floor dropped from $4.8$ to $1.2$, i.e. $3.6$ of improvement came from lowering $\rho$.
• The $\frac{1}{B}$ terms actually rose slightly ($0.032\to 0.068$) because $(1-\rho )$ grew when $\rho$ shrank — a tiny $-0.036$ offset.
• Net: $3.6-0.036=3.564$. ✓

Why this matters: essentially 100% of the win came from de-correlation, not from the number of trees. This is the parent's headline — "the real magic isn't more trees, it's lowering $\rho$" — proven with numbers.

Verify: $4.832-1.268=3.564$; floor change $4.8-1.2=3.6$; $B$-term change $0.068-0.032=+0.036$; $3.6-0.036=3.564$. ✓

---

Active Recall

Recall With

${\sigma }^2=9$, $\rho =0$, $B=3$, what is $\mathrm{Var}(\bar{f})$? (Ex 1) $0+\frac{1}{3}\cdot 9=3$ — pure ${\sigma }^2/B$, the independent case.

Recall Why does

$\rho =1$ make more trees useless? (Ex 2) The slope $(1-\rho ){\sigma }^2=0$: the line is flat, variance stays at the floor $\rho {\sigma }^2={\sigma }^2$.

Recall Smallest

$B$ to hit $\mathrm{Var}\le 3.5$ when ${\sigma }^2=10,\rho =0.3$? (Ex 9) $B=14$, from $7/B\le 0.5$. The floor $\rho {\sigma }^2=3$ makes $3.5$ reachable.

Recall OOB fraction for

$n=5$ vs the limit? (Ex 6) $(0.8{)}^5=32.8\%$ vs $e^{-1}\approx 36.8\%$; it rises toward the limit.

Recall What fraction of RF's variance win over bagging came from lowering

$\rho$? (Ex 10) Essentially all of it: floor dropped $3.6$ out of a $3.564$ net gain.

---

Connections

• Random forest algorithm — the parent theory these examples exercise.
• Bagging (Bootstrap Aggregating) — the high-$\rho$ baseline in Ex 10.
• Bias-Variance Tradeoff — the variance we keep computing.
• Cross-Validation — what OOD/OOB (Ex 6) replaces for free.
• Decision Trees — the single tree that Ex 5 collapses to.
• Feature Importance · Gini Impurity & Entropy · Gradient Boosting — neighbouring topics.

---