Log and power transformations: Worked examples

Definition Notation & precision conventions (read once, applies everywhere)

$ln$ = natural log, base $e$ (the ML default). $lo g_{10}$ = base-10 log. A bare $lo g$ in a formula box copied from the parent means natural log unless a base is written; in this page I always write $ln$ or $lo g_{10}$ explicitly so there is never ambiguity.
log1p is just the computer function name for $ln (x + 1)$ — "log of 1 plus x." Libraries provide it separately because it stays accurate when $x$ is tiny (e.g. $x = 1 0^{- 9}$ ), where naively adding 1 then logging loses digits. Whenever you see log1p, read it as $ln (x + 1)$ .
MLE = Maximum Likelihood Estimation: an automatic search for the parameter value (here $λ$ ) that makes the observed data look most probable under an assumed model (here "the transformed data is normal"). You do not compute it by hand — a library does.
Precision policy: every intermediate is rounded to 4 significant figures; final answers to 3–4 sig figs. All numbers below are freshly recomputed to this policy, so ignore any looser rounding in the parent note.

The scenario matrix

Every case a log/power transform can throw at you falls into one of these cells. The columns are the input situation; the rows are which tool answers it.

Cell	Input situation	Right tool	Worked in
A	$x > 0$ , big right skew	$lo g_{10} (x)$	Ex 1
B	$x \geq 0$ , contains a zero	$ln (x + 1)$ (`log1p`)	Ex 2
C	$x > 0$ , mild skew	$x$ (power $λ = 0.5$ )	Ex 3
D	$x > 0$ , unknown best shape	Box-Cox, general $λ$	Ex 4
E	Limiting case $λ \to 0$	Box-Cox collapses to $ln$	Ex 5
F	Data has negative values	Yeo-Johnson, $x < 0$ branch	Ex 6
G	Yeo-Johnson positive branch + the seam at $x = 0$	Yeo-Johnson continuity	Ex 7
H	Real-world word problem: model trained on log target, must report dollars	inverse transform + Jensen bias	Ex 8
I	Exam twist: proportion in $(0, 1)$ , edge behaviour at $0$ and $1$	logit $ln \frac{p}{1 - p}$	Ex 9

Two figures carry the geometry: Figure 1 shows how each power $λ$ bends the number line (used in Cells C–E), and Figure 2 shows the logit stretching the interval $(0, 1)$ to the whole line (used in Cell I).

Look at the curves above. The straight diagonal is $λ = 1$ (do nothing). As $λ$ drops below 1, big inputs get pulled down more than small ones — that is exactly what "compress the right tail" means. When $λ$ passes through $0$ the family becomes the log curve (dashed). Below $0$ (reciprocal) it flattens even harder.

Cell A — big positive right skew → $lo g_{10}$

Worked example Ex 1 · House prices spanning 100×

Data: $[5000, 12000, 25000, 85000, 5, 000, 000]$ . Apply $x^{'} = lo g_{10} (x)$ and report the transformed spread.

Forecast: guess first — will the gap between the biggest number and the rest shrink or grow? My prediction: it will shrink hard, because log turns "1000× bigger" into "only 3 units bigger." Hold that guess and let's check it.

Step 1 — Take $lo g_{10}$ of each value. $lo g_{10} (5000) = 3.699, lo g_{10} (12000) = 4.079$ $lo g_{10} (25000) = 4.398, lo g_{10} (85000) = 4.929, lo g_{10} (5, 000, 000) = 6.699$ Why this step? $lo g_{10} (x)$ answers the question "how many powers of 10 is $x$ ?" A number that is $1000 \times$ bigger is only 3 units bigger in log-space. That is precisely how you crush a 100× spread into a small range.

Step 2 — Measure the spread before and after. Raw range $= 5, 000, 000 - 5000 = 4, 995, 000$ . Log range $= 6.699 - 3.699 = 3.0$ . Why this step? Range is the crude symptom of skew. We went from ~5 million down to 3, so the giant outlier no longer dominates — the forecast (shrink) is confirmed.

Verify: $1 0^{3.0} = 1000$ , and indeed the biggest raw value is $5, 000, 000/5000 = 1000 \times$ the smallest — the log range must equal $lo g_{10} (1000) = 3$ . Units check: log of dollars is dimensionless "orders of magnitude." ✓

Cell B — a zero in the data → $ln (x + 1)$

Worked example Ex 2 · Website visits including a silent day

Data (daily visits): $[0, 9, 99, 999]$ . You cannot take $ln (0)$ . Apply log1p, which means $x^{'} = ln (x + 1)$ .

Forecast: what does the zero become after the transform — does it stay finite? My prediction: yes, it becomes exactly $0$ , because we log $0 + 1 = 1$ .

Step 1 — Shift every value up by 1, then take natural log. $ln (0 + 1) = ln (1) = 0, ln (9 + 1) = ln 10 = 2.3026$ $ln (99 + 1) = ln 100 = 4.6052, ln (999 + 1) = ln 1000 = 6.9078$ Why this step? $ln (0) = - \infty$ would blow up the column. Adding 1 first means the worst case is $ln (1) = 0$ — a clean, finite anchor. We use $ln$ (base $e$ ) because it is the ML default, and the library routine log1p computes exactly this $ln (x + 1)$ while staying accurate for tiny $x$ (see the notation box).

Step 2 — Sanity: the transform is still monotonic (bigger in → bigger out). $0 < 2.30 < 4.61 < 6.91$ . Why this step? If a transform ever reordered points it would destroy rank information; confirming order is preserved proves we only rescaled, never scrambled.

Verify: $ln (9 + 1) = ln 10$ and $e^{2.3026} = 10.00$ ✓. The zero maps to exactly $0$ , finite as predicted. ✓

Cell C — mild skew → square root

Worked example Ex 3 · Photon counts (Poisson), gentle tail

Data (counts): $[4, 9, 16, 100]$ . Apply $x^{'} = x$ (Box-Cox with $λ = 0.5$ ; look at the mint curve in Figure 1).

Forecast: square root is "gentler than log." My prediction: $100$ will still stand out more than it did in Ex 1's log, because sqrt compresses less.

Step 1 — Take the square root of each. $4 = 2, 9 = 3, 16 = 4, 100 = 10$ Why this step? For count data the variance grows with the mean (Poisson behaviour). $x$ is the classic variance-stabilising transform for counts — it pulls the tail in but less aggressively than log, so structure survives.

Step 2 — Compare compression to log. Log would give $lo g_{10} 100 - lo g_{10} 4 = 2 - 0.602 = 1.398$ ; sqrt gives $10 - 2 = 8$ . Why this step? This is the whole point of "gentler": the sqrt-transformed $100$ is still $5 \times$ the smallest, whereas in log it would be much closer. Sqrt keeps more of the original contrast — forecast confirmed.

Verify: $2^{2} = 4$ , $1 0^{2} = 100$ ✓ — squaring the outputs returns the inputs, so is the true inverse of $x^{2}$ . ✓

Cell D — unknown shape → Box-Cox with a general $λ$

Worked example Ex 4 · Customer spending, chosen

$λ = - 0.1$ Data point: $x = 450$ (positive, so Box-Cox is legal — see the domain box). Optimal $λ = - 0.1$ was found by MLE (the automatic "most-normal" search defined in the notation box). Apply Box-Cox $x^{'} = \frac{x ^{λ} - 1}{λ} .$

Forecast: $λ$ is negative. My prediction: a huge $x$ gives a bounded $x^{'}$ , because $x^{- 0.1} \to 0$ leaves a finite ceiling.

Step 1 — Write the formula for this $λ$ . $x^{'} = \frac{45 0 ^{- 0.1} - 1}{- 0.1} = 10 (1 - 45 0^{- 0.1}) .$ Why this step? Dividing by the negative $λ = - 0.1$ flips the sign: $1/ (- 0.1) = - 10$ , and distributing gives the tidy $10 (1 - x^{- 0.1})$ form. This makes the ceiling obvious.

Step 2 — Evaluate $45 0^{- 0.1}$ and finish. $45 0^{- 0.1} = e^{- 0.1 l n 450} = e^{- 0.1 \times 6.109} = e^{- 0.6109} = 0.5427$ . $x^{'} = 10 (1 - 0.5427) = 4.573.$ Why this step? We convert the awkward power to $e^{λ l n x}$ because exponentials are what a calculator/CPU actually evaluates — this is the same $x^{λ} = e^{λ l n x}$ identity that makes the $λ \to 0$ limit work in Ex 5.

Step 3 — Note the ceiling. As $x \to \infty$ , $x^{- 0.1} \to 0$ , so $x^{'} \to 10 (1 - 0) = 10$ . Why this step? A negative $λ$ gives Box-Cox a horizontal asymptote — every giant outlier is squeezed under the value $10$ . That is why negative $λ$ tames heavy tails so well — forecast (bounded) confirmed.

Verify: freshly recomputed at 4-sig-fig precision, $10 (1 - 45 0^{- 0.1}) = 4.573$ . Bounds check: since $0 < 45 0^{- 0.1} < 1$ , we need $0 < x^{'} < 10$ — and $4.573$ sits in range. ✓

Cell E — the limiting case $λ \to 0$

Worked example Ex 5 · Why Box-Cox becomes log at

$λ = 0$ Task: show numerically that $\frac{x ^{λ} - 1}{λ} \to ln x$ as $λ \to 0$ , using $x = 100$ (watch the family in Figure 1 approach the dashed log curve).

Forecast: $ln 100 = 4.605$ . My prediction: smaller $λ$ pushes the Box-Cox value down toward $4.605$ .

Step 1 — Evaluate at shrinking $λ$ . $λ = 0.5 : \frac{10 0 ^{0.5} - 1}{0.5} = \frac{10 - 1}{0.5} = 18.0$ $λ = 0.1 : \frac{10 0 ^{0.1} - 1}{0.1} = \frac{1.5849 - 1}{0.1} = 5.849$ $λ = 0.01 : \frac{10 0 ^{0.01} - 1}{0.01} = \frac{1.04713 - 1}{0.01} = 4.713$ Why this step? We march $λ$ toward $0$ and watch the output. It is sliding down: $18 \to 5.85 \to 4.71$ , heading for $ln 100 = 4.605$ — forecast confirmed.

Step 2 — Explain why with L'Hôpital. At $λ = 0$ the fraction is $\frac{1 - 1}{0} = \frac{0}{0}$ , undefined. Differentiate top and bottom with respect to $λ$ : $\frac{d}{d λ} (x^{λ} - 1) = x^{λ} ln x$ , and $\frac{d}{d λ} (λ) = 1$ . So the limit is $x^{0} ln x = ln x$ . Why this step? L'Hôpital is the exact tool for a $0/0$ hole: it replaces the ratio of values by the ratio of slopes, which is what "fill the gap smoothly" means. That is why the domain box defines the $λ = 0$ case as $ln x$ — it is not a separate rule, it is the only value that keeps the curve continuous.

Verify: sequence $18.0, 5.849, 4.713$ is decreasing toward $ln 100 = 4.6052$ , and each successive term is closer. ✓

Cell F — negative input → Yeo-Johnson, $x < 0$ branch

Worked example Ex 6 · Temperature anomaly

$x = - 8.2$ , $λ = 0.8$ Box-Cox is illegal here ( $x \leq 0$ violates the domain box). Since $λ = 0.8 \neq = 2$ , use the third case: $x^{'} = - \frac{( - x + 1 ) ^{2 - λ} - 1}{2 - λ} .$

Forecast: the output has a leading minus. My prediction: $x^{'}$ stays negative like the input (the branch is a sign-preserving mirror).

Step 1 — Plug in $x = - 8.2$ , $λ = 0.8$ , so $2 - λ = 1.2$ and $- x + 1 = 9.2$ . $x^{'} = - \frac{9. 2 ^{1.2} - 1}{1.2} .$ Why this step? For negatives, Yeo-Johnson mirrors the positive rule: it flips $x$ to $- x$ , adds 1 to keep the base $\geq 1$ , applies a power, then flips the sign back with the outer minus. This mirroring is what keeps negatives negative and preserves order across zero.

Step 2 — Compute $9. 2^{1.2}$ . $9. 2^{1.2} = e^{1.2 l n 9.2} = e^{1.2 \times 2.2192} = e^{2.6631} = 14.34$ . $x^{'} = - \frac{14.34 - 1}{1.2} = - \frac{13.34}{1.2} = - 11.12.$ Why this step? Again $a^{b} = e^{b l n a}$ to actually evaluate. The result $- 11.12$ is more negative than the input, stretching (not compressing) here because $λ < 2$ on the negative side.

Verify: freshly recomputed at 4-sig-fig precision, $9. 2^{1.2} = 14.34$ , giving $x^{'} = - 11.12$ . Sign check: input negative → output negative ✓ (forecast confirmed), and the monotonic mirror preserved order. ✓

Cell G — Yeo-Johnson positive branch + the seam at zero

Worked example Ex 7 ·

$x = 7.8$ positive, and continuity at $x = 0$ Positive branch (first case, $λ = 0.8 \neq = 0$ ): $x^{'} = \frac{( x + 1 ) ^{λ} - 1}{λ}$ . Then check both branches agree at $x = 0$ .

Forecast: at exactly $x = 0$ , do the " $x \geq 0$ " and " $x < 0$ " formulas give the same answer? My prediction: yes, both give $0$ — otherwise there'd be a jump.

Step 1 — Positive value $x = 7.8$ . $x^{'} = \frac{8. 8 ^{0.8} - 1}{0.8}, 8. 8^{0.8} = e^{0.8 l n 8.8} = e^{0.8 \times 2.1748} = e^{1.7398} = 5.696.$ $x^{'} = \frac{5.696 - 1}{0.8} = \frac{4.696}{0.8} = 5.870.$ Why this step? Same machinery as Box-Cox but on the shifted base $(x + 1)$ , so zeros are legal. The tail value $7.8$ compresses to $5.870$ .

Step 2 — Evaluate both branches at $x = 0$ . Positive branch: $\frac{( 0 + 1 ) ^{0.8} - 1}{0.8} = \frac{1 - 1}{0.8} = 0$ . Negative branch: $- \frac{( 0 + 1 ) ^{2 - 0.8} - 1}{1.2} = - \frac{1 - 1}{1.2} = 0$ . Why this step? Both give $0$ , so the two pieces meet at the origin — no gap, no jump (forecast confirmed). This is exactly the "preserves continuity at $x = 0$ " property, now checked by hand.

Verify: freshly recomputed at 4-sig-fig precision, $8. 8^{0.8} = 5.696$ so $x^{'} = 5.870$ . Both branches equal $0$ at $x = 0$ ✓ — continuity confirmed. ✓

Cell H — real-world word problem: inverse transform a prediction

Worked example Ex 8 · From log-dollars back to dollars (with Jensen fix)

Setup: you trained a regressor on $ln$ -price. For one house it predicts $\overset{y}{^}_{l o g} = 13.0$ (natural log of dollars), and the residual variance is $σ^{2} = 0.20$ . Report the price in dollars.

Forecast: is the honest dollar prediction just $e^{13.0}$ , or a bit higher? My prediction: a bit higher, because exponentiating a mean under-shoots.

Step 1 — Naive back-transform. $\overset{y}{^}_{naive} = e^{13.0} = 442, 413 dollars .$ Why this step? The model lives in log-space; to speak dollars we undo the natural log with its inverse $exp$ . But this only recovers the log-space median, not the mean.

Step 2 — Jensen bias correction. $\overset{y}{^}_{mean} = e^{\overset{y}{^}_{l o g} + σ^{2} /2} = e^{13.0 + 0.10} = e^{13.1} = 488, 938 dollars .$ Why this step? Because $exp$ is convex, averaging in log-space then exponentiating underestimates the true mean: $E [y] \neq = e^{E [l o g y]}$ . Adding $σ^{2} /2$ inside the exponent is the exact correction for log-normal data. Skipping it systematically under-predicts every price — forecast confirmed.

Verify: $e^{13} = 442, 413$ and $e^{13.1} = 488, 938$ ; the corrected value is larger by factor $e^{0.1} = 1.1052$ ✓ — corrected exceeds naive, as convexity demands. ✓

Cell I — exam twist: proportions with the logit

Worked example Ex 9 · A conversion rate

$p = 0.8$ , and the edges $p \to 0, 1$ Proportions live in $(0, 1)$ ; models often want $(- \infty, \infty)$ . Use the logit: $x^{'} = ln \frac{p}{1 - p} (natural log) .$

Forecast: $p = 0.8$ is above the midpoint $0.5$ . My prediction: the logit is positive, and as $p \to 1$ it runs to $+ \infty$ .

Step 1 — Transform $p = 0.8$ . $x^{'} = ln \frac{0.8}{0.2} = ln 4 = 1.3863.$ Why this step? $\frac{p}{1 - p}$ is the odds (successes per failure). Logit takes the log of odds, turning a bounded probability into an unbounded real number — a linear model can then move it freely without ever predicting an impossible probability.

Step 2 — Check the two edges. As $p \to 1^{-}$ , $1 - p \to 0^{+}$ , so $\frac{p}{1 - p} \to + \infty$ and $x^{'} \to + \infty$ . As $p \to 0^{+}$ , $\frac{p}{1 - p} \to 0^{+}$ and $x^{'} \to - \infty$ . At $p = 0.5$ , $ln 1 = 0$ . Why this step? This maps the whole interval: the two forbidden endpoints $0$ and $1$ get pushed to $\mp \infty$ , and the centre $0.5$ lands at $0$ . Every proportion has a finite image strictly inside $(0, 1)$ ; only the degenerate endpoints escape to infinity — which is exactly why you must never feed a raw $0$ or $1$ into a logit. Trace this on the figure below.

Look at the lavender curve: the mint dot at $p = 0.5$ sits at height $0$ , the coral dot at $p = 0.8$ sits at $1.386$ , and the curve shoots to $\pm \infty$ as you approach the two edges — the geometry of "stretch $(0, 1)$ onto the whole line."

Verify: $ln (0.8/0.2) = ln 4 = 1.3863$ ✓ (positive, forecast confirmed). Inverse (sigmoid) recovers it: $\frac{1}{1 + e ^{- 1.3863}} = 0.8$ ✓. ✓

Recall Self-test

Why does adding 1 before a log fix a zero? ::: $ln 0 = - \infty$ but $ln (0 + 1) = ln 1 = 0$ , finite and clean. What single value of $λ$ turns Box-Cox into $ln$ , and why? ::: $λ = 0$ ; the $0/0$ limit resolves by L'Hôpital to $ln x$ , the only value keeping the curve continuous. Which Yeo-Johnson $λ$ makes the negative-branch denominator zero, and how is it handled? ::: $λ = 2$ ; the $0/0$ hole is filled by L'Hôpital to give $- ln (- x + 1)$ . When is Yeo-Johnson mandatory over Box-Cox? ::: When the data contains negative or zero values — Box-Cox needs $x > 0$ . Why add $σ^{2} /2$ when un-logging a prediction? ::: $exp$ is convex, so $E [y] > e^{E [l o g y]}$ (Jensen); $σ^{2} /2$ corrects the resulting under-prediction. What do $p \to 0$ and $p \to 1$ map to under logit? ::: $- \infty$ and $+ \infty$ respectively.

Connections

Parent: Log and power transformations · Hinglish
Transforms stabilise variance, complementing 2.1.05-Feature-Scaling.
Compressing tails is an alternative to clipping in 2.1.08-Handling-Outliers.
Normality/homoscedasticity matter for 3.2.01-Linear-Regression-Assumptions.
Power features relate to 2.2.03-Polynomial-Features.
Monotone transforms leave 4.1.02-Decision-Trees split points unchanged — a useful contrast.