Performance profiling — CPU, memory, I - O profiling: Worked examples

The scenario matrix

Before touching numbers, let us list the shapes a profiling problem comes in. Think of this table as a checklist: every cell must get at least one fully-worked example below, so you never meet a scenario cold.

#	Case class	What makes it tricky	Example
A	Big slice ( $p$ large)	optimization pays off	Ex 1
B	Small slice ( $p$ small)	Amdahl says don't bother	Ex 2
C	Limit $s \to \infty$	ceiling / best-case speedup	Ex 3
D	Degenerate $p = 0$ or $p = 1$	formula edge behaviour	Ex 4
E	Sampling: how many samples?	Binomial error, solve for $N$	Ex 5
E′	Sampling worst case $p = 0.5$	$p (1 - p)$ maximal → most samples	Ex 5b
F	Sampling: tiny hotspot	error explodes for small $p$	Ex 6
G	Wall ≫ CPU (I/O-bound)	the profiler "lies"	Ex 7
H	Wall ≈ CPU (CPU-bound)	trust the profiler	Ex 7
I	Self vs cumulative time	wrapper vs real worker	Ex 8
J	Real-world word problem	translate English → $p, s$	Ex 9
K	Exam twist (multi-stage, combine laws)	chain two optimizations	Ex 10

Let us first pin down the symbols so nothing is used before it is defined.

Case A — the big slice pays off

Worked example Ex 1 · cell A

An API request takes 200 ms. A profiler attributes 160 ms to image resizing. You can make resizing 5× faster. What is the overall speedup and new time?

Forecast: Guess before reading. Resizing is 80% of the work — will 5× on it give roughly 5×, or much less? Write your guess down.

Find $p$ . Why this step? $S$ needs the fraction, not the raw ms — Amdahl is scale-free. $p = \frac{160}{200} = 0.8$
Read off $s$ . Why? "5× faster" is exactly the speedup factor. $s = 5$ .
Apply Amdahl. Why this formula? It is the only one that accounts for the untouched 20% dragging you down. $S = \frac{1}{( 1 - 0.8 ) + \frac{0.8}{5}} = \frac{1}{0.2 + 0.16} = \frac{1}{0.36} \approx 2.78 \times$
New time. Why? By definition $T_{new} = T_{old} / S$ — speedup means old time ÷ $S$ . $T_{new} = \frac{200}{2.78} \approx 72 ms$

Verify: Compute the new time directly — untouched 40 ms + resize $160/5 = 32$ ms $= 72$ ms. ✓ Matches. Note $2.78 ≪ 5$ : the 20% you left alone caps you. That is the whole lesson.

Case B — the small slice: Amdahl says skip it

Case C — the limit $s \to \infty$ : your best-case ceiling

Worked example Ex 3 · cell C

For the Ex-2 request ( $p = 0.1$ ), what is the maximum possible speedup, even if resizing became instant?

Forecast: If a part vanishes entirely, is the ceiling infinite or finite?

Take the limit. Why this tool — a limit? "Instant" means $s$ grows without bound; a limit is precisely the machinery for "as $s$ gets arbitrarily large, what does $S$ approach?" The term $p / s \to 0$ . $S_{m a x} = lim_{s \to \infty} \frac{1}{( 1 - p ) + p / s} = \frac{1}{1 - p} = \frac{1}{0.9} \approx 1.11 \times$
Read the geometry. Why a figure? Seeing $S$ flatten against a wall as $s$ rises makes "diminishing returns" concrete. The figure below plots overall speedup $S$ (y-axis) against the part's speedup $s$ (x-axis) for $p = 0.1$ .

Figure s01 — alt description. X-axis: $s$ , the speedup of the optimized part, running 1→60. Y-axis: $S$ , the overall speedup. The blue curve $S (s)$ rises steeply then bends over. A pink dashed horizontal line marks the ceiling $1/ (1 - p) = 1.111$ . A yellow dot at $s = 5$ sits at $\approx 1.087$ , already 98% of the ceiling — the annotation arrow points to it. The take-away the picture carries: the curve flattens against the ceiling, so pushing $s$ from 20 to 200 barely moves $S$ .

Verify: $1/ (1 - 0.1) = 1.1111$ . And Ex 2's $s = 5$ gave $1.087$ — already 98% of the ceiling. ✓ So even $5 \times$ nearly maxed out this small slice; more effort is wasted.

Case D — degenerate inputs $p = 0$ and $p = 1$

Worked example Ex 4 · cell D

Sanity-test Amdahl at its extremes. What is $S$ when (a) $p = 0$ , and (b) $p = 1$ ?

Forecast: One extreme should give "no change ever," the other "speedup equals $s$ ." Which is which?

$p = 0$ (you optimize a part that takes no time). Why check this? Degenerate inputs expose whether a formula is sane. $S = \frac{1}{( 1 - 0 ) + 0/ s} = \frac{1}{1} = 1 \times$ Optimizing nothing changes nothing. ✓ Sensible.
$p = 1$ (the whole program is the part). Why? The opposite extreme — no untouched code to hold you back. $S = \frac{1}{( 1 - 1 ) + 1/ s} = \frac{1}{0 + 1/ s} = s$ Overall speedup equals the part's speedup. ✓ Also sensible.
Watch the pole. Why mention it? If $p = 1$ and $s \to \infty$ , denominator $\to 0$ and $S \to \infty$ — the only way to get unbounded speedup is to optimize 100% of the program by an unbounded factor. That never happens in reality, which is why real speedups are always finite.

Verify: $p = 0 \Rightarrow S = 1$ ; $p = 1, s = 3 \Rightarrow S = 3$ . ✓ Both boundary cases behave.

Case E — sampling: how many samples do I need?

Now switch tools. A sampling profiler peeks at "what's running now?" every so often. Each peek is a coin flip: landed-in-our-function (heads) or not (tails). With $N$ flips and true fraction $p$ , the count of heads follows the Binomial distribution, and the spread of our estimate $\overset{p}{^}$ is the standard error — the typical distance between our measured fraction and the truth: $S E (\overset{p}{^}) = \frac{p ( 1 - p )}{N}$

Worked example Ex 5 · cell E

A hotspot is about 30% of runtime. You want the estimate accurate to $S E \leq 1.5%$ . How many samples $N$ ?

Forecast: Hundreds? Thousands? Guess the order of magnitude.

Solve $S E$ for $N$ . Why algebra here? We know the target error and want the sample count — invert the formula. Square both sides, then divide. $N = \frac{p ( 1 - p )}{S E ^{2}}$
Plug in $p = 0.3$ , $S E = 0.015$ . Why these numbers? $p (1 - p)$ is largest near $p = 0.5$ , so 0.3 is a realistic, not worst-case, hotspot. $N = \frac{0.3 \times 0.7}{0.01 5 ^{2}} = \frac{0.21}{0.000225} \approx 933.3 \Rightarrow N = 934$
Convert to time. Why this step? A sample count is abstract; engineers budget in seconds of running. Since the profiler fires at a fixed rate, dividing samples by that rate turns "934 samples" into a concrete "how long must I let it run?" At 1000 samples/second, $934/1000 \approx$ 0.93 s of profiling.

Verify: Put $N = 934$ back: $0.21/934 = 0.0002249 = 0.015$ . ✓ Under a second of running nails a 30% hotspot to ±1.5%.

Case E′ — the worst case: $p = 0.5$ needs the most samples

Worked example Ex 5b · cell E′

Same $S E \leq 1.5%$ target, but now the hotspot is bang on 50% of runtime. How many samples now — and why is this the hardest fraction to pin down?

Forecast: More samples than Ex 5's 934, or fewer? By how much?

Ask where $p (1 - p)$ peaks. Why this step? The required $N = p (1 - p) / S E^{2}$ grows with the factor $p (1 - p)$ . That factor is a downward parabola in $p$ : zero at $p = 0$ and $p = 1$ , and maximal at $p = 0.5$ , where $p (1 - p) = 0.25$ . So $p = 0.5$ is the single worst case — the fraction that demands the most samples for a given accuracy.
Plug in $p = 0.5$ , $S E = 0.015$ . Why? We want the upper bound on samples: whatever $N$ works here works for every other $p$ . $N = \frac{0.5 \times 0.5}{0.01 5 ^{2}} = \frac{0.25}{0.000225} \approx 1111.1 \Rightarrow N = 1112$
Compare to Ex 5. Why this step? To feel the size of the penalty. $1112$ vs $934$ — the worst case costs only $\approx 19%$ more samples than the $p = 0.3$ case, because the parabola is flat near its top.

Verify: $0.5 \times 0.5 = 0.25$ is the max of $p (1 - p)$ ; $N = 0.25/0.01 5^{2} = 1111.1 \Rightarrow 1112$ . ✓ Rule of thumb: to be safe for any hotspot, size $N$ for $p = 0.5$ — that upper bound never under-samples.

Case F — sampling breaks down for a tiny hotspot

Worked example Ex 6 · cell F

You took $N = 1000$ samples. A function got just 3 hits, so $\overset{p}{^} = 0.003$ . How trustworthy is that?

Forecast: Is "0.3% of CPU" a solid reading or noise?

Estimate the error at this $\overset{p}{^}$ . Why use $\overset{p}{^}$ in place of $p$ ? We do not know the true $p$ ; plugging the estimate in is standard. $S E = \frac{0.003 \times 0.997}{1000} = 0.000002991 \approx 0.00173$
Compare error to the estimate. Why this step? An error near the size of the value itself means the value is basically guesswork. $\frac{S E}{p ^} = \frac{0.00173}{0.003} \approx 0.58 = 58% relative error$
Interpret. A ±58% relative error means the "true" share could plausibly be anywhere from ~0.13% to ~0.47%. Why care? Tiny slices are exactly where sampling is worst — you cannot rank the bottom of a flame graph from samples alone. (See Flame Graphs.)

Verify: $0.003 \cdot 0.997/1000 = 0.001729$ , and $0.001729/0.003 = 0.576$ . ✓ Roughly 58%. Rule of thumb: trust a sampled fraction only when its hit count is in the dozens+.

Case G & H — wall ≫ CPU vs wall ≈ CPU (does the profiler lie?)

Recall the elapsed-time split, where $T_{wall}$ is real clock time you'd measure with a stopwatch, $T_{CPU}$ is time cores were actually computing (user + sys), and the rest is waiting (disk, network, locks, sleep): $T_{wall} = T_{CPU} + T_{wait}$

Worked example Ex 7 · cells G & H

time ./prog prints for two runs:

Run G: real 10.0 s, user 0.6 s, sys 0.4 s.
Run H: real 8.0 s, user 7.2 s, sys 0.5 s. For each, is it I/O-bound or CPU-bound, and would a CPU profiler help?

Forecast: In which run does the CPU profiler lie?

Compute CPU time and the wait. Why? $T_{CPU} = user + sys$ ; the gap is waiting.
- Run G: $T_{CPU} = 0.6 + 0.4 = 1.0$ s; $T_{wait} = 10.0 - 1.0 = 9.0$ s.
- Run H: $T_{CPU} = 7.2 + 0.5 = 7.7$ s; $T_{wait} = 8.0 - 7.7 = 0.3$ s.
Compute CPU utilization $= T_{CPU} / T_{wall}$ . Why a ratio? It's the fraction of real time cores were busy — the tell-tale.
- Run G: $1.0/10.0 = 0.10 = 10%$ → mostly waiting → I/O-bound (cell G).
- Run H: $7.7/8.0 = 0.9625 = 96.25%$ → cores pinned → CPU-bound (cell H).
Verdict on the profiler. Why this matters? A CPU profiler only sees $T_{CPU}$ . In Run G it can explain just 1 of 10 seconds — it lies by omission; profile I/O (strace, DB logs, the N+1 Query Problem). In Run H, trust the CPU profiler / Flame Graphs.

Figure s02 — alt description. Two stacked bars, y-axis in seconds (0→11). Left bar "Run G (I/O-bound)": a thin yellow base = CPU 1.0 s, a tall blue stack on top = wait 9.0 s, labelled "util 10%" in pink. Right bar "Run H (CPU-bound)": a tall yellow base = CPU 7.7 s, a thin blue cap = wait 0.3 s, labelled "util 96%". The picture's message: Run G is almost all pale-blue wait (profiler blind), Run H is almost all yellow CPU (profiler trustworthy).

Verify: G util $= 1.0/10 = 0.10$ ; H util $= 7.7/8 = 0.9625$ . ✓ Cross the 50% line and the diagnosis flips.

Case I — self time vs cumulative time

Worked example Ex 8 · cell I

A profiler reports for handle(): cumulative 900 ms, self 5 ms. Its only child crunch() shows cumulative 890 ms, self 890 ms. Which do you optimize?

Forecast: The big number is on handle(). Trap or truth?

Define the two times. Why re-define? They are the #1 source of wrong conclusions. Self (exclusive) = time inside a function, not counting callees. Cumulative (inclusive) = self + everything it calls.
Read handle(). Cumulative 900 but self only 5 ms. Why does this matter? $900 - 5 = 895$ ms happens inside children, not in handle itself. handle is a wrapper.
Read crunch(). Self 890 ms — the cycles actually burn here. Why? Self time points at the real worker; cumulative can be inflated by a thin parent.
Decide: optimize crunch(). Why this step? Because self time is the only quantity that bounds your possible win inside a given function: editing handle's own body touches just its 5 ms of self time (max 5 ms saved), whereas crunch's 890 ms of self time is where the cycles actually vanish, so that is where effort converts into real speedup.

Verify: Children of handle $= 900 - 5 = 895 \approx$ crunch's cumulative 890 (the 5 ms slack is other tiny calls). Optimizing crunch by even $2 \times$ saves $\approx 445$ ms; optimizing handle's body saves $\leq 5$ ms. ✓ Always sort by self time to find hotspots.

Case J — real-world word problem

Worked example Ex 9 · cell J

Your checkout page loads in 1.2 s. Traces show: 720 ms rendering the product list, which fires 1 list query + 40 per-item queries (an N+1 pattern), each query ~16 ms; the other 480 ms is unrelated JS. You batch the 41 queries into 1 join taking 20 ms. What's the new page time and overall speedup?

Forecast: Translate the English into $p$ and $s$ before computing.

Isolate the optimizable part's time. Why? The 720 ms rendering isn't all fixable — only the query time inside it is. $41 \times 16 = 656$ ms of queries → 20 ms after batching.
Compute the part's speedup $s$ . Why? $s$ = old part time ÷ new part time. $s = \frac{656}{20} = 32.8$
Compute $p$ (the query part as a fraction of the whole 1.2 s = 1200 ms). Why this step? Amdahl is scale-free — it needs the fraction the queries occupy, not their raw ms, so we normalize by the total page time. $p = \frac{656}{1200} \approx 0.5467$
Apply Amdahl. Why this formula here? We have a single optimized part ( $p, s$ ) plus an untouched remainder; Amdahl is exactly the tool that folds both into one overall speedup — the untouched $(1 - p)$ JS cannot shrink, so it must appear in the denominator alongside the shrunk query term $p / s$ . $S = \frac{1}{( 1 - 0.5467 ) + \frac{0.5467}{32.8}} = \frac{1}{0.4533 + 0.01667} = \frac{1}{0.47} \approx 2.13 \times$
New page time. Why? By definition $T_{new} = T_{old} / S = 1200 \times (1/ S)$ . So $1200 \times 0.47 \approx 564$ ms.

Verify (units too): new time directly $= (1200 - 656) + 20 = 544 + 20 = 564$ ms; $1200/564 = 2.13$ . ✓ ms in, ms out. Fixing the N+1 Query Problem roughly halved the page — this is why I/O and query batching beat micro-optimizing JS here. Consider also Caching Strategies to remove the query entirely.

Case K — exam twist: chaining two optimizations

Worked example Ex 10 · cell K

Exam-style. A job splits as: serialization 50%, DB I/O 30%, logging 20%. You apply two fixes at once: serialization $2 \times$ faster, DB I/O $3 \times$ faster. Logging untouched. Overall speedup?

Forecast: You cannot just add two speedups. Guess whether $S$ lands nearer 1.5 or 3.

Generalize Amdahl to multiple parts. Why this step? The single-part formula only shrinks one term; here two independent parts shrink at once, so we go back to first principles — new total time is the sum of every part's post-fix share $p_{i} / s_{i}$ (untouched parts keep $s_{i} = 1$ ). $T_{new} = \sum_{i} \frac{p _{i}}{s _{i}}, S = \frac{1}{T _{new}}$
List the terms. Why tabulate? Keeps signs and untouched parts ( $s = 1$ ) straight so nothing is dropped.
- Serialization: $p = 0.5, s = 2 \Rightarrow 0.5/2 = 0.25$
- DB I/O: $p = 0.3, s = 3 \Rightarrow 0.3/3 = 0.10$
- Logging: $p = 0.2, s = 1 \Rightarrow 0.2/1 = 0.20$
Sum and invert. Why? The summed $T_{new}$ is the new total time (old time normalized to 1); its reciprocal is the overall speedup $S$ . $T_{new} = 0.25 + 0.10 + 0.20 = 0.55, S = \frac{1}{0.55} \approx 1.82 \times$
Interpret the ceiling. Why this step? To know what to profile next, ask what caps you: untouched logging (0.20) can never shrink, so even infinite speedups on the other two give $S_{m a x} = 1/0.20 = 5 \times$ . Logging is now the next target.

Verify: $0.25 + 0.10 + 0.20 = 0.55$ ; $1/0.55 = 1.818$ . ✓ And the single-part formula is just this sum with one non-unit $s$ — the two are consistent.

Recall Which time do you sort by to find hotspots?

Self time — why? ::: Self (exclusive) time is the work inside a function; cumulative can be inflated by a thin wrapper whose children do the real work (Ex 8).

Recall Wall 10 s, CPU 1 s — I/O or CPU bound, and does the CPU profiler help?

Answer ::: I/O-bound (10% utilization); the CPU profiler explains only 1 s and effectively lies — profile I/O instead (Ex 7).

Recall Which hotspot fraction needs the

most samples for a fixed accuracy? Answer ::: $p = 0.5$ , because $p (1 - p)$ peaks there; size $N$ for that worst case and you never under-sample (Ex 5b).

Recall Why can't you trust a sampled function that got 3 of 1000 hits?

Answer ::: Its relative standard error is ~58% — tiny slices are where sampling is least reliable (Ex 6).