Building a portfolio and research roadmap: Exercises

Before we start, one symbol we will reuse. When we compare "how much you gain for what you give up", we use a ratio:

Definition Ratio (gain per cost)

A ratio $\frac{a}{b}$ just means "how many units of $a$ do I get for one unit of $b$ ". If a project saves you $37.5%$ compute for $0.6%$ accuracy lost, the ratio $\frac{37.5}{0.6} \approx 62.5$ reads "sixty-two-and-a-half percent of compute saved for every one percent of accuracy given up". Bigger ratio = better trade. We will lean on this a lot, so it is defined once here and reused everywhere below.

Edge case — cost $b = 0$ . If accuracy lost is exactly $0$ , the ratio $\frac{a}{0}$ is undefined — informally "infinite". This is not a mathematical bonus; it is a signal that the ratio has stopped being meaningful. A trade that costs nothing is not "infinitely good", it is a free win you should just take, and you can no longer use the ratio to rank it against other options (everything divided by zero blows up). Practical rule: when a cost rounds to zero, compare by absolute gain instead — grab the free win, then rank the remaining paid options by their (finite) ratios.

Level 1 — Recognition

Exercise 1.1

Which of the following is a portfolio artifact and which is a roadmap element? Classify each: (a) "Reproduce ResNet-50 by Month 3", (b) a GitHub repo with a narrative README, (c) a blog post explaining InfoNCE, (d) "Master attention mechanisms in 6 weeks".

Recall Solution

What we are doing: separating evidence you have built (portfolio) from plans you intend to build (roadmap). Why: the parent note's core split is "Portfolio as Evidence, Roadmap as Strategy". Anything time-bound and future-facing is roadmap; anything that already exists as a concrete object is portfolio.

(a) Roadmap — it is a time-bound goal, not a finished thing.
(b) Portfolio — a concrete artifact (a repo).
(c) Portfolio — a concrete artifact (an explanation post).
(d) Roadmap — a skill milestone with a duration. Pattern: if it has a deadline attached, it is roadmap; if you can click a link and see it, it is portfolio.

Exercise 1.2

The parent note lists three pillars of a strong portfolio. Name them and give the one-line "what it proves" for each.

Recall Solution

Reproductions — prove you can implement state-of-the-art faithfully.
Extensions — prove you can generate novel insight (do science: hypothesis → experiment → interpretation).
Explanations — prove you can teach (Feynman: teaching forces deep understanding). Mnemonic: R-E-E — Rebuild it, Extend it, Explain it.

Level 2 — Application

Exercise 2.1

In the Efficient-Attention extension example: baseline is $92.3%$ accuracy at $0.8$ GFLOPs/img; treatment is $91.7%$ at $0.5$ GFLOPs/img. Compute (a) the compute savings as a percent, (b) the accuracy lost in percentage points, (c) the "savings-per-point-lost" ratio.

Recall Solution

(a) Compute savings. We saved $0.8 - 0.5 = 0.3$ GFLOPs out of the original $0.8$ . $\frac{0.3}{0.8} = 0.375 = 37.5%.$ Why divide by the original? Percent-saved always means "fraction of what you started with", so the denominator is the baseline $0.8$ , not the new $0.5$ . (b) Accuracy lost. $92.3 - 91.7 = 0.6$ percentage points. (Note: "points" is the raw difference of two percentages — not a percent-of-a-percent.) (c) Ratio. Using our defined gain-per-cost ratio: $\frac{37.5}{0.6} = 62.5.$ Read: $62.5%$ of compute saved for every $1$ point of accuracy surrendered — an excellent trade.

Exercise 2.2

A roadmap has $N = 4$ phases, each with duration $3$ months. The Roadmap formula from the parent is a set ${Phase_{i}}_{i = 1}^{N}$ . (a) What is the total planned duration? (b) If Phase 2 slips by $50%$ of its duration, and you refuse to move the final deadline, by what percent must you compress the remaining phases' combined time?

Recall Solution

(a) Total duration. $4 \times 3 = 12$ months. (This is why the parent calls it a "12-Month Example".) (b) The slip. Phase 2 grows by $50%$ of $3 = 1.5$ months, so it now takes $4.5$ months. Phases 1, 3, 4 originally take $3 + 3 + 3 = 9$ months combined. To keep the total at $12$ with Phase 2 now eating $4.5$ , the other three must fit into $12 - 4.5 = 7.5$ months. Compression needed: $\frac{9 - 7.5}{9} = \frac{1.5}{9} = 0.1 \overline{6} \approx 16.7%.$ Why is the denominator the original $9$ months, not the new $7.5$ ? This is the same logic as percent-saved in Exercise 2.1: "how much did I have to shrink" is always measured as a fraction of what I started with. I started with $9$ months of work for those three phases; I must remove $1.5$ of them; so the shrink is $1.5$ out of the original $9$ . Dividing by $7.5$ (the new value) would answer a different, misleading question ("by how much would I have to grow $7.5$ back up to $9$ ") and inflate the number to $20%$ . Interpretation: a single $1.5$ -month slip forces a $\approx 16.7%$ squeeze on everything else — this is exactly why the parent insists on Duration + monthly reflection checkpoints. Time-boxing is not bureaucracy; slips are contagious.

Level 3 — Analysis

Exercise 3.1

You have three candidate extension projects. Rank them by the gain-per-cost ratio and say which you'd pick.

Project	Compute saved	Accuracy lost (pts)
A (window=7)	$37.5%$	$0.6$
B (window=3)	$55%$	$2.2$
C (window=9)	$22%$	$0.1$

Recall Solution

Compute each ratio (savings per point lost): $A = \frac{37.5}{0.6} = 62.5, B = \frac{55}{2.2} = 25, C = \frac{22}{0.1} = 220.$ Ranking: $C (220) > A (62.5) > B (25)$ . But ratio is not the whole story. C loses almost nothing ( $0.1$ pt) — if your goal is "publishable efficiency result", C's tiny accuracy cost is safest to defend. (Note how C's tiny denominator already pushes its ratio toward the "blows up" regime warned about in the ratio definition — at exactly $0$ loss the ratio would be undefined and you'd just take the free win.) B saves the most raw compute but pays $2.2$ points — only worth it if you are compute-starved and can tolerate the drop. Decision: pick C for a clean, defensible headline; keep B as an ablation point to show the whole trade-off curve (the parent literally says "what if window = 3? 5? 9?" — you need all of them for the ablation figure).

Now read the trade-off picture below.

Figure — Building a portfolio and research roadmap

Intuition How to read the trade-off figure

The axes. The horizontal axis is accuracy lost in percentage points — moving right means the project sacrifices more accuracy (worse). The vertical axis is compute saved as a percent — moving up means the project frees more compute (better). So the ideal corner is the top-left: lots saved, little lost. The yellow arrow points exactly that way. The three dots. Each colored dot is one project, placed at its (loss, saving) coordinates, and its label shows the ratio = saving ÷ loss (the slope of the line from the origin to that dot). Green C sits far up-left with a near-zero horizontal position — that is why its ratio ( $220$ ) is huge: it is almost against the left wall. Blue A is a solid middle trade ( $62.5$ ). Red B is high up (saves the most raw compute) but pushed far right (costs $2.2$ pts), so despite its height its ratio ( $25$ ) is the worst — being far right drags a point away from the good corner faster than height helps. The takeaway the picture makes obvious: "highest saving" (B, tallest dot) is not the same as "best trade" (C, most top-left dot). The eye sees the trade-off that a single number can hide.

Exercise 3.2

Match each portfolio "angle" from the parent to the vault topic it most naturally serves: "visualize what each GAN layer learned"; "does this model fail on low-resource medical images and why"; "can this model train without centralizing user data".

Recall Solution

What we are doing: connecting the three idea-generation angles (interpretability / data / efficiency) to research subfields.

"Visualize what each layer learned" → the interpretability angle → 6.5.1-Interpretability-and-explainability.
"Fails on low-resource medical images and why" → a data + fairness question (who does the model underserve?) → 6.5.2-Fairness-and-bias-in-AI.
"Train without centralizing user data" → the privacy/distributed angle → 6.5.3-Federated-learning. Why this matters: a portfolio project that plugs into a live research frontier (see 6.5.10-Open-problemsin-AI-research) is far more "credible through specificity" than a generic MNIST classifier.

Level 4 — Synthesis

Exercise 4.1

Design a 3-phase, 6-month roadmap whose end deliverable is one publishable extension. For each phase give all five fields the parent's formula requires: Goal, Deliverable, Skills, Duration, Success Metric.

Recall Solution

Recall the phase tuple: $Phase_{i} = (Goal_{i}, Deliverable_{i}, Skills_{i}, Duration_{i}, Success Metric_{i})$ .

Phase 1 (Months 1–2): Reproduce.

Goal: faithfully rebuild a recent efficient-attention paper.
Deliverable: open-source repo matching reported CIFAR-10 accuracy within $0.5$ pts.
Skills: PyTorch nn.Module, custom attention masks, FLOP counting via fvcore.
Duration: $2$ months.
Success Metric: my accuracy is within $0.5$ pts of the paper's.

Phase 2 (Months 3–4): Extend.

Goal: test whether a smaller local-attention window keeps most accuracy.
Deliverable: ablation over windows ${3, 5, 7, 9}$ with a trade-off plot.
Skills: experiment design, controlling confounds, reading a Pareto curve.
Duration: $2$ months.
Success Metric: at least one window gives $\geq 30%$ compute saving for $\leq 1$ pt loss.

Phase 3 (Months 5–6): Explain & ship.

Goal: communicate the finding to a broad audience.
Deliverable: blog post + Colab + a short workshop-style writeup.
Skills: technical writing, t-SNE visualisation, "mistakes-first" pedagogy.
Duration: $2$ months.
Success Metric: one external reader reproduces my headline number from the Colab.

Consistency check: $2 + 2 + 2 = 6$ months total ✓, and every phase names a concrete deliverable + an objective metric — exactly what prevents the "vague I'll learn CNNs" failure mode the parent warns against.

Exercise 4.2

Your Phase 2 metric was " $\geq 30%$ compute saving for $\leq 1$ pt loss." Project A (from Ex 3.1) gives $37.5%$ for $0.6$ pt. Does A pass the metric? Show the two inequalities.

Recall Solution

Metric is a conjunction (both conditions must hold): $37.5 \geq 30 ✓ and 0.6 \leq 1 ✓ .$ Both true, so A passes. This is the value of an objective success metric: no hand-waving — a boolean settles it.

Level 5 — Mastery

Exercise 5.1

A recruiter says: "Your portfolio has ten MNIST classifiers." Steel-man why that is weak, then rewrite the portfolio into three stronger artifacts using the R-E-E pillars — and justify each choice against a specific vault frontier.

Recall Solution

Why ten MNIST classifiers is weak: it demonstrates repetition, not range. Ten copies of the same easy task prove one skill (basic training) ten times — zero marginal evidence of rigor, novelty, or communication. "Credibility through specificity" means depth on hard, specific problems, not breadth of trivial ones. Rewrite (R-E-E):

Reproduce a recent efficient-Transformer paper end-to-end with a LESSONS.md. Proves implementation depth. Ties to 6.5.10-Open-problemsin-AI-research (scalable architectures is an open frontier).
Extend it with an interpretability probe: visualise which attention heads a fairness-sensitive input activates. Proves you do science. Ties to 6.5.1-Interpretability-and-explainability and 6.5.2-Fairness-and-bias-in-AI.
Explain the whole thing as a mistakes-first blog + Colab. Proves teaching. Adds an 6.5.8-AI-safetyand-alignment "what could go wrong if this is deployed" section for scope. Net effect: one deep, connected thread beats ten shallow copies — a reviewer sees a researcher, not a tutorial-follower.

Exercise 5.2

Two candidates apply. Candidate X: $8$ reproductions, $0$ extensions, $0$ explanations. Candidate Y: $2$ reproductions, $2$ extensions, $2$ explanations. Using only the "three pillars proves three distinct capabilities" idea, argue quantitatively (via distinct capabilities covered) who is the stronger research candidate, and name the one capability X entirely lacks.

Recall Solution

Capability count. Each pillar proves a distinct capability: implement / discover / teach.

X covers ${implement}$ → 1 of 3 distinct capabilities (repeated 8 times = no extra coverage).
Y covers ${implement, discover, teach}$ → 3 of 3 distinct capabilities. Coverage ratio: X $= 1/3 \approx 33.3%$ ; Y $= 3/3 = 100%$ . Verdict: Y is stronger — research demands all three, and Y demonstrably shows all three. The capability X entirely lacks: generating novel insight (the discover/extension pillar). X can rebuild what exists but has shown no evidence of asking a new question — the single most important trait for a researcher (versus an implementer).

Recall Self-Test Cloze Review

A portfolio proves capability through specificity, not breadth of trivial work ::: yes — depth on hard problems beats many easy copies. The three pillars are Reproductions, Extensions, Explanations ::: Rebuild it, Extend it, Explain it (R-E-E). A roadmap is best understood as a hypothesis about your learning path that you update with evidence ::: not a rigid contract. "Savings-percent" divides the saved amount by the baseline (starting) value ::: not by the new value. Percentage points are computed by plain subtraction of two percentages ::: not by a percent-of-a-percent.