Two words we lean on throughout, defined up front:
Before the questions, three pictures carry the ideas the text keeps referring to. Each has a short verbal walkthrough so you can follow it even without seeing the image clearly.
Figure 1 — The three pillars, as a cycle. Three circles read left to right: Reproduce (black, "prove you can IMPLEMENT"), Extend (drawn in red — the key object — "prove you can do SCIENCE"), and Explain (black, "prove you can TEACH"). Curved black arrows loop Reproduce → Extend → Explain → back to Reproduce, with a caption noting that each project feeds the next: what you extend, you then explain, and explaining reveals the next thing to reproduce. Read it as a loop, not a checklist — a portfolio missing any circle leaves that skill unevidenced.
Figure 2 — The accuracy-vs-compute tradeoff. A paired bar chart with two groups on the x-axis: "Baseline (full attention)" and "Efficient (local + strided)". For each, a black bar shows top-1 accuracy (92.3% vs 91.7%) and a red bar shows compute in GFLOPs, scaled ×100 (0.8 vs 0.5). Red annotations point out the accuracy drops only 0.6% while compute falls 37.5% — the picture's whole message is that a tiny accuracy loss can buy a large compute saving, so it is a win.
Figure 3 — A roadmap phase with a real checkpoint. A stacked list showing one phase's fields — Goal, Deliverable, Skills (the 20%), Duration — in black, and at the bottom a red-outlined box holding the Success metric: "CIFAR-10 top-1 within 1% of published ResNet-50, repo public, ≥3 documented bugs." A red arrow labels it "checkable at the checkpoint". The point of the picture: a good phase ends in something you can pass or fail, not a vague "get good".
TF — "A longer portfolio (more projects) is always more impressive."
False. A portfolio is read like a first impression, not counted; five clear projects beat twenty half-finished ones because each weak item lowers the average signal a reviewer infers about your judgement.
TF — "If my reproduction matches the paper's numbers, the reproduction is done."
False. Matching numbers is necessary but the value is in the documented narrative — the bugs you hit and why — because that is what proves understanding rather than luck.
TF — "A reproduction that fails to match the paper is a wasted project."
False. A carefully documented negative result ("I could not reproduce X and here is exactly where it diverged") is a genuine scientific contribution and often more respected than yet another success.
TF — "An extension is only worthwhile if the modification beats the baseline."
False. An extension's worth is the cleanly-run experiment, not the outcome; a well-designed test showing your idea does NOT help still demonstrates you can do science (this is the Extend circle in Figure 1).
TF — "Writing an explanation post is just self-promotion, not research skill."
False. Teaching a concept clearly forces you to expose gaps in your own understanding (the Feynman effect — explaining reveals what you don't really know), so a good explanation is evidence of depth, not a substitute for it.
TF — "A roadmap should be fixed once written, or it isn't a real plan."
False. A roadmap is a hypothesis about your learning, meant to be updated at reflection checkpoints; a plan that never changes means you stopped incorporating evidence.
TF — "I should master all the maths in the textbook before starting my first project."
False. The roadmap's whole logic is the Pareto 80/20 rule — you learn the ~20% of skills a specific project actually forces you to use, because that unlocks most of your research capability and unmotivated study rarely sticks or produces an artifact.
TF — "Grouping portfolio projects by date is the clearest layout."
False. Lead with your best work grouped by theme; a reviewer spends seconds and should land on your strongest signal, not your oldest attempt.
TF — "Reporting my reproduction's single best test-accuracy run is honest reporting."
False. Cherry-picking the peak run inflates the number and hides variance — report the mean over several seeds (or best and spread), because a reviewer who reruns your code should not find your headline number was a lucky outlier.
Error — "My README says: 'Implemented a Transformer. It works great.'"
The claim is unverifiable and vague — no artifact-level evidence. Fix: show a results table (your BLEU translation score vs. the paper's), training curves, and a concrete bug you fixed, so a stranger can check rather than trust.
Error — "I picked a brand-new, highly complex Vision Transformer variant as my very first reproduction."
Strategic mismatch: too-complex a first target means you can't tell whether a gap is a bug or a misunderstanding. Fix: reproduce a simpler influential paper (e.g. a ResNet variant) first to build a debugging baseline.
Error — "My extension changed the architecture, the optimizer, and the dataset all at once, and accuracy went up."
You've confounded the variables — you cannot attribute the gain to any one change. Fix: change one thing at a time (an ablation), so the experiment actually answers a question.
Error — "Baseline 92.3%, my efficient model 91.7%. I reported only the accuracy drop and called it a loss."
Incomplete accounting: efficiency work trades accuracy for compute, so omitting the FLOPs/time saved hides the actual result (0.6% for 37.5% compute is a win — see Figure 2). Report both axes of the tradeoff.
Error — "I reran my reproduction with 5 random seeds, then reported only the seed that hit the highest test accuracy."
Cherry-picking — one lucky seed is not your model's true performance, so the headline is unreproducible. Fix: report the mean and spread across seeds, so the number survives someone else rerunning it.
Error — "My roadmap phase says: 'Get good at deep learning by Month 6.'"
No deliverable and no success metric — "get good" can't be checked, so you can't tell if the phase succeeded. Fix: name a concrete artifact and an objective checkpoint you either hit or don't (see Figure 3).
Error — "I set the contrastive temperature to a round number and reported the bad accuracy as the method failing."
You blamed the method for a hyperparameter choice — a large temperature τ flattens the loss so gradients vanish, which is a tuning issue, not a fault of contrastive learning. Fix: grid-search the sensitive knob (try τ≈0.07) before concluding.
Error — "I open-sourced the repo but the README is my raw research notebook with commented-out dead code."
Research spaghetti buries the signal; a reviewer bounces. Fix: separate clean, commented code from a narrative README (what it does, results, lessons), because readability is part of the artifact.
Why does "building in public" force deeper learning than private study?
Because you cannot fake a working implementation or a clear explanation — the artifact publicly fails if you misunderstood, so the act of shipping is itself a correctness check.
Why choose a paper less than ~2 years old for a reproduction rather than a classic?
Recent work signals you can operate at the current frontier and the details are often under-documented, which is exactly where the hard, understanding-revealing debugging lives.
Why does a roadmap need reflection checkpoints and not just goals?
Because your estimate of the learning path is a guess made before you learned anything; checkpoints are where you compare predicted vs. actual progress and correct the plan — without them the roadmap can't adapt.
Why is time-boxing each phase (fixed duration) part of the design?
Parkinson's Law — work expands to fill the time available — so a hard duration forces prioritisation and produces a finished deliverable instead of an endless polish loop.
Why do extensions demonstrate research ability more than reproductions do?
A reproduction shows you can implement (the first circle of Figure 1); an extension shows you can formulate a hypothesis, design a test, and interpret the outcome — the full scientific loop rather than just the engineering half.
Why should an explanation post lead with the mistakes the author made?
Because readers share those exact misconceptions; showing the wrong mental model and then why the correct one fixes it teaches far more than presenting the polished truth alone.
Why prioritise the 20% of skills a project needs over covering everything?
The Pareto principle — a small slice of skills unlocks most of your capability — so front-loading that slice gets you to a shippable artifact fastest, and the rest can be learned on demand.
Edge — "My reproduction is 0.5% below the paper. Bug, or fundamental misunderstanding?"
You can't know from the number alone — that's the point of obsessive documentation; isolate one component at a time (unit-test self-attention, check the mask, verify the LR schedule) until the gap is explained rather than guessed.
Edge — "The paper omits the initialization scheme and learning-rate schedule entirely."
This is the normal case, not an obstacle — papers routinely drop 'obvious' details; treat filling these gaps (and recording your choices) as part of the reproduction's value.
Edge — "My extension's result is exactly equal to the baseline — nothing changed."
Still informative: either your modification is a no-op (implementation bug — check it actually fires) or the model genuinely doesn't use the capacity you altered, which is itself an interpretability insight worth writing up.
Edge — "Some of my reproductions succeeded and some failed — how do I present a mixed portfolio?"
Present both openly and label the lesson each taught; a reviewer reads a mix of documented successes and documented failures as evidence of honesty and range, whereas hiding the failures makes the successes look cherry-picked. Group them by theme, not by outcome.
Edge — "An old project no longer represents my skill. Delete it or keep it?"
Archive it, don't hide it — a portfolio is alive; leading with current best work while keeping history available shows growth without diluting your top signal.
Edge — "I have zero compute budget for the full WMT'14 dataset."
Scale down the problem, not the rigor — train on a small dataset (e.g. IWSLT, the compact spoken-language translation benchmark) and compare against a matched baseline, since the reasoning and documentation transfer even when the numbers are smaller.
Edge — "I finished every roadmap phase early. Success?"
Only if the deliverables and success metrics were actually met — finishing early with vague or missing metrics means the phases were under-specified, so tighten the next phase's checkpoints rather than celebrate speed.
Recall Fast self-test
One-word tells of a weak portfolio item? ::: Unverifiable claims, no results table, research-spaghetti code, confounded experiments, and cherry-picked best runs — all mean a reviewer must trust rather than check.
The single sentence that captures the whole page? ::: Reviewers trust artifacts and discount claims, so every project must carry its own inspectable evidence and justified reasoning.
Related traps live in:6.5.1-Interpretability-and-explainability (the "visualize what the model learned" extension angle), 6.5.2-Fairness-and-bias-in-AI and 6.5.3-Federated-learning (data-angle extension ideas), AI safety and alignment, and Open problems in AI research (where to source high-value hypotheses).
