Building a portfolio and research roadmap: Foundations

The parent note throws around a lot of words as if you already know them: artifact, hypothesis, milestone, success metric, reproduction, extension, ablation, FLOPs, BLEU, 80/20, feedback loop, phase. This page defines every single one from absolute zero, in the order that each one leans on the one before it. Nothing later uses a word we have not already earned.

Parent topic: Building a Portfolio and Research Roadmap.

Layer 0 — The words that describe doing science

1. Artifact

The picture. Imagine an empty shelf. Every time you finish something and put it out in the world, one solid object appears on the shelf. Your portfolio is literally that shelf, not a list of things you intend to do.

Why the topic needs it. The parent note says a portfolio is "evidence." Evidence has to be a thing someone else can pick up and inspect. That thing is an artifact.

2. Hypothesis

The picture. Think of a hypothesis as a line drawn in the sand before you run the test. When results come in, they land on one side of the line or the other — you were right, or you were wrong. If a statement is so vague that no result could contradict it, it is not a hypothesis.

Why the topic needs it. An extension project (Pillar 2) is exactly "form a hypothesis, then test it." Without the line-in-the-sand, you cannot tell whether your project succeeded.

Figure — Building a portfolio and research roadmap

Layer 1 — Measuring things: metrics

Before we can say a project "worked," we need numbers that go up or down. Here are the exact ones the parent note uses.

3. Accuracy and Top-1 accuracy

The picture. A model looks at $100$ images. It puts a checkmark or a cross on each. Accuracy $= \frac{checkmarks}{100}$ . The parent's " $92.3%$ " just means $92.3$ checkmarks per $100$ .

Why " $92.3%$ vs $91.7%$ " matters. The parent's whole "extension" example hangs on a $0.6%$ gap. That is the difference in checkmark counts between two models — small, but real.

4. FLOPs

The picture. Picture the model as a factory. Each little arithmetic step is one turn of one gear. FLOPs is the total number of gear-turns to push one image through. More gear-turns = slower and more expensive.

Working the parent's numbers. The parent's extension example gives us two measurements to compare:

Old value (baseline, full attention): $0.8$ GFLOPs per image.
New value (treatment, local attention): $0.5$ GFLOPs per image.

To turn " $0.8$ down to $0.5$ " into a percentage saved, we ask: what fraction of the old cost did we get rid of? The fraction we still pay is $\frac{new}{old} = \frac{0.5}{0.8}$ , so the fraction we saved is one minus that:

$compute saving = 1 - \frac{new}{old} = 1 - \frac{0.5}{0.8} = 0.375 = 37.5%.$

5. BLEU

The picture. Lay your machine's sentence on top of the human's sentence. BLEU counts how many little sliding windows of words match (part 1), then docks points if your sentence is too short (part 2). The parent's "Transformer is 2–3 BLEU points better" means its sentences overlap the human reference by a few more points after that length check.

Layer 2 — Studying trade-offs: ablation

6. Ablation

The picture. You have a machine with knobs. You freeze every knob except one, turn only that knob, and watch the output meter. Then reset and try the next knob. Because only one thing changed, whatever moved the meter must be caused by that one knob.

Why the topic needs it. The parent's extension example says "what if window size $= 3$ ? $5$ ? $9$ ?" — that is an ablation on the window-size knob. It is how you turn a single result into a scientific claim about cause and effect.

Layer 3 — The three portfolio project types

Now that we can measure and ablate, we can name the three shapes a project can take.

7. Reproduction

The picture. Someone hands you a photo of a finished cake and a rough recipe. You bake it yourself. If your cake matches the photo, you have proven you understand the recipe — including the steps the recipe quietly left out.

Why first? It is the safest project: the "right answer" already exists, so a gap between your number and theirs is a bug you can hunt. This is why the parent makes Phase 1 a reproduction.

8. Extension

The picture. Same cake — but now you swap butter for oil on purpose to test a guess ("oil makes it moister"). You taste-test (metric) and try three oil amounts (ablation). This is real science because the outcome was unknown before you tried.

Notice this definition reuses hypothesis (Layer 0), metric (Layer 1), and ablation (Layer 2). Each layer stacks on the last.

9. Explanation

The picture. A reproduction proves you can build; an extension proves you can discover; an explanation proves you can transfer what you know into someone else's head. Three different muscles, three different artifacts.

Layer 4 — Planning: the roadmap vocabulary

10. Milestone

The picture. On a long road, milestones are the numbered stones you pass. They tell you both how far you've come and whether you're on pace. A goal with no date is a wish, not a milestone.

11. Success metric

The picture. It is a finish-line tape stretched across the track before the race. When you cross the line (or don't), there is no arguing — you either hit the number or you didn't. This connects straight back to hypothesis (Layer 0): both are lines drawn in advance.

12. Feedback loop

The picture. Think of steering a bicycle. You do not compute the perfect path once; you constantly look, correct, look, correct. The roadmap is a bicycle, not a train track: the parent calls each project "a hypothesis about your learning path that you continuously update with evidence."

13. Phase, and the roadmap formula

The parent writes: $Roadmap = {Phase_{i}}_{i = 1}^{N} .$

Let us earn every symbol in that line.

What each of the five slots means (all carry the same subscript $i$ , meaning "for phase number $i$ "):

Definition The five slots of a Phase

== $Goal_{i}$ == — the "why" of the phase, stated in plain words: the capability you want to gain (e.g. "go from training in Keras to implementing a paper from scratch"). Picture: the destination sign at the far end of this stretch of road.
== $Deliverable_{i}$ == — the single artifact the phase must produce (e.g. "an open-source ResNet-50 reproduction"). Picture: the object that lands on your portfolio shelf when the phase ends. This is what stops a phase from being a vague wish.
== $Skills_{i}$ == — the specific abilities you will actually practise during the phase (e.g. "PyTorch autograd, custom datasets, learning-rate schedules") — the vital few, not the whole textbook. Picture: the small set of tools you pack for this leg of the trip.
== $Duration_{i}$ == — the time budget, a fixed number of weeks or months (e.g. "3 months"). Picture: a countdown timer on the phase; time-boxing stops the work from expanding forever.
== $Success Metric_{i}$ == — the pass/fail number decided in advance (defined in slot 11 above), e.g. "match the paper's accuracy within $0.5%$ ." Picture: the finish-line tape for this phase.

Why a formula for a life plan? It is not maths for its own sake — it is a checklist that guarantees no phase is vague. If any of the five slots is empty, that phase is not ready. The subscript $i$ just lets us talk about "any phase" without writing them all out.

Layer 5 — The 80/20 idea

14. The 80/20 rule (Pareto)

The picture. Picture a bar chart of "topics you could study," sorted by usefulness. The first few bars are tall (huge payoff); a long tail of tiny bars follows. The 80/20 move is to spend your time on the tall bars only. Research is infinite; your time is not.

Why the topic needs it. The parent's roadmap deliberately lists "the $20%$ of skills you'll actually practice, not everything in the textbook." Without 80/20, you drown.

How the foundations feed the topic

Read it top to bottom: the small vocabulary bricks (left) assemble into the two big machines — Portfolio and Roadmap — which together aim at the one goal.

Equipment checklist

Cover the right side and answer each. If any is fuzzy, re-read that layer above.

What makes something an artifact rather than just an idea?

It is finished and shareable — someone else can pick it up and inspect it (a public repo, a post, a plot).

What is the one property every good hypothesis must have?

It must be falsifiable — stated clearly enough that an experiment could prove it wrong.

Accuracy measures ___, FLOPs measures ___, BLEU measures ___.

quality of predictions; compute cost (arithmetic steps); translation-overlap quality.

Why report FLOPs instead of seconds?

FLOPs is hardware-independent, so results are reproducible and comparable across machines.

What are the two moving parts of BLEU, and why does the second one matter?

N-gram overlap plus a brevity penalty; the penalty stops a too-short translation from scoring high, which is one reason BLEU behaves counterintuitively at the edges.

What is the golden rule of an ablation?

Change one knob at a time so any change in the metric is caused by that one knob.

Order the three project types by what they prove.

Reproduction proves you can build; extension proves you can discover; explanation proves you can teach.

{Phase_{i}}_{i = 1}^{N}

, what do

i

and

N

mean?

i

is the phase counter/label;

N

is the total number of phases — "run from phase 1 to the last."

Name the five slots inside one Phase tuple, and one sentence for each.

Goal (the why/target capability); Deliverable (the artifact produced); Skills (the vital few abilities practised); Duration (the time budget); Success Metric (the pass/fail number set in advance).

What distinguishes a milestone from a wish?

A milestone is dated — it says what by when; a wish has no deadline.

What is a success metric and when is it fixed?

The number that decides pass/fail, decided in advance of running the experiment.

What shape is a feedback loop, and what turns each cycle?

A cycle: do → measure → learn → adjust → do again; the output of one round feeds the next.

State the 80/20 idea in one line.

Roughly

20%

of efforts produce roughly

80%

of results — so do that vital

20%

Once these foundations are solid, the kinds of problems worth putting in your portfolio come from:

6.5.1-Interpretability-and-explainability — the "interpretability angle" for extensions.
6.5.2-Fairness-and-bias-in-AI — the "data angle" (does the model fail on some groups?).
6.5.3-Federated-learning — a rich reproduction/extension playground.
6.5.8-AI-safetyand-alignment — high-impact research directions.
6.5.10-Open-problemsin-AI-research — where to point your roadmap's later phases.