Contributing to open-source ML: Question bank

Related deep skills: Code Review Best Practices, Testing ML Systems, Version Control with Git, Software Engineering for ML, ML Research Paper Implementation.

Related deep skills: Testing ML Systems, Software Engineering for ML.

Two trade-offs, visualised first

Before the cards, look at the two pictures the hardest questions are really about.

Figure 1 draws $Q_{PR}$ (just defined above) as four vertical switch-bars along the x-axis, one per factor ( $C_{correct}$ , $C_{clear}$ , $C_{tested}$ , $C_{justified}$ ); the y-axis is the switch value $0$ or $1$ . Inspect the red bar: three switches are green ( $1$ ) but the "tested" switch is red ( $0$ ), and the title shows $1 \times 1 \times 0 \times 1 = 0$ . The lesson to read off the picture: excellence in three columns cannot rescue a single failed column, because we multiply rather than add.

Figure 2 plots the checkpointing trade-off against the checkpoint interval $k$ on the x-axis. Inspect the blue curve (left y-axis, memory units): saved memory $O (\frac{L}{k} \cdot B \cdot H)$ falls as $k$ grows. Inspect the red curve (right y-axis, compute units): the compute cost rises toward $2 L$ as $k$ shrinks toward $1$ ; the dashed yellow line marks the normal "store everything" memory level $\sim L$ . Why does the total work reach $O (2 L)$ ? The forward pass runs once ( $O (L)$ work). Normally the backward pass reuses stored activations, so no recompute is needed — the backward pass's own gradient arithmetic is the same $O (L)$ whether or not we checkpoint, so it is not the thing that changes and is not counted twice here. The only extra cost checkpointing adds is re-running the forward computation of the discarded layers to rebuild their activations — a second forward-style pass of $\approx O (L)$ . Adding that recompute to the original forward pass gives $O (L) + O (L) = O (2 L)$ : that is the origin of "roughly doubles the time."

True or false — justify

Bigger, more sophisticated contributions are always more valuable than tiny ones.

False. A one-line documentation fix that unblocks thousands of confused users can outweigh a clever feature nobody asked for; value is measured by usefulness to real users, not lines of code.

You should start coding on the main branch of your fork so your work stays visible.

False. Always branch (see Version Control with Git). main should mirror upstream so you can sync and start clean features; committing straight to it entangles unrelated work and makes rebasing painful.

A pull request that passes all existing tests is ready to merge.

False. Existing tests only cover old behaviour. New behaviour needs new tests; without them your fix can silently regress the moment someone else edits the file.

If your code is correct, clear documentation is optional.

False. In the PR quality product

Q_{PR} = C_{correct} \times C_{clear} \times C_{tested} \times C_{justified}

, an unclear PR scores

C_{clear} = 0

and the whole product collapses — reviewers who can't understand it can't confidently merge it.

Forking a repository automatically gives you permission to merge into the original project.

False. A fork is your personal copy; merge rights belong only to maintainers of the upstream repo. You propose; they dispose.

Gradient checkpointing makes training faster.

False. It trades compute for memory: it recomputes the discarded activations in the backward pass, so time rises from

O (L)

to about

O (2 L)

(one extra forward-style pass), while memory drops from

O (L \cdot B \cdot H)

O (\frac{L}{k} \cdot B \cdot H)

, where

L

=layers,

B

=batch size,

H

=hidden dimension,

k

=checkpoint interval.

Claiming an issue before coding is just polite etiquette with no practical benefit.

False. It prevents duplicate work and confirms maintainers actually want the change before you spend hours — a "wontfix" reply after a big PR is the worst outcome.

Once a PR gets review comments, revisions mean you did something wrong.

False. Review is where most learning happens; iteration is the norm even for expert contributors. Comments are teaching, not a verdict on your worth.

Spot the error

"I'll benchmark my feature on my laptop CPU and report the speedup." — what's wrong?

A memory/speed feature like checkpointing must be measured in its real regime (large model, GPU). Laptop-CPU numbers don't reflect where the tradeoff actually matters and can mislead reviewers.

A contributor has two 1-D tensors, a of shape (3,) and b of shape (3,), and writes dot = (a * b).sum(dim=1). Why does this break?

Both tensors have rank 1 — a single axis indexed as dim=0. The code sums over dim=1, a batch axis that doesn't exist, so it raises an "IndexError/expected 2D" symptom. Fix: either unsqueeze(0) to reshape each to (1, 3) and keep dim=1, or sum over dim=0 for genuinely 1-D inputs (see Testing ML Systems).

Someone divides by the norm as dot / (norm1 * norm2) with no eps. Where does this fail?

On a zero vector the norm is

0

, giving division by zero (NaN/Inf). Clamping the denominator with a small floor eps (e.g. .clamp(min=1e-8)) keeps it finite. The scale

1 0^{- 8}

is chosen well below any real vector norm yet safely above float32's smallest normal value (

\approx 1.2 \times 1 0^{- 38}

), so for normal inputs the clamp never fires and the result is undistorted; it only rescues the degenerate zero case.

"My PR is huge — 40 files changed — because I also reformatted the whole module." What's the mistake?

Mixing unrelated changes (a fix + a mass reformat) makes review nearly impossible and buries the real diff. Keep one PR to one logical change.

A test reads assert not result.isnan(). Why is this a weak test?

It only checks the output isn't broken, not that it's correct. A good test compares against a hand-computed expected value, e.g.

32/ (1477)

for that cosine example.

"I picked TensorFlow's C++ backend as my first project because it's the most impressive." Diagnose.

Violates the RAMP Read-comfortably rule (see the RAMP mnemonic above). A codebase you can't yet comprehend produces slow, error-prone contributions; start where you can read and reason fluently.

A PR description says only "fixes bug." What's missing?

The justification and context: which issue, what the bug was, why this fix, and how it was tested. Reviewers need the why, not just the what (see Technical Writing for ML).

Why questions

Why is the PR quality formula a product rather than a sum?

Because the factors are non-substitutable: being extra correct can't compensate for being untested. A product collapses to

0

if any factor is

0

, correctly modelling "one fatal flaw kills the PR"; a sum would let strengths mask a fatal gap.

Why do maintainers value tests even more than the fix itself?

The fix solves today's bug once; the test prevents regression forever, guarding the code against future edits by people who never saw the original bug.

Why does documentation-fixing "force you to understand internals" despite needing little code?

To explain a parameter clearly you must read the source and grasp the real tradeoff (e.g. that max_features='sqrt' trades predictive power for decorrelation) — you can't fake understanding when writing an explanation.

Why prefer a project you already use for your first contribution?

You already carry domain context and know the pain points from a user's view, so you spot real issues and understand the API's intent without a long ramp-up (see Building ML Portfolios).

Why is doubling compute an acceptable price for gradient checkpointing on large models?

Because for big models memory, not compute, is the bottleneck — spare GPU cycles are cheap, but running out of VRAM stops training entirely. Fitting a larger batch is worth the extra time.

Why link the original issue in your PR description?

It gives reviewers the full context and prior discussion, proves the change was wanted, and lets the issue auto-close on merge — tying the conversation to the code.

Edge cases

What happens if you submit a PR for an issue nobody claimed but two people were quietly working on?

Only one gets merged; the other's effort is wasted. This is exactly the duplicate-work scenario that public claiming prevents — comment "I'll take this" first.

Your fix works, but the project supports a Python version where your syntax is invalid. What now?

You've broken backwards compatibility. Contributions must respect the project's declared support matrix; CI will fail, and you'll need to rewrite using compatible constructs.

You benchmark checkpointing with

k

(checkpoint_every) equal to the number of layers

L

. What's the degenerate result?

With

k = L

you checkpoint only the input (about

L / k \approx 1

saved slice), so memory savings are maximised but you recompute nearly everything — full second forward pass, so compute roughly doubles. The opposite extreme

k = 1

checkpoints every layer, giving no recompute and no memory savings. The knob spans that whole spectrum.

Cosine similarity of a vector with itself — what should the code return, and does the eps clamp distort it?

It should return

1.0

(a vector is perfectly aligned with itself). Since a nonzero self-norm greatly exceeds eps (

1 0^{- 8}

), the clamp never activates and the answer is undistorted; eps only guards the zero-vector corner.

The good-first-issue you picked turns out to require redesigning a core API. Should you push through?

No — the "good first issue" label was mislabelled. Flag it to maintainers and choose a truly scoped task; heroic scope-creep on a first PR usually stalls in review.

Two reviewers give contradictory change requests on your PR. What's the right move?

Don't silently pick one. Surface the conflict in the thread and ask maintainers to decide the direction — resolving disagreement is their call, and it keeps your PR from ping-ponging (see Code Review Best Practices).

Recall One-line self-test

The four PR factors, in order ::: Correct, Clear, Tested, Justified — multiplied, so any zero sinks it. Checkpointing trades ___ for ___ ::: Compute for memory (recompute activations instead of storing them). The symbols in $O (\frac{L}{k} B H)$ ::: $L$ layers, $k$ checkpoint interval, $B$ batch size, $H$ hidden dimension.