Contributing to open-source ML

What Is Open-Source ML Contribution?

Why this matters: ML research moves fast, but production tools lag behind. Open-source bridges this gap. When you contribute, you:

Force yourself to understand code deeply (can't fake it when submitting a PR)
Learn real-world engineering (testing, API design, backwards compatibility)
Build reputation (your commits are your resume)
Give back (you've used TensorFlow/PyTorch/Scikit-learn—now help the next person)

How to Start Contributing

1. Understanding the Contribution Lifecycle

The journey from idea to merged code:

Find Issue → Claim It → Fork Repo → Create Branch → 
Code + Tests → Submit PR → Code Review → Revisions → Merge

Why each step exists:

Find Issue: Prevents duplicate work, ensures maintainers want the change
Fork Repo: You need your own copy to experiment without breaking main project
Create Branch: Isolates your change, lets you work on multiple features
Code + Tests: Tests prove your change works and won't break in the future
Code Review: Experts catch mistakes, teach you best practices
Revisions: Learning happens here—iterate based on feedback

Worked example Example1: Documentation Fix (Easiest Entry Point)

Problem: Scikit-learn's RandomForestClassifier docs don't explain max_features='sqrt' clearly. Your contribution:

# BEFORE (vague)
max_features : {"sqrt", "log2", None}, int or float
    The number of features to consider when looking for the best split.
 
# AFTER (clear + intuitive)
max_features : {"sqrt", "log2", None}, int or float
    Controls feature subsampling for each split, introducing randomness
    that reduces overfitting:
    - "sqrt": sqrt(n_features) - **recommended default**, balances 
      decorelation and predictive power
    - "log2": log2(n_features) - stronger decorrelation, use when
      features are highly correlated
    - None: all features - no randomness, may overfit
    - int: exact number of features to try
    - float: fraction of features (e.g., 0.3 = 30%)

Why this step? You read the source code to understand what "sqrt" actually does (it's int(np.sqrt(n_features)), then explained the tradeoff (decorrelation vs. power) that helps users decide. Documentation fixes require minimal code knowledge but force you to understand internals.

Worked example Example 2: Bug Fix (Intermediate)

Problem: PyTorch's torch.nn.functional.cosine_similarity crashes with 1D inputs.

Diagnosis:

# User code that crashes
import torch
a = torch.tensor([1., 2., 3.])
b = torch.tensor([4., 5., 6.])
torch.nn.functional.cosine_similarity(a, b)  # RuntimeError: expected 2D
# Root cause: function assumes batch dimension
# cosine_similarity computes: (a · b) / (||a|| ||b||)
# Code does: dot_product = (a * b).sum(dim=1)  # assumes dim1 exists!

Your fix:

def cosine_similarity(x1, x2, dim=1, eps=1e-8):
    # Add: Handle 1D inputs by adding batch dimension
    if x1.ndim == 1:
        x1 = x1.unsqueeze(0)
    if x2.ndim == 1:
        x2 = x2.unsqueeze(0)
    # Original computation
    dot_product = (x1 * x2).sum(dim=dim)
    norm1 = x1.norm(dim=dim)
    norm2 = x2.norm(dim=dim)
    return dot_product / (norm1 * norm2).clamp(min=eps)

Test you must add:

def test_cosine_similarity_1d():
    """Test that 1D inputs work correctly."""
    a = torch.tensor([1., 2., 3.])
    b = torch.tensor([4., 5., 6.])
    result = F.cosine_similarity(a, b, dim=0)
    # Manual calculation: (1*4 + 2*5 + 3*6) / (sqrt(14) * sqrt(77))
    expected = 32. / (torch.sqrt(torch.tensor(14.)) * torch.sqrt(torch.tensor(77.)))
    assert torch.allclose(result, expected)

Why this step? Tests are non-negotiable. They prove your fix works AND prevent future changes from reintroducing the bug. The test checks the mathematical correctness (manual calculation) not just "doesn't crash".

Worked example Example 3: Feature Implementation (Advanced)

Problem: Hugging Face Transformers lacks a GradientCheckpointing utility for custom models.

What you're building: Let users trade compute for memory by recomputing activations during backward pass instead of storing them. Derivation from scratch: Normal training stores all activations: $\text{Memory} = O(L \cdot B \cdot H)$ where $L$ = layers, $B$ = batch size, $H$ = hidden size.

Gradient checkpointing stores only checkpoint activations (every $k$ layers), recomputes intermediate ones: $\text{Memory} = O\left(\frac{L}{k} \cdot B \cdot H\right)$ $\text{Compute} = O(L) \rightarrow O(L + L) = O(2L)$

Why this tradeoff works: Memory is the bottleneck for large models (can't fit batch on GPU), but GPUs have spare compute cycles. Doubling compute is acceptable if it lets you use4x larger batch.

Your implementation:

import torch
from torch.utils.checkpoint import checkpoint
 
class GradientCheckpointedModel(torch.nn.Module):
    def __init__(self, model, checkpoint_every=1):
        super().__init__()
        self.model = model
        self.checkpoint_every = checkpoint_every
    def forward(self, x):
        # Wrap layer chunks in checkpoint
        for i, layer in enumerate(self.model.layers):
            if i % self.checkpoint_every == 0and self.training:
                # checkpoint() recomputes this layer during backward
                x = checkpoint(layer, x)
            else:
                x = layer(x)
        return x

Benchmarking you must provide:

# Show memory savings vs. compute cost
model_normal = TransformerModel(layers=24)
model_ckpt = GradientCheckpointedModel(model_normal, checkpoint_every=4)
 
# Results to include PR:
# Config: 24 layers, batch=32, seq_len=512, hidden=1024, A100 GPU
# Normal:  23.4 GB memory, 180 ms/batch
# Ckpt=4:  8.1 GB memory,  245 ms/batch  (65% memory, +36% time)
# Ckpt=1:  4.2 GB memory,  410 ms/batch  (82% memory, +128% time)

Why this step? Feature PRs need evidence that the tradeoff is worth it. You derived the memory/compute relationship, then empirically verified it with benchmarks.

2. Choosing the Right Project

80/20 Rule: 80% of learning comes from 20% of projects. Pick based on:

You already use it (PyTorch, Scikit-learn, Pandas)
Active maintainers (PRs get reviewed within days, not months)
Good first issues tagged (shows they welcome newcomers)
Codebase you can comprehend (don't start with TensorFlow's C++ backend)

3. The Anatomy of a Good Pull Request

A PR is not just code—it's a persuasive argument that your change should be merged.

Formula PR Quality Formula

$Q_{\text{PR}} = \underbrace{C_{\text{correct}}}_{\text{does it work?}} \times \underbrace{C_{\text{clear}}}_{\text{can others understand?}} \times \underbrace{C_{\text{tested}}}_{\text{will it stay working?}} \times \underbrace{C_{\text{justified}}}_{\text{why this way?}}$

Derivation: Each component is multiplicative because zero on any dimension makes the PR unmergeable:

Correct but unclear → maintainers can't review confidently
Clear but untested → will break in production
Tested but unjustified → might be solving the wrong problem Each component is binary (0 or 1), so maximum $Q_{\text{PR}} = 1$ when all satisfied.

Template for a Strong PR:

## Problem
[Link to issue] Users can't load >16GB datasets because DataLoader loads 
everything into memory at once.
 
## Solution
Implement streaming mode that yields batches from disk using memory-mapped files.
 
## Changes
- Added `StreamingDataset` class wrapping `np.memmap`
- Modified `DataLoader` to detect streaming datasets and skip preloading
- Added `stream=True` parameter to dataset constructors
 
## Testing
- Unit test: 50GB dataset loads with<2GB peak memory (test_streaming.py)
- Benchmark: 1% slower iteration vs. in-memory (acceptable tradeoff)
- Backwards compatible: all existing tests pass
 
## Tradeoffs
- 1% slower due to disk I/O, but enables datasets10x larger than RAM
- Requires random-access file format (won't work with compressed .tar.gz)
 
## Docs
- Added "Working with Large Datasets" section to user guide
- Updated docstrings with `stream` parameter explanation

Why each section?:

Problem: Proves the change is necessary (not just "I thought this would be cool")
Solution: One-sentence summary (reviewer knows what to expect)
Changes: Technical detail (what files did you touch?)
Testing: Proof it works (maintainers can't merge untested code)
Tradeoffs: Shows you thought critically (every design has downsides)
Docs: Users need to know the feature exists

4. Code Review Etiquette

Code review is where learning happens. Maintainers have seen patterns you haven't.

Recall Feynman's Explanation (Explain to a 12-year-old)

Imagine you're building a LEGO spaceship and you show it to an expert LEGO builder. They say "Cool! But if you use this piece instead of that one, the wings won't fall off when you pick it up."

You have two choices:

Get defensive: "My design is fine! You just don't get my vision!"
Get curious: "Oh! Why does that piece work better? Teach me!"

Code review is choice #2. The expert has built 100 spaceships. They know which pieces break. When they suggest changes, they're giving you years of experience compressed into one comment. Your job is to ask questions until you understand why their way is better, then improve your spaceship.

Sometimes you'll disagree—that's okay! Explain your reasoning. But if three experts all say "wings will fall off", they're probably right. Trust the process.

How to respond to feedback:

Feedback Type	Good Response	Bad Response
"This could be clearer"	"Good catch! What if I rename X to Y and add comment explaining Z?"	"It's clear to me."
"This will break edge case X"	"You're right! I'll add a test for that. Should I handle it by doing Y?"	"That's an unlikely case."
"Use library function F instead"	"Didn't know that existed! Does it handle Z differently than my implementation?"	"My implementation works fine."
"This is out of scope for this PR"	"Agree, I'll split this into a separate PR. Should I open an issue first?"	"But it's related!"

Core principle: Assume good intent. Comments aren't attacks—they're collaboration. Maintainers want your PR to succeed (merging it is less work than rejecting it).

Worked example Example 4: Learning from Review Feedback

Your original code:

def compute_accuracy(predictions, labels):
    correct = 0
    for in range(len(predictions)):
        if predictions[i] == labels[i]:
            correct += 1
    return correct / len(predictions)

Reviewer comment: "This is slow for large arrays. Use NumPy vectorization."

Your learning moment: "I don't know NumPy well—can you show me what vectorized looks like, or point me to an example in the codebase?" Reviewer's response:

def compute_accuracy(predictions, labels):
    return (predictions == labels).mean()

What you learned:

== on arrays does element-wise comparison (returns boolean array)
.mean() on boolean array gives fraction of True values
This is 100x faster for1M predictions (no Python loop) Why this step? You didn't just copy the code—you asked WHY, understood the principle (vectorization eliminates loops), and can now apply it everywhere. This is how OS contribution teaches you.

5. Types of Contributions (Beyond Code)

Not all contributions require coding ML algorithms.

Value ladder (increasing difficulty):

Documentation (easiest, highest impact per hour)
- Fix typos, clarify confusing sections
- Add examples showing how to use features
- Translate docs to other languages
Issue triage (helps maintainers prioritize)
- Reproduce bug reports (add "confirmed" label)
- Ask for clarifications on vague issues
- Close duplicates
Tests (improves reliability)
- Add test cases for uncovered code
- Write regression tests for fixed bugs
Bug fixes (requires understanding codebase)
- Fix crashes, incorrect outputs
- Handle edge cases
Features (highest difficulty, most visible)
- Implement new algorithms
- Add configuration options
- Improve performance

80/20 insight: Documentation and tests are undervalued but high-impact. 80% of contributors try to implement flashy features; 20% improve docs. But users hit docs10x more often than they need new features.

Real-World Workflow Example

Let's trace a complete contribution from finding the issue to merged PR.

Scenario: You're using scikit-learn for a project and notice KMeans is slow.

Step 1: Find the Issue (1 hour)

You profile your code:

import cProfile
cProfile.run('model.fit(X)')
 
# Output shows80% time in _euclidean_distances()

Check GitHub issues: Someone already filed #12345 "KMeans slow for large n_features".

Step 2: Understand the Problem (2 hours)

Read the source code:

# sklearn/cluster/_kmeans.py
def _euclidean_distances(X, centers):
    # Current implementation: pure Python loops
    distances = np.zeros((X.shape[0], centers.shape[0]))
    for i in range(X.shape[0]):
        for j in range(centers.shape[0]):
            distances[i, j] = np.sqrt(np.sum((X[i] - centers[j])**2))
    return distances

Issue: Nested Python loops are slow. NumPy has vectorized cdist for this.

Step 3: Implement Fix (3 hours)

from scipy.spatial.distance import cdist
 
def _euclidean_distances(X, centers):
    # Vectorized computation: 100x faster for large arrays
    return cdist(X, centers, metric='euclidean')

Add test:

def test_euclidean_distances_correctness():
    X = np.random.rand(100, 50)
    centers = np.random.rand(10, 50)
    
    # Compare old (slow but trusted) vs. new (fast
    old_result = _euclidean_distances_old(X, centers)
    new_result = _euclidean_distances(X, centers)
    
    np.testing.assert_allclose(old_result, new_result)
 
def test_euclidean_distances_performance():
    X = np.random.rand(1000, 1000)
    centers = np.random.rand(100, 1000)
    
    import time
    start = time.time()
    _euclidean_distances(X, centers)
    elapsed = time.time() - start
    
    # Should complete in <1 second (old version took 45s)
    assert elapsed < 1.0

Step 4: Submit PR (1 hour)

Write the description (from template above), push to your fork, open PR.

Step 5: Code Review (2 days, async)

Reviewer comment: "This adds a scipy dependency. We want scikit-learn to depend only on NumPy. Can you implement this with NumPy's broadcasting instead?"

Your response: "Good point! Let me try NumPy broadcasting. Is this the right approach?"

def _euclidean_distances(X, centers):
    # Broadcasting: X is (n, d), centers is (k, d)
    # X[:, None, :] is (n, 1, d), centers[None, :, :] is (1, k, d)
    # Subtraction broadcasts to (n, k, d)
    diff = X[:, None, :] - centers[None, :, :]
    return np.sqrt((diff**2).sum(axis=2))

Reviewer: "Perfect! One more thing: add a benchmark to the docs showing the speedup."

You: "Added benchmark section. Speedup is 87x for n=1000, d=1000."

Step 6: Merged! (1 week total)

Your contribution is now in scikit-learn==1.5.0. Thousands of users benefit.

What you learned:

How to profile Python code (cProfile)
NumPy broadcasting mechanics
Dependency management tradeoffs
How to write performance tests
How to collaborate asynchronously

Why this matters: This is real ML engineering. Research papers don't teach you this.

Common Pitfalls and How to Avoid Them

Building Your OS Portfolio

Why it matters: Hiring managers look at GitHub profiles. Your commits are proof you can:

Write production-quality code
Collaborate with strangers
Learn new codebases quickly
Take feedback gracefully

Strategic approach (80/20):

20% of your contributions (features in major projects like PyTorch) generate 80% of visibility
But you need the other 80% (small fixes) to build credibility that unlocks the 20%

Roadmap:

Months 1-2: 10 documentation fixes across 3-5 projects (build confidence)
Months 3-4: 5 bug fixes (learn codebases deeply)
Months 5-6: 2 small features (demonstrate design skills)
Months 7+: 1 substantial feature per quarter (become recognized contributor)

By month 12, you have 30+ merged PRs across major projects. That's a portfolio that opens doors.

Connections

Code Review Best Practices - Applies to both giving and receiving feedback
Software Engineering for ML - OS teaches production ML skills
Version Control with Git - Foundation for all OS work
Testing ML Systems - OS demands rigorous testing
ML Research Paper Implementation - Often starts with OSS reimplementation
Collaborative Machine Learning - OSS is ultimate collaboration practice
Building ML Portfolios - OS contributions are portfolio centerpieces
Technical Writing for ML - Documentation work improves your explanations

Flashcards

What are the 7 steps in the OS contribution lifecycle? :: Find Issue → Claim It → Fork Repo → Create Branch → Code + Tests → Submit PR → Code Review → Revisions → Merge

Why must every PR include tests, not just code?

Tests prove correctness, prevent regressions, and give maintainers confidence to merge. Untested code will break in production as the codebase evolves.

What is gradient checkpointing's memory/compute tradeoff?

Reduces memory from O(L·B·H) to O(L/k·B·H) by storing only every k-th layer activation and recomputing intermediate ones during backward pass. Doubles compute but enables larger batches.

What is the "RAMP" mnemonic for choosing OS projects?

Read code comfortably? Active maintainers? Makes your work easier? People tagged good-first-issues? 3+ yes = good fit.

Why is documentation often higher-impact than features?

Users encounter docs 10x more than they need new features. 80% of contributors chase flashy features, so docs are underserved. Small time investment, large user benefit.

What's the "PR Quality Formula" and why is it multiplicative?

Q = C_correct × C_clear × C_tested × C_justified. Multiplicative because zero on any dimension (e.g., untested) makes PR unmergeable. All must be 1 for merge.

Why start with small PRs rather than large features?

Large PRs are hard to review, sit unreviewed for months.87% of first-timers with 500+ line PRs never return. 73% starting with <50 lines become regular contributors.

What should you do when a code review comment seems wrong?

Assume good intent and ask clarifying questions: "Can you explain why this approach is better? I'm concerned about X." Engage in discussion, don't defend. If 3 experts agree, they're probably right.

How does contributing to OSS teach "real ML engineering"?

You learn profiling, debugging production code, API design, backwards compatibility, performance optimization, testing strategies, and async collaboration—skills research papers don't cover.

What's the strategic OS roadmap for the first year?

Months 1-2: 10 doc fixes (confidence). Months 3-4: 5 bugs (codebase knowledge). Months 5-6: 2 small features (design skills). Months 7+: 1 large feature/quarter (recognition). 30+ PRs by month 12.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho yaar, is note ka core idea simple hai: machine learning ki duniya mein sabse zyada growth collaboration aur reproducibility se aati hai. Open-source projects jaise PyTorch, TensorFlow, ya Scikit-learn — yeh wahi jagah hai jaha cutting-edge research actual production tools ban jaate hain. Jab tum contribute karte ho, tum sirf knowledge consume nahi kar rahe, balki poore field ki infrastructure build karne mein help kar rahe ho. Aur sabse important baat — jab tum ek PR (pull request) submit karte ho, tum code ko fake nahi kar sakte. Tumhe usse deeply samajhna padta hai, isliye learning-by-doing sabse powerful tarika ban jaata hai.

Ab yeh matter kyun karta hai, khaaskar tumhare jaise regional student ke liye? Simple — open-source contributions tumhara real portfolio ban jaate hain jo kisi bhi degree ya certificate se zyada bolta hai. Duniya bhar ke experts tumhare code ko review karte hain, tumhe best practices sikhate hain (jaise testing, API design, backwards compatibility), aur yeh sab bilkul free mein. Contribution lifecycle bhi samajhna zaroori hai: pehle ek issue dhoondo, use claim karo, repo ko fork karo, apni branch banao, code plus tests likho, PR submit karo, review lo, revise karo, aur phir merge. Har step ka apna reason hai — jaise fork isliye taaki tum main project ko break kiye bina experiment kar sako.

Shuruaat ke liye pressure mat lo. Sabse easy entry point hota hai documentation fix — jaise note mein dikhaya gaya Scikit-learn ka max_features='sqrt' waala example, jaha tumhe zyada code likhne ki zaroorat nahi, par tumhe internals samajhne padte hain. Uske baad thoda intermediate level pe jaao — bug fixes, jaise PyTorch ka cosine_similarity waala 1D input crash. Yaha tum root cause diagnose karte ho aur ek clean fix likhte ho. Toh point yeh hai: chhote se shuru karo, consistency rakho, aur dheere-dheere tumhare commits hi tumhari sabse strong resume ban jaayenge. Give back to the community jisne tumhe seekhne ke tools diye!

What Is Open-Source ML Contribution?

How to Start Contributing

1. Understanding the Contribution Lifecycle

2. Choosing the Right Project

3. The Anatomy of a Good Pull Request

4. Code Review Etiquette

5. Types of Contributions (Beyond Code)

Real-World Workflow Example

Step 1: Find the Issue (1 hour)

Step 2: Understand the Problem (2 hours)

Step 3: Implement Fix (3 hours)

Step 4: Submit PR (1 hour)

Step 5: Code Review (2 days, async)

Step 6: Merged! (1 week total)

Common Pitfalls and How to Avoid Them

Building Your OS Portfolio

Connections

Flashcards

Concept Map

Hinglish (regional understanding)

Go deeper — visual, from zero

Test yourself — Research Frontiers & Practice

Connections