Foundations — Open problems and future directions

This page is the toolbox. Before you can read the parent note Open Problems and Future Directions you must be able to read its symbols out loud in plain words and see a picture for each one. We build them from zero, in an order where each one leans only on the ones before it.

Layer 0 — The three raw objects

Everything starts with three things: an input, an output, and a rule connecting them.

Why the topic needs these: every open problem is phrased as "$f$ maps $x$ to $y$ well here but badly there." Without naming the input, the output, and the box between them, you cannot even state a problem.

Before we can write $f$'s "type" we need the words $\mathbb{R}$ and ${\mathbb{R}}^n$.

With those words in hand, the parent's notation is now readable: $f:{\mathbb{R}}^n\to {\mathbb{R}}^m$ is read "$f$ takes an input made of $n$ numbers and returns an output made of $m$ numbers." So $x$ lives in ${\mathbb{R}}^n$ and the guess lives in ${\mathbb{R}}^m$.

Layer 1 — Where the examples live: distributions

We never get one example; we get a pile of them, and the pile has a shape. That shape is a distribution.

Now we can read the parent's key symbols:

Why "sparse in high dimensions" is scary (foreshadowing problem #1): in a space of a million directions, any finite pile of examples touches almost none of the paper. Figure s02 shows this shrinking in a way you can actually see.

Layer 2 — Measuring wrongness: the loss

To improve $f$ we need a number saying how wrong a guess is. That number is the loss.

Why we need loss at all: you cannot fix what you cannot measure. The loss turns the vague wish "be right" into a number we can push down. Every "$\min$" you see next is "push this number down."

Layer 3 — Averaging over the fog: expectation

We don't care about the loss on one example; we care about the typical loss across the whole fog. "Typical, weighted by how likely" is exactly what expectation means.

Why expectation and not a plain average? A plain average treats every example equally. But rare regions (thin fog) should count less than common regions (thick fog). $\mathbb{E}$ builds that weighting in automatically — which is precisely why rare-but-important events (rare diseases, equipment failure) get under-weighted, the seed of problem #1's importance. See Statistical Learning Theory for the formal treatment of this average-over-a-distribution idea.

Now the parent's central formula becomes readable:

So ${\mathcal{R}}_{\text{train}}=\mathcal{R}(P_{\text{train}})$ = average wrongness of the finished model on the training fog, and ${\mathcal{R}}_{\text{test}}=\mathcal{R}(P_{\text{test}})$ = average wrongness on the deployment fog. The gap between these two — same dials $\theta$, different fog — is the star of the whole chapter.

Layer 4 — Comparing two fogs: KL divergence

Problem #3 (robustness) and #4 (causality) both ask "how different are two distributions?" We need a number for that: the KL divergence.

Here is the actual formula it hides — for outcomes $i$ with fog-thicknesses $P(i)$ and $Q(i)$: $D_{\text{KL}}(P\Vert Q)={\sum }_{i}P(i)\ln \frac{P(i)}{Q(i)}\text{(sums)},\int P(x)\ln \frac{P(x)}{Q(x)}dx\text{(smooth fogs)}.$

Why this exact tool? We could subtract fogs, but $D_{\text{KL}}$ answers the specific question "if I expected fog $Q$ but reality is fog $P$, how surprised will I be on average?" — the natural currency for "train fog vs test fog." When the parent writes $D_{\text{KL}}(P_{\text{source}}\Vert P_{\text{target}})\approx 0$ it means "make the two fogs look identical after squeezing inputs through a feature map." Deeper: Bayesian Methods.

Layer 5 — The slope tools: gradient, Hessian, eigenvalue

To turn the dials sensibly you need to know which way is downhill on the wrongness surface. That is the gradient. To know how sharply it curves you need the Hessian.

Why "gradient/difference of a distribution" even makes sense. To take slopes and steps of fogs, we quietly treat each fog as a giant list of numbers — one thickness per outcome bucket. A list of numbers is a vector, and vectors can be subtracted (bucket-by-bucket) and can have slopes. So $\Delta =P_{\text{test}}-P_{\text{train}}$ means "at each bucket, subtract the two fog-thicknesses" — a fog of differences (thicker where test exceeds train, negative where it falls short). This is the one structural assumption we lean on: distributions, written as lists of bucket-weights, live in a vector space where subtraction and slopes are defined.

Why ${\nabla }_P\mathcal{R}$ even exists — derived, not asserted. Recall from Layer 3 that with the dials $\theta$ frozen, risk is genuinely a function of the fog: writing $P(i)$ for the fog-thickness in bucket $i$ and ${\ell }_i=\mathcal{L}(f_{\theta }(x_i),y_i)$ for the (now-fixed) loss in that bucket, $\mathcal{R}(P)={\sum }_{i}P(i){\ell }_i.$ This is a plain weighted sum — linear in the fog-weights $P(i)$. Differentiating a weighted sum with respect to one weight $P(i)$ just leaves its multiplier: $\frac{\partial \mathcal{R}}{\partial P(i)}={\ell }_i⟹{\nabla }_P\mathcal{R}=({\ell }_1,{\ell }_2,\dots ),$ i.e. the gradient of risk with respect to the fog is nothing but the list of per-bucket losses. So ${\nabla }_P\mathcal{R}$ is not a mysterious new object — it exists because "risk" is literally a dot-product of fog-weights with losses, and it equals the losses themselves. That is the justification the parent note skips.

Why a Taylor expansion, and what it is. A Taylor expansion is the standard trick for asking "if I nudge the input a little, how does the output change?" It approximates any smooth quantity near a point by: starting value $+$ slope $\times$ step $+$ half $\times$ curvature $\times$ step-squared. We use it here because the exact test-risk is hopeless to compute, but we only need to know how fast it grows as the fog drifts by a small $\Delta$. The parent's formula is just this recipe with "input" $=$ the fog: $\mathcal{R}(P_{\text{test}})\approx \underset{\text{start height}}{\underbrace{\mathcal{R}(P_{\text{train}})}}+\underset{\text{slope}\times \text{step}}{\underbrace{{\nabla }_P\mathcal{R}\cdot \Delta }}+\underset{\text{curvature}\times {\text{step}}^2}{\underbrace{\frac{1}{2}{\Delta }^{\top }H\Delta }}$ where ${\nabla }_P\mathcal{R}$ = the per-bucket losses we just derived, $\Delta =P_{\text{test}}-P_{\text{train}}$ is "how far the fog moved," and $H$ = the Hessian in fog-space. (The multivariable slope machinery this rests on lives in Optimization Landscape.)

Why the topic needs curvature: if the wrongness surface is a sharp ridge (big eigenvalues, whose eigenvectors point along the ridge) in fog-space, then a tiny shove from $P_{\text{train}}$ to $P_{\text{test}}$ sends the loss flying up. Figure s03 shows sharp-vs-flat side by side — that single picture is the entire argument for why small fog-changes can cause huge errors.

Layer 6 — The one truly new idea: intervention, $do(\cdot )$

Problem #4 introduces a symbol you have never seen in ordinary probability: $do$. It is the whole reason Causal Inference is separate from statistics. First, two new names.

Why a brand-new symbol? No amount of ordinary probability notation can express "physically forcing a value." $do$ is the minimum extra vocabulary needed to talk about actions, which is exactly the capability today's models lack.

How it all feeds the topic

The named research directions each lean on one branch above: Transfer Learning and Meta-Learning attack the "different fog" gap; Reinforcement Learning lives in the $do$ / action world; Cognitive Science motivates the human sample-efficiency comparison; AI Ethics asks which fog we should serve well. Prefer Hindi? See 6.5.10 Open problems and future directions (Hinglish).

Equipment checklist

Say each answer aloud before reading the parent note. If any one stalls, re-read its layer above.