Foundations — Importance of interpretability

Before you can appreciate why interpretability matters, you must be able to read every symbol the parent note (Importance of interpretability) throws at you. This page assumes you have seen none of them. We build each one from a picture.

1. What even is a "model"? — the box with inputs and an output

Look at the figure. On the left we feed in facts. In the middle sits the box (the model). On the right a single answer pops out. Everything in this topic is a comment on some part of that picture:

• The facts on the left → we will call these features.
• The box in the middle → the parent note asks: is it see-through (transparent) or painted black (a black box)?
• The answer on the right → we will call this the prediction.

2. Features — the facts we feed in ($\text{features}$, $x$)

We write a single feature as $x$. When there are many of them we imagine them stacked in a list: $x=(x_1,x_2,\dots ,x_n)$

Why the topic needs this: the parent note's whole point about the husky-vs-wolf classifier is that the model used the wrong feature (background snow instead of the animal). You cannot discuss "which feature carried the bias" until you can name features.

3. The prediction — the guess coming out ($\hat{y}$)

So the whole box is: $\text{features }x⟶\text{model}⟶\hat{y}$.

4. Probability — how sure the guess is ($P(\dots )$, "78% chance")

The parent note says "78% chance of heart disease." A guess is rarely a flat yes/no — it comes with a confidence.

Look at the bar in the figure: empty at the left ($0$, impossible), full at the right ($1$, certain). "78%" is just $0.78$ — nearly four-fifths of the way along.

5. The "$\ne$" and "$⟹$" symbols — how we state a problem

6. Functions and the "value" formula ($f(\dots )$, $\lambda$)

The parent note writes $\text{Trust}=f(\text{Accuracy},\text{Interpretability},\dots )$.

The figure shows three dials (Accuracy, Interpretability, Alignment) all feeding one output gauge (Trust). Turning any dial moves the needle — that is exactly what $f(\text{Accuracy},\text{Interpretability},\text{Alignment})$ claims: trust is not accuracy alone.

Why the topic needs this: the parent's whole trade-off argument (an 85%-accurate interpretable model beating a 90%-accurate black box) is captured by choosing large $\lambda$ for "Deployment Risk" in high-stakes settings. Without $\lambda$ you cannot express "it depends on the domain."

7. Accuracy, false positives, and "spurious correlation" — plain words

These aren't symbols but they are the load-bearing ideas of the husky-wolf and COMPAS examples. See Fairness in Machine Learning and Adversarial Examples for where these bite hardest.

8. Global vs Local — two ways to "look inside"

This distinction leads directly to Post-hoc Interpretability Methods and Inherently Interpretable Models — two different strategies for opening the box, and to Explainable AI (XAI) Techniques generally.

How the foundations feed the topic

Every arrow says "you need the earlier idea to understand the later one." All roads lead to the parent topic. For the accuracy-vs-transparency tension specifically, continue to Model Transparency vs Model Accuracy, and to record all of this for auditors see Model Cards and Documentation.

Equipment checklist

Test yourself — cover the right side and answer before revealing.