Foundations — ROC curve and AUC

Before you can read the parent note, you need to earn every symbol it fires at you. This page builds each one from nothing — plain words first, then a picture, then why the topic needs it.

1. The two groups: positives and negatives

Every problem here is a yes/no question. "Is this email spam?" "Does this patient have cancer?" There are only two real answers in the world, and we give them names.

• Picture: imagine $n$ dots split into two buckets. One bucket holds all the true positives, the other all the true negatives.
• Why we need it: every rate in this topic is "of one bucket, what fraction did we do X to?" You cannot count fractions until you know which bucket each dot belongs to.

The little symbols so far:

We will need to count how many dots land in a bucket. Counting needs two more tiny symbols, which we build next before using them.

2. Two counting symbols: $1[\cdot ]$ and $\sum$

Everything in this topic is really just "count the dots that satisfy some condition." Two symbols do that job.

• Picture: a lamp. Feed it a statement; it lights up ($1$) if the statement holds, stays dark ($0$) otherwise.

• Why the topic needs both: counting "how many dots satisfy something" becomes flipping the switch on each dot and summing the $1$s. For example, the sizes of the two buckets are exactly the counts of dots with each true answer:

$n_{+}={\sum }_{i=1}^{n}1[y_i=1]n_{-}={\sum }_{i=1}^{n}1[y_i=0]$

The first reads "walk through all $n$ dots, add $1$ each time the true answer is positive"; the second does the same for negatives. Because every dot is either positive or negative, these two counts always add back up to the whole: $n_{+}+n_{-}=n$.

Notice these two symbols only depend on conditions being true or false — they do not yet need scores or thresholds. Those come next.

3. The score $s_i$, the threshold $t$, and our guess ${\hat{y}}_i$

• Why two different letters ($y$ vs $\hat{y}$)? $y_i$ is reality; ${\hat{y}}_i$ is our decision. They can disagree — and every mistake type in the confusion matrix is exactly a way they disagree.
• How the counting symbols return: now that $s_i$ and $t$ exist, $1[s_i\ge t]$ is $1$ exactly for the dots we predict positive, so ${\sum }_{i}1[s_i\ge t]$ counts our predicted positives.

Look at the figure. Positive dots (magenta) and negative dots (violet) are spread along a score line from $0$ to $1$. The orange vertical line is the threshold $t$. Everything to its right we call positive; everything to its left, negative. Slide the line left and you sweep more dots into "predicted positive." That sliding line is the engine of the whole ROC curve.

4. The four outcomes — the Confusion Matrix

When truth $y_i$ meets guess ${\hat{y}}_i$, there are exactly four combinations. These are the atoms everything is built from — the full story lives in the Confusion Matrix.

• Picture: a 2×2 grid. Rows are truth (positive/negative), columns are our guess (positive/negative). The green diagonal (TP, TN) is where we're right; the red off-diagonal (FP, FN) is where we're wrong.
• Sanity check: every positive dot is either caught or missed, so $TP+FN=n_{+}$; every negative dot is either falsely flagged or correctly passed, so $FP+TN=n_{-}$. All four counts together cover every dot exactly once: $TP+FP+FN+TN=n$.
• Why the topic needs it: the two ROC axes are just ratios of these four boxes. No confusion matrix, no ROC.

Notice from the sliding line: as $t$ drops, dots move from "predicted negative" into "predicted positive." A positive dot doing this turns an FN into a TP (good). A negative dot doing this turns a TN into an FP (bad). That trade is the heart of ROC.

5. The two rates: TPR (y-axis) and FPR (x-axis)

Raw counts (TP, FP) depend on how big your test set is. To compare fairly, we turn them into fractions of their own bucket.

• Why the denominators? $TP+FN$ is every real positive (caught + missed) $=n_{+}$. So TPR is caught-out-of-all-positives. Likewise $FP+TN=n_{-}$, so FPR is false-alarms-out-of-all-negatives. Dividing by the bucket size is what makes these rates, immune to the raw size of each group.
• Aliases you'll meet: TPR is also called sensitivity or recall; $1-\text{FPR}$ is specificity. These come from Sensitivity and Specificity, and recall reappears in F1-Score.
• We want TPR high (catch everything) and FPR low (never cry wolf). The whole tension is that lowering $t$ pushes both up together.

6. Putting a point on the ROC plane

For one threshold $t$ you compute one pair $(\text{FPR},\text{TPR})$ — that's one point on a graph. FPR runs along the horizontal, TPR up the vertical, both from $0$ to $1$.

• Where the corners come from — cover every case:
  • $t=1$ (strictest): nobody clears the bar, $TP=FP=0$, so the point is $(0,0)$ — bottom-left.
  • $t=0$ (loosest): everybody clears the bar, $TP=n_{+}$, $FP=n_{-}$, so the point is $(1,1)$ — top-right.
  • In between: each distinct score in your data drops the line past one dot and nudges the point up (if that dot was positive) or right (if negative).
• The dashed diagonal from $(0,0)$ to $(1,1)$ is the "coin-flip" line: a model with no skill lands here. A good model bows up toward the top-left corner $(0,1)$ — high catch, zero false alarms.

7. The curve, and the area under it (AUC)

Connect all those points in order and you get the ROC curve. The single number that summarizes it is the AUC.

• Picture: shade everything beneath the curve. More shaded area = curve bows higher = better separation.
• On the "area under a curve" notation: you may see this written with an integral sign $\int$ (a tall stretched S) elsewhere. That sign just means "area under the graph" — the continuous version of adding up thin vertical strips. In practice we add up little trapezoids between consecutive ROC points, which is honest arithmetic, not scary calculus.

Why AUC equals a probability — the intuition, built from the sliding line. Here is the "magic fact" and why it holds:

So AUC $=1$ means positives always outscore negatives (perfect ordering); AUC $=0.5$ means the ordering is no better than a coin flip.

When negatives massively outnumber positives, this ROC picture can flatter a model — that's when you switch to the Precision-Recall Curve. And when there are more than two classes, you compute one ROC per class (see Multi-Class Classification).

8. How the foundations feed the topic

Read it top-down: labels and scores define the buckets and cutoff; the cutoff turns scores into predicted labels; those labels (counted with the indicator switch) fill the confusion matrix; the four boxes give the two rates; the two rates give one point; sliding the cutoff sweeps out the curve; the area under it is AUC.

Equipment checklist

Cover the right side and answer before revealing.