2.6.7 · D1Model Evaluation & Selection

Foundations — Accuracy, precision, recall, F1-score

2,410 words11 min readBack to topic

Before you can trust a single formula on the parent page Accuracy, Precision, Recall, F1-Score, you must be able to read every symbol in it without a dictionary. This page builds each one from nothing, in the order that lets the next one make sense.


0. What is a "classifier" and what is a "class"?

The picture to hold in your head: a sorting machine with two output bins.

Figure — Accuracy, precision, recall, F1-score
Figure s01 — Notice the flow: one thing goes IN, the machine picks ONE bin. The amber bin is "positive", the white bin is "negative". Every example lands in exactly one bin.

Every example that goes in must have a true label (reality) and gets a predicted label (the machine's guess). When we compare the two, four situations can happen — and those four situations are the whole foundation of this topic.


1. Positive vs negative — a naming choice, not a value judgement


2. The four outcomes — the atoms everything is built from

Compare truth (what the thing really is) against the guess (what the machine said). Two questions, each yes/no, give boxes.

Figure — Accuracy, precision, recall, F1-score
Figure s02 — Notice the layout: columns = what the model predicted, rows = what is actually true. The two cyan boxes on the diagonal are the correct guesses (TP, TN); the two amber boxes off the diagonal are the mistakes (FP, FN). Memorise this grid — it is the confusion matrix.

Read the two-word decoder for the names:

  • The second word (Positive / Negative) = what the machine guessed.
  • The first word (True / False) = whether that guess was correct.

So "False Negative" = a negative guess that was wrong = we said "no" but the answer was "yes" = a miss.

Recall Self-check: which box is a missed cancer case?

A patient who has cancer but the model says "healthy" ::: False Negative (guessed negative, wrongly)

The confusion matrix itself

The four counts are not loose numbers — they live in a fixed table called the Confusion Matrix. By convention we put actual (true) classes down the rows and predicted classes across the columns:


3. The symbols — "fraction" vs "rate"

Every metric on the parent page has the shape . So you must be fluent with what a fraction means here, and how the word "rate" is used.

Picture a bar of length with a coloured chunk of length ; the fraction is how much of the bar is coloured.

Figure — Accuracy, precision, recall, F1-score
Figure s03 — Notice the whole white bar is the denominator (100 predictions) and the cyan chunk is the numerator (93 correct). The fraction is simply how much of the bar is filled: , or .


4. Sums like — reading a total as a picture

The parent page constantly writes things like . The plus sign here just means "pile these boxes together and count".

Knowing which sum is a row and which is a column is the single trick that tells precision and recall apart. Recall divides by a row (reality); precision divides by a column (the model's claims).

Recall Self-check: which sum is "everything the model flagged as positive"?

Predicted-positive column :::


5. Precision , recall , and the F1 symbol

Now we can name the two fractions the parent page combines. Both put TP on top; they differ only in the "whole" underneath.

The F1 symbol

(also written ) is one number that summarises both and . It is defined as their harmonic mean — the special average we build next.

To see why the harmonic mean (and not a plain average) is the right tool, we compare the two.

Figure — Accuracy, precision, recall, F1-score
Figure s04 — Notice how, as recall shrinks toward (left edge), the cyan arithmetic-mean line stays comfortably near while the amber harmonic-mean (F1) line dives toward . F1 will not let a great precision hide a terrible recall.


Prerequisite map

Each box below is a foundation from this page; each arrow means "understand the box the arrow leaves before the box it points to". Read it top-to-bottom: everything funnels down into the parent topic's metrics.

Class and classifier

Positive vs negative class

Four outcomes TP TN FP FN

Confusion matrix table

Fraction a over b

Accuracy precision recall

Row and column totals

Harmonic mean

F1 score

Parent topic metrics

Every arrow means "you must understand the box it leaves before the box it enters". Notice all paths converge on the four outcome counts — that is why we spent the most effort there.


Where these foundations lead next

Once TP, TN, FP, FN feel obvious, these vault topics all reuse them directly:

  • Confusion Matrix — the table that holds the four counts.
  • ROC Curve and AUC and Precision-Recall Curve — plot these metrics as a threshold moves.
  • Threshold Optimization — choosing the cut-off that sets how many FP vs FN you accept.
  • Class Imbalance Handling and Cost-Sensitive Learning — what to do when one class is rare or one error type is expensive.
  • Cross-Validation — how to measure these numbers reliably on unseen data.

Equipment checklist

Read each question, answer out loud, then reveal.

What does the "positive" class mean, and does it mean "good"?
The class we are trying to detect; it does not mean good (a positive disease test is bad).
Decode "False Negative" into plain words.
A negative guess that was wrong — i.e. a miss (really positive, but we said negative).
In the confusion matrix, what do the rows mean and what do the columns mean?
Rows = the actual (true) class; columns = the model's predicted class.
Which sum gives "everything the model predicted positive"?
(the predicted-positive column).
Which sum gives "everything that is actually positive"?
(the actual-positive row).
Write the formulas for precision and recall .
and .
What happens to precision when the model never predicts positive, and how is it handled?
so precision is (undefined); by convention it is set to .
How do you turn a fraction like into a percentage?
Multiply by .
What is the difference between a "fraction" and a "rate" here?
Numerically the same (a part-of-whole between 0 and 1); "rate" just means that fraction used as a performance score.
Before writing the denominator of any metric, what must you ask?
"A fraction of what?" — which whole are we dividing by.
Write the F1 formula and say why it uses the harmonic mean.
; the harmonic mean is dragged down by any single small value, so F1 won't be fooled when one of / is terrible.
Compute the harmonic mean of and .
.