Anomaly detection methods: Visual walkthrough

We only need one idea: normal data lives in a fuzzy cloud. Anything far from the cloud is suspicious. Everything else is turning that sentence into arithmetic.

Step 1 — Start with a cloud of dots

WHAT. Draw the data. Each server-reading is a dot. We use two numbers per reading so we can see it: $x_{1}$ = CPU usage, $x_{2}$ = memory usage. So one reading is a point $x = (x_{1}, x_{2})$ on a flat sheet.

WHY. Before any formula, we must look. The whole method is "learn what normal looks like", and normal is literally the shape of this cloud. If you can't see the cloud you can't say what's far from it.

PICTURE. The white dots below cluster in the middle. The lonely amber dot at the top-right is the reading we suspect — it sits far from the pack.

Step 2 — Find the centre of the cloud

WHAT. Compute the mean $μ$ : the average position of all the dots.

$μ = share equally \frac{1}{m} \sum_{i = 1}^{m} each dot x^{(i)}$

$m$ ::: the number of training dots.
$x^{(i)}$ ::: the $i$ -th dot (the little $(i)$ up top is just a label, not a power).
$\frac{1}{m} \sum$ ::: "add them all up, then split evenly" — the balance point.

WHY. Normal behaviour has a "typical" value. The mean is the single point that sits closest (in squared-distance) to everything at once — the natural stake-in-the-ground for "centre of normal".

PICTURE. The cyan cross marks $μ$ . Notice it lands right in the thick of the white dots, not on any single dot.

Step 3 — Measure how the cloud is stretched and tilted

WHAT. Compute the covariance matrix $Σ$ .

$Σ = \frac{1}{m} \sum_{i = 1}^{m} arrow from centre to dot (x^{(i)} - μ) same arrow, laid sideways (x^{(i)} - μ)^{T}$

$(x^{(i)} - μ)$ ::: the arrow pointing from the centre out to one dot.
The little $T$ (transpose) ::: turns a standing column into a lying-down row so the multiplication produces a $2 \times 2$ table, not a single number.
Averaging these tables ::: gives one table describing the cloud's overall spread.

Read the table as: $Σ = [spread in x_{1} how x_{1}, x_{2} move together how x_{1}, x_{2} move together spread in x_{2}]$

WHY. A round cloud and a stretched, tilted cloud both have the same centre but very different "shapes". The mean alone can't tell a point that's far along the fat direction (still normal) from one that's far across the thin direction (very abnormal). $Σ$ records the shape so we can judge distance fairly.

PICTURE. The cyan ellipse traces the shape $Σ$ describes. Its long axis is the direction of biggest spread; its short axis, the tightest.

Step 4 — Turn "far from the cloud" into one honest number

WHAT. Compute the Mahalanobis distance squared:

$D^{2} = arrow, sideways (x - μ)^{T} shape-correcting lens Σ^{- 1} arrow (x - μ)$

$(x - μ)$ ::: arrow from centre to the point we're testing.
$Σ^{- 1}$ ::: the inverse of the shape matrix — it undoes the stretch and tilt, so distance is measured in "cloud units" not raw units.
The whole sandwich ::: one number: how many cloud-widths away $x$ sits.

WHY. Plain straight-line distance is unfair: 50 units along a fat direction is barely unusual, but 50 across a thin direction is wildly unusual. Multiplying by $Σ^{- 1}$ squeezes the fat directions and stretches the thin ones, so every direction is scored on the same "how surprising is this?" scale. We use $Σ^{- 1}$ (not $Σ$ ) precisely because we want to cancel the spread, not re-apply it.

PICTURE. Two dots sit the same ruler-distance from centre — but one is inside the cloud, one outside. In cloud-units (the dashed rings) they land on different rings: Mahalanobis sees what your eye sees.

Step 5 — Convert distance into a probability

WHAT. Feed $D^{2}$ into the Gaussian density:

$p (x) = fixed scaling so total area = 1 \frac{1}{( 2 π ) ^{n /2} ∣Σ ∣ ^{1/2}} exp (- \frac{1}{2} cloud-distance^{2} D^{2})$

$exp (- \frac{1}{2} D^{2})$ ::: a hill that is tall at the centre ( $D^{2} = 0 \Rightarrow exp (0) = 1$ ) and shrinks fast as you walk out.
$∣Σ∣$ ::: the "volume" of the cloud; dividing by it keeps the total probability equal to $1$ no matter how big the cloud.
$n$ ::: number of features.

WHY use $exp$ ? We need something that (a) is largest at the centre, (b) never goes negative (probabilities can't), and (c) fades smoothly and quickly far away. $e^{- distance^{2}}$ does exactly this — small distance → near 1, large distance → crashes toward 0. No polynomial fades this cleanly.

PICTURE. The surface is a smooth bell sitting over the cloud: high in the middle, hugging zero at the edges. Height = probability = "how normal".

Step 6 — Slice the bell with a threshold

WHAT. Pick a small cutoff $ϵ$ . Flag $x$ as an anomaly when $p (x) < ϵ$ .

$Anomaly ⟺ p (x) < ϵ$

$ϵ$ ::: a tiny probability floor we choose (e.g. $1 0^{- 5}$ ).
$p (x) < ϵ$ ::: "the bell is too low here — the point lives where almost no normal data does".

WHY. Because $p$ shrinks as $D^{2}$ grows, " $p$ below a floor" is exactly the same as " $D^{2}$ above a ceiling". So the threshold on probability is really a ring drawn on the plane: inside the ring = normal, outside = anomaly. Choosing $ϵ$ = choosing how tight that ring is.

PICTURE. The amber contour is the level $p = ϵ$ . Dots inside are green (normal); the amber dot outside is flagged.

Step 7 — The edge and degenerate cases

WHAT. Three ways the picture can break, and what happens.

Case A — a dead feature ( $Σ$ has a zero-spread direction). If every training reading has, say, the same value in some direction, the cloud is a flat line, its "volume" $∣Σ∣ = 0$ , and $Σ^{- 1}$ doesn't exist — you'd divide by zero. Fix: drop the useless feature, or add a tiny $λ$ to the diagonal ( $Σ + λ I$ ) so the ellipse always has thickness.

Case B — a lonely point on the fat axis. A point can be far from centre yet still land inside the amber ring, because it lies along the cloud's long direction. Straight-line distance would panic; Mahalanobis correctly says "normal". This is the whole reason we bothered with $Σ^{- 1}$ .

Case C — $ϵ$ too large or too small. Huge $ϵ$ → tiny ring → half the normal cloud gets flagged (false alarms). Tiny $ϵ$ → giant ring → real anomalies slip through. The ring's radius is the false-alarm dial.

WHY. The reader must never meet a scenario we hid. Zero-variance, along-axis outliers, and threshold extremes are the three traps in real deployments.

PICTURE. Left: a collapsed line-cloud (Case A) with the amber warning. Middle: the along-axis point sitting safely inside the ring (Case B). Right: same data, two rings for two $ϵ$ values (Case C).

The one-picture summary

Everything above, on a single frame: the cloud (Step 1), its centre $μ$ (Step 2), its shape ellipse $Σ$ (Step 3), the cloud-distance rings $D^{2}$ (Step 4), the bell $p (x)$ shrinking outward (Step 5), and the amber threshold ring that flags the outsider (Step 6).

Recall Feynman retelling — say it back in plain words

We dumped all the normal readings as dots and found their middle. Then we noticed the dots weren't a neat circle — they were a stretched, tilted blob, and we wrote that shape down in a little table. To ask "is a new reading weird?", we didn't just measure straight-line distance to the middle, because that's unfair to the blob's shape. Instead we measured distance through the blob — squeezing the fat directions, stretching the thin ones — so one honest number says how many blob-widths away the point is. We fed that number into a bell curve: near the middle it's tall (very normal), far out it drops to almost nothing (very rare). Finally we drew a ring at a chosen height; anything landing outside the ring — living where normal data essentially never goes — gets flagged. The only trap: never forget to measure through the blob's shape, or a normal point lying along the long axis will scare you for no reason.

Prev: Parent · Anomaly detection methods