2.5.13 · D5Unsupervised Learning
Question bank — Anomaly detection methods
Before we start, three plain-words reminders so every symbol below is earned:
- = the probability density our model assigns to a point — think "how tall is the bell curve right where sits." High = typical, low = weird.
- (epsilon) = the threshold we pick. Anything with gets flagged.
- (Sigma) = the covariance matrix — a table describing the spread of normal data and how features lean together.
True or false — justify
Anomaly detection needs labeled anomalies to train.
False. The whole appeal is that it learns "normal" from unlabeled data; anomalies are so rare and varied that collecting labels for them is usually impossible.
A point with low is definitely an anomaly.
False. Low only means "rare under our model." If the model is wrong (e.g. data isn't Gaussian), a perfectly normal point can land in a low-density region and be wrongly flagged.
If we lower the threshold , we catch more anomalies.
False. Lower is stricter: fewer points fall below it, so we flag fewer points — fewer false alarms but more missed anomalies.
Isolation Forest gives short path lengths to anomalies.
True. Anomalies sit "out alone," so a few random splits isolate them; normal points are buried in the crowd and need many splits.
A high anomaly score in Isolation Forest means the point is normal.
False. The mapping makes short path lengths give scores near 1, so means anomaly, not normal.
The Gaussian method treats all directions of deviation equally.
False. The Mahalanobis distance uses , so a fixed step counts as "far" in low-variance directions and "near" in high-variance ones — it is not plain Euclidean distance.
One-Class SVM separates two labeled classes.
False. It only sees one class ("normal") and separates that data from the rest of space (the origin side is "everything else"), which is why it's called one-class.
Increasing in One-Class SVM makes the model more permissive.
False. is roughly an upper bound on the fraction flagged as outliers; larger shrinks the "normal" region and flags more points.
Multivariate Gaussian anomaly detection can capture correlations between features.
True. The off-diagonal entries of encode how features co-vary (e.g. CPU and memory rising together), so the elliptical contours can tilt — something an independent per-feature model cannot do.
If the Gaussian formula still works fine.
False. means is singular (some feature is redundant or constant), so and blow up — the density is undefined.
Spot the error
"Since was , the point is times more anomalous than one with ."
Error: density is not a linear "anomaly odometer." We compare against a threshold ; the raw magnitude ratio has no calibrated meaning, and densities can even exceed 1.
"I'll build one giant isolation tree instead of 100 small ones — same result, less work."
Error: a single random tree is high-variance; one unlucky split order gives noisy path lengths. The forest averages many trees so the score becomes stable.
"The covariance is ."
Error: missing the transpose. It must be — an outer product giving an matrix, not a scalar.
"One-Class SVM with a linear kernel handles a ring-shaped normal region."
Error: a ring is not linearly separable from its center. You need a nonlinear kernel (e.g. RBF) so the curved boundary can wrap the ring.
"Anomaly score — I'll skip , it's just a constant."
Error: depends on dataset size , normalizing path lengths so scores compare across datasets. Dropping it makes scores meaningless across different .
"RBF kernel : bigger means smoother, wider boundary."
Error: larger makes the kernel decay faster, so influence stays local and the boundary gets tighter and wigglier — more prone to overfitting, not smoother.
Why questions
Why do we invert instead of just using raw distance ?
Because features have different scales and correlations. rescales space so "one unit of distance" means "one standard deviation of normal spread" in every direction — making the notion of far fair.
Why does Isolation Forest use random feature and split choices rather than informative ones?
We're not classifying; we only need to partition space. Random splits avoid modeling the fine structure of normal data and let outliers fall out in the first few cuts, keeping it fast and unbiased.
Why is the anomaly threshold tuned on a labeled validation set even though training is unsupervised?
We learn without labels, but to choose we need a few known anomalies to measure precision/recall — otherwise we're guessing the cutoff blindly.
Why does One-Class SVM push the hyperplane away from the origin (the term)?
The origin represents "not-normal" in feature space. Maximizing carves out the largest possible region on the data's side, so normal points sit comfortably inside and everything else is outside.
Why can the exponential in the Gaussian density make astronomically small (like )?
The term is where is the Mahalanobis distance. Because the exponent grows with the square of distance, moving a few standard deviations out crushes the density exponentially fast.
Why do we average path length over many trees before scoring, not per tree?
One tree's path length is noisy due to random splits. Averaging estimates the expected isolation depth , which is the quantity that reliably separates normal from anomalous.
Edge cases
What happens if a feature is constant for all training points (zero variance)?
becomes singular (), so the Gaussian density is undefined. Fix: drop that feature or add a tiny value to the diagonal (regularization).
What if you run Gaussian detection on data that is bimodal (two clusters of normal)?
A single Gaussian centers between the clusters, so the empty gap gets high density and both real clusters look anomalous. You'd need a mixture model, not one Gaussian.
What score does Isolation Forest give when every point is identical?
No random split can separate identical points, so path lengths saturate at max depth; all scores are similar and low — the method simply finds nothing anomalous, which is correct.
In One-Class SVM, what if is set to 1?
upper-bounds the outlier fraction, so permits all training points to be treated as outliers — the "normal" region collapses and the model is useless.
What happens to Gaussian detection as the number of features grows very large (curse of dimensionality)?
In high dimensions almost every point is far from in Mahalanobis terms, densities all become tiny, and needs samples to estimate reliably — so the method degrades and often needs feature reduction.
What if a genuine anomaly happens to look exactly like an average normal point on all measured features?
No density- or isolation-based method can catch it — it's invisible in the chosen feature space. The lesson: detection is only as good as the features that expose the anomaly.
What is the anomaly score of a point at the exact mean under the Gaussian model?
Mahalanobis distance is , so is at its maximum — the least anomalous point possible, and it will never be flagged for any sensible .
Recall Quick self-test
Does in Isolation Forest mean normal or anomaly? ::: Anomaly — short paths give scores near 1. Does lowering flag more or fewer points? ::: Fewer (stricter). What does do in the Mahalanobis distance? ::: Rescales space so distance is measured in standard-deviations, accounting for correlations.