Visual walkthrough — Data drift and concept drift detection

---

Step 1 — Two piles of data, one honest question

WHY start here. You never see the true distribution $P(X)$ — only samples. So drift detection is honestly just: lay two heaps of numbers side by side and ask "same shape or not?" Everything technical below is a careful way of answering that one plain question.

PICTURE. Two clouds of dots for one feature (say, applicant age). The reference cloud sits low; the current cloud has drifted right. Same axis, different pile-up.

---

Step 2 — Turn each pile into bars (binning)

WHY bin at all? Raw dots are hard to compare. But if both piles use the same bin edges, then "shape" becomes "the list of bar heights," and comparing two shapes becomes comparing two lists of numbers — something arithmetic can chew on.

PICTURE. The two dot clouds from Step 1 are now two histograms sharing identical bin edges: reference bars in one colour, current bars in another, standing in the same slots.

---

Step 3 — From counts to proportions (make piles comparable)

WHY proportions and not counts. We care about shape, not size. Fractions strip out how many samples you happened to collect and keep only "what share of the pile sits here." Now both piles live on the same $0$-to-$1$ footing.

PICTURE. The two histograms re-scaled so each one's bars add up to a full unit of height — same silhouettes as Step 2, but now directly stackable against each other.

---

Step 4 — How "surprised" is one bin? Enter the log-ratio

WHY a ratio, not a difference? A difference like $c_i-r_i$ treats "$0.01$ vs $0.02$" the same as "$0.50$ vs $0.51$". But doubling a bin's share (going from $1\%$ to $2\%$) is a big relative event; adding $1\%$ to a $50\%$ bin is a shrug. Ratios capture relative change — the thing that actually signals drift.

WHY the logarithm of the ratio? Look at the picture: a ratio of $2$ (doubled) and a ratio of $\frac{1}{2}$ (halved) are equally dramatic surprises, but as plain ratios they sit at wildly different distances from $1$ ($2$ vs $0.5$). The $\ln$ (natural log) is the tool that answers "how many e-foldings of change is this?" — and it makes doubling and halving symmetric: $\ln 2=0.693$ and $\ln \frac{1}{2}=-0.693$, mirror images across zero. That symmetry is exactly what "surprise" should have.

PICTURE. One bin, drawn three ways: current below reference ($\ln$ negative), current equal ($\ln =0$), current above ($\ln$ positive). The $\ln$ curve passing through $(1,0)$ shows how ratios fold into a signed surprise.

---

Step 5 — Fix the asymmetry: weight the log-ratio by the actual gap

WHY multiply by $(c_i-r_i)$? Two payoffs at once:

1. It down-weights trivial bins. A tiny bin has a tiny gap $(c_i-r_i)$, so its contribution shrinks no matter how dramatic its ratio.

2. It forces every term positive. Here is the beautiful part — watch the signs:

   • If the bin grew ($c_i>r_i$): gap is $+$, and $\ln \frac{c_i}{r_i}$ is also $+$. Product: $(+)(+)=+$.
   • If the bin shrank ($c_i<r_i$): gap is $-$, and $\ln \frac{c_i}{r_i}$ is also $-$. Product: $(-)(-)=+$.
   • If the bin is unchanged ($c_i=r_i$): gap is $0$. Product: $0$.

   The gap and the log-ratio always share the same sign, so their product can never be negative. Drift can only add up, never cancel.

PICTURE. The sign-agreement drawn explicitly: a "grew" bin (both arrows up-right, green product) and a "shrank" bin (both arrows down-left, still positive product) — showing why the term is a valid distance-like quantity.

---

Step 6 — Sum over all bins: the whole PSI

WHY sum? Each ${\text{PSI}}_i\ge 0$ is an independent "amount of surprise" in bin $i$. Adding them gathers the whole shape-mismatch into one score. Because no term is negative, a big surprise in any region cannot be hidden by calm elsewhere — the alarm can only get louder.

PICTURE. A row of the per-bin bars from Step 5 being stacked into a single total column labelled PSI, with the rule-of-thumb thresholds ($0.1$, $0.25$) drawn as horizontal lines.

---

Step 7 — The degenerate case: an empty bin ($r_i=0$ or $c_i=0$)

WHY it happens & WHY it matters. A brand-new region of feature space appearing (or an old region vanishing) is genuinely infinite "surprise" — the model has literally never seen it. But an infinite score is useless for a threshold, and one empty bin caused by small samples shouldn't nuke your whole metric.

THE FIX (standard practice). Add a tiny floor $\epsilon$ (e.g. $0.0001$) to any empty proportion before taking the log, or merge bins so none are empty. This keeps PSI finite while still reporting a large-but-usable value for genuinely new regions.

PICTURE. A bin where reference is flat-zero and current pokes up; the $\ln$ arrow shooting to infinity, then the $\epsilon$-floor clamping it down to a big-but-finite bar.

---

Step 8 — Walk the worked example on the picture

WHY re-visit it. Numbers only stick once you watch them come from geometry. Each bin's contribution is literally "gap $\times$ log-ratio," and the picture shows which bins are shouting.

$$\begin{array}{rl}\text{Bin 1:}& (0.3-0.5)\ln \frac{0.3}{0.5}=(-0.2)(-0.5108)=0.1022\\ \text{Bin 2:}& (0.3-0.3)\ln \frac{0.3}{0.3}=0\times 0=0\\ \text{Bin 3:}& (0.4-0.2)\ln \frac{0.4}{0.2}=(0.2)(0.6931)=0.1386\\ \text{Total:}& \text{PSI}\approx 0.2408\end{array}$$

• Bin 1 shrank ($0.5\to 0.3$): gap $-0.2$, log $-0.51$, product positive $0.102$. Sign-agreement in action (Step 5).
• Bin 2 unchanged: contributes exactly $0$.
• Bin 3 grew ($0.2\to 0.4$, doubled): the doubling gives $\ln 2$, product $0.139$.
• Total $\approx 0.24$ → moderate-to-significant. Investigate before triggering retraining.

PICTURE. The three bins with each term annotated on top of its bar; bins 1 and 3 lit up, bin 2 dark, and the running sum climbing to $0.24$ against the threshold lines.

---

The one-picture summary

Recall Feynman retelling — the whole walk in plain words

Imagine you have two boxes of the same toy sorted into the same shelves. The old box is how the world looked when you trained your model; the new box is today. You want one number for "did the toys move shelves?"

First you count what fraction sits on each shelf in each box — that's $r_i$ (old) and $c_i$ (new), using fractions so a bigger box doesn't cheat. For each shelf you ask two things: which way did it change and by how much? (that's the gap $c_i-r_i$) and how dramatic is the change relative to before? (that's the log-ratio $\ln \frac{c_i}{r_i}$, which treats "doubled" and "halved" as equally shocking). You multiply them — and magic: because a shelf that grew has both pieces positive, and a shelf that shrank has both pieces negative, the product is always positive. Surprise can only pile up, never cancel. Add every shelf's surprise and you get PSI. Under $0.1$ nothing moved; over $0.25$ the toys really rearranged — go check your model. And if a shelf is suddenly used that was empty before, the log tries to explode to infinity, so you tuck a tiny cushion $\epsilon$ under it to keep the number sane.

---

Connections

• Parent topic (Hinglish)
• KL Divergence and Cross-Entropy
• Kolmogorov-Smirnov Test
• Covariate Shift and Importance Weighting
• Feature Stores and Reference Windows
• Model Monitoring in Production
• Retraining Pipelines and Triggers