Data drift and concept drift detection: Foundations

This page assumes nothing. Before you can read " $P (Y ∣ X)$ changes" or " $PSI = \sum (c_{i} - r_{i}) ln \frac{c _{i}}{r _{i}}$ ", every one of those squiggles has to earn its place. We build them one at a time, each on top of the last, each with a picture.

0. What is a "distribution" (the seed of everything)

Before any symbol, the single mental image you must own is a distribution: not a number, but a shape describing how often each value shows up.

Figure — Data drift and concept drift detection

Look at the picture: the horizontal axis is the value (age), the vertical axis is how common that value is. The whole topic of drift is nothing but comparing two of these shapes — one from the past, one from now — and measuring how far apart they drifted.

Why does the topic need this? Because a model was tuned to one shape. If the shape moves, the model's assumptions crack.

Definition Discrete vs. continuous distributions (PMF vs. PDF)

Values come in two textures:

Discrete (categories or whole counts, e.g. "loan approved / declined", "number of missed payments"). Here the shape is a PMF (probability mass function): a real height sitting on each value, and those heights add to 1. This is the bar-chart picture above.
Continuous (any real number, e.g. income = $41{,}382.17). No single exact value has non-zero probability — there are infinitely many. So the shape is a PDF (probability density function): a smooth curve where area under a stretch gives the probability of landing in that stretch. The total area under the whole curve is 1.

Why this matters for drift: PSI, KL and chi-square all want heights that add to 1. A continuous feature has no such heights — so we first bin it (Section 6): chop the curve into slices and let each slice's area become a discrete height. That is the bridge from the smooth world to the bar-chart tools. When we do not want to bin at all, we compare the smooth curves directly with the KS test (Section 7), which is built for continuous features.

1. The symbols $X$ and $Y$

The picture: imagine a machine with input slots ( $X$ ) on the left and a single output light ( $Y$ ) on the right. The model is the wiring in between.

Why the topic needs this: drift is defined entirely by which of these two things — the inputs $X$ or the input→answer wiring — has changed.

2. The symbol $P (\cdot)$ — "probability of"

3. Conditional probability $P (Y ∣ X)$ — the "rule"

This is the most important symbol on the whole parent page, so we build it slowly.

The picture separates the two ideas cleanly:

$P (X)$ = which people walk through the door (the input shape).
$P (Y ∣ X)$ = for a fixed person, how the world decides their outcome (the rule).

Why the topic needs this split: drift comes in exactly two flavours, and they map perfectly onto these two objects. Data drift = the door-crowd $P (X)$ changed. Concept drift = the deciding rule $P (Y ∣ X)$ changed. Keep this figure in your head for the entire topic.

4. The joint distribution $P (X, Y)$ and the chain rule

5. Sample vs. true distribution — why we compare windows

Why the topic needs this: it turns an impossible question ("did the true $P$ change?") into a doable one ("do these two finite samples look alike?").

6. Binning, and the symbols $r_{i}$ , $c_{i}$

To compare shapes numerically we chop the value axis into slices called bins. This is exactly the "turn a continuous PDF into discrete heights" bridge promised in Section 0.

The picture shows the same axis binned twice — reference bars and current bars side by side. PSI, KL, and chi-square all just walk bin-by-bin and add up how much the bars disagree. Once you see $r_{i}$ and $c_{i}$ as "two bar heights in the same bin," those formulas stop being scary.

7. The tools that combine those bins

Now that $r_{i}$ and $c_{i}$ exist as pictures, we can name the machinery the parent uses — but only name it here (D2 derives them fully).

Two symbols still need unpacking: $F$ and $sup$ (for KS), and $O_{i}, E_{i}$ (for chi-square). We define them right where each tool uses them.

Formula Chi-square — categorical, with the symbols defined

$χ^{2} = \sum_{i = 1}^{k} \frac{( O _{i} - E _{i} ) ^{2}}{E _{i}}$ Here we count whole points, not fractions:

$O_{i}$ (Observed) = the actual number of current points in category $i$ . If the current window has $N$ points, then $O_{i} = c_{i} N$ — the same $c_{i}$ from Section 6, un-normalised back into a raw count.
$E_{i}$ (Expected) = the count we'd expect if nothing drifted, i.e. the reference proportion applied to the current window: $E_{i} = r_{i} N$ .

Why this form? $(O_{i} - E_{i})$ is the raw mismatch between "what we saw" and "what the old distribution predicted." Squaring makes every mismatch positive and punishes big gaps harder. Dividing by $E_{i}$ scales the gap to what's normal: a mismatch of 10 is huge in a category expecting 5 but trivial in one expecting 5000. Summing over categories gives one number that grows only when the categorical mix genuinely shifts.

8. Symbols for the concept-drift side

Concept drift needs the answer $Y$ , so its symbols live around error. First, pin down the index.

The prerequisite map

The graph below is deliberately small: read it in three streams. (1) The far-left column is the conceptual spine — a distribution splits into the two factors, which combine and then split again into the two drift types. (2) The middle stream is the data-drift toolkit — samples become bins, bins feed the log-ratio scores. (3) The right stream is the two specialist tools — CDF-based KS and error-based DDM. All three streams pour into "full drift detection" at the bottom.

Equipment checklist

Cover the answers; if you can state each, you are ready for D2.

What is a "distribution" in one sentence?

A description of how often each value occurs — a bucket-shape / histogram (discrete) or density curve (continuous) over the values.

What is the difference between a PMF and a PDF?

A PMF gives a real probability height on each discrete value (heights sum to 1); a PDF is a smooth curve where area under a range gives the probability (total area 1) and any single exact value has probability 0.

How do you feed a continuous feature into PSI/KL, which need heights that sum to 1?

Bin it — chop the range into slices and let each slice's fraction of points become a discrete height; or skip binning and use the KS test directly.

What is the difference between

P (X)

and

P (X = x)

P (X)

is the whole shape (one height per value);

P (X = x)

is the single height at the specific value

x

Read

P (Y ∣ X)

aloud and say what it is.

"Probability of answer

Y

given inputs

X

" — the model's input→output rule.

Write the chain rule and say why it forces exactly two drift types.

P (X, Y) = P (Y ∣ X) P (X)

; the world can only drift if one factor drifts, so it's data drift or concept drift, never a third.

Which factor moves in data drift? In concept drift?

Data drift =

P (X)

moves; concept drift =

P (Y ∣ X)

moves.

Why compare a reference window to a current window instead of measuring

P

directly?

We never observe the true

P

, only finite samples, so we compare two samples.

What are

r_{i}

and

c_{i}

, and what breaks if a bin is empty?

Fractions of reference/current points in bin

i

; an empty bin gives

ln 0 = - \infty

or divide-by-zero, so add a small floor

ε

(smoothing) or merge the bin.

In chi-square, what are

O_{i}

and

E_{i}

, and how do they relate to

r_{i}, c_{i}

O_{i} = c_{i} N

= observed current counts;

E_{i} = r_{i} N

= counts expected from reference proportions on a window of

N

points.

Why does chi-square divide

(O_{i} - E_{i})^{2}

E_{i}

To scale each mismatch relative to what's normal for that category — a gap of 10 is huge if you expected 5, trivial if you expected 5000.

What does

F (x)

mean and what does

sup

pick out in the KS test?

F (x)

= fraction of points

\leq x

(the cumulative staircase);

sup

picks the largest vertical gap between the two staircases.

In the concept-drift formulas, what does the index

i

count?

A running time step / sample number as predictions stream in — not a bin.

What does

s_{i} = p_{i} (1 - p_{i}) / n_{i}

measure?

The natural random wobble (standard deviation) of the error-rate estimate at step

i

; it shrinks as

n_{i}

grows.

Why does DDM wait for a

3 σ

jump?

A rise that large is a ~0.1% luck event, so it's almost surely real degradation, not noise.