Joint, marginal, and conditional distributions: Question bank

True or false — justify

The joint

p_{X, Y}

contains strictly more information than the pair of marginals

p_{X}

and

p_{Y}

True — the marginals throw away how the variables move together; many different joints share the same two marginals but differ in their dependence structure.

If you know both marginals

p_{X}

and

p_{Y}

, you can always reconstruct the joint.

False — only if you also assume independence, giving

p_{X, Y} = p_{X} p_{Y}

; without that assumption the coupling between

X

and

Y

is unrecoverable.

A conditional distribution

p_{Y ∣ X} (\cdot ∣ x)

is a genuine probability distribution over

y

True — after dividing the slice by

p_{X} (x)

it sums (or integrates) to exactly 1, so it satisfies all the axioms in the variable

y

p_{Y ∣ X} (y ∣ x)

must be

\leq 1

for continuous variables.

False — that is a density, not a probability, so like any pdf it can exceed 1 as long as it integrates to 1 over

y

X

and

Y

are independent then

p_{Y ∣ X} (y ∣ x) = p_{Y} (y)

for every

x

with

p_{X} (x) > 0

True — independence means conditioning on

X

changes nothing, so the conditional collapses to the marginal.

Uncorrelated random variables are always independent.

False — independence implies zero correlation, but the converse fails; e.g.

Y = X^{2}

with symmetric

X

is uncorrelated yet fully dependent (see Covariance and correlation).

p (X ∣ Y)

and

p (Y ∣ X)

are just the same thing written two ways.

False — they have different denominators (

p_{Y}

p_{X}

) and answer opposite questions; equating them is the prosecutor's fallacy, fixed only by Bayes' theorem.

The chain rule

p_{X, Y} = p_{Y ∣ X} p_{X}

requires

X

and

Y

to be dependent.

False — it holds for any joint; if they happen to be independent it simply reduces to

p_{Y} p_{X}

Summing a conditional

p_{Y ∣ X} (y ∣ x)

over all

x

gives 1.

False — a conditional normalizes over its free variable

y

, not the conditioning variable

x

; summing over

x

has no reason to give 1.

Marginalizing out

Y

from

p_{X, Y}

and marginalizing out

X

from the same joint gives the two marginals of a single consistent model.

True — both are collapses of the one joint, so they are automatically compatible; no separate consistency check is needed.

Spot the error

"To get the marginal of spam, I just read the entry

p (1, 1) = 0.25

from the joint table."

Wrong — a marginal is the sum over the whole row,

p_{X} (1) = p (1, 0) + p (1, 1)

; picking one cell forgets the other value of the summed-out variable.

"The conditional is literally the

X = x

slice of the joint."

Incomplete — that slice sums to

p_{X} (x)

, not 1, so it is not yet a distribution; you must divide by the marginal to renormalize (slice then rescale).

f_{X, Y} (x, y) = 4 x y

can't be a density because

4 x y

reaches 4, which exceeds 1."

Wrong — densities may exceed 1; validity is about integrating to 1 over the region, and

\iint_{0}^{1} 4 x y d x d y = 1

"Since

p (X ∣ Y) = 0.714

, we know

p (Y ∣ X) = 0.714

too."

Wrong — reversing the bar changes the denominator; you must convert via Bayes' theorem, and the two values genuinely differ.

X

and

Y

are independent because their covariance is 0."

Wrong — zero covariance only rules out linear association; independence requires the full factorization

p_{X, Y} = p_{X} p_{Y}

(see Independence and conditional independence).

"To check independence I confirmed

p (1, 1) = p_{X} (1) p_{Y} (1)

, so they're independent."

Wrong — factorization must hold for every cell

(x, y)

; a single matching entry can occur by coincidence while other cells disagree.

"The joint density

f_{X, Y}

is a probability, so

f_{X, Y} (0.5, 0.5)

is the chance of that point."

Wrong — for continuous variables any single point has probability 0; the density must be integrated over a region to yield a probability.

p_{Y ∣ X} (y ∣ x) = 2 y

depends on

y

, therefore

X

and

Y

are dependent."

Wrong — dependence is about whether the conditional depends on the conditioning variable

x

; here it doesn't, so they are independent.

Why questions

Why do we divide by the marginal rather than by some other constant when conditioning?

Because

p_{X} (x)

is exactly the total mass sitting in the

X = x

slice, so dividing by it is the unique rescaling that makes the slice sum to 1.

Why is the marginal called "marginal"?

In a joint table the row and column sums are written in the margins, and those edge totals are precisely the single-variable distributions.

Why does the joint let you answer any question about

X

and

Y

, but the marginals don't?

The joint records every combination's probability, so you can sum any subset you like; marginals have already discarded the co-occurrence information.

Why does the product rule give two different factorizations of the same joint?

You can peel off either variable first —

p_{Y ∣ X} p_{X}

p_{X ∣ Y} p_{Y}

— and both must equal the joint, which is exactly what forces Bayes' theorem.

Why does the law of total probability sum

p_{Y ∣ X} (y ∣ x) p_{X} (x)

over

x

Each term is the joint

p (x, y)

(chain rule), and summing the joint over

x

marginalizes

X

away to leave

p_{Y} (y)

— see Marginalization and the law of total probability.

Why can two people with the same marginals still disagree about

P (spam ∣ "free")

The conditional depends on the joint's dependence structure, which the marginals do not pin down; identical margins allow different couplings.

Why does the Naive Bayes classifier only need conditionals given the label, not the full joint over features?

It assumes features are conditionally independent given the label, so the joint over features factorizes into a product of one-dimensional conditionals — the modelling shortcut behind Naive Bayes classifier.

Edge cases

What happens to

p_{Y ∣ X} (y ∣ x)

when

p_{X} (x) = 0

It is undefined — you cannot condition on an event of probability zero via division, since dividing by 0 has no meaning (continuous cases need limiting definitions).

X

and

Y

are independent, what does the joint table "look like"?

Every cell equals the product of its row and column margins, so the table is a rank-1 outer product

p_{X} (x) p_{Y} (y)

with no extra structure.

What is the conditional

p_{Y ∣ X} (y ∣ x)

when

Y

is a deterministic function of

X

, say

Y = g (X)

It is a spike: probability 1 on

y = g (x)

and 0 elsewhere, since knowing

X

removes all uncertainty about

Y

Can a marginal be uniform while the joint is highly dependent?

Yes — e.g.

X, Y

each uniform but perfectly correlated (

Y = X

); flat margins say nothing about the coupling inside the joint.

If every conditional

p_{Y ∣ X} (\cdot ∣ x)

is identical across all

x

, what does that imply?

X

carries no information about

Y

, which is exactly independence:

p_{Y ∣ X} = p_{Y}

and hence

p_{X, Y} = p_{X} p_{Y}

What is the smallest joint that can still exhibit dependence?

2 \times 2

table already can — dependence appears the moment even one cell deviates from the product of its margins.

For a valid joint pdf on a region, must the density be positive everywhere in that region?

No — it may be zero on parts of the region (assigning them no mass); only the integral over the whole region must equal 1 and the density must stay nonnegative.

p_{X} (x) > 0

but

p_{X, Y} (x, y) = 0

for a particular

y

, is that a contradiction?

No — it just means the value

y

never co-occurs with that

x

, so the conditional

p_{Y ∣ X} (y ∣ x) = 0

there, perfectly consistent.

Connections

Bayes' theorem — the fix for every "swapped the bar" trap above.
Independence and conditional independence — factorization is the real test, not correlation.
Covariance and correlation — why uncorrelated ≠ independent.
Naive Bayes classifier — leans on conditional independence given the label.
Marginalization and the law of total probability — the "why sum over $x$ " questions.
Probabilistic graphical models — which conditionals factor the joint.
Expectation and variance — quantities computed against these distributions.