4.9.10 · D5Probability Theory & Statistics

Question bank — Joint distributions — joint PMF - PDF, marginal, conditional

1,273 words6 min readBack to topic

True or false — justify

A joint PDF value is a probability.
False — it is a density (probability per unit area). It can even exceed ; only over a region is a probability.
If for all , then and are independent.
True — factoring of the joint into the product of marginals for every point is exactly the definition of Independence of Random Variables.
If and have identical marginal distributions, they must be independent.
False — marginals describe each variable alone and say nothing about their joint behaviour; equal marginals can coexist with strong dependence.
Every valid marginal comes from some joint, but a given pair of marginals determines a unique joint.
False — many different joints share the same two marginals (they differ in how the variables move together), so the pair of marginals does not pin the joint down.
for every fixed with .
True — the conditional is renormalized by dividing by , so as a function of it integrates to exactly ; it is a genuine distribution in .
.
False — integrating the conditional over as well double-counts; it need not equal . Only the integral over (with frozen) equals .
If then and are independent.
False — zero Covariance and Correlation means no linear association; nonlinear dependence (e.g. with symmetric ) can give zero covariance yet full dependence.
If and are independent then .
True — independence makes , so covariance vanishes. (The converse fails, as above.)
can be larger than .
True — dividing the joint entry by inflates it, so the conditional probability is at least as large as the joint value.
A conditional PDF is a function of as well as .
True in form, but for a fixed it is a distribution in ; different conditioning values generally give different-shaped slices.

Spot the error

"To get , just delete the column you don't care about."
Wrong — deleting keeps one -value; marginalizing means summing over all -values: . You collapse, never drop.
"The conditional density is simply the joint evaluated along the line ."
Wrong — that slice integrates to , not . You must divide by to renormalize it into a valid distribution.
"For on the triangle , ."
Wrong bounds — on this triangle runs only from to , so . Ignoring the support gives the wrong marginal.
"Since splits as (a function of ) times (a function of ), and are independent."
Wrong — the support couples the variables (the allowed range of depends on ). Factoring must hold including a product-shaped support; here it does not.
" because both slice the same joint entry."
Wrong — both use on top but divide by different marginals ( vs ), so they are generally unequal (this is the seed of Bayes' Theorem).
"A density integrates to , so it must stay below everywhere."
Wrong — a tall narrow density (e.g. a spike on a short interval) exceeds in height while still enclosing unit area. Height is unbounded; only total area is fixed.
"If I can still condition on by dividing."
Wrong — dividing by is undefined; the conditional is only defined where , i.e. where that value of actually carries density.

Why questions

Why do we use a density instead of point probabilities in the continuous case?
Because a single point has zero area, so ; density measures probability per unit area, recovering real probability only when integrated over a region.
Why does summing/integrating out a variable give the marginal?
Because is a union of disjoint events; additivity of probability turns that union into a sum (discrete) or integral (continuous) — this is the Law of Total Probability.
Why must we divide by the marginal when forming a conditional?
The frozen slice has total mass (or ), not ; dividing rescales it so the new sample space integrates to and is a legitimate distribution.
Why does independence let us write ?
Under independence, knowing tells you nothing about , so ; substituting into the chain rule gives the product form.
Why is the chain rule symmetric in the two orders?
Both expressions reconstruct the same joint from a slice times its weight; conditioning on first or first are two routes to the identical surface — equating them yields Bayes' rule.
Why can the support region alone destroy independence even when the formula factors?
Independence needs the joint to factor as over a rectangular support; a triangular or curved support makes the allowed values of one variable depend on the other, which is dependence.
Why does knowing only marginals leave Covariance and Correlation undetermined?
Covariance depends on , a joint quantity; the marginals fix and but many joints with those marginals give different .

Edge cases

What happens to at a value where ?
It is left undefined — there is no probability mass to condition on, so the ratio has no meaning there.
For on , what is the support of once is fixed?
Only — freezing forces , so the conditional lives on the shrunken interval, not all of .
If takes a single value with probability (a degenerate/constant variable), is it independent of any ?
Yes — a constant carries no randomness, so ; the factorization holds trivially.
In a discrete joint table, what does a full row of zeros for mean, and what is there?
It means (that -value never occurs); the conditional for every , and conditioning on is undefined.
If the conditional is the same function of for every , what does that tell you?
That 's distribution does not react to , i.e. — the definition of independence.
What is at any point outside the stated support region?
Exactly — the "and elsewhere" clause is part of the density; forgetting it wrongly extends integration bounds and breaks the total-mass- check.