Marginals and conditionals
Section of Covariance and the multivariate normal, part of Statistical learning
Marginals and conditionals
| Type | Section |
| Taught in | Covariance and the multivariate normal |
| Part | Statistical learning |
Two operations act on a joint distribution. Stopping the recording of a coordinate produces a marginal distribution: that coordinate is integrated out. Knowing the coordinate’s value produces a conditional distribution: the joint is restricted to that value and renormalised. Dropping information and using information are different operations.
The joint density of the first q<p coordinates is
g(x_1,\ldots,x_q)=\int_{-\infty}^{\infty}\cdots\int_{-\infty}^{\infty} f(x_1,\ldots,x_p)\,\,\mathrm{d} x_{q+1}\cdots\,\mathrm{d} x_p.
The corresponding distribution function sends the remaining arguments to infinity,
G(x_1,\ldots,x_q)=\mathbb{P}(X_1\le x_1,\ldots,X_q\le x_q)=F(x_1,\ldots,x_q,\infty,\ldots,\infty).
For conditioning, the ratio \mathbb{P}(A\mid B)=\mathbb{P}(A\cap B)/\mathbb{P}(B) needs care, because an exact value of a continuous coordinate has probability zero. The conditional density is a ratio of densities, defined where the marginal density in the denominator is positive. Split the coordinates into blocks A and B. Then
f_{A\mid B}(\mathbf{x}_A)=\frac{f_{AB}(\mathbf{x}_A,\mathbf{x}_B)}{f_B(\mathbf{x}_B)}, \qquad f_B(\mathbf{x}_B)>0,
and the denominator is the marginal density of the conditioning block,
f_B(\mathbf{x}_B)=\int\cdots\int f(\mathbf{x}_A,\mathbf{x}_B)\,\,\mathrm{d}\mathbf{x}_A.
In the notation of the first q coordinates and the last p-q,
f(x_1,\ldots,x_q\mid x_{q+1},\ldots,x_p) =\frac{f(x_1,\ldots,x_p)}{g(x_{q+1},\ldots,x_p)}.
Here g(x_{q+1},\ldots,x_p) is the marginal density of the last p-q variables, evaluated at the conditioning values. Whenever both marginal densities are positive,
f_{AB}=f_{A\mid B}\,f_B=f_{B\mid A}\,f_A,
and Bayes’ rule for densities is
f_{A\mid B}(\mathbf{x}_A)=\frac{f_{B\mid A}(\mathbf{x}_B)\,f_A(\mathbf{x}_A)}{f_B(\mathbf{x}_B)}.
Used in
See also
Joint distribution and density · Mean vector and covariance matrix · Correlation · Multivariate normal distribution · Squared Mahalanobis distance · Linear images of a normal vector · Conditional distribution of a multivariate normal · Partial correlation