Jointly normal variables
Section of Functions of random variables and limit theorems, part of Probability
Jointly normal variables
| Type | Section |
| Taught in | Functions of random variables and limit theorems |
| Part | Probability |
Definition 1 (Multivariate normal distribution) A random vector \mathbf{Z} in \mathbb{R}^n is multivariate normal with mean \boldsymbol\mu and covariance matrix \Sigma, when \Sigma is positive definite and the density is
f(\mathbf{z})=(2\pi)^{-n/2}\,|\Sigma|^{-1/2}\,\exp\Bigl(-\tfrac12(\mathbf{z}-\boldsymbol\mu)^\top\Sigma^{-1}(\mathbf{z}-\boldsymbol\mu)\Bigr).
If \mathbf{X} has independent standard normal coordinates and \Sigma=BB^\top, then \boldsymbol\mu+B\mathbf{X} has this distribution. The representation is the definition used when \Sigma is only positive semidefinite and no density exists.
In two dimensions, with means \mu_1,\mu_2, variances \sigma_1^2,\sigma_2^2 and correlation \rho, the density has level sets that are ellipses tilted by \rho. The case \rho=0 is independence. That is special to the normal family: for jointly normal coordinates, zero covariance is independence, because the quadratic form in the exponent then splits into a sum.
Theorem 1 (Affine images of normal vectors) If X_i\sim\operatorname{N}(\mu_i,\sigma_i^2) are independent and a,b_1,\ldots,b_n are constants, then
a+\sum_{i=1}^n b_i X_i \sim \operatorname{N}\Bigl(a+\sum_{i=1}^n b_i\mu_i,\;\sum_{i=1}^n b_i^2\sigma_i^2\Bigr).
The same conclusion holds for a jointly normal vector and any affine image, with the output variance \mathbf{b}^\top\Sigma\mathbf{b}.
A machine that produces diameters X\sim\operatorname{N}(1,0.01) and then records a+bX is still producing a normal measurement. The mean and variance are the only things the affine map can change.
Example 1 (Ball bearings and a sieve) Ball bearings have diameters X\sim\operatorname{N}(1,0.01) cm. They are dropped through a sieve whose holes have diameters Y\sim\operatorname{N}(1.1,0.04) cm, independent of the balls.
Question. What is the probability that a ball falls through a hole?
Solution. The ball falls through when Y>X. Because X and Y are independent normals, Y-X is normal with mean 1.1-1=0.1 and variance 0.04+0.01=0.05. So
\mathbb{P}(Y-X>0)=\mathbb{P}\Bigl(Z>\frac{-0.1}{\sqrt{0.05}}\Bigr)=\Phi(0.447)=0.67.
The variances add even though the variable is a difference.
Used in
See also
Transformation rule · Multivariate normal distribution · Affine images of normal vectors · Normal linear model · Law of large numbers · Central limit theorem · Distribution of a function or a sum · Limit theorems