Bivariate regression lines
Section of Covariance and the multivariate normal, part of Statistical learning
Bivariate regression lines
| Type | Section |
| Taught in | Covariance and the multivariate normal |
| Part | Statistical learning |
Specialise the conditional theorem to p=2, with A=\{1\} and B=\{2\}. Then \Sigma_{AB}=\rho\sigma_1\sigma_2 and \Sigma_{BB}=\sigma_2^2, so
\Sigma_{AB}\Sigma_{BB}^{-1}=\rho\frac{\sigma_1}{\sigma_2}
and
X_1\mid(X_2=x_2) \sim\operatorname{N}\Bigl(\mu_1+\rho\frac{\sigma_1}{\sigma_2}(x_2-\mu_2),\;\sigma_1^2(1-\rho^2)\Bigr).
The regression of X_1 on X_2 predicts x_1 from x_2 and has slope \rho\sigma_1/\sigma_2 in the (x_2,x_1) plane. The regression of X_2 on X_1 is
x_2=\mu_2+\rho\frac{\sigma_2}{\sigma_1}(x_1-\mu_1).
Solving that equation for x_1 gives slope \sigma_1/(\rho\sigma_2) in the same plane. For \rho\neq 0 the ratio of the two slopes is \rho^2, so the lines agree only in the limit \rho^2=1. When \rho=0 they are the horizontal and vertical lines through the mean. A principal axis may coincide with one of these lines when the covariance is diagonal, and there is no unique principal axis when \Sigma is a multiple of the identity. Each line answers a different question.
Used in
See also
Joint distribution and density · Mean vector and covariance matrix · Correlation · Multivariate normal distribution · Squared Mahalanobis distance · Linear images of a normal vector · Conditional distribution of a multivariate normal · Partial correlation