Conditional expectation
Topic of Joint distributions, part of Probability · Also in Covariance and the multivariate normal
Conditional expectation
| Type | Topic |
| Taught in | Joint distributions · Covariance and the multivariate normal |
| Part | Probability · Statistical learning |
| Questions using it | 16 |
\mathbb{E}(Y\mid X=x) is the mean of Y under its conditional distribution given X=x. As a function of X it is the random variable \mathbb{E}(Y\mid X), whose average is \mathbb{E}Y by the tower property. Among functions of X it is the best predictor of Y in mean squared error, and for a jointly normal pair it is linear in x.
In Joint distributions
From Joint distributions
The conditional expectation \mathbb{E}(Y\mid X=x) is the expectation of Y under that conditional distribution. As a function of the random point X it is a random variable, written \mathbb{E}(Y\mid X).
In Covariance and the multivariate normal
Least squares of a response Y on a predictor X minimises \sum_i(y_i-a-bx_i)^2 and treats the observed x_i as fixed. The unconstrained bivariate normal, fitted to the same complete pairs, estimates (\widehat\mu_X,\widehat\mu_Y) and a covariance with one divisor in every entry, whether that divisor is n or n-1. Its conditional mean of Y given X=x is
\widehat\mu_Y+\frac{s_{YX}}{s_{XX}}(x-\widehat\mu_X).
The slope s_{YX}/s_{XX} and the intercept \widehat\mu_Y-(s_{YX}/s_{XX})\widehat\mu_X are exactly the least-squares slope and intercept. The Gaussian fit and the least-squares fit draw the same conditional-mean line. They do not estimate the same object. Least squares does not model the marginal distribution of X, and it does not by itself supply the conditional variance s_{YY}-s_{YX}^2/s_{XX}. The major axis of the fitted ellipse is a different line, and so is the line obtained by regressing X on Y and solving for Y. The laboratory checks the agreement of the first two lines on Iris setosa.
Used in
- Taught in Joint distributions · Covariance and the multivariate normal
- Also used in Stochastic processes · Galton–Watson branching · Linear regression · Bayesian linear regression · Introduction to generalised linear models · Covariance and the multivariate normal · Mixtures and the EM algorithm · Hierarchical clustering and kernel density estimation · Logistic regression · Graphical models
- Questions that use it (16) — The variance recurrence (answer) · The bivariate conditional (answer) · Conditional normal prediction (answer) · Normal conditioning and whitening (answer) · Q23 (answer) · Q30 (answer) · Q33 (answer) · Q45 (answer) · Q54 · Q55 · Question 19 (answer) · Question 22 (answer) · C17 (answer) · C23 (answer) · C33 (answer) · C50 (answer)
- All 8 questions on Joint distributions
See also
Conditional distribution · Random variable · Tower property · Mean squared error · Multivariate normal distribution · Point forecast · Response variable · Joint probability mass function