What a regression statement is hiding

Section of Introduction to generalised linear models, part of Generalised linear models

Type	Section
Taught in	Introduction to generalised linear models
Part	Generalised linear models

If someone says “let us fit a regression” and says nothing else, they usually mean Y_i\mid {\boldsymbol{X}}_i={\boldsymbol{X}}_i^\top\boldsymbol{\beta}+\varepsilon_i. That one line is hiding four assumptions. We number independence as zero because people so rarely say it out loud.

The responses are independent given the covariates.
The conditional mean is linear in the covariates: \mathrm{E}(Y\mid {\boldsymbol{X}})={\boldsymbol{X}}^\top\boldsymbol{\beta}.
The errors, and therefore the responses, have constant variance.
The errors, and therefore the responses, are normal.

A transparent way to write the same model is Y_i\mid {\boldsymbol{X}}_i\stackrel{\mathrm{ind}}{\sim} \mathrm{N}({\boldsymbol{X}}_i^\top\boldsymbol{\beta},\sigma^2). Independence is the \mathrm{ind}. Normality and the mean are inside the \mathrm{N}(\cdot). Constant variance is the single \sigma^2, not a \sigma_i^2. If you are uncomfortable with {\boldsymbol{X}}^\top\boldsymbol{\beta}, read it as \beta_0+\beta_1 x_1+\beta_2 x_2+\cdots.

Equation Equation (1) in Introduction to generalised linear models looks innocent, and it is not. Before we replace it, we will watch it fail on three data sets, with a different assumption failing each time.

What a regression statement is hiding

Used in

See also