Linear mixed model
Topic of Random intercepts, part of Correlated data · Also in Random intercepts
Linear mixed model
| Type | Topic |
| Taught in | Random intercepts · Random intercepts |
| Part | Correlated data |
| Questions using it | 1 |
A linear model with fixed effects, shared across all the data, and random effects drawn afresh for each block, so that observations in one block are correlated: \mathbf{y}=\mathbf{X}\boldsymbol\beta+\mathbf{Z}\mathbf{b}+\boldsymbol\varepsilon. The random intercept is the simplest case.
In Random intercepts
From Random intercepts
We generate the dependence with a mixed model rather than write down a correlation matrix and stop. A correlation matrix describes association. For a normal response, a mean and a covariance do define a distribution. For a Poisson or a binomial they do not: there is no unique multivariate Poisson with a given set of correlations. A mixed model gives a mechanism. The price is that the correlations it can produce are a narrower set.
In Random intercepts, continued
From Random intercepts
Index the block by j and the unit inside the block by i: Y_{ji}=\beta_0+{\boldsymbol{X}}_{ji}^\top\boldsymbol{\beta}+\alpha_j+\varepsilon_{ji}, \alpha_j\stackrel{\mathrm{ind}}{\sim}\mathrm{N}(0,\sigma_A^2), \qquad \varepsilon_{ji}\stackrel{\mathrm{ind}}{\sim}\mathrm{N}(0,\sigma^2), with the block effects independent of the residuals. Drop the distribution of \alpha_j and this is an ordinary model with a separate intercept for each block. The extra line says those intercepts are themselves a sample from a population of blocks.
The mean is still the straight line, because both \alpha_j and \varepsilon_{ji} have mean zero. The variance of one response is \sigma_A^2+\sigma^2. Two people in different blocks share nothing, so their covariance is zero. Two people in the same block share \alpha_j, so \mathrm{Cov}(Y_{ji},Y_{ji'})=\sigma_A^2, \qquad \mathrm{Corr}=\frac{\sigma_A^2}{\sigma_A^2+\sigma^2}. The correlation is between 0 and 1. If the blocks are identical, \sigma_A^2\to 0 and the correlation vanishes: it was one big sample. If the block effect swamps the person-to-person noise, the correlation tends to 1: everyone in the hospital looks the same. A linear mixed model of this form cannot produce a negative within-block correlation.
Negative dependence does happen, when units in a block compete for a fixed resource. It is an awkward correlation to write down for every pair at once. A 3\times 3 equicorrelation matrix is positive definite for any correlation in [0,1), but if every pair has correlation -\rho, the matrix is only valid for \rho\le 1/2. In a block of size K the lower bound is -1/(K-1). Add one more person to the block and a correlation that used to be legal becomes illegal. That dependence on the block size is why the random-intercept model’s refusal of negative correlations is acceptable. Competition can be modelled later by letting some pairs differ. A single negative number for every pair in a large block is not a structure worth having.
Used in
- Taught in Random intercepts · Random intercepts
- Also used in Exponential families and conjugacy · Hierarchical Bayes · Random slopes, and a binary mixed model · Marginal models, and two responses at once · How to use the final preparation
- Questions that use it (1) — C34
- All 1 questions on Random intercepts
See also
Fixed and random effects · Random intercept model · Repeated measures · Restricted maximum likelihood · Intraclass correlation