Random intercept model

Section of Random intercepts, part of Correlated data

Random intercept model

Type Section
Taught in Random intercepts
Part Correlated data
Questions using it 6

Index the block by j and the unit inside the block by i: Y_{ji}=\beta_0+{\boldsymbol{X}}_{ji}^\top\boldsymbol{\beta}+\alpha_j+\varepsilon_{ji}, \alpha_j\stackrel{\mathrm{ind}}{\sim}\mathrm{N}(0,\sigma_A^2), \qquad \varepsilon_{ji}\stackrel{\mathrm{ind}}{\sim}\mathrm{N}(0,\sigma^2), with the block effects independent of the residuals. Drop the distribution of \alpha_j and this is an ordinary model with a separate intercept for each block. The extra line says those intercepts are themselves a sample from a population of blocks.

The mean is still the straight line, because both \alpha_j and \varepsilon_{ji} have mean zero. The variance of one response is \sigma_A^2+\sigma^2. Two people in different blocks share nothing, so their covariance is zero. Two people in the same block share \alpha_j, so \mathrm{Cov}(Y_{ji},Y_{ji'})=\sigma_A^2, \qquad \mathrm{Corr}=\frac{\sigma_A^2}{\sigma_A^2+\sigma^2}. The correlation is between 0 and 1. If the blocks are identical, \sigma_A^2\to 0 and the correlation vanishes: it was one big sample. If the block effect swamps the person-to-person noise, the correlation tends to 1: everyone in the hospital looks the same. A linear mixed model of this form cannot produce a negative within-block correlation.

Figure 1: A random intercept. Each block’s line is the population line \beta_0+\beta_1x (dashed) shifted by its own \alpha_j, so the lines are parallel. Observations in one block share that shift, which is the source of the within-block correlation \sigma_A^2/(\sigma_A^2+\sigma^2).

Negative dependence does happen, when units in a block compete for a fixed resource. It is an awkward correlation to write down for every pair at once. A 3\times 3 equicorrelation matrix is positive definite for any correlation in [0,1), but if every pair has correlation -\rho, the matrix is only valid for \rho\le 1/2. In a block of size K the lower bound is -1/(K-1). Add one more person to the block and a correlation that used to be legal becomes illegal. That dependence on the block size is why the random-intercept model’s refusal of negative correlations is acceptable. Competition can be modelled later by letting some pairs differ. A single negative number for every pair in a large block is not a structure worth having.

Used in

See also

Hierarchical Bayesian model · Blocks in analysis of variance · Randomised block design · Linear mixed model · Fixed and random effects · Restricted maximum likelihood · Intraclass correlation · Repeated measures

Glossary and index