Maximum likelihood for a normal mean vector
Section of Estimation and inference for the multivariate normal, part of Statistical learning
Maximum likelihood for a normal mean vector
| Type | Section |
| Taught in | Estimation and inference for the multivariate normal |
| Part | Statistical learning |
The log-likelihood depends on \boldsymbol{\mu} only through the sum of quadratic forms, and \Sigma is an arbitrary positive definite matrix. Split each residual into a deviation from the sample mean and a deviation of the sample mean from \boldsymbol{\mu}:
\begin{aligned} \sum_{i=1}^n(\mathbf{x}_i-\boldsymbol{\mu})^\top\Sigma^{-1}(\mathbf{x}_i-\boldsymbol{\mu}) &=\sum_{i=1}^n\bigl[(\mathbf{x}_i-\bar{\mathbf{x}})+(\bar{\mathbf{x}}-\boldsymbol{\mu})\bigr]^\top \Sigma^{-1} \bigl[(\mathbf{x}_i-\bar{\mathbf{x}})+(\bar{\mathbf{x}}-\boldsymbol{\mu})\bigr]\\ &=\sum_{i=1}^n(\mathbf{x}_i-\bar{\mathbf{x}})^\top\Sigma^{-1}(\mathbf{x}_i-\bar{\mathbf{x}}) +n(\bar{\mathbf{x}}-\boldsymbol{\mu})^\top\Sigma^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu}). \end{aligned}
The cross terms contain \sum_i(\mathbf{x}_i-\bar{\mathbf{x}})=\mathbf{0}, so they vanish. The first sum does not involve \boldsymbol{\mu}. The second is n times a quadratic form in the positive definite matrix \Sigma^{-1}, so it is nonnegative and equals zero only at \boldsymbol{\mu}=\bar{\mathbf{x}}. The maximiser in \boldsymbol{\mu} is therefore
\widehat{\boldsymbol{\mu}}=\bar{\mathbf{x}},
for every positive definite \Sigma. The covariance does not have to be known before the mean is estimated. The next step substitutes this maximiser and maximises in \Sigma alone.
Used in
See also
Invariance of the maximum likelihood estimator · Wishart distribution · Sampling law of the normal mean and scatter · Multivariate central limit theorem · The limit of Hotelling’s statistic · Null distribution of Hotelling’s · Two-sample Hotelling test · Likelihood of a multivariate normal sample