Log density of a Gaussian graphical model
Section of Graphical models, part of Statistical learning
Log density of a Gaussian graphical model
| Type | Section |
| Taught in | Graphical models |
| Part | Statistical learning |
Write the multivariate normal density in terms of K:
\begin{aligned} f(x) &=\frac{1}{(2\pi)^{p/2}\,|\Sigma|^{1/2}} \exp\Bigl(-\tfrac12(x-\mu)^\top\Sigma^{-1}(x-\mu)\Bigr)\\ &=\frac{|K|^{1/2}}{(2\pi)^{p/2}} \exp\Bigl(-\tfrac12(x-\mu)^\top K(x-\mu)\Bigr). \end{aligned}
Treating \mu and K as constants, the logarithm of the density, up to constants c and c_1, is
\begin{aligned} \log f(x) &=c-\tfrac12(x-\mu)^\top K(x-\mu) =c-\tfrac12 x^\top Kx+x^\top K\mu-\tfrac12\mu^\top K\mu\\ &=c_1-\tfrac12\sum_{j\in V} k_{jj} x_j^2 -\sum_{j\in V}\sum_{k<j} k_{jk} x_j x_k +\sum_{j\in V} x_j\Bigl(\sum_{k\in V} k_{jk}\mu_k\Bigr)\\ &=c_1-\tfrac12\sum_{j\in V} k_{jj} x_j^2 +\sum_{j\in V} h_j x_j -\sum_{j\in V,\,k<j} k_{jk} x_j x_k, \end{aligned}
with h=K\mu. The log density is a sum of quadratic main effects with coefficients -k_{jj}/2, linear main effects with coefficients h_j, and quadratic interactions with coefficients -k_{jk}. Taking the exponential, f is a product over single variables and over pairs, with a pair appearing only if k_{jk}\neq 0. Each pair with k_{jk}\neq 0 is an edge, so the factors involve complete subsets of size at most 2, singletons or edges. Such subsets need not be maximal cliques. Their factors can be absorbed into containing clique potentials. This is an explicit instance of the factorisation, and of the Hammersley–Clifford theorem, with \psi_{jk}(x_j,x_k)=\exp(-k_{jk} x_j x_k).
Used in
See also
Undirected graph · Hammersley–Clifford · A zero precision entry is conditional independence · Maximum likelihood for a known graph · Graphical lasso · Markov graphs · Graph terminology · Conditional independence