Extended information criterion
Section of Graphical models, part of Statistical learning
Extended information criterion
| Type | Section |
| Taught in | Graphical models |
| Part | Statistical learning |
| Questions using it | 1 |
One option for choosing \lambda is the extended Bayesian information criterion (Foygel and Drton, 2010). For a Gaussian graphical model with edge set E and control parameter \gamma\in[0,1],
\mathrm{BIC}_\gamma(E)=-2\ell\bigl(\hat\theta(E)\bigr)+|E|\log n+4|E|\gamma\log p.
Here |E| counts unordered edges once. Diagonal precision parameters and any freely estimated means have the same dimension in every candidate graph, so their common terms are omitted. The estimate \hat\theta(E) is the unpenalised constrained maximum-likelihood estimate when it exists. At \gamma=0, the criterion agrees with the ordinary information criterion up to those common terms. The extra 4|E|\gamma\log p charge accounts for the growing collection of graph candidates. Consistency results impose assumptions on sample size, dimension, signal, and the candidate graphs.
Generate n=120 observations from the p=8 Gaussian autoregression with correlation 0.6^{|j-k|}. Its true graph is a chain with seven edges out of 28 possible pairs. A fit over 18 penalties from 0.02 to 0.65, followed by an unpenalised refit of each candidate edge set and a comparison of the ordinary and extended criteria, produces a small-penalty graph, an intermediate graph, and the extended-criterion choice. The refit matters: a penalised precision estimate is shrunk, so its likelihood is not the constrained maximum-likelihood value required by this version of the criterion. The simulation has a known population graph. Neither the penalty nor the criterion guarantees its exact recovery in a finite sample.
Used in
- Taught in Graphical models
- Questions that use it (1) — Conditional association and sparsity (answer)
- All 8 questions on Graphical models
See also
Akaike information criterion · Bayesian information criterion · Undirected graph · Hammersley–Clifford · A zero precision entry is conditional independence · Maximum likelihood for a known graph · Graphical lasso · Markov graphs