A chain with dense marginal correlations

Section of Graphical models, part of Statistical learning

Type	Section
Taught in	Graphical models
Part	Statistical learning

Specify an autoregressive correlation matrix with \Sigma_{ij}=0.6^{|i-j|}. Its inverse has zero entries except on the diagonal and the neighbouring off-diagonals. The marginal correlations are dense, and the conditional independences are the missing edges of a chain. Verify that algebra before estimating a graph from simulated data.

p <- 6
Sigma <- 0.6^abs(outer(1:p, 1:p, "-"))
K <- solve(Sigma)
partial <- -cov2cor(K)
diag(partial) <- 1
round(partial, 3)
#>       [,1]  [,2]  [,3]  [,4]  [,5]  [,6]
#> [1,] 1.000 0.514 0.000 0.000 0.000 0.000
#> [2,] 0.514 1.000 0.441 0.000 0.000 0.000
#> [3,] 0.000 0.441 1.000 0.441 0.000 0.000
#> [4,] 0.000 0.000 0.441 1.000 0.441 0.000
#> [5,] 0.000 0.000 0.000 0.441 1.000 0.514
#> [6,] 0.000 0.000 0.000 0.000 0.514 1.000
stopifnot(max(abs(K[abs(outer(1:p, 1:p, "-")) > 1])) < 1e-10)
set.seed(19006)
X <- MASS::mvrnorm(200, rep(0, p), Sigma)
S <- cov(X) * (nrow(X) - 1) / nrow(X)
fit <- glasso::glasso(S, rho = 0.1, penalize.diagonal = FALSE)
edges <- abs(fit$wi) > 1e-6
diag(edges) <- FALSE
round(-cov2cor(fit$wi) + diag(2, p), 2)
#>      [,1] [,2] [,3] [,4] [,5] [,6]
#> [1,] 1.00 0.44 0.07 0.00 0.00 0.00
#> [2,] 0.44 1.00 0.36 0.06 0.00 0.00
#> [3,] 0.07 0.36 1.00 0.28 0.00 0.05
#> [4,] 0.00 0.06 0.28 1.00 0.41 0.08
#> [5,] 0.00 0.00 0.00 0.41 1.00 0.36
#> [6,] 0.00 0.00 0.05 0.08 0.36 1.00
c(estimated_edges = sum(edges) / 2, true_edges = p - 1)
#> estimated_edges      true_edges
#>              10               5

The population partial correlations match the four-variable chain: neighbours at about 0.514 and 0.441, and exact zeros elsewhere. The call sets penalize.diagonal = FALSE, so it uses the off-diagonal convention, unlike the all-entry penalty in the displayed algorithm. Its fitted covariance diagonal should match S to numerical tolerance. At \rho=0.1 the fit reports 10 edges against the 5 true chain edges. Several small partial correlations, such as 0.07 and 0.06, survive.

The penalty was fixed for demonstration. An estimated missing edge may reflect shrinkage or sampling noise. In the population model the zero pattern has an exact conditional-independence meaning because Gaussianity and positive definiteness were assumed. An analogous inverse-covariance calculation on pooled iris flowers does not automatically inherit that interpretation: pooling distinct species mixes several conditional distributions.

Key idea

In a Gaussian graphical model the graph is the sparsity pattern of the precision matrix K=\Sigma^{-1}. A zero k_{jk} means X_j\perp X_k given all the other variables, and -k_{jk}/\sqrt{k_{jj}k_{kk}} is their partial correlation. For a known graph, the maximum-likelihood estimate matches the sample covariance on the diagonal and on the edges. For an unknown graph, an L^1 penalty on K estimates the edges and the parameters together. The penalty parameter is chosen by cross-validation or by an information criterion, ordinary or extended.

Used in

Taught in Graphical models
All 8 questions on Graphical models

Used in

See also