A Gram matrix

Topic: Support vector machines · more questions on this topic
‹ Kernel as an inner product Interpreting multipliers ›

For the Gaussian kernel with \sigma=1 and the points 0,1,2 in one dimension, compute the Gram matrix and show that it is positive semi-definite by checking that its leading principal minors are positive, or that its eigenvalues are nonnegative.

Worked answer (click to show)

K_{ij}=\exp\bigl(-(i-j)^2/2\bigr) gives K_{12}=K_{23}=e^{-1/2}=0.6065 and K_{13}=e^{-2}=0.1353, with ones on the diagonal. The leading principal minors are 1, then 1-0.6065^2=0.632, and

\begin{aligned} \det K &=1(0.632)-0.6065\bigl(0.6065-0.6065\times 0.1353\bigr)+0.1353(0.3679-0.1353)\\ &=0.632-0.318+0.031=0.346. \end{aligned}

All are positive, so K is positive definite by Sylvester’s criterion. Its eigenvalues are 0.207, 0.865 and 1.928, all positive and summing to 3=\operatorname{tr} K.