Kernel as an inner product

Topic: Support vector machines · more questions on this topic
‹ Which points matter? A Gram matrix ›

Show that K(x,x')=(1+xx')^2 for scalars equals \phi(x)^\top\phi(x') with \phi(x)=(1,\sqrt{2}\,x,x^2). Using it, map the one-dimensional sandwiched data (middle class with |x|\le 0.9, tail class with |x|\ge 1.5) to (\phi_2,\phi_3) and explain why a linear classifier in \phi-space can separate the classes.

Worked answer (click to show)

(1+xx')^2=1+2xx'+x^2{x'}^2 =(1)(1)+(\sqrt{2}\,x)(\sqrt{2}\,x')+(x^2)({x'}^2) =\phi(x)^\top\phi(x').

For the sandwiched sample, the middle class has |x|\le 0.9, so \phi_3=x^2\le 0.81, while the tails have |x|\ge 1.5, so \phi_3\ge 2.25. The horizontal line \phi_3=1.5, a hyperplane in \phi-space, separates them. The displayed boundary at 1.5 has distances at least 1.5-0.81=0.69 and 2.25-1.5=0.75 on its two sides. Centring it at (0.81+2.25)/2=1.53 gives equal distances 0.72 and full slab width 1.44 in this coordinate direction. In the original space it is the pair of points x=\pm\sqrt{1.5}.