Interpreting multipliers

Topic: Support vector machines · more questions on this topic
‹ A Gram matrix A tempting wrong turn ›

In a soft-margin machine with C=2, three training points have \alpha=0, \alpha=1.3, and \alpha=2. For each, state where the point is relative to the margin and its slack. Which can be used to compute b?

Worked answer (click to show)

With C=2:

• \alpha=0: the point is correctly classified and lies on or outside the margin, yf\ge 1, with slack \xi=0. It does not influence the solution.
• \alpha=1.3, strictly between 0 and C: the point lies exactly on the margin, yf=1, with slack \xi=0. It is a free support vector and can be used to compute b=y_k-\sum_i\alpha_i y_i x_i^\top x_k.
• \alpha=2=C: a bound support vector, inside the margin or misclassified, yf\le 1, with slack \xi=1-yf\ge 0, and \xi>1 if it is misclassified. Its slack is not known in advance, so it cannot be used for b without computing \xi.