Read the geometry

Topic: Support vector machines · more questions on this topic
‹ A tempting wrong turn A unit-norm constraint ›

A hard-margin linear machine uses the canonical constraints y_i(w^\top x_i+b)\ge 1. For a fitted model, w=(3,4)^\top and b=-10.

  1. Explain w and b, and give the decision-boundary equation.

  2. Find the full distance between the two margin hyperplanes.

  3. Explain what distinguishes support vectors and determine whether a positive observation at (2,1)^\top satisfies the hard-margin constraint.

  1. The vector w is a normal to the decision boundary and determines its orientation. The scalar b offsets it. The boundary is 3x_1+4x_2-10=0, and the predicted sign is the sign of this expression, with a stated tie rule at zero.

  2. \lVert w\rVert=5. The hyperplanes w^\top x+b=\pm 1 are separated by 2/\lVert w\rVert=0.4, not by 2, unless w is a unit vector.

  3. Active margin observations satisfy y_i(w^\top x_i+b)=1. Observations with nonzero dual coefficients contribute to w=\sum_i\alpha_i y_i x_i. Degeneracy can give an active constraint with a zero multiplier. At (2,1) the score is 6+4-10=0, so the positive observation violates y_i f_i\ge 1 and could not be part of this hard-margin training fit.