A unit-norm constraint

Topic: Support vector machines · more questions on this topic
‹ Read the geometry Nonlinear separation of XOR ›

For two linearly separable classes, suppose \lVert w\rVert=1, y_i\in\{-1,+1\} and y_i(w^\top x_i+b)\ge\rho/2 with \rho>0. The unit-norm convention is explicit.

  1. Explain the geometric meaning of w, b and \rho.

  2. Give the equation of the decision boundary and the prediction rule away from it.

  3. Explain what distinguishes support vectors from the remaining training observations and how they relate to the maximum margin.

  1. The vector w is a unit normal, fixing the orientation. The scalar b fixes the signed offset of the hyperplane. The number \rho is the full geometric width between its two supporting parallel margin planes. The unit norm is what makes the given \rho a distance.

  2. The decision boundary is w^\top x+b=0. Positive scores are predicted as +1 and negative scores as -1.

  3. In the hard-margin geometry, the closest observations lie on w^\top x+b=\pm\rho/2. Their constraints limit enlargement of the separating margin. The optimal normal can be represented through observations with nonzero dual coefficients. In a degenerate configuration an active boundary observation may have a zero coefficient. Observations strictly beyond the margin do not directly contribute to that representation.