Which points matter?

Topic: Support vector machines · more questions on this topic
‹ A two-point machine Kernel as an inner product ›

In the three-point example, add a fourth point x_4=(6,0) with y_4=+1. Does the separator change? What are the new support vectors and \alpha_4?

Worked answer (click to show)

The separator is x_1=2, with margin boundaries x_1=1 and x_1=3. For x_4=(6,0), w^\top x_4+b=6-2=4>1, so y_4 f(x_4)=4, well outside the margin. By complementary slackness \alpha_4=0. The three-point solution, with the same multipliers and \alpha_4=0, still satisfies all the constraints and the Karush–Kuhn–Tucker conditions, so it is still optimal. By uniqueness of w it is the solution. The support vectors remain x_1,x_2,x_3.