Distance to a hyperplane
Section of Support vector machines, part of Statistical learning
Distance to a hyperplane
| Type | Section |
| Taught in | Support vector machines |
| Part | Statistical learning |
| Questions using it | 1 |
Theorem 1 (Signed distance) For w\neq 0, the signed distance from a point x to the hyperplane \{x:w^\top x+b=0\} is (w^\top x+b)/\lVert w\rVert, where \lVert\cdot\rVert is the Euclidean norm. In particular the distance between the parallel hyperplanes w^\top x+b=-1 and w^\top x+b=1 is 2/\lVert w\rVert.
The hyperplane w\cdot x=0 goes through the origin, so w is normal to it. The hyperplane w\cdot x=c with c>0 is parallel to it, and w is normal to this also. Let x_1 be a point on w\cdot x=0. Follow a normal until it meets the second hyperplane: find x_2=x_1+aw with w\cdot x_2=c. Then w\cdot x_1+a\,w\cdot w=c, so a=c/\lVert w\rVert^2 and the distance between x_1 and x_2 is \lVert aw\rVert=c/\lVert w\rVert. That is the distance between the hyperplanes.
Likewise the hyperplanes w^\top x+b=\mp\rho/2 are \rho/\lVert w\rVert apart. The offset b only translates them. Dividing w and b by \rho/2 turns this into w^\top x+b=\mp 1, a distance of 2/\lVert w\rVert apart. For the signed distance of an arbitrary point, write x=x_\perp+r\,w/\lVert w\rVert with x_\perp on the hyperplane. Then
w^\top x+b=w^\top x_\perp+b+r\lVert w\rVert=r\lVert w\rVert,
so r=(w^\top x+b)/\lVert w\rVert.
Rescaling (w,b) by a positive constant leaves the separating plane unchanged. It also rescales the numerator and denominator of the distance formula by the same factor. A functional score w^\top x+b is not itself a geometric distance until the norm of w has been accounted for.
The margin \rho of a separator is the distance between the closest points of the two classes, measured perpendicular to the separator. Those closest examples are the support vectors. The maximum-margin separator maximises \rho. For hard-margin separation, only the points on the margin determine the solution. Moving a point that is strictly beyond the margin, without crossing it, does not change the optimum.
Used in
- Taught in Support vector machines
- Questions that use it (1) — Distance to a hyperplane
- All 10 questions on Support vector machines
See also
Signed distance · Vapnik’s radius–margin bound · Hard-margin primal · Hard-margin dual · Soft-margin primal · Kernel · Linear separators and the margin · Why maximise the margin?