Support vectors and their multipliers
Section of Support vector machines, part of Statistical learning
Support vectors and their multipliers
| Type | Section |
| Taught in | Support vector machines |
| Part | Statistical learning |
| Questions using it | 5 |
Complementary slackness gives three cases for a training point.
| Multiplier | Position | Slack |
|---|---|---|
| \alpha_i=0 | y_i f(x_i)\ge 1: outside the margin or on it, correctly classified | \xi_i=0 |
| 0<\alpha_i<C | y_i f(x_i)=1: exactly on the margin, a free support vector | \xi_i=0 |
| \alpha_i=C | y_i f(x_i)\le 1: inside the margin or misclassified, a bound support vector | \xi_i=1-y_i f(x_i)\ge 0 |
Only points on or inside the margin influence the fit. On one overlapping Gaussian sample, a linear fit at C=0.03 uses a wide slab and many support vectors, a fit at C=1 tightens the slab, and a fit at C=100 hugs the overlapping points. The rings in such a picture are the points with nonzero multipliers.
Used in
- Taught in Support vector machines
- Questions that use it (5) — A two-point machine (answer) · Which points matter? · Interpreting multipliers (answer) · Read the geometry · A unit-norm constraint
- All 10 questions on Support vector machines
See also
Gram matrix · Signed distance · Vapnik’s radius–margin bound · Hard-margin primal · Hard-margin dual · Soft-margin primal · Kernel · Linear separators and the margin