Practice, computation, and extensions
Section of Support vector machines, part of Statistical learning
Practice, computation, and extensions
| Type | Section |
| Taught in | Support vector machines |
| Part | Statistical learning |
Decomposition methods optimise subsets of the dual coefficients while preserving feasibility. Sequential minimal optimisation updates a pair at a time, which allows the equality \sum_i\alpha_i y_i=0 to be maintained (Platt, 1999). A fully stored Gram matrix uses n^2 entries, but implementations can cache or recompute selected entries. Quadratic storage is a property of full-matrix storage, not a requirement of every implementation.
Tune the kernel, C, and any kernel width inside the training data. Refit scaling within each validation fold because distances and inner products depend on units. For the Gaussian kernel, \gamma=1/(2\sigma^2): small \sigma makes similarity local. As \sigma\to\infty at fixed inputs, K(x,x')\to 1, the constant kernel. A centred and rescaled leading-order expansion contains a linear inner product, but that is a different limiting argument and can require rescaling C. The unqualified statement that the radial kernel tends to the linear kernel is incorrect.
Kernels can encode similarities for sequences, graphs, and other structured inputs, provided the resulting kernel meets the positivity condition. Related constructions include support vector regression and kernel principal components (Schölkopf, Smola and Müller, 1999; Burges, 1998). Comparative accuracy must be assessed for the actual representation, tuning procedure, and target population.
The convex support-vector objective and a network’s nonconvex objective have different optimisation geometry. Convexity does not make a fitted classifier insensitive to data or kernel choices, so ensembles can still differ through resampling, features, and tuning. A support vector machine produces a decision score. Its sign gives a binary label. For a linear machine with w\neq 0, divide the score by \lVert w\rVert to obtain signed distance. A probability requires a separate calibration model assessed on data not used to fit that calibration. Multiclass fitting also requires a construction, such as one-versus-rest or pairwise classifiers.
On the iris petal measurements, a linear kernel, a degree-two polynomial kernel, and a radial kernel, each at a fixed illustrative setting, cut the plane differently. That comparison is descriptive. It is not a tuned assessment.
Used in
See also
Signed distance · Vapnik’s radius–margin bound · Hard-margin primal · Hard-margin dual · Soft-margin primal · Kernel · Linear separators and the margin · Distance to a hyperplane