Linear separability
Section of Neural units, part of Statistical learning
Linear separability
| Type | Section |
| Taught in | Neural units |
| Part | Statistical learning |
With two inputs and one threshold output, the output changes where the net input changes sign. The decision boundary is the zero set of w_0+w^\top x. For one threshold unit the partition is always a half-space, and the geometric question is where the bounding hyperplane sits.
Without a bias
Main article: Threshold unit without a bias
With w_0=0 the boundary is \{x:\sum_j x_j w_j=0\}, which passes through the origin. A separator that ought to miss the origin cannot be represented. For w=(1,-2)^\top the boundary is x_1-2x_2=0, the line x_2=\tfrac12 x_1. The weights (0.5,1), (1,1) and (2,1) instead give the three lines x_2=-x_1/2, x_2=-x_1 and x_2=-2x_1.
With a bias
Main article: Threshold unit with a bias
The bias is the constant term that frees the boundary from the origin. With it, a threshold unit can realise any linear separator. For w=(1,1)^\top the boundary is x_1+x_2+w_0=0, and changing w_0 translates that line. In general the boundary is the hyperplane \{x:w_0+w^\top x=0\}, which has normal vector w, and the unit outputs 1 on the side the normal points to. The distance from a point to this hyperplane is derived with the support vector machine; the same normal is the object that derivation starts from.
Used in
- Taught in Neural units
- All 7 questions on Neural units
See also
Artificial neuron · Perceptron learning rule · A single threshold unit cannot compute XOR · One hidden layer approximates continuous functions · History of the neural unit · Neural unit · Threshold unit without a bias · Threshold unit with a bias