Check the XOR features in R
Section of Neural units, part of Statistical learning
Check the XOR features in R
| Type | Section |
| Taught in | Neural units |
| Part | Statistical learning |
A hand-built network is a specification one can check before trusting an optimiser. The first hidden unit detects OR, the second detects AND, and the output keeps the cases where OR is on and AND is off. That is the second construction of this page, written as thresholds on row sums.
X <- rbind(c(0, 0), c(0, 1), c(1, 0), c(1, 1))
h_or <- as.integer(rowSums(X) >= 0.5)
h_and <- as.integer(rowSums(X) >= 1.5)
out <- as.integer(h_or - 2 * h_and >= 0.5)
data.frame(X, OR = h_or, AND = h_and, XOR = out)
#> X1 X2 OR AND XOR
#> 1 0 0 0 0 0
#> 2 0 1 1 0 1
#> 3 1 0 1 0 1
#> 4 1 1 1 1 0
stopifnot(identical(out, c(0L, 1L, 1L, 0L)))The four rows match XOR, so a hidden layer of two nonlinear units is sufficient for this function. The inequality argument earlier is the matching necessity result for a single affine threshold. Nothing here was optimised. The weights were chosen so that the hidden units would be the OR and AND features, and the check confirms that those features do what the arithmetic said they would.
XOR shows why changing the representation can matter: two hidden logical features turn an inseparable input pattern into a separable one. The approximation theorem extends representational capacity to continuous functions on a compact domain. Neither argument tells us that a particular training run will find useful features or generalise to new data.
Used in
- Taught in Neural units
- All 7 questions on Neural units
See also
Artificial neuron · Perceptron learning rule · A single threshold unit cannot compute XOR · One hidden layer approximates continuous functions · History of the neural unit · Neural unit · Linear separability · Threshold unit without a bias