A small softmax network for iris
Section of Networks in practice, part of Statistical learning
A small softmax network for iris
| Type | Section |
| Taught in | Networks in practice |
| Part | Statistical learning |
Use scaled predictors, one hidden layer, three output classes and weight decay. Architecture and decay are fixed for this illustration. A production comparison would tune these choices inside training data, by the validation discipline above. The objects train, test and iris_scaled are the training flowers, the test flowers, and their scaled predictors.
set.seed(16006)
y_onehot <- nnet::class.ind(train$Species)
net <- nnet::nnet(iris_scaled$train, y_onehot,
size = 3, softmax = TRUE, decay = 0.1,
maxit = 1000, trace = FALSE)
prob <- predict(net, iris_scaled$test, type = "raw")
pred <- factor(colnames(y_onehot)[max.col(prob)],
levels = levels(train$Species))
c(convergence_code = net$convergence,
test_error = mean(pred != test$Species),
max_probability_sum_error = max(abs(rowSums(prob) - 1)))
#> convergence_code test_error
#> 0.000e+00 8.889e-02
#> max_probability_sum_error
#> 2.220e-16
stopifnot(net$convergence == 0,
max(abs(rowSums(prob) - 1)) < 1e-8)The convergence code reports the optimiser’s stopping status. It is not a proof that the global optimum was found. Weight decay shrinks weights. The species probabilities must sum to one for each flower, which the explicit check verifies. The reported test error is 0.08889, and the largest departure of a probability row from summing to 1 is 2.220\times 10^{-16}.
Initialise with small random weights scaled by the fan-in, normalise the inputs, and use cross-entropy with softmax for classification. Choose when to stop, how many hidden units, and how much weight decay by a validation set. Then report performance on a test set that was not used in any of those choices.
Used in
See also
Softmax is multinomial logistic regression · What has to be settled before a step · Weight initialisation · Variance argument for weight initialisation · Batch and online learning · Preparing data for a network · Output coding, softmax, and cross-entropy · Constructing and training a network