Evaluating the network
Section of Networks in practice, part of Statistical learning
Evaluating the network
| Type | Section |
| Taught in | Networks in practice |
| Part | Statistical learning |
A trained network can be evaluated in three ways.
- Record generalisation performance on different test sets. This provides no model-specific explanation.
- Inspect internal activations, for example by visualising the spatial similarity between hidden-unit activations for different inputs. This provides knowledge on the effect of different inputs on the model.
- Study the network weights, for example with a Hinton diagram that shows, by the size and colour of boxes, how the weights are used and which units are relevant. This provides knowledge on the decisions made in the model.
Training loss, held-out classification error, and stability across starts answer different questions. In particular, the smallest training sum of squares need not identify the smallest future classification error: the losses differ, and training data have already influenced the fit. Compare repeated starts within the training and validation procedure and report the variability relevant to the claim. If the test set is repeatedly used to pick a seed, it has become another validation set (Cawley and Talbot, 2010). Examine errors by class and by relevant subgroups as well as reporting a single average.
Used in
See also
Softmax is multinomial logistic regression · What has to be settled before a step · Weight initialisation · Variance argument for weight initialisation · Batch and online learning · Preparing data for a network · Output coding, softmax, and cross-entropy · Constructing and training a network