Choices in a neural network
Section of Neural units, part of Statistical learning
Choices in a neural network
| Type | Section |
| Taught in | Neural units |
| Part | Statistical learning |
To use a network one must choose:
- whether to have a hidden layer at all (a network is generally more capable with one);
- how many hidden layers;
- any other structural choices;
- how many hidden units in each layer;
- values for each of the weights and biases.
Choose an architecture by the function it must represent and the evidence available for fitting it. More hidden units enlarge a nested model class, so its ideal minimum training loss cannot increase. An actual optimisation run can still finish worse. Compare architecture and regularisation choices using validation within the training process, then assess the selected procedure separately.
Used in
- Taught in Neural units
- All 7 questions on Neural units
See also
Artificial neuron · Perceptron learning rule · A single threshold unit cannot compute XOR · One hidden layer approximates continuous functions · History of the neural unit · Neural unit · Linear separability · Threshold unit without a bias