Epoch
Topic of Networks in practice, part of Statistical learning · Also in Backpropagation
Epoch
| Type | Topic |
| Taught in | Networks in practice · Backpropagation |
| Part | Statistical learning |
| Questions using it | 2 |
One pass through the whole training set in iterative training. Batch learning updates the weights once per epoch; online learning updates after every pattern, many times per epoch.
In Networks in practice
From Networks in practice
The choices below are how to initialise the weights, whether to update after every pattern or after every epoch, how to present data, how to encode classes and obtain probabilities, how to limit specialisation while training, how to check a trained network, and where the same ideas have been used. The last section is the set of changes that lead to deep networks. A sigmoid output with no hidden layer is the log-odds map of logistic regression. This page adds hidden layers, the softmax, and the cross-entropy gradient.
In Backpropagation
From Backpropagation
A randomly sampled mini-batch can give an unbiased estimate of the gradient of the mean loss. That expectation, with the current weights held fixed, is the relevant property. The angle of one noisy draw is not a convergence argument. Its variance and the learning-rate schedule still affect the trajectory. One pass through all the training patterns is an epoch. NETtalk, on networks in practice, is the count used later.
Used in
- Taught in Networks in practice · Backpropagation
- Also used in Backpropagation
- Questions that use it (2) — Online versus batch · Complexity and early stopping
- All 7 questions on Networks in practice
See also
Training set · Stochastic gradient descent · Softmax is multinomial logistic regression · What has to be settled before a step · Weight initialisation · Variance argument for weight initialisation · Batch and online learning · Preparing data for a network