Basic gradient-descent learning
Definition taught in Backpropagation, part of Statistical learning
Basic gradient-descent learning
| Type | Definition |
| Taught in | Backpropagation |
| Part | Statistical learning |
- Choose a random set of initial weights, for example from the uniform distribution on [-0.5,0.5].
- Repeat:
- Run every input training pattern through the network and record the output values of the hidden and output units.
- Calculate E for the current weights.
- Update every weight by w'=w-\varepsilon\frac{\partial E}{\partial w}=w+\Delta w, until a stopping condition holds: the performance is acceptable, or the weights are barely changing.
Used in
- Taught in Backpropagation, where the derivation and examples are.
- All 7 questions on Backpropagation
See also
Weight initialisation · Uniform distribution · Squared error is not the logistic likelihood · Backpropagation, batch form · Local minima, plateaus, and symmetry · Training error of a network · Gradient descent · A single layer of sigmoid units