Backpropagation, batch form
Definition taught in Backpropagation, part of Statistical learning
Backpropagation, batch form
| Type | Definition |
| Taught in | Backpropagation |
| Part | Statistical learning |
For one pass through the data with current weights:
- Forward: for every pattern compute and store u_{iq}, h_{iq}, z_{ik}, o_{ik}.
- Output deltas: \delta_{ik}^{(o)} from the definition above.
- Backward: hidden deltas \delta_{iq}^{(h)} from the definition above.
- Gradients: G_{qk}=\sum_i \delta_{ik}^{(o)} h_{iq} and G_{jq}=\sum_i \delta_{iq}^{(h)} x_{ij}, with h_{i0}=x_{i0}=1.
- Update: w\leftarrow w-\varepsilon G for every weight.
Used in
- Taught in Backpropagation, where the derivation and examples are.
- All 7 questions on Backpropagation
See also
Delta rule · Basic gradient-descent learning · Squared error is not the logistic likelihood · Local minima, plateaus, and symmetry · Training error of a network · Gradient descent · A single layer of sigmoid units · Backpropagation through one hidden layer