Counting weights
Use (p+1)l+(l+1)m to count the weights of a network with p=10 inputs, l=5 hidden units, and m=3 outputs. Compare with the number of parameters of a multinomial logistic regression on the same inputs and outputs.
Worked answer (click to show)
(p+1)l+(l+1)m=11\times 5+6\times 3=55+18=73.
A multinomial logistic regression with p=10 inputs and G=3 classes, with one class as reference, has (G-1)(p+1)=2\times 11=22 parameters. The network with five hidden units has more than three times as many.