Derivative of the sigmoid

Topic: Neural units · more questions on this topic
Logic gates ›

Show that g(z)=1/(1+e^{-z}) satisfies g'(z)=g(z)\bigl(1-g(z)\bigr)=e^{-z}g(z)^2. Find its maximum value and where it occurs.

Worked answer (click to show)

g'(z)=\frac{e^{-z}}{(1+e^{-z})^2}=e^{-z}g(z)^2.

Since 1-g(z)=e^{-z}/(1+e^{-z}),

g(z)\bigl(1-g(z)\bigr)=\frac{e^{-z}}{(1+e^{-z})^2},

the same expression. The product g(1-g) is largest when g=\tfrac12, that is at z=0, where it equals \tfrac14.