One hidden layer approximates continuous functions
Theorem taught in Neural units, part of Statistical learning
One hidden layer approximates continuous functions
| Type | Theorem |
| Taught in | Neural units |
| Part | Statistical learning |
For a continuous target f on [0,1]^p and any \varepsilon>0, there is a finite sum
F(x)=\sum_{q=1}^{l} a_q\,\sigma(w_q^\top x+b_q)
with a continuous sigmoidal activation such that \sup_{x\in[0,1]^p}|F(x)-f(x)|<\varepsilon (Cybenko, 1989, Theorem 2). The output here is a linear combination, not a bounded sigmoid forced to represent arbitrary real values. Results for measurable targets use weaker notions of approximation. They do not assert uniform approximation of every discontinuous function by continuous networks.
Used in
- Taught in Neural units, where the derivation and examples are.
- All 7 questions on Neural units
See also
Activation function · Sigmoid function · Artificial neuron · Perceptron learning rule · A single threshold unit cannot compute XOR · History of the neural unit · Neural unit · Linear separability