On the approximation of functions by tanh neural networks

Understand

We derive bounds on the error, in high-order Sobolev norms, incurred in the approximation of Sobolev-regular as well as analytic functions by neural networks with the hyperbolic tangent activation function.

• These bounds provide explicit estimates on the approximation error with respect to the size of the neural networks.
• We show that tanh neural networks with only two hidden layers suffice to approximate functions at comparable or better rates than much deeper ReLU neural networks.