Statistical Guarantees for Regularized Neural Networks

Understand

Neural networks have become standard tools in the analysis of data, but they lack comprehensive mathematical theories.

• For example, there are very few statistical guarantees for learning neural networks from data, especially for classes of estimators that are used in practice or at least similar to such.
• In this paper, we develop a general statistical guarantee for estimators that consist of a least-squares term and a regularizer.
• We then exemplify this guarantee with $\ell_1$-regularization, showing that the corresponding prediction error increases at most sub-linearly in the number of layers and at most logarithmically in the total number of parameters.