Finite Sample Identification of Wide Shallow Neural Networks with Biases

Understand

Artificial neural networks are functions depending on a finite number of parameters typically encoded as weights and biases.

• The identification of the parameters of the network from finite samples of input-output pairs is often referred to as the \emph{teacher-student model}, and this model has represented a popular framework for understanding training and generalization.
• Even if the problem is NP-complete in the worst case, a rapidly growing literature -- after adding suitable distributional assumptions -- has established finite sample identification of two-layer networks with a number of neurons $m=\mathcal O(D)$, $D$ being the input dimension.
• For the range $D<m<D^2$ the problem becomes harder, and truly little is known for networks parametrized by biases as well.