Generalization Error of Generalized Linear Models in High Dimensions

Understand

At the heart of machine learning lies the question of generalizability of learned rules over previously unseen data.

• While over-parameterized models based on neural networks are now ubiquitous in machine learning applications, our understanding of their generalization capabilities is incomplete.
• This task is made harder by the non-convexity of the underlying learning problems.
• We provide a general framework to characterize the asymptotic generalization error for single-layer neural networks (i.e., generalized linear models) with arbitrary non-linearities, making it applicable to regression as well as classification problems.