Implicit Regularization of Random Feature Models

Understand

Random Feature (RF) models are used as efficient parametric approximations of kernel methods.

• We investigate, by means of random matrix theory, the connection between Gaussian RF models and Kernel Ridge Regression (KRR).
• For a Gaussian RF model with $P$ features, $N$ data points, and a ridge $\lambda$, we show that the average (i.e.
• expected) RF predictor is close to a KRR predictor with an effective ridge $\tilde{\lambda}$.