Generalized equivalences between subsampling and ridge regularization

Understand

We establish precise structural and risk equivalences between subsampling and ridge regularization for ensemble ridge estimators.

• Specifically, we prove that linear and quadratic functionals of subsample ridge estimators, when fitted with different ridge regularization levels $\lambda$ and subsample aspect ratios $\psi$, are asymptotically equivalent along specific paths in the $(\lambda,\psi)$-plane (where $\psi$ is the ratio of the feature dimension to the subsample size).
• Our results only require bounded moment assumptions on feature and response distributions and allow for arbitrary joint distributions.
• Furthermore, we provide a data-dependent method to determine the equivalent paths of $(\lambda,\psi)$.