High Dimensional Classification via Regularized and Unregularized Empirical Risk Minimization: Precise Error and Optimal Loss

Understand

This article provides, through theoretical analysis, an in-depth understanding of the classification performance of the empirical risk minimization framework, in both ridge-regularized and unregularized cases, when high dimensional data are considered.

• Focusing on the fundamental problem of separating a two-class Gaussian mixture, the proposed analysis allows for a precise prediction of the classification error for a set of numerous data vectors $\mathbf{x} \in \mathbb R^p$ of sufficiently large dimension $p$.
• This precise error depends on the loss function, the number of training samples, and the statistics of the mixture data model.
• It is shown to hold beyond Gaussian distribution under some additional non-sparsity condition of the data statistics.