To tune or not to tune, a case study of ridge logistic regression in small or sparse datasets

Understand

For finite samples with binary outcomes penalized logistic regression such as ridge logistic regression (RR) has the potential of achieving smaller mean squared errors (MSE) of coefficients and predictions than maximum likelihood estimation.

• There is evidence, however, that RR is sensitive to small or sparse data situations, yielding poor performance in individual datasets.
• In this paper, we elaborate this issue further by performing a comprehensive simulation study, investigating the performance of RR in comparison to Firth's correction that has been shown to perform well in low-dimensional settings.
• Performance of RR strongly depends on the choice of complexity parameter that is usually tuned by minimizing some measure of the out-of-sample prediction error or information criterion.