Muddling Label Regularization: Deep Learning for Tabular Datasets

Understand

Deep Learning (DL) is considered the state-of-the-art in computer vision, speech recognition and natural language processing.

• Until recently, it was also widely accepted that DL is irrelevant for learning tasks on tabular data, especially in the small sample regime where ensemble methods are acknowledged as the gold standard.
• We present a new end-to-end differentiable method to train a standard FFNN.
• Our method, \textbf{Muddling labels for Regularization} (\texttt{MLR}), penalizes memorization through the generation of uninformative labels and the application of a differentiable close-form regularization scheme on the last hidden layer during training.