Dropout Training for Support Vector Machines

Understand

Dropout and other feature noising schemes have shown promising results in controlling over-fitting by artificially corrupting the training data.

• Though extensive theoretical and empirical studies have been performed for generalized linear models, little work has been done for support vector machines (SVMs), one of the most successful approaches for supervised learning.
• This paper presents dropout training for linear SVMs.
• To deal with the intractable expectation of the non-smooth hinge loss under corrupting distributions, we develop an iteratively re-weighted least square (IRLS) algorithm by exploring data augmentation techniques.