Revisiting Discriminative vs. Generative Classifiers: Theory and Implications

Understand

A large-scale deep model pre-trained on massive labeled or unlabeled data transfers well to downstream tasks.

• Linear evaluation freezes parameters in the pre-trained model and trains a linear classifier separately, which is efficient and attractive for transfer.
• However, little work has investigated the classifier in linear evaluation except for the default logistic regression.
• Inspired by the statistical efficiency of naive Bayes, the paper revisits the classical topic on discriminative vs.