Using Self-Supervised Auxiliary Tasks to Improve Fine-Grained Facial Representation

Understand

Facial emotion recognition (FER) is a fine-grained problem where the value of transfer learning is often assumed.

• We first quantify this assumption and show that, on AffectNet, training from random initialization with sufficiently strong augmentation consistently matches or surpasses fine-tuning from ImageNet.
• Motivated by this result, we propose Hybrid Multi-Task Learning (HMTL) for FER in the wild.
• HMTL augments supervised learning (SL) with self-supervised learning (SSL) objectives during training, while keeping the inference-time model unchanged.