Cross-Task Multi-Branch Vision Transformer for Facial Expression and Mask Wearing Classification

Understand

With wearing masks becoming a new cultural norm, facial expression recognition (FER) while taking masks into account has become a significant challenge.

• In this paper, we propose a unified multi-branch vision transformer for facial expression recognition and mask wearing classification tasks.
• Our approach extracts shared features for both tasks using a dual-branch architecture that obtains multi-scale feature representations.
• Furthermore, we propose a cross-task fusion phase that processes tokens for each task with separate branches, while exchanging information using a cross attention module.