Representation learning through cross-modal conditional teacher-student training for speech emotion recognition

Understand

Generic pre-trained speech and text representations promise to reduce the need for large labeled datasets on specific speech and language tasks.

• However, it is not clear how to effectively adapt these representations for speech emotion recognition.
• Recent public benchmarks show the efficacy of several popular self-supervised speech representations for emotion classification.
• In this study, we show that the primary difference between the top-performing representations is in predicting valence while the differences in predicting activation and dominance dimensions are less pronounced.