Deep neural networks for emotion recognition combining audio and transcripts

Understand

In this paper, we propose to improve emotion recognition by combining acoustic information and conversation transcripts.

• On the one hand, an LSTM network was used to detect emotion from acoustic features like f0, shimmer, jitter, MFCC, etc.
• On the other hand, a multi-resolution CNN was used to detect emotion from word sequences.
• This CNN consists of several parallel convolutions with different kernel sizes to exploit contextual information at different levels.