Is Cross-Attention Preferable to Self-Attention for Multi-Modal Emotion Recognition?

“Automatic facial expression analysis: a survey,”

Beat Fasel and Juergen Luettin, · 2003

Earlier work this paper cites.

“Multimodal emotion recognition from expressive faces, body gestures and speech,”

George Caridakis, Ginevra Castellano, Loic Kessous, Amaryllis Raouzaiou, Lori Malatesta, Stelios Asteriadis, and Kostas Karpouzis, · 2007

Earlier work this paper cites.

“IEMOCAP: Interactive emotional dyadic motion capture database,”

Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan, · 2008

Earlier work this paper cites.

“Efficient estimation of word representations in vector space,”

Tomas Mikolov, Kai Chen, Greg S. Corrado, and Jeffrey Dean, · 2013

Earlier work this paper cites.

“Glove: Global vectors for word representation,”

Jeffrey Pennington, Richard Socher, and Christopher D Manning, · 2014

Earlier work this paper cites.

“Neural machine translation by jointly learning to align and translate,”

Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio, · 2015

Earlier work this paper cites.

“Adam: A method for stochastic optimization,”

Diederik P Kingma and Jimmy Ba, · 2015

Earlier work this paper cites.

“Adieu features? end-to-end speech emotion recognition using a deep convolutional recurrent network,”

George Trigeorgis, Fabien Ringeval, Raymond Brueckner, Erik Marchi, Mihalis A Nicolaou, Björn Schuller, and Stefanos Zafeiriou, · 2016

Earlier work this paper cites.

“Deep residual learning for image recognition,”

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, · 2016

Cited alongside, same era.

“End-to-end multimodal emotion recognition using deep neural networks,”

Panagiotis Tzirakis, George Trigeorgis, Mihalis A Nicolaou, Björn Schuller, and Stefanos Zafeiriou, · 2017

Cited alongside, same era.

“Attention is all you need,”

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin, · 2017

Cited alongside, same era.

“Automatic differentiation in PyTorch,”

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer, · 2017

Cited alongside, same era.

“Multimodal machine learning: A survey and taxonomy,”

Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency, · 2018

Cited alongside, same era.

“3-d convolutional recurrent neural networks with attention model for speech emotion recognition,”

“Conflictnet: End-to-end learning for speech-based conflict intensity estimation,”

Vandana Rajan, Alessio Brutti, and Andrea Cavallaro, · 2019

Later among the works it cites.

“Speech emotion recognition using multi-hop attention mechanism,”

Seunghyun Yoon, Seokhyun Byun, Subhadeep Dey, and Kyomin Jung, · 2019

Later among the works it cites.

“Multimodal transformer for unaligned multimodal language sequences,”

Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J Zico Kolter, Louis-Philippe Morency, and Ruslan Salakhutdinov, · 2019

Later among the works it cites.

“Attentive modality hopping mechanism for speech emotion recognition,”

Seunghyun Yoon, Subhadeep Dey, Hwanhee Lee, and Kyomin Jung, · 2020

Later among the works it cites.

“Multimodal transformer fusion for continuous emotion recognition,”

Jian Huang, Jianhua Tao, Bin Liu, Zheng Lian, and Mingyue Niu, · 2020

Later among the works it cites.

“Cross-modal knowledge transfer via inter-modal translation and alignment for affect recognition,”