Audio-Visual Person Verification based on Recursive Fusion of Joint Cross-Attention

Understand

Person or identity verification has been recently gaining a lot of attention using audio-visual fusion as faces and voices share close associations with each other.

• Conventional approaches based on audio-visual fusion rely on score-level or early feature-level fusion techniques.
• Though existing approaches showed improvement over unimodal systems, the potential of audio-visual fusion for person verification is not fully exploited.
• In this paper, we have investigated the prospect of effectively capturing both the intra- and inter-modal relationships across audio and visual modalities, which can play a crucial role in significantly improving the fusion performance over unimodal systems.