Generating Diverse Audio-Visual 360 Soundscapes for Sound Event Localization and Detection

Understand

We present SELDVisualSynth, a tool for generating synthetic videos for audio-visual sound event localization and detection (SELD).

• Our approach incorporates real-world background images to improve realism in synthetic audio-visual SELD data while also ensuring audio-visual spatial alignment.
• The tool creates 360 synthetic videos where objects move matching synthetic SELD audio data and its annotations.
• Experimental results demonstrate that a model trained with this data attains performance gains across multiple metrics, achieving superior localization recall (56.4 LR) and competitive localization error (21.9deg LE).