MambaFoley: Foley Sound Generation using Selective State-Space Models

Understand

Recent advancements in deep learning have led to widespread use of techniques for audio content generation, notably employing Denoising Diffusion Probabilistic Models (DDPM) across various tasks.

• Among these, Foley Sound Synthesis is of particular interest for its role in applications for the creation of multimedia content.
• Given the temporal-dependent nature of sound, it is crucial to design generative models that can effectively handle the sequential modeling of audio samples.
• Selective State Space Models (SSMs) have recently been proposed as a valid alternative to previously proposed techniques, demonstrating competitive performance with lower computational complexity.