Latent Diffusion Model Based Foley Sound Generation System For DCASE Challenge 2023 Task 7

Understand

Foley sound presents the background sound for multimedia content and the generation of Foley sound involves computationally modelling sound effects with specialized techniques.

• In this work, we proposed a system for DCASE 2023 challenge task 7: Foley Sound Synthesis.
• The proposed system is based on AudioLDM, which is a diffusion-based text-to-audio generation model.
• To alleviate the data-hungry problem, the system first trained with large-scale datasets and then downstreamed into this DCASE task via transfer learning.