Latent Space Explorations of Singing Voice Synthesis using DDSP

Understand

Machine learning based singing voice models require large datasets and lengthy training times.

• In this work we present a lightweight architecture, based on the Differentiable Digital Signal Processing (DDSP) library, that is able to output song-like utterances conditioned only on pitch and amplitude, after twelve hours of training using small datasets of unprocessed audio.
• The results are promising, as both the melody and the singer's voice are recognizable.
• In addition, we present two zero-configuration tools to train new models and experiment with them.