Finding Strength in Weakness: Learning to Separate Sounds with Weak Supervision

Understand

While there has been much recent progress using deep learning techniques to separate speech and music audio signals, these systems typically require large collections of isolated sources during the training process.

• When extending audio source separation algorithms to more general domains such as environmental monitoring, it may not be possible to obtain isolated signals for training.
• Here, we propose objective functions and network architectures that enable training a source separation system with weak labels.
• In this scenario, weak labels are defined in contrast with strong time-frequency (TF) labels such as those obtained from isolated sources, and refer either to frame-level weak labels where one only has access to the time periods when different sources are active in an audio mixture, or to clip-level weak labels that only indicate the presence or absence of sounds in an entire audio clip.