2022

Unsupervised vocal dereverberation with diffusion-based generative models

Saito, Koichi, Murata, Naoki, Uesaka, Toshimitsu et al.

Understand

Removing reverb from reverberant music is a necessary technique to clean up audio for downstream music manipulations.

Reverberation of music contains two categories, natural reverb, and artificial reverb.
Artificial reverb has a wider diversity than natural reverb due to its various parameter setups and reverberation types.
However, recent supervised dereverberation methods may fail because they rely on sufficiently diverse and numerous pairs of reverberant observations and retrieved data for training in order to be generalizable to unseen observations during inference.

Built on

“Recommendation itu-r bs.1534-1: Method for the subjective assessment of intermediate quality level of coding systems,”
I. T. Union, · 2003
Earlier work this paper cites.
“Linear inverse problems in imaging,”
A. Ribes and F. Schmitt, · 2008
Earlier work this paper cites.
“Speech dereverberation based on variance-normalized delayed linear prediction,”
T. Nakatani, T. Yoshioka, K. Kinoshita, M. Miyoshi, and B.-H. Juang, · 2010
Earlier work this paper cites.
“Music dereverberation using harmonic structure source model and wiener filter,”
N. Yasuraoka, T. Yoshioka, T. Nakatani, A. Nakamura, and H. G. Okuno, · 2010
Earlier work this paper cites.
DAFX: Digital Audio Effects: Second Edition
U. Zölzer, · 2011
Earlier work this paper cites.
“I-divergence-based dereverberation method with auxiliary function approach,”
N. Yasuraoka, H. Kameoka, T. Yoshioka, and H. G. Okuno, · 2011
Earlier work this paper cites.
“Blind single-channel dereverberation for music post-processing,”
A. Tsilfidis and J. Mourjopoulos, · 2011
Earlier work this paper cites.
“Audio inpainting,”
A. Adler, V. Emiya, M. G. Jafari, M. Elad, R. Gribonval, and M. D. Plumbley, · 2011
Earlier work this paper cites.
“Fifty years of artificial reverberation,”
V. Valimaki, J. D. Parker, L. Savioja, J. O. Smith, and J. S. Abel, · 2012
Earlier work this paper cites.
“Wide-band dereverberation method based on multichannel linear prediction using prewhitening filter,”
T. Okamoto, Y. Iwaya, and Y. Suzuki, · 2012
Earlier work this paper cites.
“Music dereverberation by spectral linear prediction in live recordings,”
K. Mahkonen, A. Eronen, T. Virtanen, E. Helander, V. Popa, and I. D. D. Curcio, · 2013
Earlier work this paper cites.
“Sparse reverberant audio source separation via reweighted analysis,”
S. Arberet, P. Vandergheynst, R. E. Carrillo, J.-P. Thiran, and Y. Wiaux, · 2013
Earlier work this paper cites.
“The NUS sung and spoken lyrics corpus: A quantitative comparison of singing and speech,”
Z. Duan, H. Fang, B. Li, and Y. Sim, K. C.and Wang, · 2013
Earlier work this paper cites.

Similar

“Nonparametric bayesian dereverberation of power spectrograms based on infinite-order autoregressive processes,”
A. Maezawa, K. Itoyama, K. Yoshii, and H. G. Okuno, · 2014
Cited alongside, same era.
“U-net: Convolutional networks for biomedical image segmentation,”
O. Ronneberger, P. Fischer, and T. Brox, · 2015
Cited alongside, same era.
“Cnn architectures for large-scale audio classification,”
S. Hershey, S. Chaudhuri, D. P. W. Ellis, J. F. Gemmeke, A. Jansen, R. C. Moore, M. Plakal, D. Platt, R. A. Saurous, B. Seybold, M. Slaney, R. J. Weiss, and K. Wilson, · 2017
Cited alongside, same era.
“Deep image prior,”
D. Ulyanov, A. Vedaldi, and V. Lempitsky, · 2018
Cited alongside, same era.
“Solving linear inverse problems using gan priors: An algorithm with provable guarantees,”
V. Shah and C. Hegde, · 2018
Cited alongside, same era.
“Fréchet Audio Distance: A metric for evaluating music enhancement algorithms,”
Original
K. Kilgour, M. Zuluaga, D. Roblek, and M. Sharifi, · 2018
Cited alongside, same era.
“webMUSHRA — A comprehensive framework for web-based listening tests.,”
M. Schoeffler and et al., · 2018
Cited alongside, same era.
“Mixed precision training,”
P. Micikevicius, S. Narang, J. Alben, G. Diamos, E. Elsen, D. Garcia, B. Ginsburg, M. Houston, O. Kuchaiev, G. Venkatesh, and H. Wu, · 2018
Cited alongside, same era.
“Decoupled weight decay regularization,”
I. Loshchilov and F. Hutter, · 2019
Cited alongside, same era.
“A history of audio effects,”
T. Wilmering, D. Moffat, A. Milo, and M. B. Sandler, · 2020
Cited alongside, same era.
“Denoising diffusion probabilistic models,”
J. Ho, A. Jain, and P. Abbeel, · 2020
Cited alongside, same era.
“NHSS: A speech and singing parallel database,”
Original
B. Sharma, X. Gao, K. Vijayan, X. Tian, and H. Li, · 2020
Cited alongside, same era.

Then

“Improved techniques for training score-based generative models,”
Y. Song and S. Ermon, · 2020
Later among the works it cites.
“Reverb conversion of mixed vocal tracks using an end-to-end convolutional deep neural network,”
J. Koo, S. Paik, and K. Lee, · 2021
Later among the works it cites.
“Score-based generative modeling through stochastic differential equations,”
Y. Song, J. S.-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, · 2021
Later among the works it cites.
“Stochastic solutions for linear inverse problems using the prior implicit in a denoiser,”
Z. Kadkhodaie and E. Simoncelli, · 2021
Later among the works it cites.
“SNIPS: Solving noisy inverse problems stochastically,”
B. Kawar, G. Vaksman, and M. Elad, · 2021
Later among the works it cites.
“Diffusion models beat GANs on image synthesis,”
P. Dhariwal and A. Nichol, · 2021
Later among the works it cites.
“Improved denoising diffusion probabilistic models,”
Original
A. Nichol and P. Dhariwal, · 2021
Later among the works it cites.
“Denoising diffusion restoration models,”
B. Kawar, M. Elad, S. Ermon, and J. Song, · 2022
Closest in time.
“Music enhancement via image translation and vocoding,”
N. Kandpal, O. Nieto, and Z. Jin, · 2022
Closest in time.
“Improving diffusion models for inverse problems using manifold constraints,”
Original
H. Chung, B. Sim, D. Ryu, and J. C. Ye, · 2022
Closest in time.
“Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction,”
H. Chung, B. Sim, and J. C. Ye, · 2022
Closest in time.
“Regularizing score-based models with score fokker-planck equations,”
Original
C.-H. Lai, Y. Takida, N. Murata, T. Uesaka, Y. Mitsufuji, and S. Ermon, · 2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…