2021

Universal Neural Vocoding with Parallel WaveNet

Jiao, Yunlong, Gabrys, Adam, Tinchev, Georgi et al.

Understand

We present a universal neural vocoder based on Parallel WaveNet, with an additional conditioning network called Audio Encoder.

Our universal vocoder offers real-time high-quality speech synthesis on a wide range of use cases.
We tested it on 43 internal speakers of diverse age and gender, speaking 20 languages in 17 unique styles, of which 7 voices and 5 styles were not exposed during training.
We show that the proposed universal vocoder significantly outperforms speaker-dependent vocoders overall.

“BS. 1534-1. method for the subjective assessment of intermediate sound quality (MUSHRA),”
I. Recommendation, · 2001
Earlier work this paper cites.
“Auto-encoding variational bayes,”
D. P. Kingma and M. Welling, · 2014
Earlier work this paper cites.
“Adam: A method for stochastic optimization,”
D. P. Kingma and J. Ba, · 2015
Earlier work this paper cites.
“Wavenet: A generative model for raw audio,”
A. van den Oord, S. Dieleman, H. Zen, et al., · 2016
Earlier work this paper cites.
“Deep voice: Real-time neural text-to-speech,”
S. Ö. Arik, M. Chrzanowski, A. Coates, et al., · 2017
Earlier work this paper cites.
“Tacotron: A fully end-to-end text-to-speech synthesis model,”
Original
Y. Wang, R. J. Skerry-Ryan, D. Stanton, et al., · 2017
Earlier work this paper cites.
“Deep voice 2: Multi-speaker neural text-to-speech,”
A. Gibiansky, S. Ö. Arik, G. F. Diamos, et al., · 2017
Earlier work this paper cites.
“An investigation of multi-speaker training for wavenet vocoder,”
T. Hayashi, A. Tamamori, K. Kobayashi, K. Takeda, and T. Toda, · 2017
Earlier work this paper cites.
“Natural TTS synthesis by conditioning wavenet on MEL spectrogram predictions,”
J. Shen, R. Pang, R. J. Weiss, et al., · 2018
Earlier work this paper cites.

“Efficient neural audio synthesis,”
N. Kalchbrenner, E. Elsen, K. Simonyan, et al., · 2018
Cited alongside, same era.
“Parallel wavenet: Fast high-fidelity speech synthesis,”
A. van den Oord, Y. Li, I. Babuschkin, et al., · 2018
Cited alongside, same era.
“A comparison of recent waveform generation and acoustic modeling methods for neural-network-based speech synthesis,”
X. Wang, J. Lorenzo-Trueba, S. Takaki, L. Juvela, and J. Yamagishi, · 2018
Cited alongside, same era.
“Multi-speaker neural vocoder,”
O. Barbany, A. Bonafonte, and S. Pascual, · 2018
Cited alongside, same era.
“Speaker-adaptive neural vocoders for statistical parametric speech synthesis systems,”
Original
E. Song, J. Kim, K. Byun, and H. Kang, · 2018
Cited alongside, same era.
“Clarinet: Parallel wave generation in end-to-end text-to-speech,”
W. Ping, K. Peng, and J. Chen, · 2019
Cited alongside, same era.
“Waveglow: A flow-based generative network for speech synthesis,”
R. Prenger, R. Valle, and B. Catanzaro, · 2019
Cited alongside, same era.
“Melgan: Generative adversarial networks for conditional waveform synthesis,”
K. Kumar, R. Kumar, T. de Boissiere, et al., · 2019
Cited alongside, same era.

“Fast spectrogram inversion using multi-head convolutional neural networks,”
S. Ö. Arik, H. Jun, and G. F. Diamos, · 2019
Later among the works it cites.
“Towards achieving robust universal neural vocoding,”
J. Lorenzo-Trueba, T. Drugman, J. Latorre, et al., · 2019
Later among the works it cites.
“LibriTTS: A corpus derived from librispeech for text-to-speech,”
H. Zen, V. Dang, R. Clark, et al., · 2019
Later among the works it cites.
“End-to-end adversarial text-to-speech,”
Original
J. Donahue, S. Dieleman, M. Binkowski, E. Elsen, and K. Simonyan, · 2020
Later among the works it cites.
“Parallel wavegan: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram,”
R. Yamamoto, E. Song, and J. Kim, · 2020
Later among the works it cites.
“Parallel WaveNet conditioned on VAE latent vectors,”
Original
J. Rohnke, T. Merritt, J. Lorenzo-Trueba, et al., · 2020
Later among the works it cites.
“https://voice.mozilla.org/,” March, 2020
Common Voice Database, · 2020
Later among the works it cites.
“https://www.clickworker.com/machine-learning-ai-artificial-intelligence/,” August, 2020
Original
Clickworker, · 2020
Later among the works it cites.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…