2018

WaveGlow: A Flow-based Generative Network for Speech Synthesis

Prenger, Ryan, Valle, Rafael, Catanzaro, Bryan

Understand

In this paper we propose WaveGlow: a flow-based network capable of generating high quality speech from mel-spectrograms.

WaveGlow combines insights from Glow and WaveNet in order to provide fast, efficient and high-quality audio synthesis, without the need for auto-regression.
WaveGlow is implemented using only a single network, trained using only a single cost function: maximizing the likelihood of the training data, which makes the training procedure simple and stable.
Our PyTorch implementation produces audio samples at a rate of more than 500 kHz on an NVIDIA V100 GPU.

“Signal estimation from modified short-time fourier transform,”
Daniel Griffin and Jae Lim, · 1984
Earlier work this paper cites.
“Nice: Non-linear independent components estimation,”
Original
Laurent Dinh, David Krueger, and Yoshua Bengio, · 2014
Earlier work this paper cites.
“Adam: A method for stochastic optimization,”
Original
Diederik P Kingma and Jimmy Ba, · 2014
Earlier work this paper cites.
“Variational inference with normalizing flows,”
Original
Danilo Jimenez Rezende and Shakir Mohamed, · 2015
Earlier work this paper cites.
“Wavenet: A generative model for raw audio,”
Original
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu, · 2016
Earlier work this paper cites.
“Improved variational inference with inverse autoregressive flow,”
Diederik P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling, · 2016
Earlier work this paper cites.
“Density estimation using real nvp,”
Original
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio, · 2016
Earlier work this paper cites.
“Weight normalization: A simple reparameterization to accelerate training of deep neural networks,”
Tim Salimans and Diederik P Kingma, · 2016
Earlier work this paper cites.

“Tacotron: A fully end-to-end text-to-speech synthesis model,”
Original
Yuxuan Wang, RJ Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Zhifeng Chen, Samy Bengio, et al., · 2017
Cited alongside, same era.
“Natural tts synthesis by conditioning wavenet on mel spectrogram predictions,”
Original
Jonathan Shen, Ruoming Pang, Ron J Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, RJ Skerry-Ryan, et al., · 2017
Cited alongside, same era.
“Deep voice: Real-time neural text-to-speech,”
Original
Sercan O Arik, Mike Chrzanowski, Adam Coates, Gregory Diamos, Andrew Gibiansky, Yongguo Kang, Xian Li, John Miller, Andrew Ng, Jonathan Raiman, et al., · 2017
Cited alongside, same era.
“The LJ speech dataset,” 2017
Keith Ito et al., · 2017
Cited alongside, same era.
“Deep voice 2: Multi-speaker neural text-to-speech,”
Original
Sercan Arik, Gregory Diamos, Andrew Gibiansky, John Miller, Kainan Peng, Wei Ping, Jonathan Raiman, and Yanqi Zhou, · 2017
Cited alongside, same era.
“Glow: Generative flow with invertible 1x1 convolutions,”
Original
Diederik P Kingma and Prafulla Dhariwal, · 2018
Cited alongside, same era.
“Waveglow,” https://nv-adlr.github.io/WaveGlow , 2018
Ryan Prenger and Rafael Valle, · 2018
Cited alongside, same era.

“Clarinet: Parallel wave generation in end-to-end text-to-speech,”
Original
Wei Ping, Kainan Peng, and Jitong Chen, · 2018
Closest in time.
“Fast spectrogram inversion using multi-head convolutional neural networks,”
Original
Sercan O Arik, Heewoo Jun, and Gregory Diamos, · 2018
Closest in time.
“Parallel WaveNet: Fast high-fidelity speech synthesis,”
Aaron van den Oord, Yazhe Li, Igor Babuschkin, Karen Simonyan, Oriol Vinyals, Koray Kavukcuoglu, George van den Driessche, Edward Lockhart, Luis Cobo, Florian Stimberg, Norman Casagrande, Dominik Grewe, Seb Noury, Sander Dieleman, Erich Elsen, Nal Kalchbrenner, Heiga Zen, Alex Graves, Helen King, Tom Walters, Dan Belov, and Demis Hassabis, · 2018
Closest in time.
“Image transformer,”
Original
Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Łukasz Kaiser, Noam Shazeer, and Alexander Ku, · 2018
Closest in time.
“Wavenet vocoder,” https://doi.org/10.5281/zenodo.1472609 , 2018
Ryuichi Yamamoto, · 2018
Closest in time.
“FFTNet: a real-time speaker-dependent neural vocoder,”
Zeyu Jin, Adam Finkelstein, Gautham J. Mysore, and Jingwan Lu, · 2018
Closest in time.
“Efficient neural audio synthesis,”
Original
Nal Kalchbrenner, Erich Elsen, Karen Simonyan, Seb Noury, Norman Casagrande, Edward Lockhart, Florian Stimberg, Aaron van den Oord, Sander Dieleman, and Koray Kavukcuoglu, · 2018
Closest in time.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…