2020

Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech

Yang, Geng, Yang, Shan, Liu, Kai et al.

Understand

In this paper, we propose multi-band MelGAN, a much faster waveform generation model targeting to high-quality text-to-speech.

Specifically, we improve the original MelGAN by the following aspects.
First, we increase the receptive field of the generator, which is proven to be beneficial to speech generation.
Second, we substitute the feature matching loss with the multi-resolution STFT loss to better measure the difference between fake and real speech.

Built on

“NICE: Non-linear Independent Components Estimation,”
Original
Laurent Dinh, David Krueger, and Yoshua Bengio, · 2014
Earlier work this paper cites.
“Generative Adversarial Nets,”
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio, · 2014
Earlier work this paper cites.
“Adam: A Method for Stochastic Optimization,”
Original
Diederik P Kingma and Jimmy Ba, · 2014
Earlier work this paper cites.
“WaveNet: A Generative Model for Raw Audio,”
Original
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu, · 2016
Earlier work this paper cites.
“SampleRNN: An Unconditional End-to-End Neural Audio Generation Model,”
Original
Soroush Mehri, Kundan Kumar, Ishaan Gulrajani, Rithesh Kumar, Shubham Jain, Jose Sotelo, Aaron Courville, and Yoshua Bengio, · 2016
Earlier work this paper cites.
“Improved Variational Inference with Inverse Autoregressive Flow,”
Durk P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling, · 2016
Earlier work this paper cites.
“Density estimation using Real NVP,”
Original
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio, · 2016
Earlier work this paper cites.
“Parallel WaveNet: Fast High-Fidelity Speech Synthesis,”
Original
Aaron van den Oord, Yazhe Li, Igor Babuschkin, Karen Simonyan, Oriol Vinyals, Koray Kavukcuoglu, George van den Driessche, Edward Lockhart, Luis C Cobo, Florian Stimberg, et al., · 2017
Earlier work this paper cites.
“Improved Training of Wasserstein GANs,”
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville, · 2017
Earlier work this paper cites.

Similar

“Progressive Growing of GANs for Improved Quality, Stability, and Variation,”
Original
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen, · 2017
Cited alongside, same era.
“Efficient Neural Audio Synthesis,”
Original
Nal Kalchbrenner, Erich Elsen, Karen Simonyan, Seb Noury, Norman Casagrande, Edward Lockhart, Florian Stimberg, Aaron van den Oord, Sander Dieleman, and Koray Kavukcuoglu, · 2018
Cited alongside, same era.
“ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech,”
Original
Wei Ping, Kainan Peng, and Jitong Chen, · 2018
Cited alongside, same era.
“Glow: Generative Flow with Invertible 1x1 Convolutions,”
Durk P Kingma and Prafulla Dhariwal, · 2018
Cited alongside, same era.
“FloWaveNet : A Generative Flow for Raw Audio,”
Original
Sungwon Kim, Sang-gil Lee, Jongyoon Song, Jaehyeon Kim, and Sungroh Yoon, · 2018
Cited alongside, same era.
“High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANS,”
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro, · 2018
Cited alongside, same era.
“Video-to-Video Synthesis,”
Original
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, and Bryan Catanzaro, · 2018
Cited alongside, same era.
“Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks,”
Sercan Ö Arık, Heewoo Jun, and Gregory Diamos, · 2018
Cited alongside, same era.
“Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions,”
Jonathan Shen, Ruoming Pang, Ron J Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, Rj Skerrv-Ryan, et al., · 2018
Cited alongside, same era.

Then

“LPCNet: Improving Neural Speech Synthesis Through Linear Prediction,”
Jean-Marc Valin and Jan Skoglund, · 2019
Later among the works it cites.
“A Real-Time Wideband Neural Vocoder at 1.6 kb/s Using LPCNet,”
Original
Valin, Jean-Marc and Skoglund, Jan, · 2019
Later among the works it cites.
“DurIAN: Duration Informed Attention Network For Multimodal Synthesis,”
Original
Chengzhu Yu, Heng Lu, Na Hu, Meng Yu, Chao Weng, Kun Xu, Peng Liu, Deyi Tuo, Shiyin Kang, Guangzhi Lei, Dan Su, and Dong Yu, · 2019
Later among the works it cites.
“WaveGlow: A Flow-based Generative Network for Speech Synthesis,”
Ryan Prenger, Rafael Valle, and Bryan Catanzaro, · 2019
Later among the works it cites.
“MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis,”
Kundan Kumar, Rithesh Kumar, Thibault de Boissiere, Lucas Gestin, Wei Zhen Teoh, Jose Sotelo, Alexandre de Brébisson, Yoshua Bengio, and Aaron C Courville, · 2019
Later among the works it cites.
“Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram,”
Original
Ryuichi Yamamoto, Eunwoo Song, and Jae-Min Kim, · 2019
Later among the works it cites.
“High Fidelity Speech Synthesis with Adversarial Networks,”
Original
Mikołaj Bińkowski, Jeff Donahue, Sander Dieleman, Aidan Clark, Erich Elsen, Norman Casagrande, Luis C Cobo, and Karen Simonyan, · 2019
Later among the works it cites.
“Probability density distillation with generative adversarial networks for high-quality parallel waveform generation,”
Original
Ryuichi Yamamoto, Eunwoo Song, and Jae-Min Kim, · 2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…