2021

Fast DCTTS: Efficient Deep Convolutional Text-to-Speech

Kang, Minsu, Lee, Jihyun, Kim, Simin et al.

Understand

We propose an end-to-end speech synthesizer, Fast DCTTS, that synthesizes speech in real time on a single CPU thread.

The proposed model is composed of a carefully-tuned lightweight network designed by applying multiple network reduction and fidelity improvement techniques.
In addition, we propose a novel group highway activation that can compromise between computational efficiency and the regularization effect of the gating mechanism.
As well, we introduce a new metric called Elastic mel-cepstral distortion (EMCD) to measure the fidelity of the output mel-spectrogram.

Built on

“Mel-cepstral distance measure for objective speech quality assessment,”
R. Kubicheck, · 1993
Earlier work this paper cites.
“Using dynamic time warping to find patterns in time series,”
D. J. Berndt and J. Clifford, · 1994
Earlier work this paper cites.
“Highway networks,”
Original
R. K. Srivastava, K. Greff, and J. Schmidhuber, · 2015
Earlier work this paper cites.
“Learning both weights and connections for efficient neural networks,”
S. Han, J. Pool, J. Tran, and W. J. Dally, · 2015
Earlier work this paper cites.
“Scheduled sampling for sequence prediction with recurrent neural networks,”
S. Bengio, O. Vinyals, N. Jaitly, and N. Shazeer, · 2015
Earlier work this paper cites.
“Pruning filters for efficient convnets,”
Original
H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, · 2016
Earlier work this paper cites.
“Weight normalization: A simple reparameterization to accelerate training of deep neural networks,”
T. Salimans and D. P. Kingma, · 2016
Earlier work this paper cites.
“Squeezenet: Alexnet-level accuracy with 50x fewer parameters and 0.5mb model size,”
Original
F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, · 2016
Earlier work this paper cites.
“Deep residual learning for image recognition,”
K. He, X. Zhang, S. Ren, and J. Sun, · 2016
Earlier work this paper cites.

Similar

“Tacotron: Towards end-to-end speech synthesis,”
Y. Wang, R. Skerry-Ryan, D. Santon, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, Q. Le, Y. Agiomyrgiannakis, R. Clark, and R. A. Saurous, · 2017
Cited alongside, same era.
“Attention is all you need,”
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, · 2017
Cited alongside, same era.
“Fastspeech: Fast, robust and controllable text to speech,”
Y. Ren, Y. Ruan, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T. Y. Liu, · 2017
Cited alongside, same era.
“Mobilenets: Efficient convolutional neural networks for mobile vision applications,”
Original
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, · 2017
Cited alongside, same era.
“The lj speech dataset,”
K. Ito, · 2017
Cited alongside, same era.
“Natural tts synthesis by conditioning wavenet on mel spectrogram predictions,”
J. Shen, R. Pang, R. J. Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, RJ Skerry-Ryan, Rif A. Saurous, Yannis Agiomyrgiannakis, and Y. Wu, · 2018
Cited alongside, same era.
“Efficiently trainable text-to-speech system based on deep convolutional networks with guided attention,”
H. Tachibana, K. Uenoyama, and S. Aihara, · 2018
Cited alongside, same era.
“Neural speech synthesis with transformer network,”
N. Li, S. Liu, Y. Liu, S. Zhao, M. Liu, and M. Zhou, · 2018
Cited alongside, same era.

Then

“Deep voice 3: Scaling text-to-speech with convolutional sequence learning,”
W. Ping, K. Peng, A. Gibiansky, S. O. Arik, A. Kannan, S. Narang, J. Raiman, and J. Miller, · 2018
Later among the works it cites.
“Shufflenet: An extremely efficient convolutional neural network for mobile devices,”
X. Zhang, X. Zhou, M. Lin, and J. Sun, · 2018
Later among the works it cites.
“Mobilenetv2: Inverted residuals and linear bottlenecks,”
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. C. Chen, · 2019
Later among the works it cites.
“Kss dataset: Korean single speaker speech dataset,”
K. Park, · 2019
Later among the works it cites.
“Aligntts: Efficient feed-forward text-to-speech system without explicit alignment,”
Z. Zeng, J. Wang, N. Cheng, T. Xia, and J. Xiao, · 2020
Later among the works it cites.
“Fastspeech 2: Fast and high-quality end-to-end text-to-speech,”
Original
Y. Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T. Y. Liu, · 2020
Later among the works it cites.
“Effective use of variational embedding capacity in expressive end-to-end speech synthesis,”
E. Battenberg, S. Mariooryad, D. Stanton, RJ Skerry-Ryan, M. Shannon, D. Kao, and T. Bagby, · 2020
Later among the works it cites.
“Parallel wavegan: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram,”
Original
R. Yamamoto, E. Song, and J.-M. Kim, · 2020
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…