2020

Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling

Shen, Jonathan, Jia, Ye, Chrzanowski, Mike et al.

Understand

This paper presents Non-Attentive Tacotron based on the Tacotron 2 text-to-speech model, replacing the attention mechanism with an explicit duration predictor.

This improves robustness significantly as measured by unaligned duration ratio and word deletion rate, two metrics introduced in this paper for large-scale robustness evaluation using a pre-trained speech recognition model.
With the use of Gaussian upsampling, Non-Attentive Tacotron achieves a 5-scale mean opinion score for naturalness of 4.41, slightly outperforming Tacotron 2.
The duration predictor enables both utterance-wide and per-phoneme control of duration at inference time.

Built on

A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J. and Zipser, D · 1989
Earlier work this paper cites.
The aligner: Text to speech alignment using markov models and a pronunciation dictionary
Talkin, D. and Wightman, C. W · 1994
Earlier work this paper cites.
Statistical Parametric Speech Synthesis
Zen, H., Tokuda, K., and Black, A · 2009
Earlier work this paper cites.
Generating Sequences with Recurrent Neural Networks
Original
Graves, A · 2013
Earlier work this paper cites.
Statistical Parametric Speech Synthesis Using Deep Neural Networks
Zen, H., Senior, A., and Schuster, M · 2013
Earlier work this paper cites.
Auto-Encoding Variational Bayes
Kingma, D. P. and Welling, M · 2014
Earlier work this paper cites.
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., and Bengio, Y · 2015
Earlier work this paper cites.
Attention-based models for speech recognition
Chorowski, J. K., Bahdanau, D., Serdyuk, D., Cho, K., and Bengio, Y · 2015
Earlier work this paper cites.
Effective approaches to attention-based neural machine translation
Original
Luong, M.-T., Pham, H., and Manning, C. D · 2015
Earlier work this paper cites.
Librispeech: An ASR Corpus Based on Public Domain Audio Books
Panayotov, V., Chen, G., Povey, D., and Khudanpur, S · 2015
Earlier work this paper cites.
Listen, Attend and Spell: A Neural Network for Large Vocabulary Conversational Speech Recognition
Chan, W., Jaitly, N., Le, Q., and Vinyals, O · 2016
Earlier work this paper cites.
WaveNet: A Generative Model for Raw Audio
Original
Oord, A. v. d., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K · 2016
Earlier work this paper cites.
Deep Voice: Real-Time Neural Text-to-Speech
Arik, S. O., Chrzanowski, M., Coates, A., Diamos, G., Gibiansky, A., Kang, Y., Li, X., Miller, J., Ng, A., Raiman, J., Sengupta, S., and Shoeybi, M · 2017
Earlier work this paper cites.
Deep voice 2: Multi-speaker neural text-to-speech
Gibiansky, A., Arik, S., Diamos, G., Miller, J., Peng, K., Ping, W., Raiman, J., and Zhou, Y · 2017
Earlier work this paper cites.
Non-autoregressive neural machine translation
Original
Gu, J., Bradbury, J., Xiong, C., Li, V. O., and Socher, R · 2017
Earlier work this paper cites.
Char2Wav: End-to-End speech synthesis
Sotelo, J., Mehri, S., Kumar, K., Santos, J. F., Kastner, K., Courville, A. C., and Bengio, Y · 2017
Earlier work this paper cites.
Tacotron: Towards End-to-End Speech Synthesis
Wang, Y., Skerry-Ryan, R., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., Le, Q., Agiomyrgiannakis, Y., Clark, R., and Saurous, R · 2017
Earlier work this paper cites.

Similar

Speech Recognition for Medical Conversations
Chiu, C.-C., Tripathi, A., Chou, K., Co, C., Jaitly, N., Jaunzeikare, D., Kannan, A., Nguyen, P., Sak, H., Sankar, A., Tansuwan, J., Wan, N., Wu, Y., and Zhang, X · 2018
Cited alongside, same era.
Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
Jia, Y., Zhang, Y., Weiss, R. J., Wang, Q., Shen, J., Ren, F., Chen, Z., Nguyen, P., Pang, R., Lopez-Moreno, I., and Wu, Y · 2018
Cited alongside, same era.
Efficient Neural Audio Synthesis
Kalchbrenner, N., Elsen, E., Simonyan, K., Noury, S., Casagrande, N., Lockhart, E., Stimberg, F., Oord, A. v. d., Dieleman, S., and Kavukcuoglu, K · 2018
Cited alongside, same era.
Deep Voice 3: Scaling text-to-speech with convolutional sequence learning
Ping, W., Peng, K., Gibiansky, A., Arik, S. O., Kannan, A., Narang, S., Raiman, J., and Miller, J · 2018
Cited alongside, same era.
Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions
Shen, J., Pang, R., Weiss, R. J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., Skerrv-Ryan, R., Saurous, R. A., Agiomyrgiannakis, Y., and Wu, Y · 2018
Cited alongside, same era.
Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron
Skerry-Ryan, R., Battenberg, E., Xiao, Y., Wang, Y., Stanton, D., Shor, J., Weiss, R., Clark, R., and Saurous, R. A · 2018
Cited alongside, same era.
Forward attention in sequence-to-sequence acoustic modeling for speech synthesis
Zhang, J.-X., Ling, Z.-H., and Dai, L.-R · 2018
Cited alongside, same era.
Parrotron: An End-to-End Speech-to-Speech Conversion Model and its Applications Hearing-Impaired Speech and Speech Separation
Biadsy, F., Weiss, R. J., Moreno, P. J., Kanvesky, D., and Jia, Y · 2019
Cited alongside, same era.
A New GAN-based End-to-End TTS Training Algorithm
Guo, H., Soong, F. K., He, L., and Xie, L · 2019
Cited alongside, same era.
Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTS
He, M., Deng, Y., and He, L · 2019
Cited alongside, same era.
Direct Speech-to-Speech Translation with a Sequence-to-Sequence Model
Jia, Y., Weiss, R. J., Biadsy, F., Macherey, W., Johnson, M., Chen, Z., and Wu, Y · 2019
Cited alongside, same era.
CHiVE: Varying Prosody in Speech Synthesis with a Linguistically Driven Dynamic Hierarchical Conditional Variational Network
Kenter, T., Wan, V., Chan, C.-A., Clark, R., and Vit, J · 2019
Cited alongside, same era.

Then

DurIAN: Duration informed attention network for multimodal synthesis
Original
Yu, C., Lu, H., Hu, N., Yu, M., Weng, C., Xu, K., Liu, P., Tuo, D., Kang, S., Lei, G., Su, D., and Yu, D · 2019
Later among the works it cites.
LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech
Zen, H., Dang, V., Clark, R., Zhang, Y., Weiss, R. J., Jia, Y., Chen, Z., and Wu, Y · 2019
Later among the works it cites.
Forward–backward decoding sequence for regularizing end-to-end tts
Zheng, Y., Tao, J., Zhengqi, W., and Yi, J · 2019
Later among the works it cites.
Location-Relative Attention Mechanisms For Robust Long-Form Speech Synthesis
Battenberg, E., Skerry-Ryan, R., Mariooryad, S., Stanton, D., Kao, D., Shannon, M., and Bagby, T · 2020
Closest in time.
TalkNet: Fully-Convolutional Non-Autoregressive Speech Synthesis Model
Original
Beliaev, S., Rebryk, Y., and Ginsburg, B · 2020
Closest in time.
WaveGrad: Estimating Gradients for Waveform Generation
Original
Chen, N., Zhang, Y., Zen, H., Weiss, R. J., Norouzi, M., and Chan, W · 2020
Closest in time.
Textual Echo Cancellation
Original
Ding, S., Jia, Y., Hu, K., and Wang, Q · 2020
Closest in time.
End-to-End Adversarial Text-to-Speech
Original
Donahue, J., Dieleman, S., Bińkowski, M., Elsen, E., and Simonyan, K · 2020
Closest in time.
Parallel tacotron: Non-autoregressive and controllable tts
Original
Elias, I., Zen, H., Shen, J., Zhang, Y., Jia, Y., Weiss, R., and Wu, Y · 2020
Closest in time.
Libri-light: A Benchmark for ASR with Limited or No Supervision
Kahn, J., Rivière, M., Zheng, W., Kharitonov, E., Xu, Q., Mazaré, P.-E., Karadayi, J., Liptchinsky, V., Collobert, R., Fuegen, C., Likhomanenko, T., Synnaeve, G., Joulin, A., Mohamed, A., and Dupoux, E · 2020
Closest in time.
JDI-T: Jointly Trained Duration Informed Transformer for Text-To-Speech without Explicit Alignment
Original
Lim, D., Jang, W., Park, H., Kim, B., and Yoon, J · 2020
Closest in time.
Improved Noisy Student Training for Automatic Speech Recognition
Park, D. S., Zhang, Y., Jia, Y., Han, W., Chiu, C.-C., Li, B., Wu, Y., and Le, Q. V · 2020
Closest in time.
FastSpeech 2: Fast and High-Quality End-to-End Text-to-Speech
Original
Ren, Y., Hu, C., Qin, T., Zhao, S., Zhao, Z., and Liu, T.-Y · 2020
Closest in time.
Fully-Hierarchical Fine-Grained Prosody Modeling for Interpretable Speech Synthesis
Sun, G., Zhang, Y., Weiss, R. J., Cao, Y., Zen, H., and Wu, Y · 2020
Closest in time.
AlignTTS: Efficient Feed-Forward Text-to-Speech System without Explicit Alignment
Zeng, Z., Wang, J., Cheng, N., Xia, T., and Xiao, J · 2020
Closest in time.
Adadurian: Few-shot adaptation for neural text-to-speech with durian
Original
Zhang, Z., Tian, Q., Lu, H., Chen, L.-H., and Liu, S · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…