Fetching the paper…

Reading the bibliography…

2019

Hierarchical Sequence to Sequence Voice Conversion with Limited Data

Narayanan, Praveen, Chakravarty, Punarjay, Charette, Francois et al.

Original
PDF
Post on X

Understand

We present a voice conversion solution using recurrent sequence to sequence modeling for DNNs.

Our solution takes advantage of recent advances in attention based modeling in the fields of Neural Machine Translation (NMT), Text-to-Speech (TTS) and Automatic Speech Recognition (ASR).
The problem consists of converting between voices in a parallel setting when {\it $<$source,target$>$} audio pairs are available.
Our seq2seq architecture makes use of a hierarchical encoder to summarize input audio frames.

Hierarchical Sequence to Sequence Voice Conversion with Limited Data · Around

Built on

E. Nachmani and L. Wolf, “Unsupervised polyglot text to speech,”
Original
1902
Earlier work this paper cites.
R. J. Williams and D. Zipser, “A learning algorithm for continually running fully recurrent neural networks,”
1989
Earlier work this paper cites.
A. Kain and M. Macon, “Spectral voice conversion for text-to-speech synthesis,” in
1998
Earlier work this paper cites.
J. Kominek and A. W. Black, “Cmu arctic databases for speechsynthesis,” Language Technology Institute, Carnegie Mellon University, Pittsburgh, PA, 2003. [Online]. Available: http://festvox.org/cmu arctic/index.html
2003
Earlier work this paper cites.
T. Toda, A. W. Black, and K. Tokuda, “Voice conversion based on maximum-likelihood estimation of spectral parameter trajectory,”
2007
Earlier work this paper cites.
M. Müller,
2007
Earlier work this paper cites.
S. Desai, E. V. Raghavendra, B. Yegnanarayana, A. W. Black, and K. Prahallad, “Voice conversion using artificial neural networks,” in
2009
Earlier work this paper cites.
S. Desai, A. W. Black, and B. Yegnanarayana, “Voice conversion using artificial neural networks,” in
2010
Earlier work this paper cites.
R. Takashima, T. Takiguchi, and Y. Ariki, “Exemplar-based voice conversion using sparse representation in noisy environments,”
2013
Earlier work this paper cites.
D. Kingma and M. Welling, “Autoencoding variational bayes,”
Original
2013
Earlier work this paper cites.
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Wade-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,”
Original
2014
Earlier work this paper cites.
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,”
2014
Earlier work this paper cites.
L. Sun, S. Yang, K. Li, and H. Meng, “Voice conversion using deep bidirectional long short-term memory,” in
2015
Earlier work this paper cites.
W. Chan, N. Jaitly, Q. V. Le, and O. Vinyals, “Listen, attend and spell,”
Original
2015
Earlier work this paper cites.
A. B. L. Larsen, S. K. Sønderby, and O. Winther, “Autoencoding beyond pixels using a learned similarity metric,”
Original
2015
Earlier work this paper cites.
S. Bengio, O. Vinyals, N. Jaitly, and N. Shazeer, “Scheduled sampling for sequence prediction with recurrent neural networks,”
Original
2015
Earlier work this paper cites.
M.-T. Luong, H. Pham, and C. D. Manning, “Effective approaches to attention-based neural machine translation,”
Original
2015
Earlier work this paper cites.
J. Chorowski, D. Bahdanau, D. Serdyuk, K. Cho, and Y. Bengio, “Attention based models for speech recognition,”
Original
2015
Earlier work this paper cites.

Similar

D. J. Rezende and S. Mohamed, “Variational normalizing flows,”
Original
2015
Cited alongside, same era.
L. Sun, K. Li, S. Kang, and H. Meng, in
2016
Cited alongside, same era.
A. Lamb, A. Goyal, Y. Zhang, S. Zhang, A. Courville, and Y. Bengio, “Professor forcing: A new algorithm for training recurrent networks,”
Original
2016
Cited alongside, same era.
J. Lee, K. Cho, and T. Hoffman, “Fully character-level neural machine translation without explicit segmentation,”
Original
2016
Cited alongside, same era.
A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. W. Senior, and K. Kavukcuoglu, “Wavenet: A generative model for raw audio,”
Original
2016

Then

C. Hsu, H. Hwang, Y. Wu, Y. Tsao, and H. Wang, “Voice conversion from unaligned corpora using variational autoencoding wasserstein generative adversarial networks,”
Original
2017
Later among the works it cites.
M. Arjrovsky, S. Chintala, and L. Bottou, “Wasserstein gan,”
Original
2017
Later among the works it cites.
T. Kaneko and H. Kameoka, “Parallel-data-free voice conversion using cycle-consistent adversarial networks,”
Original
2017
Later among the works it cites.
Y. Taigman, L. Wolf, A. Polyak, and E. Nachmani, “Voice synthesis for in-the-wild speakers via a phonological loop,”
Original
2017
Later among the works it cites.
K. Ito, “The lj speech dataset,” 2017. [Online]. Available: https://keithito.com/LJ-Speech-Dataset/

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Cited alongside, same era.

N. Kalchbrenner, L. Espeholt, K. Simonyan, A. van den Oord, A. Graves, and K. Kavukcuoglu, “Neural machine translation in linear time,”

2016

Cited alongside, same era.

A. van den Oord, N. Kalchbrenner, O. Vinyals, L. Espeholt, A. Graves, and K. Kavukcuoglu, “Conditional image generation with pixelcnn decoders,”

2016

Cited alongside, same era.

N. Kalchbrenner, A. van den Oord, K. Simonyan, I. Danihelka, O. Vinyals, A. Graves, and K. Kavukcuoglu, “Video pixel networks,”

2016

Cited alongside, same era.

D. P. Kingma, T. Salimans, and M. Welling, “Improving variational inference with inverse autoregressive flow,”

2016

Cited alongside, same era.

Y. Wang, R. J. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Y. Xiao, Z. Chen, S. Bengio, Q. Le, Y. Agiomyrgiannakis, R. Clark, and R. A. Saurous, “Tacotron: Towards end to end speech synthesis,”

2017

Cited alongside, same era.

J. Shen, R. Pang, R. J. Weiss, N. Jaitly, Z. Yang, Z. Chen, Y. Zhang, Y. Wang, R. Skerry-Ryan, R. A. Sauros, Y. Agiomyrgiannakis, and Y. Wu, “Natural tts synthesis by conditioning wavenet on mel spectrogram predictions,”

2017

Cited alongside, same era.

S. O. Arik, M. Chrzanowski, A. Coates, G. Diamos, A. Gibiansky, Y. Kang, X. Li, J. Miller, A. Ng, J. Raiman, S. Sengupta, and M. Shoeybi, “Deep voice: Real-time neural text-to-speech,”

2017

Cited alongside, same era.

2017

Later among the works it cites.

A. van den Oord, Y. Li, I. Babuschkin, K. Simonyan, O. Vinyals, K. Kavukcuoglu, G. van den Driessche, E. Lockhart, L. C. Cobo, F. Stimberg, N. Casagrande, D. Grewe, S. Noury, S. Dieleman, E. Elsen, N. Kalchbrenner, H. Zen, A. Graves, H. King, T. Walters, D. Belov, and D. Hassabis, “Parallel wavenet: Fast high-fidelity speech synthesis,”

2017

Later among the works it cites.

J.-X. Zhang, Z.-H. Ling, L.-J. Liu, Y. Jiang, and L.-R. Dai, “Sequence-to-sequence acoustic modeling for voice conversion,”

2018

Later among the works it cites.

J. Zhang, Z. Ling, Y. Jiang, L. Liu, C. Liang, and L. Dai, “Improving sequence-to-sequence acoustic modeling by adding text-supervision,”

2018

Later among the works it cites.

K. Tanaka, H. Kameoka, T. Kaneko, and N. Hojo, “Atts2s-vc: Sequence-to-sequence voice conversion with attention and context preservation mechanisms,”

2018

Later among the works it cites.

H. Kameoka, K. Tanaka, T. Kaneko, and N. Hojo, “Convs2s-vc fully convolutional sequence-to-sequence voice conversion,”

2018

Later among the works it cites.

H. Kameoka and T. Kaneko, “Stargan-vc: Non-parallel many-to-many voice conversion with star generative adversarial networks,”

2018

Later among the works it cites.

Y. Chen, Y. M. Assael, B. Shillingford, D. Budden, S. E. Reed, H. Zen, Q. Wang, L. C. Cobo, A. Trask, B. Laurie, Ç. Gülçehre, A. van den Oord, O. Vinyals, and N. de Freitas, “Sample efficient adaptive text-to-speech,”

2018

Later among the works it cites.

Y. Jia, Y. Zhang, R. J. Weiss, Q. Wang, J. Shen, F. Ren, Z. Chen, P. Nguyen, R. Pang, I. Lopez-Moreno, and Y. Wu, “Transfer learning from speaker verification to multispeaker text-to-speech synthesis,”

2018

Later among the works it cites.

S. Ö. Arik, J. Chen, K. Peng, W. Ping, and Y. Zhou, “Neural voice cloning with a few samples,”

2018

Later among the works it cites.

E. Nachmani, A. Polyak, Y. Taigman, and L. Wolf, “Fitting new speakers based on a short untranscribed sample,”

2018

Later among the works it cites.

R. Yamamoto, “Wavenet vocoder,” 2018. [Online]. Available: https://github.com/r9y9/wavenet_vocoder

2018

Later among the works it cites.

R. Prenger, R. Valle, and B. Catanzaro, “Waveglow: A flow-based generative network for speech synthesis,”

2018

Later among the works it cites.

Y. Li, M. Sun, H. Van Hamme, X. Zhang, and J. Yang, “Robust hierarchical learning for non-negative matrix factorization with outliers,”

2019

Closest in time.