2020

Self-Training for End-to-End Speech Translation

Pino, Juan, Xu, Qiantong, Ma, Xutai et al.

Understand

One of the main challenges for end-to-end speech translation is data scarcity.

We leverage pseudo-labels generated from unlabeled audio by a cascade and an end-to-end speech translation model.
This provides 8.3 and 5.7 BLEU gains over a strong semi-supervised baseline on the MuST-C English-French and English-German datasets, reaching state-of-the art performance.
The effect of the quality of the pseudo-labels is investigated.

Built on

H. Scudder, “Probability of error of some adaptive pattern-recognition machines,”
1965
Earlier work this paper cites.
A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks,” in
2006
Earlier work this paper cites.
K. Heafield, “KenLM: Faster and smaller language model queries,” in
2011
Earlier work this paper cites.
M. Schuster and K. Nakajima, “Japanese and Korean voice search,” in
2012
Earlier work this paper cites.
V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An ASR corpus based on public domain audio books,” in
2015
Earlier work this paper cites.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in
Original
2015
Earlier work this paper cites.
R. J. Weiss, J. Chorowski, N. Jaitly, Y. Wu, and Z. Chen, “Sequence-to-sequence models can directly translate foreign speech,” in
2017
Earlier work this paper cites.
M. A. Di Gangi, R. Cattoni, L. Bentivogli, M. Negri, and M. Turchi, “MuST-C: a Multilingual Speech Translation Corpus,” in
2017
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in
2017
Earlier work this paper cites.
A. Bérard, L. Besacier, A. C. Kocabiyikoglu, and O. Pietquin, “End-to-end automatic speech translation of audiobooks,” in
2018
Earlier work this paper cites.

Similar

T. Kudo and J. Richardson, “Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing,” in
2018
Cited alongside, same era.
M. Post, “A call for clarity in reporting BLEU scores,” in
2018
Cited alongside, same era.
V. Pratap, A. Hannun, Q. Xu
Original
2018
Cited alongside, same era.
M. Ott, S. Edunov, D. Grangier, and M. Auli, “Scaling neural machine translation,” in
2018
Cited alongside, same era.
B. Zhang, D. Xiong, and J. Su, “Accelerating neural transformer via an average attention network,”
Original
2018
Cited alongside, same era.
Y. Jia, M. Johnson, W. Macherey, R. J. Weiss, Y. Cao, C.-C. Chiu, N. Ari, S. Laurenzo, and Y. Wu, “Leveraging weakly supervised data to improve end-to-end speech-to-text translation,” in
2019
Cited alongside, same era.
J. Pino, L. Puzon, J. Gu, X. Ma, A. D. McCarthy, and D. Gopinath, “Harnessing indirect training data for end-to-end automatic speech translation: Tricks of the trade,” in
2019
Cited alongside, same era.
S. Bansal, H. Kamper, K. Livescu, A. Lopez, and S. Goldwater, “Pre-training on high-resource speech recognition improves low-resource speech-to-text translation,” in
2019
Cited alongside, same era.
M. A. Di Gangi, M. Negri, and M. Turchi, “One-to-many multilingual end-to-end speech translation,” in
2019
Cited alongside, same era.
H. Inaguma, K. Duh, T. Kawahara, and S. Watanabe, “Multilingual end-to-end speech translation,” in
2019
Cited alongside, same era.

Then

J. Kahn, A. Lee, and A. Hannun, “Self-training for end-to-end speech recognition,”
Original
2019
Later among the works it cites.
I. Z. Yalniz, H. Jégou, K. Chen, M. Paluri, and D. Mahajan, “Billion-scale semi-supervised learning for image classification,”
Original
2019
Later among the works it cites.
A. Mohamed, D. Okhonko, and L. Zettlemoyer, “Transformers with convolutional context for ASR,”
Original
2019
Later among the works it cites.
A. Hannun, A. Lee, Q. Xu, and R. Collobert, “Sequence-to-sequence speech recognition with time-depth separable convolutions,”
2019
Later among the works it cites.
M. C. Stoian, S. Bansal, and S. Goldwater, “Analyzing ASR pretraining for low-resource speech-to-text translation,” in
2020
Closest in time.
C. Wang, J. Pino, A. Wu, and J. Gu, “Covost: A diverse multilingual speech-to-text translation corpus,”
Original
2020
Closest in time.
J. He, J. Gu, J. Shen, and M. Ranzato, “Revisiting self-training for neural sequence generation,” in
2020
Closest in time.
G. Synnaeve, Q. Xu, J. Kahn, T. Likhomanenko, E. Grave, V. Pratap, A. Sriram, V. Liptchinsky, and R. Collobert, “End-to-end ASR: from supervised to semi-supervised learning with modern architectures,” in
2020
Closest in time.
J. Kahn, M. Rivière, W. Zheng, E. Kharitonov, Q. Xu, P. E. Mazaré, J. Karadayi, V. Liptchinsky, R. Collobert, C. Fuegen, T. Likhomanenko, G. Synnaeve, A. Joulin, A. Mohamed, and E. Dupoux, “Libri-light: A benchmark for ASR with limited or no supervision,” in
2020
Closest in time.
S. Indurthi, H. Han, N. K. Lakumarapu, B. Lee, I. Chung, S. Kim, and C. Kim, “End-end speech-to-text translation with modality agnostic meta-learning,” in
2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…