2019

Massively Multilingual Neural Machine Translation

Aharoni, Roee, Johnson, Melvin, Firat, Orhan

Understand

Multilingual neural machine translation (NMT) enables training a single model that supports translation from multiple source languages into multiple target languages.

In this paper, we push the limits of multilingual NMT in terms of number of languages being used.
We perform extensive experiments in training massively multilingual NMT models, translating up to 102 languages to and from English within a single model.
We explore different setups for training such models and analyze the trade-offs between translation quality and various modeling decisions.

Built on

Bleu: a method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002
Earlier work this paper cites.
Recurrent continuous translation models
Nal Kalchbrenner and Phil Blunsom. 2013 · 2013
Earlier work this paper cites.
Neural machine translation by jointly learning to align and translate
Original
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014 · 2014
Earlier work this paper cites.
Sequence to sequence learning with neural networks
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014 · 2014
Earlier work this paper cites.
Multi-task learning for multiple language translation
Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. 2015 · 2015
Earlier work this paper cites.
Findings of the 2016 conference on machine translation
Ondřej Bojar, Rajen Chatterjee, Christian Federmann, et al. 2016 · 2016
Earlier work this paper cites.
Multi-way, multilingual neural machine translation with a shared attention mechanism
Orhan Firat, Kyunghyun Cho, and Yoshua Bengio. 2016a · 2016
Earlier work this paper cites.
Zero-resource translation with multi-lingual neural machine translation
Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan, Fatos T Yarman Vural, and Kyunghyun Cho. 2016b · 2016
Earlier work this paper cites.
Toward multilingual neural machine translation with universal encoder and decoder
Original
Thanh-Le Ha, Jan Niehues, and Alexander Waibel. 2016 · 2016
Earlier work this paper cites.
Neural machine translation of rare words with subword units
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016 · 2016
Earlier work this paper cites.
Google’s neural machine translation system: Bridging the gap between human and machine translation
Original
Yonghui Wu, Mike Schuster, Zhifeng Chen, et al. 2016 · 2016
Earlier work this paper cites.
Transfer learning for low-resource neural machine translation
Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016 · 2016
Earlier work this paper cites.
Findings of the 2017 conference on machine translation (wmt17)
Ondřej Bojar, Rajen Chatterjee, Christian Federmann, et al. 2017 · 2017
Earlier work this paper cites.

Similar

Overview of the iwslt 2017 evaluation campaign
Mauro Cettolo, Federico Marcello, Bentivogli Luisa, Niehues Jan, Stüker Sebastian, Sudoh Katsuitho, Yoshino Koichiro, and Federmann Christian. 2017 · 2017
Cited alongside, same era.
Google’s multilingual neural machine translation system: Enabling zero-shot translation
Melvin Johnson, Mike Schuster, Quoc V Le, et al. 2017 · 2017
Cited alongside, same era.
Learning language representations for typology prediction
Chaitanya Malaviya, Graham Neubig, and Patrick Littell. 2017 · 2017
Cited alongside, same era.
Learned in translation: Contextualized word vectors
Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017 · 2017
Cited alongside, same era.
Transfer learning across low-resource, related languages for neural machine translation
Toan Q. Nguyen and David Chiang. 2017 · 2017
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Cited alongside, same era.
Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond
Original
Mikel Artetxe and Holger Schwenk. 2018 · 2018
Cited alongside, same era.
Multilingual neural machine translation with task-specific attention
Graeme Blackwood, Miguel Ballesteros, and Todd Ward. 2018 · 2018
Cited alongside, same era.
Findings of the 2018 conference on machine translation (wmt18)
Ondřej Bojar, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Philipp Koehn, and Christof Monz. 2018 · 2018
Cited alongside, same era.
Zero-shot cross-lingual classification using multilingual neural machine translation
Original
Akiko Eriguchi, Melvin Johnson, Orhan Firat, Hideto Kazawa, and Wolfgang Macherey. 2018 · 2018
Cited alongside, same era.
Universal neural machine translation for extremely low resource languages
Jiatao Gu, Hany Hassan, Jacob Devlin, and Victor O.K. Li. 2018 · 2018
Cited alongside, same era.
Achieving human parity on automatic chinese to english news translation
Original
Hany Hassan, Anthony Aue, Chang Chen, et al. 2018 · 2018
Cited alongside, same era.

Then

A comparison of transformer and recurrent neural networks on multilingual neural machine translation
Surafel Melaku Lakew, Mauro Cettolo, and Marcello Federico. 2018 · 2018
Later among the works it cites.
A neural interlingua for multilingual machine translation
Yichao Lu, Phillip Keung, Faisal Ladhak, Vikas Bhardwaj, Shaonan Zhang, and Jason Sun. 2018 · 2018
Later among the works it cites.
Rapid adaptation of neural machine translation to new languages
Graham Neubig and Junjie Hu. 2018 · 2018
Later among the works it cites.
Scaling neural machine translation
Myle Ott, Sergey Edunov, David Grangier, and Michael Auli. 2018 · 2018
Later among the works it cites.
Contextual parameter generation for universal neural machine translation
Emmanouil Antonios Platanios, Mrinmaya Sachan, Graham Neubig, and Tom Mitchell. 2018 · 2018
Later among the works it cites.
When and why are pre-trained word embeddings useful for neural machine translation
Ye Qi, Sachan Devendra, Felix Matthieu, Padmanabhan Sarguna, and Neubig Graham. 2018 · 2018
Later among the works it cites.
Parameter sharing methods for multilingual self-attentional translation models
Devendra Sachan and Graham Neubig. 2018 · 2018
Later among the works it cites.
Emerging language spaces learned from massively multilingual corpora
Original
Jörg Tiedemann. 2018 · 2018
Later among the works it cites.
Three strategies to improve one-to-many multilingual translation
Yining Wang, Jiajun Zhang, Feifei Zhai, Jingfang Xu, and Chengqing Zong. 2018 · 2018
Later among the works it cites.
Adaptive knowledge sharing in multi-task learning: Improving low-resource neural machine translation
Poorya Zaremoodi, Wray Buntine, and Gholamreza Haffari. 2018 · 2018
Later among the works it cites.
Lingvo: a modular and scalable framework for sequence-to-sequence modeling
Original
Jonathan Shen, Patrick Nguyen, Yonghui Wu, Zhifeng Chen, et al. 2019 · 2019
Closest in time.
Multilingual neural machine translation with soft decoupled encoding
Xinyi Wang, Hieu Pham, Philip Arthur, and Graham Neubig. 2019 · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…