2020

Complete Multilingual Neural Machine Translation

Freitag, Markus, Firat, Orhan

Understand

Multilingual Neural Machine Translation (MNMT) models are commonly trained on a joint set of bilingual corpora which is acutely English-centric (i.e.

English either as the source or target language).
While direct data between two languages that are non-English is explicitly available at times, its use is not common.
In this paper, we first take a step back and look at the commonly used bilingual corpora (WMT), and resurface the existence and importance of implicit structure that existed in it: multi-way alignment across examples (the same sentence in more than two languages).

Built on

The missing ingredient in zero-shot neural machine translation
Original
Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Roee Aharoni, Melvin Johnson, and Wolfgang Macherey. 2019a · 1903
Earlier work this paper cites.
Improved zero-shot neural machine translation via ignoring spurious correlations
Original
Jiatao Gu, Yong Wang, Kyunghyun Cho, and Victor O. K. Li. 2019 · 1906
Earlier work this paper cites.
Massively multilingual neural machine translation in the wild: Findings and challenges
Original
Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Dmitry Lepikhin, Melvin Johnson, Maxim Krikun, Mia Xu Chen, Yuan Cao, George Foster, Colin Cherry, Wolfgang Macherey, Zhifeng Chen, and Yonghui Wu. 2019b · 1907
Earlier work this paper cites.
A Teacher-Student Framework for Zero-Resource Neural Machine Translation
Yun Chen, Yang Liu, Yong Cheng, and Victor OK Li. 2017 · 1935
Earlier work this paper cites.
BLEU: a method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002
Earlier work this paper cites.
Machine Translation by Triangulation: Making Effective Use of Multi-Parallel Corpora
Trevor Cohn and Mirella Lapata. 2007 · 2007
Earlier work this paper cites.
A comparison of pivot methods for phrase-based statistical machine translation
Masao Utiyama and Hitoshi Isahara. 2007 · 2007
Earlier work this paper cites.
Pivot language approach for phrase-based statistical machine translation
Hua Wu and Haifeng Wang. 2007 · 2007
Earlier work this paper cites.
Improving Arabic-Chinese statistical machine translation using English as pivot language
Nizar Habash and Jun Hu. 2009 · 2009
Earlier work this paper cites.
Findings of the 2013 Workshop on Statistical Machine Translation
Ondřej Bojar, Christian Buck, Chris Callison-Burch, Christian Federmann, Barry Haddow, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2013 · 2013
Earlier work this paper cites.
Multi-task learning for multiple language translation
Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. 2015 · 2015
Earlier work this paper cites.
Neural machine translation with pivot languages
Original
Yong Cheng, Yang Liu, Qian Yang, Maosong Sun, and Wei Xu. 2016 · 2016
Earlier work this paper cites.

Similar

Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism
Orhan Firat, Kyunghyun Cho, and Yoshua Bengio. 2016a · 2016
Cited alongside, same era.
Zero-resource translation with multi-lingual neural machine translation
Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan, Fatos T Yarman Vural, and Kyunghyun Cho. 2016b · 2016
Cited alongside, same era.
Fast domain adaptation for neural machine translation
Original
Markus Freitag and Yaser Al-Onaizan. 2016 · 2016
Cited alongside, same era.
Toward multilingual neural machine translation with universal encoder and decoder
Original
Thanh-Le Ha, Jan Niehues, and Alexander Waibel. 2016 · 2016
Cited alongside, same era.
Fully character-level neural machine translation without explicit segmentation
Original
Jason Lee, Kyunghyun Cho, and Thomas Hofmann. 2016 · 2016
Cited alongside, same era.
Multi-source neural translation
Barret Zoph and Kevin Knight. 2016 · 2016
Cited alongside, same era.
Google’s multilingual neural machine translation system: Enabling zero-shot translation
Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, et al. 2017 · 2017
Cited alongside, same era.
Attention Is All You Need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Cited alongside, same era.
BERT: pre-training of deep bidirectional transformers for language understanding
Original
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018
Cited alongside, same era.
Iterative back-translation for neural machine translation
Vu Cong Duy Hoang, Philipp Koehn, Gholamreza Haffari, and Trevor Cohn. 2018 · 2018
Cited alongside, same era.
Scheduled multi-task learning: From syntax to translation
Eliyahu Kiperwasser and Miguel Ballesteros. 2018 · 2018
Cited alongside, same era.

Then

Multi-source neural machine translation with missing data
Yuta Nishimura, Katsuhito Sudoh, Graham Neubig, and Satoshi Nakamura. 2018 · 2018
Later among the works it cites.
A Call for Clarity in Reporting Bleu Scores
Matt Post. 2018 · 2018
Later among the works it cites.
Emerging language spaces learned from massively multilingual corpora
Original
Jörg Tiedemann. 2018 · 2018
Later among the works it cites.
Massively Multilingual Neural Machine Translation
Roee Aharoni, Melvin Johnson, and Orhan Firat. 2019 · 2019
Later among the works it cites.
Consistency by agreement in zero-shot neural machine translation
Maruan Al-Shedivat and Ankur Parikh. 2019 · 2019
Later among the works it cites.
Zero-resource neural machine translation with monolingual pivot data
Anna Currey and Kenneth Heafield. 2019 · 2019
Later among the works it cites.
Exploiting multilingualism through multistage fine-tuning for low-resource neural machine translation
Raj Dabre, Atsushi Fujita, and Chenhui Chu. 2019 · 2019
Later among the works it cites.
Adaptive scheduling for multi-task learning
Original
Sébastien Jean, Orhan Firat, and Melvin Johnson. 2019 · 2019
Later among the works it cites.
Opustools and parallel corpus diagnostics
Mikko Aulamo, Umut Sulubacak, Sami Virpioja, and Jörg Tiedemann. 2020 · 2020
Closest in time.
Balancing training for multilingual neural machine translation
Original
Xinyi Wang, Yulia Tsvetkov, and Graham Neubig. 2020 · 2020
Closest in time.
Improving massively multilingual neural machine translation and zero-shot translation
Original
Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. 2020 · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…