Fetching the paper…
Reading the bibliography…
This paper demonstrates that multilingual pretraining and multilingual fine-tuning are both critical for facilitating cross-lingual transfer in zero-shot translation, where the neural machine translation (NMT) model is tested on source languages unseen during supervised training.
The missing ingredient in zero-shot neural machine translation
N. Arivazhagan, Ankur Bapna, Orhan Firat, Roee Aharoni, Melvin Johnson, and Wolfgang Macherey. 2019 · 1903
Earlier work this paper cites.
A teacher-student framework for zero-resource neural machine translation
Yun Chen, Yang Liu, Yong Cheng, and Victor O.K. Li. 2017 · 1935
Earlier work this paper cites.
Unsupervised neural machine translation with indirect supervision
Hongxiao Bai, Mingxuan Wang, Hai Zhao, and Lei Li. 2020 · 2004
Earlier work this paper cites.
Multilingual translation with extensible multilingual pretraining and finetuning
Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020 · 2008
Earlier work this paper cites.
Beyond English-Centric multilingual machine translation
Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Edouard Grave, Michael Auli, and Armand Joulin. 2020 · 2010
Earlier work this paper cites.
Sequence to sequence learning with neural networks
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014 · 2014
Earlier work this paper cites.
Neural machine translation by jointly learning to align and translate
Dzmitry Bahdanau, KyungHyun Cho, and Yoshua Bengio. 2015 · 2015
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P. Kingma and Jimmy Ba. 2015 · 2015
Earlier work this paper cites.
Toward multilingual neural machine translation with universal encoder and decoder
Thanh-Le Ha, Jan Niehues, and Alexander Waibel. 2016 · 2016
Earlier work this paper cites.
Google’s multilingual neural machine translation system: Enabling zero-shot translation
Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, et al. 2017 · 2017
Earlier work this paper cites.
Six challenges for neural machine translation
Philipp Koehn and Rebecca Knowles. 2017 · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Earlier work this paper cites.
Unsupervised neural machine translation
Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018 · 2018
Earlier work this paper cites.
Subword regularization: Improving neural network translation models with multiple subword candidates
Taku Kudo. 2018 · 2018
Earlier work this paper cites.
A neural interlingua for multilingual machine translation
Yichao Lu, Phillip Keung, Faisal Ladhak, Vikas Bhardwaj, Shaonan Zhang, and Jason Sun. 2018 · 2018
Earlier work this paper cites.
Zero-shot cross-lingual neural headline generation
Shi-qi Shen, Yun Chen, Cheng Yang, Zhi-yuan Liu, Mao-song Sun, et al. 2018 · 2018
Earlier work this paper cites.
Consistency by agreement in zero-shot neural machine translation
Maruan Al-Shedivat and Ankur P. Parikh. 2019 · 2019
Cited alongside, same era.
Cross-lingual language model pretraining
Alexis Conneau and Guillaume Lample. 2019 · 2019
Cited alongside, same era.
Zero-shot cross-lingual abstractive sentence summarization through teaching generation and attention
Xiangyu Duan, Mingming Yin, Min Zhang, Boxing Chen, and Weihua Luo. 2019 · 2019
Cited alongside, same era.
Improved zero-shot neural machine translation via ignoring spurious correlations
Jiatao Gu, Yong Wang, Kyunghyun Cho, and Victor O. K. Li. 2019 · 2019
Cited alongside, same era.
The FLoRes evaluation datasets for low-resource machine translation: Nepali–English and Sinhala–English
Francisco Guzmán, Peng-Jen Chen, Myle Ott, Juan Pino, Guillaume Lample, Philipp Koehn, Vishrav Chaudhary, and Marc’Aurelio Ranzato. 2019 · 2019
Cited alongside, same era.
Multilingual denoising pre-training for neural machine translation
Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020 · 2020
Later among the works it cites.
When does unsupervised machine translation work?
Kelly Marchisio, Kevin Duh, and Philipp Koehn. 2020 · 2020
Later among the works it cites.
English intermediate-task training improves zero-shot cross-lingual transfer too
Jason Phang, Iacer Calixto, Phu Mon Htut, Yada Pruksachatkun, Haokun Liu, Clara Vania, Katharina Kann, and Samuel R. Bowman. 2020 · 2020
Later among the works it cites.
Intermediate-task transfer learning with pretrained language models: When and why does it work?
Yada Pruksachatkun, Jason Phang, Haokun Liu, Phu Mon Htut, Xiaoyi Zhang, Richard Yuanzhe Pang, Clara Vania, Katharina Kann, and Samuel Bowman. 2020 · 2020
Later among the works it cites.
Cross-lingual retrieval for iterative self-supervised training
Chau Tran, Yuqing Tang, Xian Li, and Jiatao Gu. 2020 · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
The FLORES evaluation datasets for low-resource machine translation: Nepali–English and Sinhala–English
Francisco Guzmán, Peng-Jen Chen, Myle Ott, Juan Pino, Guillaume Lample, Philipp Koehn, Vishrav Chaudhary, and Marc’Aurelio Ranzato. 2019 · 2019
Cited alongside, same era.
fairseq: A fast, extensible toolkit for sequence modeling
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019 · 2019
Cited alongside, same era.
Improving zero-shot translation with language-independent constraints
Ngoc-Quan Pham, Jan Niehues, Thanh-Le Ha, and Alexander H. Waibel. 2019 · 2019
Cited alongside, same era.
Mass: Masked sequence to sequence pre-training for language generation
Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2019 · 2019
Cited alongside, same era.
Beto, bentz, becas: The surprising cross-lingual effectiveness of bert
Shijie Wu and Mark Dredze. 2019 · 2019
Cited alongside, same era.
Unsupervised cross-lingual representation learning at scale
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020 · 2020
Cited alongside, same era.
A multilingual view of unsupervised machine translation
Xavier García, Pierre Foret, Thibault Sellam, and Ankur P. Parikh. 2020 · 2020
Cited alongside, same era.
CCNet: Extracting high quality monolingual datasets from web crawl data
Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, and Edouard Grave. 2020 · 2020
Later among the works it cites.
Improving massively multilingual neural machine translation and zero-shot translation
Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. 2020 · 2020
Later among the works it cites.
Zero-shot cross-lingual transfer of neural machine translation with multilingual pretrained encoders
Guanhua Chen, Shuming Ma, Yun Chen, Li Dong, Dongdong Zhang, Jia Pan, Wenping Wang, and Furu Wei. 2021 · 2021
Closest in time.
Harnessing multilinguality in unsupervised machine translation for rare languages
Xavier Garcia, Aditya Siddhant, Orhan Firat, and Ankur Parikh. 2021 · 2021
Closest in time.
The flores-101 evaluation benchmark for low-resource and multilingual machine translation
Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng-Jen Chen, Guillaume Wenzek, Da Ju, Sanjan Krishnan, Marc’Aurelio Ranzato, Francisco Guzmán, and Angela Fan. 2021 · 2021
Closest in time.
Adapting high-resource nmt models to translate low-resource related languages without parallel data
Wei-Jen Ko, Ahmed El-Kishky, Adithya Renduchintala, Vishrav Chaudhary, Naman Goyal, Francisco Guzmán, Pascale Fung, Philipp Koehn, and Mona Diab. 2021 · 2021
Closest in time.
Improving zero-shot translation by disentangling positional information
Danni Liu, Jan Niehues, James Cross, Francisco Guzmán, and Xian Li. 2021 · 2021
Closest in time.
Contrastive learning for many-to-many multilingual neural machine translation
Xiao Pan, Mingxuan Wang, Liwei Wu, and Lei Li. 2021 · 2021
Closest in time.
ProphetNet-X: Large-scale pre-training models for English, Chinese, multi-lingual, dialog, and code generation
Weizhen Qi, Yeyun Gong, Yu Yan, Can Xu, Bolun Yao, Bartuer Zhou, Biao Cheng, Daxin Jiang, Jiusheng Chen, Ruofei Zhang, Houqiang Li, and Nan Duan. 2021 · 2021
Closest in time.
On learning universal representations across languages
Xiangpeng Wei, Yue Hu, Rongxiang Weng, Luxi Xing, Heng Yu, and Weihua Luo. 2021 · 2021
Closest in time.
mT5: A massively multilingual pre-trained text-to-text transformer
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021 · 2021
Closest in time.