Fetching the paper…
Reading the bibliography…
There are two primary approaches to addressing cross-lingual transfer: multilingual pre-training, which implicitly aligns the hidden representations of various languages, and translate-test, which explicitly translates different languages into an intermediate language, such as English.
Roberta: A robustly optimized bert pretraining approach
Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019 · 1907
Earlier work this paper cites.
Cross-lingual name tagging and linking for 282 languages
Pan, X.; Zhang, B.; May, J.; Nothman, J.; Knight, K.; and Ji, H. 2017 · 1958
Earlier work this paper cites.
Electra: Pre-training text encoders as discriminators rather than generators
Clark, K.; Luong, M.-T.; Le, Q. V.; and Manning, C. D. 2020 · 2003
Earlier work this paper cites.
An evaluation exercise for word alignment
Mihalcea, R.; and Pedersen, T. 2003 · 2003
Earlier work this paper cites.
AER: Do we need to “improve” our alignments?
Vilar, D.; Popović, M.; and Ney, H. 2006 · 2006
Earlier work this paper cites.
Abstract meaning representation for sembanking
Banarescu, L.; Bonial, C.; Cai, S.; Georgescu, M.; Griffitt, K.; Hermjakob, U.; Knight, K.; Koehn, P.; Palmer, M.; and Schneider, N. 2013 · 2013
Earlier work this paper cites.
Not an Interlingua, But Close: Comparison of English AMRs to Chinese and Czech
Xue, N.; Bojar, O.; Hajič, J.; Palmer, M.; Urešová, Z.; and Zhang, X. 2014 · 2014
Earlier work this paper cites.
Contrastive unsupervised word alignment with non-local features
Liu, Y.; and Sun, M. 2015 · 2015
Earlier work this paper cites.
Neural Discrete Representation Learning
van den Oord, A.; Vinyals, O.; and Kavukcuoglu, K. 2017 · 2017
Earlier work this paper cites.
XNLI: Evaluating Cross-lingual Sentence Representations
Conneau, A.; Rinott, R.; Lample, G.; Williams, A.; Bowman, S.; Schwenk, H.; and Stoyanov, V. 2018 · 2018
Earlier work this paper cites.
Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
Artetxe, M.; and Schwenk, H. 2019 · 2019
Earlier work this paper cites.
Cross-lingual language model pretraining
Conneau, A.; and Lample, G. 2019 · 2019
Cited alongside, same era.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019a · 2019
Cited alongside, same era.
Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks
Huang, H.; Liang, Y.; Duan, N.; Gong, M.; Shou, L.; Jiang, D.; and Zhou, M. 2019 · 2019
Cited alongside, same era.
wav2vec 2.0: A framework for self-supervised learning of speech representations
Baevski, A.; Zhou, Y.; Mohamed, A.; and Auli, M. 2020 · 2020
Cited alongside, same era.
Unsupervised Cross-lingual Representation Learning at Scale
Conneau, A.; Khandelwal, K.; Goyal, N.; Chaudhary, V.; Wenzek, G.; Guzmán, F.; Grave, É.; Ott, M.; Zettlemoyer, L.; and Stoyanov, V. 2020 · 2020
Cited alongside, same era.
MLQA: Evaluating Cross-lingual Extractive Question Answering
InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
Chi, Z.; Dong, L.; Wei, F.; Yang, N.; Singhal, S.; Wang, W.; Song, X.; Mao, X.-L.; Huang, H.-Y.; and Zhou, M. 2021 · 2021
Later among the works it cites.
Word Alignment by Fine-tuning Embeddings on Parallel Corpora
Dou, Z.; and Neubig, G. 2021 · 2021
Later among the works it cites.
Taming transformers for high-resolution image synthesis
Esser, P.; Rombach, R.; and Ommer, B. 2021 · 2021
Later among the works it cites.
Analysis and Evaluation of Language Models for Word Sense Disambiguation
Loureiro, D.; Rezaee, K.; Pilehvar, M. T.; and Camacho-Collados, J. 2021 · 2021
Later among the works it cites.
Zero-shot text-to-image generation
Ramesh, A.; Pavlov, M.; Goh, G.; Gray, S.; Voss, C.; Radford, A.; Chen, M.; and Sutskever, I. 2021 · 2021
Later among the works it cites.
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Lewis, P.; Oğuz, B.; Rinott, R.; Riedel, S.; and Schwenk, H. 2020 · 2020
Cited alongside, same era.
XGLUE: A new benchmark datasetfor cross-lingual pre-training, understanding and generation
Liang, Y.; Duan, N.; Gong, Y.; Wu, N.; Guo, F.; Qi, W.; Gong, M.; Shou, L.; Jiang, D.; Cao, G.; et al. 2020 · 2020
Cited alongside, same era.
Exploring the limits of transfer learning with a unified text-to-text transformer
Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; Liu, P. J.; et al. 2020 · 2020
Cited alongside, same era.
CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data
Wenzek, G.; Lachaux, M.-A.; Conneau, A.; Chaudhary, V.; Guzmán, F.; Joulin, A.; and Grave, É. 2020 · 2020
Cited alongside, same era.
Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation
Zhang, B.; Williams, P.; Titov, I.; and Sennrich, R. 2020 · 2020
Cited alongside, same era.
Multilingual Bert
Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019b
Cited in the paper.
Xue, L.; Constant, N.; Roberts, A.; Kale, M.; Al-Rfou, R.; Siddhant, A.; Barua, A.; and Raffel, C. 2021 · 2021
Later among the works it cites.
Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structure
Chai, Y.; Liang, Y.; and Duan, N. 2022 · 2022
Later among the works it cites.
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA
Chi, Z.; Huang, S.; Dong, L.; Ma, S.; Zheng, B.; Singhal, S.; Bajaj, P.; Song, X.; Mao, X.-L.; Huang, H.-Y.; et al. 2022 · 2022
Later among the works it cites.
XLM-K: Improving Cross-Lingual Language Model Pre-training with Multilingual Knowledge
Jiang, X.; Liang, Y.; Chen, W.; and Duan, N. 2022 · 2022
Later among the works it cites.
Nüwa: Visual synthesis pre-training for neural visual world creation
Wu, C.; Liang, J.; Ji, L.; Yang, F.; Fang, Y.; Jiang, D.; and Duan, N. 2022 · 2022
Later among the works it cites.