Fetching the paper…
Reading the bibliography…
We show that the choice of pretraining languages affects downstream cross-lingual transfer for BERT-based models.
Orthography to phonology and meaning: Comparisons across and within writing systems
Charles A Perfetti and Ying Liu. 2005 · 2005
Earlier work this paper cites.
WALS Online
Matthew S. Dryer and Martin Haspelmath, editors. 2013 · 2013
Earlier work this paper cites.
The Stanford CoreNLP natural language processing toolkit
Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014 · 2014
Earlier work this paper cites.
Wikiextractor
Giusepppe Attardi. 2015 · 2015
Earlier work this paper cites.
Google’s neural machine translation system: Bridging the gap between human and machine translation
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016 · 2016
Earlier work this paper cites.
spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing
Matthew Honnibal and Ines Montani. 2017 · 2017
Earlier work this paper cites.
Deep contextualized word representations
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018 · 2018
Earlier work this paper cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
How multilingual is multilingual BERT?
Telmo Pires, Eva Schlinger, and Dan Garrette. 2019 · 2019
Cited alongside, same era.
Massively multilingual transfer for NER
Afshin Rahimi, Yuan Li, and Trevor Cohn. 2019 · 2019
Cited alongside, same era.
XTREME: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation
Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020 · 2020
Cited alongside, same era.
The state and fate of linguistic diversity and inclusion in the NLP world
Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020 · 2020
Cited alongside, same era.
Cross-lingual ability of multilingual BERT: an empirical study
Are all languages created equal in multilingual BERT?
Shijie Wu and Mark Dredze. 2020 · 2020
Later among the works it cites.
Filling the gaps in Ancient Akkadian texts: A masked language modelling approach
Koren Lazar, Benny Saret, Asaf Yehudai, Wayne Horowitz, Nathan Wasserman, and Gabriel Stanovsky. 2021 · 2021
Later among the works it cites.
Examining cross-lingual contextual embeddings with orthogonal structural probes
Tomasz Limisiewicz and David Mareček. 2021 · 2021
Later among the works it cites.
Revisiting the primacy of english in zero-shot cross-lingual transfer
Iulia Turc, Kenton Lee, Jacob Eisenstein, Ming-Wei Chang, and Kristina Toutanova. 2021 · 2021
Later among the works it cites.
Language contamination explains the cross-lingual capabilities of english pretrained models
Terra Blevins and Luke Zettlemoyer. 2022 · 2022
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Karthikeyan K, Zihan Wang, Stephen Mayhew, and Dan Roth. 2020 · 2020
Cited alongside, same era.
Universal Dependencies v2: An evergrowing multilingual treebank collection
Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. 2020 · 2020
Cited alongside, same era.
Unsupervised cross-lingual representation learning at scale
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020a
Cited in the paper.
Emerging cross-lingual structure in pretrained language models
Alexis Conneau, Shijie Wu, Haoran Li, Luke Zettlemoyer, and Veselin Stoyanov. 2020b
Cited in the paper.
Oolong: Investigating what makes crosslingual transfer hard with controlled studies
Zhengxuan Wu, Isabel Papadimitriou, and Alex Tamkin. 2022 · 2022
Closest in time.