Fetching the paper…
Reading the bibliography…
Many pretrained multilingual models exhibit cross-lingual transfer ability, which is often attributed to a learned language-neutral representation during pretraining.
Understanding intermediate layers using linear classifier probes
Guillaume Alain and Yoshua Bengio. 2017 · 2017
Earlier work this paper cites.
Analysis methods in neural language processing: A survey
Yonatan Belinkov and James Glass. 2019 · 2019
Earlier work this paper cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Earlier work this paper cites.
How multilingual is multilingual BERT?
Telmo Pires, Eva Schlinger, and Dan Garrette. 2019 · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019 · 2019
Earlier work this paper cites.
What do you learn from context? probing for sentence structure in contextualized word representations
Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R. Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R. Bowman, Dipanjan Das, and Ellie Pavlick. 2019 · 2019
Earlier work this paper cites.
Beto, bentz, becas: The surprising cross-lingual effectiveness of BERT
Shijie Wu and Mark Dredze. 2019 · 2019
Cited alongside, same era.
Identifying elements essential for BERT’s multilinguality
Philipp Dufter and Hinrich Schütze. 2020 · 2020
Cited alongside, same era.
Cross-lingual ability of multilingual bert: An empirical study
Karthikeyan K, Zihan Wang, Stephen Mayhew, and Dan Roth. 2020 · 2020
Cited alongside, same era.
On the language neutrality of pre-trained multilingual representations
Jindřich Libovický, Rudolf Rosa, and Alexander Fraser. 2020 · 2020
Cited alongside, same era.
English intermediate-task training improves zero-shot cross-lingual transfer too
Jason Phang, Iacer Calixto, Phu Mon Htut, Yada Pruksachatkun, Haokun Liu, Clara Vania, Katharina Kann, and Samuel R. Bowman. 2020 · 2020
Cited alongside, same era.
Wikipedia entities as rendezvous across languages: Grounding multilingual language models by predicting Wikipedia hyperlinks
mT5: A massively multilingual pre-trained text-to-text transformer
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021 · 2021
Later among the works it cites.
The geometry of multilingual language model representations
Tyler Chang, Zhuowen Tu, and Benjamin Bergen. 2022 · 2022
Later among the works it cites.
When is BERT multilingual? isolating crucial ingredients for cross-lingual transfer
Ameet Deshpande, Partha Talukdar, and Karthik Narasimhan. 2022 · 2022
Later among the works it cites.
Emergent world representations: Exploring a sequence model trained on a synthetic task
Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2023 · 2023
Later among the works it cites.
Why does zero-shot cross-lingual generation fail? an explanation and a solution
Tianjian Li and Kenton Murray. 2023 · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Iacer Calixto, Alessandro Raganato, and Tommaso Pasini. 2021 · 2021
Cited alongside, same era.
Unsupervised cross-lingual representation learning at scale
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020a
Cited in the paper.
Emerging cross-lingual structure in pretrained language models
Alexis Conneau, Shijie Wu, Haoran Li, Luke Zettlemoyer, and Veselin Stoyanov. 2020b
Cited in the paper.