Fetching the paper…
Reading the bibliography…
We investigate how Multilingual BERT (mBERT) encodes grammar by examining how the high-order grammatical feature of morphosyntactic alignment (how different languages define what counts as a "subject") is manifested across the embedding spaces of different languages.
How language-neutral is multilingual bert?
Jindřich Libovickỳ, Rudolf Rosa, and Alexander Fraser. 2019 · 1911
Earlier work this paper cites.
Noun phrase accessibility and universal grammar
Edward L. Keenan and Bernard Comrie. 1977 · 1977
Earlier work this paper cites.
Ergativity
Robert MW Dixon. 1979 · 1979
Earlier work this paper cites.
Transitivity in grammar and discourse
Paul J Hopper and Sandra A Thompson. 1980 · 1980
Earlier work this paper cites.
Language Universals and Linguistic Typology , 1st edition
Bernard Comrie. 1981 · 1981
Earlier work this paper cites.
The discourse basis of ergativity
John W Du Bois. 1987 · 1987
Earlier work this paper cites.
The noun phrase accessibility hierarchy reinterpreted: Subject primacy or the absolutive hypothesis?
Barbara A Fox. 1987 · 1987
Earlier work this paper cites.
Linguistic typology
Bernard Comrie. 1988 · 1988
Earlier work this paper cites.
Radical construction grammar: Syntactic theory in typological perspective
William Croft. 2001 · 2001
Earlier work this paper cites.
Differential case-marking in hindi
Helen De Hoop and Bhuvana Narasimhan. 2005 · 2005
Earlier work this paper cites.
Assessing the ability of LSTMs to learn syntax-sensitive dependencies
Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016 · 2016
Earlier work this paper cites.
Universal dependencies v1: A multilingual treebank collection
Joakim Nivre, Marie-Catherine De Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, et al. 2016 · 2016
Cited alongside, same era.
Colorless green recurrent networks dream hierarchically
Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018 · 2018
Cited alongside, same era.
What do RNN language models learn about filler–gap dependencies?
Ethan Wilcox, Roger Levy, Takashi Morita, and Richard Futrell. 2018 · 2018
Cited alongside, same era.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
Neural language models as psycholinguistic subjects: Representations of syntactic state
Richard Futrell, Ethan Wilcox, Takashi Morita, Peng Qian, Miguel Ballesteros, and Roger Levy. 2019 · 2019
Cited alongside, same era.
How multilingual is multilingual BERT?
Telmo Pires, Eva Schlinger, and Dan Garrette. 2019 · 2019
Later among the works it cites.
Studying the inductive biases of RNNs with synthetic variations of natural languages
Shauli Ravfogel, Yoav Goldberg, and Tal Linzen. 2019 · 2019
Later among the works it cites.
Beto, bentz, becas: The surprising cross-lingual effectiveness of bert
Shijie Wu and Mark Dredze. 2019 · 2019
Later among the works it cites.
Finding universal grammatical relations in multilingual BERT
Ethan A. Chi, John Hewitt, and Christopher D. Manning. 2020 · 2020
Later among the works it cites.
Emergent linguistic structure in artificial neural networks trained by self-supervision
Christopher D Manning, Kevin Clark, John Hewitt, Urvashi Khandelwal, and Omer Levy. 2020 · 2020
Later among the works it cites.
Cross-linguistic syntactic evaluation of word prediction models
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Designing and interpreting probes with control tasks
John Hewitt and Percy Liang. 2019 · 2019
Cited alongside, same era.
A structural probe for finding syntax in word representations
John Hewitt and Christopher D. Manning. 2019 · 2019
Cited alongside, same era.
What does BERT learn about the structure of language?
Ganesh Jawahar, Benoît Sagot, and Djamé Seddah. 2019 · 2019
Cited alongside, same era.
Revealing the dark secrets of BERT
Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019 · 2019
Cited alongside, same era.
Linguistic knowledge and transferability of contextual representations
Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. 2019 · 2019
Cited alongside, same era.
Aaron Mueller, Garrett Nicolai, Panayiota Petrou-Zeniou, Natalia Talmina, and Tal Linzen. 2020 · 2020
Later among the works it cites.
Learning music helps you read: Using transfer to study linguistic structure in language models
Isabel Papadimitriou and Dan Jurafsky. 2020 · 2020
Later among the works it cites.
Information-theoretic probing for linguistic structure
Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, and Ryan Cotterell. 2020 · 2020
Later among the works it cites.
A primer in bertology: What we know about how bert works
Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020 · 2020
Later among the works it cites.
Information-theoretic probing with minimum description length
Elena Voita and Ivan Titov. 2020 · 2020
Later among the works it cites.