Fetching the paper…
Reading the bibliography…
Targeted syntactic evaluations have demonstrated the ability of language models to perform subject-verb agreement given difficult contexts.
Assessing BERT’s syntactic abilities
Yoav Goldberg. 2019 · 1901
Earlier work this paper cites.
DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2020 · 1910
Earlier work this paper cites.
Do attention heads in BERT track syntactic dependencies?
Phu Mon Htut, Jason Phang, Shikha Bordia, and Samuel R. Bowman. 2019 · 1911
Earlier work this paper cites.
Identifiability and exchangeability for direct and indirect effects
James M. Robins and Sander Greenland. 1992 · 1992
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber. 1997 · 1997
Earlier work this paper cites.
Direct and indirect effects
Judea Pearl. 2001 · 2001
Earlier work this paper cites.
Semantics of causal DAG models and the identification of direct and indirect effects
James M. Robins. 2003 · 2003
Earlier work this paper cites.
Causal mediation analysis for interpreting neural NLP: the case of gender bias
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart M. Shieber. 2020b · 2004
Earlier work this paper cites.
Argument-head distance and processing complexity: Explaining both locality and antilocality effects
Shravan Vasishth and Richard L Lewis. 2006 · 2006
Earlier work this paper cites.
LSTM neural networks for language modeling
Martin Sundermeyer, Ralf Schlüter, and Hermann Ney. 2012 · 2012
Earlier work this paper cites.
Assessing the ability of LSTMs to learn syntax-sensitive dependencies
Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016 · 2016
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Earlier work this paper cites.
On Internal Language Representations in Deep Learning: An Analysis of Machine Translation and Speech Recognition
Yonatan Belinkov. 2018 · 2018
Earlier work this paper cites.
Under the hood: Using diagnostic classifiers to investigate and improve how language models track agreement information
Mario Giulianelli, Jack Harding, Florian Mohnert, Dieuwke Hupkes, and Willem Zuidema. 2018 · 2018
Cited alongside, same era.
Colorless green recurrent networks dream hierarchically
Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018 · 2018
Cited alongside, same era.
Targeted syntactic evaluation of language models
Rebecca Marvin and Tal Linzen. 2018 · 2018
Cited alongside, same era.
GLUE: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018 · 2018
Cited alongside, same era.
What do RNN language models learn about filler–gap dependencies?
Ethan Wilcox, Roger Levy, Takashi Morita, and Richard Futrell. 2018 · 2018
Cited alongside, same era.
Linguistic knowledge and transferability of contextual representations
Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. 2019 · 2019
Later among the works it cites.
Language models are unsupervised multitask learners
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019
Later among the works it cites.
Quantity doesn’t buy quality syntax with neural language models
Marten van Schijndel, Aaron Mueller, and Tal Linzen. 2019 · 2019
Later among the works it cites.
Structural supervision improves learning of non-local grammatical dependencies
Ethan Wilcox, Peng Qian, Richard Futrell, Miguel Ballesteros, and Roger Levy. 2019 · 2019
Later among the works it cites.
XLNet: Generalized autoregressive pretraining for language understanding
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019 · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Yonatan Belinkov and James Glass. 2019 · 2019
Cited alongside, same era.
Transformer-XL: Attentive language models beyond a fixed-length context
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc Le, and Ruslan Salakhutdinov. 2019 · 2019
Cited alongside, same era.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
Neural language models as psycholinguistic subjects: Representations of syntactic state
Richard Futrell, Ethan Wilcox, Takashi Morita, Peng Qian, Miguel Ballesteros, and Roger Levy. 2019 · 2019
Cited alongside, same era.
Designing and interpreting probes with control tasks
John Hewitt and Percy Liang. 2019 · 2019
Cited alongside, same era.
A structural probe for finding syntax in word representations
John Hewitt and Christopher D. Manning. 2019 · 2019
Cited alongside, same era.
Analysing neural language models: Contextual decomposition reveals default reasoning in number and gender assignment
Jaap Jumelet, Willem Zuidema, and Dieuwke Hupkes. 2019 · 2019
Cited alongside, same era.
Finding universal grammatical relations in multilingual BERT
Ethan A. Chi, John Hewitt, and Christopher D. Manning. 2020 · 2020
Later among the works it cites.
A systematic assessment of syntactic generalization in neural language models
Jennifer Hu, Jon Gauthier, Peng Qian, Ethan Wilcox, and Roger Levy. 2020 · 2020
Later among the works it cites.
Influence paths for characterizing subject-verb number agreement in LSTM language models
Kaiji Lu, Piotr Mardziel, Klas Leino, Matt Fredrikson, and Anupam Datta. 2020 · 2020
Later among the works it cites.
Cross-linguistic syntactic evaluation of word prediction models
Aaron Mueller, Garrett Nicolai, Panayiota Petrou-Zeniou, Natalia Talmina, and Tal Linzen. 2020 · 2020
Later among the works it cites.
Probing classifiers: Promises, shortcomings, and alternatives
Yonatan Belinkov. 2021 · 2021
Closest in time.
Mechanisms for handling nested dependencies in neural-network language models and humans
Yair Lakretz, Dieuwke Hupkes, Alessandra Vergallito, Marco Marelli, Marco Baroni, and Stanislas Dehaene. 2021 · 2021
Closest in time.
Refining targeted syntactic evaluation of language models
Benjamin Newman, Kai-Siang Ang, Julia Gong, and John Hewitt. 2021 · 2021
Closest in time.
Koustuv Sinha, Robin Jia, Dieuwke Hupkes, Joelle Pineau, Adina Williams, and Douwe Kiela. 2021 · 2021
Closest in time.