Fetching the paper…
Reading the bibliography…
There is increasing interest in assessing the linguistic knowledge encoded in neural representations.
HuggingFace’s Transformers: State-of-the-art Natural Language Processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019 · 1910
Earlier work this paper cites.
Information-Theoretic Probing with Minimum Description Length
Elena Voita and Ivan Titov. 2020 · 2003
Earlier work this paper cites.
Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies
Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016 · 2016
Earlier work this paper cites.
Fine-grained analysis of sentence embeddings using auxiliary prediction tasks
Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2017 · 2017
Earlier work this paper cites.
Understanding intermediate layers using linear classifier probes
Guillaume Alain and Yoshua Bengio. 2017 · 2017
Earlier work this paper cites.
What do Neural Machine Translation Models Learn about Morphology?
Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, and James Glass. 2017 · 2017
Earlier work this paper cites.
Contextual String Embeddings for Sequence Labeling
Alan Akbik, Duncan Blythe, and Roland Vollgraf. 2018 · 2018
Earlier work this paper cites.
Evaluation Benchmarks and Learning Criteria for Discourse-Aware Sentence Representations
Mingda Chen, Zewei Chu, and Kevin Gimpel. 2019 · 2019
Earlier work this paper cites.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
Designing and Interpreting Probes with Control Tasks
John Hewitt and Percy Liang. 2019 · 2019
Cited alongside, same era.
A Structural Probe for Finding Syntax in Word Representations
John Hewitt and Christopher D Manning. 2019 · 2019
Cited alongside, same era.
Open Sesame: Getting Inside BERT’s Linguistic Knowledge
Yongjie Lin, Yi Chern Tan, and Robert Frank. 2019 · 2019
Cited alongside, same era.
Linguistic Knowledge and Transferability of Contextual Representations
Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. 2019 · 2019
Cited alongside, same era.
Language Models as Knowledge Bases?
Probing for Semantic Classes: Diagnosing the Meaning Content of Word Embeddings
Yadollah Yaghoobzadeh, Katharina Kann, T J Hazen, Eneko Agirre, and Hinrich Schütze. 2019 · 2019
Later among the works it cites.
Universal dependencies 2.5
Daniel Zeman, Joakim Nivre, and Mitchell et al Abrams. 2019 · 2019
Later among the works it cites.
Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction
Christoph Alt, Aleksandra Gabryszak, and Leonhard Hennig. 2020 · 2020
Closest in time.
Finding Universal Grammatical Relations in Multilingual BERT
Ethan A Chi, John Hewitt, and Christopher D Manning. 2020 · 2020
Closest in time.
Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly
Nora Kassner and Hinrich Schütze. 2020 · 2020
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019 · 2019
Cited alongside, same era.
BERT is Not a Knowledge Base (Yet): Factual Knowledge vs. Name-Based Reasoning in Unsupervised QA
Nina Poerner, Ulli Waltinger, and Hinrich Schütze. 2019 · 2019
Cited alongside, same era.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019
Cited alongside, same era.
BERT Rediscovers the Classical NLP Pipeline
Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019a
Cited in the paper.
Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R. Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R. Bowman, Dipanjan Das, and Ellie Pavlick. 2019b
Cited in the paper.
Closest in time.
A Tale of a Probe and a Parser
Rowan Hall Maudslay, Josef Valvoda, Tiago Pimentel, Adina Williams, and Ryan Cotterell. 2020 · 2020
Closest in time.
Information-Theoretic Probing for Linguistic Structure
Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, and Ryan Cotterell. 2020 · 2020
Closest in time.