Fetching the paper…
Reading the bibliography…
We present minicons, an open source library that provides a standard API for researchers interested in conducting behavioral and representational analyses of transformer-based language models (LMs).
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1907
Earlier work this paper cites.
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019 · 1910
Earlier work this paper cites.
CrowS-pairs: A challenge dataset for measuring social biases in masked language models
Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020 · 1967
Earlier work this paper cites.
Collected Papers of Charles Sanders Peirce , volume 5
Charles Sanders Peirce. 1974 · 1974
Earlier work this paper cites.
Natural language grammatical inference with recurrent neural networks
Steve Lawrence, C Lee Giles, and Sandiway Fong. 2000 · 2000
Earlier work this paper cites.
English gigaword
David Graff, Junbo Kong, Ke Chen, and Kazuaki Maeda. 2003 · 2003
Earlier work this paper cites.
Representational similarity analysis-connecting the branches of systems neuroscience
Nikolaus Kriegeskorte, Marieke Mur, and Peter A Bandettini. 2008 · 2008
Earlier work this paper cites.
Clueweb09 data set
Jamie Callan, Mark Hoy, Changkuk Yoo, and Le Zhao. 2009 · 2009
Earlier work this paper cites.
The effect of word predictability on reading time is logarithmic
Nathaniel J Smith and Roger Levy. 2013 · 2013
Earlier work this paper cites.
Aligning books and movies: Towards story-like visual explanations by watching movies and reading books
Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015 · 2015
Earlier work this paper cites.
Probing for semantic evidence of composition by means of simple classification tasks
Allyson Ettinger, Ahmed Elgohary, and Philip Resnik. 2016 · 2016
Earlier work this paper cites.
Assessing the ability of lstms to learn syntax-sensitive dependencies
Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016 · 2016
Earlier work this paper cites.
CC-News
Sebastian Nagel. 2016 · 2016
Earlier work this paper cites.
Grammaticality, acceptability, and probability: A probabilistic view of linguistic knowledge
Jey Han Lau, Alexander Clark, and Shalom Lappin. 2017 · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Earlier work this paper cites.
What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties
Alexis Conneau, Germán Kruszewski, Guillaume Lample, Loïc Barrault, and Marco Baroni. 2018 · 2018
Earlier work this paper cites.
Targeted syntactic evaluation of language models
Rebecca Marvin and Tal Linzen. 2018 · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018 · 2018
Earlier work this paper cites.
A simple method for commonsense reasoning
Trieu H Trinh and Quoc V Le. 2018 · 2018
Earlier work this paper cites.
Blackbox meets blackbox: Representational similarity & stability analysis of neural language models and brains
Samira Abnar, Lisa Beinborn, Rochelle Choenni, and Willem Zuidema. 2019 · 2019
Cited alongside, same era.
Analyzing and interpreting neural networks for nlp: A report on the first blackboxnlp workshop
Afra Alishahi, Grzegorz Chrupała, and Tal Linzen. 2019 · 2019
Cited alongside, same era.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
Neural language models as psycholinguistic subjects: Representations of syntactic state
Richard Futrell, Ethan Wilcox, Takashi Morita, Peng Qian, Miguel Ballesteros, and Roger Levy. 2019 · 2019
Cited alongside, same era.
Openwebtext corpus
Aaron Gokaslan and Vanya Cohen. 2019 · 2019
Cited alongside, same era.
Exploring BERT’s sensitivity to lexical cues using tests from semantic priming
Kanishka Misra, Allyson Ettinger, and Julia Rayz. 2020 · 2020
Later among the works it cites.
Cross-linguistic syntactic evaluation of word prediction models
Aaron Mueller, Garrett Nicolai, Panayiota Petrou-Zeniou, Natalia Talmina, and Tal Linzen. 2020 · 2020
Later among the works it cites.
Contextualized word embeddings encode aspects of human-like word sense knowledge
Sathvik Nair, Mahesh Srinivasan, and Stephan Meylan. 2020 · 2020
Later among the works it cites.
jiant: A software toolkit for research on general-purpose text understanding models
Yada Pruksachatkun, Phil Yeres, Haokun Liu, Jason Phang, Phu Mon Htut, Alex Wang, Ian Tenney, and Samuel R. Bowman. 2020 · 2020
Later among the works it cites.
Masked language model scoring
Julian Salazar, Davis Liang, Toan Q. Nguyen, and Katrin Kirchhoff. 2020 · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
John Hewitt and Christopher D. Manning. 2019 · 2019
Cited alongside, same era.
Albert: A lite bert for self-supervised learning of language representations
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019 · 2019
Cited alongside, same era.
Language modelling makes sense: Propagating representations through WordNet for full-coverage word sense disambiguation
Daniel Loureiro and Alípio Jorge. 2019 · 2019
Cited alongside, same era.
Pytorch: An imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019 · 2019
Cited alongside, same era.
Language models as knowledge bases?
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019 · 2019
Cited alongside, same era.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019
Cited alongside, same era.
Neural network acceptability judgments
Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2019 · 2019
Cited alongside, same era.
Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020 · 2020
Later among the works it cites.
BLiMP: The benchmark of linguistic minimal pairs for English
Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020 · 2020
Later among the works it cites.
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020 · 2020
Later among the works it cites.
Beyond the imitation game: Measuring and extrapolating the capabilities of language models
BIG-bench collaboration. 2021 · 2021
Later among the works it cites.
GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow
Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. 2021 · 2021
Later among the works it cites.
Surface form competition: Why the highest probability answer isn’t always right
Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. 2021 · 2021
Later among the works it cites.
Towards zero-shot commonsense reasoning with self-supervised refinement of language models
Tassilo Klein and Moin Nabi. 2021 · 2021
Later among the works it cites.
Human sentence processing: Recurrence or attention?
Danny Merkx and Stefan L. Frank. 2021 · 2021
Later among the works it cites.
Do language models learn typicality judgments from text?
Kanishka Misra, Allyson Ettinger, and Julia Rayz. 2021 · 2021
Later among the works it cites.
Refining targeted syntactic evaluation of language models
Benjamin Newman, Kai-Siang Ang, Julia Gong, and John Hewitt. 2021 · 2021
Later among the works it cites.
GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model
Ben Wang and Aran Komatsuzaki. 2021 · 2021
Later among the works it cites.
When do you need billions of words of pretraining data?
Yian Zhang, Alex Warstadt, Xiaocheng Li, and Samuel R. Bowman. 2021 · 2021
Later among the works it cites.
The MultiBERTs: BERT Reproductions for Robustness Analysis
Thibault Sellam, Steve Yadlowsky, Ian Tenney, Jason Wei, Naomi Saphra, Alexander D’Amour, Tal Linzen, Jasmijn Bastings, Iulia Raluca Turc, Jacob Eisenstein, Dipanjan Das, and Ellie Pavlick. 2022 · 2022
Closest in time.