Fetching the paper…
Reading the bibliography…
To quantitatively and intuitively explore the generalization ability of pre-trained language models (PLMs), we have designed several tasks of arithmetic and logical reasoning.
Roberta: A robustly optimized bert pretraining approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, M. Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1907
Earlier work this paper cites.
Albert: A lite bert for self-supervised learning of language representations
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020 · 1909
Earlier work this paper cites.
Measuring arithmetic extrapolation performance
Andreas Madsen and alexander rosenberg johansen. 2019 · 1910
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber. 1997 · 1997
Earlier work this paper cites.
Commonsense knowledge graph reasoning by selection or generation? why?
Cunxiang Wang, Jinhang Wu, Luxin Liu, and Yue Zhang. 2020b · 2008
Earlier work this paper cites.
Ningyu Zhang, Luoqiu Li, Shumin Deng, H. Yu, Xu Cheng, W. Zhang, and Huajun Chen. 2020 · 2009
Earlier work this paper cites.
Ape210k: A large-scale and template-rich dataset of math word problems
Wei Zhao, Mingyue Shang, Yang Liu, Liang Wang, and Jingming Liu. 2020 · 2009
Earlier work this paper cites.
Empirical evaluation of gated recurrent neural networks on sequence modeling
J. Chung, Çaglar Gülçehre, Kyunghyun Cho, and Yoshua Bengio. 2014 · 2014
Earlier work this paper cites.
GloVe: Global vectors for word representation
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014 · 2014
Earlier work this paper cites.
Sequence to sequence learning with neural networks
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014 · 2014
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, undefinedukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Earlier work this paper cites.
Deep neural solver for math word problems
Yan Wang, Xiaojiang Liu, and Shuming Shi. 2017 · 2017
Cited alongside, same era.
Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks
B. Lake and Marco Baroni. 2018 · 2018
Cited alongside, same era.
Attending to mathematical language with transformers
A. Wangperawong. 2018 · 2018
Cited alongside, same era.
COMET: Commonsense transformers for automatic knowledge graph construction
Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019 · 2019
Cited alongside, same era.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
Language models are few-shot learners
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020 · 2020
Later among the works it cites.
Transformers as soft reasoners over language
Peter Clark, Oyvind Tafjord, and Kyle Richardson. 2020 · 2020
Later among the works it cites.
Injecting numerical reasoning skills into language models
Mor Geva, Ankit Gupta, and Jonathan Berant. 2020 · 2020
Later among the works it cites.
How can we accelerate progress towards human-like linguistic generalization?
Tal Linzen. 2020 · 2020
Later among the works it cites.
BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs
Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019 · 2019
Cited alongside, same era.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019
Cited alongside, same era.
Atomic: An atlas of machine commonsense for if-then reasoning
Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A. Smith, and Yejin Choi. 2019 · 2019
Cited alongside, same era.
Do NLP models know numbers? probing numeracy in embeddings
Eric Wallace, Yizhong Wang, Sujian Li, Sameer Singh, and Matt Gardner. 2019 · 2019
Cited alongside, same era.
Does it make sense? and why? a pilot study for sense making and explanation
Cunxiang Wang, Shuailong Liang, Yue Zhang, Xiaonan Li, and Tian Gao. 2019 · 2019
Cited alongside, same era.
BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020a
Cited in the paper.
Question and answer test-train overlap in open-domain question answering datasets
Patrick Lewis, Pontus Stenetorp, and Sebastian Riedel. 2020b
Cited in the paper.
R. Thomas McCoy, Junghyun Min, and Tal Linzen. 2020 · 2020
Later among the works it cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020 · 2020
Later among the works it cites.
How much knowledge can you pack into the parameters of a language model?
Adam Roberts, Colin Raffel, and Noam Shazeer. 2020 · 2020
Later among the works it cites.
Semeval-2020 task 4: Commonsense validation and explanation
Cunxiang Wang, Shuailong Liang, Yili Jin, Yilong Wang, Xiao-Dan Zhu, and Y. Zhang. 2020a · 2020
Later among the works it cites.
Investigating the limitations of the transformers with simple arithmetic tasks
Rodrigo Nogueira, Zhiying Jiang, and J. Li. 2021 · 2021
Closest in time.
Can generative pre-trained language models serve as knowledge bases for closed-book QA?
Cunxiang Wang, Pai Liu, and Yue Zhang. 2021 · 2021
Closest in time.