Fetching the paper…
Reading the bibliography…
Recent studies on transformer-based language models show that they can answer questions by reasoning over knowledge provided as part of the context (i.e., in-context reasoning).
Cyc: Using common sense knowledge to overcome brittleness and knowledge acquisition bottlenecks
Douglas B. Lenat, Mayank Prakash, and Mary Shepherd · 1985
Earlier work this paper cites.
Inductive logic programming: Theory and methods
Stephen Muggleton and Luc de Raedt · 1994
Earlier work this paper cites.
Abductive and inductive reasoning: Background and issues
Peter A. Flach and Antonis C. Kakas · 2000
Earlier work this paper cites.
Expert systems in production planning and scheduling: A state-of-the-art survey
K. S. Metaxiotis, Dimitris Askounis, and John Psarras · 2002
Earlier work this paper cites.
Expert systems in production planning and scheduling: A state-of-the-art survey
Kostas S. Metaxiotis, Dimitris Askounis, and John E. Psarras · 2002
Earlier work this paper cites.
Natural logic for textual inference
Bill MacCartney and Christopher D. Manning · 2007
Earlier work this paper cites.
A tableau prover for natural logic and language
Lasha Abzianidze · 2015
Earlier work this paper cites.
Recursive neural networks can learn logical semantics
Samuel R. Bowman, Christopher Potts, and Christopher D. Manning · 2015
Earlier work this paper cites.
Harnessing deep neural networks with logic rules
Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard Hovy, and Eric Xing · 2016
Earlier work this paper cites.
LangPro: Natural language theorem prover
Lasha Abzianidze · 2017
Earlier work this paper cites.
Searchqa: A new q&a dataset augmented with context from a search engine, 2017
Matthew Dunn, Levent Sagun, Mike Higgins, V. Ugur Guney, Volkan Cirik, and Kyunghyun Cho · 2017
Earlier work this paper cites.
Editorial: The reasoning brain: The interplay between cognitive neuroscience and theories of reasoning
Vinod Goel, Gorka Navarrete, Ira A. Noveck, and Jérôme Prado · 2017
Earlier work this paper cites.
TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer · 2017
Earlier work this paper cites.
Decoupled weight decay regularization
Ilya Loshchilov and Frank Hutter · 2017
Earlier work this paper cites.
On-demand injection of lexical knowledge for recognising textual entailment
Pascual Martínez-Gómez, Koji Mineshima, Yusuke Miyao, and Daisuke Bekki · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Acquisition of phrase correspondences using natural deduction proofs
Hitomi Yanaka, Koji Mineshima, Pascual Martínez-Gómez, and Daisuke Bekki · 2018
Earlier work this paper cites.
How to train your MAML
Antreas Antoniou, Harrison Edwards, and Amos Storkey · 2019
Earlier work this paper cites.
Comet: Commonsense transformers for automatic knowledge graph construction
Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi · 2019
Earlier work this paper cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Earlier work this paper cites.
Natural questions: A benchmark for question answering research
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov · 2019
Earlier work this paper cites.
A logic-driven framework for consistency of neural models
Tao Li, Vivek Gupta, Maitrey Mehta, and Vivek Srikumar · 2019
Earlier work this paper cites.
Language models as knowledge bases?
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever · 2019
Earlier work this paper cites.
Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead
Cynthia Rudin · 2019
Earlier work this paper cites.
CLUTRR: A diagnostic benchmark for inductive reasoning from text
Koustuv Sinha, Shagun Sodhani, Jin Dong, Joelle Pineau, and William L. Hamilton · 2019
Earlier work this paper cites.
Meta-learning with adaptive hyperparameters
Sungyong Baik, Myungsub Choi, Janghoon Choi, Heewon Kim, and Kyoung Mu Lee · 2020
Earlier work this paper cites.
Abductive commonsense reasoning, 2020
Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Scott Wen tau Yih, and Yejin Choi · 2020
Earlier work this paper cites.
Transformers as soft reasoners over language
Peter Clark, Oyvind Tafjord, and Kyle Richardson · 2020
Cited alongside, same era.
Measuring systematic generalization in neural proof generation with transformers
Nicolas Gontier, Koustuv Sinha, Siva Reddy, and Christopher Joseph Pal · 2020
Cited alongside, same era.
MonaLog: a lightweight system for natural language inference based on monotonicity
Hai Hu, Qi Chen, Kyle Richardson, Atreyee Mukherjee, Lawrence S. Moss, and Sandra Kuebler · 2020
Cited alongside, same era.
How can we know what language models know?
Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig · 2020
Cited alongside, same era.
How context affects language models’ factual predictions, 2020
Fabio Petroni, Patrick Lewis, Aleksandra Piktus, Tim Rocktäschel, Yuxiang Wu, Alexander H. Miller, and Sebastian Riedel · 2020
Cited alongside, same era.
multiPRover: Generating multiple proofs for improved interpretability in rule reasoning
Swarnadeep Saha, Prateek Yadav, and Mohit Bansal · 2021
Later among the works it cites.
ProofWriter: Generating implications, proofs, and abductive statements over natural language
Oyvind Tafjord, Bhavana Dalvi, and Peter Clark · 2021
Later among the works it cites.
Transformer inference arithmetic
Carol Chen · 2022
Later among the works it cites.
Curriculum: A broad-coverage benchmark for linguistic phenomena in natural language understanding
Zeming Chen and Qiyue Gao · 2022
Later among the works it cites.
Knowledge neurons in pretrained transformers
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei · 2022
Later among the works it cites.
Logically consistent adversarial attacks for soft theorem provers
Alexander Gaskell, Yishu Miao, Francesca Toni, and Lucia Specia · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Exploring the limits of transfer learning with a unified text-to-text transformer, 2020
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu · 2020
Cited alongside, same era.
How much knowledge can you pack into the parameters of a language model?
Adam Roberts, Colin Raffel, and Noam Shazeer · 2020
Cited alongside, same era.
A primer in BERTology: What we know about how BERT works
Anna Rogers, Olga Kovaleva, and Anna Rumshisky · 2020
Cited alongside, same era.
AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh · 2020
Cited alongside, same era.
Balancing training for multilingual neural machine translation
Xinyi Wang, Yulia Tsvetkov, and Graham Neubig · 2020
Cited alongside, same era.
On negative interference in multilingual models: Findings and a meta-learning treatment
Zirui Wang, Zachary C. Lipton, and Yulia Tsvetkov · 2020
Cited alongside, same era.
Huggingface’s transformers: State-of-the-art natural language processing, 2020
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush · 2020
Cited alongside, same era.
Later among the works it cites.
Folio: Natural language reasoning with first-order logic, 2022
Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Luke Benson, Lucy Sun, Ekaterina Zubova, Yujie Qiao, Matthew Burtell, David Peng, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Shafiq Joty, Alexander R. Fabbri, Wojciech Kryscinski, Xi Victoria Lin, Caiming Xiong, and Dragomir Radev · 2022
Later among the works it cites.
METGEN: A module-based entailment tree generation framework for answer explanation
Ruixin Hong, Hongming Zhang, Xintong Yu, and Changshui Zhang · 2022
Later among the works it cites.
LoRA: Low-rank adaptation of large language models
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen · 2022
Later among the works it cites.
Maieutic prompting: Logically consistent reasoning with recursive explanations
Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras, and Yejin Choi · 2022
Later among the works it cites.
Locating and editing factual associations in GPT
Kevin Meng, David Bau, Alex J Andonian, and Yonatan Belinkov · 2022
Later among the works it cites.
Training language models to follow instructions with human feedback, 2022
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe · 2022
Later among the works it cites.
Interpretable proof generation via iterative backward reasoning
Hanhao Qu, Yu Cao, Jun Gao, Liang Ding, and Ruifeng Xu · 2022
Later among the works it cites.
Pushing the limits of rule reasoning in transformers through natural language satisfiability
Kyle Richardson and Ashish Sabharwal · 2022
Later among the works it cites.
RobustLR: A diagnostic benchmark for evaluating logical robustness of deductive reasoners
Soumya Sanyal, Zeyi Liao, and Xiang Ren · 2022
Later among the works it cites.
FaiRR: Faithful and robust deductive reasoning over natural language
Soumya Sanyal, Harman Singh, and Xiang Ren · 2022
Later among the works it cites.
Entailer: Answering questions with faithful and truthful chains of reasoning
Oyvind Tafjord, Bhavana Dalvi Mishra, and Peter Clark · 2022
Later among the works it cites.
Generating natural language proofs with verifier-guided search
Kaiyu Yang, Jia Deng, and Danqi Chen · 2022
Later among the works it cites.
Generating natural language proofs with verifier-guided search, 2022
Kaiyu Yang, Jia Deng, and Danqi Chen · 2022
Later among the works it cites.
Kformer: Knowledge injection in transformer feed-forward layers, 2022
Yunzhi Yao, Shaohan Huang, Li Dong, Furu Wei, Huajun Chen, and Ningyu Zhang · 2022
Later among the works it cites.
Discovering knowledge-critical subnetworks in pretrained language models
Deniz Bayazit, Negar Foroutan, Zeming Chen, Gail Weiss, and Antoine Bosselut · 2023
Closest in time.
Quantifying memorization across neural language models
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang · 2023
Closest in time.
Evaluating the logical reasoning ability of chatgpt and gpt-4, 2023
Hanmeng Liu, Ruoxi Ning, Zhiyang Teng, Jian Liu, Qiji Zhou, and Yue Zhang · 2023
Closest in time.
Mass-editing memory in a transformer
Kevin Meng, Arnab Sen Sharma, Alex J Andonian, Yonatan Belinkov, and David Bau · 2023
Closest in time.
Large language models can be easily distracted by irrelevant context
Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, and Denny Zhou · 2023
Closest in time.
Logical reasoning over natural language as knowledge representation: A survey, 2023
Zonglin Yang, Xinya Du, Rui Mao, Jinjie Ni, and Erik Cambria · 2023
Closest in time.
Generate rather than retrieve: Large language models are strong context generators
Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang · 2023
Closest in time.