Fetching the paper…
Reading the bibliography…
Transformer large language models (LLMs) have sparked admiration for their exceptional performance on tasks that demand intricate multi-step reasoning.
The architecture of complexity
Herbert A. Simon · 1962
Earlier work this paper cites.
Human problem solving: The state of the theory in 1970
Herbert A Simon and Allen Newell · 1971
Earlier work this paper cites.
Bias in human reasoning: Causes and consequences
Jonathan St BT Evans · 1989
Earlier work this paper cites.
Hybrid algorithms for the constraint satisfaction problem
Patrick Prosser · 1993
Earlier work this paper cites.
Time series analysis
James Douglas Hamilton · 1994
Earlier work this paper cites.
On small perturbations of stable markov operators: Unbounded case
B. Delyon and A. Juditsky · 1999
Earlier work this paper cites.
Iterated random functions
Persi Diaconis and David Freedman · 1999
Earlier work this paper cites.
A neural probabilistic language model
Yoshua Bengio, Réjean Ducharme, and Pascal Vincent · 2000
Earlier work this paper cites.
Arithmetic and the brain
Stanislas Dehaene, Nicolas Molko, Laurent Cohen, and Anna J Wilson · 2004
Earlier work this paper cites.
Algorithm Design
Jon Kleinberg and Eva Tardos · 2005
Earlier work this paper cites.
Z3: An efficient smt solver
Leonardo De Moura and Nikolaj Bjørner · 2008
Earlier work this paper cites.
Human reasoning and cognitive science
Keith Stenning and Michiel Van Lambalgen · 2012
Earlier work this paper cites.
Conceptual and procedural knowledge: The case of mathematics
James Hiebert · 2013
Earlier work this paper cites.
Nonlinear Time Series: Theory, Methods and Applications with R Examples
R. Douc, E. Moulines, and D. Stoffer · 2014
Earlier work this paper cites.
Logic, probability, and human reasoning
Philip N Johnson-Laird, Sangeet S Khemlani, and Geoffrey P Goodwin · 2015
Earlier work this paper cites.
Program induction by rationale generation: Learning to solve and explain algebraic word problems
Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom · 2017
Earlier work this paper cites.
On the practical ability of recurrent neural networks to recognize hierarchical languages
Satwik Bhattamishra, Kabir Ahuja, and Navin Goyal · 2020
Earlier work this paper cites.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei · 2020
Earlier work this paper cites.
Location attention for extrapolation to longer sequences
Yann Dubois, Gautier Dagan, Dieuwke Hupkes, and Elia Bruni · 2020
Earlier work this paper cites.
Shortcut learning in deep neural networks
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann · 2020
Earlier work this paper cites.
Theoretical limitations of self-attention in neural sequence models
Michael Hahn · 2020
Earlier work this paper cites.
Scaling laws for autoregressive generative modeling
Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B. Brown, Prafulla Dhariwal, Scott Gray, Chris Hallacy, Benjamin Mann, Alec Radford, Aditya Ramesh, Nick Ryder, Daniel M. Ziegler, John Schulman, Dario Amodei, and Sam McCandlish · 2020
Earlier work this paper cites.
Scaling laws for neural language models
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei · 2020
Earlier work this paper cites.
The eos decision and length extrapolation
Benjamin Newman, John Hewitt, Percy Liang, and Christopher D Manning · 2020
Earlier work this paper cites.
Critical thinking for language models
Gregor Betz, Christian Voigt, and Kyle Richardson · 2021
Earlier work this paper cites.
Transformers as soft reasoners over language
Peter Clark, Oyvind Tafjord, and Kyle Richardson · 2021
Earlier work this paper cites.
Reasoning with transformer-based models: Deep learning, but shallow reasoning
Chadi Helwe, Chloé Clavel, and Fabian M. Suchanek · 2021
Earlier work this paper cites.
Delphi: Towards machine ethics and norms
Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Maxwell Forbes, Jon Borchardt, Jenny Liang, Oren Etzioni, Maarten Sap, and Yejin Choi · 2021
Earlier work this paper cites.
Limitations of autoregressive models and their alternatives
Chu-Cheng Lin, Aaron Jaech, Xin Li, Matthew R. Gormley, and Jason Eisner · 2021
Cited alongside, same era.
Show your work: Scratchpads for intermediate computation with language models, 2021
Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena · 2021
Cited alongside, same era.
Show your work: Scratchpads for intermediate computation with language models
Maxwell I. Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena · 2021
Cited alongside, same era.
Can you learn an algorithm? generalizing from easy to hard problems with recurrent networks
Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Furong Huang, Uzi Vishkin, Micah Goldblum, and Tom Goldstein · 2021
Cited alongside, same era.
Proofwriter: Generating implications, proofs, and abductive statements over natural language
NaturalProver: Grounded mathematical proof generation with language models
Sean Welleck, Jiacheng Liu, Ximing Lu, Hannaneh Hajishirzi, and Yejin Choi · 2022
Later among the works it cites.
Symbolic brittleness in sequence models: on systematic generalization in symbolic mathematics
Sean Welleck, Peter West, Jize Cao, and Yejin Choi · 2022
Later among the works it cites.
Unveiling transformers with LEGO: a synthetic reasoning task
Yi Zhang, Arturs Backurs, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, and Tal Wagner · 2022
Later among the works it cites.
Scaling laws for generative mixed-modal language models
Armen Aghajanyan, Lili Yu, Alexis Conneau, Wei-Ning Hsu, Karen Hambardzumyan, Susan Zhang, Stephen Roller, Naman Goyal, Omer Levy, and Luke Zettlemoyer · 2023
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Oyvind Tafjord, Bhavana Dalvi, and Peter Clark · 2021
Cited alongside, same era.
Neural algorithmic reasoning
Petar Veličković and Charles Blundell · 2021
Cited alongside, same era.
Exploring length generalization in large language models
Cem Anil, Yuhuai Wu, Anders Andreassen, Aitor Lewkowycz, Vedant Misra, Vinay Ramasesh, Ambrose Slone, Guy Gur-Ari, Ethan Dyer, and Behnam Neyshabur · 2022
Cited alongside, same era.
Exploring length generalization in large language models
Cem Anil, Yuhuai Wu, Anders Johan Andreassen, Aitor Lewkowycz, Vedant Misra, Vinay Venkatesh Ramasesh, Ambrose Slone, Guy Gur-Ari, Ethan Dyer, and Behnam Neyshabur · 2022
Cited alongside, same era.
End-to-end algorithm synthesis with recurrent networks: Extrapolation without overthinking
Arpit Bansal, Avi Schwarzschild, Eitan Borgnia, Zeyad Emam, Furong Huang, Micah Goldblum, and Tom Goldstein · 2022
Cited alongside, same era.
Abstract Dynamic Programming: 3rd Edition
D. Bertsekas · 2022
Cited alongside, same era.
Unobserved local structures make compositional generalization hard
Ben Bogin, Shivanshu Gupta, and Jonathan Berant · 2022
Cited alongside, same era.
Mirelle Bueno, Carlos Gemmel, Jeffrey Dalton, Roberto de Alencar Lotufo, and Rodrigo Frassetto Nogueira · 2022
Cited alongside, same era.
Ning Bian, Xianpei Han, Le Sun, Hongyu Lin, Yaojie Lu, and Ben He · 2023
Closest in time.
Sparks of artificial general intelligence: Early experiments with GPT-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott M. Lundberg, Harsha Nori, Hamid Palangi, Marco Túlio Ribeiro, and Yi Zhang · 2023
Closest in time.
Tighter bounds on the expressivity of transformer encoders
David Chiang, Peter Cholak, and Anand Pillay · 2023
Closest in time.
ChatGPT goes to law school
Jonathan H Choi, Kristin E Hickman, Amy Monahan, and Daniel Schwarcz · 2023
Closest in time.
Selection-inference: Exploiting large language models for interpretable logical reasoning
Antonia Creswell, Murray Shanahan, and Irina Higgins · 2023
Closest in time.
Draft, sketch, and prove: Guiding formal theorem provers with informal proofs
Albert Qiaochu Jiang, Sean Welleck, Jin Peng Zhou, Timothee Lacroix, Jiacheng Liu, Wenda Li, Mateja Jamnik, Guillaume Lample, and Yuhuai Wu · 2023
Closest in time.
Humans in humans out: On GPT converging toward common sense in both success and failure
Philipp E. Koralus and Vincent Wang-Mascianica · 2023
Closest in time.
Diverse demonstrations improve in-context compositional generalization
Itay Levy, Ben Bogin, and Jonathan Berant · 2023
Closest in time.
Zhengzhong Liang, Zeyu Zhang, Steven Bethard, and Mihai Surdeanu · 2023
Closest in time.
Transformers learn shortcuts to automata
Bingbin Liu, Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy, and Cyril Zhang · 2023
Closest in time.
Self-refine: Iterative refinement with self-feedback
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, and Peter Clark · 2023
Closest in time.
The expresssive power of transformers with chain of thought
William Merrill and Ashish Sabharwal · 2023
Closest in time.
A logic for expressing log-precision transformers
William Merrill and Ashish Sabharwal · 2023
Closest in time.
The parallelism tradeoff: Limitations of log-precision transformers
William Merrill and Ashish Sabharwal · 2023
Closest in time.
A tale of two circuits: Grokking as competition of sparse and dense subnetworks
William Merrill, Nikolaos Tsilivis, and Aman Shukla · 2023
Closest in time.
Grokking of hierarchical structure in vanilla transformers
Shikhar Murty, Pratyusha Sharma, Jacob Andreas, and Christopher Manning · 2023
Closest in time.
GPT-4 technical report, 2023
OpenAI · 2023
Closest in time.
Is ChatGPT a general-purpose natural language processing task solver?
Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang · 2023
Closest in time.
Language models are greedy reasoners: A systematic formal analysis of chain-of-thought
Abulhair Saparov and He He · 2023
Closest in time.
LLaMA: Open and efficient foundation language models
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample · 2023
Closest in time.
Explaining grokking through circuit efficiency
Vikrant Varma, Rohin Shah, Zachary Kenton, János Kramár, and Ramana Kumar · 2023
Closest in time.
Generating sequences by learning to self-correct
Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi, and Yejin Choi · 2023
Closest in time.
ChatGPT in scientific writing: a cautionary tale
Haoyi Zheng and Huichun Zhan · 2023
Closest in time.
Teaching algorithmic reasoning via in-context learning, 2023
Hattie Zhou, Azade Nova, Aaron Courville, Hugo Larochelle, Behnam Neyshabur, and Hanie Sedghi · 2023
Closest in time.