2023

AI for Mathematics: A Cognitive Science Perspective

Zhang, Cedegao E., Collins, Katherine M., Weller, Adrian et al.

Understand

Mathematics is one of the most powerful conceptual systems developed and used by the human species.

Dreams of automated mathematicians have a storied history in artificial intelligence (AI).
Rapid progress in AI, particularly propelled by advances in large language models (LLMs), has sparked renewed, widespread interest in building such systems.
In this work, we reflect on these goals from a \textit{cognitive science} perspective.

Built on

Intelligent machinery
Alan M. Turing · 1948
Earlier work this paper cites.
Two dogmas of empiricism
W. V. O. Quine · 1951
Earlier work this paper cites.
Functional fixedness as related to problem solving: a repetition of three experiments
Robert E Adamson · 1952
Earlier work this paper cites.
The logic theory machine–a complex information processing system
Allen Newell and Herbert A. Simon · 1956
Earlier work this paper cites.
Empirical explorations of the logic theory machine: A case study in heuristic
A. Newell, J. C. Shaw, and H. A. Simon · 1957
Earlier work this paper cites.
Toward mechanical mathematics
Hao Wang · 1960
Earlier work this paper cites.
A machine program for theorem-proving
Martin Davis, George Logemann, and Donald Loveland · 1962
Earlier work this paper cites.
Truth and proof
Alfred Tarski · 1969
Earlier work this paper cites.
Human problem solving
Allen Newell and Herbert A. Simon · 1972
Earlier work this paper cites.
The language of thought
Jerry A Fodor · 1975
Earlier work this paper cites.
Logic and conversation
Herbert P. Grice · 1975
Earlier work this paper cites.
AM: An artificial intelligence approach to discovery in mathematics as heuristic search
Douglas B. Lenat · 1976
Earlier work this paper cites.
Non-resolution theorem proving
W. W. Bledsoe · 1977
Earlier work this paper cites.
The Computer Modelling of Mathematical Reasoning
Alan Bundy · 1983
Earlier work this paper cites.
Pragmatics
Stephen C. Levinson · 1983
Earlier work this paper cites.
The psychology of sunk cost
Hal R. Arkes and Catherine Blumer · 1985
Earlier work this paper cites.
Advertisement for a semantics for psychology
Ned Block · 1986
Earlier work this paper cites.
Learning mathematics from examples and by doing
Xinming Zhu and Herbert A. Simon · 1987
Earlier work this paper cites.
Intention, plans, and practical reason
Michael Bratman · 1987
Earlier work this paper cites.
The use of explicit plans to guide inductive proofs
Alan Bundy · 1988
Earlier work this paper cites.
The Robbers Cave experiment: Intergroup conflict and cooperation
Muzafer Sherif, O.J. Harvey, B. Jack White, William R. Hood, and Carolyn W. Sherif · 1988
Earlier work this paper cites.
Mental models
Philip N. Johnson-Laird · 1989
Earlier work this paper cites.
Self-explanations: How students study and use examples in learning to solve problems
Michelene T.H. Chi, Miriam Bassok, Matthew W Lewis, Peter Reimann, and Robert Glaser · 1989
Earlier work this paper cites.
Varieties of numerical abilities
Stanislas Dehaene · 1992
Earlier work this paper cites.
Rippling: A heuristic for guiding inductive proofs
Alan Bundy, Andrew Stevens, F. V. Harmelen, Andrew Ireland, and Alan Smaill · 1993
Earlier work this paper cites.
Eliciting self-explanations improves understanding
Michelene T.H. Chi, Nicholas De Leeuw, Mei-Hung Chiu, and Christian LaVancher · 1994
Earlier work this paper cites.
Relevance: Communication and cognition
Dan Sperber and Wilson Deirdre · 1995
Earlier work this paper cites.
An analysis of arithmetic problem posing by middle school students
Edward A. Silver and Jinfa Cai · 1996
Earlier work this paper cites.
Core knowledge
Elizabeth S. Spelke · 2000
Earlier work this paper cites.
Mathematical explanation: Problems and prospects
Paolo Mancosu · 2001
Earlier work this paper cites.
E - a brainiac theorem prover
Stephan Schulz · 2002
Earlier work this paper cites.
Rationality
Eldar Shafir and Robyn A. LeBoeuf · 2002
Earlier work this paper cites.
Deep Blue
Murray Campbell, A Joseph Hoane Jr., and Feng-hsiung Hsu · 2002
Earlier work this paper cites.
The theory theory as an alternative to the innateness hypothesis
Alison Gopnik · 2003
Earlier work this paper cites.
The big book of concepts
Gregory Murphy · 2004
Earlier work this paper cites.
Core systems of number
Lisa Feigenson, Stanislas Dehaene, and Elizabeth Spelke · 2004
Earlier work this paper cites.
Moves in mind: The psychology of board games
Fernand Gobet, Jean Retschitzki, and Alex de Voogt · 2004
Earlier work this paper cites.
Theory-based bayesian models of inductive learning and reasoning
Joshua B. Tenenbaum, Thomas L. Griffiths, and Charles Kemp · 2006
Earlier work this paper cites.
The structure and function of explanations
Tania Lombrozo · 2006
Earlier work this paper cites.
Core knowledge
Elizabeth S. Spelke and Katherine D. Kinzler · 2007
Earlier work this paper cites.

Similar

Z3: An efficient smt solver
Leonardo Mendonça de Moura and Nikolaj S. Bjørner · 2008
Cited alongside, same era.
The advantage of abstract examples in learning math
Jennifer A. Kaminski, Vladimir M. Sloutsky, and Andrew F. Heckler · 2008
Cited alongside, same era.
Origins of mathematical intuitions: The case of arithmetic
Stanislas Dehaene · 2009
Cited alongside, same era.
The Origin of Concepts
Susan Carey · 2009
Cited alongside, same era.
The role of explanation in discovery and generalization: Evidence from category learning
Joseph J. Williams and Tania Lombrozo · 2010
Cited alongside, same era.
Running experiments on Amazon Mechanical Turk
Gabriele Paolacci, Jesse Chandler, and Panagiotis G. Ipeirotis · 2010
Cited alongside, same era.
How to grow a mind: Statistics, structure, and abstraction
Joshua B. Tenenbaum, Charles Kemp, Thomas L. Griffiths, and Noah D. Goodman · 2011
Cited alongside, same era.
The number sense: How the mind creates mathematics
Stanislas Dehaene · 2011
Cited alongside, same era.
The instrumental value of explanations
Tania Lombrozo · 2011
Cited alongside, same era.
Finding new facts; thinking new thoughts
Laura Schulz · 2012
Cited alongside, same era.
Children balance theories and evidence in exploration, explanation, and learning
Elizabeth Baraff Bonawitz, Tessa J.P. van Schijndel, Daniel Friel, and Laura Schulz · 2012
Cited alongside, same era.
A fully automatic problem solver with human-style output
Original
Mohan Ganesalingam and William Timothy Gowers · 2013
Cited alongside, same era.

Then

Naturalprover: Grounded mathematical proof generation with language models
Sean Welleck, Jiacheng Liu, Ximing Lu, Hannaneh Hajishirzi, and Yejin Choi · 2022
Later among the works it cites.
Draft, sketch, and prove: Guiding formal theorem provers with informal proofs
Original
Albert Q. Jiang, Sean Welleck, Jin Peng Zhou, Wenda Li, Jiacheng Liu, Mateja Jamnik, Timothée Lacroix, Yuhuai Wu, and Guillaume Lample · 2022
Later among the works it cites.
Thor: Wielding hammers to integrate language models and automated theorem provers
Albert Q. Jiang, Wenda Li, Szymon Tworkowski, Konrad Czechowski, Tomasz Odrzygóźdź, Piotr Miłoś, Yuhuai Wu, and Mateja Jamnik · 2022
Later among the works it cites.
LEMMA: Bootstrapping high-level mathematical reasoning with learned symbolic abstractions
Original
Zhening Li, Gabriel Poesia, Omar Costilla-Reyes, Noah Goodman, and Armando Solar-Lezama · 2022
Later among the works it cites.
Capturing advanced human cognitive abilities with deep neural networks
James L. McClelland · 2022
Later among the works it cites.
Symbols and mental programs: a hypothesis about human singularity
Stanislas Dehaene, Fosca Al Roumi, Yair Lakretz, Samuel Planton, and Mathias Sablé-Meyer · 2022
Later among the works it cites.
A language of thought for the mental representation of geometric shapes
Mathias Sablé-Meyer, Kevin Ellis, Josh Tenenbaum, and Stanislas Dehaene · 2022
Later among the works it cites.
Emergent world representations: Exploring a sequence model trained on a synthetic task
Original
Kenneth Li, Aspen K. Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg · 2022
Later among the works it cites.
Deep learning, reinforcement learning, and world models
Yutaka Matsuo, Yann LeCun, Maneesh Sahani, Doina Precup, David Silver, Masashi Sugiyama, Eiji Uchibe, and Jun Morimoto · 2022
Later among the works it cites.
The evolution of agency: Behavioral organization from lizards to humans
Michael Tomasello · 2022
Later among the works it cites.
People construct simplified mental representations to plan
Mark K. Ho, David Abel, Carlos G. Correa, Michael L. Littman, Jonathan D. Cohen, and Thomas L. Griffiths · 2022
Later among the works it cites.
Planning with theory of mind
Mark K. Ho, Rebecca Saxe, and Fiery Cushman · 2022
Later among the works it cites.
Training compute-optimal large language models
Original
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al · 2022
Later among the works it cites.
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed H. Chi, Quoc V. Le, Denny Zhou, et al · 2022
Later among the works it cites.
Star: Bootstrapping reasoning with reasoning
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman · 2022
Later among the works it cites.
Human-level play in the game of diplomacy by combining language models with strategic reasoning
Meta Fundamental AI Research Diplomacy Team (FAIR)†, Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, et al · 2022
Later among the works it cites.
Gpt-4 technical report, 2023
OpenAI · 2023
Closest in time.
Sparks of artificial general intelligence: Early experiments with gpt-4
Original
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al · 2023
Closest in time.
Evaluating language models for mathematics through interactions
Original
Katherine M Collins, Albert Q Jiang, Simon Frieder, Lionel Wong, Miri Zilka, Umang Bhatt, Thomas Lukasiewicz, Yuhuai Wu, Joshua B Tenenbaum, William Hart, et al · 2023
Closest in time.
Mathematical capabilities of chatgpt
Original
Simon Frieder, Luca Pinchetti, Ryan-Rhys Griffiths, Tommaso Salvatori, Thomas Lukasiewicz, Philipp Christian Petersen, Alexis Chevalier, and Julius Berner · 2023
Closest in time.
An empirical study on challenging math problem solving with gpt-4
Original
Yiran Wu, Feiran Jia, Shaokun Zhang, Qingyun Wu, Hangyu Li, Erkang Zhu, Yue Wang, Yin Tat Lee, Richard Peng, and Chi Wang · 2023
Closest in time.
Faith and fate: Limits of transformers on compositionality
Original
Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jian, Bill Yuchen Lin, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D Hwang, et al · 2023
Closest in time.
Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct
Original
Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang · 2023
Closest in time.
Let’s verify step by step
Original
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe · 2023
Closest in time.
Metamath: Bootstrap your own mathematical questions for large language models
Original
Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu · 2023
Closest in time.
Textbooks are all you need
Original
Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al · 2023
Closest in time.
Testing GPT-4 with Wolfram Alpha and Code Interpreter plug-ins on math and science problems
Original
Ernest Davis and Scott Aaronson · 2023
Closest in time.
Baldur: Whole-proof generation and repair with large language models
Original
Emily First, Markus N. Rabe, Talia Ringer, and Yuriy Brun · 2023
Closest in time.
Solving math word problems by combining language models with symbolic solvers
Original
Joy He-Yueya, Gabriel Poesia, Rose E. Wang, and Noah D. Goodman · 2023
Closest in time.
Peano: learning formal mathematical reasoning
Gabriel Poesia and Noah D. Goodman · 2023
Closest in time.
Word meaning in minds and machines
Brenden M. Lake and Gregory L. Murphy · 2023
Closest in time.
From word models to world models: Translating from natural language to the probabilistic language of thought
Original
Lionel Wong, Gabriel Grand, Alexander K. Lew, Noah D. Goodman, Vikash K. Mansinghka, Jacob Andreas, and Joshua B. Tenenbaum · 2023
Closest in time.
Augmented language models: A survey
Original
Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis, Ram Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, et al · 2023
Closest in time.
What makes mathematicians believe unproved mathematical statements?
Timothy Gowers · 2023
Closest in time.
Humans decompose tasks by trading off utility and computational cost
Carlos G. Correa, Mark K. Ho, Frederick Callaway, Nathaniel D. Daw, and Thomas L. Griffiths · 2023
Closest in time.
Rational simplification and rigidity in human planning
Mark K. Ho, Jonathan D. Cohen, and Thomas L. Griffiths · 2023
Closest in time.
Resource rationality
Thomas Icard · 2023
Closest in time.
Reflexion: Language agents with verbal reinforcement learning
Original
Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao · 2023
Closest in time.
Using games to understand the mind
Kelsey Allen, Franziska Brändle, Matthew Botvinick, Judith E. Fan, Samuel J. Gershman, Alison Gopnik, Thomas L. Griffiths, Joshua K. Hartshorne, Tobias U. Hauser, Mark K. Ho, Joshua R. de Leeuw, Wei Ji Ma, Kou Murayama, Jonathan D. Nelson, Bas van Opheusden, Thomas Pouncy, Janet Rafner, Iyad Rahwan, Robb Rutledge, Jacob Friis Sherson, Ozgur Simsek, Hugo Spiers, Christopher Summerfield, Mirko Thalmann, Natalia Vélez, Andrew J. Watrous, Joshua B. Tenenbaum, and Eric Schulz · 2023
Closest in time.
Research agenda
Gabriel Poesia · 2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…