Fetching the paper…
Reading the bibliography…
We have recently witnessed a number of impressive results on hard mathematical reasoning problems with language models.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020 · 1901
Earlier work this paper cites.
HuggingFace’s transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019 · 1910
Earlier work this paper cites.
Causal diagrams for empirical research
Judea Pearl. 1995 · 1995
Earlier work this paper cites.
Direct and indirect effects
Judea Pearl. 2001 · 2001
Earlier work this paper cites.
The independence of language and mathematical reasoning
Elizabeth M. Brannon. 2005 · 2005
Earlier work this paper cites.
Causality
Judea Pearl. 2009 · 2009
Earlier work this paper cites.
Thought beyond language: Neural dissociation of algebra and natural language
Martin M Monti, Lawrence M Parsons, and Daniel N Osherson. 2012 · 2012
Earlier work this paper cites.
Solving geometry problems: Combining text and diagram interpretation
Minjoon Seo, Hannaneh Hajishirzi, Ali Farhadi, Oren Etzioni, and Clint Malcolm. 2015 · 2015
Earlier work this paper cites.
MAWPS: A math word problem repository
Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. 2016 · 2016
Earlier work this paper cites.
Elements of causal inference: Foundations and learning algorithms
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. 2017 · 2017
Earlier work this paper cites.
From textbooks to knowledge: A case study in harvesting axiomatic knowledge from textbooks to solve geometry problems
Mrinmaya Sachan, Kumar Dubey, and Eric Xing. 2017 · 2017
Earlier work this paper cites.
Learning to solve geometry problems from natural language demonstrations in textbooks
Mrinmaya Sachan and Eric Xing. 2017 · 2017
Earlier work this paper cites.
Learning pipelines with limited data and domain knowledge: A study in parsing physics problems
Mrinmaya Sachan, Kumar Avinava Dubey, Tom M Mitchell, Dan Roth, and Eric P Xing. 2018 · 2018
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019
Earlier work this paper cites.
DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019 · 2019
Earlier work this paper cites.
Do NLP models know numbers? probing numeracy in embeddings
Eric Wallace, Yizhong Wang, Sujian Li, Sameer Singh, and Matt Gardner. 2019 · 2019
Earlier work this paper cites.
An empirical investigation of contextualized number prediction
Taylor Berg-Kirkpatrick and Daniel Spokoyny. 2020 · 2020
Earlier work this paper cites.
The pile: An 800gb dataset of diverse text for language modeling
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. 2020 · 2020
Cited alongside, same era.
Injecting numerical reasoning skills into language models
Mor Geva, Ankit Gupta, and Jonathan Berant. 2020 · 2020
Cited alongside, same era.
Learning the difference that makes A difference with counterfactually-augmented data
Divyansh Kaushik, Eduard H. Hovy, and Zachary Chase Lipton. 2020 · 2020
Cited alongside, same era.
A diverse corpus for evaluating and developing English math word problem solvers
Shen-yun Miao, Chao-Chun Liang, and Keh-Yih Su. 2020 · 2020
Cited alongside, same era.
Adjusting for confounding with text matching
Margaret E Roberts, Brandon M Stewart, and Richard A Nielsen. 2020 · 2020
Cited alongside, same era.
Masked measurement prediction: Learning to jointly predict quantities and units from textual context
Daniel Spokoyny, Ivan Lee, Zhao Jin, and Taylor Berg-Kirkpatrick. 2021 · 2021
Later among the works it cites.
Representing numbers in NLP: a survey and a vision
Avijit Thawani, Jay Pujara, Filip Ilievski, and Pedro Szekely. 2021 · 2021
Later among the works it cites.
GPT-J-6B: A 6 billion parameter autoregressive language model
Ben Wang and Aran Komatsuzaki. 2021 · 2021
Later among the works it cites.
GPT-NeoX-20B: An open-source autoregressive language model
Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, et al. 2022 · 2022
Closest in time.
Palm: Scaling language modeling with pathways
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022 · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Adapting text embeddings for causal inference
Victor Veitch, Dhanya Sridhar, and David M. Blei. 2020 · 2020
Cited alongside, same era.
Investigating gender bias in language models using causal mediation analysis
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020 · 2020
Cited alongside, same era.
Do language embeddings capture scales?
Xikun Zhang, Deepak Ramachandran, Ian Tenney, Yanai Elazar, and Dan Roth. 2020 · 2020
Cited alongside, same era.
GPT-Neo: Large scale autoregressive language modeling with mesh-tensorflow
Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. 2021 · 2021
Cited alongside, same era.
Training verifiers to solve math word problems
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021 · 2021
Cited alongside, same era.
Causal analysis of syntactic agreement mechanisms in neural language models
Matthew Finlayson, Aaron Mueller, Sebastian Gehrmann, Stuart Shieber, Tal Linzen, and Yonatan Belinkov. 2021 · 2021
Cited alongside, same era.
Mining the cause of political decision-making from social media: A case study of COVID-19 policies across the US states
Zhijing Jin, Zeyu Peng, Tejas Vaidhya, Bernhard Schoelkopf, and Rada Mihalcea. 2021b · 2021
Cited alongside, same era.
Closest in time.
CausalNLP tutorial: An introduction to causality for natural language processing
Zhijing Jin, Amir Feder, and Kun Zhang. 2022 · 2022
Closest in time.
Large language models are zero-shot reasoners
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022 · 2022
Closest in time.
Locating and editing factual associations in GPT
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022 · 2022
Closest in time.
Original or translated? A causal analysis of the impact of translationese on machine translation performance
Jingwei Ni, Zhijing Jin, Markus Freitag, Mrinmaya Sachan, and Bernhard Schölkopf. 2022 · 2022
Closest in time.
Training language models to follow instructions with human feedback
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022 · 2022
Closest in time.
Impact of pretraining term frequencies on few-shot reasoning
Yasaman Razeghi, Robert L Logan IV, Matt Gardner, and Sameer Singh. 2022 · 2022
Closest in time.
Symbolic brittleness in sequence models: on systematic generalization in symbolic mathematics
Sean Welleck, Peter West, Jize Cao, and Yejin Choi. 2022 · 2022
Closest in time.
Psychologically-inspired causal prompts
Zhiheng Lyu, Zhijing Jin, Justus Mattern, Rada Mihalcea, Mrinmaya Sachan, and Bernhard Schölkopf. 2023 · 2023
Closest in time.
Stanford alpaca: An instruction-following llama model
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023 · 2023
Closest in time.
Llama: Open and efficient foundation language models
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023 · 2023
Closest in time.
Are NLP models really able to solve simple math word problems?
Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021 · 2094
Closest in time.