We convert the DeepMind Mathematics Dataset into a reinforcement learning environment by interpreting it as a program synthesis problem.
Each action taken in the environment adds an operator or an input into a discrete compute graph.
Graphs which compute correct answers yield positive reward, enabling the optimization of a policy to construct compute graphs conditioned on problem statements.
Baseline models are trained using Double DQN on various subsets of problem types, demonstrating the capability to learn to correctly construct graphs despite the challenges of combinatorial explosion and noisy rewards.
Chuntao Jiang, Frans Coenen, and Michele Zito. A Survey of Frequent Subgraph Mining Algorithms. Knowledge Engineering Review, vol. 28, pp. 75-105, 2013. URL https://livrepository.liverpool.ac.uk/3005601/1/ker-jct-6-May-11.pdf
2013
Earlier work this paper cites.
Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929–1958, 2014
Sumit Gulwani, Alex Polozov and Rishabh Singh. Program Synthesis. Foundations and Trends in Programming Languages, vol. 4, no. 1-2, pp. 1–119, 2017. URL https://www.microsoft.com/en-us/research/publication/program-synthesis
David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli. Analysing Mathematical Reasoning Abilities of Neural Models. In International Conference on Learning Representations, 2019
Nitish Gupta, Kevin Lin, Dan Roth, Sameer Singh, and Matt Gardner. 2020. Neural Module Networks for Reasoning over Text. In International Conference on Learning Representations
2020
Later among the works it cites.
Beyond the bibliography
alphaXiv searches the wider corpus for related work and actual follow-ups.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, pp. 6000–6010, 2017