2021

SymbolicGPT: A Generative Transformer Model for Symbolic Regression

Valipour, Mojtaba, You, Bowen, Panju, Maysum et al.

Understand

Symbolic regression is the task of identifying a mathematical expression that best fits a provided dataset of input and output values.

Due to the richness of the space of mathematical expressions, symbolic regression is generally a challenging problem.
While conventional approaches based on genetic evolution algorithms have been used for decades, deep learning-based methods are relatively new and an active research area.
In this work, we present SymbolicGPT, a novel transformer-based language model for symbolic regression.

Built on

Using a tree structured genetic algorithm to perform symbolic regression
B. McKay, M. J. Willis, and G. W. Barton · 1995
Earlier work this paper cites.
Symbolic regression via genetic programming
D. A. Augusto and H. J. Barbosa · 2000
Earlier work this paper cites.
Distilling free-form natural laws from experimental data
M. Schmidt and H. Lipson · 2009
Earlier work this paper cites.
Scikit-learn: Machine learning in python
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and Édouard Duchesnay · 2011
Earlier work this paper cites.
Symbolic regression via genetic programming for data driven derivation of confinement scaling laws without any assumption on their mathematical form
A. Murari, E. Peluso, M. Gelfusa, I. Lupelli, M. Lungaroni, and P. Gaudio · 2014
Earlier work this paper cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Earlier work this paper cites.
Extrapolation and learning equations
G. S. Martius and C. Lampert · 2017
Earlier work this paper cites.
Pointnet: Deep learning on point sets for 3d classification and segmentation
C. R. Qi, H. Su, K. Mo, and L. J. Guibas · 2017
Earlier work this paper cites.

Similar

Bert: Pre-training of deep bidirectional transformers for language understanding
Original
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova · 2018
Cited alongside, same era.
Improving language understanding by generative pre-training
A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever · 2018
Cited alongside, same era.
Learning equations for extrapolation and control
S. Sahoo, C. Lampert, and G. Martius · 2018
Cited alongside, same era.
Parameter identification for symbolic regression using nonlinear least squares
M. Kommenda, B. Burlacu, G. Kronberger, and M. Affenzeller · 2019
Cited alongside, same era.
Language models are unsupervised multitask learners
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever · 2019
Cited alongside, same era.
Symbolic regression in materials science
Original
Y. Wang, N. Wagner, and J. M. Rondinelli · 2019
Cited alongside, same era.
A seq2seq approach to symbolic regression
L. Biggio, T. Bendinelli, A. Lucchi, and G. Parascandolo · 2020
Cited alongside, same era.

Then

Language models are few-shot learners
Original
T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al · 2020
Later among the works it cites.
Learning symbolic expressions via gumbel-max equation learner network
Original
G. Chen · 2020
Later among the works it cites.
Integration of neural network-based symbolic regression in deep learning for scientific discovery
S. Kim, P. Y. Lu, S. Mukherjee, M. Gilbert, L. Jing, V. Čeperić, and M. Soljačić · 2020
Later among the works it cites.
Deep learning for symbolic mathematics
G. Lample and F. Charton · 2020
Later among the works it cites.
SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python
P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, İ. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, and SciPy 1.0 Contributors · 2020
Later among the works it cites.
Probabilistic grammars for equation discovery
J. Brence, L. Todorovski, and S. Džeroski · 2021
Closest in time.
Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients
B. K. Petersen, M. L. Larma, T. N. Mundhenk, C. P. Santiago, S. K. Kim, and J. T. Kim · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…