Towards Scientific Discovery with Generative AI: Progress, Opportunities, and Challenges

SciBERT: A pretrained language model for scientific text

Original

Beltagy, I.; Lo, K.; and Cohan, A. 2019 · 1903

Earlier work this paper cites.

Lagrangian neural networks

Original

Cranmer, M.; Greydanus, S.; Hoyer, S.; Battaglia, P.; Spergel, D.; and Ho, S. 2020a · 2003

Earlier work this paper cites.

Language models are few-shot learners

Original

Brown, T. B. 2020 · 2005

Earlier work this paper cites.

ProbLog: a probabilistic prolog and its application in link discovery

De Raedt, L.; Kimmig, A.; and Toivonen, H. 2007 · 2007

Earlier work this paper cites.

Generative language modeling for automated theorem proving

Original

Polu, S.; and Sutskever, I. 2020 · 2009

Earlier work this paper cites.

Symbolic regression of implicit equations

Schmidt, M.; and Lipson, H. 2009 · 2009

Earlier work this paper cites.

Sledgehammer: judgement day

Böhme, S.; and Nipkow, T. 2010 · 2010

Earlier work this paper cites.

Searching for exotic particles in high-energy physics with deep learning

Baldi, P.; Sadowski, P.; and Whiteson, D. 2014 · 2014

Earlier work this paper cites.

Probabilistic (logic) programming concepts

De Raedt, L.; and Kimmig, A. 2015 · 2015

Earlier work this paper cites.

Planning chemical syntheses with deep neural networks and symbolic AI

Segler, M. H.; Preuss, M.; and Waller, M. P. 2018 · 2018

Earlier work this paper cites.

BioBERT: a pre-trained biomedical language representation model for biomedical text mining

Lee, J.; Yoon, W.; Kim, S.; Kim, D.; Kim, S.; So, C. H.; and Kang, J. 2020 · 2020

Earlier work this paper cites.

AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity

Udrescu, S.-M.; Tan, A.; Feng, J.; Neto, O.; Wu, T.; and Tegmark, M. 2020 · 2020

Earlier work this paper cites.

AI Feynman: A physics-inspired method for symbolic regression

Udrescu, S.-M.; and Tegmark, M. 2020 · 2020

Earlier work this paper cites.

Neural symbolic regression that scales

Biggio, L.; Bendinelli, T.; Neitz, A.; Lucchi, A.; and Parascandolo, G. 2021 · 2021

Earlier work this paper cites.

On the opportunities and risks of foundation models

Original

Bommasani, R.; Hudson, D. A.; Adeli, E.; Altman, R.; Arora, S.; von Arx, S.; Bernstein, M. S.; Bohg, J.; Bosselut, A.; Brunskill, E.; et al. 2021 · 2021

Earlier work this paper cites.

Extracting training data from large language models

Carlini, N.; Tramer, F.; Wallace, E.; Jagielski, M.; Herbert-Voss, A.; Lee, K.; Roberts, A.; Brown, T.; Song, D.; Erlingsson, U.; et al. 2021 · 2021

Earlier work this paper cites.

Text2mol: Cross-modal molecule retrieval with natural language queries

Edwards, C.; Zhai, C.; and Ji, H. 2021 · 2021

Earlier work this paper cites.

Domain-specific language model pretraining for biomedical natural language processing

Gu, Y.; Tinn, R.; Cheng, H.; Lucas, M.; Usuyama, N.; Liu, X.; Naumann, T.; Gao, J.; and Poon, H. 2021 · 2021

Earlier work this paper cites.

Proof artifact co-training for theorem proving with language models

Original

Han, J. M.; Rute, J.; Wu, Y.; Ayers, E. W.; and Polu, S. 2021 · 2021

Earlier work this paper cites.

Measuring mathematical problem solving with the math dataset

Original

Hendrycks, D.; Burns, C.; Kadavath, S.; Arora, A.; Basart, S.; Tang, E.; Song, D.; and Steinhardt, J. 2021 · 2021

Earlier work this paper cites.

Highly accurate protein structure prediction with AlphaFold

Jumper, J.; Evans, R.; Pritzel, A.; Green, T.; Figurnov, M.; Ronneberger, O.; Tunyasuvunakool, K.; Bates, R.; Žídek, A.; Potapenko, A.; et al. 2021 · 2021

Earlier work this paper cites.

Semantic probabilistic layers for neuro-symbolic learning

Ahmed, K.; Teso, S.; Chang, K.-W.; Van den Broeck, G.; and Vergari, A. 2022 · 2022

Earlier work this paper cites.

Quantifying memorization across neural language models

Original

Carlini, N.; Ippolito, D.; Jagielski, M.; Lee, K.; Tramer, F.; and Zhang, C. 2022 · 2022

Cited alongside, same era.

Transformer-based protein generation with regularized latent space optimization

Castro, E.; Godavarthi, A.; Rubinfien, J.; Givechian, K.; Bhaskar, D.; and Krishnaswamy, S. 2022 · 2022

Cited alongside, same era.

End-to-end symbolic regression with transformers

Kamienny, P.-A.; d’Ascoli, S.; Lample, G.; and Charton, F. 2022 · 2022

Cited alongside, same era.

Hypertree proof search for neural theorem proving

Lample, G.; Lacroix, T.; Lachaux, M.-A.; Rodriguez, A.; Hayat, A.; Lavril, T.; Ebner, G.; and Martinet, X. 2022 · 2022

Cited alongside, same era.

A unified framework for deep symbolic regression

Landajuela, M.; Lee, C. S.; Yang, J.; Glatt, R.; Santiago, C. P.; Aravena, I.; Mundhenk, T.; Mulcahy, G.; and Petersen, B. K. 2022 · 2022

Cited alongside, same era.

Reasoning or reciting? exploring the capabilities and limitations of language models through counterfactual tasks

Original

Wu, Z.; Qiu, L.; Ross, A.; Akyürek, E.; Chen, B.; Wang, B.; Kim, N.; Andreas, J.; and Kim, Y. 2023 · 2023

Later among the works it cites.

Protst: Multi-modality learning of protein sequences and biomedical texts

Xu, M.; Yuan, X.; Miret, S.; and Tang, J. 2023 · 2023

Later among the works it cites.

Desirable molecule discovery via generative latent space exploration

Zheng, W.; Li, J.; and Zhang, Y. 2023 · 2023

Later among the works it cites.

Multi-objective latent space optimization of generative molecular design models

Abeer, A. N.; Urban, N. M.; Weil, M. R.; Alexander, F. J.; and Yoon, B.-J. 2024 · 2024

Closest in time.

Meta-Designing Quantum Experiments with Language Models

Original

Arlt, S.; Duan, H.; Li, F.; Xie, S. M.; Wu, Y.; and Krenn, M. 2024 · 2024

Closest in time.