Fetching the paper…
Reading the bibliography…
This paper investigates the ability of transformer-based models to learn structural recursion from examples.
A logical calculus of the ideas immanent in nervous activity
Warren S McCulloch and Walter Pitts · 1943
Earlier work this paper cites.
Representation of events in nerve nets and finite automata
Stephen Cole Kleene · 1951
Earlier work this paper cites.
On understanding types, data abstraction, and polymorphism
Luca Cardelli and Peter Wegner · 1985
Earlier work this paper cites.
Encoding sequential structure: experience with the real-time recurrent learning algorithm
Smith and Zipser · 1989
Earlier work this paper cites.
Inductively defined types
Thierry Coquand and Christine Paulin · 1990
Earlier work this paper cites.
Evolving Algebras 1993: Lipari Guide , pp. 9–36
Yuri Gurevich · 1993
Earlier work this paper cites.
The Dynamics of Discrete-Time Computation, with Application to Recurrent Neural Networks and Finite State Machine Extraction
Mike Casey · 1996
Earlier work this paper cites.
Changing data structures in type theory:a study of natural numbers
Nicolas Magaud and Yves Bertot · 2000
Earlier work this paper cites.
Abstract state machines: a unifying view of models of computation and of system design frameworks
Egon Börger · 2004
Earlier work this paper cites.
A call-by-name lambda-calculus machine
Jean-Louis Krivine · 2007
Earlier work this paper cites.
Transformer feed-forward layers are key-value memories, 2020
Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy · 2012
Earlier work this paper cites.
Interactive theorem proving and program development: Coq’Art: the calculus of inductive constructions
Yves Bertot and Pierre Castéran · 2013
Earlier work this paper cites.
Certified Programming with Dependent Types: A Pragmatic Introduction to the Coq Proof Assistant
Adam Chlipala · 2013
Earlier work this paper cites.
Type-and-example-directed program synthesis
Peter-Michael Osera and Steve Zdancewic · 2015
Earlier work this paper cites.
CompCert - A Formally Verified Optimizing Compiler
Xavier Leroy, Sandrine Blazy, Daniel Kästner, Bernhard Schommer, Markus Pister, and Christian Ferdinand · 2016
Earlier work this paper cites.
Program synthesis
Sumit Gulwani, Oleksandr Polozov, and Rishabh Singh · 2017
Earlier work this paper cites.
DeepFix: Fixing common C language errors by deep learning
Rahul Gupta, Soham Pal, Aditya Kanade, and Shirish K. Shevade · 2017
Earlier work this paper cites.
ELIXIR: Effective object oriented program repair
Ripon K. Saha, Yingjun Lyu, Hiroaki Yoshida, and Mukul R. Prasad · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Universal transformers, 2019
Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser · 2019
Earlier work this paper cites.
A structural probe for finding syntax in word representations
John Hewitt and Christopher D. Manning · 2019
Earlier work this paper cites.
Synthesizing symmetric lenses
Anders Miltner, Solomon Maina, Kathleen Fisher, Benjamin C. Pierce, David Walker, and Steve Zdancewic · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever · 2019
Earlier work this paper cites.
Analyzing the structure of attention in a transformer language model
Jesse Vig and Yonatan Belinkov · 2019
Earlier work this paper cites.
Learning to prove theorems via interacting with proof assistants
Kaiyu Yang and Jia Deng · 2019
Earlier work this paper cites.
On the Ability and Limitations of Transformers to Recognize Formal Languages
Satwik Bhattamishra, Kabir Ahuja, and Navin Goyal · 2020
Earlier work this paper cites.
On the computational power of transformers and its implications in sequence modeling
Satwik Bhattamishra, Arkil Patel, and Navin Goyal · 2020
Earlier work this paper cites.
A behavioural theory of recursive algorithms
Egon Börger and Klaus-Dieter Schewe · 2020
Earlier work this paper cites.
How can self-attention networks recognize Dyck-n languages?
Javid Ebrahimi, Dhruv Gelda, and Wei Zhang · 2020
Earlier work this paper cites.
Theoretical limitations of self-attention in neural sequence models
Michael Hahn · 2020
Earlier work this paper cites.
Generative language modeling for automated theorem proving, 2020
Stanislas Polu and Ilya Sutskever · 2020
Cited alongside, same era.
Exploring the limits of transfer learning with a unified text-to-text transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu · 2020
Cited alongside, same era.
Generating correctness proofs with neural networks
Alex Sanchez-Stern, Yousef Alhessi, Lawrence Saul, and Sorin Lerner · 2020
Cited alongside, same era.
Intellicode compose: Code generation using transformer
Alexey Svyatkovskiy, Shao Kun Deng, Shengyu Fu, and Neel Sundaresan · 2020
Cited alongside, same era.
Are transformers universal approximators of sequence-to-sequence functions?
Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank Reddi, and Sanjiv Kumar · 2020
Cited alongside, same era.
Unified pre-training for program understanding and generation
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al · 2022
Later among the works it cites.
A systematic evaluation of large language models of code
Frank F Xu, Uri Alon, Graham Neubig, and Vincent Josua Hellendoorn · 2022
Later among the works it cites.
Byt5: Towards a token-free future with pre-trained byte-to-byte models
Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel · 2022
Later among the works it cites.
Proofster: Automated formal verification
Arpan Agrawal, Emily First, Zhanna Kaufman, Tom Reichel, Shizhuo Zhang, Timothy Zhou, Alex Sanchez-Stern, Talia Ringer, and Yuriy Brun · 2023
Later among the works it cites.
Santacoder: don’t reach for the stars!
Loubna Ben Allal, Raymond Li, Denis Kocetkov, Chenghao Mou, Christopher Akiki, Carlos Munoz Ferrandis, Niklas Muennighoff, Mayank Mishra, Alex Gu, Manan Dey, et al · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang · 2021
Cited alongside, same era.
Neurosymbolic programming
Swarat Chaudhuri, Kevin Ellis, Oleksandr Polozov, Rishabh Singh, Armando Solar-Lezama, Yisong Yue, et al · 2021
Cited alongside, same era.
Evaluating large language models trained on code
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba · 2021
Cited alongside, same era.
Inductive logic programming at 30
Andrew Cropper, Sebastijan Dumancic, Richard Evans, and Stephen H. Muggleton · 2021
Cited alongside, same era.
Codexglue: A machine learning benchmark dataset for code understanding and generation
Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al · 2021
Cited alongside, same era.
Show your work: Scratchpads for intermediate computation with language models
Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al · 2021
Cited alongside, same era.
Attention is turing-complete
Jorge Pérez, Pablo Barceló, and Javier Marinkovic · 2021
Cited alongside, same era.
A toy model of universality: Reverse engineering how networks learn group operations, 2023
Bilal Chughtai, Lawrence Chan, and Neel Nanda · 2023
Later among the works it cites.
Towards automated circuit discovery for mechanistic interpretability, 2023
Arthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso · 2023
Later among the works it cites.
Traced: Execution-aware pre-training for source code
Yangruibo Ding, Ben Steenhoek, Kexin Pei, Gail Kaiser, Wei Le, and Baishakhi Ray · 2023
Later among the works it cites.
Baldur: Whole-proof generation and repair with large language models, 2023
Emily First, Markus N. Rabe, Talia Ringer, and Yuriy Brun · 2023
Later among the works it cites.
Inductive synthesis of structurally recursive functional programs from non-recursive expressions
Woosuk Lee and Hangyeol Cho · 2023
Later among the works it cites.
Starcoder: may the source be with you!, 2023
Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhskii, Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Mishig Davaadorj, Joel Lamy-Poirier, João Monteiro, Oleh Shliazhko, Nicolas Gontier, Nicholas Meade, Armel Zebaze, Ming-Ho Yee, Logesh Kumar Umapathi, Jian Zhu, Benjamin Lipkin, Muhtasham Oblokulov, Zhiruo Wang, Rudra Murthy, Jason Stillerman, Siva Sankalp Patel, Dmitry Abulkhanov, Marco Zocca, Manan Dey, Zhihan Zhang, Nour Fahmy, Urvashi Bhattacharyya, Wenhao Yu, Swayam Singh, Sasha Luccioni, Paulo Villegas, Maxim Kunakov, Fedor Zhdanov, Manuel Romero, Tony Lee, Nadav Timor, Jennifer Ding, Claire Schlesinger, Hailey Schoelkopf, Jan Ebert, Tri Dao, Mayank Mishra, Alex Gu, Jennifer Robinson, Carolyn Jane Anderson, Brendan Dolan-Gavitt, Danish Contractor, Siva Reddy, Daniel Fried, Dzmitry Bahdanau, Yacine Jernite, Carlos Muñoz Ferrandis, Sean Hughes, Thomas Wolf, Arjun Guha, Leandro von Werra, and Harm de Vries · 2023
Later among the works it cites.
Code execution with pre-trained language models
Chenxiao Liu, Shuai Lu, Weizhu Chen, Daxin Jiang, Alexey Svyatkovskiy, Shengyu Fu, Neel Sundaresan, and Nan Duan · 2023
Later among the works it cites.
Locating and editing factual associations in gpt, 2023
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov · 2023
Later among the works it cites.
Formal languages and the nlp black box
William Merrill · 2023
Later among the works it cites.
Progress measures for grokking via mechanistic interpretability, 2023
Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt · 2023
Later among the works it cites.
Is self-repair a silver bullet for code generation?, 2023
Theo X. Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama · 2023
Later among the works it cites.
Automatically correcting large language models: Surveying the landscape of diverse self-correction strategies, 2023
Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang · 2023
Later among the works it cites.
Can large language models reason about program invariants?
Kexin Pei, David Bieber, Kensen Shi, Charles Sutton, and Pengcheng Yin · 2023
Later among the works it cites.
Passport: Improving automated formal verification using identifiers
Alex Sanchez-Stern, Emily First, Timothy Zhou, Zhanna Kaufman, Yuriy Brun, and Talia Ringer · 2023
Later among the works it cites.
Lexecutor: Learning-guided execution
Beatriz Souza and Michael Pradel · 2023
Later among the works it cites.
Transformers as recognizers of formal languages: A survey on expressivity, 2023
Lena Strobl, William Merrill, Gail Weiss, David Chiang, and Dana Angluin · 2023
Later among the works it cites.
Mlregtest: A benchmark for the machine learning of regular languages, 2023
Sam van der Poel, Dakotah Lambert, Kalina Kostyszyn, Tiantian Gao, Rahul Verma, Derek Andersen, Joanne Chau, Emily Peterson, Cody St. Clair, Paul Fodor, Chihiro Shibata, and Jeffrey Heinz · 2023
Later among the works it cites.
Statistically meaningful approximation: a case study on approximating turing machines with transformers, 2023
Colin Wei, Yining Chen, and Tengyu Ma · 2023
Later among the works it cites.
Automated program repair in the era of large pre-trained language models
Chunqiu Steven Xia, Yuxiang Wei, and Lingming Zhang · 2023
Later among the works it cites.
Wizardlm: Empowering large language models to follow complex instructions
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang · 2023
Later among the works it cites.
Self-edit: Fault-aware code editor for code generation, 2023
Kechi Zhang, Zhuo Li, Jia Li, Ge Li, and Zhi Jin · 2023
Later among the works it cites.
Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x
Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Lei Shen, Zihan Wang, Andi Wang, Yang Li, et al · 2023
Later among the works it cites.
Cruxeval: A benchmark for code reasoning, understanding and execution
Alex Gu, Baptiste Rozière, Hugh Leather, Armando Solar-Lezama, Gabriel Synnaeve, and Sida I Wang · 2024
Closest in time.
Software Foundations
Benjamin C. Pierce et al · 2024
Closest in time.