Fetching the paper…
Reading the bibliography…
Large pre-trained language models perform remarkably well on tasks that can be done "in one pass", such as generating realistic text or synthesizing computer programs.
Analysing mathematical reasoning abilities of neural models
David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli · 1904
Earlier work this paper cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu · 1910
Earlier work this paper cites.
Learning to parse database queries using inductive logic programming
J Zelle and R Mooney · 1996
Earlier work this paper cites.
Language models are few-shot learners
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei · 2005
Earlier work this paper cites.
Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars
Luke S Zettlemoyer and Michael Collins · 2005
Earlier work this paper cites.
Learning for semantic parsing with statistical machine translation
Yuk Wah Wong and Raymond J Mooney · 2006
Earlier work this paper cites.
Inducing probabilistic CCG grammars from logical form with higher-order unification
Tom Kwiatkowksi, Luke Zettlemoyer, Sharon Goldwater, and Mark Steedman · 2010
Earlier work this paper cites.
Long range arena: A benchmark for efficient transformers
Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, and Donald Metzler · 2011
Earlier work this paper cites.
Alex Graves, Greg Wayne, and Ivo Danihelka · 2014
Earlier work this paper cites.
Wojciech Zaremba and Ilya Sutskever · 2014
Earlier work this paper cites.
Adaptive computation time for recurrent neural networks
Alex Graves · 2016
Cited alongside, same era.
Hybrid computing using a neural network with dynamic external memory
Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwinska, Sergio Gomez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, Adrià Puigdomènech Badia, Karl Moritz Hermann, Yori Zwols, Georg Ostrovski, Adam Cain, Helen King, Christopher Summerfield, Phil Blunsom, Koray Kavukcuoglu, and Demis Hassabis · 2016
Cited alongside, same era.
Neural gpus learn algorithms
Lukasz Kaiser and Ilya Sutskever · 2016
Cited alongside, same era.
Neural random-access machines
Karol Kurach, Marcin Andrychowicz, and Ilya Sutskever · 2016
Cited alongside, same era.
Neural programmer-interpreters
Scott Reed and Nando de Freitas · 2016
Cited alongside, same era.
Learning to execute programs with instruction pointer attention graph neural networks
David Bieber, Charles Sutton, Hugo Larochelle, and Daniel Tarlow · 2020
Later among the works it cites.
Learning semantic program embeddings with graph interval neural network, 2020
Yu Wang, Fengjuan Gao, Linzhang Wang, and Ke Wang · 2020
Later among the works it cites.
Program synthesis with large language models
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al · 2021
Closest in time.
Pondernet: Learning to ponder
Andrea Banino, Jan Balaguer, and Charles Blundell · 2021
Closest in time.
Evaluating large language models trained on code, July 2021
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde, Jared Kaplan, Harri Edwards, Yura Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, Will Guss, Alex Nichol, Igor Babuschkin, Suchir Balaji, Shantanu Jain, Andrew Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba · 2021
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, and Pushmeet Kohli · 2017
Cited alongside, same era.
Learning to represent programs with graphs
Miltiadis Allamanis, Marc Brockschmidt, and Mahmoud Khademi · 2018
Cited alongside, same era.
Universal transformers
Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser · 2018
Cited alongside, same era.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Cited alongside, same era.
Climbing towards NLU: On meaning, form, and understanding in the age of data
Emily M. Bender and Alexander Koller · 2020
Cited alongside, same era.
Pointer graph networks, 2020a
Petar Veličković, Lars Buesing, Matthew C. Overlan, Razvan Pascanu, Oriol Vinyals, and Charles Blundell
Cited in the paper.
Neural execution of graph algorithms, 2020b
Petar Veličković, Rex Ying, Matilde Padovano, Raia Hadsell, and Charles Blundell
Cited in the paper.
Closest in time.
Implicit representations of meaning in neural language models
Belinda Z. Li, Maxwell Nye, and Jacob Andreas · 2021
Closest in time.
Pretrained transformers as universal computation engines
Kevin Lu, Aditya Grover, Pieter Abbeel, and Igor Mordatch · 2021
Closest in time.
Project CodeNet: A Large-Scale AI for code dataset for learning a diversity of coding tasks
Ruchir Puri, David S Kung, Geert Janssen, Wei Zhang, Giacomo Domeniconi, Vladmir Zolotov, Julian Dolby, Jie Chen, Mihir Choudhury, Lindsey Decker, Veronika Thost, Luca Buratti, Saurabh Pujar, and Ulrich Finkler · 2021
Closest in time.
Petar Velickovic and Charles Blundell · 2021
Closest in time.