Fetching the paper…
Reading the bibliography…
The instruction learning paradigm -- where a model learns to perform new tasks from task descriptions alone -- has become popular in general-purpose model research.
Programming techniques: Regular expression search algorithm
Ken Thompson. 1968 · 1968
Earlier work this paper cites.
Counter-Free Automata (MIT research monograph no. 65)
Robert McNaughton and Seymour A Papert. 1971 · 1971
Earlier work this paper cites.
Introduction to formal language theory
Michael A Harrison. 1978 · 1978
Earlier work this paper cites.
Adapting language models for zero-shot learning by meta-tuning on dataset and prompt collections
Ruiqi Zhong, Kristy Lee, Zheng Zhang, and Dan Klein. 2021 · 1978
Earlier work this paper cites.
Generating hard satisfiability problems
Bart Selman, David G Mitchell, and Hector J Levesque. 1996 · 1996
Earlier work this paper cites.
State complexity of regular languages
Sheng Yu. 2001 · 2001
Earlier work this paper cites.
The Turking Test: Can language models understand instructions?
Avia Efrat and Omer Levy. 2020 · 2010
Earlier work this paper cites.
Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks
Brenden M. Lake and Marco Baroni. 2018 · 2018
Earlier work this paper cites.
Synthetic datasets for neural program synthesis
Richard Shin, Neel Kant, Kavi Gupta, Christopher Bender, Brandon Trabucco, Rishabh Singh, and Dawn Song. 2019 · 2019
Earlier work this paper cites.
On the ability of self-attention networks to recognize counter languages
Satwik Bhattamishra, Kabir Ahuja, and Navin Goyal. 2020 · 2020
Earlier work this paper cites.
Language models are few-shot learners
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, T. J. Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020 · 2020
Cited alongside, same era.
Transformers as soft reasoners over language
Peter Clark, Oyvind Tafjord, and Kyle Richardson. 2020 · 2020
Cited alongside, same era.
Theoretical limitations of self-attention in neural sequence models
Michael Hahn. 2020 · 2020
Cited alongside, same era.
Measuring compositional generalization: A comprehensive method on realistic data
Daniel Keysers, Nathanael Schärli, Nathan Scales, Hylke Buisman, Daniel Furrer, Sergii Kashubin, Nikola Momchev, Danila Sinopalnikov, Lukasz Stafiniak, Tibor Tihon, Dmitry Tsarkov, Xiao Wang, Marc van Zee, and Olivier Bousquet. 2020 · 2020
Compositional generalization and natural language variation: Can a semantic parsing approach handle both?
Peter Shaw, Ming-Wei Chang, Panupong Pasupat, and Kristina Toutanova. 2021 · 2021
Later among the works it cites.
Dyna-bAbI: unlocking bAbI’s potential with dynamic synthetic benchmarking
Ronen Tamari, Kyle Richardson, Aviad Sar-Shalom, Noam Kahlon, Nelson Liu, Reut Tsarfaty, and Dafna Shahaf. 2021 · 2021
Later among the works it cites.
Unobserved local structures make compositional generalization hard
Ben Bogin, Shivanshu Gupta, and Jonathan Berant. 2022 · 2022
Closest in time.
Understanding robust generalization in learning regular languages
Soham Dan, Osbert Bastani, and Dan Roth. 2022 · 2022
Closest in time.
Cross-task generalization via natural language crowdsourcing instructions
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
What do neural networks learn when trained with random labels?
Hartmut Maennel, Ibrahim M. Alabdulmohsin, Ilya O. Tolstikhin, Robert J. N. Baldock, Olivier Bousquet, Sylvain Gelly, and Daniel Keysers. 2020 · 2020
Cited alongside, same era.
Exploring the limits of transfer learning with a unified text-to-text transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020 · 2020
Cited alongside, same era.
Learning from task descriptions
Orion Weller, Nicholas Lourie, Matt Gardner, and Matthew E. Peters. 2020 · 2020
Cited alongside, same era.
Aligning ai with shared human values
Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Zheng Li, Dawn Xiaodong Song, and Jacob Steinhardt. 2021 · 2021
Cited alongside, same era.
Does pretraining for summarization require knowledge transfer?
Kundan Krishna, Jeffrey P. Bigham, and Zachary Chase Lipton. 2021 · 2021
Cited alongside, same era.
Formal language theory meets modern NLP
William Merrill. 2021 · 2021
Cited alongside, same era.
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022 · 2022
Closest in time.
Pushing the limits of rule reasoning in transformers through natural language satisfiability
Kyle Richardson and Ashish Sabharwal. 2022 · 2022
Closest in time.
Multitask prompted training enables zero-shot task generalization
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang A. Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M SAIFUL BARI, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Févry, Jason Alan Fries, Ryan Teehan, Stella Rose Biderman, Leo Gao, T. G. Owe Bers, Thomas Wolf, and Alexander M. Rush. 2022 · 2022
Closest in time.
Finetuned language models are zero-shot learners
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022 · 2022
Closest in time.
Byt5: Towards a token-free future with pre-trained byte-to-byte models
Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. 2022 · 2022
Closest in time.