Fetching the paper…
Reading the bibliography…
We observe that pre-trained large language models (LLMs) are capable of autoregressively completing complex token sequences -- from arbitrary ones procedurally generated by probabilistic context-free grammars (PCFG), to more rich spatial patterns found in the Abstraction and Reasoning Corpus (ARC), a general AI benchmark, prompted in the style of ASCII art.
Stimulus generalization in the learning of classifications
R. N. Shepard and J.-J. Chang · 1963
Earlier work this paper cites.
Varieties of perceptual independence
F. G. Ashby and J. T. Townsend · 1986
Earlier work this paper cites.
Robotic clicker training
F. Kaplan, P.-Y. Oudeyer, E. Kubinyi, and A. Miklósi · 2002
Earlier work this paper cites.
Neural machine translation of rare words with subword units
R. Sennrich, B. Haddow, and A. Birch · 2015
Earlier work this paper cites.
Can clicker training facilitate conditioning in dogs?
C. Chiandetti, S. Avella, E. Fongaro, and F. Cerri · 2016
Earlier work this paper cites.
Attention is all you need
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin · 2017
Earlier work this paper cites.
SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing
T. Kudo and J. Richardson · 2018
Earlier work this paper cites.
On the measure of intelligence
F. Chollet · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al · 2019
Earlier work this paper cites.
Q8bert: Quantized 8bit bert
O. Zafrir, G. Boudoukh, P. Izsak, and M. Wasserblat · 2019
Earlier work this paper cites.
Gen: A General-Purpose Probabilistic Programming System with Programmable Inference
M. F. Cusumano-Towner, F. A. Saad, A. K. Lew, and V. K. Mansinghka · 2019
Earlier work this paper cites.
Neural abstract reasoner
V. Kolev, B. Georgiev, and S. Penkov · 2020
Earlier work this paper cites.
Language models are few-shot learners
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al · 2020
Earlier work this paper cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu · 2020
Earlier work this paper cites.
Pretrained transformers improve out-of-distribution robustness
D. Hendrycks, X. Liu, E. Wallace, A. Dziedzic, R. Krishnan, and D. Song · 2020
Earlier work this paper cites.
https://www.kaggle.com/competitions/abstraction-and-reasoning-challenge/discussion/154597 , 2020
Abstraction and Rasoning Challenge 1st place solution · 2020
Earlier work this paper cites.
Compositionality decomposed: How do neural networks generalise?
D. Hupkes, V. Dankers, M. Mul, and E. Bruni · 2020
Earlier work this paper cites.
What Makes Good In-Context Examples for GPT-3
J. Liu, D. Shen, Y. Zhang, B. Dolan, L. Carin, and W. Chen · 2021
Earlier work this paper cites.
Calibrate before use: Improving few-shot performance of language models
Z. Zhao, E. Wallace, S. Feng, D. Klein, and S. Singh · 2021
Earlier work this paper cites.
S. Ferré · 2021
Earlier work this paper cites.
A Neurosymbolic Approach to Abstraction and Reasoning
S. Alford · 2021
Earlier work this paper cites.
On the opportunities and risks of foundation models
R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, et al · 2021
Earlier work this paper cites.
Evaluating large language models trained on code
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al · 2021
Earlier work this paper cites.
RoFormer: Enhanced Transformer with Rotary Position Embedding
J. Su, Y. Lu, S. Pan, A. Murtadha, B. Wen, and Y. Liu · 2021
Earlier work this paper cites.
Decision transformer: Reinforcement learning via sequence modeling
L. Chen, K. Lu, A. Rajeswaran, K. Lee, A. Grover, M. Laskin, P. Abbeel, A. Srinivas, and I. Mordatch · 2021
Earlier work this paper cites.
DreamCoder: Bootstrapping Inductive Program Synthesis with Wake-Sleep Library Learning
K. Ellis, C. Wong, M. Nye, M. Sablé-Meyer, L. Morales, L. Hewitt, L. Cary, A. Solar-Lezama, and J. B. Tenenbaum · 2021
Earlier work this paper cites.
Chain of thought prompting elicits reasoning in large language models
J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. Chi, Q. Le, and D. Zhou · 2022
Earlier work this paper cites.
Large language models are zero-shot reasoners
T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa · 2022
Earlier work this paper cites.
Challenging BIG-Bench tasks and whether chain-of-thought can solve them
M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou, et al · 2022
Earlier work this paper cites.
Selection-Inference: Exploiting large language models for interpretable logical reasoning
A. Creswell, M. Shanahan, and I. Higgins · 2022
Earlier work this paper cites.
Solving quantitative reasoning problems with language models
A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, et al · 2022
Cited alongside, same era.
Do as I can, not as I say: Grounding language in robotic affordances
M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, et al · 2022
Cited alongside, same era.
Language models as zero-shot planners: Extracting actionable knowledge for embodied agents
W. Huang, P. Abbeel, D. Pathak, and I. Mordatch · 2022
Cited alongside, same era.
Graphs, Constraints, and Search for the Abstraction and Reasoning Corpus
Y. Xu, E. B. Khalil, and S. Sanner · 2022
Cited alongside, same era.
Object-centric Compositional Imagination for Visual Abstract Reasoning
R. Assouel, P. Rodriguez, P. Taslakian, D. Vazquez, and Y. Bengio · 2022
Cited alongside, same era.
LLM+P: Empowering large language models with optimal planning proficiency
B. Liu, Y. Jiang, X. Zhang, Q. Liu, S. Zhang, J. Biswas, and P. Stone · 2023
Closest in time.
Parsel: A (de-) compositional framework for algorithmic reasoning with language models
E. Zelikman, Q. Huang, G. Poesia, N. D. Goodman, and N. Haber · 2023
Closest in time.
Text2Motion: From natural language instructions to feasible plans
K. Lin, C. Agia, T. Migimatsu, M. Pavone, and J. Bohg · 2023
Closest in time.
Code as Policies: Language model programs for embodied control
J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng · 2023
Closest in time.
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models
I. Singh, V. Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer · 2022
Cited alongside, same era.
Pretrained transformers as universal computation engines
K. Lu, A. Grover, P. Abbeel, and I. Mordatch · 2022
Cited alongside, same era.
RT-1: Robotics transformer for real-world control at scale
A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu, et al · 2022
Cited alongside, same era.
R3M: A universal visual representation for robot manipulation
S. Nair, A. Rajeswaran, V. Kumar, C. Finn, and A. Gupta · 2022
Cited alongside, same era.
LIFT: Language-interfaced fine-tuning for non-language machine learning tasks
T. Dinh, Y. Zeng, R. Zhang, Z. Lin, M. Gira, S. Rajput, J.-y. Sohn, D. Papailiopoulos, and K. Lee · 2022
Cited alongside, same era.
Data distributional properties drive emergent in-context learning in transformers
S. Chan, A. Santoro, A. Lampinen, J. Wang, A. Singh, P. Richemond, J. McClelland, and F. Hill · 2022
Cited alongside, same era.
What can transformers learn in-context? a case study of simple function classes
S. Garg, D. Tsipras, P. S. Liang, and G. Valiant · 2022
Cited alongside, same era.
Closest in time.
Reward Design with Language Models
M. Kwon, S. M. Xie, K. Bullard, and D. Sadigh · 2023
Closest in time.
Language Instructed Reinforcement Learning for Human-AI Coordination
H. Hu and D. Sadigh · 2023
Closest in time.
TidyBot: Personalized Robot Assistance with Large Language Models
J. Wu, R. Antonova, A. Kan, M. Lepert, A. Zeng, S. Song, J. Bohg, S. Rusinkiewicz, and T. Funkhouser · 2023
Closest in time.
An approach for solving tasks on the Abstract Reasoning Corpus
J. Ainooson, D. Sanyal, J. P. Michelson, Y. Yang, and M. Kunda · 2023
Closest in time.
The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domain
A. Moskvichev, V. V. Odouard, and M. Mitchell · 2023
Closest in time.
ARC Competition : EDA + PyTorch CNN
T. Paparaju · 2023
Closest in time.
What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning
J. Pan, T. Gao, H. Chen, and D. Chen · 2023
Closest in time.
Can wikipedia help offline reinforcement learning?
M. Reid, Y. Yamada, and S. S. Gu · 2023
Closest in time.
Real-world robot learning with masked visual pre-training
I. Radosavovic, T. Xiao, S. James, P. Abbeel, J. Malik, and T. Darrell · 2023
Closest in time.
Language-Driven Representation Learning for Robotics
S. Karamcheti, S. Nair, A. S. Chen, T. Kollar, C. Finn, D. Sadigh, and P. Liang · 2023
Closest in time.
Transformers learn in-context by gradient descent
J. Von Oswald, E. Niklasson, E. Randazzo, J. Sacramento, A. Mordvintsev, A. Zhmoginov, and M. Vladymyrov · 2023
Closest in time.
X. Wang, W. Zhu, and W. Y. Wang · 2023
Closest in time.
R. Anil, A. M. Dai, O. Firat, M. Johnson, D. Lepikhin, A. Passos, S. Shakeri, E. Taropa, P. Bailey, Z. Chen, et al · 2023
Closest in time.
Socratic Models: Composing zero-shot multimodal reasoning with language
A. Zeng, A. Wong, S. Welker, K. Choromanski, F. Tombari, A. Purohit, M. Ryoo, V. Sindhwani, J. Lee, V. Vanhoucke, et al · 2023
Closest in time.
Grounded Decoding: Guiding text generation with grounded models for robot control
W. Huang, F. Xia, D. Shah, D. Driess, A. Zeng, Y. Lu, P. Florence, I. Mordatch, S. Levine, K. Hausman, et al · 2023
Closest in time.
Voyager: An Open-Ended Embodied Agent with Large Language Models
G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar · 2023
Closest in time.
In-context reinforcement learning with algorithm distillation
M. Laskin, L. Wang, J. Oh, E. Parisotto, S. Spencer, R. Steigerwald, D. Strouse, S. Hansen, A. Filos, E. Brooks, et al · 2023
Closest in time.
MotionGPT: Finetuned LLMs are General-Purpose Motion Generators
Y. Zhang, D. Huang, B. Liu, S. Tang, Y. Lu, L. Chen, L. Bai, Q. Chu, N. Yu, and W. Ouyang · 2023
Closest in time.
Supervised Pretraining Can Learn In-Context Reinforcemenet Learning
J. N. Lee, A. Xie, A. Pacchiano, Y. Chandak, C. Finn, O. Nachum, and E. Brunskill · 2023
Closest in time.
Emergent analogical reasoning in large language models
T. Webb, K. J. Holyoak, and H. Lu · 2023
Closest in time.
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, O. Bousquet, Q. Le, and E. Chi · 2023
Closest in time.
Y. Xu, W. Li, P. Vaezipoor, S. Sanner, and E. B. Khalil · 2023
Closest in time.
Hypothesis search: Inductive reasoning with language models
R. Wang, E. Zelikman, G. Poesia, Y. Pu, N. Haber, and N. D. Goodman · 2023
Closest in time.
Physics of Language Models: Part 1, Context-Free Grammar
Z. Allen-Zhu and Y. Li · 2023
Closest in time.
PaLM-E: An embodied multimodal language model
D. Driess, F. Xia, M. S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, et al · 2023
Closest in time.