Fetching the paper…
Reading the bibliography…
The ability of language models to learn a task from a few examples in context has generated substantial interest.
The redundancy of english
C. E. Shannon · 1951
Earlier work this paper cites.
Structure-mapping: A theoretical framework for analogy
D. Gentner · 1983
Earlier work this paper cites.
Parallel structure: A source of facilitation in sentence comprehension
L. Frazier, L. Taft, T. Roeper, C. Clifton, and K. Ehrlich · 1984
Earlier work this paper cites.
Distributed representations, simple recurrent networks, and grammatical structure
J. L. Elman · 1991
Earlier work this paper cites.
Statistical learning by 8-month-old infants
J. R. Saffran, R. N. Aslin, and E. L. Newport · 1996
Earlier work this paper cites.
Long short-term memory
S. Hochreiter and J. Schmidhuber · 1997
Earlier work this paper cites.
The hierarchical hidden markov model: Analysis and applications
S. Fine, Y. Singer, and N. Tishby · 1998
Earlier work this paper cites.
Syntactic priming in language production
M. J. Pickering and H. P. Branigan · 1999
Earlier work this paper cites.
The effects of parallelism and prosody in the processing of gapping structures
K. Carlson · 2001
Earlier work this paper cites.
Dynamic topic models
D. M. Blei and J. D. Lafferty · 2006
Earlier work this paper cites.
Reinforcement learning for mapping instructions to actions
S. R. Branavan, H. Chen, L. Zettlemoyer, and R. Barzilay · 2009
Earlier work this paper cites.
Word sense disambiguation: A survey
R. Navigli · 2009
Earlier work this paper cites.
Analogy and relational reasoning
K. J. Holyoak · 2012
Earlier work this paper cites.
The winograd schema challenge
H. Levesque, E. Davis, and L. Morgenstern · 2012
Earlier work this paper cites.
RL 2 : Fast reinforcement learning via slow reinforcement learning
Y. Duan, J. Schulman, X. Chen, P. L. Bartlett, I. Sutskever, and P. Abbeel · 2016
Earlier work this paper cites.
A meta-analysis of syntactic priming in language production
K. Mahowald, A. James, R. Futrell, and E. Gibson · 2016
Earlier work this paper cites.
Listen, attend, and walk: Neural mapping of navigational instructions to action sequences
H. Mei, M. Bansal, and M. Walter · 2016
Earlier work this paper cites.
Meta-learning with memory-augmented neural networks
A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, and T. Lillicrap · 2016
Earlier work this paper cites.
Matching networks for one shot learning
O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra, et al · 2016
Earlier work this paper cites.
Learning to reinforcement learn
J. X. Wang, Z. Kurth-Nelson, D. Tirumala, H. Soyer, J. Z. Leibo, R. Munos, C. Blundell, D. Kumaran, and M. Botvinick · 2016
Earlier work this paper cites.
Model-agnostic meta-learning for fast adaptation of deep networks
C. Finn, P. Abbeel, and S. Levine · 2017
Earlier work this paper cites.
Improving language understanding with unsupervised learning
A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever · 2018
Earlier work this paper cites.
Reinforcement learning: An introduction
R. S. Sutton · 2018
Earlier work this paper cites.
Causal reasoning from meta-reinforcement learning
I. Dasgupta, J. Wang, S. Chiappa, J. Mitrovic, P. Ortega, D. Raposo, E. Hughes, P. Battaglia, M. Botvinick, and Z. Kurth-Nelson · 2019
Earlier work this paper cites.
Roberta: A robustly optimized bert pretraining approach
Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov · 2019
Earlier work this paper cites.
Meta-learning of sequential strategies
P. A. Ortega, J. X. Wang, M. Rowland, T. Genewein, Z. Kurth-Nelson, R. Pascanu, N. Heess, J. Veness, A. Pritzel, P. Sprechmann, et al · 2019
Earlier work this paper cites.
Continual lifelong learning with neural networks: A review
G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al · 2019
Earlier work this paper cites.
Training agents using upside-down reinforcement learning
R. K. Srivastava, P. Shyam, F. Mutz, W. Jaśkowski, and J. Schmidhuber · 2019
Earlier work this paper cites.
Language models are few-shot learners
T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al · 2020
Earlier work this paper cites.
Shortcut learning in deep neural networks
R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann · 2020
Earlier work this paper cites.
Environmental drivers of systematicity and generalization in a situated agent
F. Hill, A. Lampinen, R. Schneider, S. Clark, M. Botvinick, J. L. McClelland, and A. Santoro · 2020
Earlier work this paper cites.
Scaling laws for neural language models
J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei · 2020
Earlier work this paper cites.
Meta-learning without memorization
M. Yin, G. Tucker, M. Zhou, S. Levine, and C. Finn · 2020
Earlier work this paper cites.
Decision transformer: Reinforcement learning via sequence modeling
L. Chen, K. Lu, A. Rajeswaran, K. Lee, A. Grover, M. Laskin, P. Abbeel, A. Srinivas, and I. Mordatch · 2021
Earlier work this paper cites.
Grounded language learning fast and slow
F. Hill, O. Tieleman, T. von Glehn, N. Wong, H. Merzic, and S. Clark · 2021
Cited alongside, same era.
Analysis and evaluation of language models for word sense disambiguation
D. Loureiro, K. Rezaee, M. T. Pilehvar, and J. Camacho-Collados · 2021
Cited alongside, same era.
Language conditioned imitation learning over unstructured data
C. Lynch and P. Sermanet · 2021
Cited alongside, same era.
Prompt programming for large language models: Beyond the few-shot paradigm
L. Reynolds and K. McDonell · 2021
Cited alongside, same era.
Winogrande: An adversarial winograd schema challenge at scale
K. Sakaguchi, R. L. Bras, C. Bhagavatula, and Y. Choi · 2021
Cited alongside, same era.
Meta-learning in natural and artificial intelligence
J. X. Wang · 2021
Cited alongside, same era.
Emergent analogical reasoning in large language models
T. Webb, K. J. Holyoak, and H. Lu · 2023
Later among the works it cites.
Larger language models do in-context learning differently
J. Wei, J. Wei, Y. Tay, D. Tran, A. Webson, Y. Lu, X. Chen, H. Liu, D. Huang, D. Zhou, et al · 2023
Later among the works it cites.
Causal parrots: Large language models may talk causality but are not causal
M. Zečević, M. Willig, D. S. Dhami, and K. Kersting · 2023
Later among the works it cites.
A definition of continual reinforcement learning
D. Abel, A. Barreto, B. Van Roy, D. Precup, H. P. van Hasselt, and S. Singh · 2024
Closest in time.
Many-shot in-context learning
R. Agarwal, A. Singh, L. M. Zhang, B. Bohnet, L. Rosias, S. C. Chan, B. Zhang, A. Faust, and H. Larochelle · 2024
Closest in time.
The surprising effectiveness of test-time training for abstract reasoning, 2024
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agents
J. X. Wang, M. King, N. P. M. Porcel, Z. Kurth-Nelson, T. Zhu, C. Deck, P. Choy, M. Cassin, M. Reynolds, H. F. Song, et al · 2021
Cited alongside, same era.
What learning algorithm is in-context learning? investigations with linear models
E. Akyürek, D. Schuurmans, J. Andreas, T. Ma, and D. Zhou · 2022
Cited alongside, same era.
Flamingo: a visual language model for few-shot learning
J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, et al · 2022
Cited alongside, same era.
Transformers generalize differently from information stored in context vs in weights
S. C. Chan, I. Dasgupta, J. Kim, D. Kumaran, A. K. Lampinen, and F. Hill · 2022
Cited alongside, same era.
A survey on in-context learning
Q. Dong, L. Li, D. Dai, C. Zheng, J. Ma, R. Li, H. Xia, J. Xu, Z. Wu, T. Liu, et al · 2022
Cited alongside, same era.
Can language models learn from explanations in context?
A. Lampinen, I. Dasgupta, S. Chan, K. Mathewson, M. Tessler, A. Creswell, J. McClelland, J. Wang, and F. Hill · 2022
Cited alongside, same era.
E. Akyürek, M. Damani, L. Qiu, H. Guo, Y. Kim, and J. Andreas · 2024
Closest in time.
Many-shot jailbreaking
C. Anil, E. Durmus, N. Rimsky, M. Sharma, J. Benton, S. Kundu, J. Batson, M. Tong, J. Mu, D. J. Ford, et al · 2024
Closest in time.
In-context learning with long-context models: An in-depth exploration
A. Bertsch, M. Ivgi, U. Alon, J. Berant, M. R. Gormley, and G. Neubig · 2024
Closest in time.
Understanding in-context learning in transformers and llms by learning to learn discrete functions
S. Bhattamishra, A. Patel, P. Blunsom, and V. Kanade · 2024
Closest in time.
Contextualizing predictive minds
M. V. Butz, M. Mittenbühler, S. Schwöbel, A. Achimova, C. Gumbsch, S. Otte, and S. Kiebel · 2024
Closest in time.
Parallel structures in pre-training data yield in-context learning
Y. Chen, C. Zhao, Z. Yu, K. McKeown, and H. He · 2024
Closest in time.
Trained transformer classifiers generalize and exhibit benign overfitting in-context
S. Frei and G. Vardi · 2024
Closest in time.
Large language models are zero-shot time series forecasters
N. Gruver, M. Finzi, S. Qiu, and A. G. Wilson · 2024
Closest in time.
Passive learning of active causal strategies in agents and language models
A. Lampinen, S. Chan, I. Dasgupta, A. Nam, and J. Wang · 2024
Closest in time.
Racing thoughts: Explaining large language model contextualization errors
M. A. Lepori, M. Mozer, and A. Ghandeharioun · 2024
Closest in time.
Evaluating the robustness of analogical reasoning in large language models
M. Lewis and M. Mitchell · 2024
Closest in time.
An incomplete loop: Instruction inference, instruction following, and in-context learning in language models
E. Liu, G. Neubig, and J. Andreas · 2024
Closest in time.
Language models learn rare phenomena from less rare phenomena: The case of the missing aanns
K. Misra and K. Mahowald · 2024
Closest in time.
Competition dynamics shape algorithmic phases of in-context learning
C. F. Park, E. S. Lubana, I. Pres, and H. Tanaka · 2024
Closest in time.
Can language models perform implicit bayesian inference over user preference states?
L. Qiu, F. Sha, K. R. Allen, Y. Kim, T. Linzen, and S. van Steenkiste · 2024
Closest in time.
Compositional capabilities of autoregressive transformers: A study on synthetic, interpretable tasks
R. Ramesh, E. S. Lubana, M. Khona, R. P. Dick, and H. Tanaka · 2024
Closest in time.
Revisiting dynamic evaluation: Online adaptation for large language models
A. Rannen-Triki, J. Bornschein, R. Pascanu, M. Hutter, A. György, A. Galashov, Y. W. Teh, and M. K. Titsias · 2024
Closest in time.
Pretraining task diversity and the emergence of non-bayesian in-context learning for regression
A. Raventós, M. Paul, F. Chen, and S. Ganguli · 2024
Closest in time.
The prompt report: A systematic survey of prompting techniques
S. Schulhoff, M. Ilie, N. Balepur, K. Kahadze, A. Liu, C. Si, Y. Li, A. Gupta, H. Han, S. Schulhoff, et al · 2024
Closest in time.
The transient nature of emergent in-context learning in transformers
A. Singh, S. Chan, T. Moskovitz, E. Grant, A. Saxe, and F. Hill · 2024
Closest in time.
Tokenization counts: the impact of tokenization on arithmetic in frontier llms
A. K. Singh and D. Strouse · 2024
Closest in time.
Schema-learning and rebinding as mechanisms of in-context learning and emergence
S. Swaminathan, A. Dedieu, R. Vasudeva Raju, M. Shanahan, M. Lazaro-Gredilla, and D. George · 2024
Closest in time.
J. Treutlein, D. Choi, J. Betley, S. Marks, C. Anil, R. Grosse, and O. Evans · 2024
Closest in time.
Evidence from counterfactual tasks supports emergent analogical reasoning in large language models
T. Webb, K. J. Holyoak, and H. Lu · 2024
Closest in time.
Understanding in-context learning from repetitions
J. Yan, J. Xu, C. Song, C. Wu, Y. Li, and Y. Zhang · 2024
Closest in time.
Trained transformers learn linear models in-context
R. Zhang, S. Frei, and P. L. Bartlett · 2024
Closest in time.
Do different prompting methods yield a common task representation in language models?
G. Davidson, T. M. Gureckis, B. M. Lake, and A. Williams · 2025
Closest in time.
Strategy coopetition explains the emergence and transience of in-context learning
A. K. Singh, T. Moskovitz, S. Dragutinovic, F. Hill, S. C. Chan, and A. M. Saxe · 2025
Closest in time.
Which attention heads matter for in-context learning?
K. Yin and J. Steinhardt · 2025
Closest in time.