Fetching the paper…
Reading the bibliography…
Prompting is one of the main ways to adapt a pretrained model to target tasks.
Meta-learning of sequential strategies
P. A. Ortega, J. X. Wang, M. Rowland, T. Genewein, Z. Kurth-Nelson, R. Pascanu, N. Heess, J. Veness, A. Pritzel, P. Sprechmann, S. M. Jayakumar, T. McGrath, K. J. Miller, M. G. Azar, I. Osband, N. C. Rabinowitz, A. György, S. Chiappa, S. Osindero, Y. W. Teh, H. van Hasselt, N. de Freitas, M. M. Botvinick, and S. Legg · 1905
Earlier work this paper cites.
Merging of opinions with increasing information
D. Blackwell and L. Dubins · 1962
Earlier work this paper cites.
Information theory, inference and learning algorithms
D. J. MacKay · 2003
Earlier work this paper cites.
Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability
M. Hutter · 2005
Earlier work this paper cites.
An introduction to Kolmogorov complexity and its applications , volume 3
M. Li, P. Vitányi, et al · 2008
Earlier work this paper cites.
A philosophical treatise of universal induction
S. Rathmanner and M. Hutter · 2011
Earlier work this paper cites.
Learning to reinforcement learn
J. X. Wang, Z. Kurth-Nelson, D. Tirumala, H. Soyer, J. Z. Leibo, R. Munos, C. Blundell, D. Kumaran, and M. Botvinick · 2016
Earlier work this paper cites.
Meta-trained agents implement bayes-optimal agents
V. Mikulik, G. Delétang, T. McGrath, T. Genewein, M. Martic, S. Legg, and P. Ortega · 2020
Earlier work this paper cites.
Lora: Low-rank adaptation of large language models, 2021
E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen · 2021
Earlier work this paper cites.
The power of scale for parameter-efficient prompt tuning
B. Lester, R. Al-Rfou, and N. Constant · 2021
Earlier work this paper cites.
Prefix-tuning: Optimizing continuous prompts for generation
X. L. Li and P. Liang · 2021
Earlier work this paper cites.
Exploring universal intrinsic task subspace via prompt tuning
Y. Qin, X. Wang, Y. Su, Y. Lin, N. Ding, J. Yi, W. Chen, Z. Liu, J. Li, L. Hou, et al · 2021
Earlier work this paper cites.
An explanation of in-context learning as implicit bayesian inference
S. M. Xie, A. Raghunathan, P. Liang, and T. Ma · 2021
Earlier work this paper cites.
What can transformers learn in-context? a case study of simple function classes
S. Garg, D. Tsipras, P. S. Liang, and G. Valiant · 2022
Earlier work this paper cites.
General-purpose in-context learning by meta-learning transformers
L. Kirsch, J. Harrison, J. Sohl-Dickstein, and L. Metz · 2022
Earlier work this paper cites.
Transformers can do bayesian inference
S. Müller, N. Hollmann, S. P. Arango, J. Grabocka, and F. Hutter · 2022
Earlier work this paper cites.
On transferability of prompt tuning for natural language processing
Y. Su, X. Wang, Y. Qin, C.-M. Chan, Y. Lin, H. Wang, K. Wen, Z. Liu, P. Li, J. Li, L. Hou, M. Sun, and J. Zhou · 2022
Earlier work this paper cites.
What learning algorithm is in-context learning? investigations with linear models
E. Akyürek, D. Schuurmans, J. Andreas, T. Ma, and D. Zhou · 2023
Cited alongside, same era.
Soft prompting might be a bug, not a feature
L. Bailey, G. Ahdritz, A. Kleiman, S. Swaroop, F. Doshi-Velez, and W. Pan · 2023
Cited alongside, same era.
Human-timescale adaptation in an open-ended task space
J. Bauer, K. Baumli, F. Behbahani, A. Bhoopchand, N. Bradley-Schmieg, M. Chang, N. Clay, A. Collister, V. Dasagi, L. Gonzalez, et al · 2023
Cited alongside, same era.
Sequential learning of neural networks for prequential mdl
J. Bornschein, Y. Li, and M. Hutter · 2023
Cited alongside, same era.
Promptbreeder: Self-referential self-improvement via prompt evolution, 2023
C. Fernando, D. Banarse, H. Michalewski, S. Osindero, and T. Rocktäschel · 2023
Cited alongside, same era.
Parameter-efficient fine-tuning for large models: A comprehensive survey
Z. Han, C. Gao, J. Liu, J. Zhang, and S. Q. Zhang · 2024
Later among the works it cites.
Compression via pre-trained transformers: A study on byte-level multimodal data
D. Heurtel-Depeiges, A. Ruoss, J. Veness, and T. Genewein · 2024
Later among the works it cites.
An Introduction to Universal Artificial Intelligence
M. Hutter, D. Quarel, and E. Catt · 2024
Later among the works it cites.
The broader spectrum of in-context learning, 2024
A. K. Lampinen, S. C. Y. Chan, A. K. Singh, and M. Shanahan · 2024
Later among the works it cites.
One step of gradient descent is provably the optimal in-context learner with one layer of linear self-attention
A. V. Mahankali, T. Hashimoto, and T. Ma · 2024
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
T. Genewein, G. Delétang, A. Ruoss, L. K. Wenliang, E. Catt, V. Dutordoir, J. Grau-Moya, L. Orseau, M. Hutter, and J. Veness · 2023
Cited alongside, same era.
In-context reinforcement learning with algorithm distillation
M. Laskin, L. Wang, J. Oh, E. Parisotto, S. Spencer, R. Steigerwald, D. Strouse, S. S. Hansen, A. Filos, E. Brooks, et al · 2023
Cited alongside, same era.
Memory augmented large language models are computationally universal
D. Schuurmans · 2023
Cited alongside, same era.
The transient nature of emergent in-context learning in transformers
A. Singh, S. Chan, T. Moskovitz, E. Grant, A. Saxe, and F. Hill · 2023
Cited alongside, same era.
Transformers learn in-context by gradient descent
J. Von Oswald, E. Niklasson, E. Randazzo, J. Sacramento, A. Mordvintsev, A. Zhmoginov, and M. Vladymyrov · 2023
Cited alongside, same era.
Large language models are latent variable models: Explaining and finding good demonstrations for in-context learning
X. Wang, W. Zhu, M. Saxon, M. Steyvers, and W. Y. Wang · 2023
Cited alongside, same era.
Many-shot in-context learning
R. Agarwal, A. Singh, L. Zhang, B. Bohnet, L. Rosias, S. Chan, B. Zhang, A. Anand, Z. Abbas, A. Nova, et al · 2024
Cited alongside, same era.
In-context learning through the bayesian prism
M. Panwar, K. Ahuja, and N. Goyal · 2024
Later among the works it cites.
When do prompting and prefix-tuning work? a theory of capabilities and limitations
A. Petrov, P. Torr, and A. Bibi · 2024
Later among the works it cites.
Autoregressive large language models are computationally universal, 2024
D. Schuurmans, H. Dai, and F. Zanini · 2024
Later among the works it cites.
Black-box prompt tuning with subspace learning
Y. Zheng, Z. Tan, P. Li, and Y. Liu · 2024
Later among the works it cites.
Algorithmic capabilities of random transformers
Z. Zhong and J. Andreas · 2024
Later among the works it cites.
Toward understanding in-context vs. in-weight learning
B. Chan, X. Chen, A. György, and D. Schuurmans · 2025
Closest in time.
A. K. Lampinen, A. Chaudhry, S. C. Y. Chan, C. Wild, D. Wan, A. Ku, J. Bornschein, R. Pascanu, M. Shanahan, and J. L. McClelland · 2025
Closest in time.
Balrog: Benchmarking agentic llm and vlm reasoning on games
D. Paglieri, B. Cupiał, S. Coward, U. Piterbarg, M. Wołczyk, A. Khan, E. Pignatelli, Ł. Kuciński, L. Pinto, R. Fergus, J. N. Foerster, J. Parker-Holder, and T. Rocktäschel · 2025
Closest in time.
Towards interpretable soft prompts
O. Patel, J. Wang, N. S. Nayak, S. Srinivas, and H. Lakkaraju · 2025
Closest in time.
LMAct: A benchmark for in-context imitation learning with long multimodal demonstrations
A. Ruoss, F. Pardo, H. Chan, B. Li, V. Mnih, and T. Genewein · 2025
Closest in time.
Strategy coopetition explains the emergence and transience of in-context learning, 2025
A. K. Singh, T. Moskovitz, S. Dragutinovic, F. Hill, S. C. Y. Chan, and A. M. Saxe · 2025
Closest in time.
Why is prompting hard? understanding prompts on binary sequence predictors
L. K. Wenliang, A. Ruoss, J. Grau-Moya, M. Hutter, and T. Genewein · 2025
Closest in time.