Fetching the paper…
Reading the bibliography…
Large language models (LLMs) excel at few-shot in-context learning (ICL) -- learning from a few examples provided in context at inference, without any weight updates.
PDDL—The Planning Domain Definition Language, 1998
M. Ghallab, A. Howe, C. Knoblock, D. Mcdermott, A. Ram, M. Veloso, D. Weld, and D. Wilkins · 1998
Earlier work this paper cites.
ROUGE: A package for automatic evaluation of summaries
C.-Y. Lin · 2004
Earlier work this paper cites.
The fast downward planning system
M. Helmert · 2006
Earlier work this paper cites.
Scikit-learn: Machine learning in python
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, et al · 2011
Earlier work this paper cites.
Good debt or bad debt: Detecting semantic orientations in economic texts
P. Malo, A. Sinha, P. Korhonen, J. Wallenius, and P. Takala · 2014
Earlier work this paper cites.
chrf++: words helping character n-grams
M. Popović · 2017
Earlier work this paper cites.
Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization
S. Narayan, S. B. Cohen, and M. Lapata · 2018
Earlier work this paper cites.
Language models are unsupervised multitask learners
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al · 2019
Earlier work this paper cites.
Language models are few-shot learners
T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei · 2020
Earlier work this paper cites.
Scaling laws for neural language models
J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei · 2020
Earlier work this paper cites.
Pegasus: Pre-training with extracted gap-sentences for abstractive summarization
J. Zhang, Y. Zhao, M. Saleh, and P. Liu · 2020
Earlier work this paper cites.
Training verifiers to solve math word problems
K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman · 2021
Earlier work this paper cites.
Xl-sum: Large-scale multilingual abstractive summarization for 44 languages
T. Hasan, A. Bhattacharjee, M. S. Islam, K. S. Mubasshir, Y. Li, Y. Kang, M. S. Rahman, and R. Shahriyar · 2021
Earlier work this paper cites.
Measuring mathematical problem solving with the math dataset
D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt · 2021
Earlier work this paper cites.
Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity
Y. Lu, M. Bartolo, A. Moore, S. Riedel, and P. Stenetorp · 2021
Earlier work this paper cites.
An explanation of in-context learning as implicit bayesian inference
S. M. Xie, A. Raghunathan, P. Liang, and T. Ma · 2021
Earlier work this paper cites.
Exploring the landscape of distributional robustness for question answering models
A. Awadalla, M. Wortsman, G. Ilharco, S. Min, I. Magnusson, H. Hajishirzi, and L. Schmidt · 2022
Earlier work this paper cites.
Transformers generalize differently from information stored in context vs in weights, Oct. 2022
S. C. Y. Chan, I. Dasgupta, J. Kim, D. Kumaran, A. K. Lampinen, and F. Hill · 2022
Earlier work this paper cites.
Lift: Language-interfaced fine-tuning for non-language machine learning tasks
T. Dinh, Y. Zeng, R. Zhang, Z. Lin, M. Gira, S. Rajput, J.-y. Sohn, D. Papailiopoulos, and K. Lee · 2022
Earlier work this paper cites.
What can transformers learn in-context? a case study of simple function classes
S. Garg, D. Tsipras, P. S. Liang, and G. Valiant · 2022
Earlier work this paper cites.
The flores-101 evaluation benchmark for low-resource and multilingual machine translation
N. Goyal, C. Gao, V. Chaudhary, P. Chen, G. Wenzek, D. Ju, S. Krishnan, M. Ranzato, F. Guzmán, and A. Fan · 2022
Earlier work this paper cites.
H. J. Kim, H. Cho, J. Kim, T. Kim, K. M. Yoo, and S. Lee · 2022
Earlier work this paper cites.
Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning
H. Liu, D. Tam, M. Muqeeth, J. Mohta, T. Huang, M. Bansal, and C. A. Raffel · 2022
Earlier work this paper cites.
Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity
Y. Lu, M. Bartolo, A. Moore, S. Riedel, and P. Stenetorp · 2022
Earlier work this paper cites.
Rethinking the role of demonstrations: What makes in-context learning work?
S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer · 2022
Cited alongside, same era.
No language left behind: Scaling human-centered machine translation
M. A. NLLB Team · 2022
Cited alongside, same era.
In-context learning and induction heads
C. Olsson, N. Elhage, N. Nanda, N. Joseph, N. DasSarma, T. Henighan, B. Mann, A. Askell, Y. Bai, A. Chen, T. Conerly, D. Drain, D. Ganguli, Z. Hatfield-Dodds, D. Hernandez, S. Johnston, A. Jones, J. Kernion, L. Lovitt, K. Ndousse, D. Amodei, T. Brown, J. Clark, J. Kaplan, S. McCandlish, and C. Olah · 2022
Cited alongside, same era.
PDDL generators
J. Seipp, Á. Torralba, and J. Hoffmann · 2022
Cited alongside, same era.
Challenging big-bench tasks and whether chain-of-thought can solve them
M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou, et al · 2022
Cited alongside, same era.
Llama 2: Open foundation and fine-tuned chat models
H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al · 2023
Later among the works it cites.
Larger language models do in-context learning differently
J. Wei, J. Wei, Y. Tay, D. Tran, A. Webson, Y. Lu, X. Chen, H. Liu, D. Huang, D. Zhou, et al · 2023
Later among the works it cites.
Effective long-context scaling of foundation models
W. Xiong, J. Liu, I. Molybog, H. Zhang, P. Bhargava, R. Hou, L. Martin, R. Rungta, K. A. Sankararaman, B. Oguz, M. Khabsa, H. Fang, Y. Mehdad, S. Narang, K. Malik, A. Fan, S. Bhosale, S. Edunov, M. Lewis, S. Wang, and H. Ma · 2023
Later among the works it cites.
Scaling relationship on learning mathematical reasoning with large language models
Z. Yuan, H. Yuan, C. Li, G. Dong, C. Tan, and C. Zhou · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
J. von Oswald, E. Niklasson, E. Randazzo, J. Sacramento, A. Mordvintsev, A. Zhmoginov, and M. Vladymyrov · 2022
Cited alongside, same era.
Chain-of-thought prompting elicits reasoning in large language models
J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al · 2022
Cited alongside, same era.
Ground-truth labels matter: A deeper look into input-label demonstrations
K. M. Yoo, J. Kim, H. J. Kim, H. Cho, H. Jo, S. Lee, S. Lee, and T. Kim · 2022
Cited alongside, same era.
An in-depth look at gemini’s language abilities
S. N. Akter, Z. Yu, A. Muhamed, T. Ou, A. Bäuerle, Á. A. Cabrera, K. Dholakia, C. Xiong, and G. Neubig · 2023
Cited alongside, same era.
Buffet: Benchmarking large language models for few-shot cross-lingual transfer
A. Asai, S. Kudugunta, X. V. Yu, T. Blevins, H. Gonen, M. Reid, Y. Tsvetkov, S. Ruder, and H. Hajishirzi · 2023
Cited alongside, same era.
Understanding in-context learning in transformers and llms by learning to learn discrete functions
S. Bhattamishra, A. Patel, P. Blunsom, and V. Kanade · 2023
Cited alongside, same era.
Gemini: A family of highly capable multimodal models
G. Gemini Team · 2023
Cited alongside, same era.
Efficient attention via control variates
L. Zheng, J. Yuan, C. Wang, and L. Kong · 2023
Later among the works it cites.
Many-shot jailbreaking
C. Anil, E. Durmus, M. Sharma, J. Benton, S. Kundu, J. Batson, N. Rimsky, M. Tong, J. Mu, D. Ford, F. Mosconi, R. Agrawal, R. Schaeffer, N. Bashkansky, S. Svenningsen, M. Lambert, A. Radhakrishnan, C. Denison, E. J. Hubinger, Y. Bai, T. Bricken, T. Maxwell, N. Schiefer, J. Sully, A. Tamkin, T. Lanham, K. Nguyen, T. Korbak, J. Kaplan, D. Ganguli, S. R. Bowman, E. Perez, R. Grosse, and D. Duvenaud · 2024
Closest in time.
The claude 3 model family: Opus, sonnet, haiku
Anthropic · 2024
Closest in time.
In-Context Learning with Long-Context Models: An In-Depth Exploration, Apr. 2024
A. Bertsch, M. Ivgi, U. Alon, J. Berant, M. R. Gormley, and G. Neubig · 2024
Closest in time.
Context caching guide, 2024
G. A. for Developers · 2024
Closest in time.
Data Engineering for Scaling Language Models to 128K Context, Feb. 2024
Y. Fu, R. Panda, X. Niu, X. Yue, H. Hajishirzi, Y. Kim, and H. Peng · 2024
Closest in time.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
G. Gemini Team · 2024
Closest in time.
V-star: Training verifiers for self-taught reasoners
A. Hosseini, X. Yuan, N. Malkin, A. Courville, A. Sordoni, and R. Agarwal · 2024
Closest in time.
An information-theoretic analysis of in-context learning
H. J. Jeon, J. D. Lee, Q. Lei, and B. Van Roy · 2024
Closest in time.
LLMTest_NeedleInAHaystack
G. Kamradt · 2024
Closest in time.
DSPy: Compiling declarative language model calls into state-of-the-art pipelines
O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. V. A, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, H. Miller, M. Zaharia, and C. Potts · 2024
Closest in time.
Are human-generated demonstrations necessary for in-context learning?
R. Li, G. Wang, and J. Li · 2024
Closest in time.
Dual operating modes of in-context learning
Z. Lin and K. Lee · 2024
Closest in time.
Beyond human data: Scaling self-training for problem-solving with language models
A. Singh, J. D. Co-Reyes, R. Agarwal, A. Anand, P. Patil, X. Garcia, P. J. Liu, J. Harrison, J. Lee, K. Xu, A. T. Parisi, A. Kumar, A. A. Alemi, A. Rizkowsky, A. Nova, B. Adlam, B. Bohnet, G. F. Elsayed, H. Sedghi, I. Mordatch, I. Simpson, I. Gur, J. Snoek, J. Pennington, J. Hron, K. Kenealy, K. Swersky, K. Mahajan, L. A. Culp, L. Xiao, M. Bileschi, N. Constant, R. Novak, R. Liu, T. Warkentin, Y. Bansal, E. Dyer, B. Neyshabur, J. Sohl-Dickstein, and N. Fiedel · 2024
Closest in time.
R. Vacareanu, V.-A. Negru, V. Suciu, and M. Surdeanu · 2024
Closest in time.
On the planning abilities of large language models-a critical investigation
K. Valmeekam, M. Marquez, S. Sreedharan, and S. Kambhampati · 2024
Closest in time.
Large language models are latent variable models: Explaining and finding good demonstrations for in-context learning
X. Wang, W. Zhu, M. Saxon, M. Steyvers, and W. Y. Wang · 2024
Closest in time.
Layer-condensed kv cache for efficient inference of large language models
H. Wu and K. Tu · 2024
Closest in time.
Addressing order sensitivity of in-context demonstration examples in causal language models
Y. Xiang, H. Yan, L. Gui, and Y. He · 2024
Closest in time.
Benefits of transformer: In-context learning in linear regression tasks with unstructured data
Y. Xing, X. Lin, N. Suh, Q. Song, and G. Cheng · 2024
Closest in time.