Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones?

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 1901

Earlier work this paper cites.

Improving language understanding by generative pre-training

Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018 · 2018

Earlier work this paper cites.

Language models are unsupervised multitask learners

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019 · 2019

Earlier work this paper cites.

Compositionality decomposed: How do neural networks generalise?

Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni. 2020 · 2020

Earlier work this paper cites.

Measuring and improving consistency in pretrained language models

Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021 · 2021

Earlier work this paper cites.

Lora: Low-rank adaptation of large language models

Original

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2021

Earlier work this paper cites.

Finetuned language models are zero-shot learners

Original

Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021 · 2021

Earlier work this paper cites.

A survey on in-context learning

Original

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022 · 2022

Earlier work this paper cites.

GLM: General language model pretraining with autoregressive blank infilling

Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2022 · 2022

Earlier work this paper cites.

BECEL: Benchmark for consistency evaluation of language models

Myeongjun Jang, Deuk Sin Kwon, and Thomas Lukasiewicz. 2022 · 2022

Earlier work this paper cites.

Data contamination: From memorization to exploitation

Inbal Magar and Roy Schwartz. 2022 · 2022

Earlier work this paper cites.

Gpt-4 technical report

Original

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023 · 2023

Cited alongside, same era.

Qwen technical report

Original

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023 · 2023

Cited alongside, same era.

Baichuan 2: Open large-scale language models

Original

Baichuan. 2023 · 2023

Cited alongside, same era.

Weak-to-strong generalization: Eliciting strong capabilities with weak supervision

Original

Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschenbrenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, et al. 2023 · 2023

Cited alongside, same era.

Llama 2: Open foundation and fine-tuned chat models

Original

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 · 2023

Later among the works it cites.

Not all demonstration examples are equally beneficial: Reweighting demonstration examples for in-context learning

Zhe Yang, Damai Dai, Peiyi Wang, and Zhifang Sui. 2023 · 2023

Later among the works it cites.

Instruction-following evaluation for large language models

Original

Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023 · 2023

Later among the works it cites.

Llama 3 model card

AI@Meta. 2024 · 2024

Closest in time.

xcot: Cross-lingual instruction tuning for cross-lingual chain-of-thought reasoning

Original

Linzheng Chai, Jian Yang, Tao Sun, Hongcheng Guo, Jiaheng Liu, Bing Wang, Xinnian Liang, Jiaqi Bai, Tongliang Li, Qiyao Peng, and Zhoujun Li. 2024 · 2024