Let's Learn Step by Step: Enhancing In-Context Learning Ability with Curriculum Learning

Curriculum learning

Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009 · 2009

Earlier work this paper cites.

Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction

Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi. 2018 · 2018

Earlier work this paper cites.

Structural scaffolds for citation intent classification in scientific publications

Arman Cohan, Waleed Ammar, Madeleine van Zuylen, and Field Cady. 2019 · 2019

Earlier work this paper cites.

Teacher algorithms for curriculum learning of deep rl in continuously parameterized environments

Rémy Portelas, Cédric Colas, Katja Hofmann, and Pierre-Yves Oudeyer. 2020 · 2020

Earlier work this paper cites.

Curriculum learning for natural language understanding

Benfeng Xu, Licheng Zhang, Zhendong Mao, Quan Wang, Hongtao Xie, and Yongdong Zhang. 2020 · 2020

Earlier work this paper cites.

Pre-training a BERT with curriculum learning by increasing block-size of input text

Koichi Nagatsuka, Clifford Broni-Bediako, and Masayasu Atsumi. 2021 · 2021

Earlier work this paper cites.

A survey for in-context learning

Original

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022 · 2022

Earlier work this paper cites.

What makes good in-context examples for GPT-3?

Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2022 · 2022

Earlier work this paper cites.

Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity

Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022 · 2022

Cited alongside, same era.

Training language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022 · 2022

Cited alongside, same era.

SciNLI: A corpus for natural language inference on scientific text

Mobashir Sadat and Cornelia Caragea. 2022 · 2022

Cited alongside, same era.

Selective annotation makes language models better few-shot learners

Original

Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, et al. 2022 · 2022

Cited alongside, same era.

Hierarchical curriculum learning for AMR parsing

Peiyi Wang, Liang Chen, Tianyu Liu, Damai Dai, Yunbo Cao, Baobao Chang, and Zhifang Sui. 2022 · 2022

Cited alongside, same era.

In-sample curriculum learning by sequence completion for natural language generation

Qi Jia, Yizhu Liu, Haifeng Tang, and Kenny Zhu. 2023 · 2023

Later among the works it cites.

Instruction tuning with human curriculum

Original

Bruce W Lee, Hyunsoo Cho, and Kang Min Yoo. 2023 · 2023

Later among the works it cites.

Llama 2: Open foundation and fine-tuned chat models

Original

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 · 2023

Later among the works it cites.

Efficient large language models: A survey

Original

Zhongwei Wan, Xin Wang, Che Liu, Samiul Alam, Yu Zheng, et al. 2023 · 2023

Later among the works it cites.

Self-adaptive in-context learning: An information compression perspective for in-context example selection and ordering