Fetching the paper…
Reading the bibliography…
In this work, we (1) introduce Curriculum Instruction Tuning, (2) explore the potential advantages of employing diverse curriculum strategies, and (3) delineate a synthetic instruction-response generation framework that complements our theoretical approach.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 1901
Earlier work this paper cites.
Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain
Benjamin S Bloom, Max D Engelhart, Edward J Furst, Walker H Hill, and David R Krathwohl. 1956 · 1956
Earlier work this paper cites.
Mind in society: The development of higher mental processes (e. rice, ed. & trans.)
Lev S Vygotsky. 1978 · 1978
Earlier work this paper cites.
Cognitive load during problem solving: Effects on learning
John Sweller. 1988 · 1988
Earlier work this paper cites.
Catastrophic interference in connectionist networks: The sequential learning problem
Michael McCloskey and Neal J Cohen. 1989 · 1989
Earlier work this paper cites.
Learning and development in neural networks: The importance of starting small
Jeffrey L Elman. 1993 · 1993
Earlier work this paper cites.
Approach to Learning and Assessment in Physics
Leslie Dickie. 1994 · 1994
Earlier work this paper cites.
Crop: Communities resolving our problems-community design for 21st century learning
Robert S Houghton. 1997 · 1997
Earlier work this paper cites.
Cognitive architecture and instructional design
John Sweller, Jeroen JG Van Merrienboer, and Fred GWC Paas. 1998 · 1998
Earlier work this paper cites.
Bloom’s taxonomy interpreted for mathematics
Lindsey Shorser. 1999 · 1999
Earlier work this paper cites.
Scaffolding language, scaffolding learning
Pauline Gibbons. 2002 · 2002
Earlier work this paper cites.
A revision of bloom’s taxonomy: An overview
David R Krathwohl. 2002 · 2002
Earlier work this paper cites.
Cognitive load theory and instructional design: Recent developments
Fred Paas, Alexander Renkl, and John Sweller. 2003 · 2003
Earlier work this paper cites.
Deep learning and education for sustainability
Kevin Warburton. 2003 · 2003
Earlier work this paper cites.
Applying the revised bloom’s taxonomy to a medical-surgical nursing lesson
Whei Ming Su, Paul J Osisek, and Beth Starnes. 2004 · 2004
Earlier work this paper cites.
Guidelines for cognitively efficient multimedia learning tools: educational strategies, cognitive load, and interface design
Tiffany Grunwald and Charisse Corsbie-Massay. 2006 · 2006
Earlier work this paper cites.
Enhancing instructional efficiency of interactive e-learning environments: A cognitive load perspective
Slava Kalyuga. 2007 · 2007
Earlier work this paper cites.
The spiral curriculum: implications for online learning
Kenneth Masters and Trevor Gibbs. 2007 · 2007
Earlier work this paper cites.
Curriculum learning
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009 · 2009
Earlier work this paper cites.
A cognitive load approach to collaborative learning: United brains for complex tasks
Femke Kirschner, Fred Paas, and Paul A Kirschner. 2009 · 2009
Earlier work this paper cites.
Flexible shaping: How learning in small steps helps
Kai A Krueger and Peter Dayan. 2009 · 2009
Earlier work this paper cites.
The effects of interleaved practice
Kelli Taylor and Doug Rohrer. 2010 · 2010
Earlier work this paper cites.
Recurrent neural network language model adaptation with curriculum learning
Yangyang Shi, Martha Larson, and Catholijn M Jonker. 2015 · 2015
Earlier work this paper cites.
The lambada dataset: Word prediction requiring a broad discourse context
Denis Paperno, German David Kruszewski Martel, Angeliki Lazaridou, Ngoc Pham Quan, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda Torrent, Fernández Raquel, et al. 2016 · 2016
Earlier work this paper cites.
Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension
Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017 · 2017
Earlier work this paper cites.
Transforming bloom’s taxonomy into classroom practice: a practical yet comprehensive approach to promote critical reading and student participation
Daniel M Mulcare and Allan Shwedel. 2017 · 2017
Cited alongside, same era.
Think you have solved question answering? try arc, the ai2 reasoning challenge
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018 · 2018
Cited alongside, same era.
Sentence-bert: Sentence embeddings using siamese bert-networks
Nils Reimers and Iryna Gurevych. 2019 · 2019
Cited alongside, same era.
CommonsenseQA: A question answering challenge targeting commonsense knowledge
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019 · 2019
Cited alongside, same era.
Hierarchical human-like strategy for aspect-level sentiment classification with sentiment linguistic knowledge and reinforcement learning
Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023 · 2023
Closest in time.
Citing: Large language models create curriculum for instruction tuning
Tao Feng, Zifeng Wang, and Jimeng Sun. 2023 · 2023
Closest in time.
Flacuna: Unleashing the problem solving power of vicuna using flan fine-tuning
Deepanway Ghosal, Yew Ken Chia, Navonil Majumder, and Soujanya Poria. 2023 · 2023
Closest in time.
Instructed to bias: Instruction-tuned language models exhibit emergent cognitive bias
Itay Itzhak, Gabriel Stanovsky, Nir Rosenfeld, and Yonatan Belinkov. 2023 · 2023
Closest in time.
Active instruction tuning: Improving cross-task generalization by training on prompt sensitive tasks
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Min Yang, Qingnan Jiang, Ying Shen, Qingyao Wu, Zhou Zhao, and Wei Zhou. 2019 · 2019
Cited alongside, same era.
Hellaswag: Can a machine really finish your sentence?
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019 · 2019
Cited alongside, same era.
Piqa: Reasoning about physical commonsense in natural language
Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. 2020 · 2020
Cited alongside, same era.
Measuring massive multitask language understanding
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020 · 2020
Cited alongside, same era.
Theory of curriculum learning, with convex loss functions
Daphna Weinshall and Dan Amir. 2020 · 2020
Cited alongside, same era.
When do curricula work?
Xiaoxia Wu, Ethan Dyer, and Behnam Neyshabur. 2020 · 2020
Cited alongside, same era.
Curriculum learning for natural language understanding
Benfeng Xu, Licheng Zhang, Zhendong Mao, Quan Wang, Hongtao Xie, and Yongdong Zhang. 2020 · 2020
Cited alongside, same era.
Pre-trained models: Past, present and future
Xu Han, Zhengyan Zhang, Ning Ding, Yuxian Gu, Xiao Liu, Yuqi Huo, Jiezhong Qiu, Yuan Yao, Ao Zhang, Liang Zhang, et al. 2021 · 2021
Cited alongside, same era.
Po-Nien Kung, Fan Yin, Di Wu, Kai-Wei Chang, and Nanyun Peng. 2023 · 2023
Closest in time.
The flan collection: Designing data and methods for effective instruction tuning
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. 2023 · 2023
Closest in time.
Instag: Instruction tagging for analyzing supervised fine-tuning of large language models
Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou. 2023 · 2023
Closest in time.
Lingbo Mo, Boshi Wang, Muhao Chen, and Huan Sun. 2023 · 2023
Closest in time.
Llm evaluation scores
MosaicML. 2023 · 2023
Closest in time.
Octopack: Instruction tuning code large language models
Niklas Muennighoff, Qian Liu, Armel Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro von Werra, and Shayne Longpre. 2023 · 2023
Closest in time.
Orca: Progressive learning from complex explanation traces of gpt-4
Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023 · 2023
Closest in time.
OpenAI. 2023 · 2023
Closest in time.
Toolllm: Facilitating large language models to master 16000+ real-world apis
Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2023 · 2023
Closest in time.
Probing the psychology of ai models
Richard Shiffrin and Melanie Mitchell. 2023 · 2023
Closest in time.
Dynamics of instruction tuning: Each ability of large language models has its own growth pace
Chiyu Song, Zhanchao Zhou, Jianhao Yan, Yuejiao Fei, Zhenzhong Lan, and Yue Zhang. 2023 · 2023
Closest in time.
Stanford alpaca: An instruction-following llama model
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023 · 2023
Closest in time.
Llama 2: Open foundation and fine-tuned chat models
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 · 2023
Closest in time.
Bioinstruct: Instruction tuning of large language models for biomedical natural language processing
Hieu Tran, Zhichao Yang, Zonghai Yao, and Hong Yu. 2023 · 2023
Closest in time.
Max J. van Duijn, Bram M. A. van Dijk, Tom Kouwenhoven, Werner de Valk, Marco R. Spruit, and Peter van der Putten. 2023 · 2023
Closest in time.
Darwin series: Domain specific large language models for natural science
Tong Xie, Yuwei Wan, Wei Huang, Zhenyu Yin, Yixuan Liu, Shaozhou Wang, Qingyuan Linghu, Chunyu Kit, Clara Grazian, Wenjie Zhang, Imran Razzak, and Bram Hoex. 2023 · 2023
Closest in time.
Wizardlm: Empowering large language models to follow complex instructions
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023 · 2023
Closest in time.
Kola: Carefully benchmarking world knowledge of large language models
Jifan Yu, Xiaozhi Wang, Shangqing Tu, Shulin Cao, Daniel Zhang-Li, Xin Lv, Hao Peng, Zijun Yao, Xiaohan Zhang, Hanming Li, et al. 2023 · 2023
Closest in time.
Judging llm-as-a-judge with mt-bench and chatbot arena
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023 · 2023
Closest in time.
Lima: Less is more for alignment
Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2023 · 2023
Closest in time.