Refine Large Language Model Fine-tuning via Instruction Vector

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 1901

Earlier work this paper cites.

Hellaswag: Can a machine really finish your sentence?

Original

Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019 · 1905

Earlier work this paper cites.

Large language models in medicine

Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023 · 1940

Earlier work this paper cites.

Catastrophic interference in connectionist networks: The sequential learning problem

Michael McCloskey and Neal J Cohen. 1989 · 1989

Earlier work this paper cites.

Measuring massive multitask language understanding

Original

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020 · 2009

Earlier work this paper cites.

Interpretation and identification of causal mediation

Judea Pearl. 2013 · 2013

Earlier work this paper cites.

Learning end-to-end goal-oriented dialog

Original

Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2016 · 2016

Earlier work this paper cites.

Overcoming catastrophic forgetting in neural networks

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. 2017 · 2017

Earlier work this paper cites.

Learning without forgetting

Zhizhong Li and Derek Hoiem. 2017 · 2017

Earlier work this paper cites.

Think you have solved question answering? try arc, the ai2 reasoning challenge

Original

Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018 · 2018

Earlier work this paper cites.

Commonsenseqa: A question answering challenge targeting commonsense knowledge

Original

Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2018 · 2018

Earlier work this paper cites.

Language models are unsupervised multitask learners

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019 · 2019

Earlier work this paper cites.

Investigating gender bias in language models using causal mediation analysis

Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020 · 2020

Earlier work this paper cites.

{CPR}: Classifier-projection regularization for continual learning

Sungmin Cha, Hsiang Hsu, Taebaek Hwang, Flavio Calmon, and Taesup Moon. 2021 · 2021

Cited alongside, same era.

Lora: Low-rank adaptation of large language models

Original

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2021

Cited alongside, same era.

Continual learning for text classification with information disentanglement based regularization

Yufan Huang, Yanzhe Zhang, Jiaao Chen, Xuezhi Wang, and Diyi Yang. 2021 · 2021

Cited alongside, same era.

Training language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022 · 2022

Cited alongside, same era.

Fine-tuned language models are continual learners

Thomas Scialom, Tuhin Chakrabarty, and Smaranda Muresan. 2022 · 2022

Cited alongside, same era.

Function vectors in large language models

Original

Eric Todd, Millicent L Li, Arnab Sen Sharma, Aaron Mueller, Byron C Wallace, and David Bau. 2023 · 2023

Later among the works it cites.

Orthogonal subspace learning for language model continual learning

Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuanjing Huang. 2023a · 2023

Later among the works it cites.

Disc-lawllm: Fine-tuning large language models for intelligent legal services

Original

Shengbin Yue, Wei Chen, Siyuan Wang, Bingxuan Li, Chenchen Shen, Shujun Liu, Yuxuan Zhou, Yao Xiao, Song Yun, Xuanjing Huang, and Zhongyu Wei. 2023 · 2023

Later among the works it cites.

Investigating the catastrophic forgetting in multimodal large language models

Original

Yuexiang Zhai, Shengbang Tong, Xiao Li, Mu Cai, Qing Qu, Yong Jae Lee, and Yi Ma. 2023 · 2023

Later among the works it cites.

Med42 – evaluating fine-tuning strategies for medical llms: Full-parameter vs. parameter-efficient approaches