Fetching the paper…
Reading the bibliography…
Transferring the reasoning capability from stronger large language models (LLMs) to smaller ones has been quite appealing, as smaller LLMs are more flexible to deploy with less expense.
Using “Annotator Rationales” to Improve Machine Learning for Text Categorization. In NAACL
Omar Zaidan, Jason Eisner, and Christine Piatko. 2007 · 2007
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al · 2015
Earlier work this paper cites.
An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition
George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, et al · 2015
Earlier work this paper cites.
Rationale-Augmented Convolutional Neural Networks for Text Classification. In EMNLP
Ye Zhang, Iain Marshall, and Byron C Wallace. 2016 · 2016
Earlier work this paper cites.
Right for the Right Reasons: Training Differentiable Models by Constraining Their Explanations
Andrew Slavin Ross, Michael C Hughes, and Finale Doshi-Velez. 2017 · 2017
Earlier work this paper cites.
e-SNLI: Natural Language Inference with Natural Language Explanations. In NeurIPS
Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018 · 2018
Earlier work this paper cites.
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018 · 2018
Earlier work this paper cites.
Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering. In EMNLP
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018 · 2018
Earlier work this paper cites.
On the efficacy of knowledge distillation. In ICCV
Jang Hyun Cho and Bharath Hariharan. 2019 · 2019
Earlier work this paper cites.
Learning from Dialogue after Deployment: Feed Yourself, Chatbot!. In ACL
Braden Hancock, Antoine Bordes, Pierre-Emmanuel Mazare, and Jason Weston. 2019 · 2019
Earlier work this paper cites.
PubMedQA: A Dataset for Biomedical Research Question Answering. In EMNLP
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019 · 2019
Earlier work this paper cites.
Explain Yourself! Leveraging Language Models for Commonsense Reasoning. In ACL
Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019 · 2019
Earlier work this paper cites.
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019 · 2019
Earlier work this paper cites.
PIQA: Reasoning about Physical Commonsense in Natural Language. In AAAI
Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. 2020 · 2020
Earlier work this paper cites.
Language models are few-shot learners. In NeurIPS
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Earlier work this paper cites.
Scaling laws for neural language models
J Kaplan, S McCandlish, T Henighan, et al · 2020
Earlier work this paper cites.
WT5?! Training Text-to-Text Models to Explain Their Predictions
Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, and Karishma Malkan. 2020 · 2020
Earlier work this paper cites.
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
C Raffel, N Shazeer, A Roberts, et al · 2020
Earlier work this paper cites.
Knowledge distillation: A survey
Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. 2021 · 2021
Earlier work this paper cites.
Ppt: Pre-trained prompt tuning for few-shot learning
Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang. 2021 · 2021
Earlier work this paper cites.
The Power of Scale for Parameter-Efficient Prompt Tuning. In EMNLP
B Lester, R Al-Rfou, and N Constant. 2021 · 2021
Earlier work this paper cites.
Prefix-Tuning: Optimizing Continuous Prompts for Generation. In ACL
X L Li and P Liang. 2021 · 2021
Earlier work this paper cites.
RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledge. In ACL Findings
Bill Yuchen Lin, Ziyi Wu, Yichi Yang, Dong-Ho Lee, and Xiang Ren. 2021 · 2021
Earlier work this paper cites.
P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2021 · 2021
Earlier work this paper cites.
Want To Reduce Labeling Cost? GPT-3 Can Help. In EMNLP Findings
Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, and Michael Zeng. 2021 · 2021
Earlier work this paper cites.
Measuring Association Between Labels and Free-Text Rationales. In EMNLP
Sarah Wiegreffe, Ana Marasovic, and Noah A Smith. 2021 · 2021
Earlier work this paper cites.
Retrieving and Reading: A Comprehensive Survey on Open-Domain Question Answering
Fengbin Zhu, Wenqiang Lei, Chao Wang, Jianming Zheng, Soujanya Poria, and Tat-S Chua. 2021 · 2021
Cited alongside, same era.
Scaling Instruction-Finetuned Language Models
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al · 2022
Cited alongside, same era.
Jacob Eisenstein, Daniel Andor, Bernd Bohnet, Michael Collins, and David Mimno. 2022 · 2022
Cited alongside, same era.
When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data. In ACL Workshop
Peter Hase and Mohit Bansal. 2022 · 2022
Cited alongside, same era.
Weize Liu, Guocong Li, Kai Zhang, Bang Du, Qiyuan Chen, Xuming Hu, Hongxia Xu, Jintai Chen, and Jian Wu. 2023 · 2023
Later among the works it cites.
What Makes it Ok to Set a Fire? Iterative Self-distillation of Contexts and Rationales for Disambiguating Defeasible Social and Moral Situations. In EMNLP Findings
Kavel Rao, Liwei Jiang, Valentina Pyatkin, Yuling Gu, Niket Tandon, Nouha Dziri, Faeze Brahman, and Yejin Choi. 2023 · 2023
Later among the works it cites.
Mededit: Model Editing for Medical Question Answering with External Knowledge Bases
Yucheng Shi, Shaochen Xu, et al · 2023
Later among the works it cites.
Stanford Alpaca: An Instruction-following LLaMA model
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023 · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Namgyu Ho, Laura Schmid, and Se-Young Yun. 2022 · 2022
Cited alongside, same era.
LoRA: Low-Rank Adaptation of Large Language Models. In ICLR
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022 · 2022
Cited alongside, same era.
Towards reasoning in large language models: A survey
Jie Huang and Kevin Chen-Chuan Chang. 2022 · 2022
Cited alongside, same era.
Large language models can self-improve
Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. 2022 · 2022
Cited alongside, same era.
Large Language Models are Zero-Shot Reasoners. In NeurIPS
Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022 · 2022
Cited alongside, same era.
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering. In NeurIPS
P Lu, S Mishra, T Xia, L Qiu, K-W Chang, S-C Zhu, O Tafjord, P Clark, and A Kalyan. 2022 · 2022
Cited alongside, same era.
Teaching Small Language Models to Reason
Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek, Eric Malmi, and Aliaksei Severyn. 2022 · 2022
Cited alongside, same era.
Evaluating Explanations: How Much Do Explanations from the Teacher Aid Students?. In ACL
Danish Pruthi, Rachit Bansal, Bhuwan Dhingra, Livio Baldini Soares, Michael Collins, Zachary C Lipton, Graham Neubig, and William W Cohen. 2022 · 2022
Cited alongside, same era.
Yijun Tian, Shichao Pei, Xiangliang Zhang, Chuxu Zhang, and Nitesh V Chawla. 2023a · 2023
Later among the works it cites.
Llama: Open and Efficient Foundation Language Models
Hugo Touvron, Thibaut Lavril, et al · 2023
Later among the works it cites.
Llama 2: Open Foundation and Fine-Tuned Chat Models
Hugo Touvron, Louis Martin, et al · 2023
Later among the works it cites.
Efficient large language models: A survey
Zhongwei Wan, Xin Wang, Che Liu, Samiul Alam, Yu Zheng, Zhongnan Qu, Shen Yan, Yi Zhu, Quanlu Zhang, Mosharaf Chowdhury, et al · 2023
Later among the works it cites.
HadSkip: Homotopic and Adaptive Layer Skipping of Pre-trained Language Models for Efficient Inference. In EMNLP Findings
Haoyu Wang, Yaqing Wang, Tianci Liu, Tuo Zhao, and Jing Gao. 2023 · 2023
Later among the works it cites.
A Survey on Large Language Models for Recommendation
Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, et al · 2023
Later among the works it cites.
Generating and Evaluating Tests for K-12 Students with Language Model Simulations: A Case Study on Sentence Reading Efficiency. In EMNLP
Eric Zelikman, Wanjing Ma, Jasmine Tran, Diyi Yang, Jason Yeatman, and Nick Haber. 2023 · 2023
Later among the works it cites.
A Survey of Large Language Models
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023 · 2023
Later among the works it cites.
Judging LLM-as-a-judge with MT-Bench and Chatbot Arena
Lianmin Zheng, Wei-Lin Chiang, et al · 2023
Later among the works it cites.
Generalized Knowledge Distillation for Auto-regressive Language Models. In The Twelfth International Conference on Learning Representations
Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. 2024 · 2024
Closest in time.
MinPrompt: Graph-based Minimal Prompt Data Augmentation for Few-shot Question Answering. In ACL
Xiusi Chen, Jyun-Yu Jiang, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, and Wei Wang. 2024 · 2024
Closest in time.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al · 2024
Closest in time.
MiniLLM: Knowledge distillation of large language models. In The Twelfth International Conference on Learning Representations
Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024 · 2024
Closest in time.
Towards Safer Large Language Models through Machine Unlearning
Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. 2024a · 2024
Closest in time.
Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning
Zhaoxuan Tan, Qingkai Zeng, Yijun Tian, Zheyuan Liu, Bing Yin, and Meng Jiang. 2024 · 2024
Closest in time.
Gemma 2: Improving open language models at a practical size
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al · 2024
Closest in time.
Graph neural prompting with large language models. In AAAI
Yijun Tian, Huan Song, Zichen Wang, Haozhu Wang, Ziqing Hu, Fang Wang, Nitesh V Chawla, and Panpan Xu. 2024 · 2024
Closest in time.
Language models don’t always say what they think: unfaithful explanations in chain-of-thought prompting
Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. 2024 · 2024
Closest in time.
Haoyu Wang, Tianci Liu, Ruirui Li, Monica Cheng, Tuo Zhao, and Jing Gao. 2024 · 2024
Closest in time.
LLMRec: Large Language Models with Graph Augmentation for Recommendation. In WSDM
W Wei, X Ren, J Tang, Q Wang, L Su, S Cheng, J Wang, D Yin, and C Huang. 2024 · 2024
Closest in time.
A survey on knowledge distillation of large language models
Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. 2024 · 2024
Closest in time.