Propulsion: Steering LLM with Tiny Fine-Tuning

Socialiqa: Commonsense reasoning about social interactions

Original

Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. 2019 · 1904

Earlier work this paper cites.

Boolq: Exploring the surprising difficulty of natural yes/no questions

Original

Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019 · 1905

Earlier work this paper cites.

Roberta: A robustly optimized bert pretraining approach

Original

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1907

Earlier work this paper cites.

Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension

Original

Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019 · 1910

Earlier work this paper cites.

Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning

Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. 2022a · 1965

Earlier work this paper cites.

Quantization

Robert M. Gray and David L. Neuhoff. 1998 · 1998

Earlier work this paper cites.

Propulsion techniques: action and reaction

Peter J Turchi. 1998 · 1998

Earlier work this paper cites.

Language models are few-shot learners

Original

Ben Mann, N Ryder, M Subbiah, J Kaplan, P Dhariwal, A Neelakantan, P Shyam, G Sastry, A Askell, S Agarwal, et al. 2020 · 2005

Earlier work this paper cites.

Deberta: Decoding-enhanced bert with disentangled attention

Original

Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020 · 2006

Earlier work this paper cites.

Learning to solve arithmetic word problems with verb categorization

Mohammad Javad Hosseini, Hannaneh Hajishirzi, Oren Etzioni, and Nate Kushman. 2014 · 2014

Earlier work this paper cites.

Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees

Original

Yudong Chen and Martin J Wainwright. 2015 · 2015

Earlier work this paper cites.

Teaching machines to read and comprehend

Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015 · 2015

Earlier work this paper cites.

Parsing algebraic word problems into equations

Rik Koncel-Kedziorski, Hannaneh Hajishirzi, Ashish Sabharwal, Oren Etzioni, and Siena Dumas Ang. 2015 · 2015

Earlier work this paper cites.

Squad: 100,000+ questions for machine comprehension of text

Original

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016 · 2016

Earlier work this paper cites.

Solving general arithmetic word problems

Original

Subhro Roy and Dan Roth. 2016 · 2016

Earlier work this paper cites.

Think you have solved question answering? try arc, the ai2 reasoning challenge

Original

Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018 · 2018

Earlier work this paper cites.

Bert: Pre-training of deep bidirectional transformers for language understanding

Original

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018

Earlier work this paper cites.

Gradient descent happens in a tiny subspace

Original

Guy Gur-Ari, Daniel A Roberts, and Ethan Dyer. 2018 · 2018

Earlier work this paper cites.

Neural tangent kernel: Convergence and generalization in neural networks

Arthur Jacot, Franck Gabriel, and Clément Hongler. 2018 · 2018

Earlier work this paper cites.

Gradient-based meta-learning with learned layerwise metric and subspace

Yoonho Lee and Seungjin Choi. 2018 · 2018

Earlier work this paper cites.

Can a suit of armor conduct electricity? a new dataset for open book question answering

Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018 · 2018

Earlier work this paper cites.

Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization

Original

Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018 · 2018

Earlier work this paper cites.

Know what you don’t know: Unanswerable questions for squad

Original

Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018 · 2018

Earlier work this paper cites.

Adafactor: Adaptive learning rates with sublinear memory cost

Noam Shazeer and Mitchell Stern. 2018 · 2018

Earlier work this paper cites.

Glue: A multi-task benchmark and analysis platform for natural language understanding

Original

Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018 · 2018

Earlier work this paper cites.

Non-convex projected gradient descent for generalized low-rank tensor regression

Han Chen, Garvesh Raskutti, and Ming Yuan. 2019 · 2019

Earlier work this paper cites.

Parameter-efficient transfer learning for nlp

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019 · 2019

Earlier work this paper cites.

Hellaswag: Can a machine really finish your sentence?

Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019 · 2019

Earlier work this paper cites.

Piqa: Reasoning about physical commonsense in natural language

Jonathan Bisk et al. 2020 · 2020

Earlier work this paper cites.

Thrusters physical model formalization with regard to situational and identification factors of motion modes

Vitalii Budashko. 2020 · 2020

Cited alongside, same era.

Continual learning in low-rank orthogonal subspaces

Arslan Chaudhry, Naeemullah Khan, Puneet Dokania, and Philip Torr. 2020 · 2020

Cited alongside, same era.

Exploring versatile generative language model via parameter-efficient transfer learning

Zhaojiang Lin, Andrea Madotto, and Pascale Fung. 2020 · 2020

Cited alongside, same era.

On the dangers of stochastic parrots: Can language models be too big?

Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021 · 2021

Cited alongside, same era.

Training verifiers to solve math word problems, 2021

Original

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021 · 2021

Cited alongside, same era.

Gpt understands, too

Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2023 · 2023

Later among the works it cites.

Efficient fine-tuning large language models for knowledge-aware response planning

Minh Nguyen, KC Kishan, Toan Nguyen, Ankit Chadha, and Thuy Vu. 2023 · 2023

Later among the works it cites.

Lcs: Learning compressible subspaces for efficient, adaptive, real-time network compression at inference time

Elvis Nunez, Maxwell Horton, Anish Prabhu, Anurag Ranjan, Ali Farhadi, and Mohammad Rastegari. 2023 · 2023

Later among the works it cites.

Tied-lora: Enhacing parameter efficiency of lora with weight tying

Original

Adithya Renduchintala, Tugrul Konuk, and Oleksii Kuchaiev. 2023 · 2023

Later among the works it cites.

Risks and benefits of large language models for the environment

Matthias C Rillig, Marlene Ågerstrand, Mohan Bi, Kenneth A Gould, and Uli Sauerland. 2023 · 2023

Later among the works it cites.