S3D: A Simple and Cost-Effective Self-Speculative Decoding Scheme for Low-Memory GPUs

Reducing transformer depth on demand with structured dropout

Original

Angela Fan, Edouard Grave, and Armand Joulin. 2019 · 1909

Earlier work this paper cites.

Zero: Memory optimizations toward training trillion parameter models

Original

Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020 · 1910

Earlier work this paper cites.

Huggingface’s transformers: State-of-the-art natural language processing

Original

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020 · 1910

Earlier work this paper cites.

Accelerating feedforward computation via parallel nonlinear equation solving

Original

Yang Song, Chenlin Meng, Renjie Liao, and Stefano Ermon. 2021 · 2002

Earlier work this paper cites.

ROUGE: A package for automatic evaluation of summaries

Chin-Yew Lin. 2004 · 2004

Earlier work this paper cites.

On the effect of dropping layers of pre-trained transformer models

Original

Hassan Sajjad, Fahim Dalvi, Nadir Durrani, and Preslav Nakov. 2022 · 2004

Earlier work this paper cites.

Teaching machines to read and comprehend

Original

Karl Moritz Hermann, Tomáš Kočiský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015 · 2015

Earlier work this paper cites.

Structured pruning of deep convolutional neural networks

Sajid Anwar, Kyuyeon Hwang, and Wonyong Sung. 2017 · 2017

Earlier work this paper cites.

BranchyNet: Fast inference via early exiting from deep neural networks

Original

Surat Teerapittayanon, Bradley McDanel, and H. T. Kung. 2017 · 2017

Earlier work this paper cites.

Learning sparse neural networks through l 0 l_{0} regularization

Original

Christos Louizos, Max Welling, and Diederik P. Kingma. 2018 · 2018

Earlier work this paper cites.

Improving language understanding by generative pre-training

Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018 · 2018

Earlier work this paper cites.

Blockwise parallel decoding for deep autoregressive models

Original

Mitchell Stern, Noam Shazeer, and Jakob Uszkoreit. 2018 · 2018

Earlier work this paper cites.

Universal transformers

Original

Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. 2019 · 2019

Earlier work this paper cites.

Decoupled weight decay regularization

Original

Ilya Loshchilov and Frank Hutter. 2019 · 2019

Earlier work this paper cites.

Evaluating large language models trained on code

Original

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021 · 2021

Earlier work this paper cites.

Lora: Low-rank adaptation of large language models

Original

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2021

Earlier work this paper cites.

Block pruning for faster transformers

Original

François Lagunas, Ella Charlaix, Victor Sanh, and Alexander M. Rush. 2021 · 2021

Earlier work this paper cites.

Instantaneous grammatical error correction with shallow aggressive decoding

Xin Sun, Tao Ge, Furu Wei, and Houfeng Wang. 2021 · 2021

Cited alongside, same era.

LLM.int8(): 8-bit matrix multiplication for transformers at scale

Original

Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022 · 2022

Cited alongside, same era.

Structured pruning learns compact and accurate models

Mengzhou Xia, Zexuan Zhong, and Danqi Chen. 2022 · 2022

Cited alongside, same era.

Gemini: A family of highly capable multimodal models

Original

Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023 · 2023

Cited alongside, same era.

Fast and robust early-exiting framework for autoregressive language models with synchronized parallel decoding

Sangmin Bae, Jongwoo Ko, Hwanjun Song, and Se-Young Yun. 2023 · 2023

Medusa: Simple LLM inference acceleration framework with multiple decoding heads

Original

Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D. Lee, Deming Chen, and Tri Dao. 2024 · 2024

Closest in time.

Sequoia: Scalable, robust, and hardware-aware speculative decoding

Original

Zhuoming Chen, Avner May, Ruslan Svirschevski, Yuhsun Huang, Max Ryabinin, Zhihao Jia, and Beidi Chen. 2024 · 2024

Closest in time.

LayerSkip: Enabling early exit inference and self-speculative decoding

Original

Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, Ahmed A Aly, Beidi Chen, and Carole-Jean Wu. 2024 · 2024

Closest in time.

Break the sequential dependency of LLM inference using lookahead decoding

Original

Yichao Fu, Peter Bailis, Ion Stoica, and Hao Zhang. 2024 · 2024

Closest in time.