Early Exit Is a Natural Capability in Transformer-based Models: An Empirical Study on Early Exit without Joint Optimization

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 1901

Earlier work this paper cites.

Learning deep transformer models for machine translation

Original

Qiang Wang, Bei Li, Tong Xiao, Jingbo Zhu, Changliang Li, Derek F Wong, and Lidia S Chao. 2019 · 1906

Earlier work this paper cites.

Once-for-all: Train one network and specialize it for efficient deployment

Original

Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. 2019 · 1908

Earlier work this paper cites.

Depth-adaptive transformer

Original

Maha Elbayad, Jiatao Gu, Edouard Grave, and Michael Auli. 2019 · 1910

Earlier work this paper cites.

Rouge: A package for automatic evaluation of summaries

Chin-Yew Lin. 2004 · 2004

Earlier work this paper cites.

The right tool for the job: Matching model and instance complexities

Original

Roy Schwartz, Gabriel Stanovsky, Swabha Swayamdipta, Jesse Dodge, and Noah A Smith. 2020 · 2004

Earlier work this paper cites.

Deebert: Dynamic early exiting for accelerating bert inference

Original

Ji Xin, Raphael Tang, Jaejun Lee, Yaoliang Yu, and Jimmy Lin. 2020 · 2004

Earlier work this paper cites.

Comet: A neural framework for mt evaluation

Original

Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020 · 2009

Earlier work this paper cites.

Transformer feed-forward layers are key-value memories

Original

Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2020 · 2012

Earlier work this paper cites.

Conditional deep learning for energy-efficient and enhanced pattern recognition

Priyadarshini Panda, Abhronil Sengupta, and Kaushik Roy. 2016 · 2016

Earlier work this paper cites.

Branchynet: Fast inference via early exiting from deep neural networks

Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. 2016 · 2016

Earlier work this paper cites.

Multi-scale dense networks for resource efficient image classification

Original

Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017 · 2017

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Cited alongside, same era.

Bert loses patience: Fast and robust inference with early exit

Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian McAuley, Ke Xu, and Furu Wei. 2020 · 2020

Cited alongside, same era.

Faster depth-adaptive transformers

Yijin Liu, Fandong Meng, Jie Zhou, Yufeng Chen, and Jinan Xu. 2021 · 2021

Cited alongside, same era.

Consistent accelerated inference via confident adaptive transformers

Original

Tal Schuster, Adam Fisch, Tommi Jaakkola, and Regina Barzilay. 2021 · 2021

Cited alongside, same era.

Early exiting with ensemble internal classifiers

Original

Tianxiang Sun, Yunhua Zhou, Xiangyang Liu, Xinyu Zhang, Hao Jiang, Zhao Cao, Xuanjing Huang, and Xipeng Qiu. 2021 · 2021

Cited alongside, same era.

Sorted llama: Unlocking the potential of intermediate layers of large language models for dynamic inference using sorted fine-tuning (soft)

Original

Parsa Kavehzadeh, Mojtaba Valipour, Marzieh Tahaei, Ali Ghodsi, Boxing Chen, and Mehdi Rezagholizadeh. 2023 · 2023

Later among the works it cites.

Fast inference from transformers via speculative decoding

Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023 · 2023

Later among the works it cites.

A global past-future early exit method for accelerating inference of pre-trained language models

Kaiyuan Liao, Yi Zhang, Xuancheng Ren, Qi Su, Xu Sun, and Bin He. 2021 · 2023

Later among the works it cites.

Gpt-4 technical report

Original

OpenAI. 2023 · 2023

Later among the works it cites.

Stanford alpaca: An instruction-following llama model

Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023 · 2023