Not All Layers of LLMs Are Necessary During Inference

Batch-shaping for learning conditional channel gated networks

Original

Babak Ehteshami Bejnordi, Tijmen Blankevoort, and Max Welling. 2019 · 1907

Earlier work this paper cites.

Depth-adaptive transformer

Original

Maha Elbayad, Jiatao Gu, Edouard Grave, and Michael Auli. 2019 · 1910

Earlier work this paper cites.

Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex

David H Hubel and Torsten N Wiesel. 1962 · 1962

Earlier work this paper cites.

Optimal brain damage

Yann LeCun, John Denker, and Sara Solla. 1989 · 1989

Earlier work this paper cites.

The processing-speed theory of adult age differences in cognition

Timothy A Salthouse. 1996 · 1996

Earlier work this paper cites.

Support vector machines

Marti A. Hearst, Susan T Dumais, Edgar Osuna, John Platt, and Bernhard Scholkopf. 1998 · 1998

Earlier work this paper cites.

Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area aip

Akira Murata, Vittorio Gallese, Giuseppe Luppino, Masakazu Kaseda, and Hideo Sakata. 2000 · 2000

Earlier work this paper cites.

Reaction times and intelligence differences: A population-based cohort study

Ian J Deary, Geoff Der, and Graeme Ford. 2001 · 2001

Earlier work this paper cites.

Conditional random fields: Probabilistic models for segmenting and labeling sequence data

John Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001 · 2001

Earlier work this paper cites.

Fastbert: a self-distilling bert with adaptive inference time

Original

Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Haotang Deng, and Qi Ju. 2020 · 2004

Earlier work this paper cites.

Cascadebert: Accelerating inference of pre-trained language models via calibrated complete models cascade

Original

Lei Li, Yankai Lin, Deli Chen, Shuhuai Ren, Peng Li, Jie Zhou, and Xu Sun. 2020 · 2012

Earlier work this paper cites.

Recursive deep models for semantic compositionality over a sentiment treebank

Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013 · 2013

Earlier work this paper cites.

Character-level convolutional networks for text classification

Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015 · 2015

Earlier work this paper cites.

SQuAD: 100,000+ Questions for Machine Comprehension of Text

Original

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016 · 2016

Earlier work this paper cites.

Branchynet: Fast inference via early exiting from deep neural networks

Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. 2016 · 2016

Earlier work this paper cites.

Adaptive neural networks for efficient inference

Tolga Bolukbasi, Joseph Wang, Ofer Dekel, and Venkatesh Saligrama. 2017 · 2017

Earlier work this paper cites.

Multi-scale dense networks for resource efficient image classification

Original

Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017 · 2017

Earlier work this paper cites.

Runtime neural pruning

Ji Lin, Yongming Rao, Jiwen Lu, and Jie Zhou. 2017 · 2017

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

Skipnet: Learning dynamic routing in convolutional networks

Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph E Gonzalez. 2018 · 2018

Earlier work this paper cites.

Channel gating neural networks

Weizhe Hua, Yuan Zhou, Christopher M De Sa, Zhiru Zhang, and G Edward Suh. 2019 · 2019

Cited alongside, same era.

CommonsenseQA: A question answering challenge targeting commonsense knowledge

Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019 · 2019

Cited alongside, same era.

Resolution adaptive networks for efficient inference

Le Yang, Yizeng Han, Xi Chen, Shiji Song, Jifeng Dai, and Gao Huang. 2020 · 2020

Cited alongside, same era.

Efficient large scale language modeling with mixtures of experts

Original

Mikel Artetxe, Shruti Bhosale, Naman Goyal, Todor Mihaylov, Myle Ott, Sam Shleifer, Xi Victoria Lin, Jingfei Du, Srinivasan Iyer, Ramakanth Pasunuru, et al. 2021 · 2021

Cited alongside, same era.

The efficiency misnomer

Original

Mostafa Dehghani, Anurag Arnab, Lucas Beyer, Ashish Vaswani, and Yi Tay. 2021 · 2021

Cited alongside, same era.

In-context learning in large language models learns label relationships but is not conventional learning

Original

Jannik Kossen, Tom Rainforth, and Yarin Gal. 2023 · 2023

Later among the works it cites.

FLM-101B: an open LLM and how to train it with $100k budget

Original

Xiang Li, Yiqun Yao, Xin Jiang, Xuezhi Fang, Xuying Meng, Siqi Fan, Peng Han, Jing Li, Li Du, Bowen Qin, Zheng Zhang, Aixin Sun, and Yequan Wang. 2023 · 2023

Later among the works it cites.

Deja vu: Contextual sparsity for efficient llms at inference time

Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Re, et al. 2023 · 2023

Later among the works it cites.

Llm-pruner: On the structural pruning of large language models

Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023 · 2023

Later among the works it cites.

Efficiently scaling transformer inference

Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. 2023 · 2023