Fetching the paper…
Reading the bibliography…
Transformer models have revolutionized natural language processing with their unparalleled ability to grasp complex contextual relationships.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
The trec-8 question answering track report
Ellen M Voorhees et al · 1999
Earlier work this paper cites.
The structure and performance of an open-domain question answering system
Dan Moldovan, Sanda Harabagiu, Marius Pasca, Rada Mihalcea, Roxana Girju, Richard Goodrum, and Vasile Rus · 2000
Earlier work this paper cites.
An analysis of the askmsr question-answering system
Eric Brill, Susan Dumais, and Michele Banko · 2002
Earlier work this paper cites.
Building watson: An overview of the deepqa project
David Ferrucci, Eric Brown, Jennifer Chu-Carroll, James Fan, David Gondek, Aditya A Kalyanpur, Adam Lally, J William Murdock, Eric Nyberg, John Prager, et al · 2010
Earlier work this paper cites.
Song Han, Huizi Mao, and William J Dally · 2015
Earlier work this paper cites.
Eie: Efficient inference engine on compressed deep neural network
Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark A Horowitz, and William J Dally · 2016
Earlier work this paper cites.
Pruning filters for efficient convnets
Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf · 2016
Earlier work this paper cites.
Pruning convolutional neural networks for resource efficient inference
Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz · 2016
Earlier work this paper cites.
Structured bayesian pruning via log-normal multiplicative noise
Kirill Neklyudov, Dmitry Molchanov, Arsenii Ashukha, and Dmitry P Vetrov · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
To prune, or not to prune: exploring the efficacy of pruning for model compression
Michael Zhu and Suyog Gupta · 2017
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Cited alongside, same era.
Rocket launching: A universal and efficient framework for training well-performing light net
Guorui Zhou, Ying Fan, Runpeng Cui, Weijie Bian, Xiaoqiang Zhu, and Kun Gai · 2018
Cited alongside, same era.
Reducing transformer depth on demand with structured dropout
Angela Fan, Edouard Grave, and Armand Joulin · 2019
Structured pruning of large language models
Ziheng Wang, Jeremy Wohlwend, and Tao Lei · 2019
Later among the works it cites.
Language models are few-shot learners
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Later among the works it cites.
Movement pruning: Adaptive sparsity by fine-tuning
Victor Sanh, Thomas Wolf, and Alexander Rush · 2020
Later among the works it cites.
Zero-shot open-book question answering
Sia Gholami and Mehdi Noori · 2021
Later among the works it cites.
Block pruning for faster transformers, 2021
François Lagunas, Ella Charlaix, Victor Sanh, and Alexander M. Rush · 2021
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
The state of sparsity in deep neural networks
Trevor Gale, Erich Elsen, and Sara Hooker · 2019
Cited alongside, same era.
Are sixteen heads really better than one?
Paul Michel, Omer Levy, and Graham Neubig · 2019
Cited alongside, same era.
Energy and policy considerations for deep learning in nlp
Emma Strubell, Ananya Ganesh, and Andrew McCallum · 2019
Cited alongside, same era.
Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned
Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov · 2019
Cited alongside, same era.
Alexa, predict my flight delay
Sia Gholami and Saba Khashe
Cited in the paper.
Flight delay prediction using deep learning and conversational voice-based agents
Sia Gholami and Saba Khashe
Cited in the paper.
Do generative large language models need billions of parameters?
Sia Gholami and Marwan Omar
Cited in the paper.
Later among the works it cites.
Text classification for online conversations with machine learning on aws
Ryan Brand, Sia Gholami, Daniel Horowitz, Liutong Zhou, and Sourav Bhabesh · 2022
Later among the works it cites.
You don’t need labeled data for open-book question answering
Sia Gholami and Mehdi Noori · 2022
Later among the works it cites.
Create, train, and deploy a billion-parameter language model on terabytes of data with tensorflow and amazon sagemaker
Sia Gholami, Danny Byrd, Francisco Calderon Rodriguez, Muhyun Kim, Yohei Nakayama, Mehdi Noori, and Nathalie Rauschmayr · 2022
Later among the works it cites.
On the effect of dropping layers of pre-trained transformer models
Hassan Sajjad, Fahim Dalvi, Nadir Durrani, and Preslav Nakov · 2023
Closest in time.