GMP*: Well-Tuned Gradual Magnitude Pruning Can Outperform Most BERT-Pruning Methods

A simple and effective method for removal of hidden units and weights

Masafumi Hagiwara. 1994 · 1994

Earlier work this paper cites.

Distilling the knowledge in a neural network

Original

Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. 2015 · 2015

Earlier work this paper cites.

SQuAD: 100,000+ questions for machine comprehension of text

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016 · 2016

Earlier work this paper cites.

First quora dataset release: Question pairs

Shankar Iyer, Nikhil Dandekar, Kornél Csernai, et al. 2017 · 2017

Earlier work this paper cites.

A broad-coverage challenge corpus for sentence understanding through inference

Original

Adina Williams, Nikita Nangia, and Samuel R Bowman. 2017 · 2017

Earlier work this paper cites.

To prune, or not to prune: exploring the efficacy of pruning for model compression

Original

Michael Zhu and Suyog Gupta. 2017 · 2017

Earlier work this paper cites.

BERT: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019

Earlier work this paper cites.

The state of sparsity in deep neural networks

Trevor Gale, Erich Elsen, and Sara Hooker. 2019 · 2019

Cited alongside, same era.

The lottery ticket hypothesis for pre-trained bert networks

Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Zhangyang Wang, and Michael Carbin. 2020 · 2020

Cited alongside, same era.

Inducing and exploiting activation sparsity for fast inference on deep neural networks

Mark Kurtz, Justin Kopinsky, Rati Gelashvili, Alexander Matveev, John Carr, Michael Goin, William Leiserson, Sage Moore, Bill Nell, Nir Shavit, and Dan Alistarh. 2020 · 2020

Cited alongside, same era.

Movement pruning: Adaptive sparsity by fine-tuning

Victor Sanh, Thomas Wolf, and Alexander Rush. 2020 · 2020

Cited alongside, same era.

Transformers: State-of-the-art natural language processing

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020 · 2020

Cited alongside, same era.

Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks

Original

Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste. 2021 · 2021

Later among the works it cites.

Block pruning for faster transformers

François Lagunas, Ella Charlaix, Victor Sanh, and Alexander Rush. 2021 · 2021

Later among the works it cites.

Datasets: A community library for natural language processing

Original

Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, Joe Davison, Mario Šaško, Gunjan Chhablani, Bhavitvya Malik, Simon Brandeis, Teven Le Scao, Victor Sanh, Canwen Xu, Nicolas Patry, Angelina McMillan-Major, Philipp Schmid, Sylvain Gugger, Clément Delangue, Théo Matussière, Lysandre Debut, Stas Bekman, Pierric Cistac, Thibault Goehringer, Victor Mustar, François Lagunas, Alexander Rush, and Thomas Wolf. 2021 · 2021

Later among the works it cites.

Rethinking network pruning–under the pre-train and fine-tune paradigm

Original

Dongkuan Xu, Ian EH Yen, Jinxi Zhao, and Zhibin Xiao. 2021 · 2021

Later among the works it cites.