Fetching the paper…
Reading the bibliography…
Model compression techniques are receiving increasing attention; however, the effect of compression on model fairness is still under explored.
Benchmarking neural network robustness to common corruptions and perturbations
Dan Hendrycks and Thomas G. Dietterich. 2019 · 1903
Earlier work this paper cites.
The generalization-stability tradeoff in neural network pruning
Brian Bartoldson, Ari S. Morcos, Adrian Barbu, and Gordon Erlebacher. 2020 · 1906
Earlier work this paper cites.
Reconciling utility and membership privacy via knowledge distillation
Virat Shejwalkar and Amir Houmansadr. 2019 · 1906
Earlier work this paper cites.
Reducing transformer depth on demand with structured dropout
Angela Fan, Edouard Grave, and Armand Joulin. 2020 · 1909
Earlier work this paper cites.
Tinybert: Distilling bert for natural language understanding
Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2020 · 1909
Earlier work this paper cites.
Albert: A lite bert for self-supervised learning of language representations
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020 · 1909
Earlier work this paper cites.
The woman worked as a babysitter: On biases in language generation
Emily Sheng, Kai-Wei Chang, P. Natarajan, and Nanyun Peng. 2019 · 1909
Earlier work this paper cites.
Improving generalization and robustness with noisy collaboration in knowledge distillation
E. Arani, Fahad Sarfraz, and Bahram Zonooz. 2019 · 1910
Earlier work this paper cites.
Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019 · 1910
Earlier work this paper cites.
When bert plays the lottery, all tickets are winning
Sai Prasanna, Anna Rogers, and Anna Rumshisky. 2020 · 2005
Earlier work this paper cites.
Squeezebert: What can computer vision teach nlp about efficient neural networks?
Forrest N. Iandola, Albert Eaton Shaw, Ravi Krishna, and Kurt Keutzer. 2020 · 2006
Earlier work this paper cites.
Compression of deep learning models for text: A survey
Manish Gupta and Puneet Agrawal. 2020 · 2008
Cited alongside, same era.
Realtoxicityprompts: Evaluating neural toxic degeneration in language models
Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020 · 2009
Cited alongside, same era.
Characterising bias in compressed models
Sara Hooker, Nyalleng Moorosi, Gregory Clark, Samy Bengio, and Emily L. Denton. 2020 · 2010
Cited alongside, same era.
Bag of tricks for adversarial training
Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. 2021 · 2010
Cited alongside, same era.
Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned
Elena Voita, David Talbot, F. Moiseev, Rico Sennrich, and Ivan Titov. 2019 · 2019
Later among the works it cites.
Adversarially robust distillation
Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. 2020 · 2020
Later among the works it cites.
Detoxify
Laura Hanu and Unitary team. 2020 · 2020
Later among the works it cites.
Going beyond classification accuracy metrics in model compression
Vinu Joseph, Shoaib Ahmed Siddiqui, Aditya Bhaskara, Ganesh Gopalakrishnan, Saurav Muralidharan, Michael Garland, Sheraz Ahmed, and Andreas R. Dengel. 2020 · 2020
Later among the works it cites.
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020 · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015 · 2015
Cited alongside, same era.
Distillation as a defense to adversarial perturbations against deep neural networks
Nicolas Papernot, Patrick Mcdaniel, Xi Wu, Somesh Jha, and Ananthram Swami. 2016 · 2016
Cited alongside, same era.
Gender bias in coreference resolution: Evaluation and debiasing methods
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018 · 2018
Cited alongside, same era.
transformers . zip : Compressing transformers with pruning and quantization
Robin Cheong. 2019 · 2019
Cited alongside, same era.
Shallow-deep networks: Understanding and mitigating network overthinking
Yigitcan Kaya, Sanghyun Hong, and Tudor Dumitras. 2019 · 2019
Cited alongside, same era.
Are sixteen heads really better than one?
Paul Michel, Omer Levy, and Graham Neubig. 2019 · 2019
Cited alongside, same era.
Language models are unsupervised multitask learners
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019
Cited alongside, same era.
Dialogpt : Large-scale generative pre-training for conversational response generation
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and William B. Dolan. 2020 · 2020
Later among the works it cites.
Redditbias: A real-world resource for bias evaluation and debiasing of conversational language models
Soumya Barikeri, Anne Lauscher, Ivan Vuli’c, and Goran Glavas. 2021 · 2021
Later among the works it cites.
On the effect of pruning on adversarial robustness
Artur Jordão and Hélio Pedrini. 2021 · 2021
Later among the works it cites.
Stereoset: Measuring stereotypical bias in pretrained language models
Moin Nadeem, Anna Bethke, and Siva Reddy. 2021 · 2021
Later among the works it cites.
Recipes for building an open-domain chatbot
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric Michael Smith, Y.-Lan Boureau, and Jason Weston. 2021 · 2021
Later among the works it cites.
Knowledge distillation with bert for image tag-based privacy prediction
Chenye Zhao and Cornelia Caragea. 2021 · 2021
Later among the works it cites.