Fetching the paper…
Reading the bibliography…
Knowledge Distillation is a commonly used Deep Neural Network (DNN) compression method, which often maintains overall generalization performance.
The generalization of ‘student’s’problem when several different population varlances are involved
Bernard L Welch · 1947
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
Building classifiers with independency constraints
Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy · 2009
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky · 2009
Earlier work this paper cites.
Reading digits in natural images with unsupervised feature learning
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng · 2011
Earlier work this paper cites.
Fairness through awareness
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel · 2012
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Earlier work this paper cites.
Tiny imagenet visual recognition challenge
Ya Le and Xuan Yang · 2015
Earlier work this paper cites.
Deep learning face attributes in the wild
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang · 2015
Earlier work this paper cites.
Big data’s disparate impact
Solon Barocas and Andrew D Selbst · 2016
Earlier work this paper cites.
Eyeriss: A spatial architecture for energy-efficient dataflow for convolutional neural networks
Yu-Hsin Chen, Joel Emer, and Vivienne Sze · 2016
Earlier work this paper cites.
Equality of opportunity in supervised learning
Moritz Hardt, Eric Price, and Nati Srebro · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
A survey of model compression and acceleration for deep neural networks
Yu Cheng, Duo Wang, Pan Zhou, and Tao Zhang · 2017
Earlier work this paper cites.
Densely connected convolutional networks
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger · 2017
Earlier work this paper cites.
An entropy-based pruning method for cnn compression
Jian-Hao Luo and Jianxin Wu · 2017
Earlier work this paper cites.
Thinet: A filter level pruning method for deep neural network compression
Jian-Hao Luo, Jianxin Wu, and Weiyao Lin · 2017
Earlier work this paper cites.
Wrpn: Wide reduced-precision networks
Asit Mishra, Eriko Nurvitadhi, Jeffrey J Cook, and Debbie Marr · 2017
Earlier work this paper cites.
Model compression and acceleration for deep neural networks: The principles, progress, and challenges
Yu Cheng, Duo Wang, Pan Zhou, and Tao Zhang · 2018
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin · 2018
Earlier work this paper cites.
A survey on addressing high-class imbalance in big data
Joffrey L Leevy, Taghi M Khoshgoftaar, Richard A Bauder, and Naeem Seliya · 2018
Cited alongside, same era.
Teacher-student training for text-independent speaker recognition
Raymond WM Ng, Xuechen Liu, and Pawel Swietojanski · 2018
Cited alongside, same era.
Mitigating unwanted biases with adversarial learning
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell · 2018
Cited alongside, same era.
Knowledge distillation with feature maps for image classification
Wei-Chun Chen, Chia-Che Chang, and Che-Rung Lee · 2019
Cited alongside, same era.
On the efficacy of knowledge distillation
Jang Hyun Cho and Bharath Hariharan · 2019
Cited alongside, same era.
An adversarial feature distillation method for audio classification
Liang Gao, Haibo Mi, Boqing Zhu, Dawei Feng, Yicong Li, and Yuxing Peng · 2019
Cited alongside, same era.
Audio-visual model distillation using acoustic images
Andres Perez, Valentina Sanguineti, Pietro Morerio, and Vittorio Murino · 2020
Later among the works it cites.
Towards fairness in visual recognition: Effective strategies for bias mitigation
Zeyu Wang, Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky · 2020
Later among the works it cites.
Data-free knowledge distillation for object detection
Akshay Chawla, Hongxu Yin, Pavlo Molchanov, and Jose Alvarez · 2021
Later among the works it cites.
Lrc-bert: latent-representation contrastive knowledge distillation for natural language understanding
Hao Fu, Shaojun Zhou, Qihong Yang, Junjie Tang, Guiquan Liu, Kaikui Liu, and Xiaolong Li · 2021
Later among the works it cites.
Knowledge distillation: A survey
Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao · 2021
Later among the works it cites.
Moving beyond “algorithmic bias is a data problem”
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
What do compressed deep neural networks forget?
Sara Hooker, Aaron Courville, Gregory Clark, Yann Dauphin, and Andrea Frome · 2019
Cited alongside, same era.
Tinybert: Distilling bert for natural language understanding
Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu · 2019
Cited alongside, same era.
Distilbert, a distilled version of bert: Smaller, faster, cheaper and lighter
V Sanh · 2019
Cited alongside, same era.
Private model compression via knowledge distillation
Ji Wang, Weidong Bao, Lichao Sun, Xiaomin Zhu, Bokai Cao, and S Yu Philip · 2019
Cited alongside, same era.
Vargfacenet: An efficient variable group convolutional neural network for lightweight face recognition
Mengjia Yan, Mengao Zhao, Zining Xu, Qian Zhang, Guoli Wang, and Zhizhong Su · 2019
Cited alongside, same era.
Low-resolution visual recognition via deep feature distillation
Mingjian Zhu, Kai Han, Chao Zhang, Jinlong Lin, and Yunhe Wang · 2019
Cited alongside, same era.
Sara Hooker · 2021
Later among the works it cites.
Fair feature distillation for visual recognition
Sangwon Jung, Donggyu Lee, Taeeon Park, and Taesup Moon · 2021
Later among the works it cites.
One label, one billion faces: Usage and consistency of racial categories in computer vision
Zaid Khan and Yun Fu · 2021
Later among the works it cites.
Teacher’s pet: understanding and mitigating biases in distillation
Michal Lukasik, Srinadh Bhojanapalli, Aditya Krishna Menon, and Sanjiv Kumar · 2021
Later among the works it cites.
Hatexplain: A benchmark dataset for explainable hate speech detection
Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee · 2021
Later among the works it cites.
A survey on bias and fairness in machine learning
Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan · 2021
Later among the works it cites.
Does knowledge distillation really work?
Samuel Stanton, Pavel Izmailov, Polina Kirichenko, Alexander A Alemi, and Andrew G Wilson · 2021
Later among the works it cites.
A framework for understanding sources of harm throughout the machine learning life cycle
Harini Suresh and John Guttag · 2021
Later among the works it cites.
Machine bias
Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner · 2022
Later among the works it cites.
Fairness without demographics through knowledge distillation
Junyi Chai, Taeuk Jang, and Xiaoqian Wang · 2022
Later among the works it cites.
Compression of deep learning models for text: A survey
Manish Gupta and Puneet Agrawal · 2022
Later among the works it cites.
A review on fairness in machine learning
Dana Pessach and Erez Shmueli · 2022
Later among the works it cites.
The effect of model compression on fairness in facial expression recognition
Samuil Stoychev and Hatice Gunes · 2022
Later among the works it cites.
Tinyvit: Fast pretraining distillation for small vision transformers
Kan Wu, Jinnian Zhang, Houwen Peng, Mengchen Liu, Bin Xiao, Jianlong Fu, and Lu Yuan · 2022
Later among the works it cites.
Efficient deep learning: A survey on making deep learning models smaller, faster, and better
Gaurav Menghani · 2023
Later among the works it cites.