Fetching the paper…
Reading the bibliography…
Knowledge distillation is widely used as a means of improving the performance of a relatively simple student model using the predictions from a complex teacher model.
Empirical margin distributions and bounding the generalization error of combined classifiers
V. Koltchinskii and D. Panchenko · 2002
Earlier work this paper cites.
Model compression
Cristian Bucilǎ, Rich Caruana, and Alexandru Niculescu-Mizil · 2006
Earlier work this paper cites.
Channel distillation: Channel-wise attention for knowledge distillation
Zaida Zhou, Chaoran Zhuge, Xinwei Guan, and Wen Liu · 2006
Earlier work this paper cites.
Three Naive Bayes approaches for discrimination-free classification
Toon Calders and Sicco Verwer · 2010
Earlier work this paper cites.
Towards understanding ensemble, knowledge distillation and self-distillation in deep learning
Zeyuan Allen-Zhu and Yuanzhi Li · 2012
Earlier work this paper cites.
Fairness through awareness
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel · 2012
Earlier work this paper cites.
On the statistical consistency of algorithms for binary classification under class imbalance
Aditya Krishna Menon, Harikrishna Narasimhan, Shivani Agarwal, and Sanjay Chawla · 2013
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean · 2015
Earlier work this paper cites.
Equality of opportunity in supervised learning
Moritz Hardt, Eric Price, and Nathan Srebro · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Unifying distillation and privileged information
D. Lopez-Paz, B. Schölkopf, L. Bottou, and V. Vapnik · 2016
Earlier work this paper cites.
A systematic study of the class imbalance problem in convolutional neural networks
Mateusz Buda, Atsuto Maki, and Maciej A. Mazurowski · 2017
Earlier work this paper cites.
On calibration of modern neural networks
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger · 2017
Earlier work this paper cites.
Cost-sensitive learning of deep feature representations from imbalanced data
Salman H. Khan, Munawar Hayat, Mohammed Bennamoun, Ferdous A. Sohel, and Roberto Togneri · 2017
Earlier work this paper cites.
Sgdr: Stochastic gradient descent with warm restarts, 2017
Ilya Loshchilov and Frank Hutter · 2017
Earlier work this paper cites.
The devil is in the tails: Fine-grained classification in the wild
Grant Van Horn and Pietro Perona · 2017
Cited alongside, same era.
Large scale distributed neural network training through online distillation
Rohan Anil, Gabriel Pereyra, Alexandre Passos, Robert Ormandi, George E. Dahl, and Geoffrey E. Hinton · 2018
Cited alongside, same era.
Gender shades: Intersectional accuracy disparities in commercial gender classification
Joy Buolamwini and Timnit Gebru · 2018
Cited alongside, same era.
Born-again neural networks
Tommaso Furlanello, Zachary Chase Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar · 2018
Cited alongside, same era.
Data distillation: Towards omni-supervised learning
Ilija Radosavovic, Piotr Dollár, Ross B. Girshick, Georgia Gkioxari, and Kaiming He · 2018
Cited alongside, same era.
The price of fair pca: One extra dimension
Knowledge distillation in wide neural networks: Risk bound, data efficiency and imperfect teacher
Guangda Ji and Zhanxing Zhu · 2020
Later among the works it cites.
Decoupling representation and classifier for long-tailed recognition
Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis · 2020
Later among the works it cites.
Long-tail learning via logit adjustment, 2020
Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar · 2020
Later among the works it cites.
Self-distillation amplifies regularization in hilbert space, 2020
Hossein Mobahi, Mehrdad Farajtabar, and Peter L. Bartlett · 2020
Later among the works it cites.
Balanced meta-softmax for long-tailed visual recognition
Jiawei Ren, Cunjun Yu, shunan sheng, Xiao Ma, Haiyu Zhao, Shuai Yi, and hongsheng Li · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Samira Samadi, Uthaipon Tantipongpipat, Jamie H Morgenstern, Mohit Singh, and Santosh Vempala · 2018
Cited alongside, same era.
The inaturalist species classification and detection dataset
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie · 2018
Cited alongside, same era.
Learning imbalanced datasets with label-distribution-aware margin loss
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Aréchiga, and Tengyu Ma · 2019
Cited alongside, same era.
On the efficacy of knowledge distillation
J. H. Cho and B. Hariharan · 2019
Cited alongside, same era.
Leveraging labeled and unlabeled data for consistent fair binary classification
Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, Luca Oneto, and Massimiliano Pontil · 2019
Cited alongside, same era.
Class-balanced loss based on effective number of samples
Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie · 2019
Cited alongside, same era.
Large-scale long-tailed recognition in an open world
Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X. Yu · 2019
Cited alongside, same era.
N. Sohoni, J. Dunnmon, G. Angus, A. Gu, and C. Ré · 2020
Later among the works it cites.
Understanding and improving knowledge distillation
Jiaxi Tang, Rakesh Shivanna, Zhe Zhao, Dong Lin, Anima Singh, Ed H. Chi, and Sagar Jain · 2020
Later among the works it cites.
Prime-aware adaptive distillation
Youcai Zhang, Zhonghao Lan, Yuchen Dai, Fangao Zeng, Yan Bai, Jie Chang, and Yichen Wei · 2020
Later among the works it cites.
Self-distillation as instance-specific label smoothing
Zhilu Zhang and Mert R. Sabuncu · 2020
Later among the works it cites.
Maintaining discrimination and fairness in class incremental learning
Bowen Zhao, Xi Xiao, Guojun Gan, Bin Zhang, and Shu-Tao Xia · 2020
Later among the works it cites.
Knowledge distillation as semiparametric inference
Tri Dao, Govinda M Kamath, Vasilis Syrgkanis, and Lester Mackey · 2021
Closest in time.
Selective classification can magnify disparities across groups
Erik Jones, Shiori Sagawa, Pang Wei Koh, Ananya Kumar, and Percy Liang · 2021
Closest in time.
Seesaw loss for long-tailed instance segmentation
Jiaqi Wang, Wenwei Zhang, Yuhang Zang, Yuhang Cao, Jiangmiao Pang, Tao Gong, Kai Chen, Ziwei Liu, Chen Change Loy, and Dahua Lin · 2021
Closest in time.
Rethinking soft labels for knowledge distillation: A bias–variance tradeoff perspective
Helong Zhou, Liangchen Song, Jiajie Chen, Ye Zhou, Guoli Wang, Junsong Yuan, and Qian Zhang · 2021
Closest in time.