Fetching the paper…
Reading the bibliography…
Knowledge distillation (KD) has been a popular and effective method for model compression.
Methods to Speed Up Error Back-Propagation Learning Algorithm
D. Sarkar · 1995
Earlier work this paper cites.
Model Compression
C. Bucilua, R. Caruana, and A. Niculescu-Mizil · 2006
Earlier work this paper cites.
ImageNet: A Large-Scale Hierarchical Image Database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
Learning Multiple Layers of Features from Tiny Images
A. Krizhevsky · 2009
Earlier work this paper cites.
OpenGL Programming Guide: The Official Guide to Learning OpenGL, Versions 3.0 and 3.1
D. Shreiner et al · 2009
Earlier work this paper cites.
Cats and Dogs
O. M. Parkhi, A. Vedaldi, A. Zisserman, and C. V. Jawahar · 2012
Earlier work this paper cites.
Fine-Grained Visual Classification of Aircraft
S. Maji, T. Chicago, E. Rahtu, J. Kannala, M. Blaschkó, and A. Vedaldi · 2013
Earlier work this paper cites.
Food-101 – Mining Discriminative Components with Random Forests
L. Bossard, M. Guillaumin, and L. Van Gool · 2014
Earlier work this paper cites.
Distilling the Knowledge in a Neural Network
G. Hinton, O. Vinyals, and J. Dean · 2015
Earlier work this paper cites.
FitNets: Hints for Thin Deep Nets
A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio · 2015
Earlier work this paper cites.
Deep Residual Learning for Image Recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Earlier work this paper cites.
Wide Residual Networks
S. Zagoruyko and N. Komodakis · 2016
Earlier work this paper cites.
On Calibration of Modern Neural Networks
C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger · 2017
Earlier work this paper cites.
Mean Teachers are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Deep Learning Results
A. Tarvainen and H. Valpola · 2017
Earlier work this paper cites.
Do Deep Convolutional Nets Really Need to be Deep and Convolutional?
G. Urban, K. J. Geras, S. E. Kahou, O. Aslan, S. Wang, A. Mohamed, M. Philipose, M. Richardson, and R. Caruana · 2017
Earlier work this paper cites.
Born Again Neural Networks
T. Furlanello, Z. Lipton, M. Tschannen, L. Itti, and A. Anandkumar · 2018
Earlier work this paper cites.
MobileNetV2: Inverted Residuals and Linear Bottlenecks
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen · 2018
Earlier work this paper cites.
Data-Free Learning of Student Networks
H. Chen, Y. Wang, C. Xu, Z. Yang, C. Liu, B. Shi, C. Xu, C. Xu, and Q. Tian · 2019
Earlier work this paper cites.
On the Efficacy of Knowledge Distillation
J. H. Cho and B. Hariharan · 2019
Cited alongside, same era.
EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification
P. Helber, B. Bischke, A. Dengel, and D. Borth · 2019
Cited alongside, same era.
Knowledge Distillation with Adversarial Samples Supporting Decision Boundary
B. Heo, M. Lee, S. Yun, and J. Y. Choi · 2019
Cited alongside, same era.
When Does Label Smoothing Help?
R. Müller, S. Kornblith, and G. E. Hinton · 2019
Cited alongside, same era.
Zero-Shot Knowledge Distillation in Deep Networks
G. K. Nayak, K. R. Mopuri, V. Shaj, V. B. Radhakrishnan, and A. Chakraborty · 2019
Cited alongside, same era.
Knowledge Extraction with No Observable Data
J. Yoo, M. Cho, T. Kim, and U. Kang · 2019
Cited alongside, same era.
Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replay
K. Binici, S. Aggarwal, N. T. Pham, K. Leman, and T. Mitra · 2022
Later among the works it cites.
Up to 100x Faster Data-Free Knowledge Distillation
G. Fang, K. Mo, X. Wang, J. Song, S. Bei, H. Zhang, and M. Song · 2022
Later among the works it cites.
A ConvNet for the 2020s
Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie · 2022
Later among the works it cites.
Target Category Agnostic Knowledge Distillation With Frequency-Domain Supervision
W. Tang, M. S. Shakeel, Z. Chen, H. Wan, and W. Kang · 2022
Later among the works it cites.
DeiT III: Revenge of the ViT
H. Touvron, M. Cord, and H. Jégou · 2022
Later among the works it cites.
J. Achiam, S. Adler, S. Agarwal, et al · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Data-Free Network Quantization with Adversarial Knowledge Distillation
Y. Choi, J. Choi, M. El-Khamy, and J. Lee · 2020
Cited alongside, same era.
RandAugment: Practical Automated Data Augmentation with a Reduced Search Space
E. D. Cubuk, B. Zoph, J. Shlens, and Q. V. Le · 2020
Cited alongside, same era.
MEAL v2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet Without Tricks
Z. Shen and M. Savvides · 2020
Cited alongside, same era.
Contrastive Representation Distillation
Y. Tian, D. Krishnan, and P. Isola · 2020
Cited alongside, same era.
Dreaming to Distill: Data-Free Knowledge Transfer via DeepInversion
H. Yin, P. Molchanov, J. M. Alvarez, Z. Li, A. Mallya, D. Hoiem, N. K. Jha, and J. Kautz · 2020
Cited alongside, same era.
Random Erasing Data Augmentation
Z. Zhong, L/ Zheng, G. Kang, S. Li, and Y. Yang · 2020
Cited alongside, same era.
Later among the works it cites.
The Augmented Image Prior: Distilling 1000 Classes by Extrapolating from a Single Image
Y. M. Asano and A. Saeed · 2023
Later among the works it cites.
How Much Data Are Augmentations Worth? An Investigation into Scaling Laws, Invariance, and Implicit Regularization
J. Geiping, M. Goldblum, G. Somepalli, R. Shwartz-Ziv, T. Goldstein, and A. Gordon-Wilson · 2023
Later among the works it cites.
Is Synthetic Data from Diffusion Models Ready for Knowledge Distillation?
Z. Li, Y. Li, P. Zhao, R. Song, X. Li, and J. Yang · 2023
Later among the works it cites.
DinoV2: Learning Robust Visual Features Without Supervision
M. Oquab, T. Darcet, T. Moutakanni, et al · 2023
Later among the works it cites.
Learning to Retain While Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation
G. Patel, K. R. Mopuri, and Q. Qiu · 2023
Later among the works it cites.
Knowledge Representation of Training Data with Adversarial Examples Supporting Decision Boundary
Z. Tian, Z. Wang, A. M. Abdelmoniem, G. Liu, and C. Wang · 2023
Later among the works it cites.
Data-Free Knowledge Distillation via Feature Exchange and Activation Region Constraint
S. Yu, J. Chen, H. Han, and s. Jiang · 2023
Later among the works it cites.
Resurrecting Old Classes with New Data for Exemplar-Free Continual Learning
D. Goswami, A. Soutif-Cormerais, Y. Liu, S. Kamath, B. Twardowski, and J. van de Weijer · 2024
Closest in time.
Dense Depth Distillation with Out-of-Distribution Simulated Images
J. Hu, C. Fan, M. Ozay, H. Jiang, and T. L. Lam · 2024
Closest in time.
Small Scale Data-Free Knowledge Distillation
H. Liu, Y. Wang, H. Liu, F. Sun, and A. Yao · 2024
Closest in time.
NAYER: Noisy Layer Data Generation for Efficient and Effective Data-Free Knowledge Distillation
M.-T. Tran, T. Le, X.-M. Le, M. Harandi, Q. H. Tran, and D. Phung · 2024
Closest in time.