Fetching the paper…
Reading the bibliography…
Knowledge distillation (KD) is a popular method to train efficient networks ("student") with the help of high-capacity networks ("teacher").
P. Indyk and R. Motwani, “Approximate nearest neighbors: towards removing the curse of dimensionality,” in Proceedings of the 30th annual ACM symposium on Theory of computing , 1998, pp. 604–613
1998
Earlier work this paper cites.
A. Gionis, P. Indyk, R. Motwani et al. , “Similarity search in high dimensions via hashing,” in Proceedings of the 25th International Conference on Very Large Data Bases (VLDB) , 1999, pp. 518–529
1999
Earlier work this paper cites.
M. Datar, N. Immorlica, P. Indyk, and V. S. Mirrokni, “Locality-sensitive hashing scheme based on p-stable distributions,” in Proceedings of the 20th ACM Symposium on Computational Geometry , 2004, pp. 253–262
2004
Earlier work this paper cites.
A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” University of Toronto, Tech. Rep., 2009
2009
Earlier work this paper cites.
M. Everingham, L. Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” International Journal of Computer Vision , vol. 88, pp. 303–338, 2009
2009
Earlier work this paper cites.
R. Xia, Y. Pan, H. Lai, C. Liu, and S. Yan, “Supervised hashing for image retrieval via image representation learning.” in Proceedings of the AAAI Conference on Artificial Intelligence , 2014, p. 2156–2162
2014
Earlier work this paper cites.
T.-Y. Lin, M. Maire, S. J. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, “Microsoft COCO: Common objects in context,” in The European Conference on Computer Vision (ECCV) , ser. LNCS, vol. 8693. Springer, 2014, pp. 740–755
2014
Earlier work this paper cites.
2015
Earlier work this paper cites.
A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio, “FitNets: Hints for thin deep nets,” in The International Conference on Learning Representations (ICLR) , 2015, pp. 1–13
2015
Earlier work this paper cites.
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “ImageNet large scale visual recognition challenge,” International Journal of Computer Vision , vol. 115, no. 3, pp. 211–252, 2015
2015
Earlier work this paper cites.
S. Ren, K. He, R. B. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 39, pp. 1137–1149, 2015
2015
Earlier work this paper cites.
F. Wang, X. Xiang, J. Cheng, and A. L. Yuille, “NormFace: L2 hypersphere embedding for face verification,” in Proceedings of the 25th ACM international conference on Multimedia , 2017, pp. 1041–1049
2017
Earlier work this paper cites.
S. Zagoruyko and N. Komodakis, “Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer,” in The International Conference on Learning Representations (ICLR) , 2017, pp. 1–13
2017
Earlier work this paper cites.
J. Yim, D. Joo, J. Bae, and J. Kim, “A gift from knowledge distillation: Fast optimization, network minimization and transfer learning,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017, pp. 7130–7138
2017
Cited alongside, same era.
Q. Li, S. Jin, and J. Yan, “Mimicking very efficient network for object detection,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017, pp. 7341–7349
2017
Cited alongside, same era.
Z. Cao, M. Long, J. Wang, and P. S. Yu, “HashNet: Deep learning to hash by continuation,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) , 2017, pp. 5608–5617
2017
Cited alongside, same era.
T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, “Focal loss for dense object detection,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) , 2017, pp. 2999–3007
2017
Cited alongside, same era.
T. Wang, L. Yuan, X. Zhang, and J. Feng, “Distilling object detectors with fine-grained feature imitation,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2019, pp. 4928–4937
2019
Later among the works it cites.
2019
Later among the works it cites.
B. Peng, X. Jin, J. Liu, D. Li, Y. Wu, Y. Liu, S. Zhou, and Z. Zhang, “Correlation congruence for knowledge distillation,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) , 2019, pp. 5007–5016
2019
Later among the works it cites.
T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in Proceedings of the International Conference on Machine Learning (ICML) , 2020, pp. 10 709–10 719
2020
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
T.-Y. Lin, P. Dollar, R. B. Girshick, K. He, B. Hariharan, and S. J. Belongie, “Feature pyramid networks for object detection,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017, pp. 936–944
2017
Cited alongside, same era.
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted residuals and linear bottlenecks,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2018, pp. 4510–4520
2018
Cited alongside, same era.
N. Ma, X. Zhang, H.-T. Zheng, and J. Sun, “ShuffleNet V2: Practical guidelines for efficient cnn architecture design,” in The European Conference on Computer Vision (ECCV) , ser. LNCS, vol. 11218. Springer, 2018, pp. 116–131
2018
Cited alongside, same era.
S. Gidaris, P. Singh, and N. Komodakis, “Unsupervised representation learning by predicting image rotations,” in The International Conference on Learning Representations (ICLR) , 2018, pp. 1–16
2018
Cited alongside, same era.
J. Kim, S. Park, and N. Kwak, “Paraphrasing complex network: Network compression via factor transfer,” in Advances in Neural Information Processing Systems 31 , 2018, pp. 2760–2769
2018
Cited alongside, same era.
Y. Cao, M. Long, B. Liu, and J. Wang, “Deep Cauchy hashing for Hamming space retrieval,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2018, pp. 1229–1237
2018
Cited alongside, same era.
X. Lan, X. Zhu, and S. Gong, “Knowledge distillation by on-the-fly native ensemble,” in Advances in Neural Information Processing Systems 31 , 2018, pp. 7517–7527
2018
Cited alongside, same era.
B. Heo, M. Lee, S. Yun, and J. Y. Choi, “Knowledge transfer via distillation of activation boundaries formed by hidden neurons,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 33, 2019, pp. 3779–3787
2019
Cited alongside, same era.
2020
Closest in time.
K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2020, pp. 9729–9738
2020
Closest in time.
Y. Tian, D. Krishnan, and P. Isola, “Contrastive representation distillation,” in The International Conference on Learning Representations (ICLR) , 2020, pp. 1–14
2020
Closest in time.
G. Xu, Z. Liu, X. Li, and C. C. Loy, “Knowledge distillation meets self-supervision,” in The European Conference on Computer Vision (ECCV) , ser. LNCS, vol. 12354. Springer, 2020, pp. 588–604
2020
Closest in time.
K. Xu, L. Rui, Y. Li, and L. Gu, “Feature normalized knowledge distillation for image classification,” in The European Conference on Computer Vision (ECCV) , ser. LNCS, vol. 12370. Springer, 2020, pp. 664–680
2020
Closest in time.
Y. Zhang, Z. Lan, Y. Dai, F. Zeng, Y. Bai, J. Chang, and Y. Wei, “Prime-aware adaptive distillation,” in The European Conference on Computer Vision (ECCV) , ser. LNCS, vol. 12364. Springer, 2020, pp. 658–674
2020
Closest in time.
T. Li, J. Li, Z. Liu, and C. Zhang, “Few sample knowledge distillation for efficient network compression,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2020, pp. 14 639–14 647
2020
Closest in time.
2020
Closest in time.
H. Liu, R. Wang, S. Shan, and X. Chen, “Deep supervised hashing for fast image retrieval,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016, pp. 2064–2072
2072
Closest in time.