Fetching the paper…
Reading the bibliography…
Knowledge Distillation (KD) is a common method for transferring the ``knowledge'' learned by one machine learning model (the \textit{teacher}) into another model (the \textit{student}), where typically, the teacher has a greater capacity (e.g., more parameters or higher bit-widths).
Bandura, A.: Social cognitive theory in cultural context. Applied Psychology (2002)
2002
Earlier work this paper cites.
Brumfiel, G.: Who has designs on your students’ minds? Nature Publishing Group (2005)
2005
Earlier work this paper cites.
Bucilua, C., Caruana, R., Niculescu-Mizil, A.: Model compression. In: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM (2006)
2006
Earlier work this paper cites.
Gurlitt, J., Renkl, A., Motes, M.A., Hauser, S.: How can we use concept maps for prior knowledge activation: different mapping-tasks lead to different cognitive processes. In: Proceedings of the 7th international conference on Learning sciences. International Society of the Learning Sciences (2006)
2006
Earlier work this paper cites.
Slotta, J.D., Chi, M.T.: Helping students understand challenging topics in science through ontology training. Cognition and instruction (2006)
2006
Earlier work this paper cites.
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images. Tech. rep., Citeseer (2009)
2009
Earlier work this paper cites.
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Proc. Adv. Neural Inf. Process. Syst. (2012)
2012
Earlier work this paper cites.
2013
Earlier work this paper cites.
Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. In: Proc. Adv. Neural Inf. Process. Syst. Workshops (2014)
2014
Earlier work this paper cites.
Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., Bengio, Y.: Fitnets: Hints for thin deep nets. In: Proc. Int. Conf. Learn. Repren. (2015)
2015
Earlier work this paper cites.
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proc. IEEE Conf. Comp. Vis. Patt. Recogn. (2016)
2016
Earlier work this paper cites.
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Proc. Eur. Conf. Comp. Vis. (2016)
2016
Earlier work this paper cites.
Zagoruyko, S., Komodakis, N.: Wide residual networks. In: BMVC (2016)
2016
Earlier work this paper cites.
Zhang, B., Wang, L., Wang, Z., Qiao, Y., Wang, H.: Real-time action recognition with enhanced motion vector cnns. In: Proc. IEEE Conf. Comp. Vis. Patt. Recogn. pp. 2718–2726 (2016)
2016
Earlier work this paper cites.
2016
Cited alongside, same era.
Cai, Z., He, X., Sun, J., Vasconcelos, N.: Deep learning with low precision by half-wave gaussian quantization. In: Proc. IEEE Conf. Comp. Vis. Patt. Recogn. pp. 5918–5926 (2017)
2017
Cited alongside, same era.
Chen, G., Choi, W., Yu, X., Han, T., Chandraker, M.: Learning efficient object detection models with knowledge distillation. In: Proc. Adv. Neural Inf. Process. Syst. (2017)
2017
Cited alongside, same era.
2017
Cited alongside, same era.
Wei, Y., Pan, X., Qin, H., Ouyang, W., Yan, J.: Quantization mimic: Towards very tiny cnn for object detection. In: Proc. Eur. Conf. Comp. Vis. (2018)
2018
Later among the works it cites.
Zhang, D., Yang, J., Ye, D., Hua, G.: Lq-nets: Learned quantization for highly accurate and compact deep neural networks. In: Proc. Eur. Conf. Comp. Vis. pp. 365–382 (2018)
2018
Later among the works it cites.
Zhang, Y., Xiang, T., Hospedales, T.M., Lu, H.: Deep mutual learning. In: Proc. IEEE Conf. Comp. Vis. Patt. Recogn. (2018)
2018
Later among the works it cites.
Zhuang, B., Shen, C., Tan, M., Liu, L., Reid, I.: Towards effective low-bitwidth convolutional neural networks. In: Proc. IEEE Conf. Comp. Vis. Patt. Recogn. (2018)
2018
Later among the works it cites.
Bai, Y., Wang, Y.X., Liberty, E.: Proxquant: Quantized neural networks via proximal operators. In: Proc. Int. Conf. Learn. Repren. (2019)
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2017
Cited alongside, same era.
Lemley, J., Bazrafkan, S., Corcoran, P.: Smart augmentation learning an optimal data augmentation strategy. Ieee Access (2017)
2017
Cited alongside, same era.
Ratner, A.J., Ehrenberg, H., Hussain, Z., Dunnmon, J., Ré, C.: Learning to compose domain-specific transformations for data augmentation. In: Proc. Adv. Neural Inf. Process. Syst. (2017)
2017
Cited alongside, same era.
Tran, T., Pham, T., Carneiro, G., Palmer, L., Reid, I.: A bayesian data augmentation approach for learning deep models. In: Proc. Adv. Neural Inf. Process. Syst. (2017)
2017
Cited alongside, same era.
2018
Cited alongside, same era.
2018
Cited alongside, same era.
Hyun Lee, S., Ha Kim, D., Cheol Song, B.: Self-supervised knowledge distillation using singular value decomposition. In: Proc. Eur. Conf. Comp. Vis. (2018)
2018
Cited alongside, same era.
2018
Cited alongside, same era.
2019
Later among the works it cites.
2019
Later among the works it cites.
He, T., Shen, C., Tian, Z., Gong, D., Sun, C., Yan, Y.: Knowledge adaptation for efficient semantic segmentation. In: Proc. IEEE Conf. Comp. Vis. Patt. Recogn. (2019)
2019
Later among the works it cites.
Ho, D., Liang, E., Stoica, I., Abbeel, P., Chen, X.: Population based augmentation: Efficient learning of augmentation policy schedules. In: Proc. Int. Conf. Mach. Learn. (2019)
2019
Later among the works it cites.
Louizos, C., Reisser, M., Blankevoort, T., Gavves, E., Welling, M.: Relaxed quantization for discretized neural networks. In: Proc. Int. Conf. Learn. Repren. (2019)
2019
Later among the works it cites.
2019
Later among the works it cites.
Park, W., Kim, D., Lu, Y., Cho, M.: Relational knowledge distillation. In: Proc. IEEE Conf. Comp. Vis. Patt. Recogn. (2019)
2019
Later among the works it cites.
Seunghyun Lee, B.C.S.: Graph-based knowledge distillation by multi-head attention network. In: Proc. Brit. Mach. Vis. Conf. (2019)
2019
Later among the works it cites.