Fetching the paper…
Reading the bibliography…
Among the neural network compression techniques, knowledge distillation is an effective one which forces a simpler student network to mimic the output of a larger teacher network.
Model compression
C. Buciluǎ, R. Caruana, and A. Niculescu-Mizil · 2006
Earlier work this paper cites.
Semantic contours from inverse detectors
B. Hariharan, P. Arbeláez, L. Bourdev, S. Maji, and J. Malik · 2011
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2012
Earlier work this paper cites.
Do deep nets really need to be deep?
J. Ba and R. Caruana · 2014
Earlier work this paper cites.
Compressing deep convolutional networks using vector quantization
Y. Gong, L. Liu, M. Yang, and L. Bourdev · 2014
Earlier work this paper cites.
Fitnets: Hints for thin deep nets
A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio · 2014
Earlier work this paper cites.
Net2net: Accelerating learning via knowledge transfer
T. Chen, I. Goodfellow, and J. Shlens · 2015
Earlier work this paper cites.
S. Han, H. Mao, and W. J. Dally · 2015
Earlier work this paper cites.
Distilling the knowledge in a neural network
G. Hinton, O. Vinyals, and J. Dean · 2015
Earlier work this paper cites.
Parsenet: Looking wider to see better
W. Liu, A. Rabinovich, and A. C. Berg · 2015
Earlier work this paper cites.
Fully convolutional networks for semantic segmentation
J. Long, E. Shelhamer, and T. Darrell · 2015
Earlier work this paper cites.
Tensorizing neural networks
A. Novikov, D. Podoprikhin, A. Osokin, and D. P. Vetrov · 2015
Earlier work this paper cites.
Faster r-cnn: Towards real-time object detection with region proposal networks
S. Ren, K. He, R. Girshick, and J. Sun · 2015
Cited alongside, same era.
An analysis of deep neural network models for practical applications
A. Canziani, A. Paszke, and E. Culurciello · 2016
Cited alongside, same era.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Cited alongside, same era.
Face model compression by distilling knowledge from neurons
P. Luo, Z. Zhu, Z. Liu, X. Wang, X. Tang, et al · 2016
Cited alongside, same era.
Xnor-net: Imagenet classification using binary convolutional neural networks
M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi · 2016
Cited alongside, same era.
In teacher we trust: Learning compressed models for pedestrian detection
Like what you like: Knowledge distill via neuron selectivity transfer
Z. Huang and N. Wang · 2017
Later among the works it cites.
Mimicking very efficient network for object detection
Q. Li, S. Jin, and J. Yan · 2017
Later among the works it cites.
Inception-v4, inception-resnet and the impact of residual connections on learning
C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi · 2017
Later among the works it cites.
Model distillation with knowledge transfer in face classification, alignment and verification
C. Wang and X. Lan · 2017
Later among the works it cites.
X. Wang, R. Girshick, A. Gupta, and K. He · 2017
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
J. Shen, N. Vesdapunt, V. N. Boddeti, and K. M. Kitani · 2016
Cited alongside, same era.
Rethinking the inception architecture for computer vision
C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna · 2016
Cited alongside, same era.
Do deep convolutional nets really need to be deep and convolutional?
G. Urban, K. J. Geras, S. E. Kahou, O. Aslan, S. Wang, R. Caruana, A. Mohamed, M. Philipose, and M. Richardson · 2016
Cited alongside, same era.
S. Zagoruyko and N. Komodakis · 2016
Cited alongside, same era.
N2n learning: Network to network compression via policy gradient reinforcement learning
A. Ashok, N. Rhinehart, F. Beainy, and K. M. Kitani · 2017
Cited alongside, same era.
Learning efficient object detection models with knowledge distillation
G. Chen, W. Choi, X. Yu, T. Han, and M. Chandraker · 2017
Cited alongside, same era.
Rethinking atrous convolution for semantic image segmentation
L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam · 2017
Cited alongside, same era.
A gift from knowledge distillation: Fast optimization, network minimization and transfer learning
J. Yim, D. Joo, J. Bae, and J. Kim · 2017
Later among the works it cites.
Dilated residual networks
F. Yu, V. Koltun, and T. A. Funkhouser · 2017
Later among the works it cites.
H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia · 2017
Later among the works it cites.
Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2018
Later among the works it cites.
Improving fast segmentation with teacher-student learning
J.-F. H. J. L. W.-S. Z. Jiafeng Xie, Bing Shuai · 2018
Later among the works it cites.
Training shallow and thin networks for acceleration via knowledge distillation with conditional adversarial networks
Z. Xu, Y.-C. Hsu, and J. Huang · 2018
Later among the works it cites.