Fetching the paper…
Reading the bibliography…
Transferring knowledge from a teacher neural network pretrained on the same or a similar task to a student neural network can significantly improve the performance of the student neural network.
An application of the principle of maximum information preservation to linear systems
R. Linsker · 1989
Earlier work this paper cites.
Mixture density networks
C. M. Bishop · 1994
Earlier work this paper cites.
The IM algorithm: a variational approach to information maximization
D. B. F. Agakov · 2004
Earlier work this paper cites.
Learning multiple layers of features from tiny images
A. Krizhevsky · 2009
Earlier work this paper cites.
Recognizing indoor scenes
A. Quattoni and A. Torralba · 2009
Earlier work this paper cites.
A survey on transfer learning
S. J. Pan, Q. Yang, et al · 2010
Earlier work this paper cites.
Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion
P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol · 2010
Earlier work this paper cites.
Caltech-UCSD Birds 200
P. Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Belongie, and P. Perona · 2010
Earlier work this paper cites.
Do deep nets really need to be deep?
J. Ba and R. Caruana · 2014
Earlier work this paper cites.
Depth map prediction from a single image using a multi-scale deep network
D. Eigen, C. Puhrsch, and R. Fergus · 2014
Earlier work this paper cites.
Fitnets: Hints for thin deep nets
A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio · 2014
Earlier work this paper cites.
Unsupervised pre-training across image domains improves lung tissue classification
T. Schlegl, J. Ofner, and G. Langs · 2014
Cited alongside, same era.
Very deep convolutional networks for large-scale image recognition
K. Simonyan and A. Zisserman · 2014
Cited alongside, same era.
Deeppose: Human pose estimation via deep neural networks
A. Toshev and C. Szegedy · 2014
Cited alongside, same era.
Flownet: Learning optical flow with convolutional networks
A. Dosovitskiy, P. Fischer, E. Ilg, P. Hausser, C. Hazirbas, V. Golkov, P. Van Der Smagt, D. Cremers, and T. Brox · 2015
Cited alongside, same era.
Fast r-CNN
R. Girshick · 2015
Cited alongside, same era.
Improved variational inference with inverse autoregressive flow
D. P. Kingma, T. Salimans, R. Jozefowicz, X. Chen, I. Sutskever, and M. Welling · 2016
Later among the works it cites.
Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer
S. Zagoruyko and N. Komodakis · 2016
Later among the works it cites.
S. Zagoruyko and N. Komodakis · 2016
Later among the works it cites.
Like what you like: Knowledge distill via neuron selectivity transfer
Z. Huang and N. Wang · 2017
Later among the works it cites.
Learning without forgetting
Z. Li and D. Hoiem · 2017
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
G. Hinton, O. Vinyals, and J. Dean · 2015
Cited alongside, same era.
How far can we go without convolution: Improving fully-connected networks
Z. Lin, R. Memisevic, and K. Konda · 2015
Cited alongside, same era.
Semi-supervised learning with ladder networks
A. Rasmus, M. Berglund, M. Honkala, H. Valpola, and T. Raiko · 2015
Cited alongside, same era.
Imagenet large scale visual recognition challenge
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al · 2015
Cited alongside, same era.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Cited alongside, same era.
A gift from knowledge distillation: Fast optimization, network minimization and transfer learning
J. Yim, D. Joo, J. Bae, and J. Kim
Cited in the paper.
G. Urban, K. J. Geras, S. E. Kahou, O. Aslan, S. Wang, A. Mohamed, M. Philipose, M. Richardson, and R. Caruana · 2017
Later among the works it cites.
Adversarial network compression
V. Belagiannis, A. Farshad, and F. Galasso · 2018
Later among the works it cites.
Mine: mutual information neural estimation
I. Belghazi, S. Rajeswar, A. Baratin, R. D. Hjelm, and A. Courville · 2018
Later among the works it cites.
Coupled end-to-end transfer learning with generalized Fisher information
S. Chen, C. Zhang, and M. Dong · 2018
Later among the works it cites.
Born again neural networks
T. Furlanello, Z. C. Lipton, M. Tschannen, L. Itti, and A. Anandkumar · 2018
Later among the works it cites.