Fetching the paper…
Reading the bibliography…
Knowledge distillation is a widely applicable technique for training a student neural network under the guidance of a trained teacher network.
Describing textures in the wild
M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi · 2014
Earlier work this paper cites.
Exploiting linear structure within convolutional networks for efficient evaluation
E. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus · 2014
Earlier work this paper cites.
Speeding up convolutional neural networks with low rank expansions
M. Jaderberg, A. Vedaldi, and A. Zisserman · 2014
Earlier work this paper cites.
Distilling the knowledge in a neural network
G. Hinton, O. Vinyals, and J. Dean · 2015
Earlier work this paper cites.
FitNets: hints for thin deep nets
A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio · 2015
Earlier work this paper cites.
Efficient and accurate approximations of nonlinear convolutional networks
X. Zhang, J. Zou, X. Ming, K. He, and J. Sun · 2015
Earlier work this paper cites.
EIE: Efficient inference engine on compressed deep neural network
S. Han, X. Liu, H. Mao, J. Pu, A. Pedram, M. A. Horowitz, and W. J. Dally · 2016
Earlier work this paper cites.
Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding
S. Han, H. Mao, and W. J. Dally · 2016
Earlier work this paper cites.
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and
F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer · 2016
Earlier work this paper cites.
Unifying distillation and privileged information
D. Lopez-Paz, L. Bottou, B. Schölkopf, and V. Vapnik · 2016
Earlier work this paper cites.
XNOR-Net: ImageNet classification using binary convolutional neural networks
M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi · 2016
Earlier work this paper cites.
Learning structured sparsity in deep neural networks
W. Wen, C. Wu, Y. Wang, Y. Chen, and H. Li · 2016
Earlier work this paper cites.
Wide residual networks
S. Zagoruyko and N. Komodakis · 2016
Earlier work this paper cites.
MobileNets: Efficient convolutional neural networks for mobile vision applications
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam · 2017
Earlier work this paper cites.
Learning from noisy labels with distillation
Y. Li, J. Yang, Y. Song, L. Cao, J. Luo, and L.-J. Li · 2017
Earlier work this paper cites.
ThiNet: A filter level pruning method for deep neural network compression
J.-H. Luo, J. Wu, and W. Lin · 2017
Earlier work this paper cites.
Exploring sparsity in recurrent neural networks
S. Narang, G. Diamos, S. Sengupta, and E. Elsen · 2017
Cited alongside, same era.
Distillation as a defense to adversarial perturbations against deep neural networks
N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami · 2017
Cited alongside, same era.
Faster CNNs with direct sparse convolutions and guided pruning
J. Park, S. Li, W. Wen, P. Tang, H. Li, Y. Chen, and P. Dubey · 2017
Cited alongside, same era.
Designing energy-efficient convolutional neural networks using energy-aware pruning
T.-J. Yang, Y.-H. Chen, and V. Sze · 2017
Cited alongside, same era.
A gift from knowledge distillation: Fast optimization, network minimization and transfer learning
J. Yim, D. Joo, J. Bae, and J. Kim · 2017
Cited alongside, same era.
Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer
Knowledge distillation by on-the-fly native ensemble
X. Lan, X. Zhu, and S. Gong · 2018
Later among the works it cites.
Frequency-domain dynamic pruning for convolutional neural networks
Z. Liu, J. Xu, X. Peng, and R. Xiong · 2018
Later among the works it cites.
Shufflenet v2: Practical guidelines for efficient cnn architecture design
N. Ma, X. Zhang, H.-T. Zheng, and J. Sun · 2018
Later among the works it cites.
Extreme network compression via filter group approximation
B. Peng, W. Tan, Z. Li, S. Zhang, D. Xie, and S. Pu · 2018
Later among the works it cites.
Model compression via distillation and quantization
A. Polino, R. Pascanu, and D. Alistarh · 2018
Later among the works it cites.
Data distillation: towards omni-supervised learning
I. Radosavovic, P. Dollár, R. Girshick, G. Gkioxari, and K. He · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
S. Zagoruyko and N. Komodakis · 2017
Cited alongside, same era.
N2N learning: Network to network compression via policy gradient reinforcement learning
A. Ashok, N. Rhinehart, F. Beainy, and K. M. Kitani · 2018
Cited alongside, same era.
Constraint-aware deep neural network compression
C. Chen, F. Tung, N. Vedula, and G. Mori · 2018
Cited alongside, same era.
Moonshine: Distilling with cheap convolutions
E. J. Crowley, G. Gray, and A. Storkey · 2018
Cited alongside, same era.
CINIC-10 is not ImageNet or CIFAR-10
L. N. Darlow, E. J. Crowley, A. Antoniou, and A. J. Storkey · 2018
Cited alongside, same era.
Coreset-based neural network compression
A. Dubey, M. Chatterjee, and N. Ahuja · 2018
Cited alongside, same era.
SYQ: Learning symmetric quantization for efficient deep neural networks
J. Faraone, N. Fraser, M. Blott, and P. H. W. Leong · 2018
Cited alongside, same era.
MobileNetV2: Inverted residuals and linear bottlenecks
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen · 2018
Later among the works it cites.
CLIP-Q: Deep network compression learning by in-parallel pruning-quantization
F. Tung and G. Mori · 2018
Later among the works it cites.
KDGAN: Knowledge distillation with generative adversarial networks
X. Wang, R. Zhang, Y. Sun, and J. Qi · 2018
Later among the works it cites.
NetAdapt: Platform-aware neural network adaptation for mobile applications
T.-J. Yang, A. Howard, B. Chen, X. Zhang, A. Go, M. Sandle, V. Sze, and H. Adam · 2018
Later among the works it cites.
NISP: Pruning networks using neuron importance score propagation
R. Yu, A. Li, C.-F. Chen, J.-H. Lai, V. I. Morariu, X. Han, M. Gao, C.-Y. Lin, and L. S. Davis · 2018
Later among the works it cites.
LQ-Nets: Learned quantization for highly accurate and compact deep neural networks
D. Zhang, J. Yang, D. Ye, and G. Hua · 2018
Later among the works it cites.
ShuffleNet: An extremely efficient convolutional neural network for mobile devices
X. Zhang, X. Zhou, M. Lin, and J. Sun · 2018
Later among the works it cites.
Explicit loss-error-aware quantization for low-bit deep neural networks
A. Zhou, A. Yao, K. Wang, and Y. Chen · 2018
Later among the works it cites.
Lifelong GAN: Continual learning for conditional image generation
M. Zhai, L. Chen, F. Tung, J. He, M. Nawhal, and G. Mori · 2019
Closest in time.