Fetching the paper…
Reading the bibliography…
We propose a novel knowledge distillation approach to facilitate the transfer of dark knowledge from a teacher to a student.
Learning Multiple Layers of Features from Tiny Images
Alex Krizhevsky · 2009
Earlier work this paper cites.
An analysis of single-layer networks in unsupervised feature learning
Adam Coates, Andrew Y. Ng, and Honglak Lee · 2011
Earlier work this paper cites.
Distilling the Knowledge in a Neural Network
Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean · 2015
Earlier work this paper cites.
FitNets: Hints for Thin Deep Nets
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio · 2015
Earlier work this paper cites.
Imagenet large scale visual recognition challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Fei-Fei Li · 2015
Earlier work this paper cites.
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan and Andrew Zisserman · 2015
Earlier work this paper cites.
Face Model Compression by Distilling Knowledge from Neurons
Ping Luo, Zhenyao Zhu, Ziwei Liu, Xiaogang Wang, and Xiaoou Tang · 2016
Earlier work this paper cites.
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Wide residual networks
Sergey Zagoruyko and Nikos Komodakis · 2016
Earlier work this paper cites.
Learning Efficient Object Detection Models with Knowledge Distillation
Guobin Chen, Wongun Choi, Xiangdong Chen, Tony X. Han, and Manmohan Krishna Chandraker · 2017
Earlier work this paper cites.
Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
Sergey Zagoruyko and Nikos Komodakis · 2017
Earlier work this paper cites.
Deep Mutual Learning
Ying Zhang, Tao Xiang, Timothy M Hospedales, and Huchuan Lu · 2018
Earlier work this paper cites.
Learning to Specialize with Knowledge Distillation for Visual Question Answering
Jonghwan Mun, Kimin Lee, Jinwoo Shin, and Bohyung Han · 2018
Earlier work this paper cites.
Through-wall human pose estimation using radio signals
Mingmin Zhao, Tianhong Li, Mohammad Abu Alsheikh, Yonglong Tian, Hang Zhao, Antonio Torralba, and Dina Katabi · 2018
Earlier work this paper cites.
Paraphrasing Complex Network: Network Compression via Factor Transfer
Jangho Kim, SeongUk Park, and Nojun Kwak · 2018
Earlier work this paper cites.
Knowledge Distillation by On-the-Fly Native Ensemble
Xu Lan, Xiatian Zhu, and Shaogang Gong · 2018
Cited alongside, same era.
Born-again neural networks
Tommaso Furlanello, Zachary Chase Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar · 2018
Cited alongside, same era.
Shufflenet: An extremely efficient convolutional neural network for mobile devices
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun · 2018
Cited alongside, same era.
Learning deep representations with probabilistic knowledge transfer
Nikolaos Passalis and Anastasios Tefas · 2018
Cited alongside, same era.
Distilled person re-identification: Towards a more scalable system
Ancong Wu, Wei-Shi Zheng, Xiaowei Guo, and Jian-Huang Lai · 2019
Cited alongside, same era.
Tinybert: Distilling BERT for natural language understanding
Mnasnet: Platform-aware neural architecture search for mobile
Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V. Le · 2019
Later among the works it cites.
Similarity-preserving knowledge distillation
Frederick Tung and Greg Mori · 2019
Later among the works it cites.
Variational information distillation for knowledge transfer
Sungsoo Ahn, Shell Xu Hu, Andreas C. Damianou, Neil D. Lawrence, and Zhenwen Dai · 2019
Later among the works it cites.
Knowledge transfer via distillation of activation boundaries formed by hidden neurons
Byeongho Heo, Minsik Lee, Sangdoo Yun, and Jin Young Choi · 2019
Later among the works it cites.
Similarity of neural network representations revisited
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey E. Hinton · 2019
Later among the works it cites.
Benchmarking neural network robustness to common corruptions and perturbations
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu · 2019
Cited alongside, same era.
Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf · 2019
Cited alongside, same era.
On knowledge distillation from complex networks for response prediction
Siddhartha Arora, Mitesh M. Khapra, and Harish G. Ramaswamy · 2019
Cited alongside, same era.
Cross-modal knowledge distillation for action recognition
Fida Mohammad Thoker and Juergen Gall · 2019
Cited alongside, same era.
A comprehensive overhaul of feature distillation
Byeongho Heo, Jeesoo Kim, Sangdoo Yun, Hyojin Park, Nojun Kwak, and Jin Young Choi · 2019
Cited alongside, same era.
Relational Knowledge Distillation
Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho · 2019
Cited alongside, same era.
Be your own teacher: Improve the performance of convolutional neural networks via self distillation
Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma · 2019
Cited alongside, same era.
Dan Hendrycks and Thomas Dietterich · 2019
Later among the works it cites.
Towards oracle knowledge distillation with neural architecture search
Minsoo Kang, Jonghwan Mun, and Bohyung Han · 2020
Later among the works it cites.
Improved knowledge distillation via teacher assistant: Bridging the gap between student and teacher
Seyed-Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, and Hassan Ghasemzadeh · 2020
Later among the works it cites.
Online knowledge distillation via collaborative learning
Qiushan Guo, Xinjiang Wang, Yichao Wu, Zhipeng Yu, Ding Liang, Xiaolin Hu, and Ping Luo · 2020
Later among the works it cites.
Peer collaborative learning for online knowledge distillation
Guile Wu and Shaogang Gong · 2020
Later among the works it cites.
Collaborative distillation for ultra-resolution universal style transfer
Huan Wang, Yijun Li, Yuehai Wang, Haoji Hu, and Ming-Hsuan Yang · 2020
Later among the works it cites.
Domain adaptation through task distillation
Brady Zhou, Nimit Kalra, and Philipp Krähenbühl · 2020
Later among the works it cites.
Contrastive representation distillation
Yonglong Tian, Dilip Krishnan, and Phillip Isola · 2020
Later among the works it cites.
Knowledge distillation meets self-supervision
Guodong Xu, Ziwei Liu, Xiaoxiao Li, and Chen Change Loy · 2020
Later among the works it cites.