Fetching the paper…
Reading the bibliography…
Knowledge distillation aims at transferring knowledge acquired in one model (a teacher) to another model (a student) that is typically smaller.
Course in general linguistics. 1916
Ferdinand de Saussure · 1983
Earlier work this paper cites.
Born again trees
Leo Breiman and Nong Shang · 1996
Earlier work this paper cites.
A short history of structural linguistics
Peter Hugoe Matthews and Peter Matthews · 2001
Earlier work this paper cites.
Model compression
Cristian Buciluǎ, Rich Caruana, and Alexandru Niculescu-Mizil · 2006
Earlier work this paper cites.
Learning multiple layers of features from tiny images
A. Krizhevsky and G. Hinton · 2009
Earlier work this paper cites.
The Caltech-UCSD Birds-200-2011 Dataset
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie · 2011
Earlier work this paper cites.
3d object representations for fine-grained categorization
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei · 2013
Earlier work this paper cites.
Do deep nets really need to be deep?
Jimmy Ba and Rich Caruana · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman · 2014
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Earlier work this paper cites.
Human-level concept learning through prob- abilistic program induction
Salakhutdinov Ruslan Lake, Brenden M and Joshua B Tenenbaum · 2015
Earlier work this paper cites.
Fitnets: Hints for thin deep nets
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio · 2015
Earlier work this paper cites.
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei · 2015
Earlier work this paper cites.
Facenet: A unified embedding for face recognition and clustering
Florian Schroff, Dmitry Kalenichenko, and James Philbin · 2015
Earlier work this paper cites.
Going deeper with convolutions
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich · 2015
Earlier work this paper cites.
Learning using privileged information: Similarity control and knowledge transfer
Vladimir Vapnik and Rauf Izmailov · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Cited alongside, same era.
Unifying distillation and privileged information
D. Lopez-Paz, B. Schölkopf, L. Bottou, and V. Vapnik · 2016
Cited alongside, same era.
Deep metric learning via lifted structured feature embedding
Hyun Oh Song, Yu Xiang, Stefanie Jegelka, and Silvio Savarese · 2016
Cited alongside, same era.
Face model compression by distilling knowledge from neurons
Ziwei Liu Xiaogang Wang Ping Luo, Zhenyao Zhu and Xiaoou Tang · 2016
Cited alongside, same era.
Deep model compression: Distilling knowledge from noisy teachers
Bharat Bhusan Sau and Vineeth N Balasubramanian · 2016
Cited alongside, same era.
Improved deep metric learning with multi-class n-pair loss objective
Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer
Sergey Zagoruyko and Nikos Komodakis · 2017
Later among the works it cites.
Label refinery: Improving imagenet classification through label progression
Hessam Bagherinezhad, Maxwell Horton, Mohammad Rastegari, and Ali Farhadi · 2018
Later among the works it cites.
Fast deep neural networks with knowledge guided training and predicted regions of interests for real-time video object detection
W. Cao, J. Yuan, Z. He, Z. Zhang, and Z. He · 2018
Later among the works it cites.
Darkrank: Accelerating deep metric learning via cross sample similarities transfer
Yuntao Chen, Naiyan Wang, and Zhaoxiang Zhang · 2018
Later among the works it cites.
Moonshine: Distilling with cheap convolutions
Elliot J Crowley, Gavin Gray, and Amos Storkey · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Kihyuk Sohn · 2016
Cited alongside, same era.
Matching networks for one shot learning
Oriol Vinyals, Charles Blundell, Tim Lillicrap, Daan Wierstra, et al · 2016
Cited alongside, same era.
Picking deep filter responses for fine-grained image recognition
Xiaopeng Zhang, Hongkai Xiong, Wengang Zhou, Weiyao Lin, and Qi Tian · 2016
Cited alongside, same era.
Learning efficient object detection models with knowledge distillation
Guobin Chen, Wongun Choi, Xiang Yu, Tony Han, and Manmohan Chandraker · 2017
Cited alongside, same era.
Like what you like: Knowledge distill via neuron selectivity transfer
Zehao Huang and Naiyan Wang · 2017
Cited alongside, same era.
Data-free knowledge distillation for deep neural networks
Raphael Gontijo Lopes, Stefano Fenu, and Thad Starner · 2017
Cited alongside, same era.
Prototypical networks for few-shot learning
Jake Snell, Kevin Swersky, and Richard Zemel · 2017
Cited alongside, same era.
Born-again neural networks
Tommaso Furlanello, Zachary Chase Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar · 2018
Later among the works it cites.
Attention-based ensemble for deep metric learning
Wonsik Kim, Bhavya Goyal, Kunal Chawla, Jungmin Lee, and Keunjoo Kwon · 2018
Later among the works it cites.
Apprentice: Using knowledge distillation techniques to improve low-precision network accuracy
Asit Mishra and Debbie Marr · 2018
Later among the works it cites.
Deep metric learning with bier: Boosting independent embeddings robustly
M. Opitz, G. Waltner, H. Possegger, and H. Bischof · 2018
Later among the works it cites.
Object-part attention model for fine-grained image classification
Yuxin Peng, Xiangteng He, and Junjie Zhao · 2018
Later among the works it cites.
Model compression via distillation and quantization
Antonio Polino, Razvan Pascanu, and Dan Alistarh · 2018
Later among the works it cites.
Data distillation: Towards omni-supervised learning
Ilija Radosavovic, Piotr Dollár, Ross Girshick, Georgia Gkioxari, and Kaiming He · 2018
Later among the works it cites.
Revisiting knowledge transfer for training object class detectors
Jasper Uijlings, Stefan Popov, and Vittorio Ferrari · 2018
Later among the works it cites.
Training shallow and thin networks for acceleration via knowledge distillation with conditional adversarial networks, 2018
Zheng Xu, Yen-Chang Hsu, and Jiawei Huang · 2018
Later among the works it cites.
Learning to navigate for fine-grained classification
Ze Yang, Tiange Luo, Dong Wang, Zhiqiang Hu, Jun Gao, and Liwei Wang · 2018
Later among the works it cites.
https://tiny-imagenet.herokuapp.com/
Tiny imagenet visual recognition challenge · 2018
Later among the works it cites.