Ensemble Knowledge Distillation for Learning Improved and Efficient Networks

James S Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl, ‘Algorithms for hyper-parameter optimization’, in NIPS

2011

Earlier work this paper cites.

Jimmy Ba and Rich Caruana, ‘Do deep nets really need to be deep?’, in NIPS

2014

Earlier work this paper cites.

Song Han, Huizi Mao, and William J Dally, ‘Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding’, arXiv preprint arXiv:1510.00149

Original

2015

Earlier work this paper cites.

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean, ‘Distilling the knowledge in a neural network’, arXiv preprint arXiv:1503.02531

Original

2015

Earlier work this paper cites.

Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio, ‘Fitnets: Hints for thin deep nets’, ICLR

2015

Earlier work this paper cites.

Forrest N Iandola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J Dally, and Kurt Keutzer, ‘Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size’, arXiv preprint arXiv:1602.07360

Original

2016

Earlier work this paper cites.

Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf, ‘Pruning filters for efficient convnets’, arXiv preprint arXiv:1608.08710

Original

2016

Earlier work this paper cites.

Gregor Urban, Krzysztof J Geras, Samira Ebrahimi Kahou, Ozlem Aslan, Shengjie Wang, Rich Caruana, Abdelrahman Mohamed, Matthai Philipose, and Matt Richardson, ‘Do deep convolutional nets really need to be deep and convolutional?’, arXiv preprint arXiv:1603.05691

Original

2016

Earlier work this paper cites.

Sergey Zagoruyko and Nikos Komodakis, ‘Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer’, arXiv preprint arXiv:1612.03928

Original

2016

Earlier work this paper cites.

Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille, ‘Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs’, PAMI

2017

Cited alongside, same era.

Wojciech M Czarnecki, Simon Osindero, Max Jaderberg, Grzegorz Swirszcz, and Razvan Pascanu, ‘Sobolev training for neural networks’, in NIPS

2017

Cited alongside, same era.

Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam, ‘Mobilenets: Efficient convolutional neural networks for mobile vision applications’, arXiv preprint arXiv:1704.04861

Original

2017

Cited alongside, same era.

Jian-Hao Luo, Jianxin Wu, and Weiyao Lin, ‘Thinet: A filter level pruning method for deep neural network compression’, in ICCV

2017

Cited alongside, same era.

Yanping Huang, Yonglong Cheng, Dehao Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, and Zhifeng Chen, ‘Gpipe: Efficient training of giant neural networks using pipeline parallelism’, arXiv preprint arXiv:1811.06965

Original

2018

Later among the works it cites.

Jangho Kim, SeongUk Park, and Nojun Kwak, ‘Paraphrasing complex network: Network compression via factor transfer’, in NIPS

2018

Later among the works it cites.

Suraj Srinivas and François Fleuret, ‘Knowledge transfer with jacobian matching’, arXiv preprint arXiv:1803.00443

Original

2018

Later among the works it cites.

Ruichi Yu, Ang Li, Chun-Fu Chen, Jui-Hsin Lai, Vlad I Morariu, Xintong Han, Mingfei Gao, Ching-Yung Lin, and Larry S Davis, ‘Nisp: Pruning networks using neuron importance score propagation’, in CVPR

2018

Later among the works it cites.