Dropout Distillation for Efficiently Estimating Model Confidence

M. H. DeGroot and S. E. Fienberg, “The comparison and evaluation of forecasters,” The Statistician: Journal of the Institute of Statisticians , vol. 32, pp. 12–22, 1983

1983

Earlier work this paper cites.

T. G. Dietterich, “Ensemble methods in machine learning,” in Multiple Classifier Systems . Berlin, Heidelberg: Springer Berlin Heidelberg, 2000, pp. 1–15

2000

Earlier work this paper cites.

C. Bucila, R. Caruana, and A. Niculescu-Mizil, “Model compression,” in Proceedings of the Twelfth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Philadelphia, PA, USA, August 20-23, 2006 , 2006, pp. 535–541. [Online]. Available: http://doi.acm.org/10.1145/1150402.1150464

2006

Earlier work this paper cites.

A. Krizhevsky, “Learning multiple layers of features from tiny images,” Tech. Rep., 2009

2009

Earlier work this paper cites.

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A Large-Scale Hierarchical Image Database,” in CVPR09 , 2009

Original

2009

Earlier work this paper cites.

I. Sutskever, J. Martens, G. Dahl, and G. Hinton, “On the importance of initialization and momentum in deep learning,” in Proceedings of the 30th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, S. Dasgupta and D. McAllester, Eds., vol. 28, no. 3. Atlanta, Georgia, USA: PMLR, 17–19 Jun 2013, pp. 1139–1147. [Online]. Available: http://proceedings.mlr.press/v28/sutskever13.html

2013

Earlier work this paper cites.

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research , vol. 15, pp. 1929–1958, 2014. [Online]. Available: http://jmlr.org/papers/v15/srivastava14a.html

2014

Earlier work this paper cites.

J. Ba and R. Caruana, “Do deep nets really need to be deep?” in Advances in Neural Information Processing Systems 27 , Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2014, pp. 2654–2662. [Online]. Available: http://papers.nips.cc/paper/5484-do-deep-nets-really-need-to-be-deep.pdf

2014

Earlier work this paper cites.

P. Ondruska and I. Posner, “Probabilistic attainability maps: Efficiently predicting driver-specific electric vehicle range,” in IEEE Intelligent Vehicles Symposium (IV) , Dearborn, MI, USA, June 2014

2014

Earlier work this paper cites.

T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in Computer Vision – ECCV 2014 , D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Eds. Cham: Springer International Publishing, 2014, pp. 740–755

2014

Cited alongside, same era.

C. Blundell, J. Cornebise, K. Kavukcuoglu, and D. Wierstra, “Weight uncertainty in neural networks,” in Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37 , ser. ICML’15. JMLR.org, 2015, pp. 1613–1622. [Online]. Available: http://dl.acm.org/citation.cfm?id=3045118.3045290

2015

Cited alongside, same era.

Y. Gal and Z. Ghahramani, “Dropout as a Bayesian approximation: Representing model uncertainty in deep learning,” arXiv:1506.02142 , 2015

Original

2015

Cited alongside, same era.

G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” in NIPS Deep Learning and Representation Learning Workshop , 2015. [Online]. Available: http://arxiv.org/abs/1503.02531

S. R. Bulò, L. Porzi, and P. Kontschieder, “Dropout distillation,” in Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 , ser. ICML’16. JMLR.org, 2016, pp. 99–107. [Online]. Available: http://dl.acm.org/citation.cfm?id=3045390.3045402

2016

Later among the works it cites.

G. Pereyra, G. Tucker, J. Chorowski, L. Kaiser, and G. E. Hinton, “Regularizing neural networks by penalizing confident output distributions,” CoRR , vol. abs/1701.06548, 2017. [Online]. Available: http://arxiv.org/abs/1701.06548

Original

2017

Later among the works it cites.

T. Furlanello, Z. C. Lipton, M. Tschannen, L. Itti, and A. Anandkumar, “Born again neural networks,” 2017

2017

Later among the works it cites.

C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” CoRR , vol. abs/1706.04599, 2017. [Online]. Available: http://arxiv.org/abs/1706.04599

Original

2017

Later among the works it cites.