2019

SelectiveNet: A Deep Neural Network with an Integrated Reject Option

Geifman, Yonatan, El-Yaniv, Ran

Understand

We consider the problem of selective prediction (also known as reject option) in deep neural networks, and introduce SelectiveNet, a deep neural architecture with an integrated reject option.

Existing rejection mechanisms are based mostly on a threshold over the prediction confidence of a pre-trained network.
In contrast, SelectiveNet is trained to optimize both classification (or regression) and rejection simultaneously, end-to-end.
The result is a deep neural network that is optimized over the covered domain.

Built on

An optimum character recognition system using decision functions
Chow, C. K · 1957
Earlier work this paper cites.
Probability inequalities for sums of bounded random variables
Hoeffding, W · 1963
Earlier work this paper cites.
The nearest neighbor classification rule with a reject option
Hellman, M. E · 1970
Earlier work this paper cites.
A method for improving classification reliability of multilayer perceptrons
Cordella, L. P., De Stefano, C., Tortorella, F., and Vento, M · 1995
Earlier work this paper cites.
To reject or not to reject: that is the question-an answer in case of neural classifiers
De Stefano, C., Sansone, C., and Vento, M · 2000
Earlier work this paper cites.
Interior-point methods
Potra, F. A. and Wright, S. J · 2000
Earlier work this paper cites.
Support vector machines with embedded reject option
Fumera, G. and Roli, F · 2002
Earlier work this paper cites.
Visualizing data using t-sne
Maaten, L. v. d. and Hinton, G · 2008
Earlier work this paper cites.

Similar

Learning multiple layers of features from tiny images
Krizhevsky, A. and Hinton, G · 2009
Cited alongside, same era.
On the foundations of noise-free selective classification
El-Yaniv, R. and Wiener, Y · 2010
Cited alongside, same era.
Reading digits in natural images with unsupervised feature learning
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng, A. Y · 2011
Cited alongside, same era.
Active learning via perfect selective classification
El-Yaniv, R. and Wiener, Y · 2012
Cited alongside, same era.
Pointwise tracking the optimal regression function
Wiener, Y. and El-Yaniv, R · 2012
Cited alongside, same era.
Adam: A method for stochastic optimization
Original
Kingma, D. P. and Ba, J · 2014
Cited alongside, same era.
Very deep convolutional networks for large-scale image recognition
Original
Simonyan, K. and Zisserman, A · 2014
Cited alongside, same era.
Dropout: a simple way to prevent neural networks from overfitting
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R · 2014
Cited alongside, same era.
Boosting with abstention
Cortes, C., DeSalvo, G., and Mohri, M
Cited in the paper.
Learning with rejection
Cortes, C., DeSalvo, G., and Mohri, M
Cited in the paper.
Deep active learning over the long tail
Original
Geifman, Y. and El-Yaniv, R
Cited in the paper.
Selective classification for deep neural networks
Geifman, Y. and El-Yaniv, R
Cited in the paper.

Then

Agnostic pointwise-competitive selective classification
El-Yaniv, R. and Wiener, Y · 2015
Later among the works it cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Original
Ioffe, S. and Szegedy, C · 2015
Later among the works it cites.
Very deep convolutional neural network based image classification using small training sample size
Liu, S. and Deng, W · 2015
Later among the works it cites.
Dropout as a bayesian approximation: representing model uncertainty in deep learning
Gal, Y. and Ghahramani, Z · 2016
Later among the works it cites.
UCI machine learning repository, 2017
Dheeru, D. and Karra Taniskidou, E · 2017
Later among the works it cites.
Simple and scalable predictive uncertainty estimation using deep ensembles
Lakshminarayanan, B., Pritzel, A., and Blundell, C · 2017
Later among the works it cites.
Deep active learning with a neural architecture search
Original
Geifman, Y. and El-Yaniv, R · 2018
Later among the works it cites.
Bias-reduced uncertainty estimation for deep neural classifiers
Geifman, Y., Uziel, G., and El-Yaniv, R · 2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…