2018

Evaluating Bayesian Deep Learning Methods for Semantic Segmentation

Mukhoti, Jishnu, Gal, Yarin

Understand

Deep learning has been revolutionary for computer vision and semantic segmentation in particular, with Bayesian Deep Learning (BDL) used to obtain uncertainty maps from deep models when predicting semantic classes.

This information is critical when using semantic segmentation for autonomous driving for example.
Standard semantic segmentation systems have well-established evaluation metrics.
However, with BDL's rising popularity in computer vision we require new metrics to evaluate whether a BDL method produces better uncertainty estimates than another method.

Built on

A real-time algorithm for signal analysis with the help of the wavelet transform
M. Holschneider, R. Kronland-Martinet, J. Morlet, and P. Tchamitchian · 1990
Earlier work this paper cites.
Learning convolutional feature hierarchies for visual recognition
K. Kavukcuoglu, P. Sermanet, Y.-L. Boureau, K. Gregor, M. Mathieu, and Y. L. Cun · 2010
Earlier work this paper cites.
Practical variational inference for neural networks
A. Graves · 2011
Earlier work this paper cites.
Towards fully autonomous driving: Systems and algorithms
J. Levinson, J. Askeland, J. Becker, J. Dolson, D. Held, S. Kammel, J. Z. Kolter, D. Langer, O. Pink, V. Pratt, et al · 2011
Earlier work this paper cites.
Bayesian learning via stochastic gradient langevin dynamics
M. Welling and Y. W. Teh · 2011
Earlier work this paper cites.
Bayesian posterior sampling via stochastic gradient fisher scoring
Original
S. Ahn, A. Korattikara, and M. Welling · 2012
Earlier work this paper cites.
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
G. E. Dahl, D. Yu, L. Deng, and A. Acero · 2012
Earlier work this paper cites.
Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups
G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, et al · 2012
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2012
Earlier work this paper cites.
Acoustic modeling using deep belief networks
A.-r. Mohamed, G. E. Dahl, G. Hinton, et al · 2012
Earlier work this paper cites.
Bayesian learning for neural networks
R. M. Neal · 2012
Earlier work this paper cites.
Learning hierarchical features for scene labeling
C. Farabet, C. Couprie, L. Najman, and Y. LeCun · 2013
Earlier work this paper cites.
Learning hierarchical features for scene labeling
C. Farabet, C. Couprie, L. Najman, and Y. LeCun · 2013
Earlier work this paper cites.
Fast image scanning with deep max-pooling convolutional neural networks
A. Giusti, D. C. Ciresan, J. Masci, L. M. Gambardella, and J. Schmidhuber · 2013
Earlier work this paper cites.
Generating sequences with recurrent neural networks
Original
A. Graves · 2013
Earlier work this paper cites.
Speech recognition with deep recurrent neural networks
A. Graves, A.-r. Mohamed, and G. Hinton · 2013
Earlier work this paper cites.
Efficient estimation of word representations in vector space
Original
T. Mikolov, K. Chen, G. Corrado, and J. Dean · 2013
Earlier work this paper cites.
Distributed representations of words and phrases and their compositionality
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean · 2013
Earlier work this paper cites.
Recursive deep models for semantic compositionality over a sentiment treebank
R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, and C. Potts · 2013
Earlier work this paper cites.
Deep learning using linear support vector machines
Original
Y. Tang · 2013
Earlier work this paper cites.
Semantic image segmentation with deep convolutional nets and fully connected crfs
Original
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2014
Earlier work this paper cites.
Distributed representations of sentences and documents
Q. Le and T. Mikolov · 2014
Earlier work this paper cites.
Deep learning of the tissue-regulated splicing code
M. K. Leung, H. Y. Xiong, L. J. Lee, and B. J. Frey · 2014
Earlier work this paper cites.

Similar

Recurrent convolutional neural networks for scene labeling
P. H. Pinheiro and R. Collobert · 2014
Cited alongside, same era.
Very deep convolutional networks for large-scale image recognition
Original
K. Simonyan and A. Zisserman · 2014
Cited alongside, same era.
Predicting the sequence specificities of dna-and rna-binding proteins by deep learning
B. Alipanahi, A. Delong, M. T. Weirauch, and B. J. Frey · 2015
Cited alongside, same era.
Weight uncertainty in neural networks
Original
C. Blundell, J. Cornebise, K. Kavukcuoglu, and D. Wierstra · 2015
Cited alongside, same era.
An empirical evaluation of deep learning on highway driving
Original
B. Huval, T. Wang, S. Tandon, J. Kiske, W. Song, J. Pazhayampallil, M. Andriluka, P. Rajpurkar, T. Migimatsu, R. Cheng-Yue, et al · 2015
Cited alongside, same era.
Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding
Original
A. Kendall, V. Badrinarayanan, , and R. Cipolla · 2015
Cited alongside, same era.
Variational dropout and the local reparameterization trick
D. P. Kingma, T. Salimans, and M. Welling · 2015
Cited alongside, same era.
Semantic image segmentation via deep parsing network
Z. Liu, X. Li, P. Luo, C.-C. Loy, and X. Tang · 2015
Cited alongside, same era.
Fully convolutional networks for semantic segmentation
J. Long, E. Shelhamer, and T. Darrell · 2015
Cited alongside, same era.
Obtaining well calibrated probabilities using bayesian binning
M. P. Naeini, G. Cooper, and M. Hauskrecht · 2015
Cited alongside, same era.
Modeling local and global deformations in deep learning: Epitomic convolution, multiple instance learning, and sliding window detection
G. Papandreou, I. Kokkinos, and P.-A. Savalle · 2015
Cited alongside, same era.
U-net: Convolutional networks for biomedical image segmentation
O. Ronneberger, P. Fischer, and T. Brox · 2015
Cited alongside, same era.

Then

Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Y. Gal and Z. Ghahramani · 2016
Later among the works it cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Later among the works it cites.
Efficient piecewise training of deep structured models for semantic segmentation
G. Lin, C. Shen, A. Van Den Hengel, and I. Reid · 2016
Later among the works it cites.
Structured and efficient variational deep learning with matrix gaussian posteriors
C. Louizos and M. Welling · 2016
Later among the works it cites.
Segnet: A deep convolutional encoder-decoder architecture for image segmentation
V. Badrinarayanan, A. Kendall, and R. Cipolla · 2017
Later among the works it cites.
Rethinking atrous convolution for semantic image segmentation
Original
L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam · 2017
Later among the works it cites.
Xception: Deep learning with depthwise separable convolutions
F. Chollet · 2017
Later among the works it cites.
Dermatologist-level classification of skin cancer with deep neural networks
A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun · 2017
Later among the works it cites.
Concrete Dropout
Y. Gal, J. Hron, and A. Kendall · 2017
Later among the works it cites.
On calibration of modern neural networks
C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger · 2017
Later among the works it cites.
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
A. Kendall and Y. Gal · 2017
Later among the works it cites.
Simple and scalable predictive uncertainty estimation using deep ensembles
B. Lakshminarayanan, A. Pritzel, and C. Blundell · 2017
Later among the works it cites.
Desire: Distant future prediction in dynamic scenes with interacting agents
N. Lee, W. Choi, P. Vernaza, C. B. Choy, P. H. Torr, and M. Chandraker · 2017
Later among the works it cites.
Refinenet: Multi-path refinement networks for high-resolution semantic segmentation
G. Lin, A. Milan, C. Shen, and I. Reid · 2017
Later among the works it cites.
Prostate cancer diagnosis using deep learning with 3d multiparametric mri
S. Liu, H. Zheng, Y. Feng, and W. Li · 2017
Later among the works it cites.
Deep network flow for multi-object tracking
S. Schulter, P. Vernaza, W. Choi, and M. Chandraker · 2017
Later among the works it cites.
Overview of deep learning in medical imaging
K. Suzuki · 2017
Later among the works it cites.
Pyramid scene parsing network
H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia · 2017
Later among the works it cites.
Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2018
Closest in time.
Encoder-decoder with atrous separable convolution for semantic image segmentation
Original
L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam · 2018
Closest in time.
Multi-task learning using uncertainty to weigh losses for scene geometry and semantics
A. Kendall, Y. Gal, and R. Cipolla · 2018
Closest in time.
Deep learning for medical image processing: Overview, challenges and the future
M. I. Razzak, S. Naz, and A. Zaib · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…