2018

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Chen, Liang-Chieh, Zhu, Yukun, Papandreou, George et al.

Understand

Spatial pyramid pooling module or encode-decoder structure are used in deep neural networks for semantic segmentation task.

The former networks are able to encode multi-scale contextual information by probing the incoming features with filters or pooling operations at multiple rates and multiple effective fields-of-view, while the latter networks can capture sharper object boundaries by gradually recovering the spatial information.
In this work, we propose to combine the advantages from both methods.
Specifically, our proposed model, DeepLabv3+, extends DeepLabv3 by adding a simple yet effective decoder module to refine the segmentation results especially along object boundaries.

Built on

A real-time algorithm for signal analysis with the help of the wavelet transform
Holschneider, M., Kronland-Martinet, R., Morlet, J., Tchamitchian, P.: · 1989
Earlier work this paper cites.
Gradient-based learning applied to document recognition
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: · 1998
Earlier work this paper cites.
Multiscale conditional random fields for image labeling
He, X., Zemel, R.S., Carreira-Perpindn, M.: · 2004
Earlier work this paper cites.
The pyramid match kernel: Discriminative classification with sets of image features
Grauman, K., Darrell, T.: · 2005
Earlier work this paper cites.
Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories
Lazebnik, S., Schmid, C., Ponce, J.: · 2006
Earlier work this paper cites.
Textonboost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context
Shotton, J., Winn, J., Rother, C., Criminisi, A.: · 2009
Earlier work this paper cites.
Robust higher order potentials for enforcing label consistency
Kohli, P., Torr, P.H., et al.: · 2009
Earlier work this paper cites.
Associative hierarchical crfs for object class image segmentation
Ladicky, L., Russell, C., Kohli, P., Torr, P.H.: · 2009
Earlier work this paper cites.
Decomposing a scene into geometric and semantically consistent regions
Gould, S., Fulton, R., Koller, D.: · 2009
Earlier work this paper cites.
Fast high-dimensional filtering using the permutohedral lattice
Adams, A., Baek, J., Davis, M.A.: · 2010
Earlier work this paper cites.
Efficient inference in fully connected crfs with gaussian edge potentials
Krähenbühl, P., Koltun, V.: · 2011
Earlier work this paper cites.
Semantic contours from inverse detectors
Hariharan, B., Arbeláez, P., Bourdev, L., Maji, S., Malik, J.: · 2011
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., Hinton, G.E.: · 2012
Earlier work this paper cites.
Describing the scene as a whole: Joint object detection, scene classification and semantic segmentation
Yao, J., Fidler, S., Urtasun, R.: · 2012
Earlier work this paper cites.
Learning hierarchical features for scene labeling
Farabet, C., Couprie, C., Najman, L., LeCun, Y.: · 2013
Earlier work this paper cites.
Fast image scanning with deep max-pooling convolutional neural networks
Giusti, A., Ciresan, D., Masci, J., Gambardella, L., Schmidhuber, J.: · 2013
Earlier work this paper cites.
The pascal visual object classes challenge – a retrospective
Everingham, M., Eslami, S.M.A., Gool, L.V., Williams, C.K.I., Winn, J., Zisserman, A.: · 2014
Earlier work this paper cites.
The role of context for object detection and semantic segmentation in the wild
Mottaghi, R., Chen, X., Liu, X., Cho, N.G., Lee, S.W., Fidler, S., Urtasun, R., Yuille, A.: · 2014
Earlier work this paper cites.
Overfeat: Integrated recognition, localization and detection using convolutional networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: · 2014
Earlier work this paper cites.
Spatial pyramid pooling in deep convolutional networks for visual recognition
He, K., Zhang, X., Ren, S., Sun, J.: · 2014
Earlier work this paper cites.
Rigid-motion scattering for image classification
Sifre, L.: · 2014
Earlier work this paper cites.
Learning visual representations at scale
Vanhoucke, V.: · 2014
Earlier work this paper cites.
Recurrent convolutional neural networks for scene labeling
Pinheiro, P., Collobert, R.: · 2014
Earlier work this paper cites.
Flattened convolutional neural networks for feedforward acceleration
Original
Jin, J., Dundar, A., Culurciello, E.: · 2014
Earlier work this paper cites.
Microsoft COCO: Common objects in context
Lin, T.Y., et al.: · 2014
Earlier work this paper cites.
Distilling the knowledge in a neural network
Hinton, G., Vinyals, O., Dean, J.: · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Simonyan, K., Zisserman, A.: · 2015
Earlier work this paper cites.
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: · 2015
Earlier work this paper cites.
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., Darrell, T.: · 2015
Earlier work this paper cites.

Similar

U-net: Convolutional networks for biomedical image segmentation
Ronneberger, O., Fischer, P., Brox, T.: · 2015
Cited alongside, same era.
Feedforward semantic segmentation with zoom-out features
Mostajabi, M., Yadollahpour, P., Shakhnarovich, G.: · 2015
Cited alongside, same era.
Convolutional feature masking for joint object and stuff segmentation
Dai, J., He, K., Sun, J.: · 2015
Cited alongside, same era.
Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture
Eigen, D., Fergus, R.: · 2015
Cited alongside, same era.
Semantic image segmentation with deep convolutional nets and fully connected crfs
Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: · 2015
Cited alongside, same era.
Material recognition in the wild with the materials in context database
Bell, S., Upchurch, P., Snavely, N., Bala, K.: · 2015
Cited alongside, same era.
Conditional random fields as recurrent neural networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., Torr, P.: · 2015
Cited alongside, same era.
Semantic image segmentation via deep parsing network
Liu, Z., Li, X., Luo, P., Loy, C.C., Tang, X.: · 2015
Cited alongside, same era.
Weakly- and semi-supervised learning of a dcnn for semantic image segmentation
Papandreou, G., Chen, L.C., Murphy, K., Yuille, A.L.: · 2015
Cited alongside, same era.
Fully connected deep structured networks
Original
Schwing, A.G., Urtasun, R.: · 2015
Cited alongside, same era.
Parsenet: Looking wider to see better
Original
Liu, W., Rabinovich, A., Berg, A.C.: · 2015
Cited alongside, same era.
Learning deconvolution network for semantic segmentation
Noh, H., Hong, S., Han, B.: · 2015
Cited alongside, same era.

Then

Segnet: A deep convolutional encoder-decoder architecture for image segmentation
Badrinarayanan, V., Kendall, A., Cipolla, R.: · 2017
Later among the works it cites.
Rethinking atrous convolution for semantic image segmentation
Original
Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: · 2017
Later among the works it cites.
Pyramid scene parsing network
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: · 2017
Later among the works it cites.
Xception: Deep learning with depthwise separable convolutions
Chollet, F.: · 2017
Later among the works it cites.
Mobilenets: Efficient convolutional neural networks for mobile vision applications
Original
Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: · 2017
Later among the works it cites.
Deformable convolutional networks – coco detection and segmentation challenge 2017 entry
Qi, H., Zhang, Z., Xiao, B., Hu, H., Cheng, B., Wei, Y., Dai, J.: · 2017
Later among the works it cites.
Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs
Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: · 2017
Later among the works it cites.
Dense and low-rank gaussian crfs using deep embeddings
Chandra, S., Usunier, N., Kokkinos, I.: · 2017
Later among the works it cites.
Feature pyramid networks for object detection
Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: · 2017
Later among the works it cites.
Dssd: Deconvolutional single shot detector
Original
Fu, C.Y., Liu, W., Ranga, A., Tyagi, A., Berg, A.C.: · 2017
Later among the works it cites.
Refinenet: Multi-path refinement networks with identity mappings for high-resolution semantic segmentation
Lin, G., Milan, A., Shen, C., Reid, I.: · 2017
Later among the works it cites.
Full-resolution residual networks for semantic segmentation in street scenes
Pohlen, T., Hermans, A., Mathias, M., Leibe, B.: · 2017
Later among the works it cites.
Large kernel matters–improve semantic segmentation by global convolutional network
Peng, C., Zhang, X., Yu, G., Luo, G., Sun, J.: · 2017
Later among the works it cites.
Gated feedback refinement network for dense image labeling
Islam, M.A., Rochan, M., Bruce, N.D., Wang, Y.: · 2017
Later among the works it cites.
The devil is in the decoder
Wojna, Z., Ferrari, V., Guadarrama, S., Silberman, N., Chen, L.C., Fathi, A., Uijlings, J.: · 2017
Later among the works it cites.
Stacked deconvolutional network for semantic segmentation
Original
Fu, J., Liu, J., Wang, Y., Lu, H.: · 2017
Later among the works it cites.
Aggregated residual transformations for deep neural networks
Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: · 2017
Later among the works it cites.
Understanding convolution for semantic segmentation
Original
Wang, P., Chen, P., Yuan, Y., Liu, D., Huang, Z., Hou, X., Cottrell, G.: · 2017
Later among the works it cites.
Deformable convolutional networks
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., Wei, Y.: · 2017
Later among the works it cites.
Revisiting unreasonable effectiveness of data in deep learning era
Sun, C., Shrivastava, A., Singh, S., Gupta, A.: · 2017
Later among the works it cites.
Not all pixels are equal: Difficulty-aware semantic segmentation via deep layer cascade
Li, X., Liu, Z., Luo, P., Loy, C.C., Tang, X.: · 2017
Later among the works it cites.
Learning object interactions and descriptions for semantic image segmentation
Wang, G., Luo, P., Lin, L., Wang, X.: · 2017
Later among the works it cites.
Deep dual learning for semantic image segmentation
Luo, P., Wang, G., Lin, L., Wang, X.: · 2017
Later among the works it cites.
COCO-Stuff: Thing and stuff classes in context
Caesar, H., Uijlings, J., Ferrari, V.: · 2018
Closest in time.
Shufflenet: An extremely efficient convolutional neural network for mobile devices
Zhang, X., Zhou, X., Lin, M., Sun, J.: · 2018
Closest in time.
Exfuse: Enhancing feature fusion for semantic segmentation
Zhang, Z., Zhang, X., Peng, C., Cheng, D., Sun, J.: · 2018
Closest in time.
Learning transferable architectures for scalable image recognition
Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: · 2018
Closest in time.
In-place activated batchnorm for memory-optimized training of dnns
Bulò, S.R., Porzi, L., Kontschieder, P.: · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…