2019

Asymmetric Non-local Neural Networks for Semantic Segmentation

Zhu, Zhen, Xu, Mengde, Bai, Song et al.

Understand

The non-local module works as a particularly useful technique for semantic segmentation while criticized for its prohibitive computation and GPU memory occupation.

In this paper, we present Asymmetric Non-local Neural Network to semantic segmentation, which has two prominent components: Asymmetric Pyramid Non-local Block (APNB) and Asymmetric Fusion Non-local Block (AFNB).
APNB leverages a pyramid sampling module into the non-local block to largely reduce the computation and memory consumption without sacrificing the performance.
AFNB is adapted from APNB to fuse the features of different levels under a sufficient consideration of long range dependencies and thus considerably improves the performance.

Built on

Conditional random fields: Probabilistic models for segmenting and labeling sequence data
John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira · 2001
Earlier work this paper cites.
A non-local algorithm for image denoising
Antoni Buades, Bartomeu Coll, and Jean-Michel Morel · 2005
Earlier work this paper cites.
Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories
Svetlana Lazebnik, Cordelia Schmid, and Jean Ponce · 2006
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li · 2009
Earlier work this paper cites.
Indoor segmentation and support inference from RGBD images
Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus · 2012
Earlier work this paper cites.
The role of context for object detection and semantic segmentation in the wild
Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-Gyu Cho, Seong-Whan Lee, Sanja Fidler, Raquel Urtasun, and Alan Yuille · 2014
Earlier work this paper cites.
Semantic image segmentation with deep convolutional nets and fully connected crfs
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille · 2015
Earlier work this paper cites.
Spatial pyramid pooling in deep convolutional networks for visual recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Earlier work this paper cites.
Parsenet: Looking wider to see better
Original
Wei Liu, Andrew Rabinovich, and Alexander C. Berg · 2015
Earlier work this paper cites.
Learning deconvolution network for semantic segmentation
Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han · 2015
Earlier work this paper cites.
U-net: Convolutional networks for biomedical image segmentation
Olaf Ronneberger, Philipp Fischer, and Thomas Brox · 2015
Earlier work this paper cites.
Multi-scale context aggregation by dilated convolutions
Original
Fisher Yu and Vladlen Koltun · 2015
Earlier work this paper cites.
Conditional random fields as recurrent neural networks
Shuai Zheng, Sadeep Jayasumana, Bernardino Romera-Paredes, Vibhav Vineet, Zhizhong Su, Dalong Du, Chang Huang, and Philip H. S. Torr · 2015
Earlier work this paper cites.
Fast, exact and multi-scale inference for semantic image segmentation with deep gaussian crfs
Siddhartha Chandra and Iasonas Kokkinos · 2016
Earlier work this paper cites.
The cityscapes dataset for semantic urban scene understanding
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Efficient piecewise training of deep structured models for semantic segmentation
Guosheng Lin, Chunhua Shen, Anton van den Hengel, and Ian D. Reid · 2016
Earlier work this paper cites.
Gaussian conditional random field network for semantic segmentation
Raviteja Vemulapalli, Oncel Tuzel, Ming-Yu Liu, and Rama Chellappa · 2016
Earlier work this paper cites.

Similar

Wider or deeper: Revisiting the resnet model for visual recognition
Original
Zifeng Wu, Chunhua Shen, and Anton van den Hengel · 2016
Cited alongside, same era.
Similarity fusion for visual tracking
Yu Zhou, Xiang Bai, Wenyu Liu, and Longin Jan Latecki · 2016
Cited alongside, same era.
Segnet: A deep convolutional encoder-decoder architecture for image segmentation
Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla · 2017
Cited alongside, same era.
Rethinking atrous convolution for semantic image segmentation
Original
Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam · 2017
Cited alongside, same era.
Refinenet: Multi-path refinement networks for high-resolution semantic segmentation
Guosheng Lin, Anton Milan, Chunhua Shen, and Ian D. Reid · 2017
Cited alongside, same era.
Automatic differentiation in pytorch
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer · 2017
Cited alongside, same era.
Large kernel matters - improve semantic segmentation by global convolutional network
Chao Peng, Xiangyu Zhang, Gang Yu, Guiming Luo, and Jian Sun · 2017
Cited alongside, same era.
Fully convolutional networks for semantic segmentation
Evan Shelhamer, Jonathan Long, and Trevor Darrell · 2017
Cited alongside, same era.
Deep semantic segmentation for automated driving: Taxonomy, roadmap and challenges
Mennatullah Siam, Sara Elkerdawy, Martin Jägersand, and Senthil Yogamani · 2017
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
Scale-adaptive convolutions for scene parsing
Rui Zhang, Sheng Tang, Yongdong Zhang, Jintao Li, and Shuicheng Yan · 2017
Cited alongside, same era.
Pyramid scene parsing network
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia · 2017
Cited alongside, same era.

Then

Dynamic-structured semantic propagation network
Xiaodan Liang, Hongfei Zhou, and Eric Xing · 2018
Later among the works it cites.
A comparative study of real-time semantic segmentation for autonomous driving
Mennatullah Siam, Mostafa Gamal, Moemen Abdel-Razek, Senthil Yogamani, Martin Jägersand, and Hong Zhang · 2018
Later among the works it cites.
Understanding convolution for semantic segmentation
Panqu Wang, Pengfei Chen, Ye Yuan, Ding Liu, Zehua Huang, Xiaodi Hou, and Garrison W. Cottrell · 2018
Later among the works it cites.
Non-local neural networks
Xiaolong Wang, Ross B. Girshick, Abhinav Gupta, and Kaiming He · 2018
Later among the works it cites.
Unified perceptual parsing for scene understanding
Tete Xiao, Yingcheng Liu, Bolei Zhou, Yuning Jiang, and Jian Sun · 2018
Later among the works it cites.
Denseaspp for semantic segmentation in street scenes
Maoke Yang, Kun Yu, Chi Zhang, Zhiwei Li, and Kuiyuan Yang · 2018
Later among the works it cites.
Bisenet: Bilateral segmentation network for real-time semantic segmentation
Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, Gang Yu, and Nong Sang · 2018
Later among the works it cites.
Learning a discriminative feature network for semantic segmentation
Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, Gang Yu, and Nong Sang · 2018
Later among the works it cites.
Deep layer aggregation
Fisher Yu, Dequan Wang, Evan Shelhamer, and Trevor Darrell · 2018
Later among the works it cites.
Ocnet: Object context network for scene parsing
Original
Yuhui Yuan and Jingdong Wang · 2018
Later among the works it cites.
Context encoding for semantic segmentation
Hang Zhang, Kristin J. Dana, Jianping Shi, Zhongyue Zhang, Xiaogang Wang, Ambrish Tyagi, and Amit Agrawal · 2018
Later among the works it cites.
Exfuse: Enhancing feature fusion for semantic segmentation
Zhenli Zhang, Xiangyu Zhang, Chao Peng, Xiangyang Xue, and Jian Sun · 2018
Later among the works it cites.
Psanet: Point-wise spatial attention network for scene parsing
Hengshuang Zhao, Yi Zhang, Shu Liu, Jianping Shi, Chen Change Loy, Dahua Lin, and Jiaya Jia · 2018
Later among the works it cites.
Torchcv: A pytorch-based framework for deep learning in computer vision
Ansheng You, Xiangtai Li, Zhen Zhu, and Yunhai Tong · 2019
Closest in time.
Self-attention generative adversarial networks
Han Zhang, Ian J. Goodfellow, Dimitris N. Metaxas, and Augustus Odena · 2019
Closest in time.
Prior-aware neural network for partially-supervised multi-organ segmentation
Yuyin Zhou, Zhe Li, Song Bai, Chong Wang, Xinlei Chen, Mei Han, Elliot Fishman, and Alan Yuille · 2019
Closest in time.
Semi-supervised 3d abdominal multi-organ segmentation via deep multi-planar co-training
Yuyin Zhou, Yan Wang, Peng Tang, Song Bai, Wei Shen, Elliot Fishman, and Alan Yuille · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…