2018

CCNet: Criss-Cross Attention for Semantic Segmentation

Huang, Zilong, Wang, Xinggang, Wei, Yunchao et al.

Understand

Contextual information is vital in visual understanding problems, such as semantic segmentation and object detection.

We propose a Criss-Cross Network (CCNet) for obtaining full-image contextual information in a very effective and efficient way.
Concretely, for each pixel, a novel criss-cross attention module harvests the contextual information of all the pixels on its criss-cross path.
By taking a further recurrent operation, each pixel can finally capture the full-image dependencies.

Built on

R. T. Azuma, “A survey of augmented reality,”
1997
Earlier work this paper cites.
A. Sperduti and A. Starita, “Supervised neural networks for the classification of structures,”
1997
Earlier work this paper cites.
J. Atwood and D. Towsley, “Diffusion-convolutional neural networks,” in
2001
Earlier work this paper cites.
M. Gori, G. Monfardini, and F. Scarselli, “A new model for learning in graph domains,” in
2005
Earlier work this paper cites.
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,”
2008
Earlier work this paper cites.
G. J. Brostow, J. Shotton, J. Fauqueur, and R. Cipolla, “Segmentation and recognition using structure from motion point clouds,” in
2008
Earlier work this paper cites.
G. J. Brostow, J. Fauqueur, and R. Cipolla, “Semantic object classes in video: A high-definition ground truth database,”
2009
Earlier work this paper cites.
M. Evening,
2012
Earlier work this paper cites.
J. Fritsch, T. Kuehnl, and A. Geiger, “A new performance measure and evaluation benchmark for road detection algorithms,” in
2013
Earlier work this paper cites.
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in
2014
Earlier work this paper cites.
J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in
2015
Earlier work this paper cites.
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Semantic image segmentation with deep convolutional nets and fully connected crfs,”
2015
Earlier work this paper cites.
O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in
2015
Earlier work this paper cites.
Z. Liu, X. Li, P. Luo, C.-C. Loy, and X. Tang, “Semantic image segmentation via deep parsing network,” in
2015
Earlier work this paper cites.
S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, and P. H. Torr, “Conditional random fields as recurrent neural networks,” in
2015
Earlier work this paper cites.
M. Henaff, J. Bruna, and Y. LeCun, “Deep convolutional networks on graph-structured data,”
Original
2015
Earlier work this paper cites.
J. Cheng, L. Dong, and M. Lapata, “Long short-term memory-networks for machine reading,” 2016
2016
Earlier work this paper cites.
M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” in
2016
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in
2016
Earlier work this paper cites.
F. Yu and V. Koltun, “Multi-scale context aggregation by dilated convolutions,”
2016
Earlier work this paper cites.
L.-C. Chen, Y. Yang, J. Wang, W. Xu, and A. L. Yuille, “Attention to scale: Scale-aware semantic image segmentation,” in
2016
Earlier work this paper cites.
M. Defferrard, X. Bresson, and P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” in
2016
Earlier work this paper cites.
T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,”
Original
2016
Earlier work this paper cites.
Z. Wu, C. Shen, and A. v. d. Hengel, “Wider or deeper: Revisiting the resnet model for visual recognition,”
Original
2016
Earlier work this paper cites.
H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” in
2017
Earlier work this paper cites.
L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Rethinking atrous convolution for semantic image segmentation,”
Original
2017
Earlier work this paper cites.

Similar

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in
2017
Cited alongside, same era.
B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba, “Scene parsing through ade20k dataset,” in
2017
Cited alongside, same era.
G. Lin, A. Milan, C. Shen, and I. D. Reid, “Refinenet: Multi-path refinement networks for high-resolution semantic segmentation.” in
2017
Cited alongside, same era.
J. Dai, H. Qi, Y. Xiong, Y. Li, G. Zhang, H. Hu, and Y. Wei, “Deformable convolutional networks,”
2017
Cited alongside, same era.
S. Liu, S. De Mello, J. Gu, G. Zhong, M.-H. Yang, and J. Kautz, “Learning affinity via spatial propagation networks,” in
2017
Cited alongside, same era.
C. Peng, X. Zhang, G. Yu, G. Luo, and J. Sun, “Large kernel matters—improve semantic segmentation by global convolutional network,” in
2017
Cited alongside, same era.
J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, “Neural message passing for quantum chemistry,” in
2017
Cited alongside, same era.
B. De Brabandere, D. Neven, and L. Van Gool, “Semantic instance segmentation with a discriminative loss function,”
Original
2017
Cited alongside, same era.
K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask r-cnn,” in
2017
Cited alongside, same era.
V. Badrinarayanan, A. Kendall, and R. Cipolla, “Segnet: A deep convolutional encoder-decoder architecture for image segmentation,”
2017
Cited alongside, same era.
X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local neural networks,” in
2018
Cited alongside, same era.
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,”
2018
Cited alongside, same era.

Then

M. Yang, K. Yu, C. Zhang, Z. Li, and K. Yang, “Denseaspp for semantic segmentation in street scenes,” in
2018
Closest in time.
L.-C. Chen, M. D. Collins, Y. Zhu, G. Papandreou, B. Zoph, F. Schroff, H. Adam, and J. Shlens, “Searching for efficient multi-scale architectures for dense image prediction,”
2018
Closest in time.
Y. Yuan and J. Wang, “Ocnet: Object context network for scene parsing,”
Original
2018
Closest in time.
R. Levie, F. Monti, X. Bresson, and M. M. Bronstein, “Cayleynets: Graph convolutional neural networks with complex rational spectral filters,”
2018
Closest in time.
P. Wang, P. Chen, Y. Yuan, D. Liu, Z. Huang, X. Hou, and G. Cottrell, “Understanding convolution for semantic segmentation,” in
2018
Closest in time.
X. Liang, H. Zhou, and E. Xing, “Dynamic-structured semantic propagation network,” in
2018
Closest in time.
T. Xiao, Y. Liu, B. Zhou, Y. Jiang, and J. Sun, “Unified perceptual parsing for scene understanding,”
2018
Closest in time.
Z. Huang, X. Wang, L. Huang, C. Huang, Y. Wei, and W. Liu, “Ccnet: Criss-cross attention for semantic segmentation,” in
2019
Closest in time.
B. Cheng, L.-C. Chen, Y. Wei, Y. Zhu, Z. Huang, J. Xiong, T. Huang, W.-M. Hwu, and H. Shi, “Spgnet: Semantic prediction guidance for scene parsing,” in
2019
Closest in time.
T. Ruan, T. Liu, Z. Huang, Y. Wei, S. Wei, and Y. Zhao, “Devil in the details: Towards accurate single and multiple human parsing,” in
2019
Closest in time.
Z. Huang, C. Wang, X. Wang, W. Liu, and J. Wang, “Semantic image segmentation by scale-adaptive networks,”
2019
Closest in time.
J. Jiao, Y. Wei, Z. Jie, H. Shi, R. W. Lau, and T. S. Huang, “Geometry-aware distillation for indoor semantic segmentation,” in
2019
Closest in time.
R. Qian, Y. Wei, H. Shi, J. Li, J. Liu, and T. Huang, “Weakly supervised scene parsing with point-based distance metric learning,” in
2019
Closest in time.
C. Liu, L.-C. Chen, F. Schroff, H. Adam, W. Hua, A. L. Yuille, and L. Fei-Fei, “Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation,” in
2019
Closest in time.
J. He, Z. Deng, L. Zhou, Y. Wang, and Y. Qiao, “Adaptive pyramid context network for semantic segmentation,” in
2019
Closest in time.
J. Fu, J. Liu, H. Tian, Z. Fang, and H. Lu, “Dual attention network for scene segmentation,”
2019
Closest in time.
Y. Chen, M. Rohrbach, Z. Yan, Y. Shuicheng, J. Feng, and Y. Kalantidis, “Graph-based global reasoning networks,” in
2019
Closest in time.
Y. Song, Z. Huang, C. Shen, H. Shi, and D. A. Lange, “Deep learning-based automated image segmentation for concrete petrographic analysis,”
2020
Closest in time.
Z. Zheng, Y. Zhong, J. Wang, and A. Ma, “Foreground-aware relation network for geospatial object segmentation in high spatial resolution remote sensing imagery,” in
2020
Closest in time.
M. T. Chiu, X. Xu, Y. Wei, Z. Huang, A. G. Schwing, R. Brunner, H. Khachatrian, H. Karapetyan, I. Dozier, G. Rose
2020
Closest in time.
M. Tik Chiu, X. Xu, K. Wang, J. Hobbs, N. Hovakimyan, S. H. Huang, Thomas S
2020
Closest in time.
Z. Wang, M. Yu, Y. Wei, R. Feris, J. Xiong, W.-m. Hwu, T. S. Huang, and H. Shi, “Differential treatment for stuff and things: A simple unsupervised domain adaptation method for semantic segmentation,” in
2020
Closest in time.
Z. Wang, Y. Wei, R. Feris, J. Xiong, W.-M. Hwu, T. S. Huang, and H. Shi, “Alleviating semantic-level shift: A semi-supervised domain adaptation method for semantic segmentation,” in
2020
Closest in time.
M. Niepert, M. Ahmed, and K. Kutzkov, “Learning convolutional neural networks for graphs,” in
2023
Closest in time.
R. Zhang, S. Tang, Y. Zhang, J. Li, and S. Yan, “Scale-adaptive convolutions for scene parsing,” in
2039
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…