2021

Conterfactual Generative Zero-Shot Semantic Segmentation

Shen, Feihong, Liu, Jun, Hu, Ping

Understand

zero-shot learning is an essential part of computer vision.

As a classical downstream task, zero-shot semantic segmentation has been studied because of its applicant value.
One of the popular zero-shot semantic segmentation methods is based on the generative model Most new proposed works added structures on the same architecture to enhance this model.
However, we found that, from the view of causal inference, the result of the original model has been influenced by spurious statistical relationships.

Built on

Visualizing data using t-sne
Laurens Van der Maaten and Geoffrey Hinton · 2008
Earlier work this paper cites.
Causal inference in statistics: An overview
Judea Pearl et al · 2009
Earlier work this paper cites.
Transforming auto-encoders
Geoffrey E Hinton, Alex Krizhevsky, and Sida D Wang · 2011
Earlier work this paper cites.
Distributed representations of words and phrases and their compositionality
Original
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean · 2013
Earlier work this paper cites.
Zero-shot learning through cross-modal transfer
Original
Richard Socher, Milind Ganjoo, Hamsa Sridhar, Osbert Bastani, Christopher D Manning, and Andrew Y Ng · 2013
Earlier work this paper cites.
Generative adversarial networks
Original
Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio · 2014
Earlier work this paper cites.
The role of context for object detection and semantic segmentation in the wild
Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-Gyu Cho, Seong-Whan Lee, Sanja Fidler, Raquel Urtasun, and Alan Yuille · 2014
Earlier work this paper cites.
Fully convolutional networks for semantic segmentation
Jonathan Long, Evan Shelhamer, and Trevor Darrell · 2015
Earlier work this paper cites.
Zero-shot learning via semantic similarity embedding
Ziming Zhang and Venkatesh Saligrama · 2015
Earlier work this paper cites.
Fasttext. zip: Compressing text classification models
Original
Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hérve Jégou, and Tomas Mikolov · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
An empirical study and analysis of generalized zero-shot learning for object recognition in the wild
Wei-Lun Chao, Soravit Changpinyo, Boqing Gong, and Fei Sha · 2016
Earlier work this paper cites.

Similar

Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille · 2017
Cited alongside, same era.
Encoder-decoder with atrous separable convolution for semantic image segmentation
Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam · 2018
Cited alongside, same era.
Feature generating networks for zero-shot learning
Yongqin Xian, Tobias Lorenz, Bernt Schiele, and Zeynep Akata · 2018
Cited alongside, same era.
Recent advances in zero-shot recognition: Toward data-efficient understanding of visual content
Yanwei Fu, Tao Xiang, Yu-Gang Jiang, Xiangyang Xue, Leonid Sigal, and Shaogang Gong · 2018
Cited alongside, same era.
Zero-shot recognition via semantic embeddings and knowledge graphs
Xiaolong Wang, Yufei Ye, and Abhinav Gupta · 2018
Cited alongside, same era.
Zero-shot semantic segmentation
Original
Maxime Bucher, Tuan-Hung Vu, Matthieu Cord, and Patrick Pérez · 2019
Cited alongside, same era.
Semantic projection network for zero-and few-label semantic segmentation
Yongqin Xian, Subhabrata Choudhury, Yang He, Bernt Schiele, and Zeynep Akata · 2019
Cited alongside, same era.
Counterfactual visual explanations
Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee · 2019
Cited alongside, same era.
Class-balanced loss based on effective number of samples
Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie · 2019
Cited alongside, same era.
Self-supervised visual feature learning with deep neural networks: A survey
Longlong Jing and Yingli Tian · 2020
Cited alongside, same era.
Learning unbiased zero-shot semantic segmentation networks via transductive transfer
Fengmao Lv, Haiyang Liu, Yichen Wang, Jiayi Zhao, and Guowu Yang · 2020
Cited alongside, same era.
Context-aware feature generation for zero-shot semantic segmentation
Zhangxuan Gu, Siyuan Zhou, Li Niu, Zihan Zhao, and Liqing Zhang · 2020
Cited alongside, same era.

Then

Cap2seg: Inferring semantic and spatial context from captions for zero-shot image segmentation
Guiyu Tian, Shuai Wang, Jie Feng, Li Zhou, and Yadong Mu · 2020
Later among the works it cites.
Long-tailed classification by keeping the good and removing the bad momentum causal effect
Original
Kaihua Tang, Jianqiang Huang, and Hanwang Zhang · 2020
Later among the works it cites.
Uncertainty-aware learning for zero-shot semantic segmentation
Ping Hu, Stan Sclaroff, and Kate Saenko · 2020
Later among the works it cites.
Interventional few-shot learning
Original
Zhongqi Yue, Hanwang Zhang, Qianru Sun, and Xian-Sheng Hua · 2020
Later among the works it cites.
Deconfounded image captioning: A causal retrospect
Original
Xu Yang, Hanwang Zhang, and Jianfei Cai · 2020
Later among the works it cites.
Unbiased scene graph generation from biased training
Kaihua Tang, Yulei Niu, Jianqiang Huang, Jiaxin Shi, and Hanwang Zhang · 2020
Later among the works it cites.
Counterfactual variable control for robust and interpretable question answering
Original
Sicheng Yu, Yulei Niu, Shuohang Wang, Jing Jiang, and Qianru Sun · 2020
Later among the works it cites.
Counterfactual vqa: A cause-effect look at language bias
Original
Yulei Niu, Kaihua Tang, Hanwang Zhang, Zhiwu Lu, Xian-Sheng Hua, and Ji-Rong Wen · 2020
Later among the works it cites.
Consistent structural relation learning for zero-shot segmentation
Peike Li, Yunchao Wei, and Yi Yang · 2020
Later among the works it cites.
Unsupervised domain adaptation in semantic segmentation: a review
Marco Toldo, Andrea Maracani, Umberto Michieli, and Pietro Zanuttigh · 2020
Later among the works it cites.
Counterfactual zero-shot and open-set visual recognition
Original
Zhongqi Yue, Tan Wang, Hanwang Zhang, Qianru Sun, and Xian-Sheng Hua · 2021
Closest in time.
Toward causal representation learning
Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…