2019

VrR-VG: Refocusing Visually-Relevant Relationships

Liang, Yuanzhi, Bai, Yalong, Zhang, Wei et al.

Understand

Relationships encode the interactions among individual instances, and play a critical role in deep visual scene understanding.

Suffering from the high predictability with non-visual information, existing methods tend to fit the statistical bias rather than ``learning'' to ``infer'' the relationships from images.
To encourage further development in visual relationships, we propose a novel method to automatically mine more valuable relationships by pruning visually-irrelevant ones.
We construct a new scene-graph dataset named Visually-Relevant Relationships Dataset (VrR-VG) based on Visual Genome.

Built on

Hierarchical clustering schemes
Stephen C Johnson · 1967
Earlier work this paper cites.
ImageNet: A Large-Scale Hierarchical Image Database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
The pascal visual object classes (VOC) challenge
Mark Everingham, Luc J. Van Gool, Christopher K. I. Williams, John M. Winn, and Andrew Zisserman · 2010
Earlier work this paper cites.
Recognition using visual phrases
Mohammad Amin Sadeghi and Ali Farhadi · 2011
Earlier work this paper cites.
Rich feature hierarchies for accurate object detection and semantic segmentation
Original
Ross B. Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik · 2013
Earlier work this paper cites.
Selective search for object recognition
Jasper R. R. Uijlings, Koen E. A. van de Sande, Theo Gevers, and Arnold W. M. Smeulders · 2013
Earlier work this paper cites.
Microsoft COCO: common objects in context
Original
Tsung-Yi Lin, Michael Maire, Serge J. Belongie, Lubomir D. Bourdev, Ross B. Girshick, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick · 2014
Earlier work this paper cites.
Glove: Global vectors for word representation
Jeffrey Pennington, Richard Socher, and Christopher D. Manning · 2014
Earlier work this paper cites.
Going deeper with convolutions
Original
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich · 2014
Earlier work this paper cites.
Aligning where to see and what to tell: image caption with region-based attention and scene factorization
Original
Junqi Jin, Kun Fu, Runpeng Cui, Fei Sha, and Changshui Zhang · 2015
Earlier work this paper cites.
Image retrieval using scene graphs
Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Fei-Fei Li · 2015
Earlier work this paper cites.
Making the V in VQA matter: Elevating the role of image understanding in visual question answering
Original
Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Visual genome: Connecting language and vision using crowdsourced dense image annotations
Original
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Fei-Fei Li · 2016
Earlier work this paper cites.

Similar

Visual relationship detection with language priors
Cewu Lu, Ranjay Krishna, Michael S. Bernstein, and Fei-Fei Li · 2016
Cited alongside, same era.
Inception-v4, inception-resnet and the impact of residual connections on learning
Original
Christian Szegedy, Sergey Ioffe, and Vincent Vanhoucke · 2016
Cited alongside, same era.
Aggregated residual transformations for deep neural networks
Original
Saining Xie, Ross B. Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He · 2016
Cited alongside, same era.
MUTAN: multimodal tucker fusion for visual question answering
Original
Hedi Ben-younes, Rémi Cadène, Matthieu Cord, and Nicolas Thome · 2017
Cited alongside, same era.
Detecting visual relationships with deep relational networks
Bo Dai, Yuqi Zhang, and Dahua Lin · 2017
Cited alongside, same era.
Scene graph generation from objects, phrases and region captions
Yikang Li, Wanli Ouyang, Bolei Zhou, Kun Wang, and Xiaogang Wang · 2017
Cited alongside, same era.
Pixels to graphs by associative embedding
Alejandro Newell and Jia Deng · 2017
Cited alongside, same era.
Knowledge graph refinement: A survey of approaches and evaluation methods
Heiko Paulheim · 2017
Cited alongside, same era.
Weakly-supervised learning of visual relations
Julia Peyre, Ivan Laptev, Cordelia Schmid, and Josef Sivic · 2017
Cited alongside, same era.
Faster R-CNN: towards real-time object detection with region proposal networks
Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun · 2017
Cited alongside, same era.
Knowledge graph embedding: A survey of approaches and applications
Quan Wang, Zhendong Mao, Bin Wang, and Li Guo · 2017
Cited alongside, same era.
Scene graph generation by iterative message passing
Danfei Xu, Yuke Zhu, Christopher B. Choy, and Li Fei-Fei · 2017
Cited alongside, same era.

Then

Beyond bilinear: Generalized multi-modal factorized high-order pooling for visual question answering
Original
Zhou Yu, Jun Yu, Chenchao Xiang, Jianping Fan, and Dacheng Tao · 2017
Later among the works it cites.
Visual translation embedding network for visual relation detection
Hanwang Zhang, Zawlin Kyaw, Shih-Fu Chang, and Tat-Seng Chua · 2017
Later among the works it cites.
http://picdataset.com/challenge/index/ , 2018
picdataset.com · 2018
Later among the works it cites.
Bottom-up and top-down attention for image captioning and visual question answering
Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang · 2018
Later among the works it cites.
Scene dynamics: Counterfactual critic multi-agent training for scene graph generation
Original
Long Chen, Hanwang Zhang, Jun Xiao, Xiangnan He, Shiliang Pu, and Shih-Fu Chang · 2018
Later among the works it cites.
Mapping images to scene graphs with permutation-invariant structured prediction
Roei Herzig, Moshiko Raboh, Gal Chechik, Jonathan Berant, and Amir Globerson · 2018
Later among the works it cites.
Learning conditioned graph structures for interpretable visual question answering
Original
Will Norcliffe-Brown, Efstathios Vafeias, and Sarah Parisot · 2018
Later among the works it cites.
Deep graph infomax
Original
Petar Veličković, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm · 2018
Later among the works it cites.
Graph R-CNN for scene graph generation
Jianwei Yang, Jiasen Lu, Stefan Lee, Dhruv Batra, and Devi Parikh · 2018
Later among the works it cites.
Exploring visual relationship for image captioning
Ting Yao, Yingwei Pan, Yehao Li, and Tao Mei · 2018
Later among the works it cites.
Neural motifs: Scene graph parsing with global context
Rowan Zellers, Mark Yatskar, Sam Thomson, and Yejin Choi · 2018
Later among the works it cites.
Large-scale visual relationship understanding
Original
Ji Zhang, Yannis Kalantidis, Marcus Rohrbach, Manohar Paluri, Ahmed M. Elgammal, and Mohamed Elhoseiny · 2018
Later among the works it cites.
Towards knowledge as a service over networks: A deep learning model communication paradigm
Z. Chen, L. Duan, S. Wang, Y. Lou, T. Huang, D. O. Wu, and W. Gao · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…