2017

Detecting and Recognizing Human-Object Interactions

Gkioxari, Georgia, Girshick, Ross, Dollár, Piotr et al.

Understand

To understand the visual world, a machine must not only recognize individual object instances but also how they interact.

Humans are often at the center of such interactions and detecting human-object interactions is an important practical and scientific problem.
In this paper, we address the task of detecting <human, verb, object> triplets in challenging everyday photos.
We propose a novel model that is driven by a human-centric approach.

Built on

Backpropagation applied to handwritten zip code recognition
Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel · 1989
Earlier work this paper cites.
Mixture density networks
C. M. Bishop · 1994
Earlier work this paper cites.
ImageNet: A large-scale hierarchical image database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
Observing human-object interactions: Using spatial and functional compatibility for recognition
A. Gupta, A. Kembhavi, and L. S. Davis · 2009
Earlier work this paper cites.
Recognizing human actions in still images: a study of bag-of-features and part-based representations
V. Delaitre, I. Laptev, and J. Sivic · 2010
Earlier work this paper cites.
The PASCAL Visual Object Classes (VOC) Challenge
M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman · 2010
Earlier work this paper cites.
Rectified linear units improve restricted boltzmann machines
V. Nair and G. E. Hinton · 2010
Earlier work this paper cites.
Modeling mutual context of object and human pose in human-object interaction activities
B. Yao and L. Fei-Fei · 2010
Earlier work this paper cites.
Recognition using visual phrases
M. Sadeghi and A. Farhadi · 2011
Earlier work this paper cites.
ImageNet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. Hinton · 2012
Earlier work this paper cites.
Predicting the location of interactees in novel human-object interactions
C.-Y. Chen and K. Grauman · 2014
Earlier work this paper cites.
Rich feature hierarchies for accurate object detection and semantic segmentation
R. Girshick, J. Donahue, T. Darrell, and J. Malik · 2014
Earlier work this paper cites.

Similar

Spatial pyramid pooling in deep convolutional networks for visual recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2014
Cited alongside, same era.
TUHOI: trento universal human object interaction dataset
D. Le, R. Bernardi, and J. Uijlings · 2014
Cited alongside, same era.
Microsoft COCO: Common objects in context
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick · 2014
Cited alongside, same era.
HICO: A benchmark for recognizing human-object interactions in images
Y.-W. Chao, Z. Wang, Y. He, J. Wang, and J. Deng · 2015
Cited alongside, same era.
Fast R-CNN
R. Girshick · 2015
Cited alongside, same era.
Contextual action recognition with R*CNN
G. Gkioxari, R. Girshick, and J. Malik · 2015
Cited alongside, same era.
Visual semantic role labeling
S. Gupta and J. Malik · 2015
Cited alongside, same era.
Learning semantic relationships for better action retrieval in images
V. Ramanathan, C. Li, J. Deng, W. Han, Z. Li, K. Gu, Y. Song, S. Bengio, C. Rossenberg, and L. Fei-Fei · 2015
Cited alongside, same era.
Faster R-CNN: Towards real-time object detection with region proposal networks
S. Ren, K. He, R. Girshick, and J. Sun · 2015
Cited alongside, same era.
Describing common human visual actions in images
M. R. Ronchi and P. Perona · 2015
Cited alongside, same era.
Very deep convolutional networks for large-scale image recognition
K. Simonyan and A. Zisserman · 2015
Cited alongside, same era.

Then

Going deeper with convolutions
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich · 2015
Later among the works it cites.
Visual Madlibs: Fill in the blank Image Generation and Question Answering
L. Yu, E. Park, A. C. Berg, and T. L. Berg · 2015
Later among the works it cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Later among the works it cites.
Visual relationship detection with language priors
C. Lu, R. Krishna, M. Bernstein, and L. Fei-Fei · 2016
Later among the works it cites.
Learning models for actions and person-object interactions with transfer to question answering
A. Mallya and S. Lazebnik · 2016
Later among the works it cites.
Convolutional pose machines
S.-E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh · 2016
Later among the works it cites.
Situation recognition: Visual semantic role labeling for image understanding
M. Yatskar, L. Zettlemoyer, and A. Farhadi · 2016
Later among the works it cites.
Realtime multi-person 2D pose estimation using part affinity fields
Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh · 2017
Closest in time.
Learning to detect human-object interactions
Original
Y.-W. Chao, Y. Liu, X. Liu, H. Zeng, and J. Deng · 2017
Closest in time.
Mask R-CNN
K. He, G. Gkioxari, P. Dollár, and R. Girshick · 2017
Closest in time.
Feature pyramid networks for object detection
T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie · 2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…