2015

Contextual Action Recognition with R*CNN

Gkioxari, Georgia, Girshick, Ross, Malik, Jitendra

Understand

There are multiple cues in an image which reveal what action a person is performing.

For example, a jogger has a pose that is characteristic for jogging, but the scene (e.g.
road, trail) and the presence of other joggers can be an additional source of information.
In this work, we exploit the simple observation that actions are accompanied by contextual cues to build a strong action recognition system.

Built on

Scene perception detecting and judging objects undergoing relational violations
I. Biederman, R. J. Mezzanotte, and J. C. Rabinowitz · 1982
Earlier work this paper cites.
Backpropagation applied to handwritten zip code recognition
Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel · 1989
Earlier work this paper cites.
A framework for multiple instance learning
O. Maron and T. Lozano-Pérez · 1998
Earlier work this paper cites.
Distinctive image features from scale-invariant keypoints
D. Lowe · 2004
Earlier work this paper cites.
Histograms of oriented gradients for human detection
N. Dalal and B. Triggs · 2005
Earlier work this paper cites.
Multiple instance boosting for object detection
P. Viola, J. Platt, and C. Zhang · 2005
Earlier work this paper cites.
The role of context in object recognition
A. Oliva and A. Torralba · 2007
Earlier work this paper cites.
The PASCAL Visual Object Classes (VOC) Challenge
M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman · 2010
Earlier work this paper cites.
Object detection with discriminatively trained part based models
P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan · 2010
Earlier work this paper cites.
Describing people: Poselet-based attribute classification
L. Bourdev, S. Maji, and J. Malik · 2011
Earlier work this paper cites.
Action recognition from a distributed representation of pose and appearance
S. Maji, L. Bourdev, and J. Malik · 2011
Earlier work this paper cites.
Human action recognition by learning bases of action attributes and parts
B. Yao, X. Jiang, A. Khosla, A. L. Lin, L. Guibas, and L. Fei-Fei · 2011
Earlier work this paper cites.

Similar

Combining randomization and discrimination for fine-grained image categorization
B. Yao, A. Khosla, and L. Fei-Fei · 2011
Cited alongside, same era.
http://pascallin.ecs.soton.ac.uk/challenges/voc/voc2012/, 2012
2012
Cited alongside, same era.
ImageNet Large Scale Visual Recognition Competition 2012 (ILSVRC2012)
J. Deng, A. Berg, S. Satheesh, H. Su, A. Khosla, and L. Fei-Fei · 2012
Cited alongside, same era.
ImageNet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. Hinton · 2012
Cited alongside, same era.
Weakly supervised learning of interactions between humans and objects
A. Prest, C. Schmid, and V. Ferrari · 2012
Cited alongside, same era.
Selective search for object recognition
J. R. R. Uijlings, K. E. A. van de Sande, T. Gevers, and A. W. M. Smeulders · 2013
Cited alongside, same era.
Action recognition with improved trajectories
H. Wang and C. Schmid · 2013
Cited alongside, same era.
2d human pose estimation: New benchmark and state of the art analysis
M. Andriluka, L. Pishchulin, P. Gehler, and B. Schiele · 2014
Cited alongside, same era.
Rich feature hierarchies for accurate object detection and semantic segmentation
R. Girshick, J. Donahue, T. Darrell, and J. Malik · 2014
Cited alongside, same era.
Spatial pyramid pooling in deep convolutional networks for visual recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2014
Cited alongside, same era.
Regularized max pooling for image categorization
M. Hoai · 2014
Cited alongside, same era.
Action recognition from weak alignment of body parts
M. Hoai, L. Ladicky, and A. Zisserman · 2014
Cited alongside, same era.

Then

Learning and transferring mid-level image representations using convolutional neural networks
M. Oquab, L. Bottou, I. Laptev, and J. Sivic · 2014
Later among the works it cites.
Weakly supervised object recognition with convolutional neural networks
M. Oquab, L. Bottou, I. Laptev, and J. Sivic · 2014
Later among the works it cites.
Fine-grained activity recognition with holistic and pose based features
L. Pishchulin, M. Andriluka, and B. Schiele · 2014
Later among the works it cites.
Two-stream convolutional networks for action recognition in videos
K. Simonyan and A. Zisserman · 2014
Later among the works it cites.
On learning to localize objects with minimal supervision
H. O. Song, R. Girshick, S. Jegelka, J. Mairal, Z. Harchaoui, and T. Darrell · 2014
Later among the works it cites.
PANDA: Pose aligned networks for deep attribute modeling
N. Zhang, M. Paluri, M. Ranzato, T. Darrell, and L. Bourdev · 2014
Later among the works it cites.
From captions to visual concepts and back
H. Fang, S. Gupta, F. N. Iandola, R. Srivastava, L. Deng, P. Dollár, J. Gao, X. He, M. Mitchell, J. C. Platt, C. L. Zitnick, and G. Zweig · 2015
Closest in time.
Fast R-CNN
R. Girshick · 2015
Closest in time.
Actions and attributes from wholes and parts
G. Gkioxari, R. Girshick, and J. Malik · 2015
Closest in time.
Finding action tubes
G. Gkioxari and J. Malik · 2015
Closest in time.
Very deep convolutional networks for large-scale image recognition
K. Simonyan and A. Zisserman · 2015
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…