2017

CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos

Shou, Zheng, Chan, Jonathan, Zareian, Alireza et al.

Understand

Temporal action localization is an important yet challenging problem.

Given a long, untrimmed video consisting of multiple action instances and complex background contents, we need not only to recognize their action categories, but also to localize the start time and end time of each instance.
Many state-of-the-art systems use segment-level classifiers to select and rank proposal segments of pre-determined boundaries.
However, a desirable model should move beyond segment-level and make dense predictions at a fine granularity in time to determine precise temporal boundaries.

Built on

Automatic annotation of human actions in video
O. Duchenne, I. Laptev, J. Sivic, F. Bach, and J. Ponce · 2007
Earlier work this paper cites.
Retrieving actions in movies
I. Laptev and P. Pérez · 2007
Earlier work this paper cites.
Aggregating local descriptors into a compact image representation
H. Jégou, M. Douze, C. Schmid, and P. Pérez · 2010
Earlier work this paper cites.
Improving the fisher kernel for large-scale image classification
F. Perronnin, J. Sánchez, and T. Mensink · 2010
Earlier work this paper cites.
A survey on vision-based human action recognition
R. Poppe · 2010
Earlier work this paper cites.
Deconvolutional networks
M. Zeiler, D. Krishnan, G. W. Taylor, and R. Fergus · 2010
Earlier work this paper cites.
Human activity analysis: A review
J. K. Aggarwal and M. S. Ryoo · 2011
Earlier work this paper cites.
Actom sequence models for efficient action detection
A. Gaidon, Z. Harchaoui, and C. Schmid · 2011
Earlier work this paper cites.
Action Recognition by Dense Trajectories
H. Wang, A. Kläser, C. Schmid, and C.-L. Liu · 2011
Earlier work this paper cites.
A survey of vision-based methods for action representation, segmentation and recognition
D. Weinland, R. Ronfard, and E. Boyer · 2011
Earlier work this paper cites.
Human focused action localization in video
A. Kläser, M. Marszałek, C. Schmid, and A. Zisserman · 2012
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2012
Earlier work this paper cites.
A database for fine grained activity detection of cooking activities
M. A. M. Rohrbach, S. Amin and B. Schiele · 2012
Earlier work this paper cites.
Temporal localization of actions with actoms
A. Gaidon, Z. Harchaoui, and C. Schmid · 2013
Earlier work this paper cites.
Action and event recognition with fisher vectors on a compact feature set
D. Oneata, J. Verbeek, and C. Schmid · 2013
Earlier work this paper cites.
Action recognition with improved trajectories
H. Wang and C. Schmid · 2013
Earlier work this paper cites.
Action localization with tubelets from motion
M. Jain, J. van Gemert, H. Jégou, P. Bouthemy, and C. Snoek · 2014
Earlier work this paper cites.
Caffe: Convolutional architecture for fast feature embedding
Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell · 2014
Earlier work this paper cites.
THUMOS challenge: Action recognition with a large number of classes
Y.-G. Jiang, J. Liu, A. R. Zamir, G. Toderici, I. Laptev, M. Shah, and R. Sukthankar · 2014
Earlier work this paper cites.
Fast saliency based pooling of fisher encoded dense trajectories
S. Karaman, L. Seidenari, and A. D. Bimbo · 2014
Earlier work this paper cites.
Large-scale video classification with convolutional neural networks
A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei · 2014
Earlier work this paper cites.
The lear submission at thumos 2014
D. Oneata, J. Verbeek, and C. Schmid · 2014
Earlier work this paper cites.
Two-stream convolutional networks for action recognition in videos
K. Simonyan and A. Zisserman · 2014
Earlier work this paper cites.
Action recognition and detection by combining motion and appearance features
L. Wang, Y. Qiao, and X. Tang · 2014
Earlier work this paper cites.
Visualizing and understanding convolutional networks
M. Zeiler and R. Fergus · 2014
Earlier work this paper cites.
http://mexculture.cnam.fr/xwiki/bin/view/Datasets/Mex+action+dataset , 2015
Mexaction2 · 2015
Earlier work this paper cites.
Semantic image segmentation with deep convolutional nets and fully connected crfs
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2015
Earlier work this paper cites.

Similar

Long-term recurrent convolutional networks for visual recognition and description
J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell · 2015
Cited alongside, same era.
Devnet: A deep event network for multimedia event detection and evidence recounting
C. Gan, N. Wang, Y. Yang, D.-Y. Yeung, and A. G. Hauptmann · 2015
Cited alongside, same era.
Finding action tubes
G. Gkioxari and J. Malik · 2015
Cited alongside, same era.
THUMOS challenge: Action recognition with a large number of classes
A. Gorban, H. Idrees, Y.-G. Jiang, A. R. Zamir, I. Laptev, M. Shah, and R. Sukthankar · 2015
Cited alongside, same era.
Activitynet: A large-scale video benchmark for human activity understanding
F. C. Heilbron, V. Escorcia, B. Ghanem, and J. C. Niebles · 2015
Cited alongside, same era.
Decoupled deep neural network for semi-supervised semantic segmentation
S. Hong, H. Noh, and B. Han · 2015
Cited alongside, same era.
Objects2action: Classifying and localizing actions without any video example
M. Jain, J. van Gemert, T. Mensink, and C. Snoek · 2015
Cited alongside, same era.
What do 15,000 object categories tell us about classifying and localizing actions?
M. Jain, J. van Gemert, and C. Snoek · 2015
Cited alongside, same era.
Fully convolutional networks for semantic segmentation
J. Long, E. Shelhamer, and T. Darrell · 2015
Cited alongside, same era.
Learning deconvolution network for semantic segmentation
H. Noh, S. Hong, and B. Han · 2015
Cited alongside, same era.
Unsupervised tube extraction using transductive learning and dense trajectories
M. M. Puscas, E. Sangineto, D. Culibrk, and N. Sebe · 2015
Cited alongside, same era.
ImageNet Large Scale Visual Recognition Challenge
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei · 2015
Cited alongside, same era.

Then

Daps: Deep action proposals for action understanding
V. Escorcia, F. C. Heilbron, J. C. Niebles, and B. Ghanem · 2016
Later among the works it cites.
Convolutional two-stream network fusion for video action recognition
C. Feichtenhofer, A. Pinz, and A. Zisserman · 2016
Later among the works it cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Later among the works it cites.
Fast temporal activity proposals for efficient detection of human actions in untrimmed videos
F. C. Heilbron, J. C. Niebles, and B. Ghanem · 2016
Later among the works it cites.
Segmental spatiotemporal cnns for fine-grained action segmentation
C. Lea, A. Reiter, R. Vidal, and G. D. Hager · 2016
Later among the works it cites.
Efficient piecewise training of deep structured models for semantic segmentation
G. Lin, C. Shen, A. van den Hengel, and I. Reid · 2016
Later among the works it cites.
Multi-scale patch aggregation (mpa) for simultaneous detection and segmentation
S. Liu, X. Qi, J. Shi, H. Zhang, and J. Jia · 2016
Later among the works it cites.
Spot on: Action localization from pointly-supervised proposals
P. Mettes, J. van Gemert, and C. Snoek · 2016
Later among the works it cites.
Temporal action detection using a statistical language model
A. Richard and J. Gall · 2016
Later among the works it cites.
Fully convolutional networks for semantic segmentation
E. Shelhamer, J. Long, and T. Darrell · 2016
Later among the works it cites.
Temporal action localization in untrimmed videos via multi-stage cnns
Z. Shou, D. Wang, and S.-F. Chang · 2016
Later among the works it cites.
Much ado about time: Exhaustive annotation of temporal data
G. A. Sigurdsson, O. Russakovsky, A. Farhadi, I. Laptev, and A. Gupta · 2016
Later among the works it cites.
Hollywood in homes: Crowdsourcing data collection for activity understanding
G. A. Sigurdsson, G. Varol, X. Wang, A. Farhadi, I. Laptev, and A. Gupta · 2016
Later among the works it cites.
Action detection
B. Singh · 2016
Later among the works it cites.
A multi-stream bi-directional recurrent neural network for fine-grained action detection
B. Singh, T. K. Marks, M. Jones, O. Tuzel, and M. Shao · 2016
Later among the works it cites.
Untrimmed classification for activity detection: submission to activitynet challenge
G. Singh and F. Cuzzolin · 2016
Later among the works it cites.
Predicting the where and what of actors and actions through online action localization
K. Soomro, H. Idrees, and M. Shah · 2016
Later among the works it cites.
What if we do not have multiple videos of the same action? – video action localization using web images
W. Sultani and M. Shah · 2016
Later among the works it cites.
Deep end2end voxel2voxel prediction
D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri · 2016
Later among the works it cites.
Temporal segment networks: Towards good practices for deep action recognition
L. Wang, Y. Xiong, Z. Wang, Y. Qiao, D. Lin, X. Tang, and L. V. Gool · 2016
Later among the works it cites.
Uts at activitynet 2016
R. Wang and D. Tao · 2016
Later among the works it cites.
Actor-action semantic segmentation with grouping process models
C. Xu and J. J. Corso · 2016
Later among the works it cites.
End-to-end learning of action detection from frame glimpses in videos
S. Yeung, O. Russakovsky, G. Mori, and L. Fei-Fei · 2016
Later among the works it cites.
Multi-scale context aggregation by dilated convolutions
F. Yu and V. Koltun · 2016
Later among the works it cites.
Temporal action localization with pyramid of score distribution features
J. Yuan, B. Ni, X. Yang, and A. Kassim · 2016
Later among the works it cites.
Cdc: Convolutional-de-convolutional networks for precise temporal action localization in untrimmed videos
Original
Z. Shou, J. Chan, A. Zareian, K. Miyazawa, and S.-F. Chang · 2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…