2019

Mining YouTube - A dataset for learning fine-grained action concepts from webly supervised video data

Kuehne, Hilde, Iqbal, Ahsan, Richard, Alexander et al.

Understand

Action recognition is so far mainly focusing on the problem of classification of hand selected preclipped actions and reaching impressive results in this field.

But with the performance even ceiling on current datasets, it also appears that the next steps in the field will have to go beyond this fully supervised classification.
One way to overcome those problems is to move towards less restricted scenarios.
In this context we present a large-scale real-world dataset designed to evaluate learning techniques for human action recognition beyond hand-crafted datasets.

Built on

Recognizing human actions: a local svm approach
C. Schuldt, I. Laptev, and B. Caputo · 2004
Earlier work this paper cites.
Actions as space-time shapes
M. Blank, L. Gorelick, E. Shechtman, M. Irani, and R. Basri · 2005
Earlier work this paper cites.
Generating typed dependency parses from phrase structure parses
M.-C. de Marneffe, B. MacCartney, and C. D. Manning · 2006
Earlier work this paper cites.
Learning realistic human actions from movies
I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld · 2008
Earlier work this paper cites.
ImageNet: A Large-Scale Hierarchical Image Database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
HMDB: a large video database for human motion recognition
H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre · 2011
Earlier work this paper cites.
True path rule hierarchical ensembles for genome-wide gene function prediction
G. Valentini · 2011
Earlier work this paper cites.
A database for fine grained activity detection of cooking activities
M. Rohrbach, S. Amin, M. Andriluka, and B. Schiele · 2012
Earlier work this paper cites.
Ucf101: A dataset of 101 human actions classes from videos in the wild
Original
K. Soomro, A. R. Zamir, and M. Shah · 2012
Earlier work this paper cites.
Action recognition with improved trajectories
H. Wang and C. Schmid · 2013
Earlier work this paper cites.
Weakly supervised action labeling in videos under ordering constraints
P. Bojanowski, R. Lajugie, F. Bach, I. Laptev, J. Ponce, C. Schmid, and J. Sivic · 2014
Earlier work this paper cites.

Similar

Empirical evaluation of gated recurrent neural networks on sequence modeling
J. Chung, C. Gulcehre, K. Cho, and Y. Bengio · 2014
Cited alongside, same era.
THUMOS challenge: Action recognition with a large number of classes, 2014
Y.-G. Jiang, J. Liu, A. Roshan Zamir, G. Toderici, I. Laptev, M. Shah, and R. Sukthankar · 2014
Cited alongside, same era.
The language of actions: Recovering the syntax and semantics of goal-directed human activities
H. Kuehne, A. Arslan, and T. Serre · 2014
Cited alongside, same era.
Two-stream convolutional networks for action recognition in videos
K. Simonyan and A. Zisserman · 2014
Cited alongside, same era.
Activitynet: A large-scale video benchmark for human activity understanding
B. G. Fabian Caba Heilbron, Victor Escorcia and J. C. Niebles · 2015
Cited alongside, same era.
What’s cookin’? Interpreting cooking videos using text, speech and vision
J. Malmaud, J. Huang, V. Rathod, N. Johnston, A. Rabinovich, and K. Murphy · 2015
Cited alongside, same era.
Unsupervised semantic parsing of video collections
O. Sener, A. Zamir, S. Savarese, and A. Saxena · 2015
Cited alongside, same era.
Unsupervised learning from narrated instruction videos
J.-B. Alayrac, P. Bojanowski, N. Agrawal, I. Laptev, J. Sivic, and S. Lacoste-Julien · 2016
Cited alongside, same era.
Connectionist temporal modeling for weakly supervised action labeling
D.-A. Huang, L. Fei-Fei, and J. C. Niebles · 2016
Cited alongside, same era.
Temporal tessellation for video annotation and summarization
Original
D. Kaufman, G. Levi, T. Hassner, and L. Wolf · 2016
Cited alongside, same era.
Weakly supervised learning of actions from transcripts
Original
H. Kuehne, A. Richard, and J. Gall · 2016
Cited alongside, same era.

Then

Hollywood in homes: Crowdsourcing data collection for activity understanding
G. A. Sigurdsson, G. Varol, X. Wang, A. Farhadi, I. Laptev, and A. Gupta · 2016
Later among the works it cites.
Temporal segment networks: Towards good practices for deep action recognition
L. Wang, Y. Xiong, Z. Wang, Y. Qiao, D. Lin, X. Tang, and L. Val Gool · 2016
Later among the works it cites.
Look, listen and learn
Original
R. Arandjelovic and A. Zisserman · 2017
Later among the works it cites.
Quo vadis, action recognition? A new model and the kinetics dataset
J. Carreira and A. Zisserman · 2017
Later among the works it cites.
Weakly-supervised action segmentation with iterative soft boundary assignment
L. Ding and C. Xu · 2017
Later among the works it cites.
AVA: A video dataset of spatio-temporally localized atomic visual actions
Original
C. Gu, C. Sun, S. Vijayanarasimhan, C. Pantofaru, D. A. Ross, G. Toderici, Y. Li, S. Ricco, R. Sukthankar, C. Schmid, and J. Malik · 2017
Later among the works it cites.
Weakly supervised semantic segmentation using web-crawled videos
S. Hong, D. Yeo, S. Kwak, H. Lee, and B. Han · 2017
Later among the works it cites.
Weakly supervised action learning with rnn based fine-to-coarse modeling
A. Richard, H. Kuehne, and J. Gall · 2017
Later among the works it cites.
Learning to learn from noisy web videos
S. Yeung, V. Ramanathan, O. Russakovsky, L. Shen, G. Mori, and L. Fei-Fei · 2017
Later among the works it cites.
Towards automatic learning of procedures from web instructional videos
Original
L. Zhou, C. Xu, and J. J. Corso · 2017
Later among the works it cites.
Neuralnetwork-viterbi: A framework for weakly supervised video learning
A. Richard, H. Kuehne, A. Iqbal, and J. Gall · 2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…