2017

Is Second-order Information Helpful for Large-scale Visual Recognition?

Li, Peihua, Xie, Jiangtao, Wang, Qilong et al.

Understand

By stacking layers of convolution and nonlinearity, convolutional networks (ConvNets) effectively learn from low-level to high-level features and discriminative representations.

Since the end goal of large-scale recognition is to delineate complex boundaries of thousands of classes, adequate exploration of feature distributions is important for realizing full potentials of ConvNets.
However, state-of-the-art works concentrate only on deeper or wider architecture design, while rarely exploring feature statistics higher than first-order.
We take a step towards addressing this problem.

Built on

Lectures on the theory of estimation of many parameters
C. Stein · 1986
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner · 1998
Earlier work this paper cites.
A multilinear singular value decomposition
L. D. Lathauwer, B. D. Moor, and J. Vandewalle · 2000
Earlier work this paper cites.
A well-conditioned estimator for large-dimensional covariance matrices
O. Ledoit and M. Wolf · 2004
Earlier work this paper cites.
Geometric means in a novel vector space structure on symmetric positive-definite matrices
V. Arsigny, P. Fillard, X. Pennec, and N. Ayache · 2006
Earlier work this paper cites.
Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories
S. Lazebnik, C. Schmid, and J. Ponce · 2006
Earlier work this paper cites.
A Riemannian framework for tensor computing
X. Pennec, P. Fillard, and N. Ayache · 2006
Earlier work this paper cites.
Non-Euclidean statistics for covariance matrices, with applications to diffusion tensor imaging
I. L. Dryden, A. Koloydenko, and D. Zhou · 2009
Earlier work this paper cites.
Shrinkage algorithms for MMSE covariance estimation
Y. Chen, A. Wiesel, Y. C. Eldar, and A. O. Hero · 2010
Earlier work this paper cites.
Semantic segmentation with second-order pooling
J. Carreira, R. Caseiro, J. Batista, and C. Sminchisescu · 2012
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2012
Earlier work this paper cites.
Image classification with the Fisher vector: Theory and practice
J. Sanchez, F. Perronnin, T. Mensink, and J. Verbeek · 2013
Earlier work this paper cites.

Similar

Return of the devil in the details: Delving deep into convolutional nets
K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman · 2014
Cited alongside, same era.
Network in network
M. Lin, Q. Chen, and S. Yan · 2014
Cited alongside, same era.
Overfeat: Integrated recognition, localization and detection using convolutional networks
P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun · 2014
Cited alongside, same era.
Elementary estimators for sparse covariance matrices and other structured moments
E. Yang, A. Lozano, and P. Ravikumar · 2014
Cited alongside, same era.
Visualizing and understanding convolutional networks
M. D. Zeiler and R. Fergus · 2014
Cited alongside, same era.
Deep filter banks for texture recognition and segmentation
M. Cimpoi, S. Maji, and A. Vedaldi · 2015
Cited alongside, same era.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
K. He, X. Zhang, S. Ren, and J. Sun · 2015
Cited alongside, same era.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
S. Ioffe and C. Szegedy · 2015
Cited alongside, same era.
Matrix backpropagation for deep networks with structured layers
C. Ionescu, O. Vantzos, and C. Sminchisescu · 2015
Cited alongside, same era.
Training deep networks with structured layers by matrix backpropagation
Original
C. Ionescu, O. Vantzos, and C. Sminchisescu · 2015
Cited alongside, same era.
From dictionary of visual words to subspaces: Locality-constrained affine subspace coding
P. Li, X. Lu, and Q. Wang · 2015
Cited alongside, same era.

Then

Bilinear CNN models for fine-grained visual recognition
T.-Y. Lin, A. RoyChowdhury, and S. Maji · 2015
Later among the works it cites.
Imagenet large scale visual recognition challenge
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei · 2015
Later among the works it cites.
Very deep convolutional networks for large-scale image recognition
K. Simonyan and A. Zisserman · 2015
Later among the works it cites.
Going deeper with convolutions
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich · 2015
Later among the works it cites.
Matconvnet – convolutional neural networks for MATLAB
A. Vedaldi and K. Lenc · 2015
Later among the works it cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Later among the works it cites.
RAID-G: Robust estimation of approximate infinite dimensional Gaussian with application to material recognition
Q. Wang, P. Li, W. Zuo, and L. Zhang · 2016
Later among the works it cites.
Regularisation, interpolation and visualisation of diffusion tensor images using non-Euclidean statistics
D. Zhou, I. L. Dryden, A. A. Koloydenko, K. M. Audenaert, and L. Bai · 2016
Later among the works it cites.
Higher-order pooling of CNN features via kernel linearization for action recognition
A. Cherian, P. Koniusz, and S. Gould · 2017
Closest in time.
Higher-order occurrence pooling for bags-of-words: Visual concept detection
P. Koniusz, F. Yan, P. H. Gosselin, and K. Mikolajczyk · 2017
Closest in time.
Local Log-Euclidean multivariate Gaussian descriptor and its application to image classification
P. Li, Q. Wang, H. Zeng, and L. Zhang · 2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…