2015

Convolutional neural networks with low-rank regularization

Tai, Cheng, Xiao, Tong, Zhang, Yi et al.

Understand

Large CNNs have delivered impressive performance in various computer vision applications.

But the storage and computation requirements make it problematic for deploying these models on mobile devices.
Recently, tensor decompositions have been used for speeding up CNNs.
In this paper, we further develop the tensor decomposition technique.

Tensor Rank and the Ill-Posedness of the Best Low-Rank Approximation Problem
de Silva, Vin and Lim, Lek-Heng · 2008
Earlier work this paper cites.
Low-rank matrix approximation with weights or missing data is np-hard
Gillis, Nicolas and Glineur, François · 2011
Earlier work this paper cites.
Improving the speed of neural networks on cpus
Vanhoucke, Vincent, Senior, Andrew, and Mao, Mark Z · 2011
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E · 2012
Earlier work this paper cites.
Learning hierarchical features for scene labeling
Farabet, Clement, Couprie, Camille, Najman, Laurent, and LeCun, Yann · 2013
Earlier work this paper cites.
Maxout networks
Original
Goodfellow, Ian J, Warde-Farley, David, Mirza, Mehdi, Courville, Aaron, and Bengio, Yoshua · 2013
Earlier work this paper cites.
Network In Network
Lin, M., Chen, Q., and Yan, S · 2013
Earlier work this paper cites.

Learning separable filters
Rigamonti, Roberto, Sironi, Amos, Lepetit, Vincent, and Fua, Pascal · 2013
Cited alongside, same era.
Learning activation functions to improve deep neural networks
Original
Agostinelli, Forest, Hoffman, Matthew, Sadowski, Peter, and Baldi, Pierre · 2014
Cited alongside, same era.
Exploiting linear structure within convolutional networks for efficient evaluation
Denton, Emily L, Zaremba, Wojciech, Bruna, Joan, LeCun, Yann, and Fergus, Rob · 2014
Cited alongside, same era.
Speeding up convolutional neural networks with low rank expansions
Original
Jaderberg, Max, Vedaldi, Andrea, and Zisserman, Andrew · 2014
Cited alongside, same era.
Caffe: Convolutional architecture for fast feature embedding
Original
Jia, Yangqing, Shelhamer, Evan, Donahue, Jeff, Karayev, Sergey, Long, Jonathan, Girshick, Ross, Guadarrama, Sergio, and Darrell, Trevor · 2014
Cited alongside, same era.
Speeding-up convolutional neural networks using fine-tuned cp-decomposition
Original
Lebedev, Vadim, Ganin, Yaroslav, Rakhuba, Maksim, Oseledets, Ivan, and Lempitsky, Victor · 2014
Cited alongside, same era.
Fully convolutional networks for semantic segmentation
Original
Long, Jonathan, Shelhamer, Evan, and Darrell, Trevor · 2014
Cited alongside, same era.

Very deep convolutional networks for large-scale image recognition
Original
Simonyan, Karen and Zisserman, Andrew · 2014
Later among the works it cites.
Dropout: A simple way to prevent neural networks from overfitting
Srivastava, Nitish, Hinton, Geoffrey, Krizhevsky, Alex, Sutskever, Ilya, and Salakhutdinov, Ruslan · 2014
Later among the works it cites.
Going deeper with convolutions
Original
Szegedy, Christian, Liu, Wei, Jia, Yangqing, Sermanet, Pierre, Reed, Scott, Anguelov, Dragomir, Erhan, Dumitru, Vanhoucke, Vincent, and Rabinovich, Andrew · 2014
Later among the works it cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Original
Ioffe, Sergey and Szegedy, Christian · 2015
Closest in time.
Tensorizing Neural Networks
Novikov, A., Podoprikhin, D., Osokin, A., and Vetrov, D · 2015
Closest in time.
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, Olga, Deng, Jia, Su, Hao, Krause, Jonathan, Satheesh, Sanjeev, Ma, Sean, Huang, Zhiheng, Karpathy, Andrej, Khosla, Aditya, Bernstein, Michael, Berg, Alexander C., and Fei-Fei, Li · 2015
Closest in time.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…