Fetching the paper…
Reading the bibliography…
This paper seeks to answer the question: as the (near-) orthogonality of weights is found to be a favorable property for training deep convolutional neural networks, how can we enforce it in more effective and easy-to-use ways? We develop novel orthogonality regularizations on training deep CNNs, utilizing various advanced analytical tools such as mutual coherence and restricted isometry property.
Decoding by linear programming
Emmanuel J Candes and Terence Tao · 2005
Earlier work this paper cites.
Special paraunitary matrices, cayley transform, and multidimensional orthogonal filter banks
Jianping Zhou, Minh N Do, and Jelena Kovacevic · 2006
Earlier work this paper cites.
Compressed sensing
David L Donoho · 2006
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio · 2010
Earlier work this paper cites.
Sparse recovery with orthogonal matching pursuit under rip
Tong Zhang · 2011
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
Andrew M Saxe, James L McClelland, and Surya Ganguli · 2013
Earlier work this paper cites.
On the difficulty of training recurrent neural networks
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio · 2013
Earlier work this paper cites.
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
Yann N Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, and Yoshua Bengio · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman · 2014
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Natural neural networks
Guillaume Desjardins, Karen Simonyan, Razvan Pascanu, et al · 2015
Earlier work this paper cites.
Dmytro Mishkin and Jiri Matas · 2015
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Earlier work this paper cites.
Optimized projections for compressed sensing via direct mutual coherence minimization
Zhouchen Lin, Canyi Lu, and Huan Li · 2015
Cited alongside, same era.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Cited alongside, same era.
Regularizing cnns with locally constrained decorrelations
Pau Rodríguez, Jordi Gonzalez, Guillem Cucurull, Josep M Gonfaus, and Xavier Roca · 2016
Cited alongside, same era.
Improving training of deep neural networks via singular value bounding
Kui Jia, Dacheng Tao, Shenghua Gao, and Xiangmin Xu · 2016
Cited alongside, same era.
Generalized backpropagation, \ \backslash ’ { \{ E } \} tude de cas: Orthogonality
On large-batch training for deep learning: Generalization gap and sharp minima
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang · 2016
Later among the works it cites.
Identity mappings in deep residual networks
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Later among the works it cites.
Di Xie, Jiang Xiong, and Shiliang Pu · 2017
Later among the works it cites.
Lei Huang, Xianglong Liu, Bo Lang, Adams Wei Yu, and Bo Li · 2017
Later among the works it cites.
Svdnet for pedestrian retrieval
Yifan Sun, Liang Zheng, Weijian Deng, and Shengjin Wang · 2017
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Mehrtash Harandi and Basura Fernando · 2016
Cited alongside, same era.
Optimization on submanifolds of convolution kernels in cnns
Mete Ozay and Takayuki Okatani · 2016
Cited alongside, same era.
Sparse Coding and its Applications in Computer Vision
Zhaowen Wang, Jianchao Yang, Haichao Zhang, Zhangyang Wang, Yingzhen Yang, Ding Liu, and Thomas S Huang · 2016
Cited alongside, same era.
Sergey Zagoruyko and Nikos Komodakis · 2016
Cited alongside, same era.
Weight normalization: A simple reparameterization to accelerate training of deep neural networks
Tim Salimans and Diederik P Kingma · 2016
Cited alongside, same era.
Dizzyrnn: Reparameterizing recurrent neural networks for norm-preserving backpropagation
Victor Dorobantu, Per Andre Stromhaug, and Jess Renteria · 2016
Cited alongside, same era.
Unitary evolution recurrent neural networks
Martin Arjovsky, Amar Shah, and Yoshua Bengio · 2016
Cited alongside, same era.
Efficient orthogonal parametrisation of recurrent neural networks using householder reflections
Zakaria Mhammedi, Andrew Hellicar, Ashfaqur Rahman, and James Bailey · 2016
Cited alongside, same era.
Later among the works it cites.
Aggregated residual transformations for deep neural networks
Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He · 2017
Later among the works it cites.
On orthogonality and learning recurrent networks with long term dependencies
Eugene Vorontsov, Chiheb Trabelsi, Samuel Kadoury, and Chris Pal · 2017
Later among the works it cites.
Spectral norm regularization for improving the generalizability of deep learning
Yuichi Yoshida and Takeru Miyato · 2017
Later among the works it cites.
Robust large margin deep neural networks
Jure Sokolić, Raja Giryes, Guillermo Sapiro, and Miguel RD Rodrigues · 2017
Later among the works it cites.
A spline theory of deep networks
Randall Balestriero and Richard Baraniuk · 2018
Closest in time.
Mad max: Affine spline insights into deep learning
Randall Balestriero and Richard Baraniuk · 2018
Closest in time.
Spectral normalization for generative adversarial networks
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida · 2018
Closest in time.
Learning simple thresholded features with sparse support recovery
Zhangyang Wang, Hongyu Xu, Haichuan Yang, Ding Liu, and Ji Liu · 2018
Closest in time.