Fetching the paper…
Reading the bibliography…
In this paper, we introduce the algorithms of Orthogonal Deep Neural Networks (OrthDNNs) to connect with recent interest of spectrally regularized deep learning methods.
Neocognitron: A self-organizing neural network for a mechanism of pattern recognition unaffected by shift in position
K. Fukushima · 1980
Earlier work this paper cites.
Multilayer feedforward networks are universal approximators
K. Hornik, M. Stinchcombe, and H. White · 1989
Earlier work this paper cites.
Arrangements of Hyperplanes
P. Orlik and H. Terao · 1992
Earlier work this paper cites.
Universal approximation bounds for superpositions of a sigmoidal function
Andrew R. Barron · 1993
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner · 1998
Earlier work this paper cites.
Efficient backprop
Yann LeCun, Léon Bottou, Genevieve B. Orr, and Klaus-Robert Müller · 1998
Earlier work this paper cites.
Stability and generalization
Olivier Bousquet and André Elisseeff · 2002
Earlier work this paper cites.
ϵ \epsilon -entropy and ϵ \epsilon -capacity of sets in functional spaces
A. N. Kolmogorov and V. Tihomirov · 2002
Earlier work this paper cites.
Decoding by linear programming
E. J. Candes and T. Tao · 2005
Earlier work this paper cites.
Optimization Algorithms on Matrix Manifolds
P. A. Absil, R. Mahony, and R. Sepulchre · 2007
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky · 2009
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio · 2010
Earlier work this paper cites.
Deep sparse rectifier neural networks
Xavier Glorot, Antoine Bordes, and Yoshua Bengio · 2011
Earlier work this paper cites.
Improving neural networks by preventing co-adaptation of feature detectors
Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov · 2012
Earlier work this paper cites.
Foundations of Machine Learning
Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar · 2012
Earlier work this paper cites.
Robustness and generalization
Huan Xu and Shie Mannor · 2012
Earlier work this paper cites.
Understanding dropout
Pierre Baldi and Peter J Sadowski · 2013
Earlier work this paper cites.
Stochastic gradient descent on riemannian manifolds
S. Bonnabel · 2013
Earlier work this paper cites.
On the importance of initialization and momentum in deep learning
Ilya Sutskever, James Martens, George E. Dahl, and Geoffrey E. Hinton · 2013
Earlier work this paper cites.
Distance Preserving Embeddings for General n-Dimensional Manifolds
Nakul Verma · 2013
Earlier work this paper cites.
Dropout training as adaptive regularization
Stefan Wager, Sida Wang, and Percy S Liang · 2013
Earlier work this paper cites.
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
Yann N. Dauphin, Razvan Pascanu, Çaglar Gülçehre, KyungHyun Cho, Surya Ganguli, and Yoshua Bengio · 2014
Cited alongside, same era.
On the number of linear regions of deep neural networks
Guido Montúfar, Razvan Pascanu, Kyunghyun Cho, and Yoshua Bengio · 2014
Cited alongside, same era.
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
Andrew M. Saxe, James L. McClelland, and Surya Ganguli · 2014
Cited alongside, same era.
Very deep convolutional networks for large-scale image recognition
K. Simonyan and A. Zisserman · 2014
Cited alongside, same era.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Cited alongside, same era.
On the effect of pooling on the geometry of representations
Gary Bécigneul · 2017
Later among the works it cites.
Parseval networks: Improving robustness to adversarial examples
Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier · 2017
Later among the works it cites.
Sharp minima can generalize for deep nets
Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio · 2017
Later among the works it cites.
A study and comparison of human and deep learning recognition performance under visual distortions
Samuel Dodge and Lina Karam · 2017
Later among the works it cites.
Comparing deep neural networks against humans: object recognition when the signal gets weaker
R. Geirhos, D. H. J. Janssen, H. H. Schütt, J. Rauber, M. Bethge, and F. A. Wichmann · 2017
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Discriminative robust transformation learning
Jiaji Huang, Qiang Qiu, Guillermo Sapiro, and Robert Calderbank · 2015
Cited alongside, same era.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Cited alongside, same era.
Adam: A method for stochastic optimization
Diederik Kingma and Jimmy Ba · 2015
Cited alongside, same era.
Deeply-supervised nets
Chen-Yu Lee, Saining Xie, Patrick W. Gallagher, Zhengyou Zhang, and Zhuowen Tu · 2015
Cited alongside, same era.
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei · 2015
Cited alongside, same era.
Unitary evolution recurrent neural networks
Martin Arjovsky, Amar Shah, and Yoshua Bengio · 2016
Cited alongside, same era.
Train faster, generalize better: Stability of stochastic gradient descent
Moritz Hardt, Ben Recht, and Yoram Singer · 2016
Cited alongside, same era.
Improving training of deep neural networks via singular value bounding
Kui Jia, Dacheng Tao, Shenghua Gao, and Xiangmin Xu · 2017
Later among the works it cites.
Generalization in deep learning
Kenji Kawaguchi, Leslie Pack Kaelbling, and Yoshua Bengio · 2017
Later among the works it cites.
On large-batch training for deep learning: Generalization gap and sharp minima
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang · 2017
Later among the works it cites.
Data-dependent stability of stochastic gradient descent
Ilja Kuzborskij and Christoph H. Lampert · 2017
Later among the works it cites.
Algorithmic stability and hypothesis complexity
Tongliang Liu, Gábor Lugosi, Gergely Neu, and Dacheng Tao · 2017
Later among the works it cites.
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice
Jeffrey Pennington, Samuel S. Schoenholz, and Surya Ganguli · 2017
Later among the works it cites.
Robust large margin deep neural networks
Jure Sokolic, Raja Giryes, Guillermo Sapiro, and Miguel R. D. Rodrigues · 2017
Later among the works it cites.
All you need is beyond a good init: Exploring better solution for training extremely deep convolutional neural networks with orthonormality and modulation
Di Xie and Jiang Xiongand Shiliang Pu · 2017
Later among the works it cites.
Aggregated residual transformations for deep neural networks
Saining Xie, Ross B. Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He · 2017
Later among the works it cites.
Understanding deep learning requires rethinking generalization
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals · 2017
Later among the works it cites.
Can we gain more from orthogonality regularizations in training deep cnns?
Nitin Bansal, Xiaohan Chen, and Zhangyang Wang · 2018
Later among the works it cites.
Can we gain more from orthogonality regularizations in training deep networks?
Nitin Bansal, Xiaohan Chen, and Zhangyang Wang · 2018
Later among the works it cites.
Global optimality conditions for deep neural networks
Chulhee Yun, Suvrit Sra, and Ali Jadbabaie · 2018
Later among the works it cites.
Theory of deep learning iib: Optimization properties of SGD
Chiyuan Zhang, Qianli Liao, Alexander Rakhlin, Brando Miranda, Noah Golowich, and Tomaso A. Poggio · 2018
Later among the works it cites.
Benchmarking neural network robustness to common corruptions and perturbations
Dan Hendrycks and Thomas Dietterich · 2019
Closest in time.
The singular values of convolutional layers
Hanie Sedghi, Vineet Gupta, and Philip M. Long · 2019
Closest in time.