Fetching the paper…
Reading the bibliography…
We analyze the convergence rate of gradient flows on objective functions induced by Dropout and Dropconnect, when applying them to shallow linear Neural Networks (NNs) - which can also be viewed as doing matrix factorization using a particular regularizer.
Inequalities: theory of majorization and its applications
Albert W. Marshall, Ingram Olkin, and Barry C. Arnold · 1979
Earlier work this paper cites.
Neuro-dynamic programming
Dimitri P. Bertsekas and John N. Tsitsiklis · 1996
Earlier work this paper cites.
An introduction to Lie groups and the geometry of homogeneous spaces
Andreas Arvanitogeōrgos · 2003
Earlier work this paper cites.
Stochastic approximation and recursive algorithms and applications
Harold J. Kushner and George G. Yin · 2003
Earlier work this paper cites.
An invitation to algebraic geometry
Karen Smith, Lauri Kahanpää, Pekka Kekäläinen, and William Traves · 2004
Earlier work this paper cites.
Stochastic approximation: a dynamical systems viewpoint
Vivek S. Borkar · 2009
Earlier work this paper cites.
Improving neural networks by preventing co-adaptation of feature detectors
Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhutdinov · 2012
Earlier work this paper cites.
ImageNet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton · 2012
Earlier work this paper cites.
Adaptive dropout for training deep neural networks
Jimmy Ba and Brendan Frey · 2013
Earlier work this paper cites.
Understanding Dropout
Pierre Baldi and Peter J. Sadowski · 2013
Earlier work this paper cites.
Real algebraic geometry
Jacek Bochnak, Michel Coste, and Marie-Françoise Roy · 2013
Earlier work this paper cites.
Smooth manifolds
John M. Lee · 2013
Earlier work this paper cites.
Dropout training as adaptive regularization
Stefan Wager, Sida Wang, and Percy S. Liang · 2013
Earlier work this paper cites.
Regularization of neural networks using dropconnect
Li Wan, Matthew Zeiler, Sixin Zhang, Yann Le Cun, and Rob Fergus · 2013
Cited alongside, same era.
The Dropout learning algorithm
Pierre Baldi and Peter J. Sadowski · 2014
Cited alongside, same era.
Dropout improves recurrent neural networks for handwriting recognition
Vu Pham, Théodore Bluche, Christopher Kermorvant, and Jérôme Louradour · 2014
Cited alongside, same era.
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov · 2014
Cited alongside, same era.
Recurrent neural network regularization
Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals · 2014
Cited alongside, same era.
Variational dropout and the local reparameterization trick
Dropout as a low-rank regularizer for matrix factorization
Jacopo Cavazza, Pietro Morerio, Benjamin Haeffele, Connor Lane, Vittorio Murino, and Rene Vidal · 2018
Later among the works it cites.
A data-driven statistical model for predicting the critical temperature of a superconductor
Kam Hamidieh · 2018
Later among the works it cites.
On the implicit bias of dropout
Poorya Mianjy, Raman Arora, and Rene Vidal · 2018
Later among the works it cites.
Deep learning for drug discovery and cancer research: Automated analysis of vascularization images
Gregor Urban, Kevin Bache, Duc T.T. Phan, Agua Sobrino, Alexander K. Shmakov, Stephanie J. Hachey, Christopher C.W. Hughes, and Pierre Baldi · 2018
Later among the works it cites.
A convergence analysis of gradient descent for deep linear neural networks
Sanjeev Arora, Noah Golowich, Nadav Cohen, and Wei Hu · 2019
Later among the works it cites.
Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Durk P Kingma, Tim Salimans, and Max Welling · 2015
Cited alongside, same era.
Dropconnected neural network trained with diverse features for classifying heart sounds
Edmund Kay and Anurag Agarwal · 2016
Cited alongside, same era.
Improved dropout for shallow and deep learning
Zhe Li, Boqing Gong, and Tianbao Yang · 2016
Cited alongside, same era.
Recurrent dropout without memory loss
Stanislau Semeniuta, Aliaksei Severyn, and Erhardt Barth · 2016
Cited alongside, same era.
Improved regularization of convolutional neural networks with cutout
Terrance DeVries and Graham W. Taylor · 2017
Cited alongside, same era.
Variational dropout sparsifies deep neural networks
Dmitry Molchanov, Arsenii Ashukha, and Dmitry Vetrov · 2017
Cited alongside, same era.
The loss surface of deep and wide neural networks
Quynh Nguyen and Matthias Hein · 2017
Cited alongside, same era.
Bubacarr Bah, Holger Rauhut, Ulrich Terstiege, and Michael Westdickenberg · 2019
Later among the works it cites.
On dropout and nuclear norm regularization
Poorya Mianjy and Raman Arora · 2019
Later among the works it cites.
Convergence rates for the stochastic gradient descent method for non-convex objective functions
Benjamin Fehrman, Benjamin Gess, and Arnulf Jentzen · 2020
Closest in time.
On convergence and generalization of dropout training
Poorya Mianjy and Raman Arora · 2020
Closest in time.
On the regularization properties of structured dropout
Ambar Pal, Connor Lane, René Vidal, and Benjamin D. Haeffele · 2020
Closest in time.
Almost sure convergence of dropout algorithms for neural networks
Albert Senen-Cerda and Jaron Sanders · 2020
Closest in time.
The implicit and explicit regularization effects of dropout
Colin Wei, Sham Kakade, and Tengyu Ma · 2020
Closest in time.