Fetching the paper…
Reading the bibliography…
By using the viewpoint of modern computational algebraic geometry, we explore properties of the optimization landscapes of the deep linear neural network models.
M. Kac, “On the average number of real roots of a random algebraic equation,” Bull. Am. Math. Soc
1943
Earlier work this paper cites.
M. Kac, “On the average number of real roots of a random algebraic equation (ii),” Proceedings of the London Mathematical Society
1948
Earlier work this paper cites.
Benjamin, 1967
R. H. Abraham and J. W. Robbin, Transversal Mappings and Flows · 1967
Earlier work this paper cites.
D. N. Bernshtein, “The number of roots of a system of equations,” Functional Analysis and its Applications
1975
Earlier work this paper cites.
R. Brockett, “Some geometric questions in the theory of linear systems,” IEEE Transactions on Automatic Control
1976
Earlier work this paper cites.
John Wiley & Sons, New York, 1979
E. L. Allgower and K. Georg, Introduction to Numerical Continuation Methods · 1979
Earlier work this paper cites.
A. Morgan and A. J. Sommese, “Computing all solutions to polynomial systems using homotopy continuation,” Applied Mathematics and Computation
1987
Earlier work this paper cites.
A. Blum and R. Rivest, “Training a 3-node neural network is np-complete,” in Proceedings of the 1st International Conference on Neural Information Processing Systems
1988
Earlier work this paper cites.
P. Baldi and K. Hornik, “Neural networks and principal component analysis: Learning from examples without local minima,” Neural networks
1989
Earlier work this paper cites.
E. D. Sontag and H. J. Sussmann, “Backpropagation can give rise to spurious local minima even for networks without hidden layers,” Complex Systems
1989
Earlier work this paper cites.
M. Gori and A. Tesi, “On the problem of local minima in backpropagation,” IEEE Transactions on Pattern Analysis and Machine Intelligence
1992
Earlier work this paper cites.
G. M. Georgiou and C. Koutsougeras, “Complex domain backpropagation,” IEEE transactions on Circuits and systems II: analog and digital signal processing
1992
Earlier work this paper cites.
H. J. Sussmann, “Uniqueness of the weights for minimal feedforward nets with a given input-output map,” Neural networks
1992
Earlier work this paper cites.
J. C. Chow, L. Udpa, and S. Udpa, “New training algorithm for neural networks,” in Review of Progress in Quantitative Nondestructive Evaluation
1992
Earlier work this paper cites.
A. M. Chen, H.-M. Lu, and R. Hecht-Nielsen, “On the geometry of feedforward neural network error surfaces,” Neural computation
1993
Earlier work this paper cites.
V. Kůrková and P. C. Kainen, “Functionally equivalent feedforward neural networks,” Neural Computation
1994
Earlier work this paper cites.
X.-H. Yu and G.-A. Chen, “On the local minima free condition of backpropagation learning,” IEEE Transactions on Neural Networks
1995
Earlier work this paper cites.
P. F. Baldi and K. Hornik, “Learning in linear neural networks: A survey,” IEEE Transactions on neural networks
1995
Earlier work this paper cites.
R. S. Zemel, C. K. Williams, and M. C. Mozer, “Lending direction to neural networks,” Neural Networks
1995
Earlier work this paper cites.
S. Hochreiter and J. Schmidhuber, “Simplifying neural nets by discovering flat minima,” in Advances in neural information processing systems
1995
Earlier work this paper cites.
D. Saad and S. A. Solla, “On-line learning in soft committee machines,” Physical Review E
1995
Earlier work this paper cites.
B. Huber and B. Sturmfels, “A polyhedral method for solving sparse polynomial systems,” Mathematics of Computation
1995
Earlier work this paper cites.
A. Edelman and E. Kostlan, “How many zeros of a random polynomial are real?,” Bulletin of the American Mathematical Society
1995
Earlier work this paper cites.
F. M. Coetzee and V. Stonick, “Homotopy approaches for the analysis and solution of neural network and other nonlinear systems of equations,” Doctoral Thesis, Carnegie Mellon University, May
1995
Earlier work this paper cites.
T. Y. Li and X. Wang, “The BKK root count in C n C^{n} , url = http://www.ams.org/mcom/1996-65-216/S0025-5718-96-00778-8/, volume = 65, year = 1996,” Mathematics of Computation
1996
Earlier work this paper cites.
M. J. Rojas and X. Wang, “Counting affine roots of polynomial systems via pointed Newton polytopes,” Journal of Complexity
1996
Earlier work this paper cites.
F. M. Coetzee and V. L. Stonick, “On a natural homotopy between linear and nonlinear single-layer networks,” IEEE Transactions on Neural Networks
1996
Earlier work this paper cites.
F. M. Coetzee and V. L. Stonick, “488 solutions to the xor problem,” Advances in Neural Information Processing Systems
1997
Earlier work this paper cites.
T. Nitta, “An extension of the back-propagation algorithm to complex numbers,” Neural Networks
1997
Earlier work this paper cites.
S. Hochreiter and J. Schmidhuber, “Flat minima,” Neural Computation
1997
Earlier work this paper cites.
Secaucus, NJ, USA: Springer-Verlag New York, Inc., 1998
D. A. Cox, J. Little, and D. O’Shea, Using Algebraic Geometry · 1998
Earlier work this paper cites.
S. Perantonis, N. Ampazis, S. Varoufakis, and G. Antoniou, “Constrained learning in neural networks: Application to stable factorization of 2-d polynomials,” Neural Processing Letters
1998
Earlier work this paper cites.
L. Blum, “F. cuck er, m. shub, and s. smale. complexity and real computation,” 1998
1998
Earlier work this paper cites.
J. P. K. Doye and D. J. Wales, “Saddle points and dynamics of lennard-jones clusters, solids, and supercooled liquids,” J. Chem. Phys
2002
Earlier work this paper cites.
J. P. K. Doye and D. J. Wales, “Saddle points and dynamics of Lennard-Jones clusters, solids, and supercooled liquids,” Journal of Chem. Phys
2002
Earlier work this paper cites.
E. Kostlan, “On the expected number of real roots of a system of random polynomial equations,” in Foundations of computational mathematics
2002
Earlier work this paper cites.
Cambridge: Cambridge University Press, 2003
D. J. Wales, Energy Landscapes · 2003
Earlier work this paper cites.
T. Kim and T. Adalı, “Approximation by fully complex multilayer perceptrons,” Neural computation
2003
Earlier work this paper cites.
World Scientific, 2003
H. Akira, Complex-valued neural networks: theories and applications · 2003
Earlier work this paper cites.
T. Nitta, “Solving the xor problem and the detection of symmetry using a single complex-valued neuron,” Neural Networks
2003
Earlier work this paper cites.
Springer, 2003
S. Basu, R. Pollack, and M. F. Roy, Algorithms in Real Algebraic Geometry · 2003
Earlier work this paper cites.
T. Y. Li, “Solving polynomial systems by the homotopy continuation method,” Handbook of numerical analysis
2003
Earlier work this paper cites.
D. J. Wales, “Some further applications of discrete path sampling to cluster isomerization,” Mol. Phys
2004
Earlier work this paper cites.
D.-S. Huang, H. H. Ip, and Z. Chi, “A neural root finder of polynomials based on root moments,” Neural Computation
2004
Earlier work this paper cites.
J.-M. Azaïs and M. Wschebor, “On the roots of a random system of equations. the theorem of shub and smale and some extensions,” Foundations of Computational Mathematics
2005
Cited alongside, same era.
H. Ninomiya, C. Tomita, and H. Asai, “An efficient learning algorithm for finding multiple solutions based on fixed-point homotopy method,” in Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005
2005
Cited alongside, same era.
World Scientific Publishing, Hackensack, NJ, 2005
A. Sommese and C. Wampler, The Numerical Solution of Systems of Polynomials Arising in Engineering and Science · 2005
Cited alongside, same era.
World Scientific Publishing Company, 2005
A. J. Sommese and C. W. Wampler, The numerical solution of systems of polynomials arising in Engineering and Science · 2005
Cited alongside, same era.
Y. Nesterov and B. T. Polyak, “Cubic regularization of newton method and its global performance,” Mathematical Programming
2006
2016
Later among the works it cites.
M. Hardt and T. Ma, “Identity matters in deep learning,” arXiv preprint arXiv:1611.04231
2016
Later among the works it cites.
N. Guberman, “On complex valued convolutional neural networks,” arXiv preprint arXiv:1602.09046
2016
Later among the works it cites.
2016
Later among the works it cites.
S. Wisdom, T. Powers, J. Hershey, J. Le Roux, and L. Atlas, “Full-capacity unitary recurrent neural networks,” in Advances in Neural Information Processing Systems
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
B. Mourrain, N. G. Pavlidis, D. K. Tasoulis, and M. N. Vrahatis, “Determining the number of real roots of polynomials through neural networks,” Computers & Mathematics with Applications
2006
Cited alongside, same era.
S.-I. Amari, H. Park, and T. Ozeki, “Singularities affect dynamics of learning in neuromanifolds,” Neural computation
2006
Cited alongside, same era.
Secaucus, NJ, USA: Springer-Verlag New York, Inc., 2007
D. A. Cox, J. Little, and D. O’Shea, Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, 3/e (Undergraduate Texts in Mathematics) · 2007
Cited alongside, same era.
S. Watanabe, “Almost all learning machines are singular,” in Foundations of Computational Intelligence, 2007. FOCI 2007. IEEE Symposium on
2007
Cited alongside, same era.
J.-P. Dedieu and G. Malajovich, “On the number of minima of a random polynomial,” Journal of Complexity
2008
Cited alongside, same era.
Cambridge University Press, 2009
S. Watanabe, Algebraic geometry and statistical learning theory · 2009
Cited alongside, same era.
D. Armentano, M. Wschebor, et al
2009
Cited alongside, same era.
2016
Later among the works it cites.
M. Arjovsky, A. Shah, and Y. Bengio, “Unitary evolution recurrent neural networks,” in International Conference on Machine Learning
2016
Later among the works it cites.
2016
Later among the works it cites.
2016
Later among the works it cites.
C. Baldassi, A. Ingrosso, C. Lucibello, L. Saglietti, and R. Zecchina, “Local entropy as a measure for sampling solutions in constraint satisfaction problems,” Journal of Statistical Mechanics: Theory and Experiment
2016
Later among the works it cites.
C. Baldassi, F. Gerace, C. Lucibello, L. Saglietti, and R. Zecchina, “Learning may need only a few bits of synaptic precision,” Physical Review E
2016
Later among the works it cites.
C. Baldassi, C. Borgs, J. T. Chayes, A. Ingrosso, C. Lucibello, L. Saglietti, and R. Zecchina, “Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes,” Proceedings of the National Academy of Sciences
2016
Later among the works it cites.
2016
Later among the works it cites.
2017
Later among the works it cites.
J. D. Lee, I. Panageas, G. Piliouras, M. Simchowitz, M. I. Jordan, and B. Recht, “First-order Methods Almost Always Avoid Saddle Points,” ArXiv e-prints
2017
Later among the works it cites.
2017
Later among the works it cites.
G. Swirszcz, W. M. Czarnecki, and R. Pascanu, “Local minima in training of neural networks,” stat
2017
Later among the works it cites.
A. Ballard, S. Martiniani, D. Mehta, J. Stevenson, and D. J. Wales, “Energy landscapes for machine learning,” Phys. Chem. Chem. Phys
2017
Later among the works it cites.
2017
Later among the works it cites.
2017
Later among the works it cites.
2017
Later among the works it cites.
2017
Later among the works it cites.
2017
Later among the works it cites.
2017
Later among the works it cites.
H. Lu and K. Kawaguchi, “Depth creates no bad local minima,” arXiv preprint arXiv:1702.08580
2017
Later among the works it cites.
2017
Later among the works it cites.
2017
Later among the works it cites.
A. Taghvaei, J. W. Kim, and P. Mehta, “How regularization affects the critical points in linear networks,” in Advances in Neural Information Processing Systems
2017
Later among the works it cites.
C. Knoll and F. Pernkopf, “On loopy belief propagation–local stability analysis for non-vanishing fields,” in Uncertainty in Artificial Intelligence
2017
Later among the works it cites.
2017
Later among the works it cites.
2017
Later among the works it cites.
2017
Later among the works it cites.
2017
Later among the works it cites.
D. Bates, D. A. Brake, J. Hauenstein, A. J. Sommese, and C. Wampler, “Homotopies for connected components of algebraic sets with application to computing critical sets,” in Mathematical Aspects of Computer and Information Sciences
2017
Later among the works it cites.
2017
Later among the works it cites.
A. Anandkumar, Y. Deng, R. Ge, and H. Mobahi, “Homotopy analysis for tensor pca,” in Conference on Learning Theory
2017
Later among the works it cites.
2017
Later among the works it cites.
D. Mehta, X. Zhao, E. A. Bernal, and D. J. Wales, “The loss surface of xor artificial neural networks,” Preprint
2018
Closest in time.
2018
Closest in time.
Z. Zhu, D. Soudry, Y. C. Eldar, and M. B. Wakin, “The Global Optimization Geometry of Shallow Linear Neural Networks,” ArXiv e-prints
2018
Closest in time.
Y. Bansal, M. Advani, D. D. Cox, and A. M. Saxe, “Minnorm training: an algorithm for training over-parameterized deep neural networks,” ArXiv e-prints
2018
Closest in time.
2018
Closest in time.
C. Knoll, D. Mehta, T. Chen, and F. Pernkopf, “Fixed points of belief propagation—an analysis via polynomial homotopy continuation,” IEEE Transactions on Pattern Analysis and Machine Intelligence
2018
Closest in time.
S. Jastrzebski, Z. Kenton, D. Arpit, N. Ballas, A. Fischer, Y. Bengio, and A. Storkey, “Finding flatter minima with sgd,” 2018
2018
Closest in time.
Y. Zhang, A. M. Saxe, M. S. Advani, and A. A. Lee, “Energy–entropy competition and the effectiveness of stochastic gradient descent in machine learning,” Molecular Physics
2018
Closest in time.