Fetching the paper…
Reading the bibliography…
It is shown that for deep neural networks, a single wide layer of width $N+1$ ($N$ being the number of training samples) suffices to prove the connectivity of sublevel sets of the training loss function.
The set of all mxn rectangular real matrices of rank-r is connected by analytic regular arcs
Jean-Claude Evard and Farhad Jafarii · 1994
Earlier work this paper cites.
Exponentially many local minima for single neurons
Peter Auer, Mark Herbster, and Manfred K. K. Warmuth · 1996
Earlier work this paper cites.
The loss surfaces of multilayer networks
Anna Choromanska, Mikael Henaff, Michael Mathieu, Gerard Ben Arous, and Yann LeCun · 2015
Earlier work this paper cites.
Deep learning without poor local minima
Kenji Kawaguchi · 2016
Earlier work this paper cites.
On the quality of the initial basin in overspecified networks
Itay Safran and Ohad Shamir · 2016
Earlier work this paper cites.
Topology and geometry of half-rectified network optimization
Daniel C. Freeman and Joan Bruna · 2017
Cited alongside, same era.
The loss surface of deep and wide neural networks
Quynh Nguyen and Matthias Hein · 2017
Cited alongside, same era.
Optimization landscape and expressivity of deep cnns
Quynh Nguyen and Matthias Hein · 2018
Cited alongside, same era.
Explaining landscape connectivity of low-cost solutions for multilayer nets
Rohith Kuditipudi, Xiang Wang, Holden Lee, Yi Zhang, Zhiyuan Li, Wei Hu, Rong Ge, and Sanjeev Arora · 2019
Cited alongside, same era.
On connected sublevel sets in deep learning
Quynh Nguyen · 2019
Cited alongside, same era.
On the loss landscape of a class of deep neural networks with no bad local valleys
Quynh Nguyen, Mahesh C. Mukkamala, and Matthias Hein · 2019
Later among the works it cites.
Spurious valleys in two-layer neural network optimization landscapes
Luca Venturi, Afonso S. Bandeira, and Joan Bruna · 2019
Later among the works it cites.
Small nonlinearities in activation functions create bad local minima in neural networks
Chulhee Yun, Suvrit Sra, and Ali Jadbabaie · 2019
Later among the works it cites.
Landscape connectivity and dropout stability of sgd solutions for over-parameterized neural networks
Alexander Shevchenko and Marco Mondelli · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…