Geometry of energy landscapes and the optimizability of deep neural networks

Understand

Deep neural networks are workhorse models in machine learning with multiple layers of non-linear functions composed in series.

• Their loss function is highly non-convex, yet empirically even gradient descent minimisation is sufficient to arrive at accurate and predictive models.
• It is hitherto unknown why are deep neural networks easily optimizable.
• We analyze the energy landscape of a spin glass model of deep neural networks using random matrix theory and algebraic geometry.