Automatic Optimisation of Normalised Neural Networks

Fast Exact Multiplication by the Hessian

Barak A. Pearlmutter · 1994

Earlier work this paper cites.

Optimization Algorithms on Matrix Manifolds

Pierre-Antoine Absil, Robert Mahony, and Rodolphe Sepulchre · 2008

Earlier work this paper cites.

Deep Learning via Hessian-Free Optimization

James Martens · 2010

Earlier work this paper cites.

MM Optimization Algorithms

Kenneth Lange · 2016

Earlier work this paper cites.

Spectrally-Normalized Margin Bounds for Neural Networks

Peter L. Bartlett, Dylan J. Foster, and Matus J. Telgarsky · 2017

Earlier work this paper cites.

Spectral Normalization for Generative Adversarial Networks

Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida · 2018

Earlier work this paper cites.

Sorting Out Lipschitz Function Approximation

Cem Anil, James Lucas, and Roger Grosse · 2019

Earlier work this paper cites.

Neural Lander: Stable Drone Landing Control Using Learned Dynamics

Guanya Shi, Xichen Shi, Michael O’Connell, Rose Yu, Kamyar Azizzadenesheli, Animashree Anandkumar, Yisong Yue, and Soon-Jo Chung · 2019

Earlier work this paper cites.

On the Distance Between Two Neural Networks and the Stability of Learning

Jeremy Bernstein, Arash Vahdat, Yisong Yue, and Ming-Yu Liu · 2020

Earlier work this paper cites.

Robust Regression for Safe Exploration in Control

Anqi Liu, Guanya Shi, Soon-Jo Chung, Anima Anandkumar, and Yisong Yue · 2020

Cited alongside, same era.

Why Spectral Normalization Stabilizes GANs: Analysis and Improvements

Zinan Lin, Vyas Sekar, and Giulia Fanti · 2021

Cited alongside, same era.

Learning by Turning: Neural Architecture Aware Optimisation

Yang Liu, Jeremy Bernstein, Markus Meister, and Yisong Yue · 2021

Cited alongside, same era.

Neural-Swarm2: Planning and Control of Heterogeneous Multirotor Swarms Using Learned Interactions

Guanya Shi, Wolfgang Hönig, Xichen Shi, Yisong Yue, and Soon-Jo Chung · 2021

Cited alongside, same era.

Spherical Motion Dynamics: Learning Dynamics of Normalized Neural Network using SGD and Weight Decay

Ruosi Wan, Zhanxing Zhu, Xiangyu Zhang, and Jian Sun · 2021

Cited alongside, same era.

Bridging the Model-Reality Gap With Lipschitz Network Adaptation

Automatic Gradient Descent: Deep Learning without Hyperparameters, 2023

Jeremy Bernstein, Chris Mingard, Kevin Huang, Navid Azizan, and Yisong Yue · 2023

Closest in time.

Optimisation & Generalisation in Networks of Neurons

Jeremy David Bernstein · 2023

Closest in time.

An Introduction to Optimization on Smooth Manifolds

Nicolas Boumal · 2023

Closest in time.

MM Optimization: Proximal Distance Algorithms, Path Following, and Trust Regions

Alfonso Landeros, Jason Xu, and Kenneth Lange · 2023

Closest in time.

Reliable Learning and Control in Dynamic Environments: Towards Unified Theory and Learned Robotic Agility

Guanya Shi · 2023

Closest in time.

Universal Majorization-Minimization Algorithms, 2023

Matthew Streeter · 2023

Closest in time.