Fetching the paper…

Reading the bibliography…

2021

The structure of conservative gradient fields

Lewis, Adrian, Tian, Tonghua

Original
PDF
Post on X

Understand

The classical Clarke subdifferential alone is inadequate for understanding automatic differentiation in nonsmooth contexts.

Instead, we can sometimes rely on enlarged generalized gradients called "conservative fields", defined through the natural path-wise chain rule: one application is the convergence analysis of gradient-based deep learning algorithms.
In the semi-algebraic case, we show that all conservative fields are in fact just Clarke subdifferentials plus normals of manifolds in underlying Whitney stratifications.

The structure of conservative gradient fields · Around

Built on

Optimization and Nonsmooth Analysis
F.H. Clarke · 1983
Earlier work this paper cites.
Entraînement unilatéral, lignes de descente, fonctions lipschitziennes non pathologiques
M. Valadier · 1989
Earlier work this paper cites.
Geometric categories and o-minimal structures
L. van den Dries and C. Miller · 1996
Earlier work this paper cites.
Clarke subgradients of stratifiable functions
J. Bolte, A. Daniilidis, A.S. Lewis, and M. Shiota · 2007
Earlier work this paper cites.

Similar

TensorFlow: Large-scale machine learning on heterogeneous systems, 2015
M. Abadi et al · 2015
Cited alongside, same era.
On Lipschitz optimization based on gray-box piecewise linearization
A. Griewank, A. Walther, S. Fiege, and T. Bosse · 2016
Cited alongside, same era.
Automatic differentiation in PyTorch
A Paszke et al · 2017
Cited alongside, same era.
Conservative set valued fields, automatic differentiation, stochastic gradient methods and deep learning
J. Bolte and E. Pauwels · 2020
Cited alongside, same era.

Then

A mathematical model for automatic differentiation in machine learning
J. Bolte and E. Pauwels · 2020
Later among the works it cites.
Stochastic subgradient method converges on tame functions
D. Davis, D. Drusvyatskiy, S. Kakade, and J.D. Lee · 2020
Later among the works it cites.
On correctness of automatic differentiation for non-differentiable functions
Wonyeol Lee, Hangyeol Yu, X. Rival, and Hongseok Yang · 2020
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…