Fetching the paper…

Reading the bibliography…

2019

A Deterministic Gradient-Based Approach to Avoid Saddle Points

Kreusser, Lisa Maria, Osher, Stanley J., Wang, Bao

Original
PDF
Post on X

Understand

Loss functions with a large number of saddle points are one of the major obstacles for training modern machine learning models efficiently.

First-order methods such as gradient descent are usually the methods of choice for training machine learning models.
However, these methods converge to saddle points for certain choices of initial guesses.
In this paper, we propose a modification of the recently proposed Laplacian smoothing gradient descent [Osher et al., arXiv:1806.06317], called modified Laplacian smoothing gradient descent (mLSGD), and demonstrate its potential to avoid saddle points without sacrificing the convergence rate.

A Deterministic Gradient-Based Approach to Avoid Saddle Points · Around

Built on

V. Vapnik
1992
Earlier work this paper cites.
Y. Nesterov
1998
Earlier work this paper cites.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams
1998
Earlier work this paper cites.
Y. Nesterov and B. T. Polyak
2006
Earlier work this paper cites.
J. Nocedal and S. Wright
2006
Earlier work this paper cites.
Y. Bengio
2009
Earlier work this paper cites.
J. Martens
2010
Earlier work this paper cites.
F. E. Curtis, D. P. Robinson, and M. Samadi
2014
Earlier work this paper cites.
Y. N. Dauphin, R. Pascanu, C. Gulcehre, K. Cho, S. Ganguli, and Y. Bengio
2014
Earlier work this paper cites.
R. Ge, F. Huang, C. Jin, and Y. Yuan
2015
Earlier work this paper cites.
R. Ge, F. Huang, C. Jin, and Y. Yuan
2015
Earlier work this paper cites.

Similar

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al
2015
Cited alongside, same era.
K. He, X. Zhang, S. Ren, and J. Sun
2016
Cited alongside, same era.
J. D. Lee, M. Simchowitz, M. I. Jordan, and B. Recht
2016
Cited alongside, same era.
K. Y. Levy
Original
2016
Cited alongside, same era.
N. Agarwal, Z. Allen-Zhu, B. Bullins, E. Hazan, and T. Ma
2017
Cited alongside, same era.
S. Du, C. Jin, J. D. Lee, M. I. Jordan, B. Poczos, and A. Singh
2017
Cited alongside, same era.

Then

Y. Carmon and J. C. Duchi
2019
Closest in time.
F. E. Curtis and D. P. Robinson
2019
Closest in time.
J. D. Lee, I. Panageas, G. Piliouras, M. Simchowitz, M. I. Jordan, and B. Recht
2019
Closest in time.
S. Paternain, A. Mokhtari, and A. Ribeiro
2019
Closest in time.
M. Iqbal, M. A. Rehman, N. Iqbal, and Z. Iqbal
2020
Closest in time.
Z. Liang, B. Wang, Q. Gu, S. Osher, and Y. Yao
Original
2020
Closest in time.
J. Ul Rahman, A. Ali, M. Ur Rehman, and R. Kazmi
2020

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

C. Jin, R. Ge, P. Netrapalli, S. Kakade, and M. I. Jordan

2017

Cited alongside, same era.

M. Liu and T. Yang

2017

Cited alongside, same era.

C. Jin, P. Netrapalli, and M. I. Jordan

2018

Cited alongside, same era.

S. Osher, B. Wang, P. Yin, X. Luo, M. Pham, and A. Lin

2018

Cited alongside, same era.

J. Sun, Q. Qu, and J. Wright

2018

Cited alongside, same era.

Closest in time.

B. Wang, Q. Gu, M. Boedihardjo, L. Wang, F. Barekat, and S. J. Osher

2020

Closest in time.

B. Wang, T. M. Nguyen, A. L. Bertozzi, R. G. Baraniuk, and S. J. Osher

2020

Closest in time.

B. Wang, D. Zou, Q. Gu, and S. Osher

2020

Closest in time.