Fetching the paper…
Reading the bibliography…
We propose a trust-region method for finite-sum minimization with an adaptive sample size adjustment technique, which is practical in the sense that it leads to a globally convergent method that shows strong performance empirically without the need for experimentation by the user.
A stochastic approximation method
H. Robbins and S. Monro · 1951
Earlier work this paper cites.
The modification of newton’s method for unconstrained optimization by bounding cubic terms
A. Griewank · 1981
Earlier work this paper cites.
Towards an efficient sparsity exploiting newton method for minimization
P. L. Toint · 1981
Earlier work this paper cites.
The conjugate gradient method and trust regions in large scale optimization
T. Steihaug · 1983
Earlier work this paper cites.
Fast exact multiplication by the hessian
B. A. Pearlmutter · 1994
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner · 1998
Earlier work this paper cites.
Fast training of support vector machines using sequential minimal optimization
J. C. Platt · 1998
Earlier work this paper cites.
Solving the trust-region subproblem using the lanczos method
N. I. M. Gould, S. Lucidi, M. Roma, and P. L. Toint · 1999
Earlier work this paper cites.
Trust-region methods , volume 1 of MPS-SIAM series on optimization
A. R. Conn, N. I. M. Gould, and P. L. Toint · 2000
Earlier work this paper cites.
A parallel mixture of svms for very large scale problems
R. Collobert, S. Bengio, and Y. Bengio · 2002
Earlier work this paper cites.
Cubic regularization of newton method and its global performance
Y. Nesterov and B. T. Polyak · 2006
Earlier work this paper cites.
Numerical Optimization
J. Nocedal and S. J. Wright · 2006
Earlier work this paper cites.
A stochastic quasi-newton method for online convex optimization
N. N. Schraudolph, Yu Jin, and S. Günter · 2007
Earlier work this paper cites.
Sgd-qn: Careful quasi-newton stochastic gradient descent
A. Bordes, L. Bottou, and P. Gallinari · 2009
Earlier work this paper cites.
Deep learning via hessian-free optimization
J. Martens · 2010
Earlier work this paper cites.
On the use of stochastic hessian information in unconstrained optimization
R. H. Byrd, G. M. Chin, W. Neveitt, and J. Nocedal · 2011
Earlier work this paper cites.
Learning recurrent neural networks with hessian-free optimization
J. Martens and I. Sutskever · 2011
Earlier work this paper cites.
Sample size selection in optimization methods for machine learning
R. H. Byrd, G. M. Chin, J. Nocedal, and Y. Wu · 2012
Cited alongside, same era.
Hybrid deterministic-stochastic methods for data fitting
M. P. Friedlander and M. Schmidt · 2012
Cited alongside, same era.
Accelerating stochastic gradient descent using predictive variance reduction
R. Johnson and T. Zhang · 2013
Cited alongside, same era.
Searching for exotic particles in high-energy physics with deep learning
P. Baldi, P. Sadowski, and D. Whiteson · 2014
Cited alongside, same era.
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
Y. N. Dauphin, R. Pascanu, C. Gulcehre, K. Cho, S. Ganguli, and Y. Bengio · 2014
Cited alongside, same era.
Saga: A fast incremental gradient method with support for non-strongly convex composite objectives
Sub-sampled cubic regularization for non-convex optimization
J. M. Kohler and A. Lucchi · 2017
Later among the works it cites.
Minimizing finite sums with the stochastic average gradient
M. Schmidt, N. Le Roux, and F. Bach · 2017
Later among the works it cites.
Second-order optimization for non-convex machine learning: An empirical study
P. Xu, F. Roosta-Khorasani, and M. W. Mahoney · 2017
Later among the works it cites.
Stochastic adaptive quasi-newton methods for minimizing expected values
C. Zhou, W. Gao, and D. Goldfarb · 2017
Later among the works it cites.
A levenberg–marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients
S. Bellavia, S. Gratton, and E. Riccietti · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
A. Defazio, F. Bach, and S. Lacoste-Julien · 2014
Cited alongside, same era.
Fast large-scale optimization by unifying stochastic gradient and quasi-newton methods
J. Sohl-Dickstein, B. Poole, and S. Ganguli · 2014
Cited alongside, same era.
Global convergence of online limited memory bfgs
A. Mokhtari, Alej, and r. Ribeiro · 2015
Cited alongside, same era.
Recent advances in trust region algorithms
Y.-X. Yuan · 2015
Cited alongside, same era.
A multi-batch l-bfgs method for machine learning
A. S. Berahas, J. Nocedal, and M. Takáč · 2016
Cited alongside, same era.
A stochastic quasi-newton method for large-scale optimization
R. H. Byrd, S. L. Hansen, J. Nocedal, and Y. Singer · 2016
Cited alongside, same era.
A self-correcting variable-metric algorithm for stochastic optimization
F. Curtis · 2016
Cited alongside, same era.
Optimization methods for large-scale machine learning
L. Bottou, F. E. Curtis, and J. Nocedal · 2018
Later among the works it cites.
Global convergence rate analysis of unconstrained optimization methods based on probabilistic models
C. Cartis and K. Scheinberg · 2018
Later among the works it cites.
Stochastic optimization using a trust-region method and random models
R. Chen, M. Menickelly, and K. Scheinberg · 2018
Later among the works it cites.
Tracking the gradients using the hessian: A new look at variance reducing stochastic methods
R. Gower, N. Le Roux, and F. Bach · 2018
Later among the works it cites.
Inexact non-convex newton-type methods
Z. Yao, P. Xu, F. Roosta-Khorasani, and M. W. Mahoney · 2018
Later among the works it cites.
A robust multi-batch l-bfgs method for machine learning
A. S. Berahas and M. Takáč · 2019
Closest in time.
Convergence rate analysis of a stochastic trust-region method via supermartingales
J. Blanchet, C. Cartis, M. Menickelly, and K. Scheinberg · 2019
Closest in time.
Exploiting negative curvature in deterministic and stochastic optimization
F. E. Curtis and D. P. Robinson · 2019
Closest in time.
Trust-region algorithms for training responses: machine learning methods using indefinite hessian approximations
J. B. Erway, J. Griffin, R. F. Marcia, and R. Omheni · 2019
Closest in time.
Sub-sampled newton methods
F. Roosta-Khorasani and M. W. Mahoney · 2019
Closest in time.
Newton-type methods for non-convex optimization under inexact hessian information
P. Xu, F. Roosta-Khorasani, and M. W. Mahoney · 2019
Closest in time.