Fetching the paper…
Reading the bibliography…
For solving large-scale non-convex problems, we propose inexact variants of trust region and adaptive cubic regularization methods, which, to increase efficiency, incorporate various approximations.
“The conjugate gradient method and trust regions in large scale optimization”
Trond Steihaug · 1983
Earlier work this paper cites.
“Some NP-complete problems in quadratic and nonlinear programming”
Katta Murty and Santosh Kabadi · 1987
Earlier work this paper cites.
“Some bounds on the complexity of gradients, Jacobians, and Hessians”
Andreas Griewank · 1993
Earlier work this paper cites.
“Fast exact multiplication by the Hessian”
Barak Pearlmutter · 1994
Earlier work this paper cites.
“Nonlinear programming”
Dimitri. Bertsekas · 1999
Earlier work this paper cites.
“Trust region methods”
Andrew Conn, Nicholas Gould and Philippe Toint · 2000
Earlier work this paper cites.
“Local minima and plateaus in hierarchical structures of multilayer perceptrons”
Kenji Fukumizu and Shun-ichi Amari · 2000
Earlier work this paper cites.
“Convex optimization”
Stephen Boyd and Lieven Vandenberghe · 2004
Earlier work this paper cites.
“Introductory lectures on convex optimization”
Yurii Nesterov · 2004
Earlier work this paper cites.
“Cubic regularization of Newton’s method for convex problems with constraints”, 2006
Yurii Nesterov · 2006
Earlier work this paper cites.
“Cubic regularization of Newton method and its global performance”
Yurii Nesterov and Boris Polyak · 2006
Earlier work this paper cites.
“Numerical optimization”
Jorge Nocedal and Stephen Wright · 2006
Earlier work this paper cites.
“Global convergence of general derivative-free trust-region algorithms to first-and second-order critical points”
Andrew Conn, Katya Scheinberg and Lu“’s Vicente · 2009
Earlier work this paper cites.
“On the complexity of steepest descent, Newton’s and regularized Newton’s methods for nonconvex unconstrained optimization problems”
Coralia Cartis, Nicholas Gould and Philippe Toint · 2010
Earlier work this paper cites.
“Deep learning via Hessian-free optimization”
James Martens · 2010
Earlier work this paper cites.
“Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results”
Coralia Cartis, Nicholas Gould and Philippe Toint · 2011
Earlier work this paper cites.
“Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function-and derivative-evaluation complexity”
Coralia Cartis, Nicholas Gould and Philippe Toint · 2011
Earlier work this paper cites.
“Optimal Newton-type methods for nonconvex smooth optimization problems”, 2011
Coralia Cartis, Nicholas Gould and Philippe Toint · 2011
Earlier work this paper cites.
“LIBSVM: A library for support vector machines” Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm
Chih-Chung Chang and Chih-Jen Lin · 2011
Cited alongside, same era.
“Complexity bounds for second-order optimality in unconstrained optimization”
Coralia Cartis, Nicholas Gould and Philippe Toint · 2012
Cited alongside, same era.
“Efficient backprop”
Yann LeCun, L“’eon Bottou, Genevieve Orr and Klaus-Robert M“”uller · 2012
Cited alongside, same era.
“Krylov Subspace Descent for Deep Learning”
Oriol Vinyals and Daniel Povey · 2012
Cited alongside, same era.
“Most tensor problems are NP-hard”
Christopher Hillar and Lek-Heng Lim · 2013
Cited alongside, same era.
“Training neural networks with stochastic Hessian-free optimization”
Ryan Kiros · 2013
“Deep residual learning for image recognition”
Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun · 2016
Later among the works it cites.
“Large scale distributed hessian-free optimization for deep neural network”
Xi He, Dheevatsa Mudigere, Mikhail Smelyanskiy and Martin Tak“’ac · 2016
Later among the works it cites.
“Stochastic derivative-free optimization using a trust region framework”
Jeffrey Larson and Stephen Billups · 2016
Later among the works it cites.
“The Power of Normalization: Faster Evasion of Saddle Points”
Kfir Levy · 2016
Later among the works it cites.
“Sub-sampled Newton methods I: globally convergent algorithms”
Farbod Roosta-Khorasani and Michael. Mahoney · 2016
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
“Exact solutions to the nonlinear dynamics of learning in deep linear neural networks”
Andrew Saxe, James McClelland and Surya Ganguli · 2013
Cited alongside, same era.
“Investigations on Hessian-free optimization for cross-entropy training of deep neural networks”
Simon Wiesler, Jinyu Li and Jian Xue · 2013
Cited alongside, same era.
“Convergence of trust-region methods based on probabilistic models”
Afonso Bandeira, Katya Scheinberg and Lu“’s Vicente · 2014
Cited alongside, same era.
“Identifying and attacking the saddle point problem in high-dimensional non-convex optimization”
Yann Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli and Yoshua Bengio · 2014
Cited alongside, same era.
“Stochastic optimization using a trust-region method and random models”
Ruobing Chen, Matt Menickelly and Katya Scheinberg · 2015
Cited alongside, same era.
“The loss surfaces of multilayer networks”
Anna Choromanska, Mikael Henaff, Michael Mathieu, G“’erard Arous and Yann LeCun · 2015
Cited alongside, same era.
“Sub-sampled Newton methods II: local convergence rates”
Farbod Roosta-Khorasani and Michael Mahoney · 2016
Later among the works it cites.
Sara Shashaani, Fatemeh Hashemi and Raghu Pasupathy · 2016
Later among the works it cites.
“Local minima in training of deep networks”
Grzegorz Swirszcz, Wojciech Czarnecki and Razvan Pascanu · 2016
Later among the works it cites.
“Sub-sampled newton methods with non-uniform sampling”
Peng Xu, Jiyan Yang, Farbod Roosta-Khorasani, Christopher R“’e and Michael Mahoney · 2016
Later among the works it cites.
“An Investigation of Newton-Sketch and Subsampled Newton Methods”
Albert Berahas, Raghu Bollapragada and Jorge Nocedal · 2017
Later among the works it cites.
“Complexity and global rates of trust-region methods based on probabilistic models”, 2017
S Gratton, CW Royer, LN Vicente and Z Zhang · 2017
Later among the works it cites.
“How to Escape Saddle Points Efficiently”
Chi Jin, Rong Ge, Praneeth Netrapalli, Sham Kakade and Michael Jordan · 2017
Later among the works it cites.
“Sub-sampled Cubic Regularization for Non-convex Optimization”
Jonas Kohler and Aurelien Lucchi · 2017
Later among the works it cites.
“Fast Black-box Variational Inference through Stochastic Trust-Region Optimization”
Jeffrey Regier, Michael Jordan and Jon McAuliffe · 2017
Later among the works it cites.
“Stochastic Cubic Regularization for Fast Nonconvex Optimization”
Nilesh Tripuraneni, Mitchell Stern, Chi Jin, Jeffrey Regier and Michael Jordan · 2017
Later among the works it cites.
“Newton-Type Methods for Non-Convex Optimization Under Inexact Hessian Information”
Peng Xu, Farbod Roosta-Khorasani and Michael. Mahoney · 2017
Later among the works it cites.
“Second-Order Optimization for Non-Convex Machine Learning: An Empirical Study”
Peng Xu, Farbod Roosta-Khorasani and Michael. Mahoney · 2017
Later among the works it cites.