A Subsampling Line-Search Method with Second-Order Results

Understand

In many contemporary optimization problems such as those arising in machine learning, it can be computationally challenging or even infeasible to evaluate an entire function or its derivatives.

• This motivates the use of stochastic algorithms that sample problem data, which can jeopardize the guarantees obtained through classical globalization techniques in optimization such as a trust region or a line search.
• Using subsampled function values is particularly challenging for the latter strategy, which relies upon multiple evaluations.
• On top of that all, there has been an increasing interest for nonconvex formulations of data-related problems, such as training deep learning models.