Online Bootstrap Inference with Nonconvex Stochastic Gradient Descent Estimator

Understand

In this paper, we investigate the theoretical properties of stochastic gradient descent (SGD) for statistical inference in the context of nonconvex optimization problems, which have been relatively unexplored compared to convex settings.

• Our study is the first to establish provable inferential procedures using the SGD estimator for general nonconvex objective functions, which may contain multiple local minima.
• We propose two novel online inferential procedures that combine SGD and the multiplier bootstrap technique.
• The first procedure employs a consistent covariance matrix estimator, and we establish its error convergence rate.