Online Statistical Inference for Nonlinear Stochastic Approximation with Markovian Data

Understand

Many stochastic approximation (SA) algorithms evolve along a single trajectory, making uncertainty quantification challenging under nonlinear dynamics and Markov dependence.

• We develop an online inference framework for nonlinear SA with decreasing step sizes.
• Under local stability and verifiable conditions, we establish a functional central limit theorem for the partial-sum path.
• The proof uses a Poisson-equation decomposition to handle Markov dependence and a uniform bound to control endpoint-dependent remainders induced by decreasing step sizes.