Constant Step Stochastic Approximations Involving Differential Inclusions: Stability, Long-Run Convergence and Applications

Understand

We consider a Markov chain $(x_n)$ whose kernel is indexed by a scaling parameter $\gamma>0$, refered to as the step size.

• The aim is to analyze the behavior of the Markov chain in the doubly asymptotic regime where $n\to\infty$ then $\gamma\to 0$.
• First, under mild assumptions on the so-called drift of the Markov chain, we show that the interpolated process converges narrowly to the solutions of a Differential Inclusion (DI) involving an upper semicontinuous set-valued map with closed and convex values.
• Second, we provide verifiable conditions which ensure the stability of the iterates.