Stochastic Recursive Inclusions in two timescales with non-additive iterate dependent Markov noise

Understand

In this paper we study the asymptotic behavior of a stochastic approximation scheme on two timescales with set-valued drift functions and in the presence of non-additive iterate-dependent Markov noise.

• It is shown that the recursion on each timescale tracks the flow of a differential inclusion obtained by averaging the set-valued drift function in the recursion with respect to a set of measures which take into account both the averaging with respect to the stationary distributions of the Markov noise terms and the interdependence between the two recursions on different timescales.
• The framework studied in this paper builds on the works of \it{A.
• Ramaswamy et al.