An Empirical Dynamic Programming Algorithm for Continuous MDPs

Understand

We propose universal randomized function approximation-based empirical value iteration (EVI) algorithms for Markov decision processes.

• The `empirical' nature comes from each iteration being done empirically from samples available from simulations of the next state.
• This makes the Bellman operator a random operator.
• A parametric and a non-parametric method for function approximation using a parametric function space and the Reproducing Kernel Hilbert Space (RKHS) respectively are then combined with EVI.