Modelling transition dynamics in MDPs with RKHS embeddings

Understand

We propose a new, nonparametric approach to learning and representing transition dynamics in Markov decision processes (MDPs), which can be combined easily with dynamic programming methods for policy optimisation and value estimation.

• This approach makes use of a recently developed representation of conditional distributions as \emph{embeddings} in a reproducing kernel Hilbert space (RKHS).
• Such representations bypass the need for estimating transition probabilities or densities, and apply to any domain on which kernels can be defined.
• This avoids the need to calculate intractable integrals, since expectations are represented as RKHS inner products whose computation has linear complexity in the number of points used to represent the embedding.