Parametric Return Density Estimation for Reinforcement Learning

Understand

Most conventional Reinforcement Learning (RL) algorithms aim to optimize decision-making rules in terms of the expected returns.

• However, especially for risk management purposes, other risk-sensitive criteria such as the value-at-risk or the expected shortfall are sometimes preferred in real applications.
• Here, we describe a parametric method for estimating density of the returns, which allows us to handle various criteria in a unified manner.
• We first extend the Bellman equation for the conditional expected return to cover a conditional probability density of the returns.