Bridging Distributional and Risk-sensitive Reinforcement Learning with Provable Regret Bounds

Understand

We study the regret guarantee for risk-sensitive reinforcement learning (RSRL) via distributional reinforcement learning (DRL) methods.

• In particular, we consider finite episodic Markov decision processes whose objective is the entropic risk measure (EntRM) of return.
• By leveraging a key property of the EntRM, the independence property, we establish the risk-sensitive distributional dynamic programming framework.
• We then propose two novel DRL algorithms that implement optimism through two different schemes, including a model-free one and a model-based one.