Constrained Reinforcement Learning via Dissipative Saddle Flow Dynamics

Understand

In constrained reinforcement learning (C-RL), an agent seeks to learn from the environment a policy that maximizes the expected cumulative reward while satisfying minimum requirements in secondary cumulative reward constraints.

• Several algorithms rooted in sampled-based primal-dual methods have been recently proposed to solve this problem in policy space.
• However, such methods are based on stochastic gradient descent ascent algorithms whose trajectories are connected to the optimal policy only after a mixing output stage that depends on the algorithm's history.
• As a result, there is a mismatch between the behavioral policy and the optimal one.