A CMDP-within-online framework for Meta-Safe Reinforcement Learning

Understand

Meta-reinforcement learning has widely been used as a learning-to-learn framework to solve unseen tasks with limited experience.

• However, the aspect of constraint violations has not been adequately addressed in the existing works, making their application restricted in real-world settings.
• In this paper, we study the problem of meta-safe reinforcement learning (Meta-SRL) through the CMDP-within-online framework to establish the first provable guarantees in this important setting.
• We obtain task-averaged regret bounds for the reward maximization (optimality gap) and constraint violations using gradient-based meta-learning and show that the task-averaged optimality gap and constraint satisfaction improve with task-similarity in a static environment or task-relatedness in a dynamic environment.