Towards Safe Reinforcement Learning Using NMPC and Policy Gradients: Part II - Deterministic Case

Understand

In this paper, we present a methodology to deploy the deterministic policy gradient method, using actor-critic techniques, when the optimal policy is approximated using a parametric optimization problem, where safety is enforced via hard constraints.

• For continuous input space, imposing safety restrictions on the exploration needed to deploying the deterministic policy gradient method poses some technical difficulties, which we address here.
• We will investigate in particular policy approximations based on robust Nonlinear Model Predictive Control (NMPC), where safety can be treated explicitly.
• For the sake of brevity, we will detail the construction of the safe scheme in the robust linear MPC context only.