LTL-Constrained Policy Optimization with Cycle Experience Replay

Understand

Linear Temporal Logic (LTL) offers a precise means for constraining the behavior of reinforcement learning agents.

• However, in many settings where both satisfaction and optimality conditions are present, LTL is insufficient to capture both.
• Instead, LTL-constrained policy optimization, where the goal is to optimize a scalar reward under LTL constraints, is needed.
• This constrained optimization problem proves difficult in deep Reinforcement Learning (DRL) settings, where learned policies often ignore the LTL constraint due to the sparse nature of LTL satisfaction.