Deep Reinforcement Learning amidst Lifelong Non-Stationarity

Understand

As humans, our goals and our environment are persistently changing throughout our lifetime based on our experiences, actions, and internal and external drives.

• In contrast, typical reinforcement learning problem set-ups consider decision processes that are stationary across episodes.
• Can we develop reinforcement learning algorithms that can cope with the persistent change in the former, more realistic problem settings? While on-policy algorithms such as policy gradients in principle can be extended to non-stationary settings, the same cannot be said for more efficient off-policy algorithms that replay past experiences when learning.
• In this work, we formalize this problem setting, and draw upon ideas from the online learning and probabilistic inference literature to derive an off-policy RL algorithm that can reason about and tackle such lifelong non-stationarity.