Testing Stationarity and Change Point Detection in Reinforcement Learning

Understand

We consider offline reinforcement learning (RL) methods in possibly nonstationary environments.

• Many existing RL algorithms in the literature rely on the stationarity assumption that requires the system transition and the reward function to be constant over time.
• However, the stationarity assumption is restrictive in practice and is likely to be violated in a number of applications, including traffic signal control, robotics and mobile health.
• In this paper, we develop a consistent procedure to test the nonstationarity of the optimal Q-function based on pre-collected historical data, without additional online data collection.