Benchmarking Batch Deep Reinforcement Learning Algorithms

Understand

Widely-used deep reinforcement learning algorithms have been shown to fail in the batch setting--learning from a fixed data set without interaction with the environment.

• Following this result, there have been several papers showing reasonable performances under a variety of environments and batch settings.
• In this paper, we benchmark the performance of recent off-policy and batch reinforcement learning algorithms under unified settings on the Atari domain, with data generated by a single partially-trained behavioral policy.
• We find that under these conditions, many of these algorithms underperform DQN trained online with the same amount of data, as well as the partially-trained behavioral policy.