Online Off-policy Prediction

Understand

This paper investigates the problem of online prediction learning, where learning proceeds continuously as the agent interacts with an environment.

• The predictions made by the agent are contingent on a particular way of behaving, represented as a value function.
• However, the behavior used to select actions and generate the behavior data might be different from the one used to define the predictions, and thus the samples are generated off-policy.
• The ability to learn behavior-contingent predictions online and off-policy has long been advocated as a key capability of predictive-knowledge learning systems but remained an open algorithmic challenge for decades.