Source Traces for Temporal Difference Learning

Understand

This paper motivates and develops source traces for temporal difference (TD) learning in the tabular setting.

• Source traces are like eligibility traces, but model potential histories rather than immediate ones.
• This allows TD errors to be propagated to potential causal states and leads to faster generalization.
• Source traces can be thought of as the model-based, backward view of successor representations (SR), and share many of the same benefits.