Temporal Difference Flows

Understand

Predictive models of the future are fundamental for an agent's ability to reason and plan.

• A common strategy learns a world model and unrolls it step-by-step at inference, where small errors can rapidly compound.
• Geometric Horizon Models (GHMs) offer a compelling alternative by directly making predictions of future states, avoiding cumulative inference errors.
• While GHMs can be conveniently learned by a generative analog to temporal difference (TD) learning, existing methods are negatively affected by bootstrapping predictions at train time and struggle to generate high-quality predictions at long horizons.