Heuristic-Guided Reinforcement Learning

Understand

We provide a framework for accelerating reinforcement learning (RL) algorithms by heuristics constructed from domain knowledge or offline data.

• Tabula rasa RL algorithms require environment interactions or computation that scales with the horizon of the sequential decision-making task.
• Using our framework, we show how heuristic-guided RL induces a much shorter-horizon subproblem that provably solves the original task.
• Our framework can be viewed as a horizon-based regularization for controlling bias and variance in RL under a finite interaction budget.