Control as Hybrid Inference

Understand

The field of reinforcement learning can be split into model-based and model-free methods.

• Here, we unify these approaches by casting model-free policy optimisation as amortised variational inference, and model-based planning as iterative variational inference, within a `control as hybrid inference' (CHI) framework.
• We present an implementation of CHI which naturally mediates the balance between iterative and amortised inference.
• Using a didactic experiment, we demonstrate that the proposed algorithm operates in a model-based manner at the onset of learning, before converging to a model-free algorithm once sufficient data have been collected.