Procedural Generalization by Planning with Self-Supervised World Models

Understand

One of the key promises of model-based reinforcement learning is the ability to generalize using an internal model of the world to make predictions in novel environments and tasks.

• However, the generalization ability of model-based agents is not well understood because existing work has focused on model-free agents when benchmarking generalization.
• Here, we explicitly measure the generalization ability of model-based agents in comparison to their model-free counterparts.
• We focus our analysis on MuZero (Schrittwieser et al., 2020), a powerful model-based agent, and evaluate its performance on both procedural and task generalization.