Visualizing MuZero Models

Understand

MuZero, a model-based reinforcement learning algorithm that uses a value equivalent dynamics model, achieved state-of-the-art performance in Chess, Shogi and the game of Go.

• In contrast to standard forward dynamics models that predict a full next state, value equivalent models are trained to predict a future value, thereby emphasizing value relevant information in the representations.
• While value equivalent models have shown strong empirical success, there is no research yet that visualizes and investigates what types of representations these models actually learn.
• Therefore, in this paper we visualize the latent representation of MuZero agents.