Transformers are Sample-Efficient World Models

Understand

Deep reinforcement learning agents are notoriously sample inefficient, which considerably limits their application to real-world problems.

• Recently, many model-based methods have been designed to address this issue, with learning in the imagination of a world model being one of the most prominent approaches.
• However, while virtually unlimited interaction with a simulated environment sounds appealing, the world model has to be accurate over extended periods of time.
• Motivated by the success of Transformers in sequence modeling tasks, we introduce IRIS, a data-efficient agent that learns in a world model composed of a discrete autoencoder and an autoregressive Transformer.