On the Importance of Feature Decorrelation for Unsupervised Representation Learning in Reinforcement Learning

Understand

Recently, unsupervised representation learning (URL) has improved the sample efficiency of Reinforcement Learning (RL) by pretraining a model from a large unlabeled dataset.

• The underlying principle of these methods is to learn temporally predictive representations by predicting future states in the latent space.
• However, an important challenge of this approach is the representational collapse, where the subspace of the latent representations collapses into a low-dimensional manifold.
• To address this issue, we propose a novel URL framework that causally predicts future states while increasing the dimension of the latent manifold by decorrelating the features in the latent space.