Unsupervised Learning of Temporal Abstractions with Slot-based Transformers

Understand

The discovery of reusable sub-routines simplifies decision-making and planning in complex reinforcement learning problems.

• Previous approaches propose to learn such temporal abstractions in a purely unsupervised fashion through observing state-action trajectories gathered from executing a policy.
• However, a current limitation is that they process each trajectory in an entirely sequential manner, which prevents them from revising earlier decisions about sub-routine boundary points in light of new incoming information.
• In this work we propose SloTTAr, a fully parallel approach that integrates sequence processing Transformers with a Slot Attention module and adaptive computation for learning about the number of such sub-routines in an unsupervised fashion.