HAMLET: Switch your Vision-Language-Action Model into a History-Aware Policy

Understand

Inherently, robotic manipulation tasks are history-dependent: leveraging past context could be beneficial.

• However, most existing Vision-Language-Action models (VLAs) have been designed without considering this aspect, i.e., they rely solely on the current observation, ignoring preceding context.
• In this paper, we propose HAMLET, a scalable framework to adapt VLAs to attend to the historical context during action prediction.
• Specifically, we introduce moment tokens that compactly encode perceptual information at each timestep.