OmniVTLA: Vision-Tactile-Language-Action Models with Semantic-Aligned Tactile Sensing

Understand

Recent vision-language-action (VLA) models build upon vision-language foundations, and have achieved promising results and exhibit the possibility of task generalization in robot manipulation.

• However, due to the heterogeneity of tactile sensors and the difficulty of acquiring tactile data, current VLA models significantly overlook the importance of tactile perception and fail in contact-rich tasks.
• To address this issue, this paper proposes OmniVTLA, a novel architecture involving tactile sensing.
• Specifically, our contributions are threefold.