ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigation

Understand

We introduce ViLAM, a novel method for distilling vision-language reasoning from large Vision-Language Models (VLMs) into spatial attention maps for socially compliant robot navigation.

• Unlike traditional methods that rely on expert demonstrations or human-annotated datasets, ViLAM performs knowledge distillation and fine-tuning at the intermediate layer representation (attention) level by aligning attention maps from a pretrained vision-action model with socially guided attention maps derived from a large VLM.
• These distilled attention maps highlight key navigational regions in a scene and serve as socially informed spatial cost maps for motion planning.
• To achieve this, we introduce a novel attention-level distillation loss that fuses knowledge from both sources, generating augmented attention maps with enhanced social awareness.