USER-VLM 360: Personalized Vision Language Models with User-aware Tuning for Social Human-Robot Interactions

Understand

The integration of vision-language models into robotic systems constitutes a significant advancement in enabling machines to interact with their surroundings in a more intuitive manner.

• While VLMs offer rich multimodal reasoning, existing approaches lack user-specific adaptability, often relying on generic interaction paradigms that fail to account for individual behavioral, contextual, or socio-emotional nuances.
• When customization is attempted, ethical concerns arise from unmitigated biases in user data, risking exclusion or unfair treatment.
• To address these dual challenges, we propose User-VLM 360{\deg}, a holistic framework integrating multimodal user modeling with bias-aware optimization.