RSVP: Reasoning Segmentation via Visual Prompting and Multi-modal Chain-of-Thought

Understand

Multi-modal Large Language Models (MLLMs) have demonstrated remarkable reasoning capability while lack explicit mechanisms for visual grounding and segmentation, creating a gap between cognitive reasoning and visual perception.

• To bridge this gap, we introduce Reasoning Segmentation via Visual Prompting (RSVP), a novel framework that unifies multi-step multimodal reasoning with grounded visual understanding.
• RSVP is a two-stage structuralized framework that integrates reasoning-driven localization with segmentation refinement.
• In the reasoning stage, RSVP employs multimodal chain-of-thought visual prompts to help MLLMs understand queries and infer targets, generating interpretable region proposals that enhance visual grounding.