ROOT: VLM based System for Indoor Scene Understanding and Beyond

Understand

Recently, Vision Language Models (VLMs) have experienced significant advancements, yet these models still face challenges in spatial hierarchical reasoning within indoor scenes.

• In this study, we introduce ROOT, a VLM-based system designed to enhance the analysis of indoor scenes.
• Specifically, we first develop an iterative object perception algorithm using GPT-4V to detect object entities within indoor scenes.
• This is followed by employing vision foundation models to acquire additional meta-information about the scene, such as bounding boxes.